|
WE1S collecting work (scraping and cleaning) can be streamlined by setting up a standard working environment on each workstation or laptop. Described below is the recommended standard set-up.
Currently, the following workstations in South Hall (English Dept.) have been configured as described. To use any of them for the project, log on as the "WE1S" user using a username in the format: [DOMAIN]\WE1S (where "domain" is the name of the individual machine, e.g., "TRANSCRIPT-1\WE1S" or "CRC-2\WE1S").
|
I. User Account
On Transcriptions Center workstations, WE1S developers should log in under the "WE1S" user name.
II. Desktop Shortcuts & Links
The WE1S user desktop should include a desktop folder called "WE1S Development Environment" that includes shortcuts or folders for the following (see screenshot). (See sections III-V below for folders and tools that need to be installed to support this development environment).
- _Info About This Workstation.txt file (contains name of the workstation, e.g., "transcript-1", OS version, plus any other useful information, such as particular traits of the machine)
(Name for icon: Info About This Workstation) - Shortcut to "workspace" folder (see III below).
(Name for icon: workspace) - Link to the online WE1S Google Drive data_archive. (This is where working data and final data from collecting runs will be uploaded.)
(Name for icon: WE1S Google Drive (online)) - Link to the local version of WE1S Google Drive data_archive. (Google Drive needs to be downloaded and installed on the local machine; and then set up under the WE1S account, which will sync it with the online Google Drive.)
(Name for icon: WE1S Google Drive (local synced to online)) - Link to the WE1S Mirrormask space on Jeremy's server: https://128.111.86.240:5001
(Name for icon: WE1S Mirrormask Site) - Link to the WE1S PBWorks site at: http://4humwhatevery1says.pbworks.com
(Name for icon: WE1S PBWorks Site)
- Link to Collection Workflow instructions.
(Name for icon: _Collection Workflow Instructions) - Link to the collection manifest Form (and also Spreadsheet)
(Name for icons: Collection Manifest Form, Collections Spreadsheet (collects responses from Collection Manifest Form) - Shortcut to the Python Integrated Development Environment on the local workstation (Enthought Canopy)
(Name for icon: Python (in Enthought Canopy IDE) - Shortcut to pythonscripts folder on local workstation
(Name for icon: pythonscripts) - Shortcut to local installation of Import.io
(Name for icon: Import.io app) - Link to Import.io online
(Name for icon: Tool - Import.io online) - Link to Import.io extractors on WE1S PBWorks site
(Name for icon: Tool - Import.io extractors) - Links to online tools (with shortcut names beginning "Tools -":
III. Local Workstation Folder Structure
- Local Workfolder: For ease of work, workstations should have a work folder set up as close to the top of the directory tree as possible (as follows). Subfolders can then be set up to collect and manipulate materials temporarily before uploading for storage to the WE1S Google Drive. (Note: the local workfolder is a temporary storage location for work. Working data and final data we want to keep should be uploaded to the proper locations on the WE1S Google Drive; see Instructions for Uploading Results to Google Drive.)
- Windows machines - Path for Working Folder:
- C:/workspace
- Mac machines - Path for Working Folder:
- User/workspace
- Local Python scripts folder: Workstations should have a Python scripts folder.
- Windows machines:
- C:/pythonscripts
- Mac machines - Path for Working Folder:
- User/pythonscripts
Note: The following subfolders and their scripts should be downloaded from the WE1S Google Drive python_scripts folder and preinstalled in the local pythonscripts folder:
- Cut Files at Delimiter
- Preprocessing
- Scott Kleiman scripts for topic model word clouds
- Scripts for NY Times
- Scripts for USA Today
- Scripts for Wall St Journal
IV. Pre-Installed Tools
WE1S collection workstations should have the following installed and configured as necessary (including the setting of "environment variables" in Windows):
- Python (version 2.x only)
- Python Integrated Development Environments (IDE)
- Canopy Enthought (installs Python plus editing environment and iPython Notebooks)
- Python packages (in addition to those included in the above distributions of Python)
- Google Drive (local version of Google Drive needs to be downloaded and installed on the local machine, and then set up under the WE1S account, which will sync it with the online Google Drive.)
- Import.io -- Note: Import.io can be used in a browser to extract data automatically from a Web page; and the browser version can also use customized extractors that someone else has created. However, creating customized extractors (e.g., able to select only particular data from a page) can only be done through Import.io's downloadable "app".
V. Additional Pre-installed Utility Tools
- LastPass password manager (for Windows, Mac, or Linux), free version -- login is username = 4humwe1s@gmail.com, passwd = [same as password for WE1S user on Transcriptions workstations]
- Should be enabled as plugin for browsers on the workstations.
- Advanced Renamer for Windows
- Chopping List for Windows
- DownloadThemAll plugin (for Firefox)
- Wget
- Also: Put shortcuts for Microsoft Word and Excel on the desktop, and open them to initiate Office for the WE1S user.