Meeting (2019-05-23) - Interpretation Exercise on New Topic Model

Advance preparation for this meeting

For those of you still on term and able to participate, please if possible do the advance preparation described on this topic-model-interpretation-exercise Start Page before our WE1S all-hands meeting on Thursday, May 23, 2019, 11am - 1 pm (Pacific). Or, if you are short of time, at least perform part of the exercise.

More details and links are on the Start Page. The gist is this: please in advance of our meeting attempt to answer the indicated research question by exploring the topic model linked from the page. To do so, operationalize the research question in ways you think would allow it to be addressed via a topic model. Then use the available modules of the WE1S Interpretation Protocol (implemented as Qualtrics surveys also linked from the Start Page) in sequence or combination to conduct the analysis.

Our main goal is to get a feeling for whether our whole process--of topic modeling, addressing and operationalizing a research question, and then using the modules of the Interpretation Protocol to answer that question--is going to work during the summer. We will then devote the all-hands meeting on Thursday to discussing problems that need to be solved, the new visualizations, the interpretation protocol modules. etc.

Thanks, everyone! --Alan

0. Preliminary Business

• Next all-hands meeting: Thursday June 6, 11am-1pm?
• Project Management Team meeting? (early or mid June?)

Purpose of today's meeting

• Status reports
• Discussion of the interpretation workflow based on experience with the topic model for today. 

1. Status reports 

• Collection work
• May 20th deadline for collection requests
• Student & humanities team requests for publications collected fChomp and form L/N 
• Primary corpus collection team recommendation for additional search term
• Creation of 1BC topic model and other models  
• WE1S platform development
• Pre-processing
• windowing
• model optimization 
• wikification
• SpaCy 
• de-duping 
• Visualizations 

1. Dfr-browser

2. TopicBubbles

3. pyLDAvis (normal version of pyLDAvis showing words in topics)

1. pyLDAvis titles (version showing article titles in topics)

2. pyLDAvis publishers (version showing publication sources in topics)

4. Metadata-7D

1. Pub-7d

5. Topic Cluster visualizations:

1. pyLDAvis with topic clusters (circles left pane represent clusters of topics; the topics in each cluster are indicated by number in the right pane)

2. pyLDAvis with topic clusters showing words in cluster (circles left pane represent clusters of topics; the words associated with each cluster are indicated by number in the right pane.)

3. Dendrograms (hierarchical clustering) -- Note:  hierarchical clustering assumes every topic is a cluster and then groups them into progressively larger clusters. The pyLDAvis visualizations coerce all topics into 20 clusters.

6.  TopicMaps

• Interpretation Protocol 
• Technical Issues & Suggestions 
1. We need to make a "dashboard" of all our tools 
1. Or make and share a browser tab collect (e.g., through Toby) 
2. Can we get the annotation-capable versions of Dfr-browser and Topic Bubbles into the notebooks?
3. Using the topic clusters tools
4. How best to see trends in "stacked" view in Dfr-browser
5. Comparing two parts of a corpus
1. pyLDAvis (publishers)
2. Pub-7D
3. how we can use Scott's WMS for this purpose?
6. Using the maps visualization
7. Still unsolved: link to articles (once we can no longer store JSON text) 
• Documentation 
• Research Questions (spreadsheet) 
• Hand coding exercise 
• Human subjects research 
• Summer Research Camp planning (planning document) 

2. Discussion of the interpretation workflow based on experience with the topic model for today

• Start Page for interpretation exercise for the May 23rd meeting
• Issues related to research method
• Research question strategy:
• Need to design research questions that meet two criteria:
• they lead to project output goals (final analyses and advocatory products) 
• they are "operationalizable" via analysis of topic models
• Note: we should begin gathering a set of "recipes" for operationalization--e.g.:
• how to find the best keyword to use as proxy for an issue (method of scouting synonyms and antonyms, perhaps through WordNet)
• how to find topics in which two keywords are both important  
• Need for companion questions to give context
• For example, if the question is "How are the humanities discussed in relation to issues of careers, employability, and/or income?", then we need companion questions like "How are the humanities discussed in relation to political concerns of students?" (Do the same kinds of concerns about the "practical" relevance of the humanities show up? Is the same vocabulary used?)
• Examples of other possible companion questions:
• "How are the humanities discussed in relation to programmatic funding concerns?" (Is student anxiety about humanities careers correlated with institutional anxiety by departments and faculty about humanities funding?) 
  
  
• Preliminary hermeneutical tactics: There needs to be a preliminary brainstorming stage in which the two following processes complement each other (in a "hermeneutical circle"):
1. Operationalizing research questions 
2.  Observing the model:
• complete Interpretation Protocol modules 1 & 2 to gain an overview of the model, and also canvass the model informally to test if the proposed operationalization for a question is going to work.
• "Next step" suggestions for other kinds of analytical/interpretive research to complete the answer to a research question: There should be a question in the surveys for "next research step" suggestions:
• E.g., having found a coherent topic on humanities and jobs like #58, that topic's top articles could be used as a guide for a next step of collecting and categorizing examples of worries about/arguments for the humanities located in the top 10 articles associated with #58. 
• Using the Interpretation Protocol modules