| 
  • If you are citizen of an European Union member nation, you may not use this service unless you are at least 16 years old.

  • Finally, you can manage your Google Docs, uploads, and email attachments (plus Dropbox and Slack files) in one convenient place. Claim a free account, and in less than 2 minutes, Dokkio (from the makers of PBworks) can automatically organize your content for you.

View
 

Pre-processing of Harvested Plain Text

Page history last edited by Alan Liu 6 years, 4 months ago

Replace:

  • . <--. (replace period with period followed by space)
  • : <--: (replace colon with period followed by space)
  • ? <-- ?  (replace ? with ?followed by space)

 

Tokenize:

 

  • Affordable_Care_Act
  • Cold_War
  • Common_Core
  • distance_learning
  • AAUP <-- American Association of University Professors
  • American_Studies_Association
  • Chronicle_of_Higher_Education
  • Harvard_University
  • HMO <--H.M.O.
  • Letters_to_the_Editor
  • liberal_arts (<--liberal arts, liberal-arts, liberal art, liberal-art)
  • M_D <-- M.D., M.D.s 
  • Middle_East (<-- Middle East, Middle Eastern)
  • M_A <-- M.A.
  • MLA <-- Modern Language Association
  • NEH <-- National Endowment for the Humanities
  • National_Humanities_Center
  • North_America (<-- North America, North_American)
  • PhD <--Ph.D.
  • Queer_Studies
  • Queer_Theory
  • Rockefeller_Foundation
  • social_sciences, social_science
  • South_America (<-- South America, South American)
  • The_National_Review
  • University_of_California
  • University_of_Chicago
  • U_S (<--U.S., United States, United States of America)
  • Wall_Street_Journal
  • Yale_University

 

Tokenize for adding to stop list:

 

  • AssociatedPress
  • RoomforDebate

 

 

Delete (or put in Stop List)

 

  • Associated Press
  • Corrections & Amplifications
  • Credit: By
  • Mr.
  • Ms.
  • njtowns@nytimes.com
  • N.Y. / Region
  • PROQUEST
  • Room for Debate
  • New York Times
  • WJS
  • wsj.com
  • Sunday New Jersey Section
  • 620 Eighth Avenue, New York, N.Y. 10018-1405

 

Comments (0)

You don't have permission to comment on this page.