Data Wrangling and Analysis

Tools Used

The following tools were used for all data work, visualization, collaboration and web publishing.

• import.io
• selectorgadget
• OpenRefine
• R Project
• R Studio
• Sci2
• Gephi
• Microsoft Excel
• Atom
• Git
• Github
• Adobe Illustrator
• Google Drive

R Packages

The following R packages were used for all data related operations:

• plyr
• dplyr
• rvest
• stringr
• reshape2
• magrittr
• gdata
• ggmap
• xlsx

R Scripts

These scripts are used, in relative order to produce the dhq_sterling_master and dhq_author_lut. Some manual processing is required, and is detailed in the code comments.

1. web scraping.R
2. LUT.R
3. geocoding.R
4. slm keyword gathering.R
5. keyword extraction.R
6. validation.R