Data Access Web Sites
1. General Comments
The controllers of data have power. Often controllers want to keep the power. For example
those who analyze the data and publish first get the credit. Those that understand the patterns
in the data may profit from their knowledge as long as others are kept in the dark.
There may be many problems with data in terms of quality, documentation and formatting.
Substantial work may be required to prepare some data sets for analysis. For example there
may be categorical variables with multiple words or misspellings for the same category label.
Some students have used OpenRefine as clean up tool. (OpenRefine began as Google tool to
address variety of recurrent clean up tasks. However, I recommend starting with “nice” data
for class projects.
Some data sets my not be safe to have on your computer. I like to think that the federal
government data is relative well-structured and safe.
Federal agencies address congressional mandates. Agencies do what they are funded to do.
Politics play a big role in congressional mandates.
Federal agencies have to address the laws addressing confidentiality.
Federal agencies have limited resources to deal with public questions. Someone may ask if the
three occurrences of brain cancer in their community constitute a hot spot? If so, what is
causing this? Many questions arise due to misunderstanding of the data or what can be
inferred. With less data available there are the fewer questions. Once in a while a pattern is
the result of a problem that needs to be addressed.
2. Recent Microsoft links to multiple data sources
Revolution Analytics blog
More Cited by Microsoft
3. Old data access links that many no longer work.
1) Department of Education
National Assessment of Educational Progress
I have often used this web ties to obtain US state data.
2) NCI and CDC Health Data:
State Cancer Profiles
CDC Wonder
In the past I have used both sites to obtain data.
3) Department Labor Statistics: has many times series
[Link]
Provides access to state and county quarterly Census of Employment and
Wages data. Easy to use.
[Link]
I used this site a fair amount in past. Directions I prepared for my students
no longer work, but the site still may be useful. BLS seems to have shifted
emphasis from states to metropolitan statistical areas. This may be an
indicator of the future.
4) Bureau of Census:
Census has lots of data and different path to data. Some paths may be disappointing.
America Fact Finder
Data
Economic Indicators
Census population type flow mapper
Info graphics
C-SPAN's America by the Numbers
5) EPA
Data Finder
Toxic Release Inventory
Finding releases for a given location is easy.
Download the [Link] software to access data. There is access to data for
state, county and zip code regions. You can enter your street address and
find release sites in a small region that contains your street address.
6) Justice
National Archive of Crime Justice Data
National Institute of Justice: Accessing Data
7) NSF: National Center for Science and Engineering Statistics
NCSES
Example: Business R&D and Innovation 2011
8) US Geological Services: USGS
The national map viewer
9) US Department of Agriculture: USDA
[Link]
National Agricultural Statistics Service
Economic Research Services
Food and expenditure, nutrition, obesity and other statistics.
10) The Earth
Goddard Space Flight Center:
[Link]
[Link]
Likely not suitable for small projects.
Global atmospheric data. Dr. Ashley (Ph.D. 2013), used Matlab to read the HDF4 files
for L3Q* compressed multivariate summaries. R could could read HDF5 files not HDF4
National Oceanic and Atmospheric Administration
National Climatic Data Center
National Snow and Ice Data Center
Note that some files are in HDF5 format. There is a capability in R to read
such data, but using MatLab may be better. One STAT763 student report
difficulty with getting data into shape using R. There were missing data and
other issues to address.
11) World Health Organization:
World Heath Observer Maps
WHO Data Access
12) World Bank
World Bank Data
13) Machine Learning
UCI Machine learning Repository Many data sets here.
14) War
Correlates of War
15) Global Warming
Skeptical Science
Getting skeptical about global warming skepticism
My quick glance at this website did not lead me immediately to data.
It seems the current hot issues still concern the identifiability of the
warming causes.
16) Federal Data
[Link]
I have not found much data here of interest to me. I not check recently.
