The Importance of The Margin of Error

Even the best poll or survey is slightly inaccurate, because a poll of a sample of people may not accurately reflect the entire population.  To account for this problem, pollsters have developed the concept of a "margin of error"- a number (usually 2 to 5 percentage points) which shows the range of likely results among the actual population, as opposed to the people who answered the survey.  (For a more technical explanation, go here).  So for example, if a poll shows that Hillary Clinton will get 48% of the vote against Chris Christie, and the poll's margin of error is 4 points, this means a poll of the entire population would probably give her between 44 and 52% of the vote. 

This concept is highly relevant to transportation data as well, since estimates of commuting habits are often based upon the American Community Survey, a group of polls performed under the auspices of the Census Bureau.  A recent story on Governing.com lists commuting data for every major American city.  If you just look at the tables with percentages, you might be surprised in all kinds of ways, noticing that transit commuting jumped in city X, or declined in city Y.  But if you consider the margin of error, you will see that some of these trends are in fact nonexistent.

For example, the ACS data shows that the percentage of Atlantans who rode public transit to work declined from 11.7% of the workforce in 2007 to 10.6% in 2012.  So you might conclude from this that something is very wrong with public transit in Atlanta (which, of course, might be the case).  But if you look at the margins of error in the 2007 and 2012 estimates you may read the data a bit differently.  

If you just look at the Census estimates, you will see that the number of transit commuters decreased from 24,346 in 2007 to 21,880 in 2012- seemingly a dramatic drop.  However, both numbers are subject to a margin of error- which, for 2007, is 3702.  So in fact, the number of 2007 transit commuters could have been anywhere from 20,644 to 28,048.  If the 2007 number is at the low end, transit commuting actually rose in Atlanta between 2007 and 2012.  If the number is at the high end, the drop in commuters is even greater.  Similarly, the 2012 estimate has a margin of error of 2469, which means the actual 2012 figure could be anywhere from 19,411 to 24,249.