Rare is the person that can find agreement in the diverse worlds of politics and sports. Nate Silver knows that first-hand.
As the journalist and statistician can attest, getting people to understand what could happen based on data can be a tricky proposition.
“People sometimes complain that polls are wrong. I think the reverse is true,” he said. “To me, it’s amazing they work at all with all the issues we have with polls.”
Silver recently spoke on the imperfect art and science of numerical forecasting at NASA’s Langley Research Center in Hampton, Virginia. Silver is a best-selling author of “The Signal and the Noise: Why So Many Predictions Fail – but Some Don’t” and is the founder of the award-winning website FiveThirtyEight.com, which uses data-driven news and analysis to cover areas such as sports, politics, economics, science and life.
“There’s more interest than ever in data-driven ways to view politics and sports and other things,” he said.
Silver’s love of data began as a child, growing up an avid Detroit Tigers fan. The Michigan native’s curiosity was piqued by baseball’s loyalty and history to numbers and figures.
“Sports is this perfectly constructed field where it’s very data-rich; it’s fairly easy to represent,” he said. “It has a tangible impact.”
When it comes to politics, people are less level-headed given the stakes involved and connections people make to candidates and causes.
“There’s more interest than ever in data-driven ways to view politics and sports and other things.”
– Nate Silver, journalist, author and statistician
“Politics invokes a lot of passions. There’s nothing wrong with passion,” he said. “Sometimes it trades off with the more cool-headed, detached thinking that you might need to make better forecasts and analysis.”
During his talk, Silver discussed what makes a good poll and how his website gathers its figures for their predictions.
“Our attitude is to use literally every poll unless we think it’s fake,” he said.
FiveThirtyEight’s purpose is to use data-driven predictions to figure out what’s happening and make it clear, concise and candid to readership. Silver notes that people are fascinated by trying to predict everything.
“Almost nothing is 100 percent or zero percent,” he said. “We’re saying, ‘What do we know and what don’t we know’.”
Silver rejects much ideology taught with statistical methods in colleges and universities today. The problem Silver finds is a belief in perfect experimental, survey, or other designs, when data often comes from a variety of sources and idealized modeling assumptions rarely hold true. Often such models reduce complex questions to overly simple hypothesis tests using arbitrary significance levels to accept or reject a single parameter value.
A good poll, according to Silver, reaches a random sample of a population, has a good response rate and contains a representative sample of different demographic groups.
“It sounds easy, but all of those are big challenges,” he said.
In contrast, the practical statistician first needs a sound understanding of how baseball, poker, elections or other uncertain processes work, what measures are reliable and which not, and what scales of aggregation are useful. This “Bayesian” approach is named for 18th century minister Thomas Bayes who discovered a simple formula for updating probabilities using new data.
For Silver, the well-known method needs revitalizing as a broader paradigm for thinking about uncertainty, founded on learning and understanding gained incrementally, rather than through any single set of observations or an ideal model summarized by just a few key parameters.
“Once you build a good process, you tend to trust that process,” he said.
However, he acknowledged the best processes always require updating, and an understanding of the uncertainty inherent in polling.
“You don’t want to outthink the polls on Election Day,” he said. “Sometimes the polls are right, sometimes they’re wrong.”
Eric Gillard
NASA Langley Research Center