The US election was a wake up call for data illiterate journalists

So Nate Silver won in 50 states; big data was the winner; and Nate Silver and data won the election. And somewhere along the lines some guy called Obama won something, too.

Elections set the pace for much of journalism’s development: predictable enough to allow for advance planning; big enough to justify the budgets to match, they are the stage on which news organisations do their growing up in public.

For most of the past decade, those elections have been about social media: the YouTube election; the Facebook election; the Twitter election. This time, it wasn’t about the campaigning (yet) so much as it was about the reporting. And how stupid some reporters ended up looking.

At the centre of all this was Nate Silver, a data journalist who had made his name by turning his skills in analysing sports data to looking at elections. Last time round he was a novelty; this time, to some, he was a threat. In fact, the witch-hunting became so ridiculous that it generated a dedicated satirical site: IsNateSilverAwitch.com

Mark Coddington summed up the culture clash perfectly: journalists used to the artificial objectivity of ‘he said/she said’ reporting could not understand the scientific objectivity of reporting the evidence:

“Journalists get access to privileged information from official sources, then evaluate, filter, and order it through the rather ineffable quality alternatively known as “news judgment,” “news sense,” or “savvy.” This norm of objectivity is how political journalists say to the public (and to themselves), “This is why you can trust what we say we know — because we found it out through this process.” (This is far from a new observation – there are decades of sociological research on this.)

“Silver’s process — his epistemology — is almost exactly the opposite of this:

“Where political journalists’ information is privileged, his is public, coming from poll results that all the rest of us see, too.

“Where political journalists’ information is evaluated through a subjective and nebulous professional/cultural sense of judgment, his evaluation is systematic and scientifically based. It involves judgment, too, but because it’s based in a scientific process, we can trace how he applied that judgment to reach his conclusions.

“Both of those different ways of knowing inevitably result in different types of conclusions. Silver’s conclusions are at once much more specific and much less certain than those of the political punditry. The process of journalistic objectivity can’t possibly produce that kind of specificity; that’s outside of its epistemological capabilities.”

They also couldn’t understand basic statistics and methodology.

That was – let’s not mince our words – an embarrassment to the profession. Journalists who professed to be political experts were shown to be well connected, well-informed perhaps, but – on the thing that ultimately decided the result: how people were planning to vote – not well educated. They were left reporting opinions, while Nate Silver and others reported research.

That embarrassment should serve as a wake up call for an industry which can do better than publishing things like this:

Or broadcasting this:

With campaigning camps communicating directly to the electorate, mere access to information has become ever more devalued. In this election it wasn’t that access which won out, but ultimately the analysis, understanding, and data literacy. The reporters who couldn’t handle this witchcraft were left with egg on their faces. When the next election swings round, let’s hope they don’t make the same mistakes again.