Vladimir Putin casts his vote at a polling station in Moscow on 4 March 2012. Photograph: KeystoneUSA-ZUMA / Rex Features

Russian election: does the data suggest Putin won through fraud?

This article is more than 12 years old

Vladimir Putin's landslide vistory in the 2012 presidential elections has been marred by allegations of fraud. Does the data support his detractors?
Get the data

To the surprise of virtually no-one, Vladimir Putin has won a landslide vistory in Russia's presidential elections. As also seemed inevitable, electoral observers and Putin's opponents alike have reported allegations of widespread electoral fraud, as the Guardian Russia correspondent Miriam Elder reports:

Two women hover over a ballot box in the industrial Russian city of Cherepovets, stuffing in ballot after ballot. On the streets of Moscow, an independent election monitor armed with an iPhone trails a van full of "carousel" voters – people bussed from polling site to polling site in order to cast multiple votes for Vladimir Putin.

Three months after Moscow exploded in a storm of fury over allegedly widespread electoral fraud during the country's parliamentary vote, Russians went to the polls to vote against or, mostly, for Vladimir Putin in his quest to return to the presidency.

Putin quickly claimed victory, waiting until just over 20% of votes were counted, but his opponents just as quickly cried foul, armed with reels of evidence of alleged fraud. They uploaded them by the thousands to their Twitter accounts and LiveJournal blogs, helping the indignation go viral.

Measuring the scale of electoral fraud is always a challenge. Confirming some irregularities have taken place is (relatively) straightforward, but proving whether fraud is systemic or isolate is vastly more difficult.

In many aspects of life, however, there is a statistical trick that can find large-scale fraud, particularly in finance.

In short, it's that far more numbers start with a "1" than you'd think: in a normal ledger book without fraud, around a third of the figures (whether £17.20 or £1.16bn) would be expected to begin with a 1.

This pattern continues throughout the digits, and is known as Benford's law (a fuller explanation can be found here)

We've taken results from just over 2,150 polling stations submitted by Russia's election observers to a Russian-language site here. These aren't the verified official results, but give us a much bigger dataset to check than those would.

We then grouped together the results for all candidates. Did the results comply to Benford's law? The short answer is no, as shown in the graph below:

The results for Putin's vote share alone were even more striking, and it's worth noting the difference was statistically significant in both cases:

Does this mean the statistics show Russia's results were subject to fraud? Unfortunately, as my colleague Ben Goldacre – who helpfully did some of the number-crunching for this piece (though all errors are mine) – is fond of saying, it's a bit more complicated than that.

Each polling station within this set of data has a relatively tight range of votes: between three and around 2,600. This makes it unlike most data in, say, financial accounts, due to lack of variance, and can mean data doesn't comply to the Benford pattern even when totally legitimate.

Whether Benford's Law has potential for finding political fraud is a matter of contention. A study by Joseph Deckert, Mikhail Myagkov and Peter C. Ordeshook published in August 2011 concluded not:

It is not simply that the Law occasionally judges a fraudulent election fair or a fair election fraudulent. Its "success rate" either way is essentially equivalent to a toss of a coin, thereby rendering it problematical at best as a forensic tool and wholly misleading at worst.

However, this piece quickly received a response from a professor at the University of Michigan, Walter Mebane:

The paper mistakenly associates such a test with Benford's Law, considers a simulation exercise that has no apparent relevance for any actual election, applies the test to inappropriate levels of aggregation, and ignores existing analysis of recent elections in Russia ...

Whether the tests are useful for detecting fraud remains an open question, but approaching this question requires an approach more nuanced and tied to careful analysis of real election data than one sees in the discussed paper.

Data-driven investigations clearly offer intriguing potential for catching widespread fraud, whether financial or political. But it also seems the scope and potential of this method and others are far from a subject of consensus.

We've published the full data from the electoral observers below. If you've picked up on anything interesting within, or run more sophisticated tests,
let us know in the comments below, by email to james.ball@guardian.co.uk, or through twitter @jamesrbuk.

Update: Ben Goldacre has published more complete tables of analysis, and a much more thorough technical overview, on his blog here.