Posts Tagged ‘election’

What the Polls got right

January 11, 2008

After Clinton’s “win from behind” in New Hampshire, those who weren’t claiming Diebold conspiracies were left wondering how the polls could get it so wrong. One popular way the media uses to report polling data is by using the Real Clear Politics average. I see two problems with the Real Clear Politics average that contributed to the discrepancy between the polls and the final result: the averages don’t report the proportion of undecideds and when there are many polls, the average will be over a very short period of time. Case in point, the final Real Clear Politics average for New Hampshire sampled polls taken spanning a period of two days (1/5 – 1/7). The problem is that all these polls were taken after Iowa, when Obama was riding a wave of hype after his caucus victory. Here’s a look at how the average of polls before Iowa (and after November) and the average of polls taken after Iowa compare to the final primary results. (NB: the data is from Real Clear Politics, but unlike them I’ve included an Other/Undecided column).

New Hampshire Polls Dec 2007 – Jan 2008
  Obama Clinton Edwards Richardson Other/Undecided Margin
Final Result 36% 39% 17% 5% 3% 3%
Pre-Iowa 28% 33% 17% 7% 16% 5%
After Iowa 38% 30% 18% 6% 8% -8%

The polls taken before Iowa, therefore, picked the margin between Clinton and Obama far more accurately than the polls taken after Obama’s success. But it’s the level of Undecideds that tell the story. Undecided is a very nebulous, poorly-defined category. Ignoring the minor candidates, an “undecided” voter sounds like one that could be equally likely to vote for either candidate. In reality this group could be very different indeed. Many, probably most, of the undecideds would in fact have a preference (be it slight or major) for either Obama or Clinton. What I suspect happened is that those tending Obama were convinced after his success in Iowa and those tending Clinton either voted for her anyway or were convinced the night before when she showed us she wasn’t just a robot programmed to become President. That would explain Obama’s increase (slightly overstated, perhaps due to hype) and the undecideds’ decrease after Iowa and why Clinton picked up the undecideds on polling day.

What is clear is that pollsters need to start examining the undecideds more closely. I propose pollsters split the Undecided column into the categories talked about above: “tending towards candidate x”. Suppose they had done that, this is what the above table may have looked like:

  Obama Clinton Edwards Richardson Other Tend Clinton Tend Obama
Final Result 36% 39% 17% 5% 3%    
Pre-Iowa 28% 33% 17% 7% 2% 7% 7%
After Iowa 38% 30% 18% 6% 1% 7% 0%

Ultimately, the same polls that seem so inaccurate could have told the story of the primary if they had had a couple more categories.

I have a love-hate relationship with polls. As a politics junkie I find them fascinating, but they also contribute the politics as a game. Voters want to back a winner, so minor candidates (who may have interesting, novel ideas) are marginalized not because of these ideas, but because they “can’t win”.

To be useful, polls have to be designed well and reported well. A design flaw in the polls (no tending x category) and deficiencies in reporting (ignoring undecideds) both contributed to the controversy over New Hampshire.


Superdelegates and Edwards

January 10, 2008

Forget Clinton, Obama and superdelegates, John Edwards could well determine who wins the Democratic nomination. Despite Obama and Clinton returning reasonably equal results in Iowa and New Hampshire, Clinton has a commanding lead in the number of delegates who will vote for her at the convention. This is because of the superdelegates, party officials who can vote for whomever they like. Superdelegates account for 792 out of the 4040 delegates who will vote at the convention. Amongst those who have pledged support, Clinton has a commanding lead. Here are how the results currently stand:

Pledged Pledged (%) Super Super (%) Total Total (%)
Clinton 24 35.82% 159 59.77% 183 54.95%
Obama 25 37.31% 53 19.92% 78 23.42%
Edwards 18 26.87% 34 12.78% 52 15.62%
Richardson 0 0.00% 19 7.14% 19 5.71%
Kucinic 0 0.00% 1 0.38% 1 0.30%

If these trends are extrapolated, no candidate will have enough votes at the convention to form a majority. Clinton, by virtue of her superior superdelegate numbers, will be ahead of Obama by about 300 votes. She will require a far smaller proportion of Edwards’s delegates in order to form a majority:

Super Pledged Total To Win % of Ed. to win
Clinton 473 1163 1637 384 39.44%
Obama 158 1212 1370 651 66.88%
Edwards 101 873 974 1047
Richardson 57 0 57 1964
Kucinic 3 0 3 2018

Indeed, Obama will need to pick up at least two thirds of Edwards’s delegates in order to seal the nomination. In reality, things will probably pan out slightly differently. Edwards is likely to gain smaller percentages of the vote in the coming states, closer to 10% than 20%. My understanding of the convention rules is that Edwards can either instruct his delegates to vote for another candidate or allow them to vote with their consciences. If he lets the delegates vote by their conscience, it is likely, in my opinion, that Hillary will win.

Should Edwards Drop Out?

With Edwards short on money and with no realistic chance of winning the nomination, there has been much speculation amongst Democrats as to whether Edwards should remain in the race. Supporters of both Clinton and Edwards seem to think their candidate would pick up the Edwards vote. Edwards himself as said he will remain in the race. It would definitely be in his interests to do so. So long as the tussle between Obama and Clinton remains close, Edwards will enter the DNC as kingmaker: his delegates will determine the nominee. Edwards will then be able to offer his delegates to either Clinton or Obama, perhaps in exchange for his name on the ticket as Vice President or a future cabinet position.

Obama needs to stay in touch with Clinton, forget about superdelegates, and concentrate on wooing Edwards. Edwards supporters should stick with him but make sure he knows who their second choice is.

Diebold and New Hampshire

January 9, 2008

At the completion of the New Hampshire primaries, certain elements are are claiming an “astonishing” discrepancy between the results tallied by hand and those tallied by Diebold machines. Naturally reddit jumped on the bandwagon as fast as possible. Here are the results as of 96% precincts reporting (NB: others include Biden, Gravel and Dodd, CNN didn’t provide info for the rest of the field when I collated the data. The rest accounted for about 1% of the vote):

  Hand Diebold Difference
Clinton 35.17% 40.71% 5.54%
Obama 39.20% 36.24% -2.96%
Edwards 17.71% 16.97% -0.74%
Richardson 5.64% 4.40% -1.24%
Kucinich 1.89% 1.25% -0.64%
Others 0.49% 0.44% -0.05%

At first glance, the results seem to backup the conspiracy theorists. Is it possible that Clinton’s vote could be so much greater in the Diebold Districts and every other candidate slightly less without foul play? Has Diebold rigged the count in Hillary’s favor? Or is there something else at play?

Whilst Hand Districts are more numerous than Diebold Districts, they tend to be in less populous areas and far fewer votes are hand-counted than tallied by machine. Note that this table currently excludes the 9 Hand Districts and 2 Diebold Districts that have yet to report results.

# of Districts Votes Cast Votes/Districts
Diebold 95 222,464 2341.73
Hand 131 56,812 433.68

The discrepency may just be a matter of demographics: urban voters may like Hillary more than rural voters. So what happens when we looks at similarly sized districts? Here are the results in districts where between 900 and 1200 votes were cast.

  Hand Diebold Difference
Clinton 35.90% 38.09% 2.19%
Obama 38.00% 37.47% -0.53%

The effect is far smaller when comparing similar districts, but probably not enough to arrest the fears of conspiracy theorists. In the end, Clinton won because she was more popular in the large precincts which happen to be tallied by Diebold machines. Correlation, not causation seems more likely to me.

  Votes Cast in Precinct
  Under 1000 1000-2000 Over 2000
Clinton 36.26% 39.20% 41.25%
Obama 38.02% 36.84% 36.29%
Difference -1.76% 2.36% 4.96%

There are so many variables in an election result that to put Hillary’s win down to jiggery-pokery without any real evidence is over the top. Demographics of the turnout and McCain siphoning Independents away from Obama at the last minute are infinitely more likely to have affected the than Diebold skullduggery.

For democracy to work, the system must be transparent and maintain the confidence of its participants. Proprietary voting machines fail both these tests. American, as far as I know, are still capable of counting, so should return exclusively to the paper ballot.