Can Twitter Predict the Virginia Governor’s Race?

The other day, I came across an article entitled, A New Study Says Twitter Can Predict US Elections. The gist of it?

…the study found a correlation between the number of times a candidate for the House of Representatives was mentioned on Twitter in the months before an election and his or her performance in that election. The more a candidate is mentioned on Twitter, the better.

In other words, the Hollywood adage, “no publicity is bad publicity” would seem to apply to politics as well. Of course, I’m not sure I agree with that adage (or others like “death is a career move”), but then again, what do I know about Hollywood?

Anyway given that I spend a ridiculously (pathetically?) large percentage of my life these days using social media – Twitter included – I was intrigued by the claims made by this study. I mean, it would be cool if we could skip the pollsters and just use Twitter to figure out who was going to win an election. On the other hand, this seems way too good to be true, so in the end I was very skeptical. It didn’t help matters that the quantity and quality of claims made for social media’s supposed influence, power, etc. are often far out of proportion to their actual influence, power, etc.

Still, I thought it might be worth looking into this a bit, and specifically whether it could be applied to the Virginia governor’s race. So, I emailed the authors – Joseph DiGrazia and Fabio Rojas of Indiana University – to see what they thought. I introduced myself, told them who I was and what I was interested in, and they graciously responded almost immediately with information on how to keep a count of Twitter mentions (don’t ask: it’s highly technical, definitely not fun for liberal arts majors like me). Instead of doing all that, I decided to just do an easy test case: count the number of Twitter mentions of each candidate in August. After doing that, I wrote a follow-up email Monday afternoon:

FYI, I just counted the number of tweets in August for the words “Cuccinelli” and “McAuliffe.” I got 130 for McAuliffe, 81 for Cuccinelli. I haven’t done any further tweaking of the data, such as adjusting for the fact that Cuccinelli is an incumbent (albeit in a different office – Attorney General – than what he’s running for – Governor) or how positive or negative the tweets were. Based on your research, would you say these results mean anything?

Here’s the response from one of the researchers, Joseph DiGrazia:

Well, assuming the results of our analysis of house races also hold for gubernatorial races, it would imply that McAuliffe has more buzz and is more likely to win. However, it’s important to remember that this isn’t deterministic and there are instances where tweet shares and vote shares don’t line up.

Interesting, but…does any of this really mean anything? I suppose it’s possible, but again, I’m highly skeptical. I became even more skeptical after reading this:

The researchers could predict the outcome in 404 out of 435 races – an astounding prediction rate of 92.8 percent.

But there’s a catch… and it’s a big one.

What the paper and its co-author Indiana University sociologist Fabio Rojas failed to take into account (or at least acknowledge in all their press on the topic) was simple statistical probability. In 19 of the past 23 elections, 90 percent of incumbents ended up winning re-election.

So the researchers improved on our existing prediction base by 2.8 percent. Not exactly newsworthy when put into that sort of perspective.

Hmmmm… In addition, what’s really weird about all this is that it seemingly doesn’t matter for the model results whether the content of the tweets is positive or negative, or even if the negative “buzz” is coming from a particular candidate’s opponent (e.g., in this case, Cuccinelli’s campaign has been relentlessly tweeting negative crap about McAuliffe, accounting for a significant percentage of the tweets about T-Mac in August). I don’t know about you, but that definitely runs smack into my “common sense screen.”

Of course, in the case of Virginia 2013, there’s really no “incumbent” per se, as neither Ken Cuccinelli nor Terry McAuliffe is currently governor. Still, does any of this make any sense, intuitively or in any other way? Again, I really have no idea, but my inclination is to be highly skeptical, certainly until we have a LOT more data, thorough examination of alternative explanatory variables, etc., probably several years from now. For the time being, I’d just chalk this one up as “potentially interesting,” and leave it at that.

P.S. It would be interesting to test these guys’ theory across various elections. Anyone interested?