Maintaining Optimism in the Face of Reality. Occasional observations on the state of the world, society, business and politics. Usually anchored by facts, always augmented by opinion.

On the Cell Phone Only Polling Bias and the '50% rule'  | e-mail post

I've done a lot of eye-rolling seeing people mention how the under-representation of voters who only have a mobile phone (no landline) is somehow invalidating all the polls, and will result in a radical shift. I just kind of chuckle when I see claims like that, because it reflects a lack of understanding about good polling methodology. I kind of chalked it up to people just dreaming up reasons why their guy might not really be behind.

But Friday evening a friend forwarded me a post from some message board, I believe to contest my claim that Bush has the race largely locked up. The posted message was:
The Washington Post has an interview with Charlie Cook, a political guru in the same league as the late Izzy Stone. Insiders pay a lot of attention to Charlie Cook. He warns that the polls are missing the cell phone-only voters, a topic that has been discussed on this site repeatedly. He also says the election may be determined by events in Iraq. If it gets much worse, Bush is in deep trouble. Cook also confirms what I have heard from many, many sources: the undecideds ultimately break at least 2:1 for the challenger. An incumbent president has to be polling 49% to 50% going into the election. A 47% to 47% tie in the polls means the challenger wins. The interview is worth reading.
This was really the last straw for me on the cell phone polling bias thing. I went to the post website to read the interview. It actually wasn't an "interview" it was a transcript of an online Q&A with Charlie Cook.

Berwyn, Ill.: With unprecedented numbers of newly registered voters all across the country, aren't ALL polls essentially irrelevant?

Charles Cook: Virtually all the polls you are seeing are using random digit dialing, not voter registeration rolls, so theoretically even newly registered voters are being polled. A far bigger problem is that as many as 18 percent of telephone subscribers today have no land lines, and since pollsters are not calling cell phones, almost one in five voters are not being included in poll samples.
OK, now that is absurd. It took about 12 seconds on Google to get a little more accurate number. IT research firm Yankee Group was citing something between 3% and 6% in result #2 in my search. In result #4, there is a number in the double digits:
In-Stat.MDR, a wireless market-research firm based in Scottsdale, Ariz., conducted a survey of wireless users in February of this year. Of the 970 people questioned, 14.4 percent were cell-phone-only users, the majority of whom were single Americans between the ages of 18 and 24, living in mostly urban areas.
OK, so among (some unknown sample of) people who own cell-phones, about one in six say that they were cell-phone only users. I still think this number is probably a little high, but even assuming one-half of two-thirds of Americans of voting age own a cell phone would suggest the wireless-only general population is in the 7%-10% range.

Charlie Cook seems a little more hack than guru, in my opinion. So I started to reply to my friend:
The point that people are missing is that all good polling organizations develop fairly detailed demographic profiles from all kinds of sources, and they make sure they talk to enough people to match. Note that when I say "good polling organizations" I would exclude any news-media run polls, as they only want content; they don't make any additional money for being accurate, and thus have limited incentives to be accurate.

What nobody ever brings up, because most people are bad with numbers and don't understand the half-science/half-black-art of polling is that the cell-phone-only issue can only cause problems if you assume that the political behavior of cell-phone-only users differs from their broader demographic peer group.

Now, you might say that, "well yes, it does because CPO people are probably generally more hip/wired than others their age, and are thus more likely to vote Democrat, because all cool hip people vote Democrat, only idiots and the terminally lame vote for Republicans"

To which I would refer you back up to the In-Stat survey that indicated the majority of CPOs were single, 18-24 and lived in urban areas. That demographic already tilts so strongly Democratic as to make it at most a rounding error, particularly when you consider that the very same demographic also produces the lowest voter turnout. In addition, because of the electoral vote system, even if they were undercounting Kerry voters, they are concentrated in areas that are already more or less locked up. NYC, LA, Chicago, Detroit are all going to go Kerry, they can go 100% and it won't matter. On the other hand, Dallas and Atlanta are in pure Bush country, again there's not a possibility of enough sampling error to make a difference.
I thought I'd actually look to see how large metro areas from CityPopulation break down on the Bush/Kerry divide when lined up versus the RealClearPolitics electoral vote projection, derived from averaging state-level polls (admittedly some media polls RCP includes are not the best, but it should be good enough to look at it).

When I looked at the data, it was about as I guessed, with Kerry-owned states having over 50% of the population of those larger metro areas. In fact, the only complete toss-up state with large cities is Ohio. I was then going to get some statistics about cell phone penetration by market, compare the percentage of total state population represented by the major cities, and do some further calculations.

When I was doing a quick search, though, I stumbled upon a great post from Friday on an experienced pollster's weblog with an analysis of the mobile phone bias that largely follows the outline of my e-mail, but has some appreciated factual detail, thanks to his access to research papers based on CDC and BLS data which are not yet available to the public (the cellphone-related questions are new in the CES, CPS and NHIS surveys in 2003 and 2004, and they tend to lag a couple of years for broad availability of the raw data). I strongly recommend you check out his weblog if you are at all interested in polling. He concludes his post writing:
We could calculate the "coverage error" that results from excluding wireless-only adults from political polls if we knew two things: (1) How the vote preferences of wireless only adults differ from those with working landlines and (2) the percentage of all likely voters with only wireless service. Unfortunately, both numbers are unknown.

Still, assume for the sake of argument that wireless adults are 5% of the electorate, that a survey of wired households shows a 48%-48% tie and that the missing wireless-only voters prefer John Kerry by a 20-point margin (58% to 38% - a pure but plausible guess based on the numbers for renters, low income, etc). If we were able to include the wireless only adults, it would change the overall preference by only one point - Kerry would lead 48.5% to 47.5%.

Keep in mind that two factors will work to reduce this small potential error: Wireless-only voters are likely to turn out at a lesser rate than those with wired phones, and pollsters typically weight to make up for overall differences in gender, age, race and education.
So, I decided it really wasn't worth spending too much more time on it, because it should be clear that for all the math I could apply, it would be impossible to determine a meaningful influence on electoral vote outcomes, especially without access to state-level data from the BLS or CDC data that might indicate particular differences among states for cell-phone only populations.

I did decide to format the spreadsheet I had made, in case you were curious about the breakdown. Note that in Ohio, I colored the cities to reflect Cleveland being a traditional Democratic stronghold and Cincinatti and Columbus being part of the state's broader Republican land mass.

The "50% Rule" For Incumbents

The other point mentioned in my friend's message, the "50% rule" for incumbents. You may have seen some mention of it, as a lot of Dems and Kerry leaners have been citing it for a few weeks. The gist is that undecided voters break heavily (around 80%) for the challenger in the race, thus if an incumbent isn't sitting at or above 50% going into the election, it's not looking too good for him.

Recent mentions are probably the first most people have heard of it; I certainly don't recall hearing it previously. However, it's actually not an entirely new idea. Nick Panagakis, President of Market Shares Research wrote an article suggesting the basic idea in "Incumbent Races: Closer Than They Appear" which appeared in 1989.

The underlying psychology behind the 50% rule is that because voters know the incumbent, their indecision is most likely based on a negative impression of the incumbent ("Another guy has got to be better") rather than true neutrality or ambivalence ("I can't decide, I need to learn more about both candidates").

Not surprisingly, just like the cell-phone-only bias claim, this "rule" has been generating a lot of chatter among Dems who are looking for another way to "prove" their guy is going to win (not that we on the other side can't be accused of the same, certainly). [MyDD-Chris Bowers (upd)] [Am Prospect-Guy Molyneux] [Mystery Poller Mark Blumenthal] Just to be clear, Guy and Mark are not shills, they are both very experienced pollsters, but they are Democrats, and could be forgiven for a lapse in complete objectivity about the issue. As Mark acknowledges at the end of his assessment: "Note: Interpretation of political survey data is even more subjective than methodology. Others may disagree, and opposing viewpoints are always welcome in the comment section."

Blumenthal does spell it out clearly in his wrap-up:
[T]he incumbent rule tells us that, at any given moment, the President's percentage of the vote relative to 50% is a better indicator of where the race stands than the margin separating Bush and Kerry. It also suggests the appropriate way to read the final polls just before the election (and these are my ranges – others may differ): If the average result of all the final polls (including undecided) puts Bush's percentage at 50% or higher, the President will likely win. If Bush's percentage is 48%-49%, the race is headed for a photo finish. At 47% or lower, the President will likely lose (add 1% to these ranges in any state where Ralph Nader is not on the ballot)

The main point: The incumbent's level of support is more important than the margin.
Of course, while most people are just grabbing this heuristic and pointing to the national horserace number, both Molyneux and Blumenthal acknowledge that you need to be looking at the state-by-state polls for the swing states and still doing the EV math.

While the psychology behind the rule more than passes a gut-check, I do wonder if the extent to which undecideds are predicted to break against Bush may be lower than the historical pattern the number show. The 80% figure for undecideds breaking to the challenger has been derived from looking at a very large sample of elections and polls, not just Presidential polls.

I would argue the number should be adjusted downard in Presidential contests because people treat their selection with more gravity than in any other race. Much higher than legislative races, because the official is ultmately only one of many in a decision-making body. Running the numbers for just gubernatorial races would probably get closer, because they are executives, but there still isn't that nationl security concern. Today, with the importance voters claim to be placing on issues of homeland security, Iraq and terrorism, undecideds may not break so hard against "the devil they know," so to speak.

Of course, anytime you focus on just presidential election behavior, its tough because we only have one every four years. If someone develops complete data sets of state polls, that would help to strengthen the predictive validity of the sample.

Finally, Nader confuses things at least a little bit, even though he is being widely ignored. If we look only at races with an incumbent and a third party candidate, it's just Perot in 1992 (v Bush & Clinton), again in 1996 (v Clinton & Dole) and Anderson (v Carter & Reagan) in 1980. Votes for either challenger were counted as part of the 80% number for undecideds breaking against he incumbent. In two of the three cases, the third party candidate was siphoning votes primarily from the incumbent (Perot from Bush and Anderson from Carter). When Chris Bowers broke down the numbers, he didn't have any 1996 polling data to assess.

In a nutshell, Perot and Anderson allowed the incumbent's base to reject the incumbent without terribly compromising their political values. In 2004, the only significant third party candidate is Ralph Nader offering an opportunity to liberals to reject big-money establishment politics. Of course, Nader has even less appeal to the Bush base than Kerry. The question Kerry supporters need to reflect on is what portion of the undecideds are torn between voting their conscience for Nader or voting pragmatically for Kerry. I was surprised, but I have heard people here in Minnesota contemplating voting for Nader, since they really haven't developed a fondness for Kerry, who they see as playing "me-too" to Bush on some issues. And the polls they watch (the Strib) show Kerry leading the state. The idealistic naïveté of some progressives may help Bush again.

As I think about it, the fact that Minnesota polls underproject Republicans could work out very well. The stronger Kerry looks in a state, the more comfortable liberal idealists will be voting for Nader. Hint: if you run into any serious liberals, encourage them to "send a message" by voting for Nader, "since Kerry's probably going to win here anyway." wink-wink

Of course, it's too early to really even tell. All the "50% rule" models are based on the undecided count from final polls within 7 days of the election, which had a 2-3% undecided level in presidential elections. Without crunching the numbers, Bush seems to be accumulating undecided voters in recent weeks.

However, this information should definitely affect our interpretations of state-level polls in about 10 days.

e-mail post | Link Cosmos | [Permalink]  |  | Saturday, October 16, 2004
Comments: Post a Comment

This page is powered by Blogger. Isn't yours?