Nate Silver sees fishy patterns in SV polling, possibly indicating fraud. SV contacts lawyers.

[jefferson_dem]

Nate has conducted some fascinating analyses and provided a terribly important service to us poll watchers. I've provided snippets. Check out the whole thing at 528 if you're inclined.

Strategic Vision Polls Exhibit Unusual Patterns, Possibly Indicating Fraud
by Nate Silver @ 9:04 AM

Share This ContentOne of the things I learned while exploring the statistical proprieties of the Iranian election, the results of which were probably forged, is that human beings are really bad at randomization. Tell a human to come up with a set of random numbers, and they will be surprisingly inept at trying to do so. Most humans, for instance, when asked to flip an imaginary coin and record the results, will succumb to the Gambler's Fallacy and be more likely to record a toss of 'tails' if the last couple of tosses had been heads, or vice versa. This feels right to most of us -- but it isn't. We're actually introducing patterns into what is supposed to be random noise.

Sometimes, as is the case with certain applications of Benford's Law, this characteristic can be used as a fraud-detection mechanism. If, for example, one of your less-trustworthy employees is submitting a series of receipts, and an unusually high number end with the trailing digit '7' ($27, $107, $297, etc.), there is a decent chance that he is falsifying his expenses. The IRS uses techniques like this to detect tax fraud.

Yesterday, I posed several pointed questions to David E. Johnson, the founder of Strategic Vision, LLC, an Atlanta-based PR firm which also occasionally releases political polls. One of the questions, in light of Strategic Vision LLC's repeated failure to disclose even basic details about its polling methodology, is whether the firm is in fact conducting polling at all, or rather, is creating fake but plausible-looking results in order to increase traffic and attention to its core business as a PR and literary firm.

I posed that question largely as a hypothetical yesterday. But today, I pose it much more literally. Certain statistical properties of the results reported by Strategic Vision, LLC suggest, perhaps strongly, the possibility of fraud, although they certainly do not prove it and further investigation will be required.

<SNIP>

I haven't really seen anyone approach polling data like this before, and I certainly haven't done so myself. So, we cannot rule out the possibility that there is some mathematical rationale for this that I haven't thought of. But it looks really, really bad. There is a substantial possibility -- far from a certainty -- that much of Strategic Vision's polling over the past several years has been forged.

I recognize the gravity of this claim. I've accused pollsters -- deservedly I think in most cases -- of all and sundry types of incompetence and bias. But that is all garden-variety stuff, as compared against the possibility that a prominent polling firm is making up numbers whole cloth.

<SNIP>

http://www.fivethirtyeight.com/2009/09/strategic-vision-polls-exhibit-unusual.html

David E. Johnson of Strategic Vision (not to be confused w former Florida GOP executive Director David Johnson) categorically denied the allegation and suggested legal action may be looming. Here's Johnson's statement:

<SNIP>

Secondly in regards to Nate Silver's statements, we categorically deny them and will refute them. We have a call into our attorney on this and fully intend to take action that will vindicate us. I wish Nate had contacted me directly yesterday when he began this tirade, I could have answered his questions fully to his satisfaction prior to damage being done to our reputation. Now that he has made these accusations and posted them online, I must and will defend our company's reputation through all legal avenues available. The reason that we are going the legal route is he has attempted to do severe damage to our reputation and what is he going to do when we disprove him just say I am sorry. That isn't enough at this point.

http://blogs.tampabay.com/buzz/2009/09/nate-silver-pollster-may-be-fraud.html

[PretzelWarrior]

1. go for it, asshole. that PR firm spokesperson or owner sounds like a hack.

[grantcart]

2. the fact that they have retained an attorney shows that they are worried

Silver's comments were always hedged "possible" etc.

go ahead and file a lawsuit.

[dave29]

3. Nate's analysis did not happen in a vacuum

SV was reprimanded by the authority for polling agencies in the last few days, prompting Nate's analysis. The idea that Nate is attempting to do severe damage ignores the severe damage already done (and deservedly so) by the AAPOR

[Mabus]

4. He's not the only who has questioned polls by Strategic Visions.

The American Association for Public Opinion Research (AAPOR) found that Strategic Vision LLC, an Atlanta-based company, repeatedly refused to release essential facts about polls it published prior to the 2008 presidential primaries in New Hampshire and Wisconsin. The AAPOR Executive Council announced today that this nondisclosure by Strategic Vision LLC was inconsistent with the association's Code of Professional Ethics and Practices (http://www.aapor.org/AAPOR_Code.htm) and contrary to basic principles of scientific research.

AAPOR’s inquiry began as part of its extensive evaluation of the methodological factors in last year’s pre-election polls, most notably those preceding the New Hampshire Democratic primary. As part of its evaluation, AAPOR requested information from 21 organizations that published pre-primary poll results in New Hampshire, South Carolina, Wisconsin or California. Strategic Vision LLC was the only polling firm that did not provide sufficient methodological information (as defined by the AAPOR Code) about its surveys and refused to provide that information in response to AAPOR’s repeated direct requests.

http://aapor.org/AAPOR_Raises_Objections_to_Actions_by_Strategic_Vision_LLC.htm

[bluestateguy]

5. Ah yes, the party of tort reform (for thee, but not for me)

nt

[Jane Austin]

6. Don't know about their statistics, but

their spokesman appears to be illiterate.