Reply #42: I agree [View All]

[mgr]

I was not certain as to what class of studies you made reference to.

I think there might be a problem in the comparison between the types of inquiry an epidemiological study makes and an exit poll does. But my mind is old and rustier than I would like. It may have to do with the determinism of the sampling.

I think you are right that both have their basis within a multivariate context, but I don't think you are being completely frank in that you do not consider how goodness of fit tests may fit into both practices. Sampling should give some sense of an expected frequency, and comparison between the two should be conducted. I do not know if a goodness of fit test between the exit poll and the results has been conducted (good old p hat and q hat).

I suspect that the average WPE reported in NEP reflects an internal attempt to do a goodness of fit test.

That said, your experience should give you pause that when a researcher reports bias in his experimental design to probably take him at his word. The experimental design requires precincts that have WPE less than 4 if they are to be aggregated into a larger population (a national or state sample), or that the WPE that varies in both directions is balanced (the error cancels itself out). If there is directional bias in excess of the design WPE, you cannot combine these samples into a larger population within the bias term being included--you cannot simply allow the central limit theorem to permit you the reward for controlling sample error or bias with a smaller MOE. It would be, in a word, unethical.

One method to do this is sum(MOE1/n1, MOEn/nn)/n which if the average WPE MOE is 6.5% makes all the aggregated samples MOE about 6.5%, and is consistent with what NEP reported on Mystery Pollster.

http://www.mysterypollster.com/main/2004/12/so_if_exit_poll.html

Mike