General Discussion
In reply to the discussion: How to Lie with Rape Statistics: America's Hidden Rape Crisis [View all]Donald Ian Rankin
(13,598 posts)Here's a concrete example.
People surveying drug use often use a trick where they give people a hat with three cards, with questions "Is the sky green?", "Have you ever used drugs?" "Are bananas yellow?". They ask people to pick a card at random, answer the question and put it back, and they tally the yesses and nos; from this you can deduce the number of drug users, without anyone confessing. So there, you can get the right answers, so it's not a fair analogy.
But now, suppose you have a hat with lots of cards, each with one of those three questions on it, and you don't know how many of each there are. This time, if you tally people's answers, there's no way to work out what the actual right answer is from the wrong answers you have. *But*, if you use the same hat in multiple cities, your numbers will all be wrong by the same linear function, so if one city produces more "yesses" than another you *can* deduce that it has more drug use, even though your numbers are wrong and you can't work out what the right numbers actually are.
And that probably *is* a good analogy for using the same, flawed-but-not-worthless methodology to try to work out incidence of rape in America in different years. It won't let you get absolute answers, but it *does* provide genuine information about comparisons.