The military wants to build a bullshit detector for social science studies

[Judi Lynn]

Artificial intelligence could potentially help psychology move past its “replication crisis.”
By Brian Resnick@[e-mail] Feb 25, 2019, 9:30am EST

How do you know if a finding in social science is, well, real?

Students of science are taught not to take the results of a single study as the absolute truth. The rule of thumb is to seek out systematic reviews, or meta-analyses, which pull together lots of studies looking at the same question and weigh their rigor to come to a more fully supported conclusion.

But can we trust all this meta-science? Not necessarily. Positive, confirmatory findings often make their way into the published literature, while negative findings collect dust in file drawers. This means meta-analyses, too, can be biased.

Adding to the confusion is the fact that in recent years, rigorous retesting of social science theories, like the willpower theory of ego depletion, has come crashing down. The resulting “replication crisis” had led many scientists to wonder: How many findings in psychological science would hold up in rigorous retesting?

They’ve been answering that question by embarking on large-scale replication efforts. But wouldn’t it be helpful — for deciding which ideas are worth funding, for deciding which ideas are worth publicizing — to be able to predict the answer without investing the time and resources to retest an experiment?

More:
https://www.vox.com/science-and-health/2019/2/25/18211125/darpa-score-center-for-open-science-ai

[PoindexterOglethorpe]

1. Some of us have excellent bullshit detectors

without the need for building a machine or invoking artificial intelligence.

Oddly enough, it seems as though men are not very good at detecting bullshit. A lot of women are. Maybe it's connected to the gossip that women overtly indulge in. Men gossip, they just don't call it that, and for reasons that escape me, don't use that as a basis for detecting bullshit.

Anyway, my answer is to hire a bunch of women.

[littlemissmartypants]

2. They should definitely try to build it.

The algorithms needed can surely be written and effective with the advances in AI and deep learning.

Inaddition to the gender related biases, age related biases also exist. I know, because I am a published researcher on the topic.

Thanks for sharing this, Judi Lynn.

❤