The E.P.A. Says It Wants Research Transparency. Scientists See an Attack on Science.

[Judi Lynn]

By Lisa Friedman
March 26, 2018

The Environmental Protection Agency is considering a major change to the way it assesses scientific work, a move that would severely restrict the research available to it when writing environmental regulations.

Under the proposed policy, the agency would no longer consider scientific research unless the underlying raw data can be made public for other scientists and industry groups to examine. As a result, regulators crafting future rules would quite likely find themselves restricted from using some of the most consequential environmental research of recent decades, such as studies linking air pollution to premature deaths or work that measures human exposure to pesticides and other chemicals.

The reason: These fields of research often require personal health information for thousands of individuals, who typically agree to participate only if the details of their lives are kept confidential.

The proposed new policy — the details of which are still being worked out — is championed by the E.P.A. administrator, Scott Pruitt, who has argued that releasing the raw data would let others test the scientific findings more thoroughly. “Mr. Pruitt believes that Americans deserve transparency,” said Liz Bowman, an E.P.A. spokeswoman.

More:
https://www.nytimes.com/2018/03/26/climate/epa-scientific-transparency-honest-act.html

[Igel]

1. This is standard in some fields.

Mostly those where truth is more important than petty things like saving the world or supporting some action agenda. I rather liked one research result where the presenters went through all their data to show that it ineluctably led to a certain conclusion, then said they didn't accept their conclusion and asked for help in finding their error.

That shows respect for the truth and humbleness before reality.

Even when I was associated with a lab group in psycholinguistics, the attitude was "our data is your data."

It cut out the possibility of data dredging, of using rigged proxies to code the actual data (since all the data was complex and a human had to assign values to the variables being investigated). The data runs had numbers attached to them, so that if you saw that a researcher with a significant result used runs 10, 15, 23, 26, 29, 35, 42, 43, 44, 49, 51, and 55 you could ask, "So why did the pick that non-random data and use statistical tools that presuppose randomness?" They could look to see if there was equipment malfunction or if the data simply didn't support the conclusion that the researcher wanted. The software used to generate the data was also available for inspection. You could take the research apart; you could replicate it using the original software, the original subject inputs, and, if you had the same equipment, the same equipment. The only thing "replicated" would be running the same experiment on additional subjects. In effect, expanding the dataset for the original experiment.

It was the same lab group where a grad student realized that numerous papers, all based on the research from his group (but preceding him by years) had misunderstood the statistics and gotten them wrong--meaning every result, every conclusion, was in error. Not a little off; no, just plain indefensible. They had no valid conclusions. Now, for a decade or more nobody had caught this mistake, but the PI immediately assigned that grad student co-authorship to the squib they'd be writing in the next two weeks and submitting to the journal where the groundbreaking research had been published. He didn't know if they'd be recanting a decades' research or re-evaluating the research, but the decision was made, and that decision was final.

Truth matters. And if you don't have access to the underlying data, all you have is trust in the other scientists. And when they have reputations and agendas, you don't have that.

However, I must add that this lab group still had the original data from the decade old research. Because it had been put on the official record and any lazy bastard any place on Earth could ask for access and download it. Because the researcher valued transparency and didn't fear having others second guess his work. And when he chucked data, it was because it was bad, invalid, data, not because it was bad, disconfirming, data.

That, I think, is the minimum level of transparency. And the more socially and politically relevant the research, (1) the more the risk that ideologues will make sure their research supports the foregone conclusion, (2) the more important it is that the research be valid and open to inspection.

Psycholinguistics? Particle physics? Relevance to public policy? Not so much. Which is why, IMHO, the researchers can focus on truth and be willing to be transparent. The only thing riding on the results is ego. Not power. Not saving the world, reforming education, creating a new world order or a millennium on Earth or a 4th Reich or a worker's paradise.