This episode exposes my biggest frustration with science right now, the complete and total disregard for good research practices, and the huge amount of fraud in science. It is honestly disturbing how prevalent this is.
Study is here, check it out.
Script:
For today’s episode we are going to talk about fraud in science. This is something that we need to keep in mind when we are analyzing studies, especially those in nutrition, because nutrition data can be manipulated pretty easily, especially with many of the smaller studies. Last time I discussed this was in episode 12 when I discussed p-hacking, today we aren’t discussing bad science, we are talking about outright fraudulent science. Meaning scientists who outright lie or manipulate their data or methods to get the results they wanted.
So first and foremost as always I like to make sure you guys know where my data is coming from and in this case it is an article called: “How Many Scientists Fabricate and Falsify Research? A Systematic Review and Meta-Analysis of Survey Data” published in PLOS One. In it there are some truly shocking numbers, the first one I want to pull out since this site is called Scientific Nutrition is a reference to the food that the Food and Drug administration in routine data audits has found flaws in 10-20% of studies. That’s a lot of studies… Even more concerning they found 2% of investigators guilty of serious misconduct. This is truly concerning for someone like me who spends so much time working with nutritional data, and the idea of 20% of it being flawed, it simultaneously shocking and completely believable.
However, the data from this study gets much worse. When asked explicitly if they had either fabricated or falsified results approximately 2% of scientists said they did. We expect the actual number is significantly higher than this as people tend to underreport their own negative behavior even in an anonymous study. The other part that makes me think this way is that approximately 14% claim they know a colleague who has falsified or fabricated results. That is a lot of false results.
Even worse is that over 33% admit to bad research practices such as eliminating only publishing data that supports a certain practice, or even failing to acknowledge a conflict of interest, or mining the data for significant relationships which is related to P-hacking that I discussed in episode 12. That is a huge number of scientists and studies that we now need to be very careful in our interpertation of.
One of the reasons this is particularly problematic is because it is often very hard to identify these studies just from reading them. Often studies don’t show the original data, and so you need to base it on other features like concurrence with existing literature, but that can actually promote fraud. Let me explain: in academia there is a pressure to either publish or perish, and so often scientists are often pushed to publish frequently, and it is easy to get something published that seems to agree with existing literature. A contrarian article is likely to go through much more intense peer review.
Now here at the end of this episode, I’m supposed to give you a solution, but I don’t really have one. This problem is endemic in this field, and it is integrated into the incentive system, the culture, and I’m starting to think even into human nature. All you can do is analyze very carefully to the absolute best of your abilities.
Find this podcast on:
Bennett's Blog 