A glass-half-full view of academic fraud in political science
[Via Monkey Cage]
Wednesday was interesting for political scientists. Our social media feeds were full of angst in response to the news that a very influential member of our discipline had requested a retraction of a very widely reported finding published by a very prestigious journal on which he had been a co-author. The data upon which the finding rested appear to have been fraudulently produced. Thus, a process of shaming has begun. It is a necessary process. Yet it misses a very important part of the story: science actually worked.
Not much political science research gets major coverage in outlets like Bloomberg, The Washington Post and “This American Life.” The now retracted finding did (here, here, and here), and that is partly because it was published in a journal that all scientists — not just social scientists — read. A retraction of an article published in such an outlet is major scientific news, and to the best of my knowledge, no political science article has ever been retracted from such a publication. And because some U.S. lawmakers oppose funding for political science research, people are particularly concerned that this “black eye” will contribute to such critiques.
“Do not fudge the data” is, of course, an important scientific norm. Public confidence in science rests in no small part upon our upholding it. So the news that the authors of one of the most widely disseminated findings our discipline has produced of late had violated that norm was met with consternation and concern. A political science study had joined the pantheon of famous academic frauds, including the 1989 cold fusion fraud, the 2011 retraction of the vaccine-autism study, and the 2013 case of serial fraud in social psychology.
The reaction to all of these cases is publish shaming. Shaming is the standard process by which human societies reproduce norms. Norms are most readily apparent when they are being violated, and if we want the norm to persist, large groups of us must raise the alarm and call out the violators for their poor behavior.
[More]
This is how the system is supposed to work. The fraud was revealed by other researchers and in pretty swift fashion. And social norms are used to make sure everyone in the community knows the penalty for fraud.
There are strong negative feedback loops to deal with fraud. I wish they were as strong in other arenas of our society.
Science worked because the research was openly published, the fraud was revealed by attempted replication and now, the most important part, shunning will be used to enforce social norms.
Not only is the career of the graduate student who made up the data destroyed (for example, he will likely never be able to get a Federal research grant) but the career of the senior researcher, who does not appear to be involved in the fraud, may well be damaged.
In fact the senior researcher may only escape universal opprobrium by having the paper retracted so swiftly. This is a plus in his favor. It was a paper in Science, something that does not happen often for anyone. He knows what the impact of the retraction will be on his career yet he swiftly did the right thing.
Good.
Richard Feynman talked about how the system is supposed to work in his commencement address Cargo Cult Science. Scientists are people, with their own faults, just like everyone else.
We’ve learned from experience that the truth will come out. Other experimenters will repeat your experiment and find out whether you were wrong or right. Nature’s phenomena will agree or they’ll disagree with your theory. And, although you may gain some temporary fame and excitement, you will not gain a good reputation as a scientist if you haven’t tried to be very careful in this kind of work.
Science creates models of the world based on data. The better the data, the more likely a good model describing the world will be supported.
If those data are wrong, it will be revealed as the model is simply not capable of accurately describing Nature. A model based on bad data will never be a good model and will eventually fall to models that are closer to reality.
“The truth will come out.” And what is fascinating is that what Feynman described in 1974 regarding things he had seen in the 40s still occur today.
Because scientists are people, with all the greatness and faults of everyone else. But science works because of the process, one that is social in nature.
We want good reputations. We fear being shunned. It is in our genes and in our communities, because those two approaches provided tremendous selective advantages to a new type of primate.
Most times, the data are wrong because the researchers allowed themselves to be fooled – confirmation bias. Feynman again:
The first principle is that you must not fool yourself–and you are the easiest person to fool. So you have to be very careful about that.
We see this in many communities, not just in science. Wishful thinking is a human trait and one that Feynman suggested researchers work hard to remove.
Now when this sort of foolishness does happen – and it always will because we are all human – everyone can recognize that good people can be led astray. In the case of cold fusion, for example, the equipment was not fully calibrated properly. Better equipment revealed the lack of the reaction.
But that can still damage the careers of anyone associated with the wrong data. They gain a reputation of not being a ‘good’ scientist. Thus the lesson for everyone is to be vigilant before the work is published.
Sometimes, though, outright fraud occurs. Here there is usually swift and harsh punishment. Not criminal – no one is unlikely to go to jail. But social – commit outright fraud and be kicked out of the community of researchers.
The negative implications of such shunning goes to the heart of being human. Being thrown out of a community they want to be a part is the greatest punishment for any researcher.
This is why science works, and why it is different from alchemy. Openness allows social norms to be applied to control maladaptive behavior, acts that harm the integrity of the community.
While scientific works are openly published for all to see and to replicate, it is the negative social aspects of being wrong that controls most aberrant behavior.
Social norms are powerful.
The science community uses the same social norms that every successful community should use to punish those who would game the system to hurt the community.
It has always been easy to fudge the data. Researchers have used all sorts of tricks to try and make their work more important or fit their personal theory.
That is human nature. Gain an edge by cheating.
But, just as with a baseball player caught using steroids, there has to be rapid and universal consequences of anyone cheating. Cheating in science goes to the core of the success we have seen over the last 400 years.
Researchers are human and some will try to seek advantage hoping no one notices. But the whole process of science requires that people notice.
Because if that fraud presents any important model for the world, it will be examined. And, in almost all cases, others will publish the real data, revealing the fraud.
Then. as here, everyone will see tremendous discordance with reality and the jig is up.
Heck, we still see discussion of possible fraud from research that is almost 200 years old. No important research will ever escape examination.
So the fraudster has to make a deadly calculus. Gain prestige from their fraudulent work – the reason to commit the fraud in the first place. But not make the work important enough for anyone to examine it and discover the fraud.
That is a balance seldom achieved. and why science works.
And an example of what makes humans such a successful species.
Image: Thomas Fisher Rare Book