Category Archives: Science

The latest fraud shows science working as expected

A glass-half-full view of academic fraud in political science
[Via Monkey Cage]

Wednesday was interesting for political scientists. Our social media feeds were full of angst in response to the news that a very influential member of our discipline had requested a retraction of a very widely reported finding published by a very prestigious journal on which he had been a co-author. The data upon which the finding rested appear to have been fraudulently produced. Thus, a process of shaming has begun. It is a necessary process. Yet it misses a very important part of the story: science actually worked.

Not much political science research gets major coverage in outlets like Bloomberg, The Washington Post and “This American Life.” The now retracted finding did (here, here, and here), and that is partly because it was published in a journal that all scientists — not just social scientists — read. A retraction of an article published in such an outlet is major scientific news, and to the best of my knowledge, no political science article has ever been retracted from such a publication. And because some U.S. lawmakers oppose funding for political science research, people are particularly concerned that this “black eye” will contribute to such critiques.

“Do not fudge the data” is, of course, an important scientific norm. Public confidence in science rests in no small part upon our upholding it. So the news that the authors of one of the most widely disseminated findings our discipline has produced of late had violated that norm was met with consternation and concern. A political science study had joined the pantheon of famous academic frauds, including the 1989 cold fusion fraud, the 2011 retraction of the vaccine-autism study, and the 2013 case of serial fraud in social psychology.

The reaction to all of these cases is publish shaming. Shaming is the standard process by which human societies reproduce norms. Norms are most readily apparent when they are being violated, and if we want the norm to persist, large groups of us must raise the alarm and call out the violators for their poor behavior.


This is how the system is supposed to work. The fraud was revealed by other researchers and in pretty swift fashion. And social norms are used to make sure everyone in the community knows the penalty for fraud.

There are strong negative feedback loops to deal with fraud. I wish they were as strong in other arenas of our society.

Science worked because the research was openly published, the fraud was revealed by attempted replication and now, the most important part, shunning will be used to enforce social norms.

Not only is the career of the graduate student who made up the data destroyed (for example, he will likely never be able to get a Federal research grant) but the career of the senior researcher, who does not appear to be involved in the fraud, may well be damaged.

In fact the senior researcher may only escape universal opprobrium by having the paper retracted so swiftly. This is a plus in his favor. It was a paper in Science, something that does not happen often for anyone. He knows what the impact of the retraction will be on his career yet he swiftly did the right thing.


Richard Feynman talked about how the system is supposed to work in his commencement address Cargo Cult Science. Scientists are people, with their own faults, just like everyone else.

We’ve learned from experience that the truth will come out. Other experimenters will repeat your experiment and find out whether you were wrong or right. Nature’s phenomena will agree or they’ll disagree with your theory. And, although you may gain some temporary fame and excitement, you will not gain a good reputation as a scientist if you haven’t tried to be very careful in this kind of work.

Science creates models of the world based on data. The better the data, the more likely a good model describing the world will be supported.

If those data are wrong, it will be revealed as the model is simply not capable of accurately describing Nature. A model based on bad data will never be a good model and will eventually fall to models that are closer to reality.

“The truth will come out.”  And what is fascinating is that what Feynman described in 1974 regarding things he had seen in the 40s still occur today. 

Because scientists are people, with all the greatness and faults of everyone else. But science works because of the process, one that is social in nature.

We want good reputations. We fear being shunned. It is in our genes and in our communities, because those two approaches provided tremendous selective advantages to a new type of primate.

Most times, the data are wrong because the researchers allowed themselves to be fooled – confirmation bias. Feynman again:

The first principle is that you must not fool yourself–and you are the easiest person to fool. So you have to be very careful about that.

We see this in many communities, not just in science. Wishful thinking is a human trait and one that Feynman suggested researchers work hard to remove.

Now when this sort of foolishness does happen – and it always will because we are all human – everyone can recognize that good people can be led astray. In the case of cold fusion, for example, the equipment was not fully calibrated properly. Better equipment revealed the lack of the reaction.

But that can still damage the careers of anyone associated with the wrong data. They gain a reputation of not being a ‘good’ scientist. Thus the lesson for everyone is to be vigilant before the work is published.

Sometimes, though, outright fraud occurs. Here there is usually swift and harsh punishment. Not criminal – no one is unlikely to go to jail. But social – commit outright fraud and be kicked out of the community of researchers.

The negative implications of such shunning goes to the heart of being human. Being thrown out of a community they want to be a part is the greatest punishment for any researcher.

This is why science works, and why it is different from alchemy. Openness allows social norms to be applied to control maladaptive behavior, acts that harm the integrity of the community.

While scientific works are openly published for all to see and to replicate, it is the negative social aspects of being wrong that controls most aberrant behavior.

Social norms are powerful.

The science community uses the same social norms that every successful community should use to punish those who would game the system to hurt the community. 

It has always been easy to fudge the data. Researchers have used all sorts of tricks to try and make their work more important  or fit their personal theory.

That is human nature. Gain an edge by cheating.

But, just as with a baseball player caught using steroids, there has to be rapid and universal consequences of anyone cheating. Cheating in science goes to the core of the success we have seen over the last 400 years.

Researchers are human and some will try to seek advantage hoping no one notices. But the whole process of science requires that people notice.

Because if that fraud presents any important model for the world, it will be examined. And, in almost all cases, others will publish the real data, revealing the fraud.

Then. as here, everyone will see tremendous discordance with reality and the jig is up.

Heck, we still see discussion of possible fraud from research that is almost 200 years old. No important research will ever escape examination. 

So the fraudster has to make a deadly calculus. Gain prestige from their fraudulent work  – the reason to commit the fraud in the first place. But not make the work important enough for anyone to examine it and discover the fraud. 

That is a balance seldom achieved. and why science works.

And an example of what makes humans such a successful species.

Image: Thomas Fisher Rare Book

The disruption of Health will be distributed

 DNA / Protein function finder from @WellcomeTrust @SangerInstitute @emblebi @YourGenome

Advances in health IT must be viewed as a whole
[Via O’Reilly Radar]

Reformers in health care claim gigantic disruption on the horizon: devices that track our movements, new treatments through massive data crunching, fluid electronic records that reflect the patient’s status wherever she goes, and even the end of the doctor’s role. But predictions in the area of health IT are singularly detached from the realities of the technical environment that are supposed to make them happen.

To help technologists, clinicians, and the rest of us judge the state of health IT, I’ve released a report titled “The Information Technology Fix for Health: Barriers and Pathways to the Use of Information Technology for Better Health Care.” It offers an overview of each area of innovation to see what’s really happening and what we need to make it progress further and faster.


Health has always been intimately connected with technology, from removing the handle of a drinking well to a handheld ultrasound wand.

Dealing with human health is probably the most complex system of endeavor mankind us currently trying to solve. Old, authoritarian, top-down approaches are giving way to newer, distributed, bottom-up paradigms.

And new digital tools are driving this.

Often we can only solve health problems because of the technological tools we have access to.

But healthcare has been slow to activate the greatest impacts of the digital revolution – to connect people and communities in ways to solve very complex problems. Healthcare’s natural attraction to the status quo for many medical needs– after all, if a doctor makes a mistake, people can die – means that therapeutic benefit often has to be shown BEFORE anything changes.

Patient healthcare data and its mining does not easily fit this paradigm. Authoritarian approaches stemming from the  medical edifice we all face still drives almost all our health concerns. So change is slow.

But it is coming. Faster than many of the authoritarian processes can deal with.

This has not stopped people from doing the mining themselves. From places like 23andme, patientslikeme to crowdfunding projects, individuals are now taking much greater control of their health data.

And finding out all sorts of interesting things.

Often without any form of mediator, because they can.

(It reminds me of the battle I was part of almost 20 years ago. We needed to connect to the internet because it was becoming critical in order to do our biomedical research. The IT department was very reluctant and stonewalled, due to fear of disrupting things. We simply said we could connect to the internet without needing them. Our IT needs had become decentralized , distributed, and we could simply dial-up without needing the IT department at all. So we did.)

Medicine is becoming a more distributed system, decentralizing access and the practice of medicine. It is, in many way, at right angles and in conflict with the authoritarian processes we find in medicine today.

This holds tremendous opportunity to revolutionize what we know about medicine. But with tremendously disruptive effects on the status quo.

There will be real battles here but the conflict between the old, authoritarian system and the new, distributed system will find a balance which eventually helps us all.

Because it has real benefits. And it cannot really be stopped anyway.

Changing the course of health care investment

rocket launch

Lean LaunchPad |
[Via  Steve Blank]

We’re deep into week 2 of teaching a Lean LaunchPad class for Life Sciences and Health Care (therapeutics, diagnostics, devices and digital health) this October at UCSF with a team of veteran venture capitalists. Part 1 of this post described the issues in the drug discovery. Part 2 covered medical devices and digital health. Part 3  described what we’re going to do about it. This is post is a brief snapshot of our progress.


I have been following Steve Blank’s journey into disrupting the way healthcare companies are funded off and on. He has brought the ‘Moneyball’ approach to funding, by using data rather than gut feelings.

As any good scientist does, he wants to do n experiment – can he teach a class on these principles to help startups get funding in a more defined way. And to change the VC industry, especially in healthcare, from looking only for home runs and perhaps consider that 3-4 singles will accomplish the same things.

As I have written before, the cost to get drugs to market today is simply too long and expensive. We must find better approaches. And this approach might just help us get there.

As with any experiment, it may not explain everything. But, as this video shows, it can help us create a better model for success.

Fundraising for our #CrowdGrant project, Consider the Facts, is almost over

Asteroid crowd funding 001

Consider the Facts: moving people to deliberative thinking is an experiment in several ways. The project itself involves experimentation to examine a hypothesis.

But the process of raising money for research through crowdfunding is also an experiment. Can a group of people in the public sphere create, vet and support active scientific research? What is required to make that happen?

SpreadingScience has learned a lot and has some answers to those questions. One final one is whetehr we will reach our goal.

$25 would help a lot. Please help us answer that question.

#CrowdGrant 23 more days to Consider the Facts

1 26 2011google

So much going on that this blog has not been updated for awhile. Consider the Facts is a fun experiment into crowdfunding research projects. It will provide the tools to examine how we might change people’s views about the future.

We have some things to do during the dead time the efforts are now in – people usually only focus their interest the first week or so and the last week or so.

The middle month is not the most interesting.

But go take a look at the site. we have some drafts of the illustrations for the short narrative.

Running a #CrowdGrant project, like Consider the Facts, can be hard work

Happy face 042

Consider the Facts is the most successful finalist in the #CrowdGrant Challenge sponsored by RocketHub and Popular Science so far. That did not just happen.

Crowdfunding projects usually succeed because they activate a community to action. Maybe it’s fans of a TV show. Or space enthusiasts who want to send up a satellite.

If that community is not already there, then it must be created. That takes some real time to figure out ways to get the word out. Comunities do not often spontaneously arise. It taks a lot fo work finding and nurturing those contacts.

People want to help but they also want to see how they might be helped. 

I’ve been planning basic research projects for some time, looking to create a community that will create, vet and support research projects independent of academia. What happens when people with good questions can get them answered?

So I had planned on doing something small, start with friends and family, and bootstrap myself to a community. Then this opportunity to work with Popular Science came along.

Now I can experiment to see if Popular Science’s Community might help help create this sort of a community faster.

We shall see.

Our #crowdgrant project is number 1. 14% of the way there. Will an asteroid save us all?

[iframe src=”” allowtransparency=”true” frameborder=”0″ scrolling=”no” width=”288″ height=”416]

I have been juggling a lot since the launch. Keeping all the social media on board can be tricky, especially since this project is an experiment.

Historically using stories—ones that engage rapid, rules of thumb thinking first and create a counterintuitive reaction—has been a way to teach people complex social lessons.

Can it be used to teach them complex scientific lessons? That is what Consider the Facts hopes to find out. To do that, we need some tools. That is why we need your help.

Consider the Facts wants to answer a basic question: Can using a modern, positive fable move people to utilize more of their slow deliberative thinking in order to engage complex problems?

Aesop’s fables, Christ’s parables and Kipling’s Just-So Stories all used the method of presenting complex ideas within a paradoxical story: “the tortoise is faster than the hare?”, “the Samaritan is good?” or “the alligator gave the elephant its trunk”?

“Really? Is that true? Let me think about that.”

All these stories make people stop using their rapid System 1 thinking (as discussed by Daniel Kahneman in his book Thinking, Fast and Slow) and utilize their System 2 as they reorient their rules of thumb to reintegrate a new view of the world surrounding them.

“I see, under some circumstances the hare could be slower than the tortoise. Good to know.” “We should not prejudge people because sometimes ‘bad’ people will do good things. Good to know.” “No way did the alligator pull the elephant’s nose and make it long. But I wonder how the trunk did form?”

In particular, the framing of Kipling’s stories actually invites deeper examination of scientific problems, not just social ones. It is this particular process that Consider the Facts will attempt to explore – can we move people to think deeply and slowly about science, not just social norms?

Scientists are trained to use slow thinking as a necessary part of their job. I cannot read any paper without dropping into System 2 thinking in order to deeply examine the data: “Do those numbers really add up?” “Does that figure actually show what the paper discusses?” “Do their procedures actually produce the results? Do the conclusions match what was described?”

System 1 thinking would read the abstract and come up with “Cancer is cured” or “Fats cause high cholesterol”. Sound familiar?

That is because most of our mainstream approaches that move information around use System 1 methods. That is why headlines are so important. Most people live in a System 1 world.

That makes sense.You had better know what to do instantly when confronted with a lion. Or wonder if that plant is good to eat. In a stable environment, System 1 thinking does a pretty good job of simulating the world. It’s good enough.

But in our complex, rapidly changing world, things are different. Our social environment is not stable. Changing technology destroys rules of thumb. System 1approaches are maladaptive, They lead people away from reality, they put people into Cargo Cult Worlds whose simulation of reality is so poor as to be dangerous.

Their rules of thumb no longer work. This results in the sort of  future shock described by Toffler. People can see that their rules of thumb do not seem to be working. Most, instead of dropping into System 2 and working things out, refuse to integrate any more information at all. They refuse to use System 2 in a way to move forward effectively.

Research shows that giving people more facts does not move them towards deliberative thinking. In truth, many people retreat even further into their Cargo Cult worlds, ignoring or rationalizing away any facts that contradict their rules of thumb. They will actually forget facts if those facts contradict their Cargo Cult World.

Shouting at them or lecturing them actually produces the opposite reaction from what is desired.

Perhaps telling them a story will.