Category Archives: Cargo Cult Worlds

The latest fraud shows science working as expected

A glass-half-full view of academic fraud in political science
[Via Monkey Cage]

Wednesday was interesting for political scientists. Our social media feeds were full of angst in response to the news that a very influential member of our discipline had requested a retraction of a very widely reported finding published by a very prestigious journal on which he had been a co-author. The data upon which the finding rested appear to have been fraudulently produced. Thus, a process of shaming has begun. It is a necessary process. Yet it misses a very important part of the story: science actually worked.

Not much political science research gets major coverage in outlets like Bloomberg, The Washington Post and “This American Life.” The now retracted finding did (here, here, and here), and that is partly because it was published in a journal that all scientists — not just social scientists — read. A retraction of an article published in such an outlet is major scientific news, and to the best of my knowledge, no political science article has ever been retracted from such a publication. And because some U.S. lawmakers oppose funding for political science research, people are particularly concerned that this “black eye” will contribute to such critiques.

“Do not fudge the data” is, of course, an important scientific norm. Public confidence in science rests in no small part upon our upholding it. So the news that the authors of one of the most widely disseminated findings our discipline has produced of late had violated that norm was met with consternation and concern. A political science study had joined the pantheon of famous academic frauds, including the 1989 cold fusion fraud, the 2011 retraction of the vaccine-autism study, and the 2013 case of serial fraud in social psychology.

The reaction to all of these cases is publish shaming. Shaming is the standard process by which human societies reproduce norms. Norms are most readily apparent when they are being violated, and if we want the norm to persist, large groups of us must raise the alarm and call out the violators for their poor behavior.

[More]

This is how the system is supposed to work. The fraud was revealed by other researchers and in pretty swift fashion. And social norms are used to make sure everyone in the community knows the penalty for fraud.

There are strong negative feedback loops to deal with fraud. I wish they were as strong in other arenas of our society.

Science worked because the research was openly published, the fraud was revealed by attempted replication and now, the most important part, shunning will be used to enforce social norms.

Not only is the career of the graduate student who made up the data destroyed (for example, he will likely never be able to get a Federal research grant) but the career of the senior researcher, who does not appear to be involved in the fraud, may well be damaged.

In fact the senior researcher may only escape universal opprobrium by having the paper retracted so swiftly. This is a plus in his favor. It was a paper in Science, something that does not happen often for anyone. He knows what the impact of the retraction will be on his career yet he swiftly did the right thing.

Good.

Richard Feynman talked about how the system is supposed to work in his commencement address Cargo Cult Science. Scientists are people, with their own faults, just like everyone else.

We’ve learned from experience that the truth will come out. Other experimenters will repeat your experiment and find out whether you were wrong or right. Nature’s phenomena will agree or they’ll disagree with your theory. And, although you may gain some temporary fame and excitement, you will not gain a good reputation as a scientist if you haven’t tried to be very careful in this kind of work.

Science creates models of the world based on data. The better the data, the more likely a good model describing the world will be supported.

If those data are wrong, it will be revealed as the model is simply not capable of accurately describing Nature. A model based on bad data will never be a good model and will eventually fall to models that are closer to reality.

“The truth will come out.”  And what is fascinating is that what Feynman described in 1974 regarding things he had seen in the 40s still occur today. 

Because scientists are people, with all the greatness and faults of everyone else. But science works because of the process, one that is social in nature.

We want good reputations. We fear being shunned. It is in our genes and in our communities, because those two approaches provided tremendous selective advantages to a new type of primate.

Most times, the data are wrong because the researchers allowed themselves to be fooled – confirmation bias. Feynman again:

The first principle is that you must not fool yourself–and you are the easiest person to fool. So you have to be very careful about that.

We see this in many communities, not just in science. Wishful thinking is a human trait and one that Feynman suggested researchers work hard to remove.

Now when this sort of foolishness does happen – and it always will because we are all human – everyone can recognize that good people can be led astray. In the case of cold fusion, for example, the equipment was not fully calibrated properly. Better equipment revealed the lack of the reaction.

But that can still damage the careers of anyone associated with the wrong data. They gain a reputation of not being a ‘good’ scientist. Thus the lesson for everyone is to be vigilant before the work is published.

Sometimes, though, outright fraud occurs. Here there is usually swift and harsh punishment. Not criminal – no one is unlikely to go to jail. But social – commit outright fraud and be kicked out of the community of researchers.

The negative implications of such shunning goes to the heart of being human. Being thrown out of a community they want to be a part is the greatest punishment for any researcher.

This is why science works, and why it is different from alchemy. Openness allows social norms to be applied to control maladaptive behavior, acts that harm the integrity of the community.

While scientific works are openly published for all to see and to replicate, it is the negative social aspects of being wrong that controls most aberrant behavior.

Social norms are powerful.

The science community uses the same social norms that every successful community should use to punish those who would game the system to hurt the community. 

It has always been easy to fudge the data. Researchers have used all sorts of tricks to try and make their work more important  or fit their personal theory.

That is human nature. Gain an edge by cheating.

But, just as with a baseball player caught using steroids, there has to be rapid and universal consequences of anyone cheating. Cheating in science goes to the core of the success we have seen over the last 400 years.

Researchers are human and some will try to seek advantage hoping no one notices. But the whole process of science requires that people notice.

Because if that fraud presents any important model for the world, it will be examined. And, in almost all cases, others will publish the real data, revealing the fraud.

Then. as here, everyone will see tremendous discordance with reality and the jig is up.

Heck, we still see discussion of possible fraud from research that is almost 200 years old. No important research will ever escape examination. 

So the fraudster has to make a deadly calculus. Gain prestige from their fraudulent work  – the reason to commit the fraud in the first place. But not make the work important enough for anyone to examine it and discover the fraud. 

That is a balance seldom achieved. and why science works.

And an example of what makes humans such a successful species.

Image: Thomas Fisher Rare Book

Too many videos in a presentation – forgetting that the audience is there to see YOU, not a TED talk video

GiardinaKARLSRUHE - Death by Powerpoint 

Think including a TED talk video can enhance your Powerpoint or Keynote presentation? Think again and think like a magician
[Via Les Posen’s Presentation Magic]

Last month, June 2014, I was part of a nationwide travelling convention intended to bring high school teachers up to date with the field of Positive Psychology and what it can offer students.

This year its organisers wanted to pay particular attention to the place of Technologies in mental health, and so I was tapped to offer a series of talks as well as presentation skills workshops.

The talks were part of a seminar featuring how to best understand how current and imminent technologies have a role to play in mental health in schools. The presentation skills effort was a one hour talk, showcasing the highlights of my half day and longer workshops.

All in all, there were about a dozen speakers, comprised of authors, psychologists, researchers and technologists including those from the Australian arms of Microsoft and Google, who visited the cities of Brisbane, Perth, Melbourne and Sydney.

The arrangement, as told to me by the organisers, was that each city would receive the same presentations. On the surface, that sounds ideal and easy: the same talk four times. But as it turns out for a few of the presenters, including myself, this wasn’t ideal and in fact we gave variations for all of our presentations. It was an iterative process, learning from each presentation what worked best and which slides and ideas appealed to the audience.

By the fourth conference I felt I had “honed” my presentations and delivered the “biggest bang for the buck”; that is, in the time I had these were my most impactful presentations.

During each of the conferences, conducted over two days including large auditoriums and break out rooms for smaller concurrent workshops, I was able to attend as an audience member and watch others in action.

For each conference, there were times when all the attendees (400+) would gather in one large auditorium to hear the speakers, including me on technologies.

What I, and the organisers, found interesting were those highly paid professional speakers who gave the same presentation each time. I was perhaps the least known to the audience of all the presenters and likely the least financially compensated, so I had to prove myself and win over the audiences with my content and presentation style. Which is why I welcomed the opportunity to present and improve each time.

[More]

Too many people, either due to fear or inexperience, forget that it is the personal interaction – you are THERE – that explains the audience’s presence. Otherwise they could just watch the video online.

Anything on the screen has to enhance your presence. not detract from it. That is what Powerpoint bullet lists are deadly. Focus moves away from the speaker to a static list, often simply reread out loud by the speaker.

Why even have them there?

If the speaker cannot provide a reason why their personal presence is important, why should the audience?

Videos need to be short and provide an easy to understand visual to support the speaker. Why show someone else speaking? That generally implies laziness, not exactly what an audience wants to see.

Even in scientific presentations, the speaker is there to provide context, not to read data out loud. To be personal.

Balance – we need both authoritarian hierarchies and distributed democracies

 

 Complexity of Life

How do we shift to a more agile organization? Podio a Case Study
[Via Robert Paterson’s Weblog]

Most people would agree that many organizations today are too stiff, too slow and too disconnected to do well in the complex world we live in now. 

Whymachineorgscannotcope

Many large organizations have placed their bet on a new technology platform that will connect all their people’s work. Some think that real change can only come from the bottom up. Many feel that any form of hierarchy is outdated. Some talk about culture but are not clear about what this means.

Few are making any progress. So what is the better way to go? 

[More]

Great discussion. We are out of balance dealing with complex problems because authoritarian hierarchies –so important for 20th century processes – are seen as the only way to get things done.  Maybe for simply processes but not the complex ones facing us.

Distributed democratic approaches using social networks are all the rage. For the first time in 10,000 years we have major tools that leverage these inherent activities of humanity’s culture. They can now overpower hierarchies especially when examining complex processes.

But, they alone cannot solve what we face. Disctibuted democracy is great at cranking the DIKW cycle to get to knowledge. The problem arises because they often want to keep turning the cycle than actually take an action.

They can spend too much time talking and not enough time doing. I’ve written about the need for a Synthetic Organization, one that is leader-full bit leaderless. 

We need some aspects of hierarchy to get things done. It is finding the right balance, designing feedback to permit leader-full approaches to survive while preventing the accretion of power that hierarchies can produce.

I have worked at organizations that found the right balance. We just did not have a firm understanding of why it worked.

Now we are getting much closer to defining how to create the balance between the two key aspects of human social interaction – authoritarian hierarchy and distrubuted democracy.

The groups that accomplish this will be the ones that truly helpus solve complex problems.

Truth invites trust. Lying invites apprehension.

 Leaves with drop of water

Complete Honesty is the Access to Ultimate Power
[Via Rands in Repose]

Rebekah Campbell via the New York Times:

A study by the University of Massachusetts found that 60 percent of adults could not have a 10-minute conversation without lying at least once. The same study found that 40 percent of people lie on their résumés and a whopping 90 percent of those looking for a date online lie on their profiles. Teenage girls lie more than any other group, which is attributed to peer pressure and expectation. The study did not investigate the number of lies told by entrepreneurs looking for investment capital, but I fear we would top the chart.

Also:

Peter maintains that telling lies is the No. 1 reason entrepreneurs fail. Not because telling lies makes you a bad person but because the act of lying plucks you from the present, preventing you from facing what is really going on in your world. Every time you overreport a metric, underreport a cost, are less than honest with a client or a member of your team, you create a false reality and you start living in it.

Drink a cup of coffee before reading this one.

[More]

Telling th truth, especially when it exposes errors you have made, can make you feel vulnerable. Especially with so many sociopathic people running things.

Looking weak is not a plus for sociopaths.

But telling lies creates a disconnect from reality, a Cargo Cult World that leads away from what actually happens. It invites more lies that divide you even more from reality.

Eventually you inhabit a world that not only does not exist but can actually prevent you from thriving in the real world.

Too many people today simply construct unsustainable Cargo Cult worlds to inhabit by lying to themselves and others. They will fall.

Here the author shows how even in business setting, telling the truth is best. Because Nature always wins. 

Perhaps by making learning easier, Apple leads the way

appleby Stephen A. Wolfe

The diffusion of iPhones as a learning process
[Via asymco]

All theoretical and empirical diffusion studies agree that an innovation diffuses along a S-shaped trajectory. Indeed, the S-shaped pattern of diffusion appears to be a basic anthropologic phenomenon.

This observation dates as far back as 1895 when the French sociologist Gabriel Tarde first described the process of social change by an imitative “group-think” mechanism and a S-shaped pattern.[1] In 1983 Everett Rogers, developed a more complete four stage model of the innovation decision process consisting of: (1) knowledge, (2) persuasion, (3) decision and implementation, and (4) confirmation.

Consequently, Rogers divided the population of potential adopters according to their adoption date and categorized them in terms of their standard deviation from the mean adoption date. He presented extensive empirical evidence to suggest a symmetric bell shaped curve for the distribution of adopters over time. This curve matches in shape the first derivative of the logistic growth and substitution curve as shown below. Screen Shot 2013-11-06 at 11-6-1.51.57 PM

In the graph above I applied the Rogers adopter characterization to the data we have on the adoption of smartphones in 

[More]

This is a very useful analysis of the way smartphones are diffusing throughout the US. I’ve written about the diffusion of innovation throughout a community many times and it is nice to see that smartphones are following the same curve.

Now, this post makes the point that the speed of adoption entails a learning stage. There have been 5 stages postulated in the personal adoption of something new: Awareness, Interest, Evaluation, Trial and Adoption.

Where someone falls along the adoption curve depends on how fast one moves through each stage. Innovators move very rapidly. The middle takes more time. In fact, they usually get stuck at the evaluation stage. They wait the thought leaders in the early adopter group to help them change.

Notice that the adoption of an innovation is slow until about 16% have made the shift. Then you see explosive and rapid growth, once the early adopters are on board.

So the faster the early adopters can evaluate and learn about the innovation, the faster it will spread. Perhaps by Apple making it easy to learn, especially for the thought leaders , allowed it to rapidly spread throughout a community. 

Other phone makers, whose platform was not as easy to evaluate and learn, suffer from churn as the evaluation process becomes muddy and undirected.

By making the evaluation process easier, Apple makes it more likely that the necessary thought leaders will convince the rest of the community to shift. and see explosive growth.

This explains why the smartphone took off so fast once Apple released the iPhone and why everyone else copied them. The same thing happened with the iPad, while Microsoft had no luck with its tablets for years.

The key step to rapid adoption is not just cool technology. It must be made very easy for the critical early adopters to evaluate. That is Apple’s real innovation.


Running a #CrowdGrant project, like Consider the Facts, can be hard work

Happy face 042

Consider the Facts is the most successful finalist in the #CrowdGrant Challenge sponsored by RocketHub and Popular Science so far. That did not just happen.

Crowdfunding projects usually succeed because they activate a community to action. Maybe it’s fans of a TV show. Or space enthusiasts who want to send up a satellite.

If that community is not already there, then it must be created. That takes some real time to figure out ways to get the word out. Comunities do not often spontaneously arise. It taks a lot fo work finding and nurturing those contacts.

People want to help but they also want to see how they might be helped. 

I’ve been planning basic research projects for some time, looking to create a community that will create, vet and support research projects independent of academia. What happens when people with good questions can get them answered?

So I had planned on doing something small, start with friends and family, and bootstrap myself to a community. Then this opportunity to work with Popular Science came along.

Now I can experiment to see if Popular Science’s Community might help help create this sort of a community faster.

We shall see.

Our #crowdgrant project is number 1. 14% of the way there. Will an asteroid save us all?

[iframe src=”http://www.rockethub.com/projects/28741-consider-the-facts-moving-people-to-deliberative-thinking/widgets/panel” allowtransparency=”true” frameborder=”0″ scrolling=”no” width=”288″ height=”416]

I have been juggling a lot since the launch. Keeping all the social media on board can be tricky, especially since this project is an experiment.

Historically using stories—ones that engage rapid, rules of thumb thinking first and create a counterintuitive reaction—has been a way to teach people complex social lessons.

Can it be used to teach them complex scientific lessons? That is what Consider the Facts hopes to find out. To do that, we need some tools. That is why we need your help.

Consider the Facts wants to answer a basic question: Can using a modern, positive fable move people to utilize more of their slow deliberative thinking in order to engage complex problems?

Aesop’s fables, Christ’s parables and Kipling’s Just-So Stories all used the method of presenting complex ideas within a paradoxical story: “the tortoise is faster than the hare?”, “the Samaritan is good?” or “the alligator gave the elephant its trunk”?

“Really? Is that true? Let me think about that.”

All these stories make people stop using their rapid System 1 thinking (as discussed by Daniel Kahneman in his book Thinking, Fast and Slow) and utilize their System 2 as they reorient their rules of thumb to reintegrate a new view of the world surrounding them.

“I see, under some circumstances the hare could be slower than the tortoise. Good to know.” “We should not prejudge people because sometimes ‘bad’ people will do good things. Good to know.” “No way did the alligator pull the elephant’s nose and make it long. But I wonder how the trunk did form?”

In particular, the framing of Kipling’s stories actually invites deeper examination of scientific problems, not just social ones. It is this particular process that Consider the Facts will attempt to explore – can we move people to think deeply and slowly about science, not just social norms?

Scientists are trained to use slow thinking as a necessary part of their job. I cannot read any paper without dropping into System 2 thinking in order to deeply examine the data: “Do those numbers really add up?” “Does that figure actually show what the paper discusses?” “Do their procedures actually produce the results? Do the conclusions match what was described?”

System 1 thinking would read the abstract and come up with “Cancer is cured” or “Fats cause high cholesterol”. Sound familiar?

That is because most of our mainstream approaches that move information around use System 1 methods. That is why headlines are so important. Most people live in a System 1 world.

That makes sense.You had better know what to do instantly when confronted with a lion. Or wonder if that plant is good to eat. In a stable environment, System 1 thinking does a pretty good job of simulating the world. It’s good enough.

But in our complex, rapidly changing world, things are different. Our social environment is not stable. Changing technology destroys rules of thumb. System 1approaches are maladaptive, They lead people away from reality, they put people into Cargo Cult Worlds whose simulation of reality is so poor as to be dangerous.

Their rules of thumb no longer work. This results in the sort of  future shock described by Toffler. People can see that their rules of thumb do not seem to be working. Most, instead of dropping into System 2 and working things out, refuse to integrate any more information at all. They refuse to use System 2 in a way to move forward effectively.

Research shows that giving people more facts does not move them towards deliberative thinking. In truth, many people retreat even further into their Cargo Cult worlds, ignoring or rationalizing away any facts that contradict their rules of thumb. They will actually forget facts if those facts contradict their Cargo Cult World.

Shouting at them or lecturing them actually produces the opposite reaction from what is desired.

Perhaps telling them a story will. 

Big data is still just data

bigdata by BBVAtech

Big data vs. big reality
[Via O’Reilly Radar]

This post originally appeared on Cumulus Partners. It’s republished with permission.

Quentin Hardy’s recent post in the Bits blog of The New York Times touched on the gap between representation and reality that is a core element of practically every human enterprise. His post is titled “Why Big Data is Not Truth,” and I recommend it for anyone who feels like joining the phony argument over whether “big data” represents reality better than traditional data.

In a nutshell, this “us” versus “them” approach is like trying to poke a fight between oil painters and water colorists. Neither oil painting nor water colors are “truth”; both are forms of representation. And here’s the important part: Representation is exactly that — a representation or interpretation of someone’s perceived reality. Pitting “big data” against traditional data is like asking you if Rembrandt is more “real” than Gainsborough. Both of them are artists and both painted representations of the world they perceived around them.

[More]

Data by itself has no meaning. It does not if it is big or traditional. Data simply exists.

It requires interaction with human beings to be transformed into information, humans to provide context, humans to provide understanding. It requires interactions between human being to transform data into information and beyond onto knowledge.

As I wrote “Information that is held by an individual, which is never revealed or acted upon, has no value. The greatest medical discovery in the world does little good if it dies with the discoverer.”

All big data is allow humans to examine data that is too large, too complex or too difficult to examine by traditional means.

But the problems with any data – confirmation bias, cherry-picking, etc. – do not simply go away because the data is big. It still requires humans to transform this data into meaningful knowledge.

That still requires open and transparent communication between people to function best.