Information overload is NOT filture failure:
This has been bothering me for a while now, dating back to last year, when I first heard Clay Shirky’s very pithy statement that information overload isn’t a real problem, the real problem is a failure to build effective filters. It’s a catchy little phrase, and like most theories from Web 2.0 gurus, it seems reasonable on the surface, but when applied to the world of scientists, it’s less than useful.
[More]
Shirky has a habit of making pithy statements. I often disagree but I have to say they lead to some interesting conversations, so I listen to what he says. He forces one to concentrate.
David is someone else I listen to. His perspective is often different than mine but it is one well worth examining. How do we deal with more information than we can individually examine? How do we figure out a way to separate the wheat from the chaff when we have no way to examine it all?
The Web is not going to replace methods of information dispersal that have stood in good stead for many years. Publication in highly regarded journals will always be an important avenue. It will most likely always be the place for the interesting stuff.
The key is not that there is more interesting stuff out there than we can read in a lifetime. The problem is that the interesting stuff is overrun with extraneous, uninteresting stuff. Important papers also get published in obscure journals. How do we find them?
In Shirky’s example, if the entire contents of a Barnes and Noble is dumped in front of us, the good stuff (i.e. Auden and Plato) will be overwhelmed with the irrelevant.
The library system has come up with some ways to help. But even characterizing books by topic does not really produce a solution. In many ways, finding the good stuff is dependent on social mechanisms. We read a review in a magazine. A friend tells us about a great new read. A speaker quotes from someone. A teacher points the way. Another book discusses the thoughts of a prior author.
Human beings act as filters to help us deal with information overload. Our social network helps us find the information relevant to us.
Similarly, in my research, I have been led to more of the important papers for my work by someone I trust saying “Hey, I read this article you might like” than I have by scouring PubMed. I see a presentation at a conference and ask the speaker a question. His answer leads me to another paper. I have a beer with a colleague who mentions an interesting paper. I read a review article and use the references to find the paper with the protocols I need.
All parts of my social network.
Published papers are not going to go away. The vetting possible by peer review is a requirement for a certain type of scientific work. Random strangers will have little impact on this.
But leveraging online social tools so that a community of scientists can more rapidly find the important papers it needs could possible create a filtering mechanism that can help deal with some of the information overload.
Personally I feel that these online communities will be more informal in nature than Shirky does. That is, they will more likely arise from a group of colleagues working together than from an organized committee. Time will tell.
Technorati Tags: Social media, Web 2.0
What I’m trying to say is that it’s not that hard to find interesting papers–we have lots of methods for this, both online and off. The problem is that there is a real “information overload”, that there are too many interesting and valuable papers that directly impact upon our research. There’s more one needs to know, there’s more one needs to follow. It can’t just be filtered out and discarded, it must be assimilated.
And that’s the real “information overload”, and it can’t be fixed by any sort of filtering. More research is being done than ever, more is being published than ever. Shouldn’t it be obvious that it’s harder to keep up with the good stuff since there’s more of it?
I’ve added a Part 2 addendum to the posting, hopefully clarifying my point:
http://www.cshblogs.org/cshprotocols/2009/01/16/information-overload-part-2/