by aslakr
[2b2k] Clay Shirky, info overload, and when filters increase the size of what’s filtered
[Via Joho the Blog]

Clay Shirky’s masterful talk at the Web 2.0 Expo in NYC last September — “It’s not information overload. It’s filter failure” — makes crucial points and makes them beautifully. [Clay explains in greater detail in this two part CJR interview: 1 2]

So I’ve been writing about information overload in the context of our traditional strategy for knowing. Clay traces information overload to the 15th century, but others have taken it back earlier than that, and there’s even a quotation from Seneca (4 BCE) that can be pressed into service: “What is the point of having countless books and libraries whose titles the owner could scarcely read through in his whole lifetime? That mass of books burdens the student without instructing…” I’m sure Clay would agree that if we take “information overload” as meaning the sense that there’s too much for any one individual to know, we can push the date back even further.

[More

David Weinberger has been one of my touchstones ever since I read The Cluetrain Manifesto. I cried when I read that book because it so simply rendered what I had achingly been trying to conceptualize.

Dealing with information glut today leverages an old way of doing things in a new way. It uses synthesis rather than analysis. Analysis gave us the industrial revolution. Breaking the complex down into small understandable bits allowed us to create the assembly line that could put together our greatest creations, such as the Space Shuttle, with more than 2.5 million parts.

Yet a single O-ring can destroy the whole thing.

Synthesis brings together facts, allows us to see them in new ways. But to attack the really complex problems of today, we need to utilize synthesis from a wide range of viewpoints, all providing their own filter. As with the story of the 5 blind men and the elephant, no one person has all the information. But a synthesis of everyone’s information provides a reasonable approximation.

David discusses this view:

A traditional filter in its strongest sense removes materials: It filters out the penny dreadful novels so that they don’t make it onto the shelves of your local library, or it filters out the crazy letters written in crayon so they don’t make it into your local newspaper. Filtering now does not remove materials. Everything is still a few clicks away. The new filtering reduces the number of clicks for some pages, while leaving everything else the same number of clicks away. Granted, that is an overly-optimistic way of putting it: Being the millionth result listed by a Google search makes it many millions of times harder to find that page than the ones that make it onto Google’s front page. Nevertheless, it’s still much much easier to access that millionth-listed page than it is to access a book that didn’t make it through the publishing system’s editorial filters.

It is through synthesis that new technologies allow us to deal with information glut. And this synthesis necessarily involves human social networks. Because humans are exquisitely positioned to filter out noise and find the signal.

I’ve discussed the DIKW model. Data simply exists. Information happens when humans interact with the data. Transformation of information, both tacit and explicit, produces knowledge, which is the ability to make a decision, to take an action. Often that action is to start the cycle again, generating more data and so on.

This can be quite analytical in approach as we try to understand something. But the final link in the cycle, wisdom, is the ability to make the RIGHT decision. This necessarily require synthesis.

New technologies allow us to deal with much more data than before, generate more information and produce more knowledge. However, without synthetic approaches that bring together a wide range of human knowledge, we will not gain the wisdom we need.

Luckily, the same technologies that produce so much data also provide us with the tools to leverage our interaction with knowledge. If we create useful social structures, ones that properly synthesize the knowledge, that employ human social networks that act as great filters, then we can more rapidly compete the DIKW cycle and take the correct actions.