Wisdom of Crowds

This forum contains all archives from the SD Mailing list (go to http://www.systemdynamics.org/forum/ for more information). This is here as a read-only resource, please post any SD related questions to the SD Discussion forum.
Locked
Kim Warren Kim strategydynamics.
Junior Member
Posts: 16
Joined: Fri Mar 29, 2002 3:39 am

Wisdom of Crowds

Post by Kim Warren Kim strategydynamics. »

Posted by ""Kim Warren"" <Kim@strategydynamics.com>
Just had a question put to me - has anyone looked at the linkage [or
conflict?] between a system dynamics view of the world and the popular
idea that 'the wisdom of crowds' can give a better view of than experts
or formal analysis.

Anyone given this any thought?

[ The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How
Collective Wisdom Shapes Business, Economies, Societies and Nations by
James Surowiecki

http://www.amazon.co.uk/exec/obidos/sea ... 74-7917206


Kim Warren | Strategy Dynamics Solutions Ltd
Posted by ""Kim Warren"" <Kim@strategydynamics.com>
posting date Tue, 7 Jun 2005 14:34:10 +0100
Fabian Fabian f_fabian yahoo.com
Junior Member
Posts: 3
Joined: Fri Mar 29, 2002 3:39 am

Wisdom of Crowds

Post by Fabian Fabian f_fabian yahoo.com »

Posted by Fabian Fabian <f_fabian@yahoo.com>
Hello Kim,

I have recently read about such a phenomenon/trend in a consumer
(in this case peer to peer) behaviour trend in :

http://www.trendwatching.com/trends/TWINSUMER.htm

I am not sure if this example of emergent collective wisdom (in this
case for optimizing the shopping experience) is aligned with your
understanding of collective wisdom.

On the other side, I am not sure if collective wisdom is a way of
analyzing or if it is an emergent phenomenon, nor if it is better
or worse than System Dynamics, in fact, I am sure that a SD model
of this kind of collective wisdom could be built.

Please let me know if I am missing some important point.

Be well...

Fabian Szulanski
Professor, System Dynamics
fszulans@itba.edu.ar
Posted by Fabian Fabian <f_fabian@yahoo.com>
posting date Wed, 8 Jun 2005 09:15:22 -0700 (PDT)
John Gunkler jgunkler sprintmail
Member
Posts: 30
Joined: Fri Mar 29, 2002 3:39 am

Wisdom of Crowds

Post by John Gunkler jgunkler sprintmail »

Posted by ""John Gunkler"" <jgunkler@sprintmail.com>
The notion that small groups of people are smarter than individuals has
certainly shown to be true in my 25 years of testing it.

I have been running group sessions over my career in which I break up the
participants into small groups (usually four to six people per group) and
give them a questionnaire to answer. I've used two very different topics:
One is a quiz about organizational culture and norms, the other is an
artificial situation where I provide them a short, one-paragraph description
of an event then ask them questions about it. In each case there are three
columns for answers: I first have people fill in the answers by themselves;
then I have them get together in their small groups and listen to each
others' answers (without taking notes), then return to their places and
individually answer the questions again; finally, I have them convene their
small groups and come up with a single, consensus answer to each question
(which everyone then fills in for column three.)

In every case, over 25 years, on average the third method (""consensus"") has
always had the highest percent correct answers for the room as a whole, the
second method (""consultative"") has the next best result, and the first
method (""individual"") does worst. Of course, there are occasional times
when a small group shows a reversal in this pattern -- usually because they
failed to use true consensus methods and let one person dominate the
discussion -- but I have never had the overall results for the entire large
group fail to show this pattern, with either kind of questionnaire!

I would think, from a System Dynamics perspective, one would look at some of
the following contributions to ""wisdom"":

* Amount of time spent thinking about a response (note: it's larger for the
consensus method than for the other two.)
* Amount of relevant information available to the responder (it's the same
(maybe?) for consensus and consultation, lower for individual.)
* Opportunity to have one's thinking challenged (highest for consensus, next
for consultation, lowest for individual.)
* Rigidity of opinion (a negative factor: highest for individual, lower for
consultation, lowest because of the rules for consensus.)
* ???
Posted by ""John Gunkler"" <jgunkler@sprintmail.com>
posting date Wed, 8 Jun 2005 11:00:56 -0500
sheldon.friedman comcast.net
Newbie
Posts: 1
Joined: Fri Mar 29, 2002 3:39 am

Wisdom of Crowds

Post by sheldon.friedman comcast.net »

Posted by sheldon.friedman@comcast.net
Hi
I noticed the discussion on the wisdom of crowds and the use of groups.
Part of this wisdom may come from what has ben called transactive memory.
People sharing bits if memory of information they all experienced. On the
other hand there is the concept of hiddent profiles where members do not
share information because they assume the information they have is not
important to a problem. Group members talk about what they all share as
knowledge , not unique information.

Shelly Friedman
Posted by sheldon.friedman@comcast.net
posting date Thu, 09 Jun 2005 15:15:16 +0000
Tim Scheffmann Tim.Scheffmann we
Newbie
Posts: 1
Joined: Fri Mar 29, 2002 3:39 am

Wisdom of Crowds

Post by Tim Scheffmann Tim.Scheffmann we »

Posted by ""Tim Scheffmann"" <Tim.Scheffmann@web.de>

Hi everyone,

I read a very interesting article about swarm intelligence. These creepy
little armies are very robust and their organization structure is nearly
unbreakable. There might be the link to the system dynamics methodology to
strengthen system structures (rather than to predict the future). I attached
an abstract from my master thesis about ""Enterprise Risk Dynamics""(please
find this further down).

On the Boston conference I read about a workshop with the title: Agent-based
Models for Crowd Dynamics
(#484)"".(http://www.systemdynamics.org/conf2005/ ... ummary.htm)
Here might be another link to how to model such behavior.

Cheers, Tim

Tim Scheffmann
IBM BCS Consulting
+49-172 - 7929242
Tim@Scheffmann.de

The abstract from my master thesis:
""Swarm intelligence"", a concept introduced to business optimisation from
scientific observa-tion, can give further insight into how an organisation
can be strengthened and how complex matters can be handled. Swarm
intelligence is based on the system of order that prevails in social insect
colonies, for example ant colonies. There is no one ant in charge and
individual ants seems to act on their own initiative, not in synchrony with
the rest of the colony. How-ever, ""ant highways,"" which can run over
hundreds of feet, represent highly coordinated forms of collective
behaviour. Humans, on the other hand, and especially organisations, suf-fer
from a ""centralised mindset""; they would like to assign the coordination of
activities to a central command. Self-organisation, by contrast, feeds upon
errors to provide the colony with flexibility and robustness. The colony can
adapt to a changing environment, and even when one or more individuals fail,
the group can still perform its tasks. ""The simple rules by which
individuals interact can generate complex group behaviour. Indeed, the
emergence of such collective behaviour out of simple rules is one the great
lessons of swarm intelligence."" If applied well, self-organisation endows
the swarm with the ability to adapt to situations that are immediate and
were not known before. This approach has proven itself in a number of
situations, ranging from network routing to supply chain optimisation.
However, it certainly requires a drastic shift in the human mindset to
become widely used.

A bank which promotes the personal responsibility of every employee can
establish a robust organisation similar to that of the ant colony mentioned
above. In the end, business is about people, and the human factor has a
strong impact on change and risk management as peo-ple execute and take the
right (or wrong) decisions. As Charles Darwin brought out:""It is not the
strongest species that survive, nor the most intelligent, but the ones most
responsive to change..""

The source is:
""Swarm Intelligence: An Interview with Eric Bonabeau"", 2003, available at
http://www.openp2p.com/pub/a/p2p/2003/0 ... abeau.html, accessed 1 March
2005.

Posted by ""Tim Scheffmann"" <Tim.Scheffmann@web.de>
posting date Thu, 9 Jun 2005 13:46:50 +0100
Conrad Stephen H shconra sandia.
Junior Member
Posts: 2
Joined: Fri Mar 29, 2002 3:39 am

Wisdom of Crowds

Post by Conrad Stephen H shconra sandia. »

Posted by ""Conrad, Stephen H"" <shconra@sandia.gov>
Surowiecki's thesis (cribbed from a Publishers Weekly review) is that
for many problems a crowd's ""collective intelligence"" will produce
better outcomes than a small group of experts provided that several
basic conditions are met -- even if members of the crowd don't know all
the facts or choose, individually, to act irrationally. ""Wise crowds""
need (1) diversity of opinion; (2) independence of members from one
another; (3) decentralization; and (4) a good method for aggregating
opinions. The diversity brings in different information; independence
keeps people from being swayed by a single opinion leader; people's
errors tend to balance each other out; and including all opinions
guarantees that the results are ""smarter"" than if a single expert had
been in charge.

I read this book several months ago and found it to be entertaining.
Although I was initially a bit skeptical of the premise, I thought the
author's arguments turned out to be fairly compelling (esp. for popular
nonfiction).

Now, as for the linkage (or conflict?) between a system dynamics view of
the world and the idea that there is some collective wisdom available to
tap into and use... I suppose that really depends on whether you view
yourself as ""the expert"" or as an analyst using a technique that not
uncommonly uncovers counterintuitive solutions (especially as compared
to conventional wisdom). I imagine the expert could view the idea of
""wise crowds"" as a threat to his authority -- except for the majority of
situations where there isn't opportunity to commission a poll to capture
diverse, independent opinions. However, for those working in an
environment where SD analysis may not (yet) have become mainstream, I
would posit that one might find it useful to make the case that SD could
offer distinct solution options, arrived at independently from more
standard approaches. And therefore, the spirit of Surowieki's thesis -
that each additional independent analysis contributes to helping an
organization arrive at better solutions - helps argue for employing an
assortment of analysis techniques (where practical), including SD. This
appeal for use of another well-founded, independent analysis technique
seems easier (and less contentious) than asserting the superiority of SD
over alternatives. Once the camel's nose is in the tent, then there
certainly is opportunity to show superior value relative to competing
analysis approaches. Remember, the author is not claiming that every
opinion is as good as any other, only that the aggregate ensemble
solutions (where available) tend to be better than that offered from any
single expert.

BTW, I don't recall the author specifically discounting formal analysis,
rather I seem to recall him comparing ""expert opinion"" to that of the
aggregate of independent opinion.
Posted by ""Conrad, Stephen H"" <shconra@sandia.gov>
posting date Thu, 9 Jun 2005 17:56:13 -0600
Jnwarfield aol.com
Newbie
Posts: 1
Joined: Fri Mar 29, 2002 3:39 am

Wisdom of Crowds

Post by Jnwarfield aol.com »

Posted by Jnwarfield@aol.com
Just to add comments to Jack's remarks, here is what we have found (and
which we know can be replicated, so you can consider these as established
scientifically for anything other than the simplest situations):

When groups are selected for their supposed expertise:

a) Each individual has a different view as to what are the most important
""problems"" involved in the situation
b) Each individual understands each problem differently, and will continue
to do so unless significant time is devoted to clarification of each
component problem
c) In the normal process, situations are describable by (on average) about
60 problems, but there is wide variation in the number
d) It is very rare that groups will vote to choose any problem as the
choice of a majority to have precedence
e) If the group runs through a facilitated session using Interpretive
Structural Modeling, consensus will emerge on the relative importance.

While the foregoing is discovered through repetitive use of the Nominal
Group Technique, we have tested it without using the NGT, and found that
""experts"" have published views that support a) and b) just as though they were
operating in group mode, even though they were operating independently. What this
shows is that people imagine themselves to be expert on situations when they
are very ill-informed, because they have built a self-image and that is what
they are being paid to display in their profession.

I wish none of this were true, because the implications for society are
staggering, and most people probably don't want to believe it.

John Warfield
Posted by Jnwarfield@aol.com
posting date Thu, 9 Jun 2005 11:06:13 EDT
Finn Jackson finn.jackson tangle
Junior Member
Posts: 7
Joined: Fri Mar 29, 2002 3:39 am

Wisdom of Crowds

Post by Finn Jackson finn.jackson tangle »

Posted by ""Finn Jackson"" <finn.jackson@tangley.com>

An interesting question, and answers.

I can easily see that ""two heads are better than one"".

But at what point does the *wisdom* of crowds transform into the *madness*
of crowds, and why?

(btw: Google found 582,000 references for ""wisdom"", and about 297,000 for
""madness"" of crowds (South Sea Bubbles, riots, ...).)


Finn Jackson
mailto:finn.jackson@tangley.com
Posted by ""Finn Jackson"" <finn.jackson@tangley.com>
posting date Thu, 9 Jun 2005 13:45:44 +0100
Jack Ring jring amug.org
Newbie
Posts: 1
Joined: Fri Mar 29, 2002 3:39 am

Wisdom of Crowds

Post by Jack Ring jring amug.org »

Posted by ""Jack Ring"" <jring@amug.org>
John Gunkler's experiences ring true with me. Interestingly, the small
group effect will vary according to the interpersonal styles in the group,
c.f., Rudolph Starkermann's ""Amity and Enmity,"" but can be mediated by
skilled facilitation. A method for orchestrating group intelligence in the
large has been described by John N. Warfield, www.jnwarfield.com as
Interactive Management with demonstrated successes in more than 200 projects
to date, c.f., http://www.cies.org/stories/s_bbroome.htm largely because it
educes an understanding of the system dynamics extant in the participants'
situation.
Jack Ring
Posted by ""Jack Ring"" <jring@amug.org>
posting date Thu, 9 Jun 2005 07:57:02 -0700
David Gillespie davidfg fidnet.c
Junior Member
Posts: 2
Joined: Fri Mar 29, 2002 3:39 am

Wisdom of Crowds

Post by David Gillespie davidfg fidnet.c »

Posted by ""David Gillespie"" <davidfg@fidnet.com>
I believe the wisdom of crowds transforms into ""madness"" when the size of
the crowd provides anonymity to the individuals in it.

David Gillespie
davidfg@fidnet.com
Posted by ""David Gillespie"" <davidfg@fidnet.com>
posting date Fri, 10 Jun 2005 08:47:46 -0500
Ahmet Ilker Soydan ilkersoydan h
Junior Member
Posts: 6
Joined: Fri Mar 29, 2002 3:39 am

Wisdom of Crowds

Post by Ahmet Ilker Soydan ilkersoydan h »

Posted by ""Ahmet Ilker Soydan"" <ilkersoydan@hotmail.com>

On 10.06.2005 Fri, 10 Jun 2005 07:07:45 Sheldon Friedman wrote:


>>.....Part of this wisdom may come from what has ben called transactive
>>memory.
>>People sharing bits if memory of information they all experienced. On the
>>other hand there is the concept of hiddent profiles where members do not
>>share information because they assume the information they have is not
>>important to a problem. Group members talk about what they all share as
>>knowledge, not unique information.


The general idea resembled the Artificial Neural Networks concept to me from
neurology science. People here represent nucleoids and their interactions
stand for dendrides (where the learning and interaction process occurs via
proteins and substances between each dendride [enzymes]), because
each interaction with the other is in fact an ""impulse"" that we give to the
system. The issue is that if these impulses accumulate to a ""critical mass""
after which learning occurs through interaction (transactions of impulses).

The collective wisdom that lies here is the systems's overall reaction to
the stimuli. Before impulse's power reaches a certain level, we do not
respond. Rather than a single nucleoid, the collection of nucleoids react.
This is, I guess, because of the biological fact that, the impulses coming
to the nucleiods via its dendrides from many other dendrides, they are being
summed. Therefore, the critical mass levels (stocks) sum up to a certain
level around THAT dendride. As you may guess, in the end, all system's
stocks are filled with impulses. Consequently, we behave in a ""collective
wisdom"" in terms of nucleoids' interaction network.

I just wanted to pinpoint a similarity from a natural science.

---------------------------------------------------------------------------
Ilker SOYDAN
Politecnico di Milano, Phd Candidate
Management, Economics and Industrial Engineering
Via Giuseppe Colombo 40, 20133, Milano

Posted by ""Ahmet Ilker Soydan"" <ilkersoydan@hotmail.com>
posting date Fri, 10 Jun 2005 17:05:13 +0200
Kim Warren Kim strategydynamics.
Junior Member
Posts: 16
Joined: Fri Mar 29, 2002 3:39 am

Wisdom of Crowds

Post by Kim Warren Kim strategydynamics. »

Posted by ""Kim Warren"" <Kim@strategydynamics.com>
thanks Ahmet - I *think* I follow the argument. An intriguing further
question is whether such a system could ever perform well when there are
important dynamic effects, such as worse-before-better, since the early
feedback from the system would be that the collective wisdom was wrong.

Kim
Posted by ""Kim Warren"" <Kim@strategydynamics.com>
posting date Sun, 12 Jun 2005 05:34:59 +0100
rgd6 cornell.edu
Junior Member
Posts: 9
Joined: Fri Mar 29, 2002 3:39 am

Wisdom of Crowds

Post by rgd6 cornell.edu »

Posted by <rgd6@cornell.edu>
There is also the problem of group-think in small groups where disagreement
is seen as disloyalty, and were like-minded individuals reinforce one
another's conceptions and misconceptions about their ""world"". This can
lead to incorrect beliefs and incorrect or even dangerous policies - if we
consider the political realm.

Are many heads better than one? Is it 'the wisdom of crowds' or is it
'group think', 'herd mentality' and the 'law of the mob'?

How can we bring out - emphasize - the wisdom?

Richard

_____________________
Richard G. Dudley
Bogor, Indonesia
please reply to
Richard.Dudley@attglobal.net
http://pws.prserv.net/RGDudley/
Posted by <rgd6@cornell.edu>
posting date Mon, 13 Jun 2005 07:32:57 +0700
Conrad Stephen H shconra sandia.
Junior Member
Posts: 2
Joined: Fri Mar 29, 2002 3:39 am

Wisdom of Crowds

Post by Conrad Stephen H shconra sandia. »

Posted by ""Conrad, Stephen H"" <shconra@sandia.gov>
Kim,

Let me try and take a stab at answering your questions:


- does this wisdom still work where there are strong dynamic effects,
e.g. heavy worse-before-better outcomes .. I can see how it might do,
with individuals spotting these effects 'infecting' others with their
own understanding

I don't recall there being any specific examples considered where there
were strong dynamic effects per se. As Surowiecki states in a recent
interview,
""The idea really came out of my writing on how markets work.
Markets are made up of diverse people with different levels of
information and intelligence, and yet when you put all those people
together and they start buying and selling, they come up with generally
intelligent decisions. Sometimes, though, they come up with remarkably
stupid decisions -- as they did during the stock-market bubble in the
late 1990s. I was interested in what explained the successes and the
failures of markets, and as I got further into it I realized that it
wasn't just markets that were smart. In fact, crowds of all sorts were
often remarkably wise.""

He addresses three types of group interaction problems: cognition,
coordination, and cooperation. Cognition problems are those where
there's (supposedly) a right, or best, answer. Examples:
1. what are the odds of U Man beating Arsenel?
2. which horse is most likely to win the Grand National?
3. how much is a share of BP worth?
The bets of people betting on a football match or a horse race determine
what the odds will be, and the choices of investors determine stock
prices. Really, any group of people who can act collectively and
independently to make decisions and solve problems -- like a company or
a government agency --can be considered to be a crowd. And according to
Surowiecki, the more diverse and independent each of the voices, the
better the decisions tend to be. So, let's consider all those
independent voices in a bit more detail. Even when considering the
solution to an inherently dynamical problem:
-- some may tend not to give much consideration to the dynamical aspects
at all, instead looking to historical analogues
-- some may use choose to ignore the dynamics because they favor a more
statistical approach such as using correlation
-- some because of their experience and expertise, may be more
sophisticated in their consideration of the dynamics, but perhaps only
intuitively using systems thinking (without explicitly using either
qualitative or quantitative systems analysis tools to help them)
-- and some, perhaps those more formally trained in SD, may be inclined
to explicitly consider system structure and favor a more mechanistic
interpretation in forming their opinions
Surowiecki would claim that experts, no matter how smart or
sophisticated their analysis tools, only have limited amounts of
information. They also, like all of us, have biases. Add together enough
independent opinions and the biases tend to cancel out. The lesson
Surowiecki would have us take away is not to rely on one or two experts
when making difficult decisions. That doesn't mean that expertise is
irrelevant. It just means that together all of us know more than any one
of us does. (I guess I believe this -- mostly. The contrarian in me
still has some trouble acquiescing all the way. As Mark Twain said,
""whenever you find you are on the side of the majority, it is time to
pause and reflect."")


- is this phenomenon itself amenable to being modeled [and is there any
value in doing so?]

Well, there seem to be lots of agent based models that attempt to
explain the behavior of markets. In SD, we tend to attempt modeling
broader problems such as the interaction of commodity markets with the
behavior of suppliers and consumers (such as in the Meadows' hogs
model).

Maybe it would be interesting to model how an organization currently
solicits opinions when making important strategic decisions. Then, look
for policies that have the effect of encouraging broader participation.
I know in my company, some decisions tend to be top-down, while others
tend to be made bottom-up. Occasionally, we bring in outside experts to
provide an independent viewpoint. I think it'd sometimes be worthwhile
to have some combination of all applied to addressing the same problem.

Also, I would think that the coordination and cooperation problems that
Surowiecki considers would be particularly amenable to agent based
modeling. An example coordination problem is negotiating traffic. An
example of good traffic coordinating behavior: I avoid the rush hour
when commuting to work (because nobody really cares exactly when I show
up). I avoid the worst of the traffic (good for me) and the traffic
during rush hour is a very little bit smaller because I'm not driving
then (marginally good for everyone else). An example of bad traffic
coordinating behavior: New Mexicans seem to insist on driving slowly in
the left lane. (Italians driving on the Autostrade seem to coordinate
much better because the collective agreement seems to be that faster
cars have the right of way in the passing lane.) As for cooperation
problems, people cooperate to uphold and enforce the law even when it's
more rational (at least in the short term) for them to turn a blind eye
or let others do the work. It seems that eliciting and programming the
decision rules of various individual agents -- and then allowing those
agents to interact and learn -- can provide insight about emergent
behaviors found when studying coordination and cooperation problems.
(Emergent behavior is defined as the collective, aggregate, sometimes
surprising, behavioral manifestation of all those little individual
decision rules.) And I would say that it is the explanation and
definition of these emergent behaviors that we operationalize as table
functions to summarize aggregate behavior in an SD model.


Interesting questions you ask. Hope this is not overkill.
Steve
Posted by ""Conrad, Stephen H"" <shconra@sandia.gov>
posting date Fri, 17 Jun 2005 20:05:46 -0600
Locked