Perceiving feedback

[Martin Schaffernicht martin utal]

Posted by Martin Schaffernicht <martin@utalca.cl>
Hi,

I have a question concerning the èrceptio (misperception) of feedback.

In ""Misperceptions of basic dynamics: the case of renewable resource
management"" (SDR 20(2): 139-162), Erling Moxnes describes an experiment
that uses a simple model of lychen, where subjects had to manage a
reindeer population. He wrote that ""the stock and flow diagram is one
of the tools of system dynamics and has typically not been available to
decision makers thus far. Consequently, a structuring of the problem
by this tool has not been available either [to the subjects in the
experiment]. Therefore, the subjects get verbal descriptions. However,
the descriptions are sufficient to construct the model (...)."" (p. 141)

Is a textual description all one needs in order to construct a model with
a stock and flows and with a feedback loop? Or does one have to think in
terms of these concepts in order to ""see"" them in the description? What
is the ""tool"": the diagram or the concept (or both).

Along with the verbal description, the subjects were given the information
feedback provided by the simulation; however, subject's failed to build
adecuate mental models despite this informatin feedback (p. 150): this
""requires a language in which to represent dynamics systems and an ability
to translate available normal language informatino into the modelling
language.""

Does this failure to percieve the relevant structure in the situation
(from text and from the simulation) mean that ""we perceive what we
(already) know""? (And so the only way to overcome the limitation is
to study SD or a comparable ""language"" with stocks and feedback?) Or
is there a way to detect or ""discover"" the structure withour ptior
training?

In his Nobel lecture (MAPS OF BOUNDED RATIONALITY: A PERSPECTIVE ON
INTUITIVE JUDGMENT AND CHOICE; 12/8/2002), Daniel Kahneman mentions
that ""The core idea of prospect theory (Section 4) is that changes
and differences are much more accessible than absolute levels of
stimulation."" (p. and ""A general property of perceptual systems
is that they are designed to enhance the accessibility of changes
and differences (Palmer, 1999). Perception is reference- dependent:
the perceived attributes of a focal stimulus reflect the contrast
between that stimulus and a context of prior and concurrent stimuli.""
(p. 11; Palmer, S. E. (1999). Vision science: Photons to phenomenology.
Cambridge, MA: The MIT Press.)


In SD, what can be known at a given point in time are the levels in
the stocks. In (visual) perception it's change, closer to
rate_of_change than to stock (am I interpreting this right?).
So: is there a change that in an experiment with a simulator
that directs the subjects' attention towards CHANGES, they
performe better (build better mental models)?

I'd appreciate your comments,

Martin Schaffernicht
Universidad de Talca
Talca - Chile
Posted by Martin Schaffernicht <martin@utalca.cl>
posting date Mon, 09 Jan 2006 12:41:19 +0100

[Matzaball50 aol.com]

Posted by Matzaball50@aol.com

Martin,

I'll take a shot at this, even though I'm not entirely convinced I fully
understand what type of comments you are seeking.

In modeling 'feedback' we have a 'problem'. For pure physical systems we can
empirically see and quantify all the components and all the connections.

In any kind of social system the 'feedback' is metaphorical. That is, we
have no pipes or wires connecting individuals and events in the 'social' realm.

This is why feedback in social systems is in the eyes of the beholder. Any
number of different components may comprise a 'system' depending on who is
constructing the system and what their purposes might be.

Most SD models are comprised of both physical and social elements but you
cannot treat them or think of them in the same way or the same thing.

You can find 'loops' in any system if you take a large enough view in space
and time.

The notion of metaphors might strike some as being 'bad' but it isn't. What
it does is place a reasonable perimeter around what you might think you know
about something and what others might think.

It is here that I believe SD models run into their most persistent problems
by trying to make believe we are not dealing in metaphors but in real 'facts'
and this is just not so.

The very old notion of ""Garbage in, garbage out"" applies to all modeling
activities.

I hope this helped Martin, but again, I'm not sure where you wanted to go
with this.

Regards,

Marc
Posted by Matzaball50@aol.com
posting date Tue, 10 Jan 2006 11:07:06 EST

[John Gunkler jgunkler sprintmail]

Posted by ""John Gunkler"" <jgunkler@sprintmail.com>
It might be time (at the risk of insulting everyone's intelligence) to remember what mathematics really is when applied to helping us understand the natural world.

Even doing something as simple as using numbers to count objects requires that we assert a one-to-one relationship (a ""metaphor"" if you will) between physical objects and these abstract things we call ""numbers."" And we assert more than that: we also assert that, in certain respects, the ""rules"" of behavior or those objects and those numbers are the same. (Here ""rules of behavior"" include things such as having an ""order relation"" for instance.]

Mathematics helps us understand the natural world to the extent that we can find, or create, a mathematical structure that is analogous to (bears a one-to-one relationship with) the structure we're trying to understand. When we assert that F=ma, we are claiming (metaphorically) that there are things in reality that we are modeling as ""force"" (or ""F""), ""mass"" (or ""m""), and ""acceleration"" (or ""a"") -- and that they are related to each other in just the same way that the mathematical quantities are related to each other in the equation and using the ""rules"" of simple algebra.

In much the same way, we can use plumbing (the flow of fluids through pipes) as an analogy for the flow of electricity in wires -- or, to put it another way, the mathematical models are quite similar.

So, every use of mathematics for the purpose of describing, predicting, or understanding the physical world is an example of ""metaphor"" -- or the use of a ""model"" of reality.

Why would we do this? There are several reasons, including:
(1) It is often easier to manipulate a mathematical model than it is to manipulate nature. [We can change the rate of flow of water by simply making one number bigger in our system of equations; to actually increase water flow in, say, a river would require significant effort!]
(2) It is often possible to see (predict) results more quickly by manipulating an equation than by making a change in the real world. [Especially when there are significant delays between changes and the effects of those changes, we can see what those effects would be almost instantaneously in mathematics (by solving an equation, or running a simulation with a speeded up time frame) whereas it could take years, decades, or centuries to see the effects in the real world.]
(3) It is often safer to change a mathematical model's parameters than to make the corresponding change in nature. [We can investigate the effects of a massive tidal wave engulfing Los Angeles without the actual loss of life and property.]

By the way, this modeling of reality (using metaphor or analogy or one-to-one relationships) is pretty much how we humans try to understand anything new. One theory of knowledge creation uses the image of ""mental Velcro"" to explain why this is so: Whenever we attempt to assimilate new information, one of the first things we do is to try to ""hang"" it on things we already know. ""Oh,"" we'll say, ""that's a lot like what happens when ... (naming something we are already familiar with.)"" Once we can see the new information as somehow ""like"" what we already know, then we're able to also see how it differs from what we knew before. But if we cannot find any hooks to hang it on, we have a lot of trouble grasping new information.

John
Posted by ""John Gunkler"" <jgunkler@sprintmail.com>
posting date Wed, 11 Jan 2006 11:55:23 -0600

[Martin Schaffernicht martin utal]

Posted by Martin Schaffernicht <martin@utalca.cl>
Hi Marc,

thanks for the ""shot""!

Let us take the simulation model used by Erling Moxnes (it can be
freely downloaded from his wab page) as a ""pure physical system"":
after downloading it and opening PowerSim, one can ""empirically
quantify all the components and all the connections"". If we repeat
Moxnes' experiment but this time allow the subjects to look at the
model, would they build different mental models and perform
diffferently. Or would thay have to learn about ""feedback"" an some
course in order to previously know such a thing may be encountered?

How do you recognize a feedback loop when you meet one? Is it like
Ponalyi's ""implicit integration"": once you start thinking about it,
you have already recognized it before (tacitly)? Then you would be
right when you say that it is in the observer's eyes (or mind) (even
though this does not make it necessary to decide if it ""really"" exists
or not).

I guess that the fact of an action being fed back towards the conditions
in which one acts, could be detected in a simulation by performing
experiments and recording the results; for example ""X reindeers in
year t is followed by Y mm of lichen in year t+1"" (which would make
sense after you realize that the lichen changes evvery time after a
change in reindeer population).

Do simulation-users recognize such connectins between changes in
variables? Do they engage in such kind of experimentation? Does
this result in some mental hypothesis including feedback and then
influence decisions? Does all of this depend on the information
offered by the simulation interface?

This is intriguing me, and I'd like to know if there have been studies
concerning these questions.

Thanks,
Martin
Posted by Martin Schaffernicht <martin@utalca.cl>
posting date Wed, 11 Jan 2006 15:49:10 +0100

[Martin Schaffernicht martin utal]

Posted by Martin Schaffernicht <martin@utalca.cl>
Hi again,

additionally to the ""velcro"" theory, we also use what we already know to
understand what we encounter. Somewhere I read ""if all you have is a hammer,
everythink starts looking like nails"".

So would it be that when a person has learned to look at the world in terms
of linear causality, it wil be specially hard to ""see"" feedback? Or, the
other way around, if you teach kids to think in terms of cycles (life cycles,
resource cycles like water and the like), they will start to put this mental
pattern over new situations?

I would say the question is not so much if feedback loops ""really"" exist: if
you can succesfully model a situation using them and you cannot discover any
reason to discard it, it is pretty much the same if it is really real - it works.

However: is the failure to perceive feedback an unavoidable feature of the
human cognition unless you undergo specific training (and so education is
the way out), or can the information feedback given by simulation intefaces
have an influence.

In the SDR 20(3), Croson and Donohue show that in the BeerGame, downstream
information is more effective than upstream information in helping to avoid
the bullwhip effect. According to the ""data-driven"" approach to supply chain
management, better information yields better decisions. In the BeerGame,
players recieve information about stocks (and the data they record directs
attention towards stocks). If it is true that the human attention is directed
towards what changes, one could imagine that directing players' attention
towards the changes of stocks and the relationships between them can have
an effect on the performance.

I am wondering if this is a question worth investigating an answer, or if
someone knows about studies in this sense.

Thanks for your comments,
Martin
Posted by Martin Schaffernicht <martin@utalca.cl>
posting date Thu, 12 Jan 2006 21:07:00 +0100

[Mihaela Tabacaru mima_tabacaru y]

Posted by Mihaela Tabacaru <mima_tabacaru@yahoo.com>

>Do simulation-users recognize such connectins between changes in
>variables? Do they engage in such kind of experimentation? Does
>this result in some mental hypothesis including feedback and then
>influence decisions? Does all of this depend on the information
>offered by the simulation interface?>

Martin,

I will try to answer some of your questins from a
psychologist's point of view.

Many of our beliefs concern the relation between one quantity and
another and we are concerned with such relationships because we want
to decide whether to manipulate one thing in order to affect another
(Barron, 1995) The normative theory points towards statistical
correlation, but people systematically violate this normative view.

As Baron (1995) points out, correlations are very often confused,
in every day reasoning, with causal relationships. It is important
to realize that they are not the same: establishing a correlation does
not establish causation, though it often provides evidence about
causation. To establish causation, other reasons must be ruled out.

Another point in perceiving dynamics is the case of the attentional
bias. Research about the probability heuristics shows that people have
a bias towards correlating two instances simply by associating the
presence of two, and ignore the combination of probabilities of one or
both being absent (Smedslund, 1963). Subjects typically attend to the
probability of the outcome given the “present” cue only. In a case of
associating the probability of a disease with the presence of one symptom,
Smedslund shows that 85% of the nurses investigated state there is
relation between the two, even though the number of times when the
symptom is absent and the disease present is almost as high as when
both are present. This might be very relevant in associating instances
in the surrounding organizational environment and picking up the ”present
cues”, thus developing very simple cause and effect relationships, that
lead to the misperception of dynamics.

A system that is limitedly transparent contains variables that cannot
be observed directly, either because we are only capable of observing
the symptoms, or because the system contains so many variables that we
have to concentrate on the few that we regard as the key ones in the system.


The choice of policies is influenced by the salience and relative easiness
of collecting quantitative data, versus the more soft, qualtative,
variables. In an experiment by Feldman (2004) about the culture of
objectivity at NASA it is shown that misunderstanding leading to the
explosion of Challenger and Columbia spaceships resulted from two general
aspects of NASA's culture: (i) an over-confidence in quantitative data
went hand-in-hand with a marginalization of no quantifiable data,
leading to an insensitivity to uncertainty and a loss of organizational
memory; and (ii) problem definition and solution creation were constructed
as if they were independent of organizational goals, resulting in an
inaccurate estimation of risk.

This over-confidence in salient data is specific to broader contexts:
accessibility of information—the ease (or effort) with which particular
mental contents come to mind is a well documented phenomenon (Kahneman,
2003). The accessibility of a thought is determined jointly by the
characteristics of the cognitive mechanisms that produce it and by the
characteristics of the stimuli and events that evoke it. Among others,
the determinants of accessibility subsume the notions of stimulus
salience, selective attention, specific training, associative activation,
and priming. Because quantitative data is and will always be more salient
than qualitative data it is highly likely to observe a bias towards the
first type. As Tversky & Kahneman put it, “the subjective assessment of
probability resembles the subjective assessments of physical quantities such as distance or size. These judgments are all based on data of
limited validity, which are processed according! to heuristic rules” (Tversky
& Kahneman, 1974, p. 1124).

Among other behavioral patterns, literature also shows that more complex
decision tasks lead to a higher deviation rate and that in the presence
of differences between the numbers of outcomes subjects prefer the simpler alternatives (Muller, 2001).

I hope these thoughts help.

As for the reading, here it is:
Baron, J. (1995), Thinking and deciding, Cambridge University Press

Feldman S.P. (2004), ‘The culture of objectivity: Quantification,
uncertainty, and the evaluation of risk at NASA’, in Human Relations
57 (6): 691-718

Kahneman, D (2003), A Perspective on Judgment and Choice
Mapping Bounded Rationality, in American Psycologist, Vol. 58, No. 9, 697–720,


Muller W (2001), ‘Strategies, heuristics, and the relevance of risk-aversion
in a dynamic decision problem’, in Journal of Economic Psychology 22 (4): 493-522

Smedslund, J. (1963), ‘The Concept of Correlation in Adults’, in Scandinavian
Journal of Psychology 4 (3): 165-173

Tversky A, Kahneman D (1974), ‘Judgment under Uncertainty - Heuristics and Biases’,
in Science, 185 (4157): 1124-1131

Best regards,

Mihaela Tabacaru
M of Phil Candidate
System Dynamics
University of Bergen
Posted by Mihaela Tabacaru <mima_tabacaru@yahoo.com>
posting date Thu, 12 Jan 2006 06:39:35 -0800 (PST)

[Keith Linard klin4960 bigpond.ne]

Posted by ""Keith Linard"" <klin4960@bigpond.net.au>
thanks Martin for the thoughtful comments on perceiving feedback. A few additional thoughts from my experience in trying to communicate feedback concepts.

<<How do you recognize a feedback loop when you meet one? >>

For those involved in any of the systems disciplines, electrical/electronic engineering, advertising etc, causal feedback may seem so blindingly obvious that you wonder what the fuss is about. My first couple of years lecturing in SDM were less than successful ... probably less than 50% really grasped the idea of feedback, while the rest continued to build 'spreadsheets' with IThink! (Parenthetically, nearly every year a 'spreadsheet model - auxiliaries only - gets through the ISD conference review process) Very quickly I was forced to introduce a new course solely on 'qualitative system dynamics', introducing systems tools from other areas such as Checkland's soft systems methodology and Eden's cognitive mapping.

<<Is it like Ponalyi's ""implicit integration"": once you start thinking about it ...>>

Once I got my teaching on feedback 'right', the consequences of the change in the students''Weltanschauung' was often dramatic.

When the concept gelled with one infantry sergeant he was like a little child who has suddenly learned how to talk. Every morning for the rest of the semester, as I opened my office door he would appear with a press clipping from the local newspaper: sport, religion, science, politics, traffic accidents, whatever ... ""Mr Linard, did you see this. A classic example of feedback ..."" One morning he started the conversation with: ""Last night in bed I was talking to my wife about feedback ..."". Who said SDM isn't romantic! At the end of the semester he wrote ""learning about feedback has transformed my whole life ..."".

Then there was the Indonesian Colonel who, to start with, was a blind apologist for that nation's transmigration program (especially the movement of large numbers of Javanes to West Irian (West New Guines) where (inter
alia) ethnic tensions and destruction of rain forest were clearly building into major problems for future generations. Over the next 2 years there was an utter transformation in his writings, with insightful quotes from Rachael Carson ('Silent Spring'), Churchman, Ackoff etc. In the introduction to his final project (an SDM model of the interactions between traditional farming of Javanese immigrants and its associated de-forestation and the effect on native subsistence agriculture, he wrote: ""Thank you Australia for introducing me to these fundamental feedback concepts. I now feel impelled to take them back to my colleagues in Indonesia ... but it will not be easy.

And the Colonel from Thailand, who found my ideas very difficult to comprehend. Then one morning, three weeks before the final exam, he came to my office smiling: ""Mr Linard, I finally understand what you are talking about! You are not talking about 'management' you are talking about 'politics'"". A bit stunned, I asked him to elaborate. ""Well, if my general says, capture that village where ther terrorists are based, I could approach the issue by doing a tactical analysis of the best way to capture the village, with the lowest loss of life on both sides. That's management! Alternatively, I could say: ""General, before we do something so drastic, should we not try to understand why the villagers are revolting; perhaps there are administrative injustices, perhaps there there are issues of land reform that need to be sorted ... if we can solve these we will not need to use military force, which will simply make the survivors even more dissatisfied."" Mr Linard, this is a profound insight that I have learned from this course, and I will never lose it. But it is not management, it is politics.""

What's the relationship between SDM & feedback thinking. First, SDM is a valuable tool in helping people take their understanding of feedback beyond the simplistic level of a dotted line on a flowchart. Secondly, the development of simple SD models exposes the assumed feedbacks to an audience in a way that they can visualise and understand ... and then challenge or suggest feedbacks that have been overlooked. (This is my greatest assistance as a modeller.) Finally, and most importantly, the SD model offers a way of testing the implications of the hypothesised feedbacks.

Keith Linard
134 Gisborne Road
Bacchus Marsh
Vic 3340
Posted by ""Keith Linard"" <klin4960@bigpond.net.au>
posting date Fri, 13 Jan 2006 08:12:54 +1000

[Matzaball50 aol.com]

Posted by Matzaball50@aol.com

John,

A very nice post, thank you, and not to nit pick, but I think you might be
mistaken in one instance here.

>>When we assert that F=ma, we are claiming (metaphorically)

I do not believe this is a metaphor. This is a physical law that cannot be
'perceived'. That is, we do not see, feel, smell, or taste a 'force' but the
equation supposedly represents 'real' physical entities.

Regards,

Marc
Posted by Matzaball50@aol.com
posting date Thu, 12 Jan 2006 10:05:07 EST