Can system dynamics models "learn"

This forum contains all archives from the SD Mailing list (go to http://www.systemdynamics.org/forum/ for more information). This is here as a read-only resource, please post any SD related questions to the SD Discussion forum.
"Ray on EV1"
Member
Posts: 29
Joined: Fri Mar 29, 2002 3:39 am

Can system dynamics models "learn"

Post by "Ray on EV1" »

My favorite approach comes from control systems engineering. It is
automatic tuning. An example could be the controller on an elevator. The
objective is to locate the correct position to stop the elevator at a floor
with an accuracy of 2mm and to do so quickly but not jerky and dont
overshoot the end point where the elevator will have to move back and forth
to get to its final stopping point. That is, the controller should
accurately place the elevator and not overshoot the target point and do so
quickly - critically damped control.

The accuracy and damping are controlled by the values of specific components
in the system. The components are adjustable so the technician can tune the
control.

A learning system could easily be designed (and many have been) that measure
the accuracy and overshoot and then tweak the adjustments until the degree
of accuracy and overshoot are obtained. Now, as system components drift in
value, the self tuning controller can readjust to account for these
variations so the elevator always approaches its best behavior.

The concept has been fully abstracted for system design where the ability to
self tune is built into the system.

So I would like to add another multi-part question:

We see that there is a learning concept where the system is comprised of a
fixed set of components and a fixed set of connections between these
components. The fixed set of components means that there are only, say, 3
components but there values my be changed. To produce learning behavior,
the systems components values my be changed in order to produce some best
results.

An alternative system configuration might be were the connections between
components may be changed to facilitate system performance. But we may see
that this is just an extension of the previous system where there are
components that switch the connection fabrics activity.

This implies that there is really just one learning method presented here -
a reconfigurable system. Narrowing the view to any subset previous address
may simplify design and analysis activities for any one system, but we
should always remember that these simplified models are just extracts from
the larger class.

The learning nature of this class of systems is really just a method of
tuning a specific stimulus
esponse mechanism. What does it take for a
system to learn to choose a different stimulus or response?

Raymond T. Joseph
Aarden Control
From: "Ray on EV1" <rtjoseph@ev1.net>
John Sterman
Senior Member
Posts: 117
Joined: Fri Mar 29, 2002 3:39 am

Can system dynamics models "learn"

Post by John Sterman »

Natarajan R C found in the beer game that a group overreacted to a
calm set of customer orders.

In related work, Rachel Croson, Karen Donahue, Elena Katok and I have
done a set of experiments with the beer game (computerized version,
but otherwise standard) in which customer demand is constant and
known to all participants. The system begins in equilibrium. Under
constant, known demand there should be no oscillation, yet in the
vast majority of games we see results that are quite similar to those
in the standard protocol. Even when everyone knows what customer
orders are, and even when those orders are constant, so there are no
external shocsk of any type impinging on the system, peoples
inability to account for the time delays and supply line of unfilled
orders leads to oscillation and amplification as they try to adjust
their inventories to zero (the optimal inventory under perfect
information). We even see this behavior when initial inventories are
zero.

We hope to present the results at the SD conference this summer.

John Sterman
From: John Sterman <jsterman@MIT.EDU>
Yaman Barlas
Member
Posts: 44
Joined: Fri Mar 29, 2002 3:39 am

Can system dynamics models "learn"

Post by Yaman Barlas »

Depending on what is exactly meant by "learning", this question may have
different answers:
- A simple negative (goal-seeking feedback) loop involves learning: An action
is taken, the result is measured and evaluated, and it is "learned" that the
action was either insufficient/inadequate or wrong and a new, compensating
action is taken... There is some learning here?
- At another level, learning occurs by the interaction of several negative and
positive feedback loops. (The shift of loop dominance as the action "learns"
about the state of the system is a classic example. See the two previous emails
addressing G.P. Richardsons paper).
- At a higher level, (near) optimal learning may be obtained by a cleverly
designed goal-adapting structures. (See goal formation, Ch. 15, Stermans
Business Dynamics book).
- A yet more optimal (but different) learning may be obtained by Kalman
Filtering (and similar) parameter optimization methods. (See the previous email
by Raymond on optimal damping of an elavator).
- And finally, the most sophisticated version would a system that modifies its
very structure (the forms of equations) as a result of action feedback. (Also
mentioned by Raymond). An extreme and most difficult version of this type of
learning is the type of "structural optimum system design" methods (as opposed
to optimum parameter estimation like K-F) rarely used in control theory. There
are a few examples of this in SD literature and in control theory literature.
(Also note that special cases of such structure modification/optimization may
involve doing this by special parameter value adjustments, which is more
feasible/possible).
Yaman Barlas
From: yaman barlas <
ybarlas@boun.edu.tr>
---------------------------------------------------------------------------
Yaman Barlas, Ph.D.
Professor, Industrial Engineering Dept.
Bogazici University,
34342 Bebek, Istanbul, TURKEY
Fax. +90-212-265 1800. Tel. +90-212-358 1540; ext.2073
http://www.ie.boun.edu.tr/~barlas
SESDYN Group: http://www.ie.boun.edu.tr/labs/sesdyn/
-----------------------------------------------------------------------------
Alan Graham
Member
Posts: 27
Joined: Fri Mar 29, 2002 3:39 am

Can system dynamics models "learn"

Post by Alan Graham »

Colleagues,

Theres a substantial literature on Adaptive Control, in which a control
system adapts its feedback responses to, e.g. changing parameters or
changing environment, i.e. dynamic systems that learn.

For example, one implementation of an adaptive control system monitors real
behavior and continually tests multiple models of the process for best fit
(in principal using maximum likelihood estimation through optimal filtering
of the Kalman/Bucy sort). Its true there are multiple unvarying (and "not
learning") individual dynamic systems in this scheme. Its also true that
if you draw the boundary a bit wider around the "system", the whole scheme
is a dynamic system that not only represents reality, but learns as it
interacts with it.

So there are whole large classes of dynamic systems that learn. The
interesting thing, if the learning process is presumed to be occuring in
real life, is how rationally to represent the learning process-what sorts of
alternate hypotheses will be looked at and chosen as the systems dynamics
march forward. You can think of many of the specific examples from
participants in this discussion list as specific representations of
different learning processes in different situations, each of them
boundedly-rational (and sub-optimally learning).

As a side topic, theres also a literature on how much its worth to perturb
the system, to learn things about it, and later be able to control it
better. This is discussed as the "dual control problem", for the dual goals
of learning about the system (for which large pertubations are often good)
and controlling the system (for which large perturbations are usually bad).

cheers,

alan

Alan K. Graham, Ph.D.
PA Consulting Group

Alan.Graham@PAConsulting.com
One Memorial Drive, Cambridge, Mass. 02142 USA
Direct phone (US) 617 - 252 - 0384
Main number (US) 617 - 225 - 2700
Mobile (US) 617 - 413 - 7801
Fax (US) 617 - 225 - 2631
Home office (US) 978 - 263 - 8719
Home fax (US) 978 - 263 - 6861
Alan Graham
Member
Posts: 27
Joined: Fri Mar 29, 2002 3:39 am

Can system dynamics models "learn"

Post by Alan Graham »

Colleagues,

John Stermans beer game example is also answering an important related
question: SHOULD realistic system dynamics models of peoples
decision-making learn?

And the answer is (if one generalizes), not very often in complex dynamic
systems. The laboratory experiment John describes is a controlled
replication of what we see in market after market: the persuasiveness and
impact of the conditions of the moment will overpower nearly any possibility
of learning (as in showing altered decision-making) about long-term results.
Has anyone EVER come across a commodity market that was cyclical until
participants understood the dynamics and how to stabilize them, and thus
ceased to have a cyclical commodity market? The results Ive found in
applications are consistent with this: that peoples intuitive policies are
far inferior to policies arrived at by analysis with a dynamic model (in
dynamically complex situations).

And this "hardly ever learning" hypothesis is well-supported by further
laboratory evidence from John and colleagues around the world: systems with
multiple inputs, delays, and feedback are much harder to deal with than
simpler systems.

All of this doesnt say learning cant occur, but its likely to be much
simpler take more than we first might expect. Rationality is indeed
bounded.

cheers,

alan

Alan K. Graham, Ph.D.
PA Consulting Group

Alan.Graham@PAConsulting.com
One Memorial Drive, Cambridge, Mass. 02142 USA
Direct phone (US) 617 - 252 - 0384
Main number (US) 617 - 225 - 2700Mobile (US) 617 - 413 - 7801
Fax (US) 617 - 225 - 2631
Home office (US) 978 - 263 - 8719
Home fax (US) 978 - 263 - 6861
"Ray on EV1"
Member
Posts: 29
Joined: Fri Mar 29, 2002 3:39 am

Can system dynamics models "learn"

Post by "Ray on EV1" »

I like the approach but I am a little lost in the detail of:
* the desired state "s" is a changing goal. It describes a learned
* response.

Is it the intent to describe this optimization process as a learning
process? So this should be valid when the end point is fixed (a given
state) or constantly changing, possibly an unbounded set of states.

I have a little difficulty in visualizing the attainment of a final state as
a learning process. Did the machine learn how to get there or did it get
there out of chance? An alternate view might be to see the hill climbing
method as the learning process. Learning how to navigate a hump or a hump
with bumps and not be distracted. But if it is the method that represents
the learning process, this brings us back to the concept of system tuning
being a learning process.

In general, it is not comfortable to say that the ability to store a value
(fix a stock), is learning. A hill climbing process can be carried out by a
simple machine with a memory location for the current step size and a memory
location for the hills slope in each dimension. More complex and robust
solutions (machines) can be envisioned that require more memory locations -
does this constitute more complex learning or is it just a more complex
machine?

To abstract the concept of attaining a changing goal as being a learning
process, lets look at a class of systems.
That is the class of set-point control systems. In such systems, a target
state in given and the system is designed to attain this target state if for
some reason, the system becomes off-target. Such a system is in our
televisions to maintain the user selected channel, or in the guidance system
of a rocket seeking a given target. Are these learning machines?

It feels like we need a simple, clear definition for learning to even hope
that some of this will make sense.

BTW, a neuron is a lot more than a stock. In fact, a neuron only conducts
when there is a sufficient build up of stimulus, in a sufficiently short
period of time. Additionally, the stimulus must be removed and then
re-applied to get a retriggering. Any single firing will only affect a
small segment of a response. It takes a multitude of firings to get a
complete response. As such, the target neuron must be pulsed, given
sufficient time to decay and then repulsed. So it has a input sensitivity
level, it has a fixed energy level that must be delivered per pulse, it
saturates after a given input level and energy delivery, and it requires a
desensitization decay time.

Raymond T. Joseph, PE
RTJoseph@ev1.net
John Sterman
Senior Member
Posts: 117
Joined: Fri Mar 29, 2002 3:39 am

Can system dynamics models "learn"

Post by John Sterman »

In the discussion of learning in system dynamics models, some folks
wonder whether procedures carried out by a machine can be called
learning:

"A hill climbing process can be carried out by a
simple machine with a memory location for the current step size and a memory
location for the hills slope in each dimension. More complex and robust
solutions (machines) can be envisioned that require more memory locations -
does this constitute more complex learning or is it just a more complex
machine?"

This touches on an old debate in philosophy and AI. Many critics of
AI assert that there is something special about humans (or perhaps
organisms of a certain complexity, e.g. primates and dolphins as
well), and are distressed by the notion that a machine can learn or
otherwise exhibit behavior we previously considered to require
intelligence. Thus every advance in AI (or control theory) caused
critics to say "that doesnt require intelligence (or learning); its
just a mechanical (algorithmic) procedure." So, when a machine could
find the optimal allocation of resources under a budget constraint
(linear programming), people said that wasnt intelligence or
learning, just the mechanical application of an algorithm (simplex).
Now games of skill, they said, those require real intelligence. Then
when a machine learned to play checkers, and to do so better than any
humans, critics said checker playing was, after all, mechanical, and
not a task requiring true intelligence or learning. Never mind that
Samuels checker program learned from its experience. Chess, they
said, now chess, that requires real intelligence. Well, now chess
too is played by machines at least as well as by the world champion
human. So it is with other tasks. It used to be that keeping a room
comfortable required a human to monitor the temperature and put
another log on the fire when it got cold. Before bimetallic strips
and thermostats, most people would have said doing so required
intelligence, and knowing when to put a new log on was something you
learned (albeit something most 6 year olds could learn pretty
quickly). As soon as a mechanical device can do it, though, we say
it is merely mechanical. There is no mystery about how the
bimetallic strip works, and some folks find this annoying. The
example may seem trivial, but consider the controls in your average
minivan today. It probably has self-adjusting fuel injection that
tunes the car to changes in fuel, air, temperature and engine
conditions. It may have a transmission that learns your driving
style and adjusts shift points accordingly. It may have a navigation
system that tells you how to get there from here. And so on.
Critics continue to say well, these are all mechanical; they dont
really represent learning.

Whats going on here is a huge hindsight bias. Prior to these things
being done, people say that doing them requires human intelligence
and learning. After they are done, they are merely mechanical, and
dont represent real learning. At bottom these reactions come
about because many are threatened by the implication that underneath
it all, we, too, are merely machines. The underlying issue is the
old debate over materialism vs. spirit. Those who object that a
model cant really be learning because it is, after all, a formal
procedure instantiated in a computer are seeking the ghost in the
machine. They prefer mystery to mechanism.

I suggest we use a form of Turing test to determine whether a model
can learn: lets take the emotional bias out of the discussion by
asking people to determine whether something represents learning
without knowing whether a human or a machine (model) did it. And
while there are clearly many situations in which humans learn far
better than machines, I further suggest that the learning
capabilities in many of our models will have to be degraded, detuned,
or turned off altogether if they are to mimic human behavior well and
pass the Turing test. There are many situations where it seems
rather easy to build a model that learns better and faster than real
people do.

John Sterman
From: John Sterman <jsterman@MIT.EDU>
"Rainer"
Junior Member
Posts: 4
Joined: Fri Mar 29, 2002 3:39 am

Can system dynamics models "learn"

Post by "Rainer" »

Hi everybody,

I´ve downloaded George Richardsons model and the slides. Unfornately there
isnt any explicative text dwelving into the issue and I´d like to know if
anybody knows a more detailed paper explaining how the learning model really
works.

I´m acompanying the discussion with a lot of interest. Here are some humble
ideas of mine :

If I describe learning as feedback process whose output is a changed
behavior due to experience, than it is quite possible to construct a model
that simulates learning.

But, it will rather be a simulation of a learning process than learning
model.

Maybe it is just hard to imagine that something heartless and mindless is
capable of learning.


Rainer Uwe Bode
From: "Rainer" <cdeh@hotlink.com.br>
CDH - Centro de Desenvolvimento Humano Ltda.
Recife - Brazil
"Jim Hines"
Senior Member
Posts: 88
Joined: Fri Mar 29, 2002 3:39 am

Can system dynamics models "learn"

Post by "Jim Hines" »

Weve been working for several years modeling social learning in
organizations. The specific question weve been asking is how can you
structure an organization so that managers tend to learn (and create)
better policies (in the SD sense).

This work combines system dynamics with agent-based modeling and genetic
algorithms (intended to represent social learning). More is available
at http://web.mit.edu/org-ev/www
esources.htm.

Jim Hines
From: "Jim Hines" <jhines@MIT.EDU>
MIT
"Marsha Price"
Newbie
Posts: 1
Joined: Fri Mar 29, 2002 3:39 am

Can system dynamics models "learn"

Post by "Marsha Price" »

JJ (and all)...

To me, learning in an SD model is about reinforcing. That does represent a
key aspect of learning. But does it meet all ”learning” needs?

Deterministic models require the modeler to specify all aspects of the
system. Therefore there are limits to the ability to learn in ways that
move outside specified structure, parameters and model boundaries. For
example, one can learn how to open bottles i.e. open one bottle and then the
next, etc. and soon one has the facility to easily open most bottles. This
kind of learning is what an SD model is good at doing. However what the
model cannot do is to move outside the model/system to similar but not same
situations and when confronted with a new problem—be able to translate the
bottle-opening skill into the ability to turn on a water faucet.

In an SD model there can be exponential growth of learning…the limit there
is the body of knowledge perceived as being expertise in opening bottles…so
there is decreasing learning as you have “learned “ all the bottles…so in
the SD model there is nothing to allow you to have a flash of insight that
goes beyond the body of knowledge and takes you and the body of knowledge to
the next level. Exponential growth with a limit…the classic “learning
curve”…does not allow for going beyond the limit.

This reminds me of a recent lecture I attended on contingency planning for a
terrorist attack on the electric supply grid where I found myself frustrated
by the rigidity of the modeling (not SD) and planning approach. The
contingency plans were of the “we do x when y occurs” character and
necessitated endless specifications of x and y possibilities and of their
individual probabilities of occurrence as defined by the modeler. And so
the first thing that occurred to me was that there could be other “y”s not
yet specified or provided for that occur. I wondered what accommodation has
been made within the system for these unforeseen events. What I am trying
to say is that if this is the case, there have to be mechanisms in place to
deal not with a specific event per se, but with any deviation from the norm
that calls for extraordinary responses. In the face of an extraordinary
event, how does one provide for the flexibility in a system to provide the
means for taking the pieces that have been prepared in advance to facilitate
recovery and to re-assemble them into different patterns as needed? I
suppose that what I am trying to say here is that I see vulnerable systems
such as electric power grids or transportation supply chains (and by
extension, SD models, and for that matter the dynamics of learning
organizations) as being composed of nodes and links. The problem that I am
seeing is how to make these nodes and links mutable, nimble and adaptive.

“Hill-climbing” approaches to learning (e.g. by running many simulations)
enable the model/learner to use experimentation to approach some optimal
condition given the specified goals and means (the model structure and
parameters). This represents one classic form of learning, but won’t allow
for the unpredictable—such as a terrorist attack, or simply what to do when
confronted with thirst and a water faucet when all you know is how to
unscrew and open bottles—though you’ve optimized your ability to open
bottles.

Agent-based simulation using genetic algorithms may prove to be one way to
move beyond deterministic model limits and to accommodate out-of-the-box
solution/learning in organizations.
This gets at the kind of “mutation” that enables moving to new levels of
evolutionary development (a la Stephen J. Gould and like-minded others in
the field of evolutionary theory)—the LIFE to which J.J. Lauble’ refers.

There are an abundance of issues to be dealt with. For example, in the
electric power grid/terrorist attack situation should the contingency
planning process be one of choosing the predictable and segmenting it so
that along with the correct toolbox it is able to be re-assembled as
needed—albeit perhaps in new and not previously thought of ways? Is it
possible to meet the challenge of the unpredictable by using mutations? Is
"the plan" robust enough to meet that challenge—to deal with the
unanticipated event (9/11)—out-imagining the clearly nimble terrorist who
can think and do the “unimaginable”? How do we construct the stockpile of
segments? Can there be a storehouse of "spare (and perhaps new) parts" for
various (and perhaps new) types of re-assembly?

Can this kind of “nimble” (out-of-the-box) preparation for recovery of a
system be cost effective if it is not built in significant measure on
existing infrastructure or common practices within the system (i.e. company
or industry)? What economic and political costs are acceptable to the many
affected businesses, communities, and regulatory bodies?

I believe that the issues of mutation (new responses of system actors)
within shifting environments (new conditions in system climate) are
universal in methodological terms—how to build models and real-world systems
that “learn” in such a way as to be rapidly flexible and adaptive to the
unpredictable.

Marsha Price
From: "Marsha Price" <
marshaprice@earthlink.net>

Marsha Jane Price
Research Affiliate, System Dynamics
Center for Transportation and Logistics
Massachusetts Institute of Technology
Telephone: +1.617.742.3221
Mobile: +1.617.642.7900
Alternate e-mail: mjprice@MIT.edu
"Jim Hines"
Senior Member
Posts: 88
Joined: Fri Mar 29, 2002 3:39 am

Can system dynamics models "learn"

Post by "Jim Hines" »

If I understood Bill Braun email, he asked if a model that learns can
remain focussed on the original problem.

The issue of problem focus is separate from the issue of representing
learning within a model. Focusing on a problem is "just" a strategy for
creating a useful model. The particular problem might or might not lead
you to represent learning. But whether you represent learning or not,
the problem thats got you going isnt going to change.

For example, maybe your is that youre afraid that your start up will
reach a certain size and then fail -- like many (but not all) other
startups youve known. You might have a hypothesis that the process by
which people learn in some companies company is a process that reduces
the potential for learning as time goes on. Consequently, companies
with these learning processes are unable to meet new challenges that
come with growth. In this case, youd want to represent your theory of
learning in a model that you hope will help you understand why startups
fail so that YOUR startup wont. A different entrepreneur might have
the same problem focus, but might be thinking of a different hypotheses:
Companies tend to grow to fast, take on too much debt, and become
vulnerable to relatively small problems. The problem focus is the same,
the models will differ in that one will represent learning and the other
might not. The fact that your model represents learning doesnt change
the fact that you are worried about your startup failing.

(Incidentally, both models might be useful).


Jim
From: "Jim Hines" <jhines@MIT.EDU>
Bill Braun
Senior Member
Posts: 73
Joined: Fri Mar 29, 2002 3:39 am

Can system dynamics models "learn"

Post by Bill Braun »

I read and understood the original message to ask if the the model itself
could learn (and thus change). With that as a reference point, I asked if a
model that learns/changes could lose its problem focus. I think what you
describe in your example below, Jim, is the learning that emerges from
various models, each of which may look at the same problem from different
reference points, and thus be different models, each of which could lead to
different insights and learning, but still "anchored" to the same problem.

Have I misconstrued the original question of the thread?

Bill Braun
From: Bill Braun <medprac@hlthsys.com>
"Finn Jackson"
Junior Member
Posts: 9
Joined: Fri Mar 29, 2002 3:39 am

Can system dynamics models "learn"

Post by "Finn Jackson" »

Hello,

Martin Schaffernicht asks whether it is possible to build a system dynamics
model that "learns".
The answer is yes, and no.

On the one hand, any balancing or negative feedback loop is one that learns:
it learns how to alter the different variables involved in order to optimise
the target value of the target variable.
This is single-loop learning, and is trivial.

Double-loop learning means questioning from time to time whether the target
value (or even the target variable) is in itself optimal.

We can imagine modelling this by having (say) two variables, A, and B, and
an overall rule which says "the main thing is to optimise A; but from time
to time you need to check up on B and make sure that its value is not too
far from its optimum; and if it is (too far) then you need to switch to
optimising B until its value is back within the required bounds; and then go
back to optimising A".

But this is just single-loop learning in disguise! The rule has simply
changed from optimise "A" to optimise "a complex combination of A and B".


The root cause is that any SD model is fixed -- it is literally "programmed"
to behave exactly as we tell it to. Although that behaviour may be complex,
it will always be the same: whether we run the model today, tomorrow, or
next week. That is the beauty of models: they are intrinsically repeatable
and deterministic. But the world (and second loop learning) is not.

For a model to truly "learn" it would have to be able to rewrite its own
equations.



Martin asked for references, and though its scope is wider than this
question, I thoroughly recommend Gregory Batesons "Steps to an Ecology of
Mind".
Reviews can be seen at:
http://www.amazon.com/exec/obidos/tg/de ... 65-8382500
?vi=glance


With regards,
Finn Jackson
From: "Finn Jackson" <Finn.Jackson@Tangley.Com>
Locked