Reverse Modelling

This forum contains all archives from the SD Mailing list (go to http://www.systemdynamics.org/forum/ for more information). This is here as a read-only resource, please post any SD related questions to the SD Discussion forum.
Locked
Gerhard Werner
Newbie
Posts: 1
Joined: Fri Mar 29, 2002 3:39 am

Reverse Modelling

Post by Gerhard Werner »

I would appreciate any references or other information that pertains to
what I am calling here reverse modelling: given the simulated faulty
output of a simulator, can one reverse the process of simulation to
identify the faulty component in the model ? (e.g.: a given physical
process of which I have a simulator produces a faulty part; I now encode
this fault in the simulator output and run the simulator in the reverse
direction, leading me back to where the error may have occurred in the
physical process ?

Gerhard Werner (gwer1@bga.com)
Jay Forrest - SDSG
Junior Member
Posts: 9
Joined: Fri Mar 29, 2002 3:39 am

Reverse Modelling

Post by Jay Forrest - SDSG »

>what I am calling here reverse modelling: given the simulated faulty
>output of a simulator, can one reverse the process of simulation to
>identify the faulty component in the model ? (e.g.: a given physical

It sounds as though you are saying, "If the model draws the right line then
it must be right." In conventional modeling the belief that "proper" output
implies validity of the model is common. And correlations are commonly used
to drive the model. Curve fitting does not necessarily add knowledge. And
there are unlimited ways to add structure to fit curves. What do you want
the reverse model to install? A connector? A new flow? New loops? New
boundaries? What priorities do you suggest we apply to each of these
alternatives? Should they always be the same? What logic do you propose as
a basis for rules for exploring alternative structures?=20

I am afraid that the answer is "nothing in, garbage out"

The result of "reverse modeling" would be models that are black box and
lack understanding. Several years ago I surveyed several companies on their
use of SD and rather consistently heard that "We used SD for a while and it
was very helpful, but over time people moved and the models became black
boxes and their value declined. So we quit." Reverse modeling would likely
shrink that period of model value to zero.

I would suggest that you and your modeling team will learn more by direct
manipulation and redesign of the model.

Good Luck!
Jay Forrest

SDSG,LLC=A0 - The Strategic Decision Simulation Group=20
11606 Highgrove Drive
Houston, Texas 77077
Tel:=A0 281-493-5022
Fax: 281-558-3228
E-mail:
jayf@sdsg.com
"John W. Gunkler"
Junior Member
Posts: 10
Joined: Fri Mar 29, 2002 3:39 am

Reverse Modelling

Post by "John W. Gunkler" »

Jay Forrest writes:
<The result of "reverse modeling" would be models that are black box and
>lack understanding.

Perhaps Jay and I understand something different in Gerhard Werners
question. I believe that many years ago, in the early 1960s, my fathers
process engineering group at Dow Chemical Co. created models on an
analog-digital hybrid computer that could do "reverse modeling." The analog
component of the computer ran actual dials and gauges that read the outputs
of the chemical process being simulated -- much like an operator in a real
plant would see. When there was a problem with the real process they could
(I was told) sometimes replicate the output characteristics of that problem
and then look for anomalous behavior within the process. Im not sure how
often they were able to do this, in practice, but Im sure my father told me
of at least one success where they subsequently went into the actual plant
and found a defective part, replaced it, and fixed the problem. [I grant
you that the technology was stupid because digital computers were so much
easier to work with and could soon do everything that analog ones could, so
their experiments did not pan out in the long run.]

I think Mr. Werner raises a very interesting problem that, unfortunately,
may not be easy to solve using standard SD methods and models -- because I
know of no way, currently, to input the output. On the other hand, why
isnt it possible to make (hundreds, if necessary) simulation runs that vary
inputs and parameters attempting to recreate "problem behavior" so as better
to understand possible cause of such behavior modes?

So, if we have created a model that replicates behavior modes of initial
interest, why couldnt we attempt to use it to simulate new problem behavior
that arises in the real system? (We could perform maximum likelihood
estimation of model paramters, for example.) And, if that doesnt work, why
cant we simply "start from scratch" to build a model that replicates the
problem behavior then use the model to understand why the problem occurs?
Isnt that what SD does all the time?

From: "John W. Gunkler" <
jgunkler@sprintmail.com>
j-d
Junior Member
Posts: 9
Joined: Fri Mar 29, 2002 3:39 am

Reverse Modelling

Post by j-d »

I found this to be a very interesting question. First, it looked like an
identification problem. Assuming a general structure of an SD model as

dx/dt = f(x,u,t,p)

where x is stock/state vector, u is control vector, p is parameter vector,
and f is the nonlinear function vector (and remembering forever Prof
Forresters caveat to me that we are not doing control theory here, but
doing SD as an art form learnt by experience like engg or medicine). So in
essence the question is, given x (and we dont like some values of some
elements of x), what changes should be made in elements of p and u and f,
so that some goals are attained. AHA!! Optimal control problem, sort of
(just as Bob Eberlein mentioned)!!! Most of the time we try to do it anyway
when we do SD modeling - look at outputs, tweak parameters (p and u) and
functions so that we like the output (the danger is - what we like can
sometimes dictate how and why we tweak certain parameters - hence my
perennial devotion to real-world data fidelity - eg, if we think the world
is going to end soon, would we tweak entities in global models to make it
show us just that - at least that was one criticism of the World3 model,
not that I agree wholly to the criticism). In very large and complex
models, there is no way to avaoid this bias except by following the US
constitution - OK what I mean here is a serious, skeptical, peer-review
process of checks and balances - though that has its own problems too (for
example, the dogmatic devotion to have nonlinear feedbacks (we love them)
even when data indicates they are not needed - I was once told why my model
was meaningless as it did not have a "commonsense" feedback. I said but
that is not true (through prior econometric studies:-( in my case, hence it
is not there. My answer was not liked. I think this is the danger of having
CLDs too - emphasizing sexy diagrams and explaining them through good
stories but the diagrams are not based in reality (DATA!!).

I also think optimization can be useful for the same reasons we do
simulations - the systems are complex and we let computers help our mental
model development through repeated simulations. BUT since the systems are
complex and large, how do you know that the mental models derived from
repeated simulations are "good" ones? So I say - do repeated optimizations
(optimlations or simulzations), so that a pattern forms in our heads as to
the limits of our models, policies and so on, EVEN when we are trying our
darn best (by optimizing) to solve the problem. The criticism of this idea
is the Bounded Rationality one (Herbert Simon) - we can never have enough
data/time to do optimizations. If we were to adhere to this critcism, wed
never even build large simulation models. The answer is simple - we do it
because we can now do it with faster and faster computers and better math
and programming (climbing Everest just because it is there). I
predict/project/forecast (dont know what after the recent discussions)
that that will be the wave of the future.

The problem with optimization is that we may not learn much about the
system or the model because the fanciness of the solution takes us away
from the mud into an ivory tower. So guard against it. I draw a very strong
analogy of this to learning martial arts, esp the practice of aikido which
I am partial to. The emphasis there is on practice, practice, practice,
just as I think should be for simulation methods such as SD. The same
criticism comes up in aikido!! - does it really work, why are we not
"thinking" about "doing" the best technique? (it happens much to students
of aikido who talk too much and "think" too much without really
experiencing the falls, the sweat, the physical differences in people, and
so on). The thinking types are the optimizers and the acting/working type
are the simulators. In real-life, both can do well (reminds me of a story
about Sir Isaac Newton who I think detested sports but was once forced into
it anyway, and managed to win the race by calculating (his form of
optimization) the best way to do it. And of course, most martial arts
training is based on the "simulation" model - the body (& mind)
automatically reacting to new, complex situations in a good way because of
deeper repeated-practice understandings). I propose a good balance between
the two for good policy choices.

Now back to the original question - I think all the above comments apply.
What type of system is it? - is it well-defined enough to do optimizations
- can we somehow isolate the problem to the relevant smaller parts of the
model by looking at model design and by simulations, etc. etc.

I have suddenly run out of ideas - what happened here?

Anyway - for those who may have checked the URL below for GPRs book
Feedback Thought in Social Science and Systems Theory may have found two
links to his book. My apologies for that - there is only one, hardback
version (1991) available (I got it about 2 weeks back). Acc to Prof GPR,
there was no later version, BUT the older version, IMHO, is great in itself.

Also, I have made the SD resource list much more user-friendly. The books
are divided into a few classes - SD CORE, SD IN TEACHING, SD IN MANAGEMENT,
SD APPLICATIONS, SD INTERFACES WITH OTHER FIELDS - APPLICATIONS, SD
INTERFACES WITH OTHER FIELDS - PROGRAMMING METHODS, and NON-SD BUT
POTENTIALLY USEFUL BOOKS. The list has links describing the books,
editions, number of pages, costs etc (most at Barnes &Nobel, some from
other companies). I think the SD listers may find it useful to have a
web-based quick way of finding SD resources in one place - this may be
useful also to build a personal library or to choose books to teach from. I
have expanded the list to include Dr. Jay Forresters book suggestions
recently on "thesis topic". Suggestions for more additions welcome.

Regards, Jaideep
From: j-d <j-d@technologist.com>
**************************************************************
Jaideep Mukherjee, Ph. D.
Virtual Office http://www.netopia.geocities.com/shunya/
**************************************************************
Locked