Heres a simple dynamic hypothesis as to why negative characterizations of
system dynamics such as the one that started this thread persist.
Before describing it, I should note that these characterizations might
persist because they are true. My view, which will surprise no one, is
that good SD work is as fully rigorous as good work in any other modeling
discipline. As Keith Linard pointed out, and as the work of folks like
George Backus, Andy Ford, PA, Jack Homer, Brian Dangerfield, Ed Anderson,
and many, many others shows, good SD does draw extensively on data,
including numerical data, uses statistical/econometric tools when
appropriate, draws on optimization and other OR methods when appropriate,
and owes no apology to anyone as to its rigor.
So heres the hypothesis: Many non-SD modelers, particularly economists,
reacted very negatively to Jays early work, particularly Urban Dynamics
and World Dynamics. We can debate later whether these particular works
were flawed, whether they drew insufficiently on data, whether Jay should
have used regression to estimate relationships and parameters, and so on,
or whether these criticisms were the manifestation of deeper underlying
differences and conflicts such as an outsider encroaching on others
intellectual turf or aversion to the conclusions of UD and WD. For the
purpose of this discussion it doesnt matter.
So suppose you were a non-SD modeler and decided that UD and WD were "bad"
models, and, since these were the most prominent and widely known SD
models, that SD as a technique was flawed. You would then say, as I heard
myself from someone just last week, something like "Oh no, not system
dynamics again. Those Forrester models were discredited in the early
1970s."
There are many logical errors in this type of statement, not least of which
is conflation of a particular model (e.g., World Dynamics) with an entire
modeling method (system dynamics). Even if we were to stipulatethat World
Dynamics was a flawed model and discredited (something I dont agree with),
the leap to "so all SD models are flawed in similar ways" is obviously a
fallacy. It is analogous to saying "Here is one particular differential
equation model that is flawed, so all differential equations models are
similarly flawed." Or more plainly, it is fallacious to argue that "there
exists one bad apple, bad person, or bad novel, so all apples, people, and
novels are bad."
Nevertheless, having made the judgment that SD models are worthless, you
then decide not to read any more literature in the field. Similarly, since
SD work is unfavorably refereed in the journals in these fields, it doesnt
get published, and (we are not blameless in this dynamic), SD authors stop
trying to publish in those journals, stop attending those conferences, and
stop giving seminars in those departments.
The field of system dynamics continues to evolve, develop, and improve, but
because you get no new information, your opinion is never updated; because
your opinion doesnt change you have no reason to seek new information on
the current work in the field.
That is, there are positive feedbacks that create a self-enforcing
equlibrium in which people exposed to SD early may have formed a negative
opinion about it that then prevents them from learning about new
developments, which, perhaps, they might view more favorably.
The signal that this dynamic is operating is when a critic of system
dynamics points to very old work such as Urban Dynamics or World Dynamics
as the "evidence" that SD is flawed. Such critics are almost always
unaware of any recent work in the field, much less of modern modeling
protocols for use of data, model testing, and so on.
Note that all agents in this framework, the economists and the SD people,
are behaving rationally from their local perspective: It is rational to
avoid reading a literature you dont think has any merit; it is rational to
avoid submitting articles to journals that wont accept it.
Note also that this structure is symmetrical and does not point a finger of
blame at economists or othes outside of SD: if SD people get defensive and
reject the utility of econometrics or economic theory they will also fail
to learn about new developments, fail to create channels of communication
across disciplinary boundaries, and become insular, out of date, and thus
unable to do the highest quality work; if this behavior continues, it soon
would justify negative assessments from these other fields.
This dynamic should be obvious to all SD folks; a formal model of it is
easily constructed, and makes a good student exercise for introductory
courses (and, interestingly, it is easy to build a simple equilibrium,
fully rational game theoretic model that gives the same outcome). It
operates also in business contexts. Consider air travel. Suppose you have
a bad experience on a particular airline and decide never to fly with that
firm again. You do, however, generate unfavorable word of mouth about
them, telling your tale of lost bags, cancelled flights, surly employees,
and so on at dinner parties for years afterwards. Meanwhile, the airline
may have improved and could even have great service today, but since you
havent personally experienced it, you dont revise your opinion and
continue to generate unfavorable word of mouth long after your opinion is
no longer accurate. Just this dynamic occurred with Continental Airlines -
it took years of high quality service, awards, etc. to overcome the bad
reputation they developed in the 1980s when they were truly awful.
Similarly, there are many people today who will not even consider buying an
American made car because they had bad experiences in the 1970s, when the
quality of US cars was poor. US car quality has improved a lot (though
there is still plenty of room for improvement), but these people dont know
it firsthand, tend to discount what they may hear from others, and continue
to buy foreign cars.
So - what then are the policy implications? First, lets stop pursuing the
low leverage policies that take a lot of energy and dont accomplish
anything. There is low leverage in whining about how "they dont
understand us" or how "this is a paradigm clash." Its also not helpful --
and its incorrect -- to defend bogus ideas such as "numerical data arent
important" or "regression doesnt apply to complex nonlinear systems."
This last point is critically important. It may be true that simple linear
OLS regressions are inappropriate for complex nonlinear dynamic systems,
but econometrics is far more sophisticated than that and has been for
years. Could it be that some SD folks have an outdated and erroneous view
of the capability of these tools because they long ago decided they werent
appropriate? Could it be that SD graduate students arent required to take
enough econometrics, economics, OR, and other relevant courses because
their advisors have outdated opinions about the relevance and suitability
of these tools?
Second, where are the high-leverage policies? Im sure I dont have the
answer to that, but one possibility is to make the effort to cross the
boundaries of the disciplines. Make sure students learn the state of the
art in OR and economic tools, not only SD tools. Learn to read other
literatures and be sure to cite them when relevant. Most importantly,
update the opinions of those who have not followed the field by doing great
work, and then making the effort to communicate it outside the boundaries
of the field. Bring your enemies into the conversation. Seek, and listen
to, their advice. You will need a thick skin and a lot of persistence, but
the effort will improve both the quality of your work and its impact.
John Sterman
From: John Sterman <
jsterman@MIT.EDU>