All models are wrong (was Strategy magazine article about Ja

[Jean-Jacques Laublé jean-jacques]

Posted by =?iso-8859-1?Q?Jean-Jacques_Laubl=E9?= <jean-jacques.lauble@wanadoo.fr>
WAS: REPLY Strategy magazine article about Jay Forrester

Hi every body

Bruce Skarin writes:
< One of the reasons System Dynamics is so unique as an approach to problem < solving is that it recognizes its faults before its benefits (all models are < wrong, not all models are useless

It was probably meant 'not all models are useful, but this is not the point of my mail.

I wanted to know how SD'ers react to the affirmation 'all models are wrong', which is so often written in the SD literature. I am not interested in knowing if they acknowledge this assertion or not, because I presume they do while they understand what the word wrong is used for.

I think that the word 'wrong' is not adequate even in an SD context, but it is another question.

But I am interested to know if SD'er think it is a good idea to use such a sentence when communicating with people without SD knowledge. That all models are not useful is a truism valuable for about anything on earth, but at least the notion of usefulness is rather clear, unlike the notion of wrongness which is not.

Recognizing that all models are wrong is like exposing to a new buyer of a product that all the products have not the quality the buyer is more sensible to. Because the objective of SD is to give the buyer the 'correct' answer to his problem, saying him that all the models are 'wrong' is counterproductive and will make him think 'if they recognize at first that all models are wrong, it is certainly true otherwise they would not write it down, but there is a possibility that it can be worse than that and because my first concern is to have the right answer, I will not take the risk of a try.'

I try to sell my products all day long, and I know that people listening to my arguments, always try to find out what disadvantages I have not been talking about, and what the so called advantages I have been talking about are wrong. Did you ever see a publicity insisting on some negative thoughts like 'wrong'?

I know that writing the word publicity may not be the right word for scientifically oriented people, but let us not be angelic.

I am not talking about something which is true or not true but about something I have always felt reading the assertion 'All models are wrong'.

Best regards to everybody.
J.J. Laublé Allocar
Strasbourg France.
Posted by =?iso-8859-1?Q?Jean-Jacques_Laubl=E9?= <jean-jacques.lauble@wanadoo.fr> posting date Thu, 15 Sep 2005 15:59:59 +0200

[McConnell George (SELEX Comms) (]

Posted by ""McConnell, George (SELEX Comms) (UK)"" <george.mcconnell@selex-comm.com>

Hi all,

I often use the ""all models are wrong but some are useful"" quote (although I recognise Jean-Jacques' concern that this can give out negative vibes).

As I see it, there is a great danger in over reliance on models without understanding of the potential ""wrongness"". People who are not 'model literate' tend to attach a high degree of surety to models which may not be suitable for their purpose. If someone asks you if you prefer Pepsi to Coke they will take your reply and accept or disregard it in accordance to their own beliefs. If, however, you have seven linked spreadsheets and a Vensim model that show that Coke is 0.037962% more likely than Pepsi to result in an attack of hiccups, then they will believe that and amend their drinking habits accordingly.

This is linked to the fact that it is important to quantify the degree to which any model should be believed - some sort of meaure of GIGO-ness. If the inputs are accurate measurements, then by all means produce answers to serveral decimal points. However, if the inputs are guesses (sorry, estimates) then that needs to be reflected in an equal degree of imprecision in the outputs.

Too often the request is ""just build me a model"". hmmm... there is a lack of understanding that the model needs to be carefully constructed; it needs a purpose; it needs decsions on what to abstract out of the problem; it needs to know the required accuracy of the results and so on. Non model litereate customers will be only too ready to accept any model regardless of how useful it really is - as long as it gives them results. If the result comes from a model ""it must be right"" - the model says so.

It is this misconception that needs to be addressed by careful use of ""all models are wrong"".

regards
George _________________________________________________________________________
george.mcconnell@selex-comm.com +44 (0)1202 404824

EUR ING George R. McConnell CEng, FBCS, CITP, FORS, MIEEE, MSc, BA Senior OR Consultant - Explore OR: The Science of Better, www.scienceofbetter.co.uk SELEX Communications, Grange Rd, Christchurch, Dorset, BH23 4JE, United Kingdom Posted by ""McConnell, George (SELEX Comms) (UK)"" <george.mcconnell@selex-comm.com> posting date Fri, 16 Sep 2005 12:06:51 +0100

[John Gunkler jgunkler sprintmail]

Posted by ""John Gunkler"" <jgunkler@sprintmail.com> Jean-Jacques,

It's not just true in academia (which prides itself on self-criticism and the scientific method, which is fundamentally skeptical) but also in commerce that being honest about one's products' shortcomings pays off. There are many cases where savvy salespeople honestly tell prospective buyers about where their products do not ""shine"" -- of course, they also emphasize their products' benefits, and they mention where their competitors' products fail to shine as well.

It is always a good idea to set people's expectations -- this helps prevent warranty and return problems later, which are very expensive to deal with. Letting people buy from you in the belief that your products will do more than they will is very dangerous to your bottom line.

Amazingly, people still respond to honesty these days -- probably because it's so damned rare.

As Mark Twain once wrote, ""Always tell the truth. This will gratify some people, and astonish the rest."" Posted by ""John Gunkler"" <jgunkler@sprintmail.com> posting date Fri, 16 Sep 2005 09:23:37 -0500

[Bill Braun bbraun hlthsys.com]

Posted by Bill Braun <bbraun@hlthsys.com>
Noting that ""all models are incomplete, some are useful"" has been received well and understood for the most part.

Bill Braun
Posted by Bill Braun <bbraun@hlthsys.com>
posting date Fri, 16 Sep 2005 07:38:00 -0500

[Paul Holmström ph holmstrom.se]

Posted by Paul Holmstr=?ISO-8859-1?B?9g==?=m <ph@holmstrom.se> I guess that many system dynamists are more humble or honest than most salesmen. Like you I am both.

When I sell anything I want the customer to be mentally present and not have their thought wandering away, thinking about what I deliberately excluded or misrepresented. So I am very straight-forward, which builds trust. Personally I mistrust everybody from vacuum cleaner salesmen to sd-model builders who represent their product at being the best thing since the invention of sliced bread.

Everybody seems to know that all models are ""wrong"", so I build on that natural suspicion. When I start talking about models I usually say that they are narrow representations of reality. Often I quote Stafford Beer who wrote quite a bit about models and their usefulness. One of his examples is the airplane model used for wind tunnel tests. It can be criticized for not having an interior at all etc. Pilots use a cockpit for flight simulation. Crews use a fuselage for training onboard service and emergency escape. All these models extremely useful for the particular purpose for which they have been created.

So from that point of view models are usually more wrong than right. The right being that particular slice of reality which we have chosen to model. We need the attitude of the honest marketer and the academic rigor of telling what is included, what is deliberately excluded and the meaning of that.

Regards
Paul Holmstrom
Posted by Paul Holmstr=?ISO-8859-1?B?9g==?=m <ph@holmstrom.se> posting date Sat, 17 Sep 2005 13:39:03 +0200

[Martin Schaffernicht martin utal]

Posted by Martin Schaffernicht <martin@utalca.cl>
Jean-Jaques,

I believe there are two different groups SD talks to. One is made up of people who are interested in SD itself, in becoming ""dynamicists""; in the other group there are those who have aproblem and want to know what best to do.

The ""all models are wrong"" message was directed to the first group, just like the chapter on model validation in ""Business dynamics"", and in my mind this is a reflex of the sufferings of a discipline that believes in remainig open to (double-loop) learning, but has to do with ""clients"" who demand to know what is the correct thing to do and want ""truths"". In my imagination this is a bit like the story of development aid: ""don't give them fish, teach them fishing"".

I agree with you that for ""selling"" SD to people of the second group, one has to transmit confidence in order to be trustworthy. Here in Chile it is rather hard to ""sell"" since people like to buy what they already know, and so many prefer econometry. As a result I would not talk a lot about SD when trying to ""sell"" aproject.

I count myself as member of the first group, and I ""force"" my students (undergraduate business students) to get used to this idea. Luckily in Chile we have quite a set of good examples of strage things that happen whan decision makers do not pay attention to the limited validity of the models they use, so with two or three stories taken from recent local news, I think everyone gets the point.

So even if I agree that this kind of talk may better be limited to the second group, I personally enjoy it. However, one thing I often wondered about is that in order to have your work published in the ""system dynamics review"", you better not send in ""my wrong model"", and I don't know of an article that showed how a modeling team learned from understanding where a previously validated (accepted) model had be found to be invalid at a later point in time.

Best greetings from Chile,

Martin Schaffernicht
Universidad de Talca
Talca - Chile
Posted by Martin Schaffernicht <martin@utalca.cl>
posting date Fri, 16 Sep 2005 10:17:50 +0200

[Bruce Skarin bruceskarin hotmail]

Posted by ""Bruce Skarin"" <bruceskarin@hotmail.com>
Thank you J.J. for sending your reaction. Very often when we try to explain a perspective, people interpret it a lot of different ways, but without receiving reactions it is hard to refine the explanation. In this case your reaction will perhaps help to illustrate my view of the statement.

I agree that beginning any kind of sales pitch with a negative is not the best strategy. But in the case of an argument, it is almost always necessary.

My approach is that I’m not in the market to sell SD. Instead my effort is to point out that we have made a lot of mistakes and we are likely to make many more. Yet, there are better ways to do things so that mistakes aren’t so bad or so frequent.

Perhaps ‘all models are wrong’ is a bit too strong. Instead ‘all models are incomplete or imperfect,’ but I still think it is important to start with this point because it seems that it is always what is forgotten.

Why should I change what I’m doing if it seems to work or nothing seems wrong? I find that nearly all innovation comes from someone recognizing that there is better way to do something. And when an innovator suggests the new model without first explaining the shortfalls of the previous model, there is usually more resistance.

A consultant or salesperson is very often in a different position, since the customer already knows that they need to do better or need something new. Yet in SD I think it is most responsible to not promise the ‘last perfect solution you’ll ever need’.

It is important to me that others know when I propose a new model or perspective that it is inherently flawed. That it will always require his or her input, refining, and most importantly ownership in order for it to help. And finally to impart that eventually it may need to be discarded and replaced.

This is perhaps a more difficult path, since many people are looking for that easy explanation. That perfect fix to their problems. But the sooner we face the reality that it is a never ending process, that there is no real finish line. The sooner we can get back to trying to make it better.

This may not change how you feel about the statement when you hear it, but perhaps that’s not such a bad thing. I find that I usually get a stronger reaction when I challenge a perspective as opposed to reinforcing one.

So how do I think J.J.’s response illustrates this point? If he hadn’t suggested that the statement ‘all models are wrong’ was not the best start to explaining SD, I wouldn’t have changed my explanation.

-Bruce
Posted by ""Bruce Skarin"" <bruceskarin@hotmail.com>
posting date Fri, 16 Sep 2005 09:06:59 -0400

[Tom Fiddaman tom vensim.com]

Posted by Tom Fiddaman <tom@vensim.com>
I usually hear this expressed as ""all models are wrong; some models are useful."" I must confess that I've grown a little wary of this statement, though not because it's bad marketing. Instead, I've heard it used as an excuse for mediocre models, in the spirit of, ""our model stinks, but what the heck ... they all stink."" Some models are a lot more wrong than others.

I think it's more productive to talk with clients about ways of building models to avoid dumb errors (e.g. units checking and extreme conditions tests), testing as many alternative ideas as possible against data (in a broad sense), learning from the process, and tracking performance for future improvement. They likely already know that models are wrong, because the world is so polluted with bad ones.

Tom

****************************************************
Tom Fiddaman
Ventana Systems, Inc.
Posted by Tom Fiddaman <tom@vensim.com>
posting date Fri, 16 Sep 2005 14:32:34 -0600

[Jean-Jacques Laublé jean-jacques]

Posted by =?iso-8859-1?Q?Jean-Jacques_Laubl=E9?= <jean-jacques.lauble@wanadoo.fr> Hi every body. Thank you to all people that answered my question. It gives me a better idea of the intention behind the formula : all models are ... Just two remarks: When I started in the business, being a scientist, I considered it a dishonour to disguise reality when dealing with clients, bankers, workers, suppliers etc. It took me a very long time, and I am still not totally cured, to understand that it could be useful to be honest at the condition that your interlocutor believes you. If he is not able to recognize the truth and your honesty, being honest has no immediate advantage and can be counterproductive. Of course being known as a liar, is not the solution. Being too concerned by self-honesty hinders the ability to consider the position of your interlocutor and his reactions. Not to manipulate him, but to influence him in a direction where I think it is his interest to go. So before telling the truth I just take some time to think about it and consider the context.

Besides I consider that the qualities that are necessary to be a good modeller (honesty, knowledge, experience etc.) are not necessary the same to sell a service that you consider as being useful for you client (this is the place where honesty is necessary). So I would separate modelling in three phases (one to verify that the client really needs my service), one to convince him and one to do the job.

I do not mean that being honest all the time is not a good method. It is certainly very courageous and can work too. There is no definite way to sell something and it depends on the service, the client, the competition, the kind of client's problem.

Another remark about the formula: all models are wrong, not all models are useless or some models are useful. There are several negative ideas. First 'all models are wrong' can be interpreted very differently depending on the knowledge of the interlocutor.

Secondly, pairing 'all models are wrong' with 'not all models are useless' or 'some models are useful', suggests that normally a model being wrong implies that it is useless. This gives automatically an incorrect importance to the notion of wrongness. And in the interlocutor mind, now there is : wrong is normally useless and all models are wrong, which implies all models are normally useless.

Thirdly: Saying that not all models are useless, implies that the majority of models are useless which is not positive.

So on one side, you have three strong negative assertions and on the other side, you have a weak assertion that is evident: some models are useful. The client certainly thinks that some models may be useful. It is like saying : Excuse me, all models are wrong and by deduction should be useless, but fortunately some come to be useful.

I hope that you will excuse my explanation that is a bit exaggerated, but it is to be better understood. Regards to everybody. J.J. Laublé. Allocar Strasbourg. France Posted by =?iso-8859-1?Q?Jean-Jacques_Laubl=E9?= <jean-jacques.lauble@wanadoo.fr> posting date Sat, 17 Sep 2005 17:20:41 +0200

[Martin Taylor mmt mmtaylor.net]

Posted by Martin Taylor <mmt@mmtaylor.net>
""All models are wrong; some models are useful."" Models can be wrong for at least two very different reasons, the first being that they don't cover the full range of phenomena of interest, or do so with low fidelity. That's not too serious, and the incompleteness or possible inaccuracy can be presented openly and fairly.

The second way a model can seem to be wrong is that the effects of nonlinearity may be important. If the model has slightly incorrect nonlinearities, or slightly wrong parameter values, its behaviour could be qualitatively different from that of the world it is modelling, even though its structure might be exactly correct.

The evolved real world (as opposed to the constructed world) seems often to have parameter values near some critical point near the border of a chaotic regime, and it seems to me that realistic models ought to have some of that same quality. With the right parameter values the fit to real-world behaviour could be excellent, but a small error in the parameter values could make the model behave very differently. The likely tendency of the analyst would then be to think that the model structure was wrong, rather than that the parameters were chosen unfortunately by a few percent.

""All models are wrong; some models are useful."" But even a model that is correctly structured might be useless if slightly wrong parameters put its behaviour the wrong side of a bifurcation point in its dynamical space.

I guess this doesn't have much to do with talking to clients, but I think it's worth keeping in mind. Few real systems are linear!

Martin Taylor
Posted by Martin Taylor <mmt@mmtaylor.net>
posting date Sat, 17 Sep 2005 10:24:10 -0400

[George Richardson gpr albany.edu]

Posted by George Richardson <gpr@albany.edu>
I tend never to say ""all models are wrong.""

My problems with the statement are that the people I'm speaking with don't usually have the same meanings and attributions to the word ""model"" that I have, or that John Sterman has when he reminds us all with this phrase.

I worry that people will hear ""mathematical models or computer models are wrong.""

But they will persist in thinking that their own thoughts aren't subject to any similar sort of disclaimer. I worry they won't think ""all models -- mental or formal -- are wrong."" If your audience doesn't interpret the phrase this way then it's a really easy slide for them to think ""This formal modeling stuff is inherently flawed, but fortunately my thinking, my mental models, are not as weak and flawed.""

I find it more useful to focus on the *thinking* that we're trying to help. If a formal model can help people think better, and they recognize that, then we never have to raise the nasty thought that the model that is helping is actually ""wrong.""

Focusing on the thinking comes to a head for me (don't you love it when you type something that amazing), when I compare the phrases ""all models are wrong"" and ""all thoughts are wrong."" We'd never agree with the second one. In fact, we're trying to use modeling and simulation to help people have better and better thoughts. We're trying to help them think rightly, in some sense.

So let's focus ourselves and our clients on the thinking that our models and analyses help engender and communicate, and not raise the specter that the formal model that is helping is wrong.

..George

George P. Richardson
Chair of public administration and policy
Rockefeller College of Public Affairs and Policy
University at Albany - SUNY, Albany, NY 12222
Posted by George Richardson <gpr@albany.edu>
posting date Sat, 17 Sep 2005 13:38:56 -0400

[Peter Hovmand phovmand wustl.edu]

Posted by Peter Hovmand <phovmand@wustl.edu>
Martin's response and George's reframing of the issue on thinking reminds me of long standing discussion in philosophy of language on the meaning of a word that has a lot to do with how we talk with clients about models being wrong. The statement ""All models are wrong"" is a playfully ambiguous proposition that only takes on a specific value once we are clear on the context in which the question, ""Is this model right/wrong?"" is understood.

It's important to understand what they mean by such terms because they will eventually get applied or emerge in discussion with their colleagues. If we don't know what would be a useful model for the clients and don't have a context for understanding their problem, then it is unlikely we will have a clear sense of what they are expecting or what will be useful for helping them think about a complex problem. Also note that sometimes the statement ""All models are wrong"" gets used to obfuscate bad modeling.

Peter
Posted by Peter Hovmand <phovmand@wustl.edu>
posting date Sun, 18 Sep 2005 10:22:09 -0500

[Justin Lyon justin1028 yahoo.com]

Posted by Justin Lyon <justin1028@yahoo.com>
Hi everyone,

It is a difficult challenge to sell and market system
dynamics. As a sales agent for several SD firms, I
thought I’d share some of my thinking.

Here's how we do it in the UK. I'd love to get
feedback from the community.

When we’re meeting with executives, we try to focus on
the challenges they are facing. It might be a desire
to improve the output of an oil field or to cut
operational costs in a bank by closing branches or to
optimise marketing spending across a portfolio of
brands.

In many cases (if not all), we're looking to release
cash for the owners of the business through
coordinated and integrated action across the
enterprise.

On behalf of HVR Consulting Services, Global Strategy
Dynamics, SDS and my other clients, I then explain
that Simulation Science helps corporations:

1. Optimise strategic planning;

2. Optimise key processes across the extended
enterprise so that the whole is greater than the sum
of the parts;

3. Optimise the management of people to increase
efficiency and generate more cash for the owners of
the business; and

4. Optimise leadership by equipping them with the
simulation technologies and management science that
enables them to create self-reinforcing business
advantage.

Fair enough, they might reply, but everyone promises
that! And, here is where the ""all models are wrong,
some are useful"" angle comes in.

Selling simulation science involves:

1. Connecting with people in the account;
2. Convincing them of your value via sales tactics;
3. Converting them into paying clients and
4. Building up and Retaining their loyalty.

Typically, I find myself talking to three types of
people in an account:

1. Executives and managers
2. Analytical specialists
3. Consultants and other influencers

Executives are not really that interested in
discussions on the models other than being convinced
that the dollars spent developing them will make a
substantial return on the investment for the owners.

That is, executives and managers are charged with
releasing cash for the owners of the business using
any and all appropriate techniques.

Analytical specialists, consultants and other
influencers are expected to debate and agree on the
appropriate techniques to help executives accomplish
that primary goal. That is, they are expected to
understand why models are wrong, yet useful to helping executives manage the business.

As Tom Fiddaman points out, many models are rather
poor (useless) and some are better (useful).

You can use a variety of techniques to create models.
The key is then to quantify the usefulness of the
model.

Techniques in the Simulation Science space are
particularly good at generating large returns on
modest investments (often less than £3 million).

Even more profitable are the System Dynamics' case
studies.

Over the past forty years, an amazing body of evidence
(a stock) has been built up regarding the success of
system dynamics' models to solving complex problems.

My clients alone have over 85 detailed case studies on
applying system dynamics to solving complex problems.

By choosing appropriate past projects and discussing
them in some detail on how they generated their
substantial returns, we usually find it possible to
overcome prospects’ consequence issues relating to
their investments in simulation technologies. The
returns on simulation investments are usually so large
that it is a 'no brainer' decision to move forward
with our solutions. (These case studies are available
upon request.)

Obviously, this is only part of the picture when
handling major account sales strategy, but it seems
clear to me that this community's techniques usually
generate a positive return on the investment.

As John Sterman said when he accepted the 2002
Forrester prize, ""What prevents us from overcoming
policy resistance is not a lack of resources,
technical knowledge, or a genuine commitment to
change. What thwarts us is our lack of a meaningful
systems thinking capability. That capability requires,
but is much more than, the ability to understand
complexity, to understand stocks and flows, feedback,
and time delays. It requires, but is much more than,
the use of formal models and simulations. It requires
an unswerving commitment to the highest standards, the
rigorous application of the scientific method, and the
inquiry skills we need to expose our hidden
assumptions and biases. It requires that we listen
with respect and empathy to others. It requires the
curiosity to keep asking those 'why' questions. It
requires the humility we need to learn and the courage
we need to lead, though all our maps are wrong. That
is the real purpose of system dynamics: To create the
future we truly desire--not just in the here and now,
but globally and for the long term.""

Thoughts?

Best,
Justin Lyon
Posted by Justin Lyon <justin1028@yahoo.com>
posting date Sun, 18 Sep 2005 06:31:30 -0700 (PDT)

[Jim Thompson james.thompson stra]

Posted by ""Jim Thompson"" <james.thompson@strath.ac.uk>
Isn't ""all models are wrong"" a reduction of a complicated set of conditions to a descriptive assertion, as in ""Model = FALSE""? If so, it's not too useful.

Posted by ""Jim Thompson"" <james.thompson@strath.ac.uk>
posting date Sun, 18 Sep 2005 11:07:38 -0400

[Mabel Fong may_belle_66 yahoo.co]

Posted by Mabel Fong <may_belle_66@yahoo.com>

Perhaps some further illumination on this topic might
follow from the question “What would constitute a
CORRECT model?”

Obviously the only completely correct model of a
system would be the system itself; and this singular
‘model’ can be problematic insofar as it might be
difficult to interrogate in terms of its responses to
novel stimuli – especially for those stimuli by which
a vital system might be destroyed.

Could we not then go on to the much more strong
statement that model building begins with a resolve to
be wrong? Is science not mostly in the business of
SUBTRACTING from what is there to be observed so as to
reveal what are the essential causes of behavior? Are
not models most likely to be foolish for their
predicting behavior modes that never happen?

Sorry to go all Zen on you here; but I think this
question is not going to be fully explored until we
turn it inside out for a while.

Best,

Mabel
Posted by Mabel Fong <may_belle_66@yahoo.com>
posting date Sun, 18 Sep 2005 06:05:02 -0700 (PDT)

[geoff coyle geoff.coyle btintern]

Posted by ""geoff coyle"" <geoff.coyle@btinternet.com>
The usual lucid good sense from George Richardson. The lad is showing promise and, in another 10 or 20 years ...

Seriously, the 'models are wrong but can be useful' phrase might be used when students are just starting, or as Senior Common Room banter, but it is potentially harmful if used too much. For instance anyone who said that an aircraft flight simulator is 'wrong' because it hasn't got a passenger cabin misses the point. It is, in fact, perfectly designed for its purpose, as extraneous factors have been simplified out, and an analytical model should be DESIGNED in the same way.

In a paper on the validation of commercial SD models (SDR, Vol 16, No 1, Spring 2000), David Exelby and I tried to distinguish between 'validation' and 'verification'. We suggested that validation meant the model was fit for its purpose of answering some well-chosen questions (a phrase that George and Jack Pugh used in their excellent text). One aspect of that is correspondence to the clients' (more than one of them ) mental models. Thus, validation is a matter of judgement. One the other hand, verification, we suggested, means that the model is working properly and that is a technical issue. It involves dimensional validity, extreme value tests and, something that is too little used, mass-balance tests. There's a lot more too it than that, of course. I do, though, recall a 'model' in which a variable that should have varied between 0 and 1 actually ranged between -3500 and -1500 (and I do mean minus!!!) We need to be ultra-careful about verification and not try to do it by fiddling with the stock/flow icons.

On mental models, I think that there is a research question. In practice, I suspect that we represent clients' mental models using the tools that SD has habitually used for that task. Often, but not exclusively, that involves using non-linear graphs and, quite frequently, multiples of such non-linearities. What other tools could we develop to make sure that we have represented the mental model and not SD's mental model of a mental model? Quite tricky, perhaps.

Regards,

Geoff

Visiting Professor of Strategic Analysis,
University of Bath
Posted by ""geoff coyle"" <geoff.coyle@btinternet.com>
posting date Mon, 19 Sep 2005 15:59:10 +0100

[Bruce Skarin bruceskarin hotmail]

Posted by ""Bruce Skarin"" <bruceskarin@hotmail.com>
>The “phrase might be used when students are just starting, or as Senior
>Common Room banter, but it is potentially harmful if used too much.”

I agree that overuse of anything is potentially harmful, but being relatively fresh from the ‘Common Room,’ I must take issue with the first part of this reply.

I think that one of the key problems that senior policy makers and managers make is assuming that somehow through years of experience they have developed a nearly ‘perfect’ understanding of business, education, etc. In reality one may have a fairly robust model, but it is still only a model that can easily be thrown off by new insight or some fundamental change in the market, organization, environment, etc.

There is often a great deal to add to seasoned experience in light of a fresh perspective that is aware that mistakes have, can, and will be made. I think it would be folly to not keep a humble awareness of the limits of our ability to solve problems using System Dynamics.

I also agree with George’s comments that focusing on the negative is not terribly productive. Yet the point I’m trying to make (and seeming to be
missed) is that many times you have to make the argument before you can be productive. Moreover, just because a model works well for its purpose, does not mean that you can’t do better.

Case in point: I actually had the opportunity to fly a multimillion dollar flight simulator at NASA Langley during a visit in the late 80’s. Now even though I’m not a pilot or flight simulator designer, I’m fairly confident that the complexity and realism have significantly improved since then, providing more effective, wider ranging training options.

Geoff’s comments on validation and verification are nice distinctions to make when evaluating models, but perhaps it should also include the need to revalidate periodically. Without revalidation, we would not have likely progressed from Newtonian mechanics to Relativity and beyond.

So Geoff and George, your warnings are well noted and regarded, but I hope mine is as well.

Highest regards,

-Bruce
Posted by ""Bruce Skarin"" <bruceskarin@hotmail.com>
posting date Tue, 20 Sep 2005 08:57:18 -0400