Is it possible to omit soft variables?

This forum contains all archives from the SD Mailing list (go to http://www.systemdynamics.org/forum/ for more information). This is here as a read-only resource, please post any SD related questions to the SD Discussion forum.
Locked
"Richard"
Junior Member
Posts: 3
Joined: Fri Mar 29, 2002 3:39 am

Is it possible to omit soft variables?

Post by "Richard" »

At last some sense from Jay on soft variables.

Much of the significant SD work my company (Cognitus) does for clients is
focussed on the identification and significance testing of strategic
variables for which little hard data exists, or where the impact of such
variables is imperfectly understood. The great value of SD, as opposed to
most other forms of modelling, is that it is an open method that enables
real managers to engage in discussion and debate around a structure - and,
in particular, to develop understanding of the real significance of soft
variables. Many SD practitioners, however, seem to be more concerned with
how many angels fit on a pin-head, than in using the methods and tools to
help real managers make decisions in an imperfect world.

There is thus a tendency for managers to think of SD in terms of old style
OR (and hence to compare/confuse it with detailed analytic tools such as
econometrics}, rather than as a powerful and structured method to develop
better shared understanding (however imperfect) of how things really work
in order to improve decision-taking. Another real problem is that SD
practitioners are often perceived - and portray themselves - as modellers.
Nobody disputes the need for competent modelling skills but good SD
practitioners need a wider range of skills, not least the skills to
facilitate diverse management groups and to communicate with them using the
language of SD. This implies that practitioners need to develop the
authority and presence to work at more senior levels, and they need the
confidence and skills to keep things simple. Few, in my experience, have
done so.

Simple SD models that address specific issues often yield much greater
insights (and therefore have more influence) than large, opaque models that
include rigorously defined data structures. Indeed I suggest that a good
objective for any SD practitioner should be to build small models - the
smaller the better, consistent with the perceived needs and realities of the
sponsoring management group.

A particularly good example of this in our consulting work, concerned
helping the managers of a huge North Sea oilfield understand how to add
value to the field over its remaining life through interdependent operating
and capital expenditure policies. A major uncertainty concerned the future
behaviour of the oil reservoir - although the engineers had hugely complex
models and vast amounts of data. In the context of future value, the
management team thought it adequate and appropriate to condense all this
data into one single graphical function in an SD model, and to test the
impact of minor variations on the value outcomes. Having thus identified
where the main sensitivities lay, the team were then able to focus the
detailed reservoir simulators at critical uncertainties. The SD model was
used interactively by the management team as an 80/20 calculator - to
develop shared understanding of the issues, and not as a replacement for
more detailed tool sets.

From: "Richard" <
richard@cognitus.co.uk>
Abboy D
Junior Member
Posts: 2
Joined: Fri Mar 29, 2002 3:39 am

Is it possible to omit soft variables?

Post by Abboy D »

"Abboy Dhanakodi" of Extended Answers wrote:

I perfectly agree with Dr.Forrester in that subjective
quantification of soft variables is better than no or infinite
quantification. Almost every science quite specifically like
organisational behaviour or psychology, have all started about
30 years ago with several variables like motivation, involvement
etc to be subjectively described and several hypethesis had been
proposed. With advent of Simulation software development, the
availability of lowcost powerful computers have played a
significant role in helping to quantify what was unthinkable
earlier. Therefore even if we start subjectively quantifying
certain parameters now, it will help in future in refining it or
to look at the structure of system more critically.

Secondly quite lot of people are quite averse to quantify
something which they feel is understood subjectively. When I
say subjectively, it does not necesarily mean it is biased. It
is more often intuitively said. Even Einstein has said he felt
that his famous energy equation was intuitively felt first and
later he marshalled it by quantifying it in equations.
Therefore to block such intuitions quantified, would surely be a
backward step. Many times, I felt, in trying to quantify
something, more insights are gained. In SD, we are writing the
critical equations in a simple graphical way to enable the
complexity structured easily and that interdependancies can be
better understood. And now to coax such intuitions into the
simulation models would be the best thing that can occur.

From: Abboy D <extans@yahoo.com>

Abboy Dhanakodi
Extended Answers
Y23/B1, 5th Avenue,
Anna Nagar,
Chennai (Madras) 600040
INDIA
Telefax: 91 44 6283708
"geoff coyle"
Senior Member
Posts: 94
Joined: Fri Mar 29, 2002 3:39 am

Is it possible to omit soft variables?

Post by "geoff coyle" »

Folks,

This important question seems to come up from time to time and gets many
excellent comments, such as Jays own remarks. My take on it is that, of
course, one cannot omit from an analysis variables which cannot easily be
quantified. If a variable such as integrity (or perceived integrity) is
relevant to the question which the analyst is trying to study then it should
be there. Perhaps, though, we sometimes too glibly assume that the soft
variable is continuous. It might be that integrity (and I dont know why Jay
chose that for his example) is a binary variable - a person is perceived as
having it or is not so perceived. There is also the need to think how a
really soft variable might be applied. Let us consider sobriety and agree
that 1 represents a teetotaller and 0 is a Central Park wino (but is the
teetotaller a recovering alcoholic or a strict believer?). What is 0.5? If
you define that, there are still the questions of that is 0.25 and 0.75, and
so on.

Ive used a similar example in a case on Y2K which involved the ability to
implement contingency plans in case of Y2K failure. One might take 1 to
imply that all contingency plans can be fully implemented and 0 to mean that
none can be implemented. Again, what is 0.5? Does it mean that half the
plans can be implemented, but which half? Does it mean that all the plans
can be half implemented, but what is half a plan? I do no more than suggest
that we need to think through such problems before happily drawing a
non-linear curve.

However, there is also the issue of how to use soft variables and there we
always beg the question by assuming that a model has to be a computer
simulation (in our discipline of a problem to do with dynamic behaviour). In
general, a model is a simplification of reality which is intended to act as
tool for thought about a reasonably well-defined problem. Thus, an ordinary
map is a model of geography and is a valid form of model and there are other
valid forms. We should accept that in SD influence diagrams and CLDs are
forms of model and we should not make a distinction between them except
insofar as both forms have characteristic advantages and limitations (and I
am NOT getting into the old and DEAD argument about whether one can predict
dynamics by looking at a diagram, of course one cannot do so). The argument
against too freely using soft variables in simulations is that the
inevitable uncertainties may be so great that the outputs from the model
(sensitivity and the significance of structure notwithstanding) may be so
dubious that one cannot make robust recommendations. It may be, in some
cases, that the only safe thing to do is to use soft variables in a diagram
and leave it at that, relying on insights emerging from the system
description. Of course, those insights will be uncertain but there may be a
danger of multiplying that uncertainty by simulation and I simply suggest
that such may not be the most responsible approach. This is clearly an issue
which is crying out for some substantial research and we should not rely on
anecdote.

I have a comment in the next issue of SDR which goes into this a little more
deeply and I hope that you will find time to read it.

I now confidently await the avalanche of Coyle knows nothing about SD.

Regards,

Geoff
From: "geoff coyle" <
geoff.coyle@btinternet.com>
shayneg@agsm.edu.au
Junior Member
Posts: 2
Joined: Fri Mar 29, 2002 3:39 am

Is it possible to omit soft variables?

Post by shayneg@agsm.edu.au »

Jay has already addressed the issue of omitting soft variables quite
effectively, but I wanted to clarify one point on operationalizing soft
concepts that Jim Thompson mentioned.

It is perfectly reasonable to include soft concepts as endogenous variables
or parameters (constants) provided the concepts capture a meaningful,
unambiguous part of the real system. Soft parameters often show up in the
behavioral policies represented in the model. For example, a Management
Bias parameter might well be used to capture the impact of certain
characteristics of the management team or board on organizational
decision-making (e.g. aggressive or conservative regarding capital
investment). Goals, targets, and perception delays are also soft
concepts that may or may not be endogenous variables. The salient point is
that EVERY concept in the model must be unambiguous and have a direct
counterpart in the real system.

---
Shayne Gary
Australian Graduate School of Management
UNSW SYDNEY NSW 2052
Australia
Tel: +61 2 9931 9247
Fax: +61 2 9663 4672
Email: shayneg@agsm.edu.au
Web: http://www.agsm.edu.au
"Jay W. Forrester"
Senior Member
Posts: 63
Joined: Fri Mar 29, 2002 3:39 am

Is it possible to omit soft variables?

Post by "Jay W. Forrester" »

At various times in this discussion forum, people have suggested that
one should not include in models soft variables that lack
quantitatively measured values.

I suggest that such omission of soft variables is not possible. If
one "omits" a variable, it has a very specific assumed value in the
model. It is being set to zero or to some other value that
inactivates the structure of which it is a part. To leave out the
variable or concept is to say explicitly that it has no importance.
Often zero significance is the most unlikely of the possible
subjective estimates.

A model is built from a structure within which numerical values are
asserted within the policies that govern action. Structure and
parameters are at least partially interrelated. By setting
parameters to zero or infinity, one can inactivate and thereby remove
structure from a model. But are those zero and infinite values the
most likely for parameters for which we have no previously
numerically measured values? Example: integrity is considered
important in hiring people, placing orders, and writing contracts,
but usually there are only comparative subjective estimates of
relative integrity.

Also, structure is at least as important, probably much more
important, than small refinements in parameter values. And, so far
as I know, there is no automatic quantitative way to establish the
relevant structure of a system that is the most efficient and
effective structure for a model that deals with a specific problem.

So, if the structure is to be determined subjectively on the basis of
experience, judgement, and observation, why should not parameters
that are only available subjectively not be used? Otherwise, one is
biasing the structure to fit a very limited body of numerically
measured data, rather than allowing the model to best represent the
vastly richer body of available data in the form of observations,
incidents, logic of extreme values, experience, and knowledge of
those intimately familiar with the system under study.

After one has a model, sensitivity analysis can then be used to
explore the importance of parameters and changes in structure. If a
parameter is found to be highly sensitive with regard to the
objectives of a model, remedial steps can then be taken. Perhaps
better measurements of the parameter can then be made, or quite
probably, the sensitivity arises because the structure within which
the parameter has been placed is not robust and the surrounding
structure should be re-examined.
--
---------------------------------------------------------
Jay W. Forrester
Professor of Management
Sloan School
Massachusetts Institute of Technology
Room E60-389
Cambridge, MA 02139
From: "Jay W. Forrester" <
jforestr@MIT.EDU>
Tom Fiddaman
Senior Member
Posts: 55
Joined: Fri Mar 29, 2002 3:39 am

Is it possible to omit soft variables?

Post by Tom Fiddaman »

Im with Jay on this one. Omission of unmeasured variables is not a helpful
strategy for most purposes. Including soft variables in models may subject
us to some uncertainty, but if we exclude them we cant even talk about 90%
of the interesting problems in the world.

The technical distinctions between SD and econometrics have grown quite
fuzzy. Whether youre an econometrician doing vector autoregressive models
or a system dynamicist using Kalman filtering and optimization youre using
the same underlying methods. Just hand-tuning a model to fit data boils
down to much the same thing as least-squares fitting in many cases.

However, there are still big differences in practice. If you look in an
econometrics journal, you will find models populated entirely with measured
variables. If you look at data-intensive SD work, you are likely to find
that only a fraction of variables are measured. Its OK to have minimal
measurement as long as you go to the trouble to compute confidence bounds
etc. so that you have an appreciation of the quality of the answer.

I think we have many important things to learn from econometrics. Two examples:
-Because econometrics is (at least historically) an analytical rather than
simulation approach, practitioners spend a LOT of time thinking about
functional forms. We in SD are quick to throw in an Us/(Us+Them) market
share allocation with a bunch of multiplicative factors determining
attractiveness, econometricians have spent years developing alternatives
with nice estimation properties (not interesting to us) and controllable,
robust behavior centrally and in extremes (critical to us).
-Econometricians have a high appreciation for data. Its enormously slow,
hard, and messy taking junk from public statistics and enterprise databases
and turning it into something usable. I thank my lucky stars every time I
find an economist whos already compiled and cleaned some macro time series
I need. Drawing a few reference modes is just way to easy, and precludes
the insight that can come from chasing puzzles in the data.

In return, I think we have many things to offer:
-We tend to use a much broader set of quality checks on models, e.g.
dimensional consistency, extreme conditions tests, etc.
-Our emphasis on simulation leads to a strong appreciation of dynamics and
weeds out many flawed model structures without wasting effort on data
collection. It also makes it vastly easier to connect the results of our
analysis to the bottom line, whatever it may be, without a lot of
intervening hand waving.
-When SD practitioners do estimation, theyre much more likely to use
methods with full appreciation of nonlinearity and feedback
-We have a large body of work containing nice, relatively reusable
formulations, especially for soft variables. Too bad its not more accessible.

Tom

P.S. For a little historical irony, read William Nordhaus critique of the
Limits to Growth, Measurement without Data (in the Economic Journal,
December 1973). He makes exactly the criticism in the Fontana entry, and
spectacularly demonstrates the foibles of econometrics at the time. If I
remember right, he takes output of World3 for income (goods per capita) and
fertility, and runs a regression on it. Because he ignores the delay
structure of the determination of desired family size he concludes wrongly
that rising income must increase the birth rate in the model. He could have
saved himself a lot of trouble by looking at the equations.

If we cant change the Fontana dictionary entry for SD, maybe we can get
the entry for econometrics changed to read "a style of inquiry in which
users gain bogus credibility by using weak numerical methods on flawed data
to assign precise values to dimensionally incorrect parameters in
irrelevant models." :)

****************************************************
Thomas Fiddaman, Ph.D.
Ventana Systems http://www.vensim.com
8105 SE Nelson Road Tel (253) 851-0124
Olalla, WA 98359 Fax (253) 851-0125
Tom@Vensim.com http://home.earthlink.net/~tomfid
****************************************************
"Jack Ring"
Junior Member
Posts: 12
Joined: Fri Mar 29, 2002 3:39 am

Is it possible to omit soft variables?

Post by "Jack Ring" »

Good point.
And even if some of the soft variables cannot be calibrated to "actuals"
the variables can still be quite educational as "what if" levers.

From: "Jack Ring" <jring@amug.org>
Locked