Application of SD to Risk Manageme

["Taylor, Roger"]

Does any one have any experience they can share of using a SD approach to
build an understanding and analysis of risk and the possible impact of
interventions to manage risk.

My experience in SD is relatively limited, although I took a paper in the
subject in my MBA a few years back and have built some high level I Think
models of parts of the electricity sector when I was working there.

The causal relationships between risk drivers and their diverse
consequences and the unforeseen impacts of some management interventions
leads me to believe there is a place for a SD approach.

I would welcome any feedback.

Roger Taylor
KPMG
Wellington, New Zealand
rtaylor@kpmg.co.nz

["Keith Linard"]

>Does any one have any experience they can share of using a SD
>approach to build an understanding and analysis of risk and the
>possible impact of interventions to manage risk.

When one moves from using SD as a mutual learning tool (creating
shared mental models with and among stakeholders) to using it as a
planning tool, identifying (& recommending) policy intervention
points, I regard the use of probabilistic analysis to be an essential
supplement to SD modelling.

My research during the 1980s into the most concrete area of
parameter estimation/forecasting - building and construction cost
estimation - showed a standard deviation (between pre-tender
estimate and actual inflation adjusted costs) of the order of 30%
consistent across a range from around $50,000 to $50,000,000
projects. My reviews of the literature and ex post evaluations of
risk and uncertainty associated with other parameters (eg return
period of floods of given magnitude, growth in passenger numbers,
propensities to consume etc etc) suggest even greater variability
in these more qualitative areas.

Use of deterministic analysis is equally, if not more, flawed in
forensic system dynamics than in traditional ops
research methodologies. In particular, the dynamics of a given
system depend critically on the relative strengths of
interrelated positive and negative feedback loops ... variability in
parameter values may change the dominance of feedback loops.

Hitherto I have addressed the risk analysis by combining the Powersim
SD software with the Evolver Genetic Algorithm software, using Visual
Basic as a linking mechanism - start the process on Friday night and
come back Monday morning to see the results.

The shortly to be released version of Powersim Solver Version 2.0
will, I understand, allow the integration of genetic algorithm
optimisation with Monte Carlo simulation with an unlimited number of
variables being permitted to vary according to defined rules (the
current version of Powersim Solver only allows 5 parameters to vary).
Thus one might seek a range of policy mixes that give, say, a 90%
probability of achieving a desired outcome. From my 12 years at the
Cabinet-Bureaucracy interface, I believe this type of capability is
essential.


Keith Linard
Keith Linard
Department of Civil Engineering
Australian Defence Force Academy
Northcott Drive CAMPBELL ACT 2601 AUSTRALIA
Phone: +61-6-268-8347
FAX: +61-6-268-8337
EMAIL: k-linard@adfa.oz.au

[Bob L Eberlein]

Just a short reply to George and Khalids postings on this subject.

First, it is important to note that the traditional method for performing
multivariat sensitvity anlysis is to take a model and run it again and
again changing each time a specified set of constants (model inputs).
This process precisely gets at sensitivities to parametric assumption to
see whether there are combinations of parameters that put the model into
different behavior regimes. It is also straigtforward to parameterize
structural changes to get at these sensitivities, though the resulting
output distributions are much harder to interpret.

Latin Hypercube simulations are a neat way to reduce dramatically the
number of simulations required while losing the minimal amount of
information. Ultimately, however, the loss is pretty big and these make
sense only when it is not practical to run several hundred to several
thousand simulations. With modern computers this means for very big models.

Sensitivity simulations dont replace the traditional experimentation and
exploration but supplement it, and allow you to have more confidence that
you have looked at a good selection of stuff.

Adding noise to a model simulation is a different issue altogether and I
tend to agree with Khalid that this is not the most effective use of time
for many traditional system dynamics projects.

Bob Eberlein
Bob@vensim.com

[Denis Smith]

Dear Khalid

I agree with your comments on the modelling of risk within organisations.
However, there are some who would argue that the presence of slack within a
system may itself contribute to risk generation by making the system and its
controls opaque for operators (outside of the "normal" operating envelope.In
addition, for low probability events, we tend to rely upon expert judgements
of risk (no apriori data) and this might give rise to the dysfunctional
feedbacks that you refer to.


Denis Smith
University of Durham
United Kingdom
From: Denis Smith <Denis.Smith@Durham.ac.uk>

["George Backus"]

I would think risk management would be an excellent use for the
Latin-Hypercube (LH) tools in Vensim. You can add the noise to the system
and add the Black-Scholes equations as desired. You can the use the LH to
make the uncertainty for the B-S equations and solve for the portfolio set
needed (using the simple optimization in EXCEL for example). But if you want
to get the Nobel Prize (Or rich), add noise to the runs exogenously (where
you think it should causally go) and let the portfolio selection be the
uncertainty for the LH sampling. I presume Vensim saves the runs (I use my
own LH package). You can then find the run that gives the greatest value,
lowest risk, etc. or whatever criteria you need. It contain the parameters
values telling you of your portfolio mix -- not the 100% optimal but about
99.5% there. You are now orders of magnitude beyond the normal risk
management tools.

But why stop here? Everybody else who is using B-S takes the variance as a
exogenous (often constant) variable/parameter that can only be measured by
looking at the past -- as in "too late." Most of the variance is truly
exogenous from the human perspective, but you have (you better have) a SD
model with some feedback. The real world feedback is controlling some of the
variance. Thus, your company has either some (albeit small) control of the
market that no one can understand, or you have can determine some leading
indicators to the variance that no one else can perceive. You have inside
information that you may be creating! This is worth mucho $. (We use this
in energy trading. Is any body watching the news lately in the Midwest and
California electric markets on unusual, extreme, price variance that no one
seems to understand.....)

For those in charge of multi-million $ asset management, I leave the rest as
a homework assignment.....

G


George Backus, President
Policy Assessment Corporation
14604 West 62nd Place
Arvada, Colorado, USA 80004-3621
Bus: +1-303-467-3566
Fax: +1-303-467-3576
George_Backus@ENERGY2020.com

[Khalid Saeed]

As much as I find George Backuss approach to using Vensim software for
risk management very sophisticated, I wish to caution that the
sophistication of an analytical process has little to do with its efficacy
for the problem it is addressing. Risk arises often from operations with
very little slack in the resources they employ or in the presence of
dysfunctional feedbacks that might be activated with a slight change in
the parameter values. Indeed, the use of noise could possibly activate
these feedbacks, but would it add to our understanding of what is the
source of risk and how risk might be managed? Wouldnt an attempt to
understand the slack problem and the dysfunctional feedback structure be
more useful. Wouldnt this be better attempted without noise adding
confusion to the analysis?

Khalid Saeed
From: Khalid Saeed <saeed@WPI.EDU>
_____________________________________
Khalid Saeed
Professor and Department Head
Social Science and Policy Studies
W. P. I., 100 Institute Road
Worcester, MA 01609, USA

Ph: 508-831-5563; fax: 508-831-5896

http://www.wpi.edu/Academics/Depts/SSPS/

["George Backus"]

Khalid Saeed writes:

"Wouldnt an attempt to understand the slack problem and the dysfunctional
feedback structure be more useful. Wouldnt this be better attempted without
noise adding confusion to the analysis?"

I appreciate and agree fully with Khalids comments, but if we were really
smart enough to do what he proposes, we would all be very rich and not on
this server. The real system has complexity in the true sense of the word.
Weather is a "piece of cake" thing to model in comparison. We can
hypothesize many reasons for the system "noises," but the interactions are
beyond forecasting ability, --- that is, other than possibly the main
underlying driver response -- and that SD is good at simulating.

Further, do to the anonymity of the market players, the market data defies
decomposition and no hypothesis of deterministic subsystem behavior be
proven. Therefore, (I think) we have no choice but to only simulate the
major drivers and our own (companys) interactions with the market, and add
the rest as noise at the boundary of the model. In this case, SD
understanding is not really the point. We simply want to do a bit better
then the "other guy" at making money in the market. Even if we bastardize
SD to do it, we can still say we took those "econometric sods" money in the
process. That must count for something

G

George Backus, President
Policy Assessment Corporation
14604 West 62nd Place
Arvada, Colorado, USA 80004-3621
Bus: +1-303-467-3566
Fax: +1-303-467-3576
George_Backus@ENERGY2020.com

[Khalid Saeed]

David,

An excellent observation. Both too little slack and too much slack can
contribute to risk. An example might be that qulaity is at risk when there
is too much inventory, efficiency is at risk when there is munificence. In
any case, risk management would require understanding the nature of the
slack and the dysfunctional feedbacks that would contribute to increase in
risk.

Khalid
From: Khalid Saeed <saeed@WPI.EDU>
_____________________________________
Khalid Saeed
Professor and Department Head
Social Science and Policy Studies
W. P. I., 100 Institute Road
Worcester, MA 01609, USA

Ph: 508-831-5563; fax: 508-831-5896

http://www.wpi.edu/Academics/Depts/SSPS/

[Jim Hines]

A short build on Bob Eberleins excellent point that multivariate
sensitivity analysis can help us ".. see whether there are combinations
of parameters that put the model into different behavior regimes."

Bob says this "allows you to have more confidence...". It also may
guide you to interesting regions of parameter space. In such cases,
the noise can then be discarded, in favor of deterministic runs
targeted at understanding why the particular parametric region behaves
as it does.

Regards,
Jim Hines
LeapTec and MIT
From: Jim Hines <jimhines@interserv.com>