Ventana software support forum

Posted: **Wed Jan 23, 2002 12:19 pm**

Mark Mackay asks if there is a role for SD in community health where
little "evidence" is quantitative. I think there is.

My first thought: Model your (and other community health
professionals) assumptions about specific problems and solutions. Even
though you say quantitative evidence is not often available, your
assumptions can be quantified. For example, you can decide how much you
think "safety" (as measured by some index or some composite index of
measures) will be improved if park lighting were improved. In fact you
can go further and quantify your assumptions about the amount of
improvement of park lighting and its relationship to improvement in
safety -- Im thinking here of drawing a graphical function relating the
two and getting professionals (and perhaps the community at large) to
agree upon the general shape of the function.

Even if the "model" goes no further, the discussion around making your
assumptions explicit (and quantified) can further peoples understanding
of the situation. Of course, if you can get further and include other
quantified assumptions, and model their relationships to each other, you
may be able to cast some light on the interactions among things such as
improved park lighting, increased police patrols, community PR and
awareness efforts, etc. By including data (even best guesses) on costs
of each of these could lead to insight about where money may be best
spent.

You can use whatever quantitative data you have to test the model, or
parts of the model. And you can run sensitivity tests on whatever data
you created (from your assumptions) to see how sensitive the outcomes of
the model are to changes in these assumptions. Often SD models are not
too sensitive to exact parameter values which gives you the ability to
say things such as, "Okay, the model wont produce appreciably different
results unless our assumptions about park lighting are off by a factor
of 10. Is it reasonable to suppose that we could be off by that much,
or can we proceed with some assurance that were in the ball park?"

John
From: "John Gunkler" <jgunkler@sprintmail.com>

Posted: **Fri Jan 25, 2002 9:36 am**

To Dr. Coyle -- HOWL!

When you write, "Even if you agree on the meaning of 1, what does 0.5 on
Johns scale mean?" it seems as if you are unaware of the decades of
research in the psychology of sense perception. Psychologists have had
very good luck with, to mention but one example, a methodology called
"magnitude estimation scaling" whereby things like perceived loudness
and perceived brightness are scaleable. Even though my scale may not
exactly match yours, by gathering data from a good sample of individuals
you can discover how much variation there is among individual
perceptions and come up with a way to measure (with some "error") the
brightness of a particular light source or the loudness of a source (or
the "greenness" or almost any other sensory perception.) Sometimes,
when there is a compelling reason, you can even get people to agree to
use a single scale -- that is, to adjust their own perceptions to match
a common scale. So, even though my "naïve" perception of the brightness
of a particular light may have been 4.8, I can "learn" to perceive that
it is a "5" on the common scale.

I believe that people of good will do this all the time in ordinary
life. Words do not have precise meanings, yet we learn to communicate
using them; money does not have identical "value" yet we learn to use it
as a medium of exchange; and colors are not perceived the same (my wife
and I are a good example of this!), yet we learn to create a common way
of using color names (or to adjust our perceptions to the way the other
person uses color names.)

John

John W. Gunkler
jgunkler@sprintmail.com

Posted: **Tue Apr 23, 2002 9:28 am**

> Hello
>
> I work in the public health sector in Australia. One of the things Im
> currently looking at is community health services and evidence. Ive been
> involved in modelling of hospital services and can see the value of
> modelling in that sector. There generally are problems in attribution of
> outcome in relation to community health services (ie a lack of evidence
> that the action or service provided worked). The main reason for this
> appears to be time: a service is provided today, but the outcome is not
> expected for a considerable period of time (may be years). This sector
> suffers that it cant prove its efforts are worthwhile. This has obvious
> political and resourcing implications for the sector.
>
> There are two sorts of work that is undertaken. 1. Working with people (in
> groups or as individuals) and 2. capacity building. Working with people
> (e.g. smoking prevention) may lend itself more readily to modelling.
> Capacity building relates to getting things right in the community so that
> the health of the community will improve. An example might be making the
> community safe - involving working with police, improving park ligthing,
> etc. (I can expand upon this further if required). Here modelling may be
> harder.
>
> Can modelling assist here to improve to
>
> - to highlight the leverage points?
> - to help show if some options might be more worthwhile than others?
>
> Much of the evidence that is available is qualitative. Some info is
> quantitative. Ive thought about qualitative SD diagrams, but Im not sure
> if these would help.
>
> Any feedback would be welcomed.
>
> Mark Mackay
> Principal Project Officer, Service Review
> Metropolitan Division
> Department of Human Services
> PO Box 65
> Rundle Mall SA 5000
> AUSTRALIA
> Mark.Mackay@dhs.sa.gov.au
>
>
>

Posted: **Wed Apr 24, 2002 9:14 am**

Dear Mark-

During the mid-to-late 1990s several colleagues and I developed a set of
health care Microworlds (simulated learning environments) based on System
Dynamics models. One module dealt with improving community health status and
included services that might only be expected to have an impact over a longer
term such as social services for dealing with family dysfunction, job
training, and juvenile justice services for kids getting into trouble with
the law. Also included were a range of public health services for dealing
with such things as environmental threats, preventive education in areas such
as smoking, and improved medical management for people who already have
chronic illnesses to reduce the frequency of acute complications. The model
was based on the best available data at the time. This module was
subsequently adapted by my colleague Sherry Immediato for use by the British
NHS.

Working through the simulations gave participants a better understanding of
the impact of various measures over time and where leverage could be found
and where it was lacking. Strategies, for example, that tried to do "a
little of everything" were spectacularly ineffective. Strategies that
focused initially on activities with a relatively quick payoff such as
improved medical management of chronically ill patients and then invested the
health care cost savings in social services for the elderly and other adults
had more significant impacts over time. "Experiencing" these lessons with
the Microworld was much more convincing than talking through qualitative SD
diagrams would have been.

Two references for learning more about the work are:

G. B. Hirsch and C. S. Immediato, "Design of Simulators to Enhance Learning:
Examples from a Health Care Microworld", International Conference of the
System Dynamics Society, Quebec City, July, 1998

G. B. Hirsch and C. S. Immediato, "Microworlds and Generic Structures as
Resources for Integrating Care and Improving Health", System Dynamics Review,
Fall, 1999

Gary Hirsch
From: GBHirsch@aol.com

Posted: **Wed Apr 24, 2002 1:07 pm**

In a message dated 1/24/02 7:38:32 AM, jgunkler@sprintmail.com writes:

<< Mark Mackay asks if there is a role for SD in community health where
little "evidence" is quantitative. I think there is.

Hello All,

I quite agree with the reply above. In System Dynamics, a basic rule is that
one should model that which is not necessarily quantifiable, but is
definitely a part of the environment being modeled. Its a way of testing
our "mental models." I think there are few better "proving grounds" for
System Dynamics than community health, with its rich mix of the "soft" and
"hard" quantitative policy issues, which effect so many people.

In the XFL Microworld (http://broadcast.forio.com/sims/xfl), there are some
relationships which are hard to quantify, but are a part of the system. For
example, increasing the number of teams impacts TV ratings. The idea is that
the more cities that have teams, the more likely it is that the ratings will
be higher. Theres no hard evidence of the perfect combination of teams and
ratings, but the relationship is generally agreed upon, and therefore must be
represented, if conservatively.

I did that after much sole searching because theres no perfect answer to the
question of what the optimum number of teams should be to sustain ratings.
For example, the Super Bowl has only two teams playing, yet commands 45
percent of the ratings of the day its played.

Now, another way of looking at that is to say that a 16 week battle of 32
league teams caused a kind of bandwagon buildup to the final game. So, were
back to the subject of league size versus TV ratings.

That "mental model" I just expressed is the common foundation of the
construction of system dynamics models before statistics are used to
"calibrate" the models. In other words, were modeling observations of
real-world phenomena in, in this case, the sports business.

Anyway, the teams factor is very important in trying to determine what the
perfect size is at which the league becomes an institution, which generates
viewers via a combination of the number of teams it has and cities its in,
and what it pays its players, and spends on marketing.

The problem is that the "novelty" effect wares off after a few years, so
ratings drop unless you do something different. Its possible to have a
ratings "pop" in later years, but generally ratings settle to around 10 to 6.
In your case, it seems like you let the carrying capacity of the size of the
league drive your ratings, rather than do anything else.

There is no perfect answer to this, so thats why one has to add teams and
"find" that systemic switch.

I think System Dynamics is the best medium for the representation of such
hard to quantify matters.

Zennie Abraham
From: zenabraham@aol.com

Posted: **Thu Apr 25, 2002 9:27 am**

This strikes me as being a bit dodgy.

Lets look at the park lighting first and imagine a scale from 0 to 1. So, 0
means no lighting at all, so thats clear (well, dark actually, at any rate
at night). At the top end , 1 must mean perfect, but what does that actually
mean? What is perfect to one person with good eyesight might be terrible to
someone else. Replacing that by X lumens per square metre does not avoid the
judgement of perfection. Is this lighting along the paths, for the whole
area or for those parts where there has been trouble in the past. Even if
you agree on the meaning of 1, what does 0.5 on John scale mean? Does it
mean that all the lights are on half the time or half are on all the time.
Maybe it mean all of them are on at half power. Even if you agree on all
that, you still have the problem of the meanings of 0.25, 0.125 and so on.

In short, its easy to make up scales for qualitative issues but hard to
give them operational meaning at the level required by a simulation model.

When you combine that with police patrols, community PR and who knows what
else you have BIG problem with making sense of what you have done and you
may, indeed, have produced what someone (not me) calls plausible nonsense.

There is also a serious problem with the interdependencies and Johns
comment about being off by a factor of 10 may be misleading (why 10 as a
cut-off - why not 2 or 22?). It may be that lighting only has an effect if
there are police patrols or that patrols are useless without lighting. Its
alright talking about drawing non-linear curves for these things but you
would need a lot of real knowledge needed about these interactions to have
any confidence at all about the results from the model and hence the
conclusions drawn.

Trying to invent knowledge by guessing about numbers and drawing curves is
worries me. Our software is as powerful as our imaginations but we need to
be a lot more rigorous in what we do for this type of problem.

Geoff Coyle (anticipating a howl of protest)
From: "geoff coyle" <geoff.coyle@btinternet.com>

Posted: **Thu Apr 25, 2002 12:44 pm**

Hello,

Im howling in protest! Just because one uses a regression analysis to
develop a rationale for using a number does not mean the number is "robust."
What do you mean by the term "robust?"

I bristle at those within the forecasting community who think and believe
that the standard static methods of analysis are perfect. As we know from so
many failed projects based on this blind marriage to numbers, those
approaches do not work.

The only reason they persist is because there are people who must have some
comfort that their way is correct, rather than look at different scenarios
and examples of possible systems behavior that may or may not materialize.
Regardless, the point is to be prepared for them, and to have a complete
understanding of the system, not an incomplete one.

Science should never replace religion; one should not use numbers as a basis
for faith, but as a tool for learning. I think this is where the rub exists
for me. The emphasis should be on good analysis and understanding of the
system that surrounds a problem. The process leading toward such
understanding is hard and frought with people who have different points of
view and who disagree. Good.

I have worked for the last two Mayor of Oakland, California as an economic
advisor. I cant tell you how many times numbers are used to protect a point
of view, but not to present an understanding of a system. I agree that in
the matter of the example of sight, one persons judgment is not the same as
anothers, but then one can use it as a representative example, or if its
the most frequently occurring outcome, use it on that basis.

But what do you do if there is no data on it? Ignore it? No. Not in System
Dynamics. Maybe in another discipline, but not in System Dynamics.

The reason I like SD is that it forces you to think about interrelationships
and the whole system, not just a part of it. It pushes you to embrace
complexity at some level.

People ask me about the internal design of the XFL Microworld
(http://broadcast.forio.com/sims/xfl) and why no two outcomes are exactly
alike. First I inform them that each outcome is based on what they do. If
they do everything EXACTLY the same way, the outcome will be the same. But
one minor change can trigger a huge event which may not show up until later
in the simulation. Thats good systems work, to pat my proud self on the
back.

Sure, I used basic data on the league, but when developing a model of a
system like a sports league, you have to be comfortable with the fact that at
a point you are going to jump off to a "unknown" series of relationships. It
happens.

Zennie Abraham
From: zenabraham@aol.com

Ventana software support forum

community health & evidence -a role for system dynamics?

community health & evidence -a role for system dynamics?

community health & evidence -a role for system dynamics?

community health & evidence -a role for system dynamics?

community health & evidence -a role for system dynamics?

community health & evidence -a role for system dynamics?

community health & evidence -a role for system dynamics?

community health & evidence -a role for system dynamics?