SD Models from Written Text

This forum contains all archives from the SD Mailing list (go to http://www.systemdynamics.org/forum/ for more information). This is here as a read-only resource, please post any SD related questions to the SD Discussion forum.
jnoble1@mmm.com
Junior Member
Posts: 4
Joined: Fri Mar 29, 2002 3:39 am

SD Models from Written Text

Post by jnoble1@mmm.com »

I would like to pose the following question to the SD list. What research as
been done in formal heuristics/methodologies/techniques for extracting, from a
system or problem domains written textual descriptions, system variables and
causal reasoning for the purpose of building SD models? By models, I?m
referring to anything from Causal Loop Diagrams to Stock-Flow system
descriptions, along with variable behavior modes. Many experts in SD have done
an excellent job of writing textual descriptions of systemic insights, both
structure and behavior (the columns of the late Donella Meadows would be
excellent examples of this), but what about technologies/methodologies to go the
other way? Would there be a significant role for fuzzy logic modeling in order
to quantify imprecise linguistic terms used into variables and values or some
other expert system technique? This may be forever the domain of the art of SD,
i.e., translation of text, as well as other "messy" data sources, into the
modeling constructs of SD, but I would like to know what serious research has
been done to make it more of a science.

Jonathan Noble
3M Supply Chain Supervisor
ph. 320-763-6521 ext. 239
Tri: 722-1239
e-mail: jnoble1@mmm.com
Bill Harris
Senior Member
Posts: 75
Joined: Fri Mar 29, 2002 3:39 am

SD Models from Written Text

Post by Bill Harris »

Pieter van der Hijden wrote:
>
> Maybe the automatic extracting from arbitrary textual descriptions is a
> few bridges too far yet. However, an intermediary solution could be
> useful.

That sounds like a good idea.

> If we could agree on some special codes to insert in our texts and have
> the discipline to apply them, intelligent querying and searching of our
> texts will be possible.
>
> The solution is called XML, short for Extensible Mark-up Language. XML

There are already approaches people use for this sort of work that
avoid the need to start with XML, which was designed to aid in the
automated transfer of information, not in the semi-automated analysis
of text. Certainly XML could be harnessed to this task, but using
existing approaches used by others in the field may avoid reinventing
the wheel, save time, and (at least in the short run) get better
results because of the insights which have built up around the use of
those methods.

The qualitative analysis community (and were arguably doing
qualitative research, at least in the front end of SD work) uses
software such as NVivo, NUD*IST, and HyperRESEARCH to do this sort of
work. By listening to that community, I gather NUD*IST is the most
popular of the three, but Im not sure. The vendors Web is at
http://www.qsr.com.au/home/home.asp. You can find out about
HyperRESEARCH at http://www.researchware.com/. I am not a user of any
of these tools, although Ive been tempted to try out NUD*IST.

Regards,

Bill
From: Bill Harris <bill_harris@facilitatedsystems.com>
--
Bill Harris 3217 102nd Place SE
Facilitated Systems Everett, WA 98208 USA
http://facilitatedsystems.com/ phone: +1 425 337-5541
George Richardson
Member
Posts: 23
Joined: Fri Mar 29, 2002 3:39 am

SD Models from Written Text

Post by George Richardson »

On Wednesday, February 27, 2002, at 10:38 AM, jnoble1@mmm.com wrote:

> I would like to pose the following question to the SD list. What
> research as
> been done in formal heuristics/methodologies/techniques for extracting,
> from a
> system or problem domains written textual descriptions, system variables
> and
> causal reasoning for the purpose of building SD models?

An interesting study would be to see if a sample of experienced system
dynamics modelers or systems thinkers would produce the same map or model
from a written textual description. Then maybe one could get them to say
what they thought they were doing.

If that bears any fruit (heuristics, ..., techniques), then we could if
teaching others those heuristics helps them extract maps or models from
text.

...George

*George P. Richardson
*Rockefeller College of Public Affairs and Policy
*University at Albany - SUNY, Albany, NY 12222
*gpr@albany.edu *518-442-3859 *http://www.albany.edu/~gpr
"Bob Walker"
Junior Member
Posts: 4
Joined: Fri Mar 29, 2002 3:39 am

SD Models from Written Text

Post by "Bob Walker" »

The discussion around fully or partially automated conversion of text
descriptions to SD models has been interesting and thought provoking.

In our practice, when we introduce new client teams to ST/SD we typically
begin with a short introduction to basic principles, then have them do a
small group exercise in translating text-based stories into systemic
language. For this purpose we have accumulated an inventory of actual
newspaper articles which seem to work.

The clients enjoy this very much... it is doing, not just passively
receiving. Over time we have revised the inventory to select those articles
that seem to work best for our training purposes. A benefit is that
we know these cases work well with newcomers to the field.

It occurs to us that our materials might form useful test cases for anyone
experimenting with or developing methods to automate the process in one form
or another. Our criteria for selection are that the articles must be no more
than one page in length, must be real situations, not contrived training
materials and must be capable of solution by newcomers within 30 minutes. We
also make no claim, now or ever, that there is a single "correct" solution.
In practice we make ourselves available to provide suggestions or
clarifications for teams that are stuck... but the success rate in
generating plausible solutions is remarkably high.

The benefits of these materials is that they cost us little to collect and,
once tested a few times, form useful materials about real situations that
people seem to enjoy "modeling".

We are willing to make our collection of exercises available to interested
parties free of charge and obligation. Just send me an e-mail. As a
community it might also be useful for us to think of establishing a
non-commercial, no obligation marketplace for such materials. Surely the
best way for the cream to rise to the top.

For the "dynamic knowledge software engineers" watching this Listserve such
examples may provide both a pragmatic test bed and lead to advancement of
the topic under discussion for us all.

Bob

Robert J. Walker
System Dynamics Practice Leader
Delsys Research Group, Inc.
Suite 400 - 45 Rideau Street
Ottawa, ON Canada. K1N 5W8
walker@delsysresearch.com
"Lazaros Petrides"
Junior Member
Posts: 2
Joined: Fri Mar 29, 2002 3:39 am

SD Models from Written Text

Post by "Lazaros Petrides" »

It seems to me that the majority of the respondents to this particular
thread are looking forward to such a development in SD... thus Im probably
in the minority when arguing that this development *at this particular time*
would be more harmful to the field than not.

The reasons for me saying so is that I think that weve reached a stage
where it has become easier to build a model than it should be. To avoid
misunderstandings, I should stress that with the previous statement I mean
that nowadays it is much easier to build a *flawed* model than it used to
be. Now, if our flawed attempts are later corrected by experts, thats
great... yet I am afraid that it has become so easy to build a model, that
people dont feel the need to ask for advice; they automatically consider
themselves as experts after the first instance they produce a running model.
Flawed models are later presented to non-experts as good SD work, since the
"modeler" has now become an "expert" in the field, and when the weaknesses
are eventually revealed the whole discipline gets degraded.

The problem becomes even greater if we consider that SD is not yet well
accepted by experts in other fields, and as a result, even one flawed model
does much more damage to the field than 10 proper models do good.

Now, if it was possible to somehow build an influence/causal diagram, or
even a stock-flow diagram from a textual description, the ones who would
benefit,in my opinion, are the true experts of the field, since they wouldnt have to waste their time trying to extract the relevant info. It should also be added
however, that they would probably be the ones who would use such a facility
the least, since the problems they tackle are simply not written down in an
A4. The ones most in need of such a development, as I see it, would
consequently be the ones who find it more difficult to convert textual info
into causal relationships and diagrams. But arent these the very people who
should be learning how to do all this?

To keep making it easier to produce (flawed and unsupervised) models while
at the same time keeping the difficulty of proper model building
(relatively) unchanged is, I think, a recipe for producing more and more
"experts" who would in turn build more and more flawed models.

Thank you

Lazaros
From: "Lazaros Petrides" <
lpetrides@cwcom.net>
Bill Harris
Senior Member
Posts: 75
Joined: Fri Mar 29, 2002 3:39 am

SD Models from Written Text

Post by Bill Harris »

Carolus Grütters wrote:

> The ratio behind the initial request seems to be the availability of a
> tool that does the hard work:
> extracting knowledge from certain sources.
> If that would be possible, it would leave more time to the real SD-work.

:-) (As if the real SD-work doesnt include--or maybe focus
on--understanding the system and how its creating the problem. :-)

> A third point is that SD - just like the legal domain - depends on
> argumentation or, as it is put
> by Forrester and Senge: building confidence.

I recall reading an impassioned article in a math journal (I think) ages
ago while in college arguing that mathematical proofs are more social
than logical constructs. That is, a mathematician writes down a
seemingly logical sequence, but its only by honing and refining in a
social process that the proof comes to be accepted (or, apparently quite
likely, to be modified into something more acceptable). Ive been told
that most of Claude Shannons proofs were wrong, and they made suitable
fodder for many a grad. students dissertation.

> This implies that a black box is never acceptable: we have to be convinced
> of the soundness of the proposed way of handling a certain problem. In
> skipping this part, leaving it
> to a sophisticated - if ever possible - knowledge-modelling tool, one
> would diminish one of the most
> important- though hard - parts in SD.

Well said.

Bill
From: Bill Harris <
bill_harris@facilitatedsystems.com>
--
Bill Harris 3217 102nd Place SE
Facilitated Systems Everett, WA 98208 USA
http://facilitatedsystems.com/ phone: +1 425 337-5541
"John Gunkler"
Member
Posts: 31
Joined: Fri Mar 29, 2002 3:39 am

SD Models from Written Text

Post by "John Gunkler" »

"John J. Voyer"
Junior Member
Posts: 5
Joined: Fri Mar 29, 2002 3:39 am

SD Models from Written Text

Post by "John J. Voyer" »

Jay Forrest makes a valuable point:

"1) There is a difference between deriving a model from text and
developing a model in an interview process."

As Jay Forrester and others have argued, perfecting the art/science of
sufacing mental models (as the precursors to SD models) is a worthy
goal. It seems particularly important to those modelers who have
followed the traditional SD modeling approach.

But it seems to me that there is also value in coming up with a good
way of deriving models from text, because some academic
practitioners (in particular) might want to test the systemic implications
of claims made by other researchers. Anjali Sastrys work at turning
OB and OT models into SD models is what I was thinking of.

Jac Vennix wrote:

"I extended the coding procedure as described in Axelrods book.
However, even after extensive training of six students in extracting
clds from written pieces of text (guided by a coding procedure
book), I was not able to get a sufficient inter coder reliability (i.e.
0.80)."

Perhaps this is not really an invitation to abandon this approach, but to
modify it. Modelers who base their work on interviews surely end up
arguing among themselves, and maybe with the clients, about what the
mental model really is. Why should it be any different among
modelers relying on written text? Much of this thread has focused on
how to "automate" this process, but why should we want to do that?
It seems to me that a more fruitful approach would be to use
something like the Axelrod/Wrightson method and then have the
coders argue (like interview-based modelers do) about what the
"correct" mental model is.

John Voyer
From: "John J. Voyer" <
voyer@usm.maine.edu>
Jay Forrest
Junior Member
Posts: 12
Joined: Fri Mar 29, 2002 3:39 am

SD Models from Written Text

Post by Jay Forrest »

I agree that a model from text process or program would be quite useful. My
facilitation experience leads me to focus on three serious hurdles to
success in generating meaningful models from text. These hurdles suggest to
me that a model parser would only be useful for suggesting preliminary
models for exploration and elaboration. The likelihood that a parsed model
would be "final" and "complete" would be quite unlikely. Additional
facilitation is almost certain to be needed. The three hurdles follow:

The first is the ambiguity of language. I only rarely encounter groups
where their vocabulary is truly aligned and where dialog related to
problems (and systems) is significantly deeper than superficial. (In group
work I find that groups are consistently surprised by the diversity of
perceptions that underlie their seeming agreement -- at a superficial,
language level.)

Second, in working with groups I find that I am much more likely to obtain
agreement on "influence" than on the nature of that influence. Thus,
influence diagrams seem more viable as an outcome of group model building
than quantifying causal relationships. I generally attribute the lack of
agreement on varying life experiences and perspectives. Closing gaps in
perspective (and experience biases) falls in the realm of the facilitator.
Written documents regarding problems/systems are typically from singular
perspectives, with boundaries and relationships deemed pertinent from that
perspective. This leads me to anticipate that a parsed model from text is
likely to have biases of perspective which need to be challenged in moving
the model forward. Thus facilitation will be needed for group models.

The third hurdle is the lack of general understanding of causality. There
seems to be multiple perspectives on what constitutes causality within the
systems community. If systems thinkers use different forms of causality,
the "unenlightened" will certainly be less clear in their application of
the concept of causality. This suggests that parsed models will be flawed
by errors of attribution which well require challenge. (While the needed
facilitation might seem to be similar to that required for point 2 --
errors of biased perspective -- IMO the underlying group needs are
different and require different approaches).

All of this is further complicated by my belief that the boundaries of the
model are critical -- and are unlikely to have been adequately explored in
developing the textual description -- and that the most insightful models
are often rather different from the description that the group would offer.
Barry Richmonds classic DEC business model (one stock, one flow) is an
immediate example.

All this is, again, not to suggest a text to model parser is not a great
idea -- more that the output may not show much value other than being a
starting point unless the authors are knowledgeable and throrough.

I look forward to comments and other views!
Jay Forrest
From: Jay Forrest <jay@jayforrest.com>
zenabraham@aol.com
Junior Member
Posts: 18
Joined: Fri Mar 29, 2002 3:39 am

SD Models from Written Text

Post by zenabraham@aol.com »

Hello,

At first crack, I cant think of any reference that perfectly matches the
desired answer to your question. But it occurs to me that if the answer were
out there, then the perfect software construct based on it, would be a
program that takes such text, and automatically crafts a system dynamics
model, regardless of loop structure, or size of the model.

But as I think of that even more, I am reminded that there are many different
ways to interpret a problem in SD, not just one.

Then, I come back to something I learned when I was interested in something
called "unified field theory" planning at the City and Regional Planning
School, at Cal (Berkeley for the "non-Bay Area people" reading this): that
social science is, by its very nature, based on the values of the researcher,
even to the point of what to study, and the methods selected. Those values
will become immediately evident as the process of research and study unfolds.
Moreover, the outcome of what is studied can be effected by the researcher.

That is different from Physics, where one cant change the course of travel
of an asteroid, just by observing it. But in social science, the future (or
present) behavior of what is studied (people and what they do) can be
effected by the researcher, as the person is working.

I personally dont have a problem with the idea that this is the nature of
the social sciences, within which system dynamics is a part. I still get the
impression, as I approach my 40th year on this earth, that such is not the
case for others.

Im at a loss to understand why this is; I think the point of SD is to offer
a way of thinking about the social system around us that openly embraces
complexity, and encourages argument.

I write this to encourage a nasty debate, because there are those who will
point to SD work in fields like Physics. But no one can challenge the
observation that SD reached the public view, when it was turned to analyze
something called "The Predicament of Mankind" by the Club of Rome, and
"Limits to Growth" was published.

In fact, that book was introduced during the era of the "environmental
movement." Its gift to the world, was the development of the idea of "Zero
Population Growth."

To further the point, one cannot challenge the fact that Jay Forresters book
Urban Dynamics focused on what people do in cities, and at a time of the
highest point of intensity of the Civil Rights movement.

Forresters model design, where the outcome of the model was interpreted by
some, though not Forrester, as anti-African American, not only placed SD in a
public light, but also caused at least one city in to adopt the urban
policies suggested in the book Urban Dynamics.

Those suggestions included the demolition of low-income housing projects,
where blacks, in disproportionate number, lived during the 60s.

Thus, the work of an SD researcher had an impact on that which was being
studied. I must also add that Jay was roundly criticized for the books
policy suggestions, leading one to wonder what he was thinking in the design
of the Urban Dynamics Model? Still, the argument was healthy, controversial,
and welcome.

Im not sure that I prefer the approach implied as desirable by some, like
Coyle. I like the current method, where we develop different representations
of the world around us, present our research and findings, and simply battle
it out, with a healthy dose of politics thrown in. Good.

To wish for "one system" to interpret text into SD, is not only...Well, the
word that comes to mind is commonly associated with Nazi Germany....But it
seems to state that SD is not a science because there is not one perfect
solution to an observed problem, or one perfect SD model that describes that
problem.

SD is as varied as our points of view. Good. The point of group modeling is
to combine those points, assuming the dynamics of the social relationships
within the group permit such a nice outcome.

I disagree with the need to transform SD into something called a "hard
science." In point of fact, given what SD is used for and the fact that
something called "hard science" exists only in fantasy, I think the endeavor
would prove to be worthless.

As long as there are people, politics and simple emotion driven desires will
always prevail. (As I think about it, we may also design machines to make
political decisions, because we are, in effect, trying to mimic our own
thought process, so the beat will go on as machines become "smarter")

This is why "hard science" does not exist. I have listened to these claims
for years, and as one who is well aware of the politics of funding research
and the hard choices one must make when work resources are constrained, the
idea of this "hard science" push is not the best use of time with respect to
SD.

I prefer to argue, to research, communicate, develop, learn, discover, and
then to do it all over again.

All the best.

Zennie Abraham
From: zenabraham@aol.com
Paul Martin
Junior Member
Posts: 11
Joined: Fri Mar 29, 2002 3:39 am

SD Models from Written Text

Post by Paul Martin »

"Pieter van der Hijden"
Junior Member
Posts: 2
Joined: Fri Mar 29, 2002 3:39 am

SD Models from Written Text

Post by "Pieter van der Hijden" »

Maybe the automatic extracting from arbitrary textual descriptions is a
few bridges too far yet. However, an intermediary solution could be
useful.

If we could agree on some special codes to insert in our texts and have
the discipline to apply them, intelligent querying and searching of our
texts will be possible.

The solution is called XML, short for Extensible Mark-up Language. XML
is a widely accepted and open standard to add so-called tags to textual
content. Tags always appear in pairs like opening and closing brackets
and maybe nested. You may define your own tags freely. However, it is
more interesting to agree on a certain document scheme (SD-related) that
describes the required and optional tags in a certain document. Your
texts can be validated automatically to such a document scheme.

A lot of software to handle XML documents is on the market and/or
available as shareware. Although XML documents are plain texts (the tags
themselves also are texts), that can be edited by a simple text editor,
the use of an XML editor is recommended. An XML editor ensures that the
texts in an XML document are syntactically correct (i.e. nested tags)
and can check that they are structured as prescribed in a certain
document scheme. XML databases can store XML documents and give you the
ability for intelligent and meaningful querying (using the tags you
entered in the document). Translation software is available to change
the structure of an XML document automatically, to change it to HTML or
to whatever output format.

When it is possible to query and search our texts in an intelligent
(SD-aware) way, it will be easier derive the interesting fragments from
our texts base, thereby facilitating (not replacing) the model building
process.

Best Regards,
Pieter van der Hijden


Pieter van der Hijden -
pvdh@sofos.nl - Sofos Consultancy - www.sofos.nl
- P. O. Box 94874, 1090 GW Amsterdam, Tel. +31-20-6941222.
=?iso-8859-1?Q?Andr=E9_Reichel?=
Junior Member
Posts: 14
Joined: Fri Mar 29, 2002 3:39 am

SD Models from Written Text

Post by =?iso-8859-1?Q?Andr=E9_Reichel?= »

Hello all,

this topic has developed into another major talk about the nature of SD. The
original question was about extracting a model from a verbal statement. The
transformation of a problem context into such a verbal statement is, of
course, model building as well. So, regardless what instruments are being
used to extract SD-relevant information from written text, this text itself
has been formulated by using (implicit) modeling methods. We can try to be
as rigorous and critical as possible to eliminate flaws and false or hidden
assumptions in the (verbal) formulization process -- group model building
with extensive discussion to avoid single-minded solutions might be one
approach. Regardless what method is used, this most essential step will
always remain linked to the thinking abilities of the model builder(s). No
software can relieve us from this.
What can be done, is to keep up with a certain (scientific) methodology that
involves precise thinking categories -- thinking about a problem in terms of
stocks and flows, feedback loops, systemic relationships and so on. This is
the language of SD and need not fear comparing itself with the language of
hard sciences -- in fact, it is the same.

A book I always like to recommend on topics like this is J. Randers,
Elements of the system dynamics method. Although from 1980 it still pretty
much says it all, and has an outstanding well written text by Dana Meadows
(as always) about SD and econometric modelling.

Mit freundlichen Grüßen /
best regards

André Reichel
A.Reichel@epost.de
Jac Vennix
Newbie
Posts: 1
Joined: Fri Mar 29, 2002 3:39 am

SD Models from Written Text

Post by Jac Vennix »

Hi all,

Just a cautionary note on extracting models from written documents. I have
done something along those lines for my PhD research. I extended the coding
procedure as described in Axelrods book. However, even after extensive
training of six students in extracting clds from written pieces of text
(guided by a coding procedure book), I was not able to get a sufficient
inter coder reliability (i.e. > 0.80). Now this was over 10 years ago, but
I do not have the impression that much has changed in the last decade wrt
this issue.

Jac Vennix
J.Vennix@maw.kun.nl
Locked