System Dynamics Certification

["geoff coyle"]

Dear All,

The idea of SD certification is a very interesting issue but it might be
fraught with difficulty.

One precedent is that the Operational Research Society (which has around
5000 members world-wide) argued for at least 15 years over the establishment
of the Fellowship of OR - equivalent to certification. Eventually a
sub-group set up a Fellowship scheme of their own but it was never
recognised by the Society and seems to have been of little practical
benefit. Maybe someone will know better, but thats my assessment. The
implication may be that our society would achieve no better outcome.

A second potential problem is the legal aspect. People such as medical
doctors, architects and the like are allowed to practice as such by
authorities which derive their legitimacy from the government. A doctor can
be struck off for malpractice and that is effectively by the authority of
the State. This is based on the need to protect the public against the
potentially serious consequences of incompetence. There are no such
consequences from the malpractice of SD. Further, I can see the prospect of
deep trouble if Bloggs is told by a self-appointed body that he is not an
acceptable system dynamicist. He might well have grounds to sue, as
individual persons, the members of the committee which had assessed him to
be incompetent. I think that there would be a need to take some serious
(and probably expensive) legal advice which would have to allow for legal
systems in many countries.

Thirdly, how a subject is taught can be influenced very heavily by how
universities require examinations to be conducted. That differs so widely
from one country to another (and we have members in no fewer than 55
countries from Argentina to Vietnam) that I shudder to think of the
complexity.

A more positive idea is the role of external examiners. In most countries
which have British-derived education systems, and maybe others, university
exams are moderated by external assessors. For written exams, they review
the questions to be set, and can over-rule the person who set them, they
then evaluate the marking and review assignments. For PhD degrees the
external examiner is normally treated as the senior person, regardless of
academic rank, and has the final say-so (there are often two externals for a
PhD). This is similar to the US system of a PhD committee except that the
external person is utterly independent. It may well, of course, be different
in other countries.

I suggest that we could make a start on improving our standards by the
Society keeping, and making known via SDR, a list of people recommended as
external examiners who could be invited to serve on US PhD committees or to
do the normal external examining in UK-based systems. A variant on such a
system might be useful elsewhere. The key idea would be for the persons on
the list to be nominated by national chapters who cold be expected to know
the system in their own countries.

I hope that helps.

Regards,

Geoff

Professor R G Coyle,
Consultant in System Dynamics and Strategic Modelling,
Telephone +44 (0) 1793 782817, Fax ... 783188
email geoff.coyle@btinternet.com

["Keith Linard"]

Competency based job profiling, competency based assessment and competency
based training are becoming the orthodoxy in english speaking countries,
especially in the primary and secondary school systems and in technical
(sub-professional) training. The US / NATO / Australian military training
systems adopted this some 20 years before the current civilian love-affair
with competencies. The US based Project Management Institute Institute
(PMI), through its "project management body of knowledge", has developed a
very comprehensively structured framework for accreditation.

Whilst there is much good in the systematic approach which underlies the
competency based training (CBT) framework, I suggest that, in its
application, it CAN BE fundamentally anti-systemic ... it CAN proceed as
though the whole is simply the sum of the individual parts.
Systematically decomposing a body of knowledge into bite sized chunks
makes it easy for the educator to tick off a list of isolated chunks of
knowledge that the student has accumulated. It does not guarantee that the
student can integrate these into a whole.

Recent coronial inquiries / commissions of inquiry into major defence and
engineering disasters in Australia have explicitly identified the
just-in-time competency based training of technical operators as one of
the systemic factors involved in the disasters. Where previously technical
training, especially apprenticeship based, exposed technicians to a wide
range of competencies, many of which were not immediately relevant to the
specific work (and hence wasteful to the economic rationalist, undoubtedly
trained on a competency based framework) ... but which gave holistic
understanding that could be called upon when the unexpected happened, the
CBT skillset left technicians ill prepared in such situations.

As an educator, I experienced extreme frustration in knowing that, over the
13 week by 3hr per week semester I was communicating the basic SD
competencies, but no real understanding of SD. For the past 6 years I
have taught these competencies in 5 consecutive all-day-Saturday sessions,
giving the students the rest of the semester to apply these in team
consultancies addressing real problems for real clients. It is in this
environment where the true education takes place, where students are forced
to integrate the basic competencies in a problem solving context.

I believe it should be a high priority for the Society to identify the skil
sets or competencies desired of system dynamicists ... but we must proceed
within a systemic mindset, not the CBT systematic mindset.

Could I suggest that a specific stream be set aside for the 2002 ISD
Conference on SD personal competencies / SD education curricula.

I would also invite practitioners / educators to submit their suggested
competency / skill-set lists to this list server.

Keith Linard
Director
UNSW Centre for Business Dynamics & Knowledge Management
University of New South Wales
(Australian Defence Force Academy)
Northcott Drive
CANBERRA ACT 2600
Phone: 02-6268-8347
Fax: 02-6268-8337
Email: k-linard@adfa.edu.au

["Rom Koziol"]

In regards to certified educational programs in SD.

At 10:08 AM 6/1/01 -0400, Jim Hines wrote in part (and Bill Braun repeated):
"A more promising route, suggested a few years ago by David Andersen, might
be to certify educational programs, rather than individuals. This seems
MUCH easier to me, and might accomplish 70% or more of the objectives."

Just consider all the locations in the world that SD has spread to (see
members list). How many of these would have, or even be near, institutions
offering educational programs that could be certified anyway? Or would
certified programs be limited to a handful of SD centres of excellence?

Isnt one of the exciting things about SD that it is still (relatively
speaking) a growing field where not everything has yet been
discovered/formalised, where one can still explore/self learn, and share at
symposia and through this medium? We have come a long way by virtue of the
fact that most of us have self learnt and then passed it on to others be it
through academia, work or otherwise. While certification may add credence to
those who need it, might it not also impede the overall diffusion process?
Perhaps we should first establish who needs certification, and the reasons
why?

Rom Koziol
romk@romk.com
+61-2-9402 5400
Sydney, NSW Australia

[Niall Palfreyman]

> On 4 Jun 2001, at 7:58, Bill Braun wrote, in part:
> > Id like to know where I am deficient and continue to hone my skills
> > in those areas. Individual certification would address a good part of that.

Surely what Bill is seeking here is "feedback" rather than
"certification". I am personally against certification, for the reasons
Mark mentions - its typically aimed at protecting the user, but not at
informing the provider. It seems to me that the best way of providing
both is the suggestion made by Jim Hines recently: to certify education
programmes.

Niall Palfreyman.
From: Niall Palfreyman <niall.palfreyman@fh-weihenstephan.de>

[KCarpen563@aol.com]

I generally agree with the thoughts that at this time SD may still be
"self-actuating," but there is a lot of poor modeling going on in the name of
learning. If we are to make progress as a discipline, there must be a role
for a quality control process.

I wonder if the issue of certification should be viewed not as an
individuals skills (for now), but as a review or audit performed by a few
groups or people that are recognized for this purpose. This would be a good
selling point to potential users of SD/ST products and remove it from snake
oil sales, which I fear that we are sometimes viewed.

There are obviously a lot of threads to this concept and various directions
for implementation, but I throw the concept out for discussion.

Ken Carpenter
kcarpen563@aol.com
Albany, NY

["Thompson, Jim B261"]

Bill Braun wrote:
"Does this leave room for the self-directed learner?...I wish to continue to
learn, and going away to school at this juncture in my life is not feasible.
Id like to know where I am deficient and continue to hone my skills in
those
areas."

I took 15.982B, Advanced Techniques for System Dynamics with Bob Eberlein,
from MITs Web-delivered offerings this semester. There were about 10 students
in the class, and we used email and BBS to exchange views. The lecturer, Bob
Eberlein, provided a thorough how-to, when-to, why-to series of 10 one-hour
lectures with high quality video so that we could follow the modeling steps. I
can recommend the course without reservation for people want to advance their
skills with subscripting, Reality Checking, optimization and calibration.
It is well worth the time investment. See <http://command.mit.edu/>

There were courses this semester that explored strategy formulation with Kim
Warren and the project model with Jim Lyneis. Maybe students of those
classes will comment?

[Hosts note. While I couldnt resist posting such blatant praise, I would
like to clarify that MIT offers a program of full credit graduate courses in
system dynamics. Information is available at

http://caes.mit.edu/asp/off_campus/syst ... index.html

The course I delivered does have prerequisites in this program, which
are themselves highly effective for building skills in system dynamics.

The major software providers also offer courses (some as distance courses)
and the Roadmaps are also available from MIT
http://sysdyn.mit.edu
oad-maps/home.html though the guided study program
is no longer offered.]

So those of us who cant afford the hiatus from work have alternatives through
the Web and email systems. The content is demanding, but there is no commute
and you schedule your class time.

From: "Thompson, Jim B261" <Jim.Thompson@CIGNA.COM>

["Mark B. Wallace"]

On 4 Jun 2001, at 7:58, Bill Braun wrote, in part:

> Id like to know where I am deficient and continue to hone my skills
> in those areas. Individual certification would address a good part of that.

Not necessarily. Every certification process I know of relies solely on the
infamous multiple-choice test to assess competence. Multiple-choice
tests reward rote memorization and superficial thinking (see "The Tyranny
of Testing" by Banesh Hoffman). If there was ever a field of human
knowledge and endeavor less suited to the certification of practitioner
competence by multiple-choice testing than SD, I havent heard of it.

Mark Wallace
wallace@acm.org

Mark B. Wallace, OCP
Oracle Certified Professional - Oracle8i DBA

[Alan Graham]

Well, the short description of the certification question is "tar baby":
well all get stuck in it if we wrestle with it too much, at least coming at
it simplistically. The reason? Most of the analogs to "certifications" in
other fields I can think of have multiple prerequisites our field isnt
anywhere close to.

Consider the certification processes of these practitioners, each of whom
delivers a highly technical service to other people, often with strong
cross-disciplinary content:*

Physicians: academic degree in medicine, successful completion of multiple
guided (and evaluated) work programs (internship, residency, and for
specialists, a fellowship)

General military officers: multiple diverse tours of duty, interleaved with
focused academic programs

Certified Public Accountant: Degree, three years of acceptable
apprenticeship performance

Tenure-track professors: academic program (usually PhD, itself involving
"on the job" research evaluated by recognized authorities in the field),
multiple steps of evaluated work (teaching and peer-reviewed publishing):
assistant, associate, tenured, full professor.

The common elements here are:

1. There is a standardized body of knowledge, delivered in explicit course
work

2. There are multiple stages involved in reaching final "certification"

3. The "certification" process is run by recognized experts, each having
more or less the same type of certification

4. At least one stage involves actual, on-the-job practice in the field,
evaluated and perhaps guided by those recognized experts

So I believe the answer to the question of "should we certify peoples
on-the-job performance, peoples test-taking ability, or the courses of
instruction that they complete is "in the long run, yes to all." In the
short run, we have a bootstrap problem that seemingly can only be addressed
very gradually.

If were going to continue having the certification question come up and get
discussed, it seems to me we should be coming up with suggestions on how to
start an adaptive bootstrapping process with a realistic chance of moving us
forward.

cheers,

alan

*footnote: as far as I understand it, law is an exception, where one can
practice after completing (an admittedly practice-oriented) academic program
and completing a written test, the Bar Exam. Perhaps those more intimate
with legal education and practice can shed light on either my understanding
of the process, or the nature of legal practice. Also, there are several
analogies made to fields that dont seem like good parallels to SD.
Engineering is one. Perhaps certification has succeeded in very stable
portions of the engineering world, but I know that the software world is
having this same kind of discussion with the same results: in the immediate
future, certification is inappropriate to this field, as the knowledge is
changing too rapidly. And this in a field that only lightly cuts across
academic disciplines!. akg

From: Alan Graham <Alan.Graham@paconsulting.com>

["Jim Hines"]

Among Rom Kozial cautions us that certifying academic programs in SD might
discourage the spontaneous popping up of courses in SD. That is a serious
concern. There is a flip side, though: An accreditation program might
actually encourage academic institutions to create explicit programs in
system dynamics where right now they have only a single course.

Many institutions already are teaching courses that could be relevant to SD
(e.g. psychology of decision making, computer programming). An explicit
accreditation might encourage them to tie these courses together into a
program, and to add the one or two additional courses that would be
necessary to get recognized.

I once asked Bob Landel at the Darden School at the University of Virginia
here in the USA about this. At the time, anyway, he believed that the
possibility of accredation would have **NO** impact on his schools desire
to offer courses in SD. If I recall correctly, David Andersen thought that
some schools **MIGHT** be influenced to create programs.

Jim Hines
jhines@mit.edu

[Jim Thompson]

About certification, Keith Linard suggests that practitioners / educators to
submit their suggested competency / skill-set lists to this list server. I
would like to see that, too.

I sat for the examination for certification as a public accountant 33 years ago
(Connecticut USA). At the time, the examination focused on the theory and
practice of accounting, business law and auditing. It was felt that candidates
had to be grounded in a certain set of principles, understand the legal
environment of their work and demonstrate skill in the art of checking the
quality of others work so that they could competently attest to the fairness of
financial presentations. It seemed like a reasonable set of requirements at the
time and it still does.*

Recently, accountants in public practice around the world have raised the
question of whether similar standards and an examination could be conjured up
for worldwide practitioners -- a subset of specialists within the profession who
demonstrate the skills to practice across international jurisdictions. Except
for the business law section, these certification requirements will be included
in international practice exams. But in many non US jurisdictions, CPA /
chartered accountant certification requirements include skills in forecasting.
(We should probably talk those people, but thats another posting.) So the
accountants are struggling with what to include in certification requirements
and what to leave out. It is an interesting debate that focuses on minimum
standards.

At a gathering of system dynamics practitioners in business, a group of us heard
about recent projects in industry. At their heart, each project included one or
more "system dynamics models". I put the term in quotes because, other than
including balancing and growth loops, the models were very, very different. The
last presenter, Jay Forrester, suggested that practitioners should have about
"20 models" as the basis for all their work. He started his list with
industrial dynamics, market growth, and world dynamics / limits to growth. We
should try to complete that list as part of this exercise.

In 1973 Jørgen Randers observed features that are common to system dynamics
projects -- issue conceptualization, model formulation, testing / validation,
and implementation. In a certification process for system dynamicists, I can
envision these four sections of an examination posed in the context of the 20 or
so models Jay Forrester suggests.
Jim Thompson

* For Mark B. Wallace: The US CPA exam includes multiple choice questions,
essays and practice problems. I think the US bar and medical boards exams are
similar in their diversity.

From: Jim Thompson <jim@globalprospectus.com>

["Mark B. Wallace"]

Alan Graham identifies certification policies in various occupations. It
should be pointed out that, in some fields (especially medicine and law),
practitioners must be not certified, but licensed. This is an entirely
different concept. Certification is voluntary, whereas licensing is
compulsory, and licenses are issued only by the government agency that
has jurisdiction in the licensees geography. For occupations that require a
license, you can be put in jail for engaging in them without one.

Note: In certain states of the U.S., you can be admitted to the State Bar
Association (i.e., get a license to practice law in that state) without ever
having attended law school. You have to serve an apprenticeship in the
office of an existing member of the Bar, and, after that lawyer approves you
as ready, you then have to take and pass the Bar Exam.

Thanks to Jim Thompson for his (somewhat reassuring) observations on
the U.S. examinations for accounting, law, and medicine. My own
speciality is computer software, and, in that field, its all "multiple-guess."
It is possible, for example, to get certified by Oracle Corporation as an
Oracle DBA (DataBase Administrator) without ever having administered an
Oracle database in the real world. I have former students who have
accomplished this.

Finally, the best explanation Ive seen in print of why (compulsory)
licensing is generally a bad idea is Chapter IX of Milton Friedmans book
"Capitalism & Freedom" (University of Chicago Press, 1962).

Mark Wallace
wallace@acm.org

["Malczynski, Leonard A"]

Mark Wallace states, his profession is, "...computer software, and, in that
field, its all "multiple-guess." It is my profession too. (I am still
aspiring in SD). Marks statement is definitely true for questions of syntax
but not for questions of semantics. The software profession has been done a
grave disservice by not distinguishing between the two distinct types of
knowledge.

Yes, it is true that cramming syntax permits one to pass "certification"
exams. However that should be the second and least important qualification
for a SD capable individual. Conceptual , or semantic knowledge is the most
important. It is harder to learn, and it is not obtained by rote
memorization.

Alas these ideas are not my own so, check out:
After the Gold Rush by Steve McConnell - certification and licensing in
software engineering
Things That Make Us Smart by Donald Norman - brains coupled with tools make
us "smart"
Ben Schnidermans classics on interface design for the syntax/semantics
debate

Len Malczynski
lamalcz@sandia.gov
Energy, Information, & Infrastructure Surety
Policy & Planning Department (6002)
Sandia National Laboratories

["Louis Macovsky"]

Has anyone modeled the value gained by the profession,
individual practioners, or some other stock, if
certification is "required" ?

[Hosts note: Louiss question is a very apt one. I think
we have worn out this discussion thread. But if anyone has
simulation model results they would like to share that would
be a useful addition. ]


Louis Macovsky, D.V.M.
Research Associate
Oregon Health and Science University
macovsky@ohsu.edu
503-721-7884