Posted by =?iso-8859-1?Q?Jean-Jacques_Laubl=E9?= <
jean-jacques.lauble@wanadoo.fr> Hi Everybody
I feel that last posts from Bill, Colm and Keith bear sufficient analogies so as to suggest the following thoughts: The post is a bit long, and I hope that you will excuse me but I have not the time to make it shorter.
The ideas exposed are strictly personal and are the reflects of my own particular experiences and I hope that if some people do not share my thoughts, they will express it so as to share our ideas and maybe correct them accordingly.
I have read the thread of the evaluation web site indicated by Bill.
The thread can be retrieved from the search button asking the following
question:
Can systems and complexiity models be less complex?
(Do not forget the double i in complexity to retrieve the thread). The question ends with these last phrases: <I fear that if systems and complexity approaches to evaluation appear too complex, then too <many people will just throw up their hands and resort to evaluation by 1 or 2easy-to-use (but <misleading, inaccurate, distorting ... but simple) performance indicators. How can we take a <systems approach in such a way that it can be understood and accepted?
I fear that the initiator of the thread after reading it may think that a systemic approach is effectively inappropriate, too complex and not practical for an evaluation method.
A question to Keith
Where can one find the exact definition of 'the Logical Framework' or 'LogFrame'? The LogFrame approach being based on causality, more then half of the job is already done, as long as one does not complicate too much the former definition. When Keith writes that the solution for SD to be better considered is marketing and hard work I would like to make a remark about the marketing side. If somebody goes to the first page of the SD association web site he finds the following definition of SD:
System dynamics is a methodology for studying and managing complex feedback systems, such as one finds in business and other social systems. In fact it has been used to address practically every sort of feedback system. While the word system has been applied to all sorts of situations, feedback is the differentiating descriptor here. Feedback refers to the situation of X affecting Y and Y in turn affecting X perhaps through a chain of causes and effects. One cannot study the link between X and Y and, independently, the link between Y and X and predict how the system will behave.
What does a reader with no knowledge of the field may understand. I am exaggerating but it is to make me better understood.
We, very clever people, will resolve your complex problems that you have never been able to solve, using very exclusive and sophisticated methods.
And what will the reader think.
That guy will play with his tools and my problems that he never heard from before, having a great intellectual pleasure, charging me a lot and leaving me with solutions that I will never be able to implement because he has never been living with the contingencies of my business and I will never understand by what way he found the proposed policies.
This is clearly a definition that is intended to be read by people with some understanding about the subject. You find on the first line three words: Complex, feedback and system.
The word complex is already not commercial.
Like the great majority of the members of the list, I have a tendency to like complexity and to think that complexity adds necessary some value like the one you can find in an aircraft engine or in a computer. It is unfortunately not automatically the case with social problems.
Somebody talked about sitting at the table. To sit at the table one must bring simple and effective ideas that can be understood by the client. I have dealt with many consultants and have always been amused how they feel necessary to bring sophisticated ideas, fearing that if they are too simple I may tell them: I can have these ideas by myself and I do not need your service anymore. They are wrong. People buy simple ideas because they feel that they will be easy to understand, not cost too much, easy to try and easy to implement.
A second idea about complexity it that one of the main idea behind SD is that it simplifies reality. Jay Forrester spoke about Sd being Spartan and that maybe people may doubt about the capacity of a Spartan method to be effective. I think that instead of insisting about complexity it should be preferable to tell them that their problem is simple if one takes away all the things that make it look complicate. This is a much more interesting idea to follow then to present their problem as being a complex one. So instead of saying that SD simplifies reality it should be better to say that reality is simple if one sees it the proper way.
The notion of feedback is too a very vague word that is probably not well understood by the vast majority of people. For instance in France the word feedback is used as a return of information. I do not know if it is the same in the English speaking language. The notion of system is very vague too and is a very theoretical word.
The three words associated together conveys to my opinion an idea of a very technical tool good for very specialised and rare problems.
<It has been used to address practically every sort of feedback system. One supposes that the reader knows what a feedback system is. Or many SDers think that people just do not understand what feedback systems are, even after they have tried desperately to explain them! And the reader is supposed to understand it the first time he reads it!
A second thought about feedback systems, is that a model without feedbacks should not be a SD model.
Kim Warren lately sent me one of his models and I was surprised by the fact that there were no feed backs in it. I had studied his book more then three years ago when it was not published and disposable by the internet. I was probably at the time not able to notice the difference because in his book many models have no or few feedback loops.
Having been always taught to build models with feedbacks, I went back to a CLD with 24 loops that I was never able to transform into a model and tried to redevelop something concentrating only on stocks. After some hours of work I had an extremely simple model with three stocks no loop and that was already a very good representation of the problem I was trying to solve. I can always add the loops further on it they are strong enough and build on a sufficient short time period. One can always in a first approximation ignore loops whose effect are not enough important or too far away. And this model has a lot of dynamic. The rest of the definition conveys the same technical impression. There is after a list of the fields where SD can be useful, but not why it will be useful.
THE READER READING THIS DEFINITION WILL NOT KNOW WHAT SD IS USEFUL FOR.
And it is the first thing that matters for him.
If you want to sell a car to a customer, you will not insist on its technicalities but how it suits his needs.
Why not instead of insisting on what SD is made off say for instance:
SD helps you to understand reality by discovering the simple structure it is made off.
There can be of course better definitions, but it is only an example.
Keith writes:
<The key limitations of LogFrame are: (1) feedback is ignored (or at best is <acknowledged by a dotted line on a diagram in Appendix X); (2) it has no <suite of 'tools' for testing the presumed causality or for rigorously <evaluating the reasons for lack of program success when, as so often has <been the case, the $billions do not achieve the desired effect.
>From the three lacks, only the last concerns a very understandable
>drawback.
The second reason about 'the suite of tools for testing the .' is more a cause of eventual drawback, but the real effect of this lack is difficult to evaluate. It would be better to explain the concrete effect of the lack of tools.
The first lack about feedback being ignored is a very technical point of view that should be transformed into concrete drawbacks. Not to mention that the lack of feedbacks is not a proof of inadequate modelling.
About Colm last post.
<I'd agree that SD alone or proof of its value alone is not sufficient. My <argument is that it can't do any harm - no matter what other credentials one <might have - to be able to prove the usefulness of the SD approach by <reference to well-documented case studies.
I agree too, and I think that if somebody having the power to decide learns that some competitors are using a tool that has results, he may decide to tell his people to use it too however the tool is functioning.
There have been many discussions about the usefulness of SD before and effectively no documentations on real applications. You can find some interesting applications on the Association paper, but documentation about the usefulness of SD are scarce.
I think that there are many reasons for it.
For the business applications, where there is a lot of competition, successful applications will probably be kept secret and unsuccessful be kept secret too. States or organisations may eventually claim for their successful results but not for their unsuccessful. Or it is to my opinion the unsuccessful that are too interesting to study.
About R.O.I.
There has been a thread about 8 months ago about:
'Evaluating expected modeling benefits' plus some others.
I asked that question, but I did not quite used the term R.O.I. Someone said that every time he proposed to a client a study the first thing he did was to evaluate the expected benefit. Another one claimed that it was impossible to do it. I think now that it depends on the definition of the benefit and that both are right.
If the benefit can be measured in money and suppose that you can evaluate the cost of the modelling process (suppose that the job is made by a consultant), you could in theory calculate the R.O.I. I think that with some projects, evaluation the R.O.I. can be done or at least a minimum and maximum of the R.O.I. which is already a good thing.
It is too essential that one tries to evaluate the expected profit of a modelling process even if it is not reducible to an amount of money, but I think too that the idea of presenting the modelling effort like a normal investment where one can usually calculate the R.O.I. can be extremely dangerous for some kind of projects especially the kind of problems SD is trying to solve.
Why it is dangerous? Because it presents SD as an investment where you pay a certain price and where you should necessarily get a predetermined result and in a certain duration of time as R.O.I is calculated on a certain time horizon.
This view forgets completely that the model cannot give more than it has, depends on the impact that it will have and on the way it will have been assimilated by the customer. It supposes that after a time the modelling effort is finished and its benefit too. It makes the customer think that his problem will be solved by a unique model after a certain time. Or it is not the real benefit of the modelling process and this is very difficult to sell.
I have a certain time looked for a concrete profit. The result is that I have tried to make very complicated models that were supposed to bring me a sure profit. I have since forgotten this idea. I build now models from whom I hope to bring me a better understanding of my problems than the one I have actually. This is an easier objective to reach and it is too an objective that will always be actualised. So there will never be any R.O.I. but more new ideas, and a better understanding. And the advantage of this way of thinking is that you are not obliged to build something complicated at first: you can first build a very simple model, use it extensively for a certain time, learn how it conforms to the reality and where is does not, and after a time decide to make it evolve.
So behind the model is how you will use it, how you are organised to apply the policies it advises, the necessary learning path that you have to comply with and to accept even if it takes years. Life is long and the ability to model and to use models builds through years. This is one of the difficulty to sell SD, but could certainly be better explained to avoid a misunderstanding of the modelling effort and reducing it to a simple investment that should bring profit, preferably quickly and that can instead bring disillusion.
To resume my thinking: and if by modelling, I decide after some years to change the stuff I sell presently and that modelling has helped me doing it? How can you calculate a R.O.I for that?
As to the documented case studies, some people will argue that there are plenty of them in the text books but the models are generally very simple or archetypal or if more elaborate do not expose the path that has been followed to develop them to their present state.
Jean-Jacques Laublé Allocar
Strasbourg France
Posted by =?iso-8859-1?Q?Jean-Jacques_Laubl=E9?= <
jean-jacques.lauble@wanadoo.fr> posting date Tue, 3 Jan 2006 12:16:42 +0100