Posted by ""Martin Testa"" <mtesta@nobelmantrich.com>
Posted by: Martin Testa <mtesta@nobelmantrich.com>
QUERY: Why is it that European Union (EU) financed research projects still use the Logical Framework (LF) Method as the prerequisite de facto standard for project design, research, monitoring, and evaluation? It is now widely used throughout most of the EU's domestic and external aid programmes.
My firm belief is that although the LF method has its usefulness at project management level, some EU research projects have been using LF to point to the underlying elements relating to a cause solely from a hierarchical perspective and miss out on the more thorough analysis and insight that SD can offer. I know of specific projects which badly missed out important issues because the feedback loops, stocks, flows, etc were missing from the analytical, monitoring and evaluation stages of the project, ending up in simplistic conclusions and recommendations.
Could something be done here to get the European Union aid agencies to understand better the real benefits of SD, to understand the limitations of LF and not to rely solely on LF?
Best Regards,
Martin Testa
------------------------------------------------------------------------
-
Adjunct Professor of Strategy and Marketing
Maastricht School of Management
Managing Director, Nobel Mantrich Ltd
Posted by ""Martin Testa"" <mtesta@nobelmantrich.com>
posting date Fri, 30 Dec 2005 14:48:28 +0100
The Logical Framework and SD
-
Martin Testa mtesta nobelmantric
- Newbie
- Posts: 1
- Joined: Fri Mar 29, 2002 3:39 am
-
Keith Linard klin4960 bigpond.ne
- Junior Member
- Posts: 7
- Joined: Fri Mar 29, 2002 3:39 am
The Logical Framework and SD
Posted by ""Keith Linard"" <klin4960@bigpond.net.au>
'The Logical Framework' (Europe) and its American and Australian equivalent 'LogFrame' offer a superb opening into what is possibly the 'market' of greatest potential for SD. The question should not be ""Why are they still using the Logical Framework', but why aren't SD academics & consultants in there saying ""We can make the 'LogFrame' work even better.
LogFrame is the basis for the evaluation (pre-project justification and post project evaluation) of aid projects by virtually all developed countries. It requires the proponent of a project to justify the 'causality' between 'inputs' ($, skills, other resources), 'processes', 'outputs' (the immediate reults, eg a new highway, or training program) and outcomes (reduction in poverty etc). When introduced in the 1960's it was a dramatic leap forward in that it explicitly focused government program developers, for the first time, on 'cause-and-effect', and enforced a structured, systematic approach to evaluation.
The key limitations of LogFrame are: (1) feedback is ignored (or at best is acknowledged by a dotted line on a diagram in Appendix X); (2) it has no suite of 'tools' for testing the presumed causality or for rigorously evaluating the reasons for lack of program success when, as so often has been the case, the $billions do not achieve the desired effect. SDM theory, the SD software (Powersim, Vensim, IThink, Consideo etc) and the associated suite of systems thinking tools (Checkland's 'Soft Systems Methodology', Colin Eden's 'Cognitive Mapping', Coyle's Influence Diagrams','Hexagons', Causal Loop Diagrams'etc) fill pricisely these gaps in LogFrame practice.
It has been my experience, when giving presentations on SDM to officials in the Foreign Aid Sector and contractors who live off this sector, that they are more receptive to SD than any other group because they have been 'sensitised' to the 'cause-and-effect' relationship; they are sensitive to the time horizon involved in program 'cause-and-effect'; and they are all too aware of the limitations of LogFrame. When presented with SDM they immediately see how it can improve their own practice.
Two related markets, with close affinity to the 'ideological approaches' of the foreign aid sector, are the Government audit offices (eg, GAO in the US), and central government program evaluation units. The officials are already sensitised to the key issues raised by SDM and, from my experience, are very open to SDM.
How do we get to these markets when they don't come to the International System Dynamics Conferences? Present SDM papers at Evaluation Conferences, Audit Conferences, Defence & Government Administration Conferences etc etc etc. Find your market, target your market, feed your markert. Do not expect the market to come to you.
I have followed with interest the extended discussion on the apparent failure of SD to make inroads. To me the issue is simply one of targetted marketing ... and hard work. In 3 years, in the early 1990's, while building up an SD consulting practice, I gave about 40 seminars in Government agencies, with audiences from 15 to 150, mainly at executive level including at least 30 from the military at '1 Star' general level or higher (including the Chief of the General Staff and a personal 2 hour presentation to the Chief of the Air Force). One result was that in 1996 Australia was the 5th highest market for Powersim (after Norway, UK, Germany & USA), notwithstanding its small population. My problem was not getting SDM consulting work, it was having the time to do it. I passed on to others probably a third of the work that came my way.
Keith Linard
134 Gisborne Road
Bacchus Marsh
Vic 3340
Posted by ""Keith Linard"" <klin4960@bigpond.net.au>
posting date Sun, 1 Jan 2006 11:23:47 +1100
'The Logical Framework' (Europe) and its American and Australian equivalent 'LogFrame' offer a superb opening into what is possibly the 'market' of greatest potential for SD. The question should not be ""Why are they still using the Logical Framework', but why aren't SD academics & consultants in there saying ""We can make the 'LogFrame' work even better.
LogFrame is the basis for the evaluation (pre-project justification and post project evaluation) of aid projects by virtually all developed countries. It requires the proponent of a project to justify the 'causality' between 'inputs' ($, skills, other resources), 'processes', 'outputs' (the immediate reults, eg a new highway, or training program) and outcomes (reduction in poverty etc). When introduced in the 1960's it was a dramatic leap forward in that it explicitly focused government program developers, for the first time, on 'cause-and-effect', and enforced a structured, systematic approach to evaluation.
The key limitations of LogFrame are: (1) feedback is ignored (or at best is acknowledged by a dotted line on a diagram in Appendix X); (2) it has no suite of 'tools' for testing the presumed causality or for rigorously evaluating the reasons for lack of program success when, as so often has been the case, the $billions do not achieve the desired effect. SDM theory, the SD software (Powersim, Vensim, IThink, Consideo etc) and the associated suite of systems thinking tools (Checkland's 'Soft Systems Methodology', Colin Eden's 'Cognitive Mapping', Coyle's Influence Diagrams','Hexagons', Causal Loop Diagrams'etc) fill pricisely these gaps in LogFrame practice.
It has been my experience, when giving presentations on SDM to officials in the Foreign Aid Sector and contractors who live off this sector, that they are more receptive to SD than any other group because they have been 'sensitised' to the 'cause-and-effect' relationship; they are sensitive to the time horizon involved in program 'cause-and-effect'; and they are all too aware of the limitations of LogFrame. When presented with SDM they immediately see how it can improve their own practice.
Two related markets, with close affinity to the 'ideological approaches' of the foreign aid sector, are the Government audit offices (eg, GAO in the US), and central government program evaluation units. The officials are already sensitised to the key issues raised by SDM and, from my experience, are very open to SDM.
How do we get to these markets when they don't come to the International System Dynamics Conferences? Present SDM papers at Evaluation Conferences, Audit Conferences, Defence & Government Administration Conferences etc etc etc. Find your market, target your market, feed your markert. Do not expect the market to come to you.
I have followed with interest the extended discussion on the apparent failure of SD to make inroads. To me the issue is simply one of targetted marketing ... and hard work. In 3 years, in the early 1990's, while building up an SD consulting practice, I gave about 40 seminars in Government agencies, with audiences from 15 to 150, mainly at executive level including at least 30 from the military at '1 Star' general level or higher (including the Chief of the General Staff and a personal 2 hour presentation to the Chief of the Air Force). One result was that in 1996 Australia was the 5th highest market for Powersim (after Norway, UK, Germany & USA), notwithstanding its small population. My problem was not getting SDM consulting work, it was having the time to do it. I passed on to others probably a third of the work that came my way.
Keith Linard
134 Gisborne Road
Bacchus Marsh
Vic 3340
Posted by ""Keith Linard"" <klin4960@bigpond.net.au>
posting date Sun, 1 Jan 2006 11:23:47 +1100
-
Bill Harris bill_harris facilita
- Junior Member
- Posts: 19
- Joined: Fri Mar 29, 2002 3:39 am
The Logical Framework and SD
Posted by Bill Harris <bill_harris@facilitatedsystems.com>
""Keith Linard klin4960 bigpond.net.au"" <system-dynamics@VENSIM.COM> writes:
>> How do we get to these markets when they don't come to the
>> International System Dynamics Conferences? Present SDM papers at
>> Evaluation Conferences, Audit Conferences, Defence & Government
>> Administration Conferences etc etc etc. Find your market, target
>> your market, feed your markert. Do not expect the market to come to
>> you.
Keith,
There's a growing appreciation for SD and other systemic approaches in the evaluation community. I've co-led a professional development workshop (along with Bob Williams [NZ], who may be here, and Glenda
Eoyang) on the topic for the American Evaluation Association and helped prepare material for at least one if not two others. Bob has spearheaded efforts to get the evaluation community worldwide to consider systemic approaches and been responsible for much of the progress in this area, and he's got some material related to systems thinking and evaluation on his Web site. The AEA now has a technical interest group (TIG) on systems thinking. There are other interesting things going on, too; perhaps Bob can fill the group in.
For more information, check out the eval-sys mailing list
(http://evaluation.wmich.edu/archives/eval-sys.html). Bob, any more comments?
Bill
- --
Bill Harris
Facilitated Systems
Posted by Bill Harris <bill_harris@facilitatedsystems.com>
posting date Sun, 01 Jan 2006 19:07:11 -0800
""Keith Linard klin4960 bigpond.net.au"" <system-dynamics@VENSIM.COM> writes:
>> How do we get to these markets when they don't come to the
>> International System Dynamics Conferences? Present SDM papers at
>> Evaluation Conferences, Audit Conferences, Defence & Government
>> Administration Conferences etc etc etc. Find your market, target
>> your market, feed your markert. Do not expect the market to come to
>> you.
Keith,
There's a growing appreciation for SD and other systemic approaches in the evaluation community. I've co-led a professional development workshop (along with Bob Williams [NZ], who may be here, and Glenda
Eoyang) on the topic for the American Evaluation Association and helped prepare material for at least one if not two others. Bob has spearheaded efforts to get the evaluation community worldwide to consider systemic approaches and been responsible for much of the progress in this area, and he's got some material related to systems thinking and evaluation on his Web site. The AEA now has a technical interest group (TIG) on systems thinking. There are other interesting things going on, too; perhaps Bob can fill the group in.
For more information, check out the eval-sys mailing list
(http://evaluation.wmich.edu/archives/eval-sys.html). Bob, any more comments?
Bill
- --
Bill Harris
Facilitated Systems
Posted by Bill Harris <bill_harris@facilitatedsystems.com>
posting date Sun, 01 Jan 2006 19:07:11 -0800
-
Jean-Jacques Laublé jean-jacques
- Senior Member
- Posts: 68
- Joined: Fri Mar 29, 2002 3:39 am
The Logical Framework and SD
Posted by =?iso-8859-1?Q?Jean-Jacques_Laubl=E9?= <jean-jacques.lauble@wanadoo.fr> Hi Everybody
I feel that last posts from Bill, Colm and Keith bear sufficient analogies so as to suggest the following thoughts: The post is a bit long, and I hope that you will excuse me but I have not the time to make it shorter.
The ideas exposed are strictly personal and are the reflects of my own particular experiences and I hope that if some people do not share my thoughts, they will express it so as to share our ideas and maybe correct them accordingly.
I have read the thread of the evaluation web site indicated by Bill.
The thread can be retrieved from the search button asking the following
question:
Can systems and complexiity models be less complex?
(Do not forget the double i in complexity to retrieve the thread). The question ends with these last phrases: <I fear that if systems and complexity approaches to evaluation appear too complex, then too <many people will just throw up their hands and resort to evaluation by 1 or 2easy-to-use (but <misleading, inaccurate, distorting ... but simple) performance indicators. How can we take a <systems approach in such a way that it can be understood and accepted?
I fear that the initiator of the thread after reading it may think that a systemic approach is effectively inappropriate, too complex and not practical for an evaluation method.
A question to Keith
Where can one find the exact definition of 'the Logical Framework' or 'LogFrame'? The LogFrame approach being based on causality, more then half of the job is already done, as long as one does not complicate too much the former definition. When Keith writes that the solution for SD to be better considered is marketing and hard work I would like to make a remark about the marketing side. If somebody goes to the first page of the SD association web site he finds the following definition of SD:
System dynamics is a methodology for studying and managing complex feedback systems, such as one finds in business and other social systems. In fact it has been used to address practically every sort of feedback system. While the word system has been applied to all sorts of situations, feedback is the differentiating descriptor here. Feedback refers to the situation of X affecting Y and Y in turn affecting X perhaps through a chain of causes and effects. One cannot study the link between X and Y and, independently, the link between Y and X and predict how the system will behave.
What does a reader with no knowledge of the field may understand. I am exaggerating but it is to make me better understood.
We, very clever people, will resolve your complex problems that you have never been able to solve, using very exclusive and sophisticated methods.
And what will the reader think.
That guy will play with his tools and my problems that he never heard from before, having a great intellectual pleasure, charging me a lot and leaving me with solutions that I will never be able to implement because he has never been living with the contingencies of my business and I will never understand by what way he found the proposed policies.
This is clearly a definition that is intended to be read by people with some understanding about the subject. You find on the first line three words: Complex, feedback and system.
The word complex is already not commercial.
Like the great majority of the members of the list, I have a tendency to like complexity and to think that complexity adds necessary some value like the one you can find in an aircraft engine or in a computer. It is unfortunately not automatically the case with social problems.
Somebody talked about sitting at the table. To sit at the table one must bring simple and effective ideas that can be understood by the client. I have dealt with many consultants and have always been amused how they feel necessary to bring sophisticated ideas, fearing that if they are too simple I may tell them: I can have these ideas by myself and I do not need your service anymore. They are wrong. People buy simple ideas because they feel that they will be easy to understand, not cost too much, easy to try and easy to implement.
A second idea about complexity it that one of the main idea behind SD is that it simplifies reality. Jay Forrester spoke about Sd being Spartan and that maybe people may doubt about the capacity of a Spartan method to be effective. I think that instead of insisting about complexity it should be preferable to tell them that their problem is simple if one takes away all the things that make it look complicate. This is a much more interesting idea to follow then to present their problem as being a complex one. So instead of saying that SD simplifies reality it should be better to say that reality is simple if one sees it the proper way.
The notion of feedback is too a very vague word that is probably not well understood by the vast majority of people. For instance in France the word feedback is used as a return of information. I do not know if it is the same in the English speaking language. The notion of system is very vague too and is a very theoretical word.
The three words associated together conveys to my opinion an idea of a very technical tool good for very specialised and rare problems.
<It has been used to address practically every sort of feedback system. One supposes that the reader knows what a feedback system is. Or many SDers think that people just do not understand what feedback systems are, even after they have tried desperately to explain them! And the reader is supposed to understand it the first time he reads it!
A second thought about feedback systems, is that a model without feedbacks should not be a SD model.
Kim Warren lately sent me one of his models and I was surprised by the fact that there were no feed backs in it. I had studied his book more then three years ago when it was not published and disposable by the internet. I was probably at the time not able to notice the difference because in his book many models have no or few feedback loops.
Having been always taught to build models with feedbacks, I went back to a CLD with 24 loops that I was never able to transform into a model and tried to redevelop something concentrating only on stocks. After some hours of work I had an extremely simple model with three stocks no loop and that was already a very good representation of the problem I was trying to solve. I can always add the loops further on it they are strong enough and build on a sufficient short time period. One can always in a first approximation ignore loops whose effect are not enough important or too far away. And this model has a lot of dynamic. The rest of the definition conveys the same technical impression. There is after a list of the fields where SD can be useful, but not why it will be useful.
THE READER READING THIS DEFINITION WILL NOT KNOW WHAT SD IS USEFUL FOR.
And it is the first thing that matters for him.
If you want to sell a car to a customer, you will not insist on its technicalities but how it suits his needs.
Why not instead of insisting on what SD is made off say for instance:
SD helps you to understand reality by discovering the simple structure it is made off.
There can be of course better definitions, but it is only an example.
Keith writes:
<The key limitations of LogFrame are: (1) feedback is ignored (or at best is <acknowledged by a dotted line on a diagram in Appendix X); (2) it has no <suite of 'tools' for testing the presumed causality or for rigorously <evaluating the reasons for lack of program success when, as so often has <been the case, the $billions do not achieve the desired effect.
>From the three lacks, only the last concerns a very understandable
>drawback.
The second reason about 'the suite of tools for testing the .' is more a cause of eventual drawback, but the real effect of this lack is difficult to evaluate. It would be better to explain the concrete effect of the lack of tools.
The first lack about feedback being ignored is a very technical point of view that should be transformed into concrete drawbacks. Not to mention that the lack of feedbacks is not a proof of inadequate modelling.
About Colm last post.
<I'd agree that SD alone or proof of its value alone is not sufficient. My <argument is that it can't do any harm - no matter what other credentials one <might have - to be able to prove the usefulness of the SD approach by <reference to well-documented case studies.
I agree too, and I think that if somebody having the power to decide learns that some competitors are using a tool that has results, he may decide to tell his people to use it too however the tool is functioning.
There have been many discussions about the usefulness of SD before and effectively no documentations on real applications. You can find some interesting applications on the Association paper, but documentation about the usefulness of SD are scarce.
I think that there are many reasons for it.
For the business applications, where there is a lot of competition, successful applications will probably be kept secret and unsuccessful be kept secret too. States or organisations may eventually claim for their successful results but not for their unsuccessful. Or it is to my opinion the unsuccessful that are too interesting to study.
About R.O.I.
There has been a thread about 8 months ago about:
'Evaluating expected modeling benefits' plus some others.
I asked that question, but I did not quite used the term R.O.I. Someone said that every time he proposed to a client a study the first thing he did was to evaluate the expected benefit. Another one claimed that it was impossible to do it. I think now that it depends on the definition of the benefit and that both are right.
If the benefit can be measured in money and suppose that you can evaluate the cost of the modelling process (suppose that the job is made by a consultant), you could in theory calculate the R.O.I. I think that with some projects, evaluation the R.O.I. can be done or at least a minimum and maximum of the R.O.I. which is already a good thing.
It is too essential that one tries to evaluate the expected profit of a modelling process even if it is not reducible to an amount of money, but I think too that the idea of presenting the modelling effort like a normal investment where one can usually calculate the R.O.I. can be extremely dangerous for some kind of projects especially the kind of problems SD is trying to solve.
Why it is dangerous? Because it presents SD as an investment where you pay a certain price and where you should necessarily get a predetermined result and in a certain duration of time as R.O.I is calculated on a certain time horizon.
This view forgets completely that the model cannot give more than it has, depends on the impact that it will have and on the way it will have been assimilated by the customer. It supposes that after a time the modelling effort is finished and its benefit too. It makes the customer think that his problem will be solved by a unique model after a certain time. Or it is not the real benefit of the modelling process and this is very difficult to sell.
I have a certain time looked for a concrete profit. The result is that I have tried to make very complicated models that were supposed to bring me a sure profit. I have since forgotten this idea. I build now models from whom I hope to bring me a better understanding of my problems than the one I have actually. This is an easier objective to reach and it is too an objective that will always be actualised. So there will never be any R.O.I. but more new ideas, and a better understanding. And the advantage of this way of thinking is that you are not obliged to build something complicated at first: you can first build a very simple model, use it extensively for a certain time, learn how it conforms to the reality and where is does not, and after a time decide to make it evolve.
So behind the model is how you will use it, how you are organised to apply the policies it advises, the necessary learning path that you have to comply with and to accept even if it takes years. Life is long and the ability to model and to use models builds through years. This is one of the difficulty to sell SD, but could certainly be better explained to avoid a misunderstanding of the modelling effort and reducing it to a simple investment that should bring profit, preferably quickly and that can instead bring disillusion.
To resume my thinking: and if by modelling, I decide after some years to change the stuff I sell presently and that modelling has helped me doing it? How can you calculate a R.O.I for that?
As to the documented case studies, some people will argue that there are plenty of them in the text books but the models are generally very simple or archetypal or if more elaborate do not expose the path that has been followed to develop them to their present state.
Jean-Jacques Laublé Allocar
Strasbourg France
Posted by =?iso-8859-1?Q?Jean-Jacques_Laubl=E9?= <jean-jacques.lauble@wanadoo.fr> posting date Tue, 3 Jan 2006 12:16:42 +0100
I feel that last posts from Bill, Colm and Keith bear sufficient analogies so as to suggest the following thoughts: The post is a bit long, and I hope that you will excuse me but I have not the time to make it shorter.
The ideas exposed are strictly personal and are the reflects of my own particular experiences and I hope that if some people do not share my thoughts, they will express it so as to share our ideas and maybe correct them accordingly.
I have read the thread of the evaluation web site indicated by Bill.
The thread can be retrieved from the search button asking the following
question:
Can systems and complexiity models be less complex?
(Do not forget the double i in complexity to retrieve the thread). The question ends with these last phrases: <I fear that if systems and complexity approaches to evaluation appear too complex, then too <many people will just throw up their hands and resort to evaluation by 1 or 2easy-to-use (but <misleading, inaccurate, distorting ... but simple) performance indicators. How can we take a <systems approach in such a way that it can be understood and accepted?
I fear that the initiator of the thread after reading it may think that a systemic approach is effectively inappropriate, too complex and not practical for an evaluation method.
A question to Keith
Where can one find the exact definition of 'the Logical Framework' or 'LogFrame'? The LogFrame approach being based on causality, more then half of the job is already done, as long as one does not complicate too much the former definition. When Keith writes that the solution for SD to be better considered is marketing and hard work I would like to make a remark about the marketing side. If somebody goes to the first page of the SD association web site he finds the following definition of SD:
System dynamics is a methodology for studying and managing complex feedback systems, such as one finds in business and other social systems. In fact it has been used to address practically every sort of feedback system. While the word system has been applied to all sorts of situations, feedback is the differentiating descriptor here. Feedback refers to the situation of X affecting Y and Y in turn affecting X perhaps through a chain of causes and effects. One cannot study the link between X and Y and, independently, the link between Y and X and predict how the system will behave.
What does a reader with no knowledge of the field may understand. I am exaggerating but it is to make me better understood.
We, very clever people, will resolve your complex problems that you have never been able to solve, using very exclusive and sophisticated methods.
And what will the reader think.
That guy will play with his tools and my problems that he never heard from before, having a great intellectual pleasure, charging me a lot and leaving me with solutions that I will never be able to implement because he has never been living with the contingencies of my business and I will never understand by what way he found the proposed policies.
This is clearly a definition that is intended to be read by people with some understanding about the subject. You find on the first line three words: Complex, feedback and system.
The word complex is already not commercial.
Like the great majority of the members of the list, I have a tendency to like complexity and to think that complexity adds necessary some value like the one you can find in an aircraft engine or in a computer. It is unfortunately not automatically the case with social problems.
Somebody talked about sitting at the table. To sit at the table one must bring simple and effective ideas that can be understood by the client. I have dealt with many consultants and have always been amused how they feel necessary to bring sophisticated ideas, fearing that if they are too simple I may tell them: I can have these ideas by myself and I do not need your service anymore. They are wrong. People buy simple ideas because they feel that they will be easy to understand, not cost too much, easy to try and easy to implement.
A second idea about complexity it that one of the main idea behind SD is that it simplifies reality. Jay Forrester spoke about Sd being Spartan and that maybe people may doubt about the capacity of a Spartan method to be effective. I think that instead of insisting about complexity it should be preferable to tell them that their problem is simple if one takes away all the things that make it look complicate. This is a much more interesting idea to follow then to present their problem as being a complex one. So instead of saying that SD simplifies reality it should be better to say that reality is simple if one sees it the proper way.
The notion of feedback is too a very vague word that is probably not well understood by the vast majority of people. For instance in France the word feedback is used as a return of information. I do not know if it is the same in the English speaking language. The notion of system is very vague too and is a very theoretical word.
The three words associated together conveys to my opinion an idea of a very technical tool good for very specialised and rare problems.
<It has been used to address practically every sort of feedback system. One supposes that the reader knows what a feedback system is. Or many SDers think that people just do not understand what feedback systems are, even after they have tried desperately to explain them! And the reader is supposed to understand it the first time he reads it!
A second thought about feedback systems, is that a model without feedbacks should not be a SD model.
Kim Warren lately sent me one of his models and I was surprised by the fact that there were no feed backs in it. I had studied his book more then three years ago when it was not published and disposable by the internet. I was probably at the time not able to notice the difference because in his book many models have no or few feedback loops.
Having been always taught to build models with feedbacks, I went back to a CLD with 24 loops that I was never able to transform into a model and tried to redevelop something concentrating only on stocks. After some hours of work I had an extremely simple model with three stocks no loop and that was already a very good representation of the problem I was trying to solve. I can always add the loops further on it they are strong enough and build on a sufficient short time period. One can always in a first approximation ignore loops whose effect are not enough important or too far away. And this model has a lot of dynamic. The rest of the definition conveys the same technical impression. There is after a list of the fields where SD can be useful, but not why it will be useful.
THE READER READING THIS DEFINITION WILL NOT KNOW WHAT SD IS USEFUL FOR.
And it is the first thing that matters for him.
If you want to sell a car to a customer, you will not insist on its technicalities but how it suits his needs.
Why not instead of insisting on what SD is made off say for instance:
SD helps you to understand reality by discovering the simple structure it is made off.
There can be of course better definitions, but it is only an example.
Keith writes:
<The key limitations of LogFrame are: (1) feedback is ignored (or at best is <acknowledged by a dotted line on a diagram in Appendix X); (2) it has no <suite of 'tools' for testing the presumed causality or for rigorously <evaluating the reasons for lack of program success when, as so often has <been the case, the $billions do not achieve the desired effect.
>From the three lacks, only the last concerns a very understandable
>drawback.
The second reason about 'the suite of tools for testing the .' is more a cause of eventual drawback, but the real effect of this lack is difficult to evaluate. It would be better to explain the concrete effect of the lack of tools.
The first lack about feedback being ignored is a very technical point of view that should be transformed into concrete drawbacks. Not to mention that the lack of feedbacks is not a proof of inadequate modelling.
About Colm last post.
<I'd agree that SD alone or proof of its value alone is not sufficient. My <argument is that it can't do any harm - no matter what other credentials one <might have - to be able to prove the usefulness of the SD approach by <reference to well-documented case studies.
I agree too, and I think that if somebody having the power to decide learns that some competitors are using a tool that has results, he may decide to tell his people to use it too however the tool is functioning.
There have been many discussions about the usefulness of SD before and effectively no documentations on real applications. You can find some interesting applications on the Association paper, but documentation about the usefulness of SD are scarce.
I think that there are many reasons for it.
For the business applications, where there is a lot of competition, successful applications will probably be kept secret and unsuccessful be kept secret too. States or organisations may eventually claim for their successful results but not for their unsuccessful. Or it is to my opinion the unsuccessful that are too interesting to study.
About R.O.I.
There has been a thread about 8 months ago about:
'Evaluating expected modeling benefits' plus some others.
I asked that question, but I did not quite used the term R.O.I. Someone said that every time he proposed to a client a study the first thing he did was to evaluate the expected benefit. Another one claimed that it was impossible to do it. I think now that it depends on the definition of the benefit and that both are right.
If the benefit can be measured in money and suppose that you can evaluate the cost of the modelling process (suppose that the job is made by a consultant), you could in theory calculate the R.O.I. I think that with some projects, evaluation the R.O.I. can be done or at least a minimum and maximum of the R.O.I. which is already a good thing.
It is too essential that one tries to evaluate the expected profit of a modelling process even if it is not reducible to an amount of money, but I think too that the idea of presenting the modelling effort like a normal investment where one can usually calculate the R.O.I. can be extremely dangerous for some kind of projects especially the kind of problems SD is trying to solve.
Why it is dangerous? Because it presents SD as an investment where you pay a certain price and where you should necessarily get a predetermined result and in a certain duration of time as R.O.I is calculated on a certain time horizon.
This view forgets completely that the model cannot give more than it has, depends on the impact that it will have and on the way it will have been assimilated by the customer. It supposes that after a time the modelling effort is finished and its benefit too. It makes the customer think that his problem will be solved by a unique model after a certain time. Or it is not the real benefit of the modelling process and this is very difficult to sell.
I have a certain time looked for a concrete profit. The result is that I have tried to make very complicated models that were supposed to bring me a sure profit. I have since forgotten this idea. I build now models from whom I hope to bring me a better understanding of my problems than the one I have actually. This is an easier objective to reach and it is too an objective that will always be actualised. So there will never be any R.O.I. but more new ideas, and a better understanding. And the advantage of this way of thinking is that you are not obliged to build something complicated at first: you can first build a very simple model, use it extensively for a certain time, learn how it conforms to the reality and where is does not, and after a time decide to make it evolve.
So behind the model is how you will use it, how you are organised to apply the policies it advises, the necessary learning path that you have to comply with and to accept even if it takes years. Life is long and the ability to model and to use models builds through years. This is one of the difficulty to sell SD, but could certainly be better explained to avoid a misunderstanding of the modelling effort and reducing it to a simple investment that should bring profit, preferably quickly and that can instead bring disillusion.
To resume my thinking: and if by modelling, I decide after some years to change the stuff I sell presently and that modelling has helped me doing it? How can you calculate a R.O.I for that?
As to the documented case studies, some people will argue that there are plenty of them in the text books but the models are generally very simple or archetypal or if more elaborate do not expose the path that has been followed to develop them to their present state.
Jean-Jacques Laublé Allocar
Strasbourg France
Posted by =?iso-8859-1?Q?Jean-Jacques_Laubl=E9?= <jean-jacques.lauble@wanadoo.fr> posting date Tue, 3 Jan 2006 12:16:42 +0100
-
Keith Linard klin4960 bigpond.ne
- Junior Member
- Posts: 7
- Joined: Fri Mar 29, 2002 3:39 am
The Logical Framework and SD
Posted by ""Keith Linard"" <klin4960@bigpond.net.au> Jean-Jacques thanks for your detailed contribution. It is a valuable contribution to the discussion. I will respond to several points:
<< Where can one find the exact definition of 'the Logical Framework' >>
Response: One detailed presentation is found on the Australian Government Aid Agency (AusAid) Website: 'The Logical Framework Approach' http://www.ausaid.gov.au/ausguide/pdf/a ... ine3.3.pdf .
<< The LogFrame approach being based on causality, more then half of the job is already done ... >>
To the extent that the thousands of people in Government aid agencies and the many consultant firms in this area are sensitised to thinking about causality, there is a real opening for dialogue on SDM and other systems thinking frameworks.
BUT ... in my experience (e.g., in the mid 1980's being tasked to reform program evaluation procedures across all Australian Federal Government
agencies) identifying ""presumed causality"" is the easy step, and all too often is simply wishful thinking. Diverse ex-post studies of innumerable foreign aid programs, welfare programs, drug programs etc etc testify to the ubiquitous nature of wishful thinking. (Dare I suggest that there was at least a measure of wishful thinking in relation to cause-and-effect issues among the US-UK-Australian coalition planners in our noble quest to introduce democratic freedoms to Iraq?)
The casue-and-effect chain for a government program (ultra simplified) shows a number of possible outcomes:
1. (GOVT PROGRAM) sets in train
(CAUSAL PROCESSES) which lead to
(DESIRED OUTCOME) >>> Program Success
1a.(GOVT PROGRAM) sets in train
(CAUSAL PROCESSES) which also lead to
(UNDESIRABLE SIDE EFFECTS) >>> Tarnished Success
2. (GOVT PROGRAM) sets in train
(CAUSAL PROCESSES) which FAILS TO lead to
(DESIRED OUTCOME) >>> Theory Failure
3. (GOVT PROGRAM) fails to set in train
(CAUSAL PROCESSES) which otherwise might have led to
(DESIRED OUTCOME) >>> Implementation Failure
Outcomes 1a, 2 and 3 are all too common. Sure, not all government programs complete failures. But (as a former Director of Evaluation in one Government department and former Chief Finance Officer in the Australian Federal Department of Finance)I have witnessed untold millions of dollars wasted as a result of wishful thinking on causality. Hence my view that a fundamental weakness of most program evaluation approaches (including the Logical Framework) is that they do not have ... "" a suite of 'tools' for testing the presumed causality or for rigorously evaluating the reasons for lack of program success ..."". SDM and allied systems thinking approaches have the tools --- let's push them into these environments where at least the professionals talk about causality. My regret is that IThink v.1.0 did not appear until 1988 --- 3 years ealier there would have been a good chance to have had it mandated for every Australian federal policy development and program evaluation unit.
Regarding Jean-Jacques' concern at the complexity of modelling, I have found it relatively easy to transform a Government program Logic Model
(qualitative) into an SD model and use it to highlight the dynamic effects that are not shown in the inevitable spreadsheet models that were used to justify the program. Talking of complexity, the size of the SD model is typically 1/10 the size of the spreadsheet. I suggest the spreadsheet model is not only more complex, it is typically 'wrong'.
Keith Linard
134 Gisborne Road
Bacchus Marsh
Vic 3340
Posted by ""Keith Linard"" <klin4960@bigpond.net.au>
posting date Thu, 5 Jan 2006 21:23:26 +1000
<< Where can one find the exact definition of 'the Logical Framework' >>
Response: One detailed presentation is found on the Australian Government Aid Agency (AusAid) Website: 'The Logical Framework Approach' http://www.ausaid.gov.au/ausguide/pdf/a ... ine3.3.pdf .
<< The LogFrame approach being based on causality, more then half of the job is already done ... >>
To the extent that the thousands of people in Government aid agencies and the many consultant firms in this area are sensitised to thinking about causality, there is a real opening for dialogue on SDM and other systems thinking frameworks.
BUT ... in my experience (e.g., in the mid 1980's being tasked to reform program evaluation procedures across all Australian Federal Government
agencies) identifying ""presumed causality"" is the easy step, and all too often is simply wishful thinking. Diverse ex-post studies of innumerable foreign aid programs, welfare programs, drug programs etc etc testify to the ubiquitous nature of wishful thinking. (Dare I suggest that there was at least a measure of wishful thinking in relation to cause-and-effect issues among the US-UK-Australian coalition planners in our noble quest to introduce democratic freedoms to Iraq?)
The casue-and-effect chain for a government program (ultra simplified) shows a number of possible outcomes:
1. (GOVT PROGRAM) sets in train
(CAUSAL PROCESSES) which lead to
(DESIRED OUTCOME) >>> Program Success
1a.(GOVT PROGRAM) sets in train
(CAUSAL PROCESSES) which also lead to
(UNDESIRABLE SIDE EFFECTS) >>> Tarnished Success
2. (GOVT PROGRAM) sets in train
(CAUSAL PROCESSES) which FAILS TO lead to
(DESIRED OUTCOME) >>> Theory Failure
3. (GOVT PROGRAM) fails to set in train
(CAUSAL PROCESSES) which otherwise might have led to
(DESIRED OUTCOME) >>> Implementation Failure
Outcomes 1a, 2 and 3 are all too common. Sure, not all government programs complete failures. But (as a former Director of Evaluation in one Government department and former Chief Finance Officer in the Australian Federal Department of Finance)I have witnessed untold millions of dollars wasted as a result of wishful thinking on causality. Hence my view that a fundamental weakness of most program evaluation approaches (including the Logical Framework) is that they do not have ... "" a suite of 'tools' for testing the presumed causality or for rigorously evaluating the reasons for lack of program success ..."". SDM and allied systems thinking approaches have the tools --- let's push them into these environments where at least the professionals talk about causality. My regret is that IThink v.1.0 did not appear until 1988 --- 3 years ealier there would have been a good chance to have had it mandated for every Australian federal policy development and program evaluation unit.
Regarding Jean-Jacques' concern at the complexity of modelling, I have found it relatively easy to transform a Government program Logic Model
(qualitative) into an SD model and use it to highlight the dynamic effects that are not shown in the inevitable spreadsheet models that were used to justify the program. Talking of complexity, the size of the SD model is typically 1/10 the size of the spreadsheet. I suggest the spreadsheet model is not only more complex, it is typically 'wrong'.
Keith Linard
134 Gisborne Road
Bacchus Marsh
Vic 3340
Posted by ""Keith Linard"" <klin4960@bigpond.net.au>
posting date Thu, 5 Jan 2006 21:23:26 +1000
-
Bill Harris bill_harris facilita
- Junior Member
- Posts: 19
- Joined: Fri Mar 29, 2002 3:39 am
The Logical Framework and SD
Posted by Bill Harris <bill_harris@facilitatedsystems.com>
""Jean-Jacques Laublé jean-jacques.lauble wanadoo.fr"" <system-dynamics@VENSIM.COM> writes:
>> A second idea about complexity it that one of the main idea behind SD
>> is that it simplifies reality. Jay Forrester spoke about Sd being
^^^^^^^^^^^^^
>> Spartan and that maybe people may doubt about the capacity of a
>> Spartan method to be effective. I think that instead of insisting
Jean-Jacques,
Actually, I think that was Jay Forrest. The names are indeed confusing.
You've got some interesting ideas; unfortunately, I have little time to think or write about them tonight -- maybe soon.
Bill
Posted by Bill Harris <bill_harris@facilitatedsystems.com>
posting date Wed, 04 Jan 2006 22:38:02 -0800
""Jean-Jacques Laublé jean-jacques.lauble wanadoo.fr"" <system-dynamics@VENSIM.COM> writes:
>> A second idea about complexity it that one of the main idea behind SD
>> is that it simplifies reality. Jay Forrester spoke about Sd being
^^^^^^^^^^^^^
>> Spartan and that maybe people may doubt about the capacity of a
>> Spartan method to be effective. I think that instead of insisting
Jean-Jacques,
Actually, I think that was Jay Forrest. The names are indeed confusing.
You've got some interesting ideas; unfortunately, I have little time to think or write about them tonight -- maybe soon.
Bill
Posted by Bill Harris <bill_harris@facilitatedsystems.com>
posting date Wed, 04 Jan 2006 22:38:02 -0800
-
Jean-Jacques Laublé jean-jacques
- Senior Member
- Posts: 68
- Joined: Fri Mar 29, 2002 3:39 am
The Logical Framework and SD
Posted by Jean-Jacques Laublé <jean-jacques.lauble@wanadoo.fr> Hi Bill
I was effectively wrong mentioning Jay Forrester instead of Jay Forrest.
I hope they will excuse me for that mistake.
Having never seen any Sders in my life I may easily mix their names.
I reread the post from Jay Forrest as I found it highly interesting and I was surprised that one of the first cause of the difficulty for SD to progress did not generate more posts as responses.
I think that the people not believing a Spartan model to be able to represent complex problems have two choices: not doing anything or doing something but building overly complicated models missing the simplicity necessary to use efficiently a model and facing later on high disillusions.
I did the same error the first years I worked with SD.
Having tried to apply many policies that cost a lot of money, time and effort and not having any results in the last 20 years, I thought the reason of my failure was necessary due to the complexity of my problem and I had to find a tool to deal with this complexity and accordingly I built complex models to represent complex situations, until after a very long time I realized that I could build much more useful models much more easily to represent a view of reality that was simple because the overall reality was in effect simple.
So I am now convinced that my problems are not complex, and the apparent complexity is the result of a hierarchy of simple piled up layers and to understand the whole it is necessary to start with the one standing on top and eventually going down if necessary.
So I think that the real challenge is to make people understand this and bring proof of it.
Regards.
J.J. Laublé Allocar
Strasbourg France
Posted by Jean-Jacques Laublé <jean-jacques.lauble@wanadoo.fr> posting date Fri, 6 Jan 2006 11:34:58 +0100
I was effectively wrong mentioning Jay Forrester instead of Jay Forrest.
I hope they will excuse me for that mistake.
Having never seen any Sders in my life I may easily mix their names.
I reread the post from Jay Forrest as I found it highly interesting and I was surprised that one of the first cause of the difficulty for SD to progress did not generate more posts as responses.
I think that the people not believing a Spartan model to be able to represent complex problems have two choices: not doing anything or doing something but building overly complicated models missing the simplicity necessary to use efficiently a model and facing later on high disillusions.
I did the same error the first years I worked with SD.
Having tried to apply many policies that cost a lot of money, time and effort and not having any results in the last 20 years, I thought the reason of my failure was necessary due to the complexity of my problem and I had to find a tool to deal with this complexity and accordingly I built complex models to represent complex situations, until after a very long time I realized that I could build much more useful models much more easily to represent a view of reality that was simple because the overall reality was in effect simple.
So I am now convinced that my problems are not complex, and the apparent complexity is the result of a hierarchy of simple piled up layers and to understand the whole it is necessary to start with the one standing on top and eventually going down if necessary.
So I think that the real challenge is to make people understand this and bring proof of it.
Regards.
J.J. Laublé Allocar
Strasbourg France
Posted by Jean-Jacques Laublé <jean-jacques.lauble@wanadoo.fr> posting date Fri, 6 Jan 2006 11:34:58 +0100