Posted by ""Ben Heslop"" <u3235557@anu.edu.au>
Hi all,
I am trying to model a human system (knowledge transfer) and have interviewed a number of practitioners.
My problem is using the right methodology, one recognised by SD that will allow me to extract commonly-perceived cause-effect linkages from interview data.
Initally, I would like to be able to build a conceptual model with dependencies between variables shown. At a later date, the intention is to turn this into a stock/flow model. I cannot go straight to the second step as the dependencies are difficult to isolate.
I have invented a methodology, whereby all concepts are listed on the X and the Y axis of a table, and each mention *by the interviewee* of a cause-effect linkage is noted. While this seems commonsensical, I don't really know whether it would produce a result suitable for publication within SD.
Does anyone know of an acknowledged methodology for extracting cause-effect information from interview data?
Thanks,
Ben Heslop (BEng, Interdisciplinary Systems)
PhD Candidate (Political Systems)
Research School of Physical Sciences and Engineering
Australian National University
Posted by ""Ben Heslop"" <u3235557@anu.edu.au>
posting date Fri, 19 Aug 2005 15:29:11 +1000
Cause-Effect Relationships from Interviews
-
Ben Heslop u3235557 anu.edu.au
- Junior Member
- Posts: 3
- Joined: Fri Mar 29, 2002 3:39 am
-
Kim Warren Kim strategydynamics.
- Junior Member
- Posts: 16
- Joined: Fri Mar 29, 2002 3:39 am
Cause-Effect Relationships from Interviews
Posted by ""Kim Warren"" <Kim@strategydynamics.com>
Relying on asking people what they *think* causes what can be risky, especially when the issue they are expressing a view about is not one in which they are personally directly involved. It does not matter one jot, for example, why I think people have stopped keeping information to themselves and started sharing it with others - the only people who's view on this question matters are those who have actually changed their behaviour.
Also not sure how you would decide what factors go on your list, whether those factors are properly defined [qualitative descriptions could mean very different things to different people] and whether that list is 'MECE' [mutually exclusive and collectively exhaustive]?.
An alternative to the kind of causal matrix you imply is to start from the question you want to answer [say, how to increase the fraction of knowledge held by individuals that is available and used by others], and push out from there. In this case, you might start with a stock of 'knowledge held by individuals', and ask individuals what has/might cause them to move that stock into 'made available to others', then ask others what might cause them to move that information into a state of 'being used by others'. That way you focus from the start on the key stocks [knowledge, people ...] and question what drives the flows of those stocks between their possible alternative states. When people don't know, I think you may be able to use statistical methods to test the influence of plausible independent variables - but the key
*dependent* variables will include the critical flows.
.. just some thoughts - others in the community know much better than this about eliciting information from people and using it in models, and I am sure there are folk out there who have worked on knowledge-sharing.
Kim Warren
Posted by ""Kim Warren"" <Kim@strategydynamics.com>
posting date Fri, 19 Aug 2005 15:48:58 +0100
Relying on asking people what they *think* causes what can be risky, especially when the issue they are expressing a view about is not one in which they are personally directly involved. It does not matter one jot, for example, why I think people have stopped keeping information to themselves and started sharing it with others - the only people who's view on this question matters are those who have actually changed their behaviour.
Also not sure how you would decide what factors go on your list, whether those factors are properly defined [qualitative descriptions could mean very different things to different people] and whether that list is 'MECE' [mutually exclusive and collectively exhaustive]?.
An alternative to the kind of causal matrix you imply is to start from the question you want to answer [say, how to increase the fraction of knowledge held by individuals that is available and used by others], and push out from there. In this case, you might start with a stock of 'knowledge held by individuals', and ask individuals what has/might cause them to move that stock into 'made available to others', then ask others what might cause them to move that information into a state of 'being used by others'. That way you focus from the start on the key stocks [knowledge, people ...] and question what drives the flows of those stocks between their possible alternative states. When people don't know, I think you may be able to use statistical methods to test the influence of plausible independent variables - but the key
*dependent* variables will include the critical flows.
.. just some thoughts - others in the community know much better than this about eliciting information from people and using it in models, and I am sure there are folk out there who have worked on knowledge-sharing.
Kim Warren
Posted by ""Kim Warren"" <Kim@strategydynamics.com>
posting date Fri, 19 Aug 2005 15:48:58 +0100
-
rgd6 cornell.edu
- Junior Member
- Posts: 9
- Joined: Fri Mar 29, 2002 3:39 am
Cause-Effect Relationships from Interviews
Posted by <rgd6@cornell.edu>
Posted on behalf of Carol Colfer:
In response to the interchange between Kim and Ben:
The first idea that came to mind, based on Ben's matrix idea, was the Galileo methodology (which is designed to produce cognitive maps). As designed it simply shows cognitive distance among concepts, but it does involve pairing concepts in a way similar to what Ben proposes. I wonder if it might be alterable to fit the cause-effect idea. The guy who'd know is Joe Woelfel jwoelfel@msn.com
Another idea came to mind, which comes from more qualitative approaches. Pete Vayda ('Andrew P. Vayda', retired from Rutgers, in Human Ecology) and others have used an approach that begins with an action and traces causal links outward from that action---through observation and discussion with people involved. Pete has (or at least had) a phobia about the term system, but from my point of view, he's looking at systems while doing these analyses. It worked very well in contexts where the researcher is in regular contact with the folks involved in the action (like participant observation contexts). Converting these conclusions to numbers is a difficulty, of course, as with many qualitative conclusions...
I think this statement of Kim's overstates the problem:
""It does not matter one jot, for example, why I think people have stopped keeping information to themselves and started sharing it with others - the only people who's view on this question matters are those who have actually changed their behaviour.""
People who have not changed their behaviour also have important views that need to be addressed in a system of knowledge transfer---though I agree that Kim's and my views are fairly irrelevant to the system under discussion.
Carol Colfer, Anthropologist (c.colfer@cgiar.org)
Center for International Forestry Research
Bogor, Indonesia
Posted by <rgd6@cornell.edu>
posting date Mon, 22 Aug 2005 10:48:58 +0700
Posted on behalf of Carol Colfer:
In response to the interchange between Kim and Ben:
The first idea that came to mind, based on Ben's matrix idea, was the Galileo methodology (which is designed to produce cognitive maps). As designed it simply shows cognitive distance among concepts, but it does involve pairing concepts in a way similar to what Ben proposes. I wonder if it might be alterable to fit the cause-effect idea. The guy who'd know is Joe Woelfel jwoelfel@msn.com
Another idea came to mind, which comes from more qualitative approaches. Pete Vayda ('Andrew P. Vayda', retired from Rutgers, in Human Ecology) and others have used an approach that begins with an action and traces causal links outward from that action---through observation and discussion with people involved. Pete has (or at least had) a phobia about the term system, but from my point of view, he's looking at systems while doing these analyses. It worked very well in contexts where the researcher is in regular contact with the folks involved in the action (like participant observation contexts). Converting these conclusions to numbers is a difficulty, of course, as with many qualitative conclusions...
I think this statement of Kim's overstates the problem:
""It does not matter one jot, for example, why I think people have stopped keeping information to themselves and started sharing it with others - the only people who's view on this question matters are those who have actually changed their behaviour.""
People who have not changed their behaviour also have important views that need to be addressed in a system of knowledge transfer---though I agree that Kim's and my views are fairly irrelevant to the system under discussion.
Carol Colfer, Anthropologist (c.colfer@cgiar.org)
Center for International Forestry Research
Bogor, Indonesia
Posted by <rgd6@cornell.edu>
posting date Mon, 22 Aug 2005 10:48:58 +0700
-
Kim Warren Kim strategydynamics.
- Junior Member
- Posts: 16
- Joined: Fri Mar 29, 2002 3:39 am
Cause-Effect Relationships from Interviews
Posted by ""Kim Warren"" <Kim@strategydynamics.com>
Hi Carol - thanks for the important clarification of my response to Ben. As stated, I got it wrong .. should have said '.. those people who have changed their behaviour, or chosen not to, or might change in future.' . i.e. the population of people personally participating in the situation, rather than outside observers.
Kim
Posted by ""Kim Warren"" <Kim@strategydynamics.com>
posting date Mon, 22 Aug 2005 15:24:32 +0100
Hi Carol - thanks for the important clarification of my response to Ben. As stated, I got it wrong .. should have said '.. those people who have changed their behaviour, or chosen not to, or might change in future.' . i.e. the population of people personally participating in the situation, rather than outside observers.
Kim
Posted by ""Kim Warren"" <Kim@strategydynamics.com>
posting date Mon, 22 Aug 2005 15:24:32 +0100
-
Jonah jfogel utk.edu
- Junior Member
- Posts: 3
- Joined: Fri Mar 29, 2002 3:39 am
Cause-Effect Relationships from Interviews
Posted by Jonah <jfogel@utk.edu>
This question is of particular interest to me and my PhD work. So, I'd be interested to hear more about the Galileo methodology but I can also offer a suggestion.
I'm using a paper by Markoczy and Goldberg (1995) to conduct a pre- and
post- test quasi-experiment with participants. It uses a matrix similar to the one described to compute distance scores for each participant. Using these I should be able to tell if my treatment (Group Model
Building) was effective at creating a shared mental model of the problem situation. This paper outlines both elicitation and analysis procedures.
Markóczy, Lívia, and Jeff Goldberg. (1995) A Method for Eliciting and Comparing Causal Maps. In Journal of Management: Elsevier Science Publishing Company, Inc.
Another more qualitative approach would be the Oval Mapping Technique as described in the book Making Strategy (1998). This technique uses a focus question to elicit means-ends relationships from an individual or group, clusters the factors, and organizes each concept into a means ends relationship. The result is a distinct list of concepts organized into named clusters. While this technique isn't a fluid as others it offers a strength in its ability to elicit and organize information quickly. Given a large complex problem you could use the cluster names to run a matrix analysis on each participant or use the entire concept list if there are fewer concepts involved.
Eden, C, Ackermann F. (1998) ""Making Strategy: the Journey of Strategic Management"". Sage, London.
What other knowledge elicitation techniques are being used in SD? I'd
be willing to compile sources into a list for everyone to share.
be well,
Jonah Fogel
University of Tennessee, USA
Posted by Jonah <jfogel@utk.edu>
posting date Mon, 22 Aug 2005 09:07:36 -0400
This question is of particular interest to me and my PhD work. So, I'd be interested to hear more about the Galileo methodology but I can also offer a suggestion.
I'm using a paper by Markoczy and Goldberg (1995) to conduct a pre- and
post- test quasi-experiment with participants. It uses a matrix similar to the one described to compute distance scores for each participant. Using these I should be able to tell if my treatment (Group Model
Building) was effective at creating a shared mental model of the problem situation. This paper outlines both elicitation and analysis procedures.
Markóczy, Lívia, and Jeff Goldberg. (1995) A Method for Eliciting and Comparing Causal Maps. In Journal of Management: Elsevier Science Publishing Company, Inc.
Another more qualitative approach would be the Oval Mapping Technique as described in the book Making Strategy (1998). This technique uses a focus question to elicit means-ends relationships from an individual or group, clusters the factors, and organizes each concept into a means ends relationship. The result is a distinct list of concepts organized into named clusters. While this technique isn't a fluid as others it offers a strength in its ability to elicit and organize information quickly. Given a large complex problem you could use the cluster names to run a matrix analysis on each participant or use the entire concept list if there are fewer concepts involved.
Eden, C, Ackermann F. (1998) ""Making Strategy: the Journey of Strategic Management"". Sage, London.
What other knowledge elicitation techniques are being used in SD? I'd
be willing to compile sources into a list for everyone to share.
be well,
Jonah Fogel
University of Tennessee, USA
Posted by Jonah <jfogel@utk.edu>
posting date Mon, 22 Aug 2005 09:07:36 -0400
-
rgd6 cornell.edu
- Junior Member
- Posts: 9
- Joined: Fri Mar 29, 2002 3:39 am
Cause-Effect Relationships from Interviews
Posted by <rgd6@cornell.edu>
The following is a response from Joe Woefel regarding the ""Galileo"" software for analyzing causal relationships obtained in interviews. Note: The web site for these software packages is: http://www.galileoco.com/index.htm
-----Original Message-----
From: Colfer, Carol (CIFOR) [mailto:c.colfer@cgiar.org]
Sent: Wednesday, August 24, 2005 1:16 PM
To: Richard G. Dudley
Cc: Ani Kartikasari
Subject: FW: REPLY Cause-Effect Relationships from Interviews (SD5462)
Joe says it's fine for people to contact him (here's the best email: jwoefel@galileoco.com
Here's what he says:
""Galileo and Catpac are the most precise technologies I know for assessing causal relationship, and I'll be happy to help your people work out how to do it.
You can give my email address to anyone you like. It's fastest and most convenient if you use jwoefel@galileoco.com. I'm here to help!
Joe
Posted by <rgd6@cornell.edu>
posting date Wed, 24 Aug 2005 13:49:50 +0700
The following is a response from Joe Woefel regarding the ""Galileo"" software for analyzing causal relationships obtained in interviews. Note: The web site for these software packages is: http://www.galileoco.com/index.htm
-----Original Message-----
From: Colfer, Carol (CIFOR) [mailto:c.colfer@cgiar.org]
Sent: Wednesday, August 24, 2005 1:16 PM
To: Richard G. Dudley
Cc: Ani Kartikasari
Subject: FW: REPLY Cause-Effect Relationships from Interviews (SD5462)
Joe says it's fine for people to contact him (here's the best email: jwoefel@galileoco.com
Here's what he says:
""Galileo and Catpac are the most precise technologies I know for assessing causal relationship, and I'll be happy to help your people work out how to do it.
You can give my email address to anyone you like. It's fastest and most convenient if you use jwoefel@galileoco.com. I'm here to help!
Joe
Posted by <rgd6@cornell.edu>
posting date Wed, 24 Aug 2005 13:49:50 +0700
-
Ben Heslop u3235557 anu.edu.au
- Junior Member
- Posts: 3
- Joined: Fri Mar 29, 2002 3:39 am
Cause-Effect Relationships from Interviews
Posted by ""Ben Heslop"" <u3235557@anu.edu.au>
Sorry, clip not allowed by server. If anyone wants the spreadsheet i refer to in the email, please get in touch
Thanks very much for your replies everyone,
Carol, I am aware of cognitive mapping and actually use it as middle-step (though i didn't mention it...), where i build up a cognitive map from each interview. I then try to interpret the variables (specified by by the
interviewee) in terms of the list of 'MECE' (thanks Kim) concept list that have been drawn up from my recollections. From here, the matrix approach is used (below) to note connectivity between the (interpreted) MECE variables
First, I identified 'common sense' linkages TOP to LEFT (green squares signify positive, orange: negative) from my personal perceptions (using my 'biological computer
. I then set out to ask two questions;
a) what linkages ARE there that I have not nominated
and
b) are there any that are there that SHOULD NOT be
the purple box is the single additional linkage that the data has (thus
far)
introduced, and I am one quarter of the way through the data.
Government Support Bureaucracy Support Commercial Support Commercial Network Commercial Participation Commercial Opportunity Research Support Research Network Research Participation Research Opportunity Cross-Industry Interaction Industry-Academic Interaction Cross-disciplinary Interaction KTO Quantity KTO Quality KTO Leadership KTO Teamwork Policy Intelligence KT Infrastructure R&D Funding Programme Co-ordination Activity Success Knowledge Sharing Project Success Perceptions of System #2
Government Support
Bureaucracy Support
Commercial Support
Commercial Network
Commercial Participation
Commercial Opportunity
Research Support
Research Network
Research Participation
Research Opportunity
Cross-Industry Interaction
Industry-Academic Interaction
Cross-disciplinary Interaction
KTO Quantity
KTO Quality
KTO Leadership
KTO Teamwork
Policy Intelligence
KT Infrastructure
R&D Funding
Programme Co-ordination
Activity Success
Knowledge Sharing
Project Success
Perceptions of System
Note: In looking at 'knowledge transfer', please be aware that I do not look at *knowledge* as a potential stock in and of itself (because such a thing is so difficult to measure) but rather the 'projects' (read: outcomes) that result from knowledge being 'put to work' (in order that innovation results). So yes, I would agree that the outcome of knowledge is the overriding (and easier to measure) aspect, and therefore more appropriate for the purposes of modelling.
Thanks all, and I will investigate the other methodologies put forward.
Ben
Posted by ""Ben Heslop"" <u3235557@anu.edu.au>
posting date Wed, 24 Aug 2005 18:48:32 +1000
Sorry, clip not allowed by server. If anyone wants the spreadsheet i refer to in the email, please get in touch
Thanks very much for your replies everyone,
Carol, I am aware of cognitive mapping and actually use it as middle-step (though i didn't mention it...), where i build up a cognitive map from each interview. I then try to interpret the variables (specified by by the
interviewee) in terms of the list of 'MECE' (thanks Kim) concept list that have been drawn up from my recollections. From here, the matrix approach is used (below) to note connectivity between the (interpreted) MECE variables
First, I identified 'common sense' linkages TOP to LEFT (green squares signify positive, orange: negative) from my personal perceptions (using my 'biological computer
. I then set out to ask two questions;a) what linkages ARE there that I have not nominated
and
b) are there any that are there that SHOULD NOT be
the purple box is the single additional linkage that the data has (thus
far)
introduced, and I am one quarter of the way through the data.
Government Support Bureaucracy Support Commercial Support Commercial Network Commercial Participation Commercial Opportunity Research Support Research Network Research Participation Research Opportunity Cross-Industry Interaction Industry-Academic Interaction Cross-disciplinary Interaction KTO Quantity KTO Quality KTO Leadership KTO Teamwork Policy Intelligence KT Infrastructure R&D Funding Programme Co-ordination Activity Success Knowledge Sharing Project Success Perceptions of System #2
Government Support
Bureaucracy Support
Commercial Support
Commercial Network
Commercial Participation
Commercial Opportunity
Research Support
Research Network
Research Participation
Research Opportunity
Cross-Industry Interaction
Industry-Academic Interaction
Cross-disciplinary Interaction
KTO Quantity
KTO Quality
KTO Leadership
KTO Teamwork
Policy Intelligence
KT Infrastructure
R&D Funding
Programme Co-ordination
Activity Success
Knowledge Sharing
Project Success
Perceptions of System
Note: In looking at 'knowledge transfer', please be aware that I do not look at *knowledge* as a potential stock in and of itself (because such a thing is so difficult to measure) but rather the 'projects' (read: outcomes) that result from knowledge being 'put to work' (in order that innovation results). So yes, I would agree that the outcome of knowledge is the overriding (and easier to measure) aspect, and therefore more appropriate for the purposes of modelling.
Thanks all, and I will investigate the other methodologies put forward.
Ben
Posted by ""Ben Heslop"" <u3235557@anu.edu.au>
posting date Wed, 24 Aug 2005 18:48:32 +1000