Hi
I wanted to study the world models that have been translated into Vensim, and found that both models from The 92 or 93 and 2003 when you run them, have lots of out of bounds lookup warnings.
The model from the Seventies works all right.
It does seem strange that the constants value drive the lookup tables out of range?
The equations have no extrapolate function.
Or are the constants value not plausible and one has to enter plausible values?
Regards.
JJ
World model
-
bob@vensim.com
- Senior Member
- Posts: 1107
- Joined: Wed Mar 12, 2003 2:46 pm
Hi JJ,
That is an interesting observation. The original World 3 model was written in Dynamo which gives warning messages whenever a table bound was exceeded. The later version were written in Stella which I do not think does this.
Lookups hold the last value constant unless an explicit extrapolate is indicated and if you look at the table overruns in the base run this seems to be consistent with the shape of the tables - or the tables are just flat placeholders.
Perhaps someone has looked into this more deeply and can comment with more insight.
That is an interesting observation. The original World 3 model was written in Dynamo which gives warning messages whenever a table bound was exceeded. The later version were written in Stella which I do not think does this.
Lookups hold the last value constant unless an explicit extrapolate is indicated and if you look at the table overruns in the base run this seems to be consistent with the shape of the tables - or the tables are just flat placeholders.
Perhaps someone has looked into this more deeply and can comment with more insight.
World model
Hi Bob
Effectively, when the lookup tables are out of bounds, it takes the bound of the lookup as value if there is no extrapolate.
All the cin files have the same particularities.
This does not look very professional for me.
The programs look too very difficult to understand, several 1000 loops!
no reference mode data for the 20th century and no way to
verify the realism of the model, but to make it work with different set of values.
I wonder if these models would stand a severe check with reality check?
Regards.
JJ
Effectively, when the lookup tables are out of bounds, it takes the bound of the lookup as value if there is no extrapolate.
All the cin files have the same particularities.
This does not look very professional for me.
The programs look too very difficult to understand, several 1000 loops!
no reference mode data for the 20th century and no way to
verify the realism of the model, but to make it work with different set of values.
I wonder if these models would stand a severe check with reality check?
Regards.
JJ
-
bob@vensim.com
- Senior Member
- Posts: 1107
- Joined: Wed Mar 12, 2003 2:46 pm
Hi JJ,
The world3 model is, in fact, probably the most thoroughly documented computer model in existence. In addition to the Limits to Growth books (three editions 1972, 1992 and 2003) the book "Dynamics of Growth in a Finite World" from 1974 has complete documentation. Another book "Toward Global Equilibrium" from 1974 is also relevant.
Of course, documenting a model in detail does not make that model good. Indeed there are some shortcomings in the World3 model that arise because of the interdependencies of the different nonlinearities. The one of these that comes to mind is the effect of life excpectency normal - which causes really odd behavior at about 70 - though the base values is 28 so it does not really mean what it is called anyway. I don't know of any of the shortcomings, however, that invalidate the basic lessons of the model.
As to simplicity - the world model from world dynamics (world.mdl) is vastly simpler, and still very rich. Whether that is enough really depends on purpose. The drive to adding in more detail to world3 was likely from criticisms of the simpler model. Still, by most standards world3 is a very aggregate model so it is much less complex than one would expect given the magnitude of the issues it addresses. Real life, unfortunately, does have thousands of feedback loops.
The world3 model is, in fact, probably the most thoroughly documented computer model in existence. In addition to the Limits to Growth books (three editions 1972, 1992 and 2003) the book "Dynamics of Growth in a Finite World" from 1974 has complete documentation. Another book "Toward Global Equilibrium" from 1974 is also relevant.
Of course, documenting a model in detail does not make that model good. Indeed there are some shortcomings in the World3 model that arise because of the interdependencies of the different nonlinearities. The one of these that comes to mind is the effect of life excpectency normal - which causes really odd behavior at about 70 - though the base values is 28 so it does not really mean what it is called anyway. I don't know of any of the shortcomings, however, that invalidate the basic lessons of the model.
As to simplicity - the world model from world dynamics (world.mdl) is vastly simpler, and still very rich. Whether that is enough really depends on purpose. The drive to adding in more detail to world3 was likely from criticisms of the simpler model. Still, by most standards world3 is a very aggregate model so it is much less complex than one would expect given the magnitude of the issues it addresses. Real life, unfortunately, does have thousands of feedback loops.
Hi JJ,
Bob pointed out this thread to me. Based on a quick look at the 03 version, most of the out-of-bounds messages appear to be of little consequence, because they occur in places where the table function is already at a logical extreme or 0 slope anyway, so the default behavior is appropriate. The technology multipliers and the assimilation half life table are good examples of this.
A few, such as jobs per hectare and jobs per industrial capital unit, do appear to clip the behavior, though this occurs at low levels of economic activity rather than at high levels where it would clearly have an important effect on the overshoot behavior. Nevertheless, as a matter of principal, they should be fixed.
I think the jobs lookups are symptomatic of the weak economic structure in the model. As Bob points out, this does not disprove the basic lessons of the model, but it opens the door for skepticism. Unfortunately, most alternative models with better representations of phenomena like capital-labor substitution also include exogenous technology specifications that dominate the behavior.
The fact that the model has thousands of feedback loops from ~400 equations should be regarded as a strength rather than a weakness, though it certainly complicates analysis. Many models with thousands of equations can muster at most a few loops; the high level of detail yields a false sense of comfort in the results, which seemingly account for "everything" while leaving out key dynamics. A quick way to reduce the feedback complexity is to run the model in Synthesim mode, and override variables with constants, which effectively eliminates any loops passing through the target variable.
I think the point of World3 could be made just as well in a model of half the size and greater internal consistency. However, that would make it more difficult to relate the model to particular real world phenomena (e.g. persistent pollution). It's important to remember that in 1973 the very idea of limits was heretical, so establishing some empirical basis for even considering the idea was important. Also, one must consider the competition - economic models which even 20 years later failed to conserve physical flows (e.g. carbon) or to present a plausible model of behavior (agents who don't know the future).
Tom
Bob pointed out this thread to me. Based on a quick look at the 03 version, most of the out-of-bounds messages appear to be of little consequence, because they occur in places where the table function is already at a logical extreme or 0 slope anyway, so the default behavior is appropriate. The technology multipliers and the assimilation half life table are good examples of this.
A few, such as jobs per hectare and jobs per industrial capital unit, do appear to clip the behavior, though this occurs at low levels of economic activity rather than at high levels where it would clearly have an important effect on the overshoot behavior. Nevertheless, as a matter of principal, they should be fixed.
I think the jobs lookups are symptomatic of the weak economic structure in the model. As Bob points out, this does not disprove the basic lessons of the model, but it opens the door for skepticism. Unfortunately, most alternative models with better representations of phenomena like capital-labor substitution also include exogenous technology specifications that dominate the behavior.
The fact that the model has thousands of feedback loops from ~400 equations should be regarded as a strength rather than a weakness, though it certainly complicates analysis. Many models with thousands of equations can muster at most a few loops; the high level of detail yields a false sense of comfort in the results, which seemingly account for "everything" while leaving out key dynamics. A quick way to reduce the feedback complexity is to run the model in Synthesim mode, and override variables with constants, which effectively eliminates any loops passing through the target variable.
I think the point of World3 could be made just as well in a model of half the size and greater internal consistency. However, that would make it more difficult to relate the model to particular real world phenomena (e.g. persistent pollution). It's important to remember that in 1973 the very idea of limits was heretical, so establishing some empirical basis for even considering the idea was important. Also, one must consider the competition - economic models which even 20 years later failed to conserve physical flows (e.g. carbon) or to present a plausible model of behavior (agents who don't know the future).
Tom
World models
Hi Bob and Tom
Thank you for your answers.
In fact, I did not understand why in the year 1940 in the 2003 model, deaths were suddenly dropping 22.5 % in one year and staying at this level the next years.
The model introduces a discontinuity at the year 40, eventually coming from the penicillin (?)
that increases suddenly the life expectation from 34.63 to 41.68 for health reasons in one year time, which is impossible for the whole world population.
Maybe this discontinuity does not change the whole results of the model, but why go to this level of detail when these details are seemingly wrong and very wrong?
Making the mortality depend on life expectancy is rather strange too.
I think it is life expectancy that depends from the current mortality and the expected future
mortality.
This is the danger of quantitative models. If you pretend to work with numbers,
you need to assume it.
To my opinion all these models are far too complicated and loose all their power of influence by adding too many details.
I can hardly imagine a politic being influenced by such complicate models and the paradox is that these big models being not able to influence political decisions, the politician revert to
highly simplistic personal evaluations of the situation.
A very simple model even very approximate would always be better than nothing.
I recognize that the out of bounds of lookups is not important. It would have been just better to add an extrapolate function to avoid these messages.
I regret that the most useful feature of Vensim, reality checks is not used here to test the overall coherence of the models.
I should anyway read the books about these models. They will eventually answer the questions: Are the results from the past have been compared to real data? Are these data available. Has the model been run regularly and its prediction compared to the reality, to improve the model?
About the number of loops, I still think that the high number of loops is a weakness, because I
think that the first quality of a model is its usability that can be measured by the fact that it is
effectively used, and that the usability is very much dependant on the visibility of the model.
The more one goes to details, the less visible and the more chance to introduce bugs in a model and the more chance that the model is never used.
I have made many complicate models that once I had to use them, discovered that they were generating more work using them than adding value.
Regards.
JJ
Thank you for your answers.
In fact, I did not understand why in the year 1940 in the 2003 model, deaths were suddenly dropping 22.5 % in one year and staying at this level the next years.
The model introduces a discontinuity at the year 40, eventually coming from the penicillin (?)
that increases suddenly the life expectation from 34.63 to 41.68 for health reasons in one year time, which is impossible for the whole world population.
Maybe this discontinuity does not change the whole results of the model, but why go to this level of detail when these details are seemingly wrong and very wrong?
Making the mortality depend on life expectancy is rather strange too.
I think it is life expectancy that depends from the current mortality and the expected future
mortality.
This is the danger of quantitative models. If you pretend to work with numbers,
you need to assume it.
To my opinion all these models are far too complicated and loose all their power of influence by adding too many details.
I can hardly imagine a politic being influenced by such complicate models and the paradox is that these big models being not able to influence political decisions, the politician revert to
highly simplistic personal evaluations of the situation.
A very simple model even very approximate would always be better than nothing.
I recognize that the out of bounds of lookups is not important. It would have been just better to add an extrapolate function to avoid these messages.
I regret that the most useful feature of Vensim, reality checks is not used here to test the overall coherence of the models.
I should anyway read the books about these models. They will eventually answer the questions: Are the results from the past have been compared to real data? Are these data available. Has the model been run regularly and its prediction compared to the reality, to improve the model?
About the number of loops, I still think that the high number of loops is a weakness, because I
think that the first quality of a model is its usability that can be measured by the fact that it is
effectively used, and that the usability is very much dependant on the visibility of the model.
The more one goes to details, the less visible and the more chance to introduce bugs in a model and the more chance that the model is never used.
I have made many complicate models that once I had to use them, discovered that they were generating more work using them than adding value.
Regards.
JJ
-
bob@vensim.com
- Senior Member
- Posts: 1107
- Joined: Wed Mar 12, 2003 2:46 pm
Hi JJ,
The penicillin thing has always bothered me as well.
To your larger question of how simple a model has to be in order to have an effect the results in my experience are more mixed. On the one hand, it is true that people can't understand a complicated model. On the other hand, it is also true that people tend not to believe a model that does not have a fair bit of detail in it.
A possible solution is to: 1) Build a model that is rich enough to capture peoples attention, give them the detail they need, and be consistent with available measurements; then 2) Create a simple model(s) that can be understood by everyone and contains the important lessons from the richer model. But that is time consuming and not always possible. Human nature seems to be opposed to sensible approaches for lots of issues.
As far as Reality Check on the World3 model. Yes it would fail a lot but it would do much better than probably every other model that has been used to inform public policy in these areas. Food for thought.
The penicillin thing has always bothered me as well.
To your larger question of how simple a model has to be in order to have an effect the results in my experience are more mixed. On the one hand, it is true that people can't understand a complicated model. On the other hand, it is also true that people tend not to believe a model that does not have a fair bit of detail in it.
A possible solution is to: 1) Build a model that is rich enough to capture peoples attention, give them the detail they need, and be consistent with available measurements; then 2) Create a simple model(s) that can be understood by everyone and contains the important lessons from the richer model. But that is time consuming and not always possible. Human nature seems to be opposed to sensible approaches for lots of issues.
As far as Reality Check on the World3 model. Yes it would fail a lot but it would do much better than probably every other model that has been used to inform public policy in these areas. Food for thought.
"To my opinion all these models are far too complicated and loose all their power of influence by adding too many details.
I can hardly imagine a politic being influenced by such complicate models and the paradox is that these big models being not able to influence political decisions, the politician revert to highly simplistic personal evaluations of the situation. "
I generally agree that simpler would be better for many purposes. I think results vary greatly in terms of political influence though. Some extremely detailed models (DOE's NEMS for example) are widely used, because they generate baseline forecasts that everyone can refer to as a safe, common set of assumptions. Of course, those forecasts have tended to be badly wrong, so this is really propaganda more than use of models for insight.
Energy-economy models for integrated assessment of climate policy vary widely in complexity, from big general equilibrium models (thousands or millions of variables and no explicit causality due to equilibrium) to Nordhaus' DICE model (4 state variables). DICE has had a wider influence on thinking in the academic community because it is so much easier to replicate and extend. But I think policymakers are relatively unaware of it; they are somewhat avid consumers of the results of the bigger models, but as far as I can tell that's because they like the answers, not because they're getting any insight.
In the scientific debate over climate change, it is absolutely necessary to have a large, spatially explicit model to correctly represent some of the key phenomena (e.g. hurricanes) and the required level of detail has not even been achieved yet. However, the basic principles of global warming can be understood from an energy balance model that fits on a napkin. Yet many American policymakers seem to have adopted the view that they will only believe climate models when they can predict next year's weather, which is of course impossible. The situation is further muddied by professional skeptics who have promoted the idea that modeling is somehow a suspect activity divorced from science.
It seems to me that mainstream politicians have little interest in having their minds changed by a small, insightful model. They just shop around for models with output they like, which are often very complex because that makes them hard to criticize. The route to influence may be to change thinking among constituents, which probably is best accomplished with fairly simple models.
With respect to World3, it has been compared to data several times that I'm aware of. That process motivated some of the revisions to the model that accompanied the release of the updated books (Beyond the Limits etc.). The revisions were minor because the model's forecasts of well-measured things (GDP, population) were reasonable. In other areas (persistent pollution, resources, etc.) it's harder to compare but some attempts were made. Unfortunately the data collected at those times is not easily available, though the books present a lot of case history. I don't think data comparisons are especially persuasive, because at a global aggregate level they don't reveal a shift in behavior indicative of overshoot (though there are certainly regional examples). The reality check approach is perhaps more productive.
I definitely recommend reading the books.
Tom
I can hardly imagine a politic being influenced by such complicate models and the paradox is that these big models being not able to influence political decisions, the politician revert to highly simplistic personal evaluations of the situation. "
I generally agree that simpler would be better for many purposes. I think results vary greatly in terms of political influence though. Some extremely detailed models (DOE's NEMS for example) are widely used, because they generate baseline forecasts that everyone can refer to as a safe, common set of assumptions. Of course, those forecasts have tended to be badly wrong, so this is really propaganda more than use of models for insight.
Energy-economy models for integrated assessment of climate policy vary widely in complexity, from big general equilibrium models (thousands or millions of variables and no explicit causality due to equilibrium) to Nordhaus' DICE model (4 state variables). DICE has had a wider influence on thinking in the academic community because it is so much easier to replicate and extend. But I think policymakers are relatively unaware of it; they are somewhat avid consumers of the results of the bigger models, but as far as I can tell that's because they like the answers, not because they're getting any insight.
In the scientific debate over climate change, it is absolutely necessary to have a large, spatially explicit model to correctly represent some of the key phenomena (e.g. hurricanes) and the required level of detail has not even been achieved yet. However, the basic principles of global warming can be understood from an energy balance model that fits on a napkin. Yet many American policymakers seem to have adopted the view that they will only believe climate models when they can predict next year's weather, which is of course impossible. The situation is further muddied by professional skeptics who have promoted the idea that modeling is somehow a suspect activity divorced from science.
It seems to me that mainstream politicians have little interest in having their minds changed by a small, insightful model. They just shop around for models with output they like, which are often very complex because that makes them hard to criticize. The route to influence may be to change thinking among constituents, which probably is best accomplished with fairly simple models.
With respect to World3, it has been compared to data several times that I'm aware of. That process motivated some of the revisions to the model that accompanied the release of the updated books (Beyond the Limits etc.). The revisions were minor because the model's forecasts of well-measured things (GDP, population) were reasonable. In other areas (persistent pollution, resources, etc.) it's harder to compare but some attempts were made. Unfortunately the data collected at those times is not easily available, though the books present a lot of case history. I don't think data comparisons are especially persuasive, because at a global aggregate level they don't reveal a shift in behavior indicative of overshoot (though there are certainly regional examples). The reality check approach is perhaps more productive.
I definitely recommend reading the books.
Tom
world model
Hi Bob
About the sudden death decrease I think that the life expectency should be modelled as a level, as it is a state. It would then behave more smoothly. The problem would be to value it.
About the just medium between a too highly complex and a too simplistic approach, this is the basic of efficiency.
I prefer to err on the simplistic side, because it is less expensive and you can always add more stuff later on.
About the two versions of a model, that should be generalized, even if it is time consuming.
This is what I do, when a model becomes obscure, I try to make it simpler.
Even being a mathematician and because I am a business man too, I am very sceptical about results that I do not fully understand, especially if I have to apply them.
Regards.
JJ
About the sudden death decrease I think that the life expectency should be modelled as a level, as it is a state. It would then behave more smoothly. The problem would be to value it.
About the just medium between a too highly complex and a too simplistic approach, this is the basic of efficiency.
I prefer to err on the simplistic side, because it is less expensive and you can always add more stuff later on.
About the two versions of a model, that should be generalized, even if it is time consuming.
This is what I do, when a model becomes obscure, I try to make it simpler.
Even being a mathematician and because I am a business man too, I am very sceptical about results that I do not fully understand, especially if I have to apply them.
Regards.
JJ
world model
Hi Tom
I know some politicians. They are more bothered by their re-election than by anything else.
They prefer as you say big models because it can help them influence other people while being sure
that these people will not been able to really understand them or see that they are wrong.
They cannot do that with simpler model easily understood by anybody.
About climate models like hurricanes, I understand the need of complexity, but it is a physical
complexity with physical laws. Then the models that predict hurricanes and how they behave will exist
one day. A hurricane has no freedom and is predictable.
I think that the physical part of the world model is probably right, it is the part that depend on human behaviour
that will always be unpredictable especially over a so long period.
I take the time to study models made by others. It is easier to criticize models that I have not made and it helps me
criticize my own models afterwards. I have nobody to criticize my models and in my business I am obliged to demonstrate the validity of my policy by very simple ways understandable by anybody even if it has been discovered by more complex methods like modelling. So I do not show my models and have no feed back about them.
So I will not be anyhow able to have a serious advice about the world model being not a specialist of the subject.
But I will try to find the time to read the books.
Regards.
JJ
I know some politicians. They are more bothered by their re-election than by anything else.
They prefer as you say big models because it can help them influence other people while being sure
that these people will not been able to really understand them or see that they are wrong.
They cannot do that with simpler model easily understood by anybody.
About climate models like hurricanes, I understand the need of complexity, but it is a physical
complexity with physical laws. Then the models that predict hurricanes and how they behave will exist
one day. A hurricane has no freedom and is predictable.
I think that the physical part of the world model is probably right, it is the part that depend on human behaviour
that will always be unpredictable especially over a so long period.
I take the time to study models made by others. It is easier to criticize models that I have not made and it helps me
criticize my own models afterwards. I have nobody to criticize my models and in my business I am obliged to demonstrate the validity of my policy by very simple ways understandable by anybody even if it has been discovered by more complex methods like modelling. So I do not show my models and have no feed back about them.
So I will not be anyhow able to have a serious advice about the world model being not a specialist of the subject.
But I will try to find the time to read the books.
Regards.
JJ