Posted by Tom Fiddaman <
tom@ventanasystems.com>
David Corben is puzzled at the SD community's apparent affection for climate models. As far as I know the SD community has little direct involvement with physical models of climate, and is mainly concerned with the policy implications of the physics given uncertainty.
Nevertheless the community should be concerned with the content and quality of the physical models because the implications are important.
Unfortunately the suggested model tests are misapplied in this case.
The very long term dynamics of glacial cycles are not the primary yardstick for climate models, because they are nearly constant on time scales important to our current climate and economy. Once this straw-man test is set aside, it's evident that climate models do pass a lot of relevant tests, and rely not at all on event-based mentality.
Waiting for unattainably perfect models to manage a system with long delays is not a robust strategy.
Let's look at the suggested criteria point by point:
> 2) Adopt an appropriate time frame for the model
>
> Ice ages can last for up to a few million years and interglacials are
> measured in 10s of thousand of years. Milankovitch identified time
> constants
>
> An analogy here would be to try and understand business cycles by
> building a model that forecasted the economy for a couple of months.
Another analogy is closer to hand: I just flipped through my copy of Industrial Dynamics and found graphs showing 3 to 5 years of behavior. Many companies have been in business for centuries (
http://en.wikipedia.org/wiki/List_of_oldest_companies ).
Does this imply that ID is invalid or ""modeling the noise""? Not at all. Choosing a time frame equal to the longest time constant evident in a real system is a poor rule of thumb. Certainly there are plenty of situations where decision makers are thinking on too short a time scale to solve their problem, and modelers should push for a broader boundary and longer horizon (one could argue that this is the case for climate - politicians are thinking in years and need to be pushed towards centuries). However, that does not mean every model needs to worry about the expansion of the sun to a red giant.
Choosing an overly long time horizon can be dangerous because the usual difficulty in representation of a system is the short time constants, which can result in a ""stiff system"" that must be simulated with short time steps. Climate is such a system, because there are important processes requiring short time scales and small spatial scales for adequate resolution. They can't be spatially aggregated or assumed to be in equilibrium due to nonlinearity, chaotic behavior, and limited understanding. Fortunately, the processes governing climate on a human time scale are reasonably separable from those governing Milankovitch cycles.
Nevertheless there are models that simulate climate in exactly the manner suggested, collapsing short term weather (and sometimes even multicentury ocean circulation) to equilibrium and focusing on the long-term dynamics of ice sheet advance and retreat with million-year horizons. No model with a 100-year time step can tell you much about the dynamics of water vapor in the atmosphere (time constant=weeks), which is a crucial determinant of the response of temperature to CO2, but these deep-time paleoclimate models agree with the short-term models on the plausible range of sensitivity of climate to external forcings.
> 1) Endogenously Replicate the Observed Reference Mode
>
> Climate model cannot replicate an ice age followed by an interglacial;
> they
As above, ""climate model"" refers to a wide variety of models ranging from low-order globally aggregated energy balance models that fit on a napkin to physically explicit 3D spatial models (GCMs) requiring supercomputers. The latter generate most of the headlines and say the most about the sensitivity of climate to greenhouse gases, but they don't operate on the kind of time scales needed to make the above reference mode relevant - they are typically run for a few centuries. They do replicate current climate with considerable fidelity, and can also replicate conditions in the far past (where the major challenge is deciding what the forcings were) and on other planets.
The assertion that models drift to a stable frozen equilibrium is not generally true.
If models are missing an important feedback loop with a time constant of 50k years, that is interesting, frustrating, limiting, etc. but doesn't imply that the shorter-term dynamics are wrong. For that you need to confront the models with other reference modes:
- Can low-order energy balance models replicate observed surface temperatures
over the last ~1000 years? (Yes, but only if greenhouse gases are included)
- Same for GCMs? (same answer)
- Can GCMs replicate tropical temperature-water vapor interactions? (Yes, but
only if water vapor behavior is consistent with the standard range of
temperature sensitivity)
- Are weather forecasts better when greenhouse signals are included in model
boundary conditions? (Yes)
- Can GCMs replicate the cooling following a large volcanic eruption like
Pinatubo? (Yes)
- Can GCMs replicate the temperature trends in upper layers of the atmosphere?
(Yes, and they predicted rising tropospheric temperatures in the face of
data showing cooling - the data was wrong) etc.
What should be of greater concern than replicating glacial-interglacial cycles is the fact that models omit many faster feedback loops as well. Carbon cycle-temperature feedbacks, ice sheet collapse dynamics, methane hydrates, and a variety of other phenomena are absent in most models. There's good reason to think that not all these omissions will turn out to be benign.
> 3) Looking at a System at a Point in Time Can be Misleading (event
> based
> thinking)
>
> System dynamics cautions against this, but global warming is a classic
> example of this mindset. People look at the melting ice caps and see
> this as
There is no doubt that in politics and media, human influence on climate does get simplified to the event level, which is detrimental to good judgment. But that has nothing to do with the scientific understanding of climate.
Anthropogenic global warming was a theory more than 100 years ago - long before it could be observed. The data used to verify models is far richer than a few events like Katrina. It includes both long aggregate time series and shorter, spatially detailed series from from satellites delivering terabytes daily.
The geological record is comparatively thin (in part because paleoclimate research has been underfunded) but there's no doubt that there have been large fluctuations in the past. Some of the changes involve time constants so long that it's hard to imagine that we should worry about them (the rock weathering cycle, continental drift, orbital changes, emergence of plants).
Past variability should not be regarded as comforting. It suggests instabilities that could be triggered by comparatively small interference on our part. Even if a warmer, ice-free world is somehow better, the transient to get there could be rough for coastal inhabitants, agriculture, forests, etc.
The last rapid excursion in temperature and carbon (the PETM, 55m years ago) was a major extinction event. The timing of the next ice age (if there is one) is not our concern so much as what will happen within the lifetime of our cars, power plants, cities, trees, etc - a few decades to a thousand years, perhaps.
Nor does historic variability say much about attribution of current changes.
Any change observed is a function of external forcings and internal dynamics.
Any important features of the system should be observable by monitoring the forcings (e.g. the sun) and the internal states (ocean heat content, sea ice extent, etc.). Large changes in the past might prompt us to look for mechanisms operating now, but when we don't find them in the present, we should look for things that do explain global temperature trends (GHGs) rather than blaming ""natural variability"" that somehow remains unmeasurable.
> The SD community has a history of strongly attacking economic
> forecasting models (a system about which we have far better data and
> far greater
The economy is actually far more complex than climate - there are more diverse actors and products than there are climate-relevant physical quantities, and we know far less about decision making than we do about the physics and chemistry of the atmosphere (except in a few areas like cloud nucleation). The economic data we do have is ambiguous about the most fundamental concepts, like GDP and prices. It's not even possible to reliably decide whether capital-output ratios are rising or falling, for example.
SD's aversion of forecasting is not due to some fundamental flaw in the very idea of forecasting. It's due to the poor practices that prevail in the creation and use of forecasts (a review of Industrial Dynamics, Appendix I is instructive). Economic forecasts are typically bad because too much is exogenous, modelers feel free to ""add factor"" (fudge) without documentation or rigor, and models are not subject to even the most basic tests of quality and robustness. Forecasts are used with the mindset, ""if I know what will happen, I'll know what to do."" As Jay has often pointed out, the interval over which it is possible to reliably forecast (due to momentum) seldom overlaps the interval over which it is possible to influence the system.
Actions taken on the basis of forecasts often become self-fulfilling or self-defeating.
SD has generally adopted a more sensible attitude toward forecasting, which we could probably argue at length but I'll assert boils down to an emphasis on behavior modes and response to policy levers rather than point forecasts.
However, it's still about prediction - only the content of the prediction has been made more reasonable. The things that make a bad forecast (wrong model, wrong inputs) can just as easily lead to a bad prediction of the qualitative response to a policy. However, we are less likely to have those bad components, because we (hopefully) spend more time getting key dynamics into the model, testing a wide range of alternative hypotheses and extreme conditions, and searching for policies that are robust to the uncertainties in inputs.
Climate modelers by and large share our attitude. They explicitly reject point prediction of weather as a reasonable critieria for model evaluation.
They clearly state that forecasts are contingent on uncertain inputs (emissions, volcanoes). They test extremes (removing all water vapor from the atmosphere, for example). I wish they would adopt some of our other habits (transparent model documentation with units of measure, for example).
If there were a significant history of failed climate forecasts - as there is for economic forecasts - I would worry more about climate model validity, but there isn't. In fact, the few early forecasts of temperature contingent on emissions (James Hansen's 1988 congressional testimony, the scenarios in the 1990 IPCC Scientific Assessment) have proven to be right on the money with respect to global temperatures.
Over the same period, critics argued that climate is insensitive to greenhouse gases and have attributed changes to measurement error, natural variablility and a variety of other forcings (solar irradiance, cosmic ray flux, etc.). Because a correct forecast doesn't prove a model - it could be just luck, modelers have examined those alternative hypotheses exhaustively.
The basic result - that current global temperatures and other features of climate can't be explained without GHGs - still stands. Some skeptic assertions - e.g. that temperatures would soon revert to a downward trend - are looking rather silly.
> In my view what is required is a climate model that can endogenously
> replicate the observed reference mode of behaviour over the last 2
> million
As described above, there are already models that can endogenously replicate ice age advance and retreat, though there is still debate about the mechanisms.
The particular trajectory over the Quaternary period will never be replicated because there is no way to recover the trajectory of inputs like solar irradiance with any accuracy.
It's a basic SD insight that systems with long delays require anticipatory action for effective control. We can't just sit on our hands until we are fully satisfied with climate models. So, what are we to do in the interim?
Seems like we should be doing what SD does best: assimilate the knowledge of domain experts (like climatologists) into a framework that makes it relevant to policy on time scales we can influence. Where there is uncertainty, that goes into the model as subjective probability distributions, along with a lot of reality checks, and we look for policies that are robust. Wait-and-see, or wait-for-perfect-models, is not a robust policy.
Tom
****************************************************
Tom Fiddaman
Ventana Systems, Inc.
Posted by Tom Fiddaman <
tom@ventanasystems.com> posting date Mon, 05 Feb 2007 14:19:20 -0700 _______________________________________________