Forecasts vs. policy design

["Jay W. Forrester"]

There have been several interesting comments on the relationship of
forecasts to system dynamics models. The subtlety and richness of the
forecast question could justify a book. However, let me comment in two
separate email messages.

First is the relation of forecasting to policy design.

Consider two very different objectives for forecasting.

1. The common meaning of forecasting calls for forecasting a future event
or the future condition of the system, for example, a forecast of what
customer orders will be received three months hence.

2. A much more important kind of forecast is to forecast that a specific
change in decision-making policies will product a better behaved system,
regardless of what the future may unfold. In other words, following Policy
A will cause orders to rise compared to following Policy B, even if future
values for A and B can not be determined.

Appendix K in my "Industrial Dynamics" book (now transferred to and
available from Pegasus Communications, Waltham, MA) illustrates the
weakness of prediction and the power of policy design. In that appendix, a
model is defined as the real system. Another model is used as the
predictor. The models are identical except for the random inputs that we
must admit exist in all real-world systems. The "predicting" model,
although a perfect replica of the "real-system" model does not give useful
predictions of the future values of variables.

However, when an improved policy, which reduces fluctuation in production
in the "forecast" model, is then adopted in the "real" model, the "real"
system also shows reduced fluctuation.

I believe that one can forecast how a policy change will affect behavior of
a system with much more confidence than one can forecast future events.
The second definition of forecasting above is feasible, the first will
seldom be useful.

Whatever is done in the name of forecasting, the process should itself be
incorporated into a system dynamics simulation model. Forecasts do not
have access to the future. They are created out of past information using
some analysis policy. Therefore, the forecast procedure is part of the
policy structure of the system and should be incorporated into a model.
Doing so will often show that the forecast procedure is creating
difficulties. For example, see Appendix M of "Industrial Dynamics" for how
forecasting can produce instability, and see Appendix N for how a forecast
can create seasonal sales fluctuation in a market that is inherently
without seasonality.

It is my observation that the higher the reputation of forecasting in a
company the greater is the harm that forecasting is doing. Forecasts that
turn out to be right are usually self-fullfilling forecasts where the
forecast creates the reality. Forecasts are driven strongly by the past
behavior. If sales have been rising, the forecast continues the trend.
Capacity is then allocated to make product for the forecast, and the sales
equal the forecast. In such a situation, one will probably find that
potential sales were substantially higher than the forecast, but sales were
held down to the capacity that was created in response to the forecast.
One cannot sell what is not produced, but that fact is overlooked in a
surprising number of companies. I have seen companies who lost market
share steadily over many years because capacity was limited by the
forecasts, sales were limited by the capacity, and excess demand went to
competitors.

---------------------------------------------------------
Jay W. Forrester
Professor of Management, Emeritus
and Senior Lecturer, Sloan School
Massachusetts Institute of Technology
Room E60-389
Cambridge, MA 02139
tel: 617-253-1571
fax: 617-258-9405
From: "Jay W. Forrester" <jforestr@MIT.EDU>