Ventana software support forum

Dear All:

I still have some questions left regarding how calibration is handled by Vensim Pro/DSS. But let's just wrap things up as far as have understood calibration in System Dynamics in general. I have enclosed three charts from the excellent paper by Peterson that is part of Jorgen Randers' "Elements of the System Dynamics Method" (1980) that should make this easier to grasp. To sum it up any SD model can be brought into a general mathematical form where you have driving functions (law of motion) for the stocks (e.g. the flow equations = dS/dt = f[Environment(t), S(t),t]) and some behavior function (e.g. how you will measure the system states = g[S(t),Environment(t),t]) - the bold would be vectors to be general.

Now, there can be error entering the model - usually called noise - in either or both functions, e.g. you can have driving noise in the flow equations or measurement error in the behavior function. According to Peterson in the case of driving noise one should stick to reinstating the state variables (e.g. the stocks) at any time you have reference data. In the case of driving and measurement error FIMLOF would be appropriate or probably Bootstrapping (?).

I am not sure whether Vensim will reinstate the state variables (e.g. the procedure suggested by Peterson for OLS which might be generalized to WLS as Vensim will allow weights...) at the points in time that there is reference data available. If Vensim does not reinstate the stocks, would this make sense and how can this achieved efficiently?

Kind regards,


Guido

If you're using Kalman Filtering in Vensim, it will move the states towards the data as needed (there's an example in the help system).

It takes some work to set up the filter, so it's pretty common to calibrate ordinary simulations, without worrying about state updating. In that case you're essentially just minimizing the squared error between model and data.

The payoffs are log likelihoods or likelihoods, so the squared error terms assume normally distributed errors. If you have some other error distribution (for example, Poisson for events in a simulation with discrete events or individuals), you can create your own error criteria and minimize it as a policy payoff.

Tom

Hi Tom,

thank you for your reply. I am aware of what the Kalman-Bucy Filter (FIMLOF) will do; what it does is - in my opionion - nicely described by Peterson in his article as provided by Randers (while it is older I believe). It is a tool very appropriate for a system where driving and measuring noice are present.

My question was rather aimed at the second chart as provided in the attachement to my post (Peterson calls it "Ordinary Least Squares") where driving noise is present only, e.g. you have no errors measuring the system's state but you have not included all factors driving it so there is noise in the flow equations. Peterson distinguishes "naive" calibration which calibrates without reinstatement vs. reinstatement during the calibration. This seems to be somewhere inbetween going to FIMLOF which needs covariances and initial error-estimates and is thus "hard" to set up.

The question simply is: Will Vensim's "regular" calibration routine reinstate the stocks during calibration - my impression is that is does not do this and thus would amount to the "naive" way of calibration as given by Peterson? If not - how can this be achieved efficiently without having to employ "heavy duty tools" like FIMLOF?

Kind regards,

Guido

Right - regular calibration is of the 'naive' variety. There isn't any straightforward way to do state updates, other than using the kalman filter (which is actually not that hard to set up as long as your model is fairly simple). You could probably implement your own state updates via additional rates that pulse in changes, but that would be a big hassle and hard to get right.

Tom

Does this mean that when doing calibration and the data is noisy (as it probably almost always will be) that Kalman filtering should be switched on - almost by default - if the model is not too big, with the extra effort required in providing the driving and measurement noise variances for the calibration variable in the payoff file? Why would one not use Kalman filtering and use the "naïve calibration" instead? To avoid the effort of estimating the variances?