how to weigh newer data more heavily in calibration?

Use this forum to post Vensim related questions.
Post Reply
kleemax
Senior Member
Posts: 125
Joined: Tue Jan 26, 2016 3:50 pm
Vensim version: PRO

how to weigh newer data more heavily in calibration?

Post by kleemax »

It's not rare that older data is less reliable, especially in my work in developing countries. REcent data may be measurements, whereas older data may be estimates only, or the measurement frequency has increased recently.
Hence when calibrating manually and I face a trade-off between of getting a good fit to old data vs. to recent data, I tend to choose the latter. I mean data for the same variable that is in one and the same vdf file.
I wonder if there is a way to do the same in automatic calibration?
In case the answer is "not yet":
I've been thinking that delays are actually do something like that, they weigh older information less and recent information more. Hence maybe we could use SD models to optimize other SD-models in the way that I am asking for above?
tomfid
Administrator
Posts: 3806
Joined: Wed May 24, 2006 4:54 am

Re: how to weigh newer data more heavily in calibration?

Post by tomfid »

Hi Max -

This is possible.

As you say, a SMOOTH creates an exponentially-weighted moving average, valuing the most recent data most, though that doesn't really help with optimization.

The key idea in calibration is that the weight or scale parameter for each data/model comparison can be a variable rather than a number.

In some cases (here's one: http://berkeleyearth.org/wp-content/upl ... -small.png) your data series will have an explicit error estimate that you can use.

If you don't have that, you can construct something in a model variable. This is fairly easy if you use the Gaussian option:
2019-05-01_1509.png
2019-05-01_1509.png (31.91 KiB) Viewed 1694 times
In this case, the variable errorSD is the std dev of the data, which you could express as f(data,time). The Robust (ABS) option is nearly the same, except that the scale parameter represents the median absolute deviation.
tomfid
Administrator
Posts: 3806
Joined: Wed May 24, 2006 4:54 am

Re: how to weigh newer data more heavily in calibration?

Post by tomfid »

For example, I might do the following:

dataVar = GET XLS ...
errorSD = dataVar * estFracError
estFracError = base error frac + data age * error slope
error slope = ... parameter describing how fast the data gets worse with age
data age = max(0,last data time - time)
last data time = INITIAL( GET DATA LAST TIME( dataVar ) )

You can also try including the base error frac and error slope as parameters in your optimization (unless you're using the Normal distribution assumption).
Post Reply