TIME STEP and historical data

Use this forum to post Vensim related questions.
Post Reply
Simplifier
Senior Member
Posts: 51
Joined: Tue Sep 11, 2018 2:04 am
Vensim version: PRO

TIME STEP and historical data

Post by Simplifier »

Since the historical data tat I am using to optimise and calibrate my model has been collected annually, should I define the TIME STEP annually too? Because the smaller TIME STEP that I use, the worse the optimisation payoff I obtain for the model.

Administrator
Super Administrator
Posts: 3522
Joined: Wed Mar 05, 2003 3:10 am

Re: TIME STEP and historical data

Post by Administrator »

The time step you select should be 1/3 of the smallest delay in your model. So if your shortest delay is one year, your time step should be 0.25 year(s).

http://www.vensim.com/documentation/ref_time_step.htm
Advice to posters seeking help (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391

Units are important!
http://www.bbc.co.uk/news/magazine-27509559

tomfid
Administrator
Posts: 2937
Joined: Wed May 24, 2006 4:54 am

Re: TIME STEP and historical data

Post by tomfid »

If shrinking the time step is changing the payoff, that's probably a sign that it's also changing the dynamics, and that in turn is changing the fit.

The choice of time step really shouldn't have anything to do with the data interval.

tomfid
Administrator
Posts: 2937
Joined: Wed May 24, 2006 4:54 am

Re: TIME STEP and historical data

Post by tomfid »

The best thing to do is create some custom graphs that compare the model to the data, and then do some hand calibration to see what's going on.

Simplifier
Senior Member
Posts: 51
Joined: Tue Sep 11, 2018 2:04 am
Vensim version: PRO

Re: TIME STEP and historical data

Post by Simplifier »

tomfid wrote:
Sun Mar 29, 2020 4:09 pm
The best thing to do is create some custom graphs that compare the model to the data, and then do some hand calibration to see what's going on.
I think I figured what the issue can be.
I'm using several RANDOM PINK NOISE functions in my model and it seems they are really sensitive to the change of TIME STEP. Note that I am optimising some of the elements that are used in any of the RANDOM PINK NOISE functions, i.e. mean, sd, and correlation time, but except seed. The outcome of these RANDOM PINK NOISE functions seems to eventually have made the value of the variables and stocks in my model fluctuate drastically with the the change of TIME STEP.

Have I observed and detected a right reason for it?? What's your recommendation to fix it?

Administrator
Super Administrator
Posts: 3522
Joined: Wed Mar 05, 2003 3:10 am

Re: TIME STEP and historical data

Post by Administrator »

Have I observed and detected a right reason for it?? What's your recommendation to fix it?
As I've suggested, you should fix all the units errors in your model first. This will lead you to where all the errors are.
Advice to posters seeking help (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391

Units are important!
http://www.bbc.co.uk/news/magazine-27509559

Simplifier
Senior Member
Posts: 51
Joined: Tue Sep 11, 2018 2:04 am
Vensim version: PRO

Re: TIME STEP and historical data

Post by Simplifier »

Administrator wrote:
Mon Mar 30, 2020 8:03 am
Have I observed and detected a right reason for it?? What's your recommendation to fix it?
As I've suggested, you should fix all the units errors in your model first. This will lead you to where all the errors are.
I have created a new small, simple model and I have used a pink noise in it. Again with the change of time step, the results and payoff function change.

Is it supposed to be working like this?

Administrator
Super Administrator
Posts: 3522
Joined: Wed Mar 05, 2003 3:10 am

Re: TIME STEP and historical data

Post by Administrator »

If you upload the model, we can take a look. Make sure you upload all files required to run it.
Advice to posters seeking help (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391

Units are important!
http://www.bbc.co.uk/news/magazine-27509559

Simplifier
Senior Member
Posts: 51
Joined: Tue Sep 11, 2018 2:04 am
Vensim version: PRO

Re: TIME STEP and historical data

Post by Simplifier »

Administrator wrote:
Mon Mar 30, 2020 10:44 am
If you upload the model, we can take a look. Make sure you upload all files required to run it.
Attachments
untitled folder.zip
(5.21 KiB) Downloaded 14 times

Administrator
Super Administrator
Posts: 3522
Joined: Wed Mar 05, 2003 3:10 am

Re: TIME STEP and historical data

Post by Administrator »

I'd suggest putting in some realistic bounds on the things you are allowing the optimizer to modify, if you do this, the optimal solution is identical for time step = 0.5 and 0.03125.
Advice to posters seeking help (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391

Units are important!
http://www.bbc.co.uk/news/magazine-27509559

tomfid
Administrator
Posts: 2937
Joined: Wed May 24, 2006 4:54 am

Re: TIME STEP and historical data

Post by tomfid »

Time step will have a big effect on any randomness - if you draw twice as many values, you get a completely different realization of the random streams.

To calibrate a model with random inputs, you really need to look at the performance over an ensemble of simulations, rather than a single run. This dramatically adds to the runtime of course.

Simplifier
Senior Member
Posts: 51
Joined: Tue Sep 11, 2018 2:04 am
Vensim version: PRO

Re: TIME STEP and historical data

Post by Simplifier »

Administrator wrote:
Mon Mar 30, 2020 12:07 pm
I'd suggest putting in some realistic bounds on the things you are allowing the optimizer to modify, if you do this, the optimal solution is identical for time step = 0.5 and 0.03125.
I put some realistic bounds. The eventual results seem identical, though not entirely. Why does the outcome of pink noise change with the change of time step? I even disconnected the pink noise and it does change every time I change the time step.

Also my real model is way larger than this. How can i select the best time step and make sure that the selected time step is the one to go with?

tomfid
Administrator
Posts: 2937
Joined: Wed May 24, 2006 4:54 am

Re: TIME STEP and historical data

Post by tomfid »

If you think about a run with 4 time steps, the random draws might be
Time 1 2 3 4
Value 3 8 2 6
If you halve the time step, you get
Time 0.5 1 1.5 2 2.5 3 3.5 4
Value 3 8 2 6 1 7 7 9
The value 2 now occurs at time 1.5 instead of time 2, etc.

In a real model, it's more complex than that, because you may have multiple random variables drawing from the same stream of numbers.

If the behavior still changes when the pink noise is disconnected (e.g., standard deviation set to 0), then that must be a change in other behavior, related to stability or perhaps discrete events (PULSE etc.).

The easiest way to test is to cut time step in half successively until the behavior stops changing.

Simplifier
Senior Member
Posts: 51
Joined: Tue Sep 11, 2018 2:04 am
Vensim version: PRO

Re: TIME STEP and historical data

Post by Simplifier »

tomfid wrote:
Mon Mar 30, 2020 12:47 pm
If you think about a run with 4 time steps, the random draws might be
Time 1 2 3 4
Value 3 8 2 6
If you halve the time step, you get
Time 0.5 1 1.5 2 2.5 3 3.5 4
Value 3 8 2 6 1 7 7 9
The value 2 now occurs at time 1.5 instead of time 2, etc.

In a real model, it's more complex than that, because you may have multiple random variables drawing from the same stream of numbers.

If the behavior still changes when the pink noise is disconnected (e.g., standard deviation set to 0), then that must be a change in other behavior, related to stability or perhaps discrete events (PULSE etc.).

The easiest way to test is to cut time step in half successively until the behavior stops changing.
Thanks Tom for clear explanation!!
I meant I disconnected the pink noise and watched whether the behaviour of the pink noise changes or not, which as a result it did. And obviously this proved your above explanation that any randomness changes with the change of time step. Its going to be a long overwhelming and frustrating optimisation process to work with all the pink noises and time step in my model i think.

Administrator
Super Administrator
Posts: 3522
Joined: Wed Mar 05, 2003 3:10 am

Re: TIME STEP and historical data

Post by Administrator »

Simplifier wrote:
Mon Mar 30, 2020 12:58 pm
Its going to be a long overwhelming and frustrating optimisation process to work with all the pink noises and time step in my model i think.
Your first action should be to correct all the unit errors in your model. These errors mean your model has bugs and is not producing the correct output at time steps other than 1.
Advice to posters seeking help (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391

Units are important!
http://www.bbc.co.uk/news/magazine-27509559

Post Reply