TIME STEP and historical data
-
- Senior Member
- Posts: 54
- Joined: Tue Sep 11, 2018 2:04 am
- Vensim version: PRO
TIME STEP and historical data
Since the historical data tat I am using to optimise and calibrate my model has been collected annually, should I define the TIME STEP annually too? Because the smaller TIME STEP that I use, the worse the optimisation payoff I obtain for the model.
-
- Super Administrator
- Posts: 4590
- Joined: Wed Mar 05, 2003 3:10 am
Re: TIME STEP and historical data
The time step you select should be 1/3 of the smallest delay in your model. So if your shortest delay is one year, your time step should be 0.25 year(s).
http://www.vensim.com/documentation/ref_time_step.htm
http://www.vensim.com/documentation/ref_time_step.htm
Advice to posters seeking help (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Units are important!
http://www.bbc.co.uk/news/magazine-27509559
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Units are important!
http://www.bbc.co.uk/news/magazine-27509559
Re: TIME STEP and historical data
If shrinking the time step is changing the payoff, that's probably a sign that it's also changing the dynamics, and that in turn is changing the fit.
The choice of time step really shouldn't have anything to do with the data interval.
The choice of time step really shouldn't have anything to do with the data interval.
/*
Advice to posters (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Blog: http://blog.metasd.com
Model library: http://models.metasd.com
Bookmarks: http://delicious.com/tomfid/SystemDynamics
*/
Advice to posters (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Blog: http://blog.metasd.com
Model library: http://models.metasd.com
Bookmarks: http://delicious.com/tomfid/SystemDynamics
*/
Re: TIME STEP and historical data
The best thing to do is create some custom graphs that compare the model to the data, and then do some hand calibration to see what's going on.
/*
Advice to posters (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Blog: http://blog.metasd.com
Model library: http://models.metasd.com
Bookmarks: http://delicious.com/tomfid/SystemDynamics
*/
Advice to posters (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Blog: http://blog.metasd.com
Model library: http://models.metasd.com
Bookmarks: http://delicious.com/tomfid/SystemDynamics
*/
-
- Senior Member
- Posts: 54
- Joined: Tue Sep 11, 2018 2:04 am
- Vensim version: PRO
Re: TIME STEP and historical data
I think I figured what the issue can be.
I'm using several RANDOM PINK NOISE functions in my model and it seems they are really sensitive to the change of TIME STEP. Note that I am optimising some of the elements that are used in any of the RANDOM PINK NOISE functions, i.e. mean, sd, and correlation time, but except seed. The outcome of these RANDOM PINK NOISE functions seems to eventually have made the value of the variables and stocks in my model fluctuate drastically with the the change of TIME STEP.
Have I observed and detected a right reason for it?? What's your recommendation to fix it?
-
- Super Administrator
- Posts: 4590
- Joined: Wed Mar 05, 2003 3:10 am
Re: TIME STEP and historical data
As I've suggested, you should fix all the units errors in your model first. This will lead you to where all the errors are.Have I observed and detected a right reason for it?? What's your recommendation to fix it?
Advice to posters seeking help (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Units are important!
http://www.bbc.co.uk/news/magazine-27509559
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Units are important!
http://www.bbc.co.uk/news/magazine-27509559
-
- Senior Member
- Posts: 54
- Joined: Tue Sep 11, 2018 2:04 am
- Vensim version: PRO
Re: TIME STEP and historical data
I have created a new small, simple model and I have used a pink noise in it. Again with the change of time step, the results and payoff function change.Administrator wrote: ↑Mon Mar 30, 2020 8:03 amAs I've suggested, you should fix all the units errors in your model first. This will lead you to where all the errors are.Have I observed and detected a right reason for it?? What's your recommendation to fix it?
Is it supposed to be working like this?
-
- Super Administrator
- Posts: 4590
- Joined: Wed Mar 05, 2003 3:10 am
Re: TIME STEP and historical data
If you upload the model, we can take a look. Make sure you upload all files required to run it.
Advice to posters seeking help (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Units are important!
http://www.bbc.co.uk/news/magazine-27509559
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Units are important!
http://www.bbc.co.uk/news/magazine-27509559
-
- Senior Member
- Posts: 54
- Joined: Tue Sep 11, 2018 2:04 am
- Vensim version: PRO
Re: TIME STEP and historical data
Administrator wrote: ↑Mon Mar 30, 2020 10:44 am If you upload the model, we can take a look. Make sure you upload all files required to run it.
- Attachments
-
- untitled folder.zip
- (5.21 KiB) Downloaded 172 times
-
- Super Administrator
- Posts: 4590
- Joined: Wed Mar 05, 2003 3:10 am
Re: TIME STEP and historical data
I'd suggest putting in some realistic bounds on the things you are allowing the optimizer to modify, if you do this, the optimal solution is identical for time step = 0.5 and 0.03125.
Advice to posters seeking help (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Units are important!
http://www.bbc.co.uk/news/magazine-27509559
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Units are important!
http://www.bbc.co.uk/news/magazine-27509559
Re: TIME STEP and historical data
Time step will have a big effect on any randomness - if you draw twice as many values, you get a completely different realization of the random streams.
To calibrate a model with random inputs, you really need to look at the performance over an ensemble of simulations, rather than a single run. This dramatically adds to the runtime of course.
To calibrate a model with random inputs, you really need to look at the performance over an ensemble of simulations, rather than a single run. This dramatically adds to the runtime of course.
/*
Advice to posters (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Blog: http://blog.metasd.com
Model library: http://models.metasd.com
Bookmarks: http://delicious.com/tomfid/SystemDynamics
*/
Advice to posters (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Blog: http://blog.metasd.com
Model library: http://models.metasd.com
Bookmarks: http://delicious.com/tomfid/SystemDynamics
*/
-
- Senior Member
- Posts: 54
- Joined: Tue Sep 11, 2018 2:04 am
- Vensim version: PRO
Re: TIME STEP and historical data
I put some realistic bounds. The eventual results seem identical, though not entirely. Why does the outcome of pink noise change with the change of time step? I even disconnected the pink noise and it does change every time I change the time step.Administrator wrote: ↑Mon Mar 30, 2020 12:07 pm I'd suggest putting in some realistic bounds on the things you are allowing the optimizer to modify, if you do this, the optimal solution is identical for time step = 0.5 and 0.03125.
Also my real model is way larger than this. How can i select the best time step and make sure that the selected time step is the one to go with?
Re: TIME STEP and historical data
If you think about a run with 4 time steps, the random draws might be
Time 1 2 3 4
Value 3 8 2 6
If you halve the time step, you get
Time 0.5 1 1.5 2 2.5 3 3.5 4
Value 3 8 2 6 1 7 7 9
The value 2 now occurs at time 1.5 instead of time 2, etc.
In a real model, it's more complex than that, because you may have multiple random variables drawing from the same stream of numbers.
If the behavior still changes when the pink noise is disconnected (e.g., standard deviation set to 0), then that must be a change in other behavior, related to stability or perhaps discrete events (PULSE etc.).
The easiest way to test is to cut time step in half successively until the behavior stops changing.
Time 1 2 3 4
Value 3 8 2 6
If you halve the time step, you get
Time 0.5 1 1.5 2 2.5 3 3.5 4
Value 3 8 2 6 1 7 7 9
The value 2 now occurs at time 1.5 instead of time 2, etc.
In a real model, it's more complex than that, because you may have multiple random variables drawing from the same stream of numbers.
If the behavior still changes when the pink noise is disconnected (e.g., standard deviation set to 0), then that must be a change in other behavior, related to stability or perhaps discrete events (PULSE etc.).
The easiest way to test is to cut time step in half successively until the behavior stops changing.
/*
Advice to posters (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Blog: http://blog.metasd.com
Model library: http://models.metasd.com
Bookmarks: http://delicious.com/tomfid/SystemDynamics
*/
Advice to posters (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Blog: http://blog.metasd.com
Model library: http://models.metasd.com
Bookmarks: http://delicious.com/tomfid/SystemDynamics
*/
-
- Senior Member
- Posts: 54
- Joined: Tue Sep 11, 2018 2:04 am
- Vensim version: PRO
Re: TIME STEP and historical data
Thanks Tom for clear explanation!!tomfid wrote: ↑Mon Mar 30, 2020 12:47 pm If you think about a run with 4 time steps, the random draws might be
Time 1 2 3 4
Value 3 8 2 6
If you halve the time step, you get
Time 0.5 1 1.5 2 2.5 3 3.5 4
Value 3 8 2 6 1 7 7 9
The value 2 now occurs at time 1.5 instead of time 2, etc.
In a real model, it's more complex than that, because you may have multiple random variables drawing from the same stream of numbers.
If the behavior still changes when the pink noise is disconnected (e.g., standard deviation set to 0), then that must be a change in other behavior, related to stability or perhaps discrete events (PULSE etc.).
The easiest way to test is to cut time step in half successively until the behavior stops changing.
I meant I disconnected the pink noise and watched whether the behaviour of the pink noise changes or not, which as a result it did. And obviously this proved your above explanation that any randomness changes with the change of time step. Its going to be a long overwhelming and frustrating optimisation process to work with all the pink noises and time step in my model i think.
-
- Super Administrator
- Posts: 4590
- Joined: Wed Mar 05, 2003 3:10 am
Re: TIME STEP and historical data
Your first action should be to correct all the unit errors in your model. These errors mean your model has bugs and is not producing the correct output at time steps other than 1.Simplifier wrote: ↑Mon Mar 30, 2020 12:58 pmIts going to be a long overwhelming and frustrating optimisation process to work with all the pink noises and time step in my model i think.
Advice to posters seeking help (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Units are important!
http://www.bbc.co.uk/news/magazine-27509559
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Units are important!
http://www.bbc.co.uk/news/magazine-27509559