Page 4 of 15

Re: Help with model calibration

Posted: Tue May 02, 2017 8:29 pm
by leon
I don't see any errors/warning other than "No values in run import - unloaded"

Re: Help with model calibration

Posted: Tue May 02, 2017 8:36 pm
by tomfid
Not sure what the problem is here. There was a bug in the initial implementation of tidy data, but I thought it was fixed in 6.4e. Maybe not - that would explain why this isn't working. I think you have two options: reformat the data in some other way, or wait for the next release (should be this week).

Re: Help with model calibration

Posted: Wed May 03, 2017 7:22 am
by Administrator
I think you've got your files confused. And I think you still need to read the help we keep posting, the original example I posted takes you through everything (including data import).

1. Download the "Drivers.zip" file that Tom uploaded.
2. Unzip the file.
3. Open Vensim.
4. Click "File->Open Model" and open "Driving1.mdl".
5. Click "File->Edit file" and select "import.cmd".
6. Click "File->Run Commands" and you should be able to import the data.

I'd really urge you to spend the time going through the tutorial first though.

Re: Help with model calibration

Posted: Wed May 03, 2017 1:39 pm
by tomfid
You're right. It's not a bug. I reverted to 6.4E and it works fine. I get:

Code: Select all

Writing 6 values for -events per month[Driver60]
Writing 6 values for -events per month[Driver57]
Writing 6 values for -events per month[Driver56]
Writing 6 values for -events per month[Driver54]
Writing 6 values for -events per month[Driver53]
Writing 6 values for -events per month[Driver52]
Writing 6 values for -events per month[Driver51]
Writing 6 values for -events per month[Driver50]
Writing 6 values for -events per month[Driver49]
Writing 6 values for -events per month[Driver48]
Writing 6 values for -monthly driving[Driver60]
Writing 6 values for -monthly driving[Driver57]
Writing 6 values for -monthly driving[Driver56]
Writing 6 values for -monthly driving[Driver54]
Writing 6 values for -monthly driving[Driver53]
Writing 6 values for -monthly driving[Driver52]
Writing 6 values for -monthly driving[Driver51]
Writing 6 values for -monthly driving[Driver50]
Writing 6 values for -monthly driving[Driver49]
Writing 6 values for -monthly driving[Driver48]
Writing 6 values for time base -Time
Loading data.vdf as data.  Origin is ..
data.csv converted to dataset data.vdf on Wed May 03 07:34:47 2017
So it has to be something else.
- running inside the .zip archive?
- files in a location without write privileges?
- wrong command?

Re: Help with model calibration

Posted: Thu May 04, 2017 5:49 am
by leon
Administrator wrote:I think you've got your files confused. And I think you still need to read the help we keep posting, the original example I posted takes you through everything (including data import).

1. Download the "Drivers.zip" file that Tom uploaded.
2. Unzip the file.
3. Open Vensim.
4. Click "File->Open Model" and open "Driving1.mdl".
5. Click "File->Edit file" and select "import.cmd".
6. Click "File->Run Commands" and you should be able to import the data.

I'd really urge you to spend the time going through the tutorial first though.
I did exactly what you described, which is almost what I was doing. The only thing new here is the step 4, which I followed. I'm still getting the same thing "No values in the run import - unloaded". Are we using the same software?

Re: Help with model calibration

Posted: Thu May 04, 2017 5:55 am
by leon
tomfid wrote:You're right. It's not a bug. I reverted to 6.4E and it works fine. I get:

Code: Select all

Writing 6 values for -events per month[Driver60]
Writing 6 values for -events per month[Driver57]
Writing 6 values for -events per month[Driver56]
Writing 6 values for -events per month[Driver54]
Writing 6 values for -events per month[Driver53]
Writing 6 values for -events per month[Driver52]
Writing 6 values for -events per month[Driver51]
Writing 6 values for -events per month[Driver50]
Writing 6 values for -events per month[Driver49]
Writing 6 values for -events per month[Driver48]
Writing 6 values for -monthly driving[Driver60]
Writing 6 values for -monthly driving[Driver57]
Writing 6 values for -monthly driving[Driver56]
Writing 6 values for -monthly driving[Driver54]
Writing 6 values for -monthly driving[Driver53]
Writing 6 values for -monthly driving[Driver52]
Writing 6 values for -monthly driving[Driver51]
Writing 6 values for -monthly driving[Driver50]
Writing 6 values for -monthly driving[Driver49]
Writing 6 values for -monthly driving[Driver48]
Writing 6 values for time base -Time
Loading data.vdf as data.  Origin is ..
data.csv converted to dataset data.vdf on Wed May 03 07:34:47 2017
So it has to be something else.
- running inside the .zip archive? I unzipped the file
- files in a location without write privileges? I'm not sure I know location without write privilege. What does this mean?
- wrong command?
I just used your command as described/suggested

Re: Help with model calibration

Posted: Thu May 04, 2017 8:04 am
by Administrator
ok, I think I know what's going on.

When you click "File->Run commands", Are you getting a message saying "Do you want to treat this as input data"?

You are using Vensim Pro, we assumed you were using DSS (and command files are not supported in Pro). There is no easy way to load tidy format data into Vensim pro at the moment.

I'd urge you again to go through the tutorial I posted originally. This takes you through data import and calibration, you could have formatted your data into a similar way to how the tutorial asks for.

I've formatted the data in a way Vensim Pro can read.

1. Click Model->Import data
2. Select data.xlsx
3. Select the "VENSIM_FORMAT_driver_data" sheet.
4. In the next dialog, there is a setting (row 2), "Time axis name : ", following this, there is "Across" and "Down. Select "Down".
5. Click ok.

Re: Help with model calibration

Posted: Fri May 05, 2017 5:58 am
by leon
Thanks so much, I appreciate. It works fine now the import.

Re: Help with model calibration

Posted: Fri May 05, 2017 8:35 pm
by leon
I've tried to follow the electric example on this link http://www.vensim.com/documentation/ind ... ?21295.htm
But I guess changes have been made to the software and the steps described are not what I'm seeing.
When I Clicked on the Optimize button on the Toolbar (there is no optimization button on the toolbar as suggested by the tutorial), I didn't get The Optimization Wizard opens with the Payoff Definition. Rather I got what I've attached to this message. Maybe there is a step I missed out
Thanks for your help

Re: Help with model calibration

Posted: Fri May 05, 2017 8:43 pm
by leon
I figured it out. Sorry

Re: Help with model calibration

Posted: Fri May 05, 2017 8:53 pm
by leon
What are the rules for setting weight during calibration? Should I use a weight of 1 for my data?

Re: Help with model calibration

Posted: Fri May 05, 2017 9:17 pm
by leon
I got this at the end of the calibration. Do you have any ideas about what went wrong?
Also, the initial values of the stocks are not really known and the software did not ask about them. I also wanted the stocks to be parameters to be estimated.
Thanks a lot, I appreciate your help

Re: Help with model calibration

Posted: Mon May 08, 2017 10:45 am
by Administrator
I'm not sure what you are doing. It looks like you've selected the data file as the payoff definition.

You really need to follow the example, it takes you through all the steps you need to go through.

Re: Help with model calibration

Posted: Mon May 08, 2017 3:24 pm
by leon
I could be wrong, but I think I've exactly followed the different steps. I've attached all the steps I went through
Thanks so much for all your help

Re: Help with model calibration

Posted: Mon May 08, 2017 3:36 pm
by Administrator
Can you upload all the files you are using as it's impossible to help with just screenshots.

And I'd really suggest you do the Vensim example I keep telling you about.

Re: Help with model calibration

Posted: Mon May 08, 2017 4:23 pm
by tomfid
It appears that you're trying to load a dataset that doesn't exist. Check what you have specified in the Data Sources field of the Advanced tab of the Simulation Control dialog.

Re: Help with model calibration

Posted: Mon May 08, 2017 4:27 pm
by tomfid
If you want the initial conditions of the stocks to be included, make them parameters:
stock = INTEG( flows, initial stock )
initial stock = 3.141
Then add the "initial ..." constant to the optimization control file.

Using a weight of 1 is probably a good start, as long as driver errors are similar in magnitude.

Re: Help with model calibration

Posted: Mon May 08, 2017 5:14 pm
by leon
Thanks Tomfid for pointing out the problem, I'm so appreciative. I think you are correct. It worked fine, but with some floating errors.

I would like to address the initial values for the stocks, which are not known.
How do you specify that constant (initial condition) in the model? The control parameter file is not giving me the option of including the initial values of the stocks as parameters. Take a look at the parameter options I got when I clicked on "add constant"

One more question: there is also an auxiliary variable I made before, and now want to remove it. Any suggestions?

Thanks so much

Re: Help with model calibration

Posted: Mon May 08, 2017 5:40 pm
by tomfid
You have to add the constant to the model first - edit your equation so that the stock initial conditions are named constants. Then you can use those constants in your control file.

You can delete a variable with the Delete tool in the toolbar, or in the Equation Editor, using the Delete Variable button.

Re: Help with model calibration

Posted: Mon May 08, 2017 6:26 pm
by leon
The equation for a stock was expressed as a function of flow and initial value, for example: cumulative miles driven=integ(monthly driving, 200) with 200 being the initial value for that stock. When I clicked on "add constant" in the optimization control file, a diag "optimization parameter" opened and has a list of variables. Stock names are not in the variable names, and I don't see a way to specify a stock initial value as a parameter.

Sorry, I wanted to ask how to delete a shadow variable. I used "time" as a shadow variable, and VENSIM said it cannot be deleted. I wanted to remove it

Re: Help with model calibration

Posted: Mon May 08, 2017 6:37 pm
by leon
I guess I have to name the initial value a variable, and use an arrow to link the variable (initial condition) with the stock variable. What do you think?

Re: Help with model calibration

Posted: Mon May 08, 2017 6:41 pm
by tomfid
Recall my example above:
stock = INTEG( flows, initial stock )
initial stock = 3.141
In the same way, rewrite your stock equation so that the 200 is a separate variable, "initial miles driven," with value 200 (or whatever). Then use that constant in your optimization setup.

To remove <time>, select it, hit the delete key, and choose "Remove from View."

Re: Help with model calibration

Posted: Mon May 08, 2017 6:41 pm
by tomfid
What do you think?
I think you are correct.

Re: Help with model calibration

Posted: Mon May 08, 2017 10:57 pm
by leon
Please, help me to improve this work. I'm so grateful for all the help you have been providing to me.
After calibration:
1) There were floating point errors
2) The results are not satisfactory as model results and actual data do not overlap very well. Issue with the weight?
3) Any inputs for improvements?

Also my thinking is to compute individual statistics and then average them to get the final statistics for the sample. Then I'm going to apply the model to a different set of drivers using the average model parameters

Please, find attached the model results. I've also attached the model

Re: Help with model calibration

Posted: Mon May 08, 2017 11:30 pm
by tomfid
3) The model you attached is the same as my original concept, so I don't have any comment on that.

Just as a matter of good source control practice, I'd recommend keeping your whole project in a folder with a version number, and periodically copying the whole thing to a new folder with an incremented number. Then it's easy to .zip up everything for archiving or posting. We can't say much without a complete package. (This is good advice for everyone posting here btw.)

2) The weights are unlikely to be the problem, at least when the data all have similar scale, as here. More likely, the structure simply can't reproduce the desired behavior precisely, and/or there's a lot of noise in the error process.

A good experiment would be to calibrate one or more drivers by hand (in Synthesim). That'll give you some insight into what matters and how to do better.

1) Generally your model should not be causing floating point errors. Sometimes they're benign, like the division by zero you get from computing market shares if all companies have 0 sales, but often they represent a conceptual problem.

To test, replicate the conditions that caused the reported error. One way to do that is to paste the error report into a .cin file and load that at startup. Here's an example, from your report:
fp error.cin
(4.85 KiB) Downloaded 150 times