Different results when using Sim Control>Set>Simulate vs Sim Control>Simulate

Use this forum to post Vensim related questions.
Post Reply
RobbieOrvis
Member
Posts: 43
Joined: Wed Dec 13, 2017 5:16 pm
Vensim version: DSS

Different results when using Sim Control>Set>Simulate vs Sim Control>Simulate

Post by RobbieOrvis »

On DSS build 8.1.2 (and perhaps earlier) we noticed today that running the model in different ways produces different results:

If we run the model by going to Sim Control>Set and then clicking Simulate, we get a different set of results than going to Sim Control>Simulate.

This is particularly an issue when the same model software is moved between directories (for example, with different regions with differing input data that rely on the same .mdl or .vpmx file). It seems as if some of the prior input data are being used, even when the model is placed in a different directory.
Administrator
Super Administrator
Posts: 4573
Joined: Wed Mar 05, 2003 3:10 am

Re: Different results when using Sim Control>Set>Simulate vs Sim Control>Simulate

Post by Administrator »

Can you email us an example of this?
Advice to posters seeking help (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391

Units are important!
http://www.bbc.co.uk/news/magazine-27509559
jrissman
Senior Member
Posts: 95
Joined: Wed Oct 16, 2013 6:04 pm
Vensim version: DSS

Re: Different results when using Sim Control>Set>Simulate vs Sim Control>Simulate

Post by jrissman »

I can provide an example of this bug. Here is how to reproduce the issue:

1. Download the public 3.0.0 release of the U.S. Energy Policy Simulator at https://github.com/Energy-Innovation/ep ... /3.0.0.zip

2. Open the EPS.vpmx file (NOT the EPS.mdl) file in Vensim DSS.

3. Click Sim Control > Changes tab > Load changes from... button and select the file "Scenario_NetZero.cin" that is included in the model folder.

4. Click the "Simulate" button that is inside the "Sim Control" pop-up box (not the "Simulate" button on the main toolbar)

5. Vensim reports that it's reading from "Scenario_NetZero.cin," runs the model, and produces results. Hit the "Table" button. Unless you have accidentally clicked on some other variable, the variable "Output Total CO2e Emissions" should be selected, and it should give you a table of numerical results. Notice the result in 2050 is -1.74853

6. If you want to verify that the "Scenario_NetZero.cin" file is still selected, you may open "Sim Control" again and look at the "Changes" tab to verify the changes file is still selected. Hit "Set" to close the "Sim Control" pane. This step is optional, as it is only to prove that we're still going to run with the same exact .cin file.

7. Click the "Simulate" button on the main toolbar.

8. Vensim reports that it's reading from "Scenario_NetZero.cin," runs the model, and produces results. Hit the "Table" button. Unless you have accidentally clicked on some other variable, the variable "Output Total CO2e Emissions" should be selected, and it should give you a table of numerical results. Notice the result in 2050 is -1.57735 (not -1.74853)

In short, the model produces different numerical results depending on whether you run the model by using the "Simulate" button in the main toolbar, or the "Simulate" button in the "Sim Control" pane. While the result is only slightly different in the example above, we've encountered cases where the results are completely, dramatically different depending on which "Simulate" button you use.

You can actually toggle back and forth between the two sets of results by running the model using one Simulate button or the other, with the same .cin file remaining selected.

Of course, the model results ought not to be different depending on which Simulate button you click.

This bug only happens with the .vpmx file. If you perform the same test above but using the .mdl file, you will get a result of -1.57735 when using either Simulate button.
Administrator
Super Administrator
Posts: 4573
Joined: Wed Mar 05, 2003 3:10 am

Re: Different results when using Sim Control>Set>Simulate vs Sim Control>Simulate

Post by Administrator »

Thank you for this.

I've already got a fix, but my internet is down at the moment preventing me from uploading it (I'm tethered to my cellphone). It only happens to lookups imported using get_direct_lookups with binary format models.

Hopefully my internet access will back up soon and I can get the fix to you.
Advice to posters seeking help (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391

Units are important!
http://www.bbc.co.uk/news/magazine-27509559
jrissman
Senior Member
Posts: 95
Joined: Wed Oct 16, 2013 6:04 pm
Vensim version: DSS

Re: Different results when using Sim Control>Set>Simulate vs Sim Control>Simulate

Post by jrissman »

That's great! Happy to hear it was a quick fix. If the fix is included in the next incremental Vensim release, that would be perfect. We can simply avoid using the broken "Simulate" button in the meanwhile.
Administrator
Super Administrator
Posts: 4573
Joined: Wed Mar 05, 2003 3:10 am

Re: Different results when using Sim Control>Set>Simulate vs Sim Control>Simulate

Post by Administrator »

I've emailed Robbie with a link to the fixed version.
Advice to posters seeking help (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391

Units are important!
http://www.bbc.co.uk/news/magazine-27509559
Post Reply