Ventana software support forum

Vensim has to open the data.vdf to read the data. Some other process may have locked the file. Restarting Vensim will probably clear the error.

First of all, thank you a lot. The weight did really improve the model fit for both measures per driver. I have some floating errors, also the model did not capture at all 1 driver. I don't know if I still need to refine the weight or if there is something I need to do. Also some of the initial conditions estimated from the model do not seem realistic at all. For example, I would not expect a new driver to have driven 3,000 miles before their formal driver license.

I think the first thing to pay attention to is these warnings:
Either the data or the model prediction is for a negative error rate, which is obviously not possible.

BTW the weight for Poisson elements in the payoff will normally be 1, because the distribution assumption already takes care of the scaling (making the mean = variance assumption).

I'm still getting the same warning after changing the weight to 1.

Let me try this and see what happens. Instead of calibrating 2 measures (errors and monthly driving) per driver, I think I will calculate error/mile and calibrate that variable.
I don't know if you will be willing to reset this model for me. I'm trying to start over, but it's not allowing me.
I got the attached when I clicked on "simulate", I don't know why it's not running

sorry I forgot to attach the model. I'm unable to reset it to start the calibration over

This isn't a model issue. The data file could be missing or corrupted, or the operating system could be locking it for some other process.

The file it's attempting to load is designated in the Data Source field of the Advanced tab of the Simulation Control dialog.

And what should I do? Please, advise. I've restarted VENSIM several times and I'm not even able to do a simple run.
Thanks

leon wrote:And what should I do? Please, advise. I've restarted VENSIM several times and I'm not even able to do a simple run.
Thanks

Tom has already told you,
"The file it's attempting to load is designated in the Data Source field of the Advanced tab of the Simulation Control dialog."

The file has probably been deleted, renamed, corrupted or locked by another process. You need to fix that - either repoint the Data Source field at the correct file (if it still exists) or restore the file from a backup (if it's damaged). Failing that, you can reimport the data or perhaps re-download the file that was posted here. But you should always have a backup.

Thanks for all your help. The file was probably corrupted. I deleted the file and downloaded a file posted here.
I was trying to calibrate errors/mile since calibrating one variable provided me a good fit before. But I didn't get good results.
1) Do you think I still need a poisson distribution when calibrating errors/mile? Or will a normal distribution better?
2) Do I need a weight other than 1 since errors/mile is very very small?
3) Do you think calibrating one measure (errors/mile) is better than calibrating 2 measures (errors per month and monthly driving)
Thanks so much for the time and efforts you have been putting to assist me

1) The distribution will not be Normal - it's Poisson/Normal. I have no idea what that yields, but it's probably asymptotically normalish, so I'd use that.
2) For a single measure, the weight doesn't matter for the central estimate (it only affects confidence bounds), unless you think the measurement error varies from point to point or driver to driver.
3) I think 2 measures is better. If calibrating with errors/mile gets you a better fit of errors & miles separately too, there's something strange going on.

What are your thoughts about sample size for my work? Since I'm trying to calibrate per driver, should I be concerned about sample size?
What are the metrics to make the model believable?

I think the only way to determine if the sample size is adequate is to check your confidence bounds on parameters. I don't know offhand what the expected values are for the Poisson distribution. Probably the simplest thing to do would be to use MCMC. https://www.vensim.com/documentation/mcmc_sa.htm

Thank you so much for your help Tom!
I actually have more drivers that I can add. So, if I fit the model to all the drivers' data, what do you suggest to validate the model, especially since the model is fitted to a set of drivers?. Should I compute the R-statistics for each driver and average across the drivers?

If you take a look at the payoff report that gets generated when you optimize, you already have a lot of stats. It's in <runname>.rep (tab delimited).

Another thing worth looking at is the parameter distributions. Presumably things like "driving need" are personal to each driver. But you'd hope that "retention time," "effect of cumulative miles" etc. would be fairly consistent across drivers. If those parameters are all over the place, that's a red flag.

Thanks a lot. I will take a look at those parameters across the drivers.

Thanks Tomfid.
If it's possible, would you be willing to take a look at the model and make me some simple suggestions? There is a lot of floating errors and I'm getting frustrated with it. The constants (retention time, event fading time, etc.) do not seem similar across the drivers and I'm worrying about it. I have loaded enough drivers in the model now
I appreciate your time and efforts you have putting into guiding me on this work. I've attached the model

Can you zip up the whole project, with the data and optimization files, and upload that?

Thanks. I've attached the file.
I have the data by gender, but I thought it would not make any difference to calibrate by gender group. Unless you think otherwise.
Thanks so much for your help, I'm so appreciative.

Hi Tomfid, I just wanted to follow-up with you, I hope you did not forget about me.
Thanks for all your help

Sorry, my travel schedule has been too busy.

I think there's a risk here of overfitting. It appears that several of the parameter vectors are bimodally distributed, with a lot of extreme values. You essentially have a unique parameterization of the model for each driver. I think that's a valuable experiment, but I'd want to try versions with fewer parameters.

Driving need, initial event rate, baseline driving and baseline memories probably should remain subscripted by [DriverID]. For the others (especially the 3 effect...s and retention), I'd try estimating a single value (remove the subscript) and see how well you can do.

The fits will degrade (they look pretty good now), but I think the reliability of the estimates will be substantially improved.

Thanks so much.
I removed the subscripts from the 3 effects, retention time, fading time. I got an error, please find attached what I got.
Thanks

leon wrote:Thanks so much.
I removed the subscripts from the 3 effects, retention time, fading time. I got an error, please find attached what I got.
Thanks

The error is displayed in your screen shot. In your optimization control file, "EFFECT OF CUMULATIVE MILES DRIVEN ON EVENT PER MILE" is not subscripted, but it is in the model. They need to match. You need to review your optimization files.