leon wrote:Sorry, one more question.
I set the sensitivity to payoff value during the calibration (optimization) and I got the attached results. Are those confidence intervals around the parameter estimates 95%CI?
Thanks
The stat tools are showing means, median, std etc.; but I'm looking for model performance metrics (eg. R-squared). Please, see the file (summary statistics) I attached to you.
Thanks
Sorry, but which file has the payoff report? I went through the different outputs generated from the model calibration and I did not see the payoff report file.
Thanks
I've have been trying to make sense of the payoff report over the last several days, but I'm not getting it.
Do I need to do some computation to get the R-square or did I check the wrong box? Please help
Thanks
I've upgraded to the new version.
Could you please help me with the interpretation?
I don't know if I need to do some computation to get the R-Square for each subscript.
What do RMSE, Um, Us, Uc, MAE, MAPE, MAEoM mean?
Are there any cutoffs for the interpretation of the metrics? What can be considered high for MAPE?
I have a lot of undefined (NA) for the variable "events/month"
Dear,
I applied my simulated driver feedback intervention to the model. I simply increased the fraction of events known to 100% for each driver and recorded the simulated intervention data. I compared the simulated data to the actual feedback intervention. I used a statistical model for the comparison and adjusted by the fact that the 2 groups of drivers have different baseline event rates. Please, see attached the mean profiles over time for both the simulated and actual intervention. I was asked to offset this in the model. Is this possible? I don't know if I clearly posed the question
I don't know if this is feasible in Vensim. I don't know if I used the right term for "offset"
Basically let's say I have 2 drivers A and B with initial event rates of 0.1/mile and 0.5/mile respectively. Driver A received the actual treatment starting at Month 2 and I simulated the treatment on Driver B using my model (I simply increased the fraction of events known to 100% starting at Month 2 and recorded the simulated intervention data). Then my research question is: do the actual treatment and simulated treatment agree?
Statistically for the comparison, I need to account for the fact that both drivers had different initial event rates to begin with. So, I will throw my initial event rate into my statistical model.
I was asked why I adjusted for the initial event rate. Isn't that initial event rate in the dynamic model? So I'm confused
Is there anyway to account for this in Vensim?
I'm still not sure if you got the issue
You are correct. But you would agree with me that the trajectory of the actual treatment response on a driver with an initial event rate of 0.5/mile would not be the same as the trajectory of the actual treatment response on a driver with an initial event rate of 0.1/mile. That's why in statistics, the initial event rates should be accounted for if I want to compare the treatment response for these 2 drivers
Another way to think about this is:
The actual treatment was applied to a group of "good" drivers. They had a very low initial event rate.
My simulation was applied to a group of "bad" drivers. They had a high initial event event rate. I would not expect the mean profiles of these 2 groups of drivers to overlap because of the differential initial event rates. But I would expect them to have the same shape. That's why I adjusted for the initial event rates using statistical approach.