I have a SD model that can be used for management control and tactical/strategic planning etc. by a large number of actual instances of the one system type. Differences between those instances are believed to be confined to their input data.
The question arises: for how many instances should the model be validated, in order to have its predictive accuracy accepted and respected? One, two, all?
Please provide guidance, and reasons for that guidance.
Thank you in advance.
John P Weldon
Model validation
-
- Member
- Posts: 39
- Joined: Sun Oct 07, 2018 7:56 pm
- Vensim version: DSS
Re: Model validation
I would always use every bit of data available (subject to cost constraints). You'll almost always learn something from the incremental case. That might depend somewhat on what a "large number" is.
As to what constitutes "validation," that should involve much more than data. I'm usually not impressed by fit metrics, unless I've first seen robustness tests, dimensional consistency, etc.
As to what constitutes "validation," that should involve much more than data. I'm usually not impressed by fit metrics, unless I've first seen robustness tests, dimensional consistency, etc.
/*
Advice to posters (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Blog: http://blog.metasd.com
Model library: http://models.metasd.com
Bookmarks: http://delicious.com/tomfid/SystemDynamics
*/
Advice to posters (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Blog: http://blog.metasd.com
Model library: http://models.metasd.com
Bookmarks: http://delicious.com/tomfid/SystemDynamics
*/