Model validation

Use this forum to post Vensim related questions.
Post Reply
Dr John P Weldon
Member
Posts: 39
Joined: Sun Oct 07, 2018 7:56 pm
Vensim version: DSS

Model validation

Post by Dr John P Weldon »

I have a SD model that can be used for management control and tactical/strategic planning etc. by a large number of actual instances of the one system type. Differences between those instances are believed to be confined to their input data.

The question arises: for how many instances should the model be validated, in order to have its predictive accuracy accepted and respected? One, two, all?

Please provide guidance, and reasons for that guidance.

Thank you in advance.

John P Weldon
tomfid
Administrator
Posts: 3806
Joined: Wed May 24, 2006 4:54 am

Re: Model validation

Post by tomfid »

I would always use every bit of data available (subject to cost constraints). You'll almost always learn something from the incremental case. That might depend somewhat on what a "large number" is.

As to what constitutes "validation," that should involve much more than data. I'm usually not impressed by fit metrics, unless I've first seen robustness tests, dimensional consistency, etc.
Post Reply