Model Validation

Use this forum to post Vensim related questions.
Post Reply
Sara T
Junior Member
Posts: 18
Joined: Thu Jun 07, 2018 7:18 am
Vensim version: PLE

Model Validation

Post by Sara T »

Hi
In order to learn how to test my model,I need tutorial files or videos about Model Validation in Vensim, including Extreme Condition, Reproduction ,Surprise Behavior and etc.
Please introduce some valid references.
Thanks
tomfid
Administrator
Posts: 3804
Joined: Wed May 24, 2006 4:54 am

Re: Model Validation

Post by tomfid »

Sara T
Junior Member
Posts: 18
Joined: Thu Jun 07, 2018 7:18 am
Vensim version: PLE

Re: Model Validation

Post by Sara T »

Thank you.
Sara T
Junior Member
Posts: 18
Joined: Thu Jun 07, 2018 7:18 am
Vensim version: PLE

Re: Model Validation

Post by Sara T »

Thank you for your all your help.
LAUJJL
Senior Member
Posts: 1421
Joined: Fri May 23, 2003 10:09 am
Vensim version: DSS

Re: Model Validation

Post by LAUJJL »

Are you working on real problems and on models that are intended to be used, or in an academic setting?
Sara T
Junior Member
Posts: 18
Joined: Thu Jun 07, 2018 7:18 am
Vensim version: PLE

Re: Model Validation

Post by Sara T »

I'm working on real problem in the field of sustainable tourism.
LAUJJL
Senior Member
Posts: 1421
Joined: Fri May 23, 2003 10:09 am
Vensim version: DSS

Re: Model Validation

Post by LAUJJL »

Hi Sara

Anybody wanting to build correct models, is concerned with model validation.

There are many papers mainly academic on that subject with structural testing, behaviour testing etc.. Vensim has a special feature called reality check to test models too.

The paradox is that if you study public models published, you never see anybody testing their models, and I have never seen a published Vensim model with reality checks. Once though with the climate interactive model C roads earlier version. The present version has no reality checks in it.

There is then a gap between the academic prescriptions and what is done in the real world, even if most models are academic.

I have an explanation to that fact, having tried to put tests in my models for years.

The problem with tests is that you must chose which tests to build, and after a certain size the number of possible tests become quickly overwhelming. This is probably why testing has never been used. I do not mean that it is not necessary to test models. Years ago, I worked with a consultant supposed to be an SD expert. I never worked with the model built by him, for some reasons. But years after, studying it again, I found a big error in an equation, that should have been discovered if mass balances checks had been correctly made. At the heart of the model was vehicle availability in a rent a car business. The model was declaring not available, vehicles that were available. The reason was an error in an equation, easily corrected.

I have learned with the time to limit these tests to a minimum, mainly mass balances and a limited number of tests that I judge mandatory.

I replace these tests by trying to improve my understanding of the model, following the Coyle's advice: 'the key to successful modelling is to keep one's understanding of the model and what it says
about the problem ahead of its size'.

It is very easy to build a model that one is not able to manage and that has no practical utility.

The way I verify that I have not built a model out of my understanding and ability to use possibilities?

A model has generally parameters. I verify that every changeable parameters has lower and upper bounds. I build graphs from the main model variables (not to much). I start simulating with synthesim or not, around the centre of the parameter interval. For instance 20% above the centre, and I verify if I am able to predict the behaviour of the main variables. If I was wrong, something that happens very often, I try to understand why. If I am obviously too often wrong, I think about
simplifying first the model, until I am able again to understand the full model.

When I have done that with every parameters, I can too use the synthesim, and override the values of the parameters by special behaviours like ramp, pulse etc… And at last, it is too possible to override the behaviours of non constants variables. If you do this systematically, you will experiment full control of your model. Of course past a certain size it is not possible to work as the computer, but having experimented with previously smaller models, will give you the certitude of working with models validated by yourself.

You can do the same for validating the model's structure.
The structure is the causal loop diagram of the model with of without the stocks and flows represented. If you have a doubt about the validity of this structure and there are other possible structures, it is often possible to represent different models structure with one model only and structural parameters that will modify the structure by modifying the structural parameters values.

The structural parameters are generally discreet, representing different structures, but they can be continuous if it is possible to move from one structure to another in a continuous fashion.

You can then experiment with these structural parameters the same way as with the ordinary ones.

Of course I am too very strict about unit testing, and use the strict testing option available in the model settings units equivalent window. (available too in Vensim Ple).

Using correct and significant variable names is important, and following Richardson's rule too: limiting the number of variables or constants influencing another variable too make the equation defining the variable easy to understand.

This all that I have learned from years of practice.

Best regards.

JJ
Sara T
Junior Member
Posts: 18
Joined: Thu Jun 07, 2018 7:18 am
Vensim version: PLE

Re: Model Validation

Post by Sara T »

Thanks for guiding me on the right path. Certainly, the tips you have taught me will be of great help to me.
LAUJJL
Senior Member
Posts: 1421
Joined: Fri May 23, 2003 10:09 am
Vensim version: DSS

Re: Model Validation

Post by LAUJJL »

Hi Sara

There is another way to verify a model. It is to verify step by step all the variables calculated, starting by the beginning of what the program calculates. See the order of calculation in the vensim documentation, but it is rather evident. It starts by initializing the levels and then looks at the parameter influencing other variables.

Joined a simplistic example taken from the user guide, that shows the three verifications that must be done. It is too very useful for the understanding of how the model works.

Best regards.

JJ
Attachments
example_POP.mdl
(3.35 KiB) Downloaded 237 times
Sara T
Junior Member
Posts: 18
Joined: Thu Jun 07, 2018 7:18 am
Vensim version: PLE

Re: Model Validation

Post by Sara T »

Hi JJ
Thanks for your tips.
I do not know how to put the boxes(time, initial rabbit population ,...) on the right of the model?
LAUJJL
Senior Member
Posts: 1421
Joined: Fri May 23, 2003 10:09 am
Vensim version: DSS

Re: Model Validation

Post by LAUJJL »

Hi Sara

I do not know which boxes you are talking about.

I have joined a second model, where the constants have a lower and upper bound and with sliders.

If you run the synthesim in Vensim PLE, you will be able to slide the sliders between their bounds and you will see the values in the tables on the right of the model changing accordingly.

Can you run the model and see the tables on the right of the model?

Regards.

JJ
Attachments
example_POP_2.mdl
(3.67 KiB) Downloaded 211 times
Sara T
Junior Member
Posts: 18
Joined: Thu Jun 07, 2018 7:18 am
Vensim version: PLE

Re: Model Validation

Post by Sara T »

Yes, thanks, I've run the model now.
I mean, how can i add tables in my model? by input/output object?
LAUJJL
Senior Member
Posts: 1421
Joined: Fri May 23, 2003 10:09 am
Vensim version: DSS

Re: Model Validation

Post by LAUJJL »

there are already three tables in the model that show the behaviours of all the variables.

You can generally add common or custom tables using the input/ouput oject. if you want to build custom tables you must go to the control panel and graphs and then build a custom table. You can too modify the existing ones in particular the three existing tables.

If you are new to that, you must study some of the Vensim user guide chapters available in the help button.

Vensim is not an easy to use tool and it requires studying seriously the user guide, at least the chapters relating to the Vensim PLE.

The Vensim user guide is very well done.

Regards.

JJ
Sara T
Junior Member
Posts: 18
Joined: Thu Jun 07, 2018 7:18 am
Vensim version: PLE

Re: Model Validation

Post by Sara T »

Thank you so much
Post Reply