Reliability and validity along with reality check option

[LAUJJL]

Hi

The model is sent to show you that there are models with Reality Checks.

I cannot show the entire model and understanding it needs anyhow a good knowledge of the field and a lot of time to study it.

I join a .mdl version with its basic run needed for some RC compare commands that works better than the .vpd version intended for the player that does not seem to work probably because I erased most views from the model.

Regards.

JJ

[mdsdoha]

Hi LAUJJL

How you doing? Your file is massive for me to follow as I am a new in modelling. Could you please set one or two RC command within s simple model? It may help me to learn more clearly. Sorry for giving you little trouble.

Doha

Hi Msdoha

About your question on validity and reliability one must first define what is validity or reliability.

For me it means approximately the same thing. My definition will certainly not be the same as most other people definitionx, because I have never seen a published model that I consider as valid.
A model to be valid must mimic sufficiently well the system that you want to study, so as to give the possibility to solve the problem.

To make the model mimic sufficiently well the system I use massively Reality Checks that I build progressively step by step with the model, starting with a simplistic model first. I use of course unit checks using the strictest testing option available in model>setting>units equiv. Mass balances are performed in some of the Reality Checks.

The other way to valid the model is to use it sufficiently at each stage of the modeling process.

One should start with a simplistic model and add some more material to the model, only when one has fully understood the model previously built. What means ‘fully understood’. To test how deeply you have understood a model, you must be able to predict the general behavior of the model, whatever the value of the parameters used and without running the model. Of course when the model gets bigger, this is more and more difficult, but one should thrive towards this achievement and how well you perform is a good indicator of your understanding.

If you do this and in parallel build your Reality Check you will not only have a model that mimics well the reality, but you will understood how it works and be able to build sound policies.
A complex model that mimics well the reality butis not deeply understood may be totally useless.

So for me a model to be valid and reliable must be correct relatively to the reality studied but too be understood which is something different.

Coyle writes in his book the following: ‘The key to successful modeling is to keep one’s understanding of the model and what it says about the problem ahead of its size.’

This rule must be respected all along the modeling process and once you achieve it, you can go along and eventually add some more material to your model.
Of course all this has a price: the price of quality.

So when one talks about validity, one must consider the model and the person who uses it.
The model may be valid, but if the person using it does not understand it fully, the person and the model are not valid together.

Joined a paper by Peterson and Eberlein about Reality Checks. Unfortunately the last paragraphs do not make sense and are in contradiction with the independence of RC and the model advocated previously and are impossible to achieve.

I join too an example of a model I am building that has already about 50 RC and that will have certainly more than hundred once finished. I have erased all the views but the Reality Checks for confidential reasons. You can still run it and run the Reality checks.

Regards.

JJ

[LAUJJL]

Hi Doha.

The model I posted was not designed to be studied as all the sketches relative to the equations was erased for confidential reasons. The objective is to show an example of using plenty of RC on a middle sized model.

Did you study correctly the chapter on RC in the used guide?

The best way is to practice on subjects that you know well. I could send you smaller models using RC but to appreciate the use of RC it is essential to know well the system studied.
After having studied the chapter, try to build some RC on some simple models or yours, and if you cannot do it, I may help you.

I build my RC, step by step starting with a small model first and I add some new material only after having fully used and understood the smaller version. It is my way of working, and I do not know anybody working this way. So you must consider the real utility building RC. Why do you need them?

I build models for myself and I find an interest in using RC, but if people do not use RC, it is probably that it does not correspond to their interest. My first priority is to keep the model as simple as possible. A simpler model is easier to debug, understand, use and modify. The caveat is that it requires more time to build. It is easy to build a complex model and difficult and time consuming to simplify it to make it useful.

A better tool and easier to use may be using Sdm-doc. Type Sdm-doc in Google and download the tool. I think too that before using a tool like RC, it is better to learn to build small models, with well defined names, simple equations and easy to read and understand. If you can do that, you will not need RC. RC’s become necessary if you need more complex models where the probability of bugs becomes bigger.

Best regards.
JJ

[mdsdoha]

Hi LAUJJL

Thanks for your mail. In fact I do not need your model with equation. I need simple model. I have read through the chapter and article. I tried to build RC in my model, but I do not trace them whether they are working alright or not. Please see the below model which I get from Vensim guide. If you hvae time, please go to view 3 where I tried myself to do the same but did not give the same result like view 2. Moreover, I do not know the way of tracing them. Can you please provide me some tips to do so. So that I can let know whether model have really checked by reality check command.

Thanks for your time.

Doha

[LAUJJL]

Hi Doha

When you crun RC, the model makes multiple runs to test all the hypotheses.

To test if one of the relaity check works properly, click on the RC button, highlight the contraints that correpsonds to the RC you want to trace. It will auotomatically highlight the corresponding test inputs on the left side. Clck on the double arrow to slide the highlighted inputs to the active test inputs window. Then click on the test inputs in the active test inputs to hilight them.
You are then ready to launch the single RC that you have chosen. Click on sim active to run the single RC. It will test only the RC that you have chosen. The run will be stored under the run name you have currently chosen. You can then study the run for example 'current'.

Regards.

JJ