Hi,
Is this a good modeling practice? To do Reality check, I first cut the model into pieces, and saved them as separate files. I fear running the Reality Check will somehow alter the original model structure.
Monte
Using Reality Check
Using Reality Check
Last edited by Monte on Sat Sep 19, 2015 4:06 am, edited 1 time in total.
reality checks
Hi
This is very good practice.
To my opinion, reality checks should be defined before the model is built. One should write the equations with a sort of pseudo language or even with the Vensim language, knowing that you cannot test them without the corresponding model being written.
Defining the reality checks before one makes the model, permits to think about the reality of the problem before making the model and that is good practice and saves a lot
of time, although looking a waste of time at first.
Reality checks can be used even with a static model, a model with no stocks and only with an influence diagram, running on two periods, the results of the two periods being the same.
The reality checks does not change anything in your model.
Using reality checks needs practice like modelling, but it pays
in the long term.
Reality check makes Vensim to my opinion the best SD software on the market.
Regards.
JJ
This is very good practice.
To my opinion, reality checks should be defined before the model is built. One should write the equations with a sort of pseudo language or even with the Vensim language, knowing that you cannot test them without the corresponding model being written.
Defining the reality checks before one makes the model, permits to think about the reality of the problem before making the model and that is good practice and saves a lot
of time, although looking a waste of time at first.
Reality checks can be used even with a static model, a model with no stocks and only with an influence diagram, running on two periods, the results of the two periods being the same.
The reality checks does not change anything in your model.
Using reality checks needs practice like modelling, but it pays
in the long term.
Reality check makes Vensim to my opinion the best SD software on the market.
Regards.
JJ
Hi JJ, thanks for your help. Vensim is the best in my opinion too.
To do reality check, I think the number of reality check equations has receive too little attention. This is crucial, as it might affect the virtue of the test, just as ones building a model of man body:
To do reality check, we know that 'no head, no eyes' is true. So, we cut the head in the model, and see no eyes; the model passes this reality check. However, models of woman, boy, or even monkey body also pass the same test equally well.
Do you think it is hard to define the number of reality check equations we need to write?
Monte
To do reality check, I think the number of reality check equations has receive too little attention. This is crucial, as it might affect the virtue of the test, just as ones building a model of man body:
To do reality check, we know that 'no head, no eyes' is true. So, we cut the head in the model, and see no eyes; the model passes this reality check. However, models of woman, boy, or even monkey body also pass the same test equally well.
Do you think it is hard to define the number of reality check equations we need to write?
Monte
-
bob@vensim.com
- Senior Member
- Posts: 1107
- Joined: Wed Mar 12, 2003 2:46 pm
First, it is important to emphasize, Reality Check equations absolutely do not interfere with regular model equations. The way they are constructed and impmented when you use them they need the model equations but the model equations do not need them. Thus the simulation of the model is exactly as it is without the added statements.
As to how many, I have had a number of long discussions on this. Conceptually, if you were to make a matrix of every meaningful model variable, including a row for every constant/exog variable but columns only for endogenous variables you should be able to fill in lots of those relationships, some with multiple entries (go up/go down and so on). Thus if the model has 20 variables and 10 constants you would have 30*20 or 600 relationships and if both up and down were included what would give 1200 Reality Check Statements. That is a very big number and in practive I have never seen anything like that.
Part of this is legitimate - there is some sparsity in the relationships. Part of it is almost legitimate - if a price decrease causes revenue to decrease, and a revenue decrease causes net revenue to decrease, why test the connection between price and net revenue. The trouble is, sometimes what seem like obviously true stuff ends up not being so. But I guess that is always lurking in many things that we do.
As to how many, I have had a number of long discussions on this. Conceptually, if you were to make a matrix of every meaningful model variable, including a row for every constant/exog variable but columns only for endogenous variables you should be able to fill in lots of those relationships, some with multiple entries (go up/go down and so on). Thus if the model has 20 variables and 10 constants you would have 30*20 or 600 relationships and if both up and down were included what would give 1200 Reality Check Statements. That is a very big number and in practive I have never seen anything like that.
Part of this is legitimate - there is some sparsity in the relationships. Part of it is almost legitimate - if a price decrease causes revenue to decrease, and a revenue decrease causes net revenue to decrease, why test the connection between price and net revenue. The trouble is, sometimes what seem like obviously true stuff ends up not being so. But I guess that is always lurking in many things that we do.
Bob's note is exactly another fact I am looking for. The logic behind the Reality Check Index is only briefly defined in the Guide (p. 209), and become so clear today. Thanks.
I see in one famous SD book, Reality Checks is simply classed as extreme condition tests. I think the checks deal with all conditions, including non-extreme events too. Another virtue of the check equations is interesting. We can use them to evaluate the performace of the system. For example, we want to see if people under an emergency event can walk (evacuate) to a given place within 25 minutes, we can use Reality Check equations to test this. We may write,
RC safe evacuation: THE CONDITIONS: TIME=25: IMPLIES: distanced travled>=TOTAL DISTANCE
However, I see a limitation of Reality Checks. The checks deal with equations (links), not a whole model (loops); they cannot detect any omitted loops that are essential in generating the reference modes. The valid model in terms of reality checks may not represent the particular system being modeled. The solution is using multiple tests.
I see in one famous SD book, Reality Checks is simply classed as extreme condition tests. I think the checks deal with all conditions, including non-extreme events too. Another virtue of the check equations is interesting. We can use them to evaluate the performace of the system. For example, we want to see if people under an emergency event can walk (evacuate) to a given place within 25 minutes, we can use Reality Check equations to test this. We may write,
RC safe evacuation: THE CONDITIONS: TIME=25: IMPLIES: distanced travled>=TOTAL DISTANCE
However, I see a limitation of Reality Checks. The checks deal with equations (links), not a whole model (loops); they cannot detect any omitted loops that are essential in generating the reference modes. The valid model in terms of reality checks may not represent the particular system being modeled. The solution is using multiple tests.
-
bob@vensim.com
- Senior Member
- Posts: 1107
- Joined: Wed Mar 12, 2003 2:46 pm
Reality Checks are behavior to behavior relationships - because they deal with behavior there is no distinction between cause and effect and feedback relationships. For example it would be perfectly legitimate to say the an increase in product attractiveness would increase the rate of growth in product sales. This would happen because of a feedback loop that is being strengthened. However, most things that are well understood about the real world, and therefore a basis for Reality Checks, do not involve loops - people rarely understand the feeback around them. It is only the complete set of statements that really gets to feedback.
There is nothing wrong with using a Test Input simply to study model behavior as you describe. Often, though, you can do the same thing in SyntheSim by overriding a varable and this is easier to do.
As far as extreme condition tests go you are right, these are a subset of Reality Checks.
There is nothing wrong with using a Test Input simply to study model behavior as you describe. Often, though, you can do the same thing in SyntheSim by overriding a varable and this is easier to do.
As far as extreme condition tests go you are right, these are a subset of Reality Checks.
reality checks
Hi
The fact that reality checks do not give any prescription about how to correct any
dysfunction in the model is in fact a good thing.
It proves that reality checks have nothing to do with SD or with any peculiar method used.
The principles of reality checks is independent of the method and comes directly from
the study of the reality. If reality checks were depending on the SD paradigm you
could distrust them if you were contesting the SD paradigm, this is why reality checks are
powerful.
About the number of reality checks, it depends on the kind of problem and the amount of knowledge bout the reality modelled.
The ideal would be to describe all the knowledge you have about the problem in question.
I am personally against any preconceived idea.
So I do not fix any number of reality checks to write, but try to start with the most important first. I write both extreme conditions test and current no extreme tests.
Sterman writes about the extreme tests but omit the non extreme ones which are even more important than the extreme ones.
Writing reality check equations is not trivial and needs a lot of practice.
It has already helped me to find bugs and helped me to better define the problem studied.
There was some months ago a discussion about reality checks that you can find on this forum.
Regards.
JJ
The fact that reality checks do not give any prescription about how to correct any
dysfunction in the model is in fact a good thing.
It proves that reality checks have nothing to do with SD or with any peculiar method used.
The principles of reality checks is independent of the method and comes directly from
the study of the reality. If reality checks were depending on the SD paradigm you
could distrust them if you were contesting the SD paradigm, this is why reality checks are
powerful.
About the number of reality checks, it depends on the kind of problem and the amount of knowledge bout the reality modelled.
The ideal would be to describe all the knowledge you have about the problem in question.
I am personally against any preconceived idea.
So I do not fix any number of reality checks to write, but try to start with the most important first. I write both extreme conditions test and current no extreme tests.
Sterman writes about the extreme tests but omit the non extreme ones which are even more important than the extreme ones.
Writing reality check equations is not trivial and needs a lot of practice.
It has already helped me to find bugs and helped me to better define the problem studied.
There was some months ago a discussion about reality checks that you can find on this forum.
Regards.
JJ