Using Statistics in Dynamics Models

[=?iso-8859-1?Q?Jean-Jacques_Laub]

Hi Jim

The 1,2,3,4 is what I effectively use with some modifications and it does
not annoy me at all. This is why models are wrong. And I am not sure that it
horrifies a statistician if you explain him that you have not the pretension
to certify that the A hypothesis is true and you use the 1,2,3,4 method with
the following precisions.

I think that most modellers are obliged to use the 1,2,3,4 method, having
nothing better.

To cope with the all models are wrong sentence, I do three things: I will
not use the word truth, I will carefully chose the B assumption and the
other assumptions specified in the 4.

That means that any B assumption is not automatically adapted, and only some
implications of A may be considered as true.

So what does a real statistician do when faced with a situation in
which he knows from first principles that the model is wrong?

I can reply to this question with some preliminary clarifications.

It is necessary to define the word true or wrong.

For me these words do not represent well the reality of the problems I am
trying to solve.

For the word true I would prefer adapted to the purpose.

For the word wrong not adapted to the purpose.

As nothing is perfectly adapted to any purpose I will use adapted to a
certain degree to the purpose. So it the degree is 0 it is not adapted at
all.

Then the problem I am faced with is not a wrong model but a model which I
want to verify that it is sufficiently adapted to a certain purpose. It is my
decision to fix the level of adaptation and the purpose.

To proceed further on I will take a concrete example, which I am presently
working on.

Suppose I want to reconsider the way I have been pricing a type of car in my
company a short term rent a truck, vans and cars company.

The purpose of the model is to increase the margin of the type of car for
the next 24 months.

It is my responsibility to fix the minimum percentage of increase. To resume
the purpose is to find a better method of pricing then the one currently
used.

If I have a finished model, I will for instance test different pricing
strategy and choose the one that has the best margin.

To be confident in the model, I will first verify that if I put the same
price applied for instance the last year, it will produce the real margin
experienced over the same period.

This is of course not the final purpose. I do not want to use that model to
prove that what happened could really happen. But I decide that if my model
is able to reproduce to a certain level the results of the past, it will
already be a first point. And I can assure you that it is not
evident, due to the number of different prices depending on the time in the
year, the kilometres per day, the duration of the renting, the kind of
customer, etc..

To verify this I can use any method I decide, for instance the classical
maximum likelihood
implemented in Vensim as I have a set of data, the real and the calculated
margin over a 12 months period.

It is too my responsibility and my experience to decide the level of
likelihood.

Of course I can verify other outcome from the model, but I can decide that
being concerned with the margin over a 24 months period, this test is
enough. This is where the purpose is important. I do not need a true model,
but a model that is adapted to a certain purpose.

If the level of adaptation is not high enough, then I have to correct my
model until it reaches the correct level of adaptation.

Once I have verified that the model can reproduce approximately the past, I
can decide that the test is sufficient enough to be used with different
prices than the one applied last year, and
observe the behaviour of the margin. I know that my model may be bugged,
because I tested the model with past data, and nothing proves that the model
is still adapted to the purpose with different prices. Suppose I test an
increase of price of 20%, my model may not describe the reaction of the
customers if I am currently pricing 20% more then the competition.

It is then up to me to decide if the first purpose is enough or not. If not
I will have to define a second purpose closer to the original one.

That kind of problem is automatic if you test a new policy. You have never
the real data that correspond to all the possible policies. It is then
necessary to believe that the policy calculated will produce sufficient
results.

I have in my case, a great chance. I can reproduce an experiment like in the
scientific word.

I can then be more confident on my model, by producing the data that
correspond to the new policy, to see if the reality is conform to the
prediction of the model.



I will have tested the model with old data and with the new data produced by
the effective application of the policy. This new testing using new data,
will of course be more adapted to the purpose which is to modify prices.



To resume, there is always a moment when some belief eventually irrational
is necessary to take a decision. The problem is to make the irrational part
as small as possible. But I can assure you that compared to the process I
used years ago (I prefer not to talk about it) this new method is much more
adapted to the purpose then the previous one, the only thing I care about.



And I have not written anything about truth.



Regards.



J.J. Laublé, Allocar rent a car company
From: =?iso-8859-1?Q?Jean-Jacques_Laubl=E9?= <JEAN-JACQUES.LAUBLE@WANADOO.FR>

Strasbourg France

[Alan Graham]

To perhaps reiterate Carolus Grutters point:

This reasoning doesnt follow:

Quote JJ.Lauble and Jim Hines :


>> If you can
>> prove A implies B with only a 5% chance of error. And, if you can prove that
>> B is not true, then youve proved that A is not true with only a 5% chance
>> of error.


If I get receipts enclosed ("B") 95% of the days I get a Fedex package ("A"),
not getting a receipt (not B) says little about whether I got a Fedex package.
Some people get them every day; others get them every two years if at all.

In probability terms, you need to know the unconditional probabilities Pr(A)
and Pr(Not B) before you can do anything with the conditional probability of
B given A (95%) and the observation of not B, through Bayes rule:

Pr(A|not B) = Pr(Not B|A)* Pr(A)/Pr(Not B)

cheers,

alan
From: Alan Graham <Alan.Graham@paconsulting.com>

[=?iso-8859-1?Q?Jean-Jacques_Laub]

Hi Carolus



There seems to be some atmosphere in that (too ?) serious list.



I wondered if I could demonstrate with your new LEGAL logic that my grand
mother was a cow.



I hope that you will excuse me: I am joking.



The assertion (A implies B) implies (not A implies not B) and reciprocally
is a basic

from the probability theory.



Another point:



Suppose that A stands for a cattle farmer in a certain area
Suppose also that B stands for a cow and that Not-B stands for a bull.



A is something that is supposed to be true or false.



Or a cattle farmer in a certain area is not susceptible to be true or false,
nor a cow

nor a bull.



So if you want to formulate a logic based on the a implies b implies not b
implies a,

you must formulate your assumptions so that they can be true or false.



Cheers.



J.J. Laublé, Allocar, rent a car company

Strasbourg, France
From: =?iso-8859-1?Q?Jean-Jacques_Laubl=E9?= <JEAN-JACQUES.LAUBLE@WANADOO.FR>

[=?iso-8859-1?Q?Jean-Jacques_Laub]

Hi Alan



I understand perfectly your argumentation and I agree.



The interest of the A implies B implies not B implies not A, is to reject a
hypothesis because there is a great probability that it is false. It is
particularly useful in SD, where the only thing possible is to reject a
model, while proving that it is right is impossible.



So the assertion that A implies B with a high probability implies that not B
implies not A with a high probability is used to reject a hypothesis and not
to prove that A is right.



In fact it seems that if A implies B with a 95% chance of success it implies
that not B implies no A with a chance of success greater or equal to 95%. I
will have to think about it this week end, my current job being light-years
away from this discussion. The fact that not A has more then 95% chance of
being true means that A has less then 5% chance of being true. This is what
explains your paradox. Somebody that receives currently a package every 10
years will not have a probability to receive one equal to 5% because he does
not get any receipt.

In fact the probability A of receiving the package is less then 5%, in that
case it will be less then 1%, if you survey things during one month even if
you do not take into account the eventual receipts. It explains too the
impossibility to prove by this method that A is right, but is can prove that
whatever the probability of A, if you can prove not B is true, then A will
have not more then a 5% probability to be true.

The fact that Prob (A) is less then or equal 0.05 and Prob (not A) is
greater then 0.95 proves that it is so easy to reject a hypothesis and why
it is impossible to prove it.



About Carolus problem it is different.



I will recapitulate the problem.



There is a population of farms and a population of cattle.



First not B does not stand for a bull, unless you suppose that A is true.



I suppose that Carolus wanted to say that if the cattle farmers have 19 cows
and 1 bull and the other farmers have only cows for instance, if I see a cow
it implies that A, the farm is a cattle farm, is not true. It is exactly the
contrary. Seeing a bull implies that you are in a cattle farm

because the other farms have only cows.

This works only if I am already in a cattle farm, otherwise being in any
farm I cannot use the assumption that there are 95% of cows and 5% of bulls.



But this fact does not prove that the assertion A implies B to a certain
extent does not imply that Not B implies not A to a certain extent.



The problem here is that if you choose a farm randomly and an animal
randomly you try to

deduce from the type of animal an information on the kind of farm. Or you
have only a conditional information, the distribution of cows and bulls in
the cattle farms. This information is not useful unless you have an
information on the distribution of the cattle

in all the farms.



I short A is not equal to if I choose a cattle farm and I choose an animal
AMONG ANY FARM it has a 95% probability to be a cow,

but if I choose a cattle farm and I choose an animal IN THE SAME KIND OF
FARM it has a 95% probability to be a cow which is very different.



It should be interesting to think about the SD case. I have not the time
right now.

When one proves that B is not true is the A is true probability equal to
0.05% or less then or equal 0.05%? True means that the case is adapted to
the results one wants to achieve.



I think that this question concerns the discussion in the list about the
subject currently discussed.



Regards.



J.J. Laublé Allocar, rent a car company
From: =?iso-8859-1?Q?Jean-Jacques_Laubl=E9?= <JEAN-JACQUES.LAUBLE@WANADOO.FR>

Strasbourg France