Rainfall probabilities

Use this forum to post Vensim related questions.
rdudley
Senior Member
Posts: 68
Joined: Mon Sep 08, 2003 2:16 am

Rainfall probabilities

Post by rdudley »

I have a small model that uses simulated rainfall as an input. Units are months. This feeds into soil moisture information. The soil dries out if there is no rain for period and that affects crops etc.....

I have one lookup that provides the AMOUNT of rainfall based on monthly averages and another look up that provides a probability that rainfall actually occurs at at any given time.

This second probability is compared to a random uniform 0 to 1 distribution to determine if rainfall occurs or not.

It works OK but.... this approach is very dependent on time step and I prefer to have models that are not dependent on time step in this way. I should be able to alter the time step within reason and get, more or less, the same results.

How might I create the same effect without the outcomes changing so much with time step?
bob@vensim.com
Senior Member
Posts: 1107
Joined: Wed Mar 12, 2003 2:46 pm

Post by bob@vensim.com »

Hi Richard,

The obvious solution is to use pink noise instead of white noise for generation of the random number (see Sterman Appendix a or b). Pink noise is normally distributed so you will need to accommodate that somehow.

An alternative, which might work, would be

cumming noise = INTEG(incoming noise - old incoming noise,mean value)
old incoming noise = DELAY FIXED(incoming noise,cumming interval)

this will retain the distribution but still give the noise some momentum. If you experiment with this let me know how it works out, I have never tried it before and am curious.
LAUJJL
Senior Member
Posts: 1421
Joined: Fri May 23, 2003 10:09 am
Vensim version: DSS

rainfall probabilities

Post by LAUJJL »

Hi Richard

If I am faced with your problem, I notice first that I must use a time step that is less that the inverse of the amount of rainfall. For instance if the amount is 10 a month, the time step must be less than 1/10. to allow at least 10 steps in the month. If the time step is 1/8, it is not possible to generate more than 8 rain falls, because there is only one or zero rain fall happening in a time step.
Providing this limitation I use the model joined that works well provided the above limitation. The difference in the model is near 0, and if the limitation is not respected it is not.
Regards.
JJ

[Edited on 20-3-2010 by LAUJJL]

[Edited on 20-3-2010 by LAUJJL]

[Edited on 20-3-2010 by LAUJJL]
Attachments
rain fall.mdl
(3.6 KiB) Downloaded 306 times
rdudley
Senior Member
Posts: 68
Joined: Mon Sep 08, 2003 2:16 am

Post by rdudley »

Bob:

I am using pink noise with the first lookup where the input is the average rainfall per month and this gives a nice variation in rainfall amount.

The second lookup determines if the is rain or not by comparing an underlying prob of rain with a random draw between zero and one. In a sense I am using time step as the time period over which rain might or might not occur, so I guess if I change time step I shouldn't be surprised that I get a different result. But I think your idea might work so I will give it a try. It will smooth out the "rain or not" probability draw so the results won't jump all over, and that is probably realistic for most situations.

Another related question is weather or not to adjust the rainfall amount per incident upward by the inverse of the probability of it occurring.

Richard
bob@vensim.com
Senior Member
Posts: 1107
Joined: Wed Mar 12, 2003 2:46 pm

Post by bob@vensim.com »

I am not an expert in rainfall, but certainly if your first variable is intended to represent the rainfall in a month, you need to adjust that for the fraction of time it is raining to bring it up to that level. I would not be inclined to do this on a per incident bases but instead just adjust up from the average by the expected faction of time it is raining (or more accurately not raining).
LAUJJL
Senior Member
Posts: 1421
Joined: Fri May 23, 2003 10:09 am
Vensim version: DSS

rainfalls

Post by LAUJJL »

Hi Richard

your problem has been puzzling me.
joined a complete model of your problem.
The probability of something happening in a period is obviously depending on the length of the period.
If something is happening 10 times in a month, it will happen 10/30 time in a day and approximately 10/4 in a week.
So what is important is to know how much rainfalls happen every month and then to generate raifalls in every time step with a probability depending on its length.
I have added the total amount of rainfall per month, but it obviously has no importance in the generation of the rainfalls.
Regards.
JJ
Attachments
rain fall4.mdl
(7.2 KiB) Downloaded 278 times
LAUJJL
Senior Member
Posts: 1421
Joined: Fri May 23, 2003 10:09 am
Vensim version: DSS

rain fall

Post by LAUJJL »

I am trying to put pink noise in my model that is white noised the way Bob adviced to do, but it is not evident. I tried to smooth the random generator but they can get over 1 or less than 0 and it has no more meaning.
I will try to smooth the random value generated. But as they are more or less discreet zero or one, it is not evident either. I will think it over to-morrow.
JJ
LAUJJL
Senior Member
Posts: 1421
Joined: Fri May 23, 2003 10:09 am
Vensim version: DSS

rain fall

Post by LAUJJL »

Here is joined a pinked loised model, using Bob's moving average. Not being able to work on the discreet values, I have been obliged to work on the moving averages of rainfalls over a certain delay time.
The result displays always a very small difference.
When one looks at the standard deviation of the moving average initial and the corrected one sees that the corrected deviation is greater than the initial. It may be due to the persistance of high deviation when they occur.
It would be good to have a way to measure if this way of calculation has effectively generated a more smooth generation of rainfalls.
I have no idea yet how to do it. I will think about it.
Regards.
JJ
Attachments
rain fall6.mdl
(12.45 KiB) Downloaded 254 times
bob@vensim.com
Senior Member
Posts: 1107
Joined: Wed Mar 12, 2003 2:46 pm

Post by bob@vensim.com »

I took another look at this an my suggested solution using DELAY FIXED does not work. This is obvious when I think about it since the integral is a collection of independent identically distributed random variables which means it moves toward normal (but won't get there since it is a finite window). Generating uniformly distributed serial correlated noise is a bit tricky it seems.
rdudley
Senior Member
Posts: 68
Joined: Mon Sep 08, 2003 2:16 am

Post by rdudley »

Yes. It looks like I will have to select a given time step and work with that.

The problem is that I need a random draw of ones and zeros to allow for days without rain which is an important consideration. But the random draws come at each time step, not really a problem except that the number of draws and random sequence over months will change if the size if the time steps changes.

The delay approach doesn't solve this problem because the delay also reflects the added draws from the random sequence.

Is there a way to adjust the number of draws downward if the time step is made smaller? In other words to keep the number of rainfall incidents within a month to say 30?

Richard
LAUJJL
Senior Member
Posts: 1421
Joined: Fri May 23, 2003 10:09 am
Vensim version: DSS

rainfalls

Post by LAUJJL »

A seemingly much better model, that now reduces the standard deviation and has always a small difference.
Regards.
JJ
Attachments
rain fall7_8.mdl
(13.95 KiB) Downloaded 259 times
bob@vensim.com
Senior Member
Posts: 1107
Joined: Wed Mar 12, 2003 2:46 pm

Post by bob@vensim.com »

Hi Richard,

Assuming the model is in days you can use

sampled random=IF THEN ELSE(Integer(Time) = Time,RANDOM UNIFORM(0,1,0),prev)
prev=INTEG((sampled random-prev)/TIME STEP,0.5)

so the noise draw is a constant during each day. This works with Euler Integration and a Time Step that is a power of 0.5.
LAUJJL
Senior Member
Posts: 1421
Joined: Fri May 23, 2003 10:09 am
Vensim version: DSS

rainfalls probabilities

Post by LAUJJL »

The last model includes structural parameters that vary the structure of the model depending on their values.
For instance the parameter K persistance measure the level of autocorrelation of rainfalls. The best thing is to simulate with varying parameter values and try to understand why the behaviour changes. It shows that it is easy to build a model especially when one has no knowledge of the subject, that there are many possibilities to represent reality and that it is much more difficult to analyze its behaviour.
The results should be compared with reference modes to at least reject some parameter combinations or do some calibration.
JJ
Attachments
rain fall7_11.mdl
(14.8 KiB) Downloaded 252 times
tomfid
Administrator
Posts: 3804
Joined: Wed May 24, 2006 4:54 am

Post by tomfid »

Another sampling approach, simpler than pink noise, is:

rainfall = rainfall amount*rainfall occurs
rainfall occurs = IF THEN ELSE( rainfall random input > rainfall prob, 1, 0)
rainfall random input = SAMPLE IF TRUE( PULSE TRAIN(...) > 0, RANDOM UNIFORM( ... ), RANDOM UNIFORM( ... ) )
rainfall amount = RANDOM SOMETHING ( ... )

That corresponds with the idea that rainfall episodes have fixed duration, consistent with the PULSE TRAIN interval.

The PINK NOISE approach would permit variable duration of rainfall episodes, but then you'd have to specify the covariance of duration and amount, which is not likely to be available.

I'm also not a rainfall expert, but I would guess that the distribution of rainfall amount is not normal.

Tom
rdudley
Senior Member
Posts: 68
Joined: Mon Sep 08, 2003 2:16 am

Post by rdudley »

Here is a little model that compares these approaches (I think). I don't see much benefit to Tom's approach, but maybe I don't understand it correctly :-)
rdudley
Senior Member
Posts: 68
Joined: Mon Sep 08, 2003 2:16 am

Rainfall probabilities

Post by rdudley »

Well, the model didn't seem to get posted. I will try again.
Attachments
rainfall testing 2.mdl
(15.39 KiB) Downloaded 213 times
LAUJJL
Senior Member
Posts: 1421
Joined: Fri May 23, 2003 10:09 am
Vensim version: DSS

rainfalls

Post by LAUJJL »

Hi Richard

I have taken the liberty to correct the unit errors, mainly the simulatneous use of year and month units. I have added the variable months per year, to allow the use of the unit year if one wants to do it.
I hope it does not change the model fundamentals.
Joined the correction.
More to follow.
Regards.
JJ
Attachments
rainfall_testing_3.mdl
(16.09 KiB) Downloaded 196 times
LAUJJL
Senior Member
Posts: 1421
Joined: Fri May 23, 2003 10:09 am
Vensim version: DSS

rainfalls

Post by LAUJJL »

Hi Richard

There are some significant differences between the number 2 and 3 version corrected when one use the run compare tool. It would be good if you could see from where come the differences.
Regards.
JJ
LAUJJL
Senior Member
Posts: 1421
Joined: Fri May 23, 2003 10:09 am
Vensim version: DSS

rainfalls

Post by LAUJJL »

I have corrected the correlation time from 3 years to 3 months that sounds more logical and there are no more differences between the two versions.
Joined the corrected model
Attachments
rainfall_testing_4.mdl
(16.11 KiB) Downloaded 164 times
tomfid
Administrator
Posts: 3804
Joined: Wed May 24, 2006 4:54 am

Post by tomfid »

Looking at rainfall_testing_4, a few observations:

The Sterman pink noise structure isn't as good as Ed Anderson's version. I'll post the latter when I get a chance.

It's weird that "effect of drought" goes negative - seems unphysical.

The realized rainfall doesn't track the input lookup "rainfall pattern within year" because the "prob it rains on a given day" is generally < 1. If the rainfall pattern lookup is data to be tracked, that's a problem - you'd probably have to take pattern/prob as the input to the random draw in order to get the expected value of rainfall to line up.

The implementation of the SAMPLE IF TRUE version isn't quite what I had in mind - the sampling was intended as a replacement for pink noise, rather than an addition to it.

Tom
rdudley
Senior Member
Posts: 68
Joined: Mon Sep 08, 2003 2:16 am

Post by rdudley »

Tom and JJ:

Thanks for your observations, they are helpful.

I'd like to see the other pink noise model.

The name of variable effect of drought within year should be change to something like rainfall variability since it can increase or decrease rainfall. Within the definition of rainfall I make the minimum possible amount zero.

I am using the two random components because rainfall follows a pattern of rainfall amount over time e.g. rainy season, dry season and in between (where the pink noise has effect) and the probability it rains at any given time (e.g. day or time step). That is both the frequency and amount of rainfall changes.

In some models I suppose that including both would be unnecessary, but in this model it is important to have the possibility that soils dry out over the short term.

At present I am using very approximate values based on monthly rainfall averages, but later I might adjust this.

By the way this is, hopefully, a part of a bigger model to look at the possibility that widespread adoption of higher yielding maize varieties in dryer parts of Africa could sometimes lead to less grain production because people might be more likely to plant maize which does not do as well as alternate grains (sorghum, millet) in dry years. Thus the need for the reasonably realistic rainfall patterns. Of course I am trying to stick to a generic fairly simple model.
rdudley
Senior Member
Posts: 68
Joined: Mon Sep 08, 2003 2:16 am

Post by rdudley »

Just another brief comment:

The pink noise correlation time should have units=months and should use something equal to one month or less.

As I indicate above, I need to change the underlying patterns of rainfall amount and rainfall probability of occurrence. I expect I need to raise the amounts and lower the prob of occurrence. That is, more rainfall with fewer occurrences.
LAUJJL
Senior Member
Posts: 1421
Joined: Fri May 23, 2003 10:09 am
Vensim version: DSS

rainfalls

Post by LAUJJL »

Hi Richard and Tom

Remarks on rainfall version 4

When one consults the statistical tool from the analysis toolset one sees that the min effect of drought within year is negative as Tom noticed which does not make sense.
It still goes negative with the following model.
The LK rainfall pattern within year goes from 0 to 12. I do not see the necessity to have the 12, because the month of December is represented by 11 and modulo (time,12) is never equal to 12.
The 12 having a low number, it does not change the overall distribution but if it was higher it would.
Same for the LK prob it rains on a given day.
The chance of rain at point in time should be transformed from
IF THEN ELSE(random draw zero to one<prob of rain, 1, 0)
To IF THEN ELSE(random draw zero to one<prob of rain* time step * 32, 1, 0)
This way it will not be dependant from the time step. 32 because the actual time step is 1/32 and the lK prob it rains on a given time step is based on that time step.
It is too important to verify the other equations to eventually change them accordingly.
But if you do not intend to modify the time step, it can work without changing it.
I do not understand the need of the PULSE TRAIN( INITIAL TIME, width, interval, FINAL TIME ) > 0, RANDOM UNIFORM(0,1,123), RANDOM UNIFORM(0,1,123) for the chance of rain at point in time 2
If you replace it by UNIFORM(0,1,123) it gives exactly the same result the pulse train being always equal to 1 then positive.
The result of the two random approaches are slightly different because the number generated even with the same random 123 are slightly different but the cumulative result is roughly the same.
There is another problem with the 4 version.
If you use the statistical analysis tool, one finds that the average rainfall is near 150 mm per month.
The total cumulated rainfall at the end of the 120 months is 12000 when it should be near 150 * 120 equal 18000. The error comes from the fact that the equation calculating rainfall is not correct.
The rainfall is the number of rains in the period multiplied by the amount of mm per rain divided by the time step because it measures a mm per month and not a mm per time step.
When one makes the correction one finds that the cumulated rainfall is near what is expected whatever the time step if one has made the above correction.
Here is the completed corrected model without any correction for the pink noise to come.
Regards.
JJ
Attachments
rainfall_testing_5.mdl
(16.88 KiB) Downloaded 172 times
tomfid
Administrator
Posts: 3804
Joined: Wed May 24, 2006 4:54 am

Post by tomfid »

I think a key question here is, what is the time time constant of the soil evaporation process? It seems like it'd be important to model rainfall at frequencies up to some cutoff given by that, and no more.
tomfid
Administrator
Posts: 3804
Joined: Wed May 24, 2006 4:54 am

Post by tomfid »

Another possible approach here would be to get some real daily rainfall data, and use that as a driver - that way you'd be sure not to miss any stochastic structure that you inadvertently failed to model.
Post Reply