Number of replications

This forum contains all archives from the SD Mailing list (go to http://www.systemdynamics.org/forum/ for more information). This is here as a read-only resource, please post any SD related questions to the SD Discussion forum.
Locked
Joel Rahn
Junior Member
Posts: 15
Joined: Fri Mar 29, 2002 3:39 am

Number of replications

Post by Joel Rahn »

You are dealing with confidence intervals. A good simulation text like
Law & Kelton will give you an answer with the usual caveats about using
such intervals.

A quick answer is: no. of replications = Variance*
(z(alpha/2)/half-interval of confidence)^2
where:
z(alpha/2) is the appropriate Normal distribution factor corresponding
to the level of confidence you want (at 95% the z is 1.96),

half-interval of confidence is half the width of the interval of
confidence that you want

Variance is the (usually estimated) variance of your observation.

Essentially, you set the half-interval of confidence (the desired
precision of your estimate of the expected value of your population)
equal the z-factor times the standard deviation of the average of
independent observations drawn from a Normal population.

Joel Rahn
jrahn@sympatico.ca
Bob Eberlein
Member
Posts: 49
Joined: Fri Mar 29, 2002 3:39 am

Number of replications

Post by Bob Eberlein »

Hi Ric,

The answer depends very much on what you are trying to do by introducing
noise as well as the model you are using. If you are looking at steady
state statistics then take a look at the textbook meantioned in the
other posts. There is a caveat on that, however. If you are dealing with
models for which the statistics you want are for low probability events
you need an amazingly large (millions) number of replications.

If your only purpose is to understand what range of behavior a model can
generate than the answer is likely to be more modest (hundreds). Unfortunately,
with nonlinear systems there is never any guarantee of coverage - it is not
hard to write an equation that have odd behavior only an arbitrarily small
fraction of the time. This means that there is always a chance that something
will lie dormant till the 10 millionth simulation. In practice, however,
a good protocal is to start with a reasonable number, like 100, then double
it to see if something new emerges. If it does, you need to keep increasing.

If there is one rule of thumb, it is that it always takes more replications
than it seems like it should. Fortunately, computers are getting so fast that
this is usually practical.

Bob Eberlein
bob@vensim.com
"Ray on EV1"
Member
Posts: 29
Joined: Fri Mar 29, 2002 3:39 am

Number of replications

Post by "Ray on EV1" »

Joel gave a very good summary. For free, try a google search on design of
experiment:
http://www.google.com/search?as_q=&num= ... btnG=Googl
e+Search&as_epq=Design+of+experiment&as_oq=&as_eq=&lr=&as_ft=i&as_filetype=&
as_qdr=all&as_occt=any&as_dt=i&as_sitesearch=&safe=images

Raymond T. Joseph, PE
RTJoseph@ev1.net

Aarden Control Engineering and Science
Locked