Random Normal

[nhoward]

Can you tell me what the Random Normal function does with values that would be negative when the minimum parameter is zero?
This happens when the mean is relatively low - at least within a few SD.
Perhaps I should use something else but I'd still like to know what the function is doing. It always produces positive values, so it might be converting values that are implied as negative into positive numbers, or is it picking another random number until it gets a positive value?

[tomfid]

It's picking another number. Reflecting negative numbers into the positive range would work if the mean=min=0, but otherwise it would distort the distribution, so we don't do that.

This won't work well if your mean & sd are such that it means most points are rejected. I think it limits to 10 attempts; I can look it up if it matters. You get a warning if that happens.

This approach applies to all the distributions (except uniform, which doesn't need it).

[nhoward]

Thanks. It would be useful to hear exactly how the function works. I have never had any warnings with simulations.
I did 10 runs using different seeds for Random Normal, with parameters (0,4, 0.25, 0.505, s), with s=0-9: please see results in attached.
The mean of the 10 runs' output comes to 0.49, SD 0.037. If you estimate the mean with Normal tables, assuming it lies at the median of the truncated distribution [area under the curve = 0.6897], I get a mean of 0.40. This suggests the result is different to a distribution truncated at zero. Perhaps, not that surprising, but if you look at the plots, there is clearly some separation between those of different seeds. In fact, this is what alerted me to this in the first place, as some scatter plots from Monte Carlo runs using different seeds were clearly different. I suspected my own error at first but apart from the functioning of Random Normal, it might be worth a look at the random number generator?

[nhoward]

Use the attached here -- previous attached numbers were referenced to my own disk. Apologies.

[tomfid]

I can't replicate your results with

Code: Select all

RANDOM NORMAL(0,4, 0.25, 0.505, seed)

Could there be a typo?

[tomfid]

Using those parameters, for samples of 101, I get:

Code: Select all

Variable	Count	Min	Max	Mean	Median	StDev	(Norm)	
"abnormal" for Time (Month) from 0 to 100 Runs:	t1	t2	t3	t4	t5	
abnormal	101	.00085	1.614	.4802	.4286	.3368	.7013	
 : t2	101	.04317	1.283	.5055	.4666	.3096	.6125	
 : t3	101	.00774	1.275	.4884	.4469	.3094	.6335	
 : t4	101	.00463	1.547	.5186	.4979	.3253	.6272	
 : t5	101	.00965	1.803	.5341	.4947	.3591	.6723	

From http://en.wikipedia.org/wiki/Truncated_ ... stribution, the expected mean/sd are .508/.352 respectively, so these seem like plausible values. If I run a large sample, I get exactly those values.

[tomfid]

abnormal.mdl

(4.72 KiB) Downloaded 626 times

This model does the math for the distribution.

[tomfid]

I should add: the model uses macros, so it'll require DSS or the Model Reader to run. You can inspect the equations with a text editor if PLE won't open it.

The random number generator is new as of 6.3. I tested it very extensively, including running the NIST test suite on the generator, but it's nice to have outside attention as well.

The old generator would have been fine for this problem as well, but it wasn't strong enough for some multivariate applications, because correlations can exist on hyperplanes that aren't evident from looking at a single series. I'm only aware of one actual demonstration of the problem, but better safe than sorry.

See http://www.vensim.com/documentation/noise_rng.htm

[tomfid]

Incidentally, the default generators in a lot of programming languages, including Java and MSVC/C++ are LCGs, http://en.wikipedia.org/wiki/Linear_con ... _generator. These are not good enough for simulation applications that are heavy consumers of random numbers, including Monte Carlo runs, MCMC and agent based models. Some of our competitors are still using LCGs though.

[nhoward]

Thank you for looking into this.

I'm sorry but the previous runs I gave were Monte Carlo and the SD would have been the standard error. The SD would have been 0.37, which is closer to what you got.

I have done 10 conventional (not monte carlo) runs of 100 steps, using different seeds, of Random normal with mean 0.25, SD 0.505, min 0, max 4, and get an overall mean of 0.500 and SD 0.337. The scatter looks reasonable but the runs' means differ and when compared with the overall mean are beyond 5% significance. This might seem statistically pedantic, I suppose, but it seems to impact on runs varying several or more random seeds with truncated Normal distributions. Please see attached.

I have had a look at Abnormal but can't run as there are no equations for NCDF2 and NPDF. I have a formula for the NPDF but would need to look up an approximation for the cumulative density function. Am using PLE+, so can't use subscripts.

[tomfid]

NCDF2 and NPDF are macros. One is at http://models.metasd.com/cumulative-nor ... tribution/

Re the 95% CI:

I think the answer is partly that the interval=+/-1.96*sd math doesn't hold exactly due to the truncation. But I think there's also an error in your spreadsheet - you're using 1.96 x SEM for the full sample of 1000, about .02, for your interval. But since your samples are 100, that's too tight. The interval should be about 2*SEM(100) = +/- .066.

When I run a large sample of 1000 runs with n=100 per run, using the attached, I get roughly the answer expected from the analytic solution:

Code: Select all

Variable	Count	Min	Max	Mean	Median	StDev	(Norm)	2.5%	97.5%	
Mean sensitivity results at time 100 Runs:	ab2	
Mean	1000	.3976	.6159	.5078	.5076	.03597	.07083	.4438	.5818	
sd sensitivity results at time 100 Runs:	ab2	
sd	1000	.2754	.4467	.3488	.3483	.02737	.07848	.2961	.4028	

abnormal2.vpm

(821.81 KiB) Downloaded 562 times

[nhoward]

Thanks. Ok, I think you're right. I had wondered whether I should have used n=10, as there were 10 runs, but with each run with 101 values, the sem for the overall mean presumably should have a denominator of root 99. I get the 5% point at 0.068, so clearly none of the 10 means are beyond it and your further runs confirm. I agree the distribution distortion spoils the significance analysis but surely only at the main data level, as the central limit theorem makes the distribution of the means Normal? I think I will look at the truncated distribution where the minimum is close to the original mean - the new minimum is in a different shape whereas the new maximum is still part of a leading tail.

Thank you for the Abnormal2 model and your macro specification.

[tomfid]

I think you're right about central limit - the interval in the large experiment (posted last) are pretty close to the 2*SEM(100) = +/- .066.

Just to beat this to death, I did another experiment, comparing half-Normal distributions implemented by (a) setting min = mean = 0, and (b) taking ABS(NORMAL) with min/max = +/- 100, using mean = 0 and sd = 1 in both cases. For a large sample (10000 runs of 4000 each), the answer is

Code: Select all

Variable	Count	Min	Max	Mean	Median	StDev	(Norm)	3%	5%	10%	90%	95%	98%	
"Mean" sensitivity results at time 4000 Runs:	ab3s	
Mean	10000	0.758493	0.83308	0.797869	0.797822	0.00938274	0.0117597	0.779508	0.782461	0.78585	0.809749	0.81332	0.815966	
"sd" sensitivity results at time 4000 Runs:	ab3s	
sd	10000	0.571667	0.633026	0.602629	0.602553	0.00802555	0.0133176	0.58684	0.589354	0.592471	0.612954	0.61585	0.618434	
"Mean 2" sensitivity results at time 4000 Runs:	ab3s	
Mean 2	10000	0.760072	0.839244	0.797883	0.797683	0.00953244	0.0119472	0.779274	0.782402	0.78582	0.810219	0.813751	0.816583	
"sd 2" sensitivity results at time 4000 Runs:	ab3s	
sd 2	10000	0.574283	0.637153	0.602573	0.602477	0.00810216	0.0134459	0.58648	0.589181	0.59218	0.613031	0.615895	0.618385	

The answers match the analytic solution to the third decimal place:

Code: Select all

Variable	Count	Min	Max	Mean	Median	StDev	(Norm)	1%	5%	15%	25%	65%	75%	95%	99%	
"truncated normal mean" results across subscripts at time 100 Runs:	a1	
truncated normal mean	1	.7979	
"truncated normal sd" results across subscripts at time 100 Runs:	a1	
truncated normal sd	1	.6028	

abnormal3.mdl

(4.5 KiB) Downloaded 517 times

[nhoward]

Thanks. Am happy to leave it there re truncated Normal.