Random number generator; any tips to replicate?

[Travis]

I'm looking into adding the RANDOM suite of functions to SDEverywhere.org. This would support converting vensim models that use those functions to the web more readily, along with targeting C and other platforms.

From the [url=https://www.vensim.com/documentation/noise_rng.html]Vensim documentation[/url], I see that Vensim uses the SIMD oriented Fast Mersenne Twister(SFMT) algorithm. The code is on GitHub [url=https://github.com/MersenneTwister-Lab/SFMT]here[/url] is seems (I haven't downloaded yet).

Before I start trying to replicate this generator, has Vensim done anything different or special? Anything that you'd flag that I should know about?

The goal, of course, is to be able to replicate a Vensim run in SDE, even with RANDOM functions being used.

TIA

[tomfid]

The version we're using is a little older, but same thing. It was very easy to implement - nothing special needed, but we did do extensive verification using the NIST suite and DieHard to be sure it was working.

I think the goal of replication with random functions may be elusive, because order and number of calls matters. If, for example,
a = random()
b = random()
y = a+b
you could equally execute this as
b = random()
a = random()
y = a+b
Equation sorting is deterministic in Vensim, but I'm not sure we could readily tell you exactly what the rules are.

Another source of variation is likely to be calls during initialization - anything that causes one more or one less will break symmetry.

The client algorithms also matter. There are several ways to implement the Normal distribution, with some variation in number of draws used.

[Travis]

I'm glad I asked this before starting the journey!

The equation ordering, and hence the order of rand() calls, seems like the most troublesome issue. SDE unrolls the the model equations, but probably in a way that is different than Vensim's implementation –– if only because there wasn't consideration for this.

I'll think about this some more and go from there. Thanks.

[Travis]

Reflecting on this a bit more...

Again I'm grateful for your reply. Prevented a lot of headaches, I'm sure.

I'm wondering how hard it would be to decipher or lookup Vensim's equation sorting. I'm not 100%, but I think the only rules SDE would need are the RANDOM-related rules. It seems like SDE would only have to worry about explicitly ordering those rules, of which I could think of several different ways to implement.

Do you think those ordering rules would be possible to uncover? I could try to reverse engineer, but any additional guidance would be useful.

[Administrator]

There are no specific rules for any of the random calls.

From memory, data, INITIAL, vector functions, then everything else.

[Travis]

So if someone renames variables or reorders views (or whatever might cause the "everything else" equations to reorder), the results of the RANDOM() function would change? I read Tom's comment above that they would be more deterministic and always ordered the same.

For instance, a model has RANDOM_UNIFORM and RANDOM_NORMAL, and is run to generate output1.vdfx.
If a user then deletes the UNIFORM() function in one portion of the model and adds a UNIFORM() function in a different part of the model, will the functions be called in the same order? Will the random numbers pulled be the same as stored in "output1"?

From Tom's reply I would assume "yes" but your reply I would assume "maybe".

[LAUJJL]

Hi Travis.

I have observed rather strange behaviors from the random generators.

For instance the generator does not generate always the same values.

For instance, if you add a new random variable in your model, the behavior of other random variables may change.

Another problem is with games.
If you stop a game at a certain time, the random generator may be reinitialized. And the behavior of some random variables may change.

If I want the behavior of random variable not to vary, I chose a base behavior that I store in a run, and I read the values of the variable in the run stored via a get vdf data or something else. It works with a game or if you modify a model.

Regards.

JJ

[Travis]

Interesting, some of that makes sense given my understanding of how RANDOM is working in Vensim.

* The generator, for a particular "Stream", should generate the same values in the same order.
* if you create a new variable that calls RANDOM() and additional time, that will use another value from the stream and change all the results from previously existing RANDOM calls.
* If you were to add that new RANDOM call to the model using a second "stream", the existing RANDOM call values should not change.

I can't explain the GAME situation because I don't use that often. Storing the random values in an exogenous file makes a lot of sense and might be useful for SDE also.

[LAUJJL]

I have sometimes used the storing of random streams for games and ordinary runs and it worked perfectly.

I think that the problem is with the initialization. Vensim changes the initialization at each new random variable even for each subscripted value if there are subscripts. The problem is how this initialization is made. It is eventually made with a given order that must stay the same if one wants the values to stay the same. One solution would be to make the random generation independent from the model itself, by adding for instance a number that references the type of generation and use a different number for each variable. But there is already a seed that should logically change the stream the same way every time it is used?

As long as one does not know how Vensim is working, it is difficult to give another solution than storing the values.

The same thing happens with the instantaneous and find zero function where one does not know the value of the variable when it converges, something that is sometimes annoying.

Other parts of Vensim work in a kind of black box too.

[Travis]

Restating what I stated above for clarity: If a modeler wanted every RANDOM-based function in a model to be independent of all of the others (not drawing from the same pool), then the functions should be called with different STREAM_IDS (https://www.vensim.com/documentation/stream_id.html). Someone from Ventana should confirm since I haven't tested this, but that is my understanding from the documentation.

[LAUJJL]

I had already seen this explanation about the streams, but I must confess that I have not really understood what it is supposed to tell.

[tomfid]

For instance, a model has RANDOM_UNIFORM and RANDOM_NORMAL, and is run to generate output1.vdfx.
If a user then deletes the UNIFORM() function in one portion of the model and adds a UNIFORM() function in a different part of the model, will the functions be called in the same order? Will the random numbers pulled be the same as stored in "output1"?

I would not expect this to be true in general. If you go back to my example,

a = random()
b = random()
y = a+b

vs.

b = random()
a = random()
y = a+b

Whether a or b executes first depends on how they appear in the ordered equation set. That in turn depends on what else uses a and b. If nothing else uses a and b, either ordering is valid. I think Vensim makes a deterministic choice, but I'm not sure what the criteria is - alphabetical, order of addition to the model, or something else.

[tomfid]

Restating what I stated above for clarity: If a modeler wanted every RANDOM-based function in a model to be independent of all of the others (not drawing from the same pool), then the functions should be called with different STREAM_IDS (https://www.vensim.com/documentation/stream_id.html). Someone from Ventana should confirm since I haven't tested this, but that is my understanding from the documentation.

This is correct. I don't think there's a good argument for having lots of streams, and there's a computational penalty, but we could easily optimize for that, if there were a compelling use case.

[tomfid]

Another problem is with games.
If you stop a game at a certain time, the random generator may be reinitialized. And the behavior of some random variables may change.

I think this would be a bug; if you have an example, we'd be interested.

[LAUJJL]

I noticed that game problem, years ago, and it is maybe no more a bug. I will try to reproduce it anyhow if possible.

[LAUJJL]

I noticed that problem in 2012 in Janauary. i was the author and the title was: question about the gaming tool