v.10 Parallel computations

[rosenm]

Hello,
I'd like to try the new options RPARALLEL/XPARALLEL in v.10 but they are not available in Multiple start menu, please see the picture. Where to find these options?

Thanks

[Administrator]

Parallel is only in DSS for now for the initial release.

[tomfid]

You can use the DSS trial key on the conference page if you'd like to experiment.

https://vensim.com/conference/

[aliakhavan89]

I have a few clarification questions:

1. Does MULTIPLE_START = XPARALLEL ignore RESTART_MAX entirely? If not, what does having MULTIPLE_START = XPARALLEL with RESTART_MAX = N mean? In this case, how does the parallelization work?

2. From the documentation: MULTIPLE_START = RPARALLEL is like RRANDOM, but with restarts delegated to multiple threads. Does setting RESTART_MAX based on the number of threads improve performance? For example, I set RESTART_MAX = N * M where N is an integer and M is the number of threads (I was thinking it could uniformly distribute the search across multiple threads)

[tomfid]

1. XPAR does ignore restart max at present. XPAR basically gets you one optimum as quickly as possible. If you want to explore multiple optima, RPAR is more efficient.

2. For RPAR I tend to leave RESTART_MAX unset (0) and just let it run until I'm tired of waiting. The performance of the search will depend only on the number of threads running. This defaults to the number of logical cores minus 2 (to leave some room for system performance).

I think RPAR has some reporting issues for individual solutions that will be fixed in the next release, but it does get you the global max correctly.

[aliakhavan89]

Hi Tom,

Thank you very much for the detailed response. This was very helpful and triggered another question. Can we set the SEED parameter in the optimization setting in a way that each thread (or each restart sequence) uses a new random value throughout the search?

Currently, all search threads with RPAR use the same SEED for random number generation, except with different initial starting point. Is that true?

Thank you very much,
Ali

[tomfid]

There are two seeds:
- One for the algorithm, set in the voc file
- One for the simulation, set in NOISE SEED or some other user variable, used in the RANDOM functions

I'm guessing that you want to vary the second one? One possibility would be to use GET TIME VALUE to randomize.
https://www.vensim.com/documentation/fn ... value.html
I think that should work as long as the model is slow enough to have more than a second between optimizations (possibly much less).

The downside of that is that it's not reproducible, so I'll look into making some kind of counter accessible. Should be easy.

[aliakhavan89]

Hi Tom,

Sorry for my late response, and thanks a lot. I have actually used the GET TIME VALUE method, and it is really helpful. Also, thanks for reminding about the reproducibility.

However, my inquiry was about the first item: for the algorithm, set in the voc file.

As far as I understand, each thread uses the same noise seed through the search process. I wondered if it has any impact on the optimization performance if each thread uses different noise seed to have more perturbation in the search alongside with varying restart points.

[tomfid]

The helper threads actually do increment the seed. However, this only affects selection of the starting points, because the Powell algorithm is deterministic. There's one exception: XPAR occasionally tests crossover directions, which convey information from one thread to another, and the crossover selection is random (using the master seed I think) and also nondeterministic because the order of arrival of information from different threads is subject to some turbulence from CPU loading.

Another way to go about exploring the space is to use MCMC as a simulated annealer, by setting a high initial temperature with a long cooldown period.