computer speed

[LAUJJL]

HI

I am presently using a laptop and a micro computer ASUS bought three years ago. I use Vensim DSS 6.3 and Windows XP.

The model I am working on takes about half an hour for one simulation, if I set the arrays dimensions to their maximum, which is necessary to build realistic policies.

I am trying to find some ways to increase my productivity.

What is presently the best solution to increase the speed of my models? The price is not a problem.

I f I have nothing better than what I have presently, I will buy a second micro computer, hopefully a bit quicker, and devote it to long batch works (several days). I may even buy a third one if working this way is productive and even a forth one, so I can run different works based on different scenarios.

Good advices will be strongly appreciated.

Regards.

JJ

[bppro]

Hi,

On the hardware side of things, I would take a look at CPU and RAM. Is your CPU reaching 100% while you're doing simulations, or is something else the bottleneck? What exactly is your CPU? Is your RAM full? How much do you have installed? Also, do you have Windows XP 32 bits or 64 bits?

With that information, it will become easier to estimate the benefits of buying new hardware.

BP

[tomfid]

Some thoughts on model speed here:
http://blog.metasd.com/2011/01/optimizi ... im-models/

CPU speed hasn't improved that much with respect to what matters for Vensim,
https://www.cpubenchmark.net/singleThread.html
However, if your existing machine is not an i7 with a high clock speed (>3GHz), there may be some room for improvement.

[gwr]

The advance in hardware performance is argueably only part of the reason software has gotten faster, the other side to it being algorithmic advances. To my knowledge, Ventity for example, will follow modern solver strategies and use a variable time step, loosely speaking making "bigger steps" when less is happening?

Preprocessing data seems indeed a very good advice, but wouldn't multi core support pay off when loading data or performing multiple runs, sensitivity analysis or optimization?

I have come to acknowledge some limits of Vensim when doing transport modeling where you will have large matrixes that are quite sparse and in the end modleing 2 Mio city inhabitants individually (mikrosimulation approach) is far faster than running through an array with more than 8 Mio. theoretical entries (here also memory was a limitation as 64Bit support was not available then - and is not optimized for run time as of now). I am not sure whether the fruits of this really carry over to Vensim in its present state (Tom?). Is the VectorSelect really a fast appraoch to sparse matrix calculations - in the end with changing entries at least the selection matrix would have to be updated completely? -- Also Boolean selection variables might be faster than '1' or '0'?

I would try to maybe employ CAS to maybe simplify flow equations so that in the end your model will be in the most compact form possible. I wonder whether scaling the numerical range might be helpful on the numerical side maybe (e.g. normalizing values and even time to 1 - cf. Hartmut Bossel's book on simulation)?

If all else fails, maybe the meta simulation approaches (cf. Kleijnen) might help (e.g. DACE / DAE)?

Best regards,
Guido

[LAUJJL]

Hi Guido

I have tested using the vector select on matrixes full of zero. I have not noticed any speed increase.

What I am currently doing to increase speed is finding the right group for each variable, to be able to erase some groups depending on what I am doing. One important group is the kernel that contains all the constants and variables that generate the dynamic behavior. To find the kernel, one just has to erase progressively all the supplementary variables, until there are no more of them.

If I have for instance to run an optimization I erase all the groups but the kernel and the group that calculates the payoff to optimize. I then generate a .cin file from the result of the optimization to save the optimized parameters and I can then run the full model with nothing erased with that .cin file.

The time saved depends on the difference between the model with only the kernel and the group calculating the payoff and the full model.

Another way when optimizing is to look at the log file to see how quickly the optimization converges.
In my case a transportation model that takes about one hour and a half to run, I have noticed that 10 simulations generate already an acceptable solution.

That same model generates an 8 gigabytes file that cannot be saved with the 32 Windows XP.
I then increase the save period from 1/64 a day to 1 day. It does not change the optimization.

The other method is to use different computers that can work during days in parallel.

There are presently many micro computers with multiple processors having multiple cores, but I do not think that it increases the speed using Vensim compilation.

One other solution is trying to simplify the model, noticing that the simplification does not affect the result of the optimization.

Of course all this takes a lot of time, but unfortunately, simulation anyway takes a lot of time if one wants to get useful results.

Regards.
JJ

[Administrator]

Have a look at the Amazon Web Services (specifically Amazon EC2).

You can configure a virtual machine in almost any way. I recently rented a machine with 32 cores (and 32 instances of Vensim, all running optimizations). It was $3 per hour of machine time so was very cost effective. You can easily change the machine type as well if needed (so could start with 1 processor and at a later date, make it 10).

Regarding computer speed. It is correct that we could use multiple cores to run parallel sensitivity/optimizations (that's essentially what I was doing on Amazon). Tom has experimented with doing this via the Vensim DLL. It's on the to-do list.

There is no point in us attempting to change the main simulation engine to run on multiple cores. The following page does a good job in explaining why (search for "There are threads, and then there are threads" in http://scalibq.wordpress.com/2012/06/01 ... threading/).


All you can really do is to compile your model if you are not already doing so.

That same model generates an 8 gigabytes file that cannot be saved with the 32 Windows XP.

Was the hard drive formatted NTFS or FAT? NTFS should allow you to save an 8GB file, FAT will not.

[gwr]

That is an interesting suggestion with regard to renting virtual machines in the cloud.

With regard to what might speed up simulation I would like to know more about how things are done internally in Vensim with regard to sensitivity runs, synthesim or optimization runs:

For example I noted that in Mathematica there is ParametricNDSolve which numerically solves a system of ODE depending upon some parameters. It will return an interpolated function object that can then be fed new parameter values. The recalculation of the numerical solution (e.g. the parametric function) will then be much faster than the initial run because the numerical function will have been simplified and preprocessed so that it is in a very compact form and "only" needs the parameter values to calculate new values for the stocks (or the goal function g depending upon the stocks of course).

To my understanding it would boil down to keeping track of the recursive determination of the stocks, e.g. s(1) = s(0) + f0( param, s(0) ) * dt , s(2) = ( s(0) + f( param, s(0) ) * dt ) + f1( param, s(0) + f0( s(0) ) * dt ) * dt ... In the end one would have a function for s(T) where everything will be a numerical expression and the only unknowns/symbols left would be the parameters. That could be simplified and stored in a preprocessed form to speed up recalculation for different parameter values, couldnt' it? Is something similar done in Vensim?

Regards,
Guido

[Administrator]

Is something similar done in Vensim?

I don't understand what you are explaining here. It sounds like pre-compiling a function, but I don't know. So if that is what you are saying, then DSS can compile models and it does exactly this. When running in interpreted mode, the model is broken down into a series of instructions to perform (sum, allocate etc).

[gwr]

So if that is what you are saying, then DSS can compile models and it does exactly this.

Yes, that is essentially what compiling should be all about. Once again a great answer even though what I was trying to say - making some effort in doing so - failed to be reasonably clear for some reason. Albeit I would like to know where my statement was maybe inconcise, given that I am not a computer scientist; how else to phrase a question that you do not know the answer to before hand?

I must admit that to our firm speed issues have so far not made top of the list -- it rather used to be memory issues given the 32 Bit limitations before. Over all I am rather interested in robustness and stability of the simulation software, in ease of reusability for partial models and model components, in analytical possibilities and in visual presentation capabilities (it is after all the business world we are trying to make an impact upon).

Great that Ventana is adressing all these issues given also that Ventity is ante portas (while the portas have not been opened yet...).

Regards,
Guido

[LAUJJL]

Hi

I use window XP 32 bits
media center edition
Version 2002
Service pack 3

The processor is a I3-2125 3.3 Ghz with 2.91 Go Ram

Regards.

JJ

[tomfid]

The advance in hardware performance is argueably only part of the reason software has gotten faster, the other side to it being algorithmic advances. To my knowledge, Ventity for example, will follow modern solver strategies and use a variable time step, loosely speaking making "bigger steps" when less is happening?

Variable time step is actually not in the initial feature set. It's possible, but made much more difficult by the variable structure of the model.

Preprocessing data seems indeed a very good advice, but wouldn't multi core support pay off when loading data or performing multiple runs, sensitivity analysis or optimization?

Generally data loading is not a big limit because it only happens once. Multicore support for multiple runs would be a big benefit. Unfortunately it's hard to implement within the existing architecture.

I have come to acknowledge some limits of Vensim when doing transport modeling where you will have large matrixes that are quite sparse and in the end modleing 2 Mio city inhabitants individually (mikrosimulation approach) is far faster than running through an array with more than 8 Mio. theoretical entries (here also memory was a limitation as 64Bit support was not available then - and is not optimized for run time as of now). I am not sure whether the fruits of this really carry over to Vensim in its present state (Tom?). Is the VectorSelect really a fast appraoch to sparse matrix calculations - in the end with changing entries at least the selection matrix would have to be updated completely? -- Also Boolean selection variables might be faster than '1' or '0'?

VECTOR SELECT should be quite a bit faster than SUM/PROD/etc. when the selection matrix is sparse, though I haven't done detailed timing. This is particularly true when the selection matrix is static. Otherwise, it's only worth it as long as updating the selection matrix is cheaper than computing the object of the sum/avg/.... Ventity is much better at this task because the 0 elements don't even exist. Whether the select is 0/1 or boolean should not be much of a speed difference; it mainly affects memory, though that could spill over to speed if the cache is filling up. Making Vensim a typed language would be tough on most uers though.

If all else fails, maybe the meta simulation approaches (cf. Kleijnen) might help (e.g. DACE / DAE)?

I've actually tried a couple such approaches, but not found something that works well on a general, unknown problem without user intervention. It's an attractive avenue though, both because of the potential improvement and that it may lend itself to paralellism.

[gwr]

Great answers to ponder, thank you, Tom!

Just for correctness: It should have read "DASE / DOE" (Design and Analysis of Simulation Experiments / Design of Experiments) in my post with DASE being used by Kleijnen having the connotations of DACE.

I understood that it is possible to start multiple threads using the Vensim DLL? If the answer here is yes, than parallelization might be established using the calling software tools at hand?

[Administrator]

I understood that it is possible to start multiple threads using the Vensim DLL? If the answer here is yes, than parallelization might be established using the calling software tools at hand?

Only the multi-context DLL supports this.

[gwr]

Only the multi-context DLL supports this.

Great. And this multi-context DLL is included where or can be obtained how?

[tomfid]

It's a separately priced item (email us for a quote). However, it may not immediately help you - its multithreading supports server usage, but doesn't paralellize the opt/sensi algorithms.

[LAUJJL]

Hi

I had just written a message after having loged in, and lost all the message, after the software asked me again to log in when I tried to submit it.

It has happen many times before. If one forgets to save the message on Word for instance, one loses the whole message!

It has just happened and I will have to wait a bit until I get less angry to write it again!

I saved this one.

Regards.

JJ

[Administrator]

What browser are you using?

Sometimes if you hit the back button, the message will still be there (and you can copy/paste it).

Tony.

[LAUJJL]

Hi Tony

I use internet explorer and using the return button does not help. i have tested different solutions but did not find any.

The solution is to write the message in Word for instance and to paste it into the forum.

I forgot to do it and was caught.

Regards.

JJ

[LAUJJL]

Hi

I have not quite understood this thread that has some contradictory opinions.

My problem is that I would like to launch different long runs at the same time, while I am for instance studying the data resulting from a preceding run.

My options are:

Try to run them on my computer that has an I3 processor and Windows XP. Is it possible?

Implement Windows 7 on my computer with additional memories, which is possible. But will it be possible to run parallel simulations, the processor being not changed. Is is necessary to use Vensim 64 to do that? Is it necessary to buy the multicontext DLL?

Buy a new computer with an I7 processor and Windows 7. But is it better to buy a multi cores processor or eventually a computer with multiple processors? What other options are needed?
The multicontext Vensim DLL?

Thanks in advance for the answers and best regards?

JJ

[Administrator]

My problem is that I would like to launch different long runs at the same time, while I am for instance studying the data resulting from a preceding run.

I believe the i3 processor is dual core. So I would only launch two instances of Vensim on this, and run one long simulation in each one. You will still be able to run a 3rd instance of Vensim and examine results, and you will also be able to use the machine for other things. My rule of thumb is to only run one instance of Vensim per core if I'm running sims/optimizing.

If you want to run much more than this, there are two options and it depends on what you feel most comfortable with.
--Option 1 is to buy additional computers, it's straightforward and you will have the machines in your possession ready to run the simulations.
--Option 2 is to use something like the Amazon cloud. The prices are very reasonable, and you can run as many processors as you need. A little more difficult to set up, but once running, things are easy.

Hope this helps.

[LAUJJL]

Hi

Thank you for the answer.

i have an I3 with one core. So if i understand, I can only run one simulation, while doing some other things, like opening another Vensim session that will only be available to see the results of a previous run, but will not be able to make a run. To run multiple sessions, one must have a multi cores processor.

What about having a multiple processor with multi cores? For instance if I have a computer with two processors with each one two cores, will I be able to run paralelly four independant sessions?
Will I be able to run for instance 4 excel sessions using the dll working on different packaged models?

Best regards?

JJ

[Administrator]

i have an I3 with one core. So if i understand, I can only run one simulation, while doing some other things, like opening another Vensim session that will only be available to see the results of a previous run, but will not be able to make a run. To run multiple sessions, one must have a multi cores processor.

Yes. I'm sure the i3 is a dual core processor. If you run more than one sim per processor, the computer tends to get overloaded (the processor runs at 100% and other tasks suffer as a result).

What about having a multiple processor with multi cores? For instance if I have a computer with two processors with each one two cores, will I be able to run paralelly four independant sessions?

Yes.

Will I be able to run for instance 4 excel sessions using the dll working on different packaged models?

Yes.

[LAUJJL]

You are right. The I3 is a dual core and i can run two separate sessions.

Thanks again.

Regards.

JJ