How can i replace the optimization algorithm?
How can i replace the optimization algorithm?
Hi
I am using the Vensim DSS 5, the optimization function is based on Powell-Brent algorithm. However, i am not satisfied with the performance.
I know that in DSS 6 some other algorithms are put into use, but now i would not like to update my vensim.
If i want to use other optimization algorithms, how can i do that?
Any help will be greatly appreciated!
Best regards.
I am using the Vensim DSS 5, the optimization function is based on Powell-Brent algorithm. However, i am not satisfied with the performance.
I know that in DSS 6 some other algorithms are put into use, but now i would not like to update my vensim.
If i want to use other optimization algorithms, how can i do that?
Any help will be greatly appreciated!
Best regards.
-
- Super Administrator
- Posts: 4590
- Joined: Wed Mar 05, 2003 3:10 am
Re: How can i replace the optimization algorithm?
You cannot use a different algorithm.
Why are you not satisfied with the existing performance?
Why are you not satisfied with the existing performance?
Advice to posters seeking help (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Units are important!
http://www.bbc.co.uk/news/magazine-27509559
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Units are important!
http://www.bbc.co.uk/news/magazine-27509559
Re: How can i replace the optimization algorithm?
Hi admin, i have three parameters a, b and c to calibrate. b is affected by a and they are correlated.
Now i would like to use optimization to estimate the proper values for the three parameters, but vensim seems to change the parameters individually. More precisely, for every loop of a, the value of b is not correspondly changed.
I am not sure, is it a bug or i have missunderstood.
Need your help, thanks in advance
Now i would like to use optimization to estimate the proper values for the three parameters, but vensim seems to change the parameters individually. More precisely, for every loop of a, the value of b is not correspondly changed.
I am not sure, is it a bug or i have missunderstood.
Need your help, thanks in advance
-
- Super Administrator
- Posts: 4590
- Joined: Wed Mar 05, 2003 3:10 am
Re: How can i replace the optimization algorithm?
Can you post the model and optimization files you are working with?
Advice to posters seeking help (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Units are important!
http://www.bbc.co.uk/news/magazine-27509559
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Units are important!
http://www.bbc.co.uk/news/magazine-27509559
Re: How can i replace the optimization algorithm?
Sorry, this model is a commercial model, i cannot give too much infromation.
I would like to know how does the optimization work and could i change it with the help of external function?
I would like to know how does the optimization work and could i change it with the help of external function?
-
- Super Administrator
- Posts: 4590
- Joined: Wed Mar 05, 2003 3:10 am
Re: How can i replace the optimization algorithm?
There is no way of changing it with an external function.
The optimization in Vensim does give good results if it is set up properly. My guess is that you just don't have it set up correctly.
Can you construct a simple model that does something similar to what you need to do and let us see that?
The optimization in Vensim does give good results if it is set up properly. My guess is that you just don't have it set up correctly.
Can you construct a simple model that does something similar to what you need to do and let us see that?
Advice to posters seeking help (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Units are important!
http://www.bbc.co.uk/news/magazine-27509559
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Units are important!
http://www.bbc.co.uk/news/magazine-27509559
Re: How can i replace the optimization algorithm?
You can run your own algorithm via the .dll, but there's quite a bit of overhead involved in setting it up, and you'll lose at least a factor of 2 in speed per iteration.
The Powell optimizer initially searches directions one at a time, but the search is iterative, so that sequential updating of a,b,c will lead to the joint maximum of f(a,b,c). In addition, after each pass, the search directions are updated, so subsequent searches will not be along the a, b, c axes, but along some linear combinations of a,b,c (i.e. rotations of the axes).
This method is not always the most efficient, but it is robust, and competitive with several other modern methods that I've tested. It will only fail in the case of multiple optima or perverse payoffs with large flat areas lacking significant gradients, which is why we provide random multistart. Other methods that converge faster if your payoff surface is well behaved will still have the multiple optimum problem.
I'd suggest,
a) turn on random multistart
if possible,
b) change your parameter set to be more independent,
c) avoid discrete/integer constraints that make the payoff surface rough.
The Powell optimizer initially searches directions one at a time, but the search is iterative, so that sequential updating of a,b,c will lead to the joint maximum of f(a,b,c). In addition, after each pass, the search directions are updated, so subsequent searches will not be along the a, b, c axes, but along some linear combinations of a,b,c (i.e. rotations of the axes).
This method is not always the most efficient, but it is robust, and competitive with several other modern methods that I've tested. It will only fail in the case of multiple optima or perverse payoffs with large flat areas lacking significant gradients, which is why we provide random multistart. Other methods that converge faster if your payoff surface is well behaved will still have the multiple optimum problem.
I'd suggest,
a) turn on random multistart
if possible,
b) change your parameter set to be more independent,
c) avoid discrete/integer constraints that make the payoff surface rough.
/*
Advice to posters (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Blog: http://blog.metasd.com
Model library: http://models.metasd.com
Bookmarks: http://delicious.com/tomfid/SystemDynamics
*/
Advice to posters (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Blog: http://blog.metasd.com
Model library: http://models.metasd.com
Bookmarks: http://delicious.com/tomfid/SystemDynamics
*/
Re: How can i replace the optimization algorithm?
Hi tomfid, thanks a lot for your suggestions.
I would like to develop my own algorithms, this means, i have to define my own external functions.
Could you please provide me some more details about how the optimization works in vensim?
Best regards
I would like to develop my own algorithms, this means, i have to define my own external functions.
Could you please provide me some more details about how the optimization works in vensim?
Best regards
Re: How can i replace the optimization algorithm?
You can't replace the optimizer with an external function. You'll have to use the .dll http://www.vensim.com/documentation/dss_dll.htm
NLopt is a useful source of algorithms relevant to dynamic modeling situations (i.e. gradient free), http://ab-initio.mit.edu/wiki/index.php/NLopt
The Powell optimizer is similar to that described in Numerical Recipes in C http://www.nr.com/, but with proprietary improvements.
There's an accessible description at http://math.fullerton.edu/mathews/n2003 ... odMod.html
I forgot to mention that, for difficult payoff surfaces with many optima, the new MCMC tool can also do Simulated Annealing, http://www.vensim.com/documentation/mcmc_sa.htm
I'm still not clear on the motivation for this, because the Powell optimizer does handle situations with covarying control parameters. It works just fine on the Rosenbrock function, for example. http://en.wikipedia.org/wiki/Rosenbrock_function
NLopt is a useful source of algorithms relevant to dynamic modeling situations (i.e. gradient free), http://ab-initio.mit.edu/wiki/index.php/NLopt
The Powell optimizer is similar to that described in Numerical Recipes in C http://www.nr.com/, but with proprietary improvements.
There's an accessible description at http://math.fullerton.edu/mathews/n2003 ... odMod.html
I forgot to mention that, for difficult payoff surfaces with many optima, the new MCMC tool can also do Simulated Annealing, http://www.vensim.com/documentation/mcmc_sa.htm
I'm still not clear on the motivation for this, because the Powell optimizer does handle situations with covarying control parameters. It works just fine on the Rosenbrock function, for example. http://en.wikipedia.org/wiki/Rosenbrock_function
/*
Advice to posters (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Blog: http://blog.metasd.com
Model library: http://models.metasd.com
Bookmarks: http://delicious.com/tomfid/SystemDynamics
*/
Advice to posters (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Blog: http://blog.metasd.com
Model library: http://models.metasd.com
Bookmarks: http://delicious.com/tomfid/SystemDynamics
*/
Re: How can i replace the optimization algorithm?
Hi nice people,
I have tried to implement the optimization algorithms myself with the external functions, but i have i have encountered some problems:
1. I use SETVAL to set values for the calibrated variables, and use MENU>RUN to get the payoff value, but the simulation is too slow! Is there some way, that i can accelarate this process?
2. I would like to run the program on our server, but i got a error message:An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B). I have searched in the forum and found that someone else had the same problem, but i could not understand the approached solution. Could you please explain it again?
Thanks a lot
I have tried to implement the optimization algorithms myself with the external functions, but i have i have encountered some problems:
1. I use SETVAL to set values for the calibrated variables, and use MENU>RUN to get the payoff value, but the simulation is too slow! Is there some way, that i can accelarate this process?
2. I would like to run the program on our server, but i got a error message:An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B). I have searched in the forum and found that someone else had the same problem, but i could not understand the approached solution. Could you please explain it again?
Thanks a lot
Last edited by skyey on Fri May 17, 2013 9:26 am, edited 1 time in total.
-
- Super Administrator
- Posts: 4590
- Joined: Wed Mar 05, 2003 3:10 am
Re: How can i replace the optimization algorithm?
The simulation will be as fast as it was before. Are you saying it's slower now that you are running it from the DLL?
Advice to posters seeking help (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Units are important!
http://www.bbc.co.uk/news/magazine-27509559
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Units are important!
http://www.bbc.co.uk/news/magazine-27509559
Re: How can i replace the optimization algorithm?
No, but i have tried the vensim's Optimization, it is clearly faster. So i would like to know, how can i make it faster? For example, do not store the simulation results into the run file
-
- Super Administrator
- Posts: 4590
- Joined: Wed Mar 05, 2003 3:10 am
Re: How can i replace the optimization algorithm?
You can use a savelist to minimise the amount if results being stored.
You can also compile the model if you have not already done so.
You can also compile the model if you have not already done so.
Advice to posters seeking help (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Units are important!
http://www.bbc.co.uk/news/magazine-27509559
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Units are important!
http://www.bbc.co.uk/news/magazine-27509559
Re: How can i replace the optimization algorithm?
Hi, what do you mean "use a savelist to minimise the amount if results being stored"? Could you please explain some more?
Thanks a lot
Thanks a lot
-
- Super Administrator
- Posts: 4590
- Joined: Wed Mar 05, 2003 3:10 am
Re: How can i replace the optimization algorithm?
http://www.vensim.com/documentation/ind ... velist.htm
http://www.vensim.com/documentation/ind ... ?25355.htm
http://www.vensim.com/documentation/ind ... ?25355.htm
Advice to posters seeking help (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Units are important!
http://www.bbc.co.uk/news/magazine-27509559
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Units are important!
http://www.bbc.co.uk/news/magazine-27509559
Re: How can i replace the optimization algorithm?
For maximum speed in the dll, you should run using the SYNTHESIM commands rather than ordinary simulations.
http://www.vensim.com/documentation/25172.htm
http://www.vensim.com/documentation/25531.htm
http://www.vensim.com/documentation/25532.htm
Turn AUTO off and use GO to launch simulations, after setting parameters with SETVAL. You can also use MEMFILE and/or a savelist to minimize write time.
This still isn't quite as fast as the native optimization simulations, but we're actively working on exposing those through the .dll.
http://www.vensim.com/documentation/25172.htm
http://www.vensim.com/documentation/25531.htm
http://www.vensim.com/documentation/25532.htm
Turn AUTO off and use GO to launch simulations, after setting parameters with SETVAL. You can also use MEMFILE and/or a savelist to minimize write time.
This still isn't quite as fast as the native optimization simulations, but we're actively working on exposing those through the .dll.
/*
Advice to posters (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Blog: http://blog.metasd.com
Model library: http://models.metasd.com
Bookmarks: http://delicious.com/tomfid/SystemDynamics
*/
Advice to posters (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Blog: http://blog.metasd.com
Model library: http://models.metasd.com
Bookmarks: http://delicious.com/tomfid/SystemDynamics
*/
Re: How can i replace the optimization algorithm?
Hallo, i have encounted a very wired problem!
My program using external functions suddenly doesn't work any more, even the simple directive "LOADMODEL" cannot be excuted. I have changed nothing and the model can be normally loaded by DSS.
It makes me so nervous, do you have any opinion?
Thanks a lot
My program using external functions suddenly doesn't work any more, even the simple directive "LOADMODEL" cannot be excuted. I have changed nothing and the model can be normally loaded by DSS.
It makes me so nervous, do you have any opinion?
Thanks a lot
Re: How can i replace the optimization algorithm?
Have you upgraded Vensim in the interim?
What is the symptom of the failure? Does the vensim_command() with the LOADMODEL return 0, or are you unable to load the dll or call the function at all?
What is the symptom of the failure? Does the vensim_command() with the LOADMODEL return 0, or are you unable to load the dll or call the function at all?
/*
Advice to posters (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Blog: http://blog.metasd.com
Model library: http://models.metasd.com
Bookmarks: http://delicious.com/tomfid/SystemDynamics
*/
Advice to posters (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Blog: http://blog.metasd.com
Model library: http://models.metasd.com
Bookmarks: http://delicious.com/tomfid/SystemDynamics
*/
Re: How can i replace the optimization algorithm?
Hi tomfid,
thanks for the response.
I havenot upgraded vensim, the error message is that the vensim_command() with the LOADMODEL return 0.
Thanks a lot
thanks for the response.
I havenot upgraded vensim, the error message is that the vensim_command() with the LOADMODEL return 0.
Thanks a lot
-
- Super Administrator
- Posts: 4590
- Joined: Wed Mar 05, 2003 3:10 am
Re: How can i replace the optimization algorithm?
Have you re-installed Vensim? Or installed Vensim model reader?
Advice to posters seeking help (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Units are important!
http://www.bbc.co.uk/news/magazine-27509559
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Units are important!
http://www.bbc.co.uk/news/magazine-27509559
Re: How can i replace the optimization algorithm?
Hi, i have uninstalled the vensim reader after the error message.
I guess that this problem is caused by the "vendll32.dll", but the file is always there, i have changed nothing.
Except for the vendll32.dll, does it need some others?
I guess that this problem is caused by the "vendll32.dll", but the file is always there, i have changed nothing.
Except for the vendll32.dll, does it need some others?
-
- Super Administrator
- Posts: 4590
- Joined: Wed Mar 05, 2003 3:10 am
Re: How can i replace the optimization algorithm?
You will need to reinstall Vensim. There are other files required, it may be that removing model reader removed other files that are needed by DSS.
Install reader, and then install DSS as then you will have all the correct files in the correct places.
Install reader, and then install DSS as then you will have all the correct files in the correct places.
Advice to posters seeking help (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Units are important!
http://www.bbc.co.uk/news/magazine-27509559
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Units are important!
http://www.bbc.co.uk/news/magazine-27509559
Re: How can i replace the optimization algorithm?
Thanks a lot It works
Re: How can i replace the optimization algorithm?
Hi, another question.
If i would like to execute the compilation with external functions, is it possible?
If yes, with which one? I haven't found it.
Thanks a lot
If i would like to execute the compilation with external functions, is it possible?
If yes, with which one? I haven't found it.
Thanks a lot
-
- Super Administrator
- Posts: 4590
- Joined: Wed Mar 05, 2003 3:10 am
Re: How can i replace the optimization algorithm?
I don't understand your question. Can you explain in a little more detail?
Advice to posters seeking help (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Units are important!
http://www.bbc.co.uk/news/magazine-27509559
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391
Units are important!
http://www.bbc.co.uk/news/magazine-27509559