Optimizing with lookups

[LAUJJL]

Hi

Question about optimizing with lookups.

Lookup tables deliver continuous functions, but the derivatives of such tables are not
Continuous. I observed that in models with plenty of lookup tables, full optimizing needed
Millions of run and an RRandom or Random optimizing method.
I suspect that the reason comes from the non continuous derivatives that stop the
Powell optimizing process and needs a Rrandom restart.

I have an example where the optimizing process was just not working because I had a constraint where X <= 0 had to be respected in a model.
I change the problem with a penalty function of the form: penalty = if then else (X > 0,1000000,0) .
If you are trying to maximize for instance Z, you change Z to Z – penalty and you are sure that x will never be positive during the course of the optimization. It is the same constraints that you find in linear programming.
The penalty function is not continuous and the optimization in my model did not work well.
I changed the penalty to a continuous function using a lookup table, and the optimization still did not work. I changed than the penalty function to a continuously derivable function.
Penalty = if then else (X > 0 , (POWER( X , 3 ) * 10000), 0)
The function is continuous everywhere and derivable everywhere and eventually not when
X = 0, But a quick calculation proves that the left and right derivative is equal to 0 when x = 0.
When using this formulation for the penalty the optimization worked very well.
I then try not to use if then else, vmax functions or lookups, but it is very cumbersome and most of the times impossible.
One solution would be to make lookup tables continuously derivable, by using polynomial interpolations that are continuously derivable. Powersim uses these kind of interpolation
Except in multidimensional lookups where there is no interpolation (maybe new releases do it now, I tested the software more than a year ago.)
By making polynomial interpolations, optimizing would be probably more stable and would not sometimes deliver strange results.
I noted that the new version 5.6D performs much better when optimizing then the later one I was using the 5.5C and gives even slightly different results.
Regards.
JJ

[bob@vensim.com]

Hi JJ,

A couple of notes

Using a penalty function you are right - usually the best results come from something diffirentiable.

Internally - away from boundary conditions - continuity is what seems to matter. I have never seen a case where the piecewise linear nature of a function caused optimization problems.

Cublicly spined lookups are on the todo list but I have never really seen their value. The one case I though was convincing involved the derivative of the lookup being used to compute another variable. In this case the solution was to include in the model not the lookup, but its derivative and where the lookup would have been used use its integral. This is pretty easy to do with the VECTOR LOOKUP function. If you are optimizing it is probably easier to optimize on the derivative than trying to force a sensible shape to exist in the integral.

Finally, we do have some improvements to the optimizer in progress which may improve perfromance in the cases you are talking about.

[LAUJJL]

Hi Bob

Thank you for your reply.

I acknowledge that having continuously derivable function doest not add any value concerning the
results, but I thought that it could make the optimization process quicker.
I will make different tests to see if changing a sliced linear function to a derivable one can accelerate the optimizing process.
I have too cases where the optimizer can just not find the optimal solution in a reasonable time (one hour for instance), that was found by modifying the algorithm. But I did not try to make the computer work during several hours.
I am glad that the optimizer will still be better in the future.
Regards.
JJ