Suggestions for Temporal Superposition

[geo_curious]

Hi all,

I am looking for implementation solutions for a problem involving time and superposition. In determining the position at each step for a variable T, I have to determine the conditions of an increasing number of inputs to the T equation with each successive timestep.

Here is an example, recalling explicit terms at each timestep to solve for T, where t=Time:

temporal_superposition.png (57.32 KiB) Viewed 53775 times

Here g is a function. The number of terms between Ts and "+q4*Rb" would increase with each new timestep.

I am still searching for valid alternatives, as I do not think temporal superposition calculations are reasonable in Vensim beyond anything more than a couple timesteps. If someone has seen a successful implementation of such problems in Vensim, I would love to hear/see more.

Thank you!

[Administrator]

I wouldn't have a clue about this. But if you can explain what you are trying to do, there may be a way.

[tomfid]

I assume we're not talking about superposition in the quantum sense.

From one point, it's not clear how the terms evolve. For example,
How would this be defined at t = ts?
Does addition of terms imply that Tf,5 = Ts + blabla + 1/2piks( ... ) * g( (t5-t0)/ts )?
Is it recursive, so Tf,n = Tf,n-1 + one more term?

If the number of terms is modest, it's possible to use some trick like putting them in an array. You can generate the code using Excel or ChatGPT. But that's obviously not general or suitable for an arbitrary number of terms.

[geo_curious]

Hi Tom,

You are correct. Definitely this is not quantum anything. It is linear algebra with an increasing number of terms in the blabla part of Ts + blabla *g((evolving time term)/ts). Something like complex addition for nonlinear time discrete systems.

Yeah, I wish maybe there was a python client for Vensim because if this. That might make this easier. I did ask around all week about alternatives to the blabla, but it appears there are no real options for this, since they are history terms. Without tracking the history terms (blabla) then I am kind of SOL. It is one of the last things I have to do for my model, a multi year saga, which bums me out.

[geo_curious]

I am not sure this helps, but I will also encounter another problem, because of these equations - solution time. As you can imagine, I have only shown you the Tf value for the 4th timestep of a model run. With hourly runs, run out to 30-50 years, that would result in more than 262,000 additional problems to solve within the same equation. Strategies involve binning/averaging the inputs for blabla into blocks. Even if I can perform the binning in Vensim, perhaps using a stock, I will have to figure out the superposition, preferably for any number of timesteps.

[tomfid]

When I see something like this, I immediately think "hidden states". Unless this is something really pathological, there can't really be 262000 meaningful terms in the expression. Presumably there's some underlying system described by this equation that could be described at a more fundamental level? But perhaps it's poorly understood, so only an empirical description is available?

[geo_curious]

I sent in a sample copy of how this works in Excel. Hopefully that helps to clarify some things. My greatest challenge is that outputs at the present timestep need to be multiplied by outputs from the initial timestep, and everything in between. Indeed, there are many terms that need to be added with temporal superposition. It is, so far, the engineering standard for the field I am studying, because other methods (FEA, FVA, FD, etc.) incur even greater computational time. I can do it on a spreadsheet, but I would like to make it dynamic in Vensim.

[geo_curious]

In effect, what I need to do is multiply two columns by each other. Each column changes with new timesteps. One column will need to be reversed and then both columns are multiplied by each other. The sum of the multiplied columns is the answer I need - at each timestep.

exampleV2.xlsm

(23.71 KiB) Downloaded 182 times

[geo_curious]

Perhaps I need to punch together an external function? Would it be possible to feed the external function the variable as a list? Example - feed list1 and list2 into external function; external function generates an inverse of list2, multiplies list1 by list2, then sums list output. Is that a reasonable task for an external function?


I have not written anything in c++ before, but I can try. I read the basic tutorial on DLL generation from VSCode, using MinGW. It seems doable, but I will need a little help. Before I go for that, however, let me know if the questions above can be answered.

[LAUJJL]

you could have more help if you explained the true nature of the problem and its envental link with the real world. Simplifying it considerably would help too.

[geo_curious]

you could have more help if you explained the true nature of the problem and its envental link with the real world. Simplifying it considerably would help too.

The application is heat exchangers. The only calculation that needs to be made is demonstrated in the yellow highlighted columns on the Excel sheet I have provided - as the clearest example I can provide. As can be seen in the Excel sheet, the following must happen:
feed list1 and list2 into external function; generate an inverse of list2, multiplies list1 by list2, then sums list output
This needs to happen with each time step.

Feel free to respond to that, knowing now the simplifications requested.

[geo_curious]

I have generated the script with an AI engine, after feeding in the vensim.h, I get the following .cpp to compile. Unfortunately, I get an error message after loading Vensim with the external functions .dll on path - "Bad external func file."

Code: Select all

#include <vector>
#include <algorithm>
#include <numeric>
#include "vensim.h"

// Define function index
#define REVERSE_AND_MULTIPLY_SUM_FUNC 0

extern "C" {

// Function implementation
int reverse_and_multiply_sum(
    int n_values,
    const double* list1,
    const double* list2,
    double* result
) {
    if (n_values <= 0 || list1 == nullptr || list2 == nullptr || result == nullptr) {
        return 1; // Error: Invalid inputs
    }

    std::vector<double> vec1(list1, list1 + n_values);
    std::vector<double> vec2(list2, list2 + n_values);

    // Reverse first vector
    std::reverse(vec1.begin(), vec1.end());

    // Compute sum of element-wise multiplication
    *result = std::inner_product(vec1.begin(), vec1.end(), vec2.begin(), 0.0);
    return 0; // Success
}

// Version information
int VEFCC version_info() {
    return EXTERNAL_VERSION; // Defined in vensim.h
}

unsigned short VEFCC funcversion_info() {
    return 1; // Custom version for this function
}

// Register function metadata
int VEFCC user_definition(
    int i,
    unsigned char** sym,
    unsigned char** argument_desc,
    int* num_arg,
    int* num_vector,
    int* func_index,
    int* dim_act,
    int* modify,
    int* num_loop,
    int* num_literal,
    int* num_lookup
) {
    if (i == 0) {
        // Define the function's metadata
        *sym = (unsigned char*)"REVERSE_AND_MULTIPLY_SUM";
        *argument_desc = (unsigned char*)"{n_values}, {list1}, {list2}";
        *num_arg = 3;        // Number of arguments
        *num_vector = 3;     // Number of vector arguments
        *func_index = REVERSE_AND_MULTIPLY_SUM_FUNC;
        *dim_act = 0;
        *modify = 0;         // Does not modify arguments
        *num_loop = 0;
        *num_literal = 0;
        *num_lookup = 0;
        return 1; // Success
    }
    return 0; // End of function list
}
}

I will continue to dig through vensim.h for reference, but I would appreciate some guidance on this. Also, I am still not sure if I can take in the variables as a list when feeding them to this eventual function. Happy New Year!

[LAUJJL]

Hi

Two questions: is the calculation in Excel correct and sufficient?

if this is the case, Why do you want to calculate it in Vensim?

Regards and happy good year for all!

JJ

[geo_curious]

This is something I have been working on for more than 2 months. My need to perform these calculations inside of Vensim is not arbitrary and has been covered in this and other threads.
1. Sufficient? Yes, for one static scenario (there are 5 major configurations and limitless other minor parameterizations which can change these results) at 1 timestep it is functional. There are 8760 hours (timesteps) in a year, and this needs to be dynamic over 30-70 years. Excel is not an option for a system analysis.
2. Without a doubt, this needs to be directly tied to Vensim, either through organic functions or through the use of external functions.

[tomfid]

The function code might be OK, but it needs to be wrapped in the Vensim external function infrastructure (there's an example in the samples directory tree).

[tomfid]

If this can be done in Excel, it almost certainly can be done in Vensim arrays, which would be a lot less work than the xfn route.

Is this by any chance heat transfer in a ground loop? Seems like a simple FEA would be a lot easier and more transparent than the "reduced" form above.

[geo_curious]

An FEA would increase the computational time beyond reason. I know, I have run the FEAs individually. The point is that an FEA or this equation is only one small component of a much larger system. I need to keep computational time low and within reason to run sensitivity. I do not see that this is possible with arrays in Vensim. I need to invert one of the lists, which I have not seen a built-in function for in Vensim. Also, the above equation form is validated with FEA and is the industry standard in many, many other pieces of ground loop software around the world.

Happy to be overruled here if you can point me in the right direction.

Also, I am not sure I follow about the wrapper. Am I supposed to put the function into the external_function.c file? I only know how to compile the .cpp Should I put the entirety of the example into the .cpp, then add the function, then compile the .dll?

[LAUJJL]

Hi

I have made models using dynamic prkogramming methods and I needed to inverse the time, starting at the end of a model and going back to the present. It may help you. To see the threads, make an advanced research on the forum with LAUJJL as the author, with Vensim and the words dynamic programming to see different threads on that topic.

Best

JJ

[geo_curious]

JJ, I went on the treasure hunt for this thread you are talking about, but could not find it. Is it this one? viewtopic.php?t=2461&hilit=array

[LAUJJL]

Hi

Your suggestion is niot the right one;

Below is a list of threads that mention system dynamics;

optimization - Ventana software support forum

Maximum value - Ventana software support forum

order of calculation of subscripts - Ventana software support forum

Discrete Event Probability - Ventana software support forum

NPVE, change TIME STEP and FIND ZERO - Ventana software support forum

Maximizing principles - Ventana software support forum

Maximizing principles - Ventana software support forum

[geo_curious]

Thank JJ. I see the only relevant thread as being "order of calculation of subscripts" from 2018. I am not using subscripts that need to be reversed. They are outputs from subscripts, for sure, but it is an interpolation at each timestep, that changes based on tabular data and parameterization.

Recalling again, the task is:

1. Take in two auxiliary variables as lists of outputs up to the present timestep, gives 2 lists
2. Reverse one of the 2 lists
3. Multiply the two lists by one another at the new index value of each list
4. Sum the new output (a third list)
5. Repeat for each timestep

I looked at the essai.mdl and it is only manipulating based on subscript values, which is not something I can do because it is effectively predeterminant, or strictly causal. I have to be able to accept parameterization which will change, resulting in novel interpolation values from subscripts that output as auxiliary variables. Subscripts are not the way I can handle and manipulate this data, so far as I can tell. I would ask those that respond to review the .cpp function in some detail, because it is intended to execute the 5 steps above. I am still unclear as to whether an auxiliary variable will be released by Vensim to the external function, if it is called. If there is a specific way of calling the whole list of auxiliary variable outputs, please let me know. Perhaps this is something Administrator is more familiar with.

[geo_curious]

Following up on Tom's comments, I have tried dumping the function into the vensim_external_function.c file, then renaming it to " ".cpp. I get a bunch or warnings upon compile:

Code: Select all

$ g++ -shared -o vensim_functions.dll vensim_external_functions.cpp -Wl,--output-def,vensim_functions.def
vensim_external_functions.cpp:191:42: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
  191 |                                         {"COSINE"," {x} ",1,0,COS_FUNC,0,0,0,0},
      |                                          ^~~~~~~~
vensim_external_functions.cpp:191:51: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
  191 |                                         {"COSINE"," {x} ",1,0,COS_FUNC,0,0,0,0},
      |                                                   ^~~~~~~
vensim_external_functions.cpp:192:42: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
  192 |                                         {"INRANGE"," {x} , {minval} , {maxval} ",3,0,INRANGE_FUNC,0,0,0,0},
      |                                          ^~~~~~~~~
vensim_external_functions.cpp:192:52: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
  192 |                                         {"INRANGE"," {x} , {minval} , {maxval} ",3,0,INRANGE_FUNC,0,0,0,0},
      |                                                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
vensim_external_functions.cpp:193:42: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
  193 |                                         {"PSUM"," {vector} , {nelm} , {nelmlimit} ",3,1,PSUM_FUNC,0,0,0,0},
      |                                          ^~~~~~
vensim_external_functions.cpp:193:49: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
  193 |                                         {"PSUM"," {vector} , {nelm} , {nelmlimit} ",3,1,PSUM_FUNC,0,0,0,0},
      |                                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
vensim_external_functions.cpp:194:42: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
  194 |                                         {"MATRIX_INVERT"," {matrix} ",1,1,INVERT_FUNC,2,0,0,0},
      |                                          ^~~~~~~~~~~~~~~
vensim_external_functions.cpp:194:58: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
  194 |                                         {"MATRIX_INVERT"," {matrix} ",1,1,INVERT_FUNC,2,0,0,0},
      |                                                          ^~~~~~~~~~~~
vensim_external_functions.cpp:195:42: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
  195 |                                         {"MATRIX_INPLACE_INVERT"," {matrix} ",1,1,INPLACE_INVERT_FUNC,0,1,0,0},
      |                                          ^~~~~~~~~~~~~~~~~~~~~~~
vensim_external_functions.cpp:195:66: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
  195 |                                         {"MATRIX_INPLACE_INVERT"," {matrix} ",1,1,INPLACE_INVERT_FUNC,0,1,0,0},
      |                                                                  ^~~~~~~~~~~~
vensim_external_functions.cpp:196:42: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
  196 |                                         {"INTERNAL_ROR"," {x} , {time} , {minror} , {maxror} , {streamid} , {compute} ",6,0,INTERNAL_ROR_FUNC,0,0,0,0},
      |                                          ^~~~~~~~~~~~~~
vensim_external_functions.cpp:196:57: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
  196 |                                         {"INTERNAL_ROR"," {x} , {time} , {minror} , {maxror} , {streamid} , {compute} ",6,0,INTERNAL_ROR_FUNC,0,0,0,0},
      |                                                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
vensim_external_functions.cpp:197:42: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
  197 |                                         {"MYMESSAGE"," {'message'} , {time} ",2,0,MYMESSAGE_FUNC,0,0,1,0},
      |                                          ^~~~~~~~~~~
vensim_external_functions.cpp:197:54: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
  197 |                                         {"MYMESSAGE"," {'message'} , {time} ",2,0,MYMESSAGE_FUNC,0,0,1,0},
      |                                                      ^~~~~~~~~~~~~~~~~~~~~~~~
vensim_external_functions.cpp:198:42: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
  198 |                                         {"MYFINDZERO"," {vector_to_zero} , {nelement} ",2,1,MYFINDZERO_FUNC,1,2,0,0},
      |                                          ^~~~~~~~~~~~
vensim_external_functions.cpp:198:55: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
  198 |                                         {"MYFINDZERO"," {vector_to_zero} , {nelement} ",2,1,MYFINDZERO_FUNC,1,2,0,0},
      |                                                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
vensim_external_functions.cpp:199:42: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
  199 |                                         {"MYLOOKUP"," {lookup} , {x} ",2,0,MYLOOKUP_FUNC,0,0,0,1},
      |                                          ^~~~~~~~~~
vensim_external_functions.cpp:199:53: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
  199 |                                         {"MYLOOKUP"," {lookup} , {x} ",2,0,MYLOOKUP_FUNC,0,0,0,1},
      |                                                     ^~~~~~~~~~~~~~~~~~
vensim_external_functions.cpp:200:42: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
  200 |                                         {"MYALLTYPES"," {'literal'} , {lookup} , {vector} , {x} ",4,1,MYALLTYPES_FUNC,1,0,1,1},
      |                                          ^~~~~~~~~~~~
vensim_external_functions.cpp:200:55: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
  200 |                                         {"MYALLTYPES"," {'literal'} , {lookup} , {vector} , {x} ",4,1,MYALLTYPES_FUNC,1,0,1,1},
      |                                                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
vensim_external_functions.cpp:201:42: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
  201 |                                         {"MYCONSTDEF"," {'literal'} ",1,0,MYCONSTDEF_FUNC,CONSTDEF_MARKER,0,1,0},
      |                                          ^~~~~~~~~~~~
vensim_external_functions.cpp:201:55: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
  201 |                                         {"MYCONSTDEF"," {'literal'} ",1,0,MYCONSTDEF_FUNC,CONSTDEF_MARKER,0,1,0},
      |                                                       ^~~~~~~~~~~~~~~
vensim_external_functions.cpp:202:42: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
  202 |                                         {"MYDATADEF"," {'literal'} ",1,0,MYDATADEF_FUNC,DATADEF_MARKER,0,1,0},
      |                                          ^~~~~~~~~~~
vensim_external_functions.cpp:202:54: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
  202 |                                         {"MYDATADEF"," {'literal'} ",1,0,MYDATADEF_FUNC,DATADEF_MARKER,0,1,0},
      |                                                      ^~~~~~~~~~~~~~~
vensim_external_functions.cpp:203:42: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
  203 |                                         {"REVERSE_AND_MULTIPLY_SUM", " {list1}, {list2}, {n_values} ", 3, 2, REVERSE_AND_MULTIPLY_SUM_FUNC, 0, 0, 0, 0},
      |                                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
vensim_external_functions.cpp:203:70: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
  203 |                                         {"REVERSE_AND_MULTIPLY_SUM", " {list1}, {list2}, {n_values} ", 3, 2, REVERSE_AND_MULTIPLY_SUM_FUNC, 0, 0, 0, 0},
      |                                                                      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

After putting the output .dll on path, I got several warnings from Vensim that the .dll could not be loaded. Without doing anything else, I closed and reopened several times, to see if the error message would disappear. After closing and reopening 4 times, the Vensim error message went away and the .dll file for external functions appears to have loaded. The functions are now available in the user defined section of the equation editor. Will begin reviewing whether this thing can actually work for what I need it to do now. I am a bit worried about what these warnings mean for a conversion to strings, however.

[geo_curious]

REVERSE AND MULTIPLY SUM( {list1}, {list2}, {n_values})
Using the above function, such that list 1 is the energy difference between time steps and list two is a shape factor (also known as a g-function output in industry specific terms) read from interpolated Excel tables, and n is Time, I am getting an error from Vensim with a crash after 3 timesteps. Inputs appear as:
REVERSE AND MULTIPLY SUM( "qi-qi-1", gfunction output, Time)

data from the first 3 timesteps for "qi-qi-1":
0 = 0; 1 = 0; 2 = -5.79858; 3 = 1.55211

data from the first 3 timesteps for gfunction output:
0 = 2.49682; 1 = 2.49682; 2 = 2.49682; 3 = 2.49682

data from the output of REVERSE AND MULTIPLY SUM():
0 = 0; 1 = 0; 2 = -14.478; 3 = 15.278

I cannot say definitively if the function is working correctly, since the simulation has not run long enough for an evolution in the gfunction output, but it is promising. I have called the variable containing the REVERSE AND MULTIPLY SUM() function "multiply sum." I get the following output errors from Vensim during the run:

ERROR: Invalid inputs to REVERSE_AND_MULTIPLY_SUM.
ERROR: Invalid inputs to REVERSE_AND_MULTIPLY_SUM.
ERROR: Invalid inputs to REVERSE_AND_MULTIPLY_SUM.
ERROR: Floating point error computing - multiply sum - at time = 3.000000.
INFO: Trying to save the results anyway

So, my questions are, why are the inputs being reported by Vensim as "invalid" and what could be the floating point error at timestep 3, since the function is not dividing by zero - or at least that is not what I intended anywhere?

[geo_curious]

Offering an update on the implemented function here for review by others:

Code: Select all

double REVERSE_AND_MULTIPLY_SUM(VECTOR_ARG *list1, VECTOR_ARG *list2, int n_values) {
    if (!list1 || !list2 || n_values <= 0) {
        (*VENGV->error_message)(VERROR, (unsigned char *)"Invalid inputs to REVERSE_AND_MULTIPLY_SUM");
        return 0.0;
    }

    validate_vector_arg(list1, 0, n_values - 1);
    validate_vector_arg(list2, 0, n_values - 1);

    double sum = 0.0;

    // Reverse list1 and compute the element-wise product with list2
    for (int i = 0; i < n_values; i++) {
        sum += list1->vals[n_values - i - 1] * list2->vals[i];
    }

    return sum;
}

[tomfid]

An FEA would increase the computational time beyond reason. I know, I have run the FEAs individually. The point is that an FEA or this equation is only one small component of a much larger system. I need to keep computational time low and within reason to run sensitivity. I do not see that this is possible with arrays in Vensim. I need to invert one of the lists, which I have not seen a built-in function for in Vensim. Also, the above equation form is validated with FEA and is the industry standard in many, many other pieces of ground loop software around the world.

Obviously id depends on how big the FEA is, but if there's a good reason to use the above form, then it makes sense to do so.

The code in the Excel file still seems underspecified to me, because a bunch of the columns are numbers rather than expressions. However, if "invert" means that you just need:
i : (i1-i5)
y[i1] = x[i1]*z[i5]
y[i2] = x[i2]*z[i4]
...
y[i5] = x[i5]*z[i1]
that's easy. You can create a second subscript with the elements in reverse order, or use VECTOR ELM MAP for arbitrary access to elements. I'd probably prefer the latter. The offsets for the elements are as in your xfn, list1->vals[n_values - i - 1] * list2->vals.

Also, I am not sure I follow about the wrapper. Am I supposed to put the function into the external_function.c file? I only know how to compile the .cpp Should I put the entirety of the example into the .cpp, then add the function, then compile the .dll?

I think the mixed approach isn't working because changing the wrapper extension from .c to .cpp causes it to compile in C++, which it isn't.

You don't necessarily need to put your function and the rest of the wrapper in one file; you can create a dll project with the c wrapper and cpp function in separate files, but you'd still need to declare function arguments etc. as in the wrapper. Making it pure c is probably easier.