Reiterating Data

[geo_curious]

Hi
I would like to use an xls file that has 8760 data points for a model with a time of 87600. After each 8760 time steps, I would like Vensim to restart the reading of data point 1 in the xls file, proceeding to final time. I cannot seem to find any examples of doing this in the files or in the documentation. Is this possible? If so, how?

Thank you!

[tomfid]

There's no built-in way to do this, but I can think of some kluges, though they might not be very efficient. How many variables are in this dataset, and what is the purpose of the repetition? Knowing more might make it easier to choose a solution.

[Administrator]

Get data at time could possibly be used. You can use the modulo function to work out the time at which to get the data.

[geo_curious]

Thanks for the replies on this. I did finally figure it out based on @Administrators reply.

To fill in the blanks here, I am taking in an energy demand load profile over the course of a year.
The solution that works to repeat this data each 8760 hours appears as:
time index=
1 + MODULO(Time - 1, 8760)
node test=
GET DATA AT TIME( load profile, time index )

Node test then repeats to FINAL TIME without any problems. Thanks again! Big help.

[geo_curious]

I do have another question relating to this. Now that I have figured out how to read in and repeat the data, I have thoroughly broken the rest of my model. It seems the data type of the "node test" in the example above is no longer a "DATA" or "CONSTANT," resulting in other floating point errors popping up that are seemingly unrelated. My question is, therefore, how can I convert the repeating values of the "node test" into something that is equivalent to the Data>Equation>Interpolate data type that was working with GET DIRECT DATA()?

I hope this makes sense. Changing the data type output with GET DATA AT TIME() from the previous version of GET DIRECT DATA() has screwed up the model run.

EDIT: I figured this out, again. I put in an IF THEN ELSE statement to ensure TIME=0, then 0, else GET DATA AT TIME(). All good again.

[tomfid]

I would expect GET DATA AT TIME to have extremely slow performance - most of the GET DATA functions cause a noticeable slowdown, unless they are wrapped in an INITIAL (which wouldn't work here). If the model is small this may not be a problem.

[LAUJJL]

I would try downkoading the 8760 data in an array first and then use the modulo to make each time point equal to the corresponding element in the array

JJ

[geo_curious]

JJ, I am not sure I follow your suggestion here. Fundamentally, what is the change you are suggesting? Add the data to the uploaded datasets, then do what? Could you show me the equation change you are suggesting?

[geo_curious]

I would expect GET DATA AT TIME to have extremely slow performance - most of the GET DATA functions cause a noticeable slowdown, unless they are wrapped in an INITIAL (which wouldn't work here). If the model is small this may not be a problem.

So far, speeds have not been a problem. I am only running with a limited number of external datasets, for now. If I wanted to convert this to an app it may become a problem for timeout reasons. Mostly, I am happy to see things running correctly, as the problems the model is solving are pretty complex. Speed will have to be addressed at some point.

[LAUJJL]

Hi

Here is the model. i have given artitrary valeus to the array that has to be dosnladed from the excel file, something that is easy to do.

i do not know if it is quicker but the utility is that the process is clearer. One downlaods first the data in an array and then one makes at each time the value of the variable equal to the correponding value in the array. I use too the stricte unit test, which explains my equations.

JJ

[geo_curious]

I appreciate you taking the time to put that together JJ, but it does not make any sense to me, insofar as there are no actual numbers being used, other than matching strings to a time index. The loading of data with subscripts, generally the use of subscripts, and all things subscripts in Vensim is a complete blackhole for me. Even this example, I see there is no point because you are using strings and not numbers. All the examples in the manual point to subscripts as objects to be selected, not numbers to be input. I cannot even simply type numbers when I am to edit the range on sub, which is simply Subscript Normal. There is nothing intuitive about the subscript functionality, when in reality it is a simple for loop that should be easier to implement. Sorry, but that sparks a lot frustration in me about Vensim. I have said it repeatedly. It is a documentation problem.

Here is a challenge:

Data:
0.01, 0.1, 0.6, 0.2, 0.5, 0.3, 0.7,0.9,0.8, 1

Iterate it with a subscript, to produce the data as a repeating sequence for 3 iterations (30 timesteps). How hard can that be? It is impossible to me. That simple problem is something I do not understand at all in Vensim. Here is the implementation in Python:

Code: Select all

data = [0.01, 0.1, 0.6, 0.2, 0.5, 0.3, 0.7, 0.9, 0.8, 1]

repeated_data = []
for i in range(3):
    for value in data:
        repeated_data.append(value)

Maybe even more intuitive in Julia:

Code: Select all

for i in 1:3
    for value in data
        push!(repeated_data, value)
    end
end

[LAUJJL]

Hi

Joined the model with the 10 values. i do not use 'sub" as a value for the index as it probably looks weird for you.

Subscripts in Vensim are a bit difficult to understand especially the mapping system; But once you have understood it you will be able to model any logic without the use of an alternate priocedural language.

JJ

[geo_curious]

Thank you again for rearranging the equations for my understanding. I think this makes more sense. I will try to recreate the scenario myself, test the time differences between GET DATA AT TIME() and the loaded data variation. I will report this back here for others to consider.

[tomfid]

After further thought, I think the right way to do this is:

- Create a subscript for the hourly instances, like i : (i1-i8760)
- Read the data as constants into a subscripted variable, like
temperature = GET XLS CONSTANT( ... )
- Use VECTOR LOOKUP to read from that table, with x=Time modulo 8760.

[geo_curious]

Hi Tom, I looked at GET XLS CONSTANTS. I am not sure how that could possibly apply, since it only reads in one cell. These values are not constant over time and they are read in down or across cells, not from one. Looking at the subscript example for GET XLS CONSTANTS[ss].mdl, it confirms that nothing changes.

Further to this point, I set it up in the model, attempted a run, and found GET XLS CONSTANTS continuously trying to extract values from cell P2. The function input was for n2. So, it just does not work. Perhaps that is because the function does not like searching through sheets.

[tomfid]

Constants aren't supposed to change over time. My suggestion is to read the constants into an array, where the subscript serves as a virtual time axis. Then read from that array with VECTOR LOOKUP to get the time dimension.

[geo_curious]

I guess what I am saying is that this does not make any sense to me, since GET XLS CONSTANTS can only read 1 cell at a time and does not read any more than 1 cell.

This is fundamentally a dead issue for me, since I have something working with GET DATA AT TIME() using modulo. The alternatives are nice to haves, and inherently undocumented nice to haves, so I have to deprioritize these changes while I have so much more to do. It is very difficult to read between the lines on what needs to happen in the absence of good documentation.

The 10 datapoint example serves as a nice reference if you want to attempt to push a model here.

[LAUJJL]

Hi

Here is the complete challenge with an excel file having the values of the challenge and the modified model that reads the value from the excel file. What else do you need as this works well? You can replace the sum that I use by a vector lookup as Tom proposes.

JJ

[tomfid]

This is a VECTOR LOOKUP version.

repeatMonth.xlsx

(16.47 KiB) Downloaded 412 times

repeatMonth.mdl

(1.85 KiB) Downloaded 405 times

[geo_curious]

Thank you both for the replies. I will have a look this week, as time permits. As long as we are in the subject of data importation and iteration, I have found a potential solution to my simultaneous error - in another recent post - could be solved by using a large lookup. This lookup, however would require that I call a dimensionless value. This value would correspond with a response. These responses are mapped in giant jsons. Is anyone successfully using jsons directly in Vensim? I did a quick review of the manual from a mobile device and did not see anything explicitly mentioned. Would this be within the ODBC procedure?