GET DIRECT LOOKUPS with subscript

[EwoutH]

I want to use subscripted lookup and tried using GET DIRECT LOOKUPS, but didn't manage to get it to work.

Basically I have a table in a CSV with years (time) on the horizontal axis and different age groups (which I want subscripted) on the Y-axis. I load in subscript values into a variable age_group using:

Code: Select all

GET DIRECT SUBSCRIPT('data/population_data.csv', ',', 'A2', 'A20', '')

I tried using GET DIRECT DATA with the subscript in population-pyramid-submodel-2.mdl, which doesn't load the data correctly over time. So I tried using GET DIRECT LOOKUPS subscripted, but that crashed and after that the model (population-pyramid-submodel-2.mdl) doesn't even open again.

So is it possible to load in lookup data subscripted? If not, what would be a work around? If so, how do you do it? If with GET DIRECT LOOKUPS, could a subscripted example model be added?

The models are attached below. The forum doesn't support attaching CSV files, the population_data.csv can be downloaded here: https://we.tl/t-ixFcuFTJkC

• GET DIRECT DATA with subscript, that doesn't load the table:

  population-pyramid-submodel-2.mdl

  (1.24 KiB) Downloaded 419 times

• GET DIRECT LOOKUPS with subscript, that doesn't open:

  population-pyramid-submodel-2b.mdl

  (1.25 KiB) Downloaded 405 times

• 

  Screenshot_707.png (85.84 KiB) Viewed 13016 times

[EwoutH]

I managed to fix it and use a GET DIRECT LOOKUPS with a subscript!

Could maybe an example model be added to the GET DIRECT LOOKUPS documentation? Since I believe it's non-trivial since you also have to define the subscript range with GET DIRECT SUBSCRIPT.

Fixed model:

population-submodel.mdl

(1.9 KiB) Downloaded 476 times

[Administrator]

I tried using GET DIRECT DATA with the subscript in population-pyramid-submodel-2.mdl, which doesn't load the data correctly over time.

What errors did you get when trying to load the data?

[tomfid]

The example models in the Help system should work in exactly the same way if you change XLS to DIRECT and resave the xlsx as csv.

[EwoutH]

What errors did you get when trying to load the data?

It just did a hard crash when I tried to open the 2b model. No idea why. Is there some way to get debug output?

The example models in the Help system should work in exactly the same way if you change XLS to DIRECT and resave the xlsx as csv.

A great to know! Maybe that can be mentioned in the docs itself?

[tomfid]

9.2 will have a crash reporter.

population_data.csv can be downloaded here

You can always zip an archive of model + supporting files.

[tomfid]

For the '2' version of the model, there are two issues:
- time axis of the model doesn't match the data
- function call is wrong: time is in row '1' not '2'

[tomfid]

2b works fine for me, after fixing the same issues. I'm currently running 9.2 though.

[EwoutH]

2b works fine for me, after fixing the same issues. I'm currently running 9.2 though.

Thanks for testing! Can you test if the population-submodel.mdl in the ZIP below works on your system with both Vensim 9.1.1 and 9.2? It now also crashes on opening on both a Windows and a macOS system, both using Vensim 9.1.1. Am I doing something wrong with reading the data?

A crash report would be very useful!

[tomfid]

This doesn't crash in 9.2, but doesn't run either. When I look at it with a text editor, I see:

Code: Select all

{UTF-8}
age group : GET DIRECT SUBSCRIPT ,'', 'A21', 'A2', ',', 'data/population_data.csv', 'data/population_data.csv'\
		, 
        ',', 
        'A2', 
        'A21', 
        ''
	~	People
	~		|

The function's parentheses are missing, and it seems like there are too many arguments.

[tomfid]

When I correct that to

Code: Select all

age group : GET DIRECT SUBSCRIPT( 'data/population_data.csv',
        ',', 
        'A2', 
        'A21', 
        '')
	~	People
	~		|

It works in 9.2.

[EwoutH]

When I correct that to

Code: Select all

age group : GET DIRECT SUBSCRIPT( 'data/population_data.csv',
        ',', 
        'A2', 
        'A21', 
        '')
	~	People
	~		|

It works in 9.2.

Thanks a lot, I adjusted the .mdl file manually, and that fixes the issue!

I think I also found the cause. We use the Verbose Equation-format to have readable diffs with Git. But apparently, the verbose saving doesn't correctly format the equation.

I have attached the manually fixed (correct) model and the verbose saved (incorrect) models below.

[tomfid]

Nice work. I'll put in a ticket to fix this. There's also a problem with reform/clean alphabetizing, so it may take a little while to rewrite the whole subsystem.

In the meantime, there are some new settings in 9.2 that may be of interest. See Model>Settings>File Format. You can move volatile settings (some of the stuff after :L <%^E!@) into a separate sidecar file to avoid false positive diffs. You can also switch to new storage for the sketch (via the >9.2 option) that avoids most spurious moves, even when using a model at multiple resolutions and scales.

[EwoutH]

Nice work. I'll put in a ticket to fix this. There's also a problem with reform/clean alphabetizing, so it may take a little while to rewrite the whole subsystem.

Thanks for putting in the ticket, do you have any update on it? It still persists in 9.2.0.

[tomfid]

It came to late in the release cycle to make the cut. I don't have a target date for a fix, because this is potentially complex. For the moment, the workaround is to avoid the Verbose setting, and use As Entered. Unfortunately this may cause at least a one-time large diff.

[EwoutH]

Thanks for getting back to me Tom. It looks for me on the surface like a relatively simple parsing issue, is it more complex than that?

[tomfid]

Short answer, yes.