Precision question

Use this forum to post Vensim related questions.
Post Reply
billh
Member
Posts: 45
Joined: Tue Jan 10, 2017 11:48 pm
Vensim version: DSS

Precision question

Post by billh »

I've attached a simple model that replicates an issue I'm having in a larger model. It's got three stocks, initialized to 315400, 5810, and 600 units. There's a Total variable that does what it implies.

However, it says the total is 321800, while my calculator and my brain say it should be 321810.

Is that the single-precision DLL at work, or is there something else going on? I'm running DSS 6.4E (x32).

As far as policy goes, this is a non-issue. I found it as part of a mass balance check to make it easy to see that I was initializing stocks to get the desired total. I knew the total I wanted (321800), and so I thought the initialization was working until I added the numbers manually.

Thanks,

Bill
Attachments
exampleprecision.mdl
(1.35 KiB) Downloaded 179 times
Administrator
Super Administrator
Posts: 4573
Joined: Wed Mar 05, 2003 3:10 am

Re: Precision question

Post by Administrator »

If you right click on the table tool and select "Scientific" instead of "Pretty" for the number format, does the correct value show up?
Advice to posters seeking help (it really helps us to help you)
http://www.ventanasystems.co.uk/forum/v ... f=2&t=4391

Units are important!
http://www.bbc.co.uk/news/magazine-27509559
billh
Member
Posts: 45
Joined: Tue Jan 10, 2017 11:48 pm
Vensim version: DSS

Re: Precision question

Post by billh »

Thanks--much better.

I learned something!

Bill
Post Reply