Modelling the effects of data quality

This forum contains all archives from the SD Mailing list (go to http://www.systemdynamics.org/forum/ for more information). This is here as a read-only resource, please post any SD related questions to the SD Discussion forum.
Locked
"George Backus"
Member
Posts: 33
Joined: Fri Mar 29, 2002 3:39 am

Modelling the effects of data quality

Post by "George Backus" »

Elaine,

I think we have "talked" before. We use HYPERSENS to determine the value=
of
data. (You can look in the SD Bibliography under Andy Ford, Jeff Amlin, a=
nd
myself to find references to HYPERSENS use in the electric utility
industry.
Andy also notes it in the appendix of his new book.)

When you have a feedback model, the interactions limit the parameters tha=
t
actually have impact. The system compensates for variance in most of the
parameters, i.e., they have little impact on results. With HYPERSENS, we
perform a relatively small series of runs based on Latin-Hypercubic sampl=
ing
of the parameter space. This parameter space can include raw data, such =
as
physical and financial data of the utility, customers and operations, as
well as the undecided decisions of competitors and (de)regulators. You =
can
then relate the uncertainty in inputs to the changes in any selected outp=
ut,
over time. Specifically you can compare the value of having better data o=
n
the such things as revenue, net income, ROI, market share, the ability to
accommodate deregulation/competition (stock prices), etc. If you are
comparing better knowledge to net income, you have an exact statement of =
how
much you should pay to obtain better data =96 if anything. Our work indic=
ates
that most of the data utilities worry about most do not have any
statistically significant affect on utility performance =96 after compens=
ation
for market counter responses are considered. For example, there is often
very little value in customer data. (We showed that a $10M Xenergy survey
would, at best, save the company $100K =96 but would most likely cause a =
lost
of money because with the data, the company would act in a manner that
increased financial risk elsewhere.)

HYPERSENS not only quantifies what is important, (the parameters and thei=
r
impacts), it places confidence intervals around the results so you can
clearly see the probabilities of what is possible. Most importantly it sh=
ows
what is not possible. We continue to find that most utility thinking abou=
t
deregulation focuses on impossible conditions.

As a side note, what is important changes over time. Inflation is importa=
nt
in the short-term, but not in the long-term. Deregulation transition issu=
es
are important for less than 4 years. HYPERSENS has a timing filter that
captures the changes in regimes over time so that you can clearly see tho=
se
few things that are important now, and those that will be important later.
In all cases, the number of key parameters (uncertainties) is small (six =
or
less) and the analysis usually indicates that the variables you currently
consider important aren=92t and those you think are irrelevant end up as
crucial (dividend policy being a prime example!).

We use ENERGY 2020 for our work because it has the customer response for =
all
fuels by end-use at a detailed economic sector level. (e.g. 2-digit SIC,
single family, multifamily, rural, etc.). It also simulates the competito=
rs
around you as well as new entrants. (It is historically validated and
parameterized from 1988 for utility data and 1975 for customer data.) We
have separated each energy supplier into its separate distribution,
transmission, generation, and marketing/trading units. We have essentiall=
y
all of the non-negligible (over 3500) suppliers in the US and Canada
simulated. Both physical and financials, as well as transmission constrai=
nts
are simulated. We use the RDI database on the US side and the CANSTATS
database on the provincial side. We have used the model many times to lo=
ok
at markets, deregulation, and M&A options in the Midwest.

I under a separate email I have sent some information on ENERGY 2020 that
does include some HYPERSENS discussions. I can provide more information t=
o
you as desired, but the analysis does require a feedback based model of t=
he
system =96 otherwise the =93value of data=94 analysis is not only meaning=
less, it
is can be dramatically counter productive.

G.

George Backus, President
Policy Assessment Corporation
14604 West 62nd Place
Arvada, Colorado, USA 80004-3621
Bus: +1-303-467-3566
Fax: +1-303-467-3576
Cell: +1-303-807-8579
George_Backus@ENERGY2020.com
Locked