On information quality and appropriateness

["Jay W. Forrester"]

On June 23, 2001, in (SD2234), Elaine Lovitt asked about modeling the
"effects of data quality on the work we perform.... if any work has
been done on value of information or data quality modelling" I
regret being so late in responding.

There are two issues here--first, the appropriateness of particular
data streams, and second, the characteristics (multiple measures of
quality) of a data stream.

Information pollution may be one of the most serious forms of
pollution. People are flooded with information, only a small
fraction of which should be used. From the flood of information,
people often choose inappropriate information as the basis for
decision-making. Faced with a flood of information, people base
decisions on what is available, without asking if even more important
information is not being presented for use.

Probably the only reliable way to judge the appropriateness of a
particular kind of information is to imbed it in a system dynamics
model to evaluate the effect of policies that respond to that
information and then consider alternative policies based on other
information inputs. Without a systems basis for evaluating
information, I believe that much of the "information technology"
activity may be wasteful or actually detrimental.

With respect to the quality of information, one must define the
characteristics of quality before proceeding. Just as one cannot
discuss the quality of an automobile in the abstract without
identifying reliability, gas mileage, stability, and durability, one
must also identify the measures of quality in information. As a
start on such a discussion, consider six characteristics of
information:

Persuasiveness
Delay
Distortion
Bias
Error
Crosstalk.

All of these terms seem to imply being desirable of undesirable, but
they should not be so construed. Under various circumstances, one
may want more or less of each one.

Persuasiveness relates to the degree of influence of the particular
information stream. There seems to be a correlation between
persuasiveness and "hardness" of information, that is, the extent to
which the information is conveniently and accurately measured. But
carefully measured information can often be much less important than
episodes, complaints, and suggestions that may be offered at a social
gathering, or by telephone to someone other than the person who can
best use the information.

Delay represents the time lag between "true" facts and when people
believe and act on the information. Delay includes the time to
change attitudes and to overcome traditional beliefs. Delay in
information streams can range from hours to decades.

Distortion is closely related to delay and refers to altering the
shape or time profile of information. For example, one averages (or
smooths) information to suppress short-term fluctuations and retain
longer-terms changes. Such processes also cause delay. Often people
try to reduce delay unnecessarily and thereby create undesirable
instability in a system.

Bias is the consistent process of interpreting information at an
offset from its true meaning. Bias often appears as interpreting
market information more favorably than justified. Example: a company
believes that it can not satisfy all the customers, that their goal
is to satisfy those who buy 90 per cent of their products. When a
favorable report comes in, naturally it came from one of the 90 per
cent and is posted on the bulletin board. When a strong negative
message arrives, obviously it came from one of the ten per cent that
should not have bought the product or who misused it, and the message
goes in the waste basket. No learning occurs because the information
channel is biased to see only favorable information.

Error is the random transposition of digits in a number or a mistaken
observation. More effort is devoted to eliminating error than for
dealing with any of the other information characteristics. However,
error is probably the least important of the six. A large error will
be recognized as such. Small errors will have little effect and will
be corrected at the next data handling. A simulation model will
usually confirm that random error is not the most important defect in
an information channel.

Crosstalk is a telephone term and I do not know of a suitable
alternative word. Seldom with todays improved telephone systems, but
frequently years ago, one might hear the conversation between two
other parties because of signals being transferred from one circuit
to another. Crosstalk is especially likely to happen in information
streams coming from the market back to a company. Consider the
following scenario. I have purchased a piece of equipment for use in
my office. It squeaks, leaks oil on the carpet, and requires too
much service. I will not buy another from that company. The
salesman returns to sell another. Do I tell him the real reason for
not buying? Perhaps not because he will counter that the designs have
been changed and he has the discussion advantage, so I tell him the
price is too high, which is my value judgement. But suppose I do
tell him the real reasons; he has had the experience of complaining
about the product to the management and been told that is none of his
business; so, he goes home and reports that the price is too high.
But suppose he does carry back the correct message; the management
does not know how to correct the problems quickly; so they reduce the
price. At several stages the true and important information is
vulnerable to being converted into a price signal.

Information sources are treacherous, misleading, and not sufficiently studied.
--
---------------------------------------------------------
Jay W. Forrester
Professor of Management
Sloan School
Massachusetts Institute of Technology
Room E60-389
Cambridge, MA 02139
From: "Jay W. Forrester" <jforestr@MIT.EDU>

["Thompson, Jim A-136"]

In SD3612 Prof. Forrester wrote: "As a start on such a discussion, consider
six characteristics of information: Persuasiveness, Delay, Distortion, Bias,
Error, and Crosstalk. All of these terms seem to imply being desirable or
undesirable, but they should not be so construed. Under various
circumstances, one may want more or less of each one."

However, the instruction to not construe the listed characteristics of
information as desirable or undesirable is difficult to impose in a
commercial and multidiscipline environment, and I would like to incorporate
at least the flavor of Prof. Forresters thinking into my work. To make
this possible, it would be helpful to establish continuums for the
characteristics. I made a first attempt at this:

Persuasiveness: ambiguous <-----> compelling
Delay: stale <-----> timely
Distortion: distorted <-----> unclear
Bias: biased <-----> impartial
Error: error-filled <-----> accurate
Crosstalk: confusing <-----> coherent

I acknowledge that this shorthand does not do justice to the richness of
Prof. Forresters descriptions and would appreciate help in remaining
faithful to the principles he suggests.
Jim Thompson
Medical Cost Analysis & Forecasting
CIGNA Information Management
900 Cottage Grove Road, A136
Hartford, CT 06152
From: "Thompson, Jim A-136" <Jim.Thompson@CIGNA.COM>

[Jay Forrest]

Hi Jim!

My immediate response to your list is that, in contradiction to Prof.
Forresters suggestion of not viewing his list of elements as "desirable" or
"undesirable", you selected scales for these words which frequently carry
clear positive and/or negative connotation. Your choice of words seems to
carry a lot of "values" and some of your word selections baffle me (such as
bounding distortion with distorted to unclear). I would suggest you revisit
your list, clarify your thinking, and develop a more neutral and meaningful
set of scales. (Note: I attempted to respond, but some of your suggestions
baffled me and I elected not to interpret your words and to overlay your
biases and connotations with my own.)

You should also note that these characteristics are not independent but
carry mutual dependencies. An extended list might include other
characteristics such as Persistence, Pertinence (value -- a perceived
characteristic, not generally inherent), and Demands (the
skills/experience/knowledge necessary for the information to be recognized
and used).

From: Jay Forrest <jay@jayforrest.com>

[Alexander Leus]

It seems like we are talking about the utilization of "fuzzy logic",
that gray area between the black and white areas. Fuzzy logic is
utilized in many areas of engineering design applications from space
crafts to washing machines and even in the area of law. Am I on the
same page as Professor Forrester and Mr. Thompson?

When I look at Mr. Thompson example I think of the concepts of "Yin &
Yang".

Sincerely,
Alex Leus
leusa@tds.net