R&D modeling and performance measures

[bh055@ctc.cummins.com]

I am interested in building a System Dynamics Model that develops a trend
for diesel engine use in the future specifically in terms of unit sales,
emissions requirements, horespower, fuel economy, etc. I then what to use
this data and the model to understand what R&D to pursue to both increase
market share and increase market size.

Does anyone have suggestions on resources that might help me structure this
model? Is it pratical to think that I can get a good enough understanding
of major trends in the future? Has anyone ever used System Dynamics to pick
R&D projects?

Note to Jim Hines: Maybe we should do this as a project in 15.875

On a related issue:
We are trying to put new Performance Measures in place to measure the
effectiveness of R&D, Manufacturing, and Product Developement. (For
those that are familiar with this we are building on Kaplans
Balanced Score Card work). It seems to me that if you had a model of
these processes you could pick out the Key Factors that dictate
success. These factors then would be used as the Performance
Measures. Does anybody have experience with this?

For those that have implemented measures before, perhaps you have
seen them initially succede, but then fail over time. Even if you
have not seen this, my experience is that the right performance
measures change over time (i.e. you use warranty cost as a measure
until your warranty costs are no longer bad, then you move up stream
to the controling factors that have emerged). It seems to me that a
Robust performance measurement system would involve building a system
dynamics model, understanding key control factors, implmeenting
performance measures, altering behavior, rebuilding the model (Do
loop).

Under such a scheme the performance measurement system would be
constantly reviewed and updated based on structural changes. Does
anyone have experience with this type of process? Does anone know of
companies using this kind of systematic approach to performance
measures?

J. Carter Williams
Email: carter.williams@ctc.cummins.com
Voice: 812-377-6683
Fax: 812-377-7808

[George backus]

TO: Carter Williams:

After three years of study, we are now implementing an R&D portfolio model for
the National Research Council of Canada (NRC) to determine R&D priorities at the
individual project level and to monitor the success of those project via leading
indicators. These indicators focus on dynamic conditions that indicate volatility
or high failure potential. One can then determine if more resources are needed to
cost-effectively reduce uncertainty, that the project scope needs changing, or
whether to drop the project before more resources are wasted. The prioritization of
a technology is based on its market behavior and market/economy-wide value. We are
working with Cambridge University (UK) who has a unique macroeconomic approach to
the differentiation of new products because of R&D. This captures the fact that
advanced characteristics can allow a technology to demand a higher price as well as
a larger market share.

Because of the uncertainty of success in R&D (and in the marketplace), the
HYPERSENS work Andy Ford has documented is used to make the model a
"risk-adjusted" portfolio and decision making tool.

Surprisingly, the orgins of this approach started nearly 20 years ago with a
similar request from your rival, Caterpillar.

If you are interested, please e-mail me directly and I will see if a direct
contact/arrangement with the NRC is possible.

George

George Backus Email: gbackus@boulder.earthnet.net
Policy Assessment Corporation phone: (303) 467-3566; fax: (303) 467-3576
14604 West 62nd Place Denver, Colorado 80004, USA

[jimhines@interserv.com]

On Fri, 2 Aug 96, bh055@ctc.cummins.com wrote:
>I am interested in building a System Dynamics Model that develops a trend
>for diesel engine use in the future specifically in terms of unit sales,
>emissions requirements, horespower, fuel economy, etc. I then what to use
...

Id be happy to explore this as a potential 15.875 project. (Note: 15.875
is MITs Application of System Dynamics course in which students take on real
projects for real companies).

15.875 projects tend to be framed a bit differently than Carters wording
might suggest to those who are unfamiliar with the course. The Standard
Approach taken in 15.875 is to first identify a problem in terms of several
reference modes; then, articulate a number of dynamic hypotheses (i.e. loop
hypotheses); then examine the hypotheses one by one in a growing simulation
model. The ultimate purpose is to help managers shape the future.

For example a principle set of reference modes might show the several
possible future trends in diesel demand. The loops would then represent
explanations for the different possible futures.

Althernatively, perhaps the focus is market share. In this case, principle
modes would include the possible paths along which market share can evolve
in the future. Loops would represent explanations for these different market
share trends.

As Carter indicates, it is possible that the focus is a companys unit
sales, which would include BOTH market demand AND market share. My guess,
though, is that the dynamics of market demand and market share are separable
enough to consider starting with one issue or the other; and then later (in a
new project?) considering the other half of the puzzle. Clearly the insights
are to some extent separable: An insight into how to increase market share
is worthwhile to pursue no matter what is happening to market demand. (As
long as it seems likely that pursuing the initiative will not adversely
affect market demand).

As Carter suggests, the ultimate purpose is not to forecast, say, future
demand for diesel sales, but to understand what managers can do to make it
more likely that the more desirable future comes to pass (e.g. make it more
likely that market share rises than falls).

Regards,
Jim Hines
JimHines@Interserv.Com
LeapTec and MIT

[jimhines@interserv.com]

To George Backus

Could you describe the modeling approach(es) that underlie your work? In
particular Im wondering what a "portfolio model" is. Is it a "typical"
system dynamics model (i.e. Is feedback central; is it largely endogenous;
does it use continuous time; is it deterministic)?

Thanks.

Jim Hines
JimHines@Interserv.Com

[George backus]

To Jim Hines/George Richardson:

The model is a rigorous SD: endogenous, deterministic, continuous time, with
feedback on market shares (qualitative choice theory), supply/demand
(stocks-and-flows) and economic growth (cointegration). (The cointegration part is
structurally SD but uses a-theoretical economics -- per the British. Thus, not
it is not really SD to us old-school purists.) The specification of the
gross portfolio of technologies is exogenous per R&D proposals. The model then uses
a latin-hypercube to sample the programs that could promote a technology, the
characteristics that cause market acceptance, and the uncertainty in in all. The
runs from the sampling are then analyzed to find that subset mix (ala portfolio
model) of technologies and associated programs that spend all the govts R&D budget
with the highest "expected" impact on country wealth.

It should be noted that the number of programs seems to be the key factor. Too few
increase the risk of total lose from uncertainty in any success; too many increase
risk that funding is too diluted to accomplish the required R&D effort. Thus, (it
appears with yet limited model experience) that the "portfolio" with the highest
expected-value of wealth creation is also the one with the least risk. It almost
rationalizes quantum mechanics, where a particle takes ALL paths but the highest
probability is in the "path of least effort." Maybe the quantum world the defies
causal logic is simply a causal-SD market competition model -- particles competing
for space and energy market share.

After having said it ever so esoterically above, the model really just simply takes
the 20 year old market share algorithms (and other baggage) we have in the ENEGRY
2020 models and applies them to something other than energy technologies.

Since this is for Canada, I hope to get one of Joel Rahns Ph.D. students a well
paid government job, if I can get Joel interested. Then maybe this work would see
the light of day.

If no one cuts the (my) budget on documentation, a report should be available at
the end of the year on the first phase effort. I believe that it could be
made available for general distribution.

G.

George Backus Email: gbackus@boulder.earthnet.net
Policy Assessment Corporation phone: (303) 467-3566; fax: (303) 467-3576
14604 West 62nd Place Denver, Colorado 80004, USA