Payoff sensitivity MCMC does not vary with payoff value (1.92, 3.84 or 6.63)

Use this forum to post Vensim related questions.
Post Reply
WayneZhou2009
Senior Member
Posts: 105
Joined: Wed Oct 25, 2017 3:52 pm
Vensim version: PRO

Payoff sensitivity MCMC does not vary with payoff value (1.92, 3.84 or 6.63)

Post by WayneZhou2009 »

Hi Tom,

I was trying to compare the samples resulting from standalone MCMC and payoff sensitivity MCMC, by following the Reference Manual (MCMC Options). While specifying standalone MCMC as the Optimizer is straightforward, I have encountered a problem with specifying MCMC as a method of payoff sensitivity analysis in the standard Powell optimizer.

I opened the .voc and specified the following (as instructed by the Manual):

:OPTIMIZER=Powell
:SENSITIVITY=payoff MCMC=6.63

Then I run it through Model - Simulate - Optimize (rather than directly clicking "Optimize - Payoff Definition - Optimization Control, which always reports an error regarding "SENSITIVITY=PAYOFF MCMC=6.63" - perhaps this is because PAYOFF MCMC is not integrated into the optimization control interface).

The real problem is that, the run results do not show any difference, regardless of the value for payoff MCMC (1.92, 3.84 or 6.63, which represents the X2 values for 90%, 95% and 99%). However, the results are clearly different from those from the standalone MCMC, as expected. Please see the attached figure.

How to deal with this issue? I believe the results for 1.92, 3.84, and 6.63 are supposed to be different.

Thank you so much.
Attachments
Scatter plot (Standalone MCMC vs Payoff sensitivity MCMC).jpg
Scatter plot (Standalone MCMC vs Payoff sensitivity MCMC).jpg (56.92 KiB) Viewed 1804 times
tomfid
Administrator
Posts: 3804
Joined: Wed May 24, 2006 4:54 am

Re: Payoff sensitivity MCMC does not vary with payoff value (1.92, 3.84 or 6.63)

Post by tomfid »

When you say "results" do you mean the sample points or the reported bounds?

I always use the standalone method and evaluate the sample directly these days. The main reason is that the bounds have limited direct utility - if there's any covariance among parameters you need the sample to get the joint distribution anyway.
WayneZhou2009
Senior Member
Posts: 105
Joined: Wed Oct 25, 2017 3:52 pm
Vensim version: PRO

Re: Payoff sensitivity MCMC does not vary with payoff value (1.92, 3.84 or 6.63)

Post by WayneZhou2009 »

Hi Tom,

Thanks for your advice. I meant the samples (datapoints). They are all exactly the same, regardless of the payoff value being set as 1.92, 3,84 or 6.63. However the bounds reported in the sensitive.tab are different, as follows:

:COM The base payoff is -7.46857
:SENSITIVITY = PAYOFF_MCMC = 1.92
PARAMETER MIN MAX

Shape parameter =2.92187 2.2660934 3.6048026
Scale parameter =33.619999 32.108877 35.247294


:COM The base payoff is -7.46857
:SENSITIVITY = PAYOFF_MCMC = 3.84
PARAMETER MIN MAX

Shape parameter =2.92187 2.1890468 3.9415925
Scale parameter =33.619999 31.499801 36.002111

:COM The base payoff is -7.46857
:SENSITIVITY = PAYOFF_MCMC = 6.63
PARAMETER MIN MAX

Shape parameter =2.92187 1.9968033 4.3559225
Scale parameter =33.619999 30.861094 36.871622


I agree with you that the bounds have limited direct utility. But I just wanted to make sure that I did things correctly. Maybe the fact the samples (datapoints) are exactly the same but the reported bounds are different is what I should have expected?

Thank you so much.
Post Reply