Mean of errors not close to 0 after optimization

Use this forum to post Vensim related questions.
Post Reply
WayneZhou2009
Senior Member
Posts: 105
Joined: Wed Oct 25, 2017 3:52 pm
Vensim version: PRO

Mean of errors not close to 0 after optimization

Post by WayneZhou2009 »

Hi,

Regarding optimization through MLE (fitting model to data), what if, after getting the true log likelihood by using the standard deviation of error term as the proper weight, it is found that the mean value of the error term is not very close to 0?

Is it a violation of the fundamental assumption that the error term should be I.I.D normally distributed with 0 mean, i.e. error ~ N (0,1) ?

Thanks!
tomfid
Administrator
Posts: 3804
Joined: Wed May 24, 2006 4:54 am

Re: Mean of errors not close to 0 after optimization

Post by tomfid »

The weighted error should be N(0,1) but the unweighted error would only have that distribution by luck or if the weights are all the same.
WayneZhou2009
Senior Member
Posts: 105
Joined: Wed Oct 25, 2017 3:52 pm
Vensim version: PRO

Re: Mean of errors not close to 0 after optimization

Post by WayneZhou2009 »

Hi Tom, thanks for your reply. Yes, it is weighted error. The weight is the standard deviation - calculated from the optimization and then plugged back into the model to replace the original weight of 1.

The resultant log likelihood is very close to half of the number of data points, namely, around 50 for 100 data points. I think this is in line with the assumption that the log likelihood function is X2 : 100 ~ SUM (N: 0, 1)^2 and therefore the mean contribution per data point is 1?

However, despite the above, the mean of the error is not zero, rather, it is around 0.3 (which is not close to zero enough). Could this be because the number of data points is not large enough, or maybe because the model structure is somewhat problematic and needs to be tweaked?

tomfid wrote: Thu Oct 04, 2018 8:12 pm The weighted error should be N(0,1) but the unweighted error would only have that distribution by luck or if the weights are all the same.
tomfid
Administrator
Posts: 3804
Joined: Wed May 24, 2006 4:54 am

Re: Mean of errors not close to 0 after optimization

Post by tomfid »

Seems odd. Could it be due to bias from the prior?
WayneZhou2009
Senior Member
Posts: 105
Joined: Wed Oct 25, 2017 3:52 pm
Vensim version: PRO

Re: Mean of errors not close to 0 after optimization

Post by WayneZhou2009 »

Hi Tom,

No, it is the model without the priors. Only MLE, no Bayesian. I have reflected on this issue a bit more. Perhaps I was fundamentally mistaken by expecting the mean of the error to be 0. This case is not linear, and hence in general the mean of error would not necessarily be close to zero?

If it is a linear situation, then the mean of errors after optimization is bound to be zero (in theory) or extremely close to zero (in modelling), as it is the feature of linear regression. I tested this by replacing the Weibull with just a simple linear regression in the same Vensim model to fit to the same Excel data and the result turns out to be exactly the same as the regression done in Excel, with the mean of errors being something around -7e-5.

So probably it should not have been a concern that the mean of error in my original case (Weibull distribution) is not close to zero?
tomfid
Administrator
Posts: 3804
Joined: Wed May 24, 2006 4:54 am

Re: Mean of errors not close to 0 after optimization

Post by tomfid »

This is true.
Post Reply