MLE: NMaximize vs FindDistributionParameters

Questions
,
1

2 simple ways to perform (simple) Maximum Likelihood Estimation in Mathematica:

(1) Construct the Log-Likelihood function and maximize it:

5393Untitled-1.png
5393Untitled-1.png690×181 5.22 KB

(2) Define the Probability Distribution and find parameters:

Untitled-2.png
Untitled-2.png652×146 5.52 KB

They are not that much different, in the sense that they yield close values for the Log-Likelihood function (shown in the plot below), but still, the difference is noticeable in the resulting values of the parameters and I am curious which should I trust more?

loglik.png
loglik.png900×506 81.8 KB

2

The two solutions are too far apart to be considered similar. The issue is that a factor of 1/2 is missing from the pdf definition. To match the likelihood the pdf needs to be

pdf[x_] := 
  Sqrt[(1 - \[Rho])/\[Rho]]
    Exp[(inN[x]^2 - ((Sqrt[1 - \[Rho]] inN[x] - inN[PD])/
        Sqrt[\[Rho]])^2)/2];

Now the result will be

{PD -> 0.013438593418697627473, \[Rho] -> 0.10995316859509961784}

which matches

{137.731, {PD -> 0.0134386, \[Rho] -> 0.109953}}
3

That’s embarrassing..

Thank you, Jim!

4

I continue to make more than my fair share of programming mistakes (especially with Mathematica). The only ways I know how to minimize such occurrences are to (1) perform lots of checks (Do the probability functions sum/integrate to 1 for arbitrary parameters?), and (2) (probably the best technique) Grab anybody and explain your problem with the code in front of both of you. Then either that person will catch the obvious (upper/lowercase, spelling, etc.) or while you’re explaining you’ll say “Oh, nevermind.” because the issue will become obvious.

5

Maybe it’s part of the documentation and I just don’t see it but it appears for FindDistributionParameters, there is no option to obtain estimates of the standard errors and covariances of the parameter estimators. A Statistics golden rule is “An estimate without an associated estimate of precision is at best of unknown value.”

Bruce Miller: Where do I send such suggestions?

6

See also:

http://mathematica.stackexchange.com/questions/6498/standard-errors-for-maximum-likelihood-estimates-in-finddistributionparameters

and

Cheers,

M.

7

Thanks, Marco. Usually the Hessian (or an approximation of it) is created during the iterative process for finding maximum likelihood estimates so I would think that it shouldn’t be too much trouble to make that available (or its inverse when it exists) as an option for FindDistributionParameters.

If Wolfram is packaging these high level functions to be of value other than being a good way to obtain starting values for a more detailed analysis, I would consider that providing these estimated standard errors is essential.

Jim

8

Jim,

Suggestions can be sent via Contact Wolfram Support by Email or Phone

A simple code example with actual output and preferred output would help clarify the issue.