Next: When can the MVB Up: Cramér-Rao Lower Bound Previous: Cramér-Rao Lower Bound   Contents

## Minimum Variance Estimation

The Theorem below gives a lower bound for the variance (or mse) of an estimator.

Theorem 8..1
Let , based on a sample X from be an estimator of (assumed to be one-dimensional). Then

 (8.7)

and
 (8.8)

where is given in (8.1) and is defined in (7.6).

Outline of Proof. [The validity depends on regularity conditions, where the interchange of integration and differentiation operations is permitted and on the existence and integrability of various partial derivatives.]

Define as in (7.7) and we note that so Var(V) and

Recall that the absolute value of the correlation coefficient, for measuring correlation between any two variables is less than or equal to and that

so that we have

or

thus proving (8.7). Now (8.8) follows using ( 8.3).

Corollary. For the class of unbiased estimators,

 (8.9)

Now inequality (8.9) is known as the Cramér-Rao lower bound, or sometimes the Information inequality. It provides (in regular estimation'' cases) a lower bound on the variance of an unbiased estimator, T. The inequality is generally attributed to Cramér's work in 1946 and Rao's work in 1945, though it was apparently first given by M. Frechet in 1937-38.

Definition 8..8
The (absolute) efficiency of an unbiased estimator T is defined as

 (8.10)

Note that, because of (8.9), , so we can think of as a measure of efficiency of any given estimator, rather than the relative efficiency of one with respect to another as in Definition 8.1.

In the case where , so that the actual lower bound of Var() is achieved, some texts refer to the estimator T as efficient. This terminology is not universally accepted. Some prefer to use the phrase minimum variance bound (MVB) for , and an estimator which is unbiased and which attains this bound is called a minimum variance bound unbiased (MVBU) estimator.

Example 8..4
In the problem of estimating in a normal distribution with mean and known variance , find the MVB of an unbiased estimator.

The MVB is where . For a sample we have for the likelihood,

So the MVB is .

Next: When can the MVB Up: Cramér-Rao Lower Bound Previous: Cramér-Rao Lower Bound   Contents
Bob Murison 2000-10-31