When can the MVB be Attained?

It is easy to establish the condition under which the minimum variance bound of an unbiased estimator, (8.9), is achieved. In the proof of Theorem8.2, it should be noted that the inequality concerning the correlation of $V$ and $T$ becomes an equality (that is, $\rho_{V,T}=+1$ or $-1$) when V is a linear function of T. Recalling that $V=\frac{\partial \log L(\theta)}{\partial \theta}$, we may write this condition as

$$\frac{\partial \log L(\theta)}{\partial \theta}=A(T-\theta),$$

where $A$ is independent of the observations but may be a function of $\theta$, so we will write it as $A(\theta)$. So the condition for the MVB to be attained is that the statistic T$=$t( $X_1, \ldots, X_n$) satisfies

$$\frac{\partial \log L(\theta)}{\partial \theta}=A(\theta)(T-E(T)).$$ (8.11)

Example 8..5
In the problem of estimating $\theta$ in a normal distribution with mean $\theta$ and known variance $\sigma^2$, where $\sigma^2$ is known, show that the MVB of an unbiased estimator can be attained.

As in Example 8.2,

$$\frac{\partial \log L(\theta)}{\partial \theta}=\frac{n(\overline{x}-\theta)}{\sigma^2}.$$

Now defining $T=t(X_1,\ldots, X_n)=\overline{X}$, we know it is an unbiased estimator of $\theta$, and we see that (8.11) is satisfied, where $A(\theta)=n/\sigma^2$, (A not being a function of $\theta$ in this case). Thus the minimum variance bound can be attained.

Comment. In the case of an unbiased estimator T where the MVB is attained, note that the inequality in (8.9) becomes an equality and we have

$$\mbox{Var}(T)=1/I_{\bf X}(\theta)=1/E\left(\frac{\partial \log L(\theta)}{\partial \theta}\right)^2.$$ (8.12)

Also, squaring (8.11) and taking expectations of both sides, we have

$$\begin{eqnarray*} E\left(\frac{\partial \log L(\theta)}{\partial \theta}\right)^2&=& [A(\theta)]^2E[(T-\theta)^2]\\ &=&[A(\theta)]^2 \mbox{Var}(T) \end{eqnarray*}$$

That is,

$$\begin{eqnarray*} \mbox{Var}(T)&=&E\left(\frac{\partial \log L(\theta)}{\partial... ...1}{[A(\theta)]^2}.\frac{1}{\mbox{Var}(T)}, \mbox{ using (2.12)} \end{eqnarray*}$$

giving

$$\mbox{Var}(T)=1/A(\theta).$$

So, if the statistic $T$ satisfies (8.11), Var(T) can be identified immediately as the multiple of $T-\theta$ on the RHS. For instance, in Example 8.2, the factor $n/\sigma^2$ multiplying $\overline{x}-\theta$ can be identified as the reciprocal of the variance of $T$, and it was not necessary to evaluate the MVB as in Example 8.2.

Example 8..6
Consider the problem of estimating the variance, $\theta$, of a normal distribution with known mean $\mu$, based on a sample of size $n$.

Now the likelihood is

$$\begin{eqnarray*} L(\theta)&=&(2 \pi)^{-n/2} \theta^{-n/2} e^{-\sum(x_i-\mu)^2/2... ...c{n}{2 \theta^2}\left(\frac{\sum(x_i-\mu)^2}{n} - \theta \right) \end{eqnarray*}$$

which is in the form (2.11) where $T=t(X_1, \ldots, X_n)=\sum_{i=1}^n(X_i-\mu)^2/n$. So, using this as the estimate of $\theta$, the MVB is achieved and it is $2 \theta^2/n$.

Note that $E(\sum(X_i-\mu)^2)=\sum E(X_i-\mu)^2=n \mbox{Var}(X_i)=n \theta$ so T is an unbiased estimator of $\theta$. Also, $\sum(X_i-\mu)^2/\theta \sim \chi^2_n$ so has variance $2n$. Hence,

$$\mbox{Var}(T)=\mbox{Var}\left[\frac{\theta}{n}.\frac{\sum(X_i... ...} {\theta}\right]=\frac{\theta^2}{n^2}.2n=\frac{2\theta^2}{n}.$$

Example 8..7

Consider the problem where we have a random sample $X_1, \ldots, X_n$ from a Poisson distribution with parameter $\theta$ and we wish to find the Cramér-Rao lower bound for the variance of an unbiased estimator of $\theta$, and identify the estimator that has his variance.

Now for $f(x;\theta)=e^{-\theta} \theta^x/x!$, the likelihood of the sample is

$$\begin{eqnarray*} L(\theta;x_1,\ldots,x_n)&=&\frac{e^{-n \theta} \theta^{\sum x_... ...\theta}[\overline{x} - \theta]\\ &=&A(\theta) \ [T - \theta]\\ \end{eqnarray*}$$

where $T(X_i)=\overline{X}$ is the statistic. This is in the correct form for the minimum variance bound to be attained and it is $1/A(\theta)=\theta/n$. We note that $\overline{X}$ is an estimator which has variance $\theta/n$.