Efficiency

We will next make some comments on the property of efficiency of estimators. The term is frequently used in comparison of two estimators where a measure of relative efficiency is used. In particular,

Definition 8..6
Given two unbiased estimators, $T_1$ and $T_2$ of $\theta$, the efficiency of $T_1$ relative to $T_2$ is defined to be

$$e(T_1, T_2)=\mbox{Var}(T_2)/\mbox{Var}(T_1),$$

and $T_2$ is more efficient than $T_1$ if Var($T_2) <$Var($T_1$).

Note that it is only reasonable to compare estimators on the basis of variance if they are both unbiased. To allow for cases where this is not so, we can use mse in the definition of efficiency. That is,

Definition 8..7
An estimator $T_2$ of $\theta$ is more efficient than $T_1$ if

$$\mbox{mse }T_2 \leq \mbox{mse }T_1,$$

with strict inequality for some $\theta$. Also the relative efficiency of $T_1$ with respect to $T_2$ is

$$e(T_1, T_2)=\frac{\mbox{mse }T_2}{\mbox{mse }T_1} =\frac{E[(T_2-\theta)^2]}{E[(T_1-\theta)^2]}.$$ (8.6)

Example 8..3
Let $X_1, \ldots, X_n$ denote a random sample from U(0, $\theta$), with $Y_1,Y_2,\ldots,Y_n$ the corresponding ordered sample.

(i)

Show that $T_1=2\overline{X}$ and $T_2=\frac{n+1}{n}Y_n$ are unbiased estimates of $\theta$.

(ii)

Find $e(T_1, T_2)$.

Solution

(i)

Now E($X_i)=\theta/2$ and Var( $X_i)=\theta^2/12$ so

$$E(T_1)=2E(\overline{X})=2E(X_i)=2.\frac{\theta}{2}=\theta.$$

To find the mean of $T_2$, first note that the probability density function of $Y_n$ is

$$\begin{eqnarray*} f_{Y_n}(y)&=&n(F_X(y))^{n-1}f_X(y), \mbox{ for }0 \leq y \leq \theta\\ &=&\frac{ny^{n-1}}{\theta^n}I_{(0,\theta)}(y). \end{eqnarray*}$$

So

$$\begin{eqnarray*} E(Y_n)&=&\frac{n}{\theta^n}\int_0^{\theta}y^n dy\\ &=& \frac{... ...ac{y^{n+1}}{n+1}\right]_0^{\theta} \\ &=& \frac{n \theta}{n+1}. \end{eqnarray*}$$

For $T_2$ defined by $T_2=\frac{n+1}{n}Y_n$, we have

$$E(T_2)=\frac{n+1}{n}E(Y_n)=\theta.$$

So both $T_1$ and $T_2$ are unbiased.

(ii)

Var($T_1)=$Var( $2\overline{X})= 4$Var( $\overline{X})=\frac{4 \mbox{Var}(X_i)}{n}=\frac{4\theta^2}{12n}=\frac{\theta^2}{3n}$.
To find Var($T_2$), first we need to find E($Y_n^2$) from

$$\begin{eqnarray*} E(Y_n^2)&=& \int_0^{\theta}y^2\frac{n}{\theta^n}y^{n-1}dy\\ &... ...{\theta^n}\frac{\theta^{n+2}}{n+2}\\ &=& \frac{n}{n+2}\theta^2. \end{eqnarray*}$$

$$\begin{eqnarray*} \mbox{Var}(Y_n )&=&\frac{n}{n+2}\theta^2 \quad - \quad \frac{n^2\theta^2} {(n+1)^2}\\ &=&\frac{n \theta^2}{(n+1)^2(n+2)}. \end{eqnarray*}$$

So

$$\begin{eqnarray*} \mbox{Var}(T_2)&=& \frac{(n+1)^2}{n^2}.\frac{n \theta^2}{(n+1)^2(n+2)}\\ &=& \frac{\theta^2}{n(n+2)} \end{eqnarray*}$$

Since these estimates are unbiased, we may use definition 2.5,

$$e(T_1,T_2)=\frac{\mbox{Var}(T_2)}{\mbox{Var}(T_1)}=\frac{\theta^2}{n(n+2)} \frac{3n}{\theta^2}=\frac{3}{n+2}.$$

This is less than $1$ for $n > 1$ so $T_2$ is more efficient than $T_1$.