Consistency

A further desirable property of estimators is that of consistency, which is an asymptotic property. To understand consistency, it is necessary to think of $T$ as really being $T_n$, the nth member of an infinite sequence of estimators, $T_1, \ldots, T_n$. Roughly speaking, an estimator is consistent if, as $n$ gets large, the probability that $T_n$ lies arbitrarily close to the parameter being estimated becomes itself arbitrarily close to $1$. More formally, we have

Definition 8..4
$T_n=t(X_1, \ldots, X_n)$ is a consistent estimator of $\theta$ if

$$\lim_{n \rightarrow \infty}P(\vert T_n-\theta\vert \geq \epsilon)=0 \mbox{ for any } \epsilon > 0.$$ (8.4)

This is often referred to as convergence in probability of $T_n$ to $\theta$.

An equivalent definition (for cases where the second moment exists) is

Definition 8..5
$T_n=t(X_1, \ldots, X_n)$ is a consistent estimator of $\theta$ if

$$\lim_{n \rightarrow \infty}E[(T_n-\theta)^2]=0.$$ (8.5)

That is, the mse of $T_n$ as an estimator of $\theta$, decreases to zero as more and more observations are incorporated into its composition. Note that, using (2.3) we see that (2.5) will be satisfied if $T_n$ is asymptotically unbiased and if Var( $T_n) \rightarrow 0$ as $n \rightarrow \infty$.

Asymptote means the truth. So as the sample size increases, $T_n$ gets closer to the true value. When $n=\infty$, we have sampled the entire population. The idea of consistency can be gleaned from the following diagram where $T_n$ converges to $\theta$. If it didn't, $T_n$ would not be a consistent estimator.

Example 8..2
Let $Y$ be a random variable with mean $\mu$ and variance $\sigma^2$. Let $\overline{Y}$ be the sample mean of $n$ random observations taken on $Y$. Is $\overline{Y}$ a consistent estimator of $\mu$?

Now $E(\overline{Y})=\mu$ so $\overline{Y}$ is unbiased. Also Var( $\overline{Y})=\sigma^2/n$ $\rightarrow0$ as $n \rightarrow \infty$, so $\overline{Y}$ is a consistent estimator of $\mu$.