Some Examples

Example 9..8
Let $X$ have a normal distribution with unknown mean $\mu$ and known variance $\sigma^2_0$. Suppose we have a random sample $x_1, x_2, \dots , x_n$ from this distribution and wish to test $H_0:\mu=3$ against $H_1:\mu \neq 3$. Now

$$\begin{eqnarray*} H_0 \cup H_1 & = & \{ (\mu , \sigma^2 ): \mu \in (-\infty , \... ...H_0 & = & \{ (\mu , \sigma^2): \mu = 3, \ \sigma^2=\sigma^2_0\} \end{eqnarray*}$$

Rather than $L(\theta)$ we have here $L(\mu ; x_1 , \dots , x_n , \sigma^2_0 )$ or more briefly, $L(\mu )$.

$$\begin{eqnarray*} L(\mu ) & = & \prod^n_{i=1} \frac{1}{(2\pi \sigma^2_0 )^{\fra... ...tstyle\frac 12} \sum^n_{i=1}(x_i -\mu )^2 /\sigma^2_0 \right\} \end{eqnarray*}$$

Now $${\max_{H_0\cup H_1 } L(\mu )}$$ is obtained by replacing $\mu$ in the above by its mle, $\overline{x}$. So

$$\max_{H_0\cup H_1} L(\mu ) = (2\pi )^{-n/2}\sigma^{-n}_0 \exp... ...c 12} \sum^n_{i=1} (x_i - \overline{x})^2/\sigma^2_0 \right\}.$$

Also $L(\mu \vert H_0)$ has only one value, obtained by replacing $\mu$ by $3$ and $\sigma^2$ by $\sigma^2_0$. So

$$\begin{eqnarray*} \max_{H_0} L(\mu ) & = & (2\pi )^{-n/2} \sigma^{-n}_0 \exp \l... ...sigma^2_0 -\frac n2 (\overline{x} - 3)^2/ \sigma^2_0 \right\} \end{eqnarray*}$$

Thus, on simplification

$$\lambda = \exp\{-n (\overline{x} -3)^2/2\sigma^2_0 \}.$$ (9.7)

Intuitively, we would expect that values of $\overline{x}$ close to 3 support the hypothesis and it can be seen that in this case $\lambda$ is close to $1$. Values of $\bar{x}$ far from $3$ lead to $\lambda$ close to $0$. We need to find the critical value $\lambda_0$ to satisfy (3.6). That is, we need to know the distribution of $\Lambda$. From (3.7), using random variables instead of observed values, we have

$$-2 \log \Lambda = \left( \frac{\overline{X} -3}{\sigma_0 / \sqrt{n}}\right)^2$$

which is the square of a $N(0,1)$ variate and therefore is distributed as $\chi^2_1$. For $\alpha=.05$, the critical region is $\{ \overline{x}:n(\overline{x}-3)^2 / \sigma^2_0 \geq 3.84\}$, or alternatively

$$\{ \overline{x}:\sqrt{n} (\overline{x} -3) / \sigma_0 > 1.96 \mbox{ or } \sqrt{n}(\overline{x} -3) / \sigma_0 < -1.96 \}.$$

The relationship between the critical region for $\lambda$ and the critical region for $-2\log \lambda$ is shown in the diagram below.

Example 9..9

Given $X_1, X_2, \dots , X_n$ is a random sample from a $N(\mu , \sigma^2 )$ distribution, where $\sigma^2$ is unknown, derive the LR test of $H_0:\mu = \mu_0$ against $H_1:\mu \neq \mu_0$.

Now the parameter space is

$$H_0 \cup H_1 = \{ (\mu , \sigma^2):\mu \in (-\infty , \infty ), \ \sigma^2 > 0\},$$

and that restricted by the hypothesis is

$$H_0 = \{ (\mu , \sigma^2 ): \mu = \mu_0, \ \sigma^2 > 0 \}.$$

We note that there are $2$ unknown parameters here, and the likelihood of the sample,
$L(\mu , \sigma^2;x_1, x_2, \dots , x_n)$ can be written as

$$L(\mu , \sigma^2) = \prod^n_{i=1} (2\pi \sigma^2)^{-\frac 12}\,e^{- \frac 12 (x_i-\mu)^2 / \sigma^2}$$

$$= (2\pi )^{-n/2} (\sigma^2)^{-n/2}\,e^{-\frac 12 \sum^n_{i=1}(x_i- \mu )^2 / \sigma^2}$$ (9.8)

Now the mle's of $\mu$ and $\sigma^2$ are

$$\hat{\mu}=\overline{x}, \ \ \ \hat{\sigma}^2 = \sum^n_{i=1} (x_i - \overline{x})^2 / n,$$

and $${\max_{H_0\cup H_1} L(\mu , \sigma^2 )}$$ is obtained by substituting these for $\mu$ and $\sigma^2$ in (3.8). This gives

$$\max_{H_0\cup H_1} L(\mu , \sigma^2) = (2\pi )^{-n/2} n^{n/... ...\sum^n_{i=1} (x_i -\overline{x})^2\right]^{-n/2}\, e^{-n/2}.$$ (9.9)

Now $${\max_{H_0}L(\mu , \sigma^2)}$$ is obtained by substituting $\mu_0$ for $\mu$ in (3.8) and replacing $\sigma^2$ by its MLE where $\mu$ is known. This is $\sum^n_{i=1}(x_i -\mu_0)^2/n = \tilde{\sigma}^2$, say.
Thus

$$\max_{H_0} L(\mu , \sigma^2) = (2\pi )^{-n/2} n^{n/2} \left[ \sum^n_{i=1}(x_i -\mu_0)^2\right]^{-n/2}\,e^{-n/2}.$$

So $${\lambda = \max_{H_0}L(\mu , \sigma^2) / \max_{H_0\cup H_1} L(\mu , \sigma^2)}$$ becomes

$$\lambda = \left[ \sum_i(x_i -\overline{x})^2 / \sum_i (x_i - \mu_0)^2\right]^{-n/2}.$$

Taking $2/n$th powers of both sides and writing $\sum_i(x_i-\mu_0)^2$ as $\sum_i\left[(x_i-\overline{x})+(\overline{x}-\mu_0)\right]^{2}$, we have

$$\lambda^{2/n} = \sum_i (x_i-\overline{x})^2 / \left[ \sum_i (x_i - \overline{x})^2+n(\overline{x}-\mu_0)^2\right]$$

$$= \frac{1}{1+\frac{n(\overline{x}-\mu_0)^2}{\sum_i(x_i-\overline{x})^2}}.$$ (9.10)

Recalling that $\lambda$ is the observed value of a random variable with a range space $[0,1]$, and that the critical region is of the form $0< \lambda < \lambda_0$, we would like to find a function of $\lambda$ (or of $\lambda^{2/n})$ whose probability distribution we recognize. Now

$$\overline{X} \sim N(\mu_0,\sigma^2/n) \mbox{ so }\displaystyl... ...tyle{\frac{n(\overline{X} - \mu_0)^2}{\sigma^2}\sim \chi^2_1}.$$

Also, $\sum^n_{i=1}(X_i-\overline{X})^2=\nu S^2$ so $\sum^n_{i=1}(X_i -\overline{X})^2/\sigma^2=\nu S^2/\sigma^2 \sim \chi^2_\nu$ where $\nu = n-1$. So, expressing the denominator of (3.10) in terms of random variables we have

$$1+\frac{n(\overline{X}-\mu_0)^2/\sigma^2}{\nu S^2/\sigma^2}\q... ...2_1}{\nu (\chi^2_\nu /\nu)} \quad\sim\quad 1+ \frac{T^2}{\nu}$$

where $T$ is a random variable with a $t$ distribution on $\nu$ degrees of freedom. Considering range spaces, the relationship between $\lambda$ (or $\lambda^{2/n}$) and $t^2$ is a strictly decreasing one, and a critical region of the form $0< \lambda < \lambda_0$ is equivalent to a CR of the form $t^2>t_0^2$, as indicated in the diagram below.

That is, the critical region is of the form $\vert t\vert>t_0$ where $t_0$ is obtained from tables, using $\nu$ degrees of freedom, and the appropriate significance level.

Example 9..10

Given the random sample $X_1,\,X_2,\dots ,\,X_n$ from a $N(\mu , \sigma^2 )$ distribution, derive the LR test of the hypothesis $H_0:\sigma^2=\sigma_0^2$, where $\mu$ is unknown, against $H_1:\sigma^2\neq \sigma_0^2$.

The parameter space, and restricted parameter space are

$$\begin{eqnarray*} H_0 \cup H_1& = & \{ (\mu , \sigma^2):\mu \in (-\infty ,\inft... ...sigma^2): \mu \in (-\infty, \infty ), \ \sigma^2=\sigma_0^2\} \end{eqnarray*}$$

$L(\mu ,\sigma^2 )$ is given by (3.8) in Example 3.2, and $${\max_{H_0\cup H_1} L(\mu , \sigma^2 )}$$ is given by (3.9). To find $${\max_{H_0}L(\mu , \sigma^2)}$$ we replace $\sigma^2$ by $\sigma_0^2$ and $\mu$ by the mle, $\overline{x}$. So

$$\max_{H_0} L(\mu ,\sigma^2)=(2\pi )^{-n/2} (\sigma_0^2)^{- n/2}\,e^{-\frac 12 \sum_i(x_i-\overline{x})^2 / \sigma_0^2}$$

So

$$\lambda = e^{n/2} \left[ \frac{\sum_i(x_i- \overline{x})^2}{... ...n/2}\,e^{-\frac 12 \sum_i(x_i - \overline{x})^2 / \sigma_0^2}$$

Again, we would like to express $\Lambda$ as a function of a random variable whose distribution we know. Denoting $\sum_i (x_i-\overline{x})^2 / \sigma_0^2$ by $w$, the random variable $W$, whose observed value this is, has a $\chi^2$ distribution with parameter $\nu = n-1$. So we have

$$\lambda = e^{n/2}(w/n)^{n/2}\,e^{-w/2},$$

and the relationship between the range spaces of $\Lambda$ and $W$ is shown in the diagram below

A critical region of the form $0< \lambda < \lambda_0$ corresponds to the pair of intervals, $0<w<a, \ b< w < \infty$. So for a size-$\alpha$ test, $H$ is rejected if

$$\nu s^2 / \sigma_0^2 < \chi^2_{\nu , \alpha / 2} \ \ \mbox{or} \ \ \nu s^2 / \sigma_0^2 > \chi^2_{\nu , 1-(\alpha /2)}.$$

[This of course is the familiar intuitive test for this problem.]