Of course, we estimate by some (appropriate) function of the observations and such a function is called a statistic or an estimator. A particular value of an estimator, say t( ), is called an estimate. We will be considering various qualities that a ``good'' estimator should possess, but firstly, it should be noted that, by virtue of it being a function of the sample values, an estimator is itself a random variable. So its behaviour for different random samples will be described by a probability distribution.
It seems reasonable to require that the distribution of the estimator be somehow centred with respect to . If it is not, the estimator will tend either to under-estimate or over-estimate . A further property that a good estimator should posssess is precision, that is, the dispersion of the distribution should be small. These two properties need to be considered together. It is not very helpful to have an estimator with small variance if it is ``centred'' far from . The difference between an estimator ) and is referred to as an error, and the ``mean squared error'' defined below is a commonly used measure of performance of an estimator.