Assumed knowledge of matrices and vector spaces

- Use of terms singular, diagonal, unit, null, symmetric.
- Operations of addition, subtraction, multiplication, inverse and
transpose.
[We will use for the transpose of .]

- ,
- .

- The
**trace**of a matrix , written tr, is defined as the sum of the diagonal elements of A. That is,

- trtrtr,
- trtr.

__Linear Independence and Rank__- Let , ..., be a set of vectors and , ...,
be scalar constants. If
only if
, the the set of vectors is
**linearly independent**. - The
**rank**of a set of vectors is the maximum number of linearly independent vectors in the set. - For a square matrix , the rank of , denoted by r, is the maximum order of non-zero subdeterminants.
- rrrr,

- Let , ..., be a set of vectors and , ...,
be scalar constants. If
only if
, the the set of vectors is
__Quadratic Forms__

For a p-vector

**x**, where , and a square matrix ,

is a quadratic form inThe matrix and the quadratic form are called:

**positive semidefinite**if for all and for some .**positive definite**if for all .- A necessary and sufficient condition for to be positive definite is that each leading diagonal sub-determinant is greater than . So a positive definite matrix is non-singular.
- A necessary and sufficient condition for a
__symmetric__matrix to be positive definite is that there exists a non-singular matrix such that .

__Orthogonality__.

A matrix is said to be

**orthogonal**if (or ).- An orthogonal matrix is non-singular.
- The determinant of an orthogonal matrix is .
- The transpose of an orthogonal matrix is also orthogonal.
- The product of two orthogonal matrices is orthogonal.
- If is orthogonal, trtrtr.
- If is orthogonal, rr.

__Eigenvalues and eigenvectors__.

Eigenvalues of a square matrix are defined as the roots of the equation

. The corresponding**x**satisfying are the eigenvectors.- The eigenvectors corresponding to two different eigenvalues are orthogonal.
- The number of non-zero eigenvalues of a square matrix is equal to the rank of .

__Reduction to diagonal form__

- Given any symmetric matrix there exists
**an orthogonal matrix**P such that where is a diagonal matrix whose elements are the eigenvalues of . We write .- If is not of full rank, some of the will be zero.
- If is positive definite (and therefore non-singular), all the will be greater than zero.
- The eigenvectors of form the columns of matrix .

- If is symmetric of rank and is orthogonal such that
, then
- tr since trtrtr.
- tr .

- For every quadratic form
there exists an
orthogonal transformation
which reduces Q to a diagonal
quadratic form so that

where is the rank of .

- Given any symmetric matrix there exists
__Idempotent Matrices__.

A matrix is said to be

**idempotent**if . In the following we shall mean symmetric idempotent matrices. Some properties are:- If A is idempotent and non-singular then . To prove this, note that and pre-multiply both sides by .
- The eigenvalues of an idempotent matrix are either or .
- If is idempotent of rank , there exists an orthogonal matrix such that where is a diagonal matrix with the first leading diagonal elements and the remainder .
- If is idempotent of rank then tr. To prove this, note that there is an orthogonal matrix such that . Now trtrtr.
- If the ith diagonal element of is zero, all elements in the ith row and column are zero.
- All idempotent matrices not of full rank are positive semi-definite. No idempotent matrix can have negative elements on its diagonal.