1 Calculation of Covariance Matrix from Data Matrix

Suppose we have a data matrix ${\bf A}$ with rows corresponding to subjects and columns corresponding to variables. We can calculate a mean for each variable and replace the data matrix with a matrix of deviations from the mean. That is, each element ${a_{ij}}$ is replaced by $a_{ij}-\mu _{j}$ where $\mu_{j}$ is the mean of the $j^{th}$ variable. Let us call the new matrix ${\bf Z}$. The covariance matrix is then simply calculated as

$$\frac{1}{N-1}{\bf Z^{\prime}Z}$$

where $N$ is the number of subjects. For example, suppose we have the following data:

$X$	$Y$	$X - \overline{X}$	$Y - \overline{Y}$
1	2	-2	-4
2	8	-1	2
3	6	0	0
4	4	1	-2
5	10	2	4

So the matrix of deviations from the mean is

$$\begin{eqnarray*} {\bf Z} &=& \left( \begin{array}{rr} -2 & -4\ -1 & 2 \ 0 & 0\ 1 & -2\ 2 & 4\\ \end{array} \right) \ \end{eqnarray*}$$

and therefore the covariance matrix of the observations is

$$\begin{eqnarray*} \frac{1}{N-1}{\bf Z^{\prime}Z} &=& \frac{1}{4} \left( \begi... ...S_{x}^{2} & S_{xy}\ S_{xy} & S_{y}^{2} \end{array} \right)\\ \end{eqnarray*}$$

The diagonal elements of this matrix are the variances of the variables, and the off-diagonal elements are the covariances between the variables. The standard deviation is the square root of the variance (see Chapter 2). The correlation is

$$\frac{S_{xy}}{\sqrt {S_{x}^{2} S_{y}^{2}}} = \frac{S_{xy}}{S_{x} S_{y}}$$

In general, a correlation matrix may be calculated from a covariance matrix by pre- and post-multiplying the covariance matrix by a diagonal matrix ${\bf D}$ in which each diagonal element $d_{ii}$ is $\frac{1}{S_{i}}$, i.e., the reciprocal of the standard deviation for that variable. Thus, in our two variable example, we have:

$$\left( \begin{array}{rr} \frac{1}{S_{x}} & 0\ 0 & \frac{1}... ...in{array}{cc} 1.0 & R_{xy}\ R_{xy} & 1.0 \end{array} \right)$$