Covariance and correlation
In probability theory and statistics, the mathematical concepts of covariance and correlation are very similar.[1][2] Both describe the degree to which two random variables or sets of random variables tend to deviate from their expected values in similar ways.
If X and Y are two random variables, with means μX and μY, and standard deviations σX and σY, respectively, then their covariance and correlation are as follows:
![{\displaystyle \sigma _{XY}=E[(X-\mu _{X})\,(Y-\mu _{Y})]}](../I/m/5f6328c2a98e12b74140dfc6fb614f7939e12a1c.svg)
![{\displaystyle \rho _{XY}=E[(X-\mu _{X})\,(Y-\mu _{Y})]/(\sigma _{X}\sigma _{Y})}](../I/m/c8ad7d4bca4314703d33deb0245732bcc249dfa4.svg)
So that
where E is the expected value operator. Notably, correlation is dimensionless while covariance is in units obtained by multiplying the units of the two variables. The covariance of a variable with itself (i.e. ) is called the variance and is more commonly denoted as the square of the standard deviation. The correlation of a variable with itself is always 1 (except in the degenerate case where the two variances are zero, in which case the correlation does not exist).
Time series analysis
In the case of a time series which is stationary in the wide sense, both the means and variances are constant over time (E(Xn+m) = E(Xn) = μX and so on). In this case the cross-covariance and cross-correlation are functions of the time difference:
![\sigma _{XY}(m)=E[(X_{n}-\mu _{X})\,(Y_{n+m}-\mu _{Y})].](../I/m/197b94cc63e5dacdea853cc4c98454775f6ea44b.svg)
![\rho _{XY}(m)=E[(X_{n}-\mu _{X})\,(Y_{n+m}-\mu _{Y})]/(\sigma _{X}\sigma _{Y}),](../I/m/abe574941d34c066f1c8d784f772408565610db9.svg)
Although the values of the theoretical covariances and correlations are linked in the above way, the probability distributions of sample estimates of these quantities are not linked in any simple way and they generally need to be treated separately. These distributions depend on the joint distribution of the pair of random quantities (X,Y) when the values are assumed independent across different pairs. In the case of a time series, the distributions depend on the joint distributions of the whole time-series.