User:Hussein Alasadi/Notebook/stephens/2013/10/14

From OpenWetWare
Jump to: navigation, search
Owwnotebook icon.png Analyzing pooled sequenced data with selection Report.pngMain project page
Resultset previous.pngPrevious entry      Next entryResultset next.png

Mean & Co-variance of random vectors

Define: [math] \vec{X} = (X_1, X_2,...,X_m) [/math] and [math] \vec{Y} = (Y_1, Y_2,...,Y_n) [/math]

[math] E[\vec{X}] = (E[X_1], E[X_2], ..., E[X_m]) [/math]

[math] Cov(X,Y) = E[(X-E[X])(Y-E[Y))^T] = m x n [/math] matrix with [math] i,j [/math] element equal to [math] Cov(X_i,Y_j) [/math]

Suppose A = p x m matrix, b [math] \in R^n [/math], C = q x n matrix, d [math] \in R^q [/math]. [math] A = (a_{i,j}), b = (b_i) [/math]

  • Claim: E[AX + b] = AE[X] + b

Proof: ith element of [math] AX + B = \sum_{j=1}^m a_{i,j}X_j + b_i [/math] so

[math] E[\sum_{j=1}^m a_{i,j}X_j + b_i] = \sum_{j=1}^m a_{i,j}E[X_j] + b_i = [/math] ith element of [math] AE[X] + b [/math]

  • Claim: Suppose [math]Cov(X,Y) = V = (v_{i,j})[/math] then [math] Cov(AX+b, CY+d) = AVC^T [/math]

Proof involves first considering the [math] i,j [/math] element of [math] AX+b [/math] and [math] CY+d [/math] then taking the covariance of both of these elements. By properties of 1D covariances, it works out that the [math]i,j[/math] element is just [math](AVC^T)_{i,j}[/math]

  • Claim: The covariance matrix [math] \sum [/math] is positive defiinite

Proof: [math] \forall b \in R^n, [/math]

[math] 0 \le Var(b^TX) = Cov(b^TX, b^TX) = b^TCov(X,X)(b^T)^T = b^T \sum b [/math]

And obviously [math] \sum [/math] is symmetric thus by definition [math]\sum[/math] is pd.

Useful results of multivariate normals

  • Definition of Multivariate normal

[math] \vec{X} = (X_1, X_2,...,X_n) [/math] is multivariate normal if [math] \vec{X} [/math] can be written as:

[math] \vec{X} = AZ+b [/math] for some non-random matrix [math] A [/math] and non-random vector [math] b [/math] with [math] \vec{Z} = (Z_1, Z_2, ..., Z_n) [/math] with [math] Z_1, .., Z_n [/math] iid N(0,1).

From previous results, [math] E[X] = A0 + b = b [/math] and [math] Cov(X,X) = AIA^T = AA^T = \sum [/math].

  • Density of MVN

Defining [math] \sum = AA^T [/math], we have [math] \| \sum \| = \| A A^T \| = \| A \| \| A^T \| = \| A \| [/math] and [math] \sum^-1 = (AA^T)^{-1} = (A^{-1})^TA^{-1} [/math]

[math] f_X(x) = \frac{1}{\sqrt{2\pi}^2\|\sum\|^{frac{1}{2}} [/math]