User:Timothee Flutre/Notebook/Postdoc/2011/11/10

From OpenWetWare

(Difference between revisions)
Jump to: navigation, search
(Entry title: move info about help2man and groff to the "templates" page)
(Bayesian model of univariate linear regression for QTL detection: start adding marginal posterior of B)
(6 intermediate revisions not shown.)
Line 6: Line 6:
| colspan="2"|
| colspan="2"|
<!-- ##### DO NOT edit above this line unless you know what you are doing. ##### -->
<!-- ##### DO NOT edit above this line unless you know what you are doing. ##### -->
-
==Entry title==
+
==Bayesian model of univariate linear regression for QTL detection==
 +
 
 +
 
 +
''See Servin & Stephens (PLoS Genetics, 2007).''
 +
 
 +
 
 +
* '''Data''': let's assume that we obtained data from N individuals. We note <math>y_1,\ldots,y_N</math> the (quantitative) phenotypes (e.g. expression levels at a given gene), and <math>g_1,\ldots,g_N</math> the genotypes at a given SNP (encoded as allele dose: 0, 1 or 2).
 +
 
 +
 
 +
* '''Goal''': we want to assess the evidence in the data for an effect of the genotype on the phenotype.
 +
 
 +
 
 +
* '''Assumptions''': the relationship between genotype and phenotype is linear; the individuals are not genetically related; there is no hidden confounding factors in the phenotypes.
 +
 
 +
 
 +
* '''Likelihood''': we start by writing the usual linear regression for one individual
 +
 
 +
<math>\forall i \in \{1,\ldots,N\}, \; y_i = \mu + \beta_1 g_i + \beta_2 \mathbf{1}_{g_i=1} + \epsilon_i \text{ with } \epsilon_i \overset{i.i.d}{\sim} \mathcal{N}(0,\tau^{-1})</math>
 +
 
 +
where <math>\beta_1</math> is in fact the additive effect of the SNP, noted <math>a</math> from now on, and <math>\beta_2</math> is the dominance effect of the SNP, <math>d = a k</math>.
 +
 
 +
Let's now write the model in matrix notation:
 +
 
 +
<math>Y = X B + E \text{ where } B = [ \mu \; a \; d ]^T</math>
 +
 
 +
This gives the following [http://en.wikipedia.org/wiki/Multivariate_normal_distribution multivariate Normal distribution] for the phenotypes:
 +
 
 +
<math>Y | X, \tau, B \sim \mathcal{N}(XB, \tau^{-1} I_N)</math>
 +
 
 +
The likelihood of the parameters given the data is therefore:
 +
 
 +
<math>\mathcal{L}(\tau, B) = \mathsf{P}(Y | X, \tau, B)</math>
 +
 
 +
<math>\mathcal{L}(\tau, B) = \left(\frac{\tau}{2 \pi}\right)^{\frac{N}{2}} exp \left( -\frac{\tau}{2} (Y - XB)^T (Y - XB) \right)</math>
 +
 
 +
 
 +
* '''Priors''': we use the usual conjugate prior
 +
 
 +
<math>\mathsf{P}(\tau, B) = \mathsf{P}(\tau) \mathsf{P}(B | \tau)</math>
 +
 
 +
A [http://en.wikipedia.org/wiki/Gamma_distribution Gamma distribution] for <math>\tau</math>:
 +
 
 +
<math>\tau \sim \Gamma(\kappa/2, \, \lambda/2)</math>
 +
 
 +
which means:
 +
 
 +
<math>\mathsf{P}(\tau) = \frac{\frac{\lambda}{2}^{\kappa/2}}{\Gamma(\frac{\kappa}{2})} \tau^{\frac{\kappa}{2}-1} e^{-\frac{\lambda}{2} \tau}</math>
 +
 
 +
And a multivariate Normal distribution for <math>B</math>:
 +
 
 +
<math>B | \tau \sim \mathcal{N}(\vec{0}, \, \tau^{-1} \Sigma_B) \text{ with } \Sigma_B = diag(\sigma_{\mu}^2, \sigma_a^2, \sigma_d^2)</math>
 +
 
 +
which means:
 +
 
 +
<math>\mathsf{P}(B | \tau) = \left(\frac{\tau}{2 \pi}\right)^{\frac{3}{2}} |\Sigma_B|^{-\frac{1}{2}} exp \left(-\frac{\tau}{2} B^T \Sigma_B^{-1} B \right)</math>
 +
 
 +
 
 +
* '''Joint posterior (1)''':
 +
 
 +
<math>\mathsf{P}(\tau, B | Y, X) = \mathsf{P}(\tau | Y, X) \mathsf{P}(B | Y, X, \tau)</math>
 +
 
 +
 
 +
* '''Conditional posterior of B''':
 +
 
 +
<math>\mathsf{P}(B | Y, X, \tau) = \mathsf{P}(B, Y | X, \tau)</math>
 +
 
 +
<math>\mathsf{P}(B | Y, X, \tau) = \frac{\mathsf{P}(B, Y | X, \tau)}{\mathsf{P}(Y | X, \tau)}</math>
 +
 
 +
<math>\mathsf{P}(B | Y, X, \tau) = \frac{\mathsf{P}(B | \tau) \mathsf{P}(Y | X, B, \tau)}{\int \mathsf{P}(B | \tau) \mathsf{P}(Y | X, \tau, B) \mathsf{d}B}</math>
 +
 
 +
Here and in the following, we neglect all constants (e.g. normalization constant, <math>Y^TY</math>, etc):
 +
 
 +
<math>\mathsf{P}(B | Y, X, \tau) \propto \mathsf{P}(B | \tau) \mathsf{P}(Y | X, \tau, B)</math>
 +
 
 +
We use the prior and likelihood and keep only the terms in <math>B</math>:
 +
 
 +
<math>\mathsf{P}(B | Y, X, \tau) \propto exp(B^T \Sigma_B^{-1} B) exp((Y-XB)^T(Y-XB))</math>
 +
 
 +
We expand:
 +
 
 +
<math>\mathsf{P}(B | Y, X, \tau) \propto exp(B^T \Sigma_B^{-1} B - Y^TXB -B^TX^TY + B^TX^TXB)</math>
 +
 
 +
We factorize some terms:
 +
 
 +
<math>\mathsf{P}(B | Y, X, \tau) \propto exp(B^T (\Sigma_B^{-1} + X^TX) B - Y^TXB -B^TX^TY)</math>
 +
 
 +
Let's define <math>\Omega = (\Sigma_B^{-1} + X^TX)^{-1}</math>. We can see that <math>\Omega^T=\Omega</math>, which means that <math>\Omega</math> is a [http://en.wikipedia.org/wiki/Symmetric_matrix symmetric matrix].
 +
This is particularly useful here because we can use the following equality: <math>\Omega^{-1}\Omega^T=I</math>.
 +
 
 +
<math>\mathsf{P}(B | Y, X, \tau) \propto exp(B^T \Omega^{-1} B - (X^TY)^T\Omega^{-1}\Omega^TB -B^T\Omega^{-1}\Omega^TX^TY)</math>
 +
 
 +
This now becomes easy to factorizes totally:
 +
 
 +
<math>\mathsf{P}(B | Y, X, \tau) \propto exp((B^T - \Omega X^TY)^T\Omega^{-1}(B - \Omega X^TY))</math>
 +
 
 +
We recognize the [http://en.wikipedia.org/wiki/Kernel_%28statistics%29 kernel] of a Normal distribution, allowing us to write the conditional posterior as:
 +
 
 +
<math>B | Y, X, \tau \sim \mathcal{N}(\Omega X^TY, \tau^{-1} \Omega)</math>
 +
 
 +
 
 +
* '''Posterior of <math>\tau</math>''':
 +
 
 +
Similarly to the equations above:
 +
 
 +
<math>\mathsf{P}(\tau | Y, X) \propto \mathsf{P}(\tau) \mathsf{P}(Y | X, \tau)</math>
 +
 
 +
But now, to handle the second term, we need to integrate over <math>B</math>, thus effectively taking into account the uncertainty in <math>B</math>:
 +
 
 +
<math>\mathsf{P}(\tau | Y, X) \propto \mathsf{P}(\tau) \int \mathsf{P}(B | \tau) \mathsf{P}(Y | X, \tau, B) \mathsf{d}B</math>
 +
 
 +
Again, we use the priors and likelihoods specified above (but everything inside the integral is kept inside it, even if it doesn't depend on <math>B</math>!):
 +
 
 +
<math>\mathsf{P}(\tau | Y, X) \propto \tau^{\frac{\kappa}{2} - 1} e^{-\frac{\lambda}{2} \tau} \int \tau^{3/2} \tau^{N/2} exp(-\frac{\tau}{2} B^T \Sigma_B^{-1} B) exp(-\frac{\tau}{2} (Y - XB)^T (Y - XB)) \mathsf{d}B</math>
 +
 
 +
As we used a conjugate prior for <math>\tau</math>, we know that we expect a Gamma distribution for the posterior.
 +
Therefore, we can take <math>\tau^{N/2}</math> out of the integral and start guessing what looks like a Gamma distribution.
 +
We also factorize inside the exponential:
 +
 
 +
<math>\mathsf{P}(\tau | Y, X) \propto \tau^{\frac{N+\kappa}{2} - 1} e^{-\frac{\lambda}{2} \tau} \int \tau^{3/2} exp \left[-\frac{\tau}{2} \left( (B - \Omega X^T Y)^T \Omega^{-1} (B - \Omega X^T Y) - Y^T X \Omega X^T Y + Y^T Y \right) \right] \mathsf{d}B</math>
 +
 
 +
We recognize the conditional posterior of <math>B</math>.
 +
This allows us to use the fact that the pdf of the Normal distribution integrates to one:
 +
 
 +
<math>\mathsf{P}(\tau | Y, X) \propto \tau^{\frac{N+\kappa}{2} - 1} e^{-\frac{\lambda}{2} \tau} exp\left[-\frac{\tau}{2} (Y^T X \Omega X^T Y + Y^T Y) \right]</math>
 +
 
 +
We finally recognize a Gamma distribution, allowing us to write the posterior as:
 +
 
 +
<math>\tau | Y, X \sim \Gamma \left( \frac{N+\kappa}{2}, \; \frac{1}{2} (Y^T X \Omega X^T Y + Y^T Y + \lambda) \right)</math>
 +
 
 +
 
 +
* '''Joint posterior (2)''': sometimes it is said that the joint posterior follows a Normal Inverse Gamma distribution:
 +
 
 +
<math>B, \tau | Y, X \sim \mathcal{N}IG(\Omega X^TY, \tau^{-1}\Omega, \frac{N+\kappa}{2}, \frac{\lambda^\ast}{2})</math>
 +
 
 +
where <math>\lambda^\ast = (Y^T X \Omega X^T Y + Y^T Y + \lambda)</math>
 +
 
 +
 
 +
* '''Marginal posterior of B''': we can now integrate out <math>\tau</math>:
 +
 
 +
<math>\mathsf{P}(B | Y, X) = \int \mathsf{P}(B, \tau | Y, X) \mathsf{d}\tau</math>
 +
 
 +
<math>\mathsf{P}(B | Y, X) = \frac{\frac{\lambda^\ast}{2}^{\frac{N+\kappa}{2}}}{(2\pi)^\frac{3}{2} |\Omega|^{\frac{1}{2}} \Gamma(\frac{N+\kappa}{2})} \int \tau^{\frac{N+\kappa+3}{2}-1} exp \left[-\tau \left( \frac{\lambda^\ast}{2} + (B - \Omega X^TY)^T \Omega^{-1} (B - \Omega X^TY) \right) \right] \mathsf{d}\tau</math>
 +
 
 +
Here we recognize the formula to [http://en.wikipedia.org/wiki/Gamma_function#Integration_problems integrate the Gamma function]:
 +
 
 +
<math>\mathsf{P}(B | Y, X) = \frac{\frac{\lambda^\ast}{2}^{\frac{N+\kappa}{2}} \Gamma(\frac{N+\kappa+3}{2})}{(2\pi)^\frac{3}{2} |\Omega|^{\frac{1}{2}} \Gamma(\frac{N+\kappa}{2})} \left( \frac{\lambda^\ast}{2} + (B - \Omega X^TY)^T \Omega^{-1} (B - \Omega X^TY) \right)^{-\frac{N+\kappa+3}{2}}</math>
<!-- ##### DO NOT edit below this line unless you know what you are doing. ##### -->
<!-- ##### DO NOT edit below this line unless you know what you are doing. ##### -->

Revision as of 22:19, 21 November 2012

Project name Main project page
Previous entry      Next entry

Bayesian model of univariate linear regression for QTL detection

See Servin & Stephens (PLoS Genetics, 2007).


  • Data: let's assume that we obtained data from N individuals. We note y_1,\ldots,y_N the (quantitative) phenotypes (e.g. expression levels at a given gene), and g_1,\ldots,g_N the genotypes at a given SNP (encoded as allele dose: 0, 1 or 2).


  • Goal: we want to assess the evidence in the data for an effect of the genotype on the phenotype.


  • Assumptions: the relationship between genotype and phenotype is linear; the individuals are not genetically related; there is no hidden confounding factors in the phenotypes.


  • Likelihood: we start by writing the usual linear regression for one individual

\forall i \in \{1,\ldots,N\}, \; y_i = \mu + \beta_1 g_i + \beta_2 \mathbf{1}_{g_i=1} + \epsilon_i \text{ with } \epsilon_i \overset{i.i.d}{\sim} \mathcal{N}(0,\tau^{-1})

where β1 is in fact the additive effect of the SNP, noted a from now on, and β2 is the dominance effect of the SNP, d = ak.

Let's now write the model in matrix notation:

Y = X B + E \text{ where } B = [ \mu \; a \; d ]^T

This gives the following multivariate Normal distribution for the phenotypes:

Y | X, \tau, B \sim \mathcal{N}(XB, \tau^{-1} I_N)

The likelihood of the parameters given the data is therefore:

\mathcal{L}(\tau, B) = \mathsf{P}(Y | X, \tau, B)

\mathcal{L}(\tau, B) = \left(\frac{\tau}{2 \pi}\right)^{\frac{N}{2}} exp \left( -\frac{\tau}{2} (Y - XB)^T (Y - XB) \right)


  • Priors: we use the usual conjugate prior

\mathsf{P}(\tau, B) = \mathsf{P}(\tau) \mathsf{P}(B | \tau)

A Gamma distribution for τ:

\tau \sim \Gamma(\kappa/2, \, \lambda/2)

which means:

\mathsf{P}(\tau) = \frac{\frac{\lambda}{2}^{\kappa/2}}{\Gamma(\frac{\kappa}{2})} \tau^{\frac{\kappa}{2}-1} e^{-\frac{\lambda}{2} \tau}

And a multivariate Normal distribution for B:

B | \tau \sim \mathcal{N}(\vec{0}, \, \tau^{-1} \Sigma_B) \text{ with } \Sigma_B = diag(\sigma_{\mu}^2, \sigma_a^2, \sigma_d^2)

which means:

\mathsf{P}(B | \tau) = \left(\frac{\tau}{2 \pi}\right)^{\frac{3}{2}} |\Sigma_B|^{-\frac{1}{2}} exp \left(-\frac{\tau}{2} B^T \Sigma_B^{-1} B \right)


  • Joint posterior (1):

\mathsf{P}(\tau, B | Y, X) = \mathsf{P}(\tau | Y, X) \mathsf{P}(B | Y, X, \tau)


  • Conditional posterior of B:

\mathsf{P}(B | Y, X, \tau) = \mathsf{P}(B, Y | X, \tau)

\mathsf{P}(B | Y, X, \tau) = \frac{\mathsf{P}(B, Y | X, \tau)}{\mathsf{P}(Y | X, \tau)}

\mathsf{P}(B | Y, X, \tau) = \frac{\mathsf{P}(B | \tau) \mathsf{P}(Y | X, B, \tau)}{\int \mathsf{P}(B | \tau) \mathsf{P}(Y | X, \tau, B) \mathsf{d}B}

Here and in the following, we neglect all constants (e.g. normalization constant, YTY, etc):

\mathsf{P}(B | Y, X, \tau) \propto \mathsf{P}(B | \tau) \mathsf{P}(Y | X, \tau, B)

We use the prior and likelihood and keep only the terms in B:

\mathsf{P}(B | Y, X, \tau) \propto exp(B^T \Sigma_B^{-1} B) exp((Y-XB)^T(Y-XB))

We expand:

\mathsf{P}(B | Y, X, \tau) \propto exp(B^T \Sigma_B^{-1} B - Y^TXB -B^TX^TY + B^TX^TXB)

We factorize some terms:

\mathsf{P}(B | Y, X, \tau) \propto exp(B^T (\Sigma_B^{-1} + X^TX) B - Y^TXB -B^TX^TY)

Let's define \Omega = (\Sigma_B^{-1} + X^TX)^{-1}. We can see that ΩT = Ω, which means that Ω is a symmetric matrix. This is particularly useful here because we can use the following equality: Ω − 1ΩT = I.

\mathsf{P}(B | Y, X, \tau) \propto exp(B^T \Omega^{-1} B - (X^TY)^T\Omega^{-1}\Omega^TB -B^T\Omega^{-1}\Omega^TX^TY)

This now becomes easy to factorizes totally:

\mathsf{P}(B | Y, X, \tau) \propto exp((B^T - \Omega X^TY)^T\Omega^{-1}(B - \Omega X^TY))

We recognize the kernel of a Normal distribution, allowing us to write the conditional posterior as:

B | Y, X, \tau \sim \mathcal{N}(\Omega X^TY, \tau^{-1} \Omega)


  • Posterior of τ:

Similarly to the equations above:

\mathsf{P}(\tau | Y, X) \propto \mathsf{P}(\tau) \mathsf{P}(Y | X, \tau)

But now, to handle the second term, we need to integrate over B, thus effectively taking into account the uncertainty in B:

\mathsf{P}(\tau | Y, X) \propto \mathsf{P}(\tau) \int \mathsf{P}(B | \tau) \mathsf{P}(Y | X, \tau, B) \mathsf{d}B

Again, we use the priors and likelihoods specified above (but everything inside the integral is kept inside it, even if it doesn't depend on B!):

\mathsf{P}(\tau | Y, X) \propto \tau^{\frac{\kappa}{2} - 1} e^{-\frac{\lambda}{2} \tau} \int \tau^{3/2} \tau^{N/2} exp(-\frac{\tau}{2} B^T \Sigma_B^{-1} B) exp(-\frac{\tau}{2} (Y - XB)^T (Y - XB)) \mathsf{d}B

As we used a conjugate prior for τ, we know that we expect a Gamma distribution for the posterior. Therefore, we can take τN / 2 out of the integral and start guessing what looks like a Gamma distribution. We also factorize inside the exponential:

\mathsf{P}(\tau | Y, X) \propto \tau^{\frac{N+\kappa}{2} - 1} e^{-\frac{\lambda}{2} \tau} \int \tau^{3/2} exp \left[-\frac{\tau}{2} \left( (B - \Omega X^T Y)^T \Omega^{-1} (B - \Omega X^T Y) - Y^T X \Omega X^T Y + Y^T Y \right) \right] \mathsf{d}B

We recognize the conditional posterior of B. This allows us to use the fact that the pdf of the Normal distribution integrates to one:

\mathsf{P}(\tau | Y, X) \propto \tau^{\frac{N+\kappa}{2} - 1} e^{-\frac{\lambda}{2} \tau} exp\left[-\frac{\tau}{2} (Y^T X \Omega X^T Y + Y^T Y) \right]

We finally recognize a Gamma distribution, allowing us to write the posterior as:

\tau | Y, X \sim \Gamma \left( \frac{N+\kappa}{2}, \; \frac{1}{2} (Y^T X \Omega X^T Y + Y^T Y + \lambda) \right)


  • Joint posterior (2): sometimes it is said that the joint posterior follows a Normal Inverse Gamma distribution:

B, \tau | Y, X \sim \mathcal{N}IG(\Omega X^TY, \tau^{-1}\Omega, \frac{N+\kappa}{2}, \frac{\lambda^\ast}{2})

where \lambda^\ast = (Y^T X \Omega X^T Y + Y^T Y + \lambda)


  • Marginal posterior of B: we can now integrate out τ:

\mathsf{P}(B | Y, X) = \int \mathsf{P}(B, \tau | Y, X) \mathsf{d}\tau

\mathsf{P}(B | Y, X) = \frac{\frac{\lambda^\ast}{2}^{\frac{N+\kappa}{2}}}{(2\pi)^\frac{3}{2} |\Omega|^{\frac{1}{2}} \Gamma(\frac{N+\kappa}{2})} \int \tau^{\frac{N+\kappa+3}{2}-1} exp \left[-\tau \left( \frac{\lambda^\ast}{2} + (B - \Omega X^TY)^T \Omega^{-1} (B - \Omega X^TY) \right) \right] \mathsf{d}\tau

Here we recognize the formula to integrate the Gamma function:

\mathsf{P}(B | Y, X) = \frac{\frac{\lambda^\ast}{2}^{\frac{N+\kappa}{2}} \Gamma(\frac{N+\kappa+3}{2})}{(2\pi)^\frac{3}{2} |\Omega|^{\frac{1}{2}} \Gamma(\frac{N+\kappa}{2})} \left( \frac{\lambda^\ast}{2} + (B - \Omega X^TY)^T \Omega^{-1} (B - \Omega X^TY) \right)^{-\frac{N+\kappa+3}{2}}


Personal tools