# User:TheLarry/Notebook/Larrys Notebook/2009/09/21

Continuous Time Markov Chains Main project page
Previous entry      Next entry

## Expectation Value of (?)

So I have been working my ass off to try to figure out how to calculate the expected value of a process completing. In the example I keep using it is A↔B↔C→D. And i want to know how long it would take on average to go from A-->D.

So I took the probability function P(t) which is exp{Qt} where Q is the generator matrix and normalized it. Then took the expectation value of t. that should be the expectation value to reach the state i want. <t> = ʃt P(t) dt. This gave me 6.03356 seconds for A-->D for all the k's are equal to 1. the inverse of this time should be k, which comes out to be .1657 1/s. Which is what i was shooting for. It also gives the correct values for B-->D and C-->D. As well as A-->D for k's equal to 2 and 3. So right now i am feeling kind of good about myself.

It didn't do so well for all forward reactions 3 and all backward reactions 1. though. So the discrete case still seems to work the best. There must be something in this that i don't understant. i am not sure what it is though (obviously). I gotta keep working. Fantastic.

why would it work for all k's 1, all k's 2, and all k's 3, but not all forward 3 and all backward 1.

I have to learn more about this shit. So apparently this process is called a birth-death process. So far i have been trying to avoid this but looks like i'll have to learn about it. I don't know why Pij(t) ≠ exp(Qt) but it might not? which would make my answers wrong? i don't know. I am grabbing at straws until i figure more of this out.

It looks like to figure out the probability as a function of time i'll need to solve differential equations. that or solve a balanced (?) equation. I am still working over here.

OK I am gonna head out, but i swear I WILL understand what it is i need to do before the week is over. I refuse to believe this is that hard.

Steve Koch 00:28, 22 September 2009 (EDT): OK, Man, I'm going to let you keep working, seems like you're on a roll. We can talk tomorrow in more detail if you want!