Implementing the new approach
- Spent just about the entire day off and on implementing the approach described in yesterday's entry. Code includes the more direct implementation of the likelihood by generating trees from the process and averaging, as well as by weighting trees by their probability. Methods appear to agree reasonably well. The second approach should generate a wider variety of trees instead of using those produced by the first method. This could be done by varying the parameters of the transition rates to populate the collection. This needs significantly more exploring.
- Note that the major time savings in the new approach stem from the fact that when the transition matrix is changed, it may not be necessary generate new trees, only to change the weights , and therefore it is not necessary to recalculate the likelihood of the OU model, since the painting hasn't changed. Indeed, even the optimization / posteriors can be saved for each tree, offering substantial computational savings. In this way, every time the algorithm generates and fits a new tree, it can use that information for the rest of time, while it uses the MCMC approach to generate more likely trees, rather than trying an exhaustive approach. Note that this is MCMC approach can be used for an entirely frequentist style if desired, not all MCMCs are used to integrate posteriors. The makov chain construction part just speeds up the integration by monte carlo approach by focusing on the meaningful area of parameter space.
- Still actually need to implement the MCMC portion, so far just calculates the likelihood.
- Also should implement the multi-tree initialization and tree-augmentation steps.
Implementation of the approach (version stable copy) (most current version).
Computational note: All in pure R, may move all or slowest parts to C after profiling and optimization. Requires the msm package (multistate/hidden markov) for its matrix exponentials.