Difference between revisions of "A comparison of methods for quantifying prediction uncertainty in systems biology"

(References)
(General aspects)
Line 17: Line 17:
 
=== Study design and evidence level ===
 
=== Study design and evidence level ===
 
==== General aspects ====
 
==== General aspects ====
Synthetic data is generated for two examplary ODE models given a true parameter set. One model is smaller (5 parameter, 5 states, 5 observables) and used as sanity check, the other is larger (27 parameter, 25 states, 20 observables). The sample correlation coefficient is used to quantify agreement between predicted and true state trajectories. For FIM and ENS the MATLAB version of the MEIGO toolbox was used for parameter estimation, whereas for PP it was used MATLAB parameter estimation toolbox PESTO.
+
Synthetic data is generated for two examplary ODE models given a true parameter set. One model is smaller (5 parameter, 5 states, 5 observables) and used as sanity check, the other is larger (27 parameter, 25 states, 20 observables). The sample correlation coefficient is used to quantify agreement between predicted and true state trajectories. For FIM and ENS the MATLAB version of the MEIGO toolbox [https://doi.org/10.1080/10618600.2013.778779 Miasojedow et al.] was used for parameter estimation, whereas for PP it was used MATLAB parameter estimation toolbox PESTO [https://doi.org/10.1186/1471-2105-15-136 Egea et al.].
  
 
==== Design for Outcome O1 ====
 
==== Design for Outcome O1 ====

Revision as of 13:50, 25 February 2020

Citation

Villaverde, Alejandro F., et al. "A comparison of methods for quantifying prediction uncertainty in systems biology." IFAC-PapersOnLine 52.26 (2019): 45-51.

Permanent link to the paper

Summary

Three methods for quantifying prediction uncertainty in ODE models are assessed. Here, prediction uncertainty does not refer to estimated parameters, but to the uncertatinty of state trajectories. The three methods are: Fisher Information Matrix (FIM), Prediction Posetrior (PP), Ensemble Consensus (ENS).

Study outcomes

Outcome O1

For a small, fully-observed ODE model (α-pinene), all three methods yield nearly same results consistent with the known true trajectories. For a larger, only-partially observed ODE model (JAK2/STAT5), PP and ENS yield better accuracy than FIM. However, even for PP and ENS, confidence levels do not cover the truth.

Outcome O2

The computational cost of the three models is differing, especially for large problems: FIM (small), ENS (intermediate), PP (high)

Study design and evidence level

General aspects

Synthetic data is generated for two examplary ODE models given a true parameter set. One model is smaller (5 parameter, 5 states, 5 observables) and used as sanity check, the other is larger (27 parameter, 25 states, 20 observables). The sample correlation coefficient is used to quantify agreement between predicted and true state trajectories. For FIM and ENS the MATLAB version of the MEIGO toolbox Miasojedow et al. was used for parameter estimation, whereas for PP it was used MATLAB parameter estimation toolbox PESTO Egea et al..

Design for Outcome O1

Sample correlation coefficient is compared for the three different methods for the two different models, respectively.

Design for Outcome O2

Computation time is compared for the different models.

Further comments and aspects

References