https://www.benchmarking.uni-freiburg.de/api.php?action=feedcontributions&user=Bwday&feedformat=atomBenchmark-Wiki - User contributions [en]2022-11-30T03:50:00ZUser contributionsMediaWiki 1.31.0https://www.benchmarking.uni-freiburg.de/index.php?title=Bioinformatics_and_Statistics:_LC%E2%80%90MS_(/MS)_Data_Preprocessing_for_Biomarker_Discovery&diff=740Bioinformatics and Statistics: LC‐MS (/MS) Data Preprocessing for Biomarker Discovery2020-02-27T15:47:15Z<p>Bwday: Created page with "__NUMBEREDHEADINGS__ === Citation === Horvatovich P, Suits F, Hoekman B, Bischoff R. Bioinformatics and Statistics: LC‐MS (/MS) Data Preprocessing for Biomarker Discovery. I..."</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
=== Citation ===<br />
Horvatovich P, Suits F, Hoekman B, Bischoff R. Bioinformatics and Statistics: LC‐MS (/MS) Data Preprocessing for Biomarker Discovery. In Comprehensive Biomarker Discovery and Validation for Clinical Application 2013 Jun 10 (pp. 199-225).<br />
<br />
[https://doi.org/10.1039/9781849734363-00199 Permanent link to the paper]<br />
<br />
<br />
=== Summary ===<br />
Briefly describe the scope of the paper, i.e. the field of research and/or application.<br />
<br />
=== Study outcomes ===<br />
List the paper results concerning method comparison and benchmarking:<br />
==== Outcome O1 ====<br />
The performance of ...<br />
<br />
Outcome O1 is presented as Figure X in the original publication. <br />
<br />
==== Outcome O2 ====<br />
...<br />
<br />
Outcome O2 is presented as Figure X in the original publication. <br />
<br />
==== Outcome On ====<br />
...<br />
<br />
Outcome On is presented as Figure X in the original publication. <br />
<br />
==== Further outcomes ====<br />
If intended, you can add further outcomes here.<br />
<br />
<br />
=== Study design and evidence level ===<br />
==== General aspects ====<br />
You can describe general design aspects here.<br />
The study designs for describing specific outcomes are listed in the following subsections:<br />
<br />
==== Design for Outcome O1 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
==== Design for Outcome O2 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
... <br />
<br />
==== Design for Outcome O ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
=== Further comments and aspects ===<br />
<br />
=== References ===<br />
The list of cited or related literature is placed here.</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Literature_Studies&diff=739Literature Studies2020-02-27T15:46:23Z<p>Bwday: /* Preprocessing high-throughput data */</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
{| class="wikitable"<br />
|-<br />
! Page summary<br />
|-<br />
| Here outcomes of benchmarking studies from the literature are collected. The primary aim is a comprehensive overview about neutral benchmark studies, i.e. assessments which were performed independenty on publication of a new approach. Studies which are not neutral are put in brackets. </br> <br />
<br />
The focus is on computational methods for analyzing experimental data (instead of comparing experimental techniques or platforms). </br><br />
<br />
Please extend this list by creating a new page and adding a link below. </br> <br />
Use the '''[[Guidelines_for_Summarizing_a_Literature_Study|guidelines described here]]'''.<br />
|}<br />
<br />
== Results from Literature ==<br />
<br />
=== Classification ===<br />
''' 2003 '''</br><br />
* [[Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data]]<br />
''' 2005 '''</br><br />
* [[A review and comparison of classification algorithms for medical decision making]]<br />
''' 2016 '''</br><br />
* [[Predicting Breast Cancer Survivability Using Data Mining Techniques]]<br />
<br />
=== Selection of Differential Features and Regions ===<br />
==== Identifying differential features ====<br />
''' 2006 '''</br><br />
* [[Rat toxicogenomic study reveals analytical consistency across microarray platforms]]<br />
''' 2010 '''</br><br />
* [[A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the sequencing Quality control consortium]]<br />
''' 2017 '''</br><br />
* [[Identification of differentially expressed peptides in high-throughput proteomics data]]<br />
* [[In-depth method assessments of differentially expressed protein detection for shotgun proteomics data with missing values]]<br />
* [[Strategies for analyzing bisulfite sequencing data]]<br />
''' 2018 '''</br><br />
* [[Identification of Differentially Methylated Sites with Weak Methylation Effects]]<br />
<br />
==== Identifying differential regions (e.g. DMRs) ====<br />
''' 2015 '''<br />
* [[De novo identification of differentially methylated regions in the human genome]]<br />
* [[MethylAction: detecting differentially methylated regions that distinguish biological subtypes]]<br />
* [[metilene: Fast and sensitive calling of differentially methylated regions from bisulfite sequencing data]]<br />
''' 2016 '''<br />
* [[seqlm: an MDL based method for identifying differentially methylated regions in high density methylation array data]]<br />
* [[Statistical methods for detecting differentially methylated regions based on MethylCap-seq data]]<br />
''' 2017 '''<br />
* [[DMRfinder: efficiently identifying differentially methylated regions from MethylC-seq data]]<br />
''' 2018 '''<br />
* [[Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus]]<br />
* [[DMRcaller: a versatile R/Bioconductor package for detection and visualization of differentially methylated regions in CpG and non-CpG contexts]]<br />
* [[MethCP: Differentially Methylated Region Detection with Change Point Models (bioRxiv)]]<br />
<br />
==== Identifying sets of features (e.g. gene set analyses) ====<br />
'''2009<br />
<br />
* [[A general modular framework for gene set enrichment analysis]]<br />
* [[Comparing gene set analysis methods on single-nucleotide polymorphism data from Genetic Analysis Workshop 16]]<br />
<br />
'''2018<br />
<br />
* [[Gene set analysis methods: a systematic comparison]]<br />
<br />
'''2020<br />
* [[Toward a gold standard for benchmarking gene set enrichment analysis]]<br />
<br />
==== Dimension reduction ====<br />
''' 2008 '''</br><br />
* [[On the Relationship Between Feature Selection and Classification Accuracy]]<br />
''' 2015 '''</br><br />
* [[Comparing feature selection methods for highdimensional imbalanced data: identifying rheumatoid arthritis cohorts from routine data]]<br />
<br />
=== Imputation methods for missing values ===<br />
<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 1996 || Schenker || [[Partially parametric techniques for multiple imputation]]<br />
|-<br />
| 1999 || Hastie T || [[Imputing Missing Data for Gene Expression Arrays]]<br />
|-<br />
| 2001 || Troyanskaya || [[Missing value estimation methods for DNA microarrays]]<br />
|-<br />
| 2002 || Engels J || [[Imputation of missing longitudinal data: a comparison of methods]]<br />
|-<br />
| 2003 || Oba || [[A Bayesian missing value estimation method for gene expression profile data]]<br />
|-<br />
| 2005 || Scholz || [[Nonlinear PCA: a missing data approach]]<br />
|-<br />
| 2007 || Stacklies || [[pcaMethods—a bioconductor package providing PCA methods for incomplete data]]<br />
|-<br />
| 2007 || Verboven || [[Sequential imputation for missing values]]<br />
|-<br />
| 2008 || Shaffer GN || [[Which missing value imputation method to use in expression profiles: a comparative study and two selection schemes]]<br />
|-<br />
| 2011 || Templ || [[Iterative stepwise regression imputation using standard and robust methods]]<br />
|-<br />
| 2012 || Hrydziuszko O || [[Missing values in mass spectrometry based metabolomics: an undervalued step in the data processing pipeline]]<br />
|-<br />
| 2012 || Stekhoven || [[MissForest—non-parametric missing value imputation for mixed-type data]]<br />
|-<br />
| 2013 || Taylor || [[Accounting for undetected compounds in statistical analyses of mass spectrometry ‘omic studies]]<br />
|-<br />
| 2013 || Waljee || [[Comparison of imputation methods for missing laboratory data in medicine]]<br />
|-<br />
| 2014 || Shah || [[Comparison of Random Forest and Parametric Imputation Models for Imputing Missing Data Using MICE: A CALIBER Study]]<br />
|-<br />
| 2014 || Rodwell || [[Comparison of methods for imputing limited-range variables: a simulation study]]<br />
|-<br />
| 2014 || Morris || [[Tuning multiple imputation by predictive mean matching and local residual draws]]<br />
|-<br />
| 2014 || Doove L || [[Recursive partitioning for missing data imputation in the presence of interaction effects]]<br />
|-<br />
| 2015 || Webb-Robertson BJM || [[Review, evaluation, and discussion of the challenges of missing value imputation for mass spectrometry-based label-free global proteomics]]<br />
|-<br />
| 2016 || Folch-Fortuny A || [[Assessment of maximum likelihood PCA missing data imputation]]<br />
|-<br />
| 2016 || Lazar C || [[Accounting for the Multiple Natures of Missing Values in Label-Free Quantitative Proteomics Data Sets to Compare Imputation Strategies]]<br />
|-<br />
| 2016 || Yin X || [[Multiple imputation and analysis for high-dimensional incomplete proteomics data]]<br />
|-<br />
| 2018 || Wei R || [[Missing Value Imputation Approach for Mass Spectrometry-based Metabolomics Data]]<br />
|-<br />
| 2018 || Poyatos R || [[Gap-filling a spatially explicit plant trait database: comparing imputation methods and different levels of environmental information]]<br />
|-<br />
| 2018 || O'Brien JJ || [[The effects of nonignorable missing data on label-free mass spectrometry proteomics experiments]]<br />
|}<br />
<br />
=== ODE-based Modelling ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2001 || Beal || [[Ways to Fit a PK Model with Some Data Below the Quantification Limit]]<br />
|-<br />
| 2008 || Balsa-Canto || [[Hybrid optimization method with general switching strategy for parameter estimation]]<br />
|-<br />
| 2011 || Tashkova || [[Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis]]<br />
|-<br />
| 2013 || Raue || [[Lessons Learned from Quantitative Dynamical Modeling in Systems Biology]]<br />
|-<br />
| 2013 || Dondelinger || [[ODE parameter inference using adaptive gradient matching with Gaussian processes]]<br />
|-<br />
| 2017 || Ballnus || [[Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems]]<br />
|-<br />
| 2017 || Henriques || [[Data-driven reverse engineering of signaling pathways using ensembles of dynamic models]]<br />
|-<br />
| 2017 || Melicher || [[Fast derivatives of likelihood functionals for ODE based models using adjoint-state method]]<br />
|-<br />
| 2017 || Penas || [[Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy]]<br />
|-<br />
| 2017 || Degasperi || [[Performance of objective functions and optimization procedures for parameter estimation in system biology models]]<br />
|-<br />
| 2017 || Fröhlich || [[Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks]]<br />
|-<br />
| 2018 || Schälte || [[Evaluation of Derivative-Free Optimizers for Parameter Estimation in Systems Biology]]<br />
|-<br />
| 2018 || Loos || [[Hierarchical optimization for the efficient parametrization of ODE models]]<br />
|-<br />
| 2018 || Stapor || [[Optimization and profile calculation of ODE models using second order adjoint sensitivity analysis]]<br />
|-<br />
| 2019 || Villaverde || [[A comparison of methods for quantifying prediction uncertainty in systems biology]]<br />
|-<br />
| 2019 || Hass || [[Benchmark problems for dynamic modeling of intracellular processes]]<br />
|-<br />
| 2019 || Villaverde || [[Benchmarking optimization methods for parameter estimation in large kinetic models]]<br />
|-<br />
| 2019 || Lines || [[Efficient computation of steady states in large-scale ODE models of biochemical reaction networks]]<br />
|-<br />
| 2019 || Stapor || [[Mini-batch optimization enables training of ODE models on large-scale datasets]]<br />
|-<br />
| 2019 || Wu || [[Parameter Estimation and Variable Selection for Big Systems of Linear Ordinary Differential Equations: A Matrix-Based Approach]]<br />
|-<br />
| 2019 || Pitt || [[Parameter estimation in models of biological oscillators: an automated regularised estimation approach]]<br />
|-<br />
| 2019 || Loos || [[Robust calibration of hierarchical population models for heterogeneous cell populations]]<br />
|-<br />
| 2019 || Clairon || [[Tracking for parameter and state estimation in possibly misspecified partially observed linear Ordinary Differential Equations]]<br />
|-<br />
| 2020 || Schmiester || [[Efficient parameterization of large-scale dynamic models based on relative measurements]]<br />
|-<br />
| 2020 || Castro || [[Testing structural identifiability by a simple scaling method]]<br />
|}<br />
<br />
=== Omics Workflows ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2008 || Neuweger H || [[MeltDB: a software platform for the analysis and integration of metabolomics experiment data]]<br />
|-<br />
| 2008 || Barla A || [[Machine learning methods for predictive proteomics]]<br />
|-<br />
| 2009 || Xia J || [[MetaboAnalyst: a web server for metabolomic data analysis and interpretation]]<br />
|-<br />
| 2013 || Weisser H || [[An Automated Pipeline for High-Throughput Label-Free Quantitative Proteomics]]<br />
|-<br />
| 2014 || Cox J || [[Accurate Proteome-wide Label-free Quantification by Delayed Normalization and Maximal Peptide Ratio Extraction, Termed MaxLFQ* ]]<br />
|-<br />
| 2015 || || [[ComparingVariant Call Files for Performance Benchmarkingof Next-Generation Sequencing Variant Calling Pipelines]]<br />
|-<br />
| 2016 || Tyanova S || [[The MaxQuant computational platform for mass spectrometry–based shotgun proteomics]]<br />
|-<br />
| 2016 || Röst HL || [[OpenMS: a flexible open-source software platform for mass spectrometry data analysis]]<br />
|-<br />
| 2017 || || [[A benchmarking of workflows for detecting differential splicing and differential expression at isoform level in human RNA-seq studies]]<br />
|-<br />
| 2018 || Välikangas T || [[A comprehensive evaluation of popular proteomics software workflows for label-free proteome quantification and imputation]]<br />
|-<br />
| 2019 || || [[A Systematic Evaluation of Single CellRNA-Seq Analysis Pipelines]]<br />
|-<br />
| 2019 || || [[Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays]]<br />
|}<br />
<br />
=== Preprocessing high-throughput data===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|- 1999 || Perkins DN || [[Probability-based protein identification by searching sequence databases using mass spectrometry data]]<br />
|-<br />
| 2003 || || [[A comparison of normalization methods for high density oligonucleotide array data based on variance and bias]]<br />
|-<br />
| 2003 || || [[Preprocessing of tandem mass spectrometric data to support automatic protein identification]]<br />
|-<br />
| 2005 || || [[Comparison of Affymetrix GeneChip Expression Measures]]<br />
|-<br />
| 2005 || Meleth S || [[The case for well-conducted experiments to validate statistical protocols for 2D gels: different pre-processing = different lists of significant proteins]]<br />
|-<br />
| 2005 || || [[Comparison of background correction and normalization procedures for high-density oligonucleotide microarrays]]<br />
|-<br />
| 2006 || || [[Using RNA sample titrations to assess microarray platform performance and normalization techniques]]<br />
|-<br />
| 2006 || Wang P || [[Normalization regarding non-random missing values in high-throughput mass spectrometry data]]<br />
|-<br />
| 2006 || Du P || [[Improved peak detection in mass spectrum by incorporating continuous wavelet transform-based pattern matching]]<br />
|-<br />
| 2007 || Carvalho B || [[Exploration, normalization, and genotype calls of high-density oligonucleotide SNP array data]]<br />
|-<br />
| 2007 || Cannataro M || [[MS‐Analyzer: preprocessing and data mining services for proteomics applications on the Grid]]<br />
|-<br />
| 2008 || || [[Comparison of preprocessing methods for the hgU133+2 chip from Affymetrix]]<br />
|-<br />
| 2009 || || [[Comparison of Affymetrix data normalization methods using 6,926 experiments across five array generations]]<br />
|-<br />
| 2009 || Mar JC || [[Data-driven normalization strategies for high-throughput quantitative RT-PCR]]<br />
|-<br />
| 2009 || Vakhrushev SY || [[Software platform for high-throughput glycomics]]<br />
|-<br />
| 2010 || || [[Consistency of predictive signature genes and classifiers generated using different microarray platforms]]<br />
|-<br />
| 2010 || || [[Detecting and correcting systematic variation in large-scale RNA sequencing data]]<br />
|-<br />
| 2010 || || [[Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments]]<br />
|-<br />
| 2010 || || [[Normalization of RNA-seq data using factor analysis of control genes or samples]]<br />
|-<br />
| 2010 || Armananzas R || [[Peakbin selection in mass spectrometry data using a consensus approach with estimation of distribution algorithms]]<br />
|-<br />
| 2011 || || [[Affymetrix GeneChip microarray preprocessing for multivariate analyses]]<br />
|-<br />
| 2011 || Zhang ZM || [[Peak alignment using wavelet pattern matching and differential evolution]]<br />
|-<br />
| 2012 || || [[A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis]]<br />
|-<br />
| 2013 || García-Torres M || [[Comparison of metaheuristic strategies for peakbin selection in proteomic mass spectrometry data]]<br />
|-<br />
| 2013 || Horvatovich P || [[Bioinformatics and Statistics: LC‐MS (/MS) Data Preprocessing for Biomarker Discovery]]<br />
|-<br />
| 2014 || || [[Normalyzer: A Tool for Rapid Evaluation of Normalization Methods for Omics Data Sets]]<br />
|-<br />
| 2014 || Zhou X || [[Prevention, diagnosis and treatment of high-throughput sequencing data pathologies]]<br />
|-<br />
| 2014 || Coble JB || [[Comparative evaluation of preprocessing freeware on chromatography/mass spectrometry data for signature discovery]]<br />
|-<br />
| 2014 || Aggio RB || [[Identifying and quantifying metabolites by scoring peaks of GC-MS data]]<br />
|-<br />
| 2014 || Cox J || [[Accurate proteome-wide label-free quantification by delayed normalization and maximal peptide ratio extraction, termed MaxLFQ]]<br />
|-<br />
| 2015 || Caraus I || [[Detecting and overcoming systematic bias in high-throughput screening technologies: a comprehensive review of practical issues and methodological solutions]]<br />
|-<br />
| 2015 || Tam S || [[Optimization of miRNA-seq data preprocessing]]<br />
|-<br />
| 2015 || Rafiei A || [[Comparison of peak‐picking workflows for untargeted liquid chromatography/high‐resolution mass spectrometry metabolomics data analysis]]<br />
|-<br />
| 2015 || Chawade A || [[Data processing has major impact on the outcome of quantitative label-free LC-MS analysis]]<br />
|-<br />
| 2015 || Wang T || [[A systematic study of normalization methods for Infinium 450K methylation data using whole-genome bisulfite sequencing data]]<br />
|-<br />
| 2015 || Lu J || [[Improved Peak Detection and Deconvolution of Native Electrospray Mass Spectra from Large Protein Complexes]]<br />
|-<br />
| 2016 || Yi L || [[Chemometric methods in data processing of mass spectrometry-based metabolomics: A review]]<br />
|-<br />
| 2016 || Tsuji J || [[Evaluation of preprocessing, mapping and postprocessing algorithms for analyzing whole genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Li B || [[Performance Evaluation and Online Realization of Data-driven Normalization Methods Used in LC/MS based Untargeted Metabolomics Analysis]]<br />
|-<br />
| 2016 || Zheng Y || [[An improved algorithm for peak detection in mass spectra based on continuous wavelet transform]]<br />
|-<br />
| 2017 || Li B || [[NOREVA: normalization and evaluation of MS-based metabolomics data]]<br />
|-<br />
| 2018 || Mazoure B || [[Identification and Correction of Additive and Multiplicative Spatial Biases in Experimental High-Throughput Screening]]<br />
|-<br />
| 2018 || Li Z || [[Comprehensive evaluation of untargeted metabolomics data processing software in feature detection, quantification and discriminating marker selection]]<br />
|-<br />
| 2018 || Willforss J || [[NormalyzerDE: Online Tool for Improved Normalization of Omics Expression Data and High-Sensitivity Differential Expression Analysis]]<br />
|}</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Literature_Studies&diff=738Literature Studies2020-02-27T15:45:41Z<p>Bwday: /* Preprocessing high-throughput data */</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
{| class="wikitable"<br />
|-<br />
! Page summary<br />
|-<br />
| Here outcomes of benchmarking studies from the literature are collected. The primary aim is a comprehensive overview about neutral benchmark studies, i.e. assessments which were performed independenty on publication of a new approach. Studies which are not neutral are put in brackets. </br> <br />
<br />
The focus is on computational methods for analyzing experimental data (instead of comparing experimental techniques or platforms). </br><br />
<br />
Please extend this list by creating a new page and adding a link below. </br> <br />
Use the '''[[Guidelines_for_Summarizing_a_Literature_Study|guidelines described here]]'''.<br />
|}<br />
<br />
== Results from Literature ==<br />
<br />
=== Classification ===<br />
''' 2003 '''</br><br />
* [[Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data]]<br />
''' 2005 '''</br><br />
* [[A review and comparison of classification algorithms for medical decision making]]<br />
''' 2016 '''</br><br />
* [[Predicting Breast Cancer Survivability Using Data Mining Techniques]]<br />
<br />
=== Selection of Differential Features and Regions ===<br />
==== Identifying differential features ====<br />
''' 2006 '''</br><br />
* [[Rat toxicogenomic study reveals analytical consistency across microarray platforms]]<br />
''' 2010 '''</br><br />
* [[A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the sequencing Quality control consortium]]<br />
''' 2017 '''</br><br />
* [[Identification of differentially expressed peptides in high-throughput proteomics data]]<br />
* [[In-depth method assessments of differentially expressed protein detection for shotgun proteomics data with missing values]]<br />
* [[Strategies for analyzing bisulfite sequencing data]]<br />
''' 2018 '''</br><br />
* [[Identification of Differentially Methylated Sites with Weak Methylation Effects]]<br />
<br />
==== Identifying differential regions (e.g. DMRs) ====<br />
''' 2015 '''<br />
* [[De novo identification of differentially methylated regions in the human genome]]<br />
* [[MethylAction: detecting differentially methylated regions that distinguish biological subtypes]]<br />
* [[metilene: Fast and sensitive calling of differentially methylated regions from bisulfite sequencing data]]<br />
''' 2016 '''<br />
* [[seqlm: an MDL based method for identifying differentially methylated regions in high density methylation array data]]<br />
* [[Statistical methods for detecting differentially methylated regions based on MethylCap-seq data]]<br />
''' 2017 '''<br />
* [[DMRfinder: efficiently identifying differentially methylated regions from MethylC-seq data]]<br />
''' 2018 '''<br />
* [[Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus]]<br />
* [[DMRcaller: a versatile R/Bioconductor package for detection and visualization of differentially methylated regions in CpG and non-CpG contexts]]<br />
* [[MethCP: Differentially Methylated Region Detection with Change Point Models (bioRxiv)]]<br />
<br />
==== Identifying sets of features (e.g. gene set analyses) ====<br />
'''2009<br />
<br />
* [[A general modular framework for gene set enrichment analysis]]<br />
* [[Comparing gene set analysis methods on single-nucleotide polymorphism data from Genetic Analysis Workshop 16]]<br />
<br />
'''2018<br />
<br />
* [[Gene set analysis methods: a systematic comparison]]<br />
<br />
'''2020<br />
* [[Toward a gold standard for benchmarking gene set enrichment analysis]]<br />
<br />
==== Dimension reduction ====<br />
''' 2008 '''</br><br />
* [[On the Relationship Between Feature Selection and Classification Accuracy]]<br />
''' 2015 '''</br><br />
* [[Comparing feature selection methods for highdimensional imbalanced data: identifying rheumatoid arthritis cohorts from routine data]]<br />
<br />
=== Imputation methods for missing values ===<br />
<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 1996 || Schenker || [[Partially parametric techniques for multiple imputation]]<br />
|-<br />
| 1999 || Hastie T || [[Imputing Missing Data for Gene Expression Arrays]]<br />
|-<br />
| 2001 || Troyanskaya || [[Missing value estimation methods for DNA microarrays]]<br />
|-<br />
| 2002 || Engels J || [[Imputation of missing longitudinal data: a comparison of methods]]<br />
|-<br />
| 2003 || Oba || [[A Bayesian missing value estimation method for gene expression profile data]]<br />
|-<br />
| 2005 || Scholz || [[Nonlinear PCA: a missing data approach]]<br />
|-<br />
| 2007 || Stacklies || [[pcaMethods—a bioconductor package providing PCA methods for incomplete data]]<br />
|-<br />
| 2007 || Verboven || [[Sequential imputation for missing values]]<br />
|-<br />
| 2008 || Shaffer GN || [[Which missing value imputation method to use in expression profiles: a comparative study and two selection schemes]]<br />
|-<br />
| 2011 || Templ || [[Iterative stepwise regression imputation using standard and robust methods]]<br />
|-<br />
| 2012 || Hrydziuszko O || [[Missing values in mass spectrometry based metabolomics: an undervalued step in the data processing pipeline]]<br />
|-<br />
| 2012 || Stekhoven || [[MissForest—non-parametric missing value imputation for mixed-type data]]<br />
|-<br />
| 2013 || Taylor || [[Accounting for undetected compounds in statistical analyses of mass spectrometry ‘omic studies]]<br />
|-<br />
| 2013 || Waljee || [[Comparison of imputation methods for missing laboratory data in medicine]]<br />
|-<br />
| 2014 || Shah || [[Comparison of Random Forest and Parametric Imputation Models for Imputing Missing Data Using MICE: A CALIBER Study]]<br />
|-<br />
| 2014 || Rodwell || [[Comparison of methods for imputing limited-range variables: a simulation study]]<br />
|-<br />
| 2014 || Morris || [[Tuning multiple imputation by predictive mean matching and local residual draws]]<br />
|-<br />
| 2014 || Doove L || [[Recursive partitioning for missing data imputation in the presence of interaction effects]]<br />
|-<br />
| 2015 || Webb-Robertson BJM || [[Review, evaluation, and discussion of the challenges of missing value imputation for mass spectrometry-based label-free global proteomics]]<br />
|-<br />
| 2016 || Folch-Fortuny A || [[Assessment of maximum likelihood PCA missing data imputation]]<br />
|-<br />
| 2016 || Lazar C || [[Accounting for the Multiple Natures of Missing Values in Label-Free Quantitative Proteomics Data Sets to Compare Imputation Strategies]]<br />
|-<br />
| 2016 || Yin X || [[Multiple imputation and analysis for high-dimensional incomplete proteomics data]]<br />
|-<br />
| 2018 || Wei R || [[Missing Value Imputation Approach for Mass Spectrometry-based Metabolomics Data]]<br />
|-<br />
| 2018 || Poyatos R || [[Gap-filling a spatially explicit plant trait database: comparing imputation methods and different levels of environmental information]]<br />
|-<br />
| 2018 || O'Brien JJ || [[The effects of nonignorable missing data on label-free mass spectrometry proteomics experiments]]<br />
|}<br />
<br />
=== ODE-based Modelling ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2001 || Beal || [[Ways to Fit a PK Model with Some Data Below the Quantification Limit]]<br />
|-<br />
| 2008 || Balsa-Canto || [[Hybrid optimization method with general switching strategy for parameter estimation]]<br />
|-<br />
| 2011 || Tashkova || [[Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis]]<br />
|-<br />
| 2013 || Raue || [[Lessons Learned from Quantitative Dynamical Modeling in Systems Biology]]<br />
|-<br />
| 2013 || Dondelinger || [[ODE parameter inference using adaptive gradient matching with Gaussian processes]]<br />
|-<br />
| 2017 || Ballnus || [[Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems]]<br />
|-<br />
| 2017 || Henriques || [[Data-driven reverse engineering of signaling pathways using ensembles of dynamic models]]<br />
|-<br />
| 2017 || Melicher || [[Fast derivatives of likelihood functionals for ODE based models using adjoint-state method]]<br />
|-<br />
| 2017 || Penas || [[Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy]]<br />
|-<br />
| 2017 || Degasperi || [[Performance of objective functions and optimization procedures for parameter estimation in system biology models]]<br />
|-<br />
| 2017 || Fröhlich || [[Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks]]<br />
|-<br />
| 2018 || Schälte || [[Evaluation of Derivative-Free Optimizers for Parameter Estimation in Systems Biology]]<br />
|-<br />
| 2018 || Loos || [[Hierarchical optimization for the efficient parametrization of ODE models]]<br />
|-<br />
| 2018 || Stapor || [[Optimization and profile calculation of ODE models using second order adjoint sensitivity analysis]]<br />
|-<br />
| 2019 || Villaverde || [[A comparison of methods for quantifying prediction uncertainty in systems biology]]<br />
|-<br />
| 2019 || Hass || [[Benchmark problems for dynamic modeling of intracellular processes]]<br />
|-<br />
| 2019 || Villaverde || [[Benchmarking optimization methods for parameter estimation in large kinetic models]]<br />
|-<br />
| 2019 || Lines || [[Efficient computation of steady states in large-scale ODE models of biochemical reaction networks]]<br />
|-<br />
| 2019 || Stapor || [[Mini-batch optimization enables training of ODE models on large-scale datasets]]<br />
|-<br />
| 2019 || Wu || [[Parameter Estimation and Variable Selection for Big Systems of Linear Ordinary Differential Equations: A Matrix-Based Approach]]<br />
|-<br />
| 2019 || Pitt || [[Parameter estimation in models of biological oscillators: an automated regularised estimation approach]]<br />
|-<br />
| 2019 || Loos || [[Robust calibration of hierarchical population models for heterogeneous cell populations]]<br />
|-<br />
| 2019 || Clairon || [[Tracking for parameter and state estimation in possibly misspecified partially observed linear Ordinary Differential Equations]]<br />
|-<br />
| 2020 || Schmiester || [[Efficient parameterization of large-scale dynamic models based on relative measurements]]<br />
|-<br />
| 2020 || Castro || [[Testing structural identifiability by a simple scaling method]]<br />
|}<br />
<br />
=== Omics Workflows ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2008 || Neuweger H || [[MeltDB: a software platform for the analysis and integration of metabolomics experiment data]]<br />
|-<br />
| 2008 || Barla A || [[Machine learning methods for predictive proteomics]]<br />
|-<br />
| 2009 || Xia J || [[MetaboAnalyst: a web server for metabolomic data analysis and interpretation]]<br />
|-<br />
| 2013 || Weisser H || [[An Automated Pipeline for High-Throughput Label-Free Quantitative Proteomics]]<br />
|-<br />
| 2014 || Cox J || [[Accurate Proteome-wide Label-free Quantification by Delayed Normalization and Maximal Peptide Ratio Extraction, Termed MaxLFQ* ]]<br />
|-<br />
| 2015 || || [[ComparingVariant Call Files for Performance Benchmarkingof Next-Generation Sequencing Variant Calling Pipelines]]<br />
|-<br />
| 2016 || Tyanova S || [[The MaxQuant computational platform for mass spectrometry–based shotgun proteomics]]<br />
|-<br />
| 2016 || Röst HL || [[OpenMS: a flexible open-source software platform for mass spectrometry data analysis]]<br />
|-<br />
| 2017 || || [[A benchmarking of workflows for detecting differential splicing and differential expression at isoform level in human RNA-seq studies]]<br />
|-<br />
| 2018 || Välikangas T || [[A comprehensive evaluation of popular proteomics software workflows for label-free proteome quantification and imputation]]<br />
|-<br />
| 2019 || || [[A Systematic Evaluation of Single CellRNA-Seq Analysis Pipelines]]<br />
|-<br />
| 2019 || || [[Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays]]<br />
|}<br />
<br />
=== Preprocessing high-throughput data===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|- 1999 || Perkins DN || [[Probability-based protein identification by searching sequence databases using mass spectrometry data]]<br />
|-<br />
| 2003 || || [[A comparison of normalization methods for high density oligonucleotide array data based on variance and bias]]<br />
|-<br />
| 2003 || || [[Preprocessing of tandem mass spectrometric data to support automatic protein identification]]<br />
|-<br />
| 2005 || || [[Comparison of Affymetrix GeneChip Expression Measures]]<br />
|-<br />
| 2005 || Meleth S || [[The case for well-conducted experiments to validate statistical protocols for 2D gels: different pre-processing = different lists of significant proteins]]<br />
|-<br />
| 2005 || || [[Comparison of background correction and normalization procedures for high-density oligonucleotide microarrays]]<br />
|-<br />
| 2006 || || [[Using RNA sample titrations to assess microarray platform performance and normalization techniques]]<br />
|-<br />
| 2006 || Wang P || [[Normalization regarding non-random missing values in high-throughput mass spectrometry data]]<br />
|-<br />
| 2006 || Du P || [[Improved peak detection in mass spectrum by incorporating continuous wavelet transform-based pattern matching]]<br />
|-<br />
| 2007 || Carvalho B || [[Exploration, normalization, and genotype calls of high-density oligonucleotide SNP array data]]<br />
|-<br />
| 2007 || Cannataro M || [[MS‐Analyzer: preprocessing and data mining services for proteomics applications on the Grid]]<br />
|-<br />
| 2008 || || [[Comparison of preprocessing methods for the hgU133+2 chip from Affymetrix]]<br />
|-<br />
| 2009 || || [[Comparison of Affymetrix data normalization methods using 6,926 experiments across five array generations]]<br />
|-<br />
| 2009 || Mar JC || [[Data-driven normalization strategies for high-throughput quantitative RT-PCR]]<br />
|-<br />
| 2009 || Vakhrushev SY || [[Software platform for high-throughput glycomics]]<br />
|-<br />
| 2010 || || [[Consistency of predictive signature genes and classifiers generated using different microarray platforms]]<br />
|-<br />
| 2010 || || [[Detecting and correcting systematic variation in large-scale RNA sequencing data]]<br />
|-<br />
| 2010 || || [[Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments]]<br />
|-<br />
| 2010 || || [[Normalization of RNA-seq data using factor analysis of control genes or samples]]<br />
|-<br />
| 2010 || Armananzas R || [[Peakbin selection in mass spectrometry data using a consensus approach with estimation of distribution algorithms]]<br />
|-<br />
| 2011 || || [[Affymetrix GeneChip microarray preprocessing for multivariate analyses]]<br />
|-<br />
| 2011 || Zhang ZM || [[Peak alignment using wavelet pattern matching and differential evolution]]<br />
|-<br />
| 2012 || || [[A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis]]<br />
|-<br />
| 2013 || García-Torres M || [[Comparison of metaheuristic strategies for peakbin selection in proteomic mass spectrometry data]]<br />
|-<br />
| 2013 || Horvatovich P || [[Data Preprocessing for Biomarker Discovery]]<br />
|-<br />
| 2014 || || [[Normalyzer: A Tool for Rapid Evaluation of Normalization Methods for Omics Data Sets]]<br />
|-<br />
| 2014 || Zhou X || [[Prevention, diagnosis and treatment of high-throughput sequencing data pathologies]]<br />
|-<br />
| 2014 || Coble JB || [[Comparative evaluation of preprocessing freeware on chromatography/mass spectrometry data for signature discovery]]<br />
|-<br />
| 2014 || Aggio RB || [[Identifying and quantifying metabolites by scoring peaks of GC-MS data]]<br />
|-<br />
| 2014 || Cox J || [[Accurate proteome-wide label-free quantification by delayed normalization and maximal peptide ratio extraction, termed MaxLFQ]]<br />
|-<br />
| 2015 || Caraus I || [[Detecting and overcoming systematic bias in high-throughput screening technologies: a comprehensive review of practical issues and methodological solutions]]<br />
|-<br />
| 2015 || Tam S || [[Optimization of miRNA-seq data preprocessing]]<br />
|-<br />
| 2015 || Rafiei A || [[Comparison of peak‐picking workflows for untargeted liquid chromatography/high‐resolution mass spectrometry metabolomics data analysis]]<br />
|-<br />
| 2015 || Chawade A || [[Data processing has major impact on the outcome of quantitative label-free LC-MS analysis]]<br />
|-<br />
| 2015 || Wang T || [[A systematic study of normalization methods for Infinium 450K methylation data using whole-genome bisulfite sequencing data]]<br />
|-<br />
| 2015 || Lu J || [[Improved Peak Detection and Deconvolution of Native Electrospray Mass Spectra from Large Protein Complexes]]<br />
|-<br />
| 2016 || Yi L || [[Chemometric methods in data processing of mass spectrometry-based metabolomics: A review]]<br />
|-<br />
| 2016 || Tsuji J || [[Evaluation of preprocessing, mapping and postprocessing algorithms for analyzing whole genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Li B || [[Performance Evaluation and Online Realization of Data-driven Normalization Methods Used in LC/MS based Untargeted Metabolomics Analysis]]<br />
|-<br />
| 2016 || Zheng Y || [[An improved algorithm for peak detection in mass spectra based on continuous wavelet transform]]<br />
|-<br />
| 2017 || Li B || [[NOREVA: normalization and evaluation of MS-based metabolomics data]]<br />
|-<br />
| 2018 || Mazoure B || [[Identification and Correction of Additive and Multiplicative Spatial Biases in Experimental High-Throughput Screening]]<br />
|-<br />
| 2018 || Li Z || [[Comprehensive evaluation of untargeted metabolomics data processing software in feature detection, quantification and discriminating marker selection]]<br />
|-<br />
| 2018 || Willforss J || [[NormalyzerDE: Online Tool for Improved Normalization of Omics Expression Data and High-Sensitivity Differential Expression Analysis]]<br />
|}</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Preprocessing_of_tandem_mass_spectrometric_data_to_support_automatic_protein_identification&diff=737Preprocessing of tandem mass spectrometric data to support automatic protein identification2020-02-27T15:36:25Z<p>Bwday: Created page with "__NUMBEREDHEADINGS__ === Citation === Gentzel M, Köcher T, Ponnusamy S, Wilm M. Preprocessing of tandem mass spectrometric data to support automatic protein identification. P..."</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
=== Citation ===<br />
Gentzel M, Köcher T, Ponnusamy S, Wilm M. Preprocessing of tandem mass spectrometric data to support automatic protein identification. PROTEOMICS: International Edition. 2003 Aug;3(8):1597-610.<br />
<br />
[https://doi.org/10.1002/pmic.200300486 Permanent link to the paper]<br />
<br />
<br />
=== Summary ===<br />
Briefly describe the scope of the paper, i.e. the field of research and/or application.<br />
<br />
=== Study outcomes ===<br />
List the paper results concerning method comparison and benchmarking:<br />
==== Outcome O1 ====<br />
The performance of ...<br />
<br />
Outcome O1 is presented as Figure X in the original publication. <br />
<br />
==== Outcome O2 ====<br />
...<br />
<br />
Outcome O2 is presented as Figure X in the original publication. <br />
<br />
==== Outcome On ====<br />
...<br />
<br />
Outcome On is presented as Figure X in the original publication. <br />
<br />
==== Further outcomes ====<br />
If intended, you can add further outcomes here.<br />
<br />
<br />
=== Study design and evidence level ===<br />
==== General aspects ====<br />
You can describe general design aspects here.<br />
The study designs for describing specific outcomes are listed in the following subsections:<br />
<br />
==== Design for Outcome O1 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
==== Design for Outcome O2 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
... <br />
<br />
==== Design for Outcome O ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
=== Further comments and aspects ===<br />
<br />
=== References ===<br />
The list of cited or related literature is placed here.</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Literature_Studies&diff=736Literature Studies2020-02-27T15:35:34Z<p>Bwday: /* Preprocessing high-throughput data */</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
{| class="wikitable"<br />
|-<br />
! Page summary<br />
|-<br />
| Here outcomes of benchmarking studies from the literature are collected. The primary aim is a comprehensive overview about neutral benchmark studies, i.e. assessments which were performed independenty on publication of a new approach. Studies which are not neutral are put in brackets. </br> <br />
<br />
The focus is on computational methods for analyzing experimental data (instead of comparing experimental techniques or platforms). </br><br />
<br />
Please extend this list by creating a new page and adding a link below. </br> <br />
Use the '''[[Guidelines_for_Summarizing_a_Literature_Study|guidelines described here]]'''.<br />
|}<br />
<br />
== Results from Literature ==<br />
<br />
=== Classification ===<br />
''' 2003 '''</br><br />
* [[Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data]]<br />
''' 2005 '''</br><br />
* [[A review and comparison of classification algorithms for medical decision making]]<br />
''' 2016 '''</br><br />
* [[Predicting Breast Cancer Survivability Using Data Mining Techniques]]<br />
<br />
=== Selection of Differential Features and Regions ===<br />
==== Identifying differential features ====<br />
''' 2006 '''</br><br />
* [[Rat toxicogenomic study reveals analytical consistency across microarray platforms]]<br />
''' 2010 '''</br><br />
* [[A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the sequencing Quality control consortium]]<br />
''' 2017 '''</br><br />
* [[Identification of differentially expressed peptides in high-throughput proteomics data]]<br />
* [[In-depth method assessments of differentially expressed protein detection for shotgun proteomics data with missing values]]<br />
* [[Strategies for analyzing bisulfite sequencing data]]<br />
''' 2018 '''</br><br />
* [[Identification of Differentially Methylated Sites with Weak Methylation Effects]]<br />
<br />
==== Identifying differential regions (e.g. DMRs) ====<br />
''' 2015 '''<br />
* [[De novo identification of differentially methylated regions in the human genome]]<br />
* [[MethylAction: detecting differentially methylated regions that distinguish biological subtypes]]<br />
* [[metilene: Fast and sensitive calling of differentially methylated regions from bisulfite sequencing data]]<br />
''' 2016 '''<br />
* [[seqlm: an MDL based method for identifying differentially methylated regions in high density methylation array data]]<br />
* [[Statistical methods for detecting differentially methylated regions based on MethylCap-seq data]]<br />
''' 2017 '''<br />
* [[DMRfinder: efficiently identifying differentially methylated regions from MethylC-seq data]]<br />
''' 2018 '''<br />
* [[Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus]]<br />
* [[DMRcaller: a versatile R/Bioconductor package for detection and visualization of differentially methylated regions in CpG and non-CpG contexts]]<br />
* [[MethCP: Differentially Methylated Region Detection with Change Point Models (bioRxiv)]]<br />
<br />
==== Identifying sets of features (e.g. gene set analyses) ====<br />
'''2009<br />
<br />
* [[A general modular framework for gene set enrichment analysis]]<br />
* [[Comparing gene set analysis methods on single-nucleotide polymorphism data from Genetic Analysis Workshop 16]]<br />
<br />
'''2018<br />
<br />
* [[Gene set analysis methods: a systematic comparison]]<br />
<br />
'''2020<br />
* [[Toward a gold standard for benchmarking gene set enrichment analysis]]<br />
<br />
==== Dimension reduction ====<br />
''' 2008 '''</br><br />
* [[On the Relationship Between Feature Selection and Classification Accuracy]]<br />
''' 2015 '''</br><br />
* [[Comparing feature selection methods for highdimensional imbalanced data: identifying rheumatoid arthritis cohorts from routine data]]<br />
<br />
=== Imputation methods for missing values ===<br />
<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 1996 || Schenker || [[Partially parametric techniques for multiple imputation]]<br />
|-<br />
| 1999 || Hastie T || [[Imputing Missing Data for Gene Expression Arrays]]<br />
|-<br />
| 2001 || Troyanskaya || [[Missing value estimation methods for DNA microarrays]]<br />
|-<br />
| 2002 || Engels J || [[Imputation of missing longitudinal data: a comparison of methods]]<br />
|-<br />
| 2003 || Oba || [[A Bayesian missing value estimation method for gene expression profile data]]<br />
|-<br />
| 2005 || Scholz || [[Nonlinear PCA: a missing data approach]]<br />
|-<br />
| 2007 || Stacklies || [[pcaMethods—a bioconductor package providing PCA methods for incomplete data]]<br />
|-<br />
| 2007 || Verboven || [[Sequential imputation for missing values]]<br />
|-<br />
| 2008 || Shaffer GN || [[Which missing value imputation method to use in expression profiles: a comparative study and two selection schemes]]<br />
|-<br />
| 2011 || Templ || [[Iterative stepwise regression imputation using standard and robust methods]]<br />
|-<br />
| 2012 || Hrydziuszko O || [[Missing values in mass spectrometry based metabolomics: an undervalued step in the data processing pipeline]]<br />
|-<br />
| 2012 || Stekhoven || [[MissForest—non-parametric missing value imputation for mixed-type data]]<br />
|-<br />
| 2013 || Taylor || [[Accounting for undetected compounds in statistical analyses of mass spectrometry ‘omic studies]]<br />
|-<br />
| 2013 || Waljee || [[Comparison of imputation methods for missing laboratory data in medicine]]<br />
|-<br />
| 2014 || Shah || [[Comparison of Random Forest and Parametric Imputation Models for Imputing Missing Data Using MICE: A CALIBER Study]]<br />
|-<br />
| 2014 || Rodwell || [[Comparison of methods for imputing limited-range variables: a simulation study]]<br />
|-<br />
| 2014 || Morris || [[Tuning multiple imputation by predictive mean matching and local residual draws]]<br />
|-<br />
| 2014 || Doove L || [[Recursive partitioning for missing data imputation in the presence of interaction effects]]<br />
|-<br />
| 2015 || Webb-Robertson BJM || [[Review, evaluation, and discussion of the challenges of missing value imputation for mass spectrometry-based label-free global proteomics]]<br />
|-<br />
| 2016 || Folch-Fortuny A || [[Assessment of maximum likelihood PCA missing data imputation]]<br />
|-<br />
| 2016 || Lazar C || [[Accounting for the Multiple Natures of Missing Values in Label-Free Quantitative Proteomics Data Sets to Compare Imputation Strategies]]<br />
|-<br />
| 2016 || Yin X || [[Multiple imputation and analysis for high-dimensional incomplete proteomics data]]<br />
|-<br />
| 2018 || Wei R || [[Missing Value Imputation Approach for Mass Spectrometry-based Metabolomics Data]]<br />
|-<br />
| 2018 || Poyatos R || [[Gap-filling a spatially explicit plant trait database: comparing imputation methods and different levels of environmental information]]<br />
|-<br />
| 2018 || O'Brien JJ || [[The effects of nonignorable missing data on label-free mass spectrometry proteomics experiments]]<br />
|}<br />
<br />
=== ODE-based Modelling ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2001 || Beal || [[Ways to Fit a PK Model with Some Data Below the Quantification Limit]]<br />
|-<br />
| 2008 || Balsa-Canto || [[Hybrid optimization method with general switching strategy for parameter estimation]]<br />
|-<br />
| 2011 || Tashkova || [[Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis]]<br />
|-<br />
| 2013 || Raue || [[Lessons Learned from Quantitative Dynamical Modeling in Systems Biology]]<br />
|-<br />
| 2013 || Dondelinger || [[ODE parameter inference using adaptive gradient matching with Gaussian processes]]<br />
|-<br />
| 2017 || Ballnus || [[Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems]]<br />
|-<br />
| 2017 || Henriques || [[Data-driven reverse engineering of signaling pathways using ensembles of dynamic models]]<br />
|-<br />
| 2017 || Melicher || [[Fast derivatives of likelihood functionals for ODE based models using adjoint-state method]]<br />
|-<br />
| 2017 || Penas || [[Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy]]<br />
|-<br />
| 2017 || Degasperi || [[Performance of objective functions and optimization procedures for parameter estimation in system biology models]]<br />
|-<br />
| 2017 || Fröhlich || [[Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks]]<br />
|-<br />
| 2018 || Schälte || [[Evaluation of Derivative-Free Optimizers for Parameter Estimation in Systems Biology]]<br />
|-<br />
| 2018 || Loos || [[Hierarchical optimization for the efficient parametrization of ODE models]]<br />
|-<br />
| 2018 || Stapor || [[Optimization and profile calculation of ODE models using second order adjoint sensitivity analysis]]<br />
|-<br />
| 2019 || Villaverde || [[A comparison of methods for quantifying prediction uncertainty in systems biology]]<br />
|-<br />
| 2019 || Hass || [[Benchmark problems for dynamic modeling of intracellular processes]]<br />
|-<br />
| 2019 || Villaverde || [[Benchmarking optimization methods for parameter estimation in large kinetic models]]<br />
|-<br />
| 2019 || Lines || [[Efficient computation of steady states in large-scale ODE models of biochemical reaction networks]]<br />
|-<br />
| 2019 || Stapor || [[Mini-batch optimization enables training of ODE models on large-scale datasets]]<br />
|-<br />
| 2019 || Wu || [[Parameter Estimation and Variable Selection for Big Systems of Linear Ordinary Differential Equations: A Matrix-Based Approach]]<br />
|-<br />
| 2019 || Pitt || [[Parameter estimation in models of biological oscillators: an automated regularised estimation approach]]<br />
|-<br />
| 2019 || Loos || [[Robust calibration of hierarchical population models for heterogeneous cell populations]]<br />
|-<br />
| 2019 || Clairon || [[Tracking for parameter and state estimation in possibly misspecified partially observed linear Ordinary Differential Equations]]<br />
|-<br />
| 2020 || Schmiester || [[Efficient parameterization of large-scale dynamic models based on relative measurements]]<br />
|-<br />
| 2020 || Castro || [[Testing structural identifiability by a simple scaling method]]<br />
|}<br />
<br />
=== Omics Workflows ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2008 || Neuweger H || [[MeltDB: a software platform for the analysis and integration of metabolomics experiment data]]<br />
|-<br />
| 2008 || Barla A || [[Machine learning methods for predictive proteomics]]<br />
|-<br />
| 2009 || Xia J || [[MetaboAnalyst: a web server for metabolomic data analysis and interpretation]]<br />
|-<br />
| 2013 || Weisser H || [[An Automated Pipeline for High-Throughput Label-Free Quantitative Proteomics]]<br />
|-<br />
| 2014 || Cox J || [[Accurate Proteome-wide Label-free Quantification by Delayed Normalization and Maximal Peptide Ratio Extraction, Termed MaxLFQ* ]]<br />
|-<br />
| 2015 || || [[ComparingVariant Call Files for Performance Benchmarkingof Next-Generation Sequencing Variant Calling Pipelines]]<br />
|-<br />
| 2016 || Tyanova S || [[The MaxQuant computational platform for mass spectrometry–based shotgun proteomics]]<br />
|-<br />
| 2016 || Röst HL || [[OpenMS: a flexible open-source software platform for mass spectrometry data analysis]]<br />
|-<br />
| 2017 || || [[A benchmarking of workflows for detecting differential splicing and differential expression at isoform level in human RNA-seq studies]]<br />
|-<br />
| 2018 || Välikangas T || [[A comprehensive evaluation of popular proteomics software workflows for label-free proteome quantification and imputation]]<br />
|-<br />
| 2019 || || [[A Systematic Evaluation of Single CellRNA-Seq Analysis Pipelines]]<br />
|-<br />
| 2019 || || [[Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays]]<br />
|}<br />
<br />
=== Preprocessing high-throughput data===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|- 1999 || Perkins DN || [[Probability-based protein identification by searching sequence databases using mass spectrometry data]]<br />
|-<br />
| 2003 || || [[A comparison of normalization methods for high density oligonucleotide array data based on variance and bias]]<br />
|-<br />
| 2003 || || [[Preprocessing of tandem mass spectrometric data to support automatic protein identification]]<br />
|-<br />
| 2005 || || [[Comparison of Affymetrix GeneChip Expression Measures]]<br />
|-<br />
| 2005 || Meleth S || [[The case for well-conducted experiments to validate statistical protocols for 2D gels: different pre-processing = different lists of significant proteins]]<br />
|-<br />
| 2005 || || [[Comparison of background correction and normalization procedures for high-density oligonucleotide microarrays]]<br />
|-<br />
| 2006 || || [[Using RNA sample titrations to assess microarray platform performance and normalization techniques]]<br />
|-<br />
| 2006 || Wang P || [[Normalization regarding non-random missing values in high-throughput mass spectrometry data]]<br />
|-<br />
| 2006 || Du P || [[Improved peak detection in mass spectrum by incorporating continuous wavelet transform-based pattern matching]]<br />
|-<br />
| 2007 || Carvalho B || [[Exploration, normalization, and genotype calls of high-density oligonucleotide SNP array data]]<br />
|-<br />
| 2007 || Cannataro M || [[MS‐Analyzer: preprocessing and data mining services for proteomics applications on the Grid]]<br />
|-<br />
| 2008 || || [[Comparison of preprocessing methods for the hgU133+2 chip from Affymetrix]]<br />
|-<br />
| 2009 || || [[Comparison of Affymetrix data normalization methods using 6,926 experiments across five array generations]]<br />
|-<br />
| 2009 || Mar JC || [[Data-driven normalization strategies for high-throughput quantitative RT-PCR]]<br />
|-<br />
| 2009 || Vakhrushev SY || [[Software platform for high-throughput glycomics]]<br />
|-<br />
| 2010 || || [[Consistency of predictive signature genes and classifiers generated using different microarray platforms]]<br />
|-<br />
| 2010 || || [[Detecting and correcting systematic variation in large-scale RNA sequencing data]]<br />
|-<br />
| 2010 || || [[Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments]]<br />
|-<br />
| 2010 || || [[Normalization of RNA-seq data using factor analysis of control genes or samples]]<br />
|-<br />
| 2010 || Armananzas R || [[Peakbin selection in mass spectrometry data using a consensus approach with estimation of distribution algorithms]]<br />
|-<br />
| 2011 || || [[Affymetrix GeneChip microarray preprocessing for multivariate analyses]]<br />
|-<br />
| 2011 || Zhang ZM || [[Peak alignment using wavelet pattern matching and differential evolution]]<br />
|-<br />
| 2012 || || [[A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis]]<br />
|-<br />
| 2013 || García-Torres M || [[Comparison of metaheuristic strategies for peakbin selection in proteomic mass spectrometry data]]<br />
|-<br />
| 2014 || || [[Normalyzer: A Tool for Rapid Evaluation of Normalization Methods for Omics Data Sets]]<br />
|-<br />
| 2014 || Zhou X || [[Prevention, diagnosis and treatment of high-throughput sequencing data pathologies]]<br />
|-<br />
| 2014 || Coble JB || [[Comparative evaluation of preprocessing freeware on chromatography/mass spectrometry data for signature discovery]]<br />
|-<br />
| 2014 || Aggio RB || [[Identifying and quantifying metabolites by scoring peaks of GC-MS data]]<br />
|-<br />
| 2014 || Cox J || [[Accurate proteome-wide label-free quantification by delayed normalization and maximal peptide ratio extraction, termed MaxLFQ]]<br />
|-<br />
| 2015 || Caraus I || [[Detecting and overcoming systematic bias in high-throughput screening technologies: a comprehensive review of practical issues and methodological solutions]]<br />
|-<br />
| 2015 || Tam S || [[Optimization of miRNA-seq data preprocessing]]<br />
|-<br />
| 2015 || Rafiei A || [[Comparison of peak‐picking workflows for untargeted liquid chromatography/high‐resolution mass spectrometry metabolomics data analysis]]<br />
|-<br />
| 2015 || Chawade A || [[Data processing has major impact on the outcome of quantitative label-free LC-MS analysis]]<br />
|-<br />
| 2015 || Wang T || [[A systematic study of normalization methods for Infinium 450K methylation data using whole-genome bisulfite sequencing data]]<br />
|-<br />
| 2015 || Lu J || [[Improved Peak Detection and Deconvolution of Native Electrospray Mass Spectra from Large Protein Complexes]]<br />
|-<br />
| 2016 || Yi L || [[Chemometric methods in data processing of mass spectrometry-based metabolomics: A review]]<br />
|-<br />
| 2016 || Tsuji J || [[Evaluation of preprocessing, mapping and postprocessing algorithms for analyzing whole genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Li B || [[Performance Evaluation and Online Realization of Data-driven Normalization Methods Used in LC/MS based Untargeted Metabolomics Analysis]]<br />
|-<br />
| 2016 || Zheng Y || [[An improved algorithm for peak detection in mass spectra based on continuous wavelet transform]]<br />
|-<br />
| 2017 || Li B || [[NOREVA: normalization and evaluation of MS-based metabolomics data]]<br />
|-<br />
| 2018 || Mazoure B || [[Identification and Correction of Additive and Multiplicative Spatial Biases in Experimental High-Throughput Screening]]<br />
|-<br />
| 2018 || Li Z || [[Comprehensive evaluation of untargeted metabolomics data processing software in feature detection, quantification and discriminating marker selection]]<br />
|-<br />
| 2018 || Willforss J || [[NormalyzerDE: Online Tool for Improved Normalization of Omics Expression Data and High-Sensitivity Differential Expression Analysis]]<br />
|}</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Accurate_proteome-wide_label-free_quantification_by_delayed_normalization_and_maximal_peptide_ratio_extraction,_termed_MaxLFQ&diff=735Accurate proteome-wide label-free quantification by delayed normalization and maximal peptide ratio extraction, termed MaxLFQ2020-02-27T15:32:20Z<p>Bwday: Created page with "__NUMBEREDHEADINGS__ === Citation === Cox J, Hein MY, Luber CA, Paron I, Nagaraj N, Mann M. Accurate proteome-wide label-free quantification by delayed normalization and maxim..."</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
=== Citation ===<br />
Cox J, Hein MY, Luber CA, Paron I, Nagaraj N, Mann M. Accurate proteome-wide label-free quantification by delayed normalization and maximal peptide ratio extraction, termed MaxLFQ. Molecular & cellular proteomics. 2014 Sep 1;13(9):2513-26.<br />
<br />
[https://doi.org/10.1074/mcp.M113.031591 Permanent link to the paper]<br />
<br />
<br />
=== Summary ===<br />
Briefly describe the scope of the paper, i.e. the field of research and/or application.<br />
<br />
=== Study outcomes ===<br />
List the paper results concerning method comparison and benchmarking:<br />
==== Outcome O1 ====<br />
The performance of ...<br />
<br />
Outcome O1 is presented as Figure X in the original publication. <br />
<br />
==== Outcome O2 ====<br />
...<br />
<br />
Outcome O2 is presented as Figure X in the original publication. <br />
<br />
==== Outcome On ====<br />
...<br />
<br />
Outcome On is presented as Figure X in the original publication. <br />
<br />
==== Further outcomes ====<br />
If intended, you can add further outcomes here.<br />
<br />
<br />
=== Study design and evidence level ===<br />
==== General aspects ====<br />
You can describe general design aspects here.<br />
The study designs for describing specific outcomes are listed in the following subsections:<br />
<br />
==== Design for Outcome O1 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
==== Design for Outcome O2 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
... <br />
<br />
==== Design for Outcome O ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
=== Further comments and aspects ===<br />
<br />
=== References ===<br />
The list of cited or related literature is placed here.</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Literature_Studies&diff=734Literature Studies2020-02-27T15:31:19Z<p>Bwday: /* Preprocessing high-throughput data */</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
{| class="wikitable"<br />
|-<br />
! Page summary<br />
|-<br />
| Here outcomes of benchmarking studies from the literature are collected. The primary aim is a comprehensive overview about neutral benchmark studies, i.e. assessments which were performed independenty on publication of a new approach. Studies which are not neutral are put in brackets. </br> <br />
<br />
The focus is on computational methods for analyzing experimental data (instead of comparing experimental techniques or platforms). </br><br />
<br />
Please extend this list by creating a new page and adding a link below. </br> <br />
Use the '''[[Guidelines_for_Summarizing_a_Literature_Study|guidelines described here]]'''.<br />
|}<br />
<br />
== Results from Literature ==<br />
<br />
=== Classification ===<br />
''' 2003 '''</br><br />
* [[Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data]]<br />
''' 2005 '''</br><br />
* [[A review and comparison of classification algorithms for medical decision making]]<br />
''' 2016 '''</br><br />
* [[Predicting Breast Cancer Survivability Using Data Mining Techniques]]<br />
<br />
=== Selection of Differential Features and Regions ===<br />
==== Identifying differential features ====<br />
''' 2006 '''</br><br />
* [[Rat toxicogenomic study reveals analytical consistency across microarray platforms]]<br />
''' 2010 '''</br><br />
* [[A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the sequencing Quality control consortium]]<br />
''' 2017 '''</br><br />
* [[Identification of differentially expressed peptides in high-throughput proteomics data]]<br />
* [[In-depth method assessments of differentially expressed protein detection for shotgun proteomics data with missing values]]<br />
* [[Strategies for analyzing bisulfite sequencing data]]<br />
''' 2018 '''</br><br />
* [[Identification of Differentially Methylated Sites with Weak Methylation Effects]]<br />
<br />
==== Identifying differential regions (e.g. DMRs) ====<br />
''' 2015 '''<br />
* [[De novo identification of differentially methylated regions in the human genome]]<br />
* [[MethylAction: detecting differentially methylated regions that distinguish biological subtypes]]<br />
* [[metilene: Fast and sensitive calling of differentially methylated regions from bisulfite sequencing data]]<br />
''' 2016 '''<br />
* [[seqlm: an MDL based method for identifying differentially methylated regions in high density methylation array data]]<br />
* [[Statistical methods for detecting differentially methylated regions based on MethylCap-seq data]]<br />
''' 2017 '''<br />
* [[DMRfinder: efficiently identifying differentially methylated regions from MethylC-seq data]]<br />
''' 2018 '''<br />
* [[Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus]]<br />
* [[DMRcaller: a versatile R/Bioconductor package for detection and visualization of differentially methylated regions in CpG and non-CpG contexts]]<br />
* [[MethCP: Differentially Methylated Region Detection with Change Point Models (bioRxiv)]]<br />
<br />
==== Identifying sets of features (e.g. gene set analyses) ====<br />
'''2009<br />
<br />
* [[A general modular framework for gene set enrichment analysis]]<br />
* [[Comparing gene set analysis methods on single-nucleotide polymorphism data from Genetic Analysis Workshop 16]]<br />
<br />
'''2018<br />
<br />
* [[Gene set analysis methods: a systematic comparison]]<br />
<br />
'''2020<br />
* [[Toward a gold standard for benchmarking gene set enrichment analysis]]<br />
<br />
==== Dimension reduction ====<br />
''' 2008 '''</br><br />
* [[On the Relationship Between Feature Selection and Classification Accuracy]]<br />
''' 2015 '''</br><br />
* [[Comparing feature selection methods for highdimensional imbalanced data: identifying rheumatoid arthritis cohorts from routine data]]<br />
<br />
=== Imputation methods for missing values ===<br />
<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 1996 || Schenker || [[Partially parametric techniques for multiple imputation]]<br />
|-<br />
| 1999 || Hastie T || [[Imputing Missing Data for Gene Expression Arrays]]<br />
|-<br />
| 2001 || Troyanskaya || [[Missing value estimation methods for DNA microarrays]]<br />
|-<br />
| 2002 || Engels J || [[Imputation of missing longitudinal data: a comparison of methods]]<br />
|-<br />
| 2003 || Oba || [[A Bayesian missing value estimation method for gene expression profile data]]<br />
|-<br />
| 2005 || Scholz || [[Nonlinear PCA: a missing data approach]]<br />
|-<br />
| 2007 || Stacklies || [[pcaMethods—a bioconductor package providing PCA methods for incomplete data]]<br />
|-<br />
| 2007 || Verboven || [[Sequential imputation for missing values]]<br />
|-<br />
| 2008 || Shaffer GN || [[Which missing value imputation method to use in expression profiles: a comparative study and two selection schemes]]<br />
|-<br />
| 2011 || Templ || [[Iterative stepwise regression imputation using standard and robust methods]]<br />
|-<br />
| 2012 || Hrydziuszko O || [[Missing values in mass spectrometry based metabolomics: an undervalued step in the data processing pipeline]]<br />
|-<br />
| 2012 || Stekhoven || [[MissForest—non-parametric missing value imputation for mixed-type data]]<br />
|-<br />
| 2013 || Taylor || [[Accounting for undetected compounds in statistical analyses of mass spectrometry ‘omic studies]]<br />
|-<br />
| 2013 || Waljee || [[Comparison of imputation methods for missing laboratory data in medicine]]<br />
|-<br />
| 2014 || Shah || [[Comparison of Random Forest and Parametric Imputation Models for Imputing Missing Data Using MICE: A CALIBER Study]]<br />
|-<br />
| 2014 || Rodwell || [[Comparison of methods for imputing limited-range variables: a simulation study]]<br />
|-<br />
| 2014 || Morris || [[Tuning multiple imputation by predictive mean matching and local residual draws]]<br />
|-<br />
| 2014 || Doove L || [[Recursive partitioning for missing data imputation in the presence of interaction effects]]<br />
|-<br />
| 2015 || Webb-Robertson BJM || [[Review, evaluation, and discussion of the challenges of missing value imputation for mass spectrometry-based label-free global proteomics]]<br />
|-<br />
| 2016 || Folch-Fortuny A || [[Assessment of maximum likelihood PCA missing data imputation]]<br />
|-<br />
| 2016 || Lazar C || [[Accounting for the Multiple Natures of Missing Values in Label-Free Quantitative Proteomics Data Sets to Compare Imputation Strategies]]<br />
|-<br />
| 2016 || Yin X || [[Multiple imputation and analysis for high-dimensional incomplete proteomics data]]<br />
|-<br />
| 2018 || Wei R || [[Missing Value Imputation Approach for Mass Spectrometry-based Metabolomics Data]]<br />
|-<br />
| 2018 || Poyatos R || [[Gap-filling a spatially explicit plant trait database: comparing imputation methods and different levels of environmental information]]<br />
|-<br />
| 2018 || O'Brien JJ || [[The effects of nonignorable missing data on label-free mass spectrometry proteomics experiments]]<br />
|}<br />
<br />
=== ODE-based Modelling ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2001 || Beal || [[Ways to Fit a PK Model with Some Data Below the Quantification Limit]]<br />
|-<br />
| 2008 || Balsa-Canto || [[Hybrid optimization method with general switching strategy for parameter estimation]]<br />
|-<br />
| 2011 || Tashkova || [[Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis]]<br />
|-<br />
| 2013 || Raue || [[Lessons Learned from Quantitative Dynamical Modeling in Systems Biology]]<br />
|-<br />
| 2013 || Dondelinger || [[ODE parameter inference using adaptive gradient matching with Gaussian processes]]<br />
|-<br />
| 2017 || Ballnus || [[Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems]]<br />
|-<br />
| 2017 || Henriques || [[Data-driven reverse engineering of signaling pathways using ensembles of dynamic models]]<br />
|-<br />
| 2017 || Melicher || [[Fast derivatives of likelihood functionals for ODE based models using adjoint-state method]]<br />
|-<br />
| 2017 || Penas || [[Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy]]<br />
|-<br />
| 2017 || Degasperi || [[Performance of objective functions and optimization procedures for parameter estimation in system biology models]]<br />
|-<br />
| 2017 || Fröhlich || [[Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks]]<br />
|-<br />
| 2018 || Schälte || [[Evaluation of Derivative-Free Optimizers for Parameter Estimation in Systems Biology]]<br />
|-<br />
| 2018 || Loos || [[Hierarchical optimization for the efficient parametrization of ODE models]]<br />
|-<br />
| 2018 || Stapor || [[Optimization and profile calculation of ODE models using second order adjoint sensitivity analysis]]<br />
|-<br />
| 2019 || Villaverde || [[A comparison of methods for quantifying prediction uncertainty in systems biology]]<br />
|-<br />
| 2019 || Hass || [[Benchmark problems for dynamic modeling of intracellular processes]]<br />
|-<br />
| 2019 || Villaverde || [[Benchmarking optimization methods for parameter estimation in large kinetic models]]<br />
|-<br />
| 2019 || Lines || [[Efficient computation of steady states in large-scale ODE models of biochemical reaction networks]]<br />
|-<br />
| 2019 || Stapor || [[Mini-batch optimization enables training of ODE models on large-scale datasets]]<br />
|-<br />
| 2019 || Wu || [[Parameter Estimation and Variable Selection for Big Systems of Linear Ordinary Differential Equations: A Matrix-Based Approach]]<br />
|-<br />
| 2019 || Pitt || [[Parameter estimation in models of biological oscillators: an automated regularised estimation approach]]<br />
|-<br />
| 2019 || Loos || [[Robust calibration of hierarchical population models for heterogeneous cell populations]]<br />
|-<br />
| 2019 || Clairon || [[Tracking for parameter and state estimation in possibly misspecified partially observed linear Ordinary Differential Equations]]<br />
|-<br />
| 2020 || Schmiester || [[Efficient parameterization of large-scale dynamic models based on relative measurements]]<br />
|-<br />
| 2020 || Castro || [[Testing structural identifiability by a simple scaling method]]<br />
|}<br />
<br />
=== Omics Workflows ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2008 || Neuweger H || [[MeltDB: a software platform for the analysis and integration of metabolomics experiment data]]<br />
|-<br />
| 2008 || Barla A || [[Machine learning methods for predictive proteomics]]<br />
|-<br />
| 2009 || Xia J || [[MetaboAnalyst: a web server for metabolomic data analysis and interpretation]]<br />
|-<br />
| 2013 || Weisser H || [[An Automated Pipeline for High-Throughput Label-Free Quantitative Proteomics]]<br />
|-<br />
| 2014 || Cox J || [[Accurate Proteome-wide Label-free Quantification by Delayed Normalization and Maximal Peptide Ratio Extraction, Termed MaxLFQ* ]]<br />
|-<br />
| 2015 || || [[ComparingVariant Call Files for Performance Benchmarkingof Next-Generation Sequencing Variant Calling Pipelines]]<br />
|-<br />
| 2016 || Tyanova S || [[The MaxQuant computational platform for mass spectrometry–based shotgun proteomics]]<br />
|-<br />
| 2016 || Röst HL || [[OpenMS: a flexible open-source software platform for mass spectrometry data analysis]]<br />
|-<br />
| 2017 || || [[A benchmarking of workflows for detecting differential splicing and differential expression at isoform level in human RNA-seq studies]]<br />
|-<br />
| 2018 || Välikangas T || [[A comprehensive evaluation of popular proteomics software workflows for label-free proteome quantification and imputation]]<br />
|-<br />
| 2019 || || [[A Systematic Evaluation of Single CellRNA-Seq Analysis Pipelines]]<br />
|-<br />
| 2019 || || [[Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays]]<br />
|}<br />
<br />
=== Preprocessing high-throughput data===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|- 1999 || Perkins DN || [[Probability-based protein identification by searching sequence databases using mass spectrometry data]]<br />
|-<br />
| 2003 || || [[A comparison of normalization methods for high density oligonucleotide array data based on variance and bias ]]<br />
|-<br />
| 2005 || || [[Comparison of Affymetrix GeneChip Expression Measures]]<br />
|-<br />
| 2005 || Meleth S || [[The case for well-conducted experiments to validate statistical protocols for 2D gels: different pre-processing = different lists of significant proteins]]<br />
|-<br />
| 2005 || || [[Comparison of background correction and normalization procedures for high-density oligonucleotide microarrays]]<br />
|-<br />
| 2006 || || [[Using RNA sample titrations to assess microarray platform performance and normalization techniques]]<br />
|-<br />
| 2006 || Wang P || [[Normalization regarding non-random missing values in high-throughput mass spectrometry data]]<br />
|-<br />
| 2006 || Du P || [[Improved peak detection in mass spectrum by incorporating continuous wavelet transform-based pattern matching]]<br />
|-<br />
| 2007 || Carvalho B || [[Exploration, normalization, and genotype calls of high-density oligonucleotide SNP array data]]<br />
|-<br />
| 2007 || Cannataro M || [[MS‐Analyzer: preprocessing and data mining services for proteomics applications on the Grid]]<br />
|-<br />
| 2008 || || [[Comparison of preprocessing methods for the hgU133+2 chip from Affymetrix]]<br />
|-<br />
| 2009 || || [[Comparison of Affymetrix data normalization methods using 6,926 experiments across five array generations]]<br />
|-<br />
| 2009 || Mar JC || [[Data-driven normalization strategies for high-throughput quantitative RT-PCR]]<br />
|-<br />
| 2009 || Vakhrushev SY || [[Software platform for high-throughput glycomics]]<br />
|-<br />
| 2010 || || [[Consistency of predictive signature genes and classifiers generated using different microarray platforms]]<br />
|-<br />
| 2010 || || [[Detecting and correcting systematic variation in large-scale RNA sequencing data]]<br />
|-<br />
| 2010 || || [[Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments]]<br />
|-<br />
| 2010 || || [[Normalization of RNA-seq data using factor analysis of control genes or samples]]<br />
|-<br />
| 2010 || Armananzas R || [[Peakbin selection in mass spectrometry data using a consensus approach with estimation of distribution algorithms]]<br />
|-<br />
| 2011 || || [[Affymetrix GeneChip microarray preprocessing for multivariate analyses]]<br />
|-<br />
| 2011 || Zhang ZM || [[Peak alignment using wavelet pattern matching and differential evolution]]<br />
|-<br />
| 2012 || || [[A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis]]<br />
|-<br />
| 2013 || García-Torres M || [[Comparison of metaheuristic strategies for peakbin selection in proteomic mass spectrometry data]]<br />
|-<br />
| 2014 || || [[Normalyzer: A Tool for Rapid Evaluation of Normalization Methods for Omics Data Sets]]<br />
|-<br />
| 2014 || Zhou X || [[Prevention, diagnosis and treatment of high-throughput sequencing data pathologies]]<br />
|-<br />
| 2014 || Coble JB || [[Comparative evaluation of preprocessing freeware on chromatography/mass spectrometry data for signature discovery]]<br />
|-<br />
| 2014 || Aggio RB || [[Identifying and quantifying metabolites by scoring peaks of GC-MS data]]<br />
|-<br />
| 2014 || Cox J || [[Accurate proteome-wide label-free quantification by delayed normalization and maximal peptide ratio extraction, termed MaxLFQ]]<br />
|-<br />
| 2015 || Caraus I || [[Detecting and overcoming systematic bias in high-throughput screening technologies: a comprehensive review of practical issues and methodological solutions]]<br />
|-<br />
| 2015 || Tam S || [[Optimization of miRNA-seq data preprocessing]]<br />
|-<br />
| 2015 || Rafiei A || [[Comparison of peak‐picking workflows for untargeted liquid chromatography/high‐resolution mass spectrometry metabolomics data analysis]]<br />
|-<br />
| 2015 || Chawade A || [[Data processing has major impact on the outcome of quantitative label-free LC-MS analysis]]<br />
|-<br />
| 2015 || Wang T || [[A systematic study of normalization methods for Infinium 450K methylation data using whole-genome bisulfite sequencing data]]<br />
|-<br />
| 2015 || Lu J || [[Improved Peak Detection and Deconvolution of Native Electrospray Mass Spectra from Large Protein Complexes]]<br />
|-<br />
| 2016 || Yi L || [[Chemometric methods in data processing of mass spectrometry-based metabolomics: A review]]<br />
|-<br />
| 2016 || Tsuji J || [[Evaluation of preprocessing, mapping and postprocessing algorithms for analyzing whole genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Li B || [[Performance Evaluation and Online Realization of Data-driven Normalization Methods Used in LC/MS based Untargeted Metabolomics Analysis]]<br />
|-<br />
| 2016 || Zheng Y || [[An improved algorithm for peak detection in mass spectra based on continuous wavelet transform]]<br />
|-<br />
| 2017 || Li B || [[NOREVA: normalization and evaluation of MS-based metabolomics data]]<br />
|-<br />
| 2018 || Mazoure B || [[Identification and Correction of Additive and Multiplicative Spatial Biases in Experimental High-Throughput Screening]]<br />
|-<br />
| 2018 || Li Z || [[Comprehensive evaluation of untargeted metabolomics data processing software in feature detection, quantification and discriminating marker selection]]<br />
|-<br />
| 2018 || Willforss J || [[NormalyzerDE: Online Tool for Improved Normalization of Omics Expression Data and High-Sensitivity Differential Expression Analysis]]<br />
|}</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Hierarchical_optimization_for_the_efficient_parametrization_of_ODE_models&diff=733Hierarchical optimization for the efficient parametrization of ODE models2020-02-26T14:11:05Z<p>Bwday: </p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
=== Citation ===<br />
C Loos, S Krause, J Hasenauer (2013) [https://doi.org/10.1093/bioinformatics/bty514 Hierarchical optimization for the efficient parametrization of ODE models] Bioinformatics, Volume 34, Issue 24, Pages 4266–4273.<br />
<br />
=== Summary ===<br />
<br />
In ODE-based modeling in the systems biology field, often only relative data is available whose measurement errors are not unknown. A common approach to deal with this setting is the introduction of scaling and noise parameters, see [[Lessons Learned from Quantitative Dynamical Modeling in Systems Biology]]. Since introducing additional parameters can decrease the performance of the parameter optimization algorithm, this paper introduced an hierarchical approach to separate the fitting of the ''nuisance'' from the ''dynamic'' parameters in every step. This was compared to the standard approach of fitting all parameters simultaneously in terms of optimizer convergence and computational efficiency for 3 systems biology models.<br />
<br />
=== Study outcomes ===<br />
<br />
==== Best Fit ====<br />
Standard and hierarchical multi-start optimization both find the same globally optimal objective function for both proposed error models (Gaussian and Laplace noise) except for one model with Laplace noise, where both approaches did not produce the same model trajectories. <br />
<br />
==== Convergence of Optimizer ====<br />
<br />
The hierarchical optimization improved the number of converged local fits from 18.4% to 29.3%, presented in Fig. 4C in the original publication.<br />
<br />
==== Computational Efficiency ====<br />
<br />
Fixing the computational budget, it was shown that the reduction in computation time per converged fit leads to 5.06 times more optimization runs reaching the best objective function value, visualized in Fig. 4D-E.<br />
<br />
=== Study design and evidence level ===<br />
* In order to generate the outcomes of the study, the '''MATLAB''' toolbox '''PESTO''' was used. In this framework, multi-start local optimization using the function ''fmincon.m'' was employed.<br />
<br />
* For error and scaling parameters, optimal estimates in each step have been derived analytically for Gaussian and Laplace noise. These analytical results were used where possible in this study.<br />
<br />
* The study used data of 3 models which were already calibrated. Application in more realistic modeling settings has not been performed.<br />
<br />
=== Further comments and aspects ===<br />
<br />
* Alternatively to using additional parameters to deal with relative data, relative data changes could be evaluated directly: [[Performance of objective functions and optimization procedures for parameter estimation in system biology models]]</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Hybrid_optimization_method_with_general_switching_strategy_for_parameter_estimation&diff=732Hybrid optimization method with general switching strategy for parameter estimation2020-02-26T14:08:11Z<p>Bwday: </p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
=== Citation ===<br />
Eva Balsa-Canto, Martin Peifer, Julio R Banga, Jens Timmer and Christian Fleck <br />
[https://doi.org/10.1186/1752-0509-2-26 Hybrid optimization method with general switching strategy for parameter estimation], 2008,<br />
BMC Systems Biology, 2:26.<br />
<br />
=== Summary ===<br />
Parameter estimation in systems biology model requires finding the global optimum of the objective function. This is usually done in terms of deterministic local optimizers which run the risk of precisely finding only a local optimum or by stochastic global optimizers which find the global optimum, but do not fully converge. Hence, the authors proposed a hybrid optimization which switches from stochastic global optimization to a local ''multiple shooting'' approach with a switching point which is determined dynamically. The hybriod method was demonstrated to vastly outperform stochastic global optimization and single as well as multiple shooting local methods in terms of computation time.<br />
<br />
=== Study outcomes ===<br />
<br />
==== Multiple vs Single Shooting ====<br />
As a deterministic optimizer, the multiple shooting method is better at finding the global optimum than a single shooting approach, as can be seen from Figure 1 and 2 in the original publication. However, this comes at the cost of increased computational cost.<br />
<br />
==== Comparison of Global Optimizers ====<br />
A comparison of single shooting, multiple shooting, a stochastic global optimizer and the hybrid method showed that the hybrid method decreased the computation time significantly compared to the alternatives in the two tested models, which is listed in Table 1 and Table 2 and demonstrated in Figure 3 in the publication.<br />
<br />
=== Study design and evidence level ===<br />
*The 2 tested models although realistic, have a rather small parameter space (less than 10 parameters).<br />
*The used data was simulated. Furthermore, noise levels are rather mild (0% and 10% noise to signal ratio).<br />
*Parameter bounds from which initial guesses for local fitting were drawn are rather narrow. Nevertheless, multiple shooting fails a considerable amount of times for the widest choice of initial conditions.<br />
<br />
=== Further comments and aspects ===<br />
*The authors claim that multistart methods are outdated, yet such a method was reported to be superior to competing algorithms in the benchmark study [[Lessons Learned from Quantitative Dynamical Modeling in Systems Biology]].</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Hybrid_optimization_method_with_general_switching_strategy_for_parameter_estimation&diff=731Hybrid optimization method with general switching strategy for parameter estimation2020-02-26T14:04:15Z<p>Bwday: /* Study design and evidence level */</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
=== Citation ===<br />
Eva Balsa-Canto, Martin Peifer, Julio R Banga, Jens Timmer and Christian Fleck <br />
[https://doi.org/10.1186/1752-0509-2-26 Hybrid optimization method with general switching strategy for parameter estimation], 2008,<br />
BMC Systems Biology, 2:26.<br />
<br />
=== Summary ===<br />
Parameter estimation in systems biology model requires finding the global optimum of the objective function. This is usually done in terms of deterministic local optimizers which run the risk of precisely finding only a local optimum or by stochastic global optimizers which find the global optimum, but do not fully converge. Hence, the authors proposed a hybrid optimization which switches from stochastic global optimization to a local ''multiple shooting'' approach with a switching point which is determined dynamically. The hybriod method was demonstrated to vastly outperform stochastic global optimization and single as well as multiple shooting local methods in terms of computation time.<br />
<br />
=== Study outcomes ===<br />
<br />
==== Multiple vs Single Shooting ====<br />
As a deterministic optimizer, the multiple shooting method is better at finding the global optimum than a single shooting approach, as can be seen from Figure 1 and 2 in the original publication. However, this comes at the cost of increased computational cost.<br />
<br />
==== Comparison of Global Optimizers ====<br />
A comparison of single shooting, multiple shooting, a stochastic global optimizer and the hybrid method showed that the hybrid method decreased the computation time significantly compared to the alternatives in the two tested models, which is listed in Table 1 and Table 2 and demonstrated in Figure 3 in the publication.<br />
<br />
=== Study design and evidence level ===<br />
* The 2 tested models although realistic, have a rather small parameter space (less than 10 parameters).<br />
* The used data was simulated. Furthermore, noise levels are rather mild (0% and 10% noise to signal ratio).<br />
* Parameter bounds from which initial guesses for local fitting were drawn are rather narrow. Nevertheless, multiple shooting fails a considerable amount of times for the widest choice of initial conditions.<br />
<br />
=== Further comments and aspects ===<br />
<br />
=== References ===<br />
The list of cited or related literature is placed here.</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Hybrid_optimization_method_with_general_switching_strategy_for_parameter_estimation&diff=730Hybrid optimization method with general switching strategy for parameter estimation2020-02-26T13:52:54Z<p>Bwday: /* Study outcomes */</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
=== Citation ===<br />
Eva Balsa-Canto, Martin Peifer, Julio R Banga, Jens Timmer and Christian Fleck <br />
[https://doi.org/10.1186/1752-0509-2-26 Hybrid optimization method with general switching strategy for parameter estimation], 2008,<br />
BMC Systems Biology, 2:26.<br />
<br />
=== Summary ===<br />
Parameter estimation in systems biology model requires finding the global optimum of the objective function. This is usually done in terms of deterministic local optimizers which run the risk of precisely finding only a local optimum or by stochastic global optimizers which find the global optimum, but do not fully converge. Hence, the authors proposed a hybrid optimization which switches from stochastic global optimization to a local ''multiple shooting'' approach with a switching point which is determined dynamically. The hybriod method was demonstrated to vastly outperform stochastic global optimization and single as well as multiple shooting local methods in terms of computation time.<br />
<br />
=== Study outcomes ===<br />
<br />
==== Multiple vs Single Shooting ====<br />
As a deterministic optimizer, the multiple shooting method is better at finding the global optimum than a single shooting approach, as can be seen from Figure 1 and 2 in the original publication. However, this comes at the cost of increased computational cost.<br />
<br />
==== Comparison of Global Optimizers ====<br />
A comparison of single shooting, multiple shooting, a stochastic global optimizer and the hybrid method showed that the hybrid method decreased the computation time significantly compared to the alternatives in the two tested models, which is listed in Table 1 and Table 2 and demonstrated in Figure 3 in the publication.<br />
<br />
=== Study design and evidence level ===<br />
==== General aspects ====<br />
You can describe general design aspects here.<br />
The study designs for describing specific outcomes are listed in the following subsections:<br />
<br />
==== Design for Outcome O1 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
==== Design for Outcome O2 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
... <br />
<br />
==== Design for Outcome O ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
=== Further comments and aspects ===<br />
<br />
=== References ===<br />
The list of cited or related literature is placed here.</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Hybrid_optimization_method_with_general_switching_strategy_for_parameter_estimation&diff=729Hybrid optimization method with general switching strategy for parameter estimation2020-02-26T13:36:16Z<p>Bwday: /* Citation */</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
=== Citation ===<br />
Eva Balsa-Canto, Martin Peifer, Julio R Banga, Jens Timmer and Christian Fleck <br />
[https://doi.org/10.1186/1752-0509-2-26 Hybrid optimization method with general switching strategy for parameter estimation], 2008,<br />
BMC Systems Biology, 2:26.<br />
<br />
=== Summary ===<br />
Parameter estimation in systems biology model requires finding the global optimum of the objective function. This is usually done in terms of deterministic local optimizers which run the risk of precisely finding only a local optimum or by stochastic global optimizers which find the global optimum, but do not fully converge. Hence, the authors proposed a hybrid optimization which switches from stochastic global optimization to a local ''multiple shooting'' approach with a switching point which is determined dynamically. The hybriod method was demonstrated to vastly outperform stochastic global optimization and single as well as multiple shooting local methods in terms of computation time.<br />
<br />
=== Study outcomes ===<br />
List the paper results concerning method comparison and benchmarking:<br />
==== Outcome O1 ====<br />
The performance of ...<br />
<br />
Outcome O1 is presented as Figure X in the original publication. <br />
<br />
==== Outcome O2 ====<br />
...<br />
<br />
Outcome O2 is presented as Figure X in the original publication. <br />
<br />
==== Outcome On ====<br />
...<br />
<br />
Outcome On is presented as Figure X in the original publication. <br />
<br />
==== Further outcomes ====<br />
If intended, you can add further outcomes here.<br />
<br />
<br />
=== Study design and evidence level ===<br />
==== General aspects ====<br />
You can describe general design aspects here.<br />
The study designs for describing specific outcomes are listed in the following subsections:<br />
<br />
==== Design for Outcome O1 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
==== Design for Outcome O2 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
... <br />
<br />
==== Design for Outcome O ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
=== Further comments and aspects ===<br />
<br />
=== References ===<br />
The list of cited or related literature is placed here.</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Hybrid_optimization_method_with_general_switching_strategy_for_parameter_estimation&diff=728Hybrid optimization method with general switching strategy for parameter estimation2020-02-26T13:25:00Z<p>Bwday: /* Summary */</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
=== Citation ===<br />
Eva Balsa-Canto, Martin Peifer, Julio R Banga, Jens Timmer and Christian Fleck <br />
Hybrid optimization method with general switching strategy for parameter estimation, 2008,<br />
BMC Systems Biology, 2:26.<br />
<br />
[https://doi.org/10.1186/1752-0509-2-26 Permanent link to the paper]<br />
<br />
=== Summary ===<br />
Parameter estimation in systems biology model requires finding the global optimum of the objective function. This is usually done in terms of deterministic local optimizers which run the risk of precisely finding only a local optimum or by stochastic global optimizers which find the global optimum, but do not fully converge. Hence, the authors proposed a hybrid optimization which switches from stochastic global optimization to a local ''multiple shooting'' approach with a switching point which is determined dynamically. The hybriod method was demonstrated to vastly outperform stochastic global optimization and single as well as multiple shooting local methods in terms of computation time.<br />
<br />
=== Study outcomes ===<br />
List the paper results concerning method comparison and benchmarking:<br />
==== Outcome O1 ====<br />
The performance of ...<br />
<br />
Outcome O1 is presented as Figure X in the original publication. <br />
<br />
==== Outcome O2 ====<br />
...<br />
<br />
Outcome O2 is presented as Figure X in the original publication. <br />
<br />
==== Outcome On ====<br />
...<br />
<br />
Outcome On is presented as Figure X in the original publication. <br />
<br />
==== Further outcomes ====<br />
If intended, you can add further outcomes here.<br />
<br />
<br />
=== Study design and evidence level ===<br />
==== General aspects ====<br />
You can describe general design aspects here.<br />
The study designs for describing specific outcomes are listed in the following subsections:<br />
<br />
==== Design for Outcome O1 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
==== Design for Outcome O2 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
... <br />
<br />
==== Design for Outcome O ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
=== Further comments and aspects ===<br />
<br />
=== References ===<br />
The list of cited or related literature is placed here.</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Peak_alignment_using_wavelet_pattern_matching_and_differential_evolution&diff=727Peak alignment using wavelet pattern matching and differential evolution2020-02-26T07:53:06Z<p>Bwday: Created page with "__NUMBEREDHEADINGS__ === Citation === Zhang ZM, Chen S, Liang YZ. Peak alignment using wavelet pattern matching and differential evolution. Talanta. 2011 Jan 30;83(4):1108-17...."</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
=== Citation ===<br />
Zhang ZM, Chen S, Liang YZ. Peak alignment using wavelet pattern matching and differential evolution. Talanta. 2011 Jan 30;83(4):1108-17.<br />
<br />
[https://doi.org/10.1016/j.talanta.2010.08.008 Permanent link to the paper]<br />
<br />
<br />
=== Summary ===<br />
Briefly describe the scope of the paper, i.e. the field of research and/or application.<br />
<br />
=== Study outcomes ===<br />
List the paper results concerning method comparison and benchmarking:<br />
==== Outcome O1 ====<br />
The performance of ...<br />
<br />
Outcome O1 is presented as Figure X in the original publication. <br />
<br />
==== Outcome O2 ====<br />
...<br />
<br />
Outcome O2 is presented as Figure X in the original publication. <br />
<br />
==== Outcome On ====<br />
...<br />
<br />
Outcome On is presented as Figure X in the original publication. <br />
<br />
==== Further outcomes ====<br />
If intended, you can add further outcomes here.<br />
<br />
<br />
=== Study design and evidence level ===<br />
==== General aspects ====<br />
You can describe general design aspects here.<br />
The study designs for describing specific outcomes are listed in the following subsections:<br />
<br />
==== Design for Outcome O1 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
==== Design for Outcome O2 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
... <br />
<br />
==== Design for Outcome O ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
=== Further comments and aspects ===<br />
<br />
=== References ===<br />
The list of cited or related literature is placed here.</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Literature_Studies&diff=726Literature Studies2020-02-26T07:50:54Z<p>Bwday: /* Preprocessing high-throughput data */</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
{| class="wikitable"<br />
|-<br />
! Page summary<br />
|-<br />
| Here outcomes of benchmarking studies from the literature are collected. The primary aim is a comprehensive overview about neutral benchmark studies, i.e. assessments which were performed independenty on publication of a new approach. Studies which are not neutral are put in brackets. </br> <br />
<br />
The focus is on computational methods for analyzing experimental data (instead of comparing experimental techniques or platforms). </br><br />
<br />
Please extend this list by creating a new page and adding a link below. </br> <br />
Use the '''[[Guidelines_for_Summarizing_a_Literature_Study|guidelines described here]]'''.<br />
|}<br />
<br />
== Results from Literature ==<br />
<br />
=== Classification ===<br />
''' 2003 '''</br><br />
* [[Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data]]<br />
''' 2005 '''</br><br />
* [[A review and comparison of classification algorithms for medical decision making]]<br />
''' 2016 '''</br><br />
* [[Predicting Breast Cancer Survivability Using Data Mining Techniques]]<br />
<br />
=== Selection of Differential Features and Regions ===<br />
==== Identifying differential features ====<br />
''' 2006 '''</br><br />
* [[Rat toxicogenomic study reveals analytical consistency across microarray platforms]]<br />
''' 2010 '''</br><br />
* [[A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the sequencing Quality control consortium]]<br />
''' 2017 '''</br><br />
* [[Identification of differentially expressed peptides in high-throughput proteomics data]]<br />
* [[In-depth method assessments of differentially expressed protein detection for shotgun proteomics data with missing values]]<br />
* [[Strategies for analyzing bisulfite sequencing data]]<br />
''' 2018 '''</br><br />
* [[Identification of Differentially Methylated Sites with Weak Methylation Effects]]<br />
<br />
==== Identifying differential regions (e.g. DMRs) ====<br />
''' 2015 '''<br />
* [[De novo identification of differentially methylated regions in the human genome]]<br />
* [[MethylAction: detecting differentially methylated regions that distinguish biological subtypes]]<br />
* [[metilene: Fast and sensitive calling of differentially methylated regions from bisulfite sequencing data]]<br />
''' 2016 '''<br />
* [[seqlm: an MDL based method for identifying differentially methylated regions in high density methylation array data]]<br />
* [[Statistical methods for detecting differentially methylated regions based on MethylCap-seq data]]<br />
''' 2017 '''<br />
* [[DMRfinder: efficiently identifying differentially methylated regions from MethylC-seq data]]<br />
''' 2018 '''<br />
* [[Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus]]<br />
* [[DMRcaller: a versatile R/Bioconductor package for detection and visualization of differentially methylated regions in CpG and non-CpG contexts]]<br />
* [[MethCP: Differentially Methylated Region Detection with Change Point Models (bioRxiv)]]<br />
<br />
==== Identifying sets of features (e.g. gene set analyses) ====<br />
'''2009<br />
<br />
* [[A general modular framework for gene set enrichment analysis]]<br />
* [[Comparing gene set analysis methods on single-nucleotide polymorphism data from Genetic Analysis Workshop 16]]<br />
<br />
'''2018<br />
<br />
* [[Gene set analysis methods: a systematic comparison]]<br />
<br />
'''2020<br />
* [[Toward a gold standard for benchmarking gene set enrichment analysis]]<br />
<br />
==== Dimension reduction ====<br />
''' 2008 '''</br><br />
* [[On the Relationship Between Feature Selection and Classification Accuracy]]<br />
''' 2015 '''</br><br />
* [[Comparing feature selection methods for highdimensional imbalanced data: identifying rheumatoid arthritis cohorts from routine data]]<br />
<br />
=== Imputation methods for missing values ===<br />
<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 1996 || Schenker || [[Partially parametric techniques for multiple imputation]]<br />
|-<br />
| 1999 || Hastie T || [[Imputing Missing Data for Gene Expression Arrays]]<br />
|-<br />
| 2001 || Troyanskaya || [[Missing value estimation methods for DNA microarrays]]<br />
|-<br />
| 2002 || Engels J || [[Imputation of missing longitudinal data: a comparison of methods]]<br />
|-<br />
| 2003 || Oba || [[A Bayesian missing value estimation method for gene expression profile data]]<br />
|-<br />
| 2005 || Scholz || [[Nonlinear PCA: a missing data approach]]<br />
|-<br />
| 2007 || Stacklies || [[pcaMethods—a bioconductor package providing PCA methods for incomplete data]]<br />
|-<br />
| 2007 || Verboven || [[Sequential imputation for missing values]]<br />
|-<br />
| 2008 || Shaffer GN || [[Which missing value imputation method to use in expression profiles: a comparative study and two selection schemes]]<br />
|-<br />
| 2011 || Templ || [[Iterative stepwise regression imputation using standard and robust methods]]<br />
|-<br />
| 2012 || Hrydziuszko O || [[Missing values in mass spectrometry based metabolomics: an undervalued step in the data processing pipeline]]<br />
|-<br />
| 2012 || Stekhoven || [[MissForest—non-parametric missing value imputation for mixed-type data]]<br />
|-<br />
| 2013 || Taylor || [[Accounting for undetected compounds in statistical analyses of mass spectrometry ‘omic studies]]<br />
|-<br />
| 2013 || Waljee || [[Comparison of imputation methods for missing laboratory data in medicine]]<br />
|-<br />
| 2014 || Shah || [[Comparison of Random Forest and Parametric Imputation Models for Imputing Missing Data Using MICE: A CALIBER Study]]<br />
|-<br />
| 2014 || Rodwell || [[Comparison of methods for imputing limited-range variables: a simulation study]]<br />
|-<br />
| 2014 || Morris || [[Tuning multiple imputation by predictive mean matching and local residual draws]]<br />
|-<br />
| 2014 || Doove L || [[Recursive partitioning for missing data imputation in the presence of interaction effects]]<br />
|-<br />
| 2015 || Webb-Robertson BJM || [[Review, evaluation, and discussion of the challenges of missing value imputation for mass spectrometry-based label-free global proteomics]]<br />
|-<br />
| 2016 || Folch-Fortuny A || [[Assessment of maximum likelihood PCA missing data imputation]]<br />
|-<br />
| 2016 || Lazar C || [[Accounting for the Multiple Natures of Missing Values in Label-Free Quantitative Proteomics Data Sets to Compare Imputation Strategies]]<br />
|-<br />
| 2016 || Yin X || [[Multiple imputation and analysis for high-dimensional incomplete proteomics data]]<br />
|-<br />
| 2018 || Wei R || [[Missing Value Imputation Approach for Mass Spectrometry-based Metabolomics Data]]<br />
|-<br />
| 2018 || Poyatos R || [[Gap-filling a spatially explicit plant trait database: comparing imputation methods and different levels of environmental information]]<br />
|-<br />
| 2018 || O'Brien JJ || [[The effects of nonignorable missing data on label-free mass spectrometry proteomics experiments]]<br />
|}<br />
<br />
=== ODE-based Modelling ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2001 || Beal || [[Ways to Fit a PK Model with Some Data Below the Quantification Limit]]<br />
|-<br />
| 2008 || Balsa-Canto || [[Hybrid optimization method with general switching strategy for parameter estimation]]<br />
|-<br />
| 2011 || Tashkova || [[Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis]]<br />
|-<br />
| 2013 || Raue || [[Lessons Learned from Quantitative Dynamical Modeling in Systems Biology]]<br />
|-<br />
| 2013 || Dondelinger || [[ODE parameter inference using adaptive gradient matching with Gaussian processes]]<br />
|-<br />
| 2017 || Ballnus || [[Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems]]<br />
|-<br />
| 2017 || Henriques || [[Data-driven reverse engineering of signaling pathways using ensembles of dynamic models]]<br />
|-<br />
| 2017 || Melicher || [[Fast derivatives of likelihood functionals for ODE based models using adjoint-state method]]<br />
|-<br />
| 2017 || Penas || [[Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy]]<br />
|-<br />
| 2017 || Degasperi || [[Performance of objective functions and optimization procedures for parameter estimation in system biology models]]<br />
|-<br />
| 2017 || Fröhlich || [[Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks]]<br />
|-<br />
| 2018 || Schälte || [[Evaluation of Derivative-Free Optimizers for Parameter Estimation in Systems Biology]]<br />
|-<br />
| 2018 || Loos || [[Hierarchical optimization for the efficient parametrization of ODE models]]<br />
|-<br />
| 2018 || Stapor || [[Optimization and profile calculation of ODE models using second order adjoint sensitivity analysis]]<br />
|-<br />
| 2019 || Villaverde || [[A comparison of methods for quantifying prediction uncertainty in systems biology]]<br />
|-<br />
| 2019 || Hass || [[Benchmark problems for dynamic modeling of intracellular processes]]<br />
|-<br />
| 2019 || Villaverde || [[Benchmarking optimization methods for parameter estimation in large kinetic models]]<br />
|-<br />
| 2019 || Lines || [[Efficient computation of steady states in large-scale ODE models of biochemical reaction networks]]<br />
|-<br />
| 2019 || Stapor || [[Mini-batch optimization enables training of ODE models on large-scale datasets]]<br />
|-<br />
| 2019 || Wu || [[Parameter Estimation and Variable Selection for Big Systems of Linear Ordinary Differential Equations: A Matrix-Based Approach]]<br />
|-<br />
| 2019 || Pitt || [[Parameter estimation in models of biological oscillators: an automated regularised estimation approach]]<br />
|-<br />
| 2019 || Loos || [[Robust calibration of hierarchical population models for heterogeneous cell populations]]<br />
|-<br />
| 2019 || Clairon || [[Tracking for parameter and state estimation in possibly misspecified partially observed linear Ordinary Differential Equations]]<br />
|-<br />
| 2020 || Schmiester || [[Efficient parameterization of large-scale dynamic models based on relative measurements]]<br />
|-<br />
| 2020 || Castro || [[Testing structural identifiability by a simple scaling method]]<br />
|}<br />
<br />
=== Omics Workflows ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2008 || Neuweger H || [[MeltDB: a software platform for the analysis and integration of metabolomics experiment data]]<br />
|-<br />
| 2008 || Barla A || [[Machine learning methods for predictive proteomics]]<br />
|-<br />
| 2009 || Xia J || [[MetaboAnalyst: a web server for metabolomic data analysis and interpretation]]<br />
|-<br />
| 2013 || Weisser H || [[An Automated Pipeline for High-Throughput Label-Free Quantitative Proteomics]]<br />
|-<br />
| 2014 || Cox J || [[Accurate Proteome-wide Label-free Quantification by Delayed Normalization and Maximal Peptide Ratio Extraction, Termed MaxLFQ* ]]<br />
|-<br />
| 2015 || || [[ComparingVariant Call Files for Performance Benchmarkingof Next-Generation Sequencing Variant Calling Pipelines]]<br />
|-<br />
| 2016 || Tyanova S || [[The MaxQuant computational platform for mass spectrometry–based shotgun proteomics]]<br />
|-<br />
| 2016 || Röst HL || [[OpenMS: a flexible open-source software platform for mass spectrometry data analysis]]<br />
|-<br />
| 2017 || || [[A benchmarking of workflows for detecting differential splicing and differential expression at isoform level in human RNA-seq studies]]<br />
|-<br />
| 2018 || Välikangas T || [[A comprehensive evaluation of popular proteomics software workflows for label-free proteome quantification and imputation]]<br />
|-<br />
| 2019 || || [[A Systematic Evaluation of Single CellRNA-Seq Analysis Pipelines]]<br />
|-<br />
| 2019 || || [[Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays]]<br />
|}<br />
<br />
=== Preprocessing high-throughput data===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|- 1999 || Perkins DN || [[Probability-based protein identification by searching sequence databases using mass spectrometry data]]<br />
|-<br />
| 2003 || || [[A comparison of normalization methods for high density oligonucleotide array data based on variance and bias ]]<br />
|-<br />
| 2005 || || [[Comparison of Affymetrix GeneChip Expression Measures]]<br />
|-<br />
| 2005 || Meleth S || [[The case for well-conducted experiments to validate statistical protocols for 2D gels: different pre-processing = different lists of significant proteins]]<br />
|-<br />
| 2005 || || [[Comparison of background correction and normalization procedures for high-density oligonucleotide microarrays]]<br />
|-<br />
| 2006 || || [[Using RNA sample titrations to assess microarray platform performance and normalization techniques]]<br />
|-<br />
| 2006 || Wang P || [[Normalization regarding non-random missing values in high-throughput mass spectrometry data]]<br />
|-<br />
| 2006 || Du P || [[Improved peak detection in mass spectrum by incorporating continuous wavelet transform-based pattern matching]]<br />
|-<br />
| 2007 || Carvalho B || [[Exploration, normalization, and genotype calls of high-density oligonucleotide SNP array data]]<br />
|-<br />
| 2007 || Cannataro M || [[MS‐Analyzer: preprocessing and data mining services for proteomics applications on the Grid]]<br />
|-<br />
| 2008 || || [[Comparison of preprocessing methods for the hgU133+2 chip from Affymetrix]]<br />
|-<br />
| 2009 || || [[Comparison of Affymetrix data normalization methods using 6,926 experiments across five array generations]]<br />
|-<br />
| 2009 || Mar JC || [[Data-driven normalization strategies for high-throughput quantitative RT-PCR]]<br />
|-<br />
| 2009 || Vakhrushev SY || [[Software platform for high-throughput glycomics]]<br />
|-<br />
| 2010 || || [[Consistency of predictive signature genes and classifiers generated using different microarray platforms]]<br />
|-<br />
| 2010 || || [[Detecting and correcting systematic variation in large-scale RNA sequencing data]]<br />
|-<br />
| 2010 || || [[Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments]]<br />
|-<br />
| 2010 || || [[Normalization of RNA-seq data using factor analysis of control genes or samples]]<br />
|-<br />
| 2010 || Armananzas R || [[Peakbin selection in mass spectrometry data using a consensus approach with estimation of distribution algorithms]]<br />
|-<br />
| 2011 || || [[Affymetrix GeneChip microarray preprocessing for multivariate analyses]]<br />
|-<br />
| 2011 || Zhang ZM || [[Peak alignment using wavelet pattern matching and differential evolution]]<br />
|-<br />
| 2012 || || [[A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis]]<br />
|-<br />
| 2013 || García-Torres M || [[Comparison of metaheuristic strategies for peakbin selection in proteomic mass spectrometry data]]<br />
|-<br />
| 2014 || || [[Normalyzer: A Tool for Rapid Evaluation of Normalization Methods for Omics Data Sets]]<br />
|-<br />
| 2014 || Zhou X || [[Prevention, diagnosis and treatment of high-throughput sequencing data pathologies]]<br />
|-<br />
| 2014 || Coble JB || [[Comparative evaluation of preprocessing freeware on chromatography/mass spectrometry data for signature discovery]]<br />
|-<br />
| 2014 || Aggio RB || [[Identifying and quantifying metabolites by scoring peaks of GC-MS data]]<br />
|-<br />
| 2015 || Caraus I || [[Detecting and overcoming systematic bias in high-throughput screening technologies: a comprehensive review of practical issues and methodological solutions]]<br />
|-<br />
| 2015 || Tam S || [[Optimization of miRNA-seq data preprocessing]]<br />
|-<br />
| 2015 || Rafiei A || [[Comparison of peak‐picking workflows for untargeted liquid chromatography/high‐resolution mass spectrometry metabolomics data analysis]]<br />
|-<br />
| 2015 || Chawade A || [[Data processing has major impact on the outcome of quantitative label-free LC-MS analysis]]<br />
|-<br />
| 2015 || Wang T || [[A systematic study of normalization methods for Infinium 450K methylation data using whole-genome bisulfite sequencing data]]<br />
|-<br />
| 2015 || Lu J || [[Improved Peak Detection and Deconvolution of Native Electrospray Mass Spectra from Large Protein Complexes]]<br />
|-<br />
| 2016 || Yi L || [[Chemometric methods in data processing of mass spectrometry-based metabolomics: A review]]<br />
|-<br />
| 2016 || Tsuji J || [[Evaluation of preprocessing, mapping and postprocessing algorithms for analyzing whole genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Li B || [[Performance Evaluation and Online Realization of Data-driven Normalization Methods Used in LC/MS based Untargeted Metabolomics Analysis]]<br />
|-<br />
| 2016 || Zheng Y || [[An improved algorithm for peak detection in mass spectra based on continuous wavelet transform]]<br />
|-<br />
| 2017 || Li B || [[NOREVA: normalization and evaluation of MS-based metabolomics data]]<br />
|-<br />
| 2018 || Mazoure B || [[Identification and Correction of Additive and Multiplicative Spatial Biases in Experimental High-Throughput Screening]]<br />
|-<br />
| 2018 || Li Z || [[Comprehensive evaluation of untargeted metabolomics data processing software in feature detection, quantification and discriminating marker selection]]<br />
|-<br />
| 2018 || Willforss J || [[NormalyzerDE: Online Tool for Improved Normalization of Omics Expression Data and High-Sensitivity Differential Expression Analysis]]<br />
|}</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Identifying_and_quantifying_metabolites_by_scoring_peaks_of_GC-MS_data&diff=725Identifying and quantifying metabolites by scoring peaks of GC-MS data2020-02-26T07:36:53Z<p>Bwday: Created page with "__NUMBEREDHEADINGS__ === Citation === Aggio RB, Mayor A, Reade S, Probert CS, Ruggiero K. Identifying and quantifying metabolites by scoring peaks of GC-MS data. BMC bioinform..."</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
=== Citation ===<br />
Aggio RB, Mayor A, Reade S, Probert CS, Ruggiero K. Identifying and quantifying metabolites by scoring peaks of GC-MS data. BMC bioinformatics. 2014 Dec;15(1):374.<br />
<br />
[https://doi.org/10.1186/s12859-014-0374-2 Permanent link to the paper]<br />
<br />
<br />
=== Summary ===<br />
Briefly describe the scope of the paper, i.e. the field of research and/or application.<br />
<br />
=== Study outcomes ===<br />
List the paper results concerning method comparison and benchmarking:<br />
==== Outcome O1 ====<br />
The performance of ...<br />
<br />
Outcome O1 is presented as Figure X in the original publication. <br />
<br />
==== Outcome O2 ====<br />
...<br />
<br />
Outcome O2 is presented as Figure X in the original publication. <br />
<br />
==== Outcome On ====<br />
...<br />
<br />
Outcome On is presented as Figure X in the original publication. <br />
<br />
==== Further outcomes ====<br />
If intended, you can add further outcomes here.<br />
<br />
<br />
=== Study design and evidence level ===<br />
==== General aspects ====<br />
You can describe general design aspects here.<br />
The study designs for describing specific outcomes are listed in the following subsections:<br />
<br />
==== Design for Outcome O1 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
==== Design for Outcome O2 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
... <br />
<br />
==== Design for Outcome O ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
=== Further comments and aspects ===<br />
<br />
=== References ===<br />
The list of cited or related literature is placed here.</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Literature_Studies&diff=724Literature Studies2020-02-26T07:36:08Z<p>Bwday: /* Preprocessing high-throughput data */</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
{| class="wikitable"<br />
|-<br />
! Page summary<br />
|-<br />
| Here outcomes of benchmarking studies from the literature are collected. The primary aim is a comprehensive overview about neutral benchmark studies, i.e. assessments which were performed independenty on publication of a new approach. Studies which are not neutral are put in brackets. </br> <br />
<br />
The focus is on computational methods for analyzing experimental data (instead of comparing experimental techniques or platforms). </br><br />
<br />
Please extend this list by creating a new page and adding a link below. </br> <br />
Use the '''[[Guidelines_for_Summarizing_a_Literature_Study|guidelines described here]]'''.<br />
|}<br />
<br />
== Results from Literature ==<br />
<br />
=== Classification ===<br />
''' 2003 '''</br><br />
* [[Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data]]<br />
''' 2005 '''</br><br />
* [[A review and comparison of classification algorithms for medical decision making]]<br />
''' 2016 '''</br><br />
* [[Predicting Breast Cancer Survivability Using Data Mining Techniques]]<br />
<br />
=== Selection of Differential Features and Regions ===<br />
==== Identifying differential features ====<br />
''' 2006 '''</br><br />
* [[Rat toxicogenomic study reveals analytical consistency across microarray platforms]]<br />
''' 2010 '''</br><br />
* [[A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the sequencing Quality control consortium]]<br />
''' 2017 '''</br><br />
* [[Identification of differentially expressed peptides in high-throughput proteomics data]]<br />
* [[In-depth method assessments of differentially expressed protein detection for shotgun proteomics data with missing values]]<br />
* [[Strategies for analyzing bisulfite sequencing data]]<br />
''' 2018 '''</br><br />
* [[Identification of Differentially Methylated Sites with Weak Methylation Effects]]<br />
<br />
==== Identifying differential regions (e.g. DMRs) ====<br />
''' 2015 '''<br />
* [[De novo identification of differentially methylated regions in the human genome]]<br />
* [[MethylAction: detecting differentially methylated regions that distinguish biological subtypes]]<br />
* [[metilene: Fast and sensitive calling of differentially methylated regions from bisulfite sequencing data]]<br />
''' 2016 '''<br />
* [[seqlm: an MDL based method for identifying differentially methylated regions in high density methylation array data]]<br />
* [[Statistical methods for detecting differentially methylated regions based on MethylCap-seq data]]<br />
''' 2017 '''<br />
* [[DMRfinder: efficiently identifying differentially methylated regions from MethylC-seq data]]<br />
''' 2018 '''<br />
* [[Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus]]<br />
* [[DMRcaller: a versatile R/Bioconductor package for detection and visualization of differentially methylated regions in CpG and non-CpG contexts]]<br />
* [[MethCP: Differentially Methylated Region Detection with Change Point Models (bioRxiv)]]<br />
<br />
==== Identifying sets of features (e.g. gene set analyses) ====<br />
'''2009<br />
<br />
* [[A general modular framework for gene set enrichment analysis]]<br />
* [[Comparing gene set analysis methods on single-nucleotide polymorphism data from Genetic Analysis Workshop 16]]<br />
<br />
'''2018<br />
<br />
* [[Gene set analysis methods: a systematic comparison]]<br />
<br />
'''2020<br />
* [[Toward a gold standard for benchmarking gene set enrichment analysis]]<br />
<br />
==== Dimension reduction ====<br />
''' 2008 '''</br><br />
* [[On the Relationship Between Feature Selection and Classification Accuracy]]<br />
''' 2015 '''</br><br />
* [[Comparing feature selection methods for highdimensional imbalanced data: identifying rheumatoid arthritis cohorts from routine data]]<br />
<br />
=== Imputation methods for missing values ===<br />
<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 1996 || Schenker || [[Partially parametric techniques for multiple imputation]]<br />
|-<br />
| 1999 || Hastie T || [[Imputing Missing Data for Gene Expression Arrays]]<br />
|-<br />
| 2001 || Troyanskaya || [[Missing value estimation methods for DNA microarrays]]<br />
|-<br />
| 2002 || Engels J || [[Imputation of missing longitudinal data: a comparison of methods]]<br />
|-<br />
| 2003 || Oba || [[A Bayesian missing value estimation method for gene expression profile data]]<br />
|-<br />
| 2005 || Scholz || [[Nonlinear PCA: a missing data approach]]<br />
|-<br />
| 2007 || Stacklies || [[pcaMethods—a bioconductor package providing PCA methods for incomplete data]]<br />
|-<br />
| 2007 || Verboven || [[Sequential imputation for missing values]]<br />
|-<br />
| 2008 || Shaffer GN || [[Which missing value imputation method to use in expression profiles: a comparative study and two selection schemes]]<br />
|-<br />
| 2011 || Templ || [[Iterative stepwise regression imputation using standard and robust methods]]<br />
|-<br />
| 2012 || Hrydziuszko O || [[Missing values in mass spectrometry based metabolomics: an undervalued step in the data processing pipeline]]<br />
|-<br />
| 2012 || Stekhoven || [[MissForest—non-parametric missing value imputation for mixed-type data]]<br />
|-<br />
| 2013 || Taylor || [[Accounting for undetected compounds in statistical analyses of mass spectrometry ‘omic studies]]<br />
|-<br />
| 2013 || Waljee || [[Comparison of imputation methods for missing laboratory data in medicine]]<br />
|-<br />
| 2014 || Shah || [[Comparison of Random Forest and Parametric Imputation Models for Imputing Missing Data Using MICE: A CALIBER Study]]<br />
|-<br />
| 2014 || Rodwell || [[Comparison of methods for imputing limited-range variables: a simulation study]]<br />
|-<br />
| 2014 || Morris || [[Tuning multiple imputation by predictive mean matching and local residual draws]]<br />
|-<br />
| 2014 || Doove L || [[Recursive partitioning for missing data imputation in the presence of interaction effects]]<br />
|-<br />
| 2015 || Webb-Robertson BJM || [[Review, evaluation, and discussion of the challenges of missing value imputation for mass spectrometry-based label-free global proteomics]]<br />
|-<br />
| 2016 || Folch-Fortuny A || [[Assessment of maximum likelihood PCA missing data imputation]]<br />
|-<br />
| 2016 || Lazar C || [[Accounting for the Multiple Natures of Missing Values in Label-Free Quantitative Proteomics Data Sets to Compare Imputation Strategies]]<br />
|-<br />
| 2016 || Yin X || [[Multiple imputation and analysis for high-dimensional incomplete proteomics data]]<br />
|-<br />
| 2018 || Wei R || [[Missing Value Imputation Approach for Mass Spectrometry-based Metabolomics Data]]<br />
|-<br />
| 2018 || Poyatos R || [[Gap-filling a spatially explicit plant trait database: comparing imputation methods and different levels of environmental information]]<br />
|-<br />
| 2018 || O'Brien JJ || [[The effects of nonignorable missing data on label-free mass spectrometry proteomics experiments]]<br />
|}<br />
<br />
=== ODE-based Modelling ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2001 || Beal || [[Ways to Fit a PK Model with Some Data Below the Quantification Limit]]<br />
|-<br />
| 2008 || Balsa-Canto || [[Hybrid optimization method with general switching strategy for parameter estimation]]<br />
|-<br />
| 2011 || Tashkova || [[Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis]]<br />
|-<br />
| 2013 || Raue || [[Lessons Learned from Quantitative Dynamical Modeling in Systems Biology]]<br />
|-<br />
| 2013 || Dondelinger || [[ODE parameter inference using adaptive gradient matching with Gaussian processes]]<br />
|-<br />
| 2017 || Ballnus || [[Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems]]<br />
|-<br />
| 2017 || Henriques || [[Data-driven reverse engineering of signaling pathways using ensembles of dynamic models]]<br />
|-<br />
| 2017 || Melicher || [[Fast derivatives of likelihood functionals for ODE based models using adjoint-state method]]<br />
|-<br />
| 2017 || Penas || [[Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy]]<br />
|-<br />
| 2017 || Degasperi || [[Performance of objective functions and optimization procedures for parameter estimation in system biology models]]<br />
|-<br />
| 2017 || Fröhlich || [[Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks]]<br />
|-<br />
| 2018 || Schälte || [[Evaluation of Derivative-Free Optimizers for Parameter Estimation in Systems Biology]]<br />
|-<br />
| 2018 || Loos || [[Hierarchical optimization for the efficient parametrization of ODE models]]<br />
|-<br />
| 2018 || Stapor || [[Optimization and profile calculation of ODE models using second order adjoint sensitivity analysis]]<br />
|-<br />
| 2019 || Villaverde || [[A comparison of methods for quantifying prediction uncertainty in systems biology]]<br />
|-<br />
| 2019 || Hass || [[Benchmark problems for dynamic modeling of intracellular processes]]<br />
|-<br />
| 2019 || Villaverde || [[Benchmarking optimization methods for parameter estimation in large kinetic models]]<br />
|-<br />
| 2019 || Lines || [[Efficient computation of steady states in large-scale ODE models of biochemical reaction networks]]<br />
|-<br />
| 2019 || Stapor || [[Mini-batch optimization enables training of ODE models on large-scale datasets]]<br />
|-<br />
| 2019 || Wu || [[Parameter Estimation and Variable Selection for Big Systems of Linear Ordinary Differential Equations: A Matrix-Based Approach]]<br />
|-<br />
| 2019 || Pitt || [[Parameter estimation in models of biological oscillators: an automated regularised estimation approach]]<br />
|-<br />
| 2019 || Loos || [[Robust calibration of hierarchical population models for heterogeneous cell populations]]<br />
|-<br />
| 2019 || Clairon || [[Tracking for parameter and state estimation in possibly misspecified partially observed linear Ordinary Differential Equations]]<br />
|-<br />
| 2020 || Schmiester || [[Efficient parameterization of large-scale dynamic models based on relative measurements]]<br />
|-<br />
| 2020 || Castro || [[Testing structural identifiability by a simple scaling method]]<br />
|}<br />
<br />
=== Omics Workflows ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2008 || Neuweger H || [[MeltDB: a software platform for the analysis and integration of metabolomics experiment data]]<br />
|-<br />
| 2008 || Barla A || [[Machine learning methods for predictive proteomics]]<br />
|-<br />
| 2009 || Xia J || [[MetaboAnalyst: a web server for metabolomic data analysis and interpretation]]<br />
|-<br />
| 2013 || Weisser H || [[An Automated Pipeline for High-Throughput Label-Free Quantitative Proteomics]]<br />
|-<br />
| 2014 || Cox J || [[Accurate Proteome-wide Label-free Quantification by Delayed Normalization and Maximal Peptide Ratio Extraction, Termed MaxLFQ* ]]<br />
|-<br />
| 2015 || || [[ComparingVariant Call Files for Performance Benchmarkingof Next-Generation Sequencing Variant Calling Pipelines]]<br />
|-<br />
| 2016 || Tyanova S || [[The MaxQuant computational platform for mass spectrometry–based shotgun proteomics]]<br />
|-<br />
| 2016 || Röst HL || [[OpenMS: a flexible open-source software platform for mass spectrometry data analysis]]<br />
|-<br />
| 2017 || || [[A benchmarking of workflows for detecting differential splicing and differential expression at isoform level in human RNA-seq studies]]<br />
|-<br />
| 2018 || Välikangas T || [[A comprehensive evaluation of popular proteomics software workflows for label-free proteome quantification and imputation]]<br />
|-<br />
| 2019 || || [[A Systematic Evaluation of Single CellRNA-Seq Analysis Pipelines]]<br />
|-<br />
| 2019 || || [[Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays]]<br />
|}<br />
<br />
=== Preprocessing high-throughput data===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|- 1999 || Perkins DN || [[Probability-based protein identification by searching sequence databases using mass spectrometry data]]<br />
|-<br />
| 2003 || || [[A comparison of normalization methods for high density oligonucleotide array data based on variance and bias ]]<br />
|-<br />
| 2005 || || [[Comparison of Affymetrix GeneChip Expression Measures]]<br />
|-<br />
| 2005 || Meleth S || [[The case for well-conducted experiments to validate statistical protocols for 2D gels: different pre-processing = different lists of significant proteins]]<br />
|-<br />
| 2005 || || [[Comparison of background correction and normalization procedures for high-density oligonucleotide microarrays]]<br />
|-<br />
| 2006 || || [[Using RNA sample titrations to assess microarray platform performance and normalization techniques]]<br />
|-<br />
| 2006 || Wang P || [[Normalization regarding non-random missing values in high-throughput mass spectrometry data]]<br />
|-<br />
| 2006 || Du P || [[Improved peak detection in mass spectrum by incorporating continuous wavelet transform-based pattern matching]]<br />
|-<br />
| 2007 || Carvalho B || [[Exploration, normalization, and genotype calls of high-density oligonucleotide SNP array data]]<br />
|-<br />
| 2007 || Cannataro M || [[MS‐Analyzer: preprocessing and data mining services for proteomics applications on the Grid]]<br />
|-<br />
| 2008 || || [[Comparison of preprocessing methods for the hgU133+2 chip from Affymetrix]]<br />
|-<br />
| 2009 || || [[Comparison of Affymetrix data normalization methods using 6,926 experiments across five array generations]]<br />
|-<br />
| 2009 || Mar JC || [[Data-driven normalization strategies for high-throughput quantitative RT-PCR]]<br />
|-<br />
| 2009 || Vakhrushev SY || [[Software platform for high-throughput glycomics]]<br />
|-<br />
| 2010 || || [[Consistency of predictive signature genes and classifiers generated using different microarray platforms]]<br />
|-<br />
| 2010 || || [[Detecting and correcting systematic variation in large-scale RNA sequencing data]]<br />
|-<br />
| 2010 || || [[Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments]]<br />
|-<br />
| 2010 || || [[Normalization of RNA-seq data using factor analysis of control genes or samples]]<br />
|-<br />
| 2010 || Armananzas R || [[Peakbin selection in mass spectrometry data using a consensus approach with estimation of distribution algorithms]]<br />
|-<br />
| 2011 || || [[Affymetrix GeneChip microarray preprocessing for multivariate analyses]]<br />
|-<br />
| 2012 || || [[A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis]]<br />
|-<br />
| 2013 || García-Torres M || [[Comparison of metaheuristic strategies for peakbin selection in proteomic mass spectrometry data]]<br />
|-<br />
| 2014 || || [[Normalyzer: A Tool for Rapid Evaluation of Normalization Methods for Omics Data Sets]]<br />
|-<br />
| 2014 || Zhou X || [[Prevention, diagnosis and treatment of high-throughput sequencing data pathologies]]<br />
|-<br />
| 2014 || Coble JB || [[Comparative evaluation of preprocessing freeware on chromatography/mass spectrometry data for signature discovery]]<br />
|-<br />
| 2014 || Aggio RB || [[Identifying and quantifying metabolites by scoring peaks of GC-MS data]]<br />
|-<br />
| 2015 || Caraus I || [[Detecting and overcoming systematic bias in high-throughput screening technologies: a comprehensive review of practical issues and methodological solutions]]<br />
|-<br />
| 2015 || Tam S || [[Optimization of miRNA-seq data preprocessing]]<br />
|-<br />
| 2015 || Rafiei A || [[Comparison of peak‐picking workflows for untargeted liquid chromatography/high‐resolution mass spectrometry metabolomics data analysis]]<br />
|-<br />
| 2015 || Chawade A || [[Data processing has major impact on the outcome of quantitative label-free LC-MS analysis]]<br />
|-<br />
| 2015 || Wang T || [[A systematic study of normalization methods for Infinium 450K methylation data using whole-genome bisulfite sequencing data]]<br />
|-<br />
| 2015 || Lu J || [[Improved Peak Detection and Deconvolution of Native Electrospray Mass Spectra from Large Protein Complexes]]<br />
|-<br />
| 2016 || Yi L || [[Chemometric methods in data processing of mass spectrometry-based metabolomics: A review]]<br />
|-<br />
| 2016 || Tsuji J || [[Evaluation of preprocessing, mapping and postprocessing algorithms for analyzing whole genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Li B || [[Performance Evaluation and Online Realization of Data-driven Normalization Methods Used in LC/MS based Untargeted Metabolomics Analysis]]<br />
|-<br />
| 2016 || Zheng Y || [[An improved algorithm for peak detection in mass spectra based on continuous wavelet transform]]<br />
|-<br />
| 2017 || Li B || [[NOREVA: normalization and evaluation of MS-based metabolomics data]]<br />
|-<br />
| 2018 || Mazoure B || [[Identification and Correction of Additive and Multiplicative Spatial Biases in Experimental High-Throughput Screening]]<br />
|-<br />
| 2018 || Li Z || [[Comprehensive evaluation of untargeted metabolomics data processing software in feature detection, quantification and discriminating marker selection]]<br />
|-<br />
| 2018 || Willforss J || [[NormalyzerDE: Online Tool for Improved Normalization of Omics Expression Data and High-Sensitivity Differential Expression Analysis]]<br />
|}</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Improved_Peak_Detection_and_Deconvolution_of_Native_Electrospray_Mass_Spectra_from_Large_Protein_Complexes&diff=723Improved Peak Detection and Deconvolution of Native Electrospray Mass Spectra from Large Protein Complexes2020-02-26T07:26:09Z<p>Bwday: Created page with "__NUMBEREDHEADINGS__ === Citation === Lu J, Trnka MJ, Roh SH, Robinson PJ, Shiau C, Fujimori DG, Chiu W, Burlingame AL, Guan S. Improved peak detection and deconvolution of na..."</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
=== Citation ===<br />
Lu J, Trnka MJ, Roh SH, Robinson PJ, Shiau C, Fujimori DG, Chiu W, Burlingame AL, Guan S. Improved peak detection and deconvolution of native electrospray mass spectra from large protein complexes. Journal of the American Society for Mass Spectrometry. 2015 Sep 1;26(12):2141-51.<br />
<br />
[https://doi.org/10.1021/jasms.8b04940 Permanent link to the paper]<br />
<br />
<br />
=== Summary ===<br />
Briefly describe the scope of the paper, i.e. the field of research and/or application.<br />
<br />
=== Study outcomes ===<br />
List the paper results concerning method comparison and benchmarking:<br />
==== Outcome O1 ====<br />
The performance of ...<br />
<br />
Outcome O1 is presented as Figure X in the original publication. <br />
<br />
==== Outcome O2 ====<br />
...<br />
<br />
Outcome O2 is presented as Figure X in the original publication. <br />
<br />
==== Outcome On ====<br />
...<br />
<br />
Outcome On is presented as Figure X in the original publication. <br />
<br />
==== Further outcomes ====<br />
If intended, you can add further outcomes here.<br />
<br />
<br />
=== Study design and evidence level ===<br />
==== General aspects ====<br />
You can describe general design aspects here.<br />
The study designs for describing specific outcomes are listed in the following subsections:<br />
<br />
==== Design for Outcome O1 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
==== Design for Outcome O2 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
... <br />
<br />
==== Design for Outcome O ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
=== Further comments and aspects ===<br />
<br />
=== References ===<br />
The list of cited or related literature is placed here.</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Literature_Studies&diff=722Literature Studies2020-02-26T07:24:57Z<p>Bwday: /* Preprocessing high-throughput data */</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
{| class="wikitable"<br />
|-<br />
! Page summary<br />
|-<br />
| Here outcomes of benchmarking studies from the literature are collected. The primary aim is a comprehensive overview about neutral benchmark studies, i.e. assessments which were performed independenty on publication of a new approach. Studies which are not neutral are put in brackets. </br> <br />
<br />
The focus is on computational methods for analyzing experimental data (instead of comparing experimental techniques or platforms). </br><br />
<br />
Please extend this list by creating a new page and adding a link below. </br> <br />
Use the '''[[Guidelines_for_Summarizing_a_Literature_Study|guidelines described here]]'''.<br />
|}<br />
<br />
== Results from Literature ==<br />
<br />
=== Classification ===<br />
''' 2003 '''</br><br />
* [[Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data]]<br />
''' 2005 '''</br><br />
* [[A review and comparison of classification algorithms for medical decision making]]<br />
''' 2016 '''</br><br />
* [[Predicting Breast Cancer Survivability Using Data Mining Techniques]]<br />
<br />
=== Selection of Differential Features and Regions ===<br />
==== Identifying differential features ====<br />
''' 2006 '''</br><br />
* [[Rat toxicogenomic study reveals analytical consistency across microarray platforms]]<br />
''' 2010 '''</br><br />
* [[A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the sequencing Quality control consortium]]<br />
''' 2017 '''</br><br />
* [[Identification of differentially expressed peptides in high-throughput proteomics data]]<br />
* [[In-depth method assessments of differentially expressed protein detection for shotgun proteomics data with missing values]]<br />
* [[Strategies for analyzing bisulfite sequencing data]]<br />
''' 2018 '''</br><br />
* [[Identification of Differentially Methylated Sites with Weak Methylation Effects]]<br />
<br />
==== Identifying differential regions (e.g. DMRs) ====<br />
''' 2015 '''<br />
* [[De novo identification of differentially methylated regions in the human genome]]<br />
* [[MethylAction: detecting differentially methylated regions that distinguish biological subtypes]]<br />
* [[metilene: Fast and sensitive calling of differentially methylated regions from bisulfite sequencing data]]<br />
''' 2016 '''<br />
* [[seqlm: an MDL based method for identifying differentially methylated regions in high density methylation array data]]<br />
* [[Statistical methods for detecting differentially methylated regions based on MethylCap-seq data]]<br />
''' 2017 '''<br />
* [[DMRfinder: efficiently identifying differentially methylated regions from MethylC-seq data]]<br />
''' 2018 '''<br />
* [[Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus]]<br />
* [[DMRcaller: a versatile R/Bioconductor package for detection and visualization of differentially methylated regions in CpG and non-CpG contexts]]<br />
* [[MethCP: Differentially Methylated Region Detection with Change Point Models (bioRxiv)]]<br />
<br />
==== Identifying sets of features (e.g. gene set analyses) ====<br />
'''2009<br />
<br />
* [[A general modular framework for gene set enrichment analysis]]<br />
* [[Comparing gene set analysis methods on single-nucleotide polymorphism data from Genetic Analysis Workshop 16]]<br />
<br />
'''2018<br />
<br />
* [[Gene set analysis methods: a systematic comparison]]<br />
<br />
'''2020<br />
* [[Toward a gold standard for benchmarking gene set enrichment analysis]]<br />
<br />
==== Dimension reduction ====<br />
''' 2008 '''</br><br />
* [[On the Relationship Between Feature Selection and Classification Accuracy]]<br />
''' 2015 '''</br><br />
* [[Comparing feature selection methods for highdimensional imbalanced data: identifying rheumatoid arthritis cohorts from routine data]]<br />
<br />
=== Imputation methods for missing values ===<br />
<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 1996 || Schenker || [[Partially parametric techniques for multiple imputation]]<br />
|-<br />
| 1999 || Hastie T || [[Imputing Missing Data for Gene Expression Arrays]]<br />
|-<br />
| 2001 || Troyanskaya || [[Missing value estimation methods for DNA microarrays]]<br />
|-<br />
| 2002 || Engels J || [[Imputation of missing longitudinal data: a comparison of methods]]<br />
|-<br />
| 2003 || Oba || [[A Bayesian missing value estimation method for gene expression profile data]]<br />
|-<br />
| 2005 || Scholz || [[Nonlinear PCA: a missing data approach]]<br />
|-<br />
| 2007 || Stacklies || [[pcaMethods—a bioconductor package providing PCA methods for incomplete data]]<br />
|-<br />
| 2007 || Verboven || [[Sequential imputation for missing values]]<br />
|-<br />
| 2008 || Shaffer GN || [[Which missing value imputation method to use in expression profiles: a comparative study and two selection schemes]]<br />
|-<br />
| 2011 || Templ || [[Iterative stepwise regression imputation using standard and robust methods]]<br />
|-<br />
| 2012 || Hrydziuszko O || [[Missing values in mass spectrometry based metabolomics: an undervalued step in the data processing pipeline]]<br />
|-<br />
| 2012 || Stekhoven || [[MissForest—non-parametric missing value imputation for mixed-type data]]<br />
|-<br />
| 2013 || Taylor || [[Accounting for undetected compounds in statistical analyses of mass spectrometry ‘omic studies]]<br />
|-<br />
| 2013 || Waljee || [[Comparison of imputation methods for missing laboratory data in medicine]]<br />
|-<br />
| 2014 || Shah || [[Comparison of Random Forest and Parametric Imputation Models for Imputing Missing Data Using MICE: A CALIBER Study]]<br />
|-<br />
| 2014 || Rodwell || [[Comparison of methods for imputing limited-range variables: a simulation study]]<br />
|-<br />
| 2014 || Morris || [[Tuning multiple imputation by predictive mean matching and local residual draws]]<br />
|-<br />
| 2014 || Doove L || [[Recursive partitioning for missing data imputation in the presence of interaction effects]]<br />
|-<br />
| 2015 || Webb-Robertson BJM || [[Review, evaluation, and discussion of the challenges of missing value imputation for mass spectrometry-based label-free global proteomics]]<br />
|-<br />
| 2016 || Folch-Fortuny A || [[Assessment of maximum likelihood PCA missing data imputation]]<br />
|-<br />
| 2016 || Lazar C || [[Accounting for the Multiple Natures of Missing Values in Label-Free Quantitative Proteomics Data Sets to Compare Imputation Strategies]]<br />
|-<br />
| 2016 || Yin X || [[Multiple imputation and analysis for high-dimensional incomplete proteomics data]]<br />
|-<br />
| 2018 || Wei R || [[Missing Value Imputation Approach for Mass Spectrometry-based Metabolomics Data]]<br />
|-<br />
| 2018 || Poyatos R || [[Gap-filling a spatially explicit plant trait database: comparing imputation methods and different levels of environmental information]]<br />
|-<br />
| 2018 || O'Brien JJ || [[The effects of nonignorable missing data on label-free mass spectrometry proteomics experiments]]<br />
|}<br />
<br />
=== ODE-based Modelling ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2001 || Beal || [[Ways to Fit a PK Model with Some Data Below the Quantification Limit]]<br />
|-<br />
| 2008 || Balsa-Canto || [[Hybrid optimization method with general switching strategy for parameter estimation]]<br />
|-<br />
| 2011 || Tashkova || [[Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis]]<br />
|-<br />
| 2013 || Raue || [[Lessons Learned from Quantitative Dynamical Modeling in Systems Biology]]<br />
|-<br />
| 2013 || Dondelinger || [[ODE parameter inference using adaptive gradient matching with Gaussian processes]]<br />
|-<br />
| 2017 || Ballnus || [[Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems]]<br />
|-<br />
| 2017 || Henriques || [[Data-driven reverse engineering of signaling pathways using ensembles of dynamic models]]<br />
|-<br />
| 2017 || Melicher || [[Fast derivatives of likelihood functionals for ODE based models using adjoint-state method]]<br />
|-<br />
| 2017 || Penas || [[Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy]]<br />
|-<br />
| 2017 || Degasperi || [[Performance of objective functions and optimization procedures for parameter estimation in system biology models]]<br />
|-<br />
| 2017 || Fröhlich || [[Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks]]<br />
|-<br />
| 2018 || Schälte || [[Evaluation of Derivative-Free Optimizers for Parameter Estimation in Systems Biology]]<br />
|-<br />
| 2018 || Loos || [[Hierarchical optimization for the efficient parametrization of ODE models]]<br />
|-<br />
| 2018 || Stapor || [[Optimization and profile calculation of ODE models using second order adjoint sensitivity analysis]]<br />
|-<br />
| 2019 || Villaverde || [[A comparison of methods for quantifying prediction uncertainty in systems biology]]<br />
|-<br />
| 2019 || Hass || [[Benchmark problems for dynamic modeling of intracellular processes]]<br />
|-<br />
| 2019 || Villaverde || [[Benchmarking optimization methods for parameter estimation in large kinetic models]]<br />
|-<br />
| 2019 || Lines || [[Efficient computation of steady states in large-scale ODE models of biochemical reaction networks]]<br />
|-<br />
| 2019 || Stapor || [[Mini-batch optimization enables training of ODE models on large-scale datasets]]<br />
|-<br />
| 2019 || Wu || [[Parameter Estimation and Variable Selection for Big Systems of Linear Ordinary Differential Equations: A Matrix-Based Approach]]<br />
|-<br />
| 2019 || Pitt || [[Parameter estimation in models of biological oscillators: an automated regularised estimation approach]]<br />
|-<br />
| 2019 || Loos || [[Robust calibration of hierarchical population models for heterogeneous cell populations]]<br />
|-<br />
| 2019 || Clairon || [[Tracking for parameter and state estimation in possibly misspecified partially observed linear Ordinary Differential Equations]]<br />
|-<br />
| 2020 || Schmiester || [[Efficient parameterization of large-scale dynamic models based on relative measurements]]<br />
|-<br />
| 2020 || Castro || [[Testing structural identifiability by a simple scaling method]]<br />
|}<br />
<br />
=== Omics Workflows ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2008 || Neuweger H || [[MeltDB: a software platform for the analysis and integration of metabolomics experiment data]]<br />
|-<br />
| 2008 || Barla A || [[Machine learning methods for predictive proteomics]]<br />
|-<br />
| 2009 || Xia J || [[MetaboAnalyst: a web server for metabolomic data analysis and interpretation]]<br />
|-<br />
| 2013 || Weisser H || [[An Automated Pipeline for High-Throughput Label-Free Quantitative Proteomics]]<br />
|-<br />
| 2014 || Cox J || [[Accurate Proteome-wide Label-free Quantification by Delayed Normalization and Maximal Peptide Ratio Extraction, Termed MaxLFQ* ]]<br />
|-<br />
| 2015 || || [[ComparingVariant Call Files for Performance Benchmarkingof Next-Generation Sequencing Variant Calling Pipelines]]<br />
|-<br />
| 2016 || Tyanova S || [[The MaxQuant computational platform for mass spectrometry–based shotgun proteomics]]<br />
|-<br />
| 2016 || Röst HL || [[OpenMS: a flexible open-source software platform for mass spectrometry data analysis]]<br />
|-<br />
| 2017 || || [[A benchmarking of workflows for detecting differential splicing and differential expression at isoform level in human RNA-seq studies]]<br />
|-<br />
| 2018 || Välikangas T || [[A comprehensive evaluation of popular proteomics software workflows for label-free proteome quantification and imputation]]<br />
|-<br />
| 2019 || || [[A Systematic Evaluation of Single CellRNA-Seq Analysis Pipelines]]<br />
|-<br />
| 2019 || || [[Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays]]<br />
|}<br />
<br />
=== Preprocessing high-throughput data===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|- 1999 || Perkins DN || [[Probability-based protein identification by searching sequence databases using mass spectrometry data]]<br />
|-<br />
| 2003 || || [[A comparison of normalization methods for high density oligonucleotide array data based on variance and bias ]]<br />
|-<br />
| 2005 || || [[Comparison of Affymetrix GeneChip Expression Measures]]<br />
|-<br />
| 2005 || Meleth S || [[The case for well-conducted experiments to validate statistical protocols for 2D gels: different pre-processing = different lists of significant proteins]]<br />
|-<br />
| 2005 || || [[Comparison of background correction and normalization procedures for high-density oligonucleotide microarrays]]<br />
|-<br />
| 2006 || || [[Using RNA sample titrations to assess microarray platform performance and normalization techniques]]<br />
|-<br />
| 2006 || Wang P || [[Normalization regarding non-random missing values in high-throughput mass spectrometry data]]<br />
|-<br />
| 2006 || Du P || [[Improved peak detection in mass spectrum by incorporating continuous wavelet transform-based pattern matching]]<br />
|-<br />
| 2007 || Carvalho B || [[Exploration, normalization, and genotype calls of high-density oligonucleotide SNP array data]]<br />
|-<br />
| 2007 || Cannataro M || [[MS‐Analyzer: preprocessing and data mining services for proteomics applications on the Grid]]<br />
|-<br />
| 2008 || || [[Comparison of preprocessing methods for the hgU133+2 chip from Affymetrix]]<br />
|-<br />
| 2009 || || [[Comparison of Affymetrix data normalization methods using 6,926 experiments across five array generations]]<br />
|-<br />
| 2009 || Mar JC || [[Data-driven normalization strategies for high-throughput quantitative RT-PCR]]<br />
|-<br />
| 2009 || Vakhrushev SY || [[Software platform for high-throughput glycomics]]<br />
|-<br />
| 2010 || || [[Consistency of predictive signature genes and classifiers generated using different microarray platforms]]<br />
|-<br />
| 2010 || || [[Detecting and correcting systematic variation in large-scale RNA sequencing data]]<br />
|-<br />
| 2010 || || [[Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments]]<br />
|-<br />
| 2010 || || [[Normalization of RNA-seq data using factor analysis of control genes or samples]]<br />
|-<br />
| 2010 || Armananzas R || [[Peakbin selection in mass spectrometry data using a consensus approach with estimation of distribution algorithms]]<br />
|-<br />
| 2011 || || [[Affymetrix GeneChip microarray preprocessing for multivariate analyses]]<br />
|-<br />
| 2012 || || [[A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis]]<br />
|-<br />
| 2013 || García-Torres M || [[Comparison of metaheuristic strategies for peakbin selection in proteomic mass spectrometry data]]<br />
|-<br />
| 2014 || || [[Normalyzer: A Tool for Rapid Evaluation of Normalization Methods for Omics Data Sets]]<br />
|-<br />
| 2014 || Zhou X || [[Prevention, diagnosis and treatment of high-throughput sequencing data pathologies]]<br />
|-<br />
| 2014 || Coble JB || [[Comparative evaluation of preprocessing freeware on chromatography/mass spectrometry data for signature discovery]]<br />
|-<br />
| 2015 || Caraus I || [[Detecting and overcoming systematic bias in high-throughput screening technologies: a comprehensive review of practical issues and methodological solutions]]<br />
|-<br />
| 2015 || Tam S || [[Optimization of miRNA-seq data preprocessing]]<br />
|-<br />
| 2015 || Rafiei A || [[Comparison of peak‐picking workflows for untargeted liquid chromatography/high‐resolution mass spectrometry metabolomics data analysis]]<br />
|-<br />
| 2015 || Chawade A || [[Data processing has major impact on the outcome of quantitative label-free LC-MS analysis]]<br />
|-<br />
| 2015 || Wang T || [[A systematic study of normalization methods for Infinium 450K methylation data using whole-genome bisulfite sequencing data]]<br />
|-<br />
| 2015 || Lu J || [[Improved Peak Detection and Deconvolution of Native Electrospray Mass Spectra from Large Protein Complexes]]<br />
|-<br />
| 2016 || Yi L || [[Chemometric methods in data processing of mass spectrometry-based metabolomics: A review]]<br />
|-<br />
| 2016 || Tsuji J || [[Evaluation of preprocessing, mapping and postprocessing algorithms for analyzing whole genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Li B || [[Performance Evaluation and Online Realization of Data-driven Normalization Methods Used in LC/MS based Untargeted Metabolomics Analysis]]<br />
|-<br />
| 2016 || Zheng Y || [[An improved algorithm for peak detection in mass spectra based on continuous wavelet transform]]<br />
|-<br />
| 2017 || Li B || [[NOREVA: normalization and evaluation of MS-based metabolomics data]]<br />
|-<br />
| 2018 || Mazoure B || [[Identification and Correction of Additive and Multiplicative Spatial Biases in Experimental High-Throughput Screening]]<br />
|-<br />
| 2018 || Li Z || [[Comprehensive evaluation of untargeted metabolomics data processing software in feature detection, quantification and discriminating marker selection]]<br />
|-<br />
| 2018 || Willforss J || [[NormalyzerDE: Online Tool for Improved Normalization of Omics Expression Data and High-Sensitivity Differential Expression Analysis]]<br />
|}</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Improved_peak_detection_in_mass_spectrum_by_incorporating_continuous_wavelet_transform-based_pattern_matching&diff=721Improved peak detection in mass spectrum by incorporating continuous wavelet transform-based pattern matching2020-02-26T07:15:20Z<p>Bwday: Created page with "__NUMBEREDHEADINGS__ === Citation === Du P, Kibbe WA, Lin SM. Improved peak detection in mass spectrum by incorporating continuous wavelet transform-based pattern matching. Bi..."</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
=== Citation ===<br />
Du P, Kibbe WA, Lin SM. Improved peak detection in mass spectrum by incorporating continuous wavelet transform-based pattern matching. Bioinformatics. 2006 Sep 1;22(17):2059-65.<br />
<br />
[https://doi.org/10.1093/bioinformatics/btl355 Permanent link to the paper]<br />
<br />
<br />
=== Summary ===<br />
Briefly describe the scope of the paper, i.e. the field of research and/or application.<br />
<br />
=== Study outcomes ===<br />
List the paper results concerning method comparison and benchmarking:<br />
==== Outcome O1 ====<br />
The performance of ...<br />
<br />
Outcome O1 is presented as Figure X in the original publication. <br />
<br />
==== Outcome O2 ====<br />
...<br />
<br />
Outcome O2 is presented as Figure X in the original publication. <br />
<br />
==== Outcome On ====<br />
...<br />
<br />
Outcome On is presented as Figure X in the original publication. <br />
<br />
==== Further outcomes ====<br />
If intended, you can add further outcomes here.<br />
<br />
<br />
=== Study design and evidence level ===<br />
==== General aspects ====<br />
You can describe general design aspects here.<br />
The study designs for describing specific outcomes are listed in the following subsections:<br />
<br />
==== Design for Outcome O1 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
==== Design for Outcome O2 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
... <br />
<br />
==== Design for Outcome O ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
=== Further comments and aspects ===<br />
<br />
=== References ===<br />
The list of cited or related literature is placed here.</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Literature_Studies&diff=720Literature Studies2020-02-26T07:14:39Z<p>Bwday: /* Preprocessing high-throughput data */</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
{| class="wikitable"<br />
|-<br />
! Page summary<br />
|-<br />
| Here outcomes of benchmarking studies from the literature are collected. The primary aim is a comprehensive overview about neutral benchmark studies, i.e. assessments which were performed independenty on publication of a new approach. Studies which are not neutral are put in brackets. </br> <br />
<br />
The focus is on computational methods for analyzing experimental data (instead of comparing experimental techniques or platforms). </br><br />
<br />
Please extend this list by creating a new page and adding a link below. </br> <br />
Use the '''[[Guidelines_for_Summarizing_a_Literature_Study|guidelines described here]]'''.<br />
|}<br />
<br />
== Results from Literature ==<br />
<br />
=== Classification ===<br />
''' 2003 '''</br><br />
* [[Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data]]<br />
''' 2005 '''</br><br />
* [[A review and comparison of classification algorithms for medical decision making]]<br />
''' 2016 '''</br><br />
* [[Predicting Breast Cancer Survivability Using Data Mining Techniques]]<br />
<br />
=== Selection of Differential Features and Regions ===<br />
==== Identifying differential features ====<br />
''' 2006 '''</br><br />
* [[Rat toxicogenomic study reveals analytical consistency across microarray platforms]]<br />
''' 2010 '''</br><br />
* [[A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the sequencing Quality control consortium]]<br />
''' 2017 '''</br><br />
* [[Identification of differentially expressed peptides in high-throughput proteomics data]]<br />
* [[In-depth method assessments of differentially expressed protein detection for shotgun proteomics data with missing values]]<br />
* [[Strategies for analyzing bisulfite sequencing data]]<br />
''' 2018 '''</br><br />
* [[Identification of Differentially Methylated Sites with Weak Methylation Effects]]<br />
<br />
==== Identifying differential regions (e.g. DMRs) ====<br />
''' 2015 '''<br />
* [[De novo identification of differentially methylated regions in the human genome]]<br />
* [[MethylAction: detecting differentially methylated regions that distinguish biological subtypes]]<br />
* [[metilene: Fast and sensitive calling of differentially methylated regions from bisulfite sequencing data]]<br />
''' 2016 '''<br />
* [[seqlm: an MDL based method for identifying differentially methylated regions in high density methylation array data]]<br />
* [[Statistical methods for detecting differentially methylated regions based on MethylCap-seq data]]<br />
''' 2017 '''<br />
* [[DMRfinder: efficiently identifying differentially methylated regions from MethylC-seq data]]<br />
''' 2018 '''<br />
* [[Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus]]<br />
* [[DMRcaller: a versatile R/Bioconductor package for detection and visualization of differentially methylated regions in CpG and non-CpG contexts]]<br />
* [[MethCP: Differentially Methylated Region Detection with Change Point Models (bioRxiv)]]<br />
<br />
==== Identifying sets of features (e.g. gene set analyses) ====<br />
'''2009<br />
<br />
* [[A general modular framework for gene set enrichment analysis]]<br />
* [[Comparing gene set analysis methods on single-nucleotide polymorphism data from Genetic Analysis Workshop 16]]<br />
<br />
'''2018<br />
<br />
* [[Gene set analysis methods: a systematic comparison]]<br />
<br />
'''2020<br />
* [[Toward a gold standard for benchmarking gene set enrichment analysis]]<br />
<br />
==== Dimension reduction ====<br />
''' 2008 '''</br><br />
* [[On the Relationship Between Feature Selection and Classification Accuracy]]<br />
''' 2015 '''</br><br />
* [[Comparing feature selection methods for highdimensional imbalanced data: identifying rheumatoid arthritis cohorts from routine data]]<br />
<br />
=== Imputation methods for missing values ===<br />
<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 1996 || Schenker || [[Partially parametric techniques for multiple imputation]]<br />
|-<br />
| 1999 || Hastie T || [[Imputing Missing Data for Gene Expression Arrays]]<br />
|-<br />
| 2001 || Troyanskaya || [[Missing value estimation methods for DNA microarrays]]<br />
|-<br />
| 2002 || Engels J || [[Imputation of missing longitudinal data: a comparison of methods]]<br />
|-<br />
| 2003 || Oba || [[A Bayesian missing value estimation method for gene expression profile data]]<br />
|-<br />
| 2005 || Scholz || [[Nonlinear PCA: a missing data approach]]<br />
|-<br />
| 2007 || Stacklies || [[pcaMethods—a bioconductor package providing PCA methods for incomplete data]]<br />
|-<br />
| 2007 || Verboven || [[Sequential imputation for missing values]]<br />
|-<br />
| 2008 || Shaffer GN || [[Which missing value imputation method to use in expression profiles: a comparative study and two selection schemes]]<br />
|-<br />
| 2011 || Templ || [[Iterative stepwise regression imputation using standard and robust methods]]<br />
|-<br />
| 2012 || Hrydziuszko O || [[Missing values in mass spectrometry based metabolomics: an undervalued step in the data processing pipeline]]<br />
|-<br />
| 2012 || Stekhoven || [[MissForest—non-parametric missing value imputation for mixed-type data]]<br />
|-<br />
| 2013 || Taylor || [[Accounting for undetected compounds in statistical analyses of mass spectrometry ‘omic studies]]<br />
|-<br />
| 2013 || Waljee || [[Comparison of imputation methods for missing laboratory data in medicine]]<br />
|-<br />
| 2014 || Shah || [[Comparison of Random Forest and Parametric Imputation Models for Imputing Missing Data Using MICE: A CALIBER Study]]<br />
|-<br />
| 2014 || Rodwell || [[Comparison of methods for imputing limited-range variables: a simulation study]]<br />
|-<br />
| 2014 || Morris || [[Tuning multiple imputation by predictive mean matching and local residual draws]]<br />
|-<br />
| 2014 || Doove L || [[Recursive partitioning for missing data imputation in the presence of interaction effects]]<br />
|-<br />
| 2015 || Webb-Robertson BJM || [[Review, evaluation, and discussion of the challenges of missing value imputation for mass spectrometry-based label-free global proteomics]]<br />
|-<br />
| 2016 || Folch-Fortuny A || [[Assessment of maximum likelihood PCA missing data imputation]]<br />
|-<br />
| 2016 || Lazar C || [[Accounting for the Multiple Natures of Missing Values in Label-Free Quantitative Proteomics Data Sets to Compare Imputation Strategies]]<br />
|-<br />
| 2016 || Yin X || [[Multiple imputation and analysis for high-dimensional incomplete proteomics data]]<br />
|-<br />
| 2018 || Wei R || [[Missing Value Imputation Approach for Mass Spectrometry-based Metabolomics Data]]<br />
|-<br />
| 2018 || Poyatos R || [[Gap-filling a spatially explicit plant trait database: comparing imputation methods and different levels of environmental information]]<br />
|-<br />
| 2018 || O'Brien JJ || [[The effects of nonignorable missing data on label-free mass spectrometry proteomics experiments]]<br />
|}<br />
<br />
=== ODE-based Modelling ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2001 || Beal || [[Ways to Fit a PK Model with Some Data Below the Quantification Limit]]<br />
|-<br />
| 2008 || Balsa-Canto || [[Hybrid optimization method with general switching strategy for parameter estimation]]<br />
|-<br />
| 2011 || Tashkova || [[Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis]]<br />
|-<br />
| 2013 || Raue || [[Lessons Learned from Quantitative Dynamical Modeling in Systems Biology]]<br />
|-<br />
| 2013 || Dondelinger || [[ODE parameter inference using adaptive gradient matching with Gaussian processes]]<br />
|-<br />
| 2017 || Ballnus || [[Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems]]<br />
|-<br />
| 2017 || Henriques || [[Data-driven reverse engineering of signaling pathways using ensembles of dynamic models]]<br />
|-<br />
| 2017 || Melicher || [[Fast derivatives of likelihood functionals for ODE based models using adjoint-state method]]<br />
|-<br />
| 2017 || Penas || [[Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy]]<br />
|-<br />
| 2017 || Degasperi || [[Performance of objective functions and optimization procedures for parameter estimation in system biology models]]<br />
|-<br />
| 2017 || Fröhlich || [[Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks]]<br />
|-<br />
| 2018 || Schälte || [[Evaluation of Derivative-Free Optimizers for Parameter Estimation in Systems Biology]]<br />
|-<br />
| 2018 || Loos || [[Hierarchical optimization for the efficient parametrization of ODE models]]<br />
|-<br />
| 2018 || Stapor || [[Optimization and profile calculation of ODE models using second order adjoint sensitivity analysis]]<br />
|-<br />
| 2019 || Villaverde || [[A comparison of methods for quantifying prediction uncertainty in systems biology]]<br />
|-<br />
| 2019 || Hass || [[Benchmark problems for dynamic modeling of intracellular processes]]<br />
|-<br />
| 2019 || Villaverde || [[Benchmarking optimization methods for parameter estimation in large kinetic models]]<br />
|-<br />
| 2019 || Lines || [[Efficient computation of steady states in large-scale ODE models of biochemical reaction networks]]<br />
|-<br />
| 2019 || Stapor || [[Mini-batch optimization enables training of ODE models on large-scale datasets]]<br />
|-<br />
| 2019 || Wu || [[Parameter Estimation and Variable Selection for Big Systems of Linear Ordinary Differential Equations: A Matrix-Based Approach]]<br />
|-<br />
| 2019 || Pitt || [[Parameter estimation in models of biological oscillators: an automated regularised estimation approach]]<br />
|-<br />
| 2019 || Loos || [[Robust calibration of hierarchical population models for heterogeneous cell populations]]<br />
|-<br />
| 2019 || Clairon || [[Tracking for parameter and state estimation in possibly misspecified partially observed linear Ordinary Differential Equations]]<br />
|-<br />
| 2020 || Schmiester || [[Efficient parameterization of large-scale dynamic models based on relative measurements]]<br />
|-<br />
| 2020 || Castro || [[Testing structural identifiability by a simple scaling method]]<br />
|}<br />
<br />
=== Omics Workflows ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2008 || Neuweger H || [[MeltDB: a software platform for the analysis and integration of metabolomics experiment data]]<br />
|-<br />
| 2008 || Barla A || [[Machine learning methods for predictive proteomics]]<br />
|-<br />
| 2009 || Xia J || [[MetaboAnalyst: a web server for metabolomic data analysis and interpretation]]<br />
|-<br />
| 2013 || Weisser H || [[An Automated Pipeline for High-Throughput Label-Free Quantitative Proteomics]]<br />
|-<br />
| 2014 || Cox J || [[Accurate Proteome-wide Label-free Quantification by Delayed Normalization and Maximal Peptide Ratio Extraction, Termed MaxLFQ* ]]<br />
|-<br />
| 2015 || || [[ComparingVariant Call Files for Performance Benchmarkingof Next-Generation Sequencing Variant Calling Pipelines]]<br />
|-<br />
| 2016 || Tyanova S || [[The MaxQuant computational platform for mass spectrometry–based shotgun proteomics]]<br />
|-<br />
| 2016 || Röst HL || [[OpenMS: a flexible open-source software platform for mass spectrometry data analysis]]<br />
|-<br />
| 2017 || || [[A benchmarking of workflows for detecting differential splicing and differential expression at isoform level in human RNA-seq studies]]<br />
|-<br />
| 2018 || Välikangas T || [[A comprehensive evaluation of popular proteomics software workflows for label-free proteome quantification and imputation]]<br />
|-<br />
| 2019 || || [[A Systematic Evaluation of Single CellRNA-Seq Analysis Pipelines]]<br />
|-<br />
| 2019 || || [[Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays]]<br />
|}<br />
<br />
=== Preprocessing high-throughput data===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|- 1999 || Perkins DN || [[Probability-based protein identification by searching sequence databases using mass spectrometry data]]<br />
|-<br />
| 2003 || || [[A comparison of normalization methods for high density oligonucleotide array data based on variance and bias ]]<br />
|-<br />
| 2005 || || [[Comparison of Affymetrix GeneChip Expression Measures]]<br />
|-<br />
| 2005 || Meleth S || [[The case for well-conducted experiments to validate statistical protocols for 2D gels: different pre-processing = different lists of significant proteins]]<br />
|-<br />
| 2005 || || [[Comparison of background correction and normalization procedures for high-density oligonucleotide microarrays]]<br />
|-<br />
| 2006 || || [[Using RNA sample titrations to assess microarray platform performance and normalization techniques]]<br />
|-<br />
| 2006 || Wang P || [[Normalization regarding non-random missing values in high-throughput mass spectrometry data]]<br />
|-<br />
| 2006 || Du P || [[Improved peak detection in mass spectrum by incorporating continuous wavelet transform-based pattern matching]]<br />
|-<br />
| 2007 || Carvalho B || [[Exploration, normalization, and genotype calls of high-density oligonucleotide SNP array data]]<br />
|-<br />
| 2007 || Cannataro M || [[MS‐Analyzer: preprocessing and data mining services for proteomics applications on the Grid]]<br />
|-<br />
| 2008 || || [[Comparison of preprocessing methods for the hgU133+2 chip from Affymetrix]]<br />
|-<br />
| 2009 || || [[Comparison of Affymetrix data normalization methods using 6,926 experiments across five array generations]]<br />
|-<br />
| 2009 || Mar JC || [[Data-driven normalization strategies for high-throughput quantitative RT-PCR]]<br />
|-<br />
| 2009 || Vakhrushev SY || [[Software platform for high-throughput glycomics]]<br />
|-<br />
| 2010 || || [[Consistency of predictive signature genes and classifiers generated using different microarray platforms]]<br />
|-<br />
| 2010 || || [[Detecting and correcting systematic variation in large-scale RNA sequencing data]]<br />
|-<br />
| 2010 || || [[Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments]]<br />
|-<br />
| 2010 || || [[Normalization of RNA-seq data using factor analysis of control genes or samples]]<br />
|-<br />
| 2010 || Armananzas R || [[Peakbin selection in mass spectrometry data using a consensus approach with estimation of distribution algorithms]]<br />
|-<br />
| 2011 || || [[Affymetrix GeneChip microarray preprocessing for multivariate analyses]]<br />
|-<br />
| 2012 || || [[A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis]]<br />
|-<br />
| 2013 || García-Torres M || [[Comparison of metaheuristic strategies for peakbin selection in proteomic mass spectrometry data]]<br />
|-<br />
| 2014 || || [[Normalyzer: A Tool for Rapid Evaluation of Normalization Methods for Omics Data Sets]]<br />
|-<br />
| 2014 || Zhou X || [[Prevention, diagnosis and treatment of high-throughput sequencing data pathologies]]<br />
|-<br />
| 2014 || Coble JB || [[Comparative evaluation of preprocessing freeware on chromatography/mass spectrometry data for signature discovery]]<br />
|-<br />
| 2015 || Caraus I || [[Detecting and overcoming systematic bias in high-throughput screening technologies: a comprehensive review of practical issues and methodological solutions]]<br />
|-<br />
| 2015 || Tam S || [[Optimization of miRNA-seq data preprocessing]]<br />
|-<br />
| 2015 || Rafiei A || [[Comparison of peak‐picking workflows for untargeted liquid chromatography/high‐resolution mass spectrometry metabolomics data analysis]]<br />
|-<br />
| 2015 || Chawade A || [[Data processing has major impact on the outcome of quantitative label-free LC-MS analysis]]<br />
|-<br />
| 2015 || Wang T || [[A systematic study of normalization methods for Infinium 450K methylation data using whole-genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Yi L || [[Chemometric methods in data processing of mass spectrometry-based metabolomics: A review]]<br />
|-<br />
| 2016 || Tsuji J || [[Evaluation of preprocessing, mapping and postprocessing algorithms for analyzing whole genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Li B || [[Performance Evaluation and Online Realization of Data-driven Normalization Methods Used in LC/MS based Untargeted Metabolomics Analysis]]<br />
|-<br />
| 2016 || Zheng Y || [[An improved algorithm for peak detection in mass spectra based on continuous wavelet transform]]<br />
|-<br />
| 2017 || Li B || [[NOREVA: normalization and evaluation of MS-based metabolomics data]]<br />
|-<br />
| 2018 || Mazoure B || [[Identification and Correction of Additive and Multiplicative Spatial Biases in Experimental High-Throughput Screening]]<br />
|-<br />
| 2018 || Li Z || [[Comprehensive evaluation of untargeted metabolomics data processing software in feature detection, quantification and discriminating marker selection]]<br />
|-<br />
| 2018 || Willforss J || [[NormalyzerDE: Online Tool for Improved Normalization of Omics Expression Data and High-Sensitivity Differential Expression Analysis]]<br />
|}</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=An_improved_algorithm_for_peak_detection_in_mass_spectra_based_on_continuous_wavelet_transform&diff=719An improved algorithm for peak detection in mass spectra based on continuous wavelet transform2020-02-26T07:11:33Z<p>Bwday: </p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
=== Citation ===<br />
Zheng Y, Fan R, Qiu C, Liu Z, Tian D. An improved algorithm for peak detection in mass spectra based on continuous wavelet transform. International Journal of Mass Spectrometry. 2016 Nov 1;409:53-8.<br />
<br />
[https://doi.org/10.1016/j.ijms.2016.09.020 Permanent link to the paper]<br />
<br />
<br />
=== Summary ===<br />
Briefly describe the scope of the paper, i.e. the field of research and/or application.<br />
<br />
=== Study outcomes ===<br />
List the paper results concerning method comparison and benchmarking:<br />
==== Outcome O1 ====<br />
The performance of ...<br />
<br />
Outcome O1 is presented as Figure X in the original publication. <br />
<br />
==== Outcome O2 ====<br />
...<br />
<br />
Outcome O2 is presented as Figure X in the original publication. <br />
<br />
==== Outcome On ====<br />
...<br />
<br />
Outcome On is presented as Figure X in the original publication. <br />
<br />
==== Further outcomes ====<br />
If intended, you can add further outcomes here.<br />
<br />
<br />
=== Study design and evidence level ===<br />
==== General aspects ====<br />
You can describe general design aspects here.<br />
The study designs for describing specific outcomes are listed in the following subsections:<br />
<br />
==== Design for Outcome O1 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
==== Design for Outcome O2 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
... <br />
<br />
==== Design for Outcome O ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
=== Further comments and aspects ===<br />
<br />
=== References ===<br />
The list of cited or related literature is placed here.</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=An_improved_algorithm_for_peak_detection_in_mass_spectra_based_on_continuous_wavelet_transform&diff=718An improved algorithm for peak detection in mass spectra based on continuous wavelet transform2020-02-26T07:11:23Z<p>Bwday: Created page with "__ NUMBEREDHEADINGS__ === Citation === Zheng Y, Fan R, Qiu C, Liu Z, Tian D. An improved algorithm for peak detection in mass spectra based on continuous wavelet transform. In..."</p>
<hr />
<div>__ NUMBEREDHEADINGS__<br />
=== Citation ===<br />
Zheng Y, Fan R, Qiu C, Liu Z, Tian D. An improved algorithm for peak detection in mass spectra based on continuous wavelet transform. International Journal of Mass Spectrometry. 2016 Nov 1;409:53-8.<br />
<br />
[https://doi.org/10.1016/j.ijms.2016.09.020 Permanent link to the paper]<br />
<br />
<br />
=== Summary ===<br />
Briefly describe the scope of the paper, i.e. the field of research and/or application.<br />
<br />
=== Study outcomes ===<br />
List the paper results concerning method comparison and benchmarking:<br />
==== Outcome O1 ====<br />
The performance of ...<br />
<br />
Outcome O1 is presented as Figure X in the original publication. <br />
<br />
==== Outcome O2 ====<br />
...<br />
<br />
Outcome O2 is presented as Figure X in the original publication. <br />
<br />
==== Outcome On ====<br />
...<br />
<br />
Outcome On is presented as Figure X in the original publication. <br />
<br />
==== Further outcomes ====<br />
If intended, you can add further outcomes here.<br />
<br />
<br />
=== Study design and evidence level ===<br />
==== General aspects ====<br />
You can describe general design aspects here.<br />
The study designs for describing specific outcomes are listed in the following subsections:<br />
<br />
==== Design for Outcome O1 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
==== Design for Outcome O2 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
... <br />
<br />
==== Design for Outcome O ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
=== Further comments and aspects ===<br />
<br />
=== References ===<br />
The list of cited or related literature is placed here.</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Literature_Studies&diff=717Literature Studies2020-02-26T07:10:43Z<p>Bwday: /* Preprocessing high-throughput data */</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
{| class="wikitable"<br />
|-<br />
! Page summary<br />
|-<br />
| Here outcomes of benchmarking studies from the literature are collected. The primary aim is a comprehensive overview about neutral benchmark studies, i.e. assessments which were performed independenty on publication of a new approach. Studies which are not neutral are put in brackets. </br> <br />
<br />
The focus is on computational methods for analyzing experimental data (instead of comparing experimental techniques or platforms). </br><br />
<br />
Please extend this list by creating a new page and adding a link below. </br> <br />
Use the '''[[Guidelines_for_Summarizing_a_Literature_Study|guidelines described here]]'''.<br />
|}<br />
<br />
== Results from Literature ==<br />
<br />
=== Classification ===<br />
''' 2003 '''</br><br />
* [[Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data]]<br />
''' 2005 '''</br><br />
* [[A review and comparison of classification algorithms for medical decision making]]<br />
''' 2016 '''</br><br />
* [[Predicting Breast Cancer Survivability Using Data Mining Techniques]]<br />
<br />
=== Selection of Differential Features and Regions ===<br />
==== Identifying differential features ====<br />
''' 2006 '''</br><br />
* [[Rat toxicogenomic study reveals analytical consistency across microarray platforms]]<br />
''' 2010 '''</br><br />
* [[A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the sequencing Quality control consortium]]<br />
''' 2017 '''</br><br />
* [[Identification of differentially expressed peptides in high-throughput proteomics data]]<br />
* [[In-depth method assessments of differentially expressed protein detection for shotgun proteomics data with missing values]]<br />
* [[Strategies for analyzing bisulfite sequencing data]]<br />
''' 2018 '''</br><br />
* [[Identification of Differentially Methylated Sites with Weak Methylation Effects]]<br />
<br />
==== Identifying differential regions (e.g. DMRs) ====<br />
''' 2015 '''<br />
* [[De novo identification of differentially methylated regions in the human genome]]<br />
* [[MethylAction: detecting differentially methylated regions that distinguish biological subtypes]]<br />
* [[metilene: Fast and sensitive calling of differentially methylated regions from bisulfite sequencing data]]<br />
''' 2016 '''<br />
* [[seqlm: an MDL based method for identifying differentially methylated regions in high density methylation array data]]<br />
* [[Statistical methods for detecting differentially methylated regions based on MethylCap-seq data]]<br />
''' 2017 '''<br />
* [[DMRfinder: efficiently identifying differentially methylated regions from MethylC-seq data]]<br />
''' 2018 '''<br />
* [[Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus]]<br />
* [[DMRcaller: a versatile R/Bioconductor package for detection and visualization of differentially methylated regions in CpG and non-CpG contexts]]<br />
* [[MethCP: Differentially Methylated Region Detection with Change Point Models (bioRxiv)]]<br />
<br />
==== Identifying sets of features (e.g. gene set analyses) ====<br />
'''2009<br />
<br />
* [[A general modular framework for gene set enrichment analysis]]<br />
* [[Comparing gene set analysis methods on single-nucleotide polymorphism data from Genetic Analysis Workshop 16]]<br />
<br />
'''2018<br />
<br />
* [[Gene set analysis methods: a systematic comparison]]<br />
<br />
'''2020<br />
* [[Toward a gold standard for benchmarking gene set enrichment analysis]]<br />
<br />
==== Dimension reduction ====<br />
''' 2008 '''</br><br />
* [[On the Relationship Between Feature Selection and Classification Accuracy]]<br />
''' 2015 '''</br><br />
* [[Comparing feature selection methods for highdimensional imbalanced data: identifying rheumatoid arthritis cohorts from routine data]]<br />
<br />
=== Imputation methods for missing values ===<br />
<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 1996 || Schenker || [[Partially parametric techniques for multiple imputation]]<br />
|-<br />
| 1999 || Hastie T || [[Imputing Missing Data for Gene Expression Arrays]]<br />
|-<br />
| 2001 || Troyanskaya || [[Missing value estimation methods for DNA microarrays]]<br />
|-<br />
| 2002 || Engels J || [[Imputation of missing longitudinal data: a comparison of methods]]<br />
|-<br />
| 2003 || Oba || [[A Bayesian missing value estimation method for gene expression profile data]]<br />
|-<br />
| 2005 || Scholz || [[Nonlinear PCA: a missing data approach]]<br />
|-<br />
| 2007 || Stacklies || [[pcaMethods—a bioconductor package providing PCA methods for incomplete data]]<br />
|-<br />
| 2007 || Verboven || [[Sequential imputation for missing values]]<br />
|-<br />
| 2008 || Shaffer GN || [[Which missing value imputation method to use in expression profiles: a comparative study and two selection schemes]]<br />
|-<br />
| 2011 || Templ || [[Iterative stepwise regression imputation using standard and robust methods]]<br />
|-<br />
| 2012 || Hrydziuszko O || [[Missing values in mass spectrometry based metabolomics: an undervalued step in the data processing pipeline]]<br />
|-<br />
| 2012 || Stekhoven || [[MissForest—non-parametric missing value imputation for mixed-type data]]<br />
|-<br />
| 2013 || Taylor || [[Accounting for undetected compounds in statistical analyses of mass spectrometry ‘omic studies]]<br />
|-<br />
| 2013 || Waljee || [[Comparison of imputation methods for missing laboratory data in medicine]]<br />
|-<br />
| 2014 || Shah || [[Comparison of Random Forest and Parametric Imputation Models for Imputing Missing Data Using MICE: A CALIBER Study]]<br />
|-<br />
| 2014 || Rodwell || [[Comparison of methods for imputing limited-range variables: a simulation study]]<br />
|-<br />
| 2014 || Morris || [[Tuning multiple imputation by predictive mean matching and local residual draws]]<br />
|-<br />
| 2014 || Doove L || [[Recursive partitioning for missing data imputation in the presence of interaction effects]]<br />
|-<br />
| 2015 || Webb-Robertson BJM || [[Review, evaluation, and discussion of the challenges of missing value imputation for mass spectrometry-based label-free global proteomics]]<br />
|-<br />
| 2016 || Folch-Fortuny A || [[Assessment of maximum likelihood PCA missing data imputation]]<br />
|-<br />
| 2016 || Lazar C || [[Accounting for the Multiple Natures of Missing Values in Label-Free Quantitative Proteomics Data Sets to Compare Imputation Strategies]]<br />
|-<br />
| 2016 || Yin X || [[Multiple imputation and analysis for high-dimensional incomplete proteomics data]]<br />
|-<br />
| 2018 || Wei R || [[Missing Value Imputation Approach for Mass Spectrometry-based Metabolomics Data]]<br />
|-<br />
| 2018 || Poyatos R || [[Gap-filling a spatially explicit plant trait database: comparing imputation methods and different levels of environmental information]]<br />
|-<br />
| 2018 || O'Brien JJ || [[The effects of nonignorable missing data on label-free mass spectrometry proteomics experiments]]<br />
|}<br />
<br />
=== ODE-based Modelling ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2001 || Beal || [[Ways to Fit a PK Model with Some Data Below the Quantification Limit]]<br />
|-<br />
| 2008 || Balsa-Canto || [[Hybrid optimization method with general switching strategy for parameter estimation]]<br />
|-<br />
| 2011 || Tashkova || [[Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis]]<br />
|-<br />
| 2013 || Raue || [[Lessons Learned from Quantitative Dynamical Modeling in Systems Biology]]<br />
|-<br />
| 2013 || Dondelinger || [[ODE parameter inference using adaptive gradient matching with Gaussian processes]]<br />
|-<br />
| 2017 || Ballnus || [[Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems]]<br />
|-<br />
| 2017 || Henriques || [[Data-driven reverse engineering of signaling pathways using ensembles of dynamic models]]<br />
|-<br />
| 2017 || Melicher || [[Fast derivatives of likelihood functionals for ODE based models using adjoint-state method]]<br />
|-<br />
| 2017 || Penas || [[Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy]]<br />
|-<br />
| 2017 || Degasperi || [[Performance of objective functions and optimization procedures for parameter estimation in system biology models]]<br />
|-<br />
| 2017 || Fröhlich || [[Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks]]<br />
|-<br />
| 2018 || Schälte || [[Evaluation of Derivative-Free Optimizers for Parameter Estimation in Systems Biology]]<br />
|-<br />
| 2018 || Loos || [[Hierarchical optimization for the efficient parametrization of ODE models]]<br />
|-<br />
| 2018 || Stapor || [[Optimization and profile calculation of ODE models using second order adjoint sensitivity analysis]]<br />
|-<br />
| 2019 || Villaverde || [[A comparison of methods for quantifying prediction uncertainty in systems biology]]<br />
|-<br />
| 2019 || Hass || [[Benchmark problems for dynamic modeling of intracellular processes]]<br />
|-<br />
| 2019 || Villaverde || [[Benchmarking optimization methods for parameter estimation in large kinetic models]]<br />
|-<br />
| 2019 || Lines || [[Efficient computation of steady states in large-scale ODE models of biochemical reaction networks]]<br />
|-<br />
| 2019 || Stapor || [[Mini-batch optimization enables training of ODE models on large-scale datasets]]<br />
|-<br />
| 2019 || Wu || [[Parameter Estimation and Variable Selection for Big Systems of Linear Ordinary Differential Equations: A Matrix-Based Approach]]<br />
|-<br />
| 2019 || Pitt || [[Parameter estimation in models of biological oscillators: an automated regularised estimation approach]]<br />
|-<br />
| 2019 || Loos || [[Robust calibration of hierarchical population models for heterogeneous cell populations]]<br />
|-<br />
| 2019 || Clairon || [[Tracking for parameter and state estimation in possibly misspecified partially observed linear Ordinary Differential Equations]]<br />
|-<br />
| 2020 || Schmiester || [[Efficient parameterization of large-scale dynamic models based on relative measurements]]<br />
|-<br />
| 2020 || Castro || [[Testing structural identifiability by a simple scaling method]]<br />
|}<br />
<br />
=== Omics Workflows ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2008 || Neuweger H || [[MeltDB: a software platform for the analysis and integration of metabolomics experiment data]]<br />
|-<br />
| 2008 || Barla A || [[Machine learning methods for predictive proteomics]]<br />
|-<br />
| 2009 || Xia J || [[MetaboAnalyst: a web server for metabolomic data analysis and interpretation]]<br />
|-<br />
| 2013 || Weisser H || [[An Automated Pipeline for High-Throughput Label-Free Quantitative Proteomics]]<br />
|-<br />
| 2014 || Cox J || [[Accurate Proteome-wide Label-free Quantification by Delayed Normalization and Maximal Peptide Ratio Extraction, Termed MaxLFQ* ]]<br />
|-<br />
| 2015 || || [[ComparingVariant Call Files for Performance Benchmarkingof Next-Generation Sequencing Variant Calling Pipelines]]<br />
|-<br />
| 2016 || Tyanova S || [[The MaxQuant computational platform for mass spectrometry–based shotgun proteomics]]<br />
|-<br />
| 2016 || Röst HL || [[OpenMS: a flexible open-source software platform for mass spectrometry data analysis]]<br />
|-<br />
| 2017 || || [[A benchmarking of workflows for detecting differential splicing and differential expression at isoform level in human RNA-seq studies]]<br />
|-<br />
| 2018 || Välikangas T || [[A comprehensive evaluation of popular proteomics software workflows for label-free proteome quantification and imputation]]<br />
|-<br />
| 2019 || || [[A Systematic Evaluation of Single CellRNA-Seq Analysis Pipelines]]<br />
|-<br />
| 2019 || || [[Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays]]<br />
|}<br />
<br />
=== Preprocessing high-throughput data===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|- 1999 || Perkins DN || [[Probability-based protein identification by searching sequence databases using mass spectrometry data]]<br />
|-<br />
| 2003 || || [[A comparison of normalization methods for high density oligonucleotide array data based on variance and bias ]]<br />
|-<br />
| 2005 || || [[Comparison of Affymetrix GeneChip Expression Measures]]<br />
|-<br />
| 2005 || Meleth S || [[The case for well-conducted experiments to validate statistical protocols for 2D gels: different pre-processing = different lists of significant proteins]]<br />
|-<br />
| 2005 || || [[Comparison of background correction and normalization procedures for high-density oligonucleotide microarrays]]<br />
|-<br />
| 2006 || || [[Using RNA sample titrations to assess microarray platform performance and normalization techniques]]<br />
|-<br />
| 2006 || Wang P || [[Normalization regarding non-random missing values in high-throughput mass spectrometry data]]<br />
|-<br />
| 2007 || Carvalho B || [[Exploration, normalization, and genotype calls of high-density oligonucleotide SNP array data]]<br />
|-<br />
| 2007 || Cannataro M || [[MS‐Analyzer: preprocessing and data mining services for proteomics applications on the Grid]]<br />
|-<br />
| 2008 || || [[Comparison of preprocessing methods for the hgU133+2 chip from Affymetrix]]<br />
|-<br />
| 2009 || || [[Comparison of Affymetrix data normalization methods using 6,926 experiments across five array generations]]<br />
|-<br />
| 2009 || Mar JC || [[Data-driven normalization strategies for high-throughput quantitative RT-PCR]]<br />
|-<br />
| 2009 || Vakhrushev SY || [[Software platform for high-throughput glycomics]]<br />
|-<br />
| 2010 || || [[Consistency of predictive signature genes and classifiers generated using different microarray platforms]]<br />
|-<br />
| 2010 || || [[Detecting and correcting systematic variation in large-scale RNA sequencing data]]<br />
|-<br />
| 2010 || || [[Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments]]<br />
|-<br />
| 2010 || || [[Normalization of RNA-seq data using factor analysis of control genes or samples]]<br />
|-<br />
| 2010 || Armananzas R || [[Peakbin selection in mass spectrometry data using a consensus approach with estimation of distribution algorithms]]<br />
|-<br />
| 2011 || || [[Affymetrix GeneChip microarray preprocessing for multivariate analyses]]<br />
|-<br />
| 2012 || || [[A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis]]<br />
|-<br />
| 2013 || García-Torres M || [[Comparison of metaheuristic strategies for peakbin selection in proteomic mass spectrometry data]]<br />
|-<br />
| 2014 || || [[Normalyzer: A Tool for Rapid Evaluation of Normalization Methods for Omics Data Sets]]<br />
|-<br />
| 2014 || Zhou X || [[Prevention, diagnosis and treatment of high-throughput sequencing data pathologies]]<br />
|-<br />
| 2014 || Coble JB || [[Comparative evaluation of preprocessing freeware on chromatography/mass spectrometry data for signature discovery]]<br />
|-<br />
| 2015 || Caraus I || [[Detecting and overcoming systematic bias in high-throughput screening technologies: a comprehensive review of practical issues and methodological solutions]]<br />
|-<br />
| 2015 || Tam S || [[Optimization of miRNA-seq data preprocessing]]<br />
|-<br />
| 2015 || Rafiei A || [[Comparison of peak‐picking workflows for untargeted liquid chromatography/high‐resolution mass spectrometry metabolomics data analysis]]<br />
|-<br />
| 2015 || Chawade A || [[Data processing has major impact on the outcome of quantitative label-free LC-MS analysis]]<br />
|-<br />
| 2015 || Wang T || [[A systematic study of normalization methods for Infinium 450K methylation data using whole-genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Yi L || [[Chemometric methods in data processing of mass spectrometry-based metabolomics: A review]]<br />
|-<br />
| 2016 || Tsuji J || [[Evaluation of preprocessing, mapping and postprocessing algorithms for analyzing whole genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Li B || [[Performance Evaluation and Online Realization of Data-driven Normalization Methods Used in LC/MS based Untargeted Metabolomics Analysis]]<br />
|-<br />
| 2016 || Zheng Y || [[An improved algorithm for peak detection in mass spectra based on continuous wavelet transform]]<br />
|-<br />
| 2017 || Li B || [[NOREVA: normalization and evaluation of MS-based metabolomics data]]<br />
|-<br />
| 2018 || Mazoure B || [[Identification and Correction of Additive and Multiplicative Spatial Biases in Experimental High-Throughput Screening]]<br />
|-<br />
| 2018 || Li Z || [[Comprehensive evaluation of untargeted metabolomics data processing software in feature detection, quantification and discriminating marker selection]]<br />
|-<br />
| 2018 || Willforss J || [[NormalyzerDE: Online Tool for Improved Normalization of Omics Expression Data and High-Sensitivity Differential Expression Analysis]]<br />
|}</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Comparison_of_metaheuristic_strategies_for_peakbin_selection_in_proteomic_mass_spectrometry_data&diff=716Comparison of metaheuristic strategies for peakbin selection in proteomic mass spectrometry data2020-02-25T16:29:57Z<p>Bwday: Created page with "__NUMBEREDHEADINGS__ === Citation === García-Torres M, Armañanzas R, Bielza C, Larrañaga P. Comparison of metaheuristic strategies for peakbin selection in proteomic mass s..."</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
=== Citation ===<br />
García-Torres M, Armañanzas R, Bielza C, Larrañaga P. Comparison of metaheuristic strategies for peakbin selection in proteomic mass spectrometry data. Information Sciences. 2013 Feb 10;222:229-46.<br />
<br />
[https://doi.org/10.1016/j.ins.2010.12.013 Permanent link to the paper]<br />
<br />
<br />
=== Summary ===<br />
Briefly describe the scope of the paper, i.e. the field of research and/or application.<br />
<br />
=== Study outcomes ===<br />
List the paper results concerning method comparison and benchmarking:<br />
==== Outcome O1 ====<br />
The performance of ...<br />
<br />
Outcome O1 is presented as Figure X in the original publication. <br />
<br />
==== Outcome O2 ====<br />
...<br />
<br />
Outcome O2 is presented as Figure X in the original publication. <br />
<br />
==== Outcome On ====<br />
...<br />
<br />
Outcome On is presented as Figure X in the original publication. <br />
<br />
==== Further outcomes ====<br />
If intended, you can add further outcomes here.<br />
<br />
<br />
=== Study design and evidence level ===<br />
==== General aspects ====<br />
You can describe general design aspects here.<br />
The study designs for describing specific outcomes are listed in the following subsections:<br />
<br />
==== Design for Outcome O1 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
==== Design for Outcome O2 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
... <br />
<br />
==== Design for Outcome O ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
=== Further comments and aspects ===<br />
<br />
=== References ===<br />
The list of cited or related literature is placed here.</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Literature_Studies&diff=715Literature Studies2020-02-25T16:29:15Z<p>Bwday: /* Preprocessing high-throughput data */</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
{| class="wikitable"<br />
|-<br />
! Page summary<br />
|-<br />
| Here outcomes of benchmarking studies from the literature are collected. The primary aim is a comprehensive overview about neutral benchmark studies, i.e. assessments which were performed independenty on publication of a new approach. Studies which are not neutral are put in brackets. </br> <br />
<br />
The focus is on computational methods for analyzing experimental data (instead of comparing experimental techniques or platforms). </br><br />
<br />
Please extend this list by creating a new page and adding a link below. </br> <br />
Use the '''[[Guidelines_for_Summarizing_a_Literature_Study|guidelines described here]]'''.<br />
|}<br />
<br />
== Results from Literature ==<br />
<br />
=== Classification ===<br />
''' 2003 '''</br><br />
* [[Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data]]<br />
''' 2005 '''</br><br />
* [[A review and comparison of classification algorithms for medical decision making]]<br />
''' 2016 '''</br><br />
* [[Predicting Breast Cancer Survivability Using Data Mining Techniques]]<br />
<br />
=== Selection of Differential Features and Regions ===<br />
==== Identifying differential features ====<br />
''' 2006 '''</br><br />
* [[Rat toxicogenomic study reveals analytical consistency across microarray platforms]]<br />
''' 2010 '''</br><br />
* [[A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the sequencing Quality control consortium]]<br />
''' 2017 '''</br><br />
* [[Identification of differentially expressed peptides in high-throughput proteomics data]]<br />
* [[In-depth method assessments of differentially expressed protein detection for shotgun proteomics data with missing values]]<br />
* [[Strategies for analyzing bisulfite sequencing data]]<br />
''' 2018 '''</br><br />
* [[Identification of Differentially Methylated Sites with Weak Methylation Effects]]<br />
<br />
==== Identifying differential regions (e.g. DMRs) ====<br />
''' 2015 '''<br />
* [[De novo identification of differentially methylated regions in the human genome]]<br />
* [[MethylAction: detecting differentially methylated regions that distinguish biological subtypes]]<br />
* [[metilene: Fast and sensitive calling of differentially methylated regions from bisulfite sequencing data]]<br />
''' 2016 '''<br />
* [[seqlm: an MDL based method for identifying differentially methylated regions in high density methylation array data]]<br />
* [[Statistical methods for detecting differentially methylated regions based on MethylCap-seq data]]<br />
''' 2017 '''<br />
* [[DMRfinder: efficiently identifying differentially methylated regions from MethylC-seq data]]<br />
''' 2018 '''<br />
* [[Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus]]<br />
* [[DMRcaller: a versatile R/Bioconductor package for detection and visualization of differentially methylated regions in CpG and non-CpG contexts]]<br />
* [[MethCP: Differentially Methylated Region Detection with Change Point Models (bioRxiv)]]<br />
<br />
==== Identifying sets of features (e.g. gene set analyses) ====<br />
'''2009<br />
<br />
* [[A general modular framework for gene set enrichment analysis]]<br />
* [[Comparing gene set analysis methods on single-nucleotide polymorphism data from Genetic Analysis Workshop 16]]<br />
<br />
'''2018<br />
<br />
* [[Gene set analysis methods: a systematic comparison]]<br />
<br />
'''2020<br />
* [[Toward a gold standard for benchmarking gene set enrichment analysis]]<br />
<br />
==== Dimension reduction ====<br />
''' 2008 '''</br><br />
* [[On the Relationship Between Feature Selection and Classification Accuracy]]<br />
''' 2015 '''</br><br />
* [[Comparing feature selection methods for highdimensional imbalanced data: identifying rheumatoid arthritis cohorts from routine data]]<br />
<br />
=== Imputation methods for missing values ===<br />
<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 1996 || Schenker || [[Partially parametric techniques for multiple imputation]]<br />
|-<br />
| 1999 || Hastie T || [[Imputing Missing Data for Gene Expression Arrays]]<br />
|-<br />
| 2001 || Troyanskaya || [[Missing value estimation methods for DNA microarrays]]<br />
|-<br />
| 2002 || Engels J || [[Imputation of missing longitudinal data: a comparison of methods]]<br />
|-<br />
| 2003 || Oba || [[A Bayesian missing value estimation method for gene expression profile data]]<br />
|-<br />
| 2005 || Scholz || [[Nonlinear PCA: a missing data approach]]<br />
|-<br />
| 2007 || Stacklies || [[pcaMethods—a bioconductor package providing PCA methods for incomplete data]]<br />
|-<br />
| 2007 || Verboven || [[Sequential imputation for missing values]]<br />
|-<br />
| 2008 || Shaffer GN || [[Which missing value imputation method to use in expression profiles: a comparative study and two selection schemes]]<br />
|-<br />
| 2011 || Templ || [[Iterative stepwise regression imputation using standard and robust methods]]<br />
|-<br />
| 2012 || Hrydziuszko O || [[Missing values in mass spectrometry based metabolomics: an undervalued step in the data processing pipeline]]<br />
|-<br />
| 2012 || Stekhoven || [[MissForest—non-parametric missing value imputation for mixed-type data]]<br />
|-<br />
| 2013 || Taylor || [[Accounting for undetected compounds in statistical analyses of mass spectrometry ‘omic studies]]<br />
|-<br />
| 2013 || Waljee || [[Comparison of imputation methods for missing laboratory data in medicine]]<br />
|-<br />
| 2014 || Shah || [[Comparison of Random Forest and Parametric Imputation Models for Imputing Missing Data Using MICE: A CALIBER Study]]<br />
|-<br />
| 2014 || Rodwell || [[Comparison of methods for imputing limited-range variables: a simulation study]]<br />
|-<br />
| 2014 || Morris || [[Tuning multiple imputation by predictive mean matching and local residual draws]]<br />
|-<br />
| 2014 || Doove L || [[Recursive partitioning for missing data imputation in the presence of interaction effects]]<br />
|-<br />
| 2015 || Webb-Robertson BJM || [[Review, evaluation, and discussion of the challenges of missing value imputation for mass spectrometry-based label-free global proteomics]]<br />
|-<br />
| 2016 || Folch-Fortuny A || [[Assessment of maximum likelihood PCA missing data imputation]]<br />
|-<br />
| 2016 || Lazar C || [[Accounting for the Multiple Natures of Missing Values in Label-Free Quantitative Proteomics Data Sets to Compare Imputation Strategies]]<br />
|-<br />
| 2016 || Yin X || [[Multiple imputation and analysis for high-dimensional incomplete proteomics data]]<br />
|-<br />
| 2018 || Wei R || [[Missing Value Imputation Approach for Mass Spectrometry-based Metabolomics Data]]<br />
|-<br />
| 2018 || Poyatos R || [[Gap-filling a spatially explicit plant trait database: comparing imputation methods and different levels of environmental information]]<br />
|-<br />
| 2018 || O'Brien JJ || [[The effects of nonignorable missing data on label-free mass spectrometry proteomics experiments]]<br />
|}<br />
<br />
=== ODE-based Modelling ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2001 || Beal || [[Ways to Fit a PK Model with Some Data Below the Quantification Limit]]<br />
|-<br />
| 2008 || Balsa-Canto || [[Hybrid optimization method with general switching strategy for parameter estimation]]<br />
|-<br />
| 2011 || Tashkova || [[Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis]]<br />
|-<br />
| 2013 || Raue || [[Lessons Learned from Quantitative Dynamical Modeling in Systems Biology]]<br />
|-<br />
| 2013 || Dondelinger || [[ODE parameter inference using adaptive gradient matching with Gaussian processes]]<br />
|-<br />
| 2017 || Ballnus || [[Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems]]<br />
|-<br />
| 2017 || Henriques || [[Data-driven reverse engineering of signaling pathways using ensembles of dynamic models]]<br />
|-<br />
| 2017 || Melicher || [[Fast derivatives of likelihood functionals for ODE based models using adjoint-state method]]<br />
|-<br />
| 2017 || Penas || [[Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy]]<br />
|-<br />
| 2017 || Degasperi || [[Performance of objective functions and optimization procedures for parameter estimation in system biology models]]<br />
|-<br />
| 2017 || Fröhlich || [[Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks]]<br />
|-<br />
| 2018 || Schälte || [[Evaluation of Derivative-Free Optimizers for Parameter Estimation in Systems Biology]]<br />
|-<br />
| 2018 || Loos || [[Hierarchical optimization for the efficient parametrization of ODE models]]<br />
|-<br />
| 2018 || Stapor || [[Optimization and profile calculation of ODE models using second order adjoint sensitivity analysis]]<br />
|-<br />
| 2019 || Villaverde || [[A comparison of methods for quantifying prediction uncertainty in systems biology]]<br />
|-<br />
| 2019 || Hass || [[Benchmark problems for dynamic modeling of intracellular processes]]<br />
|-<br />
| 2019 || Villaverde || [[Benchmarking optimization methods for parameter estimation in large kinetic models]]<br />
|-<br />
| 2019 || Lines || [[Efficient computation of steady states in large-scale ODE models of biochemical reaction networks]]<br />
|-<br />
| 2019 || Stapor || [[Mini-batch optimization enables training of ODE models on large-scale datasets]]<br />
|-<br />
| 2019 || Wu || [[Parameter Estimation and Variable Selection for Big Systems of Linear Ordinary Differential Equations: A Matrix-Based Approach]]<br />
|-<br />
| 2019 || Pitt || [[Parameter estimation in models of biological oscillators: an automated regularised estimation approach]]<br />
|-<br />
| 2019 || Loos || [[Robust calibration of hierarchical population models for heterogeneous cell populations]]<br />
|-<br />
| 2019 || Clairon || [[Tracking for parameter and state estimation in possibly misspecified partially observed linear Ordinary Differential Equations]]<br />
|-<br />
| 2020 || Schmiester || [[Efficient parameterization of large-scale dynamic models based on relative measurements]]<br />
|-<br />
| 2020 || Castro || [[Testing structural identifiability by a simple scaling method]]<br />
|}<br />
<br />
=== Omics Workflows ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2008 || Neuweger H || [[MeltDB: a software platform for the analysis and integration of metabolomics experiment data]]<br />
|-<br />
| 2008 || Barla A || [[Machine learning methods for predictive proteomics]]<br />
|-<br />
| 2009 || Xia J || [[MetaboAnalyst: a web server for metabolomic data analysis and interpretation]]<br />
|-<br />
| 2013 || Weisser H || [[An Automated Pipeline for High-Throughput Label-Free Quantitative Proteomics]]<br />
|-<br />
| 2014 || Cox J || [[Accurate Proteome-wide Label-free Quantification by Delayed Normalization and Maximal Peptide Ratio Extraction, Termed MaxLFQ* ]]<br />
|-<br />
| 2015 || || [[ComparingVariant Call Files for Performance Benchmarkingof Next-Generation Sequencing Variant Calling Pipelines]]<br />
|-<br />
| 2016 || Tyanova S || [[The MaxQuant computational platform for mass spectrometry–based shotgun proteomics]]<br />
|-<br />
| 2016 || Röst HL || [[OpenMS: a flexible open-source software platform for mass spectrometry data analysis]]<br />
|-<br />
| 2017 || || [[A benchmarking of workflows for detecting differential splicing and differential expression at isoform level in human RNA-seq studies]]<br />
|-<br />
| 2018 || Välikangas T || [[A comprehensive evaluation of popular proteomics software workflows for label-free proteome quantification and imputation]]<br />
|-<br />
| 2019 || || [[A Systematic Evaluation of Single CellRNA-Seq Analysis Pipelines]]<br />
|-<br />
| 2019 || || [[Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays]]<br />
|}<br />
<br />
=== Preprocessing high-throughput data===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|- 1999 || Perkins DN || [[Probability-based protein identification by searching sequence databases using mass spectrometry data]]<br />
|-<br />
| 2003 || || [[A comparison of normalization methods for high density oligonucleotide array data based on variance and bias ]]<br />
|-<br />
| 2005 || || [[Comparison of Affymetrix GeneChip Expression Measures]]<br />
|-<br />
| 2005 || Meleth S || [[The case for well-conducted experiments to validate statistical protocols for 2D gels: different pre-processing = different lists of significant proteins]]<br />
|-<br />
| 2005 || || [[Comparison of background correction and normalization procedures for high-density oligonucleotide microarrays]]<br />
|-<br />
| 2006 || || [[Using RNA sample titrations to assess microarray platform performance and normalization techniques]]<br />
|-<br />
| 2006 || Wang P || [[Normalization regarding non-random missing values in high-throughput mass spectrometry data]]<br />
|-<br />
| 2007 || Carvalho B || [[Exploration, normalization, and genotype calls of high-density oligonucleotide SNP array data]]<br />
|-<br />
| 2007 || Cannataro M || [[MS‐Analyzer: preprocessing and data mining services for proteomics applications on the Grid]]<br />
|-<br />
| 2008 || || [[Comparison of preprocessing methods for the hgU133+2 chip from Affymetrix]]<br />
|-<br />
| 2009 || || [[Comparison of Affymetrix data normalization methods using 6,926 experiments across five array generations]]<br />
|-<br />
| 2009 || Mar JC || [[Data-driven normalization strategies for high-throughput quantitative RT-PCR]]<br />
|-<br />
| 2009 || Vakhrushev SY || [[Software platform for high-throughput glycomics]]<br />
|-<br />
| 2010 || || [[Consistency of predictive signature genes and classifiers generated using different microarray platforms]]<br />
|-<br />
| 2010 || || [[Detecting and correcting systematic variation in large-scale RNA sequencing data]]<br />
|-<br />
| 2010 || || [[Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments]]<br />
|-<br />
| 2010 || || [[Normalization of RNA-seq data using factor analysis of control genes or samples]]<br />
|-<br />
| 2010 || Armananzas R || [[Peakbin selection in mass spectrometry data using a consensus approach with estimation of distribution algorithms]]<br />
|-<br />
| 2011 || || [[Affymetrix GeneChip microarray preprocessing for multivariate analyses]]<br />
|-<br />
| 2012 || || [[A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis]]<br />
|-<br />
| 2013 || García-Torres M || [[Comparison of metaheuristic strategies for peakbin selection in proteomic mass spectrometry data]]<br />
|-<br />
| 2014 || || [[Normalyzer: A Tool for Rapid Evaluation of Normalization Methods for Omics Data Sets]]<br />
|-<br />
| 2014 || Zhou X || [[Prevention, diagnosis and treatment of high-throughput sequencing data pathologies]]<br />
|-<br />
| 2014 || Coble JB || [[Comparative evaluation of preprocessing freeware on chromatography/mass spectrometry data for signature discovery]]<br />
|-<br />
| 2015 || Caraus I || [[Detecting and overcoming systematic bias in high-throughput screening technologies: a comprehensive review of practical issues and methodological solutions]]<br />
|-<br />
| 2015 || Tam S || [[Optimization of miRNA-seq data preprocessing]]<br />
|-<br />
| 2015 || Rafiei A || [[Comparison of peak‐picking workflows for untargeted liquid chromatography/high‐resolution mass spectrometry metabolomics data analysis]]<br />
|-<br />
| 2015 || Chawade A || [[Data processing has major impact on the outcome of quantitative label-free LC-MS analysis]]<br />
|-<br />
| 2015 || Wang T || [[A systematic study of normalization methods for Infinium 450K methylation data using whole-genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Yi L || [[Chemometric methods in data processing of mass spectrometry-based metabolomics: A review]]<br />
|-<br />
| 2016 || Tsuji J || [[Evaluation of preprocessing, mapping and postprocessing algorithms for analyzing whole genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Li B || [[Performance Evaluation and Online Realization of Data-driven Normalization Methods Used in LC/MS based Untargeted Metabolomics Analysis]]<br />
|-<br />
| 2017 || Li B || [[NOREVA: normalization and evaluation of MS-based metabolomics data]]<br />
|-<br />
| 2018 || Mazoure B || [[Identification and Correction of Additive and Multiplicative Spatial Biases in Experimental High-Throughput Screening]]<br />
|-<br />
| 2018 || Li Z || [[Comprehensive evaluation of untargeted metabolomics data processing software in feature detection, quantification and discriminating marker selection]]<br />
|-<br />
| 2018 || Willforss J || [[NormalyzerDE: Online Tool for Improved Normalization of Omics Expression Data and High-Sensitivity Differential Expression Analysis]]<br />
|}</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Peakbin_selection_in_mass_spectrometry_data_using_a_consensus_approach_with_estimation_of_distribution_algorithms&diff=714Peakbin selection in mass spectrometry data using a consensus approach with estimation of distribution algorithms2020-02-25T16:24:43Z<p>Bwday: Created page with "__NUMBEREDHEADINGS__ === Citation === Armananzas R, Saeys Y, Inza I, Garcia-Torres M, Bielza C, Van de Peer Y, Larranaga P. Peakbin selection in mass spectrometry data using a..."</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
=== Citation ===<br />
Armananzas R, Saeys Y, Inza I, Garcia-Torres M, Bielza C, Van de Peer Y, Larranaga P. Peakbin selection in mass spectrometry data using a consensus approach with estimation of distribution algorithms. IEEE/ACM Transactions on Computational Biology and Bioinformatics. 2010 Mar 25;8(3):760-74.<br />
<br />
[https://doi.org/10.1109/TCBB.2010.18 Permanent link to the paper]<br />
<br />
<br />
=== Summary ===<br />
Briefly describe the scope of the paper, i.e. the field of research and/or application.<br />
<br />
=== Study outcomes ===<br />
List the paper results concerning method comparison and benchmarking:<br />
==== Outcome O1 ====<br />
The performance of ...<br />
<br />
Outcome O1 is presented as Figure X in the original publication. <br />
<br />
==== Outcome O2 ====<br />
...<br />
<br />
Outcome O2 is presented as Figure X in the original publication. <br />
<br />
==== Outcome On ====<br />
...<br />
<br />
Outcome On is presented as Figure X in the original publication. <br />
<br />
==== Further outcomes ====<br />
If intended, you can add further outcomes here.<br />
<br />
<br />
=== Study design and evidence level ===<br />
==== General aspects ====<br />
You can describe general design aspects here.<br />
The study designs for describing specific outcomes are listed in the following subsections:<br />
<br />
==== Design for Outcome O1 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
==== Design for Outcome O2 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
... <br />
<br />
==== Design for Outcome O ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
=== Further comments and aspects ===<br />
<br />
=== References ===<br />
The list of cited or related literature is placed here.</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Literature_Studies&diff=713Literature Studies2020-02-25T16:21:37Z<p>Bwday: /* Preprocessing high-throughput data */</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
{| class="wikitable"<br />
|-<br />
! Page summary<br />
|-<br />
| Here outcomes of benchmarking studies from the literature are collected. The primary aim is a comprehensive overview about neutral benchmark studies, i.e. assessments which were performed independenty on publication of a new approach. Studies which are not neutral are put in brackets. </br> <br />
<br />
The focus is on computational methods for analyzing experimental data (instead of comparing experimental techniques or platforms). </br><br />
<br />
Please extend this list by creating a new page and adding a link below. </br> <br />
Use the '''[[Guidelines_for_Summarizing_a_Literature_Study|guidelines described here]]'''.<br />
|}<br />
<br />
== Results from Literature ==<br />
<br />
=== Classification ===<br />
''' 2003 '''</br><br />
* [[Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data]]<br />
''' 2005 '''</br><br />
* [[A review and comparison of classification algorithms for medical decision making]]<br />
''' 2016 '''</br><br />
* [[Predicting Breast Cancer Survivability Using Data Mining Techniques]]<br />
<br />
=== Selection of Differential Features and Regions ===<br />
==== Identifying differential features ====<br />
''' 2006 '''</br><br />
* [[Rat toxicogenomic study reveals analytical consistency across microarray platforms]]<br />
''' 2010 '''</br><br />
* [[A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the sequencing Quality control consortium]]<br />
''' 2017 '''</br><br />
* [[Identification of differentially expressed peptides in high-throughput proteomics data]]<br />
* [[In-depth method assessments of differentially expressed protein detection for shotgun proteomics data with missing values]]<br />
* [[Strategies for analyzing bisulfite sequencing data]]<br />
''' 2018 '''</br><br />
* [[Identification of Differentially Methylated Sites with Weak Methylation Effects]]<br />
<br />
==== Identifying differential regions (e.g. DMRs) ====<br />
''' 2015 '''<br />
* [[De novo identification of differentially methylated regions in the human genome]]<br />
* [[MethylAction: detecting differentially methylated regions that distinguish biological subtypes]]<br />
* [[metilene: Fast and sensitive calling of differentially methylated regions from bisulfite sequencing data]]<br />
''' 2016 '''<br />
* [[seqlm: an MDL based method for identifying differentially methylated regions in high density methylation array data]]<br />
* [[Statistical methods for detecting differentially methylated regions based on MethylCap-seq data]]<br />
''' 2017 '''<br />
* [[DMRfinder: efficiently identifying differentially methylated regions from MethylC-seq data]]<br />
''' 2018 '''<br />
* [[Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus]]<br />
* [[DMRcaller: a versatile R/Bioconductor package for detection and visualization of differentially methylated regions in CpG and non-CpG contexts]]<br />
* [[MethCP: Differentially Methylated Region Detection with Change Point Models (bioRxiv)]]<br />
<br />
==== Identifying sets of features (e.g. gene set analyses) ====<br />
'''2009<br />
<br />
* [[A general modular framework for gene set enrichment analysis]]<br />
* [[Comparing gene set analysis methods on single-nucleotide polymorphism data from Genetic Analysis Workshop 16]]<br />
<br />
'''2018<br />
<br />
* [[Gene set analysis methods: a systematic comparison]]<br />
<br />
'''2020<br />
* [[Toward a gold standard for benchmarking gene set enrichment analysis]]<br />
<br />
==== Dimension reduction ====<br />
''' 2008 '''</br><br />
* [[On the Relationship Between Feature Selection and Classification Accuracy]]<br />
''' 2015 '''</br><br />
* [[Comparing feature selection methods for highdimensional imbalanced data: identifying rheumatoid arthritis cohorts from routine data]]<br />
<br />
=== Imputation methods for missing values ===<br />
<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 1996 || Schenker || [[Partially parametric techniques for multiple imputation]]<br />
|-<br />
| 1999 || Hastie T || [[Imputing Missing Data for Gene Expression Arrays]]<br />
|-<br />
| 2001 || Troyanskaya || [[Missing value estimation methods for DNA microarrays]]<br />
|-<br />
| 2002 || Engels J || [[Imputation of missing longitudinal data: a comparison of methods]]<br />
|-<br />
| 2003 || Oba || [[A Bayesian missing value estimation method for gene expression profile data]]<br />
|-<br />
| 2005 || Scholz || [[Nonlinear PCA: a missing data approach]]<br />
|-<br />
| 2007 || Stacklies || [[pcaMethods—a bioconductor package providing PCA methods for incomplete data]]<br />
|-<br />
| 2007 || Verboven || [[Sequential imputation for missing values]]<br />
|-<br />
| 2008 || Shaffer GN || [[Which missing value imputation method to use in expression profiles: a comparative study and two selection schemes]]<br />
|-<br />
| 2011 || Templ || [[Iterative stepwise regression imputation using standard and robust methods]]<br />
|-<br />
| 2012 || Hrydziuszko O || [[Missing values in mass spectrometry based metabolomics: an undervalued step in the data processing pipeline]]<br />
|-<br />
| 2012 || Stekhoven || [[MissForest—non-parametric missing value imputation for mixed-type data]]<br />
|-<br />
| 2013 || Taylor || [[Accounting for undetected compounds in statistical analyses of mass spectrometry ‘omic studies]]<br />
|-<br />
| 2013 || Waljee || [[Comparison of imputation methods for missing laboratory data in medicine]]<br />
|-<br />
| 2014 || Shah || [[Comparison of Random Forest and Parametric Imputation Models for Imputing Missing Data Using MICE: A CALIBER Study]]<br />
|-<br />
| 2014 || Rodwell || [[Comparison of methods for imputing limited-range variables: a simulation study]]<br />
|-<br />
| 2014 || Morris || [[Tuning multiple imputation by predictive mean matching and local residual draws]]<br />
|-<br />
| 2014 || Doove L || [[Recursive partitioning for missing data imputation in the presence of interaction effects]]<br />
|-<br />
| 2015 || Webb-Robertson BJM || [[Review, evaluation, and discussion of the challenges of missing value imputation for mass spectrometry-based label-free global proteomics]]<br />
|-<br />
| 2016 || Folch-Fortuny A || [[Assessment of maximum likelihood PCA missing data imputation]]<br />
|-<br />
| 2016 || Lazar C || [[Accounting for the Multiple Natures of Missing Values in Label-Free Quantitative Proteomics Data Sets to Compare Imputation Strategies]]<br />
|-<br />
| 2016 || Yin X || [[Multiple imputation and analysis for high-dimensional incomplete proteomics data]]<br />
|-<br />
| 2018 || Wei R || [[Missing Value Imputation Approach for Mass Spectrometry-based Metabolomics Data]]<br />
|-<br />
| 2018 || Poyatos R || [[Gap-filling a spatially explicit plant trait database: comparing imputation methods and different levels of environmental information]]<br />
|-<br />
| 2018 || O'Brien JJ || [[The effects of nonignorable missing data on label-free mass spectrometry proteomics experiments]]<br />
|}<br />
<br />
=== ODE-based Modelling ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2001 || Beal || [[Ways to Fit a PK Model with Some Data Below the Quantification Limit]]<br />
|-<br />
| 2008 || Balsa-Canto || [[Hybrid optimization method with general switching strategy for parameter estimation]]<br />
|-<br />
| 2011 || Tashkova || [[Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis]]<br />
|-<br />
| 2013 || Raue || [[Lessons Learned from Quantitative Dynamical Modeling in Systems Biology]]<br />
|-<br />
| 2013 || Dondelinger || [[ODE parameter inference using adaptive gradient matching with Gaussian processes]]<br />
|-<br />
| 2017 || Ballnus || [[Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems]]<br />
|-<br />
| 2017 || Henriques || [[Data-driven reverse engineering of signaling pathways using ensembles of dynamic models]]<br />
|-<br />
| 2017 || Melicher || [[Fast derivatives of likelihood functionals for ODE based models using adjoint-state method]]<br />
|-<br />
| 2017 || Penas || [[Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy]]<br />
|-<br />
| 2017 || Degasperi || [[Performance of objective functions and optimization procedures for parameter estimation in system biology models]]<br />
|-<br />
| 2017 || Fröhlich || [[Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks]]<br />
|-<br />
| 2018 || Schälte || [[Evaluation of Derivative-Free Optimizers for Parameter Estimation in Systems Biology]]<br />
|-<br />
| 2018 || Loos || [[Hierarchical optimization for the efficient parametrization of ODE models]]<br />
|-<br />
| 2018 || Stapor || [[Optimization and profile calculation of ODE models using second order adjoint sensitivity analysis]]<br />
|-<br />
| 2019 || Villaverde || [[A comparison of methods for quantifying prediction uncertainty in systems biology]]<br />
|-<br />
| 2019 || Hass || [[Benchmark problems for dynamic modeling of intracellular processes]]<br />
|-<br />
| 2019 || Villaverde || [[Benchmarking optimization methods for parameter estimation in large kinetic models]]<br />
|-<br />
| 2019 || Lines || [[Efficient computation of steady states in large-scale ODE models of biochemical reaction networks]]<br />
|-<br />
| 2019 || Stapor || [[Mini-batch optimization enables training of ODE models on large-scale datasets]]<br />
|-<br />
| 2019 || Wu || [[Parameter Estimation and Variable Selection for Big Systems of Linear Ordinary Differential Equations: A Matrix-Based Approach]]<br />
|-<br />
| 2019 || Pitt || [[Parameter estimation in models of biological oscillators: an automated regularised estimation approach]]<br />
|-<br />
| 2019 || Loos || [[Robust calibration of hierarchical population models for heterogeneous cell populations]]<br />
|-<br />
| 2019 || Clairon || [[Tracking for parameter and state estimation in possibly misspecified partially observed linear Ordinary Differential Equations]]<br />
|-<br />
| 2020 || Schmiester || [[Efficient parameterization of large-scale dynamic models based on relative measurements]]<br />
|-<br />
| 2020 || Castro || [[Testing structural identifiability by a simple scaling method]]<br />
|}<br />
<br />
=== Omics Workflows ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2008 || Neuweger H || [[MeltDB: a software platform for the analysis and integration of metabolomics experiment data]]<br />
|-<br />
| 2008 || Barla A || [[Machine learning methods for predictive proteomics]]<br />
|-<br />
| 2009 || Xia J || [[MetaboAnalyst: a web server for metabolomic data analysis and interpretation]]<br />
|-<br />
| 2013 || Weisser H || [[An Automated Pipeline for High-Throughput Label-Free Quantitative Proteomics]]<br />
|-<br />
| 2014 || Cox J || [[Accurate Proteome-wide Label-free Quantification by Delayed Normalization and Maximal Peptide Ratio Extraction, Termed MaxLFQ* ]]<br />
|-<br />
| 2015 || || [[ComparingVariant Call Files for Performance Benchmarkingof Next-Generation Sequencing Variant Calling Pipelines]]<br />
|-<br />
| 2016 || Tyanova S || [[The MaxQuant computational platform for mass spectrometry–based shotgun proteomics]]<br />
|-<br />
| 2016 || Röst HL || [[OpenMS: a flexible open-source software platform for mass spectrometry data analysis]]<br />
|-<br />
| 2017 || || [[A benchmarking of workflows for detecting differential splicing and differential expression at isoform level in human RNA-seq studies]]<br />
|-<br />
| 2018 || Välikangas T || [[A comprehensive evaluation of popular proteomics software workflows for label-free proteome quantification and imputation]]<br />
|-<br />
| 2019 || || [[A Systematic Evaluation of Single CellRNA-Seq Analysis Pipelines]]<br />
|-<br />
| 2019 || || [[Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays]]<br />
|}<br />
<br />
=== Preprocessing high-throughput data===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|- 1999 || Perkins DN || [[Probability-based protein identification by searching sequence databases using mass spectrometry data]]<br />
|-<br />
| 2003 || || [[A comparison of normalization methods for high density oligonucleotide array data based on variance and bias ]]<br />
|-<br />
| 2005 || || [[Comparison of Affymetrix GeneChip Expression Measures]]<br />
|-<br />
| 2005 || Meleth S || [[The case for well-conducted experiments to validate statistical protocols for 2D gels: different pre-processing = different lists of significant proteins]]<br />
|-<br />
| 2005 || || [[Comparison of background correction and normalization procedures for high-density oligonucleotide microarrays]]<br />
|-<br />
| 2006 || || [[Using RNA sample titrations to assess microarray platform performance and normalization techniques]]<br />
|-<br />
| 2006 || Wang P || [[Normalization regarding non-random missing values in high-throughput mass spectrometry data]]<br />
|-<br />
| 2007 || Carvalho B || [[Exploration, normalization, and genotype calls of high-density oligonucleotide SNP array data]]<br />
|-<br />
| 2007 || Cannataro M || [[MS‐Analyzer: preprocessing and data mining services for proteomics applications on the Grid]]<br />
|-<br />
| 2008 || || [[Comparison of preprocessing methods for the hgU133+2 chip from Affymetrix]]<br />
|-<br />
| 2009 || || [[Comparison of Affymetrix data normalization methods using 6,926 experiments across five array generations]]<br />
|-<br />
| 2009 || Mar JC || [[Data-driven normalization strategies for high-throughput quantitative RT-PCR]]<br />
|-<br />
| 2009 || Vakhrushev SY || [[Software platform for high-throughput glycomics]]<br />
|-<br />
| 2010 || || [[Consistency of predictive signature genes and classifiers generated using different microarray platforms]]<br />
|-<br />
| 2010 || || [[Detecting and correcting systematic variation in large-scale RNA sequencing data]]<br />
|-<br />
| 2010 || || [[Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments]]<br />
|-<br />
| 2010 || || [[Normalization of RNA-seq data using factor analysis of control genes or samples]]<br />
|-<br />
| 2010 || Armananzas R || [[Peakbin selection in mass spectrometry data using a consensus approach with estimation of distribution algorithms]]<br />
|-<br />
| 2011 || || [[Affymetrix GeneChip microarray preprocessing for multivariate analyses]]<br />
|-<br />
| 2012 || || [[A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis]]<br />
|-<br />
| 2014 || || [[Normalyzer: A Tool for Rapid Evaluation of Normalization Methods for Omics Data Sets]]<br />
|-<br />
| 2014 || Zhou X || [[Prevention, diagnosis and treatment of high-throughput sequencing data pathologies]]<br />
|-<br />
| 2014 || Coble JB || [[Comparative evaluation of preprocessing freeware on chromatography/mass spectrometry data for signature discovery]]<br />
|-<br />
| 2015 || Caraus I || [[Detecting and overcoming systematic bias in high-throughput screening technologies: a comprehensive review of practical issues and methodological solutions]]<br />
|-<br />
| 2015 || Tam S || [[Optimization of miRNA-seq data preprocessing]]<br />
|-<br />
| 2015 || Rafiei A || [[Comparison of peak‐picking workflows for untargeted liquid chromatography/high‐resolution mass spectrometry metabolomics data analysis]]<br />
|-<br />
| 2015 || Chawade A || [[Data processing has major impact on the outcome of quantitative label-free LC-MS analysis]]<br />
|-<br />
| 2015 || Wang T || [[A systematic study of normalization methods for Infinium 450K methylation data using whole-genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Yi L || [[Chemometric methods in data processing of mass spectrometry-based metabolomics: A review]]<br />
|-<br />
| 2016 || Tsuji J || [[Evaluation of preprocessing, mapping and postprocessing algorithms for analyzing whole genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Li B || [[Performance Evaluation and Online Realization of Data-driven Normalization Methods Used in LC/MS based Untargeted Metabolomics Analysis]]<br />
|-<br />
| 2017 || Li B || [[NOREVA: normalization and evaluation of MS-based metabolomics data]]<br />
|-<br />
| 2018 || Mazoure B || [[Identification and Correction of Additive and Multiplicative Spatial Biases in Experimental High-Throughput Screening]]<br />
|-<br />
| 2018 || Li Z || [[Comprehensive evaluation of untargeted metabolomics data processing software in feature detection, quantification and discriminating marker selection]]<br />
|-<br />
| 2018 || Willforss J || [[NormalyzerDE: Online Tool for Improved Normalization of Omics Expression Data and High-Sensitivity Differential Expression Analysis]]<br />
|}</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Literature_Studies&diff=711Literature Studies2020-02-25T15:51:33Z<p>Bwday: /* ODE-based Modelling */</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
{| class="wikitable"<br />
|-<br />
! Page summary<br />
|-<br />
| Here outcomes of benchmarking studies from the literature are collected. The primary aim is a comprehensive overview about neutral benchmark studies, i.e. assessments which were performed independenty on publication of a new approach. Studies which are not neutral are put in brackets. </br> <br />
<br />
The focus is on computational methods for analyzing experimental data (instead of comparing experimental techniques or platforms). </br><br />
<br />
Please extend this list by creating a new page and adding a link below. </br> <br />
Use the '''[[Guidelines_for_Summarizing_a_Literature_Study|guidelines described here]]'''.<br />
|}<br />
<br />
== Results from Literature ==<br />
<br />
=== Classification ===<br />
''' 2003 '''</br><br />
* [[Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data]]<br />
''' 2005 '''</br><br />
* [[A review and comparison of classification algorithms for medical decision making]]<br />
''' 2016 '''</br><br />
* [[Predicting Breast Cancer Survivability Using Data Mining Techniques]]<br />
<br />
=== Selection of Differential Features and Regions ===<br />
==== Identifying differential features ====<br />
''' 2006 '''</br><br />
* [[Rat toxicogenomic study reveals analytical consistency across microarray platforms]]<br />
''' 2010 '''</br><br />
* [[A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the sequencing Quality control consortium]]<br />
''' 2017 '''</br><br />
* [[Identification of differentially expressed peptides in high-throughput proteomics data]]<br />
* [[In-depth method assessments of differentially expressed protein detection for shotgun proteomics data with missing values]]<br />
* [[Strategies for analyzing bisulfite sequencing data]]<br />
''' 2018 '''</br><br />
* [[Identification of Differentially Methylated Sites with Weak Methylation Effects]]<br />
<br />
==== Identifying differential regions (e.g. DMRs) ====<br />
''' 2015 '''<br />
* [[De novo identification of differentially methylated regions in the human genome]]<br />
* [[MethylAction: detecting differentially methylated regions that distinguish biological subtypes]]<br />
* [[metilene: Fast and sensitive calling of differentially methylated regions from bisulfite sequencing data]]<br />
''' 2016 '''<br />
* [[seqlm: an MDL based method for identifying differentially methylated regions in high density methylation array data]]<br />
* [[Statistical methods for detecting differentially methylated regions based on MethylCap-seq data]]<br />
''' 2017 '''<br />
* [[DMRfinder: efficiently identifying differentially methylated regions from MethylC-seq data]]<br />
''' 2018 '''<br />
* [[Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus]]<br />
* [[DMRcaller: a versatile R/Bioconductor package for detection and visualization of differentially methylated regions in CpG and non-CpG contexts]]<br />
* [[MethCP: Differentially Methylated Region Detection with Change Point Models (bioRxiv)]]<br />
<br />
==== Identifying sets of features (e.g. gene set analyses) ====<br />
'''2009<br />
<br />
* [[A general modular framework for gene set enrichment analysis]]<br />
* [[Comparing gene set analysis methods on single-nucleotide polymorphism data from Genetic Analysis Workshop 16]]<br />
<br />
'''2018<br />
<br />
* [[Gene set analysis methods: a systematic comparison]]<br />
<br />
'''2020<br />
* [[Toward a gold standard for benchmarking gene set enrichment analysis]]<br />
<br />
==== Dimension reduction ====<br />
''' 2008 '''</br><br />
* [[On the Relationship Between Feature Selection and Classification Accuracy]]<br />
''' 2015 '''</br><br />
* [[Comparing feature selection methods for highdimensional imbalanced data: identifying rheumatoid arthritis cohorts from routine data]]<br />
<br />
=== Imputation methods for missing values ===<br />
<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 1996 || Schenker || [[Partially parametric techniques for multiple imputation]]<br />
|-<br />
| 1999 || Hastie T || [[Imputing Missing Data for Gene Expression Arrays]]<br />
|-<br />
| 2001 || Troyanskaya || [[Missing value estimation methods for DNA microarrays]]<br />
|-<br />
| 2002 || Engels J || [[Imputation of missing longitudinal data: a comparison of methods]]<br />
|-<br />
| 2003 || Oba || [[A Bayesian missing value estimation method for gene expression profile data]]<br />
|-<br />
| 2005 || Scholz || [[Nonlinear PCA: a missing data approach]]<br />
|-<br />
| 2007 || Stacklies || [[pcaMethods—a bioconductor package providing PCA methods for incomplete data]]<br />
|-<br />
| 2007 || Verboven || [[Sequential imputation for missing values]]<br />
|-<br />
| 2008 || Shaffer GN || [[Which missing value imputation method to use in expression profiles: a comparative study and two selection schemes]]<br />
|-<br />
| 2011 || Templ || [[Iterative stepwise regression imputation using standard and robust methods]]<br />
|-<br />
| 2012 || Hrydziuszko O || [[Missing values in mass spectrometry based metabolomics: an undervalued step in the data processing pipeline]]<br />
|-<br />
| 2012 || Stekhoven || [[MissForest—non-parametric missing value imputation for mixed-type data]]<br />
|-<br />
| 2013 || Taylor || [[Accounting for undetected compounds in statistical analyses of mass spectrometry ‘omic studies]]<br />
|-<br />
| 2013 || Waljee || [[Comparison of imputation methods for missing laboratory data in medicine]]<br />
|-<br />
| 2014 || Shah || [[Comparison of Random Forest and Parametric Imputation Models for Imputing Missing Data Using MICE: A CALIBER Study]]<br />
|-<br />
| 2014 || Rodwell || [[Comparison of methods for imputing limited-range variables: a simulation study]]<br />
|-<br />
| 2014 || Morris || [[Tuning multiple imputation by predictive mean matching and local residual draws]]<br />
|-<br />
| 2014 || Doove L || [[Recursive partitioning for missing data imputation in the presence of interaction effects]]<br />
|-<br />
| 2015 || Webb-Robertson BJM || [[Review, evaluation, and discussion of the challenges of missing value imputation for mass spectrometry-based label-free global proteomics]]<br />
|-<br />
| 2016 || Folch-Fortuny A || [[Assessment of maximum likelihood PCA missing data imputation]]<br />
|-<br />
| 2016 || Lazar C || [[Accounting for the Multiple Natures of Missing Values in Label-Free Quantitative Proteomics Data Sets to Compare Imputation Strategies]]<br />
|-<br />
| 2016 || Yin X || [[Multiple imputation and analysis for high-dimensional incomplete proteomics data]]<br />
|-<br />
| 2018 || Wei R || [[Missing Value Imputation Approach for Mass Spectrometry-based Metabolomics Data]]<br />
|-<br />
| 2018 || Poyatos R || [[Gap-filling a spatially explicit plant trait database: comparing imputation methods and different levels of environmental information]]<br />
|-<br />
| 2018 || O'Brien JJ || [[The effects of nonignorable missing data on label-free mass spectrometry proteomics experiments]]<br />
|}<br />
<br />
=== ODE-based Modelling ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2001 || Beal || [[Ways to Fit a PK Model with Some Data Below the Quantification Limit]]<br />
|-<br />
| 2008 || Balsa-Canto || [[Hybrid optimization method with general switching strategy for parameter estimation]]<br />
|-<br />
| 2011 || Tashkova || [[Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis]]<br />
|-<br />
| 2013 || Raue || [[Lessons Learned from Quantitative Dynamical Modeling in Systems Biology]]<br />
|-<br />
| 2013 || Dondelinger || [[ODE parameter inference using adaptive gradient matching with Gaussian processes]]<br />
|-<br />
| 2017 || Ballnus || [[Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems]]<br />
|-<br />
| 2017 || Henriques || [[Data-driven reverse engineering of signaling pathways using ensembles of dynamic models]]<br />
|-<br />
| 2017 || Melicher || [[Fast derivatives of likelihood functionals for ODE based models using adjoint-state method]]<br />
|-<br />
| 2017 || Penas || [[Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy]]<br />
|-<br />
| 2017 || Degasperi || [[Performance of objective functions and optimization procedures for parameter estimation in system biology models]]<br />
|-<br />
| 2017 || Fröhlich || [[Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks]]<br />
|-<br />
| 2018 || Schälte || [[Evaluation of Derivative-Free Optimizers for Parameter Estimation in Systems Biology]]<br />
|-<br />
| 2018 || Loos || [[Hierarchical optimization for the efficient parametrization of ODE models]]<br />
|-<br />
| 2018 || Stapor || [[Optimization and profile calculation of ODE models using second order adjoint sensitivity analysis]]<br />
|-<br />
| 2019 || Villaverde || [[A comparison of methods for quantifying prediction uncertainty in systems biology]]<br />
|-<br />
| 2019 || Hass || [[Benchmark problems for dynamic modeling of intracellular processes]]<br />
|-<br />
| 2019 || Villaverde || [[Benchmarking optimization methods for parameter estimation in large kinetic models]]<br />
|-<br />
| 2019 || Lines || [[Efficient computation of steady states in large-scale ODE models of biochemical reaction networks]]<br />
|-<br />
| 2019 || Stapor || [[Mini-batch optimization enables training of ODE models on large-scale datasets]]<br />
|-<br />
| 2019 || Wu || [[Parameter Estimation and Variable Selection for Big Systems of Linear Ordinary Differential Equations: A Matrix-Based Approach]]<br />
|-<br />
| 2019 || Pitt || [[Parameter estimation in models of biological oscillators: an automated regularised estimation approach]]<br />
|-<br />
| 2019 || Loos || [[Robust calibration of hierarchical population models for heterogeneous cell populations]]<br />
|-<br />
| 2019 || Clairon || [[Tracking for parameter and state estimation in possibly misspecified partially observed linear Ordinary Differential Equations]]<br />
|-<br />
| 2020 || Schmiester || [[Efficient parameterization of large-scale dynamic models based on relative measurements]]<br />
|-<br />
| 2020 || Castro || [[Testing structural identifiability by a simple scaling method]]<br />
|}<br />
<br />
=== Omics Workflows ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2008 || Neuweger H || [[MeltDB: a software platform for the analysis and integration of metabolomics experiment data]]<br />
|-<br />
| 2008 || Barla A || [[Machine learning methods for predictive proteomics]]<br />
|-<br />
| 2009 || Xia J || [[MetaboAnalyst: a web server for metabolomic data analysis and interpretation]]<br />
|-<br />
| 2013 || Weisser H || [[An Automated Pipeline for High-Throughput Label-Free Quantitative Proteomics]]<br />
|-<br />
| 2014 || Cox J || [[Accurate Proteome-wide Label-free Quantification by Delayed Normalization and Maximal Peptide Ratio Extraction, Termed MaxLFQ* ]]<br />
|-<br />
| 2015 || || [[ComparingVariant Call Files for Performance Benchmarkingof Next-Generation Sequencing Variant Calling Pipelines]]<br />
|-<br />
| 2016 || Tyanova S || [[The MaxQuant computational platform for mass spectrometry–based shotgun proteomics]]<br />
|-<br />
| 2016 || Röst HL || [[OpenMS: a flexible open-source software platform for mass spectrometry data analysis]]<br />
|-<br />
| 2017 || || [[A benchmarking of workflows for detecting differential splicing and differential expression at isoform level in human RNA-seq studies]]<br />
|-<br />
| 2018 || Välikangas T || [[A comprehensive evaluation of popular proteomics software workflows for label-free proteome quantification and imputation]]<br />
|-<br />
| 2019 || || [[A Systematic Evaluation of Single CellRNA-Seq Analysis Pipelines]]<br />
|-<br />
| 2019 || || [[Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays]]<br />
|}<br />
<br />
=== Preprocessing high-throughput data===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|- 1999 || Perkins DN || [[Probability-based protein identification by searching sequence databases using mass spectrometry data]]<br />
|-<br />
| 2003 || || [[A comparison of normalization methods for high density oligonucleotide array data based on variance and bias ]]<br />
|-<br />
| 2005 || || [[Comparison of Affymetrix GeneChip Expression Measures]]<br />
|-<br />
| 2005 || Meleth S || [[The case for well-conducted experiments to validate statistical protocols for 2D gels: different pre-processing = different lists of significant proteins]]<br />
|-<br />
| 2005 || || [[Comparison of background correction and normalization procedures for high-density oligonucleotide microarrays]]<br />
|-<br />
| 2006 || || [[Using RNA sample titrations to assess microarray platform performance and normalization techniques]]<br />
|-<br />
| 2006 || Wang P || [[Normalization regarding non-random missing values in high-throughput mass spectrometry data]]<br />
|-<br />
| 2007 || Carvalho B || [[Exploration, normalization, and genotype calls of high-density oligonucleotide SNP array data]]<br />
|-<br />
| 2007 || Cannataro M || [[MS‐Analyzer: preprocessing and data mining services for proteomics applications on the Grid]]<br />
|-<br />
| 2008 || || [[Comparison of preprocessing methods for the hgU133+2 chip from Affymetrix]]<br />
|-<br />
| 2009 || || [[Comparison of Affymetrix data normalization methods using 6,926 experiments across five array generations]]<br />
|-<br />
| 2009 || Mar JC || [[Data-driven normalization strategies for high-throughput quantitative RT-PCR]]<br />
|-<br />
| 2009 || Vakhrushev SY || [[Software platform for high-throughput glycomics]]<br />
|-<br />
| 2010 || || [[Consistency of predictive signature genes and classifiers generated using different microarray platforms]]<br />
|-<br />
| 2010 || || [[Detecting and correcting systematic variation in large-scale RNA sequencing data]]<br />
|-<br />
| 2010 || || [[Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments]]<br />
|-<br />
| 2010 || || [[Normalization of RNA-seq data using factor analysis of control genes or samples]]<br />
|-<br />
| 2011 || || [[Affymetrix GeneChip microarray preprocessing for multivariate analyses]]<br />
|-<br />
| 2012 || || [[A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis]]<br />
|-<br />
| 2014 || || [[Normalyzer: A Tool for Rapid Evaluation of Normalization Methods for Omics Data Sets]]<br />
|-<br />
| 2014 || Zhou X || [[Prevention, diagnosis and treatment of high-throughput sequencing data pathologies]]<br />
|-<br />
| 2014 || Coble JB || [[Comparative evaluation of preprocessing freeware on chromatography/mass spectrometry data for signature discovery]]<br />
|-<br />
| 2015 || Caraus I || [[Detecting and overcoming systematic bias in high-throughput screening technologies: a comprehensive review of practical issues and methodological solutions]]<br />
|-<br />
| 2015 || Tam S || [[Optimization of miRNA-seq data preprocessing]]<br />
|-<br />
| 2015 || Rafiei A || [[Comparison of peak‐picking workflows for untargeted liquid chromatography/high‐resolution mass spectrometry metabolomics data analysis]]<br />
|-<br />
| 2015 || Chawade A || [[Data processing has major impact on the outcome of quantitative label-free LC-MS analysis]]<br />
|-<br />
| 2015 || Wang T || [[A systematic study of normalization methods for Infinium 450K methylation data using whole-genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Yi L || [[Chemometric methods in data processing of mass spectrometry-based metabolomics: A review]]<br />
|-<br />
| 2016 || Tsuji J || [[Evaluation of preprocessing, mapping and postprocessing algorithms for analyzing whole genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Li B || [[Performance Evaluation and Online Realization of Data-driven Normalization Methods Used in LC/MS based Untargeted Metabolomics Analysis]]<br />
|-<br />
| 2017 || Li B || [[NOREVA: normalization and evaluation of MS-based metabolomics data]]<br />
|-<br />
| 2018 || Mazoure B || [[Identification and Correction of Additive and Multiplicative Spatial Biases in Experimental High-Throughput Screening]]<br />
|-<br />
| 2018 || Li Z || [[Comprehensive evaluation of untargeted metabolomics data processing software in feature detection, quantification and discriminating marker selection]]<br />
|-<br />
| 2018 || Willforss J || [[NormalyzerDE: Online Tool for Improved Normalization of Omics Expression Data and High-Sensitivity Differential Expression Analysis]]<br />
|}</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Literature_Studies&diff=710Literature Studies2020-02-25T15:51:10Z<p>Bwday: /* ODE-based Modelling */</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
{| class="wikitable"<br />
|-<br />
! Page summary<br />
|-<br />
| Here outcomes of benchmarking studies from the literature are collected. The primary aim is a comprehensive overview about neutral benchmark studies, i.e. assessments which were performed independenty on publication of a new approach. Studies which are not neutral are put in brackets. </br> <br />
<br />
The focus is on computational methods for analyzing experimental data (instead of comparing experimental techniques or platforms). </br><br />
<br />
Please extend this list by creating a new page and adding a link below. </br> <br />
Use the '''[[Guidelines_for_Summarizing_a_Literature_Study|guidelines described here]]'''.<br />
|}<br />
<br />
== Results from Literature ==<br />
<br />
=== Classification ===<br />
''' 2003 '''</br><br />
* [[Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data]]<br />
''' 2005 '''</br><br />
* [[A review and comparison of classification algorithms for medical decision making]]<br />
''' 2016 '''</br><br />
* [[Predicting Breast Cancer Survivability Using Data Mining Techniques]]<br />
<br />
=== Selection of Differential Features and Regions ===<br />
==== Identifying differential features ====<br />
''' 2006 '''</br><br />
* [[Rat toxicogenomic study reveals analytical consistency across microarray platforms]]<br />
''' 2010 '''</br><br />
* [[A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the sequencing Quality control consortium]]<br />
''' 2017 '''</br><br />
* [[Identification of differentially expressed peptides in high-throughput proteomics data]]<br />
* [[In-depth method assessments of differentially expressed protein detection for shotgun proteomics data with missing values]]<br />
* [[Strategies for analyzing bisulfite sequencing data]]<br />
''' 2018 '''</br><br />
* [[Identification of Differentially Methylated Sites with Weak Methylation Effects]]<br />
<br />
==== Identifying differential regions (e.g. DMRs) ====<br />
''' 2015 '''<br />
* [[De novo identification of differentially methylated regions in the human genome]]<br />
* [[MethylAction: detecting differentially methylated regions that distinguish biological subtypes]]<br />
* [[metilene: Fast and sensitive calling of differentially methylated regions from bisulfite sequencing data]]<br />
''' 2016 '''<br />
* [[seqlm: an MDL based method for identifying differentially methylated regions in high density methylation array data]]<br />
* [[Statistical methods for detecting differentially methylated regions based on MethylCap-seq data]]<br />
''' 2017 '''<br />
* [[DMRfinder: efficiently identifying differentially methylated regions from MethylC-seq data]]<br />
''' 2018 '''<br />
* [[Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus]]<br />
* [[DMRcaller: a versatile R/Bioconductor package for detection and visualization of differentially methylated regions in CpG and non-CpG contexts]]<br />
* [[MethCP: Differentially Methylated Region Detection with Change Point Models (bioRxiv)]]<br />
<br />
==== Identifying sets of features (e.g. gene set analyses) ====<br />
'''2009<br />
<br />
* [[A general modular framework for gene set enrichment analysis]]<br />
* [[Comparing gene set analysis methods on single-nucleotide polymorphism data from Genetic Analysis Workshop 16]]<br />
<br />
'''2018<br />
<br />
* [[Gene set analysis methods: a systematic comparison]]<br />
<br />
'''2020<br />
* [[Toward a gold standard for benchmarking gene set enrichment analysis]]<br />
<br />
==== Dimension reduction ====<br />
''' 2008 '''</br><br />
* [[On the Relationship Between Feature Selection and Classification Accuracy]]<br />
''' 2015 '''</br><br />
* [[Comparing feature selection methods for highdimensional imbalanced data: identifying rheumatoid arthritis cohorts from routine data]]<br />
<br />
=== Imputation methods for missing values ===<br />
<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 1996 || Schenker || [[Partially parametric techniques for multiple imputation]]<br />
|-<br />
| 1999 || Hastie T || [[Imputing Missing Data for Gene Expression Arrays]]<br />
|-<br />
| 2001 || Troyanskaya || [[Missing value estimation methods for DNA microarrays]]<br />
|-<br />
| 2002 || Engels J || [[Imputation of missing longitudinal data: a comparison of methods]]<br />
|-<br />
| 2003 || Oba || [[A Bayesian missing value estimation method for gene expression profile data]]<br />
|-<br />
| 2005 || Scholz || [[Nonlinear PCA: a missing data approach]]<br />
|-<br />
| 2007 || Stacklies || [[pcaMethods—a bioconductor package providing PCA methods for incomplete data]]<br />
|-<br />
| 2007 || Verboven || [[Sequential imputation for missing values]]<br />
|-<br />
| 2008 || Shaffer GN || [[Which missing value imputation method to use in expression profiles: a comparative study and two selection schemes]]<br />
|-<br />
| 2011 || Templ || [[Iterative stepwise regression imputation using standard and robust methods]]<br />
|-<br />
| 2012 || Hrydziuszko O || [[Missing values in mass spectrometry based metabolomics: an undervalued step in the data processing pipeline]]<br />
|-<br />
| 2012 || Stekhoven || [[MissForest—non-parametric missing value imputation for mixed-type data]]<br />
|-<br />
| 2013 || Taylor || [[Accounting for undetected compounds in statistical analyses of mass spectrometry ‘omic studies]]<br />
|-<br />
| 2013 || Waljee || [[Comparison of imputation methods for missing laboratory data in medicine]]<br />
|-<br />
| 2014 || Shah || [[Comparison of Random Forest and Parametric Imputation Models for Imputing Missing Data Using MICE: A CALIBER Study]]<br />
|-<br />
| 2014 || Rodwell || [[Comparison of methods for imputing limited-range variables: a simulation study]]<br />
|-<br />
| 2014 || Morris || [[Tuning multiple imputation by predictive mean matching and local residual draws]]<br />
|-<br />
| 2014 || Doove L || [[Recursive partitioning for missing data imputation in the presence of interaction effects]]<br />
|-<br />
| 2015 || Webb-Robertson BJM || [[Review, evaluation, and discussion of the challenges of missing value imputation for mass spectrometry-based label-free global proteomics]]<br />
|-<br />
| 2016 || Folch-Fortuny A || [[Assessment of maximum likelihood PCA missing data imputation]]<br />
|-<br />
| 2016 || Lazar C || [[Accounting for the Multiple Natures of Missing Values in Label-Free Quantitative Proteomics Data Sets to Compare Imputation Strategies]]<br />
|-<br />
| 2016 || Yin X || [[Multiple imputation and analysis for high-dimensional incomplete proteomics data]]<br />
|-<br />
| 2018 || Wei R || [[Missing Value Imputation Approach for Mass Spectrometry-based Metabolomics Data]]<br />
|-<br />
| 2018 || Poyatos R || [[Gap-filling a spatially explicit plant trait database: comparing imputation methods and different levels of environmental information]]<br />
|-<br />
| 2018 || O'Brien JJ || [[The effects of nonignorable missing data on label-free mass spectrometry proteomics experiments]]<br />
|}<br />
<br />
=== ODE-based Modelling ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2001 || Beal || [[Ways to Fit a PK Model with Some Data Below the Quantification Limit]]<br />
|-<br />
| 2008 || Balsa-Canto || [[Hybrid optimization method with general switching strategy for parameter estimation]]<br />
|-<br />
| 2011 || Tashkova || [[Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis]]<br />
|-<br />
| 2013 || Raue || [[Lessons Learned from Quantitative Dynamical Modeling in Systems Biology]]<br />
|-<br />
| 2013 || Dondelinger || [[ODE parameter inference using adaptive gradient matching with Gaussian processes]]<br />
|-<br />
| 2017 || Ballnus || [[Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems]]<br />
|-<br />
| 2017 || Henriques || [[Data-driven reverse engineering of signaling pathways using ensembles of dynamic models]]<br />
|-<br />
| 2017 || Melicher || [[Fast derivatives of likelihood functionals for ODE based models using adjoint-state method]]<br />
|-<br />
| 2017 || Penas || [[Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy]]<br />
|-<br />
| 2017 || Degasperi || [[Performance of objective functions and optimization procedures for parameter estimation in system biology models]]<br />
|-<br />
| 2017 || Fröhlich || [[Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks]]<br />
|-<br />
| 2018 || Schälte || [[Evaluation of Derivative-Free Optimizers for Parameter Estimation in Systems Biology]]<br />
|-<br />
| 2018 || Loos || [[Hierarchical optimization for the efficient parametrization of ODE models]]<br />
|-<br />
| 2018 || Villaverde || [[Input-dependent structural identifiability of nonlinear systems]]<br />
|-<br />
| 2018 || Stapor || [[Optimization and profile calculation of ODE models using second order adjoint sensitivity analysis]]<br />
|-<br />
| 2019 || Villaverde || [[A comparison of methods for quantifying prediction uncertainty in systems biology]]<br />
|-<br />
| 2019 || Hass || [[Benchmark problems for dynamic modeling of intracellular processes]]<br />
|-<br />
| 2019 || Villaverde || [[Benchmarking optimization methods for parameter estimation in large kinetic models]]<br />
|-<br />
| 2019 || Lines || [[Efficient computation of steady states in large-scale ODE models of biochemical reaction networks]]<br />
|-<br />
| 2019 || Stapor || [[Mini-batch optimization enables training of ODE models on large-scale datasets]]<br />
|-<br />
| 2019 || Wu || [[Parameter Estimation and Variable Selection for Big Systems of Linear Ordinary Differential Equations: A Matrix-Based Approach]]<br />
|-<br />
| 2019 || Pitt || [[Parameter estimation in models of biological oscillators: an automated regularised estimation approach]]<br />
|-<br />
| 2019 || Loos || [[Robust calibration of hierarchical population models for heterogeneous cell populations]]<br />
|-<br />
| 2019 || Clairon || [[Tracking for parameter and state estimation in possibly misspecified partially observed linear Ordinary Differential Equations]]<br />
|-<br />
| 2020 || Schmiester || [[Efficient parameterization of large-scale dynamic models based on relative measurements]]<br />
|-<br />
| 2020 || Castro || [[Testing structural identifiability by a simple scaling method]]<br />
|}<br />
<br />
=== Omics Workflows ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2008 || Neuweger H || [[MeltDB: a software platform for the analysis and integration of metabolomics experiment data]]<br />
|-<br />
| 2008 || Barla A || [[Machine learning methods for predictive proteomics]]<br />
|-<br />
| 2009 || Xia J || [[MetaboAnalyst: a web server for metabolomic data analysis and interpretation]]<br />
|-<br />
| 2013 || Weisser H || [[An Automated Pipeline for High-Throughput Label-Free Quantitative Proteomics]]<br />
|-<br />
| 2014 || Cox J || [[Accurate Proteome-wide Label-free Quantification by Delayed Normalization and Maximal Peptide Ratio Extraction, Termed MaxLFQ* ]]<br />
|-<br />
| 2015 || || [[ComparingVariant Call Files for Performance Benchmarkingof Next-Generation Sequencing Variant Calling Pipelines]]<br />
|-<br />
| 2016 || Tyanova S || [[The MaxQuant computational platform for mass spectrometry–based shotgun proteomics]]<br />
|-<br />
| 2016 || Röst HL || [[OpenMS: a flexible open-source software platform for mass spectrometry data analysis]]<br />
|-<br />
| 2017 || || [[A benchmarking of workflows for detecting differential splicing and differential expression at isoform level in human RNA-seq studies]]<br />
|-<br />
| 2018 || Välikangas T || [[A comprehensive evaluation of popular proteomics software workflows for label-free proteome quantification and imputation]]<br />
|-<br />
| 2019 || || [[A Systematic Evaluation of Single CellRNA-Seq Analysis Pipelines]]<br />
|-<br />
| 2019 || || [[Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays]]<br />
|}<br />
<br />
=== Preprocessing high-throughput data===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|- 1999 || Perkins DN || [[Probability-based protein identification by searching sequence databases using mass spectrometry data]]<br />
|-<br />
| 2003 || || [[A comparison of normalization methods for high density oligonucleotide array data based on variance and bias ]]<br />
|-<br />
| 2005 || || [[Comparison of Affymetrix GeneChip Expression Measures]]<br />
|-<br />
| 2005 || Meleth S || [[The case for well-conducted experiments to validate statistical protocols for 2D gels: different pre-processing = different lists of significant proteins]]<br />
|-<br />
| 2005 || || [[Comparison of background correction and normalization procedures for high-density oligonucleotide microarrays]]<br />
|-<br />
| 2006 || || [[Using RNA sample titrations to assess microarray platform performance and normalization techniques]]<br />
|-<br />
| 2006 || Wang P || [[Normalization regarding non-random missing values in high-throughput mass spectrometry data]]<br />
|-<br />
| 2007 || Carvalho B || [[Exploration, normalization, and genotype calls of high-density oligonucleotide SNP array data]]<br />
|-<br />
| 2007 || Cannataro M || [[MS‐Analyzer: preprocessing and data mining services for proteomics applications on the Grid]]<br />
|-<br />
| 2008 || || [[Comparison of preprocessing methods for the hgU133+2 chip from Affymetrix]]<br />
|-<br />
| 2009 || || [[Comparison of Affymetrix data normalization methods using 6,926 experiments across five array generations]]<br />
|-<br />
| 2009 || Mar JC || [[Data-driven normalization strategies for high-throughput quantitative RT-PCR]]<br />
|-<br />
| 2009 || Vakhrushev SY || [[Software platform for high-throughput glycomics]]<br />
|-<br />
| 2010 || || [[Consistency of predictive signature genes and classifiers generated using different microarray platforms]]<br />
|-<br />
| 2010 || || [[Detecting and correcting systematic variation in large-scale RNA sequencing data]]<br />
|-<br />
| 2010 || || [[Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments]]<br />
|-<br />
| 2010 || || [[Normalization of RNA-seq data using factor analysis of control genes or samples]]<br />
|-<br />
| 2011 || || [[Affymetrix GeneChip microarray preprocessing for multivariate analyses]]<br />
|-<br />
| 2012 || || [[A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis]]<br />
|-<br />
| 2014 || || [[Normalyzer: A Tool for Rapid Evaluation of Normalization Methods for Omics Data Sets]]<br />
|-<br />
| 2014 || Zhou X || [[Prevention, diagnosis and treatment of high-throughput sequencing data pathologies]]<br />
|-<br />
| 2014 || Coble JB || [[Comparative evaluation of preprocessing freeware on chromatography/mass spectrometry data for signature discovery]]<br />
|-<br />
| 2015 || Caraus I || [[Detecting and overcoming systematic bias in high-throughput screening technologies: a comprehensive review of practical issues and methodological solutions]]<br />
|-<br />
| 2015 || Tam S || [[Optimization of miRNA-seq data preprocessing]]<br />
|-<br />
| 2015 || Rafiei A || [[Comparison of peak‐picking workflows for untargeted liquid chromatography/high‐resolution mass spectrometry metabolomics data analysis]]<br />
|-<br />
| 2015 || Chawade A || [[Data processing has major impact on the outcome of quantitative label-free LC-MS analysis]]<br />
|-<br />
| 2015 || Wang T || [[A systematic study of normalization methods for Infinium 450K methylation data using whole-genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Yi L || [[Chemometric methods in data processing of mass spectrometry-based metabolomics: A review]]<br />
|-<br />
| 2016 || Tsuji J || [[Evaluation of preprocessing, mapping and postprocessing algorithms for analyzing whole genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Li B || [[Performance Evaluation and Online Realization of Data-driven Normalization Methods Used in LC/MS based Untargeted Metabolomics Analysis]]<br />
|-<br />
| 2017 || Li B || [[NOREVA: normalization and evaluation of MS-based metabolomics data]]<br />
|-<br />
| 2018 || Mazoure B || [[Identification and Correction of Additive and Multiplicative Spatial Biases in Experimental High-Throughput Screening]]<br />
|-<br />
| 2018 || Li Z || [[Comprehensive evaluation of untargeted metabolomics data processing software in feature detection, quantification and discriminating marker selection]]<br />
|-<br />
| 2018 || Willforss J || [[NormalyzerDE: Online Tool for Improved Normalization of Omics Expression Data and High-Sensitivity Differential Expression Analysis]]<br />
|}</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Mini-batch_optimization_enables_training_of_ODE_models_on_large-scale_datasets&diff=709Mini-batch optimization enables training of ODE models on large-scale datasets2020-02-25T15:50:16Z<p>Bwday: </p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
=== Citation ===<br />
<br />
Mini-batch optimization enables training of ODE models on large-scale datasets, Paul Stapor, Leonard Schmiester, Christoph Wierling, Bodo M.H. Lange, Daniel Weindl, Jan Hasenauer, bioRxiv 859884.<br />
<br />
[https://doi.org/10.1101/859884 Permanent link to paper]<br />
<br />
=== Summary ===</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Mini-batch_optimization_enables_training_of_ODE_models_on_large-scale_datasets&diff=708Mini-batch optimization enables training of ODE models on large-scale datasets2020-02-25T15:49:52Z<p>Bwday: </p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
=== Citation ===<br />
<br />
Mini-batch optimization enables training of ODE models on large-scale datasets, Paul Stapor, Leonard Schmiester, Christoph Wierling, Bodo M.H. Lange, Daniel Weindl, Jan Hasenauer, bioRxiv 859884; <br />
<br />
[https://doi.org/10.1101/859884 Permanent link to paper]</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Literature_Studies&diff=707Literature Studies2020-02-25T15:48:26Z<p>Bwday: /* ODE-based Modelling */</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
{| class="wikitable"<br />
|-<br />
! Page summary<br />
|-<br />
| Here outcomes of benchmarking studies from the literature are collected. The primary aim is a comprehensive overview about neutral benchmark studies, i.e. assessments which were performed independenty on publication of a new approach. Studies which are not neutral are put in brackets. </br> <br />
<br />
The focus is on computational methods for analyzing experimental data (instead of comparing experimental techniques or platforms). </br><br />
<br />
Please extend this list by creating a new page and adding a link below. </br> <br />
Use the '''[[Guidelines_for_Summarizing_a_Literature_Study|guidelines described here]]'''.<br />
|}<br />
<br />
== Results from Literature ==<br />
<br />
=== Classification ===<br />
''' 2003 '''</br><br />
* [[Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data]]<br />
''' 2005 '''</br><br />
* [[A review and comparison of classification algorithms for medical decision making]]<br />
''' 2016 '''</br><br />
* [[Predicting Breast Cancer Survivability Using Data Mining Techniques]]<br />
<br />
=== Selection of Differential Features and Regions ===<br />
==== Identifying differential features ====<br />
''' 2006 '''</br><br />
* [[Rat toxicogenomic study reveals analytical consistency across microarray platforms]]<br />
''' 2010 '''</br><br />
* [[A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the sequencing Quality control consortium]]<br />
''' 2017 '''</br><br />
* [[Identification of differentially expressed peptides in high-throughput proteomics data]]<br />
* [[In-depth method assessments of differentially expressed protein detection for shotgun proteomics data with missing values]]<br />
* [[Strategies for analyzing bisulfite sequencing data]]<br />
''' 2018 '''</br><br />
* [[Identification of Differentially Methylated Sites with Weak Methylation Effects]]<br />
<br />
==== Identifying differential regions (e.g. DMRs) ====<br />
''' 2015 '''<br />
* [[De novo identification of differentially methylated regions in the human genome]]<br />
* [[MethylAction: detecting differentially methylated regions that distinguish biological subtypes]]<br />
* [[metilene: Fast and sensitive calling of differentially methylated regions from bisulfite sequencing data]]<br />
''' 2016 '''<br />
* [[seqlm: an MDL based method for identifying differentially methylated regions in high density methylation array data]]<br />
* [[Statistical methods for detecting differentially methylated regions based on MethylCap-seq data]]<br />
''' 2017 '''<br />
* [[DMRfinder: efficiently identifying differentially methylated regions from MethylC-seq data]]<br />
''' 2018 '''<br />
* [[Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus]]<br />
* [[DMRcaller: a versatile R/Bioconductor package for detection and visualization of differentially methylated regions in CpG and non-CpG contexts]]<br />
* [[MethCP: Differentially Methylated Region Detection with Change Point Models (bioRxiv)]]<br />
<br />
==== Identifying sets of features (e.g. gene set analyses) ====<br />
'''2009<br />
<br />
* [[A general modular framework for gene set enrichment analysis]]<br />
* [[Comparing gene set analysis methods on single-nucleotide polymorphism data from Genetic Analysis Workshop 16]]<br />
<br />
'''2018<br />
<br />
* [[Gene set analysis methods: a systematic comparison]]<br />
<br />
'''2020<br />
* [[Toward a gold standard for benchmarking gene set enrichment analysis]]<br />
<br />
==== Dimension reduction ====<br />
''' 2008 '''</br><br />
* [[On the Relationship Between Feature Selection and Classification Accuracy]]<br />
''' 2015 '''</br><br />
* [[Comparing feature selection methods for highdimensional imbalanced data: identifying rheumatoid arthritis cohorts from routine data]]<br />
<br />
=== Imputation methods for missing values ===<br />
<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 1996 || Schenker || [[Partially parametric techniques for multiple imputation]]<br />
|-<br />
| 1999 || Hastie T || [[Imputing Missing Data for Gene Expression Arrays]]<br />
|-<br />
| 2001 || Troyanskaya || [[Missing value estimation methods for DNA microarrays]]<br />
|-<br />
| 2002 || Engels J || [[Imputation of missing longitudinal data: a comparison of methods]]<br />
|-<br />
| 2003 || Oba || [[A Bayesian missing value estimation method for gene expression profile data]]<br />
|-<br />
| 2005 || Scholz || [[Nonlinear PCA: a missing data approach]]<br />
|-<br />
| 2007 || Stacklies || [[pcaMethods—a bioconductor package providing PCA methods for incomplete data]]<br />
|-<br />
| 2007 || Verboven || [[Sequential imputation for missing values]]<br />
|-<br />
| 2008 || Shaffer GN || [[Which missing value imputation method to use in expression profiles: a comparative study and two selection schemes]]<br />
|-<br />
| 2011 || Templ || [[Iterative stepwise regression imputation using standard and robust methods]]<br />
|-<br />
| 2012 || Hrydziuszko O || [[Missing values in mass spectrometry based metabolomics: an undervalued step in the data processing pipeline]]<br />
|-<br />
| 2012 || Stekhoven || [[MissForest—non-parametric missing value imputation for mixed-type data]]<br />
|-<br />
| 2013 || Taylor || [[Accounting for undetected compounds in statistical analyses of mass spectrometry ‘omic studies]]<br />
|-<br />
| 2013 || Waljee || [[Comparison of imputation methods for missing laboratory data in medicine]]<br />
|-<br />
| 2014 || Shah || [[Comparison of Random Forest and Parametric Imputation Models for Imputing Missing Data Using MICE: A CALIBER Study]]<br />
|-<br />
| 2014 || Rodwell || [[Comparison of methods for imputing limited-range variables: a simulation study]]<br />
|-<br />
| 2014 || Morris || [[Tuning multiple imputation by predictive mean matching and local residual draws]]<br />
|-<br />
| 2014 || Doove L || [[Recursive partitioning for missing data imputation in the presence of interaction effects]]<br />
|-<br />
| 2015 || Webb-Robertson BJM || [[Review, evaluation, and discussion of the challenges of missing value imputation for mass spectrometry-based label-free global proteomics]]<br />
|-<br />
| 2016 || Folch-Fortuny A || [[Assessment of maximum likelihood PCA missing data imputation]]<br />
|-<br />
| 2016 || Lazar C || [[Accounting for the Multiple Natures of Missing Values in Label-Free Quantitative Proteomics Data Sets to Compare Imputation Strategies]]<br />
|-<br />
| 2016 || Yin X || [[Multiple imputation and analysis for high-dimensional incomplete proteomics data]]<br />
|-<br />
| 2018 || Wei R || [[Missing Value Imputation Approach for Mass Spectrometry-based Metabolomics Data]]<br />
|-<br />
| 2018 || Poyatos R || [[Gap-filling a spatially explicit plant trait database: comparing imputation methods and different levels of environmental information]]<br />
|-<br />
| 2018 || O'Brien JJ || [[The effects of nonignorable missing data on label-free mass spectrometry proteomics experiments]]<br />
|}<br />
<br />
=== ODE-based Modelling ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2001 || Beal || [[Ways to Fit a PK Model with Some Data Below the Quantification Limit]]<br />
|-<br />
| 2008 || Balsa-Canto || [[Hybrid optimization method with general switching strategy for parameter estimation]]<br />
|-<br />
| 2011 || Tashkova || [[Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis]]<br />
|-<br />
| 2013 || Raue || [[Lessons Learned from Quantitative Dynamical Modeling in Systems Biology]]<br />
|-<br />
| 2013 || Dondelinger || [[ODE parameter inference using adaptive gradient matching with Gaussian processes]]<br />
|-<br />
| 2017 || Ballnus || [[Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems]]<br />
|-<br />
| 2017 || Henriques || [[Data-driven reverse engineering of signaling pathways using ensembles of dynamic models]]<br />
|-<br />
| 2017 || Melicher || [[Fast derivatives of likelihood functionals for ODE based models using adjoint-state method]]<br />
|-<br />
| 2017 || Penas || [[Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy]]<br />
|-<br />
| 2017 || Degasperi || [[Performance of objective functions and optimization procedures for parameter estimation in system biology models]]<br />
|-<br />
| 2017 || Fröhlich || [[Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks]]<br />
|-<br />
| 2018 || Schälte || [[Evaluation of Derivative-Free Optimizers for Parameter Estimation in Systems Biology]]<br />
|-<br />
| 2018 || Loos || [[Hierarchical optimization for the efficient parametrization of ODE models]]<br />
|-<br />
| 2018 || Villaverde || [[Input-dependent structural identifiability of nonlinear systems]]<br />
|-<br />
| 2018 || Stapor || [[Optimization and profile calculation of ODE models using second order adjoint sensitivity analysis]]<br />
|-<br />
| 2019 || Villaverde || [[A comparison of methods for quantifying prediction uncertainty in systems biology]]<br />
|-<br />
| 2019 || Hass || [[Benchmark problems for dynamic modeling of intracellular processes]]<br />
|-<br />
| 2019 || Villaverde || [[Benchmarking optimization methods for parameter estimation in large kinetic models]]<br />
|-<br />
| 2019 || Lines || [[Efficient computation of steady states in large-scale ODE models of biochemical reaction networks]]<br />
|-<br />
| 2019 || Stapor || [[Mini-batch optimization enables training of ODE models on large-scale datasets]]<br />
|-<br />
| 2019 || Wu || [[Parameter Estimation and Variable Selection for Big Systems of Linear Ordinary Differential Equations: A Matrix-Based Approach]]<br />
|-<br />
| 2019 || Pitt || [[Parameter estimation in models of biological oscillators: an automated regularised estimation approach]]<br />
|-<br />
| 2019 || Loos || [[Robust calibration of hierarchical population models for heterogeneous cell populations]]<br />
|-<br />
| 2019 || Clairon || [[Tracking for parameter and state estimation in possibly misspecified partially observed linear Ordinary Differential Equations]]<br />
|-<br />
| 2020 || Bianconi || [[An application of Conditional RobustCalibration (CRC) to ordinary differential equations (ODEs) models in computational systems biology: a comparison of two sampling strategies]]<br />
|-<br />
| 2020 || Schmiester || [[Efficient parameterization of large-scale dynamic models based on relative measurements]]<br />
|-<br />
| 2020 || Castro || [[Testing structural identifiability by a simple scaling method]]<br />
|}<br />
<br />
=== Omics Workflows ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2008 || Neuweger H || [[MeltDB: a software platform for the analysis and integration of metabolomics experiment data]]<br />
|-<br />
| 2008 || Barla A || [[Machine learning methods for predictive proteomics]]<br />
|-<br />
| 2009 || Xia J || [[MetaboAnalyst: a web server for metabolomic data analysis and interpretation]]<br />
|-<br />
| 2013 || Weisser H || [[An Automated Pipeline for High-Throughput Label-Free Quantitative Proteomics]]<br />
|-<br />
| 2014 || Cox J || [[Accurate Proteome-wide Label-free Quantification by Delayed Normalization and Maximal Peptide Ratio Extraction, Termed MaxLFQ* ]]<br />
|-<br />
| 2015 || || [[ComparingVariant Call Files for Performance Benchmarkingof Next-Generation Sequencing Variant Calling Pipelines]]<br />
|-<br />
| 2016 || Tyanova S || [[The MaxQuant computational platform for mass spectrometry–based shotgun proteomics]]<br />
|-<br />
| 2016 || Röst HL || [[OpenMS: a flexible open-source software platform for mass spectrometry data analysis]]<br />
|-<br />
| 2017 || || [[A benchmarking of workflows for detecting differential splicing and differential expression at isoform level in human RNA-seq studies]]<br />
|-<br />
| 2018 || Välikangas T || [[A comprehensive evaluation of popular proteomics software workflows for label-free proteome quantification and imputation]]<br />
|-<br />
| 2019 || || [[A Systematic Evaluation of Single CellRNA-Seq Analysis Pipelines]]<br />
|-<br />
| 2019 || || [[Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays]]<br />
|}<br />
<br />
=== Preprocessing high-throughput data===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|- 1999 || Perkins DN || [[Probability-based protein identification by searching sequence databases using mass spectrometry data]]<br />
|-<br />
| 2003 || || [[A comparison of normalization methods for high density oligonucleotide array data based on variance and bias ]]<br />
|-<br />
| 2005 || || [[Comparison of Affymetrix GeneChip Expression Measures]]<br />
|-<br />
| 2005 || Meleth S || [[The case for well-conducted experiments to validate statistical protocols for 2D gels: different pre-processing = different lists of significant proteins]]<br />
|-<br />
| 2005 || || [[Comparison of background correction and normalization procedures for high-density oligonucleotide microarrays]]<br />
|-<br />
| 2006 || || [[Using RNA sample titrations to assess microarray platform performance and normalization techniques]]<br />
|-<br />
| 2006 || Wang P || [[Normalization regarding non-random missing values in high-throughput mass spectrometry data]]<br />
|-<br />
| 2007 || Carvalho B || [[Exploration, normalization, and genotype calls of high-density oligonucleotide SNP array data]]<br />
|-<br />
| 2007 || Cannataro M || [[MS‐Analyzer: preprocessing and data mining services for proteomics applications on the Grid]]<br />
|-<br />
| 2008 || || [[Comparison of preprocessing methods for the hgU133+2 chip from Affymetrix]]<br />
|-<br />
| 2009 || || [[Comparison of Affymetrix data normalization methods using 6,926 experiments across five array generations]]<br />
|-<br />
| 2009 || Mar JC || [[Data-driven normalization strategies for high-throughput quantitative RT-PCR]]<br />
|-<br />
| 2009 || Vakhrushev SY || [[Software platform for high-throughput glycomics]]<br />
|-<br />
| 2010 || || [[Consistency of predictive signature genes and classifiers generated using different microarray platforms]]<br />
|-<br />
| 2010 || || [[Detecting and correcting systematic variation in large-scale RNA sequencing data]]<br />
|-<br />
| 2010 || || [[Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments]]<br />
|-<br />
| 2010 || || [[Normalization of RNA-seq data using factor analysis of control genes or samples]]<br />
|-<br />
| 2011 || || [[Affymetrix GeneChip microarray preprocessing for multivariate analyses]]<br />
|-<br />
| 2012 || || [[A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis]]<br />
|-<br />
| 2014 || || [[Normalyzer: A Tool for Rapid Evaluation of Normalization Methods for Omics Data Sets]]<br />
|-<br />
| 2014 || Zhou X || [[Prevention, diagnosis and treatment of high-throughput sequencing data pathologies]]<br />
|-<br />
| 2014 || Coble JB || [[Comparative evaluation of preprocessing freeware on chromatography/mass spectrometry data for signature discovery]]<br />
|-<br />
| 2015 || Caraus I || [[Detecting and overcoming systematic bias in high-throughput screening technologies: a comprehensive review of practical issues and methodological solutions]]<br />
|-<br />
| 2015 || Tam S || [[Optimization of miRNA-seq data preprocessing]]<br />
|-<br />
| 2015 || Rafiei A || [[Comparison of peak‐picking workflows for untargeted liquid chromatography/high‐resolution mass spectrometry metabolomics data analysis]]<br />
|-<br />
| 2015 || Chawade A || [[Data processing has major impact on the outcome of quantitative label-free LC-MS analysis]]<br />
|-<br />
| 2015 || Wang T || [[A systematic study of normalization methods for Infinium 450K methylation data using whole-genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Yi L || [[Chemometric methods in data processing of mass spectrometry-based metabolomics: A review]]<br />
|-<br />
| 2016 || Tsuji J || [[Evaluation of preprocessing, mapping and postprocessing algorithms for analyzing whole genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Li B || [[Performance Evaluation and Online Realization of Data-driven Normalization Methods Used in LC/MS based Untargeted Metabolomics Analysis]]<br />
|-<br />
| 2017 || Li B || [[NOREVA: normalization and evaluation of MS-based metabolomics data]]<br />
|-<br />
| 2018 || Mazoure B || [[Identification and Correction of Additive and Multiplicative Spatial Biases in Experimental High-Throughput Screening]]<br />
|-<br />
| 2018 || Li Z || [[Comprehensive evaluation of untargeted metabolomics data processing software in feature detection, quantification and discriminating marker selection]]<br />
|-<br />
| 2018 || Willforss J || [[NormalyzerDE: Online Tool for Improved Normalization of Omics Expression Data and High-Sensitivity Differential Expression Analysis]]<br />
|}</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Literature_Studies&diff=706Literature Studies2020-02-25T15:42:32Z<p>Bwday: /* ODE-based Modelling */</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
{| class="wikitable"<br />
|-<br />
! Page summary<br />
|-<br />
| Here outcomes of benchmarking studies from the literature are collected. The primary aim is a comprehensive overview about neutral benchmark studies, i.e. assessments which were performed independenty on publication of a new approach. Studies which are not neutral are put in brackets. </br> <br />
<br />
The focus is on computational methods for analyzing experimental data (instead of comparing experimental techniques or platforms). </br><br />
<br />
Please extend this list by creating a new page and adding a link below. </br> <br />
Use the '''[[Guidelines_for_Summarizing_a_Literature_Study|guidelines described here]]'''.<br />
|}<br />
<br />
== Results from Literature ==<br />
<br />
=== Classification ===<br />
''' 2003 '''</br><br />
* [[Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data]]<br />
''' 2005 '''</br><br />
* [[A review and comparison of classification algorithms for medical decision making]]<br />
''' 2016 '''</br><br />
* [[Predicting Breast Cancer Survivability Using Data Mining Techniques]]<br />
<br />
=== Selection of Differential Features and Regions ===<br />
==== Identifying differential features ====<br />
''' 2006 '''</br><br />
* [[Rat toxicogenomic study reveals analytical consistency across microarray platforms]]<br />
''' 2010 '''</br><br />
* [[A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the sequencing Quality control consortium]]<br />
''' 2017 '''</br><br />
* [[Identification of differentially expressed peptides in high-throughput proteomics data]]<br />
* [[In-depth method assessments of differentially expressed protein detection for shotgun proteomics data with missing values]]<br />
* [[Strategies for analyzing bisulfite sequencing data]]<br />
''' 2018 '''</br><br />
* [[Identification of Differentially Methylated Sites with Weak Methylation Effects]]<br />
<br />
==== Identifying differential regions (e.g. DMRs) ====<br />
''' 2015 '''<br />
* [[De novo identification of differentially methylated regions in the human genome]]<br />
* [[MethylAction: detecting differentially methylated regions that distinguish biological subtypes]]<br />
* [[metilene: Fast and sensitive calling of differentially methylated regions from bisulfite sequencing data]]<br />
''' 2016 '''<br />
* [[seqlm: an MDL based method for identifying differentially methylated regions in high density methylation array data]]<br />
* [[Statistical methods for detecting differentially methylated regions based on MethylCap-seq data]]<br />
''' 2017 '''<br />
* [[DMRfinder: efficiently identifying differentially methylated regions from MethylC-seq data]]<br />
''' 2018 '''<br />
* [[Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus]]<br />
* [[DMRcaller: a versatile R/Bioconductor package for detection and visualization of differentially methylated regions in CpG and non-CpG contexts]]<br />
* [[MethCP: Differentially Methylated Region Detection with Change Point Models (bioRxiv)]]<br />
<br />
==== Identifying sets of features (e.g. gene set analyses) ====<br />
'''2009<br />
<br />
* [[A general modular framework for gene set enrichment analysis]]<br />
* [[Comparing gene set analysis methods on single-nucleotide polymorphism data from Genetic Analysis Workshop 16]]<br />
<br />
'''2018<br />
<br />
* [[Gene set analysis methods: a systematic comparison]]<br />
<br />
'''2020<br />
* [[Toward a gold standard for benchmarking gene set enrichment analysis]]<br />
<br />
==== Dimension reduction ====<br />
''' 2008 '''</br><br />
* [[On the Relationship Between Feature Selection and Classification Accuracy]]<br />
''' 2015 '''</br><br />
* [[Comparing feature selection methods for highdimensional imbalanced data: identifying rheumatoid arthritis cohorts from routine data]]<br />
<br />
=== Imputation methods for missing values ===<br />
<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 1996 || Schenker || [[Partially parametric techniques for multiple imputation]]<br />
|-<br />
| 1999 || Hastie T || [[Imputing Missing Data for Gene Expression Arrays]]<br />
|-<br />
| 2001 || Troyanskaya || [[Missing value estimation methods for DNA microarrays]]<br />
|-<br />
| 2002 || Engels J || [[Imputation of missing longitudinal data: a comparison of methods]]<br />
|-<br />
| 2003 || Oba || [[A Bayesian missing value estimation method for gene expression profile data]]<br />
|-<br />
| 2005 || Scholz || [[Nonlinear PCA: a missing data approach]]<br />
|-<br />
| 2007 || Stacklies || [[pcaMethods—a bioconductor package providing PCA methods for incomplete data]]<br />
|-<br />
| 2007 || Verboven || [[Sequential imputation for missing values]]<br />
|-<br />
| 2008 || Shaffer GN || [[Which missing value imputation method to use in expression profiles: a comparative study and two selection schemes]]<br />
|-<br />
| 2011 || Templ || [[Iterative stepwise regression imputation using standard and robust methods]]<br />
|-<br />
| 2012 || Hrydziuszko O || [[Missing values in mass spectrometry based metabolomics: an undervalued step in the data processing pipeline]]<br />
|-<br />
| 2012 || Stekhoven || [[MissForest—non-parametric missing value imputation for mixed-type data]]<br />
|-<br />
| 2013 || Taylor || [[Accounting for undetected compounds in statistical analyses of mass spectrometry ‘omic studies]]<br />
|-<br />
| 2013 || Waljee || [[Comparison of imputation methods for missing laboratory data in medicine]]<br />
|-<br />
| 2014 || Shah || [[Comparison of Random Forest and Parametric Imputation Models for Imputing Missing Data Using MICE: A CALIBER Study]]<br />
|-<br />
| 2014 || Rodwell || [[Comparison of methods for imputing limited-range variables: a simulation study]]<br />
|-<br />
| 2014 || Morris || [[Tuning multiple imputation by predictive mean matching and local residual draws]]<br />
|-<br />
| 2014 || Doove L || [[Recursive partitioning for missing data imputation in the presence of interaction effects]]<br />
|-<br />
| 2015 || Webb-Robertson BJM || [[Review, evaluation, and discussion of the challenges of missing value imputation for mass spectrometry-based label-free global proteomics]]<br />
|-<br />
| 2016 || Folch-Fortuny A || [[Assessment of maximum likelihood PCA missing data imputation]]<br />
|-<br />
| 2016 || Lazar C || [[Accounting for the Multiple Natures of Missing Values in Label-Free Quantitative Proteomics Data Sets to Compare Imputation Strategies]]<br />
|-<br />
| 2016 || Yin X || [[Multiple imputation and analysis for high-dimensional incomplete proteomics data]]<br />
|-<br />
| 2018 || Wei R || [[Missing Value Imputation Approach for Mass Spectrometry-based Metabolomics Data]]<br />
|-<br />
| 2018 || Poyatos R || [[Gap-filling a spatially explicit plant trait database: comparing imputation methods and different levels of environmental information]]<br />
|-<br />
| 2018 || O'Brien JJ || [[The effects of nonignorable missing data on label-free mass spectrometry proteomics experiments]]<br />
|}<br />
<br />
=== ODE-based Modelling ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2001 || Beal || [[Ways to Fit a PK Model with Some Data Below the Quantification Limit]]<br />
|-<br />
| 2008 || Balsa-Canto || [[Hybrid optimization method with general switching strategy for parameter estimation]]<br />
|-<br />
| 2011 || Tashkova || [[Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis]]<br />
|-<br />
| 2013 || Raue || [[Lessons Learned from Quantitative Dynamical Modeling in Systems Biology]]<br />
|-<br />
| 2013 || Dondelinger || [[ODE parameter inference using adaptive gradient matching with Gaussian processes]]<br />
|-<br />
| 2017 || Ballnus || [[Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems]]<br />
|-<br />
| 2017 || Henriques || [[Data-driven reverse engineering of signaling pathways using ensembles of dynamic models]]<br />
|-<br />
| 2017 || Melicher || [[Fast derivatives of likelihood functionals for ODE based models using adjoint-state method]]<br />
|-<br />
| 2017 || Penas || [[Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy]]<br />
|-<br />
| 2017 || Degasperi || [[Performance of objective functions and optimization procedures for parameter estimation in system biology models]]<br />
|-<br />
| 2017 || Fröhlich || [[Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks]]<br />
|-<br />
| 2018 || Schälte || [[Evaluation of Derivative-Free Optimizers for Parameter Estimation in Systems Biology]]<br />
|-<br />
| 2018 || Loos || [[Hierarchical optimization for the efficient parametrization of ODE models]]<br />
|-<br />
| 2018 || Villaverde || [[Input-dependent structural identifiability of nonlinear systems]]<br />
|-<br />
| 2018 || Stapor || [[Optimization and profile calculation of ODE models using second order adjoint sensitivity analysis]]<br />
|-<br />
| 2019 || Villaverde || [[A comparison of methods for quantifying prediction uncertainty in systems biology]]<br />
|-<br />
| 2019 || Hass || [[Benchmark problems for dynamic modeling of intracellular processes]]<br />
|-<br />
| 2019 || Villaverde || [[Benchmarking optimization methods for parameter estimation in large kinetic models]]<br />
|-<br />
| 2019 || Lines || [[Efficient computation of steady states in large-scale ODE models of biochemical reaction networks]]<br />
|-<br />
| 2019 || Villaverde || [[Full observability and estimation of unknown inputs, states and parameters of nonlinear biological models]]<br />
|-<br />
| 2019 || Stapor || [[Mini-batch optimization enables training of ODE models on large-scale datasets]]<br />
|-<br />
| 2019 || Wu || [[Parameter Estimation and Variable Selection for Big Systems of Linear Ordinary Differential Equations: A Matrix-Based Approach]]<br />
|-<br />
| 2019 || Pitt || [[Parameter estimation in models of biological oscillators: an automated regularised estimation approach]]<br />
|-<br />
| 2019 || Loos || [[Robust calibration of hierarchical population models for heterogeneous cell populations]]<br />
|-<br />
| 2019 || Shin || [[Scalable nonlinear programming framework for parameter estimation in dynamic biological system models]]<br />
|-<br />
| 2019 || Clairon || [[Tracking for parameter and state estimation in possibly misspecified partially observed linear Ordinary Differential Equations]]<br />
|-<br />
| 2020 || Bianconi || [[An application of Conditional RobustCalibration (CRC) to ordinary differential equations (ODEs) models in computational systems biology: a comparison of two sampling strategies]]<br />
|-<br />
| 2020 || Schmiester || [[Efficient parameterization of large-scale dynamic models based on relative measurements]]<br />
|-<br />
| 2020 || Castro || [[Testing structural identifiability by a simple scaling method]]<br />
|}<br />
<br />
=== Omics Workflows ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2008 || Neuweger H || [[MeltDB: a software platform for the analysis and integration of metabolomics experiment data]]<br />
|-<br />
| 2008 || Barla A || [[Machine learning methods for predictive proteomics]]<br />
|-<br />
| 2009 || Xia J || [[MetaboAnalyst: a web server for metabolomic data analysis and interpretation]]<br />
|-<br />
| 2013 || Weisser H || [[An Automated Pipeline for High-Throughput Label-Free Quantitative Proteomics]]<br />
|-<br />
| 2014 || Cox J || [[Accurate Proteome-wide Label-free Quantification by Delayed Normalization and Maximal Peptide Ratio Extraction, Termed MaxLFQ* ]]<br />
|-<br />
| 2015 || || [[ComparingVariant Call Files for Performance Benchmarkingof Next-Generation Sequencing Variant Calling Pipelines]]<br />
|-<br />
| 2016 || Tyanova S || [[The MaxQuant computational platform for mass spectrometry–based shotgun proteomics]]<br />
|-<br />
| 2016 || Röst HL || [[OpenMS: a flexible open-source software platform for mass spectrometry data analysis]]<br />
|-<br />
| 2017 || || [[A benchmarking of workflows for detecting differential splicing and differential expression at isoform level in human RNA-seq studies]]<br />
|-<br />
| 2018 || Välikangas T || [[A comprehensive evaluation of popular proteomics software workflows for label-free proteome quantification and imputation]]<br />
|-<br />
| 2019 || || [[A Systematic Evaluation of Single CellRNA-Seq Analysis Pipelines]]<br />
|-<br />
| 2019 || || [[Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays]]<br />
|}<br />
<br />
=== Preprocessing high-throughput data===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|- 1999 || Perkins DN || [[Probability-based protein identification by searching sequence databases using mass spectrometry data]]<br />
|-<br />
| 2003 || || [[A comparison of normalization methods for high density oligonucleotide array data based on variance and bias ]]<br />
|-<br />
| 2005 || || [[Comparison of Affymetrix GeneChip Expression Measures]]<br />
|-<br />
| 2005 || Meleth S || [[The case for well-conducted experiments to validate statistical protocols for 2D gels: different pre-processing = different lists of significant proteins]]<br />
|-<br />
| 2005 || || [[Comparison of background correction and normalization procedures for high-density oligonucleotide microarrays]]<br />
|-<br />
| 2006 || || [[Using RNA sample titrations to assess microarray platform performance and normalization techniques]]<br />
|-<br />
| 2006 || Wang P || [[Normalization regarding non-random missing values in high-throughput mass spectrometry data]]<br />
|-<br />
| 2007 || Carvalho B || [[Exploration, normalization, and genotype calls of high-density oligonucleotide SNP array data]]<br />
|-<br />
| 2007 || Cannataro M || [[MS‐Analyzer: preprocessing and data mining services for proteomics applications on the Grid]]<br />
|-<br />
| 2008 || || [[Comparison of preprocessing methods for the hgU133+2 chip from Affymetrix]]<br />
|-<br />
| 2009 || || [[Comparison of Affymetrix data normalization methods using 6,926 experiments across five array generations]]<br />
|-<br />
| 2009 || Mar JC || [[Data-driven normalization strategies for high-throughput quantitative RT-PCR]]<br />
|-<br />
| 2009 || Vakhrushev SY || [[Software platform for high-throughput glycomics]]<br />
|-<br />
| 2010 || || [[Consistency of predictive signature genes and classifiers generated using different microarray platforms]]<br />
|-<br />
| 2010 || || [[Detecting and correcting systematic variation in large-scale RNA sequencing data]]<br />
|-<br />
| 2010 || || [[Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments]]<br />
|-<br />
| 2010 || || [[Normalization of RNA-seq data using factor analysis of control genes or samples]]<br />
|-<br />
| 2011 || || [[Affymetrix GeneChip microarray preprocessing for multivariate analyses]]<br />
|-<br />
| 2012 || || [[A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis]]<br />
|-<br />
| 2014 || || [[Normalyzer: A Tool for Rapid Evaluation of Normalization Methods for Omics Data Sets]]<br />
|-<br />
| 2014 || Zhou X || [[Prevention, diagnosis and treatment of high-throughput sequencing data pathologies]]<br />
|-<br />
| 2014 || Coble JB || [[Comparative evaluation of preprocessing freeware on chromatography/mass spectrometry data for signature discovery]]<br />
|-<br />
| 2015 || Caraus I || [[Detecting and overcoming systematic bias in high-throughput screening technologies: a comprehensive review of practical issues and methodological solutions]]<br />
|-<br />
| 2015 || Tam S || [[Optimization of miRNA-seq data preprocessing]]<br />
|-<br />
| 2015 || Rafiei A || [[Comparison of peak‐picking workflows for untargeted liquid chromatography/high‐resolution mass spectrometry metabolomics data analysis]]<br />
|-<br />
| 2015 || Chawade A || [[Data processing has major impact on the outcome of quantitative label-free LC-MS analysis]]<br />
|-<br />
| 2015 || Wang T || [[A systematic study of normalization methods for Infinium 450K methylation data using whole-genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Yi L || [[Chemometric methods in data processing of mass spectrometry-based metabolomics: A review]]<br />
|-<br />
| 2016 || Tsuji J || [[Evaluation of preprocessing, mapping and postprocessing algorithms for analyzing whole genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Li B || [[Performance Evaluation and Online Realization of Data-driven Normalization Methods Used in LC/MS based Untargeted Metabolomics Analysis]]<br />
|-<br />
| 2017 || Li B || [[NOREVA: normalization and evaluation of MS-based metabolomics data]]<br />
|-<br />
| 2018 || Mazoure B || [[Identification and Correction of Additive and Multiplicative Spatial Biases in Experimental High-Throughput Screening]]<br />
|-<br />
| 2018 || Li Z || [[Comprehensive evaluation of untargeted metabolomics data processing software in feature detection, quantification and discriminating marker selection]]<br />
|-<br />
| 2018 || Willforss J || [[NormalyzerDE: Online Tool for Improved Normalization of Omics Expression Data and High-Sensitivity Differential Expression Analysis]]<br />
|}</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Literature_Studies&diff=705Literature Studies2020-02-25T15:41:38Z<p>Bwday: /* ODE-based Modelling */</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
{| class="wikitable"<br />
|-<br />
! Page summary<br />
|-<br />
| Here outcomes of benchmarking studies from the literature are collected. The primary aim is a comprehensive overview about neutral benchmark studies, i.e. assessments which were performed independenty on publication of a new approach. Studies which are not neutral are put in brackets. </br> <br />
<br />
The focus is on computational methods for analyzing experimental data (instead of comparing experimental techniques or platforms). </br><br />
<br />
Please extend this list by creating a new page and adding a link below. </br> <br />
Use the '''[[Guidelines_for_Summarizing_a_Literature_Study|guidelines described here]]'''.<br />
|}<br />
<br />
== Results from Literature ==<br />
<br />
=== Classification ===<br />
''' 2003 '''</br><br />
* [[Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data]]<br />
''' 2005 '''</br><br />
* [[A review and comparison of classification algorithms for medical decision making]]<br />
''' 2016 '''</br><br />
* [[Predicting Breast Cancer Survivability Using Data Mining Techniques]]<br />
<br />
=== Selection of Differential Features and Regions ===<br />
==== Identifying differential features ====<br />
''' 2006 '''</br><br />
* [[Rat toxicogenomic study reveals analytical consistency across microarray platforms]]<br />
''' 2010 '''</br><br />
* [[A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the sequencing Quality control consortium]]<br />
''' 2017 '''</br><br />
* [[Identification of differentially expressed peptides in high-throughput proteomics data]]<br />
* [[In-depth method assessments of differentially expressed protein detection for shotgun proteomics data with missing values]]<br />
* [[Strategies for analyzing bisulfite sequencing data]]<br />
''' 2018 '''</br><br />
* [[Identification of Differentially Methylated Sites with Weak Methylation Effects]]<br />
<br />
==== Identifying differential regions (e.g. DMRs) ====<br />
''' 2015 '''<br />
* [[De novo identification of differentially methylated regions in the human genome]]<br />
* [[MethylAction: detecting differentially methylated regions that distinguish biological subtypes]]<br />
* [[metilene: Fast and sensitive calling of differentially methylated regions from bisulfite sequencing data]]<br />
''' 2016 '''<br />
* [[seqlm: an MDL based method for identifying differentially methylated regions in high density methylation array data]]<br />
* [[Statistical methods for detecting differentially methylated regions based on MethylCap-seq data]]<br />
''' 2017 '''<br />
* [[DMRfinder: efficiently identifying differentially methylated regions from MethylC-seq data]]<br />
''' 2018 '''<br />
* [[Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus]]<br />
* [[DMRcaller: a versatile R/Bioconductor package for detection and visualization of differentially methylated regions in CpG and non-CpG contexts]]<br />
* [[MethCP: Differentially Methylated Region Detection with Change Point Models (bioRxiv)]]<br />
<br />
==== Identifying sets of features (e.g. gene set analyses) ====<br />
'''2009<br />
<br />
* [[A general modular framework for gene set enrichment analysis]]<br />
* [[Comparing gene set analysis methods on single-nucleotide polymorphism data from Genetic Analysis Workshop 16]]<br />
<br />
'''2018<br />
<br />
* [[Gene set analysis methods: a systematic comparison]]<br />
<br />
'''2020<br />
* [[Toward a gold standard for benchmarking gene set enrichment analysis]]<br />
<br />
==== Dimension reduction ====<br />
''' 2008 '''</br><br />
* [[On the Relationship Between Feature Selection and Classification Accuracy]]<br />
''' 2015 '''</br><br />
* [[Comparing feature selection methods for highdimensional imbalanced data: identifying rheumatoid arthritis cohorts from routine data]]<br />
<br />
=== Imputation methods for missing values ===<br />
<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 1996 || Schenker || [[Partially parametric techniques for multiple imputation]]<br />
|-<br />
| 1999 || Hastie T || [[Imputing Missing Data for Gene Expression Arrays]]<br />
|-<br />
| 2001 || Troyanskaya || [[Missing value estimation methods for DNA microarrays]]<br />
|-<br />
| 2002 || Engels J || [[Imputation of missing longitudinal data: a comparison of methods]]<br />
|-<br />
| 2003 || Oba || [[A Bayesian missing value estimation method for gene expression profile data]]<br />
|-<br />
| 2005 || Scholz || [[Nonlinear PCA: a missing data approach]]<br />
|-<br />
| 2007 || Stacklies || [[pcaMethods—a bioconductor package providing PCA methods for incomplete data]]<br />
|-<br />
| 2007 || Verboven || [[Sequential imputation for missing values]]<br />
|-<br />
| 2008 || Shaffer GN || [[Which missing value imputation method to use in expression profiles: a comparative study and two selection schemes]]<br />
|-<br />
| 2011 || Templ || [[Iterative stepwise regression imputation using standard and robust methods]]<br />
|-<br />
| 2012 || Hrydziuszko O || [[Missing values in mass spectrometry based metabolomics: an undervalued step in the data processing pipeline]]<br />
|-<br />
| 2012 || Stekhoven || [[MissForest—non-parametric missing value imputation for mixed-type data]]<br />
|-<br />
| 2013 || Taylor || [[Accounting for undetected compounds in statistical analyses of mass spectrometry ‘omic studies]]<br />
|-<br />
| 2013 || Waljee || [[Comparison of imputation methods for missing laboratory data in medicine]]<br />
|-<br />
| 2014 || Shah || [[Comparison of Random Forest and Parametric Imputation Models for Imputing Missing Data Using MICE: A CALIBER Study]]<br />
|-<br />
| 2014 || Rodwell || [[Comparison of methods for imputing limited-range variables: a simulation study]]<br />
|-<br />
| 2014 || Morris || [[Tuning multiple imputation by predictive mean matching and local residual draws]]<br />
|-<br />
| 2014 || Doove L || [[Recursive partitioning for missing data imputation in the presence of interaction effects]]<br />
|-<br />
| 2015 || Webb-Robertson BJM || [[Review, evaluation, and discussion of the challenges of missing value imputation for mass spectrometry-based label-free global proteomics]]<br />
|-<br />
| 2016 || Folch-Fortuny A || [[Assessment of maximum likelihood PCA missing data imputation]]<br />
|-<br />
| 2016 || Lazar C || [[Accounting for the Multiple Natures of Missing Values in Label-Free Quantitative Proteomics Data Sets to Compare Imputation Strategies]]<br />
|-<br />
| 2016 || Yin X || [[Multiple imputation and analysis for high-dimensional incomplete proteomics data]]<br />
|-<br />
| 2018 || Wei R || [[Missing Value Imputation Approach for Mass Spectrometry-based Metabolomics Data]]<br />
|-<br />
| 2018 || Poyatos R || [[Gap-filling a spatially explicit plant trait database: comparing imputation methods and different levels of environmental information]]<br />
|-<br />
| 2018 || O'Brien JJ || [[The effects of nonignorable missing data on label-free mass spectrometry proteomics experiments]]<br />
|}<br />
<br />
=== ODE-based Modelling ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2001 || || [[Ways to Fit a PK Model with Some Data Below the Quantification Limit]]<br />
|-<br />
| 2008 || Balsa-Canto || [[Hybrid optimization method with general switching strategy for parameter estimation]]<br />
|-<br />
| 2011 || Tashkova || [[Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis]]<br />
|-<br />
| 2013 || Raue || [[Lessons Learned from Quantitative Dynamical Modeling in Systems Biology]]<br />
|-<br />
| 2013 || || [[ODE parameter inference using adaptive gradient matching with Gaussian processes]]<br />
|-<br />
| 2017 || Ballnus || [[Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems]]<br />
|-<br />
| 2017 || Henriques || [[Data-driven reverse engineering of signaling pathways using ensembles of dynamic models]]<br />
|-<br />
| 2017 || Melicher || [[Fast derivatives of likelihood functionals for ODE based models using adjoint-state method]]<br />
|-<br />
| 2017 || Penas || [[Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy]]<br />
|-<br />
| 2017 || Degasperi || [[Performance of objective functions and optimization procedures for parameter estimation in system biology models]]<br />
|-<br />
| 2017 || Fröhlich || [[Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks]]<br />
|-<br />
| 2018 || Schälte || [[Evaluation of Derivative-Free Optimizers for Parameter Estimation in Systems Biology]]<br />
|-<br />
| 2018 || Loos || [[Hierarchical optimization for the efficient parametrization of ODE models]]<br />
|-<br />
| 2018 || Villaverde || [[Input-dependent structural identifiability of nonlinear systems]]<br />
|-<br />
| 2018 || Stapor || [[Optimization and profile calculation of ODE models using second order adjoint sensitivity analysis]]<br />
|-<br />
| 2019 || Villaverde || [[A comparison of methods for quantifying prediction uncertainty in systems biology]]<br />
|-<br />
| 2019 || Hass || [[Benchmark problems for dynamic modeling of intracellular processes]]<br />
|-<br />
| 2019 || Villaverde || [[Benchmarking optimization methods for parameter estimation in large kinetic models]]<br />
|-<br />
| 2019 || Lines || [[Efficient computation of steady states in large-scale ODE models of biochemical reaction networks]]<br />
|-<br />
| 2019 || Villaverde || [[Full observability and estimation of unknown inputs, states and parameters of nonlinear biological models]]<br />
|-<br />
| 2019 || Stapor || [[Mini-batch optimization enables training of ODE models on large-scale datasets]]<br />
|-<br />
| 2019 || Wu || [[Parameter Estimation and Variable Selection for Big Systems of Linear Ordinary Differential Equations: A Matrix-Based Approach]]<br />
|-<br />
| 2019 || Pitt || [[Parameter estimation in models of biological oscillators: an automated regularised estimation approach]]<br />
|-<br />
| 2019 || Loos || [[Robust calibration of hierarchical population models for heterogeneous cell populations]]<br />
|-<br />
| 2019 || Shin || [[Scalable nonlinear programming framework for parameter estimation in dynamic biological system models]]<br />
|-<br />
| 2019 || Clairon || [[Tracking for parameter and state estimation in possibly misspecified partially observed linear Ordinary Differential Equations]]<br />
|-<br />
| 2020 || Bianconi || [[An application of Conditional RobustCalibration (CRC) to ordinary differential equations (ODEs) models in computational systems biology: a comparison of two sampling strategies]]<br />
|-<br />
| 2020 || Schmiester || [[Efficient parameterization of large-scale dynamic models based on relative measurements]]<br />
|-<br />
| 2020 || Castro || [[Testing structural identifiability by a simple scaling method]]<br />
|}<br />
<br />
=== Omics Workflows ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2008 || Neuweger H || [[MeltDB: a software platform for the analysis and integration of metabolomics experiment data]]<br />
|-<br />
| 2008 || Barla A || [[Machine learning methods for predictive proteomics]]<br />
|-<br />
| 2009 || Xia J || [[MetaboAnalyst: a web server for metabolomic data analysis and interpretation]]<br />
|-<br />
| 2013 || Weisser H || [[An Automated Pipeline for High-Throughput Label-Free Quantitative Proteomics]]<br />
|-<br />
| 2014 || Cox J || [[Accurate Proteome-wide Label-free Quantification by Delayed Normalization and Maximal Peptide Ratio Extraction, Termed MaxLFQ* ]]<br />
|-<br />
| 2015 || || [[ComparingVariant Call Files for Performance Benchmarkingof Next-Generation Sequencing Variant Calling Pipelines]]<br />
|-<br />
| 2016 || Tyanova S || [[The MaxQuant computational platform for mass spectrometry–based shotgun proteomics]]<br />
|-<br />
| 2016 || Röst HL || [[OpenMS: a flexible open-source software platform for mass spectrometry data analysis]]<br />
|-<br />
| 2017 || || [[A benchmarking of workflows for detecting differential splicing and differential expression at isoform level in human RNA-seq studies]]<br />
|-<br />
| 2018 || Välikangas T || [[A comprehensive evaluation of popular proteomics software workflows for label-free proteome quantification and imputation]]<br />
|-<br />
| 2019 || || [[A Systematic Evaluation of Single CellRNA-Seq Analysis Pipelines]]<br />
|-<br />
| 2019 || || [[Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays]]<br />
|}<br />
<br />
=== Preprocessing high-throughput data===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|- 1999 || Perkins DN || [[Probability-based protein identification by searching sequence databases using mass spectrometry data]]<br />
|-<br />
| 2003 || || [[A comparison of normalization methods for high density oligonucleotide array data based on variance and bias ]]<br />
|-<br />
| 2005 || || [[Comparison of Affymetrix GeneChip Expression Measures]]<br />
|-<br />
| 2005 || Meleth S || [[The case for well-conducted experiments to validate statistical protocols for 2D gels: different pre-processing = different lists of significant proteins]]<br />
|-<br />
| 2005 || || [[Comparison of background correction and normalization procedures for high-density oligonucleotide microarrays]]<br />
|-<br />
| 2006 || || [[Using RNA sample titrations to assess microarray platform performance and normalization techniques]]<br />
|-<br />
| 2006 || Wang P || [[Normalization regarding non-random missing values in high-throughput mass spectrometry data]]<br />
|-<br />
| 2007 || Carvalho B || [[Exploration, normalization, and genotype calls of high-density oligonucleotide SNP array data]]<br />
|-<br />
| 2007 || Cannataro M || [[MS‐Analyzer: preprocessing and data mining services for proteomics applications on the Grid]]<br />
|-<br />
| 2008 || || [[Comparison of preprocessing methods for the hgU133+2 chip from Affymetrix]]<br />
|-<br />
| 2009 || || [[Comparison of Affymetrix data normalization methods using 6,926 experiments across five array generations]]<br />
|-<br />
| 2009 || Mar JC || [[Data-driven normalization strategies for high-throughput quantitative RT-PCR]]<br />
|-<br />
| 2009 || Vakhrushev SY || [[Software platform for high-throughput glycomics]]<br />
|-<br />
| 2010 || || [[Consistency of predictive signature genes and classifiers generated using different microarray platforms]]<br />
|-<br />
| 2010 || || [[Detecting and correcting systematic variation in large-scale RNA sequencing data]]<br />
|-<br />
| 2010 || || [[Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments]]<br />
|-<br />
| 2010 || || [[Normalization of RNA-seq data using factor analysis of control genes or samples]]<br />
|-<br />
| 2011 || || [[Affymetrix GeneChip microarray preprocessing for multivariate analyses]]<br />
|-<br />
| 2012 || || [[A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis]]<br />
|-<br />
| 2014 || || [[Normalyzer: A Tool for Rapid Evaluation of Normalization Methods for Omics Data Sets]]<br />
|-<br />
| 2014 || Zhou X || [[Prevention, diagnosis and treatment of high-throughput sequencing data pathologies]]<br />
|-<br />
| 2014 || Coble JB || [[Comparative evaluation of preprocessing freeware on chromatography/mass spectrometry data for signature discovery]]<br />
|-<br />
| 2015 || Caraus I || [[Detecting and overcoming systematic bias in high-throughput screening technologies: a comprehensive review of practical issues and methodological solutions]]<br />
|-<br />
| 2015 || Tam S || [[Optimization of miRNA-seq data preprocessing]]<br />
|-<br />
| 2015 || Rafiei A || [[Comparison of peak‐picking workflows for untargeted liquid chromatography/high‐resolution mass spectrometry metabolomics data analysis]]<br />
|-<br />
| 2015 || Chawade A || [[Data processing has major impact on the outcome of quantitative label-free LC-MS analysis]]<br />
|-<br />
| 2015 || Wang T || [[A systematic study of normalization methods for Infinium 450K methylation data using whole-genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Yi L || [[Chemometric methods in data processing of mass spectrometry-based metabolomics: A review]]<br />
|-<br />
| 2016 || Tsuji J || [[Evaluation of preprocessing, mapping and postprocessing algorithms for analyzing whole genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Li B || [[Performance Evaluation and Online Realization of Data-driven Normalization Methods Used in LC/MS based Untargeted Metabolomics Analysis]]<br />
|-<br />
| 2017 || Li B || [[NOREVA: normalization and evaluation of MS-based metabolomics data]]<br />
|-<br />
| 2018 || Mazoure B || [[Identification and Correction of Additive and Multiplicative Spatial Biases in Experimental High-Throughput Screening]]<br />
|-<br />
| 2018 || Li Z || [[Comprehensive evaluation of untargeted metabolomics data processing software in feature detection, quantification and discriminating marker selection]]<br />
|-<br />
| 2018 || Willforss J || [[NormalyzerDE: Online Tool for Improved Normalization of Omics Expression Data and High-Sensitivity Differential Expression Analysis]]<br />
|}</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Literature_Studies&diff=702Literature Studies2020-02-25T15:40:34Z<p>Bwday: /* ODE-based Modelling */</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
{| class="wikitable"<br />
|-<br />
! Page summary<br />
|-<br />
| Here outcomes of benchmarking studies from the literature are collected. The primary aim is a comprehensive overview about neutral benchmark studies, i.e. assessments which were performed independenty on publication of a new approach. Studies which are not neutral are put in brackets. </br> <br />
<br />
The focus is on computational methods for analyzing experimental data (instead of comparing experimental techniques or platforms). </br><br />
<br />
Please extend this list by creating a new page and adding a link below. </br> <br />
Use the '''[[Guidelines_for_Summarizing_a_Literature_Study|guidelines described here]]'''.<br />
|}<br />
<br />
== Results from Literature ==<br />
<br />
=== Classification ===<br />
''' 2003 '''</br><br />
* [[Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data]]<br />
''' 2005 '''</br><br />
* [[A review and comparison of classification algorithms for medical decision making]]<br />
''' 2016 '''</br><br />
* [[Predicting Breast Cancer Survivability Using Data Mining Techniques]]<br />
<br />
=== Selection of Differential Features and Regions ===<br />
==== Identifying differential features ====<br />
''' 2006 '''</br><br />
* [[Rat toxicogenomic study reveals analytical consistency across microarray platforms]]<br />
''' 2010 '''</br><br />
* [[A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the sequencing Quality control consortium]]<br />
''' 2017 '''</br><br />
* [[Identification of differentially expressed peptides in high-throughput proteomics data]]<br />
* [[In-depth method assessments of differentially expressed protein detection for shotgun proteomics data with missing values]]<br />
* [[Strategies for analyzing bisulfite sequencing data]]<br />
''' 2018 '''</br><br />
* [[Identification of Differentially Methylated Sites with Weak Methylation Effects]]<br />
<br />
==== Identifying differential regions (e.g. DMRs) ====<br />
''' 2015 '''<br />
* [[De novo identification of differentially methylated regions in the human genome]]<br />
* [[MethylAction: detecting differentially methylated regions that distinguish biological subtypes]]<br />
* [[metilene: Fast and sensitive calling of differentially methylated regions from bisulfite sequencing data]]<br />
''' 2016 '''<br />
* [[seqlm: an MDL based method for identifying differentially methylated regions in high density methylation array data]]<br />
* [[Statistical methods for detecting differentially methylated regions based on MethylCap-seq data]]<br />
''' 2017 '''<br />
* [[DMRfinder: efficiently identifying differentially methylated regions from MethylC-seq data]]<br />
''' 2018 '''<br />
* [[Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus]]<br />
* [[DMRcaller: a versatile R/Bioconductor package for detection and visualization of differentially methylated regions in CpG and non-CpG contexts]]<br />
* [[MethCP: Differentially Methylated Region Detection with Change Point Models (bioRxiv)]]<br />
<br />
==== Identifying sets of features (e.g. gene set analyses) ====<br />
'''2009<br />
<br />
* [[A general modular framework for gene set enrichment analysis]]<br />
* [[Comparing gene set analysis methods on single-nucleotide polymorphism data from Genetic Analysis Workshop 16]]<br />
<br />
'''2018<br />
<br />
* [[Gene set analysis methods: a systematic comparison]]<br />
<br />
'''2020<br />
* [[Toward a gold standard for benchmarking gene set enrichment analysis]]<br />
<br />
==== Dimension reduction ====<br />
''' 2008 '''</br><br />
* [[On the Relationship Between Feature Selection and Classification Accuracy]]<br />
''' 2015 '''</br><br />
* [[Comparing feature selection methods for highdimensional imbalanced data: identifying rheumatoid arthritis cohorts from routine data]]<br />
<br />
=== Imputation methods for missing values ===<br />
<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 1996 || Schenker || [[Partially parametric techniques for multiple imputation]]<br />
|-<br />
| 1999 || Hastie T || [[Imputing Missing Data for Gene Expression Arrays]]<br />
|-<br />
| 2001 || Troyanskaya || [[Missing value estimation methods for DNA microarrays]]<br />
|-<br />
| 2002 || Engels J || [[Imputation of missing longitudinal data: a comparison of methods]]<br />
|-<br />
| 2003 || Oba || [[A Bayesian missing value estimation method for gene expression profile data]]<br />
|-<br />
| 2005 || Scholz || [[Nonlinear PCA: a missing data approach]]<br />
|-<br />
| 2007 || Stacklies || [[pcaMethods—a bioconductor package providing PCA methods for incomplete data]]<br />
|-<br />
| 2007 || Verboven || [[Sequential imputation for missing values]]<br />
|-<br />
| 2008 || Shaffer GN || [[Which missing value imputation method to use in expression profiles: a comparative study and two selection schemes]]<br />
|-<br />
| 2011 || Templ || [[Iterative stepwise regression imputation using standard and robust methods]]<br />
|-<br />
| 2012 || Hrydziuszko O || [[Missing values in mass spectrometry based metabolomics: an undervalued step in the data processing pipeline]]<br />
|-<br />
| 2012 || Stekhoven || [[MissForest—non-parametric missing value imputation for mixed-type data]]<br />
|-<br />
| 2013 || Taylor || [[Accounting for undetected compounds in statistical analyses of mass spectrometry ‘omic studies]]<br />
|-<br />
| 2013 || Waljee || [[Comparison of imputation methods for missing laboratory data in medicine]]<br />
|-<br />
| 2014 || Shah || [[Comparison of Random Forest and Parametric Imputation Models for Imputing Missing Data Using MICE: A CALIBER Study]]<br />
|-<br />
| 2014 || Rodwell || [[Comparison of methods for imputing limited-range variables: a simulation study]]<br />
|-<br />
| 2014 || Morris || [[Tuning multiple imputation by predictive mean matching and local residual draws]]<br />
|-<br />
| 2014 || Doove L || [[Recursive partitioning for missing data imputation in the presence of interaction effects]]<br />
|-<br />
| 2015 || Webb-Robertson BJM || [[Review, evaluation, and discussion of the challenges of missing value imputation for mass spectrometry-based label-free global proteomics]]<br />
|-<br />
| 2016 || Folch-Fortuny A || [[Assessment of maximum likelihood PCA missing data imputation]]<br />
|-<br />
| 2016 || Lazar C || [[Accounting for the Multiple Natures of Missing Values in Label-Free Quantitative Proteomics Data Sets to Compare Imputation Strategies]]<br />
|-<br />
| 2016 || Yin X || [[Multiple imputation and analysis for high-dimensional incomplete proteomics data]]<br />
|-<br />
| 2018 || Wei R || [[Missing Value Imputation Approach for Mass Spectrometry-based Metabolomics Data]]<br />
|-<br />
| 2018 || Poyatos R || [[Gap-filling a spatially explicit plant trait database: comparing imputation methods and different levels of environmental information]]<br />
|-<br />
| 2018 || O'Brien JJ || [[The effects of nonignorable missing data on label-free mass spectrometry proteomics experiments]]<br />
|}<br />
<br />
=== ODE-based Modelling ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2001 || || [[Ways to Fit a PK Model with Some Data Below the Quantification Limit]]<br />
|-<br />
| 2008 || Balsa-Canto || [[Hybrid optimization method with general switching strategy for parameter estimation]]<br />
|-<br />
| 2011 || Tashkova || [[Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis]]<br />
|-<br />
| 2013 || Raue || [[Lessons Learned from Quantitative Dynamical Modeling in Systems Biology]]<br />
|-<br />
| 2013 || || [[ODE parameter inference using adaptive gradient matching with Gaussian processes]]<br />
|-<br />
| 2017 || Ballnus || [[Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems]]<br />
|-<br />
| 2017 || Henriques || [[Data-driven reverse engineering of signaling pathways using ensembles of dynamic models]]<br />
|-<br />
| 2017 || Melicher || [[Fast derivatives of likelihood functionals for ODE based models using adjoint-state method]]<br />
|-<br />
| 2017 || Penas || [[Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy]]<br />
|-<br />
| 2017 || Degasperi || [[Performance of objective functions and optimization procedures for parameter estimation in system biology models]]<br />
|-<br />
| 2017 || Fröhlich || [[Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks]]<br />
|-<br />
| 2018 || Schälte || [[Evaluation of Derivative-Free Optimizers for Parameter Estimation in Systems Biology]]<br />
|-<br />
| 2018 || Loos || [[Hierarchical optimization for the efficient parametrization of ODE models]]<br />
|-<br />
| 2018 || Villaverde || [[Input-dependent structural identifiability of nonlinear systems]]<br />
|-<br />
| 2018 || Stapor || [[Optimization and profile calculation of ODE models using second order adjoint sensitivity analysis]]<br />
|-<br />
| 2019 || Villaverde || [[A comparison of methods for quantifying prediction uncertainty in systems biology]]<br />
|-<br />
| 2019 || Hass || [[Benchmark problems for dynamic modeling of intracellular processes]]<br />
|-<br />
| 2019 || Villaverde || [[Benchmarking optimization methods for parameter estimation in large kinetic models]]<br />
|-<br />
| 2019 || Lines || [[Efficient computation of steady states in large-scale ODE models of biochemical reaction networks]]<br />
|-<br />
| 2019 || Villaverde || [[Full observability and estimation of unknown inputs, states and parameters of nonlinear biological models]]<br />
|-<br />
| 2019 || Stapor || [[Mini-batch optimization enables training of ODE models on large-scale datasets]]<br />
|-<br />
| 2019 || Wu || [[Parameter Estimation and Variable Selection for Big Systems of Linear Ordinary Differential Equations: A Matrix-Based Approach]]<br />
|-<br />
| 2019 || Pitt || [[Parameter estimation in models of biological oscillators: an automated regularised estimation approach]]<br />
|-<br />
| 2019 || Loos || [[Robust calibration of hierarchical population models for heterogeneous cell populations]]<br />
|-<br />
| 2019 || Shin || [[Scalable nonlinear programming framework for parameter estimation in dynamic biological system models]]<br />
|-<br />
| 2019 || Clairon || [[Tracking for parameter and state estimation in possibly misspecified partially observed linear Ordinary Differential Equations]]<br />
|-<br />
| 2020 || Schenker || [[An application of Conditional RobustCalibration (CRC) to ordinary differential equations (ODEs) models in computational systems biology: a comparison of two sampling strategies]]<br />
|-<br />
| 2020 || Schmiester || [[Efficient parameterization of large-scale dynamic models based on relative measurements]]<br />
|-<br />
| 2020 || Castro || [[Testing structural identifiability by a simple scaling method]]<br />
|}<br />
<br />
=== Omics Workflows ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2008 || Neuweger H || [[MeltDB: a software platform for the analysis and integration of metabolomics experiment data]]<br />
|-<br />
| 2008 || Barla A || [[Machine learning methods for predictive proteomics]]<br />
|-<br />
| 2009 || Xia J || [[MetaboAnalyst: a web server for metabolomic data analysis and interpretation]]<br />
|-<br />
| 2013 || Weisser H || [[An Automated Pipeline for High-Throughput Label-Free Quantitative Proteomics]]<br />
|-<br />
| 2014 || Cox J || [[Accurate Proteome-wide Label-free Quantification by Delayed Normalization and Maximal Peptide Ratio Extraction, Termed MaxLFQ* ]]<br />
|-<br />
| 2015 || || [[ComparingVariant Call Files for Performance Benchmarkingof Next-Generation Sequencing Variant Calling Pipelines]]<br />
|-<br />
| 2016 || Tyanova S || [[The MaxQuant computational platform for mass spectrometry–based shotgun proteomics]]<br />
|-<br />
| 2016 || Röst HL || [[OpenMS: a flexible open-source software platform for mass spectrometry data analysis]]<br />
|-<br />
| 2017 || || [[A benchmarking of workflows for detecting differential splicing and differential expression at isoform level in human RNA-seq studies]]<br />
|-<br />
| 2018 || Välikangas T || [[A comprehensive evaluation of popular proteomics software workflows for label-free proteome quantification and imputation]]<br />
|-<br />
| 2019 || || [[A Systematic Evaluation of Single CellRNA-Seq Analysis Pipelines]]<br />
|-<br />
| 2019 || || [[Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays]]<br />
|}<br />
<br />
=== Preprocessing high-throughput data===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|- 1999 || Perkins DN || [[Probability-based protein identification by searching sequence databases using mass spectrometry data]]<br />
|-<br />
| 2003 || || [[A comparison of normalization methods for high density oligonucleotide array data based on variance and bias ]]<br />
|-<br />
| 2005 || || [[Comparison of Affymetrix GeneChip Expression Measures]]<br />
|-<br />
| 2005 || Meleth S || [[The case for well-conducted experiments to validate statistical protocols for 2D gels: different pre-processing = different lists of significant proteins]]<br />
|-<br />
| 2005 || || [[Comparison of background correction and normalization procedures for high-density oligonucleotide microarrays]]<br />
|-<br />
| 2006 || || [[Using RNA sample titrations to assess microarray platform performance and normalization techniques]]<br />
|-<br />
| 2006 || Wang P || [[Normalization regarding non-random missing values in high-throughput mass spectrometry data]]<br />
|-<br />
| 2007 || Carvalho B || [[Exploration, normalization, and genotype calls of high-density oligonucleotide SNP array data]]<br />
|-<br />
| 2007 || Cannataro M || [[MS‐Analyzer: preprocessing and data mining services for proteomics applications on the Grid]]<br />
|-<br />
| 2008 || || [[Comparison of preprocessing methods for the hgU133+2 chip from Affymetrix]]<br />
|-<br />
| 2009 || || [[Comparison of Affymetrix data normalization methods using 6,926 experiments across five array generations]]<br />
|-<br />
| 2009 || Mar JC || [[Data-driven normalization strategies for high-throughput quantitative RT-PCR]]<br />
|-<br />
| 2009 || Vakhrushev SY || [[Software platform for high-throughput glycomics]]<br />
|-<br />
| 2010 || || [[Consistency of predictive signature genes and classifiers generated using different microarray platforms]]<br />
|-<br />
| 2010 || || [[Detecting and correcting systematic variation in large-scale RNA sequencing data]]<br />
|-<br />
| 2010 || || [[Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments]]<br />
|-<br />
| 2010 || || [[Normalization of RNA-seq data using factor analysis of control genes or samples]]<br />
|-<br />
| 2011 || || [[Affymetrix GeneChip microarray preprocessing for multivariate analyses]]<br />
|-<br />
| 2012 || || [[A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis]]<br />
|-<br />
| 2014 || || [[Normalyzer: A Tool for Rapid Evaluation of Normalization Methods for Omics Data Sets]]<br />
|-<br />
| 2014 || Zhou X || [[Prevention, diagnosis and treatment of high-throughput sequencing data pathologies]]<br />
|-<br />
| 2014 || Coble JB || [[Comparative evaluation of preprocessing freeware on chromatography/mass spectrometry data for signature discovery]]<br />
|-<br />
| 2015 || Caraus I || [[Detecting and overcoming systematic bias in high-throughput screening technologies: a comprehensive review of practical issues and methodological solutions]]<br />
|-<br />
| 2015 || Tam S || [[Optimization of miRNA-seq data preprocessing]]<br />
|-<br />
| 2015 || Rafiei A || [[Comparison of peak‐picking workflows for untargeted liquid chromatography/high‐resolution mass spectrometry metabolomics data analysis]]<br />
|-<br />
| 2015 || Chawade A || [[Data processing has major impact on the outcome of quantitative label-free LC-MS analysis]]<br />
|-<br />
| 2015 || Wang T || [[A systematic study of normalization methods for Infinium 450K methylation data using whole-genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Yi L || [[Chemometric methods in data processing of mass spectrometry-based metabolomics: A review]]<br />
|-<br />
| 2016 || Tsuji J || [[Evaluation of preprocessing, mapping and postprocessing algorithms for analyzing whole genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Li B || [[Performance Evaluation and Online Realization of Data-driven Normalization Methods Used in LC/MS based Untargeted Metabolomics Analysis]]<br />
|-<br />
| 2017 || Li B || [[NOREVA: normalization and evaluation of MS-based metabolomics data]]<br />
|-<br />
| 2018 || Mazoure B || [[Identification and Correction of Additive and Multiplicative Spatial Biases in Experimental High-Throughput Screening]]<br />
|-<br />
| 2018 || Li Z || [[Comprehensive evaluation of untargeted metabolomics data processing software in feature detection, quantification and discriminating marker selection]]<br />
|-<br />
| 2018 || Willforss J || [[NormalyzerDE: Online Tool for Improved Normalization of Omics Expression Data and High-Sensitivity Differential Expression Analysis]]<br />
|}</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Tracking_for_parameter_and_state_estimation_in_possibly_misspecified_partially_observed_linear_Ordinary_Differential_Equations&diff=701Tracking for parameter and state estimation in possibly misspecified partially observed linear Ordinary Differential Equations2020-02-25T15:40:14Z<p>Bwday: </p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
=== Citation ===<br />
<br />
Clairon, Quentin, and Nicolas J-B. Brunel. "Tracking for parameter and state estimation in possibly misspecified partially observed linear Ordinary Differential Equations." Journal of Statistical Planning and Inference 199 (2019): 188-206.<br />
<br />
[https://doi.org/10.1016/j.jspi.2018.06.005 Permanent link to the paper]</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Testing_structural_identifiability_by_a_simple_scaling_method&diff=700Testing structural identifiability by a simple scaling method2020-02-25T15:40:05Z<p>Bwday: </p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
=== Citation ===<br />
Castro, Mario, and Rob J. de Boer. "Testing structural identifiability by a simple scaling method." bioRxiv (2020).<br />
<br />
[https://doi.org/10.1101/2020.02.04.933630 Permanent link to the paper]</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Efficient_parameterization_of_large-scale_dynamic_models_based_on_relative_measurements&diff=699Efficient parameterization of large-scale dynamic models based on relative measurements2020-02-25T15:39:48Z<p>Bwday: </p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
=== Citation ===<br />
<br />
Leonard Schmiester, Yannik Schälte, Fabian Fröhlich, Jan Hasenauer, Daniel Weindl, [https://doi.org/10.1093/bioinformatics/btz581 Efficient parameterization of large-scale dynamic models based on relative measurements], Bioinformatics, Volume 36, Issue 2, 15 January 2020, Pages 594–602.</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Robust_calibration_of_hierarchical_population_models_for_heterogeneous_cell_populations&diff=696Robust calibration of hierarchical population models for heterogeneous cell populations2020-02-25T15:38:05Z<p>Bwday: </p>
<hr />
<div>=== Citation === <br />
Carolin Loos, Jan Hasenauer, Robust calibration of hierarchical population models for heterogeneous cell populations, Journal of Theoretical Biology (488), 2020.<br />
<br />
[https://doi.org/10.1016/j.jtbi.2019.110118 Permanent link to the paper]</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=A_systematic_study_of_normalization_methods_for_Infinium_450K_methylation_data_using_whole-genome_bisulfite_sequencing_data&diff=695A systematic study of normalization methods for Infinium 450K methylation data using whole-genome bisulfite sequencing data2020-02-25T15:36:48Z<p>Bwday: Created page with "__NUMBEREDHEADINGS__ === Citation === Wang T, Guan W, Lin J, Boutaoui N, Canino G, Luo J, Celedón JC, Chen W. A systematic study of normalization methods for Infinium 450K me..."</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
=== Citation ===<br />
Wang T, Guan W, Lin J, Boutaoui N, Canino G, Luo J, Celedón JC, Chen W. A systematic study of normalization methods for Infinium 450K methylation data using whole-genome bisulfite sequencing data. Epigenetics. 2015 Jul 3;10(7):662-9.<br />
<br />
[https://doi.org/10.1080/15592294.2015.1057384 Permanent link to the paper]<br />
<br />
<br />
=== Summary ===<br />
Briefly describe the scope of the paper, i.e. the field of research and/or application.<br />
<br />
=== Study outcomes ===<br />
List the paper results concerning method comparison and benchmarking:<br />
==== Outcome O1 ====<br />
The performance of ...<br />
<br />
Outcome O1 is presented as Figure X in the original publication. <br />
<br />
==== Outcome O2 ====<br />
...<br />
<br />
Outcome O2 is presented as Figure X in the original publication. <br />
<br />
==== Outcome On ====<br />
...<br />
<br />
Outcome On is presented as Figure X in the original publication. <br />
<br />
==== Further outcomes ====<br />
If intended, you can add further outcomes here.<br />
<br />
<br />
=== Study design and evidence level ===<br />
==== General aspects ====<br />
You can describe general design aspects here.<br />
The study designs for describing specific outcomes are listed in the following subsections:<br />
<br />
==== Design for Outcome O1 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
==== Design for Outcome O2 ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
... <br />
<br />
==== Design for Outcome O ====<br />
* The outcome was generated for ...<br />
* Configuration parameters were chosen ...<br />
* ...<br />
<br />
=== Further comments and aspects ===<br />
<br />
=== References ===<br />
The list of cited or related literature is placed here.</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Literature_Studies&diff=694Literature Studies2020-02-25T15:35:17Z<p>Bwday: /* Preprocessing high-throughput data */</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
{| class="wikitable"<br />
|-<br />
! Page summary<br />
|-<br />
| Here outcomes of benchmarking studies from the literature are collected. The primary aim is a comprehensive overview about neutral benchmark studies, i.e. assessments which were performed independenty on publication of a new approach. Studies which are not neutral are put in brackets. </br> <br />
<br />
The focus is on computational methods for analyzing experimental data (instead of comparing experimental techniques or platforms). </br><br />
<br />
Please extend this list by creating a new page and adding a link below. </br> <br />
Use the '''[[Guidelines_for_Summarizing_a_Literature_Study|guidelines described here]]'''.<br />
|}<br />
<br />
== Results from Literature ==<br />
<br />
=== Classification ===<br />
''' 2003 '''</br><br />
* [[Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data]]<br />
''' 2005 '''</br><br />
* [[A review and comparison of classification algorithms for medical decision making]]<br />
''' 2016 '''</br><br />
* [[Predicting Breast Cancer Survivability Using Data Mining Techniques]]<br />
<br />
=== Selection of Differential Features and Regions ===<br />
==== Identifying differential features ====<br />
''' 2006 '''</br><br />
* [[Rat toxicogenomic study reveals analytical consistency across microarray platforms]]<br />
''' 2010 '''</br><br />
* [[A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the sequencing Quality control consortium]]<br />
''' 2017 '''</br><br />
* [[Identification of differentially expressed peptides in high-throughput proteomics data]]<br />
* [[In-depth method assessments of differentially expressed protein detection for shotgun proteomics data with missing values]]<br />
* [[Strategies for analyzing bisulfite sequencing data]]<br />
''' 2018 '''</br><br />
* [[Identification of Differentially Methylated Sites with Weak Methylation Effects]]<br />
<br />
==== Identifying differential regions (e.g. DMRs) ====<br />
''' 2015 '''<br />
* [[De novo identification of differentially methylated regions in the human genome]]<br />
* [[MethylAction: detecting differentially methylated regions that distinguish biological subtypes]]<br />
* [[metilene: Fast and sensitive calling of differentially methylated regions from bisulfite sequencing data]]<br />
''' 2016 '''<br />
* [[seqlm: an MDL based method for identifying differentially methylated regions in high density methylation array data]]<br />
* [[Statistical methods for detecting differentially methylated regions based on MethylCap-seq data]]<br />
''' 2017 '''<br />
* [[DMRfinder: efficiently identifying differentially methylated regions from MethylC-seq data]]<br />
''' 2018 '''<br />
* [[Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus]]<br />
* [[DMRcaller: a versatile R/Bioconductor package for detection and visualization of differentially methylated regions in CpG and non-CpG contexts]]<br />
* [[MethCP: Differentially Methylated Region Detection with Change Point Models (bioRxiv)]]<br />
<br />
==== Identifying sets of features (e.g. gene set analyses) ====<br />
'''2009<br />
<br />
* [[A general modular framework for gene set enrichment analysis]]<br />
* [[Comparing gene set analysis methods on single-nucleotide polymorphism data from Genetic Analysis Workshop 16]]<br />
<br />
'''2018<br />
<br />
* [[Gene set analysis methods: a systematic comparison]]<br />
<br />
'''2020<br />
* [[Toward a gold standard for benchmarking gene set enrichment analysis]]<br />
<br />
==== Dimension reduction ====<br />
''' 2008 '''</br><br />
* [[On the Relationship Between Feature Selection and Classification Accuracy]]<br />
''' 2015 '''</br><br />
* [[Comparing feature selection methods for highdimensional imbalanced data: identifying rheumatoid arthritis cohorts from routine data]]<br />
<br />
=== Imputation methods for missing values ===<br />
<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 1996 || Schenker || [[Partially parametric techniques for multiple imputation]]<br />
|-<br />
| 1999 || Hastie T || [[Imputing Missing Data for Gene Expression Arrays]]<br />
|-<br />
| 2001 || Troyanskaya || [[Missing value estimation methods for DNA microarrays]]<br />
|-<br />
| 2002 || Engels J || [[Imputation of missing longitudinal data: a comparison of methods]]<br />
|-<br />
| 2003 || Oba || [[A Bayesian missing value estimation method for gene expression profile data]]<br />
|-<br />
| 2005 || Scholz || [[Nonlinear PCA: a missing data approach]]<br />
|-<br />
| 2007 || Stacklies || [[pcaMethods—a bioconductor package providing PCA methods for incomplete data]]<br />
|-<br />
| 2007 || Verboven || [[Sequential imputation for missing values]]<br />
|-<br />
| 2008 || Shaffer GN || [[Which missing value imputation method to use in expression profiles: a comparative study and two selection schemes]]<br />
|-<br />
| 2011 || Templ || [[Iterative stepwise regression imputation using standard and robust methods]]<br />
|-<br />
| 2012 || Hrydziuszko O || [[Missing values in mass spectrometry based metabolomics: an undervalued step in the data processing pipeline]]<br />
|-<br />
| 2012 || Stekhoven || [[MissForest—non-parametric missing value imputation for mixed-type data]]<br />
|-<br />
| 2013 || Taylor || [[Accounting for undetected compounds in statistical analyses of mass spectrometry ‘omic studies]]<br />
|-<br />
| 2013 || Waljee || [[Comparison of imputation methods for missing laboratory data in medicine]]<br />
|-<br />
| 2014 || Shah || [[Comparison of Random Forest and Parametric Imputation Models for Imputing Missing Data Using MICE: A CALIBER Study]]<br />
|-<br />
| 2014 || Rodwell || [[Comparison of methods for imputing limited-range variables: a simulation study]]<br />
|-<br />
| 2014 || Morris || [[Tuning multiple imputation by predictive mean matching and local residual draws]]<br />
|-<br />
| 2014 || Doove L || [[Recursive partitioning for missing data imputation in the presence of interaction effects]]<br />
|-<br />
| 2015 || Webb-Robertson BJM || [[Review, evaluation, and discussion of the challenges of missing value imputation for mass spectrometry-based label-free global proteomics]]<br />
|-<br />
| 2016 || Folch-Fortuny A || [[Assessment of maximum likelihood PCA missing data imputation]]<br />
|-<br />
| 2016 || Lazar C || [[Accounting for the Multiple Natures of Missing Values in Label-Free Quantitative Proteomics Data Sets to Compare Imputation Strategies]]<br />
|-<br />
| 2016 || Yin X || [[Multiple imputation and analysis for high-dimensional incomplete proteomics data]]<br />
|-<br />
| 2018 || Wei R || [[Missing Value Imputation Approach for Mass Spectrometry-based Metabolomics Data]]<br />
|-<br />
| 2018 || Poyatos R || [[Gap-filling a spatially explicit plant trait database: comparing imputation methods and different levels of environmental information]]<br />
|-<br />
| 2018 || O'Brien JJ || [[The effects of nonignorable missing data on label-free mass spectrometry proteomics experiments]]<br />
|}<br />
<br />
=== ODE-based Modelling ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2001 || Schenker || [[Ways to Fit a PK Model with Some Data Below the Quantification Limit]]<br />
|-<br />
| 2008 || Balsa-Canto || [[Hybrid optimization method with general switching strategy for parameter estimation]]<br />
|-<br />
| 2011 || Tashkova || [[Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis]]<br />
|-<br />
| 2013 || Raue || [[Lessons Learned from Quantitative Dynamical Modeling in Systems Biology]]<br />
|-<br />
| 2013 || Schenker || [[ODE parameter inference using adaptive gradient matching with Gaussian processes]]<br />
|-<br />
| 2017 || Ballnus || [[Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems]]<br />
|-<br />
| 2017 || Henriques || [[Data-driven reverse engineering of signaling pathways using ensembles of dynamic models]]<br />
|-<br />
| 2017 || Melicher || [[Fast derivatives of likelihood functionals for ODE based models using adjoint-state method]]<br />
|-<br />
| 2017 || Schenker || [[Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy]]<br />
|-<br />
| 2017 || Schenker || [[Performance of objective functions and optimization procedures for parameter estimation in system biology models]]<br />
|-<br />
| 2017 || Schenker || [[Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks]]<br />
|-<br />
| 2018 || Schenker || [[Evaluation of Derivative-Free Optimizers for Parameter Estimation in Systems Biology]]<br />
|-<br />
| 2018 || Schenker || [[Hierarchical optimization for the efficient parametrization of ODE models]]<br />
|-<br />
| 2018 || Schenker || [[Input-dependent structural identifiability of nonlinear systems]]<br />
|-<br />
| 2018 || Schenker || [[Optimization and profile calculation of ODE models using second order adjoint sensitivity analysis]]<br />
|-<br />
| 2019 || Schenker || [[A comparison of methods for quantifying prediction uncertainty in systems biology]]<br />
|-<br />
| 2019 || Schenker || [[Benchmark problems for dynamic modeling of intracellular processes]]<br />
|-<br />
| 2019 || Schenker || [[Benchmarking optimization methods for parameter estimation in large kinetic models]]<br />
|-<br />
| 2019 || Schenker || [[Efficient computation of steady states in large-scale ODE models of biochemical reaction networks]]<br />
|-<br />
| 2019 || Schenker || [[Full observability and estimation of unknown inputs, states and parameters of nonlinear biological models]]<br />
|-<br />
| 2019 || Schenker || [[Mini-batch optimization enables training of ODE models on large-scale datasets]]<br />
|-<br />
| 2019 || Schenker || [[Parameter Estimation and Variable Selection for Big Systems of Linear Ordinary Differential Equations: A Matrix-Based Approach]]<br />
|-<br />
| 2019 || Schenker || [[Parameter estimation in models of biological oscillators: an automated regularised estimation approach]]<br />
|-<br />
| 2019 || Schenker || [[Robust calibration of hierarchical population models for heterogeneous cell populations]]<br />
|-<br />
| 2019 || Schenker || [[Scalable nonlinear programming framework for parameter estimation in dynamic biological system models]]<br />
|-<br />
| 2019 || Schenker || [[Tracking for parameter and state estimation in possibly misspecified partially observed linear Ordinary Differential Equations]]<br />
|-<br />
| 2020 || Schenker || [[An application of Conditional RobustCalibration (CRC) to ordinary differential equations (ODEs) models in computational systems biology: a comparison of two sampling strategies]]<br />
|-<br />
| 2020 || Schenker || [[Efficient parameterization of large-scale dynamic models based on relative measurements]]<br />
|-<br />
| 2020 || Schenker || [[Testing structural identifiability by a simple scaling method]]<br />
|}<br />
<br />
=== Omics Workflows ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2008 || Neuweger H || [[MeltDB: a software platform for the analysis and integration of metabolomics experiment data]]<br />
|-<br />
| 2008 || Barla A || [[Machine learning methods for predictive proteomics]]<br />
|-<br />
| 2009 || Xia J || [[MetaboAnalyst: a web server for metabolomic data analysis and interpretation]]<br />
|-<br />
| 2013 || Weisser H || [[An Automated Pipeline for High-Throughput Label-Free Quantitative Proteomics]]<br />
|-<br />
| 2014 || Cox J || [[Accurate Proteome-wide Label-free Quantification by Delayed Normalization and Maximal Peptide Ratio Extraction, Termed MaxLFQ* ]]<br />
|-<br />
| 2015 || || [[ComparingVariant Call Files for Performance Benchmarkingof Next-Generation Sequencing Variant Calling Pipelines]]<br />
|-<br />
| 2016 || Tyanova S || [[The MaxQuant computational platform for mass spectrometry–based shotgun proteomics]]<br />
|-<br />
| 2016 || Röst HL || [[OpenMS: a flexible open-source software platform for mass spectrometry data analysis]]<br />
|-<br />
| 2017 || || [[A benchmarking of workflows for detecting differential splicing and differential expression at isoform level in human RNA-seq studies]]<br />
|-<br />
| 2018 || Välikangas T || [[A comprehensive evaluation of popular proteomics software workflows for label-free proteome quantification and imputation]]<br />
|-<br />
| 2019 || || [[A Systematic Evaluation of Single CellRNA-Seq Analysis Pipelines]]<br />
|-<br />
| 2019 || || [[Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays]]<br />
|}<br />
<br />
=== Preprocessing high-throughput data===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|- 1999 || Perkins DN || [[Probability-based protein identification by searching sequence databases using mass spectrometry data]]<br />
|-<br />
| 2003 || || [[A comparison of normalization methods for high density oligonucleotide array data based on variance and bias ]]<br />
|-<br />
| 2005 || || [[Comparison of Affymetrix GeneChip Expression Measures]]<br />
|-<br />
| 2005 || Meleth S || [[The case for well-conducted experiments to validate statistical protocols for 2D gels: different pre-processing = different lists of significant proteins]]<br />
|-<br />
| 2005 || || [[Comparison of background correction and normalization procedures for high-density oligonucleotide microarrays]]<br />
|-<br />
| 2006 || || [[Using RNA sample titrations to assess microarray platform performance and normalization techniques]]<br />
|-<br />
| 2006 || Wang P || [[Normalization regarding non-random missing values in high-throughput mass spectrometry data]]<br />
|-<br />
| 2007 || Carvalho B || [[Exploration, normalization, and genotype calls of high-density oligonucleotide SNP array data]]<br />
|-<br />
| 2007 || Cannataro M || [[MS‐Analyzer: preprocessing and data mining services for proteomics applications on the Grid]]<br />
|-<br />
| 2008 || || [[Comparison of preprocessing methods for the hgU133+2 chip from Affymetrix]]<br />
|-<br />
| 2009 || || [[Comparison of Affymetrix data normalization methods using 6,926 experiments across five array generations]]<br />
|-<br />
| 2009 || Mar JC || [[Data-driven normalization strategies for high-throughput quantitative RT-PCR]]<br />
|-<br />
| 2009 || Vakhrushev SY || [[Software platform for high-throughput glycomics]]<br />
|-<br />
| 2010 || || [[Consistency of predictive signature genes and classifiers generated using different microarray platforms]]<br />
|-<br />
| 2010 || || [[Detecting and correcting systematic variation in large-scale RNA sequencing data]]<br />
|-<br />
| 2010 || || [[Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments]]<br />
|-<br />
| 2010 || || [[Normalization of RNA-seq data using factor analysis of control genes or samples]]<br />
|-<br />
| 2011 || || [[Affymetrix GeneChip microarray preprocessing for multivariate analyses]]<br />
|-<br />
| 2012 || || [[A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis]]<br />
|-<br />
| 2014 || || [[Normalyzer: A Tool for Rapid Evaluation of Normalization Methods for Omics Data Sets]]<br />
|-<br />
| 2014 || Zhou X || [[Prevention, diagnosis and treatment of high-throughput sequencing data pathologies]]<br />
|-<br />
| 2014 || Coble JB || [[Comparative evaluation of preprocessing freeware on chromatography/mass spectrometry data for signature discovery]]<br />
|-<br />
| 2015 || Caraus I || [[Detecting and overcoming systematic bias in high-throughput screening technologies: a comprehensive review of practical issues and methodological solutions]]<br />
|-<br />
| 2015 || Tam S || [[Optimization of miRNA-seq data preprocessing]]<br />
|-<br />
| 2015 || Rafiei A || [[Comparison of peak‐picking workflows for untargeted liquid chromatography/high‐resolution mass spectrometry metabolomics data analysis]]<br />
|-<br />
| 2015 || Chawade A || [[Data processing has major impact on the outcome of quantitative label-free LC-MS analysis]]<br />
|-<br />
| 2015 || Wang T || [[A systematic study of normalization methods for Infinium 450K methylation data using whole-genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Yi L || [[Chemometric methods in data processing of mass spectrometry-based metabolomics: A review]]<br />
|-<br />
| 2016 || Tsuji J || [[Evaluation of preprocessing, mapping and postprocessing algorithms for analyzing whole genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Li B || [[Performance Evaluation and Online Realization of Data-driven Normalization Methods Used in LC/MS based Untargeted Metabolomics Analysis]]<br />
|-<br />
| 2017 || Li B || [[NOREVA: normalization and evaluation of MS-based metabolomics data]]<br />
|-<br />
| 2018 || Mazoure B || [[Identification and Correction of Additive and Multiplicative Spatial Biases in Experimental High-Throughput Screening]]<br />
|-<br />
| 2018 || Li Z || [[Comprehensive evaluation of untargeted metabolomics data processing software in feature detection, quantification and discriminating marker selection]]<br />
|-<br />
| 2018 || Willforss J || [[NormalyzerDE: Online Tool for Improved Normalization of Omics Expression Data and High-Sensitivity Differential Expression Analysis]]<br />
|}</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Parameter_Estimation_and_Variable_Selection_for_Big_Systems_of_Linear_Ordinary_Differential_Equations:_A_Matrix-Based_Approach&diff=693Parameter Estimation and Variable Selection for Big Systems of Linear Ordinary Differential Equations: A Matrix-Based Approach2020-02-25T15:34:19Z<p>Bwday: </p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
=== Citation ===<br />
Wu, Leqin, et al. "Parameter estimation and variable selection for big systems of linear ordinary differential equations: A matrix-based approach." Journal of the American Statistical Association 114.526 (2019): 657-667.<br />
<br />
[https://doi.org/10.1080/01621459.2017.1423074 Permanent link to the paper]</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Literature_Studies&diff=691Literature Studies2020-02-25T15:30:38Z<p>Bwday: /* ODE-based Modelling */</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
{| class="wikitable"<br />
|-<br />
! Page summary<br />
|-<br />
| Here outcomes of benchmarking studies from the literature are collected. The primary aim is a comprehensive overview about neutral benchmark studies, i.e. assessments which were performed independenty on publication of a new approach. Studies which are not neutral are put in brackets. </br> <br />
<br />
The focus is on computational methods for analyzing experimental data (instead of comparing experimental techniques or platforms). </br><br />
<br />
Please extend this list by creating a new page and adding a link below. </br> <br />
Use the '''[[Guidelines_for_Summarizing_a_Literature_Study|guidelines described here]]'''.<br />
|}<br />
<br />
== Results from Literature ==<br />
<br />
=== Classification ===<br />
''' 2003 '''</br><br />
* [[Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data]]<br />
''' 2005 '''</br><br />
* [[A review and comparison of classification algorithms for medical decision making]]<br />
''' 2016 '''</br><br />
* [[Predicting Breast Cancer Survivability Using Data Mining Techniques]]<br />
<br />
=== Selection of Differential Features and Regions ===<br />
==== Identifying differential features ====<br />
''' 2006 '''</br><br />
* [[Rat toxicogenomic study reveals analytical consistency across microarray platforms]]<br />
''' 2010 '''</br><br />
* [[A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the sequencing Quality control consortium]]<br />
''' 2017 '''</br><br />
* [[Identification of differentially expressed peptides in high-throughput proteomics data]]<br />
* [[In-depth method assessments of differentially expressed protein detection for shotgun proteomics data with missing values]]<br />
* [[Strategies for analyzing bisulfite sequencing data]]<br />
''' 2018 '''</br><br />
* [[Identification of Differentially Methylated Sites with Weak Methylation Effects]]<br />
<br />
==== Identifying differential regions (e.g. DMRs) ====<br />
''' 2015 '''<br />
* [[De novo identification of differentially methylated regions in the human genome]]<br />
* [[MethylAction: detecting differentially methylated regions that distinguish biological subtypes]]<br />
* [[metilene: Fast and sensitive calling of differentially methylated regions from bisulfite sequencing data]]<br />
''' 2016 '''<br />
* [[seqlm: an MDL based method for identifying differentially methylated regions in high density methylation array data]]<br />
* [[Statistical methods for detecting differentially methylated regions based on MethylCap-seq data]]<br />
''' 2017 '''<br />
* [[DMRfinder: efficiently identifying differentially methylated regions from MethylC-seq data]]<br />
''' 2018 '''<br />
* [[Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus]]<br />
* [[DMRcaller: a versatile R/Bioconductor package for detection and visualization of differentially methylated regions in CpG and non-CpG contexts]]<br />
* [[MethCP: Differentially Methylated Region Detection with Change Point Models (bioRxiv)]]<br />
<br />
==== Identifying sets of features (e.g. gene set analyses) ====<br />
'''2009<br />
<br />
* [[A general modular framework for gene set enrichment analysis]]<br />
* [[Comparing gene set analysis methods on single-nucleotide polymorphism data from Genetic Analysis Workshop 16]]<br />
<br />
'''2018<br />
<br />
* [[Gene set analysis methods: a systematic comparison]]<br />
<br />
==== Dimension reduction ====<br />
''' 2008 '''</br><br />
* [[On the Relationship Between Feature Selection and Classification Accuracy]]<br />
''' 2015 '''</br><br />
* [[Comparing feature selection methods for highdimensional imbalanced data: identifying rheumatoid arthritis cohorts from routine data]]<br />
<br />
=== Imputation methods for missing values ===<br />
<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 1996 || Schenker || [[Partially parametric techniques for multiple imputation]]<br />
|-<br />
| 1999 || Hastie T || [[Imputing Missing Data for Gene Expression Arrays]]<br />
|-<br />
| 2001 || Troyanskaya || [[Missing value estimation methods for DNA microarrays]]<br />
|-<br />
| 2002 || Engels J || [[Imputation of missing longitudinal data: a comparison of methods]]<br />
|-<br />
| 2003 || Oba || [[A Bayesian missing value estimation method for gene expression profile data]]<br />
|-<br />
| 2005 || Scholz || [[Nonlinear PCA: a missing data approach]]<br />
|-<br />
| 2007 || Stacklies || [[pcaMethods—a bioconductor package providing PCA methods for incomplete data]]<br />
|-<br />
| 2007 || Verboven || [[Sequential imputation for missing values]]<br />
|-<br />
| 2008 || Shaffer GN || [[Which missing value imputation method to use in expression profiles: a comparative study and two selection schemes]]<br />
|-<br />
| 2011 || Templ || [[Iterative stepwise regression imputation using standard and robust methods]]<br />
|-<br />
| 2012 || Hrydziuszko O || [[Missing values in mass spectrometry based metabolomics: an undervalued step in the data processing pipeline]]<br />
|-<br />
| 2012 || Stekhoven || [[MissForest—non-parametric missing value imputation for mixed-type data]]<br />
|-<br />
| 2013 || Taylor || [[Accounting for undetected compounds in statistical analyses of mass spectrometry ‘omic studies]]<br />
|-<br />
| 2013 || Waljee || [[Comparison of imputation methods for missing laboratory data in medicine]]<br />
|-<br />
| 2014 || Shah || [[Comparison of Random Forest and Parametric Imputation Models for Imputing Missing Data Using MICE: A CALIBER Study]]<br />
|-<br />
| 2014 || Rodwell || [[Comparison of methods for imputing limited-range variables: a simulation study]]<br />
|-<br />
| 2014 || Morris || [[Tuning multiple imputation by predictive mean matching and local residual draws]]<br />
|-<br />
| 2014 || Doove L || [[Recursive partitioning for missing data imputation in the presence of interaction effects]]<br />
|-<br />
| 2015 || Webb-Robertson BJM || [[Review, evaluation, and discussion of the challenges of missing value imputation for mass spectrometry-based label-free global proteomics]]<br />
|-<br />
| 2016 || Folch-Fortuny A || [[Assessment of maximum likelihood PCA missing data imputation]]<br />
|-<br />
| 2016 || Lazar C || [[Accounting for the Multiple Natures of Missing Values in Label-Free Quantitative Proteomics Data Sets to Compare Imputation Strategies]]<br />
|-<br />
| 2016 || Yin X || [[Multiple imputation and analysis for high-dimensional incomplete proteomics data]]<br />
|-<br />
| 2018 || Wei R || [[Missing Value Imputation Approach for Mass Spectrometry-based Metabolomics Data]]<br />
|-<br />
| 2018 || Poyatos R || [[Gap-filling a spatially explicit plant trait database: comparing imputation methods and different levels of environmental information]]<br />
|-<br />
| 2018 || O'Brien JJ || [[The effects of nonignorable missing data on label-free mass spectrometry proteomics experiments]]<br />
|}<br />
<br />
=== ODE-based Modelling ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2001 || Schenker || [[Ways to Fit a PK Model with Some Data Below the Quantification Limit]]<br />
|-<br />
| 2008 || Balsa-Canto || [[Hybrid optimization method with general switching strategy for parameter estimation]]<br />
|-<br />
| 2011 || Tashkova || [[Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis]]<br />
|-<br />
| 2013 || Raue || [[Lessons Learned from Quantitative Dynamical Modeling in Systems Biology]]<br />
|-<br />
| 2013 || Schenker || [[ODE parameter inference using adaptive gradient matching with Gaussian processes]]<br />
|-<br />
| 2017 || Ballnus || [[Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems]]<br />
|-<br />
| 2017 || Henriques || [[Data-driven reverse engineering of signaling pathways using ensembles of dynamic models]]<br />
|-<br />
| 2017 || Melicher || [[Fast derivatives of likelihood functionals for ODE based models using adjoint-state method]]<br />
|-<br />
| 2017 || Schenker || [[Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy]]<br />
|-<br />
| 2017 || Schenker || [[Performance of objective functions and optimization procedures for parameter estimation in system biology models]]<br />
|-<br />
| 2017 || Schenker || [[Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks]]<br />
|-<br />
| 2018 || Schenker || [[Evaluation of Derivative-Free Optimizers for Parameter Estimation in Systems Biology]]<br />
|-<br />
| 2018 || Schenker || [[Hierarchical optimization for the efficient parametrization of ODE models]]<br />
|-<br />
| 2018 || Schenker || [[Input-dependent structural identifiability of nonlinear systems]]<br />
|-<br />
| 2018 || Schenker || [[Optimization and profile calculation of ODE models using second order adjoint sensitivity analysis]]<br />
|-<br />
| 2019 || Schenker || [[A comparison of methods for quantifying prediction uncertainty in systems biology]]<br />
|-<br />
| 2019 || Schenker || [[Benchmark problems for dynamic modeling of intracellular processes]]<br />
|-<br />
| 2019 || Schenker || [[Benchmarking optimization methods for parameter estimation in large kinetic models]]<br />
|-<br />
| 2019 || Schenker || [[Efficient computation of steady states in large-scale ODE models of biochemical reaction networks]]<br />
|-<br />
| 2019 || Schenker || [[Full observability and estimation of unknown inputs, states and parameters of nonlinear biological models]]<br />
|-<br />
| 2019 || Schenker || [[Mini-batch optimization enables training of ODE models on large-scale datasets]]<br />
|-<br />
| 2019 || Schenker || [[Parameter Estimation and Variable Selection for Big Systems of Linear Ordinary Differential Equations: A Matrix-Based Approach]]<br />
|-<br />
| 2019 || Schenker || [[Parameter estimation in models of biological oscillators: an automated regularised estimation approach]]<br />
|-<br />
| 2019 || Schenker || [[Robust calibration of hierarchical population models for heterogeneous cell populations]]<br />
|-<br />
| 2019 || Schenker || [[Scalable nonlinear programming framework for parameter estimation in dynamic biological system models]]<br />
|-<br />
| 2019 || Schenker || [[Tracking for parameter and state estimation in possibly misspecified partially observed linear Ordinary Differential Equations]]<br />
|-<br />
| 2020 || Schenker || [[An application of Conditional RobustCalibration (CRC) to ordinary differential equations (ODEs) models in computational systems biology: a comparison of two sampling strategies]]<br />
|-<br />
| 2020 || Schenker || [[Efficient parameterization of large-scale dynamic models based on relative measurements]]<br />
|-<br />
| 2020 || Schenker || [[Testing structural identifiability by a simple scaling method]]<br />
|}<br />
<br />
=== Omics Workflows ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2008 || Neuweger H || [[MeltDB: a software platform for the analysis and integration of metabolomics experiment data]]<br />
|-<br />
| 2008 || Barla A || [[Machine learning methods for predictive proteomics]]<br />
|-<br />
| 2009 || Xia J || [[MetaboAnalyst: a web server for metabolomic data analysis and interpretation]]<br />
|-<br />
| 2013 || Weisser H || [[An Automated Pipeline for High-Throughput Label-Free Quantitative Proteomics]]<br />
|-<br />
| 2014 || Cox J || [[Accurate Proteome-wide Label-free Quantification by Delayed Normalization and Maximal Peptide Ratio Extraction, Termed MaxLFQ* ]]<br />
|-<br />
| 2015 || || [[ComparingVariant Call Files for Performance Benchmarkingof Next-Generation Sequencing Variant Calling Pipelines]]<br />
|-<br />
| 2016 || Tyanova S || [[The MaxQuant computational platform for mass spectrometry–based shotgun proteomics]]<br />
|-<br />
| 2016 || Röst HL || [[OpenMS: a flexible open-source software platform for mass spectrometry data analysis]]<br />
|-<br />
| 2017 || || [[A benchmarking of workflows for detecting differential splicing and differential expression at isoform level in human RNA-seq studies]]<br />
|-<br />
| 2018 || Välikangas T || [[A comprehensive evaluation of popular proteomics software workflows for label-free proteome quantification and imputation]]<br />
|-<br />
| 2019 || || [[A Systematic Evaluation of Single CellRNA-Seq Analysis Pipelines]]<br />
|-<br />
| 2019 || || [[Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays]]<br />
|}<br />
<br />
=== Preprocessing high-throughput data===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|- 1999 || Perkins DN || [[Probability-based protein identification by searching sequence databases using mass spectrometry data]]<br />
|-<br />
| 2003 || || [[A comparison of normalization methods for high density oligonucleotide array data based on variance and bias ]]<br />
|-<br />
| 2005 || || [[Comparison of Affymetrix GeneChip Expression Measures]]<br />
|-<br />
| 2005 || Meleth S || [[The case for well-conducted experiments to validate statistical protocols for 2D gels: different pre-processing = different lists of significant proteins]]<br />
|-<br />
| 2005 || || [[Comparison of background correction and normalization procedures for high-density oligonucleotide microarrays]]<br />
|-<br />
| 2006 || || [[Using RNA sample titrations to assess microarray platform performance and normalization techniques]]<br />
|-<br />
| 2006 || Wang P || [[Normalization regarding non-random missing values in high-throughput mass spectrometry data]]<br />
|-<br />
| 2007 || Carvalho B || [[Exploration, normalization, and genotype calls of high-density oligonucleotide SNP array data]]<br />
|-<br />
| 2007 || Cannataro M || [[MS‐Analyzer: preprocessing and data mining services for proteomics applications on the Grid]]<br />
|-<br />
| 2008 || || [[Comparison of preprocessing methods for the hgU133+2 chip from Affymetrix]]<br />
|-<br />
| 2009 || || [[Comparison of Affymetrix data normalization methods using 6,926 experiments across five array generations]]<br />
|-<br />
| 2009 || Mar JC || [[Data-driven normalization strategies for high-throughput quantitative RT-PCR]]<br />
|-<br />
| 2009 || Vakhrushev SY || [[Software platform for high-throughput glycomics]]<br />
|-<br />
| 2010 || || [[Consistency of predictive signature genes and classifiers generated using different microarray platforms]]<br />
|-<br />
| 2010 || || [[Detecting and correcting systematic variation in large-scale RNA sequencing data]]<br />
|-<br />
| 2010 || || [[Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments]]<br />
|-<br />
| 2010 || || [[Normalization of RNA-seq data using factor analysis of control genes or samples]]<br />
|-<br />
| 2011 || || [[Affymetrix GeneChip microarray preprocessing for multivariate analyses]]<br />
|-<br />
| 2012 || || [[A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis]]<br />
|-<br />
| 2014 || || [[Normalyzer: A Tool for Rapid Evaluation of Normalization Methods for Omics Data Sets]]<br />
|-<br />
| 2014 || Zhou X || [[Prevention, diagnosis and treatment of high-throughput sequencing data pathologies]]<br />
|-<br />
| 2014 || Coble JB || [[Comparative evaluation of preprocessing freeware on chromatography/mass spectrometry data for signature discovery]]<br />
|-<br />
| 2015 || Caraus I || [[Detecting and overcoming systematic bias in high-throughput screening technologies: a comprehensive review of practical issues and methodological solutions]]<br />
|-<br />
| 2015 || Tam S || [[Optimization of miRNA-seq data preprocessing]]<br />
|-<br />
| 2015 || Rafiei A || [[Comparison of peak‐picking workflows for untargeted liquid chromatography/high‐resolution mass spectrometry metabolomics data analysis]]<br />
|-<br />
| 2015 || Chawade A || [[Data processing has major impact on the outcome of quantitative label-free LC-MS analysis]]<br />
|-<br />
| 2016 || Yi L || [[Chemometric methods in data processing of mass spectrometry-based metabolomics: A review]]<br />
|-<br />
| 2016 || Tsuji J || [[Evaluation of preprocessing, mapping and postprocessing algorithms for analyzing whole genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Li B || [[Performance Evaluation and Online Realization of Data-driven Normalization Methods Used in LC/MS based Untargeted Metabolomics Analysis]]<br />
|-<br />
| 2017 || Li B || [[NOREVA: normalization and evaluation of MS-based metabolomics data]]<br />
|-<br />
| 2018 || Mazoure B || [[Identification and Correction of Additive and Multiplicative Spatial Biases in Experimental High-Throughput Screening]]<br />
|-<br />
| 2018 || Li Z || [[Comprehensive evaluation of untargeted metabolomics data processing software in feature detection, quantification and discriminating marker selection]]<br />
|-<br />
| 2018 || Willforss J || [[NormalyzerDE: Online Tool for Improved Normalization of Omics Expression Data and High-Sensitivity Differential Expression Analysis]]<br />
|}</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Literature_Studies&diff=690Literature Studies2020-02-25T15:30:20Z<p>Bwday: /* ODE-based Modelling */</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
{| class="wikitable"<br />
|-<br />
! Page summary<br />
|-<br />
| Here outcomes of benchmarking studies from the literature are collected. The primary aim is a comprehensive overview about neutral benchmark studies, i.e. assessments which were performed independenty on publication of a new approach. Studies which are not neutral are put in brackets. </br> <br />
<br />
The focus is on computational methods for analyzing experimental data (instead of comparing experimental techniques or platforms). </br><br />
<br />
Please extend this list by creating a new page and adding a link below. </br> <br />
Use the '''[[Guidelines_for_Summarizing_a_Literature_Study|guidelines described here]]'''.<br />
|}<br />
<br />
== Results from Literature ==<br />
<br />
=== Classification ===<br />
''' 2003 '''</br><br />
* [[Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data]]<br />
''' 2005 '''</br><br />
* [[A review and comparison of classification algorithms for medical decision making]]<br />
''' 2016 '''</br><br />
* [[Predicting Breast Cancer Survivability Using Data Mining Techniques]]<br />
<br />
=== Selection of Differential Features and Regions ===<br />
==== Identifying differential features ====<br />
''' 2006 '''</br><br />
* [[Rat toxicogenomic study reveals analytical consistency across microarray platforms]]<br />
''' 2010 '''</br><br />
* [[A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the sequencing Quality control consortium]]<br />
''' 2017 '''</br><br />
* [[Identification of differentially expressed peptides in high-throughput proteomics data]]<br />
* [[In-depth method assessments of differentially expressed protein detection for shotgun proteomics data with missing values]]<br />
* [[Strategies for analyzing bisulfite sequencing data]]<br />
''' 2018 '''</br><br />
* [[Identification of Differentially Methylated Sites with Weak Methylation Effects]]<br />
<br />
==== Identifying differential regions (e.g. DMRs) ====<br />
''' 2015 '''<br />
* [[De novo identification of differentially methylated regions in the human genome]]<br />
* [[MethylAction: detecting differentially methylated regions that distinguish biological subtypes]]<br />
* [[metilene: Fast and sensitive calling of differentially methylated regions from bisulfite sequencing data]]<br />
''' 2016 '''<br />
* [[seqlm: an MDL based method for identifying differentially methylated regions in high density methylation array data]]<br />
* [[Statistical methods for detecting differentially methylated regions based on MethylCap-seq data]]<br />
''' 2017 '''<br />
* [[DMRfinder: efficiently identifying differentially methylated regions from MethylC-seq data]]<br />
''' 2018 '''<br />
* [[Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus]]<br />
* [[DMRcaller: a versatile R/Bioconductor package for detection and visualization of differentially methylated regions in CpG and non-CpG contexts]]<br />
* [[MethCP: Differentially Methylated Region Detection with Change Point Models (bioRxiv)]]<br />
<br />
==== Identifying sets of features (e.g. gene set analyses) ====<br />
'''2009<br />
<br />
* [[A general modular framework for gene set enrichment analysis]]<br />
* [[Comparing gene set analysis methods on single-nucleotide polymorphism data from Genetic Analysis Workshop 16]]<br />
<br />
'''2018<br />
<br />
* [[Gene set analysis methods: a systematic comparison]]<br />
<br />
==== Dimension reduction ====<br />
''' 2008 '''</br><br />
* [[On the Relationship Between Feature Selection and Classification Accuracy]]<br />
''' 2015 '''</br><br />
* [[Comparing feature selection methods for highdimensional imbalanced data: identifying rheumatoid arthritis cohorts from routine data]]<br />
<br />
=== Imputation methods for missing values ===<br />
<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 1996 || Schenker || [[Partially parametric techniques for multiple imputation]]<br />
|-<br />
| 1999 || Hastie T || [[Imputing Missing Data for Gene Expression Arrays]]<br />
|-<br />
| 2001 || Troyanskaya || [[Missing value estimation methods for DNA microarrays]]<br />
|-<br />
| 2002 || Engels J || [[Imputation of missing longitudinal data: a comparison of methods]]<br />
|-<br />
| 2003 || Oba || [[A Bayesian missing value estimation method for gene expression profile data]]<br />
|-<br />
| 2005 || Scholz || [[Nonlinear PCA: a missing data approach]]<br />
|-<br />
| 2007 || Stacklies || [[pcaMethods—a bioconductor package providing PCA methods for incomplete data]]<br />
|-<br />
| 2007 || Verboven || [[Sequential imputation for missing values]]<br />
|-<br />
| 2008 || Shaffer GN || [[Which missing value imputation method to use in expression profiles: a comparative study and two selection schemes]]<br />
|-<br />
| 2011 || Templ || [[Iterative stepwise regression imputation using standard and robust methods]]<br />
|-<br />
| 2012 || Hrydziuszko O || [[Missing values in mass spectrometry based metabolomics: an undervalued step in the data processing pipeline]]<br />
|-<br />
| 2012 || Stekhoven || [[MissForest—non-parametric missing value imputation for mixed-type data]]<br />
|-<br />
| 2013 || Taylor || [[Accounting for undetected compounds in statistical analyses of mass spectrometry ‘omic studies]]<br />
|-<br />
| 2013 || Waljee || [[Comparison of imputation methods for missing laboratory data in medicine]]<br />
|-<br />
| 2014 || Shah || [[Comparison of Random Forest and Parametric Imputation Models for Imputing Missing Data Using MICE: A CALIBER Study]]<br />
|-<br />
| 2014 || Rodwell || [[Comparison of methods for imputing limited-range variables: a simulation study]]<br />
|-<br />
| 2014 || Morris || [[Tuning multiple imputation by predictive mean matching and local residual draws]]<br />
|-<br />
| 2014 || Doove L || [[Recursive partitioning for missing data imputation in the presence of interaction effects]]<br />
|-<br />
| 2015 || Webb-Robertson BJM || [[Review, evaluation, and discussion of the challenges of missing value imputation for mass spectrometry-based label-free global proteomics]]<br />
|-<br />
| 2016 || Folch-Fortuny A || [[Assessment of maximum likelihood PCA missing data imputation]]<br />
|-<br />
| 2016 || Lazar C || [[Accounting for the Multiple Natures of Missing Values in Label-Free Quantitative Proteomics Data Sets to Compare Imputation Strategies]]<br />
|-<br />
| 2016 || Yin X || [[Multiple imputation and analysis for high-dimensional incomplete proteomics data]]<br />
|-<br />
| 2018 || Wei R || [[Missing Value Imputation Approach for Mass Spectrometry-based Metabolomics Data]]<br />
|-<br />
| 2018 || Poyatos R || [[Gap-filling a spatially explicit plant trait database: comparing imputation methods and different levels of environmental information]]<br />
|-<br />
| 2018 || O'Brien JJ || [[The effects of nonignorable missing data on label-free mass spectrometry proteomics experiments]]<br />
|}<br />
<br />
=== ODE-based Modelling ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2001 || Schenker || [[Ways to Fit a PK Model with Some Data Below the Quantification Limit]]<br />
|-<br />
| 2008 || Balsa-Canto || [[Hybrid optimization method with general switching strategy for parameter estimation]]<br />
|-<br />
| 2011 || Tashkova || [[Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis]]<br />
|-<br />
| 2013 || Raue || [[Lessons Learned from Quantitative Dynamical Modeling in Systems Biology]]<br />
|-<br />
| 2013 || Schenker || [[ODE parameter inference using adaptive gradient matching with Gaussian processes]]<br />
|-<br />
| 2017 || Ballnus || [[Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems]]<br />
|-<br />
| 2017 || Henriques || [[Data-driven reverse engineering of signaling pathways using ensembles of dynamic models]]<br />
|-<br />
| 2017 || Schenker || [[Fast derivatives of likelihood functionals for ODE based models using adjoint-state method]]<br />
|-<br />
| 2017 || Schenker || [[Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy]]<br />
|-<br />
| 2017 || Schenker || [[Performance of objective functions and optimization procedures for parameter estimation in system biology models]]<br />
|-<br />
| 2017 || Schenker || [[Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks]]<br />
|-<br />
| 2018 || Schenker || [[Evaluation of Derivative-Free Optimizers for Parameter Estimation in Systems Biology]]<br />
|-<br />
| 2018 || Schenker || [[Hierarchical optimization for the efficient parametrization of ODE models]]<br />
|-<br />
| 2018 || Schenker || [[Input-dependent structural identifiability of nonlinear systems]]<br />
|-<br />
| 2018 || Schenker || [[Optimization and profile calculation of ODE models using second order adjoint sensitivity analysis]]<br />
|-<br />
| 2019 || Schenker || [[A comparison of methods for quantifying prediction uncertainty in systems biology]]<br />
|-<br />
| 2019 || Schenker || [[Benchmark problems for dynamic modeling of intracellular processes]]<br />
|-<br />
| 2019 || Schenker || [[Benchmarking optimization methods for parameter estimation in large kinetic models]]<br />
|-<br />
| 2019 || Schenker || [[Efficient computation of steady states in large-scale ODE models of biochemical reaction networks]]<br />
|-<br />
| 2019 || Schenker || [[Full observability and estimation of unknown inputs, states and parameters of nonlinear biological models]]<br />
|-<br />
| 2019 || Schenker || [[Mini-batch optimization enables training of ODE models on large-scale datasets]]<br />
|-<br />
| 2019 || Schenker || [[Parameter Estimation and Variable Selection for Big Systems of Linear Ordinary Differential Equations: A Matrix-Based Approach]]<br />
|-<br />
| 2019 || Schenker || [[Parameter estimation in models of biological oscillators: an automated regularised estimation approach]]<br />
|-<br />
| 2019 || Schenker || [[Robust calibration of hierarchical population models for heterogeneous cell populations]]<br />
|-<br />
| 2019 || Schenker || [[Scalable nonlinear programming framework for parameter estimation in dynamic biological system models]]<br />
|-<br />
| 2019 || Schenker || [[Tracking for parameter and state estimation in possibly misspecified partially observed linear Ordinary Differential Equations]]<br />
|-<br />
| 2020 || Schenker || [[An application of Conditional RobustCalibration (CRC) to ordinary differential equations (ODEs) models in computational systems biology: a comparison of two sampling strategies]]<br />
|-<br />
| 2020 || Schenker || [[Efficient parameterization of large-scale dynamic models based on relative measurements]]<br />
|-<br />
| 2020 || Schenker || [[Testing structural identifiability by a simple scaling method]]<br />
|}<br />
<br />
=== Omics Workflows ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2008 || Neuweger H || [[MeltDB: a software platform for the analysis and integration of metabolomics experiment data]]<br />
|-<br />
| 2008 || Barla A || [[Machine learning methods for predictive proteomics]]<br />
|-<br />
| 2009 || Xia J || [[MetaboAnalyst: a web server for metabolomic data analysis and interpretation]]<br />
|-<br />
| 2013 || Weisser H || [[An Automated Pipeline for High-Throughput Label-Free Quantitative Proteomics]]<br />
|-<br />
| 2014 || Cox J || [[Accurate Proteome-wide Label-free Quantification by Delayed Normalization and Maximal Peptide Ratio Extraction, Termed MaxLFQ* ]]<br />
|-<br />
| 2015 || || [[ComparingVariant Call Files for Performance Benchmarkingof Next-Generation Sequencing Variant Calling Pipelines]]<br />
|-<br />
| 2016 || Tyanova S || [[The MaxQuant computational platform for mass spectrometry–based shotgun proteomics]]<br />
|-<br />
| 2016 || Röst HL || [[OpenMS: a flexible open-source software platform for mass spectrometry data analysis]]<br />
|-<br />
| 2017 || || [[A benchmarking of workflows for detecting differential splicing and differential expression at isoform level in human RNA-seq studies]]<br />
|-<br />
| 2018 || Välikangas T || [[A comprehensive evaluation of popular proteomics software workflows for label-free proteome quantification and imputation]]<br />
|-<br />
| 2019 || || [[A Systematic Evaluation of Single CellRNA-Seq Analysis Pipelines]]<br />
|-<br />
| 2019 || || [[Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays]]<br />
|}<br />
<br />
=== Preprocessing high-throughput data===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|- 1999 || Perkins DN || [[Probability-based protein identification by searching sequence databases using mass spectrometry data]]<br />
|-<br />
| 2003 || || [[A comparison of normalization methods for high density oligonucleotide array data based on variance and bias ]]<br />
|-<br />
| 2005 || || [[Comparison of Affymetrix GeneChip Expression Measures]]<br />
|-<br />
| 2005 || Meleth S || [[The case for well-conducted experiments to validate statistical protocols for 2D gels: different pre-processing = different lists of significant proteins]]<br />
|-<br />
| 2005 || || [[Comparison of background correction and normalization procedures for high-density oligonucleotide microarrays]]<br />
|-<br />
| 2006 || || [[Using RNA sample titrations to assess microarray platform performance and normalization techniques]]<br />
|-<br />
| 2006 || Wang P || [[Normalization regarding non-random missing values in high-throughput mass spectrometry data]]<br />
|-<br />
| 2007 || Carvalho B || [[Exploration, normalization, and genotype calls of high-density oligonucleotide SNP array data]]<br />
|-<br />
| 2007 || Cannataro M || [[MS‐Analyzer: preprocessing and data mining services for proteomics applications on the Grid]]<br />
|-<br />
| 2008 || || [[Comparison of preprocessing methods for the hgU133+2 chip from Affymetrix]]<br />
|-<br />
| 2009 || || [[Comparison of Affymetrix data normalization methods using 6,926 experiments across five array generations]]<br />
|-<br />
| 2009 || Mar JC || [[Data-driven normalization strategies for high-throughput quantitative RT-PCR]]<br />
|-<br />
| 2009 || Vakhrushev SY || [[Software platform for high-throughput glycomics]]<br />
|-<br />
| 2010 || || [[Consistency of predictive signature genes and classifiers generated using different microarray platforms]]<br />
|-<br />
| 2010 || || [[Detecting and correcting systematic variation in large-scale RNA sequencing data]]<br />
|-<br />
| 2010 || || [[Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments]]<br />
|-<br />
| 2010 || || [[Normalization of RNA-seq data using factor analysis of control genes or samples]]<br />
|-<br />
| 2011 || || [[Affymetrix GeneChip microarray preprocessing for multivariate analyses]]<br />
|-<br />
| 2012 || || [[A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis]]<br />
|-<br />
| 2014 || || [[Normalyzer: A Tool for Rapid Evaluation of Normalization Methods for Omics Data Sets]]<br />
|-<br />
| 2014 || Zhou X || [[Prevention, diagnosis and treatment of high-throughput sequencing data pathologies]]<br />
|-<br />
| 2014 || Coble JB || [[Comparative evaluation of preprocessing freeware on chromatography/mass spectrometry data for signature discovery]]<br />
|-<br />
| 2015 || Caraus I || [[Detecting and overcoming systematic bias in high-throughput screening technologies: a comprehensive review of practical issues and methodological solutions]]<br />
|-<br />
| 2015 || Tam S || [[Optimization of miRNA-seq data preprocessing]]<br />
|-<br />
| 2015 || Rafiei A || [[Comparison of peak‐picking workflows for untargeted liquid chromatography/high‐resolution mass spectrometry metabolomics data analysis]]<br />
|-<br />
| 2015 || Chawade A || [[Data processing has major impact on the outcome of quantitative label-free LC-MS analysis]]<br />
|-<br />
| 2016 || Yi L || [[Chemometric methods in data processing of mass spectrometry-based metabolomics: A review]]<br />
|-<br />
| 2016 || Tsuji J || [[Evaluation of preprocessing, mapping and postprocessing algorithms for analyzing whole genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Li B || [[Performance Evaluation and Online Realization of Data-driven Normalization Methods Used in LC/MS based Untargeted Metabolomics Analysis]]<br />
|-<br />
| 2017 || Li B || [[NOREVA: normalization and evaluation of MS-based metabolomics data]]<br />
|-<br />
| 2018 || Mazoure B || [[Identification and Correction of Additive and Multiplicative Spatial Biases in Experimental High-Throughput Screening]]<br />
|-<br />
| 2018 || Li Z || [[Comprehensive evaluation of untargeted metabolomics data processing software in feature detection, quantification and discriminating marker selection]]<br />
|-<br />
| 2018 || Willforss J || [[NormalyzerDE: Online Tool for Improved Normalization of Omics Expression Data and High-Sensitivity Differential Expression Analysis]]<br />
|}</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Literature_Studies&diff=689Literature Studies2020-02-25T15:30:02Z<p>Bwday: /* ODE-based Modelling */</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
{| class="wikitable"<br />
|-<br />
! Page summary<br />
|-<br />
| Here outcomes of benchmarking studies from the literature are collected. The primary aim is a comprehensive overview about neutral benchmark studies, i.e. assessments which were performed independenty on publication of a new approach. Studies which are not neutral are put in brackets. </br> <br />
<br />
The focus is on computational methods for analyzing experimental data (instead of comparing experimental techniques or platforms). </br><br />
<br />
Please extend this list by creating a new page and adding a link below. </br> <br />
Use the '''[[Guidelines_for_Summarizing_a_Literature_Study|guidelines described here]]'''.<br />
|}<br />
<br />
== Results from Literature ==<br />
<br />
=== Classification ===<br />
''' 2003 '''</br><br />
* [[Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data]]<br />
''' 2005 '''</br><br />
* [[A review and comparison of classification algorithms for medical decision making]]<br />
''' 2016 '''</br><br />
* [[Predicting Breast Cancer Survivability Using Data Mining Techniques]]<br />
<br />
=== Selection of Differential Features and Regions ===<br />
==== Identifying differential features ====<br />
''' 2006 '''</br><br />
* [[Rat toxicogenomic study reveals analytical consistency across microarray platforms]]<br />
''' 2010 '''</br><br />
* [[A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the sequencing Quality control consortium]]<br />
''' 2017 '''</br><br />
* [[Identification of differentially expressed peptides in high-throughput proteomics data]]<br />
* [[In-depth method assessments of differentially expressed protein detection for shotgun proteomics data with missing values]]<br />
* [[Strategies for analyzing bisulfite sequencing data]]<br />
''' 2018 '''</br><br />
* [[Identification of Differentially Methylated Sites with Weak Methylation Effects]]<br />
<br />
==== Identifying differential regions (e.g. DMRs) ====<br />
''' 2015 '''<br />
* [[De novo identification of differentially methylated regions in the human genome]]<br />
* [[MethylAction: detecting differentially methylated regions that distinguish biological subtypes]]<br />
* [[metilene: Fast and sensitive calling of differentially methylated regions from bisulfite sequencing data]]<br />
''' 2016 '''<br />
* [[seqlm: an MDL based method for identifying differentially methylated regions in high density methylation array data]]<br />
* [[Statistical methods for detecting differentially methylated regions based on MethylCap-seq data]]<br />
''' 2017 '''<br />
* [[DMRfinder: efficiently identifying differentially methylated regions from MethylC-seq data]]<br />
''' 2018 '''<br />
* [[Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus]]<br />
* [[DMRcaller: a versatile R/Bioconductor package for detection and visualization of differentially methylated regions in CpG and non-CpG contexts]]<br />
* [[MethCP: Differentially Methylated Region Detection with Change Point Models (bioRxiv)]]<br />
<br />
==== Identifying sets of features (e.g. gene set analyses) ====<br />
'''2009<br />
<br />
* [[A general modular framework for gene set enrichment analysis]]<br />
* [[Comparing gene set analysis methods on single-nucleotide polymorphism data from Genetic Analysis Workshop 16]]<br />
<br />
'''2018<br />
<br />
* [[Gene set analysis methods: a systematic comparison]]<br />
<br />
==== Dimension reduction ====<br />
''' 2008 '''</br><br />
* [[On the Relationship Between Feature Selection and Classification Accuracy]]<br />
''' 2015 '''</br><br />
* [[Comparing feature selection methods for highdimensional imbalanced data: identifying rheumatoid arthritis cohorts from routine data]]<br />
<br />
=== Imputation methods for missing values ===<br />
<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 1996 || Schenker || [[Partially parametric techniques for multiple imputation]]<br />
|-<br />
| 1999 || Hastie T || [[Imputing Missing Data for Gene Expression Arrays]]<br />
|-<br />
| 2001 || Troyanskaya || [[Missing value estimation methods for DNA microarrays]]<br />
|-<br />
| 2002 || Engels J || [[Imputation of missing longitudinal data: a comparison of methods]]<br />
|-<br />
| 2003 || Oba || [[A Bayesian missing value estimation method for gene expression profile data]]<br />
|-<br />
| 2005 || Scholz || [[Nonlinear PCA: a missing data approach]]<br />
|-<br />
| 2007 || Stacklies || [[pcaMethods—a bioconductor package providing PCA methods for incomplete data]]<br />
|-<br />
| 2007 || Verboven || [[Sequential imputation for missing values]]<br />
|-<br />
| 2008 || Shaffer GN || [[Which missing value imputation method to use in expression profiles: a comparative study and two selection schemes]]<br />
|-<br />
| 2011 || Templ || [[Iterative stepwise regression imputation using standard and robust methods]]<br />
|-<br />
| 2012 || Hrydziuszko O || [[Missing values in mass spectrometry based metabolomics: an undervalued step in the data processing pipeline]]<br />
|-<br />
| 2012 || Stekhoven || [[MissForest—non-parametric missing value imputation for mixed-type data]]<br />
|-<br />
| 2013 || Taylor || [[Accounting for undetected compounds in statistical analyses of mass spectrometry ‘omic studies]]<br />
|-<br />
| 2013 || Waljee || [[Comparison of imputation methods for missing laboratory data in medicine]]<br />
|-<br />
| 2014 || Shah || [[Comparison of Random Forest and Parametric Imputation Models for Imputing Missing Data Using MICE: A CALIBER Study]]<br />
|-<br />
| 2014 || Rodwell || [[Comparison of methods for imputing limited-range variables: a simulation study]]<br />
|-<br />
| 2014 || Morris || [[Tuning multiple imputation by predictive mean matching and local residual draws]]<br />
|-<br />
| 2014 || Doove L || [[Recursive partitioning for missing data imputation in the presence of interaction effects]]<br />
|-<br />
| 2015 || Webb-Robertson BJM || [[Review, evaluation, and discussion of the challenges of missing value imputation for mass spectrometry-based label-free global proteomics]]<br />
|-<br />
| 2016 || Folch-Fortuny A || [[Assessment of maximum likelihood PCA missing data imputation]]<br />
|-<br />
| 2016 || Lazar C || [[Accounting for the Multiple Natures of Missing Values in Label-Free Quantitative Proteomics Data Sets to Compare Imputation Strategies]]<br />
|-<br />
| 2016 || Yin X || [[Multiple imputation and analysis for high-dimensional incomplete proteomics data]]<br />
|-<br />
| 2018 || Wei R || [[Missing Value Imputation Approach for Mass Spectrometry-based Metabolomics Data]]<br />
|-<br />
| 2018 || Poyatos R || [[Gap-filling a spatially explicit plant trait database: comparing imputation methods and different levels of environmental information]]<br />
|-<br />
| 2018 || O'Brien JJ || [[The effects of nonignorable missing data on label-free mass spectrometry proteomics experiments]]<br />
|}<br />
<br />
=== ODE-based Modelling ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2001 || Schenker || [[Ways to Fit a PK Model with Some Data Below the Quantification Limit]]<br />
|-<br />
| 2008 || Balsa-Canto || [[Hybrid optimization method with general switching strategy for parameter estimation]]<br />
|-<br />
| 2011 || Tashkova || [[Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis]]<br />
|-<br />
| 2013 || Raue || [[Lessons Learned from Quantitative Dynamical Modeling in Systems Biology]]<br />
|-<br />
| 2013 || Schenker || [[ODE parameter inference using adaptive gradient matching with Gaussian processes]]<br />
|-<br />
| 2017 || Ballnus || [[Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems]]<br />
|-<br />
| 2017 || Schenker || [[Data-driven reverse engineering of signaling pathways using ensembles of dynamic models]]<br />
|-<br />
| 2017 || Schenker || [[Fast derivatives of likelihood functionals for ODE based models using adjoint-state method]]<br />
|-<br />
| 2017 || Schenker || [[Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy]]<br />
|-<br />
| 2017 || Schenker || [[Performance of objective functions and optimization procedures for parameter estimation in system biology models]]<br />
|-<br />
| 2017 || Schenker || [[Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks]]<br />
|-<br />
| 2018 || Schenker || [[Evaluation of Derivative-Free Optimizers for Parameter Estimation in Systems Biology]]<br />
|-<br />
| 2018 || Schenker || [[Hierarchical optimization for the efficient parametrization of ODE models]]<br />
|-<br />
| 2018 || Schenker || [[Input-dependent structural identifiability of nonlinear systems]]<br />
|-<br />
| 2018 || Schenker || [[Optimization and profile calculation of ODE models using second order adjoint sensitivity analysis]]<br />
|-<br />
| 2019 || Schenker || [[A comparison of methods for quantifying prediction uncertainty in systems biology]]<br />
|-<br />
| 2019 || Schenker || [[Benchmark problems for dynamic modeling of intracellular processes]]<br />
|-<br />
| 2019 || Schenker || [[Benchmarking optimization methods for parameter estimation in large kinetic models]]<br />
|-<br />
| 2019 || Schenker || [[Efficient computation of steady states in large-scale ODE models of biochemical reaction networks]]<br />
|-<br />
| 2019 || Schenker || [[Full observability and estimation of unknown inputs, states and parameters of nonlinear biological models]]<br />
|-<br />
| 2019 || Schenker || [[Mini-batch optimization enables training of ODE models on large-scale datasets]]<br />
|-<br />
| 2019 || Schenker || [[Parameter Estimation and Variable Selection for Big Systems of Linear Ordinary Differential Equations: A Matrix-Based Approach]]<br />
|-<br />
| 2019 || Schenker || [[Parameter estimation in models of biological oscillators: an automated regularised estimation approach]]<br />
|-<br />
| 2019 || Schenker || [[Robust calibration of hierarchical population models for heterogeneous cell populations]]<br />
|-<br />
| 2019 || Schenker || [[Scalable nonlinear programming framework for parameter estimation in dynamic biological system models]]<br />
|-<br />
| 2019 || Schenker || [[Tracking for parameter and state estimation in possibly misspecified partially observed linear Ordinary Differential Equations]]<br />
|-<br />
| 2020 || Schenker || [[An application of Conditional RobustCalibration (CRC) to ordinary differential equations (ODEs) models in computational systems biology: a comparison of two sampling strategies]]<br />
|-<br />
| 2020 || Schenker || [[Efficient parameterization of large-scale dynamic models based on relative measurements]]<br />
|-<br />
| 2020 || Schenker || [[Testing structural identifiability by a simple scaling method]]<br />
|}<br />
<br />
=== Omics Workflows ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2008 || Neuweger H || [[MeltDB: a software platform for the analysis and integration of metabolomics experiment data]]<br />
|-<br />
| 2008 || Barla A || [[Machine learning methods for predictive proteomics]]<br />
|-<br />
| 2009 || Xia J || [[MetaboAnalyst: a web server for metabolomic data analysis and interpretation]]<br />
|-<br />
| 2013 || Weisser H || [[An Automated Pipeline for High-Throughput Label-Free Quantitative Proteomics]]<br />
|-<br />
| 2014 || Cox J || [[Accurate Proteome-wide Label-free Quantification by Delayed Normalization and Maximal Peptide Ratio Extraction, Termed MaxLFQ* ]]<br />
|-<br />
| 2015 || || [[ComparingVariant Call Files for Performance Benchmarkingof Next-Generation Sequencing Variant Calling Pipelines]]<br />
|-<br />
| 2016 || Tyanova S || [[The MaxQuant computational platform for mass spectrometry–based shotgun proteomics]]<br />
|-<br />
| 2016 || Röst HL || [[OpenMS: a flexible open-source software platform for mass spectrometry data analysis]]<br />
|-<br />
| 2017 || || [[A benchmarking of workflows for detecting differential splicing and differential expression at isoform level in human RNA-seq studies]]<br />
|-<br />
| 2018 || Välikangas T || [[A comprehensive evaluation of popular proteomics software workflows for label-free proteome quantification and imputation]]<br />
|-<br />
| 2019 || || [[A Systematic Evaluation of Single CellRNA-Seq Analysis Pipelines]]<br />
|-<br />
| 2019 || || [[Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays]]<br />
|}<br />
<br />
=== Preprocessing high-throughput data===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|- 1999 || Perkins DN || [[Probability-based protein identification by searching sequence databases using mass spectrometry data]]<br />
|-<br />
| 2003 || || [[A comparison of normalization methods for high density oligonucleotide array data based on variance and bias ]]<br />
|-<br />
| 2005 || || [[Comparison of Affymetrix GeneChip Expression Measures]]<br />
|-<br />
| 2005 || Meleth S || [[The case for well-conducted experiments to validate statistical protocols for 2D gels: different pre-processing = different lists of significant proteins]]<br />
|-<br />
| 2005 || || [[Comparison of background correction and normalization procedures for high-density oligonucleotide microarrays]]<br />
|-<br />
| 2006 || || [[Using RNA sample titrations to assess microarray platform performance and normalization techniques]]<br />
|-<br />
| 2006 || Wang P || [[Normalization regarding non-random missing values in high-throughput mass spectrometry data]]<br />
|-<br />
| 2007 || Carvalho B || [[Exploration, normalization, and genotype calls of high-density oligonucleotide SNP array data]]<br />
|-<br />
| 2007 || Cannataro M || [[MS‐Analyzer: preprocessing and data mining services for proteomics applications on the Grid]]<br />
|-<br />
| 2008 || || [[Comparison of preprocessing methods for the hgU133+2 chip from Affymetrix]]<br />
|-<br />
| 2009 || || [[Comparison of Affymetrix data normalization methods using 6,926 experiments across five array generations]]<br />
|-<br />
| 2009 || Mar JC || [[Data-driven normalization strategies for high-throughput quantitative RT-PCR]]<br />
|-<br />
| 2009 || Vakhrushev SY || [[Software platform for high-throughput glycomics]]<br />
|-<br />
| 2010 || || [[Consistency of predictive signature genes and classifiers generated using different microarray platforms]]<br />
|-<br />
| 2010 || || [[Detecting and correcting systematic variation in large-scale RNA sequencing data]]<br />
|-<br />
| 2010 || || [[Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments]]<br />
|-<br />
| 2010 || || [[Normalization of RNA-seq data using factor analysis of control genes or samples]]<br />
|-<br />
| 2011 || || [[Affymetrix GeneChip microarray preprocessing for multivariate analyses]]<br />
|-<br />
| 2012 || || [[A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis]]<br />
|-<br />
| 2014 || || [[Normalyzer: A Tool for Rapid Evaluation of Normalization Methods for Omics Data Sets]]<br />
|-<br />
| 2014 || Zhou X || [[Prevention, diagnosis and treatment of high-throughput sequencing data pathologies]]<br />
|-<br />
| 2014 || Coble JB || [[Comparative evaluation of preprocessing freeware on chromatography/mass spectrometry data for signature discovery]]<br />
|-<br />
| 2015 || Caraus I || [[Detecting and overcoming systematic bias in high-throughput screening technologies: a comprehensive review of practical issues and methodological solutions]]<br />
|-<br />
| 2015 || Tam S || [[Optimization of miRNA-seq data preprocessing]]<br />
|-<br />
| 2015 || Rafiei A || [[Comparison of peak‐picking workflows for untargeted liquid chromatography/high‐resolution mass spectrometry metabolomics data analysis]]<br />
|-<br />
| 2015 || Chawade A || [[Data processing has major impact on the outcome of quantitative label-free LC-MS analysis]]<br />
|-<br />
| 2016 || Yi L || [[Chemometric methods in data processing of mass spectrometry-based metabolomics: A review]]<br />
|-<br />
| 2016 || Tsuji J || [[Evaluation of preprocessing, mapping and postprocessing algorithms for analyzing whole genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Li B || [[Performance Evaluation and Online Realization of Data-driven Normalization Methods Used in LC/MS based Untargeted Metabolomics Analysis]]<br />
|-<br />
| 2017 || Li B || [[NOREVA: normalization and evaluation of MS-based metabolomics data]]<br />
|-<br />
| 2018 || Mazoure B || [[Identification and Correction of Additive and Multiplicative Spatial Biases in Experimental High-Throughput Screening]]<br />
|-<br />
| 2018 || Li Z || [[Comprehensive evaluation of untargeted metabolomics data processing software in feature detection, quantification and discriminating marker selection]]<br />
|-<br />
| 2018 || Willforss J || [[NormalyzerDE: Online Tool for Improved Normalization of Omics Expression Data and High-Sensitivity Differential Expression Analysis]]<br />
|}</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Literature_Studies&diff=688Literature Studies2020-02-25T15:29:38Z<p>Bwday: /* ODE-based Modelling */</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
{| class="wikitable"<br />
|-<br />
! Page summary<br />
|-<br />
| Here outcomes of benchmarking studies from the literature are collected. The primary aim is a comprehensive overview about neutral benchmark studies, i.e. assessments which were performed independenty on publication of a new approach. Studies which are not neutral are put in brackets. </br> <br />
<br />
The focus is on computational methods for analyzing experimental data (instead of comparing experimental techniques or platforms). </br><br />
<br />
Please extend this list by creating a new page and adding a link below. </br> <br />
Use the '''[[Guidelines_for_Summarizing_a_Literature_Study|guidelines described here]]'''.<br />
|}<br />
<br />
== Results from Literature ==<br />
<br />
=== Classification ===<br />
''' 2003 '''</br><br />
* [[Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data]]<br />
''' 2005 '''</br><br />
* [[A review and comparison of classification algorithms for medical decision making]]<br />
''' 2016 '''</br><br />
* [[Predicting Breast Cancer Survivability Using Data Mining Techniques]]<br />
<br />
=== Selection of Differential Features and Regions ===<br />
==== Identifying differential features ====<br />
''' 2006 '''</br><br />
* [[Rat toxicogenomic study reveals analytical consistency across microarray platforms]]<br />
''' 2010 '''</br><br />
* [[A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the sequencing Quality control consortium]]<br />
''' 2017 '''</br><br />
* [[Identification of differentially expressed peptides in high-throughput proteomics data]]<br />
* [[In-depth method assessments of differentially expressed protein detection for shotgun proteomics data with missing values]]<br />
* [[Strategies for analyzing bisulfite sequencing data]]<br />
''' 2018 '''</br><br />
* [[Identification of Differentially Methylated Sites with Weak Methylation Effects]]<br />
<br />
==== Identifying differential regions (e.g. DMRs) ====<br />
''' 2015 '''<br />
* [[De novo identification of differentially methylated regions in the human genome]]<br />
* [[MethylAction: detecting differentially methylated regions that distinguish biological subtypes]]<br />
* [[metilene: Fast and sensitive calling of differentially methylated regions from bisulfite sequencing data]]<br />
''' 2016 '''<br />
* [[seqlm: an MDL based method for identifying differentially methylated regions in high density methylation array data]]<br />
* [[Statistical methods for detecting differentially methylated regions based on MethylCap-seq data]]<br />
''' 2017 '''<br />
* [[DMRfinder: efficiently identifying differentially methylated regions from MethylC-seq data]]<br />
''' 2018 '''<br />
* [[Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus]]<br />
* [[DMRcaller: a versatile R/Bioconductor package for detection and visualization of differentially methylated regions in CpG and non-CpG contexts]]<br />
* [[MethCP: Differentially Methylated Region Detection with Change Point Models (bioRxiv)]]<br />
<br />
==== Identifying sets of features (e.g. gene set analyses) ====<br />
'''2009<br />
<br />
* [[A general modular framework for gene set enrichment analysis]]<br />
* [[Comparing gene set analysis methods on single-nucleotide polymorphism data from Genetic Analysis Workshop 16]]<br />
<br />
'''2018<br />
<br />
* [[Gene set analysis methods: a systematic comparison]]<br />
<br />
==== Dimension reduction ====<br />
''' 2008 '''</br><br />
* [[On the Relationship Between Feature Selection and Classification Accuracy]]<br />
''' 2015 '''</br><br />
* [[Comparing feature selection methods for highdimensional imbalanced data: identifying rheumatoid arthritis cohorts from routine data]]<br />
<br />
=== Imputation methods for missing values ===<br />
<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 1996 || Schenker || [[Partially parametric techniques for multiple imputation]]<br />
|-<br />
| 1999 || Hastie T || [[Imputing Missing Data for Gene Expression Arrays]]<br />
|-<br />
| 2001 || Troyanskaya || [[Missing value estimation methods for DNA microarrays]]<br />
|-<br />
| 2002 || Engels J || [[Imputation of missing longitudinal data: a comparison of methods]]<br />
|-<br />
| 2003 || Oba || [[A Bayesian missing value estimation method for gene expression profile data]]<br />
|-<br />
| 2005 || Scholz || [[Nonlinear PCA: a missing data approach]]<br />
|-<br />
| 2007 || Stacklies || [[pcaMethods—a bioconductor package providing PCA methods for incomplete data]]<br />
|-<br />
| 2007 || Verboven || [[Sequential imputation for missing values]]<br />
|-<br />
| 2008 || Shaffer GN || [[Which missing value imputation method to use in expression profiles: a comparative study and two selection schemes]]<br />
|-<br />
| 2011 || Templ || [[Iterative stepwise regression imputation using standard and robust methods]]<br />
|-<br />
| 2012 || Hrydziuszko O || [[Missing values in mass spectrometry based metabolomics: an undervalued step in the data processing pipeline]]<br />
|-<br />
| 2012 || Stekhoven || [[MissForest—non-parametric missing value imputation for mixed-type data]]<br />
|-<br />
| 2013 || Taylor || [[Accounting for undetected compounds in statistical analyses of mass spectrometry ‘omic studies]]<br />
|-<br />
| 2013 || Waljee || [[Comparison of imputation methods for missing laboratory data in medicine]]<br />
|-<br />
| 2014 || Shah || [[Comparison of Random Forest and Parametric Imputation Models for Imputing Missing Data Using MICE: A CALIBER Study]]<br />
|-<br />
| 2014 || Rodwell || [[Comparison of methods for imputing limited-range variables: a simulation study]]<br />
|-<br />
| 2014 || Morris || [[Tuning multiple imputation by predictive mean matching and local residual draws]]<br />
|-<br />
| 2014 || Doove L || [[Recursive partitioning for missing data imputation in the presence of interaction effects]]<br />
|-<br />
| 2015 || Webb-Robertson BJM || [[Review, evaluation, and discussion of the challenges of missing value imputation for mass spectrometry-based label-free global proteomics]]<br />
|-<br />
| 2016 || Folch-Fortuny A || [[Assessment of maximum likelihood PCA missing data imputation]]<br />
|-<br />
| 2016 || Lazar C || [[Accounting for the Multiple Natures of Missing Values in Label-Free Quantitative Proteomics Data Sets to Compare Imputation Strategies]]<br />
|-<br />
| 2016 || Yin X || [[Multiple imputation and analysis for high-dimensional incomplete proteomics data]]<br />
|-<br />
| 2018 || Wei R || [[Missing Value Imputation Approach for Mass Spectrometry-based Metabolomics Data]]<br />
|-<br />
| 2018 || Poyatos R || [[Gap-filling a spatially explicit plant trait database: comparing imputation methods and different levels of environmental information]]<br />
|-<br />
| 2018 || O'Brien JJ || [[The effects of nonignorable missing data on label-free mass spectrometry proteomics experiments]]<br />
|}<br />
<br />
=== ODE-based Modelling ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2001 || Schenker || [[Ways to Fit a PK Model with Some Data Below the Quantification Limit]]<br />
|-<br />
| 2008 || Balsa-Canto || [[Hybrid optimization method with general switching strategy for parameter estimation]]<br />
|-<br />
| 2011 || Tashkova || [[Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis]]<br />
|-<br />
| 2013 || Raue || [[Lessons Learned from Quantitative Dynamical Modeling in Systems Biology]]<br />
|-<br />
| 2013 || Schenker || [[ODE parameter inference using adaptive gradient matching with Gaussian processes]]<br />
|-<br />
| 2017 || Schenker || [[Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems]]<br />
|-<br />
| 2017 || Schenker || [[Data-driven reverse engineering of signaling pathways using ensembles of dynamic models]]<br />
|-<br />
| 2017 || Schenker || [[Fast derivatives of likelihood functionals for ODE based models using adjoint-state method]]<br />
|-<br />
| 2017 || Schenker || [[Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy]]<br />
|-<br />
| 2017 || Schenker || [[Performance of objective functions and optimization procedures for parameter estimation in system biology models]]<br />
|-<br />
| 2017 || Schenker || [[Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks]]<br />
|-<br />
| 2018 || Schenker || [[Evaluation of Derivative-Free Optimizers for Parameter Estimation in Systems Biology]]<br />
|-<br />
| 2018 || Schenker || [[Hierarchical optimization for the efficient parametrization of ODE models]]<br />
|-<br />
| 2018 || Schenker || [[Input-dependent structural identifiability of nonlinear systems]]<br />
|-<br />
| 2018 || Schenker || [[Optimization and profile calculation of ODE models using second order adjoint sensitivity analysis]]<br />
|-<br />
| 2019 || Schenker || [[A comparison of methods for quantifying prediction uncertainty in systems biology]]<br />
|-<br />
| 2019 || Schenker || [[Benchmark problems for dynamic modeling of intracellular processes]]<br />
|-<br />
| 2019 || Schenker || [[Benchmarking optimization methods for parameter estimation in large kinetic models]]<br />
|-<br />
| 2019 || Schenker || [[Efficient computation of steady states in large-scale ODE models of biochemical reaction networks]]<br />
|-<br />
| 2019 || Schenker || [[Full observability and estimation of unknown inputs, states and parameters of nonlinear biological models]]<br />
|-<br />
| 2019 || Schenker || [[Mini-batch optimization enables training of ODE models on large-scale datasets]]<br />
|-<br />
| 2019 || Schenker || [[Parameter Estimation and Variable Selection for Big Systems of Linear Ordinary Differential Equations: A Matrix-Based Approach]]<br />
|-<br />
| 2019 || Schenker || [[Parameter estimation in models of biological oscillators: an automated regularised estimation approach]]<br />
|-<br />
| 2019 || Schenker || [[Robust calibration of hierarchical population models for heterogeneous cell populations]]<br />
|-<br />
| 2019 || Schenker || [[Scalable nonlinear programming framework for parameter estimation in dynamic biological system models]]<br />
|-<br />
| 2019 || Schenker || [[Tracking for parameter and state estimation in possibly misspecified partially observed linear Ordinary Differential Equations]]<br />
|-<br />
| 2020 || Schenker || [[An application of Conditional RobustCalibration (CRC) to ordinary differential equations (ODEs) models in computational systems biology: a comparison of two sampling strategies]]<br />
|-<br />
| 2020 || Schenker || [[Efficient parameterization of large-scale dynamic models based on relative measurements]]<br />
|-<br />
| 2020 || Schenker || [[Testing structural identifiability by a simple scaling method]]<br />
|}<br />
<br />
=== Omics Workflows ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2008 || Neuweger H || [[MeltDB: a software platform for the analysis and integration of metabolomics experiment data]]<br />
|-<br />
| 2008 || Barla A || [[Machine learning methods for predictive proteomics]]<br />
|-<br />
| 2009 || Xia J || [[MetaboAnalyst: a web server for metabolomic data analysis and interpretation]]<br />
|-<br />
| 2013 || Weisser H || [[An Automated Pipeline for High-Throughput Label-Free Quantitative Proteomics]]<br />
|-<br />
| 2014 || Cox J || [[Accurate Proteome-wide Label-free Quantification by Delayed Normalization and Maximal Peptide Ratio Extraction, Termed MaxLFQ* ]]<br />
|-<br />
| 2015 || || [[ComparingVariant Call Files for Performance Benchmarkingof Next-Generation Sequencing Variant Calling Pipelines]]<br />
|-<br />
| 2016 || Tyanova S || [[The MaxQuant computational platform for mass spectrometry–based shotgun proteomics]]<br />
|-<br />
| 2016 || Röst HL || [[OpenMS: a flexible open-source software platform for mass spectrometry data analysis]]<br />
|-<br />
| 2017 || || [[A benchmarking of workflows for detecting differential splicing and differential expression at isoform level in human RNA-seq studies]]<br />
|-<br />
| 2018 || Välikangas T || [[A comprehensive evaluation of popular proteomics software workflows for label-free proteome quantification and imputation]]<br />
|-<br />
| 2019 || || [[A Systematic Evaluation of Single CellRNA-Seq Analysis Pipelines]]<br />
|-<br />
| 2019 || || [[Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays]]<br />
|}<br />
<br />
=== Preprocessing high-throughput data===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|- 1999 || Perkins DN || [[Probability-based protein identification by searching sequence databases using mass spectrometry data]]<br />
|-<br />
| 2003 || || [[A comparison of normalization methods for high density oligonucleotide array data based on variance and bias ]]<br />
|-<br />
| 2005 || || [[Comparison of Affymetrix GeneChip Expression Measures]]<br />
|-<br />
| 2005 || Meleth S || [[The case for well-conducted experiments to validate statistical protocols for 2D gels: different pre-processing = different lists of significant proteins]]<br />
|-<br />
| 2005 || || [[Comparison of background correction and normalization procedures for high-density oligonucleotide microarrays]]<br />
|-<br />
| 2006 || || [[Using RNA sample titrations to assess microarray platform performance and normalization techniques]]<br />
|-<br />
| 2006 || Wang P || [[Normalization regarding non-random missing values in high-throughput mass spectrometry data]]<br />
|-<br />
| 2007 || Carvalho B || [[Exploration, normalization, and genotype calls of high-density oligonucleotide SNP array data]]<br />
|-<br />
| 2007 || Cannataro M || [[MS‐Analyzer: preprocessing and data mining services for proteomics applications on the Grid]]<br />
|-<br />
| 2008 || || [[Comparison of preprocessing methods for the hgU133+2 chip from Affymetrix]]<br />
|-<br />
| 2009 || || [[Comparison of Affymetrix data normalization methods using 6,926 experiments across five array generations]]<br />
|-<br />
| 2009 || Mar JC || [[Data-driven normalization strategies for high-throughput quantitative RT-PCR]]<br />
|-<br />
| 2009 || Vakhrushev SY || [[Software platform for high-throughput glycomics]]<br />
|-<br />
| 2010 || || [[Consistency of predictive signature genes and classifiers generated using different microarray platforms]]<br />
|-<br />
| 2010 || || [[Detecting and correcting systematic variation in large-scale RNA sequencing data]]<br />
|-<br />
| 2010 || || [[Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments]]<br />
|-<br />
| 2010 || || [[Normalization of RNA-seq data using factor analysis of control genes or samples]]<br />
|-<br />
| 2011 || || [[Affymetrix GeneChip microarray preprocessing for multivariate analyses]]<br />
|-<br />
| 2012 || || [[A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis]]<br />
|-<br />
| 2014 || || [[Normalyzer: A Tool for Rapid Evaluation of Normalization Methods for Omics Data Sets]]<br />
|-<br />
| 2014 || Zhou X || [[Prevention, diagnosis and treatment of high-throughput sequencing data pathologies]]<br />
|-<br />
| 2014 || Coble JB || [[Comparative evaluation of preprocessing freeware on chromatography/mass spectrometry data for signature discovery]]<br />
|-<br />
| 2015 || Caraus I || [[Detecting and overcoming systematic bias in high-throughput screening technologies: a comprehensive review of practical issues and methodological solutions]]<br />
|-<br />
| 2015 || Tam S || [[Optimization of miRNA-seq data preprocessing]]<br />
|-<br />
| 2015 || Rafiei A || [[Comparison of peak‐picking workflows for untargeted liquid chromatography/high‐resolution mass spectrometry metabolomics data analysis]]<br />
|-<br />
| 2015 || Chawade A || [[Data processing has major impact on the outcome of quantitative label-free LC-MS analysis]]<br />
|-<br />
| 2016 || Yi L || [[Chemometric methods in data processing of mass spectrometry-based metabolomics: A review]]<br />
|-<br />
| 2016 || Tsuji J || [[Evaluation of preprocessing, mapping and postprocessing algorithms for analyzing whole genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Li B || [[Performance Evaluation and Online Realization of Data-driven Normalization Methods Used in LC/MS based Untargeted Metabolomics Analysis]]<br />
|-<br />
| 2017 || Li B || [[NOREVA: normalization and evaluation of MS-based metabolomics data]]<br />
|-<br />
| 2018 || Mazoure B || [[Identification and Correction of Additive and Multiplicative Spatial Biases in Experimental High-Throughput Screening]]<br />
|-<br />
| 2018 || Li Z || [[Comprehensive evaluation of untargeted metabolomics data processing software in feature detection, quantification and discriminating marker selection]]<br />
|-<br />
| 2018 || Willforss J || [[NormalyzerDE: Online Tool for Improved Normalization of Omics Expression Data and High-Sensitivity Differential Expression Analysis]]<br />
|}</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Literature_Studies&diff=687Literature Studies2020-02-25T15:29:21Z<p>Bwday: /* ODE-based Modelling */</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
{| class="wikitable"<br />
|-<br />
! Page summary<br />
|-<br />
| Here outcomes of benchmarking studies from the literature are collected. The primary aim is a comprehensive overview about neutral benchmark studies, i.e. assessments which were performed independenty on publication of a new approach. Studies which are not neutral are put in brackets. </br> <br />
<br />
The focus is on computational methods for analyzing experimental data (instead of comparing experimental techniques or platforms). </br><br />
<br />
Please extend this list by creating a new page and adding a link below. </br> <br />
Use the '''[[Guidelines_for_Summarizing_a_Literature_Study|guidelines described here]]'''.<br />
|}<br />
<br />
== Results from Literature ==<br />
<br />
=== Classification ===<br />
''' 2003 '''</br><br />
* [[Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data]]<br />
''' 2005 '''</br><br />
* [[A review and comparison of classification algorithms for medical decision making]]<br />
''' 2016 '''</br><br />
* [[Predicting Breast Cancer Survivability Using Data Mining Techniques]]<br />
<br />
=== Selection of Differential Features and Regions ===<br />
==== Identifying differential features ====<br />
''' 2006 '''</br><br />
* [[Rat toxicogenomic study reveals analytical consistency across microarray platforms]]<br />
''' 2010 '''</br><br />
* [[A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the sequencing Quality control consortium]]<br />
''' 2017 '''</br><br />
* [[Identification of differentially expressed peptides in high-throughput proteomics data]]<br />
* [[In-depth method assessments of differentially expressed protein detection for shotgun proteomics data with missing values]]<br />
* [[Strategies for analyzing bisulfite sequencing data]]<br />
''' 2018 '''</br><br />
* [[Identification of Differentially Methylated Sites with Weak Methylation Effects]]<br />
<br />
==== Identifying differential regions (e.g. DMRs) ====<br />
''' 2015 '''<br />
* [[De novo identification of differentially methylated regions in the human genome]]<br />
* [[MethylAction: detecting differentially methylated regions that distinguish biological subtypes]]<br />
* [[metilene: Fast and sensitive calling of differentially methylated regions from bisulfite sequencing data]]<br />
''' 2016 '''<br />
* [[seqlm: an MDL based method for identifying differentially methylated regions in high density methylation array data]]<br />
* [[Statistical methods for detecting differentially methylated regions based on MethylCap-seq data]]<br />
''' 2017 '''<br />
* [[DMRfinder: efficiently identifying differentially methylated regions from MethylC-seq data]]<br />
''' 2018 '''<br />
* [[Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus]]<br />
* [[DMRcaller: a versatile R/Bioconductor package for detection and visualization of differentially methylated regions in CpG and non-CpG contexts]]<br />
* [[MethCP: Differentially Methylated Region Detection with Change Point Models (bioRxiv)]]<br />
<br />
==== Identifying sets of features (e.g. gene set analyses) ====<br />
'''2009<br />
<br />
* [[A general modular framework for gene set enrichment analysis]]<br />
* [[Comparing gene set analysis methods on single-nucleotide polymorphism data from Genetic Analysis Workshop 16]]<br />
<br />
'''2018<br />
<br />
* [[Gene set analysis methods: a systematic comparison]]<br />
<br />
==== Dimension reduction ====<br />
''' 2008 '''</br><br />
* [[On the Relationship Between Feature Selection and Classification Accuracy]]<br />
''' 2015 '''</br><br />
* [[Comparing feature selection methods for highdimensional imbalanced data: identifying rheumatoid arthritis cohorts from routine data]]<br />
<br />
=== Imputation methods for missing values ===<br />
<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 1996 || Schenker || [[Partially parametric techniques for multiple imputation]]<br />
|-<br />
| 1999 || Hastie T || [[Imputing Missing Data for Gene Expression Arrays]]<br />
|-<br />
| 2001 || Troyanskaya || [[Missing value estimation methods for DNA microarrays]]<br />
|-<br />
| 2002 || Engels J || [[Imputation of missing longitudinal data: a comparison of methods]]<br />
|-<br />
| 2003 || Oba || [[A Bayesian missing value estimation method for gene expression profile data]]<br />
|-<br />
| 2005 || Scholz || [[Nonlinear PCA: a missing data approach]]<br />
|-<br />
| 2007 || Stacklies || [[pcaMethods—a bioconductor package providing PCA methods for incomplete data]]<br />
|-<br />
| 2007 || Verboven || [[Sequential imputation for missing values]]<br />
|-<br />
| 2008 || Shaffer GN || [[Which missing value imputation method to use in expression profiles: a comparative study and two selection schemes]]<br />
|-<br />
| 2011 || Templ || [[Iterative stepwise regression imputation using standard and robust methods]]<br />
|-<br />
| 2012 || Hrydziuszko O || [[Missing values in mass spectrometry based metabolomics: an undervalued step in the data processing pipeline]]<br />
|-<br />
| 2012 || Stekhoven || [[MissForest—non-parametric missing value imputation for mixed-type data]]<br />
|-<br />
| 2013 || Taylor || [[Accounting for undetected compounds in statistical analyses of mass spectrometry ‘omic studies]]<br />
|-<br />
| 2013 || Waljee || [[Comparison of imputation methods for missing laboratory data in medicine]]<br />
|-<br />
| 2014 || Shah || [[Comparison of Random Forest and Parametric Imputation Models for Imputing Missing Data Using MICE: A CALIBER Study]]<br />
|-<br />
| 2014 || Rodwell || [[Comparison of methods for imputing limited-range variables: a simulation study]]<br />
|-<br />
| 2014 || Morris || [[Tuning multiple imputation by predictive mean matching and local residual draws]]<br />
|-<br />
| 2014 || Doove L || [[Recursive partitioning for missing data imputation in the presence of interaction effects]]<br />
|-<br />
| 2015 || Webb-Robertson BJM || [[Review, evaluation, and discussion of the challenges of missing value imputation for mass spectrometry-based label-free global proteomics]]<br />
|-<br />
| 2016 || Folch-Fortuny A || [[Assessment of maximum likelihood PCA missing data imputation]]<br />
|-<br />
| 2016 || Lazar C || [[Accounting for the Multiple Natures of Missing Values in Label-Free Quantitative Proteomics Data Sets to Compare Imputation Strategies]]<br />
|-<br />
| 2016 || Yin X || [[Multiple imputation and analysis for high-dimensional incomplete proteomics data]]<br />
|-<br />
| 2018 || Wei R || [[Missing Value Imputation Approach for Mass Spectrometry-based Metabolomics Data]]<br />
|-<br />
| 2018 || Poyatos R || [[Gap-filling a spatially explicit plant trait database: comparing imputation methods and different levels of environmental information]]<br />
|-<br />
| 2018 || O'Brien JJ || [[The effects of nonignorable missing data on label-free mass spectrometry proteomics experiments]]<br />
|}<br />
<br />
=== ODE-based Modelling ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2001 || Schenker || [[Ways to Fit a PK Model with Some Data Below the Quantification Limit]]<br />
|-<br />
| 2008 || Balsa-Canto || [[Hybrid optimization method with general switching strategy for parameter estimation]]<br />
|-<br />
| 2011 || Tashkova || [[Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis]]<br />
|-<br />
| 2013 || Schenker || [[Lessons Learned from Quantitative Dynamical Modeling in Systems Biology]]<br />
|-<br />
| 2013 || Schenker || [[ODE parameter inference using adaptive gradient matching with Gaussian processes]]<br />
|-<br />
| 2017 || Schenker || [[Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems]]<br />
|-<br />
| 2017 || Schenker || [[Data-driven reverse engineering of signaling pathways using ensembles of dynamic models]]<br />
|-<br />
| 2017 || Schenker || [[Fast derivatives of likelihood functionals for ODE based models using adjoint-state method]]<br />
|-<br />
| 2017 || Schenker || [[Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy]]<br />
|-<br />
| 2017 || Schenker || [[Performance of objective functions and optimization procedures for parameter estimation in system biology models]]<br />
|-<br />
| 2017 || Schenker || [[Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks]]<br />
|-<br />
| 2018 || Schenker || [[Evaluation of Derivative-Free Optimizers for Parameter Estimation in Systems Biology]]<br />
|-<br />
| 2018 || Schenker || [[Hierarchical optimization for the efficient parametrization of ODE models]]<br />
|-<br />
| 2018 || Schenker || [[Input-dependent structural identifiability of nonlinear systems]]<br />
|-<br />
| 2018 || Schenker || [[Optimization and profile calculation of ODE models using second order adjoint sensitivity analysis]]<br />
|-<br />
| 2019 || Schenker || [[A comparison of methods for quantifying prediction uncertainty in systems biology]]<br />
|-<br />
| 2019 || Schenker || [[Benchmark problems for dynamic modeling of intracellular processes]]<br />
|-<br />
| 2019 || Schenker || [[Benchmarking optimization methods for parameter estimation in large kinetic models]]<br />
|-<br />
| 2019 || Schenker || [[Efficient computation of steady states in large-scale ODE models of biochemical reaction networks]]<br />
|-<br />
| 2019 || Schenker || [[Full observability and estimation of unknown inputs, states and parameters of nonlinear biological models]]<br />
|-<br />
| 2019 || Schenker || [[Mini-batch optimization enables training of ODE models on large-scale datasets]]<br />
|-<br />
| 2019 || Schenker || [[Parameter Estimation and Variable Selection for Big Systems of Linear Ordinary Differential Equations: A Matrix-Based Approach]]<br />
|-<br />
| 2019 || Schenker || [[Parameter estimation in models of biological oscillators: an automated regularised estimation approach]]<br />
|-<br />
| 2019 || Schenker || [[Robust calibration of hierarchical population models for heterogeneous cell populations]]<br />
|-<br />
| 2019 || Schenker || [[Scalable nonlinear programming framework for parameter estimation in dynamic biological system models]]<br />
|-<br />
| 2019 || Schenker || [[Tracking for parameter and state estimation in possibly misspecified partially observed linear Ordinary Differential Equations]]<br />
|-<br />
| 2020 || Schenker || [[An application of Conditional RobustCalibration (CRC) to ordinary differential equations (ODEs) models in computational systems biology: a comparison of two sampling strategies]]<br />
|-<br />
| 2020 || Schenker || [[Efficient parameterization of large-scale dynamic models based on relative measurements]]<br />
|-<br />
| 2020 || Schenker || [[Testing structural identifiability by a simple scaling method]]<br />
|}<br />
<br />
=== Omics Workflows ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2008 || Neuweger H || [[MeltDB: a software platform for the analysis and integration of metabolomics experiment data]]<br />
|-<br />
| 2008 || Barla A || [[Machine learning methods for predictive proteomics]]<br />
|-<br />
| 2009 || Xia J || [[MetaboAnalyst: a web server for metabolomic data analysis and interpretation]]<br />
|-<br />
| 2013 || Weisser H || [[An Automated Pipeline for High-Throughput Label-Free Quantitative Proteomics]]<br />
|-<br />
| 2014 || Cox J || [[Accurate Proteome-wide Label-free Quantification by Delayed Normalization and Maximal Peptide Ratio Extraction, Termed MaxLFQ* ]]<br />
|-<br />
| 2015 || || [[ComparingVariant Call Files for Performance Benchmarkingof Next-Generation Sequencing Variant Calling Pipelines]]<br />
|-<br />
| 2016 || Tyanova S || [[The MaxQuant computational platform for mass spectrometry–based shotgun proteomics]]<br />
|-<br />
| 2016 || Röst HL || [[OpenMS: a flexible open-source software platform for mass spectrometry data analysis]]<br />
|-<br />
| 2017 || || [[A benchmarking of workflows for detecting differential splicing and differential expression at isoform level in human RNA-seq studies]]<br />
|-<br />
| 2018 || Välikangas T || [[A comprehensive evaluation of popular proteomics software workflows for label-free proteome quantification and imputation]]<br />
|-<br />
| 2019 || || [[A Systematic Evaluation of Single CellRNA-Seq Analysis Pipelines]]<br />
|-<br />
| 2019 || || [[Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays]]<br />
|}<br />
<br />
=== Preprocessing high-throughput data===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|- 1999 || Perkins DN || [[Probability-based protein identification by searching sequence databases using mass spectrometry data]]<br />
|-<br />
| 2003 || || [[A comparison of normalization methods for high density oligonucleotide array data based on variance and bias ]]<br />
|-<br />
| 2005 || || [[Comparison of Affymetrix GeneChip Expression Measures]]<br />
|-<br />
| 2005 || Meleth S || [[The case for well-conducted experiments to validate statistical protocols for 2D gels: different pre-processing = different lists of significant proteins]]<br />
|-<br />
| 2005 || || [[Comparison of background correction and normalization procedures for high-density oligonucleotide microarrays]]<br />
|-<br />
| 2006 || || [[Using RNA sample titrations to assess microarray platform performance and normalization techniques]]<br />
|-<br />
| 2006 || Wang P || [[Normalization regarding non-random missing values in high-throughput mass spectrometry data]]<br />
|-<br />
| 2007 || Carvalho B || [[Exploration, normalization, and genotype calls of high-density oligonucleotide SNP array data]]<br />
|-<br />
| 2007 || Cannataro M || [[MS‐Analyzer: preprocessing and data mining services for proteomics applications on the Grid]]<br />
|-<br />
| 2008 || || [[Comparison of preprocessing methods for the hgU133+2 chip from Affymetrix]]<br />
|-<br />
| 2009 || || [[Comparison of Affymetrix data normalization methods using 6,926 experiments across five array generations]]<br />
|-<br />
| 2009 || Mar JC || [[Data-driven normalization strategies for high-throughput quantitative RT-PCR]]<br />
|-<br />
| 2009 || Vakhrushev SY || [[Software platform for high-throughput glycomics]]<br />
|-<br />
| 2010 || || [[Consistency of predictive signature genes and classifiers generated using different microarray platforms]]<br />
|-<br />
| 2010 || || [[Detecting and correcting systematic variation in large-scale RNA sequencing data]]<br />
|-<br />
| 2010 || || [[Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments]]<br />
|-<br />
| 2010 || || [[Normalization of RNA-seq data using factor analysis of control genes or samples]]<br />
|-<br />
| 2011 || || [[Affymetrix GeneChip microarray preprocessing for multivariate analyses]]<br />
|-<br />
| 2012 || || [[A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis]]<br />
|-<br />
| 2014 || || [[Normalyzer: A Tool for Rapid Evaluation of Normalization Methods for Omics Data Sets]]<br />
|-<br />
| 2014 || Zhou X || [[Prevention, diagnosis and treatment of high-throughput sequencing data pathologies]]<br />
|-<br />
| 2014 || Coble JB || [[Comparative evaluation of preprocessing freeware on chromatography/mass spectrometry data for signature discovery]]<br />
|-<br />
| 2015 || Caraus I || [[Detecting and overcoming systematic bias in high-throughput screening technologies: a comprehensive review of practical issues and methodological solutions]]<br />
|-<br />
| 2015 || Tam S || [[Optimization of miRNA-seq data preprocessing]]<br />
|-<br />
| 2015 || Rafiei A || [[Comparison of peak‐picking workflows for untargeted liquid chromatography/high‐resolution mass spectrometry metabolomics data analysis]]<br />
|-<br />
| 2015 || Chawade A || [[Data processing has major impact on the outcome of quantitative label-free LC-MS analysis]]<br />
|-<br />
| 2016 || Yi L || [[Chemometric methods in data processing of mass spectrometry-based metabolomics: A review]]<br />
|-<br />
| 2016 || Tsuji J || [[Evaluation of preprocessing, mapping and postprocessing algorithms for analyzing whole genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Li B || [[Performance Evaluation and Online Realization of Data-driven Normalization Methods Used in LC/MS based Untargeted Metabolomics Analysis]]<br />
|-<br />
| 2017 || Li B || [[NOREVA: normalization and evaluation of MS-based metabolomics data]]<br />
|-<br />
| 2018 || Mazoure B || [[Identification and Correction of Additive and Multiplicative Spatial Biases in Experimental High-Throughput Screening]]<br />
|-<br />
| 2018 || Li Z || [[Comprehensive evaluation of untargeted metabolomics data processing software in feature detection, quantification and discriminating marker selection]]<br />
|-<br />
| 2018 || Willforss J || [[NormalyzerDE: Online Tool for Improved Normalization of Omics Expression Data and High-Sensitivity Differential Expression Analysis]]<br />
|}</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Literature_Studies&diff=686Literature Studies2020-02-25T15:29:03Z<p>Bwday: /* ODE-based Modelling */</p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
{| class="wikitable"<br />
|-<br />
! Page summary<br />
|-<br />
| Here outcomes of benchmarking studies from the literature are collected. The primary aim is a comprehensive overview about neutral benchmark studies, i.e. assessments which were performed independenty on publication of a new approach. Studies which are not neutral are put in brackets. </br> <br />
<br />
The focus is on computational methods for analyzing experimental data (instead of comparing experimental techniques or platforms). </br><br />
<br />
Please extend this list by creating a new page and adding a link below. </br> <br />
Use the '''[[Guidelines_for_Summarizing_a_Literature_Study|guidelines described here]]'''.<br />
|}<br />
<br />
== Results from Literature ==<br />
<br />
=== Classification ===<br />
''' 2003 '''</br><br />
* [[Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data]]<br />
''' 2005 '''</br><br />
* [[A review and comparison of classification algorithms for medical decision making]]<br />
''' 2016 '''</br><br />
* [[Predicting Breast Cancer Survivability Using Data Mining Techniques]]<br />
<br />
=== Selection of Differential Features and Regions ===<br />
==== Identifying differential features ====<br />
''' 2006 '''</br><br />
* [[Rat toxicogenomic study reveals analytical consistency across microarray platforms]]<br />
''' 2010 '''</br><br />
* [[A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the sequencing Quality control consortium]]<br />
''' 2017 '''</br><br />
* [[Identification of differentially expressed peptides in high-throughput proteomics data]]<br />
* [[In-depth method assessments of differentially expressed protein detection for shotgun proteomics data with missing values]]<br />
* [[Strategies for analyzing bisulfite sequencing data]]<br />
''' 2018 '''</br><br />
* [[Identification of Differentially Methylated Sites with Weak Methylation Effects]]<br />
<br />
==== Identifying differential regions (e.g. DMRs) ====<br />
''' 2015 '''<br />
* [[De novo identification of differentially methylated regions in the human genome]]<br />
* [[MethylAction: detecting differentially methylated regions that distinguish biological subtypes]]<br />
* [[metilene: Fast and sensitive calling of differentially methylated regions from bisulfite sequencing data]]<br />
''' 2016 '''<br />
* [[seqlm: an MDL based method for identifying differentially methylated regions in high density methylation array data]]<br />
* [[Statistical methods for detecting differentially methylated regions based on MethylCap-seq data]]<br />
''' 2017 '''<br />
* [[DMRfinder: efficiently identifying differentially methylated regions from MethylC-seq data]]<br />
''' 2018 '''<br />
* [[Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus]]<br />
* [[DMRcaller: a versatile R/Bioconductor package for detection and visualization of differentially methylated regions in CpG and non-CpG contexts]]<br />
* [[MethCP: Differentially Methylated Region Detection with Change Point Models (bioRxiv)]]<br />
<br />
==== Identifying sets of features (e.g. gene set analyses) ====<br />
'''2009<br />
<br />
* [[A general modular framework for gene set enrichment analysis]]<br />
* [[Comparing gene set analysis methods on single-nucleotide polymorphism data from Genetic Analysis Workshop 16]]<br />
<br />
'''2018<br />
<br />
* [[Gene set analysis methods: a systematic comparison]]<br />
<br />
==== Dimension reduction ====<br />
''' 2008 '''</br><br />
* [[On the Relationship Between Feature Selection and Classification Accuracy]]<br />
''' 2015 '''</br><br />
* [[Comparing feature selection methods for highdimensional imbalanced data: identifying rheumatoid arthritis cohorts from routine data]]<br />
<br />
=== Imputation methods for missing values ===<br />
<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 1996 || Schenker || [[Partially parametric techniques for multiple imputation]]<br />
|-<br />
| 1999 || Hastie T || [[Imputing Missing Data for Gene Expression Arrays]]<br />
|-<br />
| 2001 || Troyanskaya || [[Missing value estimation methods for DNA microarrays]]<br />
|-<br />
| 2002 || Engels J || [[Imputation of missing longitudinal data: a comparison of methods]]<br />
|-<br />
| 2003 || Oba || [[A Bayesian missing value estimation method for gene expression profile data]]<br />
|-<br />
| 2005 || Scholz || [[Nonlinear PCA: a missing data approach]]<br />
|-<br />
| 2007 || Stacklies || [[pcaMethods—a bioconductor package providing PCA methods for incomplete data]]<br />
|-<br />
| 2007 || Verboven || [[Sequential imputation for missing values]]<br />
|-<br />
| 2008 || Shaffer GN || [[Which missing value imputation method to use in expression profiles: a comparative study and two selection schemes]]<br />
|-<br />
| 2011 || Templ || [[Iterative stepwise regression imputation using standard and robust methods]]<br />
|-<br />
| 2012 || Hrydziuszko O || [[Missing values in mass spectrometry based metabolomics: an undervalued step in the data processing pipeline]]<br />
|-<br />
| 2012 || Stekhoven || [[MissForest—non-parametric missing value imputation for mixed-type data]]<br />
|-<br />
| 2013 || Taylor || [[Accounting for undetected compounds in statistical analyses of mass spectrometry ‘omic studies]]<br />
|-<br />
| 2013 || Waljee || [[Comparison of imputation methods for missing laboratory data in medicine]]<br />
|-<br />
| 2014 || Shah || [[Comparison of Random Forest and Parametric Imputation Models for Imputing Missing Data Using MICE: A CALIBER Study]]<br />
|-<br />
| 2014 || Rodwell || [[Comparison of methods for imputing limited-range variables: a simulation study]]<br />
|-<br />
| 2014 || Morris || [[Tuning multiple imputation by predictive mean matching and local residual draws]]<br />
|-<br />
| 2014 || Doove L || [[Recursive partitioning for missing data imputation in the presence of interaction effects]]<br />
|-<br />
| 2015 || Webb-Robertson BJM || [[Review, evaluation, and discussion of the challenges of missing value imputation for mass spectrometry-based label-free global proteomics]]<br />
|-<br />
| 2016 || Folch-Fortuny A || [[Assessment of maximum likelihood PCA missing data imputation]]<br />
|-<br />
| 2016 || Lazar C || [[Accounting for the Multiple Natures of Missing Values in Label-Free Quantitative Proteomics Data Sets to Compare Imputation Strategies]]<br />
|-<br />
| 2016 || Yin X || [[Multiple imputation and analysis for high-dimensional incomplete proteomics data]]<br />
|-<br />
| 2018 || Wei R || [[Missing Value Imputation Approach for Mass Spectrometry-based Metabolomics Data]]<br />
|-<br />
| 2018 || Poyatos R || [[Gap-filling a spatially explicit plant trait database: comparing imputation methods and different levels of environmental information]]<br />
|-<br />
| 2018 || O'Brien JJ || [[The effects of nonignorable missing data on label-free mass spectrometry proteomics experiments]]<br />
|}<br />
<br />
=== ODE-based Modelling ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2001 || Schenker || [[Ways to Fit a PK Model with Some Data Below the Quantification Limit]]<br />
|-<br />
| 2008 || Balsa-Canto || [[Hybrid optimization method with general switching strategy for parameter estimation]]<br />
|-<br />
| 2011 || Schenker || [[Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis]]<br />
|-<br />
| 2013 || Schenker || [[Lessons Learned from Quantitative Dynamical Modeling in Systems Biology]]<br />
|-<br />
| 2013 || Schenker || [[ODE parameter inference using adaptive gradient matching with Gaussian processes]]<br />
|-<br />
| 2017 || Schenker || [[Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems]]<br />
|-<br />
| 2017 || Schenker || [[Data-driven reverse engineering of signaling pathways using ensembles of dynamic models]]<br />
|-<br />
| 2017 || Schenker || [[Fast derivatives of likelihood functionals for ODE based models using adjoint-state method]]<br />
|-<br />
| 2017 || Schenker || [[Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy]]<br />
|-<br />
| 2017 || Schenker || [[Performance of objective functions and optimization procedures for parameter estimation in system biology models]]<br />
|-<br />
| 2017 || Schenker || [[Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks]]<br />
|-<br />
| 2018 || Schenker || [[Evaluation of Derivative-Free Optimizers for Parameter Estimation in Systems Biology]]<br />
|-<br />
| 2018 || Schenker || [[Hierarchical optimization for the efficient parametrization of ODE models]]<br />
|-<br />
| 2018 || Schenker || [[Input-dependent structural identifiability of nonlinear systems]]<br />
|-<br />
| 2018 || Schenker || [[Optimization and profile calculation of ODE models using second order adjoint sensitivity analysis]]<br />
|-<br />
| 2019 || Schenker || [[A comparison of methods for quantifying prediction uncertainty in systems biology]]<br />
|-<br />
| 2019 || Schenker || [[Benchmark problems for dynamic modeling of intracellular processes]]<br />
|-<br />
| 2019 || Schenker || [[Benchmarking optimization methods for parameter estimation in large kinetic models]]<br />
|-<br />
| 2019 || Schenker || [[Efficient computation of steady states in large-scale ODE models of biochemical reaction networks]]<br />
|-<br />
| 2019 || Schenker || [[Full observability and estimation of unknown inputs, states and parameters of nonlinear biological models]]<br />
|-<br />
| 2019 || Schenker || [[Mini-batch optimization enables training of ODE models on large-scale datasets]]<br />
|-<br />
| 2019 || Schenker || [[Parameter Estimation and Variable Selection for Big Systems of Linear Ordinary Differential Equations: A Matrix-Based Approach]]<br />
|-<br />
| 2019 || Schenker || [[Parameter estimation in models of biological oscillators: an automated regularised estimation approach]]<br />
|-<br />
| 2019 || Schenker || [[Robust calibration of hierarchical population models for heterogeneous cell populations]]<br />
|-<br />
| 2019 || Schenker || [[Scalable nonlinear programming framework for parameter estimation in dynamic biological system models]]<br />
|-<br />
| 2019 || Schenker || [[Tracking for parameter and state estimation in possibly misspecified partially observed linear Ordinary Differential Equations]]<br />
|-<br />
| 2020 || Schenker || [[An application of Conditional RobustCalibration (CRC) to ordinary differential equations (ODEs) models in computational systems biology: a comparison of two sampling strategies]]<br />
|-<br />
| 2020 || Schenker || [[Efficient parameterization of large-scale dynamic models based on relative measurements]]<br />
|-<br />
| 2020 || Schenker || [[Testing structural identifiability by a simple scaling method]]<br />
|}<br />
<br />
=== Omics Workflows ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2008 || Neuweger H || [[MeltDB: a software platform for the analysis and integration of metabolomics experiment data]]<br />
|-<br />
| 2008 || Barla A || [[Machine learning methods for predictive proteomics]]<br />
|-<br />
| 2009 || Xia J || [[MetaboAnalyst: a web server for metabolomic data analysis and interpretation]]<br />
|-<br />
| 2013 || Weisser H || [[An Automated Pipeline for High-Throughput Label-Free Quantitative Proteomics]]<br />
|-<br />
| 2014 || Cox J || [[Accurate Proteome-wide Label-free Quantification by Delayed Normalization and Maximal Peptide Ratio Extraction, Termed MaxLFQ* ]]<br />
|-<br />
| 2015 || || [[ComparingVariant Call Files for Performance Benchmarkingof Next-Generation Sequencing Variant Calling Pipelines]]<br />
|-<br />
| 2016 || Tyanova S || [[The MaxQuant computational platform for mass spectrometry–based shotgun proteomics]]<br />
|-<br />
| 2016 || Röst HL || [[OpenMS: a flexible open-source software platform for mass spectrometry data analysis]]<br />
|-<br />
| 2017 || || [[A benchmarking of workflows for detecting differential splicing and differential expression at isoform level in human RNA-seq studies]]<br />
|-<br />
| 2018 || Välikangas T || [[A comprehensive evaluation of popular proteomics software workflows for label-free proteome quantification and imputation]]<br />
|-<br />
| 2019 || || [[A Systematic Evaluation of Single CellRNA-Seq Analysis Pipelines]]<br />
|-<br />
| 2019 || || [[Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays]]<br />
|}<br />
<br />
=== Preprocessing high-throughput data===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|- 1999 || Perkins DN || [[Probability-based protein identification by searching sequence databases using mass spectrometry data]]<br />
|-<br />
| 2003 || || [[A comparison of normalization methods for high density oligonucleotide array data based on variance and bias ]]<br />
|-<br />
| 2005 || || [[Comparison of Affymetrix GeneChip Expression Measures]]<br />
|-<br />
| 2005 || Meleth S || [[The case for well-conducted experiments to validate statistical protocols for 2D gels: different pre-processing = different lists of significant proteins]]<br />
|-<br />
| 2005 || || [[Comparison of background correction and normalization procedures for high-density oligonucleotide microarrays]]<br />
|-<br />
| 2006 || || [[Using RNA sample titrations to assess microarray platform performance and normalization techniques]]<br />
|-<br />
| 2006 || Wang P || [[Normalization regarding non-random missing values in high-throughput mass spectrometry data]]<br />
|-<br />
| 2007 || Carvalho B || [[Exploration, normalization, and genotype calls of high-density oligonucleotide SNP array data]]<br />
|-<br />
| 2007 || Cannataro M || [[MS‐Analyzer: preprocessing and data mining services for proteomics applications on the Grid]]<br />
|-<br />
| 2008 || || [[Comparison of preprocessing methods for the hgU133+2 chip from Affymetrix]]<br />
|-<br />
| 2009 || || [[Comparison of Affymetrix data normalization methods using 6,926 experiments across five array generations]]<br />
|-<br />
| 2009 || Mar JC || [[Data-driven normalization strategies for high-throughput quantitative RT-PCR]]<br />
|-<br />
| 2009 || Vakhrushev SY || [[Software platform for high-throughput glycomics]]<br />
|-<br />
| 2010 || || [[Consistency of predictive signature genes and classifiers generated using different microarray platforms]]<br />
|-<br />
| 2010 || || [[Detecting and correcting systematic variation in large-scale RNA sequencing data]]<br />
|-<br />
| 2010 || || [[Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments]]<br />
|-<br />
| 2010 || || [[Normalization of RNA-seq data using factor analysis of control genes or samples]]<br />
|-<br />
| 2011 || || [[Affymetrix GeneChip microarray preprocessing for multivariate analyses]]<br />
|-<br />
| 2012 || || [[A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis]]<br />
|-<br />
| 2014 || || [[Normalyzer: A Tool for Rapid Evaluation of Normalization Methods for Omics Data Sets]]<br />
|-<br />
| 2014 || Zhou X || [[Prevention, diagnosis and treatment of high-throughput sequencing data pathologies]]<br />
|-<br />
| 2014 || Coble JB || [[Comparative evaluation of preprocessing freeware on chromatography/mass spectrometry data for signature discovery]]<br />
|-<br />
| 2015 || Caraus I || [[Detecting and overcoming systematic bias in high-throughput screening technologies: a comprehensive review of practical issues and methodological solutions]]<br />
|-<br />
| 2015 || Tam S || [[Optimization of miRNA-seq data preprocessing]]<br />
|-<br />
| 2015 || Rafiei A || [[Comparison of peak‐picking workflows for untargeted liquid chromatography/high‐resolution mass spectrometry metabolomics data analysis]]<br />
|-<br />
| 2015 || Chawade A || [[Data processing has major impact on the outcome of quantitative label-free LC-MS analysis]]<br />
|-<br />
| 2016 || Yi L || [[Chemometric methods in data processing of mass spectrometry-based metabolomics: A review]]<br />
|-<br />
| 2016 || Tsuji J || [[Evaluation of preprocessing, mapping and postprocessing algorithms for analyzing whole genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Li B || [[Performance Evaluation and Online Realization of Data-driven Normalization Methods Used in LC/MS based Untargeted Metabolomics Analysis]]<br />
|-<br />
| 2017 || Li B || [[NOREVA: normalization and evaluation of MS-based metabolomics data]]<br />
|-<br />
| 2018 || Mazoure B || [[Identification and Correction of Additive and Multiplicative Spatial Biases in Experimental High-Throughput Screening]]<br />
|-<br />
| 2018 || Li Z || [[Comprehensive evaluation of untargeted metabolomics data processing software in feature detection, quantification and discriminating marker selection]]<br />
|-<br />
| 2018 || Willforss J || [[NormalyzerDE: Online Tool for Improved Normalization of Omics Expression Data and High-Sensitivity Differential Expression Analysis]]<br />
|}</div>Bwdayhttps://www.benchmarking.uni-freiburg.de/index.php?title=Literature_Studies&diff=685Literature Studies2020-02-25T15:28:05Z<p>Bwday: </p>
<hr />
<div>__NUMBEREDHEADINGS__<br />
{| class="wikitable"<br />
|-<br />
! Page summary<br />
|-<br />
| Here outcomes of benchmarking studies from the literature are collected. The primary aim is a comprehensive overview about neutral benchmark studies, i.e. assessments which were performed independenty on publication of a new approach. Studies which are not neutral are put in brackets. </br> <br />
<br />
The focus is on computational methods for analyzing experimental data (instead of comparing experimental techniques or platforms). </br><br />
<br />
Please extend this list by creating a new page and adding a link below. </br> <br />
Use the '''[[Guidelines_for_Summarizing_a_Literature_Study|guidelines described here]]'''.<br />
|}<br />
<br />
== Results from Literature ==<br />
<br />
=== Classification ===<br />
''' 2003 '''</br><br />
* [[Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data]]<br />
''' 2005 '''</br><br />
* [[A review and comparison of classification algorithms for medical decision making]]<br />
''' 2016 '''</br><br />
* [[Predicting Breast Cancer Survivability Using Data Mining Techniques]]<br />
<br />
=== Selection of Differential Features and Regions ===<br />
==== Identifying differential features ====<br />
''' 2006 '''</br><br />
* [[Rat toxicogenomic study reveals analytical consistency across microarray platforms]]<br />
''' 2010 '''</br><br />
* [[A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the sequencing Quality control consortium]]<br />
''' 2017 '''</br><br />
* [[Identification of differentially expressed peptides in high-throughput proteomics data]]<br />
* [[In-depth method assessments of differentially expressed protein detection for shotgun proteomics data with missing values]]<br />
* [[Strategies for analyzing bisulfite sequencing data]]<br />
''' 2018 '''</br><br />
* [[Identification of Differentially Methylated Sites with Weak Methylation Effects]]<br />
<br />
==== Identifying differential regions (e.g. DMRs) ====<br />
''' 2015 '''<br />
* [[De novo identification of differentially methylated regions in the human genome]]<br />
* [[MethylAction: detecting differentially methylated regions that distinguish biological subtypes]]<br />
* [[metilene: Fast and sensitive calling of differentially methylated regions from bisulfite sequencing data]]<br />
''' 2016 '''<br />
* [[seqlm: an MDL based method for identifying differentially methylated regions in high density methylation array data]]<br />
* [[Statistical methods for detecting differentially methylated regions based on MethylCap-seq data]]<br />
''' 2017 '''<br />
* [[DMRfinder: efficiently identifying differentially methylated regions from MethylC-seq data]]<br />
''' 2018 '''<br />
* [[Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus]]<br />
* [[DMRcaller: a versatile R/Bioconductor package for detection and visualization of differentially methylated regions in CpG and non-CpG contexts]]<br />
* [[MethCP: Differentially Methylated Region Detection with Change Point Models (bioRxiv)]]<br />
<br />
==== Identifying sets of features (e.g. gene set analyses) ====<br />
'''2009<br />
<br />
* [[A general modular framework for gene set enrichment analysis]]<br />
* [[Comparing gene set analysis methods on single-nucleotide polymorphism data from Genetic Analysis Workshop 16]]<br />
<br />
'''2018<br />
<br />
* [[Gene set analysis methods: a systematic comparison]]<br />
<br />
==== Dimension reduction ====<br />
''' 2008 '''</br><br />
* [[On the Relationship Between Feature Selection and Classification Accuracy]]<br />
''' 2015 '''</br><br />
* [[Comparing feature selection methods for highdimensional imbalanced data: identifying rheumatoid arthritis cohorts from routine data]]<br />
<br />
=== Imputation methods for missing values ===<br />
<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 1996 || Schenker || [[Partially parametric techniques for multiple imputation]]<br />
|-<br />
| 1999 || Hastie T || [[Imputing Missing Data for Gene Expression Arrays]]<br />
|-<br />
| 2001 || Troyanskaya || [[Missing value estimation methods for DNA microarrays]]<br />
|-<br />
| 2002 || Engels J || [[Imputation of missing longitudinal data: a comparison of methods]]<br />
|-<br />
| 2003 || Oba || [[A Bayesian missing value estimation method for gene expression profile data]]<br />
|-<br />
| 2005 || Scholz || [[Nonlinear PCA: a missing data approach]]<br />
|-<br />
| 2007 || Stacklies || [[pcaMethods—a bioconductor package providing PCA methods for incomplete data]]<br />
|-<br />
| 2007 || Verboven || [[Sequential imputation for missing values]]<br />
|-<br />
| 2008 || Shaffer GN || [[Which missing value imputation method to use in expression profiles: a comparative study and two selection schemes]]<br />
|-<br />
| 2011 || Templ || [[Iterative stepwise regression imputation using standard and robust methods]]<br />
|-<br />
| 2012 || Hrydziuszko O || [[Missing values in mass spectrometry based metabolomics: an undervalued step in the data processing pipeline]]<br />
|-<br />
| 2012 || Stekhoven || [[MissForest—non-parametric missing value imputation for mixed-type data]]<br />
|-<br />
| 2013 || Taylor || [[Accounting for undetected compounds in statistical analyses of mass spectrometry ‘omic studies]]<br />
|-<br />
| 2013 || Waljee || [[Comparison of imputation methods for missing laboratory data in medicine]]<br />
|-<br />
| 2014 || Shah || [[Comparison of Random Forest and Parametric Imputation Models for Imputing Missing Data Using MICE: A CALIBER Study]]<br />
|-<br />
| 2014 || Rodwell || [[Comparison of methods for imputing limited-range variables: a simulation study]]<br />
|-<br />
| 2014 || Morris || [[Tuning multiple imputation by predictive mean matching and local residual draws]]<br />
|-<br />
| 2014 || Doove L || [[Recursive partitioning for missing data imputation in the presence of interaction effects]]<br />
|-<br />
| 2015 || Webb-Robertson BJM || [[Review, evaluation, and discussion of the challenges of missing value imputation for mass spectrometry-based label-free global proteomics]]<br />
|-<br />
| 2016 || Folch-Fortuny A || [[Assessment of maximum likelihood PCA missing data imputation]]<br />
|-<br />
| 2016 || Lazar C || [[Accounting for the Multiple Natures of Missing Values in Label-Free Quantitative Proteomics Data Sets to Compare Imputation Strategies]]<br />
|-<br />
| 2016 || Yin X || [[Multiple imputation and analysis for high-dimensional incomplete proteomics data]]<br />
|-<br />
| 2018 || Wei R || [[Missing Value Imputation Approach for Mass Spectrometry-based Metabolomics Data]]<br />
|-<br />
| 2018 || Poyatos R || [[Gap-filling a spatially explicit plant trait database: comparing imputation methods and different levels of environmental information]]<br />
|-<br />
| 2018 || O'Brien JJ || [[The effects of nonignorable missing data on label-free mass spectrometry proteomics experiments]]<br />
|}<br />
<br />
=== ODE-based Modelling ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2001 || Schenker || [[Ways to Fit a PK Model with Some Data Below the Quantification Limit]]<br />
|-<br />
| 2008 || Schenker || [[Hybrid optimization method with general switching strategy for parameter estimation]]<br />
|-<br />
| 2011 || Schenker || [[Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis]]<br />
|-<br />
| 2013 || Schenker || [[Lessons Learned from Quantitative Dynamical Modeling in Systems Biology]]<br />
|-<br />
| 2013 || Schenker || [[ODE parameter inference using adaptive gradient matching with Gaussian processes]]<br />
|-<br />
| 2017 || Schenker || [[Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems]]<br />
|-<br />
| 2017 || Schenker || [[Data-driven reverse engineering of signaling pathways using ensembles of dynamic models]]<br />
|-<br />
| 2017 || Schenker || [[Fast derivatives of likelihood functionals for ODE based models using adjoint-state method]]<br />
|-<br />
| 2017 || Schenker || [[Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy]]<br />
|-<br />
| 2017 || Schenker || [[Performance of objective functions and optimization procedures for parameter estimation in system biology models]]<br />
|-<br />
| 2017 || Schenker || [[Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks]]<br />
|-<br />
| 2018 || Schenker || [[Evaluation of Derivative-Free Optimizers for Parameter Estimation in Systems Biology]]<br />
|-<br />
| 2018 || Schenker || [[Hierarchical optimization for the efficient parametrization of ODE models]]<br />
|-<br />
| 2018 || Schenker || [[Input-dependent structural identifiability of nonlinear systems]]<br />
|-<br />
| 2018 || Schenker || [[Optimization and profile calculation of ODE models using second order adjoint sensitivity analysis]]<br />
|-<br />
| 2019 || Schenker || [[A comparison of methods for quantifying prediction uncertainty in systems biology]]<br />
|-<br />
| 2019 || Schenker || [[Benchmark problems for dynamic modeling of intracellular processes]]<br />
|-<br />
| 2019 || Schenker || [[Benchmarking optimization methods for parameter estimation in large kinetic models]]<br />
|-<br />
| 2019 || Schenker || [[Efficient computation of steady states in large-scale ODE models of biochemical reaction networks]]<br />
|-<br />
| 2019 || Schenker || [[Full observability and estimation of unknown inputs, states and parameters of nonlinear biological models]]<br />
|-<br />
| 2019 || Schenker || [[Mini-batch optimization enables training of ODE models on large-scale datasets]]<br />
|-<br />
| 2019 || Schenker || [[Parameter Estimation and Variable Selection for Big Systems of Linear Ordinary Differential Equations: A Matrix-Based Approach]]<br />
|-<br />
| 2019 || Schenker || [[Parameter estimation in models of biological oscillators: an automated regularised estimation approach]]<br />
|-<br />
| 2019 || Schenker || [[Robust calibration of hierarchical population models for heterogeneous cell populations]]<br />
|-<br />
| 2019 || Schenker || [[Scalable nonlinear programming framework for parameter estimation in dynamic biological system models]]<br />
|-<br />
| 2019 || Schenker || [[Tracking for parameter and state estimation in possibly misspecified partially observed linear Ordinary Differential Equations]]<br />
|-<br />
| 2020 || Schenker || [[An application of Conditional RobustCalibration (CRC) to ordinary differential equations (ODEs) models in computational systems biology: a comparison of two sampling strategies]]<br />
|-<br />
| 2020 || Schenker || [[Efficient parameterization of large-scale dynamic models based on relative measurements]]<br />
|-<br />
| 2020 || Schenker || [[Testing structural identifiability by a simple scaling method]]<br />
|}<br />
<br />
=== Omics Workflows ===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|-<br />
| 2008 || Neuweger H || [[MeltDB: a software platform for the analysis and integration of metabolomics experiment data]]<br />
|-<br />
| 2008 || Barla A || [[Machine learning methods for predictive proteomics]]<br />
|-<br />
| 2009 || Xia J || [[MetaboAnalyst: a web server for metabolomic data analysis and interpretation]]<br />
|-<br />
| 2013 || Weisser H || [[An Automated Pipeline for High-Throughput Label-Free Quantitative Proteomics]]<br />
|-<br />
| 2014 || Cox J || [[Accurate Proteome-wide Label-free Quantification by Delayed Normalization and Maximal Peptide Ratio Extraction, Termed MaxLFQ* ]]<br />
|-<br />
| 2015 || || [[ComparingVariant Call Files for Performance Benchmarkingof Next-Generation Sequencing Variant Calling Pipelines]]<br />
|-<br />
| 2016 || Tyanova S || [[The MaxQuant computational platform for mass spectrometry–based shotgun proteomics]]<br />
|-<br />
| 2016 || Röst HL || [[OpenMS: a flexible open-source software platform for mass spectrometry data analysis]]<br />
|-<br />
| 2017 || || [[A benchmarking of workflows for detecting differential splicing and differential expression at isoform level in human RNA-seq studies]]<br />
|-<br />
| 2018 || Välikangas T || [[A comprehensive evaluation of popular proteomics software workflows for label-free proteome quantification and imputation]]<br />
|-<br />
| 2019 || || [[A Systematic Evaluation of Single CellRNA-Seq Analysis Pipelines]]<br />
|-<br />
| 2019 || || [[Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays]]<br />
|}<br />
<br />
=== Preprocessing high-throughput data===<br />
{| class="wikitable sortable"<br />
|-<br />
! Year || First Author || Title<br />
|- 1999 || Perkins DN || [[Probability-based protein identification by searching sequence databases using mass spectrometry data]]<br />
|-<br />
| 2003 || || [[A comparison of normalization methods for high density oligonucleotide array data based on variance and bias ]]<br />
|-<br />
| 2005 || || [[Comparison of Affymetrix GeneChip Expression Measures]]<br />
|-<br />
| 2005 || Meleth S || [[The case for well-conducted experiments to validate statistical protocols for 2D gels: different pre-processing = different lists of significant proteins]]<br />
|-<br />
| 2005 || || [[Comparison of background correction and normalization procedures for high-density oligonucleotide microarrays]]<br />
|-<br />
| 2006 || || [[Using RNA sample titrations to assess microarray platform performance and normalization techniques]]<br />
|-<br />
| 2006 || Wang P || [[Normalization regarding non-random missing values in high-throughput mass spectrometry data]]<br />
|-<br />
| 2007 || Carvalho B || [[Exploration, normalization, and genotype calls of high-density oligonucleotide SNP array data]]<br />
|-<br />
| 2007 || Cannataro M || [[MS‐Analyzer: preprocessing and data mining services for proteomics applications on the Grid]]<br />
|-<br />
| 2008 || || [[Comparison of preprocessing methods for the hgU133+2 chip from Affymetrix]]<br />
|-<br />
| 2009 || || [[Comparison of Affymetrix data normalization methods using 6,926 experiments across five array generations]]<br />
|-<br />
| 2009 || Mar JC || [[Data-driven normalization strategies for high-throughput quantitative RT-PCR]]<br />
|-<br />
| 2009 || Vakhrushev SY || [[Software platform for high-throughput glycomics]]<br />
|-<br />
| 2010 || || [[Consistency of predictive signature genes and classifiers generated using different microarray platforms]]<br />
|-<br />
| 2010 || || [[Detecting and correcting systematic variation in large-scale RNA sequencing data]]<br />
|-<br />
| 2010 || || [[Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments]]<br />
|-<br />
| 2010 || || [[Normalization of RNA-seq data using factor analysis of control genes or samples]]<br />
|-<br />
| 2011 || || [[Affymetrix GeneChip microarray preprocessing for multivariate analyses]]<br />
|-<br />
| 2012 || || [[A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis]]<br />
|-<br />
| 2014 || || [[Normalyzer: A Tool for Rapid Evaluation of Normalization Methods for Omics Data Sets]]<br />
|-<br />
| 2014 || Zhou X || [[Prevention, diagnosis and treatment of high-throughput sequencing data pathologies]]<br />
|-<br />
| 2014 || Coble JB || [[Comparative evaluation of preprocessing freeware on chromatography/mass spectrometry data for signature discovery]]<br />
|-<br />
| 2015 || Caraus I || [[Detecting and overcoming systematic bias in high-throughput screening technologies: a comprehensive review of practical issues and methodological solutions]]<br />
|-<br />
| 2015 || Tam S || [[Optimization of miRNA-seq data preprocessing]]<br />
|-<br />
| 2015 || Rafiei A || [[Comparison of peak‐picking workflows for untargeted liquid chromatography/high‐resolution mass spectrometry metabolomics data analysis]]<br />
|-<br />
| 2015 || Chawade A || [[Data processing has major impact on the outcome of quantitative label-free LC-MS analysis]]<br />
|-<br />
| 2016 || Yi L || [[Chemometric methods in data processing of mass spectrometry-based metabolomics: A review]]<br />
|-<br />
| 2016 || Tsuji J || [[Evaluation of preprocessing, mapping and postprocessing algorithms for analyzing whole genome bisulfite sequencing data]]<br />
|-<br />
| 2016 || Li B || [[Performance Evaluation and Online Realization of Data-driven Normalization Methods Used in LC/MS based Untargeted Metabolomics Analysis]]<br />
|-<br />
| 2017 || Li B || [[NOREVA: normalization and evaluation of MS-based metabolomics data]]<br />
|-<br />
| 2018 || Mazoure B || [[Identification and Correction of Additive and Multiplicative Spatial Biases in Experimental High-Throughput Screening]]<br />
|-<br />
| 2018 || Li Z || [[Comprehensive evaluation of untargeted metabolomics data processing software in feature detection, quantification and discriminating marker selection]]<br />
|-<br />
| 2018 || Willforss J || [[NormalyzerDE: Online Tool for Improved Normalization of Omics Expression Data and High-Sensitivity Differential Expression Analysis]]<br />
|}</div>Bwday