Treffer: MLcps: machine learning cumulative performance score for classification problems.

Title:
MLcps: machine learning cumulative performance score for classification problems.
Authors:
Akshay A; Functional Urology Research Group, Department for BioMedical Research DBMR, University of Bern, 3008 Bern, Switzerland.; Graduate School for Cellular and Biomedical Sciences, University of Bern, 3012 Bern, Switzerland., Abedi M; Department of Medical Data Science, Leipzig University Medical Centre, 04107 Leipzig, Germany., Shekarchizadeh N; Department of Medical Data Science, Leipzig University Medical Centre, 04107 Leipzig, Germany.; Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI) Dresden/Leipzig, 04105 Leipzig, Germany., Burkhard FC; Functional Urology Research Group, Department for BioMedical Research DBMR, University of Bern, 3008 Bern, Switzerland.; Department of Urology, Inselspital University Hospital, 3010 Bern, Switzerland., Katoch M; Institute of Neuropathology, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), 91054 Erlangen, Germany., Bigger-Allen A; Biological & Biomedical Sciences Program, Division of Medical Sciences, Harvard Medical School, 02115 Boston, MA, USA.; Urological Diseases Research Center, Boston Children's Hospital, 02115 Boston, MA, USA.; Department of Surgery, Harvard Medical School, 02115 Boston, MA, USA.; Broad Institute of MIT and Harvard, 02142 Cambridge, MA, USA., Adam RM; Urological Diseases Research Center, Boston Children's Hospital, 02115 Boston, MA, USA.; Department of Surgery, Harvard Medical School, 02115 Boston, MA, USA.; Broad Institute of MIT and Harvard, 02142 Cambridge, MA, USA., Monastyrskaya K; Functional Urology Research Group, Department for BioMedical Research DBMR, University of Bern, 3008 Bern, Switzerland.; Department of Urology, Inselspital University Hospital, 3010 Bern, Switzerland., Gheinani AH; Functional Urology Research Group, Department for BioMedical Research DBMR, University of Bern, 3008 Bern, Switzerland.; Department of Urology, Inselspital University Hospital, 3010 Bern, Switzerland.; Urological Diseases Research Center, Boston Children's Hospital, 02115 Boston, MA, USA.; Department of Surgery, Harvard Medical School, 02115 Boston, MA, USA.; Broad Institute of MIT and Harvard, 02142 Cambridge, MA, USA.
Source:
GigaScience [Gigascience] 2022 Dec 28; Vol. 12. Date of Electronic Publication: 2023 Dec 13.
Publication Type:
Journal Article; Research Support, Non-U.S. Gov't; Research Support, N.I.H., Extramural
Language:
English
Journal Info:
Publisher: Oxford University Press Country of Publication: United States NLM ID: 101596872 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 2047-217X (Electronic) Linking ISSN: 2047217X NLM ISO Abbreviation: Gigascience Subsets: MEDLINE
Imprint Name(s):
Publication: 2017- : New York : Oxford University Press
Original Publication: London : BioMed Central
MeSH Terms:
Machine Learning*
References:
Circ Cardiovasc Qual Outcomes. 2020 Oct;13(10):e006556. (PMID: 33079589)
Mol Pharm. 2018 Oct 1;15(10):4361-4370. (PMID: 30114914)
NPJ Digit Med. 2020 Mar 9;3:30. (PMID: 32195365)
Int Conf Affect Comput Intell Interact Workshops. 2013;2013:245-251. (PMID: 25574450)
Gigascience. 2024 Jan 2;13:. (PMID: 38206587)
Diagnostics (Basel). 2023 Feb 08;13(4):. (PMID: 36832106)
Bioinformatics. 2010 Jan 1;26(1):139-40. (PMID: 19910308)
Nucleic Acids Res. 2016 May 5;44(8):e71. (PMID: 26704973)
Molecules. 2019 Aug 01;24(15):. (PMID: 31374986)
IEEE Trans Pattern Anal Mach Intell. 2023 Feb;45(2):1501-1513. (PMID: 35417345)
J Clin Invest. 2018 Jan 2;128(1):427-445. (PMID: 29227286)
Stat Methods Med Res. 2021 Oct;30(10):2352-2366. (PMID: 34468239)
Sci Rep. 2022 Apr 8;12(1):5979. (PMID: 35395867)
Kidney360. 2021 Mar 03;2(5):878-880. (PMID: 35373058)
BMC Biol. 2010 May 11;8:58. (PMID: 20459774)
Front Oncol. 2023 Feb 14;13:1130229. (PMID: 36845729)
Gigascience. 2022 Dec 28;12:. (PMID: 38091508)
Grant Information:
310030 National Science Foundation; R01 DK077195 United States DK NIDDK NIH HHS; R01 DK127673 United States DK NIDDK NIH HHS
Contributed Indexing:
Keywords: Python package; classification problems; machine learning; model evaluation; unified evaluation score
Entry Date(s):
Date Created: 20231213 Date Completed: 20231216 Latest Revision: 20250106
Update Code:
20250114
PubMed Central ID:
PMC10716825
DOI:
10.1093/gigascience/giad108
PMID:
38091508
Database:
MEDLINE

Weitere Informationen

Background: Assessing the performance of machine learning (ML) models requires careful consideration of the evaluation metrics used. It is often necessary to utilize multiple metrics to gain a comprehensive understanding of a trained model's performance, as each metric focuses on a specific aspect. However, comparing the scores of these individual metrics for each model to determine the best-performing model can be time-consuming and susceptible to subjective user preferences, potentially introducing bias.
Results: We propose the Machine Learning Cumulative Performance Score (MLcps), a novel evaluation metric for classification problems. MLcps integrates several precomputed evaluation metrics into a unified score, enabling a comprehensive assessment of the trained model's strengths and weaknesses. We tested MLcps on 4 publicly available datasets, and the results demonstrate that MLcps provides a holistic evaluation of the model's robustness, ensuring a thorough understanding of its overall performance.
Conclusions: By utilizing MLcps, researchers and practitioners no longer need to individually examine and compare multiple metrics to identify the best-performing models. Instead, they can rely on a single MLcps value to assess the overall performance of their ML models. This streamlined evaluation process saves valuable time and effort, enhancing the efficiency of model evaluation. MLcps is available as a Python package at https://pypi.org/project/MLcps/.
(© The Author(s) 2023. Published by Oxford University Press GigaScience.)