Cohort | Model | AUC | 95% CI | Optimal cutoff | Accuracy | Sensitivity | Specificity | Precision | F1 score |
---|---|---|---|---|---|---|---|---|---|
Internal validation cohort | LR | 0.806 | (0.787–0.825) | 0.571 | 0.728 | 0.749 | 0.713 | 0.659 | 0.701 |
MLP | 0.809 | (0.790–0.827) | 0.531 | 0.735 | 0.724 | 0.744 | 0.676 | 0.699 | |
XGBoost | 0.806 | (0.788–0.825) | 0.566 | 0.727 | 0.730 | 0.725 | 0.662 | 0.694 | |
Bagging | 0.801 | (0.782–0.820) | 0.578 | 0.727 | 0.709 | 0.740 | 0.669 | 0.688 | |
RF | 0.801 | (0.782–0.820) | 0.578 | 0.719 | 0.750 | 0.697 | 0.646 | 0.694 | |
NB | 0.797 | (0.778–0.816) | 0.464 | 0.714 | 0.747 | 0.689 | 0.640 | 0.689 | |
LightGBM | 0.796 | (0.776–0.815) | 0.577 | 0.716 | 0.731 | 0.705 | 0.647 | 0.687 | |
SVM | 0.769 | (0.747–0.791) | 0.594 | 0.736 | 0.704 | 0.760 | 0.685 | 0.694 | |
KNN | 0.767 | (0.746–0.788) | 0.619 | 0.694 | 0.753 | 0.650 | 0.614 | 0.677 | |
DT | 0.714 | (0.694–0.734) | 0.569 | 0.711 | 0.736 | 0.692 | 0.639 | 0.684 | |
Internal testing cohort | LR | 0.798 | (0.771–0.825) | 0.617 | 0.711 | 0.793 | 0.649 | 0.626 | 0.700 |
MLP | 0.796 | (0.769–0.823) | 0.579 | 0.697 | 0.847 | 0.585 | 0.602 | 0.704 | |
XGBoost | 0.798 | (0.771–0.825) | 0.641 | 0.709 | 0.857 | 0.599 | 0.612 | 0.714 | |
Bagging | 0.791 | (0.763–0.818) | 0.597 | 0.722 | 0.718 | 0.724 | 0.658 | 0.687 | |
RF | 0.792 | (0.764–0.819) | 0.556 | 0.720 | 0.700 | 0.734 | 0.661 | 0.680 | |
NB | 0.786 | (0.759–0.814) | 0.664 | 0.695 | 0.826 | 0.597 | 0.603 | 0.697 | |
LightGBM | 0.789 | (0.762–0.817) | 0.582 | 0.719 | 0.749 | 0.696 | 0.646 | 0.693 | |
SVM | 0.739 | (0.707–0.770) | 0.610 | 0.717 | 0.674 | 0.748 | 0.664 | 0.669 | |
KNN | 0.748 | (0.718–0.778) | 0.614 | 0.685 | 0.704 | 0.670 | 0.612 | 0.655 | |
DT | 0.721 | (0.693–0.749) | 0.569 | 0.718 | 0.746 | 0.696 | 0.645 | 0.692 | |
External validation cohort | LR | 0.831 | (0.825–0.837) | 0.923 | 0.777 | 0.756 | 0.790 | 0.695 | 0.724 |
MLP | 0.823 | (0.817–0.829) | 0.691 | 0.755 | 0.734 | 0.769 | 0.667 | 0.699 | |
XGBoost | 0.784 | (0.777–0.791) | 0.788 | 0.763 | 0.574 | 0.883 | 0.755 | 0.652 | |
Bagging | 0.757 | (0.751–0.764) | 0.874 | 0.758 | 0.633 | 0.837 | 0.710 | 0.669 | |
RF | 0.763 | (0.755–0.770) | 0.726 | 0.756 | 0.571 | 0.873 | 0.740 | 0.645 | |
NB | 0.798 | (0.792–0.805) | 0.997 | 0.742 | 0.762 | 0.729 | 0.640 | 0.696 | |
LightGBM | 0.807 | (0.800-0.814) | 0.912 | 0.766 | 0.723 | 0.794 | 0.689 | 0.706 | |
SVM | 0.607 | (0.598–0.615) | 0.641 | 0.731 | 0.436 | 0.917 | 0.769 | 0.556 | |
KNN | 0.774 | (0.767–0.781) | 0.880 | 0.738 | 0.628 | 0.807 | 0.673 | 0.650 | |
DT | 0.670 | (0.664–0.676) | 0.569 | 0.733 | 0.391 | 0.949 | 0.829 | 0.532 |