Clinical Prediction Models for Surgical Outcomes: Addressing Methodological Challenges in Implementation, Data, and Analysis
Research Project | 8 Project Members
Clinical prediction models can support surgical decision-making by identifying patient profiles that are at an increased risk of poor outcomes. While artificial intelligence (AI)-enabled decision aids demonstrated potential in enhancing shared decision-making and improving patient satisfaction, there remains a significant gap in implementation. Furthermore, few studies describe stakeholders’ preferences regarding the use of clinical prediction models. The adoption of decision support tools needs to be accelerated by (i) improving their transparency and reporting standards, (ii) addressing their implementation barriers, and (iii) aligning their development and implementation with stakeholder expectations to maximize their clinical impact.
Simultaneously, the growing deployment and development of modern machine learning and AI models have sparked debates about their performance in comparison to regression models, particularly when citing issues relevant for surgical databases, including varying sample size, atypical outcome distributions (for example ceiling effects in patient-reported outcome measures), and number and strength of predictive ability of prognostic factors.
The Clinical Prediction Models for Surgical Outcomes (CPMSO) project aims to address the challenges of implementation, data, and analysis by establishing standardized guidelines, ultimately improving the reliability and adoption of AI-driven decision support tools encompassing prediction models for surgical outcomes.
Research Objective (RO) 1: To develop and validate a consensus-driven set of recommendations that support the implementation of prediction models for surgical outcomes in patients with musculoskeletal disorders.
RO2: To assess the impact of data characteristics ((i) sample size, (ii) outcome distribution and (iii) number and predictive ability of prognostic factors) on the performance of machine-learning, regression, and Bayesian models for ordinal/continuous outcomes.
METHODS. In working package (WP) 1 related to RO1, we will (1) systematically map existing evidence on the implementation of prediction models in joint surgery through a scoping review, identifying methodological challenges and factors influencing their clinical adoption. We will also (2) explore the perceptions, needs, and implementation barriers of key stakeholders—including patients, healthcare professionals, and researchers— regarding prediction models for surgical outcomes, using focus group discussions. Based on these findings, we will then (3) develop and assess the relevance of a set of consensus recommendations addressing key issues for the implementation of prediction models for surgical outcomes. In WP2 related to RO2, the apparent and bootstrap-validated overall, discrimination and calibration performance of models predicting ordinal/continuous outcomes will be compared, including machine-learning, regression, and Bayesian models. First, data simulations will be generated, for which key data characteristics will be controlled ((i) sample size, (ii) outcome distribution and (iii) number and strength of predictive ability of prognostic factors). The validity of our findings will then be assessed using four nationwide surgical databases focusing on the prediction of pain (Numeric Rating Scale [0-10]) one year after surgery. Databases will be standardized to the Observational Health Data Sciences and Informatics (OHDSI) format, facilitating the further steps of analyses, which will include handling of missing data, predictor selection, model development, and internal validation.
EXPECTED RESULTS AND IMPACT. RO1, linked to WP1, will produce key recommendations for implementing prediction models for surgical outcomes, a crucial step toward enhanced decision-making and achieving better health and socioeconomic outcomes for patients and the society. RO2, linked to WP2, will provide methodological guidance on selecting optimal models for ordinal/continuous outcomes based on key data characteristics. The findings will be published in peer-reviewed journals and presented at international scientific conferences. The CPMSO project aims also to develop a professional network of experts, and support the integration of currently developed predictive analytics tools supporting surgical decision-making in Europe.