DELIRIUM remains a critical yet often underrecognized condition in acute care settings with direct links to increased mortality and extended hospital stays. Now a new scoping review raises important questions about the reliability of predictive models designed to detect delirium in adult patients. The study, led by Schöler LM et al., reveals how inconsistent labeling and underreporting are undermining the very foundation on which machine learning models are built.
Analyzing 120 studies across Cochrane, PubMed, and IEEE databases, the review highlights that only 42.5% of these models used primary data sources while 40.8% relied on routine clinical data. The most widely used assessment tool was the Confusion Assessment Method, applied in 60% of studies. Despite this, a striking 35% of the studies experienced label or data leakage, compromising the integrity of the predictive models.
The researchers found that nearly one-third of the studies (31.7%) lacked sufficient detail in their reporting, and 36.7% showed flawed outcome determination. Studies that utilized primary data demonstrated a lower risk of bias, while those with unclear labeling practices were more susceptible to high bias, a critical concern for clinicians relying on these models for real-time decision-making.
The team emphasizes the need for rigorous planning in how labels are determined in model development. To address these ongoing issues, the authors propose a practical decision-support flowchart to help guide future research and model training efforts, offering a path toward more accurate and clinically useful tools.
This study sheds light on a crucial gap in clinical informatics and should prompt U.S. healthcare professionals and hospital systems to scrutinize how AI-driven prediction tools are developed and validated. With delirium continuing to pose high risks to patient outcomes, the findings underscore the urgent need to build more transparent and reliable prediction frameworks.
Reference:
Schöler LM et al. Determining the ground truth for the prediction of delirium in adult patients in acute care: a scoping review. JAMIA Open. 2025;8(3):ooaf037.
