Human-supervised, large language model-based clinical decision support aligned to national newborn protocols in Kenya: a pragmatic, early-stage evaluation

Résumé

Introduction: Timely, protocol-adherent clinical decisions are crucial for reducing neonatal mortality in low-resource settings. Translating extensive national guidelines into bedside practice remains challenging. Objective: We developed and evaluated AIFYA, a human-supervised, large language model (LLM)-based clinical decision support system (CDSS) aligned with Kenya's national newborn care protocols. Methods: This prospective, mixed-methods, early-stage evaluation, guided by the DECIDE-AI framework, embedded AIFYA into routine workflows at two public health facilities (Level 5 and Level 4) in Bungoma County, Kenya, from September 2024 to June 2025. Primary outcomes were: (1) adoption, measured by cumulative neonatal cases managed; (2) training reach, assessed by credentialed healthcare workers (HCWs); and (3) guideline and citation concordance, evaluated through blinded review of 118 AI-generated recommendations by two neonatologists, with adjudication by a third. Secondary outcomes included protocol adherence and triage-to-decision time. Results: = 33), 76% were female (mean age 32.1 years). Expert review found 75% of recommendations were correct and 15% partially correct, with strong inter-rater reliability (weighted Cohen's kappa 0.85; 95% CI 0.79-0.91) between reviewers. Citation accuracy was 96%. In 40 complex dosing scenarios, 75% of outputs were rated correct. The median triage-to-decision time was 23 min (IQR 18-31). Implementation was supported by an offline-first architecture and a facility-based coaching model, sustaining engagement despite staff turnover. Conclusion: A human-supervised, guideline-aligned AI CDSS can be feasibly implemented in routine neonatal care in low-resource settings, with high adoption and guideline concordance. Citation-linked recommendations enhance transparency and support clinical verification with most clinically appropriate recommendations. However, variability in complex scenarios highlights the need for ongoing refinement and strict clinical oversight. These findings support progression to control trials to evaluate clinical effectiveness and safety.

Citer ce document

Accès au document

Texte intégral en lecture en ligne, réservé aux abonnés SPHAERO et aux membres de l'institution. Se connecter

Auteur(s)

Statistiques

Consultations : 1

Téléchargements : 0