Can ChatGPT Perform HAI Surveillance? A New Study Offers Important Insights

Artificial intelligence is rapidly finding its way into healthcare workflows, and infection prevention is no exception. As large language models (LLMs) such as ChatGPT become increasingly accessible, many infection preventionists (IPs) are asking an important question: Can these tools help support healthcare-associated infection (HAI) surveillance?

A recently published study in the American Journal of Infection Control by IP&MA team members Kelly Holmes, Mishga Moinuddin, and Sandi Steinfeld, with Xiaoyu Liu, explored that question using a simulation-based evaluation of ChatGPT-4 and validated NHSN surveillance case scenarios. In their article, Simulation-based evaluation of ChatGPT for healthcare associated infection surveillance using validated case scenarios, the team evaluated how accurately and consistently ChatGPT could apply NHSN definitions to 20 de-identified surveillance vignettes commonly used to build infection prevention competency.

Across three identical testing runs, ChatGPT-4 achieved an average accuracy of 45%. More notably, only 20% of scenarios were answered correctly in all three attempts, highlighting variability in performance. The model performed best on laboratory-identified (LABID) events but struggled with cases requiring temporal reasoning, interpretation of multiple criteria, and complex surveillance logic. Trained infection preventionists significantly outperformed the AI on the same scenarios.

These findings demonstrate the current reality of generative AI in infection prevention: promising as a supplemental tool, but not yet reliable enough to replace the expertise, judgment, and critical thinking of trained infection prevention professionals. The study serves as both a proof-of-concept and a reminder that surveillance remains a highly specialized practice requiring nuanced interpretation of evolving NHSN definitions.

As AI technologies continue to evolve, infection prevention leaders will need to balance innovation with validation, ensuring that new tools enhance—not compromise—the accuracy of surveillance programs.

At Infection Prevention & Management Associates, we help organizations strengthen surveillance accuracy, regulatory readiness, and infection prevention program performance through expert consulting, NHSN surveillance support, competency-based education and training, and comprehensive program assessments. If your organization is evaluating how emerging technologies fit into your infection prevention strategy, our team can help you navigate the opportunities while maintaining the rigor and reliability that patient safety demands.