Beyond the Hype: Wielding AI as a Precise Tool in Anesthesiology Research and Practice

Christian S. Guay, MD

The integration of artificial intelligence (AI) into medical literature has evolved from theoretical possibility to commonplace, yet implementing these powerful tools effectively requires nuanced understanding, proper reporting and careful application. During the session, “Using the Force – Applying Artificial Intelligence Effectively in Research and in Clinical Practice,” on Saturday, March 22, at the 2025 Annual Meeting, presented by IARS and SOCCA, three experts shared complementary insights on selecting appropriate analytical methods, publishing high-quality AI research, and implementing AI systems in clinical environments.

Hannah Lonsdale, MBChB, an assistant professor in pediatric anesthesiology, attending pediatric anesthesiologist and Director of Pediatric Simulation at Vanderbilt University Medical Center, moderated this dynamic session.

Kelly Michaelsen, MD, PhD, Assistant Professor in the Department of Anesthesiology and Pain Medicine at the University of Washington, opened with a comparison between traditional statistics and AI, characterizing them as “scalpel versus sledgehammer” approaches to data analysis. “The dirty secret of AI is that at a fundamental level, it’s using traditional statistics, albeit in a complex way,” Dr. Michaelsen explained, noting that while AI excels at handling numerous variables simultaneously, its results can be more challenging to interpret.

Dr. Michaelsen candidly addressed AI’s limitations, including its tendency to propagate biases present in training data, substantial computational costs, requirements for high-volume datasets, and the fact that many models are controlled by corporations with limited customizability. She advocated for hybrid approaches that leverage each method’s strengths, citing natural language processing of electronic medical records as an example where AI can extract data that subsequently feeds traditional statistical analyses.

To guide researchers in choosing appropriate analytical techniques, Dr. Michaelsen recommended asking four key questions: What specific outcome are you seeking? Can traditional statistics provide the answer? Is machine learning truly essential to addressing your research question? Are there ways to augment your dataset or analysis with machine learning components?

Thomas Hemmerling, MD, Professor of Anesthesiology at McGill University, shifted focus to publishing AI research, noting that “the quality bar for AI in scientific articles has substantially moved up over the last few years. It’s no longer just hype; it needs to be done well.” He drew parallels to publication patterns during the COVID-19 pandemic, describing a similar transition from quantity to quality in AI-focused research.

For researchers aiming to publish AI studies, Dr. Hemmerling highlighted three critical aspects: novelty of approach, clinical relevance of findings, and demonstrated generalizability — ideally through retrospective validation in one setting followed by prospective testing in another. Common rejection reasons for AI manuscripts include inadequate validation, limited clinical utility, poor reproducibility due to insufficient methodological details, and overstated conclusions.

To avoid these pitfalls, Dr. Hemmerling recommended following established reporting guidelines such as TRIPOD-AI, clearly articulating clinical impact, performing external validation, addressing implementation challenges, and engaging clinical experts early in the research process.

Theodora Wingert, MD, a practicing pediatric anesthesiologist and clinical informaticist in the Department of Anesthesiology and Perioperative Medicine at the UCLA, concluded the session with practical insights on implementing AI/ML models in research and clinical settings. She emphasized the “five rights” of clinical decision support: right information, right people, right format, right channel, and right time. “A good clinical decision support tool goes unnoticed in clinical workflow, whereas a bad tool becomes annoying,” Dr. Wingert observed.

Dr. Wingert outlined a four-step implementation process: starting with a well-defined clinical problem, creating a model in an idealized in-silico environment, embedding the model in production systems, and evaluating real-world deployment. Successfully navigating this pathway requires several key ingredients: analytic-ready datasets, AI-enabled platforms, effective communication channels for results, and recent data to ensure relevance.

She illustrated these principles by reviewing a recent publication that developed an automated machine learning model to predict postoperative mortality using preoperative electronic health record data, demonstrating how thoughtful implementation can transform theoretical AI capabilities into practical clinical tools.

As AI continues to transform anesthesiology research and practice, this session highlighted the importance of selecting appropriate analytical approaches, maintaining scientific rigor in AI research, and designing implementation strategies that seamlessly integrate into clinical workflows. By addressing these considerations, anesthesiologists can move beyond the hype to harness AI’s genuine potential for improving patient care and advancing scientific understanding.