Demystifying the Study Section: No Need to Fear Reviewer #2

Young May Cha, MD

The study section is the first level of scientific review and can be daunting for new grant applicants. On Sunday, May 19, moderator Douglas A. Colquhoun, MB ChB, MPH, MSc, assistant professor of anesthesiology at University of Michigan, led an illuminating session “Mock Study Section – An Interactive Panel,” cosponsored by Early-Stage Anesthesiology Scholars (eSAS), at the 2024 Annual Meeting, presented by IARS and SOCCA. Using a real grant application, the study section reviewers discussed the process and their insights to help demystify this process. In this mock study section, Daniel S. Rubin, MD, associate professor of anesthesia and critical care at University of Chicago, served as the Study Section Chair. Lauriane Guichard, MD, assistant professor of anesthesiology at the University of North Carolina-Chapel Hill served as Reviewer 1, Saul Siller, MD, PhD, clinical fellow at Yale University served as Reviewer 2, and Cameron R. Bosinski, MD, MS, clinical fellow at Brigham and Women’s Hospital served as Reviewer 3.

A study section is a confidential process to assess the scientific merit of an application. Its members have broad-based expertise on the subject matter. For each application, there are three reviewers who review, provide preliminary scores, and give critiques. Like golf, lower scores are better scores. After preliminary scoring, the top 50% of applications are discussed at the study section meeting. The bottom 50% of applications are not discussed. Each application discussion lasts 12-15 minutes. The entire study section, which has 20 standing members and 10 ad hoc members, then provide final scores. The process is confidential, and much effort goes into making it as fair as possible. Reviewers are trained on bias and there are strict rules around conflict of interest. Applicants may not contact anyone on the study section and any materials generated by the study section are destroyed after the review. Afterwards, the applicant receives a summary statement, which includes the scores and narrative critiques from the reviewers. Funding decisions are based on percentiles or impact score, which are institute- and mechanism-specific.

The second level of review is the Institute Council Review, which is an advisory board meeting by program officers. Early-stage investigators tend to be funded at lower percentile scores and K applications have approximately a 35% success rate when including new applications and resubmission. It is very common to have a first-time K submission unfunded, but the program officer can provide information and guidance about next steps for resubmission or creation of a new application.

The panel provided some helpful tips for investigators looking to submit a proposal. One, the career development plan is often overlooked by applicants. Having a mentor in any area of deficit can strengthen your application. Second, often the science itself gets the highest score (worst score), but the research plan is also the easiest thing to change in a resubmission. Third is the importance of being very clear in your application with no ambiguity or assumptions. In an NIH study section, the group of reviewers can be very diverse. Unlike a study section for an anesthesia society research award, hardly anyone on an NIH study section may be an anesthesiologist.

The study section criteria are also changing. “Significance and innovation” will become “importance of research.” “Approach” will be “rigor and feasibility.” “Investigator and environment” will now be “expertise and resources.” The panel also encourages researchers to get involved in the Early Career Reviewer Program. This program seeks to help early-career scientists become more competitive as grant applicants by providing practical experience with the review system. Early-career scientists with at least one year of experience as faculty, with a first or last author publication, and who have submitted a prior NIH grant but not served on a study section in the past are qualified to apply.