In my last post I tried to emphasize how my job, and the jobs of many of my colleagues, is done in response to a request of the federal government of the USA. By the wonders of a representative democratic republic, this means I work for you, the taxpayer.
Mostly this is by way of proposing research projects that one or more of the Institutes or Centers (ICs) of the National Institutes of Health decide are worthy of selecting for funding. In my case this mostly means the National Institute on Drug Abuse (NIDA) but the process is roughly the same across all of the NIH’s ICs, including the National Cancer Institute, the National Institute of Mental Health, the National Institute of Diabetes and Digestive and Kidney Diseases…etc. Not everything we scientists propose gets selected for funding. The NIH is selective about what it chooses to request we scientists work on.
Highly selective.
In very brief outline, I am able to submit grant proposals (applications) roughly three times per year, Fall, Winter and Summer. These are assigned to a review panel of my peers (these are called Study Sections, I’ve served on these panels myself) which meets three times a year. As an example, about 80 major grant proposals are assigned to a panel of 25-30 scientists with expertise relevant to the pool of applications. Three panelists review each proposal in depth, assigning an overall merit score and writing some formalized assessments. Each panelist might review 7-10 proposals. The scores are then averaged for the three reviewers of a given proposal and all of the 80 applications assigned to a study section meeting are ranked.
The lower half of these proposals are given a “Not Discussed” or ND designation and considered no further. That is correct. About 50% of proposals are simply discarded* at this phase. NIH is highly selective.
It is worth mentioning that these grant proposals are not just one-page thought experiments. They are very difficult to prepare, have to convince a wide scientific audience that the work is important, they rest on a lot of supporting data and tend to describe a 5 year plan of attack on a problem. Preparing one proposal per round (three per year) is not simple. I work pretty hard at this, at times, and I think I’ve only reached six proposals in a given Fiscal Year once or twice in the past 25 years.
The top ~half of the proposals then go to a meeting of the panel where the merits and weaknesses are discussed, primarily by three assigned peers. The rest of the panel can then ask questions, discuss key points, etc. At the end of this discussion the three assigned people give another overall merit score recommendation, which can be modified from the original score based on the discussion. The entire panel then votes on a score, generally (but not always) within the range of the three primary reviewers. This average score then becomes the overall merit score of that proposal.
There is a further complication. The scores are turned into a percentile based on a moving three-meeting average of the grant proposals reviewed in that panel, e.g. in the current round and the two prior rounds. The reason for the percentile is to try to account for differences in scoring behavior across all of the various study sections that review grants. If the possible scores range from 10 to 90, with lower being higher merit, one section might tend to score their best grants at a 10 (perfect) whereas other sections might tend to score their best grants at a 20, with the 10 reserved for extremely rare circumstances. The percentile within study section adjusts for this.
The NIH provides a lot of information about their funding process, including this depiction of how many grants are funded and not funded at each of these percentile ranks. The bars indicate the number of applications of major grants (R01 or equivalent) scored at each percentile from 1 (left end, “good”) to 50 (right end, “not meritorious”). The light blue bars indicate proposals that were funded and the yellow bars indicate proposals that were not funded at each percentile rank. (We’ll ignore the dark blue bars for now, these are special cases of partially funded awards.) This chart is for Fiscal Year 2024 and it shows that at about 10%ile and below (i.e., the top 10% of proposals as evaluated by peer review) almost every proposal was funded. Above about 25%ile, almost no proposals were funded. In between the 10%ile and 25%ile, you can see that the chances of the grant being awarded are variable, but roughly aligned with percentile- the better the percentile, the better the chances of being funded. Somewhere around 14-16%ile, half of the proposals are funded and half are not.
The reasons for one proposal being funded and one not being funded at a given peer review rank are multiple and range far beyond this discussion. But some of it has to do with giving a break to Early Stage Investigators (who have never won a grant before), covering specific topic interests of the IC, reacting to Congressionally specified priorities, or avoiding unnecessary duplication of effort.
These relationships hold true, more or less, at the level of each IC, although the patterns are harder to see as the number of funded grant proposals declines. The National Cancer Institute gets about 15% of the overall budget, the most of any IC and therefore funds the most grants. Their data are here. NIDA is smaller and there is more apparent variability but you can make out the pattern.
In the NIH-wide data above, you can see that a proposal had to score in the top 10% of proposals to have an exceptionally high chance of being funded. Considering all funded and not-funded proposals at every percentile rank (and ND scores), NIH funded about** 19% of R01equiv applications in FY2024.
Perhaps obviously, the selectivity of the NIH is related to the dollars that Congress chooses to appropriate for the NIH. From 2002 to about 2013 the NIH budget was merely flatlined in nominal dollars. This means that the purchasing power of the NIH eroded with inflation. Success rates went from 32% in 2000-2001 to a low of 17% in 2013.
Again, this was while the NIH appropriation was merely held steady.
The Administration is proposing a 40% reduction in the overall NIH appropriation for Fiscal Year 2026. Congress has recessed in August without significant progress on the appropriations for Health and Human Services, which contains the NIH. While Congress appears likely to moderate the size of that reduction, it seems quite likely that a significant cut will occur. The National Cancer Institute has just announced it will adopt a payline of 4%ile for the final round of grants being considered for this fiscal year. The payline is the rank at which almost everything is expected to win funding and this is down from about a 10%ile payline for 2024.
*There may be a temptation to assume these proposals are half junk anyway. This is incorrect. I have reviewed grants and in my experience the truly terrible proposal is quite rare, maybe 5% of the ones I have seen. There is something of value in the majority of proposals, I’d estimate 66% or more of them.
**There is a weirdness here in the calculation, having to do with proposals that are submitted once, reviewed and then resubmitted in revised form within a single Fiscal Year being considered as one application- this raises the apparent success rate by an unknown amount.
TL neuro 