Writing the AI/ML component of your research application – a 12-point checklist

Applications often budget clinician and research-nurse time in detail yet include no explicit budget for compute, cloud or GPU costs, data engineering, or ML-specific personnel time, even when the method clearly requires it. If your institution provides compute for free, say so explicitly and name the resource rather than omitting the cost. A missing AI budget line reads as unscoped work and hurts your feasibility score.

What this means

This one needs less translation, but the pattern is worth naming: applications routinely budget clinician time and research-nurse time down to the hour, then say nothing at all about the cost of the compute, cloud hosting, GPU time, data engineering, or ML-specific personnel that the AI component actually requires. It’s the equivalent of budgeting a kitchen renovation in detail (cabinets, tile, countertops) while forgetting to budget the electrician, even though rewiring the kitchen was the entire point of the renovation. A missing line doesn’t read as “we’ll figure it out”; it reads as “this part of the project wasn’t scoped,” which directly hurts your feasibility score.

If your institution provides free compute, that’s a perfectly good answer, but it has to be written down explicitly, naming the actual resource, not simply omitted.

This is the most consequential and most mishandled competence issue. Two failure modes recur:

• Adjacent-field borrowing: an outstanding publication record in a nearby field (control theory, robotics, general computer vision) does not substitute for direct expertise in the specific method proposed.
• The absent methodologist: the approach cites specific prior work, but the lead author of that work is not a funded co-applicant. If the person whose work you are building on is not on the team, the reviewer will ask who will execute the method.

Before submitting, check that everyone whose technique you cite is named in the application with real allocated time and funding, not just “in-kind” or covered by a different grant. If that person is really an unnamed PhD student, name them and fund them properly.

What this means

Adjacent-field borrowing. Having an excellent publication record in a nearby field doesn’t automatically transfer to the specific method you’re proposing. A world-class violin maker and a world-class guitar maker both work with wood, glue, and strings, and both are genuinely skilled. However, you wouldn’t assume the violin maker can walk in and build you a concert-quality guitar. Reviewers who know the field can tell the difference between “this person works in machine learning broadly” and “this person has actually built and validated the specific type of model this application proposes,” and they will look for the second, more specific thing.

The absent methodologist. This happens when an application cites a specific published method as its foundation (for example, “we will build on the approach of Smith et al. (2024)”), but the authors of that work, or anyone else who has implemented it, are nowhere on the funded team. A reviewer’s very next question is: who, concretely, is going to execute this? Citing a technique is not the same as being able to build it. It’s the equivalent of a restaurant’s business plan citing a Michelin-starred chef’s signature recipe, without a single person on the kitchen staff who has ever actually cooked it.

The fix is concrete: everyone whose technique your application leans on should be named, with real allocated time and real funding. Not just “in-kind,” not “covered by another grant,” not an uncredited PhD student doing the actual work behind the scenes. If that unnamed person exists, name them and fund them.

It is common to have ethics approval for the clinical parts of a study but not yet for the work packages that involve training models on patient data. If approval is pending, state exactly what stage it is at and by when it will be resolved: a named milestone reads very differently from a gap the reviewer must notice.

What this means

It’s common for a study to already have ethics approval for its clinical components, such as recruiting patients, collecting samples, running an intervention. But the specific work of training a model on that patient data hasn’t been separately approved. These are not automatically the same approval, because training and storing models on identifiable or re-identifiable health data raises its own governance questions. 

If your AI-specific approval is still pending, the fix is simple: say exactly what stage it’s at, and by when it will be resolved. A named milestone reads completely differently to a reviewer than an unexplained gap they must notice and flag themselves..

“AI-supported”, “powered by AI” and similar phrases cannot be evaluated, so they are worth close to nothing on their own. If you genuinely don’t know the exact architecture yet (for example, it depends on a vendor choice), say so and explain your decision criteria: that is honest and reviewable, whereas silence reads as not having thought it through. Some simplification is expected given reviewer expertise and page limits, but if you know the method, name it.

For example, instead of: “The platform uses AI to generate personalised content.” Write: “We will fine-tune [specific pretrained model/architecture] on [specific dataset], generating [specific output], evaluated against [specific reference].”

What this means

“AI” is a category, like “vehicle” or “medicine”, not a description of what you’re going to build. Saying you’ll “use AI” is like a construction firm telling a funder, “we will use tools.” A reviewer with technical expertise cannot evaluate whether your approach is sound, appropriately sized, or even sensible for your data, because you haven’t told them what the approach is.

A few terms worth pinning down here:

• Model: the specific mathematical system that takes your data in and produces a prediction out. “The model” is the thing you are building.
• Architecture: the general design or blueprint the model follows. A “convolutional neural network” is one architecture (good at finding patterns in images); a “transformer” is another (good at finding patterns in sequences, like text or time-series). Naming the architecture is like naming the make and model of a car, not just saying “vehicle.”
• Pretrained model: a model that someone else already trained on a large, general dataset before you got involved. You don’t have to start from nothing.
• Fine-tuning: taking that pretrained model and continuing to train it, but now on your own, smaller, specific dataset, so it specializes in your task. This is the difference between hiring a chef who already knows how to cook broadly and teaching them your restaurant’s specific menu, versus training a chef from absolute zero. Fine-tuning is usually faster and needs less data than starting from scratch, which is directly relevant to point 7 below.

If you genuinely don’t know the exact method yet (say, it depends on which vendor you end up contracting), that’s a legitimate thing to write, but say so explicitly in that case, and explain what will determine the choice, make sure this task is included in a work package and has budget hours. A reviewer can evaluate “we will choose between X and Y based on Z” much more easily than they can evaluate silence.

Vague validation language (“the model will be validated”) and mislabeled validation are both costly. A temporal holdout from the same single-site dataset is internal validation, often not enough on its own. External validation means a genuinely independent cohort: a different site, different vendor or instrument, ideally a different population or period. Mislabeling one as the other costs more than a point; it makes the reviewer read the rest of the application more skeptically.

Be explicit about the train/test/validation split methodology (patient-level, not just record-level, to avoid leakage), and what happens if the external validation fails: do you have a contingency plan, or does the whole clinical claim collapse?

What this means

“Validation” just means testing whether the model’s predictions hold up on data it did not learn from. The whole point of testing on unseen data is to check that the model learned a real pattern, not just memorized the specific patients it was shown.

Three data splits are usually involved, and it helps to think of them like preparing for an exam:

• Training set: the textbook you study from. The model adjusts itself based on this data.
• Validation set: a practice exam you use while studying, to decide which study strategy is working (which settings, which version of the model). You look at your practice-exam score and adjust your approach.
• Test set: the real final exam. You look at it exactly once, at the end, and you don’t get to go back and restudy based on how you did.

Internal vs. external validation is where a lot of applications quietly overstate themselves. Internal validation means testing on more data from the same source and same hospital, same scanner, maybe even a later period at the same site. That’s like practicing free throws on your own home court: useful, but it doesn’t tell you whether you can shoot on an unfamiliar court, with a different rim height, in front of a hostile crowd. External validation means testing on a genuinely independent cohort, like a different hospital, a different equipment vendor, ideally a different patient population or period. Reviewers read “externally validated” as a specific, strong claim. If what you actually have is a later batch of patients from the same hospital and the same scanner, that is internal validation and calling it “external” doesn’t just lose you a point; it makes the reviewer distrust the rest of the application, because now they have to wonder what else was labeled generously.

Patient-level splitting and data leakage. If one patient contributes multiple data points, several scans, several visits, several lab draws, and some of those end up in your training set while others from the same patient end up in your test set, the model can partly recognize that individual patient rather than learn a pattern that generalizes to new patients. This is called data leakage, and it inflates your reported performance in a way that won’t hold up once the model meets a genuinely new patient. It’s analogous to a student who has secretly seen several of the exact exam questions in advance, courtesy of a friend who took the same class last semester. Their score looks great, but it doesn’t reflect what they know.

“Explainability will be prioritised” is an aspiration, not a method. If interpretability matters for clinical adoption, which will be assumed for anything touching a treatment decision, name the actual technique (SHAP, attention visualisation, saliency maps, counterfactual explanations, or an explicitly interpretable model class) and show how you will validate that the explanation is correct, not just generate one as a checkbox.

What this means

“Explainability will be prioritised” is a stated intention, not a method. It’s the equivalent of a construction firm saying “safety will be prioritised” instead of naming which safety code they’re building to. If a clinician is going to act on the model’s output, and for anything touching a treatment decision reviewers will assume they are, the application needs to name the actual technique that will show why the model flagged a given patient, and how you’ll check that the explanation is actually correct, rather than just plausible-looking.

A few concrete techniques, in plain terms:

• SHAP (SHapley Additive exPlanations): for one individual prediction, SHAP shows how much each input feature pushed the risk score up or down from a baseline. Think of it like an itemized receipt: age contributed +0.08 to this patient’s risk score, elevated lactate contributed +0.15, normal white blood cell count contributed −0.04, and so on, all summing to the final total.
• Saliency maps / attention visualization: for models working on images (like a chest X-ray) or sequences, these produce a heat-map overlay showing which regions of the input most influenced the prediction, so a radiologist can see, roughly, where the model “looked.”
• An explicitly interpretable model class: sometimes the simplest fix is choosing a model that is transparent by construction (like a logistic regression with a small number of clinically meaningful inputs), so that no separate explanation tool is even needed. This is a legitimate answer, and sometimes the strongest one.

Naming the technique is only half of it. Show how you’ll validate that the explanation is correct (for example, checking that SHAP-highlighted features match known clinical risk factors), not just generated as a box-ticking exercise.

A large aggregate cohort (“>1,300 cases,” “25 years of registry data”) does not make a rare-event classifier feasible. What matters is the positive-class count for the specific outcome you are modelling, after inclusion and exclusion criteria have been applied.

State explicitly how many labelled cases you have, how many positive events exist for the rarest outcome you are modelling, and what you will do about class imbalance (reweighting, resampling, metrics such as PR-AUC rather than only ROC-AUC); if you don’t know this yet, flag it as a risk with a mitigation plan. Justify N against the model’s prior evidence base: a pretrained foundation model may credibly need a smaller confirmatory set than one trained from scratch.

What this means

"“We have 1,300 patients in our registry” and “we have 1,300 patients to learn from” are very different claims when the thing you’re trying to predict is rare. What matters statistically is not the size of the haystack, but how many needles are in it.

Think of it as fishing in a large lake: you might catch 1,300 fish, but if you’re trying to teach a model to recognize one rare species, and only 12 of those 1,300 fish belong to that species, your model is really learning from 12 examples, not 1,300. Everything about how much you can trust the model’s performance on that rare species depends on the number 12, not the number 1,300.

This is also where two related metrics come in, and it’s worth knowing the difference:

• ROC-AUC measures how well the model ranks patients who have the outcome above patients who don’t, across all possible decision thresholds. It’s a widely reported number, but when the outcome is rare, ROC-AUC can look deceptively good, and the model gets a lot of “easy credit” just from correctly recognizing most patients who don’t have the outcome.
• PR-AUC (precision-recall AUC) looks specifically at how well the model finds the rare positive cases without generating too many false alarms. When the outcome you care about is rare, PR-AUC is generally the more honest, more clinically relevant number to report. A reviewer who knows this will specifically look for it, and its absence will read as a gap.

Finally: how much data you need isn’t a fixed number. It depends on how much your model already “knows” going in. A pretrained foundation model, per point 4, may need a much smaller confirmatory dataset than a model you’re training completely from scratch, because it isn’t starting from zero. Justify your sample size against that baseline, not against a generic rule of thumb.

Even with adequately labelled n, many clinically important outcomes (rare diseases, rare adverse events, rare complications) have severe class imbalance. Naming a concrete mitigation strategy, rather than gesturing at “conventional regularisation and class-imbalance handling,” scores better and is more credible. Stratifying by subtype, treatment line or vendor, often necessary to avoid confounding, can shrink a headline cohort of thousands to a few hundred, sometimes with only a handful of positive cases: that shrunk number is what the reviewer needs to see.

What this means

Even when your labeled dataset is technically large enough, many clinically important outcomes, like rare diseases, rare adverse events, rare complications, are heavily imbalanced: the vast majority of patients don’t have the outcome, and only a small minority do. This creates a specific, well-known trap: a model can score 99% “accuracy” by simply always predicting “no sepsis,” because 99% of patients genuinely don’t develop sepsis. That’s a technically true number that is clinically worthless, since the entire point of the tool is to catch the 1%.

Vague language like “we will apply conventional regularisation and class-imbalance handling” doesn’t tell a reviewer anything concrete. Two real mitigation strategies worth naming:

• Reweighting: telling the model, during training, to treat mistakes on the rare class as more costly than mistakes on the common class, so it doesn’t get to shrug off the rare cases.
• Resampling: either showing the model more copies of the rare cases (oversampling) or fewer copies of the common cases (undersampling) so the training process pays proportionally more attention to the rare outcome.

If you need to stratify your data, then split it by subtype, treatment line, or equipment vendor to avoid the model learning the wrong pattern. That stratification can shrink an already-modest cohort down further, sometimes to a genuinely small handful of positive cases in each stratum. Reviewers need to see that shrunk number, not just the headline cohort size.

“We will develop an accurate model” is not a criterion; “AUC ≥ 0.85” or “Dice ≥ 0.9 on held-out segmentation, matched against inter-rater variability” is. For anything safety-relevant, state a pre-specified quantitative bar and what happens if the method fails to clear it, set for a real clinical or operational reason and not just to have a number: an unjustified bar invites tuning the analysis until it clears (p-hacking by another name), so justify it and pre-register the threshold where the design allows.

What this means

“We will develop an accurate model” isn’t a criterion, it’s a hope. A criterion is a specific number, set before you see your results, that defines success or failure. Two metrics come up often enough to be worth explaining:

• AUC (area under the ROC curve) ranges from 0.5 to 1.0. A rough, genuinely useful way to think about it: AUC is approximately the probability that, if you picked one random patient who had the outcome and one random patient who didn’t, the model would correctly rank the one who had the outcome as higher-risk. 0.5 means the model is no better than a coin flip; 1.0 means perfect separation.
• Dice coefficient is used mainly for image segmentation tasks (like outlining a tumor on a scan). It measures how much the model’s drawn boundary overlaps with the true, expert-drawn boundary. Imagine laying the model’s outline and the radiologist’s outline on top of each other and asking what fraction of the combined area is shared between them. 0 means no overlap at all; 1 means a perfect match.

The threshold also needs to be set for a real clinical or operational reason, not chosen arbitrarily just to have a number on the page, and it needs to be set before you run the analysis, not adjusted afterward to match whatever result you happened to get. Picking your target after seeing your results is a close cousin of p-hacking: it’s the equivalent of a runner moving the finish line to wherever they happened to stop. Where your study design allows it, pre-registering the threshold (committing to it publicly and in writing before analysis) is the strongest way to show a reviewer you mean it10

If your preliminary evidence comes from a foreign public dataset, or your method will run on a specific language, population or equipment vendor, say so plainly and describe your plan for demonstrating transfer to the population you will deploy in. Do not let the reviewer discover the substitution themselves.

What this means

A model that works well in the exact conditions it was trained on can fail badly outside them, and reviewers specifically look for whether you’ve thought about this. A useful analogy: a self-driving car trained entirely on sunny California roads may perform beautifully there and still fail badly the first time it encounters a snowy Swedish winter, with different lighting, different road markings, different weather entirely. The same thing happens with clinical AI: a model trained on English-language clinical notes may not transfer to Swedish notes; a model trained on images from one MRI scanner vendor can perform noticeably worse on a different vendor’s scanner, because of subtle differences in image characteristics that have nothing to do with the underlying biology (sometimes called “domain shift”)1

If your preliminary evidence comes from a public dataset collected somewhere else, or your method is built around a specific language, population, or equipment vendor, say so plainly, and describe concretely how you’ll test whether it transfers to the population you intend to deploy in. Don’t make the reviewer discover the substitution on their own.

Overclaiming is easy to spot and expensive: promising transformative clinical impact in the lay summary when the funded scope is a narrow, single-outcome pilot reads as a credibility problem, not ambition. Conversely, disclosing a limitation, such as a data issue caught and corrected, or a weaker-than-hoped preliminary signal, reads as rigour, not weakness. Do not bury known weak points; frame them with your mitigation.

What this means

Overclaiming is one of the easiest problems for a reviewer to spot, and one of the most damaging. If your lay summary promises transformative, hospital-wide clinical impact, but the funded scope is a narrow, single-outcome, single-site pilot, that mismatch reads as a credibility problem and not as ambition. It’s the equivalent of a funding pitch for a lane-departure warning system that talks entirely about “revolutionizing autonomous transportation.”

The flip side matters just as much: disclosing a real limitation (such as a data quality issue you caught and fixed, or a preliminary signal that’s weaker than you’d hoped) reads as rigor, not weakness. Reviewers who evaluate methodology for a living have seen enough applications to recognize honest limitation-reporting as a good sign, not a bad one. Don’t bury known weak points; name them alongside your mitigation plan.

Disclose any investigator’s equity or founder role in a company supplying data, IP or the commercialisation pathway, and any overlap in team, dataset or technology with another pending or funded grant, stating what marginal work this application buys. Silence does not avoid the question; it just means the reviewer raises it, less charitably.

What this means

If any investigator holds equity in, or a founder role at, a company that supplies data, IP, or the eventual commercialisation pathway for this work, that needs to be disclosed explicitly. Not because it necessarily disqualifies the application, but because reviewers need to weigh it, and silence reads far worse than disclosure. The same goes for overlap with another pending or funded grant: if this application shares a team, a dataset, or a technology with something already funded elsewhere, say so, and explain exactly what new, marginal work this specific application buys that the other one doesn’t already cover. Not disclosing it doesn’t make the question go away, it just means the reviewer raises it themselves, and less charitably than if you’d raised it first.