Real-world evidence needs real-world discipline: the FDA raises the bar

We all have recognised it: real-world data (RWD) are having a moment, some would say lasting for quite some time. They are abundant, cheap-ish, and, unlike trial datasets, they arrive with all the messiness that makes clinicians nod approvingly: this, at last, is medicine as it is practised. The trouble is that regulators do not approve sentiment. They approve evidence.

That is the spirit of the FDA’s newly issued guidance on using real-world evidence (RWE) to support regulatory decision-making for medical devices (issued on December 17th 2025). The document is not a love letter to innovation. It is a reminder that, for all the talk of “modernising evidence,” the FDA remains stubbornly old-fashioned about one thing: credibility.

The guidance also does something quietly important. It frames “fit-for-purpose” not as a vibe, but as a two-part argument of relevance and reliability, and more importantly, then tells sponsors to show their working. It expands prior guidance in drug development by providing more detail on what is permissible when establishing the feasibility of using RWD.  And it even warns readers not to mistake this for a lowering of the evidentiary bar. It “should not be construed” to alter existing standards; it merely describes when clinical evidence from RWD can meet them.

The first hurdle is relevance: can the data even see what you are claiming?

The FDA’s relevance expectations are, at heart, about basic vision: can the source observe the exposure, outcomes and confounders needed to answer the question?

Start with the exposure. For devices, the agency points to the device identifier (the DI portion of the UDI) as a preferred route; where that is not captured, it acknowledges that alternative identification methods may be necessary (other structured fields, or even clinical notes). A dataset that cannot reliably distinguish what was used from what was not is not “real-world”; it is simply unclear.

Then come outcomes and covariates. The guidance is explicit that sponsors should ensure the RWD contain outcomes of interest and covariates that may influence exposure or outcomes (signs, symptoms, treatments, diagnoses, labs, demographics, we all know the usual suspects). It also nods to an awkward truth: many sources were never built for research, and a registry’s “minimum dataset” can be insufficient for a particular study question, meaning additional fields may be needed, and changes to collection should be documented.

Finally, the FDA cares about time, not just calendar time, but clinical time. Relevance includes longitudinality and continuity of care: you need enough follow-up, and enough visibility across the patient journey, to capture exposures and outcomes as they occur. A database that sees only a fragment of care can deliver fragmentary truth.

Linkages, timeliness and generalisability: where “big data” often breaks

Relevance is also where ambitious studies tend to acquire their first scar tissue: linkage.

Sponsors often talk about “linking datasets” the way children talk about connecting Lego bricks. The FDA is less enchanted. It asks sponsors to assess whether and how data from different sources can be obtained and integrated, recognising heterogeneity in populations, practice patterns and coding, and to describe that assessment in the submission.

If linkage is performed, the agency wants it to be treated as a method, not a miracle: a predefined linkage methodology that is scientifically valid, privacy-protecting, interoperable, and able to cope with coding differences. It even specifies the kinds of due diligence it expects: checking that the same individuals are being matched, using predefined consistency rules, correcting redundant data, resolving inconsistencies, and assessing missingness introduced by linkage.

Timeliness is another trap for the unwary. Data may be plentiful, but medicine moves. The FDA notes that the time between data collection and release should be reasonable, and that the RWD should reflect the current clinical environment. If the data continue accruing during the study, sponsors should specify update timing in the protocol and define a reporting schedule, with a rationale.

And then there is generalisability. Once a question is defined, the FDA expects the study sample to be representative of the intended-use population; if quantitative assessment shows it is not, analyses should be conducted to evaluate generalisability. This is less philosophical than it sounds. It is a warning against “RWE” that is really evidence about a narrow, conveniently observed subset.

Reliability is the second hurdle: can you defend how the data were made and handled?

If relevance is about whether the data can answer the question, reliability is about whether anyone should trust the answer.

Here the FDA sounds almost like an auditor. In thinking about data quality, it recognises that RWD sources may not permit classic line-item source verification. That does not let sponsors off the hook. It changes the nature of the work: sponsors should consider the systems used to ensure sufficient data quality, and they should retain records showing adherence.

Then the guidance goes straight for the jugular: missingness. Data should be captured in a way that minimises it.  And, most importantly, sponsors should perform a quantitative assessment of potential bias associated with high missingness and carry it into interpretation. In other words, “we had missing data” is an insufficient explanation.

Robustness also depends on provenance, knowing where the numbers came from, and what happened to them along the way. It expects documentation of transformations into common data models, where applicable, and calls for a data audit trail including discrepancy assessment.

Feasibility is allowed. Fishing is not.

The FDA’s guidance guides in more detail what can be done during feasibility.  It recommends that sponsors finalise the protocol and analysis plan before reviewing outcome data and before performing prespecified analyses, and to state in the submission whether that happened. While the protocol is being developed, sponsors “should not have access to the outcome measure results.” Any protocol revisions should be dated, time-stamped and justified.

This draws a clean line between feasibility; i.e., confirming that key fields exist, that linkage is possible, that follow-up is adequate, that missingness is manageable, and outcome-peeking, where “feasibility” becomes a synonym for early results.

Why drug developers should read this too; and why no one should read it alone

Formally, the guidance is about devices. It even states that it does not apply to drugs and biologics, and points readers instead to the FDA’s RWE programme for those products. Still, drug developers would be unwise to treat this as someone else’s paperwork.

Why? Because the agency’s logic is portable. The guidance anchors RWE best practices in the same principles as traditional clinical studies, good clinical practice, and emphasises that RWE will be judged as part of the totality of evidence, under standards of valid scientific evidence. The discipline it demands; i.e., pre-specification, access controls, audit trails, quantified uncertainty, is exactly what makes RWD persuasive in any product area.

The real message, then, is not that RWD are welcome. Everyone already knows that. The message is that RWD are welcome only if they are governable, and that “governable” means the boring things: definitions, documentation, provenance, and restraint.

In the real world, there is no free lunch. The FDA has finally printed the full menu.