The first question almost every team asks about its real-world evidence is whether it is good enough. Is the database clean enough, is it "regulatory-grade"? Fair question, and we spent a whole post answering it [1]. It is also the second question.
The first one is quieter, and it decides far more. What job does this evidence actually have to do?
Because a study built to carry effectiveness and a study built to add context are different builds, not the same build at two quality settings: the comparator, the provenance, the pre-specification and the statistical power all change with the job. And here is the number that should worry you. When Purpura and colleagues went through 88 real-world evidence studies the FDA actually leant on across a run of approvals, only 8 of them, about 9%, carried substantial or primary weight. Fifty-seven, roughly two-thirds, were supportive, and thirteen were judged inadequate [2].
Most teams are drifting toward that 9% case by default. They generate RWE opportunistically, then hope it can be promoted into a load-bearing role when the submission needs one. This post argues you have to fix the role first, primary or supportive, before you design the study, because a primary-role study and a supportive-role study answering the very same clinical question are architecturally different studies. The weight you need, not the data you happen to hold, sets the design.
Consider blinatumomab in relapsed or refractory B-cell acute lymphoblastic leukaemia: same molecule, same disease, same brutal patient population, and two studies built to do very different jobs.
Study '211 (NCT01466179) was single-arm: around 225 patients, primary endpoint complete remission or CR with partial haematological recovery (CR/CRh*) within two cycles, with historical and external data brought in to give the response rate a reference point [3]. TOWER (NCT02013167) was randomised 2:1 against standard-of-care chemotherapy, with 405 patients, an internal concurrent control, and overall survival as the primary endpoint [4].
Look at what moved. The clinical question never changed: does blinatumomab help these patients? What changed was the evidentiary job, and that single shift in the required weight reshaped everything downstream, from the comparator and the control to the endpoint and the enrolment. One study was built to demonstrate a response signal early, the other to prove a survival difference against a live comparator.
Let me be clear about what this exhibit does and does not show. It is not a story about RWE carrying pivotal weight. In '211 the external data was contextual, sitting alongside a single-arm trial, and was not itself the primary evidence of effectiveness. The point is narrower and more useful: when the weight you need goes up, the architecture changes underneath you, and you cannot reverse that arrow after the fact.
You decide which walls carry the roof before you pour the foundation, not after you have framed the whole house and are choosing, retrospectively, which partition was meant to be load-bearing. RWE is no different: decide what the evidence has to hold up before you build it.
This is not a binary someone invented for a slide; it sits in the statute.
Under section 505(d) of the FD&C Act (21 U.S.C. §355(d)), substantial evidence of effectiveness can rest on "adequate and well-controlled investigations" (note the plural) or, since the FDA Modernization Act of 1997 (FDAMA), on "one adequate and well-controlled clinical investigation and confirmatory evidence." Read those two clauses again: primary and supportive map almost exactly onto them. In the first, your evidence is one of the load-bearing investigations; in the second, it is the confirmatory evidence backing a single pivotal trial. Two different jobs, written into one sentence of law.
The FDA has a draft guidance operationalising each clause. "Demonstrating Substantial Evidence of Effectiveness" (draft December 2019, reissued as a revised draft on 24 June 2026, and still never finalised) speaks to the load-bearing case [5]. "Demonstrating Substantial Evidence of Effectiveness Based on One Adequate and Well-Controlled Clinical Investigation and Confirmatory Evidence" (draft, 19 September 2023) formalises the confirmatory, supportive role [6]. Both remain draft, and you should cite them as such.
Now the honest part: the agencies do not keep a crisp two-box binary, and you will tie yourself in knots pretending they do. CDER tends to say "substantial evidence"; CBER reaches for "primary evidence" for much the same idea. Worse for planners, Subramaniam and colleagues documented the FDA and EMA classifying the same kind of single-arm paediatric-extrapolation design differently, one treating it as pivotal, the other as supportive [7]. The line moves by centre, by agency, by context.
That is precisely why you have to decide the role deliberately. The regulator will not draw the line for you in advance, and if you do not commit to which side of it you are standing on, you will find out at review, which is the worst possible time.
Free download
The RWE Briefing Document Template
The section-by-section structure for the RWE part of a regulatory briefing, built around the questions reviewers actually ask.
Get the template →The same category of data can be a load-bearing wall in one submission and a partition in the next. What decides it is the role you assigned before you built.
Primary role: Prograf. When tacrolimus was approved for prophylaxis of organ rejection in lung transplant recipients (July 2021), a non-interventional study drawn from the SRTR transplant registry was reportedly the primary basis of effectiveness for the new indication [8]. When a registry has to carry effectiveness itself, everything about it, from provenance and comparator to pre-specification, has to sit at pivotal standard, and there is nowhere to hide.
Supportive role: Ibrance. Palbociclib was approved for men with HR-positive, HER2-negative metastatic breast cancer in April 2019. The pivotal evidence was the PALOMA-2 and PALOMA-3 randomised trials, run in women, while real-world data from EHRs and claims did the narrower job of bridging that evidence to male patients [9]. Because the RWE only had to answer a bounded extrapolation question, it was built to a bounded bar, fit-for-purpose for that job and no further.
Same raw material in both cases, observational data about real patients, but opposite roles and opposite builds. And when RWE genuinely has carried primary weight, it has been engineered for it from the start the retrospective natural-history cohorts used as external controls for products like Zolgensma, Defitelio and Crysvita were assembled to serve as the pivotal comparator, not repurposed into one late in the day [10].
Here is the failure mode I see most, and it deserves a name. Call it "role creep": evidence generated for context, then quietly asked to carry pivotal weight later, once the pivotal trial reads thinner than anyone hoped. The study was never controlled or powered for that job. Promoting it will not make it load-bearing. The label changes; the study still prays nobody checks what's underneath.
Role creep is one of two symmetrical and expensive mistakes.
The mechanism connecting the two is worth internalising. Arondekar and colleagues found that regulatory scrutiny of RWE scales inversely with the strength of the pivotal evidence: the weaker your trial, the heavier the role your RWE must carry, and the more rigour the agency will demand of it [11]. So the standard you owe your real-world evidence is set by its role, and its role is set by how strong the rest of your package honestly is. That said, there is no Tier-1 cost figure I can put on either mistake, and I am not going to invent one. But you do not need a number to see that spending pivotal money on a supportive study, or supportive money on evidence you need to be pivotal, are both ways to lose.
Many years ago I worked on an early-phase programme in a rare indication where we generated a rich seam of real-world data, mostly to characterise the natural history and sharpen the trial design, which was genuinely useful work. Then the clinical data came in softer than the team had banked on, and the conversation turned, almost overnight, to whether that real-world dataset could be leaned on to support effectiveness. It could not. The data itself was perfectly good; it had simply been assembled for a different job, with no pre-specified comparator, no analysis locked while we were blind to outcomes, and no power calculation aimed at an effectiveness claim. We had built a very good partition and then wished it were a wall, and that wish cost time we did not have.
Match the rigour to the role. Do not hang the roof on a partition, and do not reinforce a partition as if it were holding the roof. This is the same discipline as knowing your minimum viable evidence [12] and knowing where the true regulatory-grade bar sits for the load-bearing pieces [1].
The strongest objection to all of this is a fair one, so let me put it at full strength.
In early development, especially in rare disease, you often genuinely do not know whether you will end up with a clean randomised confirmatory trial or whether the RWE will have to carry pivotal weight. Data landscapes shift, recruitment stalls, and the agency's posture can move between the pre-IND and the End-of-Phase-II meeting. Telling a resource-strapped biotech to "fix the role first" can sound like a luxury it cannot afford, a demand for certainty that early development simply does not offer.
Here is the answer: you do not need certainty, you need a committed primary hypothesis about the role, one you revisit at every regulatory interaction and change deliberately when the facts change. Two things make that workable. First, the discipline of pre-specifying your analysis while still blinded to outcomes, which lets you commit to the harder role early without waiting for the fog to clear [13]. Second, the plain fact that the same single-arm design has been treated as pivotal by one regulator and supportive by another: the role is a commitment you make, not a property the data has [7]. Deciding late does not spare you the choice. What it forfeits are the design levers (the comparator, the power, the pre-specification) that make the harder role achievable at all.
And the asymmetry is stark. Over-specify for the heavier role and the worst case is a bit of wasted rigour. Under-specify, then find out at submission that you needed it after all, and the worst case is the submission itself. If the EMA is in your plans, the calculus is stricter still, because the randomised trial is its default and a single-arm or external-control design needs justifying [14], so get that read early [15].
This is the part to run against your current RWE plan this week. It is a sequence, not a taxonomy.
Role is a design input, not a label you reach for at submission when you finally see what the package needs.
So decide it in the room where you choose the comparator, not the room where you write the cover letter. The blinatumomab programme did not run two studies because someone doubled the budget for fun; it ran two because the job changed, and the architecture had to change with it. Make that call well at the start and your evidence will hold the weight you actually put on it. The alternative is discovering the gap at review, when there is nothing left to redesign.
That first call, primary or supportive, is exactly the sort of decision worth pressure-testing before you commit real spend to a study design. That pressure-testing is where Inovia Bio earns their keep.
Get the monthly digest
The 5 things evidence leads need to know each month: regulatory moves, RWE developments and what they mean in practice. No pitch, one email a month.