Qualify the Reviewer Before the AI Goes Live

The question now facing pharma is whether the human in the loop still exercises independent judgment. On October 27, Anthropic published a preregistered randomized trial that tested exactly that. Professional analysts went with an AI agent’s secretly assigned lean on 86% of analytical judgment calls, and more than half rated the final recommendation as more their own than the agent’s.¹ The agent’s lean settled most of the calls. The analysts signed for them.

Every AI deployment plan in this industry rests on one sentence: a qualified human reviews the output. That sentence carries the validation strategy, the risk assessment, and the answer an inspector will hear. It assumes the person reviewing can find what the system got wrong, including what it left out. Very few organizations have tested that assumption. They train people to use the tool, record the training, and treat the signature that follows as evidence of control.

A signature from a reviewer who was never qualified to catch the error is a record of approval, and nothing more.

I made this argument in this column in August,² when I wrote that pharma’s AI return-on-investment problem is a qualification problem, and I built a method around it in Governing AI in Pharmaceutical Operations.³ Since then, the same conclusion has come from conference stages, quality journals, executive interviews, and the AI developers themselves. The industry now agrees that human review is the control. What remains to be settled is whether we will qualify the people who perform it with the same rigor we apply to every other control that protects a patient.

The answer is a rule this industry already knows how to apply: qualification before operational use. Every operator demonstrates proficiency before touching patient product. Every person who reviews or approves AI-supported quality work should do the same, against a higher bar, because an AI error spreads further than any single deviation.

What the Field Is Now Saying

The enforcement record came first. In April 2026, FDA cited a manufacturer that “used AI to create drug product specifications, procedures, and master production or control records” and failed to review them for accuracy and cGMP compliance, a violation of 21 CFR 211.22(c).^4,5 David Hubmayr of Takeda drew the lesson in the PDA Letter: “no one had enough cGMP fluency to recognize the errors.”⁶ Reviewers scrutinize polished output less, he wrote, “because the surface signals of error are absent.”

The quality literature followed. A PDA Letter article put the point in its title, “AI Upskilling Is Now a GMP Capability,” and argued that “AI literacy must include critical review skills, not only tool usage” and that upskilling “acts as a control mechanism.”⁷ Frank Henrichmann and Oliver Herrmann, writing for the International Society for Pharmaceutical Engineering (ISPE), concluded that “human oversight under real-world working conditions is not inherently reliable,” and that automation bias and review fatigue “can reduce HITL [human in the loop] to a formal gesture.”⁸ Their definition of the control itself names “qualified personnel.”

Practitioners at this year’s conferences put numbers on it. Reporting from the 2026 ISPE Annual Meeting, Janeen Skutnik-Wilkinson relayed a point from a CIMCON session: “If an AI tool is 80% right and you don’t know which 20% is wrong, you haven’t saved 80% of your review time.”⁹ The same session described an AI check of a SOP against a regulation that covered 14 of 17 requirements and never flagged the other 3. A reviewer working only from the tool’s output would have approved the gap. From the PDA/FDA Joint Regulatory Conference, she reported an FDA speaker’s caution against using AI for anything a user is not qualified to check independently.¹⁰

McKinsey’s cost analyses add a financial reason. One puts ~60% of an agentic task’s cost into refining answers, a figure drawn from software engineering. The analysis attributed that share to the fact that “the expensive part is not the first answer generated but the checking, repairing, and reverifying that follows.”¹¹ In a banking customer-service example, human oversight made up 70% to 75% of an agent’s variable cost.¹² If review is most of the cost, the first review has to be right; an error that clears review and enters a record costs more to find and fix later.

The Anthropic trial is the strongest evidence of all, and it comes from an AI developer. Its researchers gave 50 professional causal-inference practitioners an AI coding agent instructed to hold and voice a view. On each analytical decision, a coin flip the participant never saw set which of 2 defensible options the agent favored.¹ The agent made the best honest case for its side and did not mislead. That is where the 86% comes from; chance alone would have produced 50%. When asked to justify their choices, 77% repeated the agent’s talking points, sometimes nearly verbatim, and only 15% mentioned the agent at all.

One finding in that study should change how this industry thinks about training. In exploratory analyses, the authors found no evidence that years of experience, self-rated expertise, education, or a skills-quiz score changed how often participants followed the agent’s lean. Knowledge alone did not appear to protect anyone. The authors’ explanation is precise: the agents “rarely explained why a competent expert could pick the other option,” and on 77% of decisions the agent voiced its view before the participant had taken a side. Their conclusion is one every quality leader should read twice: “Institutions often treat an expert’s sign-off as evidence of independent judgment, and as AI does more of the work, that assumption becomes shakier.”¹

Operations leaders outside pharma have watched this pattern before. Tyson Heaton, who leads the Lean Enterprise Institute’s LeanTech/AI initiative, compares today’s AI rollouts to the business process reengineering wave of the 1990s. Of that era, he wrote, “Studies established that 70% or more of reengineering efforts failed or made things worse.” His diagnosis is that reengineering “treated people as cost to be eliminated rather than capability to be developed.”¹³ In pharmaceutical quality operations, that principle carries a regulatory consequence. When an AI-assisted investigation enters the quality record, the reviewer must be capable of independently evaluating the work they approve.

Executives see the skills gap mostly as an adoption problem. Jim Swanson, chief information officer, Johnson & Johnson, described building “a curriculum and a mindset around upskilling,”¹⁴ and Parexel’s Tala Fakhouri, a former FDA official, criticized rollouts in which “staff are supposed to figure it out.”^15,16 Few of the executive statements I found connect training to the reliability of human review; that connection is the one that matters.

Why An AI Error Is Larger Than a Deviation

A process deviation has a boundary. It happened on a batch, a line, a piece of equipment, or within a window of time. The investigation starts from that boundary and works outward, and 21 CFR 211.192 requires it to extend to other batches and other drug products that may have been associated with the failure.¹⁷ The system is built to find the edge of the damage.

An error introduced through a generative AI system has no natural edge. A missed requirement in an AI-drafted SOP is present in every batch run under that procedure. A flawed impact-assessment template repeats in every deviation it touches. A retrieval gap that drops a contradicting record from one investigation drops it from every investigation that asks a similar question. The error recurs every time the tool is used, across sites and products, until someone notices it.

Take a hypothetical site that deploys an assistant to draft deviation investigations. On ambiguous events, the model tends to favor procedural nonadherence as the most probable root cause, a defensible call in any single case. Investigators accept it because it is plausible and well argued, and retraining becomes the corrective and preventive action (CAPA). During a year, 40 investigations at 3 sites close on the same conclusion, and each one passes review. The trend report now shows a training problem; the equipment and process causes a different reviewer might have pursued never enter the record. No single document contains an error an auditor could point to. The pattern is the error.

The Anthropic trial shows a second way the scope grows. Run with no assigned lean, the same agent made the same 5 analytical choices in 91% of 123 runs.¹ When many reviewers work with one model, their conclusions converge on that model’s preferences. Agreement between 2 investigators normally counts as independent confirmation. When both deferred to the same system, it confirms nothing.

The error is also harder to see. A deviation announces itself through an out-of-specification result, an alarm, or a failed check. A generative system produces fluent, well-structured output whether its reasoning is sound or not, so the defect looks exactly like correct work.

Many of the decisions AI now supports have no single right answer: the most probable root cause, the severity rating, the scope of a CAPA, the conclusion of an impact assessment. On calls like these, an AI’s lean can settle the outcome without producing any error a reviewer could catch. The record shows a qualified person’s signature. The judgment belonged to the model.

The Anthropic authors could measure deferral only because they randomized the agent’s lean, which no real deployment can do.¹ A reviewer reading the finished investigation cannot see it either, because the output looks sound whichever option was chosen. Review of the output therefore cannot be the only control on these calls. The record has to show, at the moment each judgment call was made, which option the system proposed, which alternative was considered, and who decided, before anyone approves the result. That capture must come from the system’s own log, because people’s accounts of where a judgment came from are what the study found unreliable. Captured this way, the hypothetical above becomes visible: 40 investigations in which the assistant proposed the root cause and the investigator accepted it form a trend a quality unit can see and act on.

A process deviation can compromise a batch. An unqualified AI user can compromise the quality system that decides whether batches are fit for release.

The Standard We Already Apply to Operators

Consider what it takes for a new operator to touch patient material in a chimeric antigen receptor T-cell (CAR-T) suite. They read the procedures and sign the training record. Then they demonstrate the work: gowning qualification, supervised practice on the aseptic manipulations, observed performance against defined criteria, and participation in a successful aseptic process simulation before they work unsupervised in Grade A or B areas, as EU GMP Annex 1 requires.¹⁸ Their qualification is tied to specific processes and equipment, and it lapses or is repeated when the process changes or performance slips. In autologous manufacturing, one batch is one patient’s therapy. Nobody argues that reading the SOP is enough.

Monoclonal antibody manufacturing applies the same logic at larger scale. Operators on a chromatography skid or a filling line are qualified on that operation before they run it, because one batch may supply thousands of patients. The core requirement is short. Under 21 CFR 211.25, personnel must have the education, training, and experience to perform their assigned functions.¹⁹ No quality unit would accept a sign-in sheet as proof of that.

Now consider what it takes, in most organizations, to use a generative AI system on a deviation investigation, an SOP revision, or a batch record review. Usually it is an e-learning module on responsible use, an acceptable-use policy, and access to the tool. Sometimes it is less. The person who will decide whether an AI-drafted root cause is sound has demonstrated nothing about their ability to recognize a fluent omission, an unsupported causal claim, a source from the wrong product, or an agent quietly settling a judgment call for them.

That person is performing an assigned function within the meaning of 21 CFR 211.25. The function is reviewing and approving AI-supported quality work. We have simply never defined the proficiency it requires.

Qualification Before Operational Use

Training alone has a weak record against this problem. In a field experiment with 758 Boston Consulting Group consultants, those given instructional material on using an AI tool did worse on a task the AI got wrong than colleagues who used the tool without that material. The authors reported the gap as significant at the 10% level.²⁰ A widely cited review of automation research concludes that automation bias “cannot be prevented by training or instructions.”²¹ Both findings concern instruction and general training. Neither tested qualification on the work itself: seeded defects, independent positions recorded before the AI’s, and pass criteria set in advance. That is the gap between a training record and a qualification record.

The fix is to treat AI-supported quality work as an operation that requires qualification before anyone performs it, and to set the bar higher than we set it for a single manufacturing step, because an AI error reaches further. In practice that means 6 things.

1. Qualify the role on a specific system. Qualification attaches to a person, a configured system, a version, and a defined task. A reviewer qualified to check AI-drafted deviation chronologies from one repository is not automatically qualified when the system starts searching new sources or drafting CAPA plans.
2. Teach the failure modes, then the tool. Users need to know how these systems fail before they learn the buttons: fluent output that omits a record, confident language with no source behind it, a true statement about the wrong product, and a one-sided case for a defensible option. They also need the reason behind each requirement they are checking. A reviewer who knows why a requirement exists can see when an AI output satisfies its wording and misses its purpose.
3. Demonstrate proficiency on representative work. Assessment uses real-format cases with seeded defects: missing records, unsupported causation, wrong-site sources, and correct outputs, so that blanket rejection fails as surely as blanket acceptance. Pass criteria are set in advance, and some failures, such as a missed critical omission, cannot be offset by success on easier items.
4. Train the judgment calls explicitly. For decisions with no single right answer, qualified reviewers record their own position before seeing the AI’s and state the strongest case for the option the AI did not choose. The Anthropic results show why this must be practiced and assessed directly: knowledge alone did not appear to prevent deferral.
5. Build the process to require the behavior. Training decays when the workflow rewards speed. The design Ferring and Kuatro Group described at ISPE shows the pattern: investigators answer SOP-based questions in their own words, quality assurance reviews the record without the tool, and acceptance faster than a defined minimum review time is flagged.⁹ The system should also log, for every judgment call, who proposed the option chosen and which alternative was recorded before approval, so that deferral can be trended across sites and model versions.
6. Requalify on change and on evidence. A model update, a new retrieval source, or a grant of new authority changes the job, and the qualification has to follow. So does a missed error found later in a record. As with an operator, the response to a failure is an investigation of the person, the system, and the conditions, with retraining only where a skill was actually missing.

Nothing here is new to this industry. It is the qualification discipline this industry already applies to every person who touches patient product, extended to the systems that now shape the records proving that product is fit for use. The training record documents that instruction took place. The qualification record documents that the person can do the work. When an inspector asks how the organization knows its reviewers can catch an AI error, only the second record answers the question.

The Signature Has to Mean Something

Generative AI is already doing real work in quality operations, and much of it is good work. In the Anthropic trial, 6% of analysts who used the agent made a serious technical error, against 55% of a smaller group who worked without it, though that comparison was not randomized.¹ The value is real. So is the risk, and the risk sits in one place: the moment a person signs for output they were never qualified to judge.

We settled this question for manufacturing decades ago. No one touches the product until they have shown they can do the work, and they show it again when the work changes. The same rule should apply to everyone who reviews what generative systems now draft: investigations, procedures, and release records.

Qualify the reviewer before the AI goes live.

References

Herdağdelen A, Joren H, Tamkin A. Who's aligned to whom? How AI agents shape experts' research judgment calls. Anthropic. October 27, 2026. Accessed [TK]. https://www-cdn.anthropic.com/519c17a5d23a794e8b6d50a854048d0b61e0d73a.pdf

2. Drapeau B. Pharma's AI ROI problem isn't a tech problem but a qualification problem. Pharmaceutical Technology. August 10, 2026. Accessed [TK]. https://www.pharmtech.com/view/pharmas-ai-roi-problem-isnt-tech-problem-but-qualification-problem

3. Drapeau B. Governing AI in Pharmaceutical Operations. Version 4.2. GxP Frame; 2026. Accessed [TK]. https://www.gxpframe.com/governing-ai/

4. FDA. Warning letter 320-26-58: Purolea Cosmetics Lab (MARCS-CMS 722591). FDA.gov. Published April 2, 2026. Accessed [TK]. https://www.fda.gov/inspections-compliance-enforcement-and-criminal-investigations/warning-letters/purolea-cosmetics-lab-722591-04022026

5. Responsibilities of quality control unit. 21 CFR §211.22 (2026). Accessed [TK]. https://www.ecfr.gov/current/title-21/chapter-I/subchapter-C/part-211/subpart-B/section-211.22

6. Hubmayr D. When the AI never told you. PDA Letter. Published [TK]. Accessed [TK]. https://www.pda.org/pda-letter-portal/home/full-article/when-the-ai-never-told-you

7. Parenteral Drug Association. AI upskilling is now a GMP capability: why drug makers must train employees before scaling AI. PDA Letter. Published [TK]. Accessed [TK]. https://www.pda.org/pda-letter-portal/home/full-article/ai-upskilling-is-now-a-gmp-capability

8. Henrichmann F, Herrmann O. Human-in-the-loop as an illusion of control? iSpeak. International Society for Pharmaceutical Engineering. September 4, 2026. Accessed [TK]. https://ispe.org/pharmaceutical-engineering/ispeak/human-loop-illusion-control

9. Skutnik-Wilkinson J. Human-in-the-loop review came up in almost every AI session I attended at ISPE 2026. LinkedIn. Posted October 28, 2026. Accessed [TK]. https://www.linkedin.com/posts/jskutnik_aiingxp-pharmaceuticalquality-gmp-share-7521136217422323712-4Xra/

10. Skutnik-Wilkinson J. What builds judgment when AI does the easy work? LinkedIn. Posted October 30, 2026. Accessed [TK]. https://www.linkedin.com/posts/jskutnik_pharmaceuticalquality-regulatoryintelligence-share-7521138010915160065--46-/

11. Hämäläinen L, Patel M, Blumberg S, Catlin T, Lala W. Is that AI agent worth it? Agentic economics and the modern operating model. McKinsey Quarterly.July 13, 2026. Accessed [TK]. https://www.mckinsey.com/capabilities/quantumblack/our-insights/is-that-ai-agent-worth-it-agentic-economics-and-the-modern-operating-model

12. Asif C, Kiewell D, Olanrewaju T, Hämäläinen L, Kolaja T. Where AI agents pay off: a practical guide to the economics of agentic workflows. McKinsey & Company. August 24, 2026. Accessed [TK]. https://www.mckinsey.com/capabilities/quantumblack/our-insights/where-ai-agents-pay-off-a-practical-guide-to-the-economics-of-agentic-workflows

13. Heaton T. The reengineering is coming: leadership negligence, not technology failure, sets the stage for AI's most expensive correction. Lean Tech & AI Journal. Lean Enterprise Institute. May 13, 2026. Accessed [TK]. https://tech.lean.org/journal/the-reengineering-is-coming

14. Kell J. How pharmaceutical companies are training their workers on AI. Business Insider. March 10, 2025. Accessed [TK]. https://finance.yahoo.com/news/pharmaceutical-companies-training-workers-ai-193045480.html

15. Parrish M. BIO 2026: where pharma leaders say AI hasn't delivered. PharmaVoice. June 26, 2026. Accessed [TK]. https://www.pharmavoice.com/news/pharma-leaders-ai-drug-discovery-development-tech/823811/

16. Trang B. A former AI regulator, now in industry, says biopharma is reading FDA's guidance wrong. STAT. July 2, 2026. Accessed [TK]. https://www.statnews.com/2026/07/02/fda-ai-guidance-pharma-industry-caution-tala-fakhouri-explains/

17. Production record review. 21 CFR §211.192 (2026). Accessed [TK]. https://www.ecfr.gov/current/title-21/chapter-I/subchapter-C/part-211/subpart-J/section-211.192

18. European Commission. EudraLex volume 4, Annex 1: manufacture of sterile medicinal products. August 25, 2022. Accessed [TK]. https://health.ec.europa.eu/system/files/2022-08/20220825_gmp-an1_en_0.pdf

19. Personnel qualifications. 21 CFR §211.25 (2026). Accessed [TK]. https://www.ecfr.gov/current/title-21/chapter-I/subchapter-C/part-211/subpart-B/section-211.25

20. Dell'Acqua F, McFowland E III, Mollick E, et al. Navigating the jagged technological frontier: field experimental evidence of the effects of artificial intelligence on knowledge worker productivity and quality. Organ Sci. 2026;37(2):403-423. doi:10.1287/orsc.2025.21838

21. Parasuraman R, Manzey DH. Complacency and bias in human use of automation: an attentional integration. Hum Factors. 2010;52(3):381-410. doi:10.1177/0018720810376055