PharmTech AI Pulse Check: AI Watermark Vulnerability

When new research shows that AI watermarking can make models more vulnerable to adversarial prompts, quality and compliance teams face a practical question: how much should they rely on detecting AI-generated content at all? PharmTech AI Pulse Check panelists argue that detection is the wrong lens for GxP environments, and that attribution, review, and accountable sign-off already answer the question.

Richard Jaenisch, senior director of education, outreach, and digital experience at Open Biopharma, pushes back on the assumption that all AI-generated content is slop, while stressing that human review remains essential regardless of who or what drafts a document. He sees the watermarking vulnerability as a temporary hurdle developers will likely engineer around, but notes that watermarking has limited reach to begin with. "The digital watermark also only really works on slop," he says, adding that edited content, or content that mixes human and AI input, tends to slip past it. The more useful question, he argues, is whether content is valuable and what it is for. An autonomous agent that compiles a daily report, for example, produces content whose entire purpose is to be reviewed, and he cautions that low effort and low value are not the same thing. For deviations, investigations, and reports, Jaenisch argues organizations should already know the data provenance of their records. The real exposure is shadow AI, a risk the panel discusses earlier in Episode 2, in which employees copy proprietary information into external tools and paste the output back in, sometimes to appear more capable and sometimes to avoid stigma from peers who view AI use as a shortcut. The panel first examined how AI-generated content enters deviation reports and standard operating procedures in Fixing AI Slop at the Source.

Florin Muraru, an independent regulatory advisor specializing in EU and US regulatory strategy and AI governance, grounds his view in batch record practice. The industry has never validated a batch record by guessing who typed it, he explains; it validates by knowing who signed it, where it came from, and what the audit trail shows. The right question for a quality function, he argues, is whether content is attributable, reviewed, and traceable to its source, and that question has the same answer whether a person or a model drafted it. Keeping humans in the loop is already the mantra of any governance process, he notes. "Detection is useful for forensicing after something eventually has gone wrong," Muraru says, but from a GxP perspective, he does not expect detection to change much.

Gourav Pandey, R&D quality lead at Takeda, builds on Muraru's analogy. Whether a standard operating procedure, batch record, or process validation document comes from AI or a skilled technical writer does not matter, he argues, as long as it follows approved verification procedures and complies with ongoing operations. He also sees detection as a losing contest. "So the detector will always fall behind the new models, and that's going to be [a] race forever," Pandey says. Even if an audit reveals that a procedure was written by AI, he believes the FDA is unlikely to object, provided the right controls are in place and the company follows its approved workflow.

For manufacturing and quality leaders, the message is that detection tools will keep chasing new models, while attribution, audit trails, and accountable signatures already provide the control GxP requires.