Implications for the pharmaceutical industry
The Supreme Court's discussions of "bright-line" decision rules and the "total mix"of information applies to the use of statistical
analysis of quality assurance and manufacturing data. Without too much difficulty, the Supreme Court's position could apply
equally to stability studies, assay validation, process validation, design space, and process capability.
Anyone who has hosted an FDA inspection knows that an investigator need not wait for a control chart to exceed one or more
of the classic "out-of-control" signals before beginning a review of suspicious current and historical data.
Sampling-plan acceptance should not be used as a defense for accepting some lots when it is clear from all the available information
that the defect rate is higher than desired for the process as a whole. Waiting for a statistically significant event before
evaluating a trend simply pushes the problem down the road; such an approach is also inconsistent with the concept of quality
Manufacturers should be looking at the total mix of information throughout process design, process qualification, and process
verification. FDA's guidance for industry on Process Validation: General Principles and Practices, issued in January 2011, makes clear that understanding the manufacturing process and potential variations—that is, the total
mix of information—is essential for product quality. Interestingly, the FDA process-validation guidance makes no mention of
statistical significance. The document does refer to statistical confidence several times, but only in a general sense. It
does not suggest or require a specific confidence level.
As stated above, statistical significance is only the determination that the difference we observe is larger than we would
expect by random chance alone given the available estimate of variability. It is not necessarily truth, fact, or causation.
It is only one tool for understanding all of the available information. FDA expects pharmaceutical manufacturers to use science-based
reasoning to determine which tools are appropriate for evaluating any given issue.
In the laboratory.
An out-of-specification (OOS) determination is the result of comparing a single reportable value collected at a specific point
in time with a specification. Statistical significance has no role in this determination, nor does practical significance.
A reportable result is either within specification or OOS. Thus, the Matrixx case has little or no implication for OOS determinations.
A determination of out-of-trend (OOT) is the result of comparing one or more reportable results collected over a time period
with a statistically defined model or to statistically summarize those historical data. If a trend is large enough to be of
practical importance to an experienced analyst, then data should be reviewed even if they do not rise to the level of statistical
significance. If the data are statistically significant, then chance is ruled out and a cause-and-effect relationship is supported
but not proven. This position is supported by Matrixx, in that a lack of statistical significance does not justify failing to investigate a change in the data that could be of
practical importance. Statistical significance cannot be used, to quote the Supreme Court, as a single "bright-line" decision
rule to the exclusion of other sources of supporting information. The evaluation of OOT data must be an evaluation of the
"total mix" of information.
Product liability lawsuits.
Given that the underlying facts in Matrixx relate to adverse drug events, it is reasonable to ask whether the Supreme Court's decision has any bearing on product liability
lawsuits. Although the case does not appear to change the Supreme Court's view of the standard for causation (i.e., that the
product "more likely than not" caused the injury, and that statistical significance is not necessary to prove such a claim),
it does seem to reinforce the Supreme Court's holding in an earlier decision, Wyeth v. Levine, regarding a drug manufacturer's responsibility to act upon new information about adverse events (24).