Chris Burgess, PhD
|
The importance of United States vs. Barr Laboratories (1) in the handling of out-of-specification (OOS) results in the pharmaceutical laboratory cannot be over estimated. It took
13 years after the judgement for FDA to issue its final guidance (2) on the matter in 2006. The European Compliance Academy
(ECA) Analytical Quality Control Working Group (3) recently launched version 2 of its standard operating procedure (SOP) on
OOS investigations (4), which harmonizes the FDA and European regulatory approaches.
In an earlier column, the question of retest sample size selection was discussed and proposals (5) were made. The article,
however, did not address the vexed question of how to calculate the reportable result. Retesting is only performed if a root
cause for the OOS cannot be established. In such cases, the FDA requirement is specific:
If no laboratory or calculation errors are identified in the first test, there is no scientific basis for invalidating initial
OOS results in favor of passing retest results. All test results, both passing and suspect, should be reported and
considered
in batch release decisions (2).
The key word is considered. How can we take into account all test results in a rational and scientific manner as required by 21 CFR §211.160 (b)? There are two well-established ways of taking an OOS result into consideration in arriving at a reportable
result.
Standard confidence interval approach
Firstly, we will look at isolating the OOS result using a standard confidence interval approach. Take as an example that the
registered specification for an assay is 95.0 to 105.0% of the claim and that an OOS result of 94.7% was obtained in the laboratory.
An investigation failed to identify a root cause. The quality unit authorized a retesting protocol for six retests and the
% results obtained were 98.0, 97.0, 96.1, 96.5, 97.4, and 96.2. In this example, the decision was made that six passing retest
results out of seven would lead to the isolation of the OOS result rather than the Barr case example of seven of eight.
The question that remains, however, is how to demonstrate that the OOS result has been successfully isolated and how to calculate
the reportable result. One thing we cannot do to arrive at a reportable result is take the average of all the results because:
In the context of additional testing performed during an OOS investigation, averaging the result(s) of the original test that
prompted the investigation and additional retest or resample results obtained during the OOS investigation is not appropriate
because it hides variability among the individual results (2).
There is also a good statistical reason for not calculating this average, which is because the arithmetic mean (the average)
is sensitive to outlying results and an OOS's inclusion will cause the reportable result to be biased. A better approach would
be to use a statistical method to calculate the robust mean and robust standard deviation.
The idea behind the isolation method using a confidence interval is that if the confidence interval of the arithmetic mean
calculated at 95% confidence from all the results lies above the lower specification limit (LSL), excluding it from the reportable
result calculation is justified although the OOS result will be recorded in the laboratory record as required by regulation.
The lower 95% confidence limit (CL) of the mean is calculated for all results (n=7) using the conventional equation:
where is the mean of all 7 values, t
(0.05,n–1)
is the value of the t distribution for 6 degrees of freedom at 95% confidence and s is the calculated sample standard deviation.
Hence for our example;
As this value is greater than the LSL of 95.0%, the isolation of the initial OOS result is achieved.
Figure 1: Illustration of a successful out-of-specification (OOS) "isolation" using the confidence interval approach. LSL
is lower specification limit, USL is upper specification limit.
|
The reportable value is the mean value of six retest results. This situation is shown in Figure 1.
Figure 2: Illustration of out-of-specification (OOS) "isolation" failure using the confidence interval approach. LSL is lower
specification limit, USL is upper specification limit.
|
Note in Figure 1 and, in particular, Figure 2, the effect of bias on the mean caused by inclusion of the original OOS result and the broadening of the normal distribution.
There is a statistical disadvantage to this method because the distance of the OOS result from the retest values causes a
point at which the "isolation" fails. For example, if the OOS result was 92.7% rather than 94.7, then repeating the calculation
would yield a lower 95% confidence limit of 94.9%. In this instance, the "isolation" would fail (see Figure 2).
