Calculating the Reportable Result from Retest Data - Pharmaceutical Technology

Latest Issue

Latest Issue
PharmTech Europe

Calculating the Reportable Result from Retest Data
Two methods to evaluate retest data following out-of-specification results.

Pharmaceutical Technology
Volume 37, Issue 2

Chris Burgess, PhD
The importance of United States vs. Barr Laboratories (1) in the handling of out-of-specification (OOS) results in the pharmaceutical laboratory cannot be over estimated. It took 13 years after the judgement for FDA to issue its final guidance (2) on the matter in 2006. The European Compliance Academy (ECA) Analytical Quality Control Working Group (3) recently launched version 2 of its standard operating procedure (SOP) on OOS investigations (4), which harmonizes the FDA and European regulatory approaches.

In an earlier column, the question of retest sample size selection was discussed and proposals (5) were made. The article, however, did not address the vexed question of how to calculate the reportable result. Retesting is only performed if a root cause for the OOS cannot be established. In such cases, the FDA requirement is specific:

If no laboratory or calculation errors are identified in the first test, there is no scientific basis for invalidating initial OOS results in favor of passing retest results. All test results, both passing and suspect, should be reported and considered in batch release decisions (2).

The key word is considered. How can we take into account all test results in a rational and scientific manner as required by 21 CFR 211.160 (b)? There are two well-established ways of taking an OOS result into consideration in arriving at a reportable result.

Standard confidence interval approach

Firstly, we will look at isolating the OOS result using a standard confidence interval approach. Take as an example that the registered specification for an assay is 95.0 to 105.0% of the claim and that an OOS result of 94.7% was obtained in the laboratory. An investigation failed to identify a root cause. The quality unit authorized a retesting protocol for six retests and the % results obtained were 98.0, 97.0, 96.1, 96.5, 97.4, and 96.2. In this example, the decision was made that six passing retest results out of seven would lead to the isolation of the OOS result rather than the Barr case example of seven of eight.

The question that remains, however, is how to demonstrate that the OOS result has been successfully isolated and how to calculate the reportable result. One thing we cannot do to arrive at a reportable result is take the average of all the results because:

In the context of additional testing performed during an OOS investigation, averaging the result(s) of the original test that prompted the investigation and additional retest or resample results obtained during the OOS investigation is not appropriate because it hides variability among the individual results (2).

There is also a good statistical reason for not calculating this average, which is because the arithmetic mean (the average) is sensitive to outlying results and an OOS's inclusion will cause the reportable result to be biased. A better approach would be to use a statistical method to calculate the robust mean and robust standard deviation.

The idea behind the isolation method using a confidence interval is that if the confidence interval of the arithmetic mean calculated at 95% confidence from all the results lies above the lower specification limit (LSL), excluding it from the reportable result calculation is justified although the OOS result will be recorded in the laboratory record as required by regulation.

The lower 95% confidence limit (CL) of the mean is calculated for all results (n=7) using the conventional equation:

where is the mean of all 7 values, t (0.05,n–1) is the value of the t distribution for 6 degrees of freedom at 95% confidence and s is the calculated sample standard deviation.

Hence for our example;

As this value is greater than the LSL of 95.0%, the isolation of the initial OOS result is achieved.

Figure 1: Illustration of a successful out-of-specification (OOS) "isolation" using the confidence interval approach. LSL is lower specification limit, USL is upper specification limit.
The reportable value is the mean value of six retest results. This situation is shown in Figure 1.

Figure 2: Illustration of out-of-specification (OOS) "isolation" failure using the confidence interval approach. LSL is lower specification limit, USL is upper specification limit.
Note in Figure 1 and, in particular, Figure 2, the effect of bias on the mean caused by inclusion of the original OOS result and the broadening of the normal distribution. There is a statistical disadvantage to this method because the distance of the OOS result from the retest values causes a point at which the "isolation" fails. For example, if the OOS result was 92.7% rather than 94.7, then repeating the calculation would yield a lower 95% confidence limit of 94.9%. In this instance, the "isolation" would fail (see Figure 2).


blog comments powered by Disqus
LCGC E-mail Newsletters

Subscribe: Click to learn more about the newsletter
| Weekly
| Monthly
| Weekly

What role should the US government play in the current Ebola outbreak?
Finance development of drugs to treat/prevent disease.
Oversee medical treatment of patients in the US.
Provide treatment for patients globally.
All of the above.
No government involvement in patient treatment or drug development.
Finance development of drugs to treat/prevent disease.
Oversee medical treatment of patients in the US.
Provide treatment for patients globally.
All of the above.
No government involvement in patient treatment or drug development.
Jim Miller Outsourcing Outlook Jim MillerCMO Industry Thins Out
Cynthia Challener, PhD Ingredients Insider Cynthia ChallenerFluorination Remains Key Challenge in API Synthesis
Marilyn E. Morris Guest EditorialMarilyn E. MorrisBolstering Graduate Education and Research Programs
Jill Wechsler Regulatory Watch Jill Wechsler Biopharma Manufacturers Respond to Ebola Crisis
Sean Milmo European Regulatory WatchSean MilmoHarmonizing Marketing Approval of Generic Drugs in Europe
Seven Steps to Solving Tabletting and Tooling ProblemsStep 1: Clean
Legislators Urge Added Incentives for Ebola Drug Development
FDA Reorganization to Promote Drug Quality
FDA Readies Quality Metrics Measures
New FDA Team to Spur Modern Drug Manufacturing
Source: Pharmaceutical Technology,
Click here