How much confidence do I have that a batch will meet specification acceptance criteria?
A design space is determined based on the knowledge gained throughout the development process. The goal of defining a design
space is to demonstrate a region for the relevant parameters such that all specification acceptance limits are met. The size
of the design space is represented by a region that is defined by parameter boundaries. Those parameter boundaries are determined
by results of multifactor experiments that demonstrate where a process can operate to generate an acceptable quality drug
product. Parameter boundaries of a design space do not necessarily represent the edge of failure (i.e., failure to meet specification
limits). Frequently, those boundaries simply reflect the region that has been systematically evaluated.
Different approaches may be used define the size of the design space. The approaches are based on the type of statistical
design that was used; the accuracy of the model used to define the operating ranges; whether the boundaries represent limits
or edges of failure; and the magnitude of other sources of variability such as analytical variance. Each approach provides
a certain level of confidence that future batches will achieve acceptable quality or a certain level of risk that future batches
will not meet acceptable quality. The approaches include using a statistical model based on regression, using an interval
based on approach, or using mechanistic models.
Which statistical approaches ensure higher confidence?
Because of the inherent variability in the process and in the analytical testing, the boundary of the mean predicted response
region also has inherent variability and thus is probably not exactly equal to the underlying true mean value of the responses.
Building a buffer into the system may provide increased confidence in reliably meeting the specification. Two types of intervals
can provide a buffer: a statistical interval around the normal operating range, and an interval on the edge of the design
space. Figure 13 illustrates a design space for Factor B and its relation to the degradate specification of 1.00%. At the
0.50 value for Factor B, three of the six observations (50%) are outside of the specification. Using an interval approach
and adjusting the operating region to 0.5 to -0.08 provides more assurance that responses from future batches will meet the
specification. Figures 12 and 14 illustrate this same principle for a two-parameter design space. The yellow region in Figure
12 shows the design space based on the predicted value that provides about a 50% probability of passing the specification
if the factors are set at the boundary of the design space. Figure 14 shows a yellow region based on an interval that although
is smaller in size, increases the probability that batches manufactured within the region will meet specifications. Quantification
of these probabilities is helpful in determining where to operate the process.
Figure 14: Two-factor (A, B) design space with confidence of passing specification.
As discussed in Question 19 and shown in Figures 13 and 14, an interval can be developed to quantify the level of assurance
of passing specifications at the boundary of the design space, and can also quantify assurance throughout the design space,
not just the boundary. Possible intervals include prediction, tolerance, or Bayesian.
Figure 15: Two-factor design space based on Bayesian contours for passing specification.
- A prediction interval captures a specified proportion of future batches.
- A tolerance interval can be developed to include a specific proportion of batches with a specific confidence level. This approach
may be more restrictive than a prediction interval because it will most likely ensure greater confidence.
- A Bayesian interval accounts for uncertainty in estimating the model parameters and calculates a probability of meeting all
specifications simultaneously. Figure 15 shows the Bayesian probability contour plot for the two-factor example.
These intervals can be thought of as providing some "buffer" around the region that is based on mean results. Although an
interval will decrease the size of the acceptable mean response region, there is greater confidence that future batches within
the reduced region would meet the specifications (providing the assurance required for a design space), especially if the
mean response is closer to the specification. Use of intervals may not reduce the acceptable mean response region on all boundaries.
There may be cases that within the experimental region for example, when the responses are not close to their specifications.
In such cases, the predicted response at the boundary of the region where the experiments were conducted may be well within
specification. Using the predicted response may be appropriate. For example, if the degradate in the example was never greater
than 0.3% and the specification was 1.0%, then the use of intervals is unlikely to reduce the region.
The use of an interval approach is a risk-based decision. Proper specification setting and use of control strategies should
also be used to increase confidence in the design space and the strategy employed should fit the entire quality system.
Are there any other considerations when using an interval approach?
The interval approach incorporates the uncertainty in the measurements and the number of batches used in the experiments.
As discussed in Question 7 (see Part I of this article series (1)), increasing the number of batches through additional design
points or replicating the same design point increases the power of the analysis. If a small experiment is used to define the
acceptable mean response region, then the predicted values will not reflect the small sample size. However, the interval approach
DoEs reflect the small sample size resulting in a smaller region. In a similar way, if the variation in the data is large,
then the interval approach will reduce the size of the region. A large difference between a region based on the acceptable
mean response and one based on intervals indicates uncertainty in the mean response region. Often, an assumption when performing
the statistical analysis is that the variability of the response is similar throughout the experimental region. In the situation
where this assumption is not true, replication of batches near the boundary of the design space may be needed to increase
the confidence that future batches will meet specifications and the variability explicitly modeled. Prior knowledge from other
products or other scales may be incorporated into the estimates to increase confidence.