Typically, to simplify the task, acceptance limits for some computed value (e.g., the sample average, sample standard deviation, or number of results outside 75–125%) are determined in advance, and the outcome of the test is determined by comparing the observed computed value with the limits. If the observation is outside the limits, then the assumption is rejected, otherwise not. The limits are determined so they correspond to a selected low probability of rejecting a correct assumption. The probability used for the decision varies based on the circumstances; however, often probabilities of 0.05, 0.01, or 0.001 (5, 1, or 0.1%) are used. This probability, often referred to as the significance level, also can be thought of as the risk of drawing the wrong conclusion by rejecting an assumption that actually is correct. In quality control for batch release, this is the producer's risk and represents a risk of failing a batch with acceptable quality. As long as there is variation in the data (e.g., as a result of manufacturing or measurement process variability), there will always be a non-zero risk of rejecting a correct assumption.

Although this type of error cannot be completely removed, widening the acceptance limits will reduce it. This change, however, has the undesirable effect of increasing the risk of accepting a batch of inferior quality. In quality control for batch release, this is called the consumer's risk. Maintaining a low consumer's risk is a key goal of both regulators and the pharmaceutical industry. A major challenge in decision-making is to find a suitable balance between these two risks. One of the advantages of increased testing is that it allows for the simultaneous reduction of both types of risks, provided acceptance criteria are well designed.

The final decision about appropriate batch quality is made by using different tests, with the proper sample sizes and acceptance criterion, to protect both the consumer and producer. When these tests are performed more frequently than prescribed without adjusting the decision rule, the producer's risk increases. This phenomenon is called multiplicity. Moreover, this increase in producer's risk often occurs without any significant associated reduction of the consumer's risk. Further, if the producer's risk is too large, the manufacturing costs cannot be recovered, and a potentially beneficial drug product may not be made available to patients.

This section illustrates typical situations in which multiplicity acts as a disincentive to collecting more data and thereby causes poor decision-making.

Release testing. At batch release, a range of properties judged important for product quality is evaluated to verify acceptable batch performance. It is not uncommon to have 10–20 properties to test, some of which may be associated with multiple acceptance criteria. For some tests, replicates in addition to final (reportable) results may be compared with criteria. In total, the specification might contain more than 30 comparisons with acceptance criteria. A failure of any one of the 30 criteria will result in a failure of the batch being tested. In this case, if each test has a 1% risk of falsely exceeding its acceptance criteria for an acceptable batch, then the multiplicity phenomenon results in as much as a 26% risk of rejecting an acceptable batch as a result of at least one criterion failure (1 – [1 – 0.01]³⁰ = 0.26). As a result, manufacturers are motivated to reduce the number of properties being studied or the amount of testing supporting a given property.

Repeated release is another multiplicity issue related to release testing. For certain product types and territories, the manufacturer's release of a batch must be followed by additional regulatory testing and re-release of the same material. This results in multiple tests when the material additionally must be tested for importation into a region, then possibly tested a third time by local authorities. The situation is exacerbated when the same batch is exported to several countries or territories.