Christopher Burgess is with Burgess Consultancy, "Rose Rae," The Lendings, Starforth Barnard Castle, Co. Durham, DL12 9AB UK.

Christopher Burgess

	Analytical people tend not to worry too much about the nature of the distribution of their data.  Statisticians, on the other hand do worry.  Some 10 years ago, Rebecca Elliott, a senior statistician with Eli Lilly, gave an excellent presentation on some ways users drive statisticians crazy (1).  This column is about just one method to prevent that happening, as we do need statisticians occasionally. 

	The central limit theorem (CLT)-actually theorems as there are more than one-is at the heart of much statistical methodology. The CLT is a lifesaver for analytical data, be it continuous or discrete.  The good news is that the part that must be understood is very simple.  The really good news is that if the CLT did not exist, many familiar statistical methods would not be valid.  The first lesson for analytical people to learn is that, from a statistical point of view, there is a world of difference between individual or replicate data from a sample and means of those data.

	It is visually apparent, therefore, that to use methods based on the assumption of normality would be invalid in many instances. Fortunately, the CLT comes to our aid.

	It is the mathematical model (shape) of data population which determines the applicability (suitability) of the statistical methodology.  A summary of some commonly met distributions are shown in 

	The simple part of the CLT is that for any sample of N independent determinations, the means of n values tend to a normal distribution irrespective of the underlying population distribution. In addition, the overall or grand mean tends to the population mean. Note that the bigger n is, the more this is true. Unlikely though this seems, some calculations can be performed in Microsoft Excel to demonstrate this property. This is not well described in books on statistics, but 

Basic Statistics and Pharmaceutical Statistical Applications

 does describe this well, albeit somewhat hidden away in the 800 pages (3). The following is one way to visualize the value of the CLT.

	Let us consider a small data set of 15 integer numbers far away from a normal distribution, namely 8,8,8,9, 9,9,10,10,10,11,11,11,12,12,12, which are shown as a histogram in 

	Let us take the decision that n is to be two and we know that there will be 105 means of two combinations of these 15 numbers. In Excel, this would be calculated using the combination formula;=COMBIN(15,2). Sadly, there is no function currently in Excel to list all of these 105 combinations, but fortunately there is a macro available on the Internet to do just that (4). Once you have these combinations listed, then you can calculate the means. The result is shown in 

	Looking good so far; so what about n=3? In Excel, this is calculated from the formula =COMBIN(15,3), which gives 455 combinations. Modifying the macro works well and generates the 455 combinations listing. We proceed as before by calculating the means of 3 resulting in the histogram shown in 

	The approximation to the normal distribution becomes even more apparent. The normal approximation becomes good when n is in the region of 25 to 30. This was demonstrated in an earlier column using the t distribution (5). This particular calculation cannot be performed in Excel because if one used N=50 data points and n=30, one would have 47,129,212,243,960 combinations, which would take, at one calculated mean per second, just under 1.5 million years to achieve even if Excel allowed that many rows.

	The central limit theorem concerning the distribution of means allows one to justify the assumption of the normal distribution so that we can use many of the statistical formulae that require normality on our datasets. As long as mean and individual data are clearly differentiated, we will help our statistician colleagues remain sane.

	1. R. Elliott, “Product Specifications: How to Drive a Statistician Crazy,” Pharmaceutical Statistics 2008: Confronting Controversy Conference, Arlington VA, 2008.
	2. A. Damodaran, Probabilistic Approaches: Scenario Analysis, Decision Trees And Simulations, people.stern.nyu.edu/adamodar/pdfiles/papers/probabilistic.pdf, Figure 6A.15: Distributional Choices.
	3. JE De Muth, 

, 3rd edition, page 123 (CRC Press, 2014).
	4. Allen Wyatt, Listing Combinations, 


	Vol. 43, No. 10
	October 2019
	Pages: 62–64

	When referring to this article, please cite it as C. Burgess, "Distribution of Data: The Central Limit Theorem," 

Articles

Using the central limit theorem concerning the distribution of means allows one to justify the assumption of the normal distribution. 

Distribution of Data: The Central Limit Theorem

	Performance metrics for processes are an area of much regulatory interest currently. There isn’t always a readily available clear definition of what is needed, however, and guidance from regulators is not always consistent. This column goes back to the basics that were first set out by Shewhart (1) nearly 90 years ago and relates them to some modern process performance and capability indices.

	Definitions are important in providing a consistent nomenclature, and the global International Organization for Standardization (ISO) standard 3534-2:2006 (2) will be used.

	All process measurement results are subject to variations that come from a variety of sources as was seen in the previous Statistical Solutions column (3). However, there are only two types as defined by Shewhart, namely common cause variation and special cause variation.

	Common cause variation is the inherent noise in a process over time due to random effects and hence predictable within statistically derived limits.

	By definition, a process that contains only common cause variation is said to be in statistical control.

	Special cause variation occurs because of specific circumstances that are not always present manifesting themselves by, for example, a shift or drift in the process mean or excessive noise. If a process contains special cause variation, it is unstable from a statistical point of view, and the overall variation observed contains both common and special cause components. Control charts are designed to detect the presence of special causes of variation. The normal distribution is characterized by two parameters: a measure of location (the arithmetic mean or average) and a measure of dispersion (the standard deviation). An unstable process means that both of these parameters could be or are changing in an uncontrolled manner (

	The task is to bring these two parameters into a state of statistical control. This would entail ensuring that the mean and the standard deviations were not varying significantly. This ideal situation is illustrated in 

. This process would then be said to be under statistical control (i.e., no special cause variation and stable common cause variation). In this state, the process is amenable to the tools of statistical process control (SPC). However, a stable process may not be statistically capable of meeting the specification limits. 

 illustrates this, showing that the red process, albeit stable, is incapable. The desired state is to arrive at the blue capable state.

	Capability is assessed using a family of quality metrics or indices called process performance and capability indices.

	Quality metrics for process performance and capability

	There are a variety of performance indices for processes in regular use. However, in this column, only four will be discussed, P

	The definition and meaning of these four will be defined later. Of these four, only two have any practical relevance, P

. The other two are of theoretical interest as they do not occur in practice other than by chance. Process performance. P

, a process performance index, relates to the output performance of a process, irrespective if it is in control or not, with the specification assuming that the long-term mean will be on the target for the product (an unbiased process).

	The index is defined as a ratio of the difference between the upper and lower specification limits (called the specified tolerance in ISO) and the 99.73% probability of a value lying within ±3 standard deviations from the target (called the reference interval in ISO).

	Hence, it can be said that this index would represent what the customer actually receives from the overall process (see 

 , is calculated from the usual formula for a sample standard deviation.

	Where X is the mean of the N data points.

 of 1.33 or more would indicate a highly capable process. A value of less than 1 would indicate an incapable process that would lead to out-of-specification (OOS) results.

	As it is highly improbable that processes are unbiased, a practical process performance index would need to take this bias into account when assessing process performance. This is done by calculating the upper and lower process performance indices P

 using not the target but the actual observed mean to calculate them from:

	Hence the process performance index, Ppk is given by the smaller of the two values above.

	Process capability. Process capability refers to the performance of the process when it is operating under statistical control. Two capability indices are usually computed: C

 in a similar way as was described with P

 measures the potential capability in the process, if the process was centered, while C

 measures the actual capability in a process, which is off-center or biased. If a process is centered, then C

	The critical thing to note is that while the formulae for P

 look very similar, the standard deviation used to calculate the reference interval for Cpk  is not S

 is the within batch standard deviation (called the within sub-group standard deviation in ISO) not the overall process standard deviation. It is usually estimated from a Shewhart mean and range control chart using the formula:

 are 0.5 to 1 for incapable processes, 1 to 2 for capable processes and >2 for highly capable processes.

	A word of caution is necessary in interpreting C

 analysis requires a normal underlying distribution and a demonstrated state of statistical process control. When reporting a C

 value, a 95% or 99% confidence interval should always be reported because this takes into account the sample size used in the calculation (5,6).

	Sadly, this is usually missing. The confidence interval is extremely important because it is not always recognized that, for reasonably small confidence intervals around C

  values, the number of data points needs to be large. 

 shows that to have a 95% confidence interval in C

  of 1.33 ±10% requires in excess of 200 data points. One commonly used approximation formula (5) for the confidence interval is:

 values for comparison of performance needs to be interpreted with great care when n is small.

	It has been shown how to differentiate between process performance and process capability. Equations, however, are not normally as clear as an example.

 shows data from 157 batches of a product with a target of 7.0 and upper and lower specification limits ± 1.5. The data are nicely normally distributed as can be seen from the normal probability plot, but the long-term mean of 7.4 is biased high. However, the process capability C

 is excellent at 1.31 and even with the bias would be unlikely to produce OOS results due to common cause variation (red curve).

	Unfortunately, the process suffers from considerable special cause variation, the dashed black curve, with P

  being an unacceptable 0.74 because the overall batch standard deviation (St ) is 0.49, whereas the within batch standard deviation (S

 is much smaller at 0.28. Note that if we could remove the mean bias, P

 would be a more acceptable 1.02. However, it would require a root cause investigation and process change(s) to remove some of the special cause variation(s) to approach a truly capable process.

Economic Control of Quality of Manufactured Product

	2. ISO 3534-2:2006; Statistics-Vocabulary and symbols-Part 2: Applied statistics

 GHTF/SG3/N99-10:2004 (Edition 2) Annex A, Statistical methods and tools for process validation, www.imdrf.org/documents/doc-ghtfsg3.asp.

	5. ASTM E2281-15, Standard Practice for Process and Measurement Capability Indices.


	Vol. 42, No. 10
	October 2018
	Pages: 64-67

	When referring to this article, please cite it as C. Burgess, "Process Stability, Performance, and Capability; What is the Difference?," 

This article applies the basics of stability, performance, and capability to modern process performance and capability indices.

Process Stability, Performance, and Capability; What is the Difference?

	In February 2005, Lynn Torbeck wrote “In Defence of USP Singlet Testing,” in which he discussed the US Pharmacopeial Convention’s (USP) philosophy of singlet testing and the clear differentiation between standards and specifications (1). Later that year, Tim Schofield, David Leblond, and Stan Altan offered comments and an alternative viewpoint (2). Much has changed over the past 12 years, although many people in industry still regard the USP standard as a specification for the release of product to the market. This article reviews these changes based on the latest USP General Notices (3) and provides a view on the meaning of ‘singlet testing’. 

	Torbeck emphasized the important philosophical difference between a standard and a specification, and he quoted from the 

) 28 (2005) General Notices (1), “Compendial standards define what is an acceptable article and give test procedures that demonstrates that the article is in compliance. These standards apply at any time in the life of the article from production to consumption.” 

	Furthermore, these General Notices defined the testing, which constitutes a singlet determination. 

	“Thus, when tested from the viewpoint of commercial or regulatory compliance, any specimen tested directed in the monograph for that article shall comply…,” and “Tests and assays in this Pharmacopeia prescribe operations on a single specimen, that is, the singlet determination which is the minimum sample on which the attributes of a compendial article should be measured.”

	It is this last sentence that has given rise to much debate and dissention. Schofield 

 took this to mean that “Every lot of a pharmaceutical products with possibly millions of dosage units per lot must be made by processes that guaranteed that every dosage unit tested by the filed analytical method at any time before expiry conforms with the stated USP standard” (2).

	Clearly, this guarantee is unattainable from both a statistical and practical viewpoint and appears to contradict the uniformity of dosage requirements, which they point out. However, what is clear is that USP monographs are standards and not specifications. The current version of the USP General Notices and Requirements doesn’t mention the singlet determination and clearly states in 3.10 Applicability of Standards (3): “Standards for an article recognized in the compendia (

) are expressed in the article’s monograph, applicable general chapters and General Notices.” Hence, there is a strict hierarchy for compliance in the order; monograph, General Chapters and General Notices: “The standards in the relevant monograph, General Chapters and General Notices’ apply at all times in the life of the article from production to expiration” (3). This has not changed from 2005. 

	“It is also noted that the manufacturer’s specifications and manufacturing practices (e.g., quality by design, process analytical technology, and real-time release testing initiatives), generally are followed to ensure that the article will comply with compendial standards until its expiration date, when stored as directed” (3). This is a significant change from 2005. This quotation addresses the valid concern of Scofield et al. in 2005 that a singlet determination may discourage a full scientific understanding of a manufacturing process and interfere with the objective of providing quality product to patient.

	USP standards are legal benchmarks of acceptable quality or performance and must be met with some acceptable degree of confidence (probability). In testing of a sample, there is no such thing as an exact value. There is only an estimate of the true value with an associated confidence or tolerance interval.

	Specifications, on the other hand, apply to reportable results and are decision limits. They must be met or failure occurs. There is no probability level associated with specifications. 

	What is not disputed is that all acceptance criteria in any monograph must be met to achieve compliance (3). However, the basis for the acceptance criteria often is disputed. USP 4.10.20 in the General Notices says, “The acceptance criteria allow for analytical error, for unavoidable variations in manufacturing and compounding, and for deterioration to an extent considered acceptable under practical conditions …” and requires in addition that, “An official product shall be formulated with the intent to provide 100% of the quantity of each ingredient declared on the label” (3). 

	The purpose of compliance testing is to demonstrate that the sample taken from the lot meets the acceptance criteria for the specific monograph. General Notices 3.10 states, “… in all cases, statements about whether the compendial standard is met apply only to the units tested” (3).

	This statement leaves no doubt that compliance with the monograph relates only to the sample tested and not to the batch or lot. Therefore, it is scientifically unsound to use the acceptance criteria of the standard as the release specification for a lot. The General Notices go on to state, “Repeats, replicates, statistical rejection of outliers, or extrapolations of results to larger populations, as well as the necessity and appropriate frequency of batch testing, are neither specified nor prescribed by the compendia; such decisions are based on the objectives of the testing. Frequency of testing and sampling are left to the preferences or direction of those performing the compliance testing …” (3).

	In 7.10, Interpretation of Requirements, the reportable value is defined as: “The reportable value, which often is a summary value for several individual determinations, is compared with the acceptance criteria. The reportable value is the end result of a completed measurement procedure, as documented” (3). 

	In addition, regarding multiple dosage unit testing, 

 states, “Some tests, such as those for Dissolution and Uniformity of Dosage units, require multiple dosage units in conjunction with a decision scheme. These tests, albeit using a number of dosage units, are in fact one determination” (3).

	The statement regarding the singlet determination was removed from the General Notices in 

	The USP requirements for sampling remains very weak. The only mention is in 6.60 Units necessary to complete a test, “Unless otherwise specified, a sufficient number of units to ensure a suitable analytical result shall be taken” (3).

	This review of changes to the meaning of singlet testing raises the following points:

		It is important to understand the difference between a standard and a specification

		A singlet determination did not mean testing must only be on one sample to determine compliance. It was the minimum sample size.

		The degree of testing replication necessary to arrive at a reportable value must be determined and justified by the manufacturer or tester in a documented procedure.

		Monograph acceptance criteria relate to the samples (units) tested for compliance and not the manufactured lot. It is therefore scientifically unsound to use the acceptance criteria of the standard as the release specification for a lot.

		The monograph acceptance criteria assume that the amount of specified material is targeted at 100% of claim.

		Sampling is not addressed well in the General Notices and only General Chapter <1010>, which is non-mandatory, considers it (4).

		The manufacturer needs to have in place risk-based in-house specifications designed to take into account sampling, analytical, and manufacturing variation with the objective of ensuring with a high degree of probability that any sample take from any manufactured lot would meet the acceptance criteria given in the monograph. Such a risk-based approach is available based on the ISO guard band principle (5, 6) and references therein.

 28 (2) 105-106 (2005).
	2. T. Schofield, D. Leblond, and S. Altan

 First Supplement, General Notices (USP, 2017) (valid from Aug. 1, 2017).
	4. USP, 

, First Supplement, General Chapter <1010>, Analytical Data--Interpretation And Treatment, (USP, 2017) (valid from Aug. 1, 2017).
	5. C. Burgess, 


	Vol. 41, No. 10
	October 2017
	Pages: 82–83, 92

	When referring to this article, please cite it as C. Burgess, "Singlet Determination Revisited," 

Is there a difference between a specification and a standard?

Singlet Determination Revisited

	Some years ago, the analytical laboratory in a small pharmaceutical company used a domestic refrigerator to store standard solutions for high-performance liquid chromatography (HPLC) analyses. The laboratory head decided that the temperature of the refrigerator was to be recorded manually twice a day using a mercury in glass thermometer positioned on the middle shelf.  As this refrigerator lacked formal qualification documentation as required by 

) General Chapter <1058>, however, the laboratory head requested that it be temperature mapped over a working week. The acceptance criteria to be adopted was that the mean temperature was to comply with the 

 requirements of 2 °C to 8 °C. Recognizing that twice-a-day monitoring was probably insufficient for qualification, the laboratory head decided that the average temperature over two hourly intervals on the middle shelf would be used to generate the data over 60 hours of routine operation.
	 

	Average temperature over two hourly intervals

	To capture the data, certified and calibrated data loggers were used with an accuracy of better than ±0.3 °C, which was considered sufficient for the qualification. The average temperature over two hourly intervals on the middle shelf from the qualification run is shown in 

	Noting that the acceptance criteria were 2 °C to 8 °C, the data logger two-hourly means were rounded to a single significant figure. However, 10 of the 30 averages were identified as failures and an investigation was undertaken.

	Fortunately, there were also data loggers on the top and bottom shelves in order to map the operational volume of the refrigerator. When these data were reviewed in the same manner as the middle shelf, a more interesting picture emerged as shown in 

	All the points on the top shelf except two meet the acceptance criteria and all the points on the bottom shelf except two fail the acceptance criteria.  At this point, the laboratory head mentioned that they did sometimes find solutions that had frozen. 

	The data loggers used had the data storage capacity to record the temperature every 4.8 minutes and 

 could be updated to include the maximum and minimum temperatures for each of the two hourly means (

	
	The temperature range over the five days was a high of 12.6 °C and a low of -6.0 °C.  Hence, there is demonstrable lack of control.

	Even more instructive is to simply trend the 1500 individual readings (

).  The thermostatic control is so poor that cycles of 10 °C are not uncommon, which is more than the 

 specification range. In addition, the lack of air circulation creates a large temperature difference between the top and bottom shelves.

	This example gives rise to some important learning points:

		Always carefully consider whether individual readings or means are appropriate for setting acceptance criteria. Note that on many occasions both are appropriate.

		Always plot the trend of individual readings to see evidence of time-related phenomena.

		Always consider the frequency of measurement of data loggers in the context of actual use.

		Always consider the positioning of data loggers in the context of actual use.

		Never purchase a piece of equipment that was not designed to perform to specific user requirements.

	Remember that a domestic refrigerator is not designed to control an environment to meet 

A case study presents important lessons for using average and individual results, as well as user requirements, when qualifying a laboratory refrigerator. 
