Christopher Burgess, PhD, is managing director, Burgess Analytical Consultancy Limited, Barnard Castle, Co Durham, UK.

Christopher Burgess

	Some years ago, the analytical laboratory in a small pharmaceutical company used a domestic refrigerator to store standard solutions for high-performance liquid chromatography (HPLC) analyses. The laboratory head decided that the temperature of the refrigerator was to be recorded manually twice a day using a mercury in glass thermometer positioned on the middle shelf.  As this refrigerator lacked formal qualification documentation as required by 

) General Chapter <1058>, however, the laboratory head requested that it be temperature mapped over a working week. The acceptance criteria to be adopted was that the mean temperature was to comply with the 

 requirements of 2 °C to 8 °C. Recognizing that twice-a-day monitoring was probably insufficient for qualification, the laboratory head decided that the average temperature over two hourly intervals on the middle shelf would be used to generate the data over 60 hours of routine operation.
	 

	Average temperature over two hourly intervals

	To capture the data, certified and calibrated data loggers were used with an accuracy of better than ±0.3 °C, which was considered sufficient for the qualification. The average temperature over two hourly intervals on the middle shelf from the qualification run is shown in 

	Noting that the acceptance criteria were 2 °C to 8 °C, the data logger two-hourly means were rounded to a single significant figure. However, 10 of the 30 averages were identified as failures and an investigation was undertaken.

	Fortunately, there were also data loggers on the top and bottom shelves in order to map the operational volume of the refrigerator. When these data were reviewed in the same manner as the middle shelf, a more interesting picture emerged as shown in 

	All the points on the top shelf except two meet the acceptance criteria and all the points on the bottom shelf except two fail the acceptance criteria.  At this point, the laboratory head mentioned that they did sometimes find solutions that had frozen. 

	The data loggers used had the data storage capacity to record the temperature every 4.8 minutes and 

 could be updated to include the maximum and minimum temperatures for each of the two hourly means (

	
	The temperature range over the five days was a high of 12.6 °C and a low of -6.0 °C.  Hence, there is demonstrable lack of control.

	Even more instructive is to simply trend the 1500 individual readings (

).  The thermostatic control is so poor that cycles of 10 °C are not uncommon, which is more than the 

 specification range. In addition, the lack of air circulation creates a large temperature difference between the top and bottom shelves.

	This example gives rise to some important learning points:

		Always carefully consider whether individual readings or means are appropriate for setting acceptance criteria. Note that on many occasions both are appropriate.

		Always plot the trend of individual readings to see evidence of time-related phenomena.

		Always consider the frequency of measurement of data loggers in the context of actual use.

		Always consider the positioning of data loggers in the context of actual use.

		Never purchase a piece of equipment that was not designed to perform to specific user requirements.

	Remember that a domestic refrigerator is not designed to control an environment to meet 

Articles

A case study presents important lessons for using average and individual results, as well as user requirements, when qualifying a laboratory refrigerator. 


For Lab Equipment Qualification, Data Reveal Important Story


	Vol. 41, No. 6
	June 2017
	Pages: 54-55

	When referring to this article, please cite it as C. Burgess, "Do You Listen to What Your Data are Telling You?," 

This column presents a data case study of a laboratory refrigerator and its qualification performance over five days, with important lessons for using average and individual results, as well as user requirements.


Do You Listen to What Your Data are Telling You?

	Operators of analytical laboratories in a regulated environment are always concerned about being in a demonstrable state of control. One important aspect of this ‘state of control’ is the proper use of certified reference materials in the calibration of instruments and the performance of procedures. This article looks at a structured approach to decision rules for user measurements with certified reference materials to guard against bias.
	
	


	Asking if one’s systems and procedures are in a state of control might seem to be an easily answerable question, especially if one used a traceable certified reference material (CRM) from the National Institute for Science and Technology (NIST), for example. After all, the CRM comes with a certified value together with an expanded measurement uncertainty doesn’t it? Actually, the methodology required to answer this question is not as well known as it should be. The question becomes: How do you construct the acceptance limits for user measurements of a CRM?
	
	Let’s look first at an example of an instrument calibration for absorbance accuracy of a spectrophotometer. The following is a quotation from a 

, which NIST calls a standard reference material (SRM), certificate for glass filters used for the calibration of the absorbance scale of a spectrophotometer (1): “An acceptable level of agreement between the user’s measurements and the certified value and its expanded uncertainty overlaps any part of the user’s tolerance band defined by the measured mean and the user-defined level of acceptability.”

	NIST appears to propose that one needs just two elements to construct an acceptable level of agreement with the certified value and the user’s requirements. The first part is simple. Take the value of the certified standard value from the certificate, x

, and its expanded uncertainty U. Hence the uncertainty interval at 95.45% confidence (k=2) is x

	NIST then introduces the concept of user acceptability based upon the measurement capability of the instrument or the laboratory, A. So if one simply adds the SRM expanded uncertainty to the spectrophotometer manufacturer’s tolerance to the measurement capability, A, then these form the acceptance limits for satisfactory calibration or method performance, i.e., x

	In addition, this certificate states that the measurement process should be “to demonstrate that a user’s measurements are traceable within acceptable limits to the accuracy defined by SRM 1930 the user must first determine the required tolerances or acceptable uncertainty for the application in question. It is recommended that a number of replicate measurements be made for each filter and wavelength, with removal and replacement of the filter between replicate measurements.”

	Assume that, at a particular wavelength, the certified value is 0.4326 absorbance units and the uncertainty is Â±0.0029 absorbance units (U). Also assume that the mean value measured on the user’s spectrometer is 0.4251 and that the spectrometer has a specification for absorbance accuracy of Â±0.0035 absorbance units (A).

 Â± (U + A) becomes the acceptance limits (AL)0.4326 Â± (0.0029 + 0.0035) or 0.4262 to 04390. As the mean measured value found was 0.4251, which falls outside this range, the spectrometer is considered inaccurate. One can clearly confirm the effectiveness of this decision rule by running a Monte Carlo simulation (MCS)model using the SRM values and setting the acceptance limits as above. The result is shown in 

	The simple approach presented, however, is not sufficient, especially as the certificate method mentions replicate measurements and a mean value. Therefore, one needs to include the metrological uncertainty of the user’s measurement in the acceptance limits.

 (2) as the source document for acceptance limits and, in section E3a, the statistically correct calculation for the AL required is defined as 


	
	This is because the above simpler approach does not take into account the metrological uncertainty of the mean measured value, which depends on the standard error of the mean of n measurements. If one takes six measurements and calculates the standard deviation, s, of 0.0015 absorbance units (


	then the ALs become AL = Â±0.0016 + (U + A) = Â±0.0080 giving an acceptance range of 0.4246 to 0.4406.
	After taking into account the metrological uncertainty, the mean measured value, 0.4251, now falls within this range, so the spectrometer is considered sufficiently accurate. This enhanced decision rule is confirmed once again by MCS modeling 

	For those readers wishing for more statistical background and excellent discourse, please read Churchill Eisenhart’s 

 on “Realistic evaluation of precision and accuracy of instrument calibration systems” (3), which is freely available from the NIST archive as is NIST 


	If this acceptance limit approach works for calibration, will it work for standards and methods? Yes, it will if one has a certified and traceable CRM with a measurement uncertainty.

	Assume one has a primary CRM with a certified value of 99.46%, with an expanded uncertainty of 0.15 (i.e., x

  Â± U . If one has generated a working standard that has been assayed six times against this primary standard, and finds a mean value of 99.23 with a sample standard deviation of 0.062, from an analytical viewpoint, the laboratory has decided that it is acceptable to have a bias between the standards of Â±0.05%.

	Using the full equation for the acceptance range, â

, and including the metrological uncertainty, this becomes 


	
	The difference between the certified value and the mean value is

therefore, at 95% confidence, there is no evidence that the working standard does not lie within the combined acceptance limits set by the laboratory requirement, the metrological uncertainty, and the measurement uncertainty of the CRM.

), it is clear that the combined acceptance limits encompass the large majority of the distribution of the CRM values and that less than 0.03% of the values is likely to lie outside these limits.


	The method for calculating acceptance limits for calibrations based on user measurements of certified reference materials has been described and the NIST basis for it discussed. The approach has been confirmed using Monte Carlo simulations. If laboratories adopt this approach, they will have a sound statistical basis for deciding if an instrument or method is ‘fit for purpose’ or not.


	2. D. Becker et al., “Use of NIST Standard Reference Materials for Decisions on Performance of Analytical Chemical Methods and Laboratories,” 


	When referring to this article, please cite it as C. Burgess, "Demonstrating a State of Control," 

Do you have acceptable control of your instruments and analytical procedures?

Demonstrating a State of Control

	Linear calibration is usually considered easy, particularly for high-performance liquid chromatography (HPLC) methods. After all, the procedure and the rules have been unchanged for 20 years (1).

	The International Conference on Harmonization (ICH) requirements (1) are simple and well known:

		“A linear relationship should be evaluated across the range of the analytical procedure.

		“Linearity should be evaluated by visual inspection of a plot of signals as a function of analyte concentration or content.

		“If there is a linear relationship, test results should be evaluated by appropriate statistical methods, for example, by calculation of a regression line by the method of least squares over a minimum of five concentrations over the analytical range.

		“Data from the regression line itself may be helpful to provide mathematical estimates of the degree of linearity.					

				The slope of the regression line and residual sum of squares

				An analysis of the deviation of the actual data points from the regression line may also be helpful for evaluating linearity.”

	The ICH requirements appear to be straightforward. Simply carry out the required experiments and put the numbers into a statistical package or even Excel. How is one to know, however, if it is a good straight line? This column discusses some of the issues, assumptions, and limitations of ordinary least squares (OLS) regression, illustrated using an extraordinary data set.

Issues, assumptions, and limitations of OLS regression

Issue one: Proving the calibration is a good straight line.

 One cannot prove the calibration is a good straight line.  The problem comes from inferential statistics in that the null hypothesis is that it is a good straight line; however, one can’t prove the null hypothesis. All one can do is show that the alternative hypothesis (i.e., that it is in fact a curve) is unlikely at a given degree of probability. 

Issue two: The assumption that the bigger the value of the correlation coefficient r

The statement that ‘the bigger the value of the correlation coefficient r

 measures amount of variation accounted for by the data model, not linearity. This truth is not always recognized by analytical chemists or by quality assurance (QA) personnel who may rely on r

	More than 40 years ago, Anscombe wrote a paper illustrating this issue of slavishly following calculated statistical parameters particularly r

 (2). He showed a table of four X,Y datasets (

), which when submitted to OLS regression gave the same calculated statistical parameters within rounding error (

	He then showed the plot of the four data sets (

). It is readily apparent that blind obedience to calculated parameters is not sensible.

Issue three: The OLS model assumes that all the error is in the Y (or response) variable.

 Issue three is another assumption that is not always valid or recognized. In HPLC, the response variable (Y) is usually peak area and the X variable is concentration. Are the concentrations really error free? 

Issue four: The model residuals are normally distributed.

 This assumption is often overlooked; although, ICH suggests plotting the residuals. A residual is simply the difference between the observed value and the value predicted by the regression model. The evaluation of normal distribution is usually relatively small because the amount of data. However, it is easy to construct a normal probability plot using standard statistical packages, for example, Minitab 17. Ideally all the data should lie closely scattered around the center line.

	Perhaps surprisingly, the plot of the residuals from Anscombe’s Quartet shows only one point beyond the 95% confidence interval (see 

). The wave-like data shape should give cause for suspicion, however, and prompt further investigation.

Issue five: The variance is constant over the analytical range (Homoscedasticity requirement).

 With main peak assays with large response function values over a limited analytical range, for example 80% to 120% of target, this is a good assumption. Even over larger ranges, for example 20% to 150% of target, this may well be a reasonable assumption. For low concentrations of impurities requiring calibration from the reporting threshold to 120% of target, however, this is likely to be untrue, and alternative least squares procedures should be used (e.g., example weighting). It should be apparent that statistical procedures deliver statistical answers, not analytical judgement.


	A user of the Minitab Network on LinkedIn posted the following example of a linearity data set (3). The raw data are shown in 

. Triplicate standard preparations were made at six concentrations covering a concentration range of 0.004 mgL-1 to 0.030 mgL-1 representing 20% to 150% of a target concentration. The OLS regression plot is shown in 

 is 0.999618, and the intercept is indistinguishable from zero at 95% confidence. How about that for a straight line? However, as the data set poster remarked, how are the residuals distributed (see 

)? Definitely not normally distributed, but why?

	First, let’s look at the residual plot, as suggested by ICH and shown in 

. Clearly there is something strange about the concentration data at 0.016%. Let us assume that we have evidence that there was a preparation error at this concentration; therefore, we exclude it from the analysis (

	It appears this has solved the problem, but is there more to learn? Let’s look at the revised regression data and its plot (Figure 7).

	Well the slope hasn’t changed a lot, and r

 has even increased to almost unity. The intercept, however, has and is now more negative and significantly different from zero. The intercept is small, approximately 0.4% of the response at 100% of target, but the possibility of a negative peak area for a positive concentration should worry chromatographers. 

	Help is at hand, because there is a useful, but little known, simple diagnostic plot not mentioned in ICH known as a Sensitivity Plot (4) . If the peak area response is divided by the concentration (a normalized response), theoretically you should get a constant value or, in practice, a straight line parallel to the X axis with a small random scatter due to measurement noise. The mean normalized response is the slope of the calibration line. If the sensitivity plots for the data in 

 are produced with and without the values at 0.016%, the results are shown in 

	The following are three important features in Figure 8 that are readily apparent:

		The normalized responses are not linear.

		Excluding the 0.016% data merely shifts the mean line (the slope of the calibration line) slightly downwards (approximately 0.5%).

		The range of values at below 0.016% are much larger than those above suggesting the variance may not constant over the analytical range (issue five). The statistical term for this is heteroscedasticity (the violation of homoscedasticity) when the size of the variance differs across values of an independent variable, in this instance, concentration.


	Statistical procedures give statistical answers not analytical judgement. The statistical tools, in this instance OLS linear regression, require sound interpretation by the analytical practitioner, not slavish adherence to statistical significance of parameters especially r

. Always look at the data and do sensitivity plots. Given the findings herein, it would be most interesting to investigate the laboratory records of the example data set to try and establish root cause(s) for the observed calibration data issues.

Validation Of Analytical Procedures: Text And Methodology

 (ICH, Oct. 27, 1994, Nov. 6, 1996, incorporated November 2005).
	2. F. J. Anscome, 

 27, 17-21 (1973).
	3. Gamal M. Mohamed, data set posted on the Minitab Network on LinkedIn, Feb. 12, 2015, www.linkedin.com/grp/post/166220-5970966595894288384.
	4.  J. Ermer and P. Nethercote (Eds), 

Method Validation in Pharmaceutical Analysis

, 2nd Edition, Section 5.5.1.2.1, p. 153 (Wiley-VCH, 2015). 

:
	When referring to this article, please cite it as C. Burgess, "Is Your Calibration Really a Good Straight Line?," 

Statistical procedures give statistical answers not analytical judgement.

Is Your Calibration Really a Good Straight Line?

	Most analytical procedures rely on the accurate preparation of reference standard solutions.  It is common practice to prepare such solutions volumetrically using standard glassware.  Usually a stock solution is prepared followed by a dilution or series of dilutions to arrive at a working concentration of the reference standard.  This column looks at the effects of choice of dilution schemes on the accuracy of the prepared reference standard.

	A previous column looked at the basics of measurement uncertainty and related it to the direct preparation of a standard solution were discussed (1).  This column will review the preparation of standard solutions by simple and serial dilution using standard pipettes and flasks. Only the uncertainties in the volumetric glassware, however, will be considered to answer the question: “Is my dilution accurate enough?”  It will be assumed that all the other factors (e.g., operator variability when pipetting or making up to the mark, temperature, and solvent effects) are constant.
	
	Further, it will be assumed that standard Grade A glassware  are being used (2). Accuracy in the context of this column is the International Organization for Standardization (ISO) definition.  In ISO, accuracy combines the concepts of unbiasedness (termed trueness) and precision (3).
	
	Suppose an analytical procedure states that a standard solution of 1 mg per liter is required by dilution of a 50 mg/L stock solution.  The question arises “What is the ‘best way’ to perform this dilution volumetrically?” The ‘best way’ would be to have minimal dilution error (i.e., smallest measurement uncertainty commensurate with the analytical need). This column will only consider some whole number dilutions, but the principles apply to any dilution or dilution sequence.
	
	

Tolerances in Grade A volumetric glassware


	
	For volumetric glassware, tolerances are an ISO Type B uncertainty and are converted to a standard deviation,
	u

, assuming a triangular distribution, as it is more likely that the value lies closer to the central value (3).  For a range of ±a, the corresponding estimate for the standard uncertainty would be 

Choosing the best combinations Grade A volumetric glassware


	Relatively little has been published on the optimum choice of pipettes and flasks (4, 5).  Indeed, there appears to be little awareness in laboratories that the selection of different combinations results in different accuracies.  At least one textbook covers the practicalities of volumetric operations (6).  Lam and Isenhour in 1980 (4) covered minimization of volumetric errors using error propagation theory but only considered combinations of 4-, 10-, 15-, 20-, and 25-mL pipettes and flasks from 25 to 1000 mL (4) with relative errors of 0.13 to 0.35% for one step to three step dilutions.  In this paper, the tolerance interval 

.  The results in their table need to be divided by 2.45 to be corrected for this issue. They defined a concentration ratio, C, for any pipette (

 )dilution sequence of n steps (Equation 1).

	
	Hence, for a sequence 20 to 1000 followed by 25 to 500, C would be 1/1000 or a dilution of 1000:1.
	
	Consider the number of combinations of standard glassware to make a single dilution. Assume that in the laboratory, there are pipettes from 1 mL to 25 mL and volumetric flasks from 5 mL to 1000 mL.  

 illustrates single step whole number dilution ratios from 5 to 1000.  The dilution ratio here is 1/C  of  Equation 2.

	Based on error propagation theory  (7), the combination of relative errors for any pipette (

)dilution combination sequence of n steps can be calculated from Equation 3 and the results are shown in 

	In this example of preparing a 1 mg per liter dilution from a 50 mg/L stock solution in a single step, five combinations can be selected from 

; 1 to 50, 2 to 100, 5 to 250, 10 to 500, and 20 to 1000. Do they all have the same accuracy? To answer this question, look up the appropriate combined relative uncertainties in 

 gives the combined uncertainties as a %.
	
	From 

, the bigger the pipette and the bigger the flask is, the lower the combined relative standard uncertainty. Larger amounts of both solvent and solute, however, are needed to achieve a factor of 5 gain from a single dilution.
	
	Returning to the 50-fold dilution, this effect is illustrated (

) as the 20 to 1000 dilution produces a lower uncertainty by a factor of four compared with the 1 to 50 dilution.  This comes at a cost of solvent and solute.  So what is ‘best for you’?  The 1 to 50 dilution may be perfectly acceptable.

	
	
	Suppose, however, the laboratory wanted to achieve this 1:50 dilution with minimum solvent and solute usage by preparing a stock solution in a 10-mL flask and serially diluting it 1 to 5 and then 1 to 10. The resulting standard uncertainty from volumetric sources would be seen in Equation 4.
	
	 

	
	
	The numerical values are taken from the 

.  Note that this value is slightly different from the value in 

.  This is because there is a small contribution from the original 10-mL flask.
	
	Ignoring the uncertainty contribution associated with weighing 0.5 mg of a reference standard, the volumetric uncertainty due to the serial dilution is 0.40%, which is 1.6 times larger than a simple 1 to 50 dilution. Bearing in mind that at a coverage factor, 

, of 2 (95.45% Confidence), the volumetric uncertainty alone would approach 1%.  In this instance, a complete error budget calculation would probably show that the overall uncertainty at a coverage factor of 2 might be too large from this sequence to be employed in practice for an assay standard.
	
	Quite often a 1000-fold dilution is needed. 

 shows three examples.  In the first one, it is a simple one-step 1 to 1000 dilution resulting in a reasonably accurate result. However, it requires a lot of solvent.  A three-stage serial dilution process is attractive to reduce solvent usage. The smallest serial dilution process is three 1.00 to 10.0 dilution steps.  This dilution sequence results in about twice the measurement uncertainty. However, three 5.00 to 50.0 dilution steps or three 10.00 to 100.0 dilution steps could also be used to achieve the same dilution.  The three 5.00 to 50.0 dilution steps shows not only an improvement of about half the measurement uncertainty of the 1000-fold dilution, it also saves approximately 75% of the solvent.  There is no advantage in using three 10.00 to 100.0 dilution steps over three 5.00 to 50.0 dilution and it uses twice as much solvent.


	As analytical procedures rely on the accurate preparation of reference standard solutions, it is important to know what the probable effect of analytical operations will be on the overall measurement uncertainty of the analytical result.  Whereas volumetric dilution errors are small, they are not insignificant.  The structure of dilution sequences can play a significant role in contributing to an uncertainty budget.  Examples in this column have demonstrated both the method of estimation and extent of uncertainty contributions.  
	
	

, 37(9) 62-64, 73 (2013).
	2. ISO 648:2008 Laboratory glassware. Single-volume pipettes.  ISO 1042:2000 Laboratory Glassware - One-mark volumetric flasks
	3. S L R Ellison and A Williams (Eds). 

Eurachem/CITAC guide: Quantifying Uncertainty in Analytical Measurement

, 3rd Edition, (2012) ISBN 978-0-948926-30-3.
	4. R. B. Lam and T. L. Isenhour, Anal. Chem., 52(7), 1158 -1161 (1980).
	5. Y Hayashi and R Matsuda,

, 10(6) 881-888 (Japan Society for Analytical Chemistry, 1994).
	6. D. A. Skoog, D.M. West, F J Holler,

, 7th edition, 798-808 (Saunders College Publishing, 1996).
	7. A.A. Clifford, 

, Applied Science Publishers (Wiley, New York, 1973).

: When referring to this article, please cite as C. Burgess, “How Accurate are Your Dilutions?,” 

The choice of dilution schemes can minimize solution preparation errors.

How Accurate are Your Dilutions?

Manufacturers rely upon analytical chemists to carry out analysis and testing of samples to ensure products conform to specifications and to predict the properties of the batch or lot based on the analysis of a sample. Suppose for a moment that a sample is representative of the batch or lot. How much testing of the sample is needed to get a reliable estimate result that is applicable to the batch? This question has financial as well as technical implications. Businesses need--and regulators require--that sufficient testing is done to deliver an answer within a level of confidence.

When testing a number of samples from a batch, analysts usually estimate the value for the batch by taking an average and the confidence in it by calculating its standard deviation. In other words, they adopt a mathematical model that describes the shape of the continuous analytical data and allows them to estimate two parameters that define the distribution: a measure of location (in this instance the mean) and a measure of dispersion (the variance or standard deviation). The distribution for analytical data of this type is usually assumed to be the normal or Gaussian distribution.

Calculating the average isn’t real statistics, is it?


On this basis, the calculation of the humble mean or average takes on a new significance. Consider 10 measurements of a particular analytical parameter of interest, such as the density of a liquid sample. In the example shown in 

, the average is calculated to be 1.503 and the standard deviation to be 0.0124, tasks easily accomplished with a pocket calculator or Microsoft Excel. How can these calculations relate to a mathematical model?

Table I: Residuals and their squares from the calculated   mean density value.

In taking a closer look at the calculations in Table I, subtracting the mean value from each of the n individual values results in a residual. The sum of the residuals is zero, as expected. However, when squaring the residuals and summing them the answer is not zero. This sum of squares of residuals, 

By use of a little calculus, it is easy to show that, according to the principle of least squares (1), the best estimate of location will be when 

 is at a minimum. Hence, the differential is set to zero (

It is now apparent that the humble average is the best least squares estimator for the data model, Y = X + ε, which is the measure of location. 

In addition, the dispersion can be calculated--in our example, the sample standard deviation, sx--from the square root of the variance (

), which is simply the sum of squares of residuals divided by the number of independent variables (degrees of freedom) n-1. This calculation can be done because there are not 10 independent pieces, but only 9, as a mean from the data has already been calculated. The standard deviation is the square root of the variance.

However, the question remains: How many determinations are needed?

Statistical analysis shows how much testing is needed to deliver a reliable estimate result.