Marion Chatfield is a manager in Statistical Sciences at GlaxoSmithKline (Hertfordshire).

Marion Chatfield

Validation of analytical procedures requires study of intermediate precision for within-laboratories variations; however, there is no global guidance on how to develop such designs. By applying science- and risk-based principles-aligned with quality by design (QbD)-to analytical procedures, studies can be designed to ensure the factors selected present the highest risk of impacting the performance of the analytical procedure. In addition, the number of independent analytical runs to include in these designs should be linked to the overall risk/complexity associated with the analytical procedure.

	Useful pre-QbD guidance (provided by the Japanese National Institute of Health Sciences) proposed the use of a generic design which utilizes six independent analytical runs. Examples are provided which instead link the design structure and the number of independent analytical runs to the risk.

, September 2019
	September 2019
	Pages: 12–22

	When referring to this article, please cite it as P.J. Borman, et al., “Risk-Based Intermediate Precision Studies for Analytical Procedure Validation," 

Pharmaceutical Technology Regulatory Sourcebook

Articles

Validation of analytical procedures require assessment of the impact of variations within laboratories; however, guidance to study intermediate precision has been lacking. Science and risk-based principles should be used in the design of intermediate precision studies. 

Risk-Based Intermediate Precision Studies for Analytical Procedure Validation

Analytical quality by design (AQbD) has been a topic of interest as quality by design (QbD), risk-based pharmaceutical development, and manufacturing principles have been promoted over the past decade (1). It has been the topic of several publications and presentations (2–8), and goals have mirrored those outlined for general QbD-based regulatory submissions: the development of well-designed, understood, and controlled processes for manufacturing and testing pharmaceutical substances and products. An initially proposed AQbD framework outlined a goal toward delivering robust, well-understood analytical methods for effective process control in the delivery of high-quality, well-characterized, and controlled pharmaceuticals along with regulatory flexibility to suitably manage analytical methods over a product’s lifecycle (9). Introduction of that framework afforded a platform for discussion and development of practices toward achieving these goals. Several concepts associated with quality-risk management as outlined in International Council for Harmonization (ICH) guidelines (10–12) were adopted from the QbD lexicon. Design spaces, risk assessments, and criticality assessments are three such concepts. Analytical QbD also introduced concepts such as an analytical target profile (ATP) to define method expectations, and may be analogous to the quality target product profile that contains critical quality attributes (CQAs) as a QbD-based concept to define a method as “fit for purpose”. The following is an example of an ATP for a small-molecule drug product assay:

Assay: The procedure must be able to quantify [analyte] in [presence of X,Y,Z] over a range of A% to B% of the nominal concentration with an accuracy and uncertainty so that the reportable result falls within +/-C% of the true value with at least a 90% probability determined with 95% confidence (13).


	
	Based upon commonality with QbD-based principles, some of these concepts have even been discussed in US Pharmacopeial Convention (USP) stimuli articles as an opportunity to illustrate desired analytical method practices in support of the proposed 

 Chapter <1220> Lifecycle Management of Analytical Procedures (13-15).  These USP-based stimuli articles support the use of ATPs and quality-risk based assessments for managing methods through their lifecycles to ensure that methods remain fit for purpose.

	While these concepts are easy to discuss and perhaps describe when specifications have been defined, implementation of such concepts has posed challenges over the past decade for several reasons:

		Often the ATPs incorporate statistical language and may require enhanced statistical support when establishing ATPs.

		During development, specifications may change and consequently ATPs may change to ensure that methods remain suitable for intended purpose.

		It was proposed that as long as analytical methods favorably deliver against the defined ATP method expectations, methods should remain suitable and could be updated with enhanced regulatory flexibility. Industry feedback, however, has suggested that regulatory submissions that used AQbD concepts have been accepted but without, or perhaps with very limited, reduction in regulatory reporting for method changes (9,16).

	To assess the status of AQbD implementation approximately 10 years after introduction of QbD/AQbD concepts into pharmaceutical analysis (1, 2), the IQ Consortium’s Analytical Quality by Design Working Group (AQbD WG) submitted a survey to IQ member companies. The IQ Consortium’s mission is to advance science and technology by augmenting the capability of member companies to develop transformational solutions that benefit patients, regulators, and the broader R&D community. Eighteen responses from 16 member companies were received to the 34-question survey. The responses represent the use of AQbD in 12 large (>10,000 employees) pharmaceutical or biopharmaceutical companies and four smaller companies.

	More than two-thirds of the companies (69%, or 11 of the 16) had implemented AQbD in some way (

). Not surprisingly, the large pharma companies had a higher implementation ratio (83%, or 10 of 12 companies) than the smaller pharma companies (25%, or 1 of 4 companies). Stated differently, 91%, or 10 of the 11 companies, implementing AQbD concepts were from large companies (although the large companies had significantly more survey responses). Of the varied company responses, implementation of AQbD concepts occurred for both large and small molecules as many companies indicated application for both small and large molecules with very little difference in approach.

	For those companies that have implemented AQbD concepts, several business drivers were noted. Improved method performance was the most important driver. For those five companies that had not implemented AQbD efforts into their development programs, concerns regarding increased cost and/or resource investments with no or low benefit were the primary reasons.

	Technical details and challenges with implementation

	Of the 11 companies that have implemented AQbD concepts, the respondents were almost equally split between implementation just recently started and those who have been practicing AQbD concepts for an extended time. Of the five companies not having implemented AQbD, three indicated that they are just beginning. Interestingly, and not surprisingly, most companies practice AQbD concepts during late-stage development (>80% in Phase III) and commercialization, versus earlier development stages, to obtain maximum value from their investment efforts. 

 shows a breakdown of AQbD implementation during the drug development lifecycle for the 18 respondents. This outcome is clearly consistent with the greatest value voiced from AQbD implementation: achieving robust methods, better validation packages, and much improved method knowledge during development. While not captured in 

, comments toward utilization indicate the application of subsets of AQbD tools (e.g., modeling) by some companies in earlier development stages (Phases I and II) but with less systematic rigor than in later phases for better knowledge development.

	The survey also clearly indicated that the primary focus for implementation of AQbD concepts is on drug substance and drug product methods with much lower application to the testing of other materials (

). Interestingly, and perhaps not surprisingly, respondents have applied AQbD concepts almost equally between chromatographic and non-chromatographic methods and clearly do not apply such concepts to compendial methods.

	Eight respondents stated that there was time saved in developing robust methods, while six respondents thought otherwise. Lifecycle management of methods was thought to take longer with a multivariate AQbD-based approach, therefore requiring additional resources. However, the developed methods were thought to be more robust, and hence fewer issues encountered during routine use.

	The majority of companies in the survey leveraged commonality amongst methods to gain AQbD implementation efficiencies. Two examples included the use of generic risk assessments based on method types (e.g., single risk assessment templates for reversed phase-high-performance liquid chromatography [RP-HPLC] assay methods) and the use of standardized approaches to method development, such as the use of standard chromatographic column screens. In addition, automated software systems were used to offset increased resource demand to develop methods. Thirteen out of 15 respondents developed methods using various commercially available software packages such as Fusion AE, DryLab, ChromSword, and ACD/AutoChrom. Respondents have applied statistics (92%) and experimental design (100%) to improve the efficiency AQbD implementation. This is consistent with numerous examples of models and logically designed experimental screens found in the literature in an effort to promote development of robust methods under an AQbD moniker (17–20). A simple design of experiment (DoE) approach was the most widely used experimental design tool, especially for robustness testing. Modeling and simulation tools were routinely used by some companies (64%) for AQbD application. Of note, most respondents have not leveraged any other approaches, other than the tools mentioned, to offset any potential increase in resource demand for AQbD implementation.

	Regarding technical guidance (e.g., the issuance of departmental guidelines or standard operating procedures [SOPs] for implementation of AQbD concepts), six of the 16 companies noted the existence of a departmental guideline or SOP on AQbD processes. Those six companies who had implemented AQbD guidance represent about half of the companies that have implemented AQbD concepts. The comments on this question suggested that some groups felt specific guidance on QbD implementation was not necessary.  In fact, some responses indicated that QbD aspects of analytical work were directly documented alongside or within those work packages (e.g., method development or validation) rather than as a separate entity.

	As companies consider implementation of AQbD concepts, most responders (13 of the 18 responses, or 11 of the 16 companies) have acknowledged practice of at least some AQbD concepts in their organization. As briefly mentioned above, there were several business drivers for companies to consider for implementation of AQbD concepts in their method development efforts. Of the member companies surveyed, 67% responded that the most important business driver was the expectation of improved performance of analytical methods and robustness of these methods, and 43% indicated this business driver produced the largest return on their investment of AQbD practices. Member companies also indicated other reasons that offer a business incentive: improved knowledge of analytical methodology, regulatory relief with operational flexibility, development of consistent processes for method management, and facilitating lab-to-lab transfers (see 

	However, 31% of the company responses (5 of 16 companies) had not yet implemented AQbD concepts as illustrated in 

 (though three were just beginning).  One of the appreciable concerns mentioned as a potential barrier to advancement involves additional investment cost (e.g., resource costs) with perceived limited value. Respondents were asked whether there was an additional resource requirement in their area associated with analytical QbD. The answer was predominantly yes (79% of respondents). These additional investments are often framed in terms of enhanced statistical support, enhanced infrastructure to support risk assessments and QbD programs, and lack of clear guidance for efficient implementation of new processes.

	For some companies, the value of well-designed method development studies to define and ensure robust methods for intended purpose has been accomplished without the formal implementation of many AQbD-associated concepts (e.g., definition of an analytical target profile and performing formal risk assessments). For those implementing AQbD approaches, focusing application of QbD principles to methods of high importance in the control strategy and/or to those that are particularly complex and therefore of higher risk are appropriate to manage resource investments. It is not felt that applying QbD to every analytical method in a control strategy is a sensible use of resources. The additional level of method understanding and control provided by QbD should be applied following a risk-based approach so that resources are not wasted on well-defined, general compendial methodology. Additionally, cultural barriers also can inhibit adoption of newer practices.

	One of the initially touted potential benefits of AQbD submissions included regulatory flexibility surrounding analytical controls and robustness of the control strategy based upon enhanced knowledge gained by QbD efforts. However, this regulatory driver has not materialized as initially proposed during the first several years of QbD-based submissions.  

 affords a summary of hurdles toward AQbD implementation.

	One of the benefits of adopting a QbD approach is highlighted in ICH guidance: “Areas where the demonstration of greater understanding of pharmaceutical and manufacturing sciences can create a basis for flexible regulatory approaches.  The degree of regulatory flexibility is predicated on the level of relevant scientific knowledge provided” (1). Following on from the adoption of this guidance, interested stakeholders began to think about the concept of regulatory flexibility as applied to analytical methods that have been designed, qualified, and continually verified using QbD-derived approaches. Survey responses related to implementation, filing practices, and regulatory responses are discussed below. While most survey responders concluded that the regulatory aspects were not the main benefit from adopting a QbD approach to analytical methods, other benefits included greater method understanding leading to better method controls. These benefits went some way to justifying the increased effort required. Overall, the survey concluded that benefits could be gained by using QbD tools and processes during the analytical method lifecycle and further discussions on experimental and statistical best practice culminating in the publication of examples, case studies, and ultimately industry guidance are warranted.

	There were four questions relating to regulatory implementation as part of the survey. Initially, the survey asked whether the company had filed analytical methods developed with an AQbD approach. Seven companies, representing both large and small companies, had indicated affirmatively and provided feedback on the questions regarding regulatory agency review of AQbD implementation. The experience of pharmaceutical companies in this area so far seems mixed but more negative than positive. Varied regulatory understanding and expectations as well as a lack of clarity on what level of detail should be gathered and provided was noted. Responding to whether the AQbD regulatory review was successful, three companies had successful reviews and three had either pending submissions or submissions not filed yet.

	There was a variety in the level of detail of AQbD filings. In increasing detail, responses ranged from:

		Outline of risk assessment approach and description of analytical studies

		Analytical target profiles, risk assessments, and statistical analyses.

	Companies are using various approaches, likely based on historical successes. This is inefficient because it leads to more work for the applicant to decide on the approach and to address questions during review. It also leads to more work for reviewers who must adjust to different approaches for presentation. One favorable approach has involved companies including a “change appendix” or protocol in the filing describing the criteria to allow changes outside of the validated operating range. Such protocols are advantageous in gaining prospective approval and assist in planning for additional resources.

	AQbD information was submitted to several countries and positive feedback was received with the exception of using ATP for greater operational flexibility. Regulators preferred the term Method Operable Design Region (MODR) to Analytical Design Space to avoid confusion with process and product design space. Some of the regulators commented on the lack of robustness testing. A few of the respondents indicated successful filing of AQbD methods in the United States, the European Union, Canada, and Switzerland mainly for marketing applications. Authorities responded, asking to clarify the information presented. For cases where AQbD methods were developed for drugs in early/mid stages of development, the method details were not presented in the submission documents, and therefore, no feedback was received.

	The regulatory concerns are best summarized in 

.  Five of eight respondents (62%) said they had not seen method flexibility/regulatory relief as a great value from the AQbD information they provided. The increased flexibility, which was supposed to be gained from the analytical target profile approach to method registration, has not been realized and is likely to be a continued area of negotiation. The use of method operable design regions to gain operational flexibility for an individual method is also an area that requires clarity around regulatory expectations.

	With respect to including AQbD information in filings, respondents were evenly split between those who do and do not include AQbD information. It was commented that the approach was well received in the United States, Europe, and Japan, perhaps in line with the fact that these regions adopt new regulatory practices more quickly.  However, global method changes remain challenging as part of an efficient overall method lifecycle management strategy at the moment.

	In summary, results from a 34-question survey indicate that a majority of responding companies have adopted AQbD concepts over the past decade.  About half of the companies were just starting implementation. A clear driver for adoption of AQbD principles has involved the development of more robust analytical methods, and adoption of varied tools that enhance experimental design and modeling are evidence of commitment toward that goal. Furthermore, most value is found with implementation in the late-development stages and commercialization portions of an asset for best return on investment. The results of this survey were consistent with a separate informal AQbD adoption survey in 2015 of pharmaceutical colleagues (21). In that November 2015 meeting of representatives from approximately 25 biopharmaceutical companies that spanned small-molecule, large-molecule, and vaccine-related organizations, clear interest was shown for advanced discussions of AQbD principles.  More robust analytical methods and enhanced regulatory flexibility (average score of 4.7 out of 5, with 5 being the highest rating) were identified as drivers for interest in the development of AQbD case studies to advance discussions on the topic.

	There have been certainly technical, business, and regulatory hurdles toward implementation. While reduced regulatory reporting requirements were desired by some survey respondents for streamlined analytical method changes through commitment of ATPs in a QbD-based submission, that benefit generally has not been realized. Current guidance from regulatory authorities on mechanism and requirements for changing analytical methods and divergent regulatory requirements between regulatory authorities continue to challenge efficient analytical method lifecycle management. Lengthy and cumbersome processes exist to globally change analytical methods post-licensure, and foster lack of adoption of modern methods/technologies even as analytical technologies/instrumentation become old and obsolete. Despite this obstacle, implementation of AQbD principles offers a potential operational benefit that encourages further interest and application.

	1. FDA, Pharmaceutical CGMPs for the 21st Century-A Risk Based Approach, 2004.
	2. P. Borman, et al., 

., 31 (10) (October 2007).
	3. J. Ermer and P. Nethercote, 

, 36 (10) (October 2012).
	4. S. Colgan, et al., 

, 38 (9) (September 2014).
	5. T. Graul et al., “Quality by Design for Analytical Methods,” in 

 D. Am Ende, editor (2011), 545-562.
	6. M. Pohl et al., 

(Aug 27, 2013).
	8. S. Chatterjee, “QbD Considerations for Analytical Methods-FDA Perspective,” presentation at IFPAC Annual Meeting Baltimore, January, 25, 2013.
	9. M. Schweitzer et al., 

, 34(2), 52- 59,(Feb 2010).
	10. ICH, Quality Guideline Q8(R2) 

 (ICH, August 2009).
	11. ICH, Quality Guideline Q9 

(ICH, November 2005).
	12. ICH, Quality Guideline Q10 

 (ICH, June 2008).
	13. G. Martin et al., “USP Stimuli to the Revision Process on Lifecycle Management of Analytical Procedures: Method Development, Procedure Performance Qualification, and Procedure Performance Verification,” USP-PF, 39 (5) (2013).
	14. K. Barnett, et al., “Analytical Target Profile: Structure and Application Throughout the Analytical Lifecycle,” Stimuli to the Revision Process, 

42 (5), (2016).
	15. G. Martin, et al., Proposed New USP General Chapter <1220>: The Analytical Procedure Lifecycle, USP  website, 2016 (http://www.usp.org/usp-nf/notices/stimuli-revision-process-proposed-new-usp-general-chapter-analytical-procedure-lifecycle), to also be published in 

 43(1), (2017).
	16. EMA, “EMA-FDA pilot program for parallel assessment of Quality-by-Design applications: lessons learnt and Q&A resulting from the first parallel assessment,” Aug. 20, 2013, EMA/430501/2013.
	17. C. Hubert et al., 

, 1395, 88-98 (2015).
	18. G. Reid et al., 

36 (18), 2612-2638 (2013).
	19. Miller et al., 

, 52(35), 12269-12284 (2013).
	20. J. Kochling et al.,

, 125, 130-139, (2016).
	21. T. Schofield et al., “AQbD Industry Teleconference Meeting 06-Oct-2015” and associated “AQbD Case Study Industry Telecon Meeting Minutes 10-6-2015” with informal survey responses from >25 companies interested in AQbD and representing large and small-molecule product offerings as well as vaccines. 

 This article was developed with the support of the 

International Consortium for Innovation and Quality in Pharmaceutical Development

 (IQ). IQ is a not-for-profit organization of pharmaceutical and biotechnology companies with a mission of advancing science and technology to augment the capability of member companies to develop transformational solutions that benefit patients, regulators and the broader research and development community.


	Vol. 41, No. 4
	April 2017
	Pages: 52–59

	When referring to this article, please cite it as M. Argentine et al.,"Evaluating Progress in Analytical Quality by Design," 

The authors present the results of a survey of small- and large-molecule pharmaceutical and biopharmaceutical companies on implementation of Analytical quality by design concepts.

Evaluating Progress in Analytical Quality by Design

To ensure the quality of drug products and patient safety, pharmaceutical companies need to manage and justify changes in chemistry, manufacturing, and controls (CMC). Analytical methods are integral parts of CMC, and some common reasons to change analytical methods used in regulated environments include applying new analytical technologies and accommodating changes in chemical or formulation processes. When changes are made to analytical methods, pharmaceutical companies need to compare the new method and the existing method, and to demonstrate that the new method will provide equivalent or better performance to that of the existing method. 

There are two concepts related to comparing two analytical methods for the same test: analytical method comparability and analytical method equivalency. Chambers et al. considered that analytical method comparability refers to studies to evaluate similarities and differences in method performance characteristics between two analytical methods (i.e., accuracy, precision, specificity, detection limit, and quantitation limit), while analytical method equivalency is a subset of analytical method comparability and refers to studies to evaluate similarities between two analytical methods in regard to generating results for the same sample (1). In other words, analytical method equivalency evaluates whether the new method can generate equivalent results to those from the existing method. In a recent publication, Chatfield et al. suggested another way to differentiate these two concepts, where analytical method equivalency is restricted to a formal statistical study to evaluate similarities in method performance characteristics (2). In a United States Pharmacopeia (USP) stimuli paper, Hauck et al. also discussed other approaches to assessing equivalence such as “result equivalence” (3).

Unlike analytical method validation, where clear regulatory guidelines such as International Conference on Harmonisation (ICH) Q2, are available, there is little regulatory guidance on how or when to perform analytical method comparability or equivalency. In a draft guidance published by FDA in 2003, 

Comparability Protocols-Chemistry, Manufacturing, and Controls Information

, the agency stated that proper validation is required to demonstrate that the new analytical method will provide similar or better performance compared with the existing method. On the other hand, the agency also stated that whether an analytical method equivalency study is needed and how to perform it will depend on the extent of the proposed change, type of product, and type of test (e.g., chemical, biological) (4). Another relevant document on analytical method comparability is 

 General Chapter <1010> “Analytical Data-Interpretation and Treatment,” which was first published in 

In this General Chapter under the section of Comparison of Analytical Methods, statistical-based approaches to compare method precision and accuracy are discussed with some examples. Most recently, an FDA draft guidance on analytical method validation was issued in 2014 (6), and the concept of analytical method comparability is discussed. However, in this draft guidance, the use of formal statistical studies is recommended whenever a change is made to an analytical method that has stability indicating properties. While this approach is valid and rigorous, the draft guidance does not consider less formal but suitable alternatives for method comparison assessments by leveraging a quality- and risk-based assessment strategy. 

Based on a 2003 Pharmaceutical Research and Manufacturers of America (PhRMA) workshop, Chambers et al. published a position paper on analytical method equivalency (1). The authors summarized industry practice at the time and provided perspectives on when and how analytical method equivalency should be performed. The authors noticed that a wide range of different practices were used by pharmaceutical companies for analytical method equivalency, including only validation of the new method, side-by-side result comparison against pre-defined acceptance criteria, as well as formal statistical demonstration. Such wide variations are clearly not preferred by either regulatory agencies or the industry and may lead to delay in regulatory review. 

The International Consortium for Innovation and Quality in Pharmaceutical Development (IQ) is a pharmaceutical industry association formed in 2010 with the mission of advancing science-based and scientifically driven standards and regulations for pharmaceutical and biotechnology products worldwide. In 2011, an IQ working group, Analytical Method Comparability, was formed to assess current practices by IQ member companies and to recommend best practices on analytical method comparability for analytical methods used in regulatory environments. Harmonized approaches on analytical method comparability, if in existence, will not only improve industry compliance to ensure product quality and patient safety, but also align with quality-by-design (QbD) principles and may also reduce the regulatory filing burden on method change control, which in turn may encourage innovation in pharmaceutical analyses. 

As a starting point, the working group purposely limited its initial scope to changes in high-performance liquid chromatography (HPLC) assay and impurities methods in registration and post-approval stages. The rationale for this decision is that new developments in instrumentation and column technologies led to the birth of ultra-high pressure liquid chromatography (UHPLC) (7). An HPLC impurities analysis that used to routinely take 30 min or more to perform by conventional HPLC can now be achieved within a few minutes by UHPLC with better sensitivity and less solvent consumption. Though UHPLC has witnessed increasing popularity by pharmaceutical companies for new products, implementation of UHPLC in legacy products has been slow, partially due to the heavy regulatory filing burden upon method change control. CMC related changes, including changes in analytical methods, are regulated most stringently in registration and post-approval stages. While HPLC assay and impurities methods were the starting point, it is reasonable to anticipate that the general concepts developed in this position paper can be applied to other analytical methods. 

The working group conducted a survey on analytical method comparability among IQ member companies, and the results are summarized in the next section. Based on the survey results and discussions among working group participants, the working group recommends a risk-based approach for analytical method comparability of HPLC assay and impurities methods in registration and post-approval stages. The views in this article are those of the authors and do not reflect the official policies of their respective companies.


A survey of industry practice on analytical method comparability for HPLC assay and impurities methods in registration and post-approval stages was carried out among IQ member companies. The survey contained 17 questions covering the following aspects with regard to industry practice on analytical method comparability: background and internal governing documents, regulatory interactions, general practice, quality assurance (QA) and statistician involvement, and detailed procedures. Nine questions were required to be answered as a “Yes” or “No” response, with the option to provide additional written details; written responses were required for the remaining eight questions. A total of 19 responses were received from 15 IQ member companies. Three companies provided multiple responses from different divisions within the same company. The survey results were discussed during subsequent working group meetings. Though this is a relatively small survey and the percentage numbers undoubtedly have a large margin of error, the authors believe that the results, described in the following section, provide a meaningful picture of current industry practice on analytical method comparability for HPLC assay and impurities methods. 

Background and internal governing documents


With regard to the definitions of analytical method comparability and analytical method equivalency, 68% of participants thought that these terms represented two different concepts. From subsequent working group discussions, it was clear that most participants agreed with Chambers et al. that analytical method equivalency is the evaluation of whether equivalent results can be generated by two different analytical methods; analytical method comparability is a broader and less defined term (1). Meanwhile, some participants also indicated that analytical method equivalency should be restricted to a statistical demonstration of equivalence, as described by Chatfield et al. (2).

With regard to internal governing documents, 79% of participants indicated that there are no specific standard operating procedures (SOPs) or guidelines on analytical method comparability within their company. On the other hand, 53% of participants mentioned that there are some general discussions on analytical method comparability within their company’s change control policies or SOPs. Some participants, for example,  indicated that it is part of their risk assessment workflow to evaluate if an analytical method equivalency study should be performed when a change is made to an analytical method.


With regard to regulatory interactions, 68% of participants indicated that they have had successful regulatory reviews of analytical method comparability packages for a registration filing or post-approval change. Additional written comments showed that a complete analytical method comparability package typically includes method information, method validation, equivalency data, and the reason for the change. Meanwhile, 47% of participants indicated that they have received questions on analytical method comparability from health authorities. Additional written comments suggested that regulatory authorities have expectations that analytical method equivalency needs to be demonstrated when changes are made to an analytical method; however, the feedback also indicated that there were no consistent requirements on what kind of data package should be provided. In general, more data are required when the method change is more significant.

In one example, when an HPLC assay method was replaced with an UHPLC assay method, method equivalency was demonstrated using a side-by-side comparison of both methods for three lots of material; this approach was accepted by FDA. In this example, the change was considered minor as the separation mechanism (i.e., reversed-phase HPLC) was not changed. In another example, however, when a normal-phase HPLC assay and impurities method was replaced by a reversed-phase HPLC method during registration stability testing, several questions were received from FDA requesting detailed explanations due to the significant difference in methodology, which impacted the impurity profile and the specifications. The agency requested detection limits and quantitation limits of all impurities by both methods and full disclosure of the impurity profiles by both methods on all lots tested. To help justify the change, the company also provided overlapping stability data for 3-6 stability time points to demonstrate that the reversed phase HPLC method provided better characterization of the drug product.


With regard to general practices, all participants stated that in addition to validation, they would also evaluate whether an analytical method comparability or equivalency study is needed when a change is made to an analytical method, while 63% of participants specified that not all method changes will need a comparability or equivalency study. From additional written comments, it was clear that risk-based approaches are adopted by most companies when evaluating whether an analytical method comparability or equivalency study is needed for method changes. The risks are typically evaluated based on the types of the changes, and in general, the more significant the change, the more likely a comparability or equivalency study will be needed. 

For example, as indicated by some participants, neither validation nor equivalency will be necessary for compendial HPLC method changes if the changes are within the ranges allowed in the 

 General Chapter <621> “Chromatography.” For non-compendial HPLC methods, such as those included in a new drug application (NDA) filing, no equivalency study will be needed if the change is within the established method robustness ranges. On the other hand, a comparability or equivalency study is typically required if the changes are outside of these ranges, such as a change in liquid chromatography (LC) stationary phase chemistry or a change in detection technique (e.g., from ultraviolet to mass spectrometry). Some companies also evaluated the risks based on the stage of the development. One participant mentioned that equivalency studies are performed for any changes to an HPLC method in post-approval stages.

If an analytical method comparability or equivalency study is deemed necessary, different practices still exist amongst the participants. A majority of participants (74%) agreed that, in addition to validation of the new method per ICH guidelines, a simple side-by-side comparison of representative batch results generated by both methods against pre-defined acceptance criteria (some companies referred this practice as cross-validation or as a bridging study) is usually sufficient. A more conservative practice (16%) is a demonstration of method equivalency through formal statistical analysis. One participant indicated that whether such a formal statistical analysis is necessary will also be evaluated based on the types of the changes as part of the risk assessment workflow. Another less common practice  (5%) is to conduct a full validation of the new method; the two methods are considered equivalent if they are both validated. 


With regard to quality assurance (QA) and statistician involvement, 68% of participants indicated that QA is involved in analytical method comparability or equivalency studies, such as approval of study protocols, while only 26% indicated that statisticians might be involved. One participant indicated that statisticians are involved in the design of the equivalency study, setting acceptance criteria, data evaluation, and report writing; while other participants stated that statistician assistance is only required when it is deemed necessary by analytical scientists.


With regard to detailed procedures, though the survey was on analytical method comparability for HPLC methods, responses from almost all participants were only focused on analytical method equivalency. This suggested that analytical method equivalency (i.e., demonstration of no practically significant bias in mean results) is the primary goal when a change is made to an HPLC method. Most participants did not use formal statistical approaches, but for those who did, statistical approaches are used throughout the study to ensure a statistically meaningful outcome.

An analytical method equivalency study typically consisted of acceptance criteria, study design (sample selection, sample size, and test plan), data evaluation, and documentation. With regard to acceptance criteria, most participants started with generic acceptance criteria derived from internal SOPs on analytical method validation or method transfer, such as 1.5-3.0% for assay, and 10-30% relative for impurities (or an absolute difference, such as 0.03%, for levels close to quantitation limits). Such acceptance criteria could be loosened or tightened based on the specific method and product involved (e.g., product types).

On the other hand, some participants indicated that a method specific criterion will be set based on the statistical evaluation of the precision of the method and the acceptance criteria of the relevant product specifications. With regard to study design, representative batches are typically used, with spiking of impurities if necessary. Some participants also suggested that stability samples over a few time points could be used (cross-over testing), though this practice was discouraged by Chambers et al. to avoid generating concerns of regulatory compliance.

A wide range of different practices exist among participants in terms of sample size (number of batches) and test plans. Some examples include using six replicate preparations of one or two batches, three replicate preparations of three batches in three runs (fresh reference standard preparations for each run), and one preparation for each of 20-30 batches, etc. Some participants indicated that statistical approaches are used to determine sample size (statistical power calculation based on acceptance criterion and method precision) and test plan (evaluation and control of other factors not related to the change). Some of these statistical approaches were discussed by Borman et al. (8). 

With regard to data evaluation, most participants simply compared difference in mean results against pre-defined acceptance criteria, while some participants indicated that a statistical test will be used. Concerning the specific statistical tests, the two one-sided t-test (TOST) is most often used (9), while the two-sample t-test is also used by some participants. With regard to documentation, most participants use a pre-approved study protocol and provide a study report once the study is completed.

In summary, it is fair to say that compared to analytical method validation, analytical method comparability or equivalency is a more science-driven and less regulated field. The survey results show that risk assessments are commonly performed by pharmaceutical companies to evaluate the impact of the changes, including whether an analytical method comparability study is necessary. If analytical method comparability is deemed necessary, different practices exist among pharmaceutical companies on how to perform the study.Though formal statistical approaches are used by some companies to provide a statistical meaningful declaration on whether the two methods were equivalent, a simple side-by-side comparison against pre-defined acceptance criteria remains the most popular approach for changes made to HPLC methods. On the other hand, the survey results also show that though regulatory authorities have expectations that analytical method equivalency needs to be demonstrated when changes are made to an analytical method, there seems to be no consistent requirements on what kind of data package should be provided. In general, more data are required when the extent of the change is more significant.

Overall, the survey results suggest that risk-based approaches should be adopted for analytical method comparability. In other words, whether an analytical method comparability or equivalency study is needed and how it is performed should be decided based on the risks associated with the changes. In the FDA draft guidance on comparability protocols, the agency states that “The need and plan for providing product testing to compare the two (analytical) procedures could vary depending on the extent of the proposed change, type of product, and type of test (e.g., chemical, biological)” (4). This statement suggests that risk-based approaches are expected by FDA as well. In the next section, a risk-based approach for analytical method comparability for HPLC assay and impurities methods in registration and post-approval stages is proposed based on the survey results and the subsequent working group discussions.

A risk-based approach for analytical method comparability


  When a change is made to an analytical method, a risk assessment should be performed to evaluate the potential impact. Though analytical method equivalency (i.e., no practically significant bias in mean results) is an important factor, other method performance characteristics, such as precision, should be evaluated as well. Impact on method performance depends upon the nature of the method as well as the type and the extent of the change. With regard to HPLC, the mechanistic understandings of both thermodynamic and kinetic aspects of HPLC separation processes have been well developed in the past few decades. In the same time period, improvements to LC instrumentation and column technologies have made HPLC one of the most reliable techniques in analytical laboratories around the world. The knowledge of HPLC separation and the reliability of modern HPLC instruments have enabled analytical scientists to better understand the potential impact on method performance from certain types of changes to HPLC methods. Therefore, changes to HPLC methods could be categorized based on the risks associated with the changes (e.g., low risk, medium risk, or high risk). These changes would be accompanied by a commensurate amount of work to assess comparability, as depicted in 

Small   changes to the high performance liquid chromatrography (HPLC) operating parameters within accepted ranges (e.g. 

 <621> or within established   method robustness ranges).

Changes   to the HPLC operating parameters outside of low risk ranges, but not   involving any change to the underlying principles of the existing analytical   method (e.g. column chemistry).

Changes   to the underlying principles of the existing analytical method (e.g. liquid chromatrography-ultraviolet (LC-UV) to   liquid chromatography-mass spectrometry (LC-MS)). 

Low-risk changes are small changes to the HPLC operating parameters, such as changes allowed by USP General Chapter <621> “Chromatography” for a compendial HPLC method or changes within the established robustness ranges in analytical method validation (10-11). Two examples of low-risk changes are the column temperature within ±10˚C or the flow rate within ±50% from the target values of a compendial isocratic HPLC method. In low-risk situations, additional method validation and comparability studies should not be required, as the changes have already been assessed within the initial validation experiments and any impact is generally understood and predicted based upon mechanistic models.  

Medium-risk changes are changes to the HPLC operating parameters outside the established robustness ranges evaluated during the method validation, but do not involve any change to the underlying principles of the existing analytical method. An example of a medium-risk change includes an HPLC column change to a smaller particle size and/or shorter column length or diameter with appropriate scaling considerations to speed up a gradient HPLC method (i.e., conversion from HPLC to UHPLC methodology without appreciable change in separation mechanism). Other medium-risk changes might include changes in column stationary phase from C18 to C8 or a change in organic solvent in mobile phases from acetonitrile to methanol as long as the characteristics of method performance (e.g., specificity, accuracy, precision) remains practically unaffected, which can be confirmed through successful completion of associated method validation activities prior to implementation of any comparability studies.  

Finally, high-risk changes are those changes that either involve a change to the underlying principles of the existing analytical method or a change in a critical method parameter that has high impact to method performance. Some examples of high risk “underlying analytical principle” changes include conversion of a microbiological potency method to a reversed-phase HPLC potency method, a normal-phase HPLC method to a reversed-phase HPLC method, a UV detection to a mass spectrometry detection, or an assay method by HPLC to a near-infrared (NIR) real time release method. 

The HPLC risk categorization examples discussed previously should serve as a general guideline for risk assessment, and it is recommended that each HPLC method change should be evaluated and categorized on a case-by-case basis. Application of QbD principles to analytical methods has been discussed recently (12-16) and can assist method change assessments. Identifying critical method parameters through a comprehensive risk assessment is an integral step in an analytical QbD workflow. The knowledge gained through analytical QbD should also be applied in risk assessment and categorization when a change is made to an existing HPLC method. For example, a change in detection wavelength for an HPLC impurities method may be an example of a medium-risk change or a high-risk change. If all impurities are quantified by calibration with external standards of each impurity, the change should be of medium risk; however, quantification of impurities based upon response-corrected areas could potentially represent a high-risk change if relative response factors are assumed to be 1.0, a common industry practice for impurities with calculated response factors between 0.8-1.2 (11). In such a situation, a change in detection wavelength could lead to an appreciable bias in impurity results. 

For changes in the low-risk category, potential impact on method performance is usually low. The risks will be further guarded by system suitability evaluation, such as the resolution between critical pairs, performed in each HPLC run; therefore, low-risk changes can take place without additional studies. Neither additional validation nor analytical method comparability studies are necessary. This approach agrees with the requirements from USP and ICH.

For any changes beyond low risk, justification and validation of affected method parameters are generally required. Additionally, both health authorities and pharmaceutical companies expect that analytical method comparability or equivalency studies should be performed. A question of interest is whether a formal statistical demonstration of analytical method equivalency is always necessary. Chambers et al. discussed the differences between validation and equivalency, and emphasized that validation alone cannot guarantee equivalence because validation experiments are not designed to detect small bias between mean results  (1). This statement is fair, but it should be emphasized that the risk of non-equivalency decreases as successful validations of both methods have been completed.

For medium-risk changes, where underlying principles remain the same for both methods (e.g., reversed-phase HPLC methods with UV detection), the risk of having a practically significant difference in mean results is rather low when both HPLC methods are successfully validated in the same manner with the same pre-defined criteria. For example, a co-elution of two impurities would potentially fail equivalency, and would fail validation on method specificity as well. In addition, validation studies also confirm improvement or similarities of other method performance characteristics, such as precision. Therefore, the authors recommend that a side-by-side comparison against pre-defined acceptance criteria should be sufficient for medium-risk changes. Though not a formal statistical approach, the side-by-side comparison study should be designed carefully. Acceptance criteria could be based on internal SOPs on analytical method validation or method transfer. Generic limits of 1.0-2.0 % for drug substance assay, 2.0-3.0 % for drug product assay, and 10-30% relative for impurities can be considered as a starting point. Such acceptance criteria can be loosened or tightened based on the specific method involved. Representative batches, preferably covering a wide range of results, should be chosen for the study. Spiking impurities should be considered, if no suitable batches can be identified containing the impurities at appropriate levels (e.g., the specification limit). QA should be consulted if active GMP samples, such as GMP release and registration stability study samples are considered to be used for comparison. Material limitation is usually not a concern at registration and post-approval stages, and typically three batches or more should be used for the study. Factors affecting intermediate precision and/or reproducibility, such as analyst, instrument, and environment, should be included in the study if possible. For example, a common test plan is triplicate preparations of three batches in three runs, with each run using fresh standard preparations, and altering factors of analyst, instrument, and/or environment as well. 

Because high-risk changes usually involve a change to the underlying principles between the existing and the new methods, the risk of being non-equivalent is high, even with successful validations of both methods. In some cases, the validation plan and acceptance criteria between the two methods have to be modified to accommodate the changes to the underlying principles. For example, the specificity evaluation for a NIR assay method will be different (test procedure and acceptance criterion) from that for an HPLC assay method. Therefore, the authors recommend that statistical principles should be followed throughout the study and a formal statistical assessment of the data should be applied for high-risk changes to ensure a statistically meaningful outcome. This includes selection of the acceptance criterion, design of the test plan, data validity evaluation (e.g., outlier testing), and an equivalency test (e.g., TOST). Note the two-sample t-test should not be used as an equivalence test (or as acceptance criteria for a side-by-side comparison) as it is designed to detect a difference of means, not to establish whether the means are equivalent based on a measure of practical importance (1, 9). Besides method equivalency, other method performance characteristics, such as precision, should also be evaluated (though note that a large difference in precision is likely to cause the TOST to fail (9). In other words, an analytical method comparability study should be performed for high-risk changes. Detailed procedures for statistically guided analytical method comparability and equivalency studies can be found in the USP General Chapter <1010> and some external scientific publications as well  (2, 5, 8, 9, 17). Due to the involvement of advanced statistics, statistician assistance should be sought. The proposed analytical comparability activities for a specific HPLC risk categorization does not preclude a more stringent assessment being applied (e.g., if the analytical method is to be applied in a situation where even a very small difference in means could be problematic). 


Analytical method comparability or equivalency remains as a more science-driven and a less regulated field, and different practices currently still exist among pharmaceutical companies. Harmonized practices will not only improve industry compliance to ensure product quality and patient safety, but also reduce the regulatory filing burden on method change control, which in turn will prompt innovation in pharmaceutical analysis. In this position paper, the IQ working group on analytical method comparability recommends a risk-based approach for analytical method comparability for HPLC assay and impurities methods in registration and post-approval stages. It is the working group’s hope that this paper can help to stimulate further discussions among the industry and health authorities to define acceptable industry practices for assessing analytical method comparability or equivalency in general.


This position paper was prepared with the support of the International Consortium for Innovation and Quality in Pharmaceutical Development (IQ). IQ is a pharmaceutical industry association formed in 2010 with the mission of advancing science-based and scientifically driven standards and regulations for pharmaceutical and biotechnology products worldwide. Visit www.iqconsortium.org for more information.

., 29 (9), 64-80 (2005).
  2.	M.J. Chatfield et al., 

Quality and Reliability Engineering International,

 27 (5), 629-640 (2011).
  3.	W.W. Hauck et al., 

Draft Guidance, Comparability Protocols-Chemistry, Manufacturing, and Controls Information

 31, General Chapter <1010>, “Analytical Data-Interpretation and Treatment,” 452-464.
  6.	FDA, 

Draft Guidance, Analytical Procedures and Methods Validation for Drugs and Biologics

 (Rockville, MD, February 2014).
  7.	B. Kleintop and Q. Wang, “Practical Application of Very High-Pressure Liquid Chromatography across the Pharmaceutical Development-Manufacturing Continuum,” in 

Characterization of Impurities and Degradants Using Mass Spectrometry

. (B. Pramanik, M S. Lee, G. Chen, Eds., (John Wiley & Sons, Inc., 2011) pp. 213-229.
  8.	P.J. Borman, et al., 

, 81 (24) 9849-9857 (2009).
  9.	G.B. Limentani et al., 

, 77 (11) 221 A-226 A (2005).
  10.	ICH, ICH Q2 (R1) 

Validation of Analytical Procedures: Text and Methodology

 31, General Chapter <621>, “Chromatography,” 268-275.
  12.	P.J. Borman et al., 

., 34 (4), 72-82 (2010).
  13.	P.J. Borman et al., 

., 34 (10), 142-152 (2010).
  14.	P.F. Gavin and B.A. Olsen,. 

Journal of Pharmaceutical and Biomedical Analysis

, 46 (3), 431-441 (2008).
  15.	S. Karmarkar et al., 

, 49 (6) 439-446 (2011).
  16.	M. Schweitzer et al., 

, 34 (2), 52-59 (2010).
17.	M.J. Chatfield and P.J. Borman, 


Hafez Abdel-Kader is a director at Sanofi; Mark Argentine is a senior research advisor, and Jeff Hofer is a research advisor, both at Eli Lilly; Nancy Benz is a director at AbbVie; Rick Burdick is a quality engineering director at Amgen; Marion Chatfield is a manager statistician at GlaxoSmithKline; Frank Diana is a vice-president at Endo Pharmaceuticals; Hui Fang and Yuan Huang are both senior scientists at Eisai; Shreekant Karmarkar and Lakshmy M. Nair are both senior research scientists at Baxter; Theresa Natishan is a director at Merck; Andrea M. Pless is an associate director at Teva; Qinggang Wang* is a principal scientist at Bristol-Myers Squibb, qinggang.wang@bms.com; and Zeena Williams is a senior associate director at Boehringer Ingelheim. * To whom all correspondence should be addressed.

A risk-based approach is recommended for analytical method comparability for HPLC assay and impurities methods.

Analytical Method Comparability in Registration and Post-Approval Stages: A Risk-Based Approach

Please refer to the full online version of this article, located 

The authors describe a novel approach for assessing method robustness. This article contains online-bonus material and was copublished with Pharmaceutical Technology Europe.

Reduced-Method Robustness Testing of Analytical Methods Driven by a Risk-Based Approach

This downloadable Failure Mode Effect Analysis tool accompanies the article titled 'The Application of Quality by Design to Analytical Methods' by Phil Borman et. al.