Defining and Managing Risk for Projects with a Contract Manufacturing Organization

This article proposes a block diagram process model that can be used to ensure that the information and input from each part of the organizational elements are collected and summarized according to standard categories of work tasks in GMP-regulated organizations. It briefly addresses some of the means to achieve quantitative and qualitative risk evaluations, including the use and limitations of the risk table for summarizing risk factors and management options. It emphasizes the importance of quantitative assessments where possible and the scientifically valid and defensible collation and analyses of the data.

In 2002, the US Food and Drug Administration pointed out the need for a new paradigm in drug development. As part of its description of a new order for current good manufacturing practices (CGMPs) regulations, the agency introduced its desire to begin using a risk-based approach for drug development, manufacturing, and control. The final report for this initiative was issued in 2004 (1).

"Risk" is now a synonym for "opportunity" in the pharmaceutical industry. Risk definition, assessment, mitigation, and elimination for all manufacturing and control processes should become the basis for thinking about product development and life-cycle management for every product. When a company can show—with supportive technical reasoning and meaningful, carefully and correctly evaluated data—that it completely understands the manufacturing process for its product and can mitigate or eliminate altogether the risk factors for failure through materials and process controls, then making future changes in manufacturing procedures becomes much easier to perform, document, and report. True assurance of consistency and value for each successive product lot will ensure greater patient confidence, disease treatment with minimum excursions, and overall cost reductions in terms of surveillance, documentation preparation and review, and industrial waste generation (i.e., failed batches, recalls, and reprocessing).

This understanding must extend to all of the materials, facilities, personnel practices, and procedures that contribute to the manufacture of the product. This knowledge applies not only to work done in-house but also to work performed by a contract manufacturing organization (CMO). For products already on the market, it may be difficult and cost prohibitive to establish retrospectively the scientific understanding for the materials, equipment, and procedures used to establish a complete risk assessment and risk-management action plan (RiskMAP) (2). Nonetheless, early development is the ideal time to start accumulating knowledge and understanding about any new product.

The concept of risk assessment and management applies not only to technical matters such as material quality assessment and lot-to-lot consistency, analytical control capability, machine performance and control, influence of facilities, and manufacturing procedures. Risk assessment also applies to interactions between organizations such as those for the procurement of starting materials and those for the procurement of manufacturing and control services for the production of the product.

A systematic and objective process for performing the risk assessment and preparing or executing the RiskMAP must be invoked by each organization. This article proposes a block diagram process model that can be used to ensure that the information and input from each part of the organizational elements are collected and summarized according to standard categories of work tasks in GMP-regulated organizations. The author also briefly addresses some of the means to achieve quantitative and qualitative risk evaluations, including the use and limitations of the risk table for summarizing risk factors and management options. It emphasizes the importance of quantitative assessments where possible and scientifically valid and defensible data collation and analyses.

Definitions of risk

Risk is a small word with a large connotation. It is used in many contexts in the pharmaceutical industry, but primarily in a nonquantified format in each case. To achieve the highest usage level of risk-based development and manufacturing as intended by FDA, industry must develop quantifiable risk algorithms for each function and process within the function.

Definitions of risk vary and give little guidance for quantification. For example, some dictionary definitions include:

• possibility of loss or injury;

• a dangerous element or factor;

• the degree of probability of loss (3).

Only the latter definition provides an idea that risk can be calculated according to a probability algorithm. Nonetheless, all the definitions connote a negative outcome whether or not it is quantifiable, and that further implies an element of action or process that might result in a negative result.

An Internet search yielded the following definitions for "risk":

• potential future harm that may arise from some present action;

• an expected after-the-fact level of regret (4).

These definitions also imply the negative consequence of a process or action.

The banking industry in 2004 recognized and defined risk associated with organizational interactions in the following definition:

Operational risk is the loss resulting from inadequate or failed internal processes, people and systems, or from external events (5).

FDA recently provided guidance with respect to the need for an action plan to identify, evaluate, and manage such consequences of risk in the following definition:

Risk Management Action Plan [FDA RiskMAP] is the strategic safety program to meet specific goals and objectives to minimize risks for a product while preserving its benefits (2).

Thus, there is an expectation that a pharmaceutical company will proactively and systematically identify risks that might negate some deliverable quality attribute of a product and have a program in place to prevent or minimize these identified risks. Although negative impact on product quality will be linked most directly to the technical aspects of production and control, one must also consider the effect of negative organizational performance leading to decreased product quality as a potential true root cause.

Facilities, production machines, and analytical equipment require a human interface in their use to produce and control the medicinal product. Personnel and procedural systems, whether in the sponsor organization or in any of the supportive supplier organizations, affect the final quality of that medicinal product. Therefore, the RiskMAP must take such organizational elements into account, at least implicitly, as well as the technical elements. These are paraphrased from the FDA guidance as follows and apply throughout the full life cycle of the product.

FDA RiskMAP:

• Goals target the achievement of outcomes to mitigate known safety risks.

• Objectives are the pragmatic, specific, and measurable steps taken to achieve a goal.

• The program requires evidence-based risk identification, assessment, and characterization throughout the product's life cycle.

Medicinal product development and registration are a continuum of tasks. This process was recently described in a pictorial that shows the pathway leading from product concept through design through development and execution (6, see Figure 1). This pictorial clearly shows the concept of quality by design for any medicinal product and the need for proactive management of all suppliers, whether they supply materials or services. It shows that each of the functional areas must be evaluated with respect to "evidence-based risk identification, assessment and characterization" (2).

Figure 1: Process pathway from product concept and definition to market supply. This pictorial has been modified to indicate that risk is a factor in each of the segments of the development pathway. Source: ref. 6.

Three schematic elements, "manage supplier quality," "build the product to customer needs," and "control process and product conformance," are the most pertinent elements for any discussion about sourcing product manufacturing and analytical control testing at a CMO.

Mental models

Humans have the ability to conceptualize fairly complex systems and tools. Such ideas are termed mental models and apply to physical objects, to regulation of interactions between people either in the allowance or constraint of actions, and to processes for achieving intellectual objectives (e.g., solving quadratic equations) or building useful items.

The identification and assessment of risk and development of a risk management plan is a process. Such evaluation determines the degrees of freedom that one has in the performance of that particular process. It is fairly clear that the risks are greater in number and potentially more complex when a sponsor uses a CMO whether for manufacturing operations, control testing, warehousing and distribution, or all of these. The organizational complexity and inherent risk of negative results are great within the sponsor company and can be even more complex and risky when the sponsor chooses to use another company to supply materials or services.

Our mental model of CGMPs has evolved over time. A pictorial view of one mental model of the process for product development and marketing is shown in Figure 2.

Figure 2: A mental model of the evolution of the product development process over time. White boxes indicate a concept just before the introduction of good manufacturing practices (GMPs). White and green boxes indicate a concept at the introduction of GMPs. White, green, and blue boxes indicate an addition of qualification and verification to developmental aspects of current good manufacturing practices (CGMPs). All boxes indicate contemporary CGMPs, including the RiskMAP concept.

GMP regulations were first introduced by FDA in 1963 and strengthened during the next decade with a major revision in 1978 that increased requirements for review and rationalization of processes and data and associated validation. GMPs were originally developed for commercial products but also were implicitly applicable to the production of clinical trial supplies for products that were not yet approved for the market place. The application of CGMPs to clinical trial materials was explicitly expressed by FDA in its guidance issued in 1991 (7). When FDA decided in the early 1990s that something such as validation was needed for product development, industry responded that this was not a realistic option and devised the concepts of qualification and verification (8). The concept of risk-based development and life-cycle management was added in 2002 in the FDA statement on CGMPs for the 21st century (1).

Quantitative versus qualitative risk assessment

The quantitative assessment of potential negative outcomes associated with a risk factor, after the risk factor has been identified, is the best approach. This method is not always possible or applicable, however. When quantification is possible, the appropriate statistic algorithm also must be used for analytical representation. Not all outcomes can be represented as a Gaussian distribution. Outcomes may be singular in nature or else the distribution of outcomes may be skewed. Some negative outcome distributions may not be describable in a simple mathematical form. A detailed understanding of the root causes of the negative outcomes associated with any particular risk factor must be achieved. Then, an accurate analytical representation of these negative outcomes and the potential for experiencing such negative outcomes for any product must be developed and used.

The physical chemistry of each material used in the manufacture of a product must be understood and controlled. The effect of the various machine-running parameters on the manufacture of the product must be understood in relation to the intended product. And, the interrelation of the material characteristics and specifications with the operating conditions, which also includes the facility environment, operating conditions and controls, must be well understood and managed to manufacture a product to meet its quality expectations. The quality expectations are defined by a multidimensional response surface of quality-attribute specifications for each of the materials and equipment or facilities factors. The limits of the quality specifications define the desired response surface of all parameters that must be met for the quality of the product to "meet specifications." This response surface lies within a multidimensional space and must be established during product development. Design space is the current terminology for this response surface. Quality by design is the current terminology for achieving the repeatable manufacture of a product for which quality parameters will always be achieved and lie, in the aggregate, in that design space. The definition and control of design space is a large topic that will be addressed elsewhere by a consortium of universities and FDA (9). Nonetheless, the assessment and management of risk to minimize or eliminate the negative impacts will be an important aspect for the full definition of any design space. Therefore, a methodology and process for performing this assessment and describing the management plan must be developed in parallel.

Understanding organization risk. Although there is every reason to believe that we can achieve an understanding of the risk factors and root causes for failures associated with physical, chemical, and apparatus or facilities parameters, there is not such confidence for understanding and describing failures associated with organizational interactions in a quantitative and predictable way. But, industry must make an attempt to categorize and identify organizational risks related to human interactions as well as the technological risks associated with material and machine interactions. For example, the development of a quality agreement as part of the contract development between a sponsor and CMO seems risk free until one begins to discuss the definitions and understanding of quality terms by the personnel in the two organizations.

The risk of failure can be exacerbated by issues such as a lack of completeness in the determination of true root cause of an error or insufficient investigation suitable for resolution of out-of-specification results during stability evaluation. Failure to recognize any result as suspect may lead to product failures and recalls. Communication and training are very important in any organization and the concepts become critical for interactions between two organizations. Tracking of successes and failures in product performance also should include an evaluation of the successes and failures in information exchange and communication for understanding within and between the organizations. Just as there are specifications, acceptance criteria, and critical control criteria for the material parameters and machine or facility operating conditions, so should the specifications, acceptance criteria, and critical control criteria for organizational interactions be identified and managed.

Quantitative estimates for negative outcomes can be developed from existing in-house data or from benchmark evaluations. For example, if moisture content for a granulation is a risk factor for a product, then the control of moisture content of all of the ingredients, including active pharmaceutical ingredients (APIs), excipients, and added granulation solvents, may contribute to successful product manufacture. If the moisture content of an excipient is a known contribution to failure, then finding a supplier that can deliver the material with the appropriate moisture content every time is crucial. The risk of failure by that supplier can be estimated from the number of times the sponsor (in-house data) or other sponsors (benchmark data) received material from the supplier that did not meet the moisture content criterion. A similar approach can be used for establishing a quantitative representation of risk for particle-size distribution, particle shape, related substances, and other parameters. Similar concepts can be defined for API and packaging components as well.

Formulation scientists often evaluate machine-running parameters for their formulations but often do not put an assessment of risk into this evaluation. The critical control criteria for finished goods such as content uniformity or dissolution for solid oral products can be affected by machine-running parameters, as well as by material characteristics. Compression running speed, compression dwell time, and segregation patterns for the granulation resulting from bulk granulation feed reservoir configuration and machine vibration are well recognized as factors. Currently, industry does not have good methods for quantifying the potential for negative results for these known risk factors in any given formulation. Developing such methods should be the goal for future product development.

Ranking risks. Once identified, risks often must be ranked to know which are most critical and must be taken care of first. This ranking is often done in a quasi-quantitative way with the establishment of the risk priority number for each risk:

Risk priority number = (S)(O)(D)

• Severity is how serious is the potential failure mode for the customer (numeric value between 1 for least serious and 5 for very serious);

• Occurrence is the likelihood that a cause will happen and will result in failure mode during the intended life and use of the product (numeric value between 1 for lowest likelihood and 5 for highest likelihood);

• Detection is the likelihood that current controls (design and process) will not detect the cause of the failure mode or of the failure mode itself (numeric value between 1 for good detection capability and 5 for poor detection capability).

This tool was originally defined in the failure mode and effects analysis (FMEA) program developed by the National Aeronautics and Space Administration for evaluating rocket and guidedmissile development programs and is well described in quality management literature and in Internet postings (10). Its use may be the most quantitative possible in some situations. Nonetheless, assumptions should be carefully and clearly challenged. For example, the potential failure of the O-ring on shuttle Columbia was recognized but considered a very low value risk. Similarly, the observation of the break-off of the approximate 1.3-lb piece of oxygen tank insulation foam on the shuttle Challenger also was discounted as a very low risk factor. It seems difficult to estimate a failure distribution curve when considering either of these two events, especially when knowing that a very serious outcome such as multiple deaths may happen but doesn't always occur. The original estimates of shuttle loss were in the range of 0.0001–1%, in the absence of data and based on the potential threat that something catastrophic would happen while in orbit. Nonetheless, the risk of loss can be estimated today at 1.8% based on those two lost flights out of the total 113 flights to that date of the Challenger flight. And, both losses occurred on launch or re-entry, not while in orbit.

Block diagram process model for risk assessment for interactions with a CMO

Process models come in many forms ranging from simple line drawings to more complex geometric displays. In all cases, the process model is intended to provide a pictorial representation of the interactions, requirements, inputs, and outputs for the tasks associated with the process. Such models are bounded by a beginning and an end, which means that the model allows the freedom to describe it in a very high-level or a very detailed view. It allows for breaking a large process into multiple parts to provide the required detail for each of these parts. Because all pharmaceutical firms require multiple processes to reach overall objectives for the organization, the process model allows for defining each subprocess as a contributing input to the "main" process, thus allowing the various functional groups to show all of the detail they need for any particular process to which they contribute.

An example of this block-diagram process model is shown in Figure 3. A typical pharmaceutical industry process, the selection and use of a CMO by a sponsor firm, was chosen. The diagram is a hypothetical construct that could be representative of the process used by many pharmaceutical firms. Some details for this hypothetical process will be shown in the model and described in the text. Nonetheless, each firm that might choose to use this approach would enter the information and requirements specific to that organization. The description here is only for the presentation of the concept.

Figure 3: Block-diagram process model for the evaluation of risks associated with the performance of a pharmaceutical project in collaboration with a contract manufacturing organization (CMO).

This model shows the bounded process, as stated in the "scope" box, from the point that a project has been defined until it is placed at a CMO. The outputs represent the successful placement of the project at the CMO with an agreed RiskMAP for the two organizations. The input represents the results of the risks evaluation and plan for the elimination or mitigation of risks associated with the performance of the project in collaboration with the CMO. These three elements of this process map form the description of the risks evaluation and the resultant RiskMAP.

Four more elements are included in this concept that must be used to achieve this risk description. These items will be discussed in detail.

• Performance standards are the specifications and acceptance criteria within which the process should run. These should be part of the corporate policy program for identifying and selecting a CMO.

• Procedures are the accepted tasks and steps to be taken to complete the tasks necessary to ensure the process will achieve its objective. In the pharmaceutical industry, these tasks are usually in the form of standard operating procedures or work instructions that are developed by the responsible functional group personnel and reviewed and approved for use by the quality unit.

• Training and knowledge is a broad category that contains all of the basis information and understanding and the associated instructional training on the background information and required procedures to be used by operating personnel to ensure that the process achieves its objectives. This training includes regulatory and, in particular, CGMP training, technical education and training, and interpersonal skills training.

• Facilities and equipment are all of the hardware and software tools and requisite equipment and facilities that are necessary for the satisfactory completion of the process to achieve its objective.

Table I shows some of the detail that might be considered for each of these category blocks. In no way is this collation a complete table of the areas of concern for the choice of a CMO and placement of a project with that firm. Rather, it is meant to represent the issues and tasks that may be considered. Each firm must develop a table for itself in a complete format and decide wherein the risks may lie. The results may be different when each potential CMO is reviewed against the categories and the risks that may be identified.

Table I: Four general categories for the process model and select representative items in each category for the risk assessment for contract manufacturing organization (CMO) use.

The identification and collation of points to consider and risk factors in each of the categories in Table I will be different for each firm. The process will depend on the firm's previous experience with projects managed in collaboration with CMO firms, the expertise and experience level of the in-house personnel, the size and complexity of the project, and the uniqueness of the manufacturing process. The cost of failure for a project will be much greater for the small firm with only one or two projects in a single therapeutic area than for a very large firm with multiple projects in many therapeutic areas.

A subprocess for the quantitative determination of each of the risks and the preparation of the RiskMAP may be of the form shown in Figure 4.

Figure 4: Process model for the quantitative evaluation of each risk and development of the RiskMAP.

The output from the process performed according to Figure 4 would then be used as the input to the process performed according to Figure 3. Figure 5, developed within the government in Texas (11) and reproduced here in its entirety, is a very good starting point for anyone to develop a framework for identification and description of risks and associated actions. Although this list was developed as a project management tool, it can be modified as a checklist for process management and used in conjunction with the process model pictorial described herein.

As stated previously, quantitative assessment of risk is not always possible by mathematical algorithm. Nonetheless, risk tables have been used for decades in the pharmaceutical industry and are invoked by the checklist shown in Figure 5. The development of the risk table usually occurs as a joint effort by the stakeholders. Brainstorming sessions are used to identify and describe each risk observed by each stakeholder. When the mathematical quantification of any risk is not possible, then the stakeholders must agree on the criticality and expectation for the risk to occur. From this assessment, a risk priority number can be evaluated as described previously, or a risk avoidance plan can be determined.

Figure 5: The Risk Management Initiation Checklist developed in the government of Texas (11).

A hypothetical risk table is shown in Table II in which the concept of risk prevention or mitigation is developed as a tactical plan for a few steps of a typical manufacturing process for solid oral tablets. This approach is often quite good for actual manufacturing operations in which the potential root cause of failure can be "guesstimated" from previous experiences in a company. Such occurrences usually are seen when inexperienced or poorly informed operators are assigned to a new process and for the transfer of process between two sites when there is inadequate understanding of the various critical steps in a process. The knowledge that such occurrences have happened in the past should provide the impetus to develop the framework and proactive approach necessary to prevent such occurrences from happening in the future.

Table II: A hypothetical risk table for a few critical steps in a typical tablet manufacturing process.

Summary

The concept presented in this article is a work in progress and is by no means complete. Comments, criticisms, and extensions or alternative approaches to the concept from others would be greatly appreciated. The industry has a great opportunity now offered by FDA to achieve the next level of good manufacturing practices through presentation of greater knowledge and control for each manufacturing process. A key contributor to that opportunity is the proactive development of the risk-managment action plan (RiskMAP) for each product. Although such a proactive process may require additional knowledge and understanding on the front end of product development it should allow much greater regulatory flexibility and decreased costs both to the firm and to society during the life-cycle management of the product.

The block diagram process model as a framework for describing and detailing any process, which is suggested for the process of risk evaluation, is not new and all of the categories of information are included in every pharmaceutical organization. The suggestion is that an application of this framework may provide some clarity for outlining what information must be considered, how it should be evaluated and used, and how it should be presented in support of the "evidence-based risk identification, assessment, and characterization . . . that continue throughout a product's life cycle" (2).

The process model will provide greater written detail, may provide the keys for quantification and management of the identified risks, and may be used as the basis for proactive rationalization of technical and organizational control procedures. The use of such an approach will ensure the development of an ongoing database of experiences and findings from which greater control can be recognized and, perhaps, future errors can be avoided.

An additional benefit of such a stylized and descriptive approach is that education and training programs can be designed to emphasize the identified risks and the required management practices. General technical or GMP principles training can be replaced with education and training which is specific to the products being produced in the facility or at the contract manufacturing organization (CMO) site. The resultant trained personnel should then be much more qualified and capable of managing their regulatory and technical role in the project, whether in-house or at the CMO site.

The ultimate advantage of this approach is that the RiskMAP requested by FDA will be readily forthcoming and easily supported. This will be a distinct advantage to the sponsor firm whether preparing the submission documentation or preparing for the next regulatory audit.

The emphasis in this article has been on the evaluation and control of risk for manufacturing operations at a CMO. Nonetheless, this approach is equally applicable to the collaboration for services supplied by a CRO, for example in the management of clinical trial sites. Although the descriptors for identified risks may look different, the concepts for identifying, characterizing, evaluating, and managing them in a proactive and systematic way are the same.

Charles F. Carney is a senior affiliate consultant with Seraphim Life Sciences Consulting LLC, 25 Head of Meadow Road, Newtown, CT 06470, tel. 203.426.1860, ccarney@seraphimlifesciences.com

References

1. US Food and Drug Administration, Pharmaceutical cGMPs for the 21st Century—A Risk-Based Approach, Final Report (FDA, Rockville, MD, 2004), http://www.fda.gov/cder/gmp/gmp2004/GMP_finalreport2004.htm, accessed July 6, 2006.

2. FDA, Guidance for Industry: Development and Use of Risk Minimization Action Plans, (FDA, Rockville, MD, 2005) http://www.fda.gov/cder/guidance/6358fnl.htm, accessed July 6, 2006.

3. Webster's New Collegiate Dictionary (G&C Merriam Company, Springfield, MA, 1980), p. 992.

4. Brainy Encyclopedia, http://www.brainyencyclopedia.com/encyclopedia/r/ri/risk.html, accessed July 6, 2006.

5. Riskglossary.com, http://www.riskglossary.com/articles/operational_risk.htm, accessed July 6, 2006.

6. "Process Approach ISO/TS 16949," Quality Digest, June 2004, p. 25.

7. FDA, Guideline on the Preparation of Investigational New Drug Products (Human and Animal), (FDA Rockville, MD, 1991).

8. C.F. Carney, M.J. Killeen, and S. Galloway-Ludwig, "Clinical Material Manufacturing Process Verification/Validation," Pharm. Engineering, May–June, 42–45 (1995).

9. For information on this initiative, contact Fernando Muzzio, PhD, Director of the Pharmaceutical Engineering Program, Department of Chemical and Biochemical Engineering, Rutgers University, 98 Brett Road, Piscataway, NJ; muzzio@sol.rutgers.edu

10. Reliasoft, http://www.reliasoft.com/newsletter/2q2003/rpns.htm, accessed July 6, 2006.

11. State of Texas, Department of Information Resources, http://www.dir.state.tx.us/eod/qa/risk/cheklst2.htm, accessed July 6, 2006.