Anne Greene, PhD, is director PRST, Technological University Dublin.

Anne Greene

 This article was peer-reviewed by a member of 

Submitted: June 11, 2023
Accepted: March 5, 2024

Cube wooden block with alphabet building the word RISK. Risk assessment | Image Credit: ©Dontree - stock.adobe.com

In November 2020, the International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH) announced it was appointing an Expert Working Group (EWG) to update the quality guideline, ICH Q9 

 (1). The concept paper, published in support of this announcement, noted a number of areas of improvement, one of which was risk-based decision-making (RBDM) (2). The concept paper stated that there was a “lack of clarity” on what good RBDM means, and that that there was a “breadth of peer reviewed research” in this area. The Pharmaceutical Regulatory Science Team (PRST), based in Technological University Dublin, embarked on an exploration of this research, and this paper presents some of the outputs of this study. The hope is to clarify or ‘demystify’ what RBDM is, and to offer some suggestions as to how to improve or enhance RBDM to improve quality risk management (QRM) effectiveness for the benefit of the patient.

There are many decisions made within a pharmaceutical product lifecycle and within a pharmaceutical manufacturing operation. Most of these decisions involve the consideration of risk, including risk to patient safety, product quality, and product availability. In this context, it is prudent to focus upon RBDM and to consider why it differs from other decision-making within a pharmaceutical quality system (PQS) and why, as is proposed by the ICH Q9(R1) concept paper, that by improving RBDM, the output of QRM processes might also be improved.

This paper is structured using the 5W1H method of problem definition (a tool with intriguing origins in a Rudyard Kipling poem [3]). The questions, illustrated in 

, are used to explore the subject of RBDM.

Figure 1: The 5W1H Problem Solving Method. [Figure courtesy of the authors]

 The most obvious answer to why one should focus on RBDM is because it is recommended in the updated risk management guideline ICH Q9(R1), which was published on Jan. 18, 2023 (4). This revision contains a new section, section 5.2, on RBDM. This alone should focus one’s curiosity on RBDM. However, the authors propose that by using another tool of problem solving, the 5Whys, perhaps further layers of ‘why?’ (at least three) are needed to fully appreciate the objective of adding RBDM to the revision.

Why add RBDM to ICH Q9(R1)? The ICH committee explained, in the 2020 concept document, why the topic of RBDM was under consideration. According to this document, it was determined that there was “a lack of clarity” with both the term and with the application. How this problem statement was determined is a little less clear.

However, the main reason for its inclusion might be that the full value of QRM, as envisaged when the original ICH Q9 was published in 2005, was not fully realized. The aspiration that QRM would enhance the effectiveness of the PQS remains a work in progress. This is evidenced by continued drug shortages, with quality problems (with both products, and the processes that manufacture them) identified as the highest contributory factor (5,6).

It is important to realize that effective QRM processes may, on rare occasion, result in unfortunate and unwanted outcomes, such as product quality issues or shortages. In these instances, it is important to demonstrate that the process (of QRM) was applied properly and diligently and that the decisions made were as informed as possible, given what was known or understood at the time. In other words, that the unwanted outcome is not a consequence of a lack of rigor. However, it does not appear that this is an easy matter to demonstrate. Kevin O’Donnell from the Irish Health Products Regulatory Authority (HPRA) noted that pharmaceutical companies “struggle to present evidence that their QRM activities are actually effective” (7). In the same publication, O’Donnell noted that, “While it is clear that the industry has done a lot of work in the area of QRM, it may have reached a plateau in that work, where advances in the area have stagnated to some extent.”

One area of QRM activity that heretofore has not received much attention is that of the decision-making within the process. The authors propose that, perhaps, this is the secret sauce to improve both the process and the output of QRM and address the shortfalls currently observed.

 As products, processes, packaging, and supply chains become more complex both technologically and logistically, the nature and likelihood of hazards, risks, and impacts, also becomes more complex. Most problems encountered in pharmaceutical operations are ‘tame problems’ (i.e., they can be described and resolved with the application of root cause analysis, the right knowledge and expertise, and the appropriate corrective and preventive actions) (8). This is not to imply that these problems are simple—both the problem and the solution can be complicated—but they fit within a ‘cause-and-effect’ paradigm.

Increasingly, however, some problems in modern operations display the characteristics of ‘messy’ or ‘wicked’ problems (9,10). Wicked problems are characterized by multiple contributary causes, with multiple knock-on effects. Consequently, the number and formality of risk analysis also typically increases, as the multiple contributary elements are analyzed. Consequently, the level of formality applied to the decision-making process should also increase. Decision complexity has also been described in terms such as convergent and divergent, where solutions either resolve into one solution or not (11).

It is in these areas of decision complexity, where formality becomes important. Research in the area of decision science informs us that humans, for many reasons, do not perform as rationally as they would like to think when faced with complex decisions (12). This is particularly true when these decisions are about risk and when there is a level of uncertainty involved (13).

Decisions in these contexts are often supported by much data, from multiple sources, a circumstance which humans often find overwhelming. In these cases, the instinct is to reduce the information to more easily interpreted formats such as graphs, charts, or dashboards. However, insights are often lost in these reductive representations.

Alternatively, complex decisions may be required in circumstances where there is little data. It has been established that when information is limited and uncertain, decision-making about risk requires the application of a different logic or mental reasoning, than those used to problem solve many routine PQS decisions (14). Typically, within the PQS, decisions rely heavily on factual evidence; therefore, deductive reasoning is widely used, and the skill of deduction is improved with much practice. However, RBDM requires inductive reasoning, which is called upon less frequently in routine PQS decision making, and therefore is not as well honed. The authors illustrate the two types of reasoning in 

, which shows the comparative characteristics of deductive and inductive reasoning processes (15).

Figure 2: The authors’ illustration of the difference between deductive and inductive reasoning. [Figure courtesy of the authors]

In data-deficient contexts, concern with respect to RBDM accuracy is further complicated when RBDM relies on knowledge acquired from experts via expert opinion (which gives rise to concerns of subjectivity), or from computerized algorithms, search engines, mathematical models, and more recently artificial intelligence (giving rise to concerns of representation), all of which, somewhat counter-intuitively, add to uncertainty rather than resolve it (16,17).

RBDM within QRM requires different mental abilities, which renders it a competency that may not be fully appreciated when important, complex, and uncertain risks are under analysis. Unless the skills are developed and honed, the more typical approaches and modes of reasoning, may overwhelm and mislead the decision-maker.

Why might improving RBDM improve QRM output? It is possible, that by appreciating the skills needed to optimize RBDM, that the quality of the decisions made in QRM might improve, thus improving the output and the effectiveness of QRM in supporting PQS decision-making. O’Donnell also observed that, “The emphasis has probably been more on the mechanics of risk assessment tools/scores rather than on the quality of decisions made based on the outcomes of risk assessments” (7).

As this is one aspect of QRM that has not been discussed and advanced, enhancing RBDM approaches are likely to realize improvement.

ICH Q9(R1) provides a definition of RBDM as, “An approach to, or a process of, making decisions that considers knowledge about risks relevant to the decision and whether risks are at an acceptable level” (4).

To fully understand the nature of a hazard (i.e., how it might arise in the future, and consequently how it might be prevented, mitigated, or controlled), the more complete the knowledge about that risk the better. Therefore, a critical input into a QRM process is knowledge. A supporting knowledge management (KM) process is needed to ensure that accurate and reliable knowledge is available for RBDM within QRM.

 (18) proposed that both QRM and KM were enablers of decision-making processes within the PQS. To date, QRM has probably evolved in organizations more rapidly than its twin enabler KM, Calnan 

 referred to the latter as the ‘orphan enabler’ (19).

 (20) proposed, using a framework referred to as the Risk-Knowledge Infinity (RKI) cycle, illustrated in 

, that these processes are both interdependent and iterative. This interconnection and its importance to QRM was discussed previously by the authors (21). The key takeaway is that robust QRM is founded on the availability of sound scientific knowledge of risk, while the output of QRM further informs the organization’s understanding of risk management and risk control. Both are essential requirements to support the risk-based decision maker (i.e., a strong understanding of the risk and the analysis of those risks).

Figure 3: The Risk-Knowledge Infinity (RKI) cycle proposed by Lipa et al. [Figure courtesy of the authors]

The skill of RBDM is to make good decisions about the control of risk, using what is known about that risk. When much is known, these decisions are not onerous.

However, when knowledge is limited, as is often the case in real-world situations, then RBDM is more uncertain and more challenging. In this circumstance, illustrated in 

, the decision maker has two options. The first is to delay the decision and to seek further knowledge. While this is perhaps the preferred option, it is not always possible. The second option is to improve the capability and accuracy of both the analysis of risk, and RBDM under uncertainty. In this circumstance, reliance on RBDM competencies becomes more critical, as the application of inductive reasoning skills is required.

Figure 4: The reasoning required for risk-based decision-making (RBDM) under uncertainty. [Figure courtesy of the authors]

A focus group, held by the authors in 2022, attended by senior quality professionals from the pharmaceutical industry, confirmed that there was confusion as to where RBDM is performed. The most common perception is that all decisions within the PQS are risk-based decisions, as most, if not all, involved some form of risk. While the underlying premise is true, it is not however an accurate perspective. Neither is it helpful.

If all decision-making processes within the PQS and QRM are considered to be similar, when in fact they are not, it is difficult to develop consistent and informed approaches. Decision types differ with respect to context, values, process, available tools, or as mentioned previously the reasoning, logic, and skills needed. If RBDM is to be improved, then a key starting point must be to recognize where it is occurring and, in that context, focus on its enhancement.

Through researching the term RBDM, and its use and application in other risk-sensitive industries, the authors found that the term RBDM is more commonly used to describe the technical characterization of risk, such as that conducted in the risk analysis phase of the QRM process. The term is used to refer, for example, to the decisions made with respect to risk analysis tools, scoring schema, acceptance criteria, control strategies, and review and monitoring mechanisms. The output of RBDM and QRM (i.e., an understanding of the ‘state-of-control’ with respect to risk) may then be used to inform and support PQS decision making, with the term risk-informed decision-making (RIDM) commonly used to describe this latter context (22).

A more bounded use of the term RBDM, confining it to the decisions made within the QRM process is recommended, thus allowing for the development of best practice, as advised by the revised ICH Q9(R1). Even in this context and within these boundaries, there are multiple decisions made within a single QRM process, and numerous QRM analyses and decisions may combine or compound to control a single product risk. This is particularly true in complex problem settings, with multiple stakeholders, dispersed operations, and long lifecycles. Therefore, even within this narrower interpretation of the term, there is still much complexity and several layers of decision-making.

When should formality be applied to RBDM?

It is clear from ICH Q9(R1) that all risk management decisions should be approached with some level of formality, but that the extent of that formality may vary along a continuum from low to high. As problems increase in complexity, so do the decision-making processes required to support them.

QRM, when performed during a pharmaceutical product lifecycle, can be an elongated process occurring at different times over a long timeframe, by different expert teams, in different operational functions with different geographies, cultures, or ownership. In such long horizon processes, early RBDM (e.g., those associated with the choice of formulation and materials) can be important, complex, and uncertain, and are often made with limited available knowledge. Yet such decisions may commit the entire product lifecycle to irrevocable pathways (i.e., it is difficult to reverse the decisions made). Often such irrevocable decision points limit the choices available to subsequent decision-makers, and thus they should be made with care.

The factors that might drive the process to the higher end of the scale and require higher formality are suggested by the authors as follows:

When the hazard or risk under analysis is important (i.e., has a high potential to impact the quality and/or the availability of the product and ultimately to compromise patient safety).

When the decision is complex due to the technical, logistical, or interdependencies in the risk or system under analysis. Complexity may also be contributed by differing stakeholder preferences or objectives.

When there is limited knowledge of the hazard, how it might arise, how frequently, and/or how it might be effectively controlled, all of which contribute to uncertainty with respect to QRM outcomes.

In cases such as these, higher formality is appropriate and more structured approaches to the RBDM process, such as those illustrated in 

, gives enhanced clarity and transparency into how decisions are made (23).

Figure 5: Authors’ illustration of higher formality applied to risk-based decision-making (RBDM). [Figure courtesy of the authors]

When taking higher formality approaches, ICH Q9(R1) recommends that RBDM be performed by cross functional teams. It is important that the understanding and analysis of risk is developed through the input of diverse views, opinions, and expertise. However, the hazards of multiple viewpoints and group decision-making must also be considered.

Formal and structured decision-making processes and tools attempt to control the challenges of multiple contributions and constrain issues such as subjectivity, biases, heuristics, cultural differences, internal and external politics, power imbalances, and differing priorities, risk tolerances, and acceptance criteria (24). Recognizing the role of the decision-maker and decision-making in the QRM processes ensures that these important steps are performed accurately. ICH Q9(R1) addressed the role of decision-makers, as follows (4):

take responsibility for coordinating quality risk management across various functions and departments of their organization.

assure that a quality risk management process is defined, deployed, and reviewed and that adequate resources and knowledge are available; and

assure that subjectivity in quality risk management activities is managed and minimized, to facilitate scientifically robust risk-based decision-making” (4).

Another consideration when decisions are important is the likelihood that decision-makers are senior in the organization and are not the same people as those performing the risk analysis. This introduces the importance of risk communication in QRM. Visualization and representation of the output of risk analysis, particularly when complex approaches are taken, should be such that the decision-maker is not misled with respect to any uncertainty associated with the analysis or with the output.

This, once again, raises the question of decision-making competency. Returning to the concept that RBDM is reliant on logic and reasoning that differs from other decision-making contexts, an understanding of the underlying skills and knowledge required to develop inductive reasoning and how to apply it to RBDM is warranted. These skills have been identified in order of influence, as numeracy, statistical numeracy, risk literacy, and problem solving (fluid intelligence) (25, 26).

The deductive decision-making approaches that are more typical in PQS decision-making are commonly less reliant on numeracy. Yet numeracy, and a knowledge of statistics, is a key asset in the interpretation of risk analysis, particularly where probability or modeling has been employed. A key argument for confining the concept of RBDM, to within the QRM process, is to develop and target the correct training and to practice RBDM competency to enhance improvement.

A final consideration when considering who performs RBDM is the assumption made in ICH Q9(R1) that decision-making is a human function. This assumption may not completely future proof RBDM. The rate of progress in the areas of data analytics, machine learning, and artificial intelligence is such that future decision-making about risk, particularly with respect to risk control, will be made by computerized agents, or by humans with increased reliance on computerized decision support systems. In these cases, an understanding of the underlying decision-making rules and the reasoning applied within any programs or algorithms used, is also key to accurate outcomes.

The risk analysis may suggest courses of action based on expected value (either cost or quality), as a balance of risk and benefits, or as risk reduction opportunities, expressed with respect to defined thresholds of acceptable risk, or in terms of avoidance of worst-case scenarios, often with little knowledge of the decision-makers preferences or risk tolerance (27). Therefore, the connection, communication, and understanding between the decision-maker and the risk analyst is important to recognize and define, irrespective of whether both are, or either is, human.

With all these considerations in mind, the authors researched RBDM in other sectors. The focus was on industries with high sensitivity to risk (e.g., nuclear, aviation, aerospace, oil and gas, etc.). This research informed the development of 21 attributes of good RBDM. These attributes are illustrated in 

. The attributes were then categorized as the characteristics of good governance and human related factors, of robust QRM, and of knowledge management (28).

Figure 6: 21 attributes of good RBDM, determined from a study of other risk sensitive industries. [Figure courtesy of the authors]

Further research into these attributes will ultimately allow for the development of a guidance or playbook on RBDM, but for now, the following four insights on how to enhance RBDM are emerging:

Decide the appropriate level of formality to support the RBDM.

Provide the best available knowledge to inform and support the decision.

If knowledge is limited, ensure the correct skills are in place to interpret any predictive analysis.

While the first three recommendations have been discussed in previous sections, the recommendation to review RBDM throughout a QRM process has not. However, a ‘lessons learned’ or review process is key to the improvement of any process or system and is an important element of a KM process. Like any skill, RBDM can benefit from the application of hindsight and reflection to determine if, with what was known at the time, the optimal decisions were made. Using this information, the process (of RBDM) can be improved with further enhancements, such as correct information flows, the right decision-makers, different tools, etc.

The concept of applying review node” to ’ process exists in other industries. Commonly at these reviews, the decision-maker determines whether the process has enough information and risk understanding to proceed to the next stage of the process with an acceptable assurance of success. These reviews also facilitate projects proceeding, but with certain ‘liens’ or assumptions, which are highlighted within the action plan for the next phase or phases until they are understood and acceptable.

For any company to improve its own decision-making capability, particularly under uncertainty, the introduction of key decision review points is a worthwhile consideration. The QRM process lends itself to natural pause points. Each review point can have an associated prompt list to assure that the understanding and control of risk is refined through the QRM process, bearing in mind that this may be over a long period of time. The concept of key decision review points is discussed in more detail in a previous publication by the authors (29).

RBDM is distinct from other types of decision-making within a PQS. As decision-making occurs in the context of risk characterization and control, factors such as prediction and probability are inherent. These factors create uncertainty, which is not an optimal context for human decision-making. Therefore, when decisions are important or complex, especially if the problems are messy or wicked, then applying the appropriate formality ensures a rigor to the decision-making process. Training to ensure that the competencies are available also provides greater assurance of successful outcomes. Supporting decision-makers with the correct knowledge is a further enhancement. Finally, reviewing RBDM in the context of QRM processes and outcomes allows for enhancements in future applications, thus fulfilling the opportunities of the RKI cycle.

 (ICH, 2005). https://www.ich.org/page/quality-guidelines

ICH. Final Concept Paper for the ICH Q9 (R1) Quality Risk Management, November 2020. https://www.ich.org

US Senate Committee on Homeland Security & Governmental Affairs. Short Supply: The Health & National Security Risk of Drug Shortages. March 2023.

O’Donnell, K. QRM in the GMP Environment beyond 2020: What Should We Strive For?, in 

An Audience with the International Regulators in the Manufacture of Medicine: QRM and KM

Rittel, H. W. J.; Webber, M. M. Dilemmas in a General Theory of Planning. 

Systems, Messes, and Interactive Planning, in Redesigning the Future

Skaburskis, A. The Origin of ‘Wicked Problems’. Plan. 

 9 (2), pp. 277–280, June 2008, DOI: 10.1080/14649350802041654.

Howard, R. A. Uncertainty about Probability: A Decision Analysis Perspective. 

Mushtaq, F.; Bland, A. R.; Schaefer, A.; Pourtois, G.; Stürmer, B.; and Krebs, R. M.; 

Holland, J. H.; Holyoak, K. J.; Nisbett, R. E.; and Thagard, P. R. Induction: Processes of Inference, Learning, and Discovery, First. MIT Press, 1989.

Hanea, A. M.; Hemming, V.; Nane, G. F. Uncertainty Quantification with Experts: Present Status and Research Needs. 

The Cambridge Handbook of Artificial Intelligence

Calnan, N.; Greene, A.; Kane, P. An Academic Perspective: Knowledge Management: The Orphan Enabler—Enabling ICH Q10 Implementation, in 

A Lifecycle Approach to Knowledge Excellence in the Biopharmaceutical Industry,

 Calnan, N.; Lipa, M. J.; Kane, P.; Cardoso Menezes, J. Eds., CRC Press, 2018, pp. 133–152.

Lipa, M.; O’Donnell, K.; Greene, A. Knowledge as the Currency of Managing Risk: A Novel Framework to Unite Quality Risk Management and Knowledge Management. 

Lipa, M.; Mulholland, V.; Greene, A. (editor). Improving Risk-Based Decision Making (RBDM) Effectiveness Through a Framework which Integrates Knowledge and Risk. 

 16 (2), February, Accessed: Feb. 28, 2022. [Online]. https://arrow.tudublin.ie/level3/vol16/iss2/2

Mulholland, V.; Greene, A.; Lipa, M. J. Steps Beyond Risk Assessment in QRM: RBDM, The Next Horizon. 

Mulholland, V.; Greene, A. Improving Risk-Based Decision Effectiveness: Determining the Level Of Formality. 

 3 (4), Accessed: May 14, 2023. https://kenx.org/kenx_insights/improving-risk-based-decision-making-effectiveness-determining-the-level-of-formality/

Mulholland, V.; Greene, A.; Lipa, M. Improving Risk-Based Decision Effectiveness: Addressing Uncertainty. 

3 (5), Accessed: May 14, 2023. https://kenx.org/kenx_insights/improving-risk-based-decision-making-effectiveness-addressing-uncertainty/

Cokely, E. T.; Ghazal, S.; Galesic, M.; Garcia-Retamero, R., & Schulz, E. How to Measure Risk Comprehension in Educated Samples. In R. Garcia-Retamero & M. Galesic (Eds.), 

Transparent Communication of Risks about Health: Overcoming Cultural Differences

Dieckmann, N. F.; Slovic, P.; Peters, E. M. The Use of Narrative Evidence and Explicit Likelihood by Decisionmakers Varying in Numeracy. 

29 (10), 1473–1488. DOI: 10.1111/J.1539-6924.2009.01279.

Articles

It is important to understand the differences between risk-based decision making and other decision making in a pharmaceutical quality system.

Steps Towards Demystifying Risk-Based Decision Making


This article was peer-reviewed by a member of 

Submitted: September 8, 2023
Accepted: January 19, 2024

Business woman holding checked icon on virtual screen. Business process automation, quality assurance management, certification, digital transformation. Service guarantee, Validation concept.| Image Credit: ©Atomic62 Studio - stock.adobe.com

In 2020, the International Council for Harmonisation (ICH) published a concept paper (1) that outlined a limited revision to the published guideline ICH Q9 

 (2) to address the fact that “the benefits of QRM [quality risk management], as envisaged by ICH Q9, have not yet been fully realized.”

Arguably, despite the publication of the companion ICH guidelines in the first decade of the 21st century, ICH Q9 (2) and ICH Q10 

(3), there are still numerous challenges in enhancing manufacturing operations, which lead to quality issues and drug shortages. Even with significant efforts in QRM by both the industry and regulators, there are signs that unmitigated risks persist within the pharmaceutical industry, resulting in quality defects as reported by Kevin O’Donnell, Market Compliance Manager at the Health Products Regulatory Authority (HPRA) in Ireland, who observed that the number of total quality defects reported to the Irish regulator between 2002 and 2021 had increased by more than 10-fold (4). In addition, a survey of chief quality officers of 30 global pharmaceutical companies found that the benefits envisioned by ICH Q10 have generally not been realized, and in some cases the situation was slightly worse or more complex (5).

FDA’s 2019 Drug Shortage Report underscored the ongoing problem of drug shortages in the United States, suggesting manufacturers might prioritize cost reduction over quality investment because quality is not viewed as a competitive marketplace advantage (6). Concurrently, the US Senate Committee on Homeland Security drew attention to the escalating trend of essential medication shortages, impacting a broad spectrum of products from emergency hospital drugs to over-the-counter remedies. By the end of 2022, drug shortages reached a new peak. This crisis was rooted in various factors, such as an overreliance on foreign sources for active ingredient, a lack of domestic manufacturing capacity, and insufficient supply chain visibility into all the companies involved, leading to a misleading appearance of supplier diversity. These shortages have profound cascading effects on patient care (7).

As a tangible step to help address some of these challenges, a targeted revision of ICH Q9 was approved. ICH Q9(R1) was published in January 2023 (8), superseding the original ICH Q9 guidance issued in 2005 (2). This revision narrowly focused on the following four topics deemed critical to advance the competency and effectiveness of the industry in risk management:

high levels of subjectivity in risk assessments and QRM outputs

lack of understanding into what constitutes formality in QRM work

lack of clarity on risk-based decision-making.

The ICH Q9(R1) concept paper further suggested risk review and hazard identification as two additional points to be addressed.

Considering the topic of subjectivity, the ICH Q9(R1) Concept Paper (1) noted the challenge associated with subjectivity as follows:

“High levels of subjectivity in risk assessments and in QRM outputs—the reasons for this can include highly subjective risk scoring methods and differences in how risks are assessed and how hazards, risk, and harms are perceived by different stakeholders. This can lead to varying levels of effectiveness in the management of risks. While subjectivity cannot be completely eliminated from risk assessment and QRM activities, it may be controlled using well recognized strategies, including addressing bias and behavioral factors.”

The Concept Paper further went on to describe potential benefits of addressing subjectivity and suggests:

“Less subjective risk assessments to support manufacturing processes should lead to fewer quality defects that could present risks to patients. Recalls are required every year to protect patients from the real and/or potential risks presented by defects in medicines, leading in some cases to shortages of important medicines. Less subjective risk assessments should also lead to more science-based manufacturing operations, control strategies, and validation activities, resulting in the potential for reduced costs and the possibility to free up resources for other necessary activities. ICH Q8, Q10, and Q11 expect science- and risk-based applications, and revising ICH Q9 to address subjectivity in QRM more explicitly will help enable/accelerate the continued implementation of Q8, Q10, Q11 (and Q12), because of the foundational relevance of QRM.”

Drivers to reduce subjectivity are compelling when one considers the opportunity to reduce defects (and, in turn, recalls). Reduced defects will ultimately lead to reduced costs and resources, and further realization of the goals of a modern PQS. Indeed, one could argue that subjectivity casts a long shadow over many facets of the pharmaceutical business not just QRM. This paper provides an introductory exploration into the topic of subjectivity in QRM by:

reviewing contemporary QRM literature on subjectivity

creating a shared understanding of subjectivity for QRM

examining how subjectivity is addressed in the original Q9 guidance, and comparing it to the revised guidance.

Subjectivity: insights from contemporary QRM-related literature

The emphasis on subjectivity in ICH Q9(R1) is not surprising when one reviews contemporary literature on the topic published since the 2005 release of ICH Q9. Notably, H. Gregg Claycamp, the Regulatory Rapporteur of the original ICH Q9 guidance document, published a paper in 2018 entitled “Controlling Subjectivity in Risk-based Decision Making: The Benefits of Multi-criteria Decision Analysis” (9). In this paper, in addition to brief discussions on subjectivity inherent in risk assessment and risk management, and linking subjectivity and uncertainty, Claycamp proposed that the problem with subjectivity in QRM stems from two elements as follows:

“A failure to recognize where and when subjectivity enters and might create problems in risk assessment and risk-based decision-making.

“A failure to implement control on subjectivity where it is known
to occur.”

Several other papers have further highlighted the challenges posed by subjectivity, including a paper by ICH Q9(R1) rapporteur Kevin O’Donnell (10). This paper discusses elements of subjectivity and uncertainty such as the impact of human heuristics, which are cognitive behaviors that can introduce bias and errors in judgment during activities like brainstorming and probability estimation. It suggests incorporating design elements into QRM tools to counteract the adverse effects of heuristics and reduce guesswork. Practical strategies and cognitive approaches are presented to improve the accuracy of risk analyses and enhance the outcomes of QRM exercises. These insights can be valuable to facilitators in achieving more accurate risk assessments.

In a follow up paper (11), O’Donnell discusses the significance of risk communication in the QRM process, highlighting potential challenges of subjectivity and uncertainty in this activity. Stakeholders’ individual perceptions can lead to subjective differences in risk judgments, introducing uncertainty in risk management outcomes. To address these issues, O’Donnell suggests adopting a more rigorous approach to risk communication. Practical strategies based on research from experimental psychology and uncertainty analysis to improve risk communication and reduce subjectivity and uncertainty are presented. These strategies include stakeholder selection, employee roles in risk communication, and essential information to be communicated within pharmaceutical companies.

In a 2016 publication (12), authors Vesper and O’Donnell highlight multiple issues encountered 10 years after the publication of ICH Q9. One of the issues discussed was the relationship between risk and uncertainty, with risk defined as the “effect of uncertainty on achieving one’s objectives.” Subjectivity in risk assessment can emerge due to differences in perceptions, values, and experiences among stakeholders. This subjectivity influences the evaluation of potential harms, probabilities, and severities in the QRM process. This may be compounded by groupthink during risk assessment activities, leading to difficulties in reaching consensus on risk acceptability and appropriate courses of action. Vesper and O’Donnell emphasize the importance of addressing subjectivity and uncertainty in risk assessment and offer strategies to do so. These strategies include fostering diverse team composition, using key words in scales, acknowledging uncertainty, and relying on validated data. Additionally, this publication suggests integrating good science into risk assessments and documenting current controls before assigning ratings to failure modes or hazards.

A 2018 paper by Emma Ramnarine and Kevin O’Donnell (13) emphasizes the role of subjectivity during risk assessment activities and how subjectivity can negatively impact the state of control and the delivery of products with consistent quality throughout their commercial life.

In addition to QRM-related literature that directly highlights subjectivity, related research by Valerie Mulholland explores the impact subjectivity plays in effective risk-based decision-making. In a paper on Key Decision Review Points (14), Mulholland proposes that each decision within a QRM process has the potential to be sub-optimal due to subjectivity (among other causes). Furthermore, in another paper from the same four-paper series focused on uncertainty (15), Mulholland discusses the presence of high uncertainty, which poses significant challenges in decision-making. In some cases, the risk itself may be uncertainty. Uncertainty can arise from various sources, including variability in data, lack of knowledge, and ambiguity in interpretations. Decision-making under uncertainty may rely on different mental abilities and competencies compared to evidence-based decision-making. Overall, addressing uncertainty in risk management requires a thoughtful and strategic approach, acknowledging the interconnectedness of risk and subjectivity.

Considering these articles in aggregate, the authors of this paper suggest that Claycamp’s 2018 assertion that the problem with subjectivity in QRM arises from two elements (9) captures the essence of an important theme across this literature. The authors propose restating these two challenges more simply as follows:

knowing what to do about it when it does.

In the opinion of the authors, creating a common understanding of subjectivity is a prerequisite for both of these challenges and is an important need for the industry and those involved in the practice of QRM, as discussed in the next section.

Subjectivity: seeking a common understanding

While high levels of subjectivity were identified as an issue, it is notable that neither ICH Q9 (2), the ICH Q9(R1) Concept Paper (1), nor ICH Q9(R1) (8) provide a definition for subjectivity. In the opinion of the authors of this paper, this lack of a common operational definition and/or associated examples poses the risk of an inconsistent comprehension and understanding of what subjectivity is. Therefore, establishing a common definition of subjectivity as it pertains to QRM is important.

Basic definitions of the word subjectivity exist, including the following
dictionary entries:

“The quality of being based on or influenced by personal feelings, tastes, or opinions” (Cambridge Dictionary)

“The influence of personal beliefs or feelings, rather than facts” (Oxford Dictionary).

While these definitions certainly provide a good starting point, the world of QRM has many different challenges, including:

leading-edge science and new types of therapies

expectations for speed (e.g., product development in response to a pandemic or other market pressures)

human behaviors (e.g., expert opinion, heuristics, bias) multi-step QRM process with many different participants over a long-time horizon.

It is difficult to recognize when subjectivity enters QRM and influences outputs that may affect product quality and patient outcomes. The authors suggest this is an important topic for future study. Notwithstanding, the lack of a definition for subjectivity ICH Q9(R1) does offer important clues by its use of other language, in particular language that evokes objectivity as the antidote for subjectivity in a scientific setting, as explored in next section.

Subjectivity: a closer examination of its emergence in ICH Q9(R1)

While ICH Q9(R1) (8) does not expressly define subjectivity, an examination of it in comparison to the 2005 version of ICH Q9 (2) reveals the emergence of the term subjectivity, which appears 10 times in the revised document in contrast to the lack of being mentioned in the original.

Furthermore, the authors have observed that the language in ICH Q9(R1) includes concepts related to subjectivity- and objectivity as its antonym, for example:

terms which invoke scientific facts which suggests minimizing subjectivity (e.g., evidence, data, information, knowledge)

knowledge management (KM) as a means to manage and deliver scientific knowledge to QRM and risk-based decision-making processes.

A simple comparison of the two documents was conducted by the authors to evaluate the frequency of relevant terms in each ICH Q9 and ICH Q9(R1), the results are presented in 

Table I: Frequency of subjectivity-related terms in International Council for Harmonisation (ICH) Q9(R1) and ICH Q9. KM is knowledge management.

As can be seen by the comparison, there is a significant increase in subjectivity related concepts in ICH Q9(R1). The authors believe this signifies a proactive desire to address subjectivity. By incorporating these related concepts, the revised guideline emphasizes the importance of basing risk assessments on objective evidence and data-driven insights. This shift toward more objective decision-making that is evidence-based, while seeking to minimize the influence of subjectivity enhances the reliability of risk analyses and risk management at large.

Subjectivity emerges in ICH Q9(R1) (8) as a concept that must be addressed to improve the effectiveness of risk management, and in turn the pharmaceutical quality system. In fact, O’Donnell challenges “can an organization do effective Risk-based Decision-making if its risk assessments are highly subjective?” (16). As mentioned by experts, acknowledging and addressing subjectivity are crucial for making informed and effective risk-based decisions, ultimately leading to improved product quality, patient safety, and product availability. Indeed, subjectivity casts a long shadow on QRM and, by extension, the pharmaceutical industry as a whole.

Updates included in the revision to ICH Q9(R1) reflect the growing recognition of the significance of subjectivity in risk management within the pharmaceutical industry. By incorporating language related to bias, evidence, data, information, knowledge, and knowledge management, the revised guideline appears positioned to enhance the objectivity and reliability of risk assessments. These changes included in ICH Q9(R1) provide an opportunity to shine a light on this issue by emphasizing the need to address subjectivity and its connection to business outcomes and patient safety.

However, while subjectivity has emerged as important in ICH Q9(R1), significant work remains to develop a clear and shared understanding of what subjectivity is, when subjectivity might exist and what to do about it when it does. These topics will be the focus of future research by the authors. Such research is also expected to explore subjectivity in other industries and study best practices. In addition, it is anticipated that training on this topic, and exploring emerging technologies and methods, would be beneficial.

The views expressed in this article are those of the authors and are not necessarily those of any organizations with which the authors are affiliated.

ICH. Final Concept Paper ICH Q9(R1): Quality Risk Management. 2020. 

https://database.ich.org/sites/default/files/Q9%20Guideline.pdf

. (ICH, 2008). https://database.ich.org/sites/default/files/Q10%20Guideline.pdf (accessed: Aug. 28, 2023).

O’Donnell, K. Quality Risk Management–An Update on the Ongoing Revision of ICH Q9 ‘Delivering Value and Resilience using QRM’ [Conference Proceeding]. in PDA Ireland, Dublin, Ireland, September 2022.

Ramnarine, E. Solving the Continual Improvement and Innovation Challenge for the Benefit of Patients: How an Effective Pharmaceutical Quality System (PQS) and Risk-Based Approach Could Transform Post-Approval Change (PAC) Management. 2021. DOI: 10.21427/960F-8S69.

https://www.fda.gov/drugs/drug-shortages/report-drug-shortages-root-causes-and-potential-solutions

Peters, G. Risk of Drug Shortages-United States Senate Committee- Homeland Security & Governmental Affairs Report. 2023. 

https://www.hsgac.senate.gov/wp-content/uploads/2023-03-20-HSGAC-Majority-Draft-Drug-Shortages-Report.pdf

https://database.ich.org/sites/default/files/ICH_Q9%28R1%29_Guideline_Step4_2023_0126_0.pdf

Claycamp, G. Controlling Subjectivity in Risk-based Decision Making: The Benefits of Multi-criteria Decision Analysis. DIT Academic Press, 2018.

O’Donnell, K. Strategies for Addressing the Problems of Subjectivity and Uncertainty in Quality Risk Management Exercises Part I-The Role of Human Heuristics. 

O’Donnell, K. Strategies for Addressing the Problems of Subjectivity and Uncertainty in Quality Risk Management–Part 2 Risk Communication and Perception Issues. 

Vesper, J. and O’Donnell, K. Current Challenges in Implementing Quality Risk Management. 

https://ispe.org/pharmaceutical-engineering/ispeak/current-challenges-implementing-quality-risk-management

Ramnarine, E. and O’Donnell, K. Demonstrating Pharmaceutical Quality System Effectiveness and Driving Continual Improvement: Evidence-based Risk Reduction.

 72 (3), pp. 338–345, May 2018, DOI: 10.5731/pdajpst.2017.008524.

Mulholland, V. and Greene, A. Improving Risk Based Decision Making Effectiveness: A Case for Risk Decision Review Points (KDRPs) In The Quality Risk Management Process. 

Mulholland, V. and Greene, A. Improving Risk Based Decision Making Effectiveness: Addressing Uncertainty. 

O’Donnell, K. ICH Q9(R1) and Some Personal Thoughts about Implementing the New Guidance [Conference Proceeding]. in TU Dublin, PRST Conference, Dublin, Ireland, June 2023.

Amin Ziaie, QRM Merck & Co., Inc, PRST; Anne Greene is director PRST; and Marty Lipa, PhD, PRST; all at the Technological University Dublin.

®
Vol. 48, No. 3
March 2024
Pages: 30-33

When referring to this article, please cite it as Ziaie, A.; Green, A.; and Lipa, M. Shining a Light on the Long Shadow of Subjectivity in Quality Risk Management. 

This article explores the emergence of subjectivity in ICH Q9 (R1).

Shining a Light on the Long Shadow of Subjectivity in Quality Risk Management

Medical concept of compliance. Pharmacy law compliance.| Image credit: wladimir1804 - stock.adobe.com 

 This article was peer reviewed by a member of 

®’s Editorial Advisory Board and was previously published in the September 2023 issue.

The current International Council for Harmonisation (ICH) Q9(R1) Quality Risk Management (QRM) guideline is a targeted revision of the initial guidance issued in 2005, intended to address subjectivity, formality, risk-based decision-making, and other specific areas for improvement. While not a formally defined area for improvement, the revised guidance invokes many new references to “knowledge” and “knowledge management (KM)” in supporting improvement in QRM practice and, by extension, to the goals of an effective pharmaceutical quality system.

Knowledge as the Currency of Managing Risk: A Smart Investment for Patients

Regulating Innovation, Quality, and Risk eBook January 2024

Recognizing that QRM and risk-based decision-making (RBDM) both depend on knowledge to be effective, it could be suggested that knowledge is the currency of managing risk. This paper presents research and perspectives on the opportunity to better connect risk and knowledge through improved integration of QRM and KM, including presenting several potential benefits of such integration. A framework is proposed, the Risk-Knowledge Infinity Cycle (RKI Cycle), which could be used to improve the connectivity of risk and knowledge along with preliminary and practical recommendations to operationalize the RKI Cycle. In addition, the paper invokes concepts of systems thinking in connecting QRM and KM as well as the opportunity to consider the role of knowledge and KM more holistically across the pharmaceutical business through application of a process classification framework.

Pharmaceutical Technology/Pharmaceutical Technology Europe Regulating Innovation, Quality, and Risk

Martin J. Lipa*, PhD, martin.lipa@prst.ie, is senior research fellow, Pharmaceutical Regulatory Science Team. Anne Greene, PhD, is professor, Technological University Dublin, and director, Pharmaceutical Regulatory Science Team.


*To whom all correspondences should be addressed.

Recent research and perspectives shed light on an opportunity to better connect risk and knowledge through improved integration of systems.

Knowledge as the Currency of Managing Risk

The current International Council for Harmonisation (ICH) Q9(R1) 

 (QRM) guideline is a targeted revision of the initial guidance issued in 2005, intended to address subjectivity, formality, risk-based decision-making, and other specific areas for improvement. While not a formally defined area for improvement, the revised guidance invokes many new references to “knowledge” and “knowledge management (KM)” in supporting improvement in QRM practice and, by extension, to the goals of an effective pharmaceutical quality system. 

This article was peer reviewed by a member of 

Submitted: April 24, 2023
Accepted: June 14, 2023

Recognizing that QRM and risk-based decision-making (RBDM) both depend on knowledge to be effective, it could be suggested that knowledge is the currency of managing risk. This paper presents research and perspectives on the opportunity to better connect risk and knowledge through improved integration of QRM and KM, including presenting several potential benefits of such integration. A framework is proposed, the 