Shining a Light on the Long Shadow of Subjectivity in Quality Risk Management

Editor’s Note:
This article was peer-reviewed by a member of Pharmaceutical Technology®’s Editorial Advisory Board.

Submitted: September 8, 2023
Accepted: January 19, 2024

In 2020, the International Council for Harmonisation (ICH) published a concept paper (1) that outlined a limited revision to the published guideline ICH Q9 Quality Risk Management (QRM) (2) to address the fact that “the benefits of QRM [quality risk management], as envisaged by ICH Q9, have not yet been fully realized.”

Arguably, despite the publication of the companion ICH guidelines in the first decade of the 21st century, ICH Q9 (2) and ICH Q10 Pharmaceutical Quality System (PQS) (3), there are still numerous challenges in enhancing manufacturing operations, which lead to quality issues and drug shortages. Even with significant efforts in QRM by both the industry and regulators, there are signs that unmitigated risks persist within the pharmaceutical industry, resulting in quality defects as reported by Kevin O’Donnell, Market Compliance Manager at the Health Products Regulatory Authority (HPRA) in Ireland, who observed that the number of total quality defects reported to the Irish regulator between 2002 and 2021 had increased by more than 10-fold (4). In addition, a survey of chief quality officers of 30 global pharmaceutical companies found that the benefits envisioned by ICH Q10 have generally not been realized, and in some cases the situation was slightly worse or more complex (5).

FDA’s 2019 Drug Shortage Report underscored the ongoing problem of drug shortages in the United States, suggesting manufacturers might prioritize cost reduction over quality investment because quality is not viewed as a competitive marketplace advantage (6). Concurrently, the US Senate Committee on Homeland Security drew attention to the escalating trend of essential medication shortages, impacting a broad spectrum of products from emergency hospital drugs to over-the-counter remedies. By the end of 2022, drug shortages reached a new peak. This crisis was rooted in various factors, such as an overreliance on foreign sources for active ingredient, a lack of domestic manufacturing capacity, and insufficient supply chain visibility into all the companies involved, leading to a misleading appearance of supplier diversity. These shortages have profound cascading effects on patient care (7).

As a tangible step to help address some of these challenges, a targeted revision of ICH Q9 was approved. ICH Q9(R1) was published in January 2023 (8), superseding the original ICH Q9 guidance issued in 2005 (2). This revision narrowly focused on the following four topics deemed critical to advance the competency and effectiveness of the industry in risk management:

• high levels of subjectivity in risk assessments and QRM outputs
• product availability risks
• lack of understanding into what constitutes formality in QRM work
• lack of clarity on risk-based decision-making.

The ICH Q9(R1) concept paper further suggested risk review and hazard identification as two additional points to be addressed.

Considering the topic of subjectivity, the ICH Q9(R1) Concept Paper (1) noted the challenge associated with subjectivity as follows:

“High levels of subjectivity in risk assessments and in QRM outputs—the reasons for this can include highly subjective risk scoring methods and differences in how risks are assessed and how hazards, risk, and harms are perceived by different stakeholders. This can lead to varying levels of effectiveness in the management of risks. While subjectivity cannot be completely eliminated from risk assessment and QRM activities, it may be controlled using well recognized strategies, including addressing bias and behavioral factors.”

The Concept Paper further went on to describe potential benefits of addressing subjectivity and suggests:

“Less subjective risk assessments to support manufacturing processes should lead to fewer quality defects that could present risks to patients. Recalls are required every year to protect patients from the real and/or potential risks presented by defects in medicines, leading in some cases to shortages of important medicines. Less subjective risk assessments should also lead to more science-based manufacturing operations, control strategies, and validation activities, resulting in the potential for reduced costs and the possibility to free up resources for other necessary activities. ICH Q8, Q10, and Q11 expect science- and risk-based applications, and revising ICH Q9 to address subjectivity in QRM more explicitly will help enable/accelerate the continued implementation of Q8, Q10, Q11 (and Q12), because of the foundational relevance of QRM.”

Drivers to reduce subjectivity are compelling when one considers the opportunity to reduce defects (and, in turn, recalls). Reduced defects will ultimately lead to reduced costs and resources, and further realization of the goals of a modern PQS. Indeed, one could argue that subjectivity casts a long shadow over many facets of the pharmaceutical business not just QRM. This paper provides an introductory exploration into the topic of subjectivity in QRM by:

1. reviewing contemporary QRM literature on subjectivity
2. creating a shared understanding of subjectivity for QRM
3. examining how subjectivity is addressed in the original Q9 guidance, and comparing it to the revised guidance.

Subjectivity: insights from contemporary QRM-related literature

The emphasis on subjectivity in ICH Q9(R1) is not surprising when one reviews contemporary literature on the topic published since the 2005 release of ICH Q9. Notably, H. Gregg Claycamp, the Regulatory Rapporteur of the original ICH Q9 guidance document, published a paper in 2018 entitled “Controlling Subjectivity in Risk-based Decision Making: The Benefits of Multi-criteria Decision Analysis” (9). In this paper, in addition to brief discussions on subjectivity inherent in risk assessment and risk management, and linking subjectivity and uncertainty, Claycamp proposed that the problem with subjectivity in QRM stems from two elements as follows:

1. “A failure to recognize where and when subjectivity enters and might create problems in risk assessment and risk-based decision-making.
2. “A failure to implement control on subjectivity where it is known
   to occur.”

Several other papers have further highlighted the challenges posed by subjectivity, including a paper by ICH Q9(R1) rapporteur Kevin O’Donnell (10). This paper discusses elements of subjectivity and uncertainty such as the impact of human heuristics, which are cognitive behaviors that can introduce bias and errors in judgment during activities like brainstorming and probability estimation. It suggests incorporating design elements into QRM tools to counteract the adverse effects of heuristics and reduce guesswork. Practical strategies and cognitive approaches are presented to improve the accuracy of risk analyses and enhance the outcomes of QRM exercises. These insights can be valuable to facilitators in achieving more accurate risk assessments.

In a follow up paper (11), O’Donnell discusses the significance of risk communication in the QRM process, highlighting potential challenges of subjectivity and uncertainty in this activity. Stakeholders’ individual perceptions can lead to subjective differences in risk judgments, introducing uncertainty in risk management outcomes. To address these issues, O’Donnell suggests adopting a more rigorous approach to risk communication. Practical strategies based on research from experimental psychology and uncertainty analysis to improve risk communication and reduce subjectivity and uncertainty are presented. These strategies include stakeholder selection, employee roles in risk communication, and essential information to be communicated within pharmaceutical companies.

In a 2016 publication (12), authors Vesper and O’Donnell highlight multiple issues encountered 10 years after the publication of ICH Q9. One of the issues discussed was the relationship between risk and uncertainty, with risk defined as the “effect of uncertainty on achieving one’s objectives.” Subjectivity in risk assessment can emerge due to differences in perceptions, values, and experiences among stakeholders. This subjectivity influences the evaluation of potential harms, probabilities, and severities in the QRM process. This may be compounded by groupthink during risk assessment activities, leading to difficulties in reaching consensus on risk acceptability and appropriate courses of action. Vesper and O’Donnell emphasize the importance of addressing subjectivity and uncertainty in risk assessment and offer strategies to do so. These strategies include fostering diverse team composition, using key words in scales, acknowledging uncertainty, and relying on validated data. Additionally, this publication suggests integrating good science into risk assessments and documenting current controls before assigning ratings to failure modes or hazards.

A 2018 paper by Emma Ramnarine and Kevin O’Donnell (13) emphasizes the role of subjectivity during risk assessment activities and how subjectivity can negatively impact the state of control and the delivery of products with consistent quality throughout their commercial life.

In addition to QRM-related literature that directly highlights subjectivity, related research by Valerie Mulholland explores the impact subjectivity plays in effective risk-based decision-making. In a paper on Key Decision Review Points (14), Mulholland proposes that each decision within a QRM process has the potential to be sub-optimal due to subjectivity (among other causes). Furthermore, in another paper from the same four-paper series focused on uncertainty (15), Mulholland discusses the presence of high uncertainty, which poses significant challenges in decision-making. In some cases, the risk itself may be uncertainty. Uncertainty can arise from various sources, including variability in data, lack of knowledge, and ambiguity in interpretations. Decision-making under uncertainty may rely on different mental abilities and competencies compared to evidence-based decision-making. Overall, addressing uncertainty in risk management requires a thoughtful and strategic approach, acknowledging the interconnectedness of risk and subjectivity.

Considering these articles in aggregate, the authors of this paper suggest that Claycamp’s 2018 assertion that the problem with subjectivity in QRM arises from two elements (9) captures the essence of an important theme across this literature. The authors propose restating these two challenges more simply as follows:

1. knowing when subjectivity might exist
2. knowing what to do about it when it does.

In the opinion of the authors, creating a common understanding of subjectivity is a prerequisite for both of these challenges and is an important need for the industry and those involved in the practice of QRM, as discussed in the next section.

Subjectivity: seeking a common understanding

While high levels of subjectivity were identified as an issue, it is notable that neither ICH Q9 (2), the ICH Q9(R1) Concept Paper (1), nor ICH Q9(R1) (8) provide a definition for subjectivity. In the opinion of the authors of this paper, this lack of a common operational definition and/or associated examples poses the risk of an inconsistent comprehension and understanding of what subjectivity is. Therefore, establishing a common definition of subjectivity as it pertains to QRM is important.

Basic definitions of the word subjectivity exist, including the following
dictionary entries:

1. “The quality of being based on or influenced by personal feelings, tastes, or opinions” (Cambridge Dictionary)
2. “The influence of personal beliefs or feelings, rather than facts” (Oxford Dictionary).

While these definitions certainly provide a good starting point, the world of QRM has many different challenges, including:

1. leading-edge science and new types of therapies
2. expectations for speed (e.g., product development in response to a pandemic or other market pressures)
3. human behaviors (e.g., expert opinion, heuristics, bias) multi-step QRM process with many different participants over a long-time horizon.

It is difficult to recognize when subjectivity enters QRM and influences outputs that may affect product quality and patient outcomes. The authors suggest this is an important topic for future study. Notwithstanding, the lack of a definition for subjectivity ICH Q9(R1) does offer important clues by its use of other language, in particular language that evokes objectivity as the antidote for subjectivity in a scientific setting, as explored in next section.

Subjectivity: a closer examination of its emergence in ICH Q9(R1)

While ICH Q9(R1) (8) does not expressly define subjectivity, an examination of it in comparison to the 2005 version of ICH Q9 (2) reveals the emergence of the term subjectivity, which appears 10 times in the revised document in contrast to the lack of being mentioned in the original.

Furthermore, the authors have observed that the language in ICH Q9(R1) includes concepts related to subjectivity- and objectivity as its antonym, for example:

1. synonyms for subjectivity (e.g., bias)
2. terms which invoke scientific facts which suggests minimizing subjectivity (e.g., evidence, data, information, knowledge)
3. knowledge management (KM) as a means to manage and deliver scientific knowledge to QRM and risk-based decision-making processes.

A simple comparison of the two documents was conducted by the authors to evaluate the frequency of relevant terms in each ICH Q9 and ICH Q9(R1), the results are presented in Table I.

As can be seen by the comparison, there is a significant increase in subjectivity related concepts in ICH Q9(R1). The authors believe this signifies a proactive desire to address subjectivity. By incorporating these related concepts, the revised guideline emphasizes the importance of basing risk assessments on objective evidence and data-driven insights. This shift toward more objective decision-making that is evidence-based, while seeking to minimize the influence of subjectivity enhances the reliability of risk analyses and risk management at large.

Conclusion

Subjectivity emerges in ICH Q9(R1) (8) as a concept that must be addressed to improve the effectiveness of risk management, and in turn the pharmaceutical quality system. In fact, O’Donnell challenges “can an organization do effective Risk-based Decision-making if its risk assessments are highly subjective?” (16). As mentioned by experts, acknowledging and addressing subjectivity are crucial for making informed and effective risk-based decisions, ultimately leading to improved product quality, patient safety, and product availability. Indeed, subjectivity casts a long shadow on QRM and, by extension, the pharmaceutical industry as a whole.

Updates included in the revision to ICH Q9(R1) reflect the growing recognition of the significance of subjectivity in risk management within the pharmaceutical industry. By incorporating language related to bias, evidence, data, information, knowledge, and knowledge management, the revised guideline appears positioned to enhance the objectivity and reliability of risk assessments. These changes included in ICH Q9(R1) provide an opportunity to shine a light on this issue by emphasizing the need to address subjectivity and its connection to business outcomes and patient safety.

However, while subjectivity has emerged as important in ICH Q9(R1), significant work remains to develop a clear and shared understanding of what subjectivity is, when subjectivity might exist and what to do about it when it does. These topics will be the focus of future research by the authors. Such research is also expected to explore subjectivity in other industries and study best practices. In addition, it is anticipated that training on this topic, and exploring emerging technologies and methods, would be beneficial.

Disclaimer

The views expressed in this article are those of the authors and are not necessarily those of any organizations with which the authors are affiliated.

References

1. ICH. Final Concept Paper ICH Q9(R1): Quality Risk Management. 2020. http://www.ich.org
2. ICH. Q9 Quality Risk Management (ICH, 2005). https://database.ich.org/sites/default/files/Q9%20Guideline.pdf
3. ICH. Q10 Pharmaceutical Quality System. (ICH, 2008). https://database.ich.org/sites/default/files/Q10%20Guideline.pdf (accessed: Aug. 28, 2023).
4. O’Donnell, K. Quality Risk Management–An Update on the Ongoing Revision of ICH Q9 ‘Delivering Value and Resilience using QRM’ [Conference Proceeding]. in PDA Ireland, Dublin, Ireland, September 2022.
5. Ramnarine, E. Solving the Continual Improvement and Innovation Challenge for the Benefit of Patients: How an Effective Pharmaceutical Quality System (PQS) and Risk-Based Approach Could Transform Post-Approval Change (PAC) Management. 2021. DOI: 10.21427/960F-8S69.
6. CDER. 2019 Drug Shortages Report. 2019. https://www.fda.gov/drugs/drug-shortages/report-drug-shortages-root-causes-and-potential-solutions
7. Peters, G. Risk of Drug Shortages-United States Senate Committee- Homeland Security & Governmental Affairs Report. 2023. https://www.hsgac.senate.gov/wp-content/uploads/2023-03-20-HSGAC-Majority-Draft-Drug-Shortages-Report.pdf
8. ICH. Q9(R1), Quality Risk Management (ICH, 2023). https://database.ich.org/sites/default/files/ICH_Q9%28R1%29_Guideline_Step4_2023_0126_0.pdf
9. Claycamp, G. Controlling Subjectivity in Risk-based Decision Making: The Benefits of Multi-criteria Decision Analysis. DIT Academic Press, 2018.
10. O’Donnell, K. Strategies for Addressing the Problems of Subjectivity and Uncertainty in Quality Risk Management Exercises Part I-The Role of Human Heuristics. Journal of Validation Technology, 2010.
11. O’Donnell, K. Strategies for Addressing the Problems of Subjectivity and Uncertainty in Quality Risk Management–Part 2 Risk Communication and Perception Issues. Journal of GXP Compliance, 2011 15 (1).
12. Vesper, J. and O’Donnell, K. Current Challenges in Implementing Quality Risk Management. iSpeak Blog 2016 (accessed: Aug. 31, 2023). https://ispe.org/pharmaceutical-engineering/ispeak/current-challenges-implementing-quality-risk-management
13. Ramnarine, E. and O’Donnell, K. Demonstrating Pharmaceutical Quality System Effectiveness and Driving Continual Improvement: Evidence-based Risk Reduction. PDA J Pharm Sci Technol, 2018 72 (3), pp. 338–345, May 2018, DOI: 10.5731/pdajpst.2017.008524.
14. Mulholland, V. and Greene, A. Improving Risk Based Decision Making Effectiveness: A Case for Risk Decision Review Points (KDRPs) In The Quality Risk Management Process. KENX Insight 2023 3 (3).
15. Mulholland, V. and Greene, A. Improving Risk Based Decision Making Effectiveness: Addressing Uncertainty. KENX Insight 2023 3 (5).
16. O’Donnell, K. ICH Q9(R1) and Some Personal Thoughts about Implementing the New Guidance [Conference Proceeding]. in TU Dublin, PRST Conference, Dublin, Ireland, June 2023.

About the authors

Amin Ziaie, QRM Merck & Co., Inc, PRST; Anne Greene is director PRST; and Marty Lipa, PhD, PRST; all at the Technological University Dublin.

Article details

Pharmaceutical Technology®
Vol. 48, No. 3
March 2024
Pages: 30-33

Citation

When referring to this article, please cite it as Ziaie, A.; Green, A.; and Lipa, M. Shining a Light on the Long Shadow of Subjectivity in Quality Risk Management. Pharmaceutical Technology 2024 48 (3).