Steps Towards Demystifying Risk-Based Decision Making

Editor’s Note: This article was peer-reviewed by a member of Pharmaceutical Technology®’s Editorial Advisory Board.

Submitted: June 11, 2023
Accepted: March 5, 2024

In November 2020, the International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH) announced it was appointing an Expert Working Group (EWG) to update the quality guideline, ICH Q9 Quality Risk Management (1). The concept paper, published in support of this announcement, noted a number of areas of improvement, one of which was risk-based decision-making (RBDM) (2). The concept paper stated that there was a “lack of clarity” on what good RBDM means, and that that there was a “breadth of peer reviewed research” in this area. The Pharmaceutical Regulatory Science Team (PRST), based in Technological University Dublin, embarked on an exploration of this research, and this paper presents some of the outputs of this study. The hope is to clarify or ‘demystify’ what RBDM is, and to offer some suggestions as to how to improve or enhance RBDM to improve quality risk management (QRM) effectiveness for the benefit of the patient.

Demystifying RBDM

There are many decisions made within a pharmaceutical product lifecycle and within a pharmaceutical manufacturing operation. Most of these decisions involve the consideration of risk, including risk to patient safety, product quality, and product availability. In this context, it is prudent to focus upon RBDM and to consider why it differs from other decision-making within a pharmaceutical quality system (PQS) and why, as is proposed by the ICH Q9(R1) concept paper, that by improving RBDM, the output of QRM processes might also be improved.

This paper is structured using the 5W1H method of problem definition (a tool with intriguing origins in a Rudyard Kipling poem [3]). The questions, illustrated in Figure 1, are used to explore the subject of RBDM.

Why focus on RBDM? The most obvious answer to why one should focus on RBDM is because it is recommended in the updated risk management guideline ICH Q9(R1), which was published on Jan. 18, 2023 (4). This revision contains a new section, section 5.2, on RBDM. This alone should focus one’s curiosity on RBDM. However, the authors propose that by using another tool of problem solving, the 5Whys, perhaps further layers of ‘why?’ (at least three) are needed to fully appreciate the objective of adding RBDM to the revision.

Why add RBDM to ICH Q9(R1)? The ICH committee explained, in the 2020 concept document, why the topic of RBDM was under consideration. According to this document, it was determined that there was “a lack of clarity” with both the term and with the application. How this problem statement was determined is a little less clear.

However, the main reason for its inclusion might be that the full value of QRM, as envisaged when the original ICH Q9 was published in 2005, was not fully realized. The aspiration that QRM would enhance the effectiveness of the PQS remains a work in progress. This is evidenced by continued drug shortages, with quality problems (with both products, and the processes that manufacture them) identified as the highest contributory factor (5,6).

It is important to realize that effective QRM processes may, on rare occasion, result in unfortunate and unwanted outcomes, such as product quality issues or shortages. In these instances, it is important to demonstrate that the process (of QRM) was applied properly and diligently and that the decisions made were as informed as possible, given what was known or understood at the time. In other words, that the unwanted outcome is not a consequence of a lack of rigor. However, it does not appear that this is an easy matter to demonstrate. Kevin O’Donnell from the Irish Health Products Regulatory Authority (HPRA) noted that pharmaceutical companies “struggle to present evidence that their QRM activities are actually effective” (7). In the same publication, O’Donnell noted that, “While it is clear that the industry has done a lot of work in the area of QRM, it may have reached a plateau in that work, where advances in the area have stagnated to some extent.”

One area of QRM activity that heretofore has not received much attention is that of the decision-making within the process. The authors propose that, perhaps, this is the secret sauce to improve both the process and the output of QRM and address the shortfalls currently observed.

Why focus on RBDM within QRM? As products, processes, packaging, and supply chains become more complex both technologically and logistically, the nature and likelihood of hazards, risks, and impacts, also becomes more complex. Most problems encountered in pharmaceutical operations are ‘tame problems’ (i.e., they can be described and resolved with the application of root cause analysis, the right knowledge and expertise, and the appropriate corrective and preventive actions) (8). This is not to imply that these problems are simple—both the problem and the solution can be complicated—but they fit within a ‘cause-and-effect’ paradigm.

Increasingly, however, some problems in modern operations display the characteristics of ‘messy’ or ‘wicked’ problems (9,10). Wicked problems are characterized by multiple contributary causes, with multiple knock-on effects. Consequently, the number and formality of risk analysis also typically increases, as the multiple contributary elements are analyzed. Consequently, the level of formality applied to the decision-making process should also increase. Decision complexity has also been described in terms such as convergent and divergent, where solutions either resolve into one solution or not (11).

It is in these areas of decision complexity, where formality becomes important. Research in the area of decision science informs us that humans, for many reasons, do not perform as rationally as they would like to think when faced with complex decisions (12). This is particularly true when these decisions are about risk and when there is a level of uncertainty involved (13).

Decisions in these contexts are often supported by much data, from multiple sources, a circumstance which humans often find overwhelming. In these cases, the instinct is to reduce the information to more easily interpreted formats such as graphs, charts, or dashboards. However, insights are often lost in these reductive representations.

Alternatively, complex decisions may be required in circumstances where there is little data. It has been established that when information is limited and uncertain, decision-making about risk requires the application of a different logic or mental reasoning, than those used to problem solve many routine PQS decisions (14). Typically, within the PQS, decisions rely heavily on factual evidence; therefore, deductive reasoning is widely used, and the skill of deduction is improved with much practice. However, RBDM requires inductive reasoning, which is called upon less frequently in routine PQS decision making, and therefore is not as well honed. The authors illustrate the two types of reasoning in Figure 2, which shows the comparative characteristics of deductive and inductive reasoning processes (15).

In data-deficient contexts, concern with respect to RBDM accuracy is further complicated when RBDM relies on knowledge acquired from experts via expert opinion (which gives rise to concerns of subjectivity), or from computerized algorithms, search engines, mathematical models, and more recently artificial intelligence (giving rise to concerns of representation), all of which, somewhat counter-intuitively, add to uncertainty rather than resolve it (16,17).

RBDM within QRM requires different mental abilities, which renders it a competency that may not be fully appreciated when important, complex, and uncertain risks are under analysis. Unless the skills are developed and honed, the more typical approaches and modes of reasoning, may overwhelm and mislead the decision-maker.

Why might improving RBDM improve QRM output? It is possible, that by appreciating the skills needed to optimize RBDM, that the quality of the decisions made in QRM might improve, thus improving the output and the effectiveness of QRM in supporting PQS decision-making. O’Donnell also observed that, “The emphasis has probably been more on the mechanics of risk assessment tools/scores rather than on the quality of decisions made based on the outcomes of risk assessments” (7).

As this is one aspect of QRM that has not been discussed and advanced, enhancing RBDM approaches are likely to realize improvement.

What is RBDM?

ICH Q9(R1) provides a definition of RBDM as, “An approach to, or a process of, making decisions that considers knowledge about risks relevant to the decision and whether risks are at an acceptable level” (4).

To fully understand the nature of a hazard (i.e., how it might arise in the future, and consequently how it might be prevented, mitigated, or controlled), the more complete the knowledge about that risk the better. Therefore, a critical input into a QRM process is knowledge. A supporting knowledge management (KM) process is needed to ensure that accurate and reliable knowledge is available for RBDM within QRM.

The ICH guideline on the PQS, ICH Q10 Pharmaceutical Quality Systems (18) proposed that both QRM and KM were enablers of decision-making processes within the PQS. To date, QRM has probably evolved in organizations more rapidly than its twin enabler KM, Calnan et al. referred to the latter as the ‘orphan enabler’ (19).

Lipa et al. (20) proposed, using a framework referred to as the Risk-Knowledge Infinity (RKI) cycle, illustrated in Figure 3, that these processes are both interdependent and iterative. This interconnection and its importance to QRM was discussed previously by the authors (21). The key takeaway is that robust QRM is founded on the availability of sound scientific knowledge of risk, while the output of QRM further informs the organization’s understanding of risk management and risk control. Both are essential requirements to support the risk-based decision maker (i.e., a strong understanding of the risk and the analysis of those risks).

The skill of RBDM is to make good decisions about the control of risk, using what is known about that risk. When much is known, these decisions are not onerous.

However, when knowledge is limited, as is often the case in real-world situations, then RBDM is more uncertain and more challenging. In this circumstance, illustrated in Figure 4, the decision maker has two options. The first is to delay the decision and to seek further knowledge. While this is perhaps the preferred option, it is not always possible. The second option is to improve the capability and accuracy of both the analysis of risk, and RBDM under uncertainty. In this circumstance, reliance on RBDM competencies becomes more critical, as the application of inductive reasoning skills is required.

Where is RBDM performed?

A focus group, held by the authors in 2022, attended by senior quality professionals from the pharmaceutical industry, confirmed that there was confusion as to where RBDM is performed. The most common perception is that all decisions within the PQS are risk-based decisions, as most, if not all, involved some form of risk. While the underlying premise is true, it is not however an accurate perspective. Neither is it helpful.

If all decision-making processes within the PQS and QRM are considered to be similar, when in fact they are not, it is difficult to develop consistent and informed approaches. Decision types differ with respect to context, values, process, available tools, or as mentioned previously the reasoning, logic, and skills needed. If RBDM is to be improved, then a key starting point must be to recognize where it is occurring and, in that context, focus on its enhancement.

Through researching the term RBDM, and its use and application in other risk-sensitive industries, the authors found that the term RBDM is more commonly used to describe the technical characterization of risk, such as that conducted in the risk analysis phase of the QRM process. The term is used to refer, for example, to the decisions made with respect to risk analysis tools, scoring schema, acceptance criteria, control strategies, and review and monitoring mechanisms. The output of RBDM and QRM (i.e., an understanding of the ‘state-of-control’ with respect to risk) may then be used to inform and support PQS decision making, with the term risk-informed decision-making (RIDM) commonly used to describe this latter context (22).

A more bounded use of the term RBDM, confining it to the decisions made within the QRM process is recommended, thus allowing for the development of best practice, as advised by the revised ICH Q9(R1). Even in this context and within these boundaries, there are multiple decisions made within a single QRM process, and numerous QRM analyses and decisions may combine or compound to control a single product risk. This is particularly true in complex problem settings, with multiple stakeholders, dispersed operations, and long lifecycles. Therefore, even within this narrower interpretation of the term, there is still much complexity and several layers of decision-making.

When should formality be applied to RBDM?

It is clear from ICH Q9(R1) that all risk management decisions should be approached with some level of formality, but that the extent of that formality may vary along a continuum from low to high. As problems increase in complexity, so do the decision-making processes required to support them.

QRM, when performed during a pharmaceutical product lifecycle, can be an elongated process occurring at different times over a long timeframe, by different expert teams, in different operational functions with different geographies, cultures, or ownership. In such long horizon processes, early RBDM (e.g., those associated with the choice of formulation and materials) can be important, complex, and uncertain, and are often made with limited available knowledge. Yet such decisions may commit the entire product lifecycle to irrevocable pathways (i.e., it is difficult to reverse the decisions made). Often such irrevocable decision points limit the choices available to subsequent decision-makers, and thus they should be made with care.

The factors that might drive the process to the higher end of the scale and require higher formality are suggested by the authors as follows:

• When the hazard or risk under analysis is important (i.e., has a high potential to impact the quality and/or the availability of the product and ultimately to compromise patient safety).
• When the decision is complex due to the technical, logistical, or interdependencies in the risk or system under analysis. Complexity may also be contributed by differing stakeholder preferences or objectives.
• When there is limited knowledge of the hazard, how it might arise, how frequently, and/or how it might be effectively controlled, all of which contribute to uncertainty with respect to QRM outcomes.

In cases such as these, higher formality is appropriate and more structured approaches to the RBDM process, such as those illustrated in Figure 5, gives enhanced clarity and transparency into how decisions are made (23).

Who performs RBDM?

When taking higher formality approaches, ICH Q9(R1) recommends that RBDM be performed by cross functional teams. It is important that the understanding and analysis of risk is developed through the input of diverse views, opinions, and expertise. However, the hazards of multiple viewpoints and group decision-making must also be considered.

Formal and structured decision-making processes and tools attempt to control the challenges of multiple contributions and constrain issues such as subjectivity, biases, heuristics, cultural differences, internal and external politics, power imbalances, and differing priorities, risk tolerances, and acceptance criteria (24). Recognizing the role of the decision-maker and decision-making in the QRM processes ensures that these important steps are performed accurately. ICH Q9(R1) addressed the role of decision-makers, as follows (4):

“Decision makers should:

• take responsibility for coordinating quality risk management across various functions and departments of their organization.
• assure that a quality risk management process is defined, deployed, and reviewed and that adequate resources and knowledge are available; and
• assure that subjectivity in quality risk management activities is managed and minimized, to facilitate scientifically robust risk-based decision-making” (4).

Another consideration when decisions are important is the likelihood that decision-makers are senior in the organization and are not the same people as those performing the risk analysis. This introduces the importance of risk communication in QRM. Visualization and representation of the output of risk analysis, particularly when complex approaches are taken, should be such that the decision-maker is not misled with respect to any uncertainty associated with the analysis or with the output.

This, once again, raises the question of decision-making competency. Returning to the concept that RBDM is reliant on logic and reasoning that differs from other decision-making contexts, an understanding of the underlying skills and knowledge required to develop inductive reasoning and how to apply it to RBDM is warranted. These skills have been identified in order of influence, as numeracy, statistical numeracy, risk literacy, and problem solving (fluid intelligence) (25, 26).

The deductive decision-making approaches that are more typical in PQS decision-making are commonly less reliant on numeracy. Yet numeracy, and a knowledge of statistics, is a key asset in the interpretation of risk analysis, particularly where probability or modeling has been employed. A key argument for confining the concept of RBDM, to within the QRM process, is to develop and target the correct training and to practice RBDM competency to enhance improvement.

A final consideration when considering who performs RBDM is the assumption made in ICH Q9(R1) that decision-making is a human function. This assumption may not completely future proof RBDM. The rate of progress in the areas of data analytics, machine learning, and artificial intelligence is such that future decision-making about risk, particularly with respect to risk control, will be made by computerized agents, or by humans with increased reliance on computerized decision support systems. In these cases, an understanding of the underlying decision-making rules and the reasoning applied within any programs or algorithms used, is also key to accurate outcomes.

The risk analysis may suggest courses of action based on expected value (either cost or quality), as a balance of risk and benefits, or as risk reduction opportunities, expressed with respect to defined thresholds of acceptable risk, or in terms of avoidance of worst-case scenarios, often with little knowledge of the decision-makers preferences or risk tolerance (27). Therefore, the connection, communication, and understanding between the decision-maker and the risk analyst is important to recognize and define, irrespective of whether both are, or either is, human.

How might RBDM be improved?

With all these considerations in mind, the authors researched RBDM in other sectors. The focus was on industries with high sensitivity to risk (e.g., nuclear, aviation, aerospace, oil and gas, etc.). This research informed the development of 21 attributes of good RBDM. These attributes are illustrated in Figure 6. The attributes were then categorized as the characteristics of good governance and human related factors, of robust QRM, and of knowledge management (28).

Further research into these attributes will ultimately allow for the development of a guidance or playbook on RBDM, but for now, the following four insights on how to enhance RBDM are emerging:

• Decide the appropriate level of formality to support the RBDM.
• Provide the best available knowledge to inform and support the decision.
• If knowledge is limited, ensure the correct skills are in place to interpret any predictive analysis.
• Review RBDM throughout the QRM process.

While the first three recommendations have been discussed in previous sections, the recommendation to review RBDM throughout a QRM process has not. However, a ‘lessons learned’ or review process is key to the improvement of any process or system and is an important element of a KM process. Like any skill, RBDM can benefit from the application of hindsight and reflection to determine if, with what was known at the time, the optimal decisions were made. Using this information, the process (of RBDM) can be improved with further enhancements, such as correct information flows, the right decision-makers, different tools, etc.

The concept of applying review node” to ’ process exists in other industries. Commonly at these reviews, the decision-maker determines whether the process has enough information and risk understanding to proceed to the next stage of the process with an acceptable assurance of success. These reviews also facilitate projects proceeding, but with certain ‘liens’ or assumptions, which are highlighted within the action plan for the next phase or phases until they are understood and acceptable.

For any company to improve its own decision-making capability, particularly under uncertainty, the introduction of key decision review points is a worthwhile consideration. The QRM process lends itself to natural pause points. Each review point can have an associated prompt list to assure that the understanding and control of risk is refined through the QRM process, bearing in mind that this may be over a long period of time. The concept of key decision review points is discussed in more detail in a previous publication by the authors (29).

Conclusion

RBDM is distinct from other types of decision-making within a PQS. As decision-making occurs in the context of risk characterization and control, factors such as prediction and probability are inherent. These factors create uncertainty, which is not an optimal context for human decision-making. Therefore, when decisions are important or complex, especially if the problems are messy or wicked, then applying the appropriate formality ensures a rigor to the decision-making process. Training to ensure that the competencies are available also provides greater assurance of successful outcomes. Supporting decision-makers with the correct knowledge is a further enhancement. Finally, reviewing RBDM in the context of QRM processes and outcomes allows for enhancements in future applications, thus fulfilling the opportunities of the RKI cycle.

References

1. ICH. Q9 Quality Risk Management (ICH, 2005). https://www.ich.org/page/quality-guidelines
2. ICH. Final Concept Paper for the ICH Q9 (R1) Quality Risk Management, November 2020. https://www.ich.org
3. Kippling, R. Just So Stories. MacMillan & Co, 1902.
4. ICH. Q9(R1) Quality Risk Management (Final Version) (ICH, 2023).
5. US Senate Committee on Homeland Security & Governmental Affairs. Short Supply: The Health & National Security Risk of Drug Shortages. March 2023.
6. FDA. 2019 Drug Shortages Report. Updated Feb. 21, 2020 (CDER, 2020).
7. O’Donnell, K. QRM in the GMP Environment beyond 2020: What Should We Strive For?, in An Audience with the International Regulators in the Manufacture of Medicine: QRM and KM, TU Dublin, 2018.
8. Rittel, H. W. J.; Webber, M. M. Dilemmas in a General Theory of Planning. Policy Sci., 1973 4, pp. 155–169.
9. Ackoff, R. Systems, Messes, and Interactive Planning, in Redesigning the Future, John Wiley & Sons, 1974.
10. Skaburskis, A. The Origin of ‘Wicked Problems’. Plan. Theory Pract., 2008 9 (2), pp. 277–280, June 2008, DOI: 10.1080/14649350802041654.
11. Schumacher, E. F. A Guide for the Perplexed. Harper & Row, 1977.
12. Simon, H. A. Administrative Behaviour. Macmillan, 1947.
13. Howard, R. A. Uncertainty about Probability: A Decision Analysis Perspective. Risk Anal., 1988 8 (1), pp. 91–98.
14. Mushtaq, F.; Bland, A. R.; Schaefer, A.; Pourtois, G.; Stürmer, B.; and Krebs, R. M.; Uncertainty and Cognitive Control. 2011, DOI: 10.3389/fpsyg.2011.00249.
15. Holland, J. H.; Holyoak, K. J.; Nisbett, R. E.; and Thagard, P. R. Induction: Processes of Inference, Learning, and Discovery, First. MIT Press, 1989.
16. Hanea, A. M.; Hemming, V.; Nane, G. F. Uncertainty Quantification with Experts: Present Status and Research Needs. Risk Anal., 2021 42 (2).
17. Amir, E. The Cambridge Handbook of Artificial Intelligence. Cambridge University Press, 2014.
18. ICH. Q10: Pharmaceutical Quality System (ICH, 2008).
19. Calnan, N.; Greene, A.; Kane, P. An Academic Perspective: Knowledge Management: The Orphan Enabler—Enabling ICH Q10 Implementation, in A Lifecycle Approach to Knowledge Excellence in the Biopharmaceutical Industry, Calnan, N.; Lipa, M. J.; Kane, P.; Cardoso Menezes, J. Eds., CRC Press, 2018, pp. 133–152.
20. Lipa, M.; O’Donnell, K.; Greene, A. Knowledge as the Currency of Managing Risk: A Novel Framework to Unite Quality Risk Management and Knowledge Management. Lev. 3, 2020 15 (2), p. 4.
21. Lipa, M.; Mulholland, V.; Greene, A. (editor). Improving Risk-Based Decision Making (RBDM) Effectiveness Through a Framework which Integrates Knowledge and Risk. Lev. 3, 2022 16 (2), February, Accessed: Feb. 28, 2022. [Online]. https://arrow.tudublin.ie/level3/vol16/iss2/2
22. Mulholland, V.; Greene, A.; Lipa, M. J. Steps Beyond Risk Assessment in QRM: RBDM, The Next Horizon. Lev. 3, 2021 16 (1).
23. Mulholland, V.; Greene, A. Improving Risk-Based Decision Effectiveness: Determining the Level Of Formality. KENX Insights, 2023 3 (4), Accessed: May 14, 2023. https://kenx.org/kenx_insights/improving-risk-based-decision-making-effectiveness-determining-the-level-of-formality/
24. Mulholland, V.; Greene, A.; Lipa, M. Improving Risk-Based Decision Effectiveness: Addressing Uncertainty. KENX Insights, 2023 3 (5), Accessed: May 14, 2023. https://kenx.org/kenx_insights/improving-risk-based-decision-making-effectiveness-addressing-uncertainty/
25. Cokely, E. T.; Ghazal, S.; Galesic, M.; Garcia-Retamero, R., & Schulz, E. How to Measure Risk Comprehension in Educated Samples. In R. Garcia-Retamero & M. Galesic (Eds.), Transparent Communication of Risks about Health: Overcoming Cultural Differences (pp. 29–52). (Springer, 2013).
26. Dieckmann, N. F.; Slovic, P.; Peters, E. M. The Use of Narrative Evidence and Explicit Likelihood by Decisionmakers Varying in Numeracy. Risk Analysis 2009 29 (10), 1473–1488. DOI: 10.1111/J.1539-6924.2009.01279.
27. Paté-Cornell, M. E.; Dillon, R. L. The Respective Roles of Risk and Decision Analyses in Decision Support. Decis. Anal., 2006 3 (4), pp. 220–232, December, DOI: 10.1287/deca.1060.0077.
28. ISPE. Good Practice Guide to Knowledge Management in the Pharmaceutical Industry (ISPE 2021).
29. Mulholland, V.; Greene, A. Improving Risk-Based Decision Effectiveness: A Case for Risk Decision Review Points (KDRPS) In the Quality Risk Management Process. KENX Insights, 2023 3 (3), https://kenx.org/kenx_insights/improving-risk-based-decision-making-effectiveness-a-case-for-risk-decision-review-points-kdrps-in-the-quality-risk-management-process/

About the authors

Valerie Mulholland is a PhD Candidate, Pharmaceutical Regulatory Science Team (PRST), Technological University Dublin. Anne Greene is director PRST, Technological University Dublin.

Article details

Pharmaceutical Technology®
Vol. 48, No. 4
April 2024
Pages: 32–39

Citation

Mulholland, V. and Greene, A. Steps Towards Demystifying Risk-Based Decision Making. Pharmaceutical Technology 2024 48 (4).