Risk identification, analysis, and evaluation
Figure 1: Fault-tree analysis (FTA) of equipment changes and associated validation impact. (ALL FIGURES ARE COURTESY OF THE
AUTHORS)
|
The risk-assessment process began with a review and analysis of the change control system to determine how equipment parts
replacements could potentially cause an unwanted or undetected change to the equipment's validated state. The analysis was
organized into the fault-tree structure (see Figure 1). This fault tree illustrates the potential means by which equipment
changes, such as parts replacements, could pose risk to the validated state of the equipment. The team concluded that many
of the potential fault pathways were already being appropriately mitigated by robust quality systems (e.g., training, validation,
and change control) that were performing as intended and that were being routinely audited. However, significant gaps and
improvement opportunities were noted around the process used for the functional-equivalence assessments (see Figure 1, yellow
pathway).
Figure 2: Fault-tree analysis (FTA) of functional equivalent assessments.
|
To further explore the risks associated with the functional-equivalence assessment process for equipment replacement parts,
the risk-assessment team continued development of the fault tree as shown in Figure 2. The team focused on two key areas of
risk: functional-equivalence assessments performed by parts vendors, and functional-equivalence assessments performed internally
by the firm's functional areas.
The detailed FTA executed by the risk-assessment team revealed two areas of significant risk where improvement was required:
- The Initiator (i.e., petitioner and preliminary data collector) for functional-equivalence evaluations should be a subject
matter expert (SME) who is appropriately trained and qualified to craft accurate initial assessments (see Figure 2, green
triangles).
- Specific roles and responsibilities for each functional area participating in functional-equivalence assessments should be
clearly defined (see Figure 2, beige triangles).
Risk control
For each of the two areas of significant risk identified in the FTAs and summarized above, associated risk-control plans were
established, as follows:
Training curricula were created to define the training and qualification criteria for personnel initiating functional-equivalence
assessments. These controls were designed to ensure that Initiators would be able to identify, compile, and/or generate the data and rationale
required to support thorough and accurate functional-equivalence assessments.
Roles and responsibilities for each functional area participating in functional-equivalence assessments were delineated in
the form of executable checklists designed to ensure that every functional-equivalence assessment will be performed in a thorough
and reproducible fashion. Each organization identified in Figure 2 (Engineering, Technical Services, Quality Assurance, and Regulatory) created a checklist
tailored to their specific roles and responsibilities that the team had collectively defined.
Table I: Checklist template for determining engineering functional equivalence.
|
This approach minimized both gaps and redundancies in the assessment efforts while also providing a common assessment record
format to facilitate overall review of the assessment package. Each functional area checklist details unique areas of consideration
for the assessment and provides spaces for the assessment conclusions and the signatures of the assessor(s). An example checklist
from the Engineering functional area is shown in Table I.
|
