A Harmonized Approach to Performing a Risk-Based Audit Trail Review

Audit trail review (ATR) is a mechanism to detect potential critical changes to data/system security settings and to ensure the quality and integrity of reported data. The authors have defined a risk-based approach to ATR where ATR is only required for high impact GxP (good manufacturing practices [GMP] and good laboratory practices [GLP] for the purposes of this paper) analytical data and possible system security changes. This approach requires a fully documented risk assessment that encompasses the technical controls and identification of data impact. Note that while analytical data are the focus of this paper, the principles outlined may be applied to other activities.

Regulatory expectations

Data integrity, particularly electronic data integrity, has become an area of increased regulatory focus. Per FDA, “For purposes of this guidance, audit trail means a secure, computer-generated, time-stamped electronic record that allows for reconstruction of the course of events relating to the creation, modification, or deletion of an electronic record” (1).

In 2018, the United Kingdom’s Medicines & Healthcare products Regulatory Agency (MHRA) and the US FDA issued guidance documents on the topic. MHRA’s ‘GXP’ Data Integrity Guidance and Definitions was issued in March 2018. FDA’s Data Integrity and Compliance with Drug CGMP—Questions and Answers was issued in December 2018. These documents join guidance issued by the World Health Organization (Guideline on Data Integrity) and the Pharmaceutical Inspection Convention Pharmaceutical Inspection Co-operation Scheme (PIC/S, Good Practices for Data Management and Integrity in Regulated GMP/GDP Environments) (1–4).

The publication of these guidance documents is associated with enforcement actions with an emphasis on data integrity that stem from a failure to follow current good manufacturing practices (CGMPs) predicate rules and existing regulations in 21 Code of Federal Regulations (CFR) 211 for electronic systems (5).

PIC/S Good Practices For Data Management And Integrity In Regulated GMP/GDP Environments—July 2021 (6)—gives an indication of the key elements to consider for an effective risk-based approach: “Data criticality (impact to decision making and product quality) and data risk (opportunity for data alteration and deletion, and likelihood of detection/visibility of changes by the manufacturer’s routine review processes).”

Therefore, regulatory expectations for audit trail review have become an established part of the GxP data lifecycle.

Scope and intended use

This article introduces a harmonized approach to performing a risk-based ATR developed by a working group of the International Consortium for Innovation and Quality in Pharmaceutical Development (IQ).

It should be noted that the scope of this article includes electronic instrument analytical data where raw data are stored in non-volatile memory (i.e., can be recalled later).Both enterprise and standalone data acquisition systems are in scope.

Systems that do not generate data are out of scope.

The following terms are defined (5):

• technical control—computerized features like audit trail, backup mechanism, user management and security, electronic signatures and/or digital signatures to assist or enforce administrative and procedural controls
• procedural control—standard operating procedures (SOPs) and work instructions for operation and administration, system user controls, computer system validation, calibration, network qualification, awareness training, etc.
• system controls—combination of procedural and technical controls for a system.

Risk-based approach

Recent regulatory guidance such as those from FDA and MHRA emphasize the implementation of risk-based approaches to ensuring data integrity. The FDA guidance reminds us that, “CGMP regulations and guidance allow for flexible and risk-based strategies to prevent and detect data integrity issues” (1).

Similarly, the MHRA guidance describes “a risk-based approach to data management that includes data risk, criticality and lifecycle” (2).

The concept of performing a data integrity risk assessment specific to a particular data acquisition and processing system is laid out in the MHRA guidance:

“An example of a suitable approach is to perform a data integrity risk assessment (DIRA) where the processes that produce data or where data are obtained are mapped out and each of the formats and their controls are identified and the data criticality and inherent risks documented” (2).

The data integrity risk assessment is seen as a driver of compliance and prioritization of any necessary remediation activities. While audit trail review is often considered an essential part of ensuring data integrity, the same guidance clarifies that routine data review should include a documented audit trail review where this is determined by a risk assessment (emphasis added) (2).

The appropriateness of any mitigation of a data integrity risk should be assessed in the context of the criticality of the gap. MHRA defines critical risks as those that impact the potential of data or metadata “to be deleted, amended, or excluded without authorization.”FDA states that, “Data integrity is critical throughout the CGMP data lifecycle, including in the creation, modification, processing, maintenance, archival, retrieval, transmission, and disposition of data after the record’s retention period ends” (1).

It should be noted that archival and retrieval are out of scope for this paper on ATR.

A decision tree has been developed (Figure 1) where data types were categorized and the need for audit trail review considered. This serves as a risk assessment that can be used to determine the need for procedural controls, and the controls should be documented within the qualification package for new equipment or in change management system for equipment updates. A risk assessment, for instance the one described in Assessing Data Integrity Risks in an R&D Environment (7), may be used to define data integrity elements for a system where audit trail review is the chosen mitigation. Figure 1 is specific to ATR and does not include data review. For GLP, data review and ATR need to happen at the same time, for GMP there may be opportunity to separate and streamline some activities with a documented risk-based approach.

Determining the need for and frequency of ATR

Data risk. ATR should be considered for electronic GxP relevant data when a technical control does not remove the need to review the audit trail. A risk-based approach should be applied to ATR, and this general approach is described in Figure 2. Tools such as the risk filtering tool in International Council for Harmonisation (ICH) Q9 (8) may be used.

When possible, there is a preference to implement technical controls to reduce/eliminate the need for ATR. It is preferred to prevent an undesirable action from occurring if this is technically feasible. In cases where prevention is not possible, detection of the undesirable action through data review (including ATR) is required. In rare (limited) cases where an action may be neither prevented nor detected, discuss with additional business/IT/quality assurance (QA) support to risk assess whether the system may be used for GxP work or if alternate controls need to be put in place such as a procedural control.

It is preferred that a technical control be employed to prevent data deletion. If deletion cannot be prevented, the ATR process should be designed to detect that specific activity. It should be noted that, in many cases, ATR is most logically performed concurrent with other data review activities. The severity of any residual risk should be assessed. The frequency for ATR should be commensurate with the probability of the risk occurrence. The frequency may be adjusted based on documented historical performance.

Intended use of the data. The intended use of the data should also have an impact on the need for and frequency of ATR. The data’s potential risk impact on patient safety and product quality should be considered, and GxP-relevant data are determined by regulatory requirements.

High risk impact data are defined as data with potential for direct impact to product quality and patient safety. Individual companies may identify other activities as high impact but at a minimum would include release, clinical stability, and cleaning verification.

While it is acknowledged that companies may assign slightly different levels of impact to the same data types, some useful guidance may be obtained from an informal poll conducted of IQ member companies. Data arising from activities such as GLP studies, cleaning verification, clinical product release, and stability were considered greater impact and may trigger ATR. Activities such as method validation may have an indirect effect on product quality and patient safety and may be less impactful and have a medium/low impact dependent on a company’s risk considerations.

Defining appropriate ATR effort/frequency. Together, the assessments of the systemcharacteristics and limitations and the impact of the data’s intended use will facilitate identification of records, steps, and changes, and enable risk classification (low to high) and a tiered audit trail review effort.

Performing data ATR

A decision tree describing the performance of ATR is provided in Figure 1. Additional explanatory comments about the decision tree are given as follows.

Where it is possible for changes to be made to the experimental conditions, metadata, or other parameters that have the potential to affect the results/data, one control strategy to ensure detection is to perform ATR. In these cases, the following should be considered as critical changes (and potentially included in a list of elements to be checked):

• changes to test parameters
• changes to data processing parameters (analytical method)
• deletion of data
• repeated analysis or reprocessing without justification
• change history of finished product test results
• changes to sample sequences
• changes to sample identification
• changes to critical process parameters.

The requirements for ATR (and data verification in general) should be proceduralized and should define requirements on a software-by-software basis dependent on the assessed data risk and impact. The frequency and responsibility for ATR should be defined in the procedure. Evidence of audit trail review should be documented, in most cases by defining the meaning of the overall review signature.

System level ATR

The purpose of a system level ATR is to ensure key configurations and settings have not been changed (either intentionally or unintentionally). It is recommended that the system audit trail (which contains, for example, system administrator actions such as deletion of data or changes to system security settings) also be reviewed at least periodically. This periodic review will ensure the system has remained configured as it was during validation/qualification. Based on the type of the system and corresponding data, the following items might be considered:

• system policies
• deactivation of audit trail
• changes to data paths or folder structure
• changes to reports or calculations
• data security management (lifecycle including archival, restoration, etc.)
• audit trail review may be used to verify appropriate access privileges have been used. Other processes may be employed to satisfy this requirement such as user access reviews (including admin privileges)
• configuration files
• library files (where applicable) where the technical controls of the library would drive the need and frequency for review.

A decision tree has been developed (Figure 3) for system-level ATR where data types were categorized with examples and the need for audit trail review considered.

Appropriate audit trail comments

Manually entered audit trail comments should be suitable for an auditor/inspector to read and should include the scientific rationale for why the change was made. GAMP 5 (9) provides audit trail requirements for an audit trail entry. Note: There may be character limitations, so it may be necessary to document justification outside of the electronic

system and include a cross-reference. Manually entered comments will also require review and should be contemporaneous (within a reasonable experiment/review time frame). If time has elapsed, then the time gap should be supported with a justification.

Conclusion

ATR is a mechanism to detect potential critical changes to data and one means to ensure the quality and integrity of reported data. The authors have defined a pragmatic risk-based approach to ATR where ATR is only needed for high impact GxP data and that ATR can be targeted to focus specifically on critical changes that may be possible. This approach requires a fully documented data risk assessment that encompasses the technical controls, identification of relevant high-risk impact data, which may vary on a per-organization basis. The preferred approach is to utilize technical controls wherever possible.

ATR inherently remains a manual process that is resource intensive, and opportunities for automated data transfer/digitalization removes opportunity for critical changes negating the need for ATR where utilized. In addition, when a system has been configured to provide visual flags for undesirable actions/states, or when the audit trail provides filtering or searching capabilities, these abilities may be leveraged to streamline ATR.

References

1. FDA, Data Integrity and Compliance with Drug CGMP—Questions and Answers (Rockville, MD, December 2018).
2. MHRA,‘GXP’ Data Integrity Guidance and Definitions (London, UK, March 2018).
3. World Health Organization, Guideline on Data Integrity (Geneva, Switzerland, October 2019).
4. PIC/S, Good Practices for Data Management and Integrity in Regulated GMP/GDP Environments (Geneva, Switzerland, November 2018).
5. US CFR, Title 21, Food and Drugs (Government Printing Office, Washington, DC), Part 211.
6. PIC/S Guidance PI 041-1: Good Practices for Data Management and Integrity in Regulated GMP/GDP Environments (July 2021).
7. J. Lippke et al., Pharm. Technol., 44 (8) (51–53) (2020).
8. ICH, Q9 Quality Risk Management (ICH, Geneva, 2005).
9. ISPE, GAMP 5: A Risk-Based Approach to Compliant GxP Computerized Systems (February 2008).

About the authors

Julie Lippke* and Joseph Mongillo both work in Analytical Research and Development at Pfizer Inc. (Groton, CT); Thomas Cullen and Christian Metz both work in Analytical Research and Development at AbbVie Inc. (North Chicago, IL and Ludwigshafen, Germany, respectively); Katria Harasewych works in Computer System Validation Quality, CoE at Merck & Co., Inc., (West Point, PA); Fouad Benamira works in Analytical Development Sciences for Biologicals at UCB S.A. Belgium. All contributors are part of the IQ Working Group.
*To whom all correspondence should be addressed.

Article details

Pharmaceutical Technology
Vol. 46, No. 9
September 2022
Pages: 48–53

Citation

When referring to this article, please cite it as J. Lippke, et al., “A Harmonized Approach to Performing a Risk-Based Audit Trail Review,” Pharmaceutical Technology 46 (9) 2022.