Katria Harasewych, quality, Merck & Co., Inc., Kenilworth, NJ

Katria Harasewych

Audit trail review (ATR) is a mechanism to detect potential critical changes to data/system security settings and to ensure the quality and integrity of reported data. The authors have defined a risk-based approach to ATR where ATR is only required for high impact GxP (good manufacturing practices [GMP] and good laboratory practices [GLP] for the purposes of this paper) analytical data and possible system security changes. This approach requires a fully documented risk assessment that encompasses the technical controls and identification of data impact. Note that while analytical data are the focus of this paper, the principles outlined may be applied to other activities.

Data integrity, particularly electronic data integrity, has become an area of increased regulatory focus. Per FDA, “For purposes of this guidance, audit trail means a secure, computer-generated, time-stamped electronic record that allows for reconstruction of the course of events relating to the creation, modification, or deletion of an electronic record” (1).

In 2018, the United Kingdom’s Medicines & Healthcare products Regulatory Agency (MHRA) and the US FDA issued guidance documents on the topic. MHRA’s 

‘GXP’ Data Integrity Guidance and Definitions

Data Integrity and Compliance with Drug CGMP—Questions and Answers

 was issued in December 2018. These documents join guidance issued by the World Health Organization (Guideline on Data Integrity) and the Pharmaceutical Inspection Convention Pharmaceutical Inspection Co-operation Scheme (PIC/S, 

Good Practices for Data Management and Integrity in Regulated GMP/GDP Environments

The publication of these guidance documents is associated with enforcement actions with an emphasis on data integrity that stem from a failure to follow current good manufacturing practices (CGMPs) predicate rules and existing regulations in 21 

Good Practices For Data Management And Integrity In Regulated GMP/GDP Environments

—July 2021 (6)—gives an indication of the key elements to consider for an effective risk-based approach: “Data criticality (impact to decision making and product quality) and data risk (opportunity for data alteration and deletion, and likelihood of detection/visibility of changes by the manufacturer’s routine review processes).”

Therefore, regulatory expectations for audit trail review have become an established part of the GxP data lifecycle.

This article introduces a harmonized approach to performing a risk-based ATR developed by a working group of the International Consortium for Innovation and Quality in Pharmaceutical Development (IQ).

It should be noted that the scope of this article includes electronic instrument analytical data where raw data are stored in non-volatile memory (i.e., can be recalled later).Both enterprise and standalone data acquisition systems are in scope.

Systems that do not generate data are out of scope.

technical control—computerized features like audit trail, backup mechanism, user management and security, electronic signatures and/or digital signatures to assist or enforce administrative and procedural controls

procedural control—standard operating procedures (SOPs) and work instructions for operation and administration, system user controls, computer system validation, calibration, network qualification, awareness training, etc.

system controls—combination of procedural and technical controls for a system.

Recent regulatory guidance such as those from FDA and MHRA emphasize the implementation of risk-based approaches to ensuring data integrity. The FDA guidance reminds us that, “CGMP regulations and guidance allow for flexible and risk-based strategies to prevent and detect data integrity issues” (1).

Similarly, the MHRA guidance describes “a risk-based approach to data management that includes data risk, criticality and lifecycle” (2).

The concept of performing a data integrity risk assessment specific to a particular data acquisition and processing system is laid out in the MHRA guidance:

“An example of a suitable approach is to perform a data integrity risk assessment (DIRA) where the processes that produce data or where data are obtained are mapped out and each of the formats and their controls are identified and the data criticality and inherent risks documented” (2).

The data integrity risk assessment is seen as a driver of compliance and prioritization of any necessary remediation activities. While audit trail review is often considered an essential part of ensuring data integrity, the same guidance clarifies that routine data review should include a documented audit trail review 

where this is determined by a risk assessment 

The appropriateness of any mitigation of a data integrity risk should be assessed in the context of the criticality of the gap. MHRA defines critical risks as those that impact the potential of data or metadata “to be deleted, amended, or excluded without authorization.”FDA states that, “Data integrity is critical throughout the CGMP data lifecycle, including in the creation, modification, processing, maintenance, archival, retrieval, transmission, and disposition of data after the record’s retention period ends” (1).

It should be noted that archival and retrieval are out of scope for this paper on ATR.

) where data types were categorized and the need for audit trail review considered. This serves as a risk assessment that can be used to determine the need for procedural controls, and the controls should be documented within the qualification package for new equipment or in change management system for equipment updates. A risk assessment, for instance the one described in 

Assessing Data Integrity Risks in an R&D Environment

 (7), may be used to define data integrity elements for a system where audit trail review is the chosen mitigation. 

 is specific to ATR and does not include data review. For GLP, data review and ATR need to happen at the same time, for GMP there may be opportunity to separate and streamline some activities with a documented risk-based approach.

Risk assessment tool for determining audit trail review requirements. Where ALCOA+ is attributable, legible, contemporaneous, original, and accurate, plus the data needs to be complete, consistent, enduring, and available. 

Determining the need for and frequency of ATR

 ATR should be considered for electronic GxP relevant data when a technical control does not remove the need to review the audit trail. A risk-based approach should be applied to ATR, and this general approach is described in 

. Tools such as the risk filtering tool in International Council for Harmonisation (ICH) Q9 (8) may be used.

Determining the rigor of audit trail review (ATR) as a function of data risk and data impact. 

When possible, there is a preference to implement technical controls to reduce/eliminate the need for ATR. It is preferred to prevent an undesirable action from occurring if this is technically feasible. In cases where prevention is not possible, detection of the undesirable action through data review (including ATR) is required. In rare (limited) cases where an action may be neither prevented nor detected, discuss with additional business/IT/quality assurance (QA) support to risk assess whether the system may be used for GxP work or if alternate controls need to be put in place such as a procedural control.

It is preferred that a technical control be employed to prevent data deletion. If deletion cannot be prevented, the ATR process should be designed to detect that specific activity. It should be noted that, in many cases, ATR is most logically performed concurrent with other data review activities. The severity of any residual risk should be assessed. The frequency for ATR should be commensurate with the probability of the risk occurrence. The frequency may be adjusted based on documented historical performance.

 The intended use of the data should also have an impact on the need for and frequency of ATR. The data’s potential risk impact on patient safety and product quality should be considered, and GxP-relevant data are determined by regulatory requirements.

High risk impact data are defined as data with potential for direct impact to product quality and patient safety. Individual companies may identify other activities as high impact but at a minimum would include release, clinical stability, and cleaning verification.

While it is acknowledged that companies may assign slightly different levels of impact to the same data types, some useful guidance may be obtained from an informal poll conducted of IQ member companies. Data arising from activities such as GLP studies, cleaning verification, clinical product release, and stability were considered greater impact and may trigger ATR. Activities such as method validation may have an indirect effect on product quality and patient safety and may be less impactful and have a medium/low impact dependent on a company’s risk considerations.

Defining appropriate ATR effort/frequency. 

Together, the assessments of the systemcharacteristics and limitations and the impact of the data’s intended use will facilitate identification of records, steps, and changes, and enable risk classification (low to high) and a tiered audit trail review effort.

A decision tree describing the performance of ATR is provided in 

. Additional explanatory comments about the decision tree are given as follows.

Where it is possible for changes to be made to the experimental conditions, metadata, or other parameters that have the potential to affect the results/data, one control strategy to ensure detection is to perform ATR. In these cases, the following should be considered as critical changes (and potentially included in a list of elements to be checked):

changes to data processing parameters (analytical method)

repeated analysis or reprocessing without justification

change history of finished product test results

The requirements for ATR (and data verification in general) should be proceduralized and should define requirements on a software-by-software basis dependent on the assessed data risk and impact. The frequency and responsibility for ATR should be defined in the procedure. Evidence of audit trail review should be documented, in most cases by defining the meaning of the overall review signature.

The purpose of a system level ATR is to ensure key configurations and settings have not been changed (either intentionally or unintentionally). It is recommended that the system audit trail (which contains, for example, system administrator actions such as deletion of data or changes to system security settings) also be reviewed at least periodically. This periodic review will ensure the system has remained configured as it was during validation/qualification. Based on the type of the system and corresponding data, the following items might be considered:

changes to data paths or folder structure

data security management (lifecycle including archival, restoration, etc.)

audit trail review may be used to verify appropriate access privileges have been used. Other processes may be employed to satisfy this requirement such as user access reviews (including admin privileges)

library files (where applicable) where the technical controls of the library would drive the need and frequency for review.

) for system-level ATR where data types were categorized with examples and the need for audit trail review considered.

Decision tree for determining minimum system level audit trail review requirements.

Manually entered audit trail comments should be suitable for an auditor/inspector to read and should include the scientific rationale for why the change was made. GAMP 5 (9) provides audit trail requirements for an audit trail entry. Note: There may be character limitations, so it may be necessary to document justification outside of the electronic

system and include a cross-reference. Manually entered comments will also require review and should be contemporaneous (within a reasonable experiment/review time frame). If time has elapsed, then the time gap should be supported with a justification.

ATR is a mechanism to detect potential critical changes to data and one means to ensure the quality and integrity of reported data. The authors have defined a pragmatic risk-based approach to ATR where ATR is only needed for high impact GxP data and that ATR can be targeted to focus specifically on critical changes that may be possible. This approach requires a fully documented data risk assessment that encompasses the technical controls, identification of relevant high-risk impact data, which may vary on a per-organization basis. The preferred approach is to utilize technical controls wherever possible.

ATR inherently remains a manual process that is resource intensive, and opportunities for automated data transfer/digitalization removes opportunity for critical changes negating the need for ATR where utilized. In addition, when a system has been configured to provide visual flags for undesirable actions/states, or when the audit trail provides filtering or searching capabilities, these abilities may be leveraged to streamline ATR.

 (Rockville, MD, December 2018).
2. MHRA,

‘GXP’ Data Integrity Guidance and Definitions 

(London, UK, March 2018).
3. World Health Organization, 

 (Geneva, Switzerland, October 2019).
4. PIC/S, 

 (Geneva, Switzerland, November 2018).
5. US 

, Title 21, Food and Drugs (Government Printing Office, Washington, DC), Part 211.
6. PIC/S Guidance PI 041-1: 

GAMP 5: A Risk-Based Approach to Compliant GxP Computerized Systems

 both work in Analytical Research and Development at Pfizer Inc. (Groton, CT); 

 both work in Analytical Research and Development at AbbVie Inc. (North Chicago, IL and Ludwigshafen, Germany, respectively); 

 works in Computer System Validation Quality, CoE at Merck & Co., Inc., (West Point, PA); 

 works in Analytical Development Sciences for Biologicals at UCB S.A. Belgium. All contributors are part of the IQ Working Group.
*To whom all correspondence should be addressed.


Vol. 46, No. 9
September 2022
Pages: 48–53

When referring to this article, please cite it as J. Lippke, et al., “A Harmonized Approach to Performing a Risk-Based Audit Trail Review,” 

Articles

The IQ Working Group has defined a pragmatic risk-based approach to audit trail review, where it is only required for high impact GxP data.

A Harmonized Approach to Performing a Risk-Based Audit Trail Review

In 2014 through 2016, inspections of pharmaceutical facilities revealed a pattern of repeated failure to follow data integrity requirements established in the current good manufacturing practices’ (CGMP’s) predicate rules and regulations in 21 US 

) 211 (1) and other international regulations. Global regulators took a number of enforcement actions against manufacturers. Since that time, data integrity, particularly for electronic data, has become an area of increased regulatory focus in the pharmaceutical industry.

In March 2018, the UK’s Medicines & Healthcare Products Regulatory Agency (MHRA) issued guidance and definitions on this topic (2), and FDA published guidance the following December, in the form of Questions and Answers (3). These documents followed guidances issued by the World Health Organization (4) and the Pharmaceutical Inspection Convention/Pharmaceutical Inspection Co-operation Scheme (PIC/PICS) (5).

A working group of the International Consortium for Innovation and Quality in Pharmaceutical Development (IQ) has developed a risk-assessment tool to help professionals working in pharmaceutical research and development assess data-integrity risks associated with stand-alone (i.e., non-enterprise) computerized analytical data acquisition and processing software. It is meant to be used with systems that generate data kept in non-transient data storage. It would not, for example, be used for transient data storage (e.g., for non-computerized pH meters and the like). Its developers expect this tool to help harmonize the process for performing data-integrity risk assessments across the pharmaceutical R&D analytical community.

They also hope that the new risk-assessment tool will promote standardization and harmonization, allowing instrument and software vendors who sell into the pharmaceutical R&D analytical marketplace to better understand the sector’s expectations and to meet the needs of their customers. This article will describe the tool in greater depth.

Recent regulatory guidances, particularly those from FDA and MHRA, place considerable emphasis on the implementation of risk-based approaches to ensuring data integrity. For example, the FDA guidance explicitly notes that CGMP regulations and guidance allow for “flexible and risk-based strategies” to prevent and detect failures to ensure data integrity. In similar spirit, the MHRA guidance describes a risk-based approach to data management that includes the assessment of “data risk, criticality, and lifecycle.”

The MHRA guidance spells out how to perform a risk assessment specific to a particular data acquisition and processing system, suggesting that all processes that produce data or from which data are obtained be mapped out so that each of the formats is identified. The objective is to identify controls, the criticality of the data, and the inherent risks involved with each of them.

Assessing the risk of lack of data integrity is seen as driving compliance and, if necessary, remediation. For example, while audit trail review is often considered an essential part of ensuring data integrity, the same guidance clarifies that risk assessment should determine a documented audit trail review as part of routine data review.

Given the complexity of audit trails for any electronic system, practitioners generally prefer to implement technical controls that will reduce or eliminate the need for audit trail review. Generally, however, when technically feasible, the goal must be to prevent a data-integrity failure.

In cases where this is not technically feasible, data collection and review is required to demonstrate that data integrity has been maintained. In rare cases in which an action can neither be prevented nor detected, a procedural control may (in some limited cases) be appropriate.

Any step taken to mitigate a data integrity risk should be assessed to determine its appropriateness in the context of the criticality of the gap between desired and actual practice. The MHRA defines critical risks as those that would potentially allow data or metadata to be “deleted, amended or excluded without authorization.” FDA’s guidance echoes this emphasis, highlighting the importance of data integrity throughout the CGMP data life cycle, from creation, through modification, processing, maintenance, archival, and on through retrieval, transmission, and disposition after the period required for data retention ends.

The risk assessment tool is divided into a number of sections, each of which is briefly discussed below.

. This section contains a set of questions that deal with access controls for users, administrators, and others. It describes expectations related to the implementation of different access levels and varying permissions. In situations where it is necessary to assign system administrator access to lab personnel, appropriate justification should be provided (e.g., ensuring that technical and procedural controls are in place to prevent the modification or deletion of files and settings). Time and date stamps and time zones are also discussed, as are steps that can be taken for the prevention of unauthorized access.

Data protection, controls, and regulatory compliance

. The focus of the data protection, controls and compliance section is on restricting the ability to delete or modify data. It also defines and discusses true copies of data and encourages the definition of the complete data (raw file, process file, etc.), including storage location, system configuration, and data flow.

. The questions in this section focus on the completeness and appropriateness of audit trails, and the required review of data and metadata. FDA defines an audit trail as being a secure, computer-generated, and time-stamped electronic record that allows the course of events leading to a data-integrity failure (e.g., the creation, modification, or deletion of an electronic record) to be documented and analyzed.

FDA guidance uses a high-performance liquid chromatography (HPLC) run as an example, stating that the audit trail should include user name, date/time of the run, the integration parameters used, and any reprocessing details. It also states that the system audit trail, including system administrator actions, should be periodically reviewed.

Archival, retrieval, back-up, disaster recovery, and contingency plans

. The questions in this section focus on ensuring that electronic data are enduring, complete, and secure from modification or loss. The electronic data (including associated metadata) should be in the original format or in a format that is compatible with the original format.

. The questions in this section focus on the use (where necessary) of electronic signatures, which should ensure attributability to the specific person who signed the document electronically.

The risk-assessment tool consists of a summary, followed by a tabular presentation as depicted in 

. The entire tool is available on the IQ Consortium web site (6). The expected responses to be filled into these sections are described below. The summary section should contain an overall evaluation of the acceptability (from a data integrity standpoint) of the system under study. This evaluation should take into account any mitigations and interim actions identified in the risk assessment.

In addition, users may find it convenient to describe (in general terms) the configuration of the system, including the data storage location and any data safeguards that are associated with it (e.g., if the data are written to a drive to which the user does not possess rights to modify or delete files). Other mitigations that apply to multiple lines in the assessment may also be conveniently discussed in the summary section and the summary section referenced in the affected lines.

. The first column contains a list of data-integrity requirements, each phrased as a question for which “yes” is the desired response. Options include yes, no, and not applicable (NA). These questions are designed to prompt a thorough investigation of the system at hand. As mentioned, the questions are phrased such that the desired response is “yes.” It is important to note that the assessment should be performed against the system (including the hardware, software, and any governing procedures) as it currently exists. If the system does not meet the expectation described in the question, a response of “no” should be selected.

In rare instances, a response of N/A may be appropriate. For example, one question asks whether libraries are included in the data backup. For software that does not utilize a library search, this question would properly be answered N/A.

. The Guidance column provides additional context for, and clarification of the questions. In some cases, it also provides suggestions on potential mitigations.

. The final column can be used to describe the manner in which the system meets the requirement (in the case of a “yes” answer). When the “no” response is selected, it should be used to describe the manner in which the risk therein identified is mitigated.

There also may be the scenario where a “no” response is selected with a rationale for why the risk is considered acceptable. Finally, in the case of an N/A response, the context which renders the question not applicable should be explained.

The IQ working group is publishing this risk assessment tool to facilitate data-integrity risk assessments and promote standardization and harmonization of practice. However, its developers believe that it may be useful in other applications as well.

For instance, pharmaceutical firms may find it helpful to use the tool (or a subset of it) when evaluating instrument/software systems to aid in purchasing decisions. Similarly, instrument vendors may find it useful in understanding the needs and expectations of the pharmaceutical industry.

The tool may also be useful in managing third-party laboratories and vendors who service equipment and in drafting technical quality agreements with them. Finally, it is expected that implementation of the tool will aid pharmaceutical firms in internal harmonization and benchmarking, across sites within a company.

The authors wish to acknowledge the contributions of the IQ Working Group for Data Integrity in R&D Analytical Laboratories.

, Title 21, Food and Drugs (Government Printing Office, Washington, DC), Part 211
2. MHRA, GXP Data Integrity Guidance and Definitions (London, UK, March 2018).
3. FDA, Data Integrity and Compliance with Drug CGMP - Questions and Answers (Rockville, MD, December 2018).
4. World Health Organization, 

 (Geneva, Switzerland, October 2019).
5. Pharmaceutical Inspection Convention Pharmaceutical Inspection Co-operation Scheme, 

 (Geneva, Switzerland, November 2018).
6. IQ Consortium, 

www.iqconsortium.org/initiatives/projects/iq-data-integrity-manuscript

 both work in analytical research and development at Pfizer Inc. (Groton, CT); 

 works in analytical research and development and 

 works in quality, both at AbbVie Inc.(North Chicago, IL); K

 both work in quality at Merck & Co., Inc., (Kenilworth, NJ and West Point, PA respectively); 

 works in quality at Boehringer Ingelheim Pharmaceuticals Inc. (Ridgefield, CT).

*To whom all correspondence should be addressed


Vol. 44, No. 8
August 2020
Pages: 51­-53

When referring to this article, please cite it as J. Lippke, et al. “Assessing Data Integrity Risks in an R&D Environment,” 

A data-integrity risk assessment tool has been developed for use with standalone R&D data-acquisition and processing software. 