Joseph Mongillo, analytical research and development at Pfizer Inc., Groton, CT

Joseph Mongillo

Audit trail review (ATR) is a mechanism to detect potential critical changes to data/system security settings and to ensure the quality and integrity of reported data. The authors have defined a risk-based approach to ATR where ATR is only required for high impact GxP (good manufacturing practices [GMP] and good laboratory practices [GLP] for the purposes of this paper) analytical data and possible system security changes. This approach requires a fully documented risk assessment that encompasses the technical controls and identification of data impact. Note that while analytical data are the focus of this paper, the principles outlined may be applied to other activities.

Data integrity, particularly electronic data integrity, has become an area of increased regulatory focus. Per FDA, “For purposes of this guidance, audit trail means a secure, computer-generated, time-stamped electronic record that allows for reconstruction of the course of events relating to the creation, modification, or deletion of an electronic record” (1).

In 2018, the United Kingdom’s Medicines & Healthcare products Regulatory Agency (MHRA) and the US FDA issued guidance documents on the topic. MHRA’s 

‘GXP’ Data Integrity Guidance and Definitions

Data Integrity and Compliance with Drug CGMP—Questions and Answers

 was issued in December 2018. These documents join guidance issued by the World Health Organization (Guideline on Data Integrity) and the Pharmaceutical Inspection Convention Pharmaceutical Inspection Co-operation Scheme (PIC/S, 

Good Practices for Data Management and Integrity in Regulated GMP/GDP Environments

The publication of these guidance documents is associated with enforcement actions with an emphasis on data integrity that stem from a failure to follow current good manufacturing practices (CGMPs) predicate rules and existing regulations in 21 

Good Practices For Data Management And Integrity In Regulated GMP/GDP Environments

—July 2021 (6)—gives an indication of the key elements to consider for an effective risk-based approach: “Data criticality (impact to decision making and product quality) and data risk (opportunity for data alteration and deletion, and likelihood of detection/visibility of changes by the manufacturer’s routine review processes).”

Therefore, regulatory expectations for audit trail review have become an established part of the GxP data lifecycle.

This article introduces a harmonized approach to performing a risk-based ATR developed by a working group of the International Consortium for Innovation and Quality in Pharmaceutical Development (IQ).

It should be noted that the scope of this article includes electronic instrument analytical data where raw data are stored in non-volatile memory (i.e., can be recalled later).Both enterprise and standalone data acquisition systems are in scope.

Systems that do not generate data are out of scope.

technical control—computerized features like audit trail, backup mechanism, user management and security, electronic signatures and/or digital signatures to assist or enforce administrative and procedural controls

procedural control—standard operating procedures (SOPs) and work instructions for operation and administration, system user controls, computer system validation, calibration, network qualification, awareness training, etc.

system controls—combination of procedural and technical controls for a system.

Recent regulatory guidance such as those from FDA and MHRA emphasize the implementation of risk-based approaches to ensuring data integrity. The FDA guidance reminds us that, “CGMP regulations and guidance allow for flexible and risk-based strategies to prevent and detect data integrity issues” (1).

Similarly, the MHRA guidance describes “a risk-based approach to data management that includes data risk, criticality and lifecycle” (2).

The concept of performing a data integrity risk assessment specific to a particular data acquisition and processing system is laid out in the MHRA guidance:

“An example of a suitable approach is to perform a data integrity risk assessment (DIRA) where the processes that produce data or where data are obtained are mapped out and each of the formats and their controls are identified and the data criticality and inherent risks documented” (2).

The data integrity risk assessment is seen as a driver of compliance and prioritization of any necessary remediation activities. While audit trail review is often considered an essential part of ensuring data integrity, the same guidance clarifies that routine data review should include a documented audit trail review 

where this is determined by a risk assessment 

The appropriateness of any mitigation of a data integrity risk should be assessed in the context of the criticality of the gap. MHRA defines critical risks as those that impact the potential of data or metadata “to be deleted, amended, or excluded without authorization.”FDA states that, “Data integrity is critical throughout the CGMP data lifecycle, including in the creation, modification, processing, maintenance, archival, retrieval, transmission, and disposition of data after the record’s retention period ends” (1).

It should be noted that archival and retrieval are out of scope for this paper on ATR.

) where data types were categorized and the need for audit trail review considered. This serves as a risk assessment that can be used to determine the need for procedural controls, and the controls should be documented within the qualification package for new equipment or in change management system for equipment updates. A risk assessment, for instance the one described in 

Assessing Data Integrity Risks in an R&D Environment

 (7), may be used to define data integrity elements for a system where audit trail review is the chosen mitigation. 

 is specific to ATR and does not include data review. For GLP, data review and ATR need to happen at the same time, for GMP there may be opportunity to separate and streamline some activities with a documented risk-based approach.

Risk assessment tool for determining audit trail review requirements. Where ALCOA+ is attributable, legible, contemporaneous, original, and accurate, plus the data needs to be complete, consistent, enduring, and available. 

Determining the need for and frequency of ATR

 ATR should be considered for electronic GxP relevant data when a technical control does not remove the need to review the audit trail. A risk-based approach should be applied to ATR, and this general approach is described in 

. Tools such as the risk filtering tool in International Council for Harmonisation (ICH) Q9 (8) may be used.

Determining the rigor of audit trail review (ATR) as a function of data risk and data impact. 

When possible, there is a preference to implement technical controls to reduce/eliminate the need for ATR. It is preferred to prevent an undesirable action from occurring if this is technically feasible. In cases where prevention is not possible, detection of the undesirable action through data review (including ATR) is required. In rare (limited) cases where an action may be neither prevented nor detected, discuss with additional business/IT/quality assurance (QA) support to risk assess whether the system may be used for GxP work or if alternate controls need to be put in place such as a procedural control.

It is preferred that a technical control be employed to prevent data deletion. If deletion cannot be prevented, the ATR process should be designed to detect that specific activity. It should be noted that, in many cases, ATR is most logically performed concurrent with other data review activities. The severity of any residual risk should be assessed. The frequency for ATR should be commensurate with the probability of the risk occurrence. The frequency may be adjusted based on documented historical performance.

 The intended use of the data should also have an impact on the need for and frequency of ATR. The data’s potential risk impact on patient safety and product quality should be considered, and GxP-relevant data are determined by regulatory requirements.

High risk impact data are defined as data with potential for direct impact to product quality and patient safety. Individual companies may identify other activities as high impact but at a minimum would include release, clinical stability, and cleaning verification.

While it is acknowledged that companies may assign slightly different levels of impact to the same data types, some useful guidance may be obtained from an informal poll conducted of IQ member companies. Data arising from activities such as GLP studies, cleaning verification, clinical product release, and stability were considered greater impact and may trigger ATR. Activities such as method validation may have an indirect effect on product quality and patient safety and may be less impactful and have a medium/low impact dependent on a company’s risk considerations.

Defining appropriate ATR effort/frequency. 

Together, the assessments of the systemcharacteristics and limitations and the impact of the data’s intended use will facilitate identification of records, steps, and changes, and enable risk classification (low to high) and a tiered audit trail review effort.

A decision tree describing the performance of ATR is provided in 

. Additional explanatory comments about the decision tree are given as follows.

Where it is possible for changes to be made to the experimental conditions, metadata, or other parameters that have the potential to affect the results/data, one control strategy to ensure detection is to perform ATR. In these cases, the following should be considered as critical changes (and potentially included in a list of elements to be checked):

changes to data processing parameters (analytical method)

repeated analysis or reprocessing without justification

change history of finished product test results

The requirements for ATR (and data verification in general) should be proceduralized and should define requirements on a software-by-software basis dependent on the assessed data risk and impact. The frequency and responsibility for ATR should be defined in the procedure. Evidence of audit trail review should be documented, in most cases by defining the meaning of the overall review signature.

The purpose of a system level ATR is to ensure key configurations and settings have not been changed (either intentionally or unintentionally). It is recommended that the system audit trail (which contains, for example, system administrator actions such as deletion of data or changes to system security settings) also be reviewed at least periodically. This periodic review will ensure the system has remained configured as it was during validation/qualification. Based on the type of the system and corresponding data, the following items might be considered:

changes to data paths or folder structure

data security management (lifecycle including archival, restoration, etc.)

audit trail review may be used to verify appropriate access privileges have been used. Other processes may be employed to satisfy this requirement such as user access reviews (including admin privileges)

library files (where applicable) where the technical controls of the library would drive the need and frequency for review.

) for system-level ATR where data types were categorized with examples and the need for audit trail review considered.

Decision tree for determining minimum system level audit trail review requirements.

Manually entered audit trail comments should be suitable for an auditor/inspector to read and should include the scientific rationale for why the change was made. GAMP 5 (9) provides audit trail requirements for an audit trail entry. Note: There may be character limitations, so it may be necessary to document justification outside of the electronic

system and include a cross-reference. Manually entered comments will also require review and should be contemporaneous (within a reasonable experiment/review time frame). If time has elapsed, then the time gap should be supported with a justification.

ATR is a mechanism to detect potential critical changes to data and one means to ensure the quality and integrity of reported data. The authors have defined a pragmatic risk-based approach to ATR where ATR is only needed for high impact GxP data and that ATR can be targeted to focus specifically on critical changes that may be possible. This approach requires a fully documented data risk assessment that encompasses the technical controls, identification of relevant high-risk impact data, which may vary on a per-organization basis. The preferred approach is to utilize technical controls wherever possible.

ATR inherently remains a manual process that is resource intensive, and opportunities for automated data transfer/digitalization removes opportunity for critical changes negating the need for ATR where utilized. In addition, when a system has been configured to provide visual flags for undesirable actions/states, or when the audit trail provides filtering or searching capabilities, these abilities may be leveraged to streamline ATR.

 (Rockville, MD, December 2018).
2. MHRA,

‘GXP’ Data Integrity Guidance and Definitions 

(London, UK, March 2018).
3. World Health Organization, 

 (Geneva, Switzerland, October 2019).
4. PIC/S, 

 (Geneva, Switzerland, November 2018).
5. US 

, Title 21, Food and Drugs (Government Printing Office, Washington, DC), Part 211.
6. PIC/S Guidance PI 041-1: 

GAMP 5: A Risk-Based Approach to Compliant GxP Computerized Systems

 both work in Analytical Research and Development at Pfizer Inc. (Groton, CT); 

 both work in Analytical Research and Development at AbbVie Inc. (North Chicago, IL and Ludwigshafen, Germany, respectively); 

 works in Computer System Validation Quality, CoE at Merck & Co., Inc., (West Point, PA); 

 works in Analytical Development Sciences for Biologicals at UCB S.A. Belgium. All contributors are part of the IQ Working Group.
*To whom all correspondence should be addressed.


Vol. 46, No. 9
September 2022
Pages: 48–53

When referring to this article, please cite it as J. Lippke, et al., “A Harmonized Approach to Performing a Risk-Based Audit Trail Review,” 

Articles

The IQ Working Group has defined a pragmatic risk-based approach to audit trail review, where it is only required for high impact GxP data.

A Harmonized Approach to Performing a Risk-Based Audit Trail Review

This article summarizes a pharmaceutical industry consensus viewpoint of the current regulations regarding data integrity as applied to analytical test data generated to support regulatory activities. In particular, the focus will be on data generated by external personnel (e.g., a third-party laboratory or instrument service technician), hereafter referred to as a vendor.

There are considerations for data integrity (1–4) when data are generated by a vendor. For example, analytical instrumentation (e.g., chromatographic, spectroscopic, spectrophotometric, thermogravimetric, electrochemical, or microscopy instrumentation) utilized in the pharmaceutical industry generates data not only during analytical sample analysis but also during routine or preventive maintenance, such as instrument calibration or qualification, and during troubleshooting associated with repairs. For preventive maintenance or repairs, data may be generated by company personnel, hereafter referred to as the sponsor, or by vendors. The authors will refer to vendor-generated data, whether generated using the sponsor’s instrumentation or the vendor’s own instrumentation, as “outsourced data”.

Although data generated during routine testing may employ well-established processes that ensure data integrity, because pharmaceutical companies deal with many vendors as well as in-house support for instrument support, the maintenance/calibration/qualification process may be fully paper based, fully paperless, or a combination of both (i.e., hybrid documentation). As such, the approaches taken to ensure data integrity during these activities may vary and should be assessed on a case-by-case basis. In addition, the possible modes of remediation may evolve as instrument manufacturers and testing/maintenance vendors evolve their approaches and capabilities.

Within this article, data are defined as electronic and/or paper records generated during GxP testing (e.g., release or stability testing) as well as during analytical instrument calibration and/or qualification activities. Outsourced (or third-party) data are data generated on qualified/validated instruments by outside personnel. These include:

Data generated by vendors when visiting a sponsor site and generating electronic and/or paper records that are created and saved outside of the sponsor’s network and support regulated activities (e.g., lot release, clinical stability, etc.)

Data that support regulated activities generated by a vendor other than at the sponsor site; these data include electronic and/or paper records

Data generated by secondary vendors contracted by a primary vendor, which may include electronic and/or paper records (e.g., in the case of hybrid systems).

All expectations should be clearly described in the written agreement (e.g., maintenance contract, quality agreement) in any of these situations.

Data should be attributable, legible, contemporaneously recorded, original or a true copy, and accurate (ALCOA). Data management, retention, and archiving should be managed based on documented risk, including the format of the data. If electronic, dynamic data are available, these data or a complete, certified true copy thereof should be maintained.

Dynamic data are formatted to allow for interaction between the user and the record content. With dynamic data, a user may be able to reprocess using different parameters or modify formulas/entries that will alter a calculation result. If it is not possible to maintain the electronic, dynamic data record, a complete static representation of all data including metadata, audit trails, etc., must be maintained. Static data (data that are fixed and allow little or no interaction between the user and the record content) may allow for a more streamlined approach than that required for dynamic data. For instance, because there is no need to monitor changes in data processing or in reported results, in many cases it may be appropriate for the third-party to supply only a printed or static electronic report (e.g., a PDF file) of the reported data for archive and retention. For data that were generated in static format, a static archived representation is appropriate if it is a true copy (including all relevant metadata) of the data. In all cases, the original data must be completely reconstructable.

The various scenarios under which outsourced data may be generated and managed are summarized in Table I and are further discussed in the following scenarios. In each case, key requirements and points to be considered are given. For all three scenarios, the vendor should be approved by the sponsor and there should be an agreement in place for services provided. The necessary details of the agreement vary depending on the service provided and the scenario.

Scenario A: Vendor-generated data on sponsor instruments using the information technology (IT) infrastructure of the sponsor.

In cases where data are acquired and stored by the vendor through the sponsor’s standard workflows and on the sponsor’s technology infrastructure, the sponsor’s standard data integrity policies and procedures should apply. The accounts and roles utilized by the vendor should be unique and configured to ensure attributability and be specific to the work being performed. The sponsor should ensure that the vendor possesses the appropriate training required for access per sponsor’s applicable standards and retain documentation of this assurance.

Scenario B: Vendor-generated data on sponsor instruments using the IT infrastructure of the sponsor and non-standard processes. 

This scenario pertains to situations where the instrument hardware, firmware, or software is being accessed or utilized in a non-routine manner (different than it would typically be used to collect or process data). Examples could include using diagnostic mode or calibration mode, accessing the software outside of the network, saving in a different file location, or using a hybrid paper/electronic process. A risk assessment and mitigation strategy should be considered to ensure compliance with ALCOA principles.

It is recognized that some of the previously mentioned examplesmay require non-standard access. The accounts and roles utilized by the vendor should be unique and configured to ensure attributability and be specific to the work being performed. The sponsor should ensure (through a written agreement, direct training, or other written process) that the vendor has appropriate technical and good documentation training per sponsor’s applicable standards and retain this documentation. Data review should follow the sponsor’s standard practices and may include a defined, documented risk-based approach for reviews of vendor-generated data. System level reviews should include reviews of vendor account access. Vendor-generated data on sponsor instruments should be in alignment with the sponsor’s standards for segregation of duties. Controls should be in place to ensure data integrity is appropriately managed. In particular:

Specific and unique user roles, permissions, and passwords for access are required.

Where appropriate, access should be provided on a temporary basis and subsequently rescinded after a defined period.

Controls for remote access, especially by vendors, should be evaluated to determine the effectiveness of physical controls and to ensure appropriate data and system protection.

Data security (including the ability to delete/modify data) should be subject to appropriate controls (technical, administrative, or procedural); consider accidental loss or corruption of sponsor data.

Changes in the data retention path (relative to that in the qualified configuration) should be subject to change management.

Reviews of vendor data (including the audit trail, if applicable) should be defined based on documented risk.

Vendors should be granted access at the lowest level that allows the performance of necessary duties.

In cases where administrative access is required for the vendor to perform their duties, their ability to make critical changes (such as deleting, editing, or modifying data without detection) should be controlled through technical, administrative, or procedural means.

Hybrid paper/electronic processes may be captured in the vendor agreement, but in any case, should be documented and assessed for data integrity risks.

Deletions or modifications of data or metadata (e.g., dates) should not obscure previously recorded information and should be attributable.

Scenario C: Vendor-generated data stored on the IT infrastructure of the vendor.

One of the critical elements of data integrity for data generated by a third-party is where the raw and reported data are stored and to whom the data are accessible. Regardless, data should be retained and archived in accordance with a quality agreement (or equivalent) and in compliance with regulatory requirements. These written agreements should establish sponsor expectations and vendor responsibilities related to data integrity controls for good manufacturing practice (GMP) or good laboratory practice (GLP) records, as well as how communication and auditing of such records should take place. The agreement should ensure the following, for the vendor:

Vendor procedures are in place to ensure that for all data, whether paper or electronic, at a minimum the ALCOA requirements are met.

Vendor testing records are reviewed by the vendor to ensure compliance with all procedures, specifications, and regulatory requirements.

Vendor investigations and vendor internal auditing include data integrity (e.g., calculations, quality of procedures/processes to uncover data integrity issues, documented training with a focus on data integrity and responsibilities for the vendor).

The sponsor should agree to regularly update, review, and communicate the following to the vendor:

Any updates to procedures reflecting data integrity practices providing a quality environment between sponsor and vendor

Audit/investigation results and reviews of vendor data integrity quality metrics.

The sponsor should clearly communicate that sponsor audits of the vendor will include a focus on data integrity elements and practices, and the sponsor should ensure that audits include assessment and evaluation of data integrity controls in place.

The written agreement should additionally delineate the record retention responsibilities of the two parties and any handoffs between vendor and sponsor at specific milestones. In cases where data are collected using software that the sponsor does not have, the vendor should retain e-records and the software necessary to make them human-readable (including metadata).

A primary concern is that of security of the raw data. Data stored on the IT infrastructure of a third party are inherently less under the control and protection of the sponsor. In cases where the third party retains the original data, it is critical that appropriate expectations and responsibilities are clearly defined in a quality agreement (or equivalent). These agreements should include the following considerations:

Because the sponsor may, in some cases, receive only reports of final data (e.g., from sample analysis or from calibration, maintenance, and qualification), an audit for selecting an external vendor should challenge the process from raw data generation through to distribution of final reports to ensure the accuracy and reliability of raw data generated by the vendor. This audit should include a review of the mechanisms used to generate and distribute data summaries and reports.

Transparency around the vendor’s IT infrastructure (including the subcontracting of IT infrastructure to cloud-based providers) should be defined in a written agreement. This agreement should include notification of the unauthorized accessing of sponsor data (e.g., through a hacking attack).

Responsibilities, including clearly defined ownership and retention requirements/schedules, should be clearly described in the written agreement.

Where data are transferred between the sponsor and their vendor site, the written agreement should specify how this is done to ensure adherence to ALCOA + principles, including management of true copies.

All electronic data (or certified true copy thereof) should be retained and include a means to retrieve/read these data (including metadata such as audit trails, etc.). In particular, the vendor must retain the ability to read data from retired instruments through the retention period of the data (as defined by the sponsor).

Electronic data should be maintained in its original format unless otherwise defined/agreed upon with the sponsor.

Timing expectations for data retrievability should be clearly described in the written agreement.

The written agreement should define who is responsible for backup and archiving features. Any incidents with or changes to data archival, backup, or restoration should be clearly documented in accordance with appropriate procedures and reported to the sponsor within an agreed time period.

The written agreement should include expectations on confidentiality of all disclosed information. The written agreement should define who in the vendor organization has access to sponsor data.

The written agreement should define the requirements for data review. This should include review against ALCOA + principles (e.g., metadata such as audit trails, where applicable) and system-level reviews.

Any vendors using cloud service providers should be assessed to ensure infrastructure controls are in place, including infrastructure and services for change control, system backup/restore, and data archiving processes.

Additional considerations for data generated during instrument calibration, qualification, repair, and troubleshooting (can be Scenario A, B or C). 

Data integrity controls for analytical instruments and equipment may be challenged during internal or regulatory authority audits and inspections. It is a regulatory expectation (1–4) that the integrity of supporting instrument data is robust and that risks to these data have been adequately mitigated. Some examples where data integrity controls (including appropriate change control) are needed include calibration and qualification of raw data, control of standards (e.g., reference standards, external calibrated test probes, etc.) used during calibration, and management of internal and vendor documentation.

There are several aspects of the data lifecycle needed to underwrite the accuracy of analytical data. Requirements include demonstration that an instrument can produce accurate results and is under adequate system controls. Adequate system controls relate to documented procedures that support initial qualification, periodic calibration and maintenance, instrument repair and troubleshooting (including change control), and periodic review of the calibration/qualification status of the instrument.

Analytical instrument qualification and/or initial calibration is critical to ensure data generated on an instrument is accurate (ALCOA). Data generated using systems other than the one under qualification or calibration (e.g., a thermocouple used to calibrate a chromatography column heater) should be subject to the same controls as other data. In a scenario where initial validation/qualification is performed prior to mitigation of data integrity concerns, consideration should be given to potential data integrity concerns of the qualification testing. In general, data generated during calibration, qualification/validation, or maintenance activities should comply with all the foregoing requirements. However, there may be scenarios where data are generated outside of the normal workflow. In these cases, deviations from the normal workflow should be evaluated for risk and documented appropriately, particularly for risks that may impact the attributability or accuracy of the resultant data.

While all three scenarios, as well as many permutations of them, are possible and may be managed, Scenarios A (vendor-generated data on sponsor instruments using IT infrastructure of the sponsor) and C (vendor-generated data stored on the IT infrastructure of the vendor) are strongly preferred from a compliance and risk perspective. If Scenario B (vendor-generated data on sponsor instruments using the IT infrastructure of the sponsor and non-standard processes) is employed, a risk assessment and mitigation strategy should be considered to ensure compliance with ALCOA principles. In any case, the considerations described in this article should be evaluated to achieve compliance.

This manuscript was developed with the support of the International Consortium for Innovation and Quality in Pharmaceutical Development (IQ, www.iqconsortium.org). IQ is a not-for-profit organization of pharmaceutical and biotechnology companies with a mission of advancing science and technology to augment the capability of member companies to develop transformational solutions that benefit patients, regulators, and the broader research and development community.

GXP’ Data Integrity Guidance and Definitions

Guidance for Industry: Data Integrity and Compliance With Drug CGMP Questions and Answers 

(Rockville, MD, December 2018).
3. World Health Organization, 

 (Geneva, Switzerland, October 2019).
4. Pharmaceutical Inspection Convention Pharmaceutical Inspection Co-operation Scheme, 

Thomas Cullen*, thomas.f.cullen@abbvie.com, and Cliff Mitchell work in Analytical Research and Development at AbbVie Inc. (North Chicago, IL); Julie Lippke and Joseph Mongillo both work in Analytical Research and Development at Pfizer Inc. (Groton, CT); Koottala S. Ramaswamy works in Global Development Quality at Merck & Co., Inc. (West Point, PA); and Thomas Purdue works in Quality at Boehringer Ingelheim Pharmaceuticals Inc. (Ridgefield, CT); all authors are members of the IQ Consortium.

*To whom all correspondence should be addressed

When referring to this article, please cite it as T. Cullen, C. Mitchell, J. Lippke, J. Mongillo, K.S. Ramaswamy, and T. Purdue, “Data Integrity Considerations for Vendor-Generated Data Associated with Analytical Testing," 

Pharmaceutical Technology Outsourcing Resources

Sponsors should consider best practices for maintaining data generated during sample analysis and instrument maintenance.

Data Integrity Considerations for Vendor-Generated Data Associated with Analytical Testing

In 2014 through 2016, inspections of pharmaceutical facilities revealed a pattern of repeated failure to follow data integrity requirements established in the current good manufacturing practices’ (CGMP’s) predicate rules and regulations in 21 US 

) 211 (1) and other international regulations. Global regulators took a number of enforcement actions against manufacturers. Since that time, data integrity, particularly for electronic data, has become an area of increased regulatory focus in the pharmaceutical industry.

In March 2018, the UK’s Medicines & Healthcare Products Regulatory Agency (MHRA) issued guidance and definitions on this topic (2), and FDA published guidance the following December, in the form of Questions and Answers (3). These documents followed guidances issued by the World Health Organization (4) and the Pharmaceutical Inspection Convention/Pharmaceutical Inspection Co-operation Scheme (PIC/PICS) (5).

A working group of the International Consortium for Innovation and Quality in Pharmaceutical Development (IQ) has developed a risk-assessment tool to help professionals working in pharmaceutical research and development assess data-integrity risks associated with stand-alone (i.e., non-enterprise) computerized analytical data acquisition and processing software. It is meant to be used with systems that generate data kept in non-transient data storage. It would not, for example, be used for transient data storage (e.g., for non-computerized pH meters and the like). Its developers expect this tool to help harmonize the process for performing data-integrity risk assessments across the pharmaceutical R&D analytical community.

They also hope that the new risk-assessment tool will promote standardization and harmonization, allowing instrument and software vendors who sell into the pharmaceutical R&D analytical marketplace to better understand the sector’s expectations and to meet the needs of their customers. This article will describe the tool in greater depth.

Recent regulatory guidances, particularly those from FDA and MHRA, place considerable emphasis on the implementation of risk-based approaches to ensuring data integrity. For example, the FDA guidance explicitly notes that CGMP regulations and guidance allow for “flexible and risk-based strategies” to prevent and detect failures to ensure data integrity. In similar spirit, the MHRA guidance describes a risk-based approach to data management that includes the assessment of “data risk, criticality, and lifecycle.”

The MHRA guidance spells out how to perform a risk assessment specific to a particular data acquisition and processing system, suggesting that all processes that produce data or from which data are obtained be mapped out so that each of the formats is identified. The objective is to identify controls, the criticality of the data, and the inherent risks involved with each of them.

Assessing the risk of lack of data integrity is seen as driving compliance and, if necessary, remediation. For example, while audit trail review is often considered an essential part of ensuring data integrity, the same guidance clarifies that risk assessment should determine a documented audit trail review as part of routine data review.

Given the complexity of audit trails for any electronic system, practitioners generally prefer to implement technical controls that will reduce or eliminate the need for audit trail review. Generally, however, when technically feasible, the goal must be to prevent a data-integrity failure.

In cases where this is not technically feasible, data collection and review is required to demonstrate that data integrity has been maintained. In rare cases in which an action can neither be prevented nor detected, a procedural control may (in some limited cases) be appropriate.

Any step taken to mitigate a data integrity risk should be assessed to determine its appropriateness in the context of the criticality of the gap between desired and actual practice. The MHRA defines critical risks as those that would potentially allow data or metadata to be “deleted, amended or excluded without authorization.” FDA’s guidance echoes this emphasis, highlighting the importance of data integrity throughout the CGMP data life cycle, from creation, through modification, processing, maintenance, archival, and on through retrieval, transmission, and disposition after the period required for data retention ends.

The risk assessment tool is divided into a number of sections, each of which is briefly discussed below.

. This section contains a set of questions that deal with access controls for users, administrators, and others. It describes expectations related to the implementation of different access levels and varying permissions. In situations where it is necessary to assign system administrator access to lab personnel, appropriate justification should be provided (e.g., ensuring that technical and procedural controls are in place to prevent the modification or deletion of files and settings). Time and date stamps and time zones are also discussed, as are steps that can be taken for the prevention of unauthorized access.

Data protection, controls, and regulatory compliance

. The focus of the data protection, controls and compliance section is on restricting the ability to delete or modify data. It also defines and discusses true copies of data and encourages the definition of the complete data (raw file, process file, etc.), including storage location, system configuration, and data flow.

. The questions in this section focus on the completeness and appropriateness of audit trails, and the required review of data and metadata. FDA defines an audit trail as being a secure, computer-generated, and time-stamped electronic record that allows the course of events leading to a data-integrity failure (e.g., the creation, modification, or deletion of an electronic record) to be documented and analyzed.

FDA guidance uses a high-performance liquid chromatography (HPLC) run as an example, stating that the audit trail should include user name, date/time of the run, the integration parameters used, and any reprocessing details. It also states that the system audit trail, including system administrator actions, should be periodically reviewed.

Archival, retrieval, back-up, disaster recovery, and contingency plans

. The questions in this section focus on ensuring that electronic data are enduring, complete, and secure from modification or loss. The electronic data (including associated metadata) should be in the original format or in a format that is compatible with the original format.

. The questions in this section focus on the use (where necessary) of electronic signatures, which should ensure attributability to the specific person who signed the document electronically.

The risk-assessment tool consists of a summary, followed by a tabular presentation as depicted in 

. The entire tool is available on the IQ Consortium web site (6). The expected responses to be filled into these sections are described below. The summary section should contain an overall evaluation of the acceptability (from a data integrity standpoint) of the system under study. This evaluation should take into account any mitigations and interim actions identified in the risk assessment.

In addition, users may find it convenient to describe (in general terms) the configuration of the system, including the data storage location and any data safeguards that are associated with it (e.g., if the data are written to a drive to which the user does not possess rights to modify or delete files). Other mitigations that apply to multiple lines in the assessment may also be conveniently discussed in the summary section and the summary section referenced in the affected lines.

. The first column contains a list of data-integrity requirements, each phrased as a question for which “yes” is the desired response. Options include yes, no, and not applicable (NA). These questions are designed to prompt a thorough investigation of the system at hand. As mentioned, the questions are phrased such that the desired response is “yes.” It is important to note that the assessment should be performed against the system (including the hardware, software, and any governing procedures) as it currently exists. If the system does not meet the expectation described in the question, a response of “no” should be selected.

In rare instances, a response of N/A may be appropriate. For example, one question asks whether libraries are included in the data backup. For software that does not utilize a library search, this question would properly be answered N/A.

. The Guidance column provides additional context for, and clarification of the questions. In some cases, it also provides suggestions on potential mitigations.

. The final column can be used to describe the manner in which the system meets the requirement (in the case of a “yes” answer). When the “no” response is selected, it should be used to describe the manner in which the risk therein identified is mitigated.

There also may be the scenario where a “no” response is selected with a rationale for why the risk is considered acceptable. Finally, in the case of an N/A response, the context which renders the question not applicable should be explained.

The IQ working group is publishing this risk assessment tool to facilitate data-integrity risk assessments and promote standardization and harmonization of practice. However, its developers believe that it may be useful in other applications as well.

For instance, pharmaceutical firms may find it helpful to use the tool (or a subset of it) when evaluating instrument/software systems to aid in purchasing decisions. Similarly, instrument vendors may find it useful in understanding the needs and expectations of the pharmaceutical industry.

The tool may also be useful in managing third-party laboratories and vendors who service equipment and in drafting technical quality agreements with them. Finally, it is expected that implementation of the tool will aid pharmaceutical firms in internal harmonization and benchmarking, across sites within a company.

The authors wish to acknowledge the contributions of the IQ Working Group for Data Integrity in R&D Analytical Laboratories.

, Title 21, Food and Drugs (Government Printing Office, Washington, DC), Part 211
2. MHRA, GXP Data Integrity Guidance and Definitions (London, UK, March 2018).
3. FDA, Data Integrity and Compliance with Drug CGMP - Questions and Answers (Rockville, MD, December 2018).
4. World Health Organization, 

 (Geneva, Switzerland, October 2019).
5. Pharmaceutical Inspection Convention Pharmaceutical Inspection Co-operation Scheme, 

 (Geneva, Switzerland, November 2018).
6. IQ Consortium, 

www.iqconsortium.org/initiatives/projects/iq-data-integrity-manuscript

 both work in analytical research and development at Pfizer Inc. (Groton, CT); 

 works in analytical research and development and 

 works in quality, both at AbbVie Inc.(North Chicago, IL); K

 both work in quality at Merck & Co., Inc., (Kenilworth, NJ and West Point, PA respectively); 

 works in quality at Boehringer Ingelheim Pharmaceuticals Inc. (Ridgefield, CT).


Vol. 44, No. 8
August 2020
Pages: 51­-53

When referring to this article, please cite it as J. Lippke, et al. “Assessing Data Integrity Risks in an R&D Environment,” 

A data-integrity risk assessment tool has been developed for use with standalone R&D data-acquisition and processing software. 