Julie Lippke, analytical research and development at Pfizer Inc., Groton, CT

Julie Lippke

This article summarizes a pharmaceutical industry consensus viewpoint of the current regulations regarding data integrity as applied to analytical test data generated to support regulatory activities. In particular, the focus will be on data generated by external personnel (e.g., a third-party laboratory or instrument service technician), hereafter referred to as a vendor.

There are considerations for data integrity (1–4) when data are generated by a vendor. For example, analytical instrumentation (e.g., chromatographic, spectroscopic, spectrophotometric, thermogravimetric, electrochemical, or microscopy instrumentation) utilized in the pharmaceutical industry generates data not only during analytical sample analysis but also during routine or preventive maintenance, such as instrument calibration or qualification, and during troubleshooting associated with repairs. For preventive maintenance or repairs, data may be generated by company personnel, hereafter referred to as the sponsor, or by vendors. The authors will refer to vendor-generated data, whether generated using the sponsor’s instrumentation or the vendor’s own instrumentation, as “outsourced data”.

Although data generated during routine testing may employ well-established processes that ensure data integrity, because pharmaceutical companies deal with many vendors as well as in-house support for instrument support, the maintenance/calibration/qualification process may be fully paper based, fully paperless, or a combination of both (i.e., hybrid documentation). As such, the approaches taken to ensure data integrity during these activities may vary and should be assessed on a case-by-case basis. In addition, the possible modes of remediation may evolve as instrument manufacturers and testing/maintenance vendors evolve their approaches and capabilities.

Within this article, data are defined as electronic and/or paper records generated during GxP testing (e.g., release or stability testing) as well as during analytical instrument calibration and/or qualification activities. Outsourced (or third-party) data are data generated on qualified/validated instruments by outside personnel. These include:

Data generated by vendors when visiting a sponsor site and generating electronic and/or paper records that are created and saved outside of the sponsor’s network and support regulated activities (e.g., lot release, clinical stability, etc.)

Data that support regulated activities generated by a vendor other than at the sponsor site; these data include electronic and/or paper records

Data generated by secondary vendors contracted by a primary vendor, which may include electronic and/or paper records (e.g., in the case of hybrid systems).

All expectations should be clearly described in the written agreement (e.g., maintenance contract, quality agreement) in any of these situations.

Data should be attributable, legible, contemporaneously recorded, original or a true copy, and accurate (ALCOA). Data management, retention, and archiving should be managed based on documented risk, including the format of the data. If electronic, dynamic data are available, these data or a complete, certified true copy thereof should be maintained.

Dynamic data are formatted to allow for interaction between the user and the record content. With dynamic data, a user may be able to reprocess using different parameters or modify formulas/entries that will alter a calculation result. If it is not possible to maintain the electronic, dynamic data record, a complete static representation of all data including metadata, audit trails, etc., must be maintained. Static data (data that are fixed and allow little or no interaction between the user and the record content) may allow for a more streamlined approach than that required for dynamic data. For instance, because there is no need to monitor changes in data processing or in reported results, in many cases it may be appropriate for the third-party to supply only a printed or static electronic report (e.g., a PDF file) of the reported data for archive and retention. For data that were generated in static format, a static archived representation is appropriate if it is a true copy (including all relevant metadata) of the data. In all cases, the original data must be completely reconstructable.

The various scenarios under which outsourced data may be generated and managed are summarized in Table I and are further discussed in the following scenarios. In each case, key requirements and points to be considered are given. For all three scenarios, the vendor should be approved by the sponsor and there should be an agreement in place for services provided. The necessary details of the agreement vary depending on the service provided and the scenario.

Scenario A: Vendor-generated data on sponsor instruments using the information technology (IT) infrastructure of the sponsor.

In cases where data are acquired and stored by the vendor through the sponsor’s standard workflows and on the sponsor’s technology infrastructure, the sponsor’s standard data integrity policies and procedures should apply. The accounts and roles utilized by the vendor should be unique and configured to ensure attributability and be specific to the work being performed. The sponsor should ensure that the vendor possesses the appropriate training required for access per sponsor’s applicable standards and retain documentation of this assurance.

Scenario B: Vendor-generated data on sponsor instruments using the IT infrastructure of the sponsor and non-standard processes. 

This scenario pertains to situations where the instrument hardware, firmware, or software is being accessed or utilized in a non-routine manner (different than it would typically be used to collect or process data). Examples could include using diagnostic mode or calibration mode, accessing the software outside of the network, saving in a different file location, or using a hybrid paper/electronic process. A risk assessment and mitigation strategy should be considered to ensure compliance with ALCOA principles.

It is recognized that some of the previously mentioned examplesmay require non-standard access. The accounts and roles utilized by the vendor should be unique and configured to ensure attributability and be specific to the work being performed. The sponsor should ensure (through a written agreement, direct training, or other written process) that the vendor has appropriate technical and good documentation training per sponsor’s applicable standards and retain this documentation. Data review should follow the sponsor’s standard practices and may include a defined, documented risk-based approach for reviews of vendor-generated data. System level reviews should include reviews of vendor account access. Vendor-generated data on sponsor instruments should be in alignment with the sponsor’s standards for segregation of duties. Controls should be in place to ensure data integrity is appropriately managed. In particular:

Specific and unique user roles, permissions, and passwords for access are required.

Where appropriate, access should be provided on a temporary basis and subsequently rescinded after a defined period.

Controls for remote access, especially by vendors, should be evaluated to determine the effectiveness of physical controls and to ensure appropriate data and system protection.

Data security (including the ability to delete/modify data) should be subject to appropriate controls (technical, administrative, or procedural); consider accidental loss or corruption of sponsor data.

Changes in the data retention path (relative to that in the qualified configuration) should be subject to change management.

Reviews of vendor data (including the audit trail, if applicable) should be defined based on documented risk.

Vendors should be granted access at the lowest level that allows the performance of necessary duties.

In cases where administrative access is required for the vendor to perform their duties, their ability to make critical changes (such as deleting, editing, or modifying data without detection) should be controlled through technical, administrative, or procedural means.

Hybrid paper/electronic processes may be captured in the vendor agreement, but in any case, should be documented and assessed for data integrity risks.

Deletions or modifications of data or metadata (e.g., dates) should not obscure previously recorded information and should be attributable.

Scenario C: Vendor-generated data stored on the IT infrastructure of the vendor.

One of the critical elements of data integrity for data generated by a third-party is where the raw and reported data are stored and to whom the data are accessible. Regardless, data should be retained and archived in accordance with a quality agreement (or equivalent) and in compliance with regulatory requirements. These written agreements should establish sponsor expectations and vendor responsibilities related to data integrity controls for good manufacturing practice (GMP) or good laboratory practice (GLP) records, as well as how communication and auditing of such records should take place. The agreement should ensure the following, for the vendor:

Vendor procedures are in place to ensure that for all data, whether paper or electronic, at a minimum the ALCOA requirements are met.

Vendor testing records are reviewed by the vendor to ensure compliance with all procedures, specifications, and regulatory requirements.

Vendor investigations and vendor internal auditing include data integrity (e.g., calculations, quality of procedures/processes to uncover data integrity issues, documented training with a focus on data integrity and responsibilities for the vendor).

The sponsor should agree to regularly update, review, and communicate the following to the vendor:

Any updates to procedures reflecting data integrity practices providing a quality environment between sponsor and vendor

Audit/investigation results and reviews of vendor data integrity quality metrics.

The sponsor should clearly communicate that sponsor audits of the vendor will include a focus on data integrity elements and practices, and the sponsor should ensure that audits include assessment and evaluation of data integrity controls in place.

The written agreement should additionally delineate the record retention responsibilities of the two parties and any handoffs between vendor and sponsor at specific milestones. In cases where data are collected using software that the sponsor does not have, the vendor should retain e-records and the software necessary to make them human-readable (including metadata).

A primary concern is that of security of the raw data. Data stored on the IT infrastructure of a third party are inherently less under the control and protection of the sponsor. In cases where the third party retains the original data, it is critical that appropriate expectations and responsibilities are clearly defined in a quality agreement (or equivalent). These agreements should include the following considerations:

Because the sponsor may, in some cases, receive only reports of final data (e.g., from sample analysis or from calibration, maintenance, and qualification), an audit for selecting an external vendor should challenge the process from raw data generation through to distribution of final reports to ensure the accuracy and reliability of raw data generated by the vendor. This audit should include a review of the mechanisms used to generate and distribute data summaries and reports.

Transparency around the vendor’s IT infrastructure (including the subcontracting of IT infrastructure to cloud-based providers) should be defined in a written agreement. This agreement should include notification of the unauthorized accessing of sponsor data (e.g., through a hacking attack).

Responsibilities, including clearly defined ownership and retention requirements/schedules, should be clearly described in the written agreement.

Where data are transferred between the sponsor and their vendor site, the written agreement should specify how this is done to ensure adherence to ALCOA + principles, including management of true copies.

All electronic data (or certified true copy thereof) should be retained and include a means to retrieve/read these data (including metadata such as audit trails, etc.). In particular, the vendor must retain the ability to read data from retired instruments through the retention period of the data (as defined by the sponsor).

Electronic data should be maintained in its original format unless otherwise defined/agreed upon with the sponsor.

Timing expectations for data retrievability should be clearly described in the written agreement.

The written agreement should define who is responsible for backup and archiving features. Any incidents with or changes to data archival, backup, or restoration should be clearly documented in accordance with appropriate procedures and reported to the sponsor within an agreed time period.

The written agreement should include expectations on confidentiality of all disclosed information. The written agreement should define who in the vendor organization has access to sponsor data.

The written agreement should define the requirements for data review. This should include review against ALCOA + principles (e.g., metadata such as audit trails, where applicable) and system-level reviews.

Any vendors using cloud service providers should be assessed to ensure infrastructure controls are in place, including infrastructure and services for change control, system backup/restore, and data archiving processes.

Additional considerations for data generated during instrument calibration, qualification, repair, and troubleshooting (can be Scenario A, B or C). 

Data integrity controls for analytical instruments and equipment may be challenged during internal or regulatory authority audits and inspections. It is a regulatory expectation (1–4) that the integrity of supporting instrument data is robust and that risks to these data have been adequately mitigated. Some examples where data integrity controls (including appropriate change control) are needed include calibration and qualification of raw data, control of standards (e.g., reference standards, external calibrated test probes, etc.) used during calibration, and management of internal and vendor documentation.

There are several aspects of the data lifecycle needed to underwrite the accuracy of analytical data. Requirements include demonstration that an instrument can produce accurate results and is under adequate system controls. Adequate system controls relate to documented procedures that support initial qualification, periodic calibration and maintenance, instrument repair and troubleshooting (including change control), and periodic review of the calibration/qualification status of the instrument.

Analytical instrument qualification and/or initial calibration is critical to ensure data generated on an instrument is accurate (ALCOA). Data generated using systems other than the one under qualification or calibration (e.g., a thermocouple used to calibrate a chromatography column heater) should be subject to the same controls as other data. In a scenario where initial validation/qualification is performed prior to mitigation of data integrity concerns, consideration should be given to potential data integrity concerns of the qualification testing. In general, data generated during calibration, qualification/validation, or maintenance activities should comply with all the foregoing requirements. However, there may be scenarios where data are generated outside of the normal workflow. In these cases, deviations from the normal workflow should be evaluated for risk and documented appropriately, particularly for risks that may impact the attributability or accuracy of the resultant data.

While all three scenarios, as well as many permutations of them, are possible and may be managed, Scenarios A (vendor-generated data on sponsor instruments using IT infrastructure of the sponsor) and C (vendor-generated data stored on the IT infrastructure of the vendor) are strongly preferred from a compliance and risk perspective. If Scenario B (vendor-generated data on sponsor instruments using the IT infrastructure of the sponsor and non-standard processes) is employed, a risk assessment and mitigation strategy should be considered to ensure compliance with ALCOA principles. In any case, the considerations described in this article should be evaluated to achieve compliance.

This manuscript was developed with the support of the International Consortium for Innovation and Quality in Pharmaceutical Development (IQ, www.iqconsortium.org). IQ is a not-for-profit organization of pharmaceutical and biotechnology companies with a mission of advancing science and technology to augment the capability of member companies to develop transformational solutions that benefit patients, regulators, and the broader research and development community.

GXP’ Data Integrity Guidance and Definitions

Guidance for Industry: Data Integrity and Compliance With Drug CGMP Questions and Answers 

(Rockville, MD, December 2018).
3. World Health Organization, 

 (Geneva, Switzerland, October 2019).
4. Pharmaceutical Inspection Convention Pharmaceutical Inspection Co-operation Scheme, 

Good Practices for Data Management and Integrity in Regulated GMP/GDP Environments

Thomas Cullen*, thomas.f.cullen@abbvie.com, and Cliff Mitchell work in Analytical Research and Development at AbbVie Inc. (North Chicago, IL); Julie Lippke and Joseph Mongillo both work in Analytical Research and Development at Pfizer Inc. (Groton, CT); Koottala S. Ramaswamy works in Global Development Quality at Merck & Co., Inc. (West Point, PA); and Thomas Purdue works in Quality at Boehringer Ingelheim Pharmaceuticals Inc. (Ridgefield, CT); all authors are members of the IQ Consortium.

*To whom all correspondence should be addressed

When referring to this article, please cite it as T. Cullen, C. Mitchell, J. Lippke, J. Mongillo, K.S. Ramaswamy, and T. Purdue, “Data Integrity Considerations for Vendor-Generated Data Associated with Analytical Testing," 

Pharmaceutical Technology Outsourcing Resources

Articles

Sponsors should consider best practices for maintaining data generated during sample analysis and instrument maintenance.

Data Integrity Considerations for Vendor-Generated Data Associated with Analytical Testing

In 2014 through 2016, inspections of pharmaceutical facilities revealed a pattern of repeated failure to follow data integrity requirements established in the current good manufacturing practices’ (CGMP’s) predicate rules and regulations in 21 US 

) 211 (1) and other international regulations. Global regulators took a number of enforcement actions against manufacturers. Since that time, data integrity, particularly for electronic data, has become an area of increased regulatory focus in the pharmaceutical industry.

In March 2018, the UK’s Medicines & Healthcare Products Regulatory Agency (MHRA) issued guidance and definitions on this topic (2), and FDA published guidance the following December, in the form of Questions and Answers (3). These documents followed guidances issued by the World Health Organization (4) and the Pharmaceutical Inspection Convention/Pharmaceutical Inspection Co-operation Scheme (PIC/PICS) (5).

A working group of the International Consortium for Innovation and Quality in Pharmaceutical Development (IQ) has developed a risk-assessment tool to help professionals working in pharmaceutical research and development assess data-integrity risks associated with stand-alone (i.e., non-enterprise) computerized analytical data acquisition and processing software. It is meant to be used with systems that generate data kept in non-transient data storage. It would not, for example, be used for transient data storage (e.g., for non-computerized pH meters and the like). Its developers expect this tool to help harmonize the process for performing data-integrity risk assessments across the pharmaceutical R&D analytical community.

They also hope that the new risk-assessment tool will promote standardization and harmonization, allowing instrument and software vendors who sell into the pharmaceutical R&D analytical marketplace to better understand the sector’s expectations and to meet the needs of their customers. This article will describe the tool in greater depth.

Recent regulatory guidances, particularly those from FDA and MHRA, place considerable emphasis on the implementation of risk-based approaches to ensuring data integrity. For example, the FDA guidance explicitly notes that CGMP regulations and guidance allow for “flexible and risk-based strategies” to prevent and detect failures to ensure data integrity. In similar spirit, the MHRA guidance describes a risk-based approach to data management that includes the assessment of “data risk, criticality, and lifecycle.”

The MHRA guidance spells out how to perform a risk assessment specific to a particular data acquisition and processing system, suggesting that all processes that produce data or from which data are obtained be mapped out so that each of the formats is identified. The objective is to identify controls, the criticality of the data, and the inherent risks involved with each of them.

Assessing the risk of lack of data integrity is seen as driving compliance and, if necessary, remediation. For example, while audit trail review is often considered an essential part of ensuring data integrity, the same guidance clarifies that risk assessment should determine a documented audit trail review as part of routine data review.

Given the complexity of audit trails for any electronic system, practitioners generally prefer to implement technical controls that will reduce or eliminate the need for audit trail review. Generally, however, when technically feasible, the goal must be to prevent a data-integrity failure.

In cases where this is not technically feasible, data collection and review is required to demonstrate that data integrity has been maintained. In rare cases in which an action can neither be prevented nor detected, a procedural control may (in some limited cases) be appropriate.

Any step taken to mitigate a data integrity risk should be assessed to determine its appropriateness in the context of the criticality of the gap between desired and actual practice. The MHRA defines critical risks as those that would potentially allow data or metadata to be “deleted, amended or excluded without authorization.” FDA’s guidance echoes this emphasis, highlighting the importance of data integrity throughout the CGMP data life cycle, from creation, through modification, processing, maintenance, archival, and on through retrieval, transmission, and disposition after the period required for data retention ends.

The risk assessment tool is divided into a number of sections, each of which is briefly discussed below.

. This section contains a set of questions that deal with access controls for users, administrators, and others. It describes expectations related to the implementation of different access levels and varying permissions. In situations where it is necessary to assign system administrator access to lab personnel, appropriate justification should be provided (e.g., ensuring that technical and procedural controls are in place to prevent the modification or deletion of files and settings). Time and date stamps and time zones are also discussed, as are steps that can be taken for the prevention of unauthorized access.

Data protection, controls, and regulatory compliance

. The focus of the data protection, controls and compliance section is on restricting the ability to delete or modify data. It also defines and discusses true copies of data and encourages the definition of the complete data (raw file, process file, etc.), including storage location, system configuration, and data flow.

. The questions in this section focus on the completeness and appropriateness of audit trails, and the required review of data and metadata. FDA defines an audit trail as being a secure, computer-generated, and time-stamped electronic record that allows the course of events leading to a data-integrity failure (e.g., the creation, modification, or deletion of an electronic record) to be documented and analyzed.

FDA guidance uses a high-performance liquid chromatography (HPLC) run as an example, stating that the audit trail should include user name, date/time of the run, the integration parameters used, and any reprocessing details. It also states that the system audit trail, including system administrator actions, should be periodically reviewed.

Archival, retrieval, back-up, disaster recovery, and contingency plans

. The questions in this section focus on ensuring that electronic data are enduring, complete, and secure from modification or loss. The electronic data (including associated metadata) should be in the original format or in a format that is compatible with the original format.

. The questions in this section focus on the use (where necessary) of electronic signatures, which should ensure attributability to the specific person who signed the document electronically.

The risk-assessment tool consists of a summary, followed by a tabular presentation as depicted in 

. The entire tool is available on the IQ Consortium web site (6). The expected responses to be filled into these sections are described below. The summary section should contain an overall evaluation of the acceptability (from a data integrity standpoint) of the system under study. This evaluation should take into account any mitigations and interim actions identified in the risk assessment.

In addition, users may find it convenient to describe (in general terms) the configuration of the system, including the data storage location and any data safeguards that are associated with it (e.g., if the data are written to a drive to which the user does not possess rights to modify or delete files). Other mitigations that apply to multiple lines in the assessment may also be conveniently discussed in the summary section and the summary section referenced in the affected lines.

. The first column contains a list of data-integrity requirements, each phrased as a question for which “yes” is the desired response. Options include yes, no, and not applicable (NA). These questions are designed to prompt a thorough investigation of the system at hand. As mentioned, the questions are phrased such that the desired response is “yes.” It is important to note that the assessment should be performed against the system (including the hardware, software, and any governing procedures) as it currently exists. If the system does not meet the expectation described in the question, a response of “no” should be selected.

In rare instances, a response of N/A may be appropriate. For example, one question asks whether libraries are included in the data backup. For software that does not utilize a library search, this question would properly be answered N/A.

. The Guidance column provides additional context for, and clarification of the questions. In some cases, it also provides suggestions on potential mitigations.

. The final column can be used to describe the manner in which the system meets the requirement (in the case of a “yes” answer). When the “no” response is selected, it should be used to describe the manner in which the risk therein identified is mitigated.

There also may be the scenario where a “no” response is selected with a rationale for why the risk is considered acceptable. Finally, in the case of an N/A response, the context which renders the question not applicable should be explained.

The IQ working group is publishing this risk assessment tool to facilitate data-integrity risk assessments and promote standardization and harmonization of practice. However, its developers believe that it may be useful in other applications as well.

For instance, pharmaceutical firms may find it helpful to use the tool (or a subset of it) when evaluating instrument/software systems to aid in purchasing decisions. Similarly, instrument vendors may find it useful in understanding the needs and expectations of the pharmaceutical industry.

The tool may also be useful in managing third-party laboratories and vendors who service equipment and in drafting technical quality agreements with them. Finally, it is expected that implementation of the tool will aid pharmaceutical firms in internal harmonization and benchmarking, across sites within a company.

The authors wish to acknowledge the contributions of the IQ Working Group for Data Integrity in R&D Analytical Laboratories.

, Title 21, Food and Drugs (Government Printing Office, Washington, DC), Part 211
2. MHRA, GXP Data Integrity Guidance and Definitions (London, UK, March 2018).
3. FDA, Data Integrity and Compliance with Drug CGMP - Questions and Answers (Rockville, MD, December 2018).
4. World Health Organization, 

 (Geneva, Switzerland, October 2019).
5. Pharmaceutical Inspection Convention Pharmaceutical Inspection Co-operation Scheme, 

 (Geneva, Switzerland, November 2018).
6. IQ Consortium, 

www.iqconsortium.org/initiatives/projects/iq-data-integrity-manuscript

 both work in analytical research and development at Pfizer Inc. (Groton, CT); 

 works in analytical research and development and 

 works in quality, both at AbbVie Inc.(North Chicago, IL); K

 both work in quality at Merck & Co., Inc., (Kenilworth, NJ and West Point, PA respectively); 

 works in quality at Boehringer Ingelheim Pharmaceuticals Inc. (Ridgefield, CT).


Vol. 44, No. 8
August 2020
Pages: 51­-53

When referring to this article, please cite it as J. Lippke, et al. “Assessing Data Integrity Risks in an R&D Environment,” 

A data-integrity risk assessment tool has been developed for use with standalone R&D data-acquisition and processing software. 