OR WAIT null SECS
MIT survey results address product and site characteristics that statistically correlate with quality performance.
Although there is awareness among the biopharmceutical industry of the challenges and trade-offs associated with monitoring the safety of drug products, there does not seem to be a clear and consistent understanding of characteristics correlated with the presence of critical-to-quality deviations of a product at the commercial scale (1). The intent of this paper is to initiate an understanding of product and site characteristics that statistically correlate with quality performance of biopharmaceutical commercial products. Towards this objective, a comprehensive survey on related topics is underway at the Massachusetts Institute of Technology (MIT) Center for Biomedical Innovation (CBI). The interim results described in this research represent survey results from 34 commercial-scale biopharmaceutical products at 11 international sites. This research builds on analysis done by the MIT CBI team earlier this year (2).
Some regulatory agencies, including FDA, have actively pursued a risk-based approach to manufacturing site inspection because their inspections are site-specific and not restricted to an individual product (1). Explicit inputs that FDA uses for its risk-based selection of sites to inspect include the types of products manufactured (e.g., prescription versus over-the-counter, approved versus unapproved, therapeutic classes); the control and/or contamination potential of the manufacturing environment (e.g., facility size, facility type, number of drug products, sterility requirements); and cost of inspection (e.g., domestic versus international locations). These inputs are logical for consideration of total risk to quality at the level of a site, but, in this research, the focus regarding quality is on the product.
A comprehensive survey on topics related to site characteristics, quality approaches, quality activities, perception of relative consistency of regulators, product-level process details, and product-level compliance performance has been developed by MIT's CBI. The survey includes questions that result in a total of 57 variables across these topic areas. The developed survey was tested with industry representatives and then deployed over the past 18 months as a secure web-based questionnaire directly to biopharmaceutical manufacturing sites through several different channels. The individuals targeted for inclusion as survey respondents have been manufacturing plant managers or their representatives.
In total, 20 sites representing 52 manufactured products have completed the survey. These responses have been filtered to focus on commercial-scale manufacturing and products for which all relevant information was provided. This reduced the number of products included thus far to 34 across 11 sites.
The research is supported by the Alfred P. Sloan Foundation of New York City.
Results and discussion
Characteristics of the products and sites. The demographics of the 34 products included in the analysis are organized into the following groups:
Product and process:
Characteristics of the manufacturing site:
Based on these overal demographics, the products in the MIT CBI survey represent a reasonably diverse set of biopharmaceutical products, processes, and manufacturing sites.
Distribution of technical personnel within the manufacturing sites. The distribution of technical personnel within a manufacturing site provides some representation of the priorities and/or focus of the site. The sites reported the number of personnel in each of the following technical areas: quality control (QC)—average 12.6%; quality assurance (QA)—average 17.4%; engineering support and services average 8.8%; technical services—average 16.3%; and manufacturing—average 44.9%.
Figure 1: Fraction of personnel associated with each of the technical areas at different sites. QA is quality assurance; QC is quality control; ESS is engineering support and services; and TS is technical services. (ALL FIGURES COURTESY OF THE AUTHORS)
A histogram with the fraction of personnel in each of the first four areas is presented in Figure 1. The histogram of fraction of technical personnel in manufacturing is shown in Figure 2. There is a wide distribution of where technical personnel are assigned within the sites (particularly visible from the distribution of technical personnel in manufacturing) suggesting one or more of the following scenarios across the sites that completed the survey:
Figure 2: Fraction of technical personnel in a manufacturing position across sites surveyed.
Site-level drivers of change. Each site was asked to identify significant sources, or drivers, of changes in quality efforts and activities at their site. The respondents indicated whether specific drivers had been experienced by the site in the past and/or are anticipated drivers for the site in the future. The results from the survey of the sites are presented in Figure 3, ordered from left to right in descending order of fraction of sites identifying the type of driver as one that has been experienced in the past. Of particular note are the following findings:
Figure 3: Experienced and expected drivers of quality activities and resources.
Correlations with critical-to-quality product deviations. Using critical-to-quality defects at the commercial scale as the dependent variable, 56 other variables were compared based on likelihood ratios for each of the 34 commercial products. Of these 56 variables, 11 were found to have statistically significant different likelihoods depending upon whether a critical-to-quality defect was experienced for that product. These 11 variables of significance are shown in Figure 4 and grouped based on confidence level of statistical significance (the top row contains variables for which the difference in likelihood ratio demonstrated greatest confidence) and grouped based on whether the correlation was negatively or positively correlated with a critical-to-quality defect at the commercial scale. If the variable is in the negatively correlated section of Figure 4, then presence of that variable is more likely to occur for a product without a reported critical defect.
Table I: Variables that were found to be statistically different in likelihood of correlating with a critical-to-quality defect.
For example, a product manufactured at commercial scale in a site having an above-median fraction of technical personnel working in QA was statistically less likely to have experienced a critical-to-quality defect at the commercial scale. On the other hand, a positively correlated variable is more likely to be present for a product that has experienced a critical defect. For example, a product manufactured in Asia is more likely to have experienced such a defect.
Consistent with prior preliminary discussion regarding variables correlated with product defects, many of the statistically significant variables presented in Figure 4 fall into the categories of history of quality issues in the product lifecycle, geographic region of manufacture, and manufacture in a contract manufacturing site (2). Of note is the dependence upon fraction of technical personnel employed in QA and employed in manufacturing. More generally, this dependence is related to the discussion of the distribution of technical personnel across a site as described above, where the lack of consistency and, perhaps, understanding of an optimal distribution was highlighted. The data presented in Figure 4 demonstrates that above median level of the fraction of technical personnel in QA and below median level of the fraction of technical personnel in manufacturing are both correlated with lower likelihood of a critical-to-quality defect at the commercial scale.
It is possible that greater emphasis on quality is demonstrated by fraction of personnel in QA compared with an emphasis on execution demonstrated by fraction of personnel in manufacturing. Further investigation in this area and clarity regarding the expectations of technical personnel roles could provide greater insight into this observation.
Although there were 11 variables significantly correlated with likelihood of a critical-to-quality defect at the commercial scale, there were also 45 variables found not to be so. Several of these variables are notable as not having been found to correlate with critical-to-quality defects at the commercial scale, including:
With the intent of increasing understanding of variables associated with commercial quality performance of individual biopharmaceutical products, a survey was conducted with questions related to the product lifecycle and site characteristics. The interim analysis presented here was based on 34 products across 11 sites with diverse characteristics.
The analysis focused on determining which product or site variables were statistically more likely to be present for products that have experienced a critical-to-quality defect at the commercial scale. The themes of variables found to be statistically significant based on the likelihood analysis were:
A number of themes not found to be statistically significant for association with commercial critical-to-quality defects, based on the data available, include:
The CBI MIT research is ongoing, both specific to this survey and more generally regarding the understanding of product quality.
For more information
To take the survey on behalf of a biomanufacturing site, use this link: https://survey.vovici.com/se.ashx?s=664A932C76367731. Or contact MIT CBI at firstname.lastname@example.org, tel. 617.253.0257.
Reuben D. Domike is affiliated with the Center for Biomedical Innovation (CBI), Massachusetts Institute of Technology (MIT), and the School of Business at the Univ. of Prince Edward Island; Jeffrey T. Macher is affilated with CBI, MIT, and the McDonough School of Business at Georgetown Univ.; Paul W. Barone and Stacy L. Springs are affiliated with CBI, MIT; Anthony J. Sinskey is affiliated with the MIT Department of Biology; and Scott Stern is affiliated with the MIT Sloan School of Management.
1. FDA, "Risk-Based Method for Prioritizing CGMP Inspections of Pharmaceutical Manufacturing Sites—A Pilot Risk Ranking Model" (FDA, Rockville, MD), (2004).
2. R. Domike et al., Bioprocessing & Sterile Mfg., supp. to Pharm. Technol. 36 (5) s12 (2012).
3. L. Yu, Pharm. Res. 25 (4) 781–791 (2008).
4. M. Hermanto, R. Braatz, and M Chiu, AIChE Journal 57 (4) 1008–1019 (2011).