A Perspective on Risk Analysis for the GMP Initiative (PDF) - PQRI

A <strong>Perspective</strong> on Risk
Analysis for the GMP
Initiative
H. Gregg Claycamp, Ph.D., CHP
Center for Veterinary Medicine
Office of New Animal Drug Evaluation
hclaycam@cvm.fda.gov
April 2003
1

Outline
• The Premise and Questions
• Basics of Risk Analysis
• Possible Stages of Risk Assessments for GMP
Initiative
• Risk Ranking Model for a GMP Initiative?
• Pilot Scale
• Conclusions
The opinions and ideas presented here are those of the author and do not
represent policy or opinion of the FDA. This material is intended for
discussion purposes only.
2

Premise: Links Among Process (GMP)
Risks and Patient Risks are Lost
cGMP
Patient
Correlation?
3
Processes
Inspection Risk
Quality (Patient)
Factors
RISK
RISK

Goal: Re-Link cGMP Risks with Actual
Risks to the Patient
cGMP Patient
4
Processes
Inspection Risk
Quality (Patient)
Factors
RISK
RISK

The Question…
• Can Risk Management theory, tools, practice and
philosophy be employed to re-link risks to the patient
with the risks identified, perceived or otherwise
implicated in the product quality regulatory process?
• How can we share a common language about risk, risk
management and science-based decision making so that
we can focus on developing a high-quality risk
management model for product quality?
5

Getting Started…
• What theories, tools and lessons learned in risk
analysis can help address these questions?
• Given the need for a significant shift in the
approach to risk management, how do we begin
the change process?
Are there off-the-shelf models and tools that
might be used, i.e., at a pilot-scale?
What kinds of RM processes can be used to
foster changes needed both the regulatory
and industrial spheres?
6

Basic Risk Analysis
7

Starting with the Some Basics
Risk is intuitive and familiar to everyone, yet few
among us define risk carefully and formally
enough for complex risk analysis.
8

Risk = “exposure to a chance of loss”
(or, Risk = “chance of losing something
we value”)
Risk = Hazard x Exposure
Risk Consequence = Hazard x Exposure
9

Contemporary Risk Analysis
• Includes four major activities:
Hazard Identification
Risk Assessment
Risk Management
Risk Communication
10

Risk Assessment Precedes Risk
Management
• Risk assessment is not a single process, but “a
systematic approach to organizing and analysing
scientific knowledge and information.” NRC (1994)
• Various paradigms exist for the execution of a
risk assessment in public health; however, all
paradigms have in common fundamental
principles.
11

Risk Assessment Asks:
• What can go wrong?
• What is the likelihood it would go wrong?
• What are the consequences?
12

Risk Management Asks:
• What can be done?
• What options are available?
• What are risk trade-offs in terms of risks,
benefits and costs?
• What are the impacts of current
management decisions on future options?
13

Risk Analysis in a Democracy
• Risk assessments provide the “facts” for risk
analysis.
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Assessment
Risk
Risk
Risk
Risk
Risk
Risk
Worst
14

Risk Analysis in a Democracy
• The risk management decisions about which
risks to manage are value-laden decisions.
Worst
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Values
Risk
Management
Costs
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk Management
Rank
15

Translating Risk Analytic Paradigms
Risk Analysis
Hazard Identification
Risk Assessment
• Release GMP Failures Assessment
• Exposure Assessment
• Consequence Assessment
• Risk Estimation
Risk Management
Risk Communication
A simple
simple
change
change
to
to
apply
apply
a
contemporary
contemporary
model.
model.
•• What What can can go go wrong? wrong?
•• What What are are the the consequences?
consequences?
•• What What is is the the likelihood likelihood that that it it
would would go go wrong? wrong?
•• What What can can be be done? done?
•• What What are are the the trade-offs trade-offs in in terms terms
of of costs, costs, benefits benefits and and risks? risks?
•• What What is is the the impact impact of of decisions decisions
on on future future RM RM options? options?
16

Possible Stages of Risk
Assessment for the
GMP Initiative
17

Hazard Identification
• What can go wrong?
Identify hazards: events
Identify hazardous agents (chemical, biological,
physical)
• How severe are the potential consequences?
Given the event occurs, is the consequence
catastrophic? Mildly annoying?
• How likely are the events to occur?
Essentially a crude risk estimate for initial prioritization
purposes.
18

Exposure Assessment
• Release Assessment: How “much” of the
hazardous event occurs?
Example: Does a “non-sterile” event involve 1
or 10,000 vials?
• Pathway analysis: If the hazardous event
occurs, what pathways are there that expose
humans to the hazard?
• Extent of exposure: If a hazardous event occurs,
how many people are potentially exposed?
19

GMP Failure (Release) Assessment
• How frequent are the identified GMP events
(hazards)?
• Boundary of release? Process line, plant,
warehouse, distributor?
• Release rates (“GMP Faults”) are obtained in
fault tree assessments, empirically, historical
data, expert analyses.
Example: FMEA
20

Consequence Assessment*
• Given exposure to the hazardous event/agent,
what is the likelihood of harm under a predefined
endpoint?
Endpoint examples:
• Death
• Illness
• Worry
• OAI
*A.K.A. “Dose-Response Assessment” (see next slide)
21

Consequence Assessment
100%
Proportion of exposed
persons who become ill
50%
0%
ED 50
Quantity of contamination (“non-sterility”)
i.e., in “bacteria counts per vial”
22

Qualitative Consequence Assessment
Relative Effect/Impact
High
Medium
Low
Quantitative
relationships known
in few cases
Low
Medium
(Exposure or Dose Metric)
High
23

Risk Estimation
• Bring together the information about
the hazard,
the extent of exposure to the hazard,
the consequences of exposures, and then
estimate the risk.
• Includes a critical analysis of uncertainty in
both the data and risk assessment models.
24

Uncertainties in Risk Assessment
Knowledge
•Data
•Parameters
•Model
Variability
•Temporal
•Spatial
•Inter-individual
25

Conceptual Models for
RM in GMP Initiative
26

The cGMP Risk Management Problem
• Diverse GMP failure (hazards) are identified.
• Wide-ranging risk (= chance that exposure to the
hazard will result in harm [adverse outcome]).
How can we objectively rank
• Wide-ranging consequences (death to worry).
“apples and oranges” among the
• Quantitative “potatoes risk analysis and hazard-by-hazard beans?” too
vast an undertaking.
• Ranking of risks for re-linking worst GMP risks
with worst health risks, etc.
27

Bulb
Fails
Fault Trees for each process?
No
electricity
Glass
Broken
Filament
Broken
Vacuum
Leak
Power Plant
Fails
Power Line
Fails
Connector
Corroded
Impurities
Vibrations
Wind Breaks
Line
Tree Breaks
Line
28

Faults Magnified N-fold for a Simple
Manufacturing Process
29

Decision Analyses for Each Hazard Multiplies
Complexity!
e.g.,
30

Solution? A Multifactor Approach to
GMP Risk Management
• Multifactor methods already exist.
• Some tools (software) already developed.
• Appropriately-scaled approach to
the question,
the data quality,
the nature of the decision, and
the understanding of the overall process.
31

State the Assumptions
• E.g., assume that health risks were linked to
GMP “compliance risks” previously, i.e., the
historical basis of regulation.
Historically based assumption:
↑compliance
↓Health risk
↑quality
• Given the assumption, can GMP “compliance
risk” be modeled as a surrogate of health risk?
32

Identify the GMP Failures (Hazards)
• What can go wrong?
• Top level organization of hazards:
Health | Compliance | Resources | Sociopolitical
• Second level (detail) organization:
Sterility (microbial contamination)
Dose (formulation)
Toxicity (chemical contamination)
Physical hazards (physical contamination/defect)
• Fine detail: “risk factor” event descriptors.
33

Sort the Hazards/Risks by Major
Categories
• Start with assumptions.
• State questions to be answered.
• Sort under the questions.
• Re-sort if new patterns emerge.
For example, (next slide)…
34

Organizing a Multi-factorial Risk Model
Risk
Health Compliance Resource
Socio-
Political
…
Death
VAI
Human
Public
Chronic Illness
OAI
Inspection $
Industry
Acute Illness
The Hill
Mental Health
35

Focused Multi-factorial Risk Model
Risk
Health
Compliance
Example
health risk
endpoints
Death
Chronic Illness
Acute Illness
VAI
OAI
Example
compliance
risk endpoints
Mental Health
36

Risk factors for a given endpoint…
Health
Compliance
Death
OAI
Sterility
Sterility
Lyophilization
Lyophilization
Final Sterility
Final Sterility
…
etc. …
…
etc. …
37

Estimate the Prevalence
• The prevalence of inspection findings for a
given type of event are initial estimates of
probabilities necessary for risk
management modeling.
• Failure analysis “in plant.”
• Failure in compliance inspections.
• Human adverse events.
38

For each hazard…
Health
Probability of Occurrence
Endpoint
Very
Low
Low
Medium
High
Very
High
Death
Medium
Medium
High
High
High
Chronic
Illness
Low
Medium
Medium
High
High
Acute
Illness
Low
Low
Medium
Medium
High
Worry
Low
Low
Low
Medium
Medium
39

The modeler’s view… (for example)
Health
Probability of Occurrence
Endpoint
Very
Low
Low
Medium
High
Very
High
Death
5
4
3
2
1
Chronic
Illness
6
5
4
3
2
Acute
Illness
7
6
5
4
3
Worry
8
7
6
5
4
40

For each hazard…
Compliance
Prior History of Actions
Endpoint
Never
Violations
Few
Viol.
Average
Viol.
Some
Viol.
Many
Viol.
OAI
Medium
Medium
High
High
High
VAI
Low
Low
Medium
High
High
Other?
Low
Low
Low
Medium
High
41

Scoring, then prioritize multiple hazards
Endpo
int
Death
Chron
ic
Illness
Acute
Illness
Worry
Ve
ry
Lo
w
Endpo Me
int diu
m
Death
Lo
w
Chron
ic
Illness
Lo
w
Acute
Illness Lo
w
Worry
Probability of Occurrence
Low Ve
ry
Lo
Med w
ium
Me
Endpo diu
int Med m
ium
Death Lo
w
Low
Chron
ic
Lo
Illness
w
Low
Acute Lo
Illness w
Ver
Probability of Occurrence y
Medi Hi Hig
um gh h Ver
y
Probability Medi of HiOccurrence
Hig
Low Hi Hig
High um gh h
Ve gh h
Ver
ry
y
Med Lo
Hi Hig
High MediProbability of Occurrence Hig
ium Medi w Low Hi Hig um gh gh h h
um gh h
Ve
Me
ry
Med
Medi
Hig
diuEndpo
Lo
Me
High
Medi
ium Medi
ium um Hig gh gh
mint
w Low h um
diu
um
h
m
Me
Lo
Medi MediMed
Hig
Low
Death diu
Me diu
High
w ium um Med
um ium
m
gh h
Low diu m
ium
m
Chron
Me
Lo
Lo MediMed
Med
Medi
Low
ic Low Low diu diu
w
w um ium
ium
um
Illness
m m
Hig
h
Hig
h
Hig
h
Hi
gh
Hi
gh
Hi
gh
Ver
y
Hig
h
Hig
h
Hig
h
Scored and Prioritized
1. GMP Fault A
2. GMP Fault T
3. GMP Fault C
4. GMP Fault D
Worry
LoAcute
Low
wIllness
Worry
Lo
w
Lo
w
Low Low
Low
Me
diu
m
Medi
um
Low
Med
ium
Me
diu
m
Me
diu
m
Hig
h
Med
ium
5. GMP Fault X
6. GMP Fault M
42

Risk Ranking & Filtering Model
Health
Compliance
Other…
(Risk Ranking and Filtering)
Scored and Prioritized
Under Multiple Criteria
1. GMP Fault M
2. GMP Fault T
3. GMP Fault C
4. GMP Fault D
5. GMP Fault X
6. GMP Fault A
43

Risk Analysis Cycle
Start
Risk Assessment
Assessments
(Data Bases)
cGMP/Compliance
Inspections
Risk Management
Work Planning
Multi-Factorial
Risk Model
Risk Risk Ranking
and and Filtering
44

Pilot Scale?
45

Example Approach to Build RRF List*
Risk Estimator
Respondents
x GMP Faults
Database
- Risk Managers
- GMP Experts
- Sr. Managers
- Industry
Risk Ranking
Analysis
46

Fold into cGMP Model
Risk Ranking
(Table)
Best Worst
Risk Management
“Cut-Off”
• Budget
• Risk Tolerance
• Benefit-Costs
• Stake holders
47

Conclusions
• Risk Assessment provides a process for
organizing information in support of risk-based
decision making.
• Risk assessment is one of the tools available for
Risk Management, the activity in which the
options for controlling risks are examined in light
of costs, benefits and risk trade-offs.
• Multifactor Risk Ranking and filtering approach
might be robust enough to employ in the GMP
Initiative.
48