Repeated-dose 90-day Oral Toxicity Studies on Whole ... - Europabio
Repeated-dose 90-day Oral Toxicity Studies on Whole ... - Europabio
Repeated-dose 90-day Oral Toxicity Studies on Whole ... - Europabio
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
<str<strong>on</strong>g>Repeated</str<strong>on</strong>g>-<str<strong>on</strong>g>dose</str<strong>on</strong>g> <str<strong>on</strong>g>90</str<strong>on</strong>g>-<str<strong>on</strong>g>day</str<strong>on</strong>g> <str<strong>on</strong>g>Oral</str<strong>on</strong>g><br />
<str<strong>on</strong>g>Toxicity</str<strong>on</strong>g> <str<strong>on</strong>g>Studies</str<strong>on</strong>g> <strong>on</strong> <strong>Whole</strong><br />
Food/Feed<br />
Presenters:<br />
Kevin Leiner, Margaret Nemeth, and<br />
Keith Ward<br />
1
Introducti<strong>on</strong><br />
• EFSA has stated that its guidance <strong>on</strong> repeated-<str<strong>on</strong>g>dose</str<strong>on</strong>g> <str<strong>on</strong>g>90</str<strong>on</strong>g>-<str<strong>on</strong>g>day</str<strong>on</strong>g> oral toxicity<br />
studies in rodents <strong>on</strong> whole food/feed is “not intended to provide<br />
prescriptive experimental test protocols.”<br />
• EuropaBio Members agree there should be flexibility in the design of future<br />
studies with whole food/feed and that justificati<strong>on</strong> for the approach to the<br />
study is important.<br />
• The new EFSA guideline will:<br />
– Impact design, c<strong>on</strong>duct, analysis and interpretati<strong>on</strong> both toxicologically and<br />
statistically.<br />
– Create challenges that require additi<strong>on</strong>al discussi<strong>on</strong> with EFSA and other<br />
stakeholders in the EU.<br />
• EuropaBio Members appreciate the opportunity to discuss these new<br />
challenges, and their proposed soluti<strong>on</strong>s to them, with key stakeholders in<br />
the EU.<br />
2
Existing safety data and the<br />
weight of evidence<br />
• Prior to the c<strong>on</strong>duct of the <str<strong>on</strong>g>90</str<strong>on</strong>g>-<str<strong>on</strong>g>day</str<strong>on</strong>g> study several safety<br />
assessments have been c<strong>on</strong>ducted including:<br />
– Compositi<strong>on</strong>al analysis, agr<strong>on</strong>omic performance, and molecular analysis of the<br />
crop.<br />
– Evaluati<strong>on</strong>s of the inserted trait and its protein product:<br />
• Bioinformatic evaluati<strong>on</strong>s, in vitro digestibility and heat stability assays as well as<br />
protein toxicity studies <strong>on</strong> a case-by-case basis.<br />
• Establishing Substantial Equivalence between the GM and<br />
c<strong>on</strong>venti<strong>on</strong>al crop, and an absence of adverse findings in the<br />
indicated tests, provides a weight of evidence for the safety of<br />
the whole food (Codex, 2009).<br />
• EuropaBio Members believe the 15 + year history of safe use of<br />
GM crops evaluated and approved under this approach<br />
attests to its strength.<br />
3
Hypothesis of safety and<br />
study justificati<strong>on</strong><br />
• In such cases, EuropaBio Members believe there is no<br />
specific hypothesis to test in the <str<strong>on</strong>g>90</str<strong>on</strong>g>-<str<strong>on</strong>g>day</str<strong>on</strong>g> study and<br />
thus no justificati<strong>on</strong> for them.<br />
– EFSA published opini<strong>on</strong> appears to agree with this positi<strong>on</strong>.<br />
• EFSA Panel <strong>on</strong> Genetically Modified Organisms (GMO). Scientific opini<strong>on</strong>-<br />
Guidance for risk assessment of food and feed from genetically modified<br />
plants. EFSA Journal 2011; 9(5):2150<br />
• However, if potential indicati<strong>on</strong>s of toxicity are<br />
present in the existing data, then EuropaBio Members<br />
believe the principles of Hazard Identificati<strong>on</strong> should<br />
be followed to maximise the possibility of detecting<br />
toxic effects of the test diet.<br />
4
Hazard Identificati<strong>on</strong> and<br />
multiple <str<strong>on</strong>g>dose</str<strong>on</strong>g> levels<br />
• For Hazard Identificati<strong>on</strong> purposes, administering a <str<strong>on</strong>g>dose</str<strong>on</strong>g> lower than the<br />
highest possible (which maintains nutriti<strong>on</strong>al balance) is of disputable<br />
value.<br />
• Testing at the highest possible c<strong>on</strong>centrati<strong>on</strong> maximises exposure and thus<br />
the potential to detect unanticipated adverse effects.<br />
– C<strong>on</strong>sistent with the limit <str<strong>on</strong>g>dose</str<strong>on</strong>g> approach detailed in OECD TG 408 (1998).<br />
– Requirement of nutriti<strong>on</strong>al balance limits c<strong>on</strong>centrati<strong>on</strong> in diet.<br />
– EuropaBio Members estimate the highest possible c<strong>on</strong>centrati<strong>on</strong> of soybean<br />
and maize will typically exceed human intake in the EU 10X to 100X (WHO,<br />
2003; EFSA 2011c).<br />
• It is difficult to scientifically justify the inclusi<strong>on</strong> of the low <str<strong>on</strong>g>dose</str<strong>on</strong>g> group if<br />
regulating <strong>on</strong> the basis of Hazard Identificati<strong>on</strong>.<br />
– A “clean” low <str<strong>on</strong>g>dose</str<strong>on</strong>g> level would then likely be irrelevant if the high <str<strong>on</strong>g>dose</str<strong>on</strong>g> is<br />
adverse.<br />
5
The regulatory paradigm for<br />
GM crops<br />
• Based <strong>on</strong> practical experience, EuropaBio<br />
Members have presumed the current<br />
regulatory paradigm for GM crops is based<br />
<strong>on</strong> Hazard Identificati<strong>on</strong> (or rather an<br />
absence of Hazard).<br />
• EFSA’s inclusi<strong>on</strong> of a low <str<strong>on</strong>g>dose</str<strong>on</strong>g> level could be<br />
interpreted as a willingness to regulate GM<br />
crops <strong>on</strong> the principle of Risk.<br />
– Risk = Hazard × exposure<br />
6
Secti<strong>on</strong> summary<br />
• EuropaBio Members believe:<br />
– When there is no evidence of unintended changes, the<br />
<str<strong>on</strong>g>90</str<strong>on</strong>g>-<str<strong>on</strong>g>day</str<strong>on</strong>g> study would not normally be expected to add<br />
value to the safety assessment – it is largely<br />
c<strong>on</strong>firmatory.<br />
– The current regulatory paradigm for GM crops is based<br />
<strong>on</strong> Hazard Identificati<strong>on</strong>.<br />
– The purpose of testing a <str<strong>on</strong>g>dose</str<strong>on</strong>g> below the highest dietary<br />
incorporati<strong>on</strong> level that maintains nutriti<strong>on</strong>al balance is<br />
unclear and needs to be resolved.<br />
7
Discussi<strong>on</strong><br />
• EuropaBio Members agree with EFSA’s Scientific<br />
Opini<strong>on</strong> (2011a) that an applicant could forego<br />
c<strong>on</strong>ducting a <str<strong>on</strong>g>90</str<strong>on</strong>g>-<str<strong>on</strong>g>day</str<strong>on</strong>g> study if data indicating the GM<br />
crop and its parental isoline:<br />
– Are Substantially Equivalent in regard to nutriti<strong>on</strong>al compositi<strong>on</strong>.<br />
– Dem<strong>on</strong>strate no indicati<strong>on</strong>s of unintended adverse effects in<br />
molecular, compositi<strong>on</strong>al, or phenotypic analyses.<br />
• EuropaBio Members request clarificati<strong>on</strong> <strong>on</strong> whether<br />
inclusi<strong>on</strong> of two (or more) <str<strong>on</strong>g>dose</str<strong>on</strong>g> levels is mandatory<br />
and, if so, how results from the low <str<strong>on</strong>g>dose</str<strong>on</strong>g> will be<br />
used.<br />
8
Part 1: Study design and c<strong>on</strong>duct<br />
9
Challenges in study design<br />
and c<strong>on</strong>duct<br />
1. Designing for appropriate power<br />
2. Experimental design and c<strong>on</strong>duct<br />
a) Practical aspects regarding cage locati<strong>on</strong><br />
b) Variati<strong>on</strong> c<strong>on</strong>trols<br />
3. Housing c<strong>on</strong>diti<strong>on</strong>s and impact <strong>on</strong> the<br />
Experimental Unit (ExpU)<br />
4. Statistical power investigati<strong>on</strong><br />
10
1. Defining toxicologically<br />
relevant differences<br />
• As per guidance, relevant endpoints, c<strong>on</strong>sistent with OECD TG<br />
408, will be m<strong>on</strong>itored.<br />
– TG 408 specifies that each test animal will be m<strong>on</strong>itored <strong>on</strong> >200 individual<br />
endpoints.<br />
– Ensures a thorough examinati<strong>on</strong> of the whole animal and that known<br />
mechanisms of toxicity will be evaluated.<br />
• Toxicologists will review the data for c<strong>on</strong>sistent trends in related<br />
endpoints associated with toxic mechanisms.<br />
– ↑ serum ALT, ↑ liver weight, histopathological altera<strong>on</strong>s of liver ssue.<br />
– ↑ serum ALT, in the absence of other indica<strong>on</strong>s of liver damage, is of equivocal<br />
importance.<br />
• To reach scientifically valid c<strong>on</strong>clusi<strong>on</strong>s, the weight of evidence<br />
must be c<strong>on</strong>sidered to determine if the difference observed is<br />
treatment-related.<br />
11
1. Defining toxicologically relevant<br />
differences (c<strong>on</strong>tinued)<br />
• Identifying a toxicologically-relevant difference for individual<br />
endpoints in isolati<strong>on</strong> is very difficult, especially when there<br />
is no evidence of adverse changes in related parameters.<br />
• However, to perform power calculati<strong>on</strong>s, relevant effect sizes<br />
must be defined.<br />
• EFSA guidance acknowledges that this is no easy task, and<br />
does not provide specific guidance in this respect.<br />
• To determine endpoints and effect sizes with the greatest<br />
potential to identify toxicity, EuropaBio toxicologists<br />
examined numerous guidance documents.<br />
• Guidance documents <strong>on</strong> determining a Maximum Tolerated<br />
Dose (MTD) provided definitive endpoints and effect sizes.<br />
12
1. Endpoints and effect<br />
sizes<br />
Endpoint Change relative to c<strong>on</strong>trol<br />
Body weight Decrease 10%<br />
Liver weight Increase 25%<br />
Kidney weight Increase 25%<br />
Leukocyte count Decrease/increase 30%<br />
Lymphocyte count Decrease/increase 30%<br />
Cholesterol Increase 200%<br />
Blood urea nitrogen Increase 50%<br />
Creatinine Increase 50%<br />
Alkaline phosphatase Increase 200%<br />
(Foster, 2002; EPA, 2002; EPA, 2003; Rhomberg et al., 2007).<br />
13
1. Toxicological endpoints<br />
for power calculati<strong>on</strong>s<br />
• The MTD has also been defined as the <str<strong>on</strong>g>dose</str<strong>on</strong>g><br />
that, “…does not produce mortality, clinical<br />
signs of toxicity, or pathologic lesi<strong>on</strong>s…”<br />
(Dorato et al., 2008; S<strong>on</strong>tag, 1976).<br />
• Hence, it is a <str<strong>on</strong>g>dose</str<strong>on</strong>g> that elicits signs more<br />
subtle than overt toxicity.<br />
• Endpoints and effect sizes are taken from the<br />
indicated guidance documents.<br />
14
1. Toxicological endpoints for<br />
power calculati<strong>on</strong>s (c<strong>on</strong>tinued)<br />
• The toxicological relevance of any differences observed will be<br />
determined <strong>on</strong> a case-by-case basis.<br />
– Alkaline phosphatase factors: AST levels, ALT levels, liver weight, liver<br />
histopathology, etc.<br />
• All effect sizes proposed have been c<strong>on</strong>sidered <strong>on</strong> the scale of<br />
original units.<br />
– This is the scale that makes most sense to toxicologists as it provides<br />
necessary c<strong>on</strong>text for evaluati<strong>on</strong> of biological significance.<br />
• Specifying effect sizes <strong>on</strong> the original units scale removes the<br />
need to c<strong>on</strong>sider effect sizes <strong>on</strong> the standardised scale.<br />
– To be discussed in greater detail later.<br />
15
1. Secti<strong>on</strong><br />
summary/discussi<strong>on</strong><br />
• EuropaBio Members believe the endpoints<br />
and toxicologically-relevant effect sizes chosen<br />
provide a robust platform <strong>on</strong> which to base<br />
subsequent power calculati<strong>on</strong>s for the study<br />
as a whole.<br />
• All effect sizes proposed have been<br />
c<strong>on</strong>sidered <strong>on</strong> the scale of original units; the<br />
scale which makes the most sense to<br />
toxicologists.<br />
16
2. Experimental design<br />
• EFSA guidance advocates use of a randomised complete block<br />
design (RCBD) in which males and females are randomised<br />
together.<br />
• However, current toxicology study animal husbandry practices<br />
intenti<strong>on</strong>ally separate males and females (e.g., by assigning<br />
genders to different cage racks).<br />
• There is a c<strong>on</strong>cern that housing opposite genders in adjacent<br />
cages, particularly in paired housing setups, may lead to intra-<br />
MSK2 MSK3<br />
cage aggressi<strong>on</strong> in males.<br />
• EuropaBio Members’ str<strong>on</strong>g recommendati<strong>on</strong> is that the<br />
current practice of gender separati<strong>on</strong> be maintained.<br />
17
Slide 17<br />
MSK2 Follow up with technical folks.<br />
Animal behavior studies.<br />
Best practices, etc.<br />
Michael Koch; 1/10/2012<br />
MSK3 Will be stated in much str<strong>on</strong>ger language if available informati<strong>on</strong> allows. Still being researched.<br />
"Housing genders in adjacent cages is specifically avoided to prevent intra-cage aggressi<strong>on</strong>.."<br />
Michael Koch; 9/10/2012
2. Experimental design<br />
(c<strong>on</strong>tinued)<br />
• EFSA guidance states that use of designs other than<br />
RCBD “may be acceptable provided that appropriate<br />
justificati<strong>on</strong> for using them is given”.<br />
• EuropaBio Members agree that, in circumstances<br />
where a good reas<strong>on</strong> for blocking exists, an RCBD is<br />
an appropriate choice.<br />
• However, in the absence of a known blocking<br />
structure, a completely randomised design (CRD) is<br />
the most obvious choice and indeed is likely to be<br />
slightly more powerful than an RCBD.<br />
18
2. Variati<strong>on</strong> c<strong>on</strong>trols<br />
• The following variati<strong>on</strong> c<strong>on</strong>trol measures are<br />
typically utilised to minimise variati<strong>on</strong> am<strong>on</strong>gst the<br />
ExpUs:<br />
– Age-matched animals (single stock, single supplier)<br />
– A small range in initial body weight is specified.<br />
– Housed in a single, envir<strong>on</strong>mentally-c<strong>on</strong>trolled room.<br />
• Temperature, humidity, photoperiod, and fresh air exchanges<br />
closely m<strong>on</strong>itored.<br />
– In some cases, cage-rack positi<strong>on</strong> is periodically rotated<br />
within the animal room to further minimise potential<br />
envir<strong>on</strong>mental variability.<br />
19
2. Secti<strong>on</strong><br />
summary/discussi<strong>on</strong><br />
• EuropaBio Members seek agreement<br />
from EFSA that:<br />
– Maintaining the current practice of<br />
physically separating males and females is<br />
acceptable.<br />
– The absence of obvious blocking factors is<br />
sufficient justificati<strong>on</strong> for the use of a CRD.<br />
20
3. Group or single housing<br />
• EFSA’s Scientific Opini<strong>on</strong> document states that “In accordance with<br />
the European Directive 2010/063, the test animals should be housed<br />
socially”, and recommends that animals of the same sex are housed in<br />
pairs.<br />
• EuropaBio Members understand this requirement for studies<br />
c<strong>on</strong>ducted within the EU.<br />
• For studies performed in the U.S., housing animals singly is acceptable<br />
from an animal welfare perspective and is comm<strong>on</strong>place.<br />
• EuropaBio Members believe group and single housing are equally valid<br />
and should remain opti<strong>on</strong>s.<br />
– Existing historical c<strong>on</strong>trol data based <strong>on</strong> single housing remains relevant.<br />
– It acknowledges different animal welfare requirements exist in testing<br />
locati<strong>on</strong>s utilized for whole food toxicity testing.<br />
21
3. Group or single housing<br />
(c<strong>on</strong>tinued)<br />
• In cases where animals are housed two (or more)<br />
per cage, the guidance makes clear that cage<br />
should be regarded as the ExpU.<br />
• The guidance then explains how it is possible to<br />
test for cage effects and, in cases where the<br />
between-cage variati<strong>on</strong> is not significantly greater<br />
than the between-animal variati<strong>on</strong>, the statistical<br />
analysis can be based <strong>on</strong> individual animals.<br />
22
3. Group or single housing<br />
(c<strong>on</strong>tinued)<br />
• While the analysis can be d<strong>on</strong>e in this way, the majority of<br />
EuropaBio statisticians believe it is more appropriate to<br />
c<strong>on</strong>duct tests of main effects at the cage level regardless of<br />
whether the cage effect is significant or not (in which case<br />
testing the significance of the cage effect is unnecessary).<br />
• EuropaBio members seek assurance that the statement<br />
“statistical analysis can be based <strong>on</strong> the individual animals” is<br />
not prescriptive and that an approach in which main effects are<br />
always tested at the cage level is acceptable.<br />
23
3. Secti<strong>on</strong><br />
summary/discussi<strong>on</strong><br />
• EuropaBio members believe single or pair housing<br />
arrangements are valid for toxicology studies with<br />
whole foods and request that EFSA permit flexibility<br />
in animal housing c<strong>on</strong>diti<strong>on</strong>s c<strong>on</strong>sistent with local<br />
jurisdicti<strong>on</strong>.<br />
• EuropaBio members seek assurance that an<br />
approach in which main effects are always tested at<br />
the cage level is acceptable (in which case testing<br />
the significance of the cage effect would be<br />
unnecessary).<br />
24
4. Statistical power<br />
investigati<strong>on</strong><br />
• EuropaBio Members have carried out an extensive<br />
power investigati<strong>on</strong> based <strong>on</strong> existing <str<strong>on</strong>g>90</str<strong>on</strong>g>-<str<strong>on</strong>g>day</str<strong>on</strong>g> feeding<br />
data.<br />
– Mean resp<strong>on</strong>se levels and estimates of variati<strong>on</strong> based <strong>on</strong> 13 studies<br />
for corn-based diets and 10 studies for soy-based diets.<br />
– Some studies based <strong>on</strong> <strong>on</strong>e animal per cage and others based <strong>on</strong> two<br />
or more animals per cage.<br />
• The endpoints and effect sizes used in this exercise<br />
were those discussed previously, that are widely<br />
c<strong>on</strong>sidered by experts in the field to be<br />
toxicologically-relevant.<br />
25
4. Endpoints and effect<br />
sizes<br />
Endpoint Change relative to c<strong>on</strong>trol<br />
Body weight Decrease 10%<br />
Liver weight Increase 25%<br />
Kidney weight Increase 25%<br />
Leukocyte count Decrease/increase 30%<br />
Lymphocyte count Decrease/increase 30%<br />
Cholesterol Increase 200%<br />
Blood urea nitrogen Increase 50%<br />
Creatinine Increase 50%<br />
Alkaline phosphatase Increase 200%<br />
26
4. Measures of variati<strong>on</strong><br />
• Given that effect sizes are expressed in percentage terms, the most<br />
appropriate measure of variati<strong>on</strong> is the coefficient of variati<strong>on</strong> (CV).<br />
• Cage was regarded as the ExpU; CV at the cage level is dependent<br />
<strong>on</strong> the number of rats per cage.<br />
• Evidence of differences in CV between males and females was<br />
detected.<br />
• For each of the endpoints c<strong>on</strong>sidered, CV’s for a given gender and<br />
number of rats per cage were found to be fairly c<strong>on</strong>sistent across<br />
studies.<br />
– Also, no evidence to suggest that CV differed between corn and<br />
soy-based diets.<br />
• So, for each endpoint, an average CV was generated for each<br />
gender / housing policy combinati<strong>on</strong>, to be used in subsequent<br />
power calculati<strong>on</strong>s.<br />
27
4. Average coefficients of<br />
variati<strong>on</strong><br />
Endpoint<br />
1 animal/cage<br />
Male Female<br />
2 animals/cage<br />
Male Female<br />
Body weight 9.4% 8.3% 4.9% 4.9%<br />
Liver weight 12.4% 10.3% 7.9% 6.9%<br />
Kidney weight 10.8% 10.5% 6.4% 6.5%<br />
Leukocyte count 24.3% 28.5% 16.6% 20.3%<br />
Lymphocyte count 27.5% 34.7% 18.1% 21.7%<br />
Cholesterol 22.6% 22.2% 12.9% 16.7%<br />
Blood urea nitrogen 14.0% 14.9% 10.0% 9.9%<br />
Creatinine 19.5% 16.9% 7.5% 7.5%<br />
Alkaline phosphatase 21.8% 28.0% 12.8% 20.8%<br />
28
4. Design scenarios<br />
c<strong>on</strong>sidered<br />
Three design scenarios were c<strong>on</strong>sidered:<br />
• 1 animal/cage; 3 treatment groups x 12 cages per treatment<br />
per gender = 72 animals in total<br />
– high <str<strong>on</strong>g>dose</str<strong>on</strong>g>, low <str<strong>on</strong>g>dose</str<strong>on</strong>g>, c<strong>on</strong>trol<br />
• 2 animals/cage; 3 treatment groups x 6 cages per treatment<br />
per gender = 72 animals in total<br />
– high <str<strong>on</strong>g>dose</str<strong>on</strong>g>, low <str<strong>on</strong>g>dose</str<strong>on</strong>g>, c<strong>on</strong>trol<br />
• 2 animals/cage; 4 treatment groups x 5 cages per treatment<br />
per gender = 80 animals in total<br />
– GM high incorporati<strong>on</strong> rate, n<strong>on</strong>-GM high incorporati<strong>on</strong> rate,<br />
GM low incorporati<strong>on</strong> rate, n<strong>on</strong>-GM low incorporati<strong>on</strong> rate<br />
29
4. Analysis details<br />
• For the 3-treatment group designs, power was based <strong>on</strong> two<br />
analysis scenarios:<br />
– the comparis<strong>on</strong> of individual treatments,<br />
– the comparis<strong>on</strong> between c<strong>on</strong>trol and mean of high and low <str<strong>on</strong>g>dose</str<strong>on</strong>g>s.<br />
• For the 4-treatment group design, power was based <strong>on</strong> the<br />
comparis<strong>on</strong> of treatments (i.e. GM vs n<strong>on</strong>-GM) averaged over<br />
incorporati<strong>on</strong> rates, in line with EFSA guidance.<br />
• In all cases, cage was regarded as the ExpU.<br />
• Males and females were c<strong>on</strong>sidered separately.<br />
• All results assume a two-sided alternative hypothesis.<br />
30
4. Results<br />
1 animal/cage, 3 treatment groups, 12 replicate<br />
cages/group; individual treatment comparis<strong>on</strong>s<br />
Endpoint Effect Size<br />
Predicted Power<br />
Males Females<br />
Body weight 10% 72% 82%<br />
Liver weight 25% >99% >99%<br />
Kidney weight 25% >99% >99%<br />
Leukocyte count 30% 84% 71%<br />
Lymphocyte count 30% 74% 54%<br />
Cholesterol 200% >99% >99%<br />
Blood urea nitrogen 50% >99% >99%<br />
Creatinine 50% >99% >99%<br />
Alkaline phosphatase 100% >99% >99%<br />
31
4. Results<br />
1 animal/cage, 3 treatment groups, 12 replicate<br />
cages/group; c<strong>on</strong>trol vs. mean of low and high <str<strong>on</strong>g>dose</str<strong>on</strong>g><br />
Endpoint Effect Size<br />
Predicted Power<br />
Males Females<br />
Body weight 10% 83% 91%<br />
Liver weight 25% >99% >99%<br />
Kidney weight 25% >99% >99%<br />
Leukocyte count 30% 92% 82%<br />
Lymphocyte count 30% 85% 66%<br />
Cholesterol 200% >99% >99%<br />
Blood urea nitrogen 50% >99% >99%<br />
Creatinine 50% >99% >99%<br />
Alkaline phosphatase 100% >99% >99%<br />
32
4. Results<br />
2 animals/cage, 3 treatment groups, 6 replicate<br />
cages/group; individual treatment comparis<strong>on</strong>s<br />
Endpoint Effect Size<br />
Predicted Power<br />
Males Females<br />
Body weight 10% 91% 91%<br />
Liver weight 25% >99% >99%<br />
Kidney weight 25% >99% >99%<br />
Leukocyte count 30% 83% 67%<br />
Lymphocyte count 30% 77% 61%<br />
Cholesterol 200% >99% >99%<br />
Blood urea nitrogen 50% >99% >99%<br />
Creatinine 50% >99% >99%<br />
Alkaline phosphatase 100% >99% >99%<br />
33
4. Results<br />
2 animals/cage, 3 treatment groups, 6 replicate<br />
cages/group; c<strong>on</strong>trol vs. mean of low and high <str<strong>on</strong>g>dose</str<strong>on</strong>g><br />
Endpoint Effect Size<br />
Predicted Power<br />
Males Females<br />
Body weight 10% 97% 97%<br />
Liver weight 25% >99% >99%<br />
Kidney weight 25% >99% >99%<br />
Leukocyte count 30% 92% 79%<br />
Lymphocyte count 30% 87% 73%<br />
Cholesterol 200% >99% >99%<br />
Blood urea nitrogen 50% >99% >99%<br />
Creatinine 50% >99% >99%<br />
Alkaline phosphatase 100% >99% >99%<br />
34
4. Results<br />
2 animals/cage, 4 treatment groups, 5 replicate cages/group;<br />
GM vs. n<strong>on</strong>-GM averaged over incorporati<strong>on</strong> rates<br />
Endpoint Effect Size<br />
Predicted Power<br />
Males Females<br />
Body weight 10% 99% 99%<br />
Liver weight 25% >99% >99%<br />
Kidney weight 25% >99% >99%<br />
Leukocyte count 30% 97% 87%<br />
Lymphocyte count 30% 94% 83%<br />
Cholesterol 200% >99% >99%<br />
Blood urea nitrogen 50% >99% >99%<br />
Creatinine 50% >99% >99%<br />
Alkaline phosphatase 100% >99% >99%<br />
35
4. Provisos for power<br />
analyses<br />
• Power investigati<strong>on</strong> for 3-treatment group scenario<br />
comparing c<strong>on</strong>trol with mean of high and low <str<strong>on</strong>g>dose</str<strong>on</strong>g>s<br />
assumes that, in reality, mean of low <str<strong>on</strong>g>dose</str<strong>on</strong>g> = mean<br />
of high <str<strong>on</strong>g>dose</str<strong>on</strong>g>.<br />
• Power investigati<strong>on</strong> for the 4-treatment group<br />
design assumes that there is no interacti<strong>on</strong> between<br />
treatment (i.e. GM vs. n<strong>on</strong>-GM) and incorporati<strong>on</strong><br />
rate.<br />
– Analysis of previous data indicates that this<br />
assumpti<strong>on</strong> should typically hold in practice.<br />
36
4. Should males and females<br />
be analysed together?<br />
• Physically separating males and females does not<br />
necessarily preclude analysing data from both genders<br />
together but there are good reas<strong>on</strong>s for doing so:<br />
– Analysis across genders for some variables is problematic due<br />
to well-established gender-specific differences in variance,<br />
which experience suggests cannot be readily overcome by the<br />
use of a data transformati<strong>on</strong>.<br />
– Large differences in gender means (e.g., for body and organ<br />
weights) may make results of a combined analysis more<br />
complicated to interpret (Gad, 2006).<br />
– Analysis across genders would be more complex and could be<br />
difficult for some CRO’s to implement correctly.<br />
37
4. Should males and females<br />
be analysed together?<br />
• Reas<strong>on</strong>s for analysing genders separately (c<strong>on</strong>tinued):<br />
– Analysis by gender is the standard approach in toxicology.<br />
– Certain endpoints (e.g., testes or uterus weights) are limited to<br />
<strong>on</strong>e gender and thus necessitate separate analyses by default.<br />
– Results from the power investigati<strong>on</strong> show that powers based<br />
<strong>on</strong> gender-specific analyses are generally more than adequate<br />
for the designated purpose.<br />
• EuropaBio Members therefore recommend analysing<br />
genders separately.<br />
38
4. Secti<strong>on</strong><br />
summary/discussi<strong>on</strong><br />
• The power profile as a whole is very positive for each of the<br />
three proposed design scenarios.<br />
– Each scenario strikes a reas<strong>on</strong>able balance between<br />
power and ethical c<strong>on</strong>siderati<strong>on</strong>s.<br />
• Findings dem<strong>on</strong>strate that the power inherent in previous<br />
studies of this size (i.e., 10-12 animals per treatment group<br />
per gender, in line with OECD TG 408) should in general be<br />
similarly adequate.<br />
• While power is important, the toxicological relevance of all<br />
observed differences are discussed in any case regardless of<br />
statistical significance.<br />
39
Part 2: Study analysis and<br />
interpretati<strong>on</strong><br />
40
Challenges in study analysis<br />
and interpretati<strong>on</strong><br />
1. Toxicological interpretati<strong>on</strong><br />
2. Standardised effect sizes (SES)<br />
3. Statistical Analysis Plan & summary<br />
statistics and raw analysis outputs<br />
41
1. Toxicological<br />
interpretati<strong>on</strong><br />
• EuropaBio Members appreciate that EFSA has commented <strong>on</strong> the<br />
imperfect relati<strong>on</strong>ship between statistical significance and<br />
toxicological relevance in the final versi<strong>on</strong> of the guidance.<br />
• However, without feedback from EFSA regarding the flexibility that<br />
will be permitted in overall design, some c<strong>on</strong>cerns about the<br />
prominence of the statistical analysis persist.<br />
• EFSA (2011b) recently stated, “Many researchers incorrectly c<strong>on</strong>clude<br />
that any statistically significant effect is biologically relevant...”<br />
– This gives EuropaBio Members c<strong>on</strong>fidence that EFSA c<strong>on</strong>siders statistical analysis<br />
<strong>on</strong>e tool of many to be used to identify toxicologically-relevant differences.<br />
• The scientific training of toxicologists, and their experience in<br />
interpreting data, must ultimately determine the relati<strong>on</strong>ship<br />
between the finding and treatment with the test substance.<br />
42
2. Presentati<strong>on</strong> of results <strong>on</strong><br />
the standardised scale<br />
• Given that all endpoints are to be analysed and presented <strong>on</strong><br />
the scale of original units, EuropaBio Members questi<strong>on</strong> the<br />
need to also present results <strong>on</strong> the standardised scale.<br />
– “If the original units of measurement are meaningful, the presentati<strong>on</strong><br />
of unstandardised effect statistics is preferable over those of<br />
standardised effect statistics”. (Nakagawa and Cuthill, 2007)<br />
• Original units are meaningful to trained toxicologists.<br />
– “If researchers are familiar with their study systems, or abundant<br />
previous research <strong>on</strong> a topic of interest exists, effect sizes in original<br />
units are more readily interpretable than standardised effect statistics”.<br />
(Nakagawa and Cuthill, 2007)<br />
• The study system is familiar to trained toxicologists and the basis of<br />
decades of previous research.<br />
43
2. Presentati<strong>on</strong> of results <strong>on</strong><br />
the standardised scale<br />
• When results are presented <strong>on</strong> the standardised<br />
scale all studies of the same design will appear to<br />
be equally sensitive regardless of any differences in<br />
underlying variability.<br />
• The guidance states that “Changes in endpoints<br />
…must also be evaluated with respect to normal<br />
biological variati<strong>on</strong> of the endpoints”; this can be<br />
d<strong>on</strong>e <strong>on</strong> the scale of original units but not <strong>on</strong> the<br />
standardised scale.<br />
44
Original Scale Standardised Scale<br />
Lab 1 Lab 2 Lab 1 Lab 2<br />
Mean C<strong>on</strong>trol 9.77 9.77 -1.45 -0.59<br />
Mean GM low 12.44 12.44 0.84 0.34<br />
Mean GM high 12.17 12.17 0.61 0.25<br />
SD 1.17 2.85 1.00 1.00<br />
CV 10% 25% NA NA<br />
p-value GM high vs C<strong>on</strong>trol 0.003 0.165 0.003 0.165<br />
GM high - C<strong>on</strong>trol 2.40 2.40 2.06 0.84<br />
95% CI<br />
GM high - C<strong>on</strong>trol<br />
0.97 - 3.83 -1.11 - 5.92 0.83 - 3.29 -0.39 - 2.07<br />
Width of 95% CI 2.86 7.03 2.46 2.46<br />
Comparis<strong>on</strong> with historical<br />
ranges possible?<br />
2. Example of original vs.<br />
standardised scaling<br />
yes yes no no<br />
45
2. Secti<strong>on</strong><br />
summary/discussi<strong>on</strong><br />
• EuropaBio Members see the presentati<strong>on</strong> of<br />
results <strong>on</strong> the standardised scale as<br />
providing no additi<strong>on</strong>al benefit, and<br />
respectfully request that this aspect of the<br />
guidance be relaxed from mandatory to<br />
opti<strong>on</strong>al.<br />
46
3. Statistical Analysis<br />
Plan<br />
• From the Scientific Opini<strong>on</strong> document it is unclear<br />
whether it is mandatory to provide a separate SAP<br />
in additi<strong>on</strong> to the details c<strong>on</strong>cerning the proposed<br />
statistical analysis that are included in the study<br />
protocol.<br />
• EuropaBio Members wish to avoid unnecessary<br />
duplicati<strong>on</strong>, and believe that the level of detail that<br />
will be provided in the study protocol should<br />
suffice.<br />
47
3. Summary statistics and<br />
raw analysis outputs<br />
• EuropaBio Members’ understanding is that EFSA wishes to see<br />
results presented both in terms of means and p-values, and<br />
differences and 95% c<strong>on</strong>fidence intervals.<br />
• However, some requirements are less clear, and others (e.g., the<br />
median) seem to be of questi<strong>on</strong>able value in most cases.<br />
• It is the opini<strong>on</strong> of EuropaBio Members that the following<br />
summary statistics would generally be appropriate:<br />
– Mean, min, max and SD for each treatment group.<br />
– P-values for all fixed effects and specific c<strong>on</strong>trasts.<br />
– Point estimate of the difference between GM and n<strong>on</strong>-GM together with<br />
95% c<strong>on</strong>fidence interval for the difference.<br />
• EuropaBio Members seek c<strong>on</strong>firmati<strong>on</strong> from EFSA as to whether<br />
the above list is acceptable.<br />
48
3. Summary statistics and<br />
raw analysis outputs<br />
• With regard to the provisi<strong>on</strong> of raw data in electr<strong>on</strong>ic<br />
format, the guidance document is inc<strong>on</strong>sistent<br />
regarding whether this is a default requirement or “<strong>on</strong><br />
request”.<br />
– Please clarify.<br />
• The guidance document states that “The statistical<br />
analysis programs, logs and outputs should be<br />
provided for the purposes of review”.<br />
– To provide all this informati<strong>on</strong> by default seems excessive.<br />
– Please clarify whether this is a default requirement or “<strong>on</strong><br />
request”.<br />
49
3. Secti<strong>on</strong><br />
summary/discussi<strong>on</strong><br />
• Please clarify if a separate Statistical Analysis Plan is<br />
mandatory or if the level of detail that can be<br />
presented in the study protocol is adequate.<br />
• Does EFSA regard the summary statistics proposed by<br />
EuropaBio Members as suitable?<br />
• Please clarify whether raw data in electr<strong>on</strong>ic format is<br />
a default requirement or “<strong>on</strong> request”.<br />
• Please clarify whether the statistical analysis<br />
programs, logs and outputs are a default requirement<br />
or “<strong>on</strong> request”.<br />
50
References<br />
• Codex Alimentarius. 2009. Foods derived from modern biotechnology. Codex<br />
Alimentarius Commissi<strong>on</strong>, Joint FAO/WHO Food Standards Programme, Food and<br />
Agriculture Organizati<strong>on</strong> of the United Nati<strong>on</strong>s: Rome, Italy.<br />
• Dorato, M.A., McMillian, C.L., Vodicnik, M.J. “The Toxicologic Assessment of<br />
Pharmaceutical and Biotechnology Products”. Principles and Methods of<br />
Toxicology. Ed. A.W. Hayes. New York: Informa Healthcare USA, Inc., 2008. pp 325-<br />
368.<br />
• EFSA. 2008. Safety and nutriti<strong>on</strong>al assessment of GM plants and derived food and<br />
feed: The role of animal feeding trials. Food and Chemical Toxicology 46:S2-S70.<br />
• EFSA. 2011a. Guidance for risk assessment of food and feed from genetically<br />
modified plants. EFSA Journal 9:2150.<br />
• EFSA. 2011b. Scientific opini<strong>on</strong>: Statistical significance and biological relevance.<br />
EFSA Journal 9:2372.<br />
51
References<br />
• EFSA. 2011c. Guidance of EFSA. Use of the EFSA Comprehensive European Food<br />
C<strong>on</strong>sumpti<strong>on</strong> Database in Exposure Assessment. EFSA Journal 2011;9:2097.<br />
Supplemental informati<strong>on</strong>:<br />
http://www.efsa.europa.eu/en/datexfooddb/datexfooddbspecificdata.htm.<br />
• EPA. 2002. Health effects Divisi<strong>on</strong> (HED) Guidance Document. # G0201.<br />
Hepatocellular Hypertrophy. October 21, 2002.<br />
• EPA. 2003. Health effects Divisi<strong>on</strong> (HED) Interim Guidance Document. # G2003.02.<br />
Rodent Carcinogenicity <str<strong>on</strong>g>Studies</str<strong>on</strong>g>: Dose selecti<strong>on</strong> and evaluati<strong>on</strong>. July 1, 2003.<br />
• Foster, P.M.D. 2002. EPA. 23 July 2002.<br />
<br />
• Gad, S. 2006. Statistics and Experimental Design for Toxicologists and<br />
Pharmacologists. Boca Rat<strong>on</strong>: Taylor & Francis Group.<br />
• Nakagawa, S., and I.C. Cuthill. 2007. Effect size, c<strong>on</strong>fidence interval and statistical<br />
significance: a practical guide for biologists. Biol Rev Camb Philos Soc 82:591-605.<br />
52
References<br />
• OECD. 1998. Guideline for the Testing of Chemicals – <str<strong>on</strong>g>Repeated</str<strong>on</strong>g> Dose <str<strong>on</strong>g>90</str<strong>on</strong>g>-<str<strong>on</strong>g>day</str<strong>on</strong>g> <str<strong>on</strong>g>Oral</str<strong>on</strong>g><br />
<str<strong>on</strong>g>Toxicity</str<strong>on</strong>g> Study in Rodents, 408.<br />
<br />
• OECD. 2012. Guidance Document 116 <strong>on</strong> the C<strong>on</strong>duct and Design of Chr<strong>on</strong>ic<br />
<str<strong>on</strong>g>Toxicity</str<strong>on</strong>g> and Carcinogenicity <str<strong>on</strong>g>Studies</str<strong>on</strong>g>, Supporting Test Guidelines 451, 452 and453<br />
2nd Editi<strong>on</strong>.<br />
• Rhomberg, L.R., K. Baetcke, J. Blancato, J. Bus, S. Cohen, R. C<strong>on</strong>olly, R. Dixit, J.<br />
Doe, K. Ekelman, P. Fenner-Crisp, P. Harvey, D. Hattis, A. Jacobs, D. Jacobs<strong>on</strong>-Kram,<br />
T. Lewandowski, R. Liteplo, O. Pelk<strong>on</strong>en, J. Rice, D. Somers, A. Turturro, W. West,<br />
and S. Olin. 2007. Issues in the design and interpretati<strong>on</strong> of chr<strong>on</strong>ic toxicity and<br />
carcinogenicity studies in rodents: approaches to <str<strong>on</strong>g>dose</str<strong>on</strong>g> selecti<strong>on</strong>. Crit Rev Toxicol<br />
37:729-837.<br />
• S<strong>on</strong>tag, J.M., Page, N.P., and Safiotti, V. 1976. Guidelines for Carcinogen Bioassays<br />
in Small Rodents, DHHS Publ. (NIH) 76-801. Nati<strong>on</strong>al Cancer Institute, Bethesda,<br />
MD.<br />
53
References<br />
• WHO. 2003. GEMS/FOOD Regi<strong>on</strong>al Diets: Regi<strong>on</strong>al per Capita C<strong>on</strong>sumpti<strong>on</strong> Of<br />
Raw and Semi-processed Agricultural Commodities. Revisi<strong>on</strong> September 2003.<br />
54
C<strong>on</strong>tributing EuropaBio<br />
• Bayer CropScience<br />
• BASF<br />
• Dow AgroSciences<br />
• DuP<strong>on</strong>t Pi<strong>on</strong>eer<br />
• M<strong>on</strong>santo<br />
• Syngenta<br />
Members<br />
55