BY NOW, YOU MAY have heard that eating genetically modified (GM) corn causes cancer. That alarming news is based on research published on 20 September, by Gilles-Eric Séralini and colleagues at the University of Caen, in France, and published in the Journal of Food and Chemical Toxicology. What you might not have heard is that last week two European food safety authorities dismissed the findings as scientifically unsound.
“The authors’ main statements are not sufficiently corroborated by experimental evidence, due to deficiencies in the study design and in the presentation and interpretation of the study results,” declared Germany’s Federal Institute for Risk Assessment, BfR. “EFSA is presently unable to regard the authors’ conclusions as scientifically sound,” echoed the European Food Safety Authority.
Since the original paper was published, some 300 scientists have demanded to know how such shoddy science could pass the peer review process. “It is appalling that such work should appear in a respected Elsevier Journal. … we would also like a comment from you as editor-in-chief as to how this paper passed peer-review successfully,” wrote Maurice Moloney, director of the prestigious UK agricultural institute, Rothamsted Research, to the editor-in-chief of the Journal of Food and Chemical Toxicology.
The indefensible conclusions of this article will no doubt inform public debate for months, perhaps years to come. They have led to a ban on importing GM corn by the Russians and the French Prime Minister Jean-Marc Ayrault announced he might try to get GM corn banned throughout the European Union. They have also reinvigorated an anti-GM lobby languishing from lack of evidence – in 2010 the European Commission concluded “that GMOs, are not per se more risky than [for example] conventional plant breeding technologies,” and in 2012 the World Health Organisation states that assessments of GMOs on the market, “are thorough, they have not indicated any risk to human health.”
Such is the modern day news cycle. Breaking stories plastered on the front of newspapers and electronic screens get the limelight; the measured response languishes in the shade.
But this recent case was an extreme example. The authors manipulated the media cycle for all it was worth.
To recap, Séralini’s paper reported that, contrary to hundreds of other studies, GM corn engineered to resist the herbicide Round-Up (chemically known as glyphosate) and Round-Up itself, cause cancer and organ failure in rats. The reason other studies had not seen this effect, the authors explained, was that standard rat studies last for 90 days but this study tracked the entire two year lifespan. This is simply not true, since the very same journal earlier that year had published a review of 12 long-term studies, between 90 days and two years, that showed no harm from GM corn or other crops in aged rats.
NEVERTHELESS, NEWSPAPERS, websites and TV screens ran pictures of rats with tumours bulging out of their chests and headlines around the world blared with variations on the theme articulated by France’s Le Monde: “New Study Says Monsanto GM Corn Causes Tumours, Kidney and Liver Disease”.
The findings appeared credible. The journal was a respectable one. The scientists were credentialed academics and had at least one scientific champion whose name was not among the list of authors on the paper: Michael Antoniou, a molecular biologist at Kings College, London. But he was hardly an “independent voice.” He is a member of the scientific council of the institute that funded the work, CRIIGEN, and had helped in the writing of the manuscript.
Séralini’s paper was a whopping media success, not only provoking politicians to react but it also served as great PR for a book and movie Séralini launched the same week: Tous Cobayes? (All of Us Guinea-Pigs Now?).
It was a sad day for science. And a sadder day for the media. In the rush for this story – in the age of online media, speed is everything – journalists yielded to some very overt manipulation. Only those who signed a confidentiality clause agreeing not to show the paper to independent experts were allowed a preview. Journalists typically receive alerts about newsworthy science publications a week or so in advance to allow them to prepare their stories; an embargo on the story prior to the publication date is standard. But a clause barring the gathering of independent opinions is extraordinary.
What it meant was that Séralini’s story, when it broke, got to prance unfettered in the media limelight before second opinions could dull its shine. By the time the storm of criticism blew in, the media limelight had moved on.
Take this example. Melbourne radio journalist, Jon Faine, renowned for his hard-hitting style, nevertheless gave Scott Kinnear (who was privy to the paper) the stage to trumpet the findings of Séralini’s paper. Kinnear is the Director of the Safe Food Foundation and the owner of Organic Wholefoods, according to the Internet “Melbourne’s best organic grocery stores in two inner city locations, Brunswick and Fitzroy”. So a hard-hitting radio jock called in a non-scientist with glaringly obvious conflicts of interest, to comment on an alarming new report on GM food?
When I shot Faine a querulous email he acknowledged, “Quite right, we got thrown a bone at short notice and took it and ran with it”. When I suggested he redress matters by interviewing a scientist with expertise in this area, his response was, “We have moved along, as the daily diet changes fast.”
Faine and many others that day broke a basic rule of journalism. Independent expert opinions mean the difference between being used as somebody’s megaphone or trying to provide a fair take on a story. The storm of criticism came but mostly it rained onto blog sites.It was largely absent from the front pages of newspapers, radio or TV.
What it showed was that the science reported in the Journal of Food and Chemical Toxicology was almost unbelievably shoddy.
If you want to see if a food or a chemical causes cancer in rats, you compare rats who have been fed the ingredient with rats who haven’t. But cancer is very common in rats, especially aged rats of the Harlan Sprague Dawley strain, which were used in this study: 74% of males and 89% of females will get cancer by the time they reach the end of their life.
With a background rate like that, you need a robust statistical analysis to make sure you’re seeing a real signal above the noise. Séralini’s paper did not provide that. Nor could it: the experiment seems to have been designed to make the analysis problematic. For one thing, he used only 20 control rats compared to 180 treated — a skewing that would make it more likely he’d pick up the tumours in the treated group. “The control group is inadequate to make any deduction. Until you know the degree of variation in 90 or 180 control rodents these results are of no value,” explained Anthony Trewavas, professor of cell biology, at the University of Edinburgh, in Scotland. David Spiegelhalter, a statistician at the University of Cambridge, in England, and Winton professor for the public understanding of risk, offered, “Frankly, the paper makes a great case study for teaching how not to design, analyse and report a scientific experiment.”
See for yourself:
200 rats were divided into 10 groups; 10 males and 10 females in each group. One group were fed ordinary rat food and served as the baseline ‘control’. Three groups were given food comprised of a percentage of the GM corn that resists Round-Up (it’s called NK603): 11%, 22% or 33%. Three groups were fed the same percentages of GM corn that had been grown in the presence of Round Up. Another three groups were dosed with Round-Up alone by adding it to their water.
A high school student would know what to do next: apply a statistical test to look for differences in the treated groups and the controls. Of course, since the background cancer rate is so high, you would also want to be reassured that the effect is real by seeing that higher doses of GM corn produce higher rates of cancer.
But the authors did not do what many of their peers consider to be an appropriate statistical test.
“There does not appear to be a statistical analysis of the mammary tumours,” noted Alan Boobis, professor of biochemical pharmacology, Imperial College London. “The statistical methods are unconventional, there is no clearly defined data analysis plan and probabilities are not adjusted for multiple comparisons,” said Tom Sanders, head of the Nutritional Sciences Research Division, King’s College London.
IN LIEU OF A ROBUST statistical analysis, Séralini seems to have cherry picked the data. The females fed 11% GM corn showed a 30% increase in the number of tumours. But the females fed a 33% s dose showed a decreased number of tumours! And the males showed no differences in tumours between treated and controls.
The authors concluded that there was a “threshold” effect seen with a low level of GM corn that disappeared at high doses. They could have equally concluded that GM corn at 33% of the diet protects you from cancer! And the fact that there was no effect seen in males? GM corn was interacting with the female hormones.
The more plausible explanation? “To be frank it looks like random variation to me in a rodent line likely to develop tumours anyway,” offered Trewavas.
The whole affair raises some acute questions. For starters, why did the authors design such a shoddy experiment? “There is so much wrong with the experimental design that the conclusion is inescapable that the investigators intended to get a spurious, preordained result,” wrote scientists Henry Miller and Bruce Chassy.
Séralini has a history of trying very hard to prove that GM corn is unsafe.
In 2007, he reanalysed the data on Monsanto’s rat feeding studies for another variety of GM corn known as Mon 863. This variety carries an inbuilt bacterial protein (Bt) that protects corn from insects like corn borers. Contrary to Monsanto’s findings, Séralini’s reanalysis concluded that Mon 863 made the rats ill. In 2007 the European Food Safety Authority investigated those claims, and found them to be spurious.
This time round, Séralini produced his own data but in another bizarre twist, he withheld much of it from the paper because he claimed “All data cannot be shown in one report.” Yet publications routinely carry a “supplementary section” that is just a hyperlink away. Now some 300 scientists have signed a petition demanding to see that data.
“I think that I speak for the vast majority of the biological sciences community in saying that this paper is desperately in need of a second review, by a world-class toxicologist and a world-class statistician. We would like to see all the data, especially the carefully hidden controls,” wrote Moloney to the editor-in-chief of the Journal of Food and Chemical Toxicology, A. Wallace Hayes.
As a science writer, I am aghast. I rely on peer review to do its job as a first pass filter. Is it broken? I asked Rick Roush, the dean of the Melbourne School of land and Environment at the University of Melbourne. “No”, he told me. “Sloppy science always gets through. That’s why you can’t rely on any one paper.”
And certainly never any one scientist, no matter how credentialed.
Why did the scientists design an experiment that produced such indefensible results? Why did the journal publish it? Both have sacrificed their credibility. But they certainly raised their profiles. As Roush points out the journal’s “impact factor” – a metric used to assess the quality of a journal – is likely to soar because everyone has been quoting the paper.
While Séralini and his colleagues, who are no doubt true believers in their cause, have achieved their goal of creating maximum alarm among the masses, most of whom don’t read beyond the first day headlines.
There’s no doubt GM varieties could theoretically have adverse health effects. That’s why there is thorough testing. Experimental varieties that hint of risk, don’t make it onto the market – like a soya bean that carried a Brazil nut protein with the potential to cause allergies. But traditionally bred varieties also carry risk. Think of the organic sprouts that caused 53 deaths in Germany and Europe last year.
We rely on good scientific analysis to assess risk. Séralini’s paper was a far cry from that. The Séralini affair was a sad episode for science and the media, and reminds me that writing about science is no walk in the park and ever more challenging in today’s frenetic media circus.
Press release from BfR, Germany’s Federal Institute for Risk Assessment
EFSA report on Séralini paper