Even good science can sometimes be bad science

Let’s take another delve into the realms of the hacktitioner, and today’s subject is food additives and hyperactivity in children as reported by Sean Poulter of the Daily Mail – although to be fair to Sean, he’s billed as the newspaper’s  consumer affairs correspondent and not as a health correspondent so we need to moderate our expectations accordingly.

And so, to the article, which sports the title ‘Additives DO harm children – and a ban could cut child hyperactivity by a third, say scientists.’

The number of hyperactive children could be cut by a third by banning suspect food additives, it is claimed today.

Actually, this was claimed more than six months ago when the research paper in question was published in The Lancet, at which time it was covered by the Guardian’s Felicity Lawrence, another consumer affairs correspondent, but what the hell, this made the front page of the print edition of the Mail so it must be a slow news day over there.

The finding by British scientists will put pressure on the Food Standards Agency to force manufacturers to stop using the “E-number” chemicals.

Maybe. Maybe not. Let’s actually see what the research says before we jump to any conclusions.

The researchers believe that removing artificial colours from children’s foods, including cakes, drinks and sweets, would bring significant health and social benefits.

Ah… the researchers believe… so they’ve seeming found something of interest but what they’ve found is short of clear proof, not a shortcoming in the research itself but a sign that we need to be a little cautious about the possibility that the significance of the research may well be oversold at some point during this article.

Thousands of children would avoid the blight on their education caused by hyperactive behaviour, which can mean they are labelled slow and disruptive.

Removing the chemicals could also help reduce anti-social behaviour in teenagers, according to the researchers from the University of Southampton, led by Professor Jim Stevenson.

That all seems fair enough if the researchers have established a connection between the consumption of food additives and behavioural problems in children, depending on how strong the connection is and how solid the research methodology and evidence turns out to be. If we ignore the Mail’s usual overblown hyperbole, it does seem that we may be on to something here.

The scientists believe the harm caused to the IQ of youngsters is equivalent to the damaging impact of lead on developing brains.

They say just as efforts were made to protect children against lead poisoning years ago, there is “justification for action now” on food colours.

So its a neurological study of some description, is it?

Because it had better be if you’re going to draw parallels between the effects of food additives and the pathological effects of lead on neurological development/function.

They are frustrated at the lack of action to tackle the harm to children posed by food additives and are calling on the board of the Food Standards Agency (FSA), which is meeting today, to take bold measures to ban them. Anything less and I’m not only going to be a little disappointed but I’ll also be duty bound to point out that you’re making a bit of an apples and oranges comparison here, and the reference to what the researchers believe rather than what they can demonstrate or evidence doesn’t fill me with the greatest optimism.

The Southampton team calculates that some 6.6 per cent of children aged three to 12, a total of 462,000, suffer from Attention Deficit Hyperactivity Disorder (ADHD). The academics believe this figure could be reduced by 30 per cent – around 140,000 – if the additives were banned.

And we suddenly have a problem.

You see, if ADHD is actually a genuine disorder then it must have one or more pathological causes, causes which could be genetic in origin or they could stem biochemical imbalances in the brain, or neurological deficits and developmental problems, or they could stem from a combination of any of these factors.

Now this gives us a couple of possibilities given that the assertion being made is that removal of these additives from the diet of children could reduce the number of children diagnosed with ADHD by 30%.

One possibility is that what we have is a fairly common form of ADHD of a kind that is predicated on neurological sensitivity to these additives, which is certainly possible and likely to be genetic in origin – one such disorder that we do know about that has these characteristics is phenylketonuria (PKU), which all newborns are tested for within a day or so of birth and which can be readily controlled by dietary means, and if that is what the researchers have uncovered that this is good news indeed.
The other possibility, however, is that our 30% figure is made of children who are not just diagnosed as having ADHD, but who have actually misdiagnosed when, in fact, their problem is nothing more than the fact that consumption of these additives gives rise to behaviour patterns which mimic the effects of ADHD.

Now if its the latter, then there’s some very good news in all this, as if this research holds up then it could mean that 30% fewer kids out there at risk of being unnecessarily doped up to the eyeballs with ritalin, but there’s also some possible bad news in the sense that this will almost certainly that some doctors are going to have a hard time with those over-anxious middle-class parents who fall into the 70% who do actually have ADHD and who will now be that bit harder to convince that their precious offspring actually needs proper clinical intervention rather than a diet of brown rice and bottles of woo.

But then you can’t make an omelette without breaking a few eggs so what’s needed is just to keep a close eye on the activities of known woo merchants until we can sort out the science behind all this properly.

Professor Stevenson and his team discovered that food chemicals caused “psychological harm” to normal healthy children.

Really? Let’s see – ‘psychological harm’ is presented in quotes so, at best, what’s being reported here is a matter of opinion rather than fact, which means that we’re no closer to understanding what kind of mechanism we’re actually dealing with here; a neurological one, in which case we’d expect evidence of persistent changes in neural biochemistry, or a behavioural mechanism, in which the vast bulk of any potential harm resides in the manner in which these children are treated by others as a consequence of their additive induced behavioural problems.

Two groups of children showed changes in behaviour when given the additives during controlled trials. They found it hard to sit still and concentrate, they had problems reading and became loud and impulsive.

Ah, good. Now we’re getting to the science and it appears that what we have here is a behavioural rather than a neurological study – so the whole lead business earlier is just a bit of the usual scaremongering bollocks that infests the work of the hacktitioner – and that means its time to take a look at the actual research.

Or rather it would be, were is not stuck behind The Lancet’s pay per view firewall.

Still, never mind, we still have the abstract to play with:

Background

We undertook a randomised, double-blinded, placebo-controlled, crossover trial to test whether intake of artificial food colour and additives (AFCA) affected childhood behaviour.

Methods

153 3-year-old and 144 8/9-year-old children were included in the study. The challenge drink contained sodium benzoate and one of two AFCA mixes (A or B) or a placebo mix. The main outcome measure was a global hyperactivity aggregate (GHA), based on aggregated z-scores of observed behaviours and ratings by teachers and parents, plus, for 8/9-year-old children, a computerised test of attention. This clinical trial is registered with Current Controlled Trials (registration number ISRCTN74481308). Analysis was per protocol.

Findings

16 3-year-old children and 14 8/9-year-old children did not complete the study, for reasons unrelated to childhood behaviour. Mix A had a significantly adverse effect compared with placebo in GHA for all 3-year-old children (effect size 0·20 [95% CI 0·01–0·39], p=0·044) but not mix B versus placebo. This result persisted when analysis was restricted to 3-year-old children who consumed more than 85% of juice and had no missing data (0·32 [0·05–0·60], p=0·02). 8/9-year-old children showed a significantly adverse effect when given mix A (0·12 [0·02–0·23], p=0·023) or mix B (0·17 [0·07–0·28], p=0·001) when analysis was restricted to those children consuming at least 85% of drinks with no missing data.

Interpretation

Artificial colours or a sodium benzoate preservative (or both) in the diet result in increased hyperactivity in 3-year-old and 8/9-year-old children in the general population.

Well, I must say that despite being a tad disappointed that this isn’t a neurological study that does all look pretty good to me. The methodology pushes all the right ‘buttons’ – double blind, placebo controlled and using a crossover method – and the numbers involved in the trials seem adequate as does the GHA-based scoring method.

Without having had chance to review the full paper, this certainly looks to be rather promising. It appears to be a well-constructed trial – subject to statistical validation that the sample populations provide a good match to what would be expected in the general population, and the results look pretty good as well.

Perhaps the only real quibble here is that the additive mixes given in the study are all in the form of two additives plus the sodium benzoate preservative, which means that what we cannot say for certain is whether or not its just a particular additive or the combination of additives that’s giving rise to the effect reported in the results, which means that it is possible that one or more of the additives used in the study might be completely harmless.

But that quibble aside, this looks like a pretty good bit of science on the limited information I have to hand, and as long as this is reflected properly in the manner of its presentation (which, sadly, already looks a little doubtful) then we may have something of a rarity in the world of the hacktitioner on our hands, a story in which the science actually stands up to scrutiny.

Professor Stevenson said: “We now have clear evidence that mixtures of certain food colours can adversely influence the behaviour of children.

Fine – a perfectly reasonable assertion in the evidence.

“We know that hyperactivity in young child is a risk factor for, for example, later difficulties in school. Certainly it is associated with difficulties in learning to read.

Yeah, no problems with that either.

“It is also associated with wider behavioural difficulties in middle childhood, such as conduct disorder.

Sorry..?

Conduct disorder? WTF is ‘conduct disorder’?

Ah, right, I get you… it’s this:

Conduct disorder is a psychiatric category to describe a pattern of repetitive behaviour where the rights of others or the social norms are violated. Symptoms include verbal and physical aggression, cruel behavior toward people and pets, destructive behaviour, lying, truancy, vandalism, and stealing.

After the age of 18, a conduct disorder may develop into antisocial personality disorder.

So, for the benefit of any Daily Mail readers who happen by, its a clinical euphemism for ‘hoodie, one that magically turns in an anti-social behaviour disorder when the individual in question reaches the age of 18, something we rather euphemistically refer to as ‘criminality’.

In short, it’s a load of pseudo-clinical bollocks of the ‘It’s not my fault I vandalised that bus shelter, M’lud, I’ve got a conduct disorder’ variety.

“I feel that the effects we are seeing here are sufficiently great to represent a threat to health.”

How so?

Certainly there are risks to individual psychological well-being arising out of any failure to recognise additive-induced behavioural problems, particularly if that is then compounded by a misdiagnosis of ADHD, but otherwise the risk to actual health are pretty marginal, unless we’re going into the realms of depression, self-harming behaviour and suicide.

But to simply state that there is a ‘threat to health’ without qualifying such a statement with indications of the nature of this alleged threat is, well, rather overselling the science.

Mind you, one does have to take into account the fact that this is the Daily Mail the guy’s talking to and that our hacktitioner, who’s doing the write-up, almost certainly hasn’t bothered to look up what a contact disorder and how this might, in turn, be relevant to the phrase ‘hug a hoodie’, so maybe this overstatement is one born of a little necessary diplomacy in order to secure the coverage.

The Southampton team has sent a report to the FSA board, which argues that a significant number of children could be prevented from developing ADHD if the additives are removed.

This research was actually commissioned by the Food Standards Agency, not just sent to it – they paid for it so, of course, they received copies of the report.

This is, in all, rather a curious misstatement of the position given that the Daily Mail’s ‘Ban the Additives’ campaign began last September and that Poulter has written something like 18-20 articles on this subject, including coverage of the publication of the research in the Lancet, the earliest of which dates back to 2002.

Children who are diagnosed with ADHD can find their entire school careers and lives suffer as a result. The report warns: “Elevated levels of hyperactivity in young children represent a risk for continuing behaviour problems into later childhood.

Well yes, but not necessarily because of the direct effect of the additives as any later behavioural problems may easily stem from the psychological effects of having been misdiagnosed with ADHD and how such a diagnosis then goes on to dictate the treatment that the individual receives from others.

“It should also be recognised that children with elevated levels of hyperactivity can be disruptive to a family and are sometimes socially isolated because peers find their behaviour unsettling.”

Well yes, obviously – but then a correct diagnosis at an early stage should go a long way towards correcting such problems.

Last month the Government announced a task force to concentrate on improving the behaviour of 1,000 particularly disruptive young people.

The Southampton team say: “It is a Government policy priority to reduce the level of disruptive behaviour by young people. We suggest… the removal of food colours might be a small, indirect contribution to such a goal.”

As might making an accurate diagnosis and advising the individual and their parents accordingly – and I should say that I’m not siding with the food industry here, simply pointing out that there is more than one way to skin this particular cat and that, therefore, I’d prefer to see a little bit stronger case being advanced if the research evidence is a solid as the abstract suggests.

And, at that point, we’ll call time on the Mail’s article, the rest of which is fairly generic guff.

However, in checking the background to this, I did come across material that sheds a little more light on the strength of the evidence provided by this piece of research, information that raises a few questions about whether or not its all the Mail’s pet hacktitioner seems to think it is:

For starters, the research was covered, last September, by a US-based specialist website called Myomancy, which puts some useful flesh on the bones of the abstract provided by The Lancet:

the UK Food Standards Agency has released the results of complex and top-rate research it has been doing into food colorings. Rather than looking for an individual cause, it looked at mixtures of chemicals commonly found in drinks aimed at children. They tested two mixtures on 260 children split into two age groups, three years old and eight years old. The children including a range of ADHD symptoms from none to extreme so that the study could assess whether the chemicals increase existing symptoms or cause ADHD in those with no symptoms. The study lasted six weeks and during which the children were assessed by the parents, teachers and most importantly, a trained independent observer. The trial was a double blind study so that none of the children, parents, teachers or observers knew whether the child was receiving mixture A, mixture B or a placebo. A second stage of the study used a subset of the children and observed them under tightly controlled laboratory conditions.

All of which confirms my own impression that, at the very least, the methodology used in the study stacks up. When it comes to the results, however, things become a little less straightforward:

The results were complex. Three year olds responded with a significantly increased level of hyperactivity to mixture A whereas the eight year old responded more to mixture B. Also not all children responded the same way and the levels of response were not connected to the child’s existing levels of hyperactive behaviour.

That all sounds a little less definite, doesn’t it?

And, in fact, if one looks at the statistical data in the abstract then its clear that the observed effects in three year olds were significantly greater than those seen in eight year olds, which is actually what one would expect anyway, for a variety of reasons.

Part of the research was to see if genetic make-up played a role in how children reacted to the drinks. They found that children with genes relating to impaired histamine clearance (histamine N-methyltransferase, HNMT Thr105le and/or HNMT T939C). Children with these genes did show a significantly greater reaction to the both mixtures.

Okay, so the write up is tad garbled, but what there appears to be is some indication of a possible genetic element in this, one relating to histamine clearance any this is clearly worth following up with further neurological research – finding such a correlation is good but not quite so good as actually understanding the mechanism that gives rise to the effect.

That said, the link to impaired histamine clearance does give us a clue as to what we may need to be looking for. There are four types of histamine receptor (H1 to H4) of which the H1 type is the one that will be (indirectly) the most familiar as this is the one that anti-histamines, taken to counter hay fever and other allergies, acts upon and the reason why taken anti-histamines tend to make people drowsy – histamine plays a role in sleep regulation.

However, its the H3 receptor that is likely to be of most interest because blocking the H3 receptor not only increases histamine levels in the brain, increasing wakefulness, but it may also impact on the levels of other neurotransmitters, particularly dopamine acetylcholine and norepinephrine, acting as an inhibitory heteroreceptor. The importance of this is that deficiencies in both dopamine and norepinephrine are thought to play a part in ADHD, hence the seemingly counter-intuitive practice of prescribing an amphetamine-based  stimulant (ritalin) to counter hyperactivity (in actual fact, what ritalin does is increase the levels of norepinephrine and dopamine which, in turn, increases the individuals attention and focus, taking care of the ‘attention deficit’ component. In addition, acetylcholine plays a part in both synaptic plasticity, specifically in learning and short-term memory, and in modulating the excitability of the individual.

So, what we have is wakefulness, lack of attentions, problems with learning and short-term memory function and heightened excitability. Does that sound familiar?

The article concludes by noting:

The study has not produced a clear culprit for ADHD and the study authors admit that the results they have seen could be down to chance. However this study and previous work does indicate diet can have an effect, and sometimes a very strong effect, on some children.

Which is broadly the conclusion of the FSA in this paper (pdf), which does provide the detailed information on the research that I couldn’t access via The Lancet, and which concludes that:

45. We consider that this study has provided supporting evidence suggesting that certain mixtures of artificial food colours together with the preservative sodium benzoate are associated with an increase in hyperactivity in children from the general population. If causal, this observation may be of significance for some individual children across the range of hyperactive behaviours, but could be of more relevance for children towards the more hyperactive end of the scales.

And, therefore that:

47. We conclude that the results of this study are consistent with, and add weight to, previous published reports of behavioural changes occurring in children following consumption of particular food additives.

48. This research has not indicated any possible biological mechanism for the observations made, which might have provided evidence of causality or of the possible effects of individual additives or of other mixtures of additives.

49. The timing and duration of any possible effects would need to be addressed by further research.

Which is what you’d expect from an organisation whose primary function is that of regulating the food industry, and yet…

Let’s put up the hypothesis that what these food additives may be causing here is not ADHD itself, but a form of temporary hyperactivity which mimics the effects of full blown ADHD.

If that hypothesis is correct then, for all that this research has its limits, it may be rather more important than anyone seems to have realised because every one looking at it is caught up in the whole business of arguing over whether it does or does not justify banning a small number of food additives rather than considering what it might have to tell us about the neurological mechanisms responsible for ADHD itself.

It certainly doesn’t offer any proof in that respect – its not that kind of study – but it does seems to be pointing in a particular direction, and a direction which looks plausible enough to merit further investigation because if these additive are somehow mimicking ADHD then the biochemical mechanisms that underpin that mimicry should be similar, if not the same, as those which cause full blown ADHD.

And yet, with that possibility suggested by the research, all the Daily Mail and, seemingly, the researchers responsible for this study seem to want to do is bang on about food additives when they may have, somewhat inadvertently, hit upon the key to unlocking, if not the actual cause of ADHD, then at least the biochemical foundations of the condition.

Even good science,  and there is much that is good in the Southampton study, can incorporate a degree of bad science when researchers (and others) get so caught up in a particular hypothesis to the extent that they fail to recognise that their evidence may be pointing them towards something of even greater value and significance – and it seems possible that they may be exactly what’s happening here.