Examining the BCA’s ‘plethora’ of evidence

Via Jack of Kent, it seems that the British Chiropractic Association have finally been stung into publishing details of the ‘plethora’ of research evidence that they claim supports the use of Chiropractic manipulation for a variety of condition other than non-specific lower back pain.

Jack’s got a copy of the full document available for download but, having checked it over, it seems that the BCA have pulled out all the stops to make its critics’ life just that bit more difficult by securing the document in question in such a way as to prevent not only unauthorised amendments but also copying and pasting information from it in the course of mounting criticisms of its content…

…and so, out comes the trusted PDF password/encryption stripper and voila, we now have a copy of the same document from which it is possible to copy/paste. (If you have any problems downloading by clicking the link, try right-clicking and choosing ‘save link as’ – WP 2.8 is behaving a tad eccentrically at the moment)

(Update: I’ve managed to fix the PDF, so it should be working properly)

So, let’s get down to business and start to examine the BCA’s evidence, and for my own contribution I’d like to make one or general points and then tackle the first three research papers cited as  ‘evidence’ for the efficacy of chiropractic manipulation as a treatment for infantile colic, leaving (hopefully) the field clear for other bloggers to examine the some of the other papers cited by the BCA, both in relation to colic and other conditions.

So, general remarks first.

In total, the BCA’s research evidence runs to 29 documents and reports of which a number are largely or entirely irrelevant.

For example, the BCA cites it own code of practice and a reference in a glossary to a definition of ‘evidence based care’ as ‘‘clinical practice that incorporates the best available evidence from research, the preferences of patients and the expertise of practitioners (including the individual chiropractor him/herself)’. Research evidence is, of course, central to evidence-based medicine but this is not necessarily also true of either the ‘preferences of patients’ or even the ‘expertise of practitioners’, neither of which can be guaranteed to be founded on or derived from actual evidence.

Also joining the pile the marked ‘objection – relevancy’ are:

An article (citation 6) that discusses the conduct of medical researchers and the issue of the misrepresentation of research evidence – the BCA are nothing if not lacking in well-defined sense of irony.

Three papers (citations 8-10 inclusive) which deal with evidence for the efficacy of osteopathy rather than chiropractic.

A paper proposing a supposedly ‘rational framework’ for the care and management of excessive infant crying (citation 24).

A general paper on the nature of evidence-based medicine (citation 26).

And finally, three papers on the side effects associated with the use of nonsteroidal anti-inflammatory drugs (citations 27-29, inclusion).

So, before we’ve even got out of the starting blocks, fully one third of the ‘evidence’ cited by the BCA turns out to provide no actual evidence to support any of its claims for the efficacy of chiropractic manipulation when dealing with conditions other than back pain.

Moving swiftly on, the first three research papers cited by the BCA, Klougart et al. (1989), Mercer and Nook (1999) and Wiberg et al. (1999) were all reviewed by Stephen Hughes and Jennifer Bolton in a short paper that was published in 2002 and is available online via the BMJ’s ‘Archives of Disease in Childhood’ under the title ‘Is Chiropractic an effective treatment in infantile colic‘. (Registration required, but free of charge). The BCA do not reference this paper in its list of ‘research evidence’.

Of these three papers…

Klougart et al. is an uncontrolled prospective study that provided the first [alleged] evidence for the use of chiropractic manipulation in treating infantile colic. Although noteworthy for its size – 316 infants were recruited to the study – it lacks blinding, randomisation and a control group and is, therefore, subject to a considerable degree of bias and offers no means of assessing its findings against either a placebo or simply allowing the condition to run it natural course, colic being a condition that typically improves over time.

The BCA’s inclusion of the paper by Mercer and Nook, which claims a 93% success rate in actually curing colic, in its list of evidence is nothing short of a complete embarrassment. The paper was actually reported only in abstract and provided very little methodological information and no actual research data to support its claimed findings. It is a very small study (30 infants diagnosed with colic by a paediatrician but no information on the diagnostic criteria used) in which 15 infants were treated with chiropractic spinal manipulation and the remaining 15 were ‘treated’ with non-functional, detuned, ultrasound machine as a placebo and has a list of flaws almost as long as your arm…

No information given on dropouts RCT (level 1b). Single blinded study. Randomisation unclear. Subjective response to treatment by parents before treatment and at each subsequent consultation. Outcomes not defined. Statistically significant difference (no data given) in response to treatment between 2 groups (assumed beneficial in experimental group). Complete resolution of symptoms in 93% of infants in (assumed) experimental group. No comparative data for placebo group.

As citations go, Mercer and Nook is the next best thing to useless as a piece of evidence.

The third study, Wiberg et al. is perhaps the best designed of the three but still not without significant flaws. It is a single blinded study of 50 infants, 25 of whom were given chiropractic treatment while 25 were treated with dimethicone, a common, over the counter, colic remedy which has been shown to perform no better than a placebo. Nine of the 25 infants treated with dimethicone dropped out of the study with their parents citing a worsening of symptoms, rather than a medication bias. The big problem with this study is that the parents were fully aware of which treatment their child was receiving and the researchers failed to conduct an ‘intention to treat’ analysis prior to beginning the study, the upshot of which being that the outcomes reported are prone to a significant degree of parental bias.

The upshot of all this is that although Wiberg et al. report that the chiropractic treatment proved to be more effective than dimethicone, its impossible to say whether these result indicate that the treatment had an genuine theraputic effect or whether it merely served as a more effective placebo than the treatment administered to the control group.

Having covered the three papers cited by the BCA, it should be noted that Hughes and Bolton included a fourth paper in their review, by Olafsdottir et al. (2001), which the BCA has omitted from its list of evidence. This last study is somewhat larger than the Wiberg study (100 infants) but uses broadly the same methodology but for one very significant difference – the Olafsdottir study is double-blinded such that neither the parents or the researchers were aware of the precise treatments given to a specific infant.

Olafsdottir found no difference in the parentally reported symptom scores between spinal manipulation and placebo, hence, on would assume, the exclusion of this study from the BCA’s list of evidence.

So, from looking at just three of the colic studies put forward by the BCA its apparent that they’ve been highly selective in the choice of evidence.

This is unproblematic in the context of their current litigation against Simon Singh where, due to the adversarial nature of the courtroom, neither side is necessarily bound to advance evidence that does not support their case, but in terms of the credibility of the BCA’s claim that is adhere to the principle of evidence-based care it rather blows a hole in the reference, in their code of conduct, to using the ‘best available evidence from research’. What Hughes and Bolton concluded in their review was that the best available evidence, at the time, was that provided by Olafsdottir et al, which showed that chiropractic spinal manipulation performs no better than a placebo when used to treat infantile colic, and yet this is the one paper of the four included in that review that the BCA have failed to cite in their list of evidence.

While it would be fair to suggest that, in writing his original article, Simon Singh could have been somewhat more precise in his criticism by asserting that the BCA’s claims in regards to the treatment of colic lack credible evidence, the all-too-obvious partiality demonstrated by the BCA in its selection of ‘evidence’ does nothing whatsoever to enhance its already damaged credibility and, in my own personal opinion, fully supports Singh’s contention that the BCA happily promotes ‘bogus’ treatments.

The BCA may well believe that the three papers I’ve looked here provide evidence that supports the use of chiropractic spinal manipulation as a treatment for colic but belief alone is not proof that such treatments are not, in reality, entirely bogus and lacking in any theraputic value over and above that provided by a simple placebo.

So, that’s my bit over and done with for the moment… I wonder who else is up for picking apart the BCA’s list of ‘evidence’?

UPDATE

Jack of Kent has now put up a commentary on the covering statement that accompanies the BCA’s list of evidence, which is well worth a read…

AND SO IT BEGINS…

The Lay Scientist has a rather more comprehensive look at the amount of irrelevant material the BCA has included in its ‘plethora’ of evidence, while Prof David Colquhoun has also given the BCA’s colic papers the once over.