The Faculty of Homeopathy have a page summarising the evidence on homeopathy. Astoundingly, it finds that homeopathy is effective for some conditions. It has been noticed that their analysis of the evidence is, to put it charitably, upbeat.
I thought I’d have a quick look at the page on randomised controlled trials. The results were predictable, but it’s actually quite remarkable how open they were about the poor quality of evidence overall. They have had their pick of the cherries, and most of them turn out to have maggots in them.
Seasonal allergic rhinitis
The studies are:
1 Aabel S, Laerum E, Dølvik S, Djupesland P (2000). Is homeopathic ‘immunotherapy’ effective? A double-blind, placebo-controlled trial with the isopathic remedy Betula 30c for patients with birch pollen allergy. British Homeopathic Journal; 89:161–168.
Flagged Green, meaning strongly positive.
No statistically significant difference between the groups was found during the first and last period of May. However, from 8 to 18 May, a clinically interesting difference was revealed between the groups, those receiving Betula 30c having fewer and less serious symptoms. For some days these differences were statistically significant.
This is classic P-hacking. Note the small sample size: “32 patients received Betula 30c tablets and 34 patients received placebo tablets”. It is unsurprising that with P=0.05, a one in twenty chance of a positive outcome by random chance, there were some days when one group experienced less symptoms.
Another red flag is the journal: British Journal of Homeopathy. This is ideologically favourable to homeopathy and peer-reviewers may be unlikely to challenge findings that are ideologically consonant, even if they are considered wildly implausible by non-believers.
Overall this is a small study with a small effect that is not uniform; there was some reported relief but also more reported “aggravations” (i.e. side-effects). It is at best very weak evidence, and I would argue it is in fact equivocal.
2 Aabel S (2000). No beneficial effect of isopathic prophylactic treatment for birch pollen allergy during a low-pollen season: a double-blind, placebo-controlled clinical trial of homeopathic Betula 30c. British Homeopathic Journal; 89:169–173.
Flagged red, meaning negative.
This is a second study in the same issue of the same journal by the same lead author, testing the same remedy on the same condition in the same place. The trial was conducted the year following the earlier study and involved 73 children.
The pollen count was very low in 1996, only three days were high enough to provoke allergic symptoms. Surprisingly, the verum treated patients fared worse than the placebo group; they used more rescue medication and had higher symptom scores during these three days. Homeopaths might attribute the findings to a putative aggravation response, but the results certainly do not lend support to the usefulness of the tested prophylactic approach, under conditions of low allergen exposure.
Excuses aside, the finding is unambiguously negative, and combined with the weak finding of the previous study certainly does not support any therapeutic effect.
3 Aabel S (2001). Prophylactic and acute treatment with the homeopathic medicine Betula 30c for birch pollen allergy: a double-blind, randomized, placebo-controlled study of consistency of VAS responses. British Homeopathic Journal; 90:73–78.
Flagged amber, meaning equivocal.
The patient groups that received either placebo or Betula 30c for two successive years showed a consistent response (r=0.75, P=0.01 and r=0.70, P=0.003, respectively). No such correlation was found in the two groups that changed remedy from one year to another (either from placebo to Betula or vice versa).
An unambiguously negative result, again from a small group (51 patients) but slightly more robust by virtue of running across two years with partial crossover. This is also the conclusion of the NHMRC review.
Taken together, the three Aabel papers are unambiguously negative: they provide no credible evidence that the selected remedy has any therapeutically useful effect.
4 Kim LS, Riedlinger JE, Baldwin CM, Hilli L, Khalsa SV, Messer SA, Waters RF (2005). Treatment of seasonal allergic rhinitis using homeopathic preparation of common allergens in the southwest region of the US: a randomized, controlled clinical trial. Annals of Pharmacotherapy; 39:617–624.
Flagged green, meaning positive.
This is another small study (40 patients) and the authors are associated with Southwest College of Naturopathic Medicine & Health Sciences, Tempe – naturopaths in the USA especially have a vested interest in the validity of homeopathy.
These preliminary findings indicate potential benefits of the homeopathic intervention in reducing symptoms and improving quality of life in patients with seasonal allergic rhinitis in the Southwestern US.
A published critique in the original journal, which therefore should form part of any diligent appraisal of the paper, notes: “The particular form of homeopathy used in The Annals‘ study is called isopathy. This differs considerably from classical homeopathy, and some homeopaths do not recognize the approach as constituting homeopathy”.
One of the signature features of sectarian medicine is the use of practices on which practitioners disagree, sometimes violently, but cannot, in the absence of any objective standards or tests, settle that disagreement. They all agree, of course, that the fundamental tenets of their sect, are ineffably true.
Not only is the remedy under test isopathic, it is also nowhere near normal homeopathic levels of dilution. The usual dilution is 30C, essentially 1 × 10−60 mg/mL. Kim et. al. use 1 × 10−6 mg/mL – a dose which is likely to be pharmacologically active and likely also to be bioavailable at some objectively testable level.
There are also specific technical problems with the statistical analysis, which invalidate the conclusions reached by the authors.
The concluding paragraph of the critique is damning:
Kim et al. present us with another small trial with somewhat significant results. Technical questions about the appropriateness of the statistical analysis aside, more practical questions must be raised about the clinical significance of the results. The biggest question is, just what did the researchers test? The nasal administration of allergens for only 4 weeks is not conventional immunotherapy.12 The dose used was in the same range as that used parenterally in immunotherapy, which is immensely higher than that used in previous isopathy studies and takes the remedy out of clearly homeopathic concentrations. Classical homeopaths will also question the lack of detailed symptom collection and treatment not being individualized. Whatever caused the alleged benefits reported here should not be called homeopathy.
Overall, this study is of no value and should have been discounted as failing to meet minimal quality standards.
h/t Alison Gaylard.
5 Reilly DT, Taylor MA, McSharry C, Aitchison T (1986). Is homeopathy a placebo response? Controlled trial of homeopathic potency, with pollen in hayfever as model. Lancet; ii:881–885.
Flagged green, meaning positive.
Another small study (51 enrolled, 50 completed). Two main outcome measures were reported, one having no significant difference. Combining multiple endpoints increases the chances of a positive outcome purely by chance. Another red flag is citing Linde et. al. (1997) and not the rather important 1999 followup, “Impact of study quality on outcome in placebo-controlled trials of homeopathy“.
The authors declare no competing interests, but the lead author, David Taylor Reilly, headed the Glasgow Homeopathic Hospital and has a clear vested interest in proving the validity of his beliefs.
This study is weakly positive but of low power and with significant red flags.
h/t Alison Gaylard.
6 Wiesenauer M, Gaus W, Häussler S (1990). Behandlung der Pollinosis mit Galphimia glauca. Eine Doppelblindstudie unter Praxisbedingungen [Treatment of pollinosis with the homeopathic preparation Galphimia glauca. A double-blind trial in clinical practice]. Allergologie; 13:359–363.
Flagged as green, meaning positive.
This article is not indexed in PubMed (others by the same authors, are; some at least use pharmacologically active doses), but it is included in reviews including Linde (1997) and the Australian NHMRC review. It is larger (N=243) but most sources find it hard to evaluate, not being available in translation. The Jadad score is 4, which is reasonable.
There is one large caveat in addition to its not being indexed: it is published under the auspices of the Karl und Veronica Carstens-Stiftung, a foundation devoted to the promotion of homeopathy and naturopathy. This is substantially different from a medical study funded by, say, the Wellcome Trust: Wellcome have no particular investment in any specific treatment, and very few medical trials aim to validate an entire class of treatment, especially one which has no plausible biomedical basis.
This trial is positive and apparently of acceptable quality.
7 Wiesenauer M, Gaus W (1985). Double-blind trial comparing the effectiveness of the homoeopathic preparation Galphimia potentization D6, Galphimia dilution 10?6 and placebo on pollinosis. Arzneimittel Forschung; 35:1745–1747.
Flagged as amber, meaning equivocal.
This is an earlier trial by the same authors using the same remedy. It is negative despite using a D6 potency, which is pharmacologically active. The amusingly ambitious conclusion is:
The study itself demonstrates that it is possible to do strictly controlled trials for homeopathic drugs and with medical practitioners.
This is true but fails to address the question: in the absence of any remotely plausible mechanism, why would you want to?
This is also included in the NHMRC review, where it is summarised thus:
No significant difference. Improvement by end of treatment in intervention group [80% ocular, 78% nasal], diluted homeopathy remedy group [66% ocular, 51% nasal], placebo group [65% ocular, 58% nasal]
This study is negative, since there is no significant difference.
8 Wiesenauer M, Lüdtke R (1995). The treatment of pollinosis with Galphimia glauca D4 – a randomized placebo-controlled double-blind clinical trial. Phytomedicine; 2: 3-6.
Flagged as amber, meaning equivocal.
A third study by Wiesenauer and once again using pharmacologically active doses (D4 potency). N=164. NHMRC summary says:
Differences between groups were statistically significant only for ocular symptoms.
This study is equivocal.
The Faculty of Homeopathy lead with seasonal allergic rhinitis as one of the signature conditions for which there is a “Clearly positive direction of RCT evidence”.
|Study||FoH rating||Reality based rating|
|Aabel (2000)i||Positive (+1)||Equivocal (0)|
|Aabel (2000)ii||Negative (-1)||Negative (-1)|
|Aabel (2001)||Equivocal (0)||Negative (-1)|
|Kim (2005)||Positive (+1)||Unusable (0)|
|Reilly (1986)||Positive (+1)||Positive (+1)|
|Weisenauer (1990)||Positive (+1)||Positive (+1)|
|Weisenauer (1985)||Equivocal (0)||Negative (-1)|
|Weisenauer (1995)||Equivocal (0)||Equivocal (0)|
|Overall||Strongly positive (+3)||Negative (-1)|
In assessing the evidence for SAR, the FoH show strong evidence of motivated reasoning. It is possible to directly compare their evaluations with those of the independent review by the Australian NHMRC, and it is clear that the FoH consistently over-rate the outcomes compared with ratings by researchers with no vested interest in the outcome.
In Aabel (2001) for example they assign an equivocal (“non-conclusive” in their parlance) rating to a study where the NHMRC summary is:
Similar improvement in verum and placebo.
And the paper itself concludes:
The patient groups that received either placebo or Betula 30c for two successive years showed a consistent response
A finding of no difference between verum and placebo arms is not “non-conclusive”, it is unambiguously negative.
As Ioannidis points out, the chances of a positive result being a false positive rise when the intervention is implausible. The FoH make no allowance for this.
Simple re-scoring their best example leads to a clear negative sum of evidence. The trajectory is also heading the wrong way, as the newer studies are less positive than the older ones.
It is hard to see this as anything other than dishonest.