Honest Universe

Superstition, pseudoscience, and scepticism

ACC and Acupuncture 2

leave a comment »

The ACC has responded to my Official Information Act request that I mentioned in my last post on ACC and acupuncture. Here’s what I asked for in my request:

Dear Accident Compensation Corporation,

In response to an Official Information Act request from Kevin McCready in June (https://fyi.org.nz/request/1749-continue…), Mrs Koleti Vae’au wrote that:

In 2011, ACC’s research team conducted a literature review of the efficacy of acupuncture in the management of musculoskeletal pain. It found the most convincing evidence for the effectiveness of acupuncture related to the treatment of chronic neck pain and the improvement of pain and mobility in chronic shoulder pain. In terms of other injuries, evidence of the benefits of acupuncture was either inconclusive or insufficient.

I have been unable to find this particular review by searching on the ACC website, although I have found other reports such as the “Effectiveness of acupuncture in selected mental health conditions” brief report from earlier this year.

Also, if ACC has any guidelines for carrying out these reviews, could you please provide me with a copy of or link to these guidelines.

Sincerely,

Mark Hanna

Here’s a (direct PDF download) link to the mental health review that I mentioned in my request: Effectiveness of acupuncture in selected mental health conditions – Brief report [2014]

The brief conclusion of this report is:

There is limited good quality evidence to conclusively determine acupuncture’s efficacy in treatment of mental health conditions such as Major Depressive Disorder, Dysthymia, Anxiety Disorder, Borderline Personality Disorder and Post Traumatic Stress Disorder.

As I mentioned in my last post on this topic, ACC released their review of the evidence for acupuncture in the management of musculoskeletal pain in response to another OIA request. That review can be read in full here: The efficacy of acupuncture in the management of musculoskeletal pain

Here’s what ACC said in response to my OIA request yesterday:

There are only two ACC literature reviews on the efficacy of acupuncture. These are:

The efficacy of acupuncture in the management of musculoskeletal pain.

I understand from your email of 25 August 2014 that you have accessed a copy of this report and therefore do not require another copy.

Effectiveness of acupuncture in selected mental health conditions — Brief report 2014.

As you have identified, this is available on the ACC website.

In regard to ACC guidelines on literature reviews, ACC follows standard practice when undertaking literature reviews, and there are no ACC specific guidelines on this practice.

After my previous post on this topic, I was contacted by Ross Mason who told me about a similar OIA request he had made some years ago. One interesting thing I read in their response to him, which was written by the same person as their response to me, was this (bolded emphasis mine):

2. Evidence of the efficacy of the use of CAM treatments/programmes;

Schedule 1 Part 1 sections 1 & 2 of the [Accident Compensation] Act detail ACC’s liability to pay for the cost of treatment. These provisions in part include the requirement that the treatment is necessary and appropriate and of the quality required for the purpose. ACC has always required that new treatments for which payment is requested are supported by evidence of effectiveness. However it must be noted that there are many treatments that treatment providers utilise that do now have a well established evidence base.

Examples of the research that ACC does in considering new treatments can be found on the “For Providers” section of the ACC website – http://www.acc.co.nz/about-acc/research-sponsorship-and-projects/research-and-development/evidence-based-healthcare-reports/index.htm

If requested to fund CAM, ACC would require that evidence be provided for the treatments [sic] efficacy. ACC evaluation process of the evidence is detailed on the ACC website in the “For Providers, Clinical Best Practice” section – http://www.acc.co.nz/for-providers/clinical-best-practice/index.htm.

The first link, to the “Evidence based healthcare reports” page, seems to provide a little more detail on what was described as “standard practice” in response to my request:

Evidence based reviews

These reports assess the effectiveness and safety of health interventions. They are developed according to a robust methodology similar to that used by the Centre for Evidence Based Medicine (external link) and the Scottish Intercollegiate Guidelines Network (external link). This includes systematic searches of the literature, critical appraisal of existing research evidence and peer review by clinical experts.


ACC has funded acupuncture treatments for over a decade, spending over $24 million on it last year alone.

In August 2011, ACC reviewed the evidence regarding the efficacy of acupuncture for musculoskeletal pain and found that (emphasis mine):

The evidence for the effectiveness of acupuncture is most convincing for the treatment of chronic neck and shoulder pain. In terms of other injuries, the evidence is either inconclusive or insufficient.

In March 2014, ACC published a brief report on the effectiveness of acupuncture in selected mental health conditions didn’t find enough good quality evidence to provide any recommendation.

No other review of the evidence for acupuncture has been undertaken by ACC. Despite this, over the last year ACC spent significant amounts of public money on acupuncture treatments for other medical issues such as burns ($30,002), lacerations and puncture wounds ($309,458), and fractures and dislocations ($591,613).

In the past year, they spent $22,592,552 on acupuncture for soft tissue injuries. Unfortunately, their recent response to another OIA request shows that they haven’t been keeping track of which body parts were treated, so we’ve been unable to determine how much of this substantial amount of money was spent on treatments that ACC’s own findings say are not supported by evidence.

As far as I’ve been able to tell, ACC’s funding scheme for acupuncture simply isn’t consistent with the evidential requirements they claim to require, and is instead largely based on the pre-scientific notions (to quote the author of their 2011 review) detailed in their Acupuncture Treatment Profiles document. When these issues were raised with the previous ACC minister Nick Smith in 2009 he promised a review of their effectiveness would be undertaken, but in the 5 years since then no such review has taken place. This latest response of theirs has only made me feel even more strongly about my recommendations for change:

I think ACC needs to review its funding scheme for acupuncture. I think their approach to this should start with reviewing their Acupuncture Treatment Profiles document, ensuring that the only treatments contained within it are those supported by rigorous evidence, and purging pseudoscientific claims from it. If they find they need to undertake further reviews of the evidence for the use of acupuncture for particular indications, then they should do that before approving funding for it.

I think ACC should then only agree to pay for acupuncture treatments that are aligned with their Treatment Profiles document, which they should commit to reviewing at regular intervals to keep it in line with the latest evidence (I’m not sure what time interval would be most appropriate, and I understand that there is a cost involved in that work).

I’m not sure, but it’s possible some changes to legislature may be required before this becomes a reality, but if that’s the case those changes should happen. A government body should not be bound by law to fund healthcare that is not supported by evidence.

There’s one last thing I’d also like to see, although I really feel like this is a long shot. I think ACC should take an active role in discouraging healthcare practice based on the “pre-scientific notions” described in their 2011 review. I think they should do this by distancing themselves from those acupuncturists who promote it and who base their practice on it, by refusing to grant them status as registered ACC practitioners if they are found to rely on it.

Written by Mark Hanna

2014/09/04 at 5:11 pm

Green Party Health Policy

with 14 comments

There was a thread recently in one of the Facebook groups I’m a member of in which, among other election-related things, the Green Party’s Health Policy was being discussed. It was mentioned that the Green Party had been essentially pro-CAM, at least in the past. As I’d been considering voting for the Greens, this was something I thought I should look into further.

Here’s a link to their Health Policy. My first impressions of this document were quite positive, especially seeing that one of their “Key Principles” seems to echo the mission statement of the Society for Science Based Healthcare where it states that:

Decisions about health services should be based on the strongest possible evidence.

However, I found myself worried that this might just be paying lip service to evidence-based policy. Particularly considering the comments I’d seen regarding past policy, and some other sections of the current Health Policy such as:

The Green Party will… Find ways to integrate complementary therapies that have a sound evidence base into health services.

The Green Party will … Increase resources for physical and mental rehabilitation in… complementary practices.

Of course, complementary therapies that have a sound evidence base should be “integrated” into real medicine. That’s exactly what medicine should be – therapies with a sound evidence base. However, as I said before, I felt worried this mention of evidence may just be lip service, or perhaps that the standard of evidence required might be low (as I’ve typically seen when it comes to the promotion of complementary therapies).

As you’ll have seen if you’ve clicked on the Health Policy link, Kevin Hague is listed as the spokesperson for this policy, as well as the person to contact. So that’s what I did. Below is the email I sent him, and the response I received (much faster than anticipated – within 2 hours):

Hello Mr Hague,

My name is Mark Hanna. I don’t think we’ve met, although we were both at the New Zealand Skeptics conference in Wellington last year. I’m currently considering giving my party vote to the Greens this election but I wanted to email you first to clarify something about one of the policy statements.

I spend a lot of my spare time working with regulatory systems to combat misinformation about health and healthcare in the public sphere. The vast majority of this misinformation comes from sources that promote “complementary therapies”.

To this end, I’ve made dozens of successful complaints to the Advertising Standards Authority and Medsafe, and have recently co-founded the Society for Science Based Healthcare as part of this continued effort. I’ve also written on some aspects of publicly funded healthcare in New Zealand that I find troubling and would very much like to see change, such as ACC’s stance on acupuncture (http://sciblogs.co.nz/honestuniverse/?p=41).

I’ve read the Green Party’s Health Policy (https://home.greens.org.nz/policy/health-policy); I’m glad to see an emphasis on evidence and I strongly support with [sic] party’s focus on equality in healthcare and making it as accessible as possible. However, I’m also wary that I don’t want my party vote to inadvertently support the kind of thing I spend so much of my time fighting against. Most particularly, I am worried about point 7 under the “Whole-of-System Healthcare” section:

“Find ways to integrate complementary therapies that have a sound evidence base into health services.”

As you may be aware, the majority of complementary therapies have a very poor evidence base. Even in cases where there is a large amount of research, in general it is of poor quality, and in very many cases the use of complementary therapies is simply not supported by sound evidence.

Other sections of the Health Policy add to my concern here, such as the 1st point under “9. Post-Acute Care” which says “The Green Party will… Increase resources for physical and mental rehabilitation in… complementary practices”. Before I commit my party vote, I’d appreciate it if you could assuage my fear that the reference to sound evidence may be no more than lip service.

In general terms, what would the Green Party consider the minimum standard of a sound evidence base in this context? Could you describe what the evidence for a treatment that does not meet this standard might look like?

As part of their commitment, will the Green Party also make an effort to prevent the integration of therapies (whether “complementary” or otherwise) into health services where there is not a sound evidence base?

I also note the party has pledged to:

“Support rongoa Māori (traditional Māori healing) practitioners and practices, and develop better linkages with other health services.”

I’m wary that when dealing traditional healing practices such as rongoa Māori and Traditional Chinese Medicine, the complex issues of science- and evidence-based healthcare and cultural sensitivity and inclusiveness can become entangled. Will the Green Party maintain their commitment to basing decisions about health services being based on the strongest possible evidence even when dealing with complex and often difficult issues such as these?

I’m under the impression that other people I know who are also considering giving their party vote to the Greens could also be swayed one way or the other depending on your response. I’d appreciate it if I could have your permission to publicly share any response you send me regarding this.

Sincerely
Mark Hanna

This is the response I received from Kevin Hague:

hi Mark,
I’m an admirer of your work. Thank you for it!

As you probably know, Green Party policy is developed by members, rather than MPs, and the Policy Committee strives for consensus if possible.

if you have compared the current Health policy with its predecessor (and my statements with those of my predecessor in the role) you will have noted substantial change that I think you would be pleased with.

The statement in the principles concerning complementary therapies, is a neat compromise I think (nobody can really argue for the use of public money on something that doesn’t work, but if it does work then it should be integrated), but does give rise to your question of what kind of evidence would be sufficient. In practice it would be me as spokesperson who would interpret the meaning of the policy (this is established practice in the Green Party). My first port of call would be Cochrane. If there is no guidance there, then it’s probably a no, but as someone who used to teach at postgrad level in research methods I would also be in a position to look at any papers that are proposed as providing evidence and make an assessment.

The obverse of this commitment does also apply ie if the state is currently funding therapies with poor or no evidence of effectiveness then it should stop doing so. This applies both to ‘complementary therapies, but also to mainstream treatments (my favourite instructive example being mammary arterial ligation as a treatment for coronary heart disease – it’s worse than ineffective).

In the case of rongoa I think matters actually are more complex. In addition to the cultural sensitivity matters to which you refer there are also Article II Treaty rights, so it’s not simply a matter of evidence. And when it comes to the evidence, far fewer studies have been undertaken. You probably know that there is significant evidence internationally that such indigenous treatment systems may be effective, even though their intervention logic and mechanism are quite different from traditional western approaches (eg Kleinmann in relation to mental illness in Taiwan). It could be that strongly held cultural beliefs may create a substantial placebo effect. Sometimes, even if we know the effect is just placebo, it could be worth having. So on this point I would interpret our policy as supporting the availability of rongoa, even though the evidence base for effectiveness may not be strong. I would also support further research to create a sound evidence base, where this doesn’t currently exist.

Hope that’s useful. Please feel free to share this if you want.

Best,

Kevin

Honestly, I’m quite happy to hear this answer. It sounds to me like the mentions of evidence are more than lip service, and hopefully indicative of a change in direction for the Greens in this area. The fact that Kevin Hague is their spokesperson on Health and ACC gives me hope as well. Especially comparing him to their past Health spokesperson Sue Kedgley, who has some ideas about healthcare that are rather wacky.

I would like to make one more comment regarding the issue of traditional medicine such as rongoa. First off, rongoa is not something I’ve looked into in much depth, I don’t know much about the state of the evidence and don’t mean to discuss it here. I would like to say though, that when I referred to issues like it being “complex and often difficult”, I meant to imply that it is not as simple as it may be in some other cases, where the only important question may be “what does the evidence say?”.

These traditions are often more than just for healing, they’re part of a wider culture and the people that use them are often already part of a disadvantaged group. If we care about what’s best for people, then we also need to care about the repercussions of discouraging or somehow diminishing such a part of their culture. As Kevin mentioned in his response, this is where things like the Treaty of Waitangi become important. There’s also an important distinction to be drawn between treatments for which there is negative evidence and those for which there is no evidence one way or the other.

I certainly wouldn’t claim to have all the answers when it comes to issues like this, and I think it’s a matter on which I would do better to listen than to speak. I do agree with Kevin that it’s not simply a matter of evidence, though, and I’d welcome a discussion on this particular issue if anyone else has anything to add.

Written by Mark Hanna

2014/09/03 at 9:53 am

The Price of Painkillers Part 2: Only Misleading in Australia

leave a comment »

Yesterday, I wrote about how various painkillers with the same active ingredients often cost different amounts. In particular I talked about Reckitt Benckiser’s “Nurofen” ibuprofen products, some of which are advertised as targeting specific types of pain even though it doesn’t seem plausible, given that they are essentially exactly the same, that they could target different types or sources of pain.

There are a few aspects of their advertising that I think gives this impression. For one, they have products that seem intended for specific types of pain that have names like “Nurofen Migraine Pain” and “Nurofen Back Pain”. For another example, if you look on their website you’ll see that the claim “Targeted relief from pain” is pretty prominently displayed alongside their logo in the upper left hand corner:

Nurofen

The packaging of their products repeats this claim, also pretty prominently:

ibuprofen lysine 342 mg (equiv. ibuprofen 200 mg)

In 2011 Professor Paul Rolan*, head of the medical school at the University of Adelaide, laid a complaint with the Therapeutic Goods Administration (TGA) regarding a TV advertisement for Nurofen as a treatment for headaches regarding this claim. The TGA is roughly Australia’s equivalent of New Zealand’s Medsafe, as they are responsible for the regulation of therapeutic goods.

For some reason (I’m assuming it’s a bug, and have emailed them about it), although the decision is still listed on the TGA’s Complaints Resolution Panel’s website, its full details aren’t currently available. However, they have been archived in full elsewhere. The thrust of the complaint was that:

The complainant accepted that the advertised product “is undoubtedly effective” as a treatment for headache, but argued that “it works by being absorbed in the blood stream and being distributed widely around the body, not only to where it is needed but to everywhere else as well”, and that “there is no ‘targeting’ to the head and hence the phrases ‘targeted pain relief’ on the packet and the claim made in the advertisement ‘goes straight to the source of the pain’ are factually incorrect and misleading”.

The relevant section of the Therapeutic Goods Advertising Code was section 4(2)(c), which states that:

An advertisement for therapeutic goods must not… mislead, or be likely to mislead, directly or by implication or through emphasis, comparisons, contrasts or omissions

Reckitt Benckiser’s response was basically that they weren’t being misleading because anyone seeing their advertisements would realise that when they say Nurofen “goes straight to the source of pain”…

an ordinary and reasonable person viewing the advertisement would interpret it as conveying “that Nurofen goes to and acts at the source of the pain associated with a headache, namely where the headache is located in the head” and not “that Nurofen will only go to and act at the source of the pain.”

I don’t find myself convinced by this rebuttal. It seems analogous (if you’ll pardon the violent example) to saying a bomb targets a single person, but that’s not misleading just because it also happens to destroy everyone nearby as well.

You can read the full findings of the panel yourself at the link above, but for a synopsis they essentially agreed with Professor Rolan in that:

26. The Panel did not accept that, in being dispersed throughout the body, the advertised product could reasonably be said to “target” or “go straight to the source of” headache pain or the site of a headache.

They also found Reckitt Benckiser’s rebuttal unconvincing:

28. The Panel noted that the advertiser had argued that consumers were generally well aware that Nurofen was suitable for many types of pain. On this basis, the advertiser argued that consumers would not interpret the advertisement as conveying a “targeted” action in the sense alleged by the complainant.

29. The Panel did not accept this argument. While the Panel accepted that consumers might generally be aware that Nurofen is suitable for many types of pain, this does not mean that a reasonable consumer would be aware that Nurofen is normally distributed throughout the body when taken. The Panel was satisfied that a reasonable consumer, noting the references to “targeted relief of pain” and “going… to the source of pain”, would conclude that the active components of the product travelled specifically to areas of the body affected by pain – in this instance, the part of the head affected by a headache. The reasonable consumer would also conclude that the active components of the product would not travel elsewhere throughout the body to any significant degree. The Panel noted that although the advertisement did not state exactly how Nurofen might physically concentrate in areas of the body affected by pain and target them, it did convey that Nurofen would do so. That Nurofen might also be understood to be beneficial for other types of pain did not alter this, because in such cases Nurofen would again be understood to concentrate at the site of pain.

As a result, Reckitt Benckiser was told to stop that particular advertisement and to not make the same representations in other advertisements. Of course, this only applies in Australia; to my knowledge Reckitt Benckiser hasn’t changed their advertising strategy here in New Zealand. As far as I know, it also hasn’t been challenged here.

I’m reminded of another case of what I consider to be misleading advertising from a pharmaceutical company that I’ve written about previously. When GlaxoSmithKline has advertised their “Panadol Extra” product, they have claimed it is “37% more powerful than standard paracetamol tablets”. The catch is that, when they say “more powerful”, they’re referring to potency, not efficacy, so instead of meaning the pills give 37% more powerful pain relief it means you can take 37% less Panadol Extra to get the same amount of pain relief. This might be difficult, however, considering that a single Panadol Extra pill contains exactly the same amount of paracetamol as a standard paracetamol tablet – 500 mg.


* Professor Rolan is also a member of Friends of Science in Medicine, a great Australian organisation dedicated to promoting science and fighting pseudoscience in the medical field. They do some great work across the ditch; I’m a member and I think they deserve your support as well.

The Price of Painkillers

with 6 comments

DISCLAIMER: Since I mention some specific branded pharmaceutical products in this article, I want to make it very clear that I don’t intend to promote any particular product and have no conflict of interests to declare.


Last Saturday, the New Zealand Herald’s Consumer Affairs reporter Morgan Tait published a very good article about a topic I’ve come across before. Many painkiller products cost different amounts despite having exactly the same active ingredients, and for some reason this isn’t common knowledge. Here’s a link to her article: Expensive and cheap pain-relief pills use same ingredients

I first discovered this from watching a Consumer Advocacy TV show from Australia called The Checkout. It’s quite light-hearted in its approach to issues, which I admit I find a bit painful at times, but that’s just a matter of taste and the information in the show is pretty consistently good. Although its content is sometimes only relevant to an Australian audience, a lot of it is also relevant to New Zealanders and I’d highly recommend you watch it if you get the chance.

Their episode that deals with painkillers is season 1 episode 5. The main point, which is also made in Ms Tait’s recent article, is that many painkillers you can find in pharmacies and supermakets cost different amounts but have exactly the same active ingredients. The prime example given in both the article and the episode is Nurofen.

Nurofen is an ibuprofen-based painkiller made by the pharmaceutical company Reckitt Benckiser. They sell a product range of “specific pain relief” products:

  • Nurofen Migraine Pain
  • Nurofen Tension Headache
  • Nurofen Back Pain
  • Nurofen Period Pain

(Before I go into any criticism of these products, I do want to note that the efficacy of these products is not something I mean to call into question. I only intend to examine issues associated with their advertising and sale.)

If you look at the Nurofen website, you’ll also see that they have a lot of marketing claiming that their products can target specific types or sources of pain. For example, here’s the Nurofen logo as it appears in the upper left corner of their New Zealand website:

Nurofen

As you can see, their slogan “Targeted relief from pain” is pretty prominently displayed. Other sections of the site, such as their “Nurofen Treatment Advisor” (which recommends Nurofen products based on what sort of pain you say you have) also imply that certain products are more applicable to certain types of pain than others. However, all of the products I listed above have exactly the same active ingredient: 342 mg ibuprofen lysine.

I hope I don’t need to explain that, as all products are taken in the same way (a “caplet”) and contain the same active ingredient, your body isn’t able to distinguish between them so they’re all going to act in exactly the same way. If you take a Nurofen “Nurofen Period Pain” caplet for your migraine pain, it will be just as effective as a “Nurofen Migraine Pain” caplet.

The obvious explanation for this is that having specific products makes it easier for consumers to understand that this product can help with their specific type of pain. It probably helps Reckitt Benckiser sell more products too, just quietly. However, I can definitely see how this could be confusing to consumers – I know I was surprised for this very reason when I found out the products were effectively identical – and if the products are available for different prices people might end up paying more than they need to for exactly the same product.

The price issue, of course, is not an issue if all the products cost the same. However, this isn’t always the case. After I first became aware of this in March this year, I had a look and found that Pharmacy Direct stocked these products for different prices:

Of course it’s not particularly surprising that the 12 caplet packs are relatively more expensive than the 24 caplet packs, but the price differences between packs of the same size seems quite odd. I also find it very strange that it’d be cheaper to by two 12 caplet packs of Nurofen Migraine Pain than a single 24 caplet pack of the same product.

I emailed Pharmacy Direct on the 27th of March to ask about the price discrepancies. Their response was basically that yes, the products do all have exactly the same ingredient, and they thought the suppliers did that for marketing reasons. They told me that the difference in price was due to them selling more of some packs than others, so they can justify buying more of them from their supplier and getting a bulk discount that allows them to sell the products to consumers for a lower price. They also offered me the option to pay the lowest price for any of the products if I mentioned our conversation in the “notes” section when buying them online.

If you’re looking for a cheaper ibuprofen-based painkiller as an alternative to Nurofen, you can buy a 24 pack of Countdown’s “Homebrand” version (just one example, I’m sure there are others) for $2.99. That’s $0.12 per pill, compared with $0.73 per pill as the cheapest Nurofen branded option from Pharmacy Direct. Now if you check the packet you’ll see that Nurofen pills contain 342 mg of ibuprofen lysine, whereas this particular unbranded option contains 200 mg of ibuprofen per pill.

I’m by no means an expert on the differences between ibuprofen and ibuprofen lysine, but I’m under the impression that 342 mg ibuprofen lysine will have a faster onset than 200 mg ibuprofen but provide the same amount of pain relief. If anyone knows more please say so in the comments. If you look at a pack of Nurofen, you’ll see that they’re considered equivalent:

ibuprofen lysine 342 mg (equiv. ibuprofen 200 mg)

ibuprofen lysine 342 mg (equiv. ibuprofen 200 mg)

Other types of painkiller can suffer from similar problems. For example, a 20 pack of Panadol (500 mg paracetamol) from Countdown costs $4.19, but a 20 pack of their own equivalent “Home Brand” paracetamol (which also contains 500 mg paracetamol per pill) costs $2.19.

While these cheaper alternatives do often contain the same amount of the same active ingredient (or equivalent), they’d unlikely to be exactly the same. For example, they may use different “fillers” to make up the rest of the pill. If you’re concerned about the differences or want to know more, your GP or pharmacist should be able to give you some advice.

It can be worth looking for an unbranded version if you’re looking to save. So-called “generics” are typically made available after the patent on a drug ends, and other companies are able to start producing and selling it. They tend to be much less expensive, and are sometimes subsidised if you have a prescription, but are often also less well-known.

Written by Mark Hanna

2014/08/28 at 1:45 pm

ACC and Acupuncture

leave a comment »

Over the past year or so, various Official Information Act (OIA) requests have been made via the website fyi.org.nz, which allows for these requests and their responses to be made public. I have written previously about the results of some of these requests regarding ACC and Alternative Medicine. I’ve also shared some more information on Twitter, which Thomas Lumley has documented on the StatsChat blog: Sticking it to ACC

Following my previous post on this matter I also wrote to the ACC Minister, Judith Collins, in May to outline my concerns. Here’s what I said in my email:

Dear Ms Collins,

My name is Mark Hanna. I’m a consumer advocate from Auckland with a particular interest in alternative medicine. Over the past year and a half I’ve been actively trying to reduce the amount of misinformation regarding alternative medicine so often seen in advertisements, mainly by lodging complaints with the Advertising Standards Authority.

Last year, prompted by an earlier Official Information Act request, two friends of mine submitted OIA requests of their own to the Accident Compensation Corporation inquiring as to how much funding it had provided to “alternative therapies”:

The latter request has attracted some media attention: Big bill for alternative health

In looking further into this issue, I found that in 2004 Dr Paul Hutchison submitted a number of very interesting questions to the Minister of the ACC regarding their funding of acupuncture, such as how much had been paid, how much they expected to pay in the future, and whether the treatments covered were supported by scientific evidence.

I’m also aware, largely due to its mention in the recent article, and also due to this article from 2009, that in 2009 then-ACC Minister Nick Smith said the ACC would review the effectiveness of these treatments. Unfortunately, it seems no such review has been conducted.

Piecing together the information released by the ACC regarding their funding of acupuncture, I believe it paints a rather disturbing picture. I have attached a chart constructed from the data released by the ACC in response to your 2004 questions, and their updated expenditure data released earlier this month.

Now, of course, this wouldn’t be a problem if the ACC’s funding of acupuncture were due to a strong evidence base in favour of its effectiveness and safety. Unfortunately, this does not seem to be the case.

The earlier OIA request I mentioned asked the ACC how they measure the effectiveness of treatments that they fund. Their answer included the statement that:

With respect to processes and requirements for measuring effectiveness, Treatment Profiles have been developed for 150 common injury types. Treatment Profiles are guides to the treatment and rehabilitation services ACC expects a practitioner to provide to a client for a particular injury. They describe current ‘good practice’ and what outcomes should be achieved.

However, viewing the ACC’s Acupuncture Treatment Profiles document, I find instead that it is based on statements such as these:

  • “Acupuncture… Relieves pain by treating stagnation of Qi and Blood in the affected areas and channels”
  • “Qi is the vital force of life which… Is the material substrate of the Universe”
  • “Brain… Is considered to be the same in substance as marrow”
  • “the Kidney produces marrow”

That is only a small selection of the many statements in that document that are clearly – even blatantly – no more than pseudoscience at best. They certainly do not give the impression of science-based decision making.

In light of all this, I believe it is high time the ACC conducted a rigorous review of the evidence regarding acupuncture so as to cease funding treatment options that do not appear to be effective or science-based.

As current ACC Minister, I hope you will be able to help facilitate what seems to me like a long overdue review of the effectiveness of acupuncture and related practices as they are funded by the ACC.

Sincerely,
Mark Hanna

Here’s the chart I attached to that email:

ACC Acupuncture Spending 1994-2013

I heard back from Ms Collins on the 18th of June, you can view her response here:

Response from Judith Collins 18 June 2014

In her response, she described review ACC undertook of some of the evidence regarding acupuncture in 2011:

In 2011, ACC’s research team also conducted a literature review of the efficacy of acupuncture in the management of musculoskeletal pain. It found the most convincing evidence for the effectiveness of acupuncture related to the treatment of chronic neck pain and the improvement of pain and mobility in chronic shoulder pain. Evidence on the effectiveness of acupuncture is not dissimilar to other physical therapies such as physiotherapy, chiropractic and osteopathy.

Since that response, another OIA request was submitted to ACC by Mark Honeychurch, co-founder of the Society for Science Based Healthcare, asking for a breakdown of acupuncture spending by condition. To make the request easier, the categories requested were the same as those used by ACC’s Injury Statistics Tool. You can view the results from that request directly as a PDF or as a Google Spreadsheet.

The vast majority of the spending (around 94% for each of the last 2 years) was for soft tissue injuries. However, without a more detailed breakdown we don’t yet know how many of this was supported by ACC’s findings regarding the evidence for acupuncture.

Although the amount of money is piddling in comparison, I have to admit I find it rather concerning to see that, over the past decade, ACC has spend $9,000 on acupuncture for deafness.

EDIT 2014/08/30

In a discussion of this post on Facebook, Jonathan Grady raised a good point about what this might have involved. It’s still just speculation, but I thought it was worth mentioning here:

Well, I know of no evidence that acupuncture is useful in the treatment of hearing impairment per se. There’s no plausible mechanism at work. But it could well, in some instances, be used plausibly to treat tinnitus, as tinnitus has a high component of subjective influences such as mood and levels of physiological and psychological arousal (similar to perceptions of chronic pain and of “phantom limb” syndrome). Placebos have some good efficacy in the treatment of tinnitus and, as we all know, acupuncture is really not much more than a placebo treatment. But it certainly won’t help you hear any better…

Jonathan Grady, Practicing Audiologist and Consulting Expert Member of the Society for Science Based Healthcare


I currently have an outstanding OIA request with ACC, asking for their 2011 review as well as any others they’ve conducted regarding acupuncture, and any guidelines they have for conducting these reviews.

Although my request asked for “copies of or links to all literature reviews regarding the effectiveness of acupuncture for any condition undertaken by ACC”, in their initial response to my request they only identified that I had asked for “A copy of The efficacy of acupuncture in the management of musculoskeletal pain conducted by ACC’s research team”.

I’m still waiting to hear back regarding if this means it’s the only such review, or if there are others as well. However, I have found other reports on the ACC website, like this one on the Effectiveness of acupuncture in selected mental health conditions, so I suspect there should be more reviews than just the 2011 one.

Yesterday, in response to a separate request from Kevin McCready, ACC has released this 2011 review. Although the names and qualifications of the author or authors were also requested, ACC denied this part of McCready’s request and removed that information from the report:

ACC declines to provide the names and qualifications of the staff members who compiled the literature review as there is a need to protect people’s privacy. We have also been unable to identify why the release of this information would be in the public interest.

You can find the full report, minus the name of the reviewer, here: The efficacy of acupuncture in the management of musculoskeletal pain

Here are the “Key findings” of this review:

General

  • There is insufficient evidence to make a recommendation for the use of acupuncture in the management of acute neck, back or shoulder pain
  • There is emerging evidence that acupuncture may enhance/facilitate other conventional therapies (including physiotherapy & exercise-based therapies)
  • There is a paucity of research for the optimal dosage of acupuncture treatment for treating shoulder, knee, neck and lower back pain
  • Studies comparing effective conservative treatments (including simple analgesics, physical therapy, exercise, heat & cold therapy) for (sub) acute and chronic non-specific low back pain (LBP) have been largely inconclusive

Lower back

  • The evidence for the use of acupuncture in (sub)acute LBP is inconclusive
  • There is limited evidence to support the use of acupuncture for pain relief in chronic LBP in the short term (up to 3 months)
  • The evidence is inconclusive for the use of acupuncture for long term (beyond 3 months) pain relief in chronic LBP
  • There is no evidence to recommend the use of acupuncture for lumbar disc herniation related radiculopathy (LDHR)

Neck

  • There is good evidence that acupuncture is effective for short term pain relief in the treatment of chronic neck pain
  • There is moderate evidence that real acupuncture is more effective than sham acupuncture for the treatment of chronic neck pain
  • There is limited evidence that acupuncture has a long term effect on chronic neck pain

Shoulder

  • There is good evidence from one pragmatic trial that acupuncture improves pain and mobility in chronic shoulder pain
  • There is limited evidence for the efficacy of acupuncture for frozen shoulder
  • There is contradictory evidence for the efficacy of acupuncture for subacromial impingement syndrome

Knee

  • There is no evidence to recommend the use of acupuncture for injury-related knee pain

Ankle

  • There is no evidence to recommend the use of acupuncture for ankle pain

Regarding the evidence found for neck and shoulder pain, it seems important to note a couple of caveats:

The “good” evidence for shoulder pain was from a single trial: Molsberger et al. 2010. This trial had a decent sample size of 424, and compared Chinese acupuncture to sham acupuncture.

The abstract doesn’t specify any details of the sham treatment, but the full paper describes it as involving the same number of needles and number and frequency of treatments, but the needles were placed at non-acupuncture points and inserted less than 5 mm (compared with 1-2 cm in the acupuncture group). I was glad to see one of the inclusion criteria was that participants be naïve to acupuncture, so they could be effectively blinded to this.

The full paper makes it clear that “Treatment assignment… was known to the acupuncturist” so this study wasn’t double-blinded.

The report also notes that its conclusions contrast with an earlier Cochrane review on Acupuncture for shoulder pain, which found that:

There is little evidence to support or refute the use of acupuncture for shoulder pain

The Cochrane review was carried out in 2005, though, prior to some of the studies examined in the ACC review such as Molsberger et al 2010.

It's surprisingly hard to find a stock photo of acupuncture involving gloves

It’s surprisingly hard to find a stock photo of acupuncture involving gloves

While the overall evidence for neck pain was described as “good”, when compared with sham acupuncture (which is the most appropriate control if you want to account for the elaborate placebo effect of acupuncture) the evidence was not so strong. It’s not clear what the sham protocol was, or if it allowed for the blinding of both participants and practitioners.

It seems Judith Collins’ description of the review’s conclusions was taken practically verbatim from its “Summary Message”, although she also seems to have opted to remove the sentence regarding the evidence for injuries other than those involving chronic neck and shoulder pain, which I’ve highlighted below:

The evidence for the effectiveness of acupuncture is most convincing for the treatment of chronic neck and shoulder pain. In terms of other injuries, the evidence is either inconclusive or insufficient. The state of the evidence of the effectiveness of acupuncture is not dissimilar to other physical therapies such as physiotherapy, chiropractic and osteopathy.

For a more direct comparison, here’s her wording again:

In 2011, ACC’s research team also conducted a literature review of the efficacy of acupuncture in the management of musculoskeletal pain. It found the most convincing evidence for the effectiveness of acupuncture related to the treatment of chronic neck pain and the improvement of pain and mobility in chronic shoulder pain. Evidence on the effectiveness of acupuncture is not dissimilar to other physical therapies such as physiotherapy, chiropractic and osteopathy.


Despite the findings of this report, ACC still seems to be spending a lot of money on treatments that their own review concludes are not supported by evidence. It’s hard to quantify exactly how much of their expenditure on acupuncture – over $24,000,000 over the 2013/14 financial year – went toward these treatments, but Daniel Ryan (also from the Society for Science Based Healthcare) has submitted another OIA request asking for a further breakdown of spending by body part.

In their response to Kevin McCready’s request, however, they have said that:

ACC funds acupuncture for conditions where prescribed by the client’s medical professional. This includes conditions other than musculoskeletal pain, and for non-chronic pain.

I’m trying to give them the benefit of the doubt, but from this it sounds like their review of the evidence may play no part in the decision of whether or not to fund acupuncture treatment for any particular condition.

Instead, it appears ACC will fund any acupuncture treatment so long as the patient was:

referred for acupuncture by a health professional who is covered by the Health Practitioners Competence Assurance (HPCA) Act.

Perhaps this explains ACC’s worrying funding of such treatments as acupuncture for deafness, but I certainly don’t think it excuses it.


Although the report makes an “Important Note” at the top that “The content [of this document] does not necessarily represent the official view of ACC or represent ACC policy”, I did find one particular section in the background section quite interesting. First off, TCM acupuncture is described as:

Drawing of the human body showing acupuncture meridians

TCM acupuncture involves inserting needles into traditional meridian points with the intention on [sic] influencing energy flow within that meridian

As far as I’m aware, that’s a fairly accurate description of how modern acupuncturists tend to claim their treatments work. For example, the New Zealand Register of Acupuncturists’ explanation of How Acupuncture Works claims that:

Any disruption or blockage of the flow of Qi along the meridians will in time affect the associated organ, resulting, for example, in decreased function or pain. Acupuncture aims to correct this flow of Qi, and thereby restore the balance within the body

This explanation also seems to be reflected in ACC’s Acupuncture Treatment Profiles document. However, the reviewer then went on to say that:

many physicians currently practicing acupuncture reject such pre-scientific notions described above

I think that description of acupuncturists’ claims to be able to manipulate qi flow with their needles as “pre-scientific notions” is absolutely spot on. The existence of qi and meridians is not supported by any evidence, and when this practice was developed it was based more on philosophy than evidence. In that way, it’s similar to extinct medical philosophies such as the “Western” medical philosophy of humorism.

One aspect of acupuncture that I find illustrates quite well the fact that its development was not supported by evidence is that horses are said to have a gall bladder meridian. This gall bladder meridian has even been the subject of published papers in journals such as the “Journal of Acupuncture and Meridian Studies”. What’s so odd about that, though? Horses don’t have a gall bladder.

Statements such as many of those found in ACC’s Acupuncture Treatment Profiles, which was “developed by the New Zealand Register of Acupuncturists Inc. in consultation with the New Zealand Acupuncture Standards Authority in a joint initiative with ACC”, are pretty self-evidently pseudoscientific. For example:

“Qi is the vital force of life which… Is the material substrate of the Universe”

If I want to know about the “vital force of life” or the “material substrate of the Universe”, I’ll ask a biologist and a physicist, respectively. So I asked a biologist and a physicist what they thought of these claims regarding “Qi”. Here’s what they had to say:

From my perspective as a biologist, that quote about Qi bears a lot more relation to the old and discredited concept of ‘vitalism’ than it does to modern biology. There is no evidence Qi exists, let alone that the lines along which Qi is supposed to flow through the body exist (and indeed, different ‘schools’ of acupuncture seem to deal in different ‘meridians’ – they can’t all be right!)

Alison Campbell, Senior biological sciences lecturer at the University of Waikato

I think it is self-evident that modern physics does not hold that qi is the “material substrate of the universe”; I would just say that qi is not something you find in physics textbooks; going beyond that might be beating a dead horse.

Richard Easther, Professor and Head of Department of Physics at the University of Auckland

(Alison and Richard often have interesting things to say. If you’d like to follow them, Alison blogs at Bioblog, and Richard at Excursionset.com. You can also follow him on Twitter @REasther.)


Now you have some insight into what I consider to be the problems with ACC’s funding of acupuncture. I’ve had people suggest to me, though, that some people do seem to benefit from acupuncture, so perhaps I shouldn’t be so interested in removing it from ACC.

Before I respond to that, I want to make one thing clear. I am not and will not argue for preventing people who want acupuncture from having acupuncture. Patient autonomy is an immensely important aspect of healthcare, and people have the right to make their own healthcare choices.

My primary motivation, as with all my involvement in dealing with dodgy healthcare claims, is echoed in the mission statement of the Society for Science Based Healthcare:

We believe that a strong basis in rigorous science is a necessary prerequisite for providing safe and effective healthcare. Decisions regarding public funding of healthcare in New Zealand should therefore be science based. We support public health measures that have a clear basis in science and evidence, and oppose those that do not.

We will work to counter misinformation about health issues propagated by individuals and organisations in New Zealand.

Consumers have the right to make an informed decision about their healthcare, and should not have to worry about being misled by unsubstantiated claims.

People have the right to make their healthcare choices, but they should not be misinformed by claims like “this needle will unblock qi flow through your meridians”.

Also, and this is more relevant to the case at hand, a treatment that is not supported by rigorous evidence is at best an experimental treatment. Generally the only ethically appropriate situation for an experimental treatment to be used is in a well-designed clinical trial, although in certain cases an argument can also be made for compassionate use exceptions.

However, this is not what is being funded. ACC is funding acupuncture for indications for which there is either insufficient evidence, no evidence, or negative evidence, and it is funding it in a clinical setting.

Furthermore, the ACC registered acupuncturists that benefit from this framework are generally proponents of the “pre-scientific notions” of qi flow and meridians. Whether they intend to or not, I think ACC is effectively promoting this via their association with these practitioners, and through official documents such as the Acupuncture Treatment Profiles. I also think this promotion is likely to do harm to the public understanding of science, particularly medical science.

ACC funding implies governmental approval, and many more obviously ineffective services such as “energy therapies” make similar justifications to acupuncturists. If consumers are led to believe that practitioners who base their practice on “pre-scientific notions” like qi flow are reliable and trustworthy, it seems reasonable to me to expect that they will think the same of practitioners who base their practice on other concepts such as aura healing.

So, what do I think should be done? I think ACC needs to review its funding scheme for acupuncture. I think their approach to this should start with reviewing their Acupuncture Treatment Profiles document, ensuring that the only treatments contained within it are those supported by rigorous evidence, and purging pseudoscientific claims from it. If they find they need to undertake further reviews of the evidence for the use of acupuncture for particular indications, then they should do that before approving funding for it.

I think ACC should then only agree to pay for acupuncture treatments that are aligned with their Treatment Profiles document, which they should commit to reviewing at regular intervals to keep it in line with the latest evidence (I’m not sure what time interval would be most appropriate, and I understand that there is a cost involved in that work).

I’m not sure, but it’s possible some changes to legislature may be required before this becomes a reality, but if that’s the case those changes should happen. A government body should not be bound by law to fund healthcare that is not supported by evidence.

There’s one last thing I’d also like to see, although I really feel like this is a long shot. I think ACC should take an active role in discouraging healthcare practice based on the “pre-scientific notions” described in their 2011 review. I think they should do this by distancing themselves from those acupuncturists who promote it and who base their practice on it, by refusing to grant them status as registered ACC practitioners if they are found to rely on it.

But I won’t hold my breath.

Osmosis Skincare’s Drinkable Sunscreen Trial

leave a comment »

If you haven’t already read my post about Osmosis Skincare’s Drinkable Sunscreen, you might want to do that before you read this one. A brief overview is that Osmosis Skincare claims drinking their “harmonized water” prevents sunburn. I complained to the Advertising Standards Authority that this claim wasn’t backed up by evidence and the complaint was upheld, now those products have been removed from their New Zealand website as a result.

In their response to this complaint (which you can read via the ASA’s website), Osmosis Skincare said that (emphasis mine):

This is a new type of technology being used in this way and Head office can reference the internal research they did showing the product to be effective, but their independent clinical trial isn’t until the 28th of June, whereby they will put 30 people outside for one hour in San Diego, CA at noon supervised by a plastic surgeon.

Osmosis Skincare NZ – Response to ASA complaint 14/287

When reading this, I noticed that the study design seemed to be lacking a control group (which prohibits randomisation and blinding) and that the sample size seemed very small. These are all properties of low-quality science, but with only this brief summary to go by I couldn’t draw in any firm conclusions. In any case, as they were making therapeutic claims without any evidence to back them up, the complaint was upheld.


A few days ago, the head office of Osmosis Skincare issued a press release regarding this “independent clinical trial”, which has now been completed. I have to say I wasn’t particularly surprised to read that the trial was actually not independent. The press release introduces it by saying:

Osmosis Pür Medical Skincare executed the [Harmonized Water UV Neutralizer] line’s first clinical trial on June 28, 2014.

Osmosis Skincare – First Drinkable Sunscreen Releases Clinical Trial Results

The press release also claimed that the trial was randomised (emphasis mine):

The randomized clinical trial was designed to evaluate a new technology…

Osmosis Skincare – First Drinkable Sunscreen Releases Clinical Trial Results

When a clinical trial is randomised, that means that participants are allocated into different groups in a way that is determined randomly. These different groups typically consist of an experimental group, which receives the treatment being tested, and a control group, which receives either a placebo or sham version of the treatment or the standard of care against which the experimental treatment is being compared. Randomly allocating participants into these groups helps avoid any systematic differences between the groups that could be a source of bias in the results.

Of course, this necessitates multiple groups for participants to be included in. With a sample size of 30, this means each group would only contain around 15 people. However, the press release isn’t done with its surprises yet:

24 patients ranging from 18 to 60 with various ethnic backgrounds and skin types were exposed to one hour of sun to one side of the body between noon and 1pm after ingesting 3ml Osmosis Harmonized Water UV Neutralizer

Osmosis Skincare – First Drinkable Sunscreen Releases Clinical Trial Results

The press release doesn’t mention 6 other participants in a control group, so it’s not clear yet if there was another (smaller) group that just isn’t mentioned here or if, for some reason, they went with 24 participants instead of the 30 they’d planned on using earlier.

The “Summary of Results” says that:

All 24 patients were evaluated before, and immediately after the exposure as well as the following morning. There was no evidence of sunburn on 16 patients, 5 had minor or partial sunburns and 3 had notable sunburns in the study.

This proves UV Neutralizer effectively limited the sun damage for a majority of the users that consumed it.

Hang on a minute, “proves”? I’m not sure how on Earth they’d expect to be able to honestly evaluate the effectiveness of their product without first establishing some baseline to use as a comparison. It reads as though they’ve assumed that, if their product didn’t work, then all participants would have received notable sunburns. If they want to gather evidence regarding the effectiveness of their product in an intellectually honest way, they’d need to either compare it to a placebo or to the standard treatment, so they can actually see if it cause different results than nothing or if it’s as good as the real deal.

Instead, though, we get quotes like this:

The definitive results from this trial prove that the scalar wave technology in Harmonized Water works.

Dr. Ben Johnson, Founder and CEO of Osmosis Pür Medical Skin Care – First Drinkable Sunscreen Releases Clinical Trial Results

The press release then provides links to the clinical trial and photos of participants.

If you click on those links, you’ll probably notice a couple of things right away. First, instead of being published in a peer-reviewed scientific journal, the link to the clinical trial takes you to a folder on “box.com”, a cloud storage website. Second, the PDF containing photos of participants only contains 16 participants, not the 24 we’re told participated in the trial. I’ve no idea why that is the case, or how they determined which 8 participants to exclude.

The paper, entitled “Evaluation of a Novel Form of Sun Protection”, seems laid out pretty much as one would expect from a real clinical trial, published in a peer-reviewed scientific journal. Its first author is Paul Ver Hoeve, the doctor that supervised the experiment, and the second author is Ben Johnson, the founder and CEO of Osmosis Skincare (although that conflict of interest isn’t stated in the paper itself). I’m really wondering why this was ever described as “independent”. Hopefully this was just a piece of honest confusion on the part of whoever was liaising with the ASA on behalf of Osmosis Skincare NZ.

Unsurprisingly, although the authors included 8 distinct references (out of 11 in total) for the claim that topical sunscreens can cause inflammation, this claim went unreferenced:

Upon ingestion of 2-3 ml of the [harmonized] water, the scalar waves reportedly work their way through the molecules of water in the body until they reach the water in the dermis. This process has been shown to take an hour on an empty stomach, 90 minutes if any food is present in the stomach.

Paul Ver Hoeve, Ben Johnson – Evaluation of a Novel Form of Sun Protection

The “Subjects and Methods” section starts with a very concerning sentence. As far as I can tell, this is the entire basis for describing the study as “randomized”:

In this study, 24 patients were randomly selected as test subjects with no consideration for their natural skin tone.

Paul Ver Hoeve, Ben Johnson – Evaluation of a Novel Form of Sun Protection

As I mentioned earlier, when a clinical trial is described as “randomised” that means participants are randomly allocated to different treatment groups. It absolutely does not mean “patients were randomly selected as test subjects”. I’m not sure what that even means. It sounds like it’s referring to their recruitment method, but no more detail is given so I can’t really tell what this is supposed to say about the study design.

In my opinion, using this as justification for describing this experiment as a “randomized clinical trial” is very misleading.

The report goes on to state that:

The decision was made to not do a double-blind test for this application because of the ethical implications of knowingly causing a sunburn in many people.

Paul Ver Hoeve, Ben Johnson – Evaluation of a Novel Form of Sun Protection

Despite this apparent ethical concern, I can see no indication of the trial being approved by an Institutional Review Board (IRB). I believe IRB approval is required of all human subject research in the USA that is publicly funded, but privately funded research like this doesn’t have the same requirement. I’m not sure how it would affect a trial’s chances of being published in a peer-reviewed journal, perhaps someone who knows more could weigh in via the comments.

Performing “a double-blind test” in this case would require giving some participants water then leaving them to burn in the sun for an hour, and they’re right to say that’s pretty obviously unethical. I think the fact that they realise this but it’s also what they did in their trial should have been a pretty big red flag that they could use some ethical oversight.

They also don’t mention what seems to me is obviously a more ethical and rigorous way to conduct the trial. Even if they didn’t manage blind the researchers, it would have been better for everyone if the experimental group were compared to a control group that had applied topical sunscreen instead. This would allow them to have a “randomized clinical trial” that is actually randomised, as well as giving them a baseline to compare their results to so they might have a chance to draw some useful conclusions.

The actual results of the experiment were that:

There was no evidence of a sunburn on 16 patients, 5 had minor or partial sunburns and 3 had significant sunburns in the study.

Paul Ver Hoeve, Ben Johnson – Evaluation of a Novel Form of Sun Protection

Now I don’t know about you, but 1/3 of participants getting sunburned doesn’t exactly sound like a rousing success to me. They went on to try to justify these failures by saying:

All of the patients who burned said they would not normally lay [sic] out in the sun for one hour. Many of them said they burn with the use of other sunscreens as well.

Paul Ver Hoeve, Ben Johnson – Evaluation of a Novel Form of Sun Protection

The report gives no indication that the other 16 participants were asked the same questions, so there’s no way of telling if this could have contributed to the results, let alone accounted for them. As far as I know, this hasn’t prompted Osmosis Skincare to put a warning label on their products that it’s not effective for people that don’t usually expose themselves to the sun very much. It also doesn’t stop the authors from putting the results down to these answers entirely:

While the results were not 100%, the authors believe this was due solely to the excessive amount of sun they received to their relatively virgin skin and their overall health.

Paul Ver Hoeve, Ben Johnson – Evaluation of a Novel Form of Sun Protection


Society for Science Based Healthcare

This trial had a tiny sample size, and was uncontrolled (therefore also non-randomised and unblinded), as well as being industry-funded and co-authored by the founder of the company that makes and sells the product. And on top of all that it didn’t even seem to have had particularly positive results.

If there’s any conclusion that can be drawn from this study, it’s that Osmosis Skincare is willing to do bad science and use its mediocre results to promote their products. Considering they were already making the same therapeutic claims before this experiment, I can’t say I find that surprising.

If Osmosis Skincare NZ stands by this research, and considers it rigorous enough to justify the sort of claims they were previously making about these products without substantiation, then they should appeal the ASA’s decision to uphold a complaint against them. I’d certainly like to see them try.

If you see this product being promoted or sold in New Zealand, please contact the Society for Science Based Healthcare with the details.

ASA Complaint: Osmosis Skincare’s Drinkable Sunscreen

with 3 comments

In May this year, One News ran a story on a US skincare company releasing what it was calling “drinkable sunscreen”. Around the world, various sceptical websites also picked up the story, such as Doubtful News and Neurological Blog. The message was roughly that a “drinkable sunscreen” is a cool idea that isn’t entirely implausible, but that this company’s “Harmonized Water” product seemed to be entirely ineffective pseudoscience.

I tweeted about the story from One News, noting that the article seemed like little more than free advertising of what really seemed like quite a dangerous product. Thomas Lumley, a professor of Biostatistics at Auckland University who runs the great blog Stats Chat, picked up this story and wrote about it there: Revolutionary new advertising success

He also pointed out to me on Twitter that this company, Osmosis Skincare, has a New Zealand distributor, and that they have a website. Here’s that website’s Harmonized Water product listing page. If you look at it now, you’ll luckily see that although it does list a large number of “Harmonized Water” products that almost certainly don’t do what is claimed about them, it does not include any products that claim to be able to be used as “drinkable sunscreen”. The reason for this is that the Advertising Standards Authority has upheld a complaint I lodged against their online advertisements for these products.


As part of submitting this complaint, I took screenshots of the advertisements. I’ve embedded these below so you can see the claims as they were originally made:

Osmosis Skincare - UV Neutralizer Tan

Osmosis Skincare - UV Protection No Tan

My full complaint is available for you to read, as well as Osmosis Skincare’s response and the ASA’s decision, on the ASA’s website. I encourage you to read it in full, but I’ve put some of the highlights in this article.


The gist of my complaint was, as usual, that I don’t believe the advertiser has any evidence to support the claims they were making about the product. I also argued that the advertisements “abuse scientific terminology in a way that seems intended to exploit consumers’ lack of knowledge”.

In my complaints I generally also argue that making misleading or unsubstantiated therapeutic claims is socially irresponsible, and when the ASA upholds my complaints they tend to agree. In this case though, I felt the advertiser went a step further:

The New Zealand Cancer Society website writes, on the dangers of unprotected sun exposure:

New Zealand has the highest rate of melanoma in the world, and other skin cancers are also very common. You can help reduce your risk of skin cancer by using sunscreen the right way.

By misleading consumers into believing they are protected when in fact they very likely are not, this advertisement is likely to increase their risk of contacting [sic] melanoma due to unprotected exposure to UV radiation from the Sun. This misrepresentation is highly irresponsible.

Soon after submitting my complaint, I saw that the British Association of Dermatologists had published a response to these products. It’s worth reading in full, but here’s a highlight:

We want to make it immediately clear at this stage, the formulation is 100% water and, as far as our experts are concerned, it is complete nonsense to suggest that drinking water will give you a Sun Protection Factor (SPF) of 30.

They also contacted Osmosis Skincare to ask what the “scientific basis” for their claims was. The full message and its response are available at the link. Osmosis Skincare confirmed that the product is 100% water and didn’t provide them with any evidence to support their claims.


In Osmosis Skincare’s response to my ASA complaint, they said they’d made some changes like calling the products “UV Neutralizer” instead of “UV Protection”. How they thought this made it acceptable is entirely beyond me.

They also said the following:

This is a new type of technology being used in this way and Head office can reference the internal research they did showing the product to be effective, but their independent clinical trial isn’t until the 28th of June, whereby they will put 30 people outside for one hour in San Diego, CA at noon supervised by a plastic surgeon. So perhaps we have some extra time to submit these results? We are told our UK distributor will also be conducting their own study. We have been selling this in New Zealand for the past couple of years without any issue.

I can’t say I was surprised to read that the “independent clinical trial” they were planning on would have a tiny sample size of 30 and be without a control group, let alone adequate randomisation and blinding.

DermNet NZ has a page on sunscreen testing and classification that says sunscreens in New Zealand are now tested in vitro. That makes perfect sense to me, partly because in vivo testing for something like sunscreen seems like it would likely be unethical (which is mentioned on DermNet’s page) and partly because the difference between sunscreen and no sunscreen – blocking UV radiation when placed on the skin surface – would presumably be much easier to test and measure than more complex medical outcomes.

I would also imagine that the placebo effect will not have a strong influence here, but that’s only a guess and I have no evidence to support that. However, a study like this would still need a control group to be able to tell how much of a difference the product made, and in such a design it would still be more rigorous to randomise participants and blind both them and the researchers to eliminate potential sources of bias.

It will be interesting to see if this trial is published, and what its methodology and findings are. Especially since they’ve publicly reported beforehand that an independent trial was due to be done.


The Advertising Standards Complaints Board seemed to agree with my complaint on all its main points. To quote the summary of their decision (which they note is not the decision itself, but the whole decision is available on their website):

The Complaints Board acknowledged the changes made by the Advertiser, however, it said that the amended advertisement was still misleading, abused the trust and exploited the knowledge of the consumer by stating that the product offered sun protection using scientific terminology without adequate substantiation. It said this was exacerbated within New Zealand as sun exposure can have significant negative effects in comparison with other countries.

Accordingly the Complaints Board said the advertisement was in breach of the Therapeutic Products Advertising Code and did not observe a high standard of social responsibility effecting a breach of the Therapeutic Products Advertising Code.

In their full decision, the complaints board noted that although Osmosis Skincare alluded to evidence, they didn’t actually provide any. They also raised the valid point that their US-based clinical trial’s “application in a New Zealand context considering the strength of the sun was questionable”. They also said that:

the advertisement was likely to abuse the trust and exploit the knowledge of the consumer by stating the product offered sun protection “30 x more than normal” and used scientific terminology like “isolates the precise frequencies” without adequate substantiation.

I have to applaud the complaints board here for taking a stand against this sort of language, which abuses scientific jargon in a way that makes the advertisement sound more authoritative than it should. As they would have been able to uphold the complaint on the sole basis that the claims are unsubstantiated, I’m glad they also decided to take on this language as well.


To their credit, Osmosis Skincare has quickly removed the advertisements for these products entirely from their New Zealand website, even before the ASA’s decision was released. However, as you’ll have seen if you clicked on the link to Osmosis Skincare’s “Harmonized Water” product listing page at the top of this article, they still sell a number of these products that also seem to make unsubstantiated therapeutic claims.

All of my complaints about misleading healthcare claims, including this one, are now submitted under the Society for Science Based Healthcare. If you’re interested in these complaints, have a look at their website. You can also keep up to date with their complaints on Twitter @SBHNZ.

Follow

Get every new post delivered to your Inbox.

Join 37 other followers