Yesterday, the Advertising Standards Authority released a decision regarding a TV advertisement for Panadol Extra. The advertisement claims that the product is “37% more powerful than standard paracetamol tablets”. Although this is not the claim that was challenged in the complaint, the advertiser, GlaxoSmithKline (GSK), provided a citation in attempt to substantiate their claim.
However, it seems to me that the citation they provided substantiates a different claim. The study they provided, Laska et al. 1984, substantiates the claim that their product is 37% more potent than standard paracetamol tablets, not 37% more effective. As far as I’ve found, in pharmacology, potency refers to the dosage required to achieve a particular effect. In claiming that their product is “37% more powerful” they didn’t mean that it is able to provide 37% more pain relief, but that you don’t have to take as much of it to get the amount of same pain relief.
In order to convince the Advertising Standards Complaints Board that saying “more powerful” when they meant “more potent” was not misleading, GSK pointed to a 2009 ASA decision in their response to the complaint:
Importantly, the claim ‘37% more powerful than regular paracetamol tablets‘ and the associated graph in question relate to the potency of Panadol Extra compared with regular paracetamol tablets and NOT its efficacy. That is, the reference to potency refers to the ratio of doses required to achieve the same analgesic effect rather than any improved efficacy result.
In October 2009, a complaint was considered by the ASA in relation to the claim that Panadol Extra is ’37% more powerful than regular paracetamol tablets’. The ASA Panel was of the view that this was an accurate description of potency and that it did not communicate efficacy improvements. The Panel was also satisfied that the claim 37% more powerful had been substantiated by the Laska 1984 study (Attachment 2). Accordingly, the Panel determined that the advertised claim was not, directly or by implication, deceiving or misleading consumers (Attachment 3).
Given the historical consideration of this claim by the ASA it is GSK’s view that the claim accurately communicates the potency of Panadol Extra and not the efficacy of this product compared to regular paracetamol tablets.
The complaints board seems to have accepted this argument, as they state in their decision that:
Firstly, the Advertiser addressed the claim “37% more powerful than standard paracetamol tablets” and the Complaints Board noted the percentage was in relation to the potency not the efficacy. It also noted the Advertiser provided robust substantiation to support the factual claim.
Partly as a result of this, the complaints board ruled to Not Uphold the complaint.
However, things aren’t quite that simple. First, the Commercial Approvals Bureau also responded to the complaint, stating that:
The claim of 37% improved efficacy over standard paracetamol is verifiable fact, and the client has sufficient data to substantiate this claim.
Apparently the Commercial Approvals Bureau was misled by the advertisement, interpreting its claim that the product is more powerful as regarding efficacy, not potency. To tell the truth, when I read the claim I made the same assumption. I was very surprised when GSK defended the claim by essentially saying they meant something else so it was okay, and honestly felt as though I had been misled.
It seems the complaints board have likely been misled as well. When GSK referred to the 2009 decision (09/626), they missed a very important point. That advertisement appeared in a publication specifically for medical professionals, and the complaints board had considered the likely interpretation of “more powerful” in that context. From their decision, they stated:
The Panel was of the view that within this informed environment, there would be a greater awareness and familiarity with analgesics, the difference between analgesic effect and potency, and a level of comfort with references to scientific studies and the capacity and the ability to access these studies, if further clarification was required of the reference to them.
Having made these observations, the Panel was of the view that medical practitioners reading the advertisement would understand the word “STRONGER” in the advertisement to mean potency.
In their response to this recent complaint, it seems GSK may have misled the complaints board when they told them that previous precedent has determined that “more powerful” means “more potent”, as they omitted the important and relevant fact that it was only decided to be the case for advertisements aimed specifically at healthcare professionals, not advertisements aimed at the general public such as this one.
It’s also relevant that, in some of the advertisements complaint 09/626 was about, GSK was making these claims:
Because Panadol Extra is 37% more powerful than regular paracetamol it provides extra pain relief and helps you break through the pain barrier
Panadol Extra…combines paracetamol with caffeine for 37% extra pain relief
That complaint was Upheld (in part) because the ASCB ruled that these claims had not been substantiated and were therefore misleading. Given that GSK has been willing to make this claim explicitly in the past, despite the fact that it seems to have been misleading, it would not surprise me at all if they intend for uninformed consumers to take away the same message from their more recent advertisement.
I also can’t help but wonder if the complaints board actually went through the details of complaint 09/626 when considering complaint 13/585, or if they just took GSK’s word for its contents. Their decision seems to imply the latter, unfortunately.
What do you think about the claim “37% more powerful”? Would you have assumed it meant “37% more pain relief”, or that it means you can take 37% less of the active ingredient in Panadol Extra than regular paracetamol to achieve the same result? Would you have been misled by this advertisement like the Commercial Approvals Bureau seems to have been?
UPDATE 2014/17/01 1:54pm NZDT:
The content of the page has now been updated for the better, although its title still says “Telescopes for astronomy or land viewing; a great gift for him”.
Over the summer holidays, for the first time ever I took some binoculars outside on a clear night and looked up at the sky. I had some idea of what I could expect to see, but still the number of stars I could see surprised me. It was marvellous and elating, and settled my resolve to (finally) take up amateur astronomy as a hobby instead of just an idle interest. As a first step, I plan to buy a telescope. Earlier today, I searched online to see what was available and was very disappointed by something that I found.
The New Zealand website telescopes.net.nz sells telescopes, which is all well and good, but they have decided to market them as a “gift for him”. The title of the page is “Telescopes for astronomy or land viewing; a great gift for him.”, and the content advertises them like this:
A telescope makes the perfect Christmas gift for him and we have picked the best brands available in the market today. Whether you want a gift for your Dad, the man in your life or you just want to indulge yourself, you will find it here.
I was thoroughly disappointed and, horribly, not particularly shocked to see such sexism regarding this. As a white straight male, I have tremendous privilege; my male privilege would let me simply ignore this if I wanted to, as it’s not an affront to me. However, I follow various fantastic (I cannot stress that enough) female astronomers, astrophysicists, and astronauts through Twitter and blogs, and have long been aware of the existence of sexism in STEM fields. In fields such as astronomy, there is a large gender gap – these fields are dominated (in terms of numbers) by men – and this is only made worse by commonplace sexist attitudes. Even if some of this behaviour is entirely innocent, it still does active harm by excluding women and girls from astronomy. Although my privilege gives me the option to ignore this, I consciously choose to be a feminist and fight these attitudes. Telescopes are for everyone.
I shared this page on Twitter, and the great Katie Mack (@AstroKatie) sent them an email. Seeing this, I also sent an email to telescopes.net.nz, which I include in full below, urging them to change the content of their website and the attitude from which it sprang. If you would like to do the same, you can contact them at email@example.com. Here is the email I sent them:
Dear Telescopes New Zealand,
I’m interested in amateur astronomy, having for the first time viewed the night sky through binoculars over the summer holidays and marvelled at how many stars I could see. I decided that in the new year, I would like to buy a telescope and take up amateur astronomy as a hobby. I found your website when searching for a telescope online.
However, when I came upon your telescopes page I was very disappointed to see it advertised telescopes as a “gift for him”, casually excluding all the women and girls that are interested in astronomy. Unfortunately, women already face significant sexism in STEM fields, including astronomy. Attitudes such as the one shown on your website only encourage this hostile environment, and harm astronomy as a whole.
I hope that you will revise the content of your website. I’m sure you can see that its current content, however innocent your intentions may have been, contributes to a harmful atmosphere of exclusion. Taking a more positive attitude toward women in astronomy will only benefit yourselves, amateur astronomers, and the field of astronomy itself.
I’ve set up a change detection service monitoring the page, and hope to see it change for the better soon. If it doesn’t then, needless to say, I certainly won’t be buying my telescope from telescopes.net.nz.
By definition, when it comes to telling the truth, advertisers have a conflict of interest. They want you to buy whatever they’re advertising, so they’re going to try to show it in its best light. Often, this means using psychological tricks, and these tricks generally work best when consumers aren’t aware of them.
For example, most people are aware that the reason why products cost $39.90 instead of $40.00 is that they sound significantly less expensive, even though the difference is miniscule. Consumers tend to be aware that advertisers are prohibited from “false advertising” – they can’t tell us anything that isn’t true – and that advertising is regulated. This tends to give us a sense of security when it comes to taking advertisers at their word. Unfortunately, there are a lot of ways in which medical advertisers in particular can, and do, take advantage of this.
In today’s edition of my local newspaper, there was a full page ad placed by “The Natural Health Co”. This advertisement contains a lot of these tricks that can be used by medical advertisers to mislead consumers without technically breaking the rules, so I thought I’d use it as a case study to point out some of these tricks.
First thing’s first, here’s the advertisement:
The most common trick is one that I’ve written about before. The industry regulators of medical advertisements in New Zealand draw a distinction between “therapeutic claims” and “health claims”. Although they sound very similar to the consumer, the important difference is that the advertiser is only required to substantiate therapeutic claims. Any health claims they make can be entirely unsubstantiated and, to my knowledge, if they’re false there is no penalty. To quote the guidelines:
Health Claims are defined as claims which support the normal physiological function.
An example of a health claim made in this advertisement is “Supports cardiovascular health”, which is said for a couple of products in this advertisement. As far as I can tell, the reason why advertisers are allowed to make these claims without oversight is that they are not well-defined, and claims like that technically just mean it won’t interfere with normal physiological function, i.e. what would happen anyway in a healthy person.
So, when this advertisement says a product “Supports muscular and nervous system health”, you should interpret it as saying “This product will not interfere with your muscular or nervous system health if you’re already healthy”, and as far as I know they’re not even required to have evidence to support that.
That’s why advertisers can get away with saying that glucosamine and chondroitin supplements work “for maintenance of healthy joints”, despite the fact that statements that these substances have beneficial effects on joint health do not seem to be strongly supported by quality evidence.
Another trick used by these advertisers is making a point out of what is contained in the product. This allows the consumer to draw certain conclusions without the advertiser having to suggest them directly. It’s important to remember that advertisers will always put their best foot forward. If there’s evidence to show that a product has a beneficial effect, then they will say that instead of only saying “Potent antioxidant”.
Most people think they understand what that means (it’s good for you, right?) but, unfortunately, most people also lack the medical expertise required to make good health judgements, and are easily influenced by information like this.
I expect that’s the main reason why health-related testimonials in medical advertisements are prohibited by section 58(1)(c)(iii) of the Medicines Act 1981. Unfortunately, Medsafe seems extremely apathetic when it comes to enforcing this; I’ve contacted them about numerous violations but as far as I can tell they’ve never done anything about them. While it seems intuitive that more information is better, some types of information tend to lead to misinformed health decisions, and testimonials are foremost amongst these.
There are a lot of examples of marketing tricks used in this advertisement. I wouldn’t be surprised if there’s not even a single piece of useful information about the therapeutic properties of one of these products that is backed up by any evidence at all.
Feel free to point out more instances of marketing tricks in this ad, or mention others that you’ve seen in medical advertisements. Unfortunately, “health claims” are almost everywhere in medical ads in New Zealand. Keep an eye out for the word “supports”, as it’s usually a strong indicator that they’re making a health claim and therefore likely don’t have evidence to support it.
In September, I found an A4 insert in the New Zealand Herald advertising for Niagara Healthcare. A big red heading: “Arthritic Relief?” caught my attention, and when I looked a little closer I found it accompanied by some big red flags. This advertisement for a “FREE TREATMENT” that seemed like it could relieve practically any type of pain, as well as several other ailments, looked a little too good to be true, and experience has taught me that when something looks too good to be true, it probably is.
My first response to this advertisement was to look for any research I could find corroborating its claims. This took me to the Niagara Healthcare website for New Zealand. They appear to be based in Australia, and have a separate but nearly identical website for their New Zealand branch. Their website’s key benefits page, which states that “Much research has been conducted on the physical benefits of Niagara’s Cycloid Vibration Therapy since 1954″, contained a convenient list of therapeutic claims for me to look at:
- Increase local area blood flow
- Assist in the reduction of musculoskeletal pain
- Increase joint mobilisation
- Reduce excess oedema (swelling) whether the cause is vascular or lymphatic
- Assist in the treatment of wounds where an improvement in circulation is a factor
- Assist in the treatment of pressure ulcers where and [sic] improvement in local circulation is a factor
The only study I was able to find (searching Google Scholar and PubMed) with the keywords “Cycloid Vibration Therapy” was a small uncontrolled preliminary study of 21 patients. That is nowhere near enough to substantiate a therapeutic claim. Luckily for me, there were also 4 other studies cited on the webpage.
I was able to find the full text of what I believed may be the first study mentioned. This study appeared to use a Niagara Healthcare product, Lymphease, but it was only a pilot study with a small sample size and no control group, not a clinical trial as claimed on the website, and therefore not rigorous enough to substantiate any therapeutic claims.
Interestingly, although this was not stated on Niagara Healthcare’s website, this study was funded by “Cyprossage Pty Ltd”, which holds the patent for the product used in the study. Both Cyprossage Pty Ltd and Niagara Healthcare are divisions of CT Healthcare Pty Ltd, and they share the same director, Anthony Thompson. Even if everything else in these advertisements checked out, this would violate the ASA’s Therapeutic Products Advertising Code Part B2 R4.3:
Publication of research results in an advertisement must identify the researcher and the financial sponsor of the research.
I was only able to find citations of the second and fourth studies, and only the abstract of the third study. As far as I was able to tell, the second and fourth studies were not clinical trials, and the third study did not adequately account for the placebo effect via its “no treatment” control group. These papers were also published in 1984, 1981, and 1961 respectively. Worryingly, the Australian version of this webpage describes those same studies as “recent”, despite the majority of them having been published years before I was born. If this was Niagara Healthcare putting their best foot forward, it wasn’t very impressive.
I was also able to find that the Advertising Standards Authority in the UK upheld a complaint against Niagara Healthcare in 2005, on the basis that the therapeutic claims they were making were not adequately substantiated. It looked like the evidence behind the advertisement didn’t live up to the claims, which was particularly worrying considering that the print advertisement claimed that the products had been “Medically proven for 60 years”, and had been approved by TAPS. The Therapeutic Advertising Pre-vetting System, TAPS, is a service provided by the Association of New Zealand Advertisers (ANZA) that is intended to help advertisers avoid publishing ads that violate the relevant codes and legislation.
The back of the print advertisement also contained a testimonial. I still don’t understand how a medical advertisement containing a testimonial could have been approved by TAPS, considering that the Medicines Act 1981 Section 58 subclause (1)(c)(iii) effectively prohibits all testimonials in medical advertisements:
no person shall publish, or cause or permit to be published, any medical advertisement that… directly or by implication claims, indicates, or suggests that… a medical device of the kind… advertised… has beneficially affected the health of a particular person or class of persons, whether named or unnamed, and whether real or fictitious, referred to in the advertisement
After finding how problematic these advertisements seemed to be, I laid a complaint with the Advertising Standards Authority. My complaint ended up being treated as two separate complaints: one for the print advertisement and a separate one for the website advertisement. On Friday, the ASA released their decision regarding both of these complaints. They were both upheld, meaning that the ASA has told Niagara Healthcare the advertisements must be removed. As I do with all my complaints, I have set up a monitoring service so I will be notified of any changes to the web advertisement. So far, the only change is that a note that the research they cite was funded by them has been added to their Key Benefits page.
I found the advertiser’s response to my complaints quite interesting and, I think, revealing. To start with, they claim that the printed material was published incorrectly, and contained obsolete material. This seems odd to me, considering that the ad had been approved by TAPS, which requires a fee, and stating that it contained obsolete material implies that the material was once correct, but this certainly does not seem to be the case.
In attempting to substantiate their therapeutic claims, it seems the advertiser provided a clinical evaluation performed by CT Healthcare, which it called “an Australian based manufacturer”. CT Healthcare is the parent company of both Niagara Healthcare and Cyprossage (the company that funded the small trial mentioned on the Niagara website). Here’s what the ASA had to say about that:
The Complaints Board also noted the substantiation provided by the Advertiser which was a “Report Review” on “Vibration Therapy.” It said while the Advertiser provided references on the subject and the claims were of a low level, the Complaints Board were of the view that it did not provide adequate substantiation particularly because the review was not conducted independently.
The advertiser also tried to substantiate their therapeutic claims by providing the ASA with certificates from the Australian Register of Therapeutic Goods (ARTG).
[The Advertising Standards Complaints Board] was of the view that the certificates provided were not categorical evaluations of the product, but rather they confirmed registration of the products.
As well as finding that the therapeutic claims made in their advertisements were not substantiated, the complaints board said that…
The Complaints Board agreed with the Complainant that the lack of the research listed under the heading “Medical Research”, its quality and the fact that some of it had been paid for by the Advertiser was not robust enough to support the statement “much research had been conducted on physical benefits of Niagara’s Cycloid Vibration Therapy since 1954″ as the overall consumer takeout of that statement would be this meant 60 years of independent peer-reviewed medical studies which was not the case.
The most interesting part of this whole thing is, I think, the way in which the advertiser tried to defend their statement that their products have been “Medically proven for 60 years”. Here is how the advertiser tried to justify this statement:
However, to provide clarification regarding the statement on the advertisement Niagara devices have been proven for 60 years, this originates from the basis that CT Healthcare has been involved in medical research relating to the product since 1952.
The complaints board responded to this by stating that the words used in the advertisement simply did not mean what the advertisers claim they meant, and therefore exploited consumers’ lack of knowledge. I think the board’s response was entirely appropriate, and consider such behaviour from a medical advertiser, whom consumers should be able to take at their word, to be utterly reprehensible.
In the end, the complaints board said that both advertisements were in breach of Principles 2 and 3, and Part B2 Requirements 4(a) and 4(b) of the Therapeutic Products Advertising Code. They also said that the website advertisement was in violation of Part B2 Requirement 4(c). Here’s a quick rundown of what those codes are (some paraphrased by me):
- Principle 2
- Must not be misleading and claims must be substantiated
- Principle 3
- Must observe a high standard of social responsibility
- Part B2
- Refers to advertisements for medical devices targeting consumers
- Requirement 4(a)
- Must not be misleading
- Requirement 4(b)
- Must not abuse trust or exploit lack of knowledge
- Requirement 4(c)
- Must not exploit the superstitious or, without justifiable reason, play on fear or cause distress.
You can read the full decision of the complaints board, including my original complaint and the advertiser’s response, on the ASA’s website:
- 13/430 – Niagara Healthcare Newspaper Advertisement
- 13/431 – NIagara Healthcare Website Advertisement
I’ve also uploaded a scanned copy of the print advertisement that you can look at: Niagara Healthcare Herald Insert
Even though the ASA’s Advertising Code of Ethics Basic Principle 1 and its Therapeutic Products Code Principle 1 both require that “All advertisements must comply with the laws of New Zealand”, the complaints board had this to say about the testimonial in the print advertisement:
The Complaints Board noted that compliance with the laws of New Zealand under Basic Principle 1 under the Code of Ethics and Principle 1 of the Therapeutic Products Advertising Code were also raised in the complaint. While acknowledging they are part of Advertising Code, the Complaints Board agreed that whether or not the advertisements complied with the laws of New Zealand was a matter for the Courts.
I’m of two minds about this. For one, I agree that it’s appropriate for the ASA not to overstep their authority, and that the courts are the appropriate place for it to be determined whether or not the law has been breached. However, this precedent effectively makes the first principles of the majority of their codes useless, by placing them outside of their own jurisdiction.
If the complaints board is not willing to consider whether or not an advertisement is in breach of New Zealand law, then the advertising codes should be modified to emulate the relevant laws. These include sections 57 and 58 of the Medicines Act 1981, particularly section 58 subclause (1)(c)(iii), which effectively prohibits the use of testimonials in medical advertisements.
This is a step that has been taken by at least one other New Zealand body that is involved in regulating advertising. The New Zealand Chiropractors Board’s Advertising Guideline section 3(f) prohibits the use of testimonials, in accordance with the Medicines Act.
In my opinion, perhaps the most important aspect of this complaint, taking into account that it was upheld, was that the print advertisement had been approved by TAPS. Even though the complaints board found that the advertisement was full of misleading claims that weren’t backed up by the required evidence, the advertiser was able to convince TAPS to approve this ad for publishing.
Another complaint (not one of mine) about an advertisement approved by TAPS was also recently upheld on the basis that it contained unsubstantiated therapeutic claims: Complaint 13/372 against BioMag.
The business of “natural health” rests heavily on the use of testimonials. They are used in advertisements by people selling therapeutic products and services, and you’ll hear them as anecdotes from people that you know telling you what worked for them. Intuitively, it makes sense to trust in this sort of experience, but unfortunately testimonials and personal experience are not good ways of evaluating a treatment option.
I don’t expect you to take my word for this. Maybe you were told by a doctor that you’d need an operation, then you had reiki therapy and after that your doctor said the problem was no longer there. Perhaps your first child had terrible teething troubles, but on your second child you used a Baltic amber teething necklace and they didn’t have the same problems, but you swear if you forget to put it on them they become agitated. Or maybe you’ve been spraying a colloidal silver solution onto the back of your throat whenever you feel a cold coming on and you haven’t been sick in years. Who am I to doubt or deny your experience?
These are all testimonials that I have heard personally, not from advertisements but from individual people relating their own experiences to me. But still, I remain unconvinced that reiki is any more than an exotic twist on faith healing (that is just as ineffective), that Baltic amber teething necklaces are anything but expensive yet inert jewellery, and that colloidal silver is much good for anything other than causing argyria.
In this series of blog posts, I intend to explain to you why I don’t consider anecdotes like these to be useful in drawing any conclusions about therapeutic interventions. But first, I’d like to point out that I am not trying to be dismissive of personal experience. I don’t think anecdotes are all lies, or anything of that nature, and personal experience can certainly be useful in drawing all sorts of conclusions in everyday life. The only conclusion I am arguing for here is that anecdotes are not useful for evaluating the efficacy of therapeutic interventions.
In searching for any truth, we have to be very careful not to jump to conclusions. There will always be a vast number of potential explanations for any observation, and if we really care about the truth then we can’t just pick the explanation that we like the most, or even the one that we think is most likely. Some possible explanations can be ruled out right from the start, if they’re impossible to test, but the explanations that can be tested are known as hypotheses. If we want to determine whether or not one particular hypothesis is correct, we should design and carry out a test that will rule out every other potential cause of our observation.
Note that this method of testing does not prove anything. Instead, it focuses on ruling out everything else, until only one idea is left standing. The key to designing a good test of an intervention is to make sure anything you observe is as unlikely as possible to be due to anything other than the intervention. This means that, in order to design a good test of an intervention, it is important to have a good understanding of what these other potential causes are.
Part 1: After This, Therefore Because of This
There’s a formal logical fallacy that’s usually known by its latin name post hoc ergo propter hoc, which translates to “After this, therefore because of this”. The fallacy is of the form:
- A happened, then B happened
- Therefore A caused B
Of course, the reason why this is a logical fallacy is that it’s entirely possible that something other than A was the cause of B. This doesn’t mean that the conclusion is false, but it does mean that it is not necessarily true.
Anecdotes take the same form as the above example: “I tried treatment X and I got better”. Although experiences like this can result in strong beliefs, the fact that the improvement happened after the treatment does not mean the treatment necessarily helped at all. Instead, the improvement could have been due to a few different things.
Many common health conditions are self-limiting. This means that, left to their own devices, they will almost always go away in time. The common cold is an example of a self-limiting illness. Unless you are seriously immunocompromised, if you catch a cold you will be fine again after a few days. This includes things like the flu, teething, colic, and acne. Pretty much everything that isn’t a chronic illness and won’t kill you is self-limiting.
Regression to the Mean
Even when nothing external seems to be changing, your health is not constant. Instead, it fluctuates over time around a baseline level of health that itself changes over longer amounts of time. This baseline is basically your average health over a certain period of time; the mean. The tendency for your wellbeing to return to this mean after a fluctuation is known as regression to the mean.
This is a picture of 300 random data points generated in Microsoft Excel. Starting with 0, I added a random number between -0.5 and 0.5 to the running total 310 times, and then took a 10 point running average to smooth the resulting curve.
As you can see, even though the changes are all random, trends do form and the data oscillate around a particular mean. Especially over longer periods of time, the data will tend to return to that mean.
I’ve indicated the 2 most prominent downward trends with arrows. As you might imagine, such low points in a person’s health could motivate a person to take a therapeutic intervention in order to reverse this trend. After the intervention, they’ll likely start to feel better, but as you can see by this graph such variations can happen randomly, and it can be very hard to say whether an improvement was caused by something in particular or if it was just the result of regression to the mean.
For example, I get frequent headaches. However, the frequency and intensity of those headaches varies from day to day, just due to random chance. I’d be more likely to decide to seek a therapeutic intervention on a particularly bad day. However, considering that my wellbeing is fluctuating around a mean value I’d expect my headaches to return to their “normal” level, unless of course something has changed to make them worse on average. If I take an intervention and then the next day my headaches are better, how can I know whether it’s due to the intervention or regression to the mean?
Even with illnesses that are not self-limiting, spontaneous remission that has no obvious cause is something that does happen occasionally. I’m not familiar with the data on this, so I won’t go into it in too much depth, but it is worth knowing that even some serious illnesses can get better on their own, so even some sudden recoveries from serious illnesses can happen on their own, whether an intervention has recently been used or not.
As you may have noticed, these things all have a common theme. They describe ways in which health can improve on its own, which make it difficult to tell whether a particular improvement is due to an intervention or if it would have happened anyway. Ideally, in order to tell the difference, we’d travel back in time in order to try without the intervention and see what would have happened in that case, but unfortunately that’s not an option. The next best method is to have what is known as a control that has the same problem but doesn’t get the treatment.
However, as I discussed earlier, health fluctuates on its own. If the person receiving the intervention improves and the person acting as the control stays the same or gets worse, we still can’t be too sure that the intervention was helping. Variations between different people can make outcomes difficult to interpret as well. Like how random fluctuations will tend to return to the mean over longer periods of time, testing more people will smooth over these random variations. The more people we include in both the treatment group and the control group, the better, as having more observations will help us to tell whether any effect we observe is due to random variation or due to the intervention itself.
Having a control group and a large sample size are 2 aspects of a good test of a therapeutic intervention, but that’s not all there is to it. In my next post, I’ll discuss some other potential confounding factors, and how we can modify our test in order to account for them.
2 days ago, the Advertising Standards Authority published 4 more decisions regarding complaints I’d made. Each of these decisions was focused on a different type of product, but they were all therapeutic products. The decisions released were:
- 13/177 Amber Teething Beads – Punga TailsSettled
- 13/180 Energy Bracelet – 6ShooterSettled
- 13/190 Mohdoh – Punga TailsSettled
- 13/011 Infrared Sauna – Innate HealthAppeal dismissed; complaint upheld
Punga Tails is a New Zealand business that, as far as I can tell, is owned and operated by naturopath Lydia Dorotich. It sells products for infants and has a focus on therapeutic products, including Baltic amber teething necklaces.
Mrs Dorotich has made various public statements claiming that Baltic amber can relieve teething, and that she personally recommends them. For example, from one of her listings on Trade Me:
As a qualified Naturopath and Medical herbalist (see my profile), I highly recommend the use of Baltic Amber for teething babies. Many parents have found that by wearing a Baltic Amber teething necklace the symptoms of teething have reduced.
On a Grabone deal some months ago, a customer stated that she was “a little unsure what these are for”. Given that Grabone generally tries very hard to avoid therapeutic claims and had not mentioned any in this deal (although the listing was for “Authentic Baltic Amber Teething Beads”), that confusion can be understood. In response, Lydia posted this:
Amber teething necklaces are to be worn against the skin. When amber is worn against the skin the benefits from the amber are absorbed into the skin and help to soothe the pain and inflammation caused by teething. They reduce drooling, red cheeks, nappy rash, swollen gums, low-grade fevers and sleeping problems associated with teething.
I have already thoroughly dealt with claims such as these in my previous post on amber teething necklaces. If you’re interested in why claims such as these are completely implausible, have a read of that post. Of course, even if these claims were plausible, it would still not be reasonable to believe them given that they are entirely unsupported by evidence.
Although both of these advertisements were in blatant violation of the Advertising Standards Authority’s Therapeutic Products Advertising Code, in that they made unsubstantiated therapeutic claims, they are not the advertisements I have complained about. Instead, I complained about the advertisements on the Punga Tails website itself.
There are various advertisements on the Punga Tails website, and they all link back to the Baltic amber FAQs page. It is on this page that most of the therapeutic claims were made. Here are some of the claims from that page that I highlighted in my complaint:
- “The therapeutic effects of Baltic amber come from the succinic acid contained in it.”
- “Baltic amber warms against the skin, releasing it’s therapeutic properties safely and naturally.”
- “The therapeutic properties of Baltic amber include analgesic, calmative, anti-inflammatory, antispasmodic, expectorant, and febrifuge (reduces fever).”
- “[Baltic amber teething necklaces] can boost the immune system and ease many ailments such as eczema, fatigue, fibromyalgia, carpal tunnel syndrome, migraines, psoriasis, menstrual cramping, pain, all types of arthritis, reduces stress, anxiety and depression.”
- “Baltic amber is a natural analgesic so is ideal for pain relief with no side effects!”
- “[Hazelwood] will help with the teething just as well.”
A couple of other pages on the site also made therapeutic claims regarding these products, such as:
Baltic amber is a natural way to reduce pain & inflammation WITH NO SIDE EFFECTS!
In response to my complaint, Punga Tails changed a lot of the content on their advertisements for Baltic amber teething necklaces. All of the claims that were not on the FAQs page seem to have been removed, and the FAQs page itself had quite a content overhaul. In light of this, the chairman of the ASA decided that the complaint should be considered settled.
When a complaint is settled that means the chairman has decided that as a result of the advertiser’s self-regulatory action “it would serve no further purpose to place the matter before the Complaints Board.”
Disappointingly, the FAQs page is still misleading on the subject of amber teething necklaces. The therapeutic claims it still contains are no longer as explicit, but they are still clearly there. They have generally been changed to claims about what is commonly believed, and references to its “effectiveness” remain. For example:
Succinic acid is the component of amber that is believed to contribute to the beneficial effects on teething.
This could explain why some amber teething necklaces are less effective.
Looking on the bright side for once, in the remnants of the FAQs page there is still one small part that I am mostly happy to see. One small piece of grudging honesty:
Because Baltic amber teething necklaces have not been scientifically proven we cannot make any claims as to their effectiveness.
I’ve uploaded my complaint and the details of the complaints board’s decision for you to read if you’re interested: 13/177 complaint and decision details
A friend on Facebook alerted me to this one. 6Shooter is a deals website, and they’d posted a deal for an “Health Nano Quantum Energy Bracelet/ Wristband”. Given the name of this product, you might not be surprised to hear that it can basically turn you into a superhero. Here are some of the unsubstantiated therapeutic claims I listed in my complaint:
- “It will transmit nutrients and oxygen to cell and expel toxin in our body,people wont have feel cold with hand and feet any more”
- “It will provide energy to blood corpuscle and lower viscosity,then reduce the chance to get cardiovascular disease,heart disease and wind-stroke”
- “Strengthen human body BIO energy field to prevent harmful electromagnetic wave.”
- “Protect us from electromagnetic waves from computer, mobile phone, electrical appliance, telecommunications and so on,recover our body with balance and coordination.”
- “Stabilize oxygen supply in blood, activate blood corpuscle”
- “Provide relief from allergies and respiratory related illnesses.”
- “Normalize hormonal imbalances.”
I also brought up the fact that the advertisement misused a lot of scientific terms. This was relevant to my complaint as the Therapeutic Products Advertising Code requires that:
Scientific terminology must be appropriate, clearly communicated and able to be readily understood by the audience to whom it is directed.
I explained that the product name misused the terms “nano”, “quantum”, and “energy”. It seems quite clear that whoever is trying to sell these bracelets has simply put some sciencey-sounding words in the name to help convince their target audience: innocent people who simply don’t know any better.
The advertisement also refered to a “human body BIO energy field”, yet there is no such thing as far as we’ve ever been able to detect. Finally, I referred the following as “an example of gratuitous use of pseudoscience”:
Negative ion is the basic element to maintain good health.It can neutralize oxidized substance,such as cells.So the cells are revived and improved the immunity of human body.
I also mentioned that the claim that the product would offer protection from “electromagnetic waves from computer, mobile phone, electrical appliance, telecommunications and so on” constituted playing on consumers’ fair without justifiable reason, since there is no evidence to suggest that such electromagnetic radiation is harmful to humans.
This complaint was settled, after the advertiser responded by permanently removing the product from their stock. They also stated that they never intended to mislead consumers.
I think this case is a good example of why it is important to support science information and to point out pseudoscience for what it is. There are many products such as these which rely on people’s ignorance in order to convince them by sounding like science without having any of the actual substance. If more people can be educated in how to tell the difference between science and pseudoscience, scammers like the people who make and sell these bracelets will have less of a chance for success.
I’ve uploaded my complaint and the details of the complaints board’s decision for you to read if you’re interested: 13/180 complaint and decision details
After submitting my complaint about their amber teething necklaces, I found this product as well on the Punga Tails website. It’s basically smelly playdough, which is claimed to provide certain specific health benefits via aromatherapy and colour therapy. I submitted a complaint about this not so much because I was worried about health fraud, since this product appears to be harmless in that aspect, but because I also feel committed to fighting pseudoscience.
Similar to their amber teething necklaces advertisements, the majority of claims here were on an FAQs page. The misleading information here mostly related to aromatherapy and colour therapy, instead of being specific to the products being advertised, although there were a few specific claims as well. Here are a couple of the most egregious pseudoscience that were on that page:
- “Colour therapy is a holistic and non-intrusive form of healing, which introduces the optimum balance of colour energies into the human organism in order to promote harmony between the body, mind and spirit.”
- “If our energy centres (Chakras) become blocked or depleted, then our body cannot function properly and this, in turn, can lead to a variety of problems.”
The products also used to have specific indications. Bizarrely, even though they appeared to be marketed toward infants (including being in the “Babies Natural Care” section), one of the products was given the following indication:
Helps you quit smoking
This complaint was settled, after the advertiser removed most of the pseudoscience and therapeutic claims from the page. I was happy to see the introduction of the following disclaimers:
While Aromatherapy is not scientifically proven…
Colour therapy is not a scientifically proven therapy.
I’ve uploaded my complaint and the details of the complaints board’s decision for you to read if you’re interested: 13/190 complaint and decision details
Recently, I wrote about 3 complaints against Innate Health. One of these complaints, number 13/011, was about an advertisement in Coffee News for an infrared sauna that made various unsubstantiated therapeutic claims, and bizarrely stated that the product could:
Activate every cell in your body to increase your sense of well-being
The complaint was originally upheld on the 18th of March. On the 9th of May, I received a notice from the ASA saying that an appeal that had been submitted on this complaint had been accepted. This doesn’t mean the complaint had been successfully upheld, it just means that the appeal will be heard by the complaints board and they would then decide whether or not their decision should be changed. I was contacted because, as the complainant, I was to be given an opportunity to respond to this appeal.
The appeal claimed to present new evidence, and was basically just one really long citation of a bad review. I got the impression that Barbara Good Hammond, owner of Innate Health, came across this article some time after the complaint was upheld and thought she saw an opportunity to have the decision changed.
The article in question is a review published in Alternative Medicine Review in 2011, entitled Sauna as a Valuable Clinical Tool for Cardiovascular, Autoimmune, Toxicant-induced and other Chronic Health Problems. It was written by a naturopath called Walter Crinnion.
The article was a (non-systematic) review of the evidence regarding saunas as a therapeutic intervention. It seemed to rely very heavily on pilot studies and unpublished research, and seemed to be a rather unreliable source.
In the original complaint, the complaints board decided that “Activate every cell in your body” could be considered puffery. This means that it is obviously meant to be a ridiculous exaggeration and would not be taken seriously by anyone, so does not require substantiation. As part of the appeal, the advertiser disagreed with this analysis, claiming that the statement is not puffery and could be substantiated. They then proceeded to attempt to substantiate the claim:
With the whole body being effected [sic] during therapy, circulation is enhanced to every cell. The modality of Far Infrared sauna therapy has been substantiated world wide [sic] for improving cardia-vascular [sic] function, and with the heart being directly responsible for pumping and circulating blood to every cell in the body (as reported in Anatomy & Physiology books and being general knowledge) this would have a direct impact to increase the bodies [sic] sense of well-being, as proven and shown in published clinical studies worldwide.
She then went on to quote a paragraph from Crinnion’s review that seemed to be pretty much irrelevant to the claim, then declared that the statement “is legitimate and should not be considered puffery”.
This appears to be the one thing on which both Ms Hammond and I can agree: that statement should not be considered puffery. Bizarrely, even though the advertiser specifically said the statement is not puffery the complaints board reiterated their previous decision on this statement, saying that it should be considered puffery and therefore doesn’t need to be substantiated.
The rest of the text of the appeal is fairly benign and unconvincing, but there are 38 references listed at the end of it. At first, I thought that seemed pretty intimidating. However, after a minute or so I noticed that they were copied and pasted directly from the review, including spelling mistakes and formatting errors. Worse than that, many of them were duplicates, and there were actually only 25 unique references.
I got the distinct impression that Barbara Good Hammond didn’t even attempt to find these references to look at them herself. I’m not sure how else I can explain the fact that she cited the same piece of unpublished research 3 separate times. Nonetheless, with the help of a contact at the University of Auckland, I was able to get my hands on the full text of 12 of the references, and the abstracts of a further 6 references.
I went through each reference one by one, and found them all to be either irrelevant or inadequate substantiation for the claims made in Innate Health’s advertisement. In their decision, the complaint board seemed to agree with me, stating that:
Turning to the Advertiser’s evidence, the Complaints Board considered the Advertiser had not adequately substantiated the claims. It noted that one research authority was a naturopath, not a doctor or scientist, and the saunas about which the research was done were not infra red saunas but Finnish saunas. While the Complaints Board noted the research said there may be some merit in using Finnish saunas, this did not reach the threshold to validate the very strong claims in the advertisement, particular [sic] as that research discussed a different type of sauna.
If you’re interested, you can read the entire appeal application and my response: Complaint 13/011: appeal 13/013, response to appeal, and decision details
On the 3rd of January 2013, I picked up a copy of the Whangarei edition of Coffee News. This is a weekly publication that has different editions for various areas of New Zealand, and seems to contain a relatively high proportion of questionable advertising. I make a habit of looking at it whenever I get the chance, in case I find something I should complain about. In this case, I found what I was looking for.
I wasn’t familiar yet with the product being advertised, but the claims sounded too good to be true so I thought it certainly warranted further investigation.
Once I got back to my computer, I looked up the Innate Health website. There I found other advertisements like the one I’d seen in Coffee News. There was one advertisement for an “Ionic Detox Foot Spa” (scroll down on that page), and another for a “Quantum Magnetic Analyser Report“.
Both of these advertisements were concerning. Fraudulent “detox foot spas” and similar pads, which change colour and appear to gunk up after placing your feet in them and are claimed that they’re removing “toxins” are nothing new. There are a few varieties, but they have essentially been shown to be fake:
As for their “Quantum Magnetic Analyser Report”, the way they described it didn’t even sound superficially plausible, and the spattering of pseudoscientific language didn’t particularly help their case. The product seems to be a hand grip connected to a computer, and allegedly when you hold onto the grip the computer will tell you what’s wrong with you.
So far, I have laid 3 complaints against Innate Health with the ASA.
Ionic Detox Foot Spa – Complaint 13/010
Innate Health made a lot of claims about what their foot spa could help with:
- Joint Pain, Arthritis
- Chronic Fatigue
- Foggy Brain, Poor Concentration
- Poor Circulation
- Heavy Metal Toxicity
- Eczema, Psoriasis
- Hormonal Imbalance
- Low Sex Drive
- Weak Immune System
- Cellular Acidosis (Ph [sic] is too high)
There was absolutely no attempt made to substantiate any of these claims, making them in violation of the Therapeutic Products Advertising Code Principle 2.
The advertisement also used scientific terminology in an inappropriate way, attempting to take advantage of consumers’ lack of knowledge by talking about such sciencey sounding things as
balancing the body’s natural energy system
utilizing principles of reflexology and the science of ionization and osmosis, create a positive cellular environment
Because these statements used scientific terminology in an inappropriate way, I considered them to be in violation of the Therapeutic Products Advertising Code Part B2 R4.3. I also considered them to be in violation of Part B2 Requirement 4(a) and 4(b), because they abused consumers’ lack on knowledge in order to deceive them.
I also argued that the advertisement violated Principle 3 of the Therapeutic Products Advertising Code, which requires that advertisements such as this exhibit a high standard of social responsibility.
In response to the complaint, the Innate Health representative seemed to think that it was sufficient to explain to the ASCB that anyone wanting to buy their products could just search on Google for the product name and research it themselves. Instead of attempting to substantiate their claims they just told the ASCB how many results Google said it found when they searched for their product name. I have no idea why they thought this was a good idea, especially given that the ASA’s website has information available on how to respond to a complaint regarding misleading claims, but I guess that’s just what they decided to do.
In complaints like this the onus is always on the advertiser to substantiate their claims. As they had failed to do so convincingly, the ASCB upheld the complaint and ruled that the advertisement be removed.
However, this complaint was a bit more interesting than that. Innate Health had actually already removed the advertisement from their website before the ASCB met to make a decision about the complaint. Normally, at least in my experience, this would result in a complaint being considered settled. However, the ASCB did something for which I commend them in this case:
The Complaints Board noted the advertisement for this product had been removed from the Advertiser’s website, but considered it was in the public interest the issues raised by the Complainant be addressed.
I hope that this precedent may be useful in future complaints that involve advertisements with a short lifespan, such as GrabOne deals, which may be removed before the ASCB is able to meet and potentially uphold a complaint against them. If you would like to refer to this precedent in a complaint of your own, remember this is complaint 13/010.
I’ve uploaded the details of this decision, which includes my original complaint, for you to read: 13/010 Decision Details
METAbolic Infrared Sauna Chamber – Complaint 13/011
The advertisement in Coffee News was similar to Innate Health’s other advertisement, in that it made various unsubstantiated health claims and misused scientific terminology. The line in particular which grabbed my attention was:
Activate every cell in your body to increase your sense of well-being
Aside from that, various therapeutic claims were also made in the advertisement, without any attempt to substantiate them. Apparently the product will:
- Promote weight loss and burn calories
- Enhance metabolism, immune system and blood circulation
- Assists with removal of toxins, heavy metals, lactic/fatty acids
- Promote relaxation, reducing stiffness, pain and fatigue
Honestly, I don’t doubt that sitting in any sort of sauna, including an infrared one like this, could be relaxing. The other claims though, particularly the claim about helping remove heavy metals and other “toxins”, were quite concerning.
As always, before I made this complaint I searched to see what evidence I could find. I didn’t find much at the time, but I did manage to find a 2008 article from the LA times that quotes relevant experts saying that, as I’d expected, sweating is not a valid method of treating heavy metal toxicity. Here’s a link to the article: You sweat, but toxins likely stay
My complaint argued that the claim about “activating” your cells was an inappropriate use of scientific terminology, that the therapeutic claims made were unsubstantiated, and that it was highly irresponsible to imply that a serious condition like heavy metal poisoning could be treated in this way.
Like in the previous complaint, the Innate Health representative’s tactic involved searching for their product on Google and reporting the number of results, so the ASCB upheld the complaint.
Unfortunately, though, instead of agreeing with me that “Activate every cell in your body” was a misuse of scientific terminology, the ASCB decided it should be treated as “puffery”. As far as I understand, that basically means a claim that is exaggerated to the point of appearing so ridiculous that no one could possibly take it seriously, and therefore doesn’t require substantiation. I disagree with their decision here, but I got the result I was after overall.
I’ve uploaded the details of the ASCB’s decision, including my complaint, for you to read: 13/011 Decision Details
Quantum Magnetic Analyser Report – Complaint 13/012
I found this one very amusing. Here’s how the product was originally described in the advertisement:
Quantum Magnetic Analyzer collect the weak magnetic field sensors of frequency and energy from the human body through the hand grip sensor. The instrument magnifies your body functions and the computer processes the information collected and compares with the diseases on record which are installed inside the instrument and compares against the standard spectrum. According to the results, the therapist is able to make an analytic judgement for the person and put forward standard advice for any preventive treatment that may be required.
The complaint about this ad focused on similar points to the previous ones: its therapeutic claims are unsubstantiated, it’s socially irresponsible, and it’s taking advantage of consumers’ lack of knowledge.
You can probably guess how the Innate Health representative responded to this complaint. That’s right, they searched for their product on Google and told the ASCB how many results they got. In light of this, the complaint was upheld.
Despite this, and despite the fact that having a complaint upheld against you by the ASA means you are asked to remove the advertisement, Innate Health has not removed the advertisement. I have submitted a new complaint to the ASA regarding this.
I always make records of the original state of advertisements against which I complain so I can tell what, if anything, is changed. For online advertisements, I use this Chrome extension: Screen Capture (by Google). In this case, the important changes were that the latter half of the advertisement was changed, removing some unsubstantiated claims and adding a new one, and a disclaimer was added. I don’t really know how Innate Health could have decided this was an acceptable route for them to take, because the ASCB clearly stated this in their decision:
The Complaints Board said that even if there had been a disclaimer, that alone would not remove the responsibility to provide substantiation.
As always, I’ve uploaded the decision details and my complaint for you to read: 13/012 Decision Details