Don’t Take Their Word for It!

About a month ago, I ran into an excellent example of the importance of scepticism. A business card for a local gym was dropped off at my workplace, and they had a “Vibration Training” section on their website. Of course, my interests being as they are, I find myself entirely unable to resist anything involving the word “vibration”, so I took a look.

This page made 2 referenced claims:

the benefits have been well proven to match and surpass conventional forms of training – especially when time is factored in.*

*Roelants, Delecluse, Goris, Veshueren, 2004

A German skin clinic noted that the appearance of cellulite reduced on average 25.68%** with vibration training alone. When combined with conventional cardio exercise this percentage rose to 32.3 % (with a considerable increase in the total time spent exercising).

** Sanderm, 2003

The second claim, in particular, seemed strange to me. I had no idea how cellulite could be measured in a quantifiable way, but even allowing for my lack of knowledge here the figure 25.68% seemed far too specific. Something smelled fishy.

The next step, of course, was to look for the references themselves. Unfortunately, no links were provided on that page. Frustratingly, both references were also misspelled. The name of the first reference’s last author is actually Verschueren, not Veshueren, and the German skin clinic is called Sanaderm, not Sanderm (as far as I can tell, its fully qualified English name is the SANADERM Professional Clinic for Skin Disease and Allergology).

The first study wasn’t too hard to find, being available on PubMed and having been published in the International Journal of Sports Medicine. Here’s a link to the paper – Roelants et al.

The Sanaderm study, however, didn’t seem to be published anywhere. Eventually, however, I managed to find what appeared to be the study in its original German, on the website of another gym offering vibration training – Anti Cellulite Untersuchung.

Unfortunately, I don’t speak a lick of German, so the only way in which I’ve been able to read the study is via Google Translate (here’s a link to the translated study).

The first thing I looked for was how cellulite was measured. To my disbelief, I found that it was not measured in any quantifiable way. Instead, subjects were sorted into 1 of 3 qualitative categories. Here is the relevant passage from the machine translated study:

Defined for this study stages of cellulite

Stage 1
Cellulite is not normally visible. But if the skin Thigh, buttocks or belly with hands is pressed together, dents appear honeycomb.

Stage 2
Cellulite is not pushing together of the skin standing visible.

Stage 3
Cellulite is not pushing together of the skin lying visible.

The word order of the machine translation is unfortunately less than perfect, but I think the meaning is clear enough.

So now my question was how on Earth could they possibly get from a qualitative measurement of cellulite to so precise a measurement as a 25.68% decrease?

Unfortunately, although their results section seems to include all other measurements for each individual subject, the “Cellulite Degree” categorisation is only given for 3 individuals selected (the method of selection is not given) from each of the non-control groups.

Interestingly, one of these example subjects is given a “Cellulite Degree” categorisation of 2.5, despite this being an undefined value. I have not been able to find a justification for this datum anywhere in the study.

The appallingly bad method these researchers have used to compare their groups is to assign arbitrary values (1, 2, and 3) to each of the “Cellulite Degree” categories. It seems these values are associated with the category names of “Stage 1”, “Stage 2”, and “Stage 3”. To illustrate just how inappropriate these values are, I’d like to point out that the categories could easily have been named “A”, “B”, and “C”. There seems to be no justification anywhere in the study for the particular values that have been assigned to each category.

The researchers then added up these values for an entire group, and compared the totals of the first and final tests (omitting the results of the intermediate test that took place halfway through the study).

The sum of these values for the first test of the group that only took part in exercise on the Power Plate vibration machine (Group 1) was 46.50 (why they felt the need to record the result to 2 d.p. is entirely beyond me, and it is not due to the machine translation) and the final result was 37.00. Using these values, which are both almost entirely arbitrary, they seem to have performed the following calculations:

 46.50
-37.00
= 9.50

  9.50
 37.00
=25.68%

Note that this is absolutely not the calculation that should have been done. By this same calculation, if I start with 1 apple and eat half of it, it has decreased by 100%.

The other calculation, done for Group 2 (the group that participated in a cardio workout routine after the same vibration exercise as Group 1) appears to be of the same form:

 43.00
-32.50
=10.50

 10.50
 32.50
=32.30%

The values that they would have gained from the correct calculations would not have been 25.68% and 32.30%, but 20.43% and 24.42%, each of which is massively less. Also recall, of course, that these percentages were still gained using the unjustified values assigned to each qualitative category.

Every single calculation resulting in a percentage change seems to have been performed in the same way, resulting in every case in a result that is wrong and overstated. They also appear to have truncated results instead of employing rounding, although the effect of this mistake is to slightly decrease results (at most by 0.01%, e.g. 1.22% instead of 1.23%).

Aside from all this, it’s also worth noting that a control group of 5 is abysmally small. On top of that, the average initial cellulite categorisation (which I realise is arbitrary, but it’s the only indication available to me of each group’s cellulite) for the control group is lower than for the other 2 groups.

The average initial value for the control group is 1.5. My first calculations for the average initial values for Groups 1 and 2 resulted in 1.94 and 1.39, respectively, but then I realised that the results tables listed “Number of volunteers” as though no participants had dropped out.

From earlier in the results section I could find that 1 participant from Group 1 and 6 participants from Group 2 dropped out. Taking these data into account, the actual average initial values for Groups 1 and 2 were 2.02 and 1.72, respectively.

It’s hard to compare these values due to their arbitrary nature, but I feel it’s worth noting that they are both significantly above the average value of 1.5 for the control group.

This study, in particular its value of 25.68%, seems to be quoted practically everywhere the product used (Power Plate) is advertised. Here are some examples I found (this list is far from exhaustive, you’ll be able to find many more by searching for such things as “25.68% vibration”):

A page on the Power Plate website that refers to the value in its Figure 1 – Power Plate® Training Helps Reduce Cellulite
A PDF on the French Power Plate website that claims (emphasis theirs): “an average reduction of 25,68 percent was achieved in the Power Plate group” – Research into the effects of vibration training in cellulite
A PDF on the website of the Windsor Spine Centre (a chiropractic organisation), that appears to have been created by Power Plate, mentioning “a 25.7% reduction of cellulite” and “a 32.3% reduction of cellulite” – Defeating Cellulite
Another PDF that also appears to have been produced by Power Plate that I found on the Body Smart website, mentioning the same phrases as the PDF on the Windsor Spine Centre website – Whole Body Vibration Helps Reduce Cellulite
A PDF on the Regeneration Point website (which peddles some very dubious sounding products, mainly Pulsed-Electro-Magnetic Technology) that makes the specific claim that “Vibration training has been shown in the laboratory to provide the following benefits: Cellulite -25.7%” – Benefits of Vibration Training
A post on the Vibrant Health Wellness Centre blog, Beyond the Great Abyss, which mentions the 25.68% and 32% figures – Losing Weight with Whole Body Vibration
A page on the Vibrant Health Wellness Centre’s website mentioning the same figures as the blog post – Whole Body Vibration—Weight Loss
A page on the Tropical Sensations site that makes the following claim: “Research shows that with just 3 sessions a week (11 hours total over 24 weeks) it’s possible to reduce cellulite on your thighs and buttocks by a massive 25.7%!” – Vibration Technology from Tropical Sensations in Dartmouth
The Vibration Training page on the website of The Exercise Room, which led me to this study, mentions the values of 25.68% and 32.3%.

I’d also like to mention that the participants in this study were all females aged 25-45, yet none of the places where I saw it referenced recognised that its conclusions should not be applied to males or to females outside of that age range.

Also, as far as I can tell from the machine translation, the control group underwent no training, so the difference attributable to the Power Plate vibration equipment cannot be assessed from this study. Instead of comparing exercise to exercise+vibration, they have compared no exercise to exercise+vibration.

And, of course, I have not “debunked” or “disproven” the type of training done in this study. All I have done is criticise a particularly bad study.

Hopefully the title and contents of this post are enough to make the conclusions that you should draw quite clear. Just in case they’re not, though, let me state them explicitly: