Evaluating Different Modalities

How Comparative Effectiveness Studies Work

By Jerrilyn Cambron
[Somatic Research]

There are many different types of massage, and each therapist has an opinion on which is the most effective. But what does the research say? Can we know what type of massage is right for a given condition? One way to determine which technique is most beneficial is by a comparative effectiveness trial: a randomized clinical trial that compares one treatment to another.
The traditional randomized clinical trial compares one treatment to a placebo control. More and more researchers are instead moving toward comparative effectiveness trials because not only do they want to know if a treatment is beneficial, they want to know if it is more beneficial than something else.
In 2011, Daniel Cherkin and his colleagues completed a comparative effectiveness trial at the Group Health Research Institute, comparing the effect of two types of massage against usual care for chronic low-back pain.1 Subjects were randomized into one of three groups: usual care or 10 weekly sessions of either structural massage or relaxation massage. At the end of the 10 weeks, subjects in both massage groups had reduced pain and disability, with the improvements lasting at least six months. People with back pain may find it encouraging that both types of massage were effective, but as therapists who want to better understand what this study means, we need to dig a little deeper.
The first things to consider in a comparative effectiveness trial are the ability of the health-care professionals, and the actual treatment provided to the subjects in each of the three groups. In Cherkin's study, there were 27 licensed massage therapists. Each had at least five years of experience and underwent one and a half days of protocol training for the study. This gives us the sense that there was capability among the group and consistency in the treatments.
The relaxation massage group received weekly 50–60 minute massages that included “effleurage, petrissage, circular friction, vibration, rocking and jostling, and holding.” Each body part was only treated for a specific time period; for example, the back and buttocks were allowed 7–20 minutes of treatment time.
The structural massage group received the same weekly amount of massage, but the therapists were allowed to treat the clients as they saw necessary, focusing on alleviating each individual’s personal contributors to back pain, with no preset protocol.
The usual care group did not receive any massage within the study and received their typical medical care, if and when desired.
Using a predetermined massage protocol in a study helps the reader know exactly what was done during the session. This control over the study design is one of the main benefits of a clinical trial: we know exactly how the massage was performed on every subject in that group. It also allows therapists in the field to replicate the massage in their own practices. However, a structured massage format is not true to life. Every massage therapist approaches their massage practice differently, and every client has varied needs. What might be beneficial to one client may not help another. This trial included one group where the massage was predetermined and one group where the treatment was based on specific client needs, so the study actually had the best of both designs, although for different forms of massage care.
Comparative effectiveness studies are like any clinical trials in that they need to specify the treatment rendered. When reviewing a study, ask yourself if you are able to replicate the treatment or at least have a good sense of what was done with the subjects. If this information is not available, applying the study results to your practice may be difficult.
Most massage therapists are willing to treat any person who is in need of care. Comparative effectiveness trials are different: to participate in the study, subjects must meet a pre-set list of criteria.
The subjects in the Cherkin study were 20 to 65 years old, had outpatient diagnoses suggesting nonspecific chronic low-back pain lasting at least three months without two or more pain-free weeks, and their back pain bothered them at a rating of at least 3 on a scale of 0 to 10. Potential subjects were excluded from the study if they had specific causes of back pain such as cancer, fractures, or spinal stenosis; complicated back problems such as sciatica, back surgery in the past three years, or medicolegal issues; conditions making treatment difficult, such as paralysis or psychoses; conditions that might confound treatment effects or interpretation of results, such as severe fibromyalgia or rheumatoid arthritis; inability to speak English; having received a massage within the past year; or had plans to visit a provider for back pain.
With each criteria, the group of people who are eligible for the study becomes smaller and smaller. The reason researchers create so many criteria is to decrease any factors that might confound the trial’s results. For example, subjects who are over 65 years old might respond slower than those who are younger. Spinal fractures might lead to a need for modification of the massage treatment protocol. People with psychosis might complete outcome measures inaccurately.
The inclusion and exclusion criteria within a comparative effectiveness trial work to standardize the subject base. However, when applying what we have learned from the research results, we have to ask ourselves if the study’s subjects are anything like our own clients. For example, study results from adult subjects may not directly apply to teenage clients. When reading the literature, paying attention to the inclusion and exclusion criteria will help you better understand to whom the results may apply.
Outcome Measures
The outcome measures used in a comparative effectiveness trial are based on what the researchers are interested in assessing. Some studies might be biomechanical in nature, with range of motion or gait measures being the main outcomes. Other studies may include anxiety or depression measures.
The outcome measures in the Cherkin study included pain (measured by the bothersomeness scale) and disability (measured by the Roland Disability Questionnaire). These measures are paper-based, easy to use, and free, making them beneficial to massage therapists, as they can be used in any clinical practice and the results can be compared with those in the research study. Subjects were assessed at the beginning of the study and at 10, 26, and 52 weeks. Treatment stopped at the end of 10 weeks, aligning with the first post-treatment assessment. The measures at 26 and 52 weeks were used to determine long-term effectiveness. Most comparative effectiveness trials will have several post-treatment outcome measures to determine the immediate effects of treatment as well as the lasting effects. Timing of outcome measures may greatly affect the results.
Results are assessed statistically to determine if there are any group differences. Some researchers also focus on the clinical significance of their results. Clinical significance deals with the amount of improvement or the clinical effectiveness of treatment. Statistical significance and clinical significance are two very different aspects of a study. Results could be statistically significant without having any clinical significance. In the Cherkin study, the researchers defined clinical significance in regards to dysfunction as an improvement of at least 2 points on a scale of 0 to 23, with 23 being the most dysfunctional. In regards to pain, a clinically significant change was defined as an improvement of at least 1.5 points on a scale of 0 to 10, with 10 indicating the most bothersome pain. Even if the study did not meet statistical significance, clinically significant changes would still be important.
In our clinical practices, we only focus on clinical significance. Studies that discuss levels of clinical significance are helpful to clinicians in that we can determine if our clients are reaching the same level of improvement when using the same measures.
In the Cherkin study, 402 subjects were randomized into three different groups. All groups improved in pain and dysfunction over the 10 weeks, but the improvement was statistically significantly greater in both massage groups compared to usual care. There was also a clinically significant change in dysfunction with an improvement of 3.6 (on a scale of 0 to 23) in the structural massage group and an improvement of 5.6 in the relaxation massage group. There was a clinically significant change in pain of 1.8 out of 10 in the structural massage group and 2.1 in the relaxation massage group. No such clinically significant change occurred in the usual care group.
The results of this comparative effectiveness study indicate that both relaxation and structural massage treatments are beneficial for individuals with chronic low-back pain. These results can be directly applied to a massage practice by sharing this information with prospective clients, as well as with referring physicians. These results can also be compared to the results achieved within your own practice when using the massage protocols described in this study.  
However, as a comparative effectiveness trial, these results do not give us an indication that one massage treatment is better than another. When compared, the two massage groups proved to be equally effective, even though most of us would have expected massage techniques tailored to the individual would have better results than a standardized massage protocol.
These results raise many questions: Does the type of massage really matter in terms of treatment effectiveness? Should all therapists use predetermined massage protocols for treatment of chronic low-back pain? Would another form of massage treatment have been more effective than the types included in this study? These questions will certainly lead to additional comparative effectiveness trials. But in the meantime, both relaxation massage and structural massage have been demonstrated to be beneficial for chronic low-back pain; therefore, either form of massage can be recommended.

1. Daniel C. Cherkin et al., “A Comparison of the Effects of 2 Types of Massage and Usual Care on Chronic Low Back Pain: A Randomized, Controlled Trial,” Annals of Internal Medicine 155, no. 1 (July 5, 2011): 1–9.

To read this article in our digital issue, click here.