"Extraordinary claims require extraordinary evidence." - Carl Sagan

Mental health professionals are inundated with aggressive promotional and marketing efforts designed to sell them new treatment technologies and philosophies.  These treatment methods are more often than not promulgated without formal evaluation or supportive research. In effect, the only testing that occurs is the use of these "new" approaches on an unknowing or unaware public. Here, we call attention to these trends and evaluate the evidence for their claims.

Evidence-based Practice: It it "all that?"

Despite the overwhelming support for the dodo bird verdict and all the research endorsing client and alliance factors, the mental health field remains dangerously enamored of flashy techniques and the promise of miracle cures. Clinicians are regularly bombarded by what’s new and different by workshop brochures and book announcements. Of late the call has been for the ultimate, all-powerful silver bullet: the evidence based treatment. This is the empirically bankrupt notion that for a particular problem, there is a specific treatment that is best. In actuality, to be designated "evidence based," the developer of a given approach need only prove it's superiority to placebo in two studies.

Superiority over placebo, however, is not really saying that much; psychotherapy has demonstrated its superiority over placebo for nearly 50 years! Therapy is about twice as efficacious as placebo, and about four times better than no treatment at all. This research, for all its pomp and circumstance, tells us nothing that we already do not know: Therapy works!   Further, demonstrating efficacy over placebo is not the same as demonstrating efficacy over other approaches. Recall the dodo bird verdict. There is no differential efficacy among approaches. Why are studies funded that tell us what we already know? Why do EBT proponents seem to pretend that efficacy over placebo means that they are better than other treatments?  When differential efficacy is claimed, be suspicious. In reality, studies which find a difference are actually no more in number than one would expect from chance. Further, closer inspection of studies that claim superiority reveals two major issues that must be considered: allegiance effects and unfair/indirect comparisons (Wampold, 2001). 

Allegiance effects are those in which superior outcomes are attributable to the therapist or researcher’s affinity for the treatment being assessed.  By the way, allegiance effects account for as much as 70% of treatment effects.  As an example, consider Emotionally Focused Therapy (EMT), the EBP queen of marital therapy.  In a testament to hyperbole, EMT is being touted as “the revolution in couple therapy” because it demonstrates “the best outcomes” (Johnson, 2003, p. 363). The developer cites but four studies to support the claim that the dodo bird verdict is dead (2003, p. 367). 

Setting aside the obvious overstatement regarding a revolution--recall, model factors account for just 1% of outcome variance--let’s consider the claim of “best outcomes.”  First, all but two EMT studies involve demonstrations of efficacy over placebo or no treatment, and are not comparisons with other bona fide couples treatments. All therapies can make similar claims of best outcomes when compared to no treatment. The remaining two studies did look at differential effects. The first pitted EMT against problem-solving intervention (PS) (Johnson & Greenberg, 1985), a questionable direct comparison to another model of therapy. In this study, EMT showed a significant difference over PS--but only on 4 of 13 measures at termination!   Moreover, at 8-week follow-up, these meager results became even less remarkable—only 2 measures showed superiority of EMT over PS.  Underwhelming, huh?  Now consider the claims of "best outcomes."  With one exception, ALL of the studies published to date have been conducted by the very person who developed the approach.  Can you say, "allegiance" here?  Finally, the one study not conducted by the developer (Goldman & Greenberg, 1992) (conspicuously absent in the "revolution" article by the way) was a head-to-head comparison between EMT and an integrated systemic approach (IST) and--suprise, surprise--found no difference in outcome--although at follow-up, IST had an advantage.  Bottom line: the dodo bird still rules. 

Goldman, A., & Greenberg,L. (1992). Comparison of integrated systemic and emotionally focused approaches to couples therapy. Journal of Consulting and Clinical Psychology, 60, 962-969
Johnson, S.M. (2003). The revolution in couple therapy: A practitioner-scientist perspective. Journal of Marital and Family Therapy,29, 365-384.
Johnson, S.M., & Greenberg, L.S. (1985). The differential effects of experiential and problem solving interventions in resolving couple conflicts. Journal of Consulting and Clinical Psychology, 58, 175-184.
Johnson, S., Hunsley, J, Greenberg, L, & Schindler, D. (1999). Emotionally Focused Couples Therapy: Status and challenges. Clinical Psychology, Science, and Practice, 6, 67-79.
Wampold, B. E. (2001). The great psychotherapy debate: Models, methods, and findings. Mahwah, NJ: Erlbaum.

More Baloney...