Saturday, December 20, 2008

Verbatim: Three conclusions about autism interventions

Five papers published or epublished so far in 2008, along with a sixth which has sat in press all year, have in one way or another reviewed the ABA group designs, sometimes alongside other approaches to autism.

The solitary RCT (where the intended comparison between randomized groups was actually carried out) in the 47-year history of ABA-based autism interventions--a small, largely failed study--is now drowned in an enormous murk of systematic, quasi-systematic, and not-systematic-at-all reviews. At this point in autism and autism advocacy history, no one should be surprised that this phenomenon encompasses dueling meta-analyses.

As with every other Verbatim, providing quotes doesn't mean I agree with them or the papers they are situated in. Here are quotes from the conclusions of three of the 2008 reviews:

Intervention studies suffer from methodological problems that preclude definitive conclusions regarding their efficacy. [...] While this review suggests that Lovaas may improve some core symptoms of ASD compared to special education, these findings are based on pooling outcomes from a few, methodologically weak studies with few participants and relatively short-term follow-up. [...] Future studies on the effectiveness of these interventions need to be more rigorous. (Ospina et al., 2008)
Given the few RCT studies, the few models that have been tested, and the large differences in interventions that are being published, it is clear that the field is still very early in the process of determining what kinds of interventions are most efficacious in early autism, the variables that mediate and moderate treatment effects, and the degree of both short-term and long-term improvement that can be expected for an individual child. (Rogers & Vismara, 2008)
Currently there is inadequate evidence that ABI [applied behavioral intervention] has better outcomes than standard care for children with autism. Appropriately powered clinical trials with broader outcomes are required. (Spreckley & Boyd, in press)

References:

Ospina, M.B., Krebs Seida, J., Clark, B., Karkhaneh, M., Hartling, L., Tjosvold, L., Vandermeer, B., & Smith, V. (2008). Behavioural and developmental interventions for autism spectrum disorder: a clinical systematic review. PLoS ONE, 3:e3755.

Rogers, S.J., & Vismara, L.A. (2008). Evidence-based comprehensive treatments for early autism. Journal of Clinical Child and Adolescent Psychology, 37, 8-38.

Spreckley, M., & Boyd, R. (in press). Efficacy of applied behavioral intervention in preschool children with autism for improving cognitive, language, and adaptive behavior: A systematic review and meta-analysis. Journal of Pediatrics.

8 comments:

Tim said...
This comment has been removed by a blog administrator.
Michelle Dawson said...

I've removed a comment from Tim Welsh, self-described autism advocate, because it was spam (and very badly placed spam).

Estee Klar said...

I can vouch for it on a personal level with Adam... who is approaching the age of seven. He began typing this year...now he is completely independent...he is talking, without forcing or Verbal Behaviour, and he is learning and growing without any particular intervention at all.

He is just learning and growing with the rest of us.

Lisa Jo Rudy said...

It seems to me that before one can determine whether a therapy is "efficacious" one must first determine the goals of the therapy.

If the goal is "to teach a child to name a fork," I'd guess that most forms of ABA would be effective. If the goal is "to teach a child emotional flexiblity and connectedness," then I'd guess ABA would have no impact whatever.

I've never seen a study in which ABA is compared head-to-head with other therapies such as floortime or RDI, to achieve specific goals. Intuitively it would seem that ABA is more effective in teaching behavioral and self-help skills (self-care, for example)while floortime/RDI is more effective in teaching emotional/communication skills.

But maybe real head-to-head comparisons would surprise us!

Lisa (autism.about.com)

Michelle Dawson said...

In response to Lisa Jo Rudy, as my collaborators and I noted elsewhere (Dawson et al., 2008), researchers have "studied the effectiveness of programs, not the appropriateness of various goals" (NRC, 2001).

If you read the literature (the primary sources), you can find out what outcome measures have been used in the ABA group designs. Some of this information is also reported in the reviews I mention, but as always I suggest reading primary sources. Also, I suggest not making claims about an autism intervention unless you are very familiar with the relevant primary sources.

There are no published (in peer-reviewed journals) papers reporting the results of controlled trials of RDI or DIR/Floortime. But ABA-based interventions have been directly compared to other services. For one recently-published example, see this post.

Also, not all autism interventions are heavily marketed the way ABA, RDI, and DIR/Floortime have been and continue to be. For an example of a manualized non-marketed approach which has a successful published pilot RCT, see Aldred et al. (2004; some information about this study is here).

This successful pilot RCT has been expanded into an MRC-funded 152-child multi-site RCT, which is ongoing. Unlike so many others (see the promotion of ABA, RDI, DIR, etc), those involved in investigating this manualized intervention have been scientifically and ethically responsible, in not making any claims for this intervention before publication of data from a reasonable quality study.

Andrew said...

I have not read the first two studies quoted, however I have read Spreckley and Boyd and was appalled at the basic flaws in the meta-analysis performed.

As an example, the Sallows and Graupner study states that there was a clinic based group and a home based group. Spreckley and Boyd instead class them as a study group and a control group. Both groups got good results, however the home based group had better results in a majority of areas measured. So this study sould be regarded as having a positive outcome but be open to criticism for not having a proper control group.

Instead, Spreckley and Boyd lists S&G research as having a negative result, as the "control" group did better than the "study" group. This significant negative result weighs heavily on the weighted total taken for the remaining studies that survived the author's initial filtering process for quality research.

If on the other hand, Spreckley and Boyd had instead assumed that the home based group was the study group and the clinic based group was the control, the research would have instead scored a positive result and the overall outcome for the metastudy would also have been resoundingly positive.

The Spreckley and Boyd paper is an awful piece of work with numerous methodological issues. I'm amazed it made it into a peer reviewed journal.

Michelle Dawson said...

In response to Andrew, after seeing their data ("there were no significant differences between groups at pre- or posttest"), Sallows and Graupner (2005) decided not to carry out their intended comparison between groups and instead lumped them together.

In my view, Sallows and Graupner (2005) should be regarded as an uncontrolled trial (see Cohen et al., 2006, who also take this position).

Spreckley and Boyd (in press) make the serious (but popular) error of using meta-analysis when the available studies are not of sufficient quality.

Andrew said...

Michelle, I agree with your comment regarding the Sallows and Graupner study, it should be regarded as an uncontrolled trial.

Why S&G expected the clinic based group to perform better than a group of highly motivated parents operating in an environment familiar to each child is beyond me.

S&G got encouraging results, but as with any uncontrolled trial, all these can do is suggest that controlled trials are warranted.

As a supporter of ABA, based on the results I have seen with my own child, I am encouraged by the results of S&G, but recognise that ABA still lacks a replication of the Lovaas study with a randomised control group and aversives excluded.

As a skeptic of therapies such as Sensory Integration, as well as alternative remedies such as homeopathy, both of which have ample uncontrolled trials supporting them but no quality, controlled trials supporting them, I recognise that better evidence is needed for ABA in order for me to avoid being guilty of double standards.