According to Canada's eminent autism advocates, anyone who criticizes ABA-based autism interventions is, to put it more politely than they do, ignorant, misguided and reprehensible (you can see this on their flagship website, which I respond to here; see also here). It looks like I'm going to overlook this forcefully-promoted pillar of autism advocacy--that ABA is above science, ethics, and scrutiny--again.
Harold Doherty, an influential and powerful Canadian autism advocate, has helpfully supplied what he sees as definitive evidence that ABA-based interventions are "Most Effective In Educating Autistic Children". This is interesting, because Mr Doherty also supports the view that ABA-based autism interventions are not education at all, but are "medically necessary" treatment.
In fact, right now, we have one set of autism advocates heavily lobbying Parliamentarians (Senators and MPs) to alter the law and mandate ABA-based autism interventions as "medically necessary" for all autistics in Canada. And we have another set of autism advocates going to the Supreme Court of Canada, just down the street, to deny that ABA-based autism interventions are medical treatment or "medically necessary" treatment or medical in any way at all. Those are the autism advocates who support the Wynberg trial decision.
But back to Mr Doherty's science.
Mr Doherty supplies three sources to support the effectiveness of ABA as education for autistic children. One is the MADSEC report. One is a list of papers or reports, and short descriptions of same, provided on Dr Lovaas' website (the Lovaas Institute, you can see this page here; Mr Doherty has copied parts of this into his post). The last is the Auton trial decision.
The MADSEC report bases its conclusions on a total of three ABA controlled trials, Lovaas (1987; and follow-up, McEachin et al., 1993), Birnbrauer and Leach (1993), and Sheinkopf and Siegel (1998). The first two are prospective, while the last is retrospective. Lovaas (1987) and follow-up are dependent on the use of aversives, and therefore represent a treatment which is no longer acceptable. Birnbrauer and Leach have presented a 10-year follow-up, showing that the few children who did well in their study (four of them, none of whom achieved "normal functioning") did not maintain these gains on follow-up (Birnbrauer and Leach, 2006). Sheinkopf and Siegel (1998), the retrospective study, features no autistics who achieved "normal functioning", and also shows no correlation between intensity of treatment and outcome measures. Lovaas (2002) forcefully criticizes this study, and does not accept its validity.
Then there is the list from Dr Lovaas' website, or the parts of it reproduced by Mr Doherty on his blog.
First there is this claim,
Between 1985 and 2005, there were over 500 articles published concerning Applied Behavior Analysis and autism.
I don't dispute this at all. I've read quite a few of these articles. Dr Lovaas does not make any claims as to what these articles have found or their quality. Arguing that quantity of articles is evidence of treatment effectiveness is akin to arguing that the ToM, EF, and/or WCC accounts of autism must be correct because there are hundreds of articles about these theories, or that social skills training must be effective because there is an impressive number of articles about this kind of intervention.
Here's the rest of the list Mr Doherty provides from Dr Lovaas' website:
Lovaas (1987): In multiple post-1987 papers (e.g., Smith & Lovaas, 1998), and in the book Dr Lovaas recommends (Lovaas, 2002), Lovaas and colleagues have underlined the problem of the 10 non-normal-functioning children profiting little from ABA. E.g., their IQs did not increase at all, even though some stayed in their ABA programs for 10 years. Lovaas (2002) has stated that these children must stay in ABA programs for life. There is also the essential role of physical punishment (contingent aversives) in this study, which you can read about here, complete with quotes from Lovaas (1987). A Rett's girl was in Control Group 1, which was not reported until Boyd (1998). The savant abilities of at least one of the experimental group children were extinguished (Epstein et al., 1985; I wrote about this here). Also of interest to Mr Doherty (who has weighed in about high and low functioning in autism, as well as about savant syndrome), five of the autistic children in Control Group 1 (a group which had poor outcomes) were high-functioning. Mr Doherty is a leading autism advocate, so I will assume he knows where that fact is reported.
McEachin, Smith & Lovaas (1993): the follow-up into school ages of Lovaas (1987): See above. One of the "normal functioning" children loses this status.
Jacobson et al. (1998): This paper lacks a factual foundation, because in 46 years of behaviour analytic research in autism, there is no peer-reviewed paper which reports data about the adult outcomes of children who received 3 years of ABA-based intervention between the ages of 2-6. There is no controlled trial of a non-aversive early ABA/IBI that has a published follow-up into school ages (the one presented follow-up presented a failure; Birnbrauer & Leach, 2006). Also, Canada's autism advocates have rejected this level of service (3 years of ABA) as unacceptable, and Dr Lovaas claims that children who do not achieve "normal functioning" by age 7 must stay in ABA programs for their entire lives (Lovaas, 2002).
The NYSDOH report: This report found only four studies (of the 232 looked at) of ABA-based early intensive interventions that met their standards: Lovaas (1987) together with McEachin et al. (1993); Birnbrauer and Leach (1993); Smith et al. (1997); and Sheinkopf and Siegel (1998). All these papers have been described above except Smith et al. (1997), a retrospective study (not a true experimental design) showing very limited results (increase in IQ of 8 points, almost half of which is accounted for by one participant; no "normal functioning").
The Surgeon General's report: The only ABA-based study cited is Lovaas (1987) together with McEachin et al. (1993).
Eikeseth et al. (2002): A ME-Book (Lovaas, 1981) based ABA program is compared to a an unknown intensity of ABA plus a lot of contradictory approaches (this is called "eclectic" treatment). The groups are unmatched, and finish with no significant differences, even though the experimental group had greater gains. The study is for one year only. None of the children achieve "normal functioning".
Howard et al. (2005): This is also a one-year study that compares an ABA-based intensive intervention (for which there is no manual), to "eclectic" treatment (ABA of unknown quantity or quality, plus contradictory approaches), as well as to generic segregated special education. The groups aren't matched. None of the children achieve "normal functioning". The reported "effectiveness" of the ABA-based treatment does not take into account the total failure of two children, who could not continue in ABA, and whose data were discarded.
Sallows & Graupner (2005): The 48% rate of "rapid learners" can only be achieved by combining the control group with the experimental group, producing an uncontrolled trial. The control group performed better than the experimental group. This was not a feature of Lovaas (1987). This paper shows that neither intensity nor quality of ABA-based interventions is relevant to outcomes. Cohen et al. (2006, see below) point out that this study does not have a comparison group. Also, Sallows & Graupner (2005) report using a wide variety of approaches apart from Lovaas ABA, including non-behaviour analytic approaches.
And finally,
Cohen et al. (2006): The limited number of significant differences between outcome measures in the unmatched groups vanishes when non-matched variables are accounted for, with the one exception of classroom placement. Kasari (2006) has pointed out that classroom placement is a measure of parent pressure, rather than of child achievement. The control condition is segregated special education, of less intensity (less hrs/wk) than the ABA treatment.
Mr Doherty's last source is the Auton trial judge, and the Auton trial decision. This is his evidence that randomized assignment, a standard used to protect and benefit all non-autistics, should not apply to autistics. Mr Doherty further adds that there are ethical implications to randomized assignment. This is on the assumption that Lovaas-type ABA is known to be effective and therefore, you cannot deprive any autistics of this treatment.
So let's sum up Mr Doherty's science.
The evidence from the MADSEC report (which did not have to pass peer review) amounts to very little in the absence of an aversive-based study. Only Sheinkopf & Siegel (1998) is left, a retrospective study that Lovaas (2002) strongly objects to. Mr Doherty may be arguing that autistics don't need controlled trials either.
The list of studies and reports from Dr Lovaas' site is selective, and the selected studies are selectively reported. Dr Lovaas does not include all the ABA controlled trials, and he includes only one uncontrolled trial. Sheer number of studies (without reference to their quality, content, relevance, etc.) is not evidence of effectiveness. Using the website of an organization which provides a particular treatment as evidence for the effectiveness of that treatment represents the kind of low standards that autism advocates demand for autistics.
Finally, Mr Doherty uses the Auton trial decision to deny the importance of randomization. This assumes that this decision is scientifically accurate, and is based on peer-reviewed science. That is demonstrably not the case (e.g., see the description of the 1 in 64 study, which was not even in evidence). However, let's accept this for now and look at the evidence in Auton. There were only two controlled trials of ABA-based interventions in the evidence in Auton (in fact, these were the only primary sources reporting data about ABA-based interventions). These would necessarily be the studies being referred to by the trial judge re the importance of randomized assignment. One of them (Lovaas, 1987, and follow-up) is dependent on aversives, and represents a treatment that is currently unacceptable. The other (Smith et al., 1997) is not a true experimental design. It is retrospective, and, as reported above, has weak results which did not support the claims made by the Auton parents. These two studies are insufficient to argue that failing to enroll all autistics in non-aversive ABA programs (on the grounds that their effectiveness has been proven) is unethical.
ABA/IBI does have one existing randomized controlled trial (Smith, Groen & Wynn, 2000, 2001). This is left off Dr Lovaas' page, even though it was conducted at UCLA by behaviour analysts he trained and oversaw, and is of greater importance than Lovaas (1987). Smith, Groen & Wynn (2000, 2001) represents both the scientifc standards that would apply to Mr Doherty and all non-autistics (a randomized controlled trial), and the outcome of a treatment that is not, unlike the treatment reported in Lovaas (1987), dependent on contingent aversives.
A description and criticism of Smith, Groen & Wynn (also, accurate information about the NYSDOH report, and about the importance of randomized assignment) can be found here. I'll add that the reported increase in IQ is confined to the PDD-NOS group. Also, there is a second erratum (Dr Gernsbacher describes one of the published errata, which eliminates the reported result in language) published by the authors to correct another important error in the text (re educational placement).
There is also the Canadian study, Eaves and Ho (2004), which shows no effect of kind or amount of any currently marketed autism treatment in the "critical" 2-5 yr range. About half the children in this study were in ABA programs, and their outcomes did not differ from the children who were not in ABA programs.
I can also suggest comparing the standards, quality, etc., of the reports and sources favoured by Mr Doherty (MADSEC report, Dr Lovaas' website, the Auton trial decision) with the sources which have concluded that when it comes to autism interventions, we do not have sufficient information to decide what is effective for all autistic individuals (e.g., NRC, 2001; Volkmar et al., 2004). The NRC also alluded to the absence of basic ethical considerations in the entire body of autism intervention research.
I agree with Mr Doherty that there has been and no doubt will continue to be a lot of incompetent, non-valid and self-interested criticism of ABA-based interventions, including from behaviour analysts. Indeed, this poverty of criticism is why I wrote The Misbehaviour of Behaviourists, almost three years ago, and also in part why I wrote a series here about aversives. Putting sourced and referenced accurate information on the record is important, so that sweeping statements (e.g., about criticisms of ABA) by influential and powerful leaders like Mr Doherty can be verified against the public record. The record shows that I've been as harsh a critic of incompetent criticisms of ABA-based interventions as any behaviour analyst. This does not mean that accurate criticism of these interventions is non-existent, unnecessary, reprehensible, or impossible.
There is also the problem of ethics, a more important issue than the "effectiveness" issue, which is written about above. It is more important because an apparently effective treatment may be unethical. This has been shown in the area of ABA-based interventions, by the strong and decisive ethical objections, including from behaviour analysts, to Dr Lovaas' other NIH-funded early ABA/IBI--which was also demanded by parents and said to be the only effective treatment for the targeted pathology. My attempt to "point out that ethical standards which have improved the circumstances of all other persons would equally benefit autistics, and would also improve the state of the science", has been met with extreme opposition (including defamation) by autism advocates.
The above does not represent anything more than a brief response to some aspects of Mr Doherty's post. I haven't touched on major issues in the behaviour analytic literature, like diagnostic standards, accounting (or not) for medication, etc. I've only provided tiny, superficial descriptions of studies and reports, though what I wrote can be verified, and they're all studies and reports that I know well (I don't know exactly which 500 studies Dr Lovaas is referring to, but I've probably read a lot of them). I haven't gone into the existence of a successful controlled trial of a non-ABA-based intervention, reported in a peer-reviewed journal. Etc. And I've barely mentioned ethical consideration, which is paramount. I've failed to suggest that Mr Doherty provide sources for the kinds of criticism he considers should not be made. I've forgotten to point out the dramatic contrast between the huge amount of existing autism research and the concurrent poverty of knowledge about autism and about how to help autistic people--a contrast which might have something to do with the effectiveness of autism advocacy. And later, I'll try to post an outline of some of the major bases on which ABA-based interventions can and should be criticized.
The onus is on autism advocates like Mr Doherty to show how discarding basic scientific and ethical standards, the standards which protect and benefit themselves, helps autistics. My own suggestion is that lousy scientific and ethical standards are bad for everyone, including those who, like autism advocates, demand that they be imposed on autistics.
(If this post looks like it was written in pieces by a person busy doing other things, that's because it was; some of the information above has already been provided on this blog, see e.g., here and here , as well as on my website, but autism advocacy involves repeating the same things over and over, which means responding can be tedious).
References:
Birnbrauer J.S., & Leach, D.J. (1993). The Murdoch Early Intervention Program after two years. Behaviour Change, 10, 63-74
Birnbrauer J.S., & Leach, D.J. (2006, June). The Murdoch Early Intervention Program at 10 years. Association for Behavior Analysis Annual Conference abstract. Atlanta, GA.
Boyd, R.D. (1998). Sex as a possible source of group inequivalence in Lovaas (1987). Journal of Autism and Developmental Disorders, 28, 211-214.
Cohen, H., Amerine-Dickins, M., Smith, T. (2006). Early intensive behavioral treatment: Replication of the UCLA model in a community setting. Journal of Developmental and Behavioral Pediatrics, 27, S145-S155.
Eaves, L.C., Ho, H.H. (2004). The very early identification of autism: Outcome to age 4 1/2-5. Journal of Autism and Developmental Disorders, 34, 367-378.
Eikeseth, S., Smith, T., Jahr, E., & Eldevik, S. (2002). Intensive behavioral treatment at school for 4- to 7-year old children with autism: A one-year comparison controlled study. Behavior Modification, 26, 49–68.
Epstein, L.J., Taubman, M.T., & Lovaas, O.I. (1985). Changes in self-stimulatory behaviors with treatment. Journal of Abnormal Child Psychology, 13, 281-294.
Howard, J.S., Sparkman, C.R., Cohen, H.G., Green, G., & Stanislaw, H. (2005). A comparison of intensive behavior analytic and eclectic treatments for young children with autism. Research in Developmental Disabilities, 26, 359–383.
Jacobson, J.W., Mulick, J.A., and Green, G. (1998). Cost-benefit estimates for early intensive behavioral intervention for young children with autism--general model and single state case. Behavioral Interventions, 13, 201-226.
Kasari, C. (2006, June). What are the active ingredients of interventions? International Meeting for Autism Research. Montreal, Canada.
Lovaas, O.I. (1981). Teaching Developmentally Disabled Children: The Me Book. Austin, TX: Pro-Ed.
Lovaas, O.I. (1987). Behavioral treatment and normal educational and intellectual functioning in young autistic children. Journal of Consulting and Clinical Psychology, 55, 3-9.
Lovaas, O.I. (2002). Teaching Individuals with Developmental Delays: Basic Intervention Techniques. Austin, TX: Pro-Ed.
McEachin, J.J., Smith, T., and Lovaas, O.I. (1993). Long-term outcome for children with autism who received early intensive behavioral treatment. American Journal on Mental Retardation, 97, 359-72.
National Research Council (2001). Educating children with autism. Committee on Educational Interventions for Children with Autism. Division of Behavioral and Social Sciences and Education. National Academy Press, Washington, DC.
Sallows, G.O., & Graupner, T.D. (2005). Intensive behavioral treatment for children with autism: Four year outcome and predictors. American Journal on Mental Retardation, 110, 417-438.
Sheinkopf, S.J., & Siegel, B. (1998). Home-based behavioral treatment of young children with autism. Journal of Autism and Developmental Disorders, 28, 15–23.
Smith, T., Eikeseth, S., Klevstrand, M., & Lovaas, O.I. (1997). Intensive behavioral treatment for preschoolers with severe mental retardation and pervasive developmental disorder. American Journal on Mental Retardation 103, 238-249.
Smith, T., Groen, A.D., and Wynn, J.W. (2000, 2001). Randomized trial of intensive early intervention for children with pervasive developmental disorder. American Journal on Mental Retardation, 105, 269-85. Erratum in American Journal on Mental Retardation, 105, 508. Erratum in American Journal on Mental Retardation, 106, 208.
Smith, T., Lovaas, O.I. (1998). Intensive and early behavioral intervention in autism: The UCLA young autism project. Infants and Young Children, 10, 67-78.
Volkmar, F.R., Lord, C., Bailey, A., Schultz, R.T., & Klin, A. (2004). Autism and pervasive developmental disorders. Journal of Child Psychology and Psychiatry, 45, 135-170.