Sunday, January 07, 2007

Mr Doherty's science

According to Canada's eminent autism advocates, anyone who criticizes ABA-based autism interventions is, to put it more politely than they do, ignorant, misguided and reprehensible (you can see this on their flagship website, which I respond to here; see also here). It looks like I'm going to overlook this forcefully-promoted pillar of autism advocacy--that ABA is above science, ethics, and scrutiny--again.

Harold Doherty, an influential and powerful Canadian autism advocate, has helpfully supplied what he sees as definitive evidence that ABA-based interventions are "Most Effective In Educating Autistic Children". This is interesting, because Mr Doherty also supports the view that ABA-based autism interventions are not education at all, but are "medically necessary" treatment.

In fact, right now, we have one set of autism advocates heavily lobbying Parliamentarians (Senators and MPs) to alter the law and mandate ABA-based autism interventions as "medically necessary" for all autistics in Canada. And we have another set of autism advocates going to the Supreme Court of Canada, just down the street, to deny that ABA-based autism interventions are medical treatment or "medically necessary" treatment or medical in any way at all. Those are the autism advocates who support the Wynberg trial decision.

But back to Mr Doherty's science.

Mr Doherty supplies three sources to support the effectiveness of ABA as education for autistic children. One is the MADSEC report. One is a list of papers or reports, and short descriptions of same, provided on Dr Lovaas' website (the Lovaas Institute, you can see this page here; Mr Doherty has copied parts of this into his post). The last is the Auton trial decision.

The MADSEC report bases its conclusions on a total of three ABA controlled trials, Lovaas (1987; and follow-up, McEachin et al., 1993), Birnbrauer and Leach (1993), and Sheinkopf and Siegel (1998). The first two are prospective, while the last is retrospective. Lovaas (1987) and follow-up are dependent on the use of aversives, and therefore represent a treatment which is no longer acceptable. Birnbrauer and Leach have presented a 10-year follow-up, showing that the few children who did well in their study (four of them, none of whom achieved "normal functioning") did not maintain these gains on follow-up (Birnbrauer and Leach, 2006). Sheinkopf and Siegel (1998), the retrospective study, features no autistics who achieved "normal functioning", and also shows no correlation between intensity of treatment and outcome measures. Lovaas (2002) forcefully criticizes this study, and does not accept its validity.

Then there is the list from Dr Lovaas' website, or the parts of it reproduced by Mr Doherty on his blog.

First there is this claim,

Between 1985 and 2005, there were over 500 articles published concerning Applied Behavior Analysis and autism.

I don't dispute this at all. I've read quite a few of these articles. Dr Lovaas does not make any claims as to what these articles have found or their quality. Arguing that quantity of articles is evidence of treatment effectiveness is akin to arguing that the ToM, EF, and/or WCC accounts of autism must be correct because there are hundreds of articles about these theories, or that social skills training must be effective because there is an impressive number of articles about this kind of intervention.

Here's the rest of the list Mr Doherty provides from Dr Lovaas' website:

Lovaas (1987): In multiple post-1987 papers (e.g., Smith & Lovaas, 1998), and in the book Dr Lovaas recommends (Lovaas, 2002), Lovaas and colleagues have underlined the problem of the 10 non-normal-functioning children profiting little from ABA. E.g., their IQs did not increase at all, even though some stayed in their ABA programs for 10 years. Lovaas (2002) has stated that these children must stay in ABA programs for life. There is also the essential role of physical punishment (contingent aversives) in this study, which you can read about here, complete with quotes from Lovaas (1987). A Rett's girl was in Control Group 1, which was not reported until Boyd (1998). The savant abilities of at least one of the experimental group children were extinguished (Epstein et al., 1985; I wrote about this here). Also of interest to Mr Doherty (who has weighed in about high and low functioning in autism, as well as about savant syndrome), five of the autistic children in Control Group 1 (a group which had poor outcomes) were high-functioning. Mr Doherty is a leading autism advocate, so I will assume he knows where that fact is reported.

McEachin, Smith & Lovaas (1993): the follow-up into school ages of Lovaas (1987): See above. One of the "normal functioning" children loses this status.

Jacobson et al. (1998): This paper lacks a factual foundation, because in 46 years of behaviour analytic research in autism, there is no peer-reviewed paper which reports data about the adult outcomes of children who received 3 years of ABA-based intervention between the ages of 2-6. There is no controlled trial of a non-aversive early ABA/IBI that has a published follow-up into school ages (the one presented follow-up presented a failure; Birnbrauer & Leach, 2006). Also, Canada's autism advocates have rejected this level of service (3 years of ABA) as unacceptable, and Dr Lovaas claims that children who do not achieve "normal functioning" by age 7 must stay in ABA programs for their entire lives (Lovaas, 2002).

The NYSDOH report: This report found only four studies (of the 232 looked at) of ABA-based early intensive interventions that met their standards: Lovaas (1987) together with McEachin et al. (1993); Birnbrauer and Leach (1993); Smith et al. (1997); and Sheinkopf and Siegel (1998). All these papers have been described above except Smith et al. (1997), a retrospective study (not a true experimental design) showing very limited results (increase in IQ of 8 points, almost half of which is accounted for by one participant; no "normal functioning").

The Surgeon General's report: The only ABA-based study cited is Lovaas (1987) together with McEachin et al. (1993).

Eikeseth et al. (2002): A ME-Book (Lovaas, 1981) based ABA program is compared to a an unknown intensity of ABA plus a lot of contradictory approaches (this is called "eclectic" treatment). The groups are unmatched, and finish with no significant differences, even though the experimental group had greater gains. The study is for one year only. None of the children achieve "normal functioning".

Howard et al. (2005): This is also a one-year study that compares an ABA-based intensive intervention (for which there is no manual), to "eclectic" treatment (ABA of unknown quantity or quality, plus contradictory approaches), as well as to generic segregated special education. The groups aren't matched. None of the children achieve "normal functioning". The reported "effectiveness" of the ABA-based treatment does not take into account the total failure of two children, who could not continue in ABA, and whose data were discarded.

Sallows & Graupner (2005): The 48% rate of "rapid learners" can only be achieved by combining the control group with the experimental group, producing an uncontrolled trial. The control group performed better than the experimental group. This was not a feature of Lovaas (1987). This paper shows that neither intensity nor quality of ABA-based interventions is relevant to outcomes. Cohen et al. (2006, see below) point out that this study does not have a comparison group. Also, Sallows & Graupner (2005) report using a wide variety of approaches apart from Lovaas ABA, including non-behaviour analytic approaches.

And finally,

Cohen et al. (2006): The limited number of significant differences between outcome measures in the unmatched groups vanishes when non-matched variables are accounted for, with the one exception of classroom placement. Kasari (2006) has pointed out that classroom placement is a measure of parent pressure, rather than of child achievement. The control condition is segregated special education, of less intensity (less hrs/wk) than the ABA treatment.

Mr Doherty's last source is the Auton trial judge, and the Auton trial decision. This is his evidence that randomized assignment, a standard used to protect and benefit all non-autistics, should not apply to autistics. Mr Doherty further adds that there are ethical implications to randomized assignment. This is on the assumption that Lovaas-type ABA is known to be effective and therefore, you cannot deprive any autistics of this treatment.

So let's sum up Mr Doherty's science.

The evidence from the MADSEC report (which did not have to pass peer review) amounts to very little in the absence of an aversive-based study. Only Sheinkopf & Siegel (1998) is left, a retrospective study that Lovaas (2002) strongly objects to. Mr Doherty may be arguing that autistics don't need controlled trials either.

The list of studies and reports from Dr Lovaas' site is selective, and the selected studies are selectively reported. Dr Lovaas does not include all the ABA controlled trials, and he includes only one uncontrolled trial. Sheer number of studies (without reference to their quality, content, relevance, etc.) is not evidence of effectiveness. Using the website of an organization which provides a particular treatment as evidence for the effectiveness of that treatment represents the kind of low standards that autism advocates demand for autistics.

Finally, Mr Doherty uses the Auton trial decision to deny the importance of randomization. This assumes that this decision is scientifically accurate, and is based on peer-reviewed science. That is demonstrably not the case (e.g., see the description of the 1 in 64 study, which was not even in evidence). However, let's accept this for now and look at the evidence in Auton. There were only two controlled trials of ABA-based interventions in the evidence in Auton (in fact, these were the only primary sources reporting data about ABA-based interventions). These would necessarily be the studies being referred to by the trial judge re the importance of randomized assignment. One of them (Lovaas, 1987, and follow-up) is dependent on aversives, and represents a treatment that is currently unacceptable. The other (Smith et al., 1997) is not a true experimental design. It is retrospective, and, as reported above, has weak results which did not support the claims made by the Auton parents. These two studies are insufficient to argue that failing to enroll all autistics in non-aversive ABA programs (on the grounds that their effectiveness has been proven) is unethical.

ABA/IBI does have one existing randomized controlled trial (Smith, Groen & Wynn, 2000, 2001). This is left off Dr Lovaas' page, even though it was conducted at UCLA by behaviour analysts he trained and oversaw, and is of greater importance than Lovaas (1987). Smith, Groen & Wynn (2000, 2001) represents both the scientifc standards that would apply to Mr Doherty and all non-autistics (a randomized controlled trial), and the outcome of a treatment that is not, unlike the treatment reported in Lovaas (1987), dependent on contingent aversives.

A description and criticism of Smith, Groen & Wynn (also, accurate information about the NYSDOH report, and about the importance of randomized assignment) can be found here. I'll add that the reported increase in IQ is confined to the PDD-NOS group. Also, there is a second erratum (Dr Gernsbacher describes one of the published errata, which eliminates the reported result in language) published by the authors to correct another important error in the text (re educational placement).

There is also the Canadian study, Eaves and Ho (2004), which shows no effect of kind or amount of any currently marketed autism treatment in the "critical" 2-5 yr range. About half the children in this study were in ABA programs, and their outcomes did not differ from the children who were not in ABA programs.

I can also suggest comparing the standards, quality, etc., of the reports and sources favoured by Mr Doherty (MADSEC report, Dr Lovaas' website, the Auton trial decision) with the sources which have concluded that when it comes to autism interventions, we do not have sufficient information to decide what is effective for all autistic individuals (e.g., NRC, 2001; Volkmar et al., 2004). The NRC also alluded to the absence of basic ethical considerations in the entire body of autism intervention research.

I agree with Mr Doherty that there has been and no doubt will continue to be a lot of incompetent, non-valid and self-interested criticism of ABA-based interventions, including from behaviour analysts. Indeed, this poverty of criticism is why I wrote The Misbehaviour of Behaviourists, almost three years ago, and also in part why I wrote a series here about aversives. Putting sourced and referenced accurate information on the record is important, so that sweeping statements (e.g., about criticisms of ABA) by influential and powerful leaders like Mr Doherty can be verified against the public record. The record shows that I've been as harsh a critic of incompetent criticisms of ABA-based interventions as any behaviour analyst. This does not mean that accurate criticism of these interventions is non-existent, unnecessary, reprehensible, or impossible.

There is also the problem of ethics, a more important issue than the "effectiveness" issue, which is written about above. It is more important because an apparently effective treatment may be unethical. This has been shown in the area of ABA-based interventions, by the strong and decisive ethical objections, including from behaviour analysts, to Dr Lovaas' other NIH-funded early ABA/IBI--which was also demanded by parents and said to be the only effective treatment for the targeted pathology. My attempt to "point out that ethical standards which have improved the circumstances of all other persons would equally benefit autistics, and would also improve the state of the science", has been met with extreme opposition (including defamation) by autism advocates.

The above does not represent anything more than a brief response to some aspects of Mr Doherty's post. I haven't touched on major issues in the behaviour analytic literature, like diagnostic standards, accounting (or not) for medication, etc. I've only provided tiny, superficial descriptions of studies and reports, though what I wrote can be verified, and they're all studies and reports that I know well (I don't know exactly which 500 studies Dr Lovaas is referring to, but I've probably read a lot of them). I haven't gone into the existence of a successful controlled trial of a non-ABA-based intervention, reported in a peer-reviewed journal. Etc. And I've barely mentioned ethical consideration, which is paramount. I've failed to suggest that Mr Doherty provide sources for the kinds of criticism he considers should not be made. I've forgotten to point out the dramatic contrast between the huge amount of existing autism research and the concurrent poverty of knowledge about autism and about how to help autistic people--a contrast which might have something to do with the effectiveness of autism advocacy. And later, I'll try to post an outline of some of the major bases on which ABA-based interventions can and should be criticized.

The onus is on autism advocates like Mr Doherty to show how discarding basic scientific and ethical standards, the standards which protect and benefit themselves, helps autistics. My own suggestion is that lousy scientific and ethical standards are bad for everyone, including those who, like autism advocates, demand that they be imposed on autistics.

(If this post looks like it was written in pieces by a person busy doing other things, that's because it was; some of the information above has already been provided on this blog, see e.g., here and here , as well as on my website, but autism advocacy involves repeating the same things over and over, which means responding can be tedious).


References:

Birnbrauer J.S., & Leach, D.J. (1993). The Murdoch Early Intervention Program after two years. Behaviour Change, 10, 63-74

Birnbrauer J.S., & Leach, D.J. (2006, June). The Murdoch Early Intervention Program at 10 years. Association for Behavior Analysis Annual Conference abstract. Atlanta, GA.

Boyd, R.D. (1998). Sex as a possible source of group inequivalence in Lovaas (1987). Journal of Autism and Developmental Disorders, 28, 211-214.

Cohen, H., Amerine-Dickins, M., Smith, T. (2006). Early intensive behavioral treatment: Replication of the UCLA model in a community setting. Journal of Developmental and Behavioral Pediatrics, 27, S145-S155.

Eaves, L.C., Ho, H.H. (2004). The very early identification of autism: Outcome to age 4 1/2-5. Journal of Autism and Developmental Disorders, 34, 367-378.

Eikeseth, S., Smith, T., Jahr, E., & Eldevik, S. (2002). Intensive behavioral treatment at school for 4- to 7-year old children with autism: A one-year comparison controlled study. Behavior Modification, 26, 49–68.

Epstein, L.J., Taubman, M.T., & Lovaas, O.I. (1985). Changes in self-stimulatory behaviors with treatment. Journal of Abnormal Child Psychology, 13, 281-294.

Howard, J.S., Sparkman, C.R., Cohen, H.G., Green, G., & Stanislaw, H. (2005). A comparison of intensive behavior analytic and eclectic treatments for young children with autism. Research in Developmental Disabilities, 26, 359–383.

Jacobson, J.W., Mulick, J.A., and Green, G. (1998). Cost-benefit estimates for early intensive behavioral intervention for young children with autism--general model and single state case. Behavioral Interventions, 13, 201-226.

Kasari, C. (2006, June). What are the active ingredients of interventions? International Meeting for Autism Research. Montreal, Canada.

Lovaas, O.I. (1981). Teaching Developmentally Disabled Children: The Me Book. Austin, TX: Pro-Ed.

Lovaas, O.I. (1987). Behavioral treatment and normal educational and intellectual functioning in young autistic children. Journal of Consulting and Clinical Psychology, 55, 3-9.

Lovaas, O.I. (2002). Teaching Individuals with Developmental Delays: Basic Intervention Techniques. Austin, TX: Pro-Ed.

McEachin, J.J., Smith, T., and Lovaas, O.I. (1993). Long-term outcome for children with autism who received early intensive behavioral treatment. American Journal on Mental Retardation, 97, 359-72.

National Research Council (2001). Educating children with autism. Committee on Educational Interventions for Children with Autism. Division of Behavioral and Social Sciences and Education. National Academy Press, Washington, DC.

Sallows, G.O., & Graupner, T.D. (2005). Intensive behavioral treatment for children with autism: Four year outcome and predictors. American Journal on Mental Retardation, 110, 417-438.

Sheinkopf, S.J., & Siegel, B. (1998). Home-based behavioral treatment of young children with autism. Journal of Autism and Developmental Disorders, 28, 15–23.

Smith, T., Eikeseth, S., Klevstrand, M., & Lovaas, O.I. (1997). Intensive behavioral treatment for preschoolers with severe mental retardation and pervasive developmental disorder. American Journal on Mental Retardation 103, 238-249.

Smith, T., Groen, A.D., and Wynn, J.W. (2000, 2001). Randomized trial of intensive early intervention for children with pervasive developmental disorder. American Journal on Mental Retardation, 105, 269-85. Erratum in American Journal on Mental Retardation, 105, 508. Erratum in American Journal on Mental Retardation, 106, 208.

Smith, T., Lovaas, O.I. (1998). Intensive and early behavioral intervention in autism: The UCLA young autism project. Infants and Young Children, 10, 67-78.

Volkmar, F.R., Lord, C., Bailey, A., Schultz, R.T., & Klin, A. (2004). Autism and pervasive developmental disorders. Journal of Child Psychology and Psychiatry, 45, 135-170.

37 comments:

Joseph said...

The evidence on the effectiveness of ABA is surprisingly poor. Are there any randomized blinded trials? If you just look at claims posted on the web, you end up with the impression that it cures close to 50% of autistic children. Of course the problem is that most people are not familiar with the way bias can skew the results of non-rigorous trials.

Anonymous said...

Joseph

Neither you, nor Ms Dawson, have actually demonstrated any actual or real bias in the long list of studies done which show gains for autistic children educated using ABA methodology.

You raise again the issue of randomized blinded trials. As a parent I would never submit my child's education to a randomized blinded process. That is the ethical problem with arriving at randomized blinded trials for ANY methodology.

Nor have you listed which education methodologies enjoy greater scientific support. Which other methodologies have been subjected to your randomixed blind trial standard? If you know of them you should make your information available to the governments who have resisted providing ABA services to autistic children. They have been unable to provide them to courts in Canada OR the US.

Ms Dawson's anti-ABA prejudices may be sufficient for readers of this blog site. The weight of study as reviewed by objective and learned academics provides overwhelming support for the effectiveness of ABA as an educational intervention for autistic children. Of course the California, New York, Maine, US Surgeon General and Association for Science in Autism Treatment authorities may all be mistaken in their assessments of these studies and maybe Ms Dawson is correct.

As a parent of a severely autistic child who has to make real decisions about the education of my child AND lobby to ensure that the resources are available for him and others in my home province of New Brunswick to receive a real education I will go with these other authorities and forgo reliance on Ms. Dawson's anti-ABA prejudices.

And BTW my son has been exposed to ABA - to his great benefit. By use of ABA we have given him language and communication skills, reduced self injurious behaviors and permitted him to function in a school environment. And yes we have used ABA for such unmentionables as toilet training.

Have a good day.

Harold Doherty
Conor's Dady

http://autisminnb.blogspot.com/

Anonymous said...

What I know about ABA could be written on the back of a postage stamp.

For many children it is completely ineffective, others only partially so, for others still, it may be helpful to address a few issues.

To me it is merely one technique of many, to permit the parent to 'perform' consistently to one or more issues, usually 'small,' specific and discrete matters.

Professionals by definition, act objectively. Parents on the other hand may have a tendency to react emotionally [subjectively] which doesn't help the child one wit.

By addressing key issues in a consistent manner over a long period of time, enables a family to function.

Lastly I don't think it is helpful to measure the success of a programme by correlating it to IQ, as autistic children's abilities [generally] are not accurately reflected in tests. [depending upon which one you use]
Best wishes

Joseph said...

Authorities are not impervious to misconceptions. That's why it's better to understand the arguments first hand and in depth. Gresham & MacMillan (1998) offers a review of the methodological questions in Lovaas (1987) and states that "It is recommended that parents and fair hearing officers adopt an attitude of healthy skepticism before proceeding to an unqualified endorsement of the EIP as a treatment for autism."

There are other papers of interest, such as one where they document a skewed male-female ratio in the groups.

I don't see what would be unethical about blinded evaluations, for example.

Dubious claims about the effectiveness of autism treatments abound and have existed from the start. Rimland had said Secretin has a 75% improvement rate. Neubrander had claimed a 90% improvement rate or so for Me-B12. It's becoming clear such claims are bogus due probably to the natural course of autism and subjective evaluations.

Michelle Dawson said...

Hi Mr Doherty,

I'd really appreciate it if you showed me where I show "anti-ABA prejudices" (quotes, please).

You are doing the equivalent of saying that I have "anti-cognitive science prejudices" because I've been an extremely harsh critic of cognitive science and neuroscience in autism--much more so than I have of ABA in autism.

Science is supposed to be criticized. If you are not interested in science, then do not make your public-policy-level demands using science-based claims.

I'd also like to know if you are certain that ABA is an education methodology, rather than a medical treatment, and why therefore you are seeking to change the law to make ABA-based autism interventions "medically necessary" for all autistics in Canada.

The absence of basic ethical considerations in the autism-ABA studies is evidence of bias. When the same treatment but a different population was involved, there were serious ethical concerns raised, including by major behaviour analysts. Do you regard Donald Baer (one of the three behaviour analysts who first defined ABA) as having "anti-ABA prejudices" because he strongly criticized Dr Lovaas' work?

Using your own logic, you are saying that the US National Academy of Sciences is wrong. Who do we believe, the National Research Council of the US, or Mr Doherty? You are saying that the major researchers working in autism are wrong. Who do we believe, scientists who have to pass peer-review, or Mr Doherty? Etc. This kind of argument is pointless.

Blinding in ABA studies is not a major concern of mine, so long as the evaluators are blind to which group children were in, and are not directly involved in the trial. Parents and therapists are going to know which children are supposed to do well, and which are not. There is no getting around this.

Eric Fombonne is conducting a randomized controlled trial of an intervention which already has a successful controlled trial (McConachie et al., 2005; this is a wait list study). I mentioned in my post an existing successful randomized controlled trial of an autism intervention (Aldred et al., 2004). This study is now being expanded to a large multi-site study involving 144 children (this is in the UK). The same kind of intervention has a large successful uncontrolled trial (Mahoney & Perales, 2005), apart from the controlled trial mentioned above. There is an excellent scientific foundation for this kind of intervention (e.g., Siller & Sigman, 2002; Keen, 2005; Keen et al., 2005) including in the ABA literature. See this for a review of this area.

The NYSDOH report found a greater number of medication studies (compared to ABA-based studies) which satisfied their criteria. Does this mean all autistic children should be medicated? I very much hope not. And there is no justification for making false or exaggerated claims about any kind of treatment, for autistics or anyone else.

Michelle Dawson said...

Hi Joseph,

Frank Gresham, as he told me, is the biggest ABA fan in the world. This did not surprise me, given his Auton testimony. He was the star witness for the BC government, and he fully supports ABA-based autism interventions. As I've reported, the Auton trial judge cited his testimony as fully supporting the parents' position.

The Misbehaviour of Behaviourists includes some very stiff criticism of the work of Dr Gresham and others similar.

Anonymous said...

Joseph

You make some general statements about professional (one could substitute "human" fallibility). I have no problem with someone objecting to claims about ABA effectiveness if the 1987 Lovaas study were the only study supporting ABA effectiveness but it is not. As you know. Forget the studies cited in the Lovaas summary that I posted in my comment on my blog site. Remember that the board of the Association for Science in Autism Treatment has also endorsed ABA as the only method that meets the evidence based standard as an intervention for autistic children.

I ask you to show me one other methodology that has anywhere near the scientific support for its efficacy that ABA enjoys.

Harold Doherty
Conor's Dad

Lucas McCarty said...

That question could be unanswerable as it is being framed in terms that make it unanswerable.

There are different methods of teaching different things. ABA has never been said to be an alternative to them except where Autism is concerned, it's promoted as the best or the only way an Autistic can be taught anything.

All claims made about ABA's positive effectiveness in Autism contradict the one near-universal consensus in Autism research, which is that there is no single research that can be reccomended as effective for Autism or all Autistic people.

r.b. said...

God Bless it.....if there is anything about ABA it is not the ABA that works, but the language learning system that was contained in Maurice's book.

Lovaas aside, I'm telling you...who is going to speak to the teaching of language visually that is a part of it. ABSOLUTELY NO_ONE ADDRESSES THE LANGUAGE COMPONENT.

Yeesh, am I the only one who's read the book?

Sorry...didn't mean to speak out of turn....

Joseph said...

NB: The other trials would tend to have similar methodological issues. Maybe not specifically the sources of bias, such as the skewed male-female ratio. I find it particularly telling that there's only one randomized ABA trial, and that happens to be the trial that did not work out.

Jennifer said...

Mr. Doherty,

Perhaps you could consider the fact that since the Lovaas paper was published, many researchers have rushed to investigate this "promising" educational technique. Because of Lovaas, a huge effort has been put into showing the benefits (or lack thereof) of ABA, almost to the exclusion of other approaches. That, in my opinion, is why other educational methodologies do not have the same publication record.

The ABA proponents have had more than 20 years to replicate the Lovaas results (without aversives). Can you show me a paper that does this? In normal science, we would refer to Lovaas as an un-replicated study. Sallows and Graupner had many, many children to choose from in the approximately 20 years that they have had to replicate Lovaas. It's not surprising that they found a few with good outcomes - good outcomes are well documented in the scientific literature for approximately 15% of children diagnosed with autistic disorder.


I have a child, orginally diagnosed with ASD, and now doing tremendously well - fully integrated in school, has friends, and a happy sociable child (and toilet trained for many years!) Almost recovered, you might say. But my child has never had ABA. Your anecotal evidence only proves that your individual child has progressed with age. Not too surprising. All children do this, even autistic ones.

Michelle Dawson said...

Hi again Mr Doherty,

If you're looking for an intervention that is effective in producing successful adults, there is no existing evidence in peer-reviewed papers which shows that early ABA/IBI does this. There are successful adults reported in the science, but they did not receive early ABA/IBI.

Using the model for investigating interventions put forward by the behaviour analyst Tristram Smith at IMFAR 2006, and then reported in Smith et al. (2006; this is with an NIMH working group), the kind of intervention I mentioned in my previous post is more advanced than ABA-based interventions. This is because ABA-based interventions lack a currently-validated manual, and also lack a successful randomized trial. On the other hand, the interventions I mentioned have an evidence base, including a case study, single subject designs, an uncontrolled trial, and a controlled trial. The group designs are all manual-based, as is the successful randomized controlled trial. There is now a large multi-site trial being funded. The quantity of studies is small (for a possible reason why, see Jennifer's post), but the quality is good, at least by autism intervention research standards.

This does not mean that I uncritically support this other kind of educational intervention. In its several forms, it needs to be greatly improved on. But the general direction (which is more or less the opposite of ABA-based interventions) is both consistently effective and consistent with what is known about autism.

Also, re efficacy, what kind(s) of ABA-based interventions do you want funded? Lovaas ABA? VB? PRT? PBS? RFT (which has an associated intervention, and I'll have to check if it has its own journal yet)? There is no agreement among behaviour analysts as to which kind(s) of ABA are effective and evidence-based, and there are also disagreements among highly respected behaviour analysts over whether some kinds of ABA are in fact behaviour analytic.

Anonymous said...

Ms Dawson

I believe we know each other's positions on the evidence basis for the efficacy of ABA as an education intervention for autistic children.

I asked you if you can name any OTHER education intervention for autistic children which is considered evidence based and effective? If so could you name it and provide some authority to support your position?

This is not just an academic question. I have a 10 year old autistic son. He has been diagnosed as having Autism Disorder, and described as proundly autistic by one of his diagnosing paediatricians as well as an assessing Psychologist. Despite his severe limitations we have accomplished much in terms of providing him with communication skills and modifying self injurious and dangerous behavior which he had previously exhibited.

We did this with ABA techniques and they did not involve aversives. And my son is a very happy, a visibly happy, boy. Visit my blog site and look at his pictures I have posted if you disbelieve me. ABA is something he looks forward to in his daily life.

But if you know of some other approach that has been empirically validated as effective then please by all means share that approach with me and provide your authorities. If your suggestion has merit I would want to consider its use with my son.

Harold Doherty
Conor's Dad

Michelle Dawson said...

Hi Mr Doherty,

No, I don't know which kind(s) of ABA you are demanding to be legislated as "medically necessary" for all autistics, even though you seem to take the position that ABA is an educational methodology.

Re a non-ABA intervention, in more than one previous comment (above), I've supplied references and a link to a pdf (which has the full references, plus many more, as well as descriptions of the interventions). I'm not sure what else you want. If you refuse to look up, read or consider the information I provide in response to your questions, there is no point in repeating it.

Do I recommend that this kind of intervention be imposed as public policy? No, I recommend that there be more research in this area, with higher standards of science and ethics. One of the programs I provided information about is provided to parents free in some places (Ottawa). This is the same intervention that Dr Fombonne is conducting a randomized trial on. It is (in my view) the most limited form of this kind of intervention, which trains parents how to respond to their autistic children's communication.

For an interesting discussion about how to help autistics, see this discussion, which links to multiple resources (and has some discussion of the non-ABA intervention I've written about here). The information is a bit out of date (Cohen et al, 2006, and Birnbrauer & Leach, 2006, were not yet available), I'm a lot better informed in some specific areas now than I was then, and the quality of the science in this discussion varies widely. But it's still a useful and interesting discussion with a lot of references (so you can verify things) and sources, the whole thing requiring (as does everything in autism) a lot of critical thinking.

Parents have described their autistic children as happy and miraculously progressing in pretty much any intervention you can name, including aversive ones (this has been pointed out by Tristram Smith; Smith, 1988). E.g., the JRC has a blog brimming with testimonials of delighted parents who are certain their children would be doomed without the treatment supplied by the JRC. Also see Maurice (1993) for a claim that an autistic child was happy to have her electric shock helmet on, and was upset when it was removed. Lichstein and Schriebman (1976) concluded (as I've reported here before) that the main side effect of electric shock treatments was that they made autistic children happy. Dr Smith has also pointed out that Bruno Bettelheim had glowing testimonials and a long waiting list (Smith, 1988). All of these things can be verified (that's what sources are for). And no, I'm not making any hysterical claims about aversives. I'm just reporting what has been reported by other people, including behaviour analysts.

There is also the observation that parents of autistic children who make no measurable progress in ABA programs still express complete satisfaction with these programs (Smith, Buch & Gamby, 2000; Boyd & Corley, 2001; Eaves & Ho, 2004; Volkmar et al., 2004).

We also know that a boy who underwent Dr Lovaas' other early ABA/IBI did, as a young adult, express gratitude for his treatment. This is striking, given what he learned in his ABA program and its later consequences for him (Green, 1987).

The extreme enthusiasm of a child for his ABA program was also reported first-hand in the Wynberg trial. I wrote about this here and here.

Anonymous said...

Ms Dawson

I asked you a simple and direct question. Can you answer that question and provide me in your response the name of another educational method that has the evidence based to support it that ABA has?

I know you are aware that the California, NY and Maine state agencies, the US Surgeon General office and the MD's and Ph. D's who make up the advisory board of the ASAT have all taken the view that ONLY ABA meets the standard of evidence based proven effective in educating autistic children. But maybe they are all wrong and maybe Michelle Dawson is right.

Please state the name of any alternative methodology and the references that you have to support it as evidence based proven effective.


Harold Doherty
Conor's Dad

http://autisminnb.blogspot.com/

jypsy said...

I have a 10 year old autistic son. He has been diagnosed as having Autism Disorder, and described as proundly autistic by one of his diagnosing paediatricians as well as an assessing Psychologist.

As the father of a severely autistic 10 year old boy( from here)

I'm thinking that Mr. Doherty would use the same terms to describe my son Alex. I'm thinking where I use the term "very much autistic" he uses "severely" and "profoundly".

Mr. Doherty, can you explain to me what makes your son "severe" and "profound"? I asked on this blog before and will ask again (and I'll accept "that's too personal a question to answer so I won't")- "At what age did Conor first begin to develop verbal communication?" (from here

Lucas McCarty said...

Mr Doherty, your question has been answered as close as possible to what it can actually be answered. The problem is in your question itself which is loaded by assuming ABA has wide scientific support that it doesn't have and then demands to know of another method that enjoys the same. It's a trick question along the lines of "This car goes fast. It doesn't start. But show me a car that goes faster".

The most widely supported consensus in Autism is that no single method is suitable for Autism and all Autistics. You should already know this, meaning you should know exactly what is wrong with your question.

Michelle Dawson said...

Hi Mr Doherty,

You're repeating yourself, but I'm not going to repeat the answers I've already given.

If your standards involve believing that if ASAT says so, it must be true, then you are helpfully confirmning what we've come to know about the standards autism advocates demand for autistics.

When you are clearly (and repetitively) not interested in the information I provided in more than one previous comment above, and when you keep denying that I've provided this information, repeating this information is not going to change anything.

I'm always amazed (and in this case, baffled) when autism advocates show so little interest in autism research, as you are showing.

I have to take a wild guess, and conclude that you are looking for a brand name or snappy acronym or easy "fact sheet" (or something you can quickly look up on ASAT's website?). Sorry. I did provide references. They are not full cites, but I did provide a link to a published review (do you know what a review is?) which does have some of the full cites and many, many more, as well as data. Also, at least one of those studies should be familiar to anyone concerned with autism interventions, because it is a successful randomized trial of an educational intervention. There are not very many of those in the literature.

If you're not interested, or if this is too much work for you, that is your business. But why you persist in claiming that I have not provided the information you asked for, when it is there to see--for anyone who actually is interested--is mystifying. What is the point of demanding over and over that I provide information that I've already provided?

Or maybe you don't know how to look up papers, unless you have a snappy acronym or brand name or website? But this is about a general evidence-based approach to autism, and I'm afraid you need to look at the actual science. But not only are you an eminent autism advocate (and therefore an expert on autism research), I made it easy, by providing a general-level pdf (it is in a peer-reviewed journal), which has full references.

But so long as you deny that I've provided the information that I've clearly provided, there isn't much I can do except conclude that you're not interested.

Finally, no, the approach to autism that I wrote about is not featured on ASAT's website. You might want to ask them why. Interestingly, Tristram Smith (the behaviour analyst) did mention the existence of this approach to autism in his IMFAR 2006 presentation, in among his list of brand names and familiar acronyms. Maybe he can alert ASAT to this (I believe he's on the board).

Michelle Dawson said...

I agree with Lucas that there is, for the time being, a consensus in the science that when it comes to interventions, there is no evidence that can justify recommending one specific kind of intervention for all autistics.

This is in a context where the goals of interventions have never been questioned (NRC, 2001). Possibly, the failure to arrive at consistently and predictably effective ways to help autistics succeed is a result of a failure to consider what the goals of interventions or approaches should be.

However, there is recent empirical evidence to support a non-ABA approach (the one lacking a snappy name or acronym that I provided information about in more than one previous message above). This evidence is good but sparse. As I wrote above, the research needs to be expanded and improved on, both in the area of science and ethics, and as with all science, the studies have to be read critically. However, this approach (in different forms) has, by currently recognized scientific standards, been shown to help autistics (and their parents).

Joseph said...

The suggestion that an autistic child would need ABA to be visibly happy strikes me as odd. I'm speaking as the father of a visibly happy, yet very much autistic, child.

Jennifer said...

Mr. Doherty,

To give it a name, the approach discussed by Michelle is "More Than Words". Here are a couple of links to the published studies on the effectiveness. You may find them interesting.
McConachie et al 2005

Girolametto et al 2006

Jennifer said...

Sorry, forgot this one:

Aldred et al 2004

Ms Clark said...

I thought the treatment that Michelle was referring to did have the snappy acronym, MOONBAT or something...

I think MOONBAT got shortened to just BAT... though, Bainian Acceptance Therapy... or maybe that's just a related therapy with a similar protocol.

No website though... not exactly.

Michelle Dawson said...

For clarity, the non-ABA approach I've mentioned isn't at all limited to "More Than Words", even though "More Than Words" (see the first 2 papers Jennifer helpfully provided, the first of which I mentioned above) is one form of this approach.

In my view, "More Than Words", as I stated above (because this is the program offered free in Ottawa), is the most limited form of this approach.

Aldred et al. (2004) is not a "More Than Words" study. Nor is Mahoney & Perales (2005), but they also represent empirical investigations (successful ones) of the approach I've been going on about. Gernsbacher (2004) is also a case study in this area, and it does not involve "More Than Words". Likewise, Gernsbacher (2006), the review I provided a link to (the pdf is free), is not about "More Than Words", but provides evidence (and a long list of references) concerning the non-ABA approach I've described here.

So while "More Than Words" is one form of this approach, it is a specific and limited one. There are also other forms of this approach which have empirical evidence to support their effectiveness in helping autistic children (and their parents).

This is why I didn't apply a name to the non-ABA approach I've been describing, and only provided one reference concerning "More Than Words", as well as several which are not.

However, "More Than Words" is the intervention which Eric Fombonne is conducting a randomized controlled trial of at McGill (as I wrote way back there somewhere in the comments).

Again, I can't think of a better introduction to the non-ABA approach to autism (which is not limited to "More Than Words") I've pointed to in these comments than Gernsbacher (2006), which I linked to above.

Anonymous said...

Without getting into some sort of internet fight with a bunch of people I do not know. I'll keep this very short.
My son was diagnosed with Autism this past spring (2006) since his ABA therapy, done by Autism Intervention services, he has done so well, that his receptive & expressive language is age appropriate. Just a few months ago, after seeing Paul McDonnell, he was 12 months behind! This was with NO speech therapy, ONLY ABA therapy done by an ASW. Not only is his speech where it "should" be, but his motor skills, social skills, are almost all age appropriate. Before his ABA therapy, he was so far behind in everything, I thought he would never "get better" and now, he blows every professional he meets away.

As a parent, you do what YOU think is best for YOUR child. And for MY child, I KNOW that ABA is the best therapy for MY son. And I am not selfish, and would NEVER take a risk on any other type of treatment, as ABA is known to work, and is PROVEN to work.

All this back and fourth arguing is just tiring. Give it up Michelle. Mr.Doherty is a very intelligent man, your out of your league.

jypsy said...

As a parent, you do what YOU think is best for YOUR child. And for MY child, I KNOW that ABA is the best therapy for MY son.

Parents (not saying you, I have no idea who you are "anonymous") advocating to make ABA "medically necessary" treatment in Canada, or the "only proven educational method" are, as I see it, telling me *they* know what's best for *my* child.

Anonymous said...

Hi Michelle,

You wrote:

"The savant abilities of at least one of the experimental group children were extinguished (Epstein et al., 1985; I wrote about this here)."

I seem to recall reading elsewhere too where you've indicated that there is evidence that ABA can extinguish savant skills.

In the case you mentioned, your link (Paragraphs 72-73) indicates that the ABA practitioners specifically targeted the savant skills for elimination, writing that "Successfully extinguishing the unwanted genius behaviour near the end of this boy’s treatment was described as requiring only "minor discouragement from his parents and the project staff"

Is there evidence that ABA can 'accidentally' eliminate savant skills, i.e. eliminate them as an unintended consequence of ABA? I’m specifically thinking about cases in which ABA is used strictly for skills acquisition – i.e. teaching – without any intention or attempt to judge or eliminate behaviours seen as autistic (e.g. making no attempt to eliminate stimming and no attempt to force eye contact, etc).

Michelle Dawson said...

Hi Mr Parker,

You would have to point me towards some published science concerning the kind of ABA program you describe.

The boy in Epstein et al. (1985) was in the experimental group in Lovaas (1987) and was therefore hit. This means you could speculate that for this child, a bit of discouragement might go further than for a child who was not systematically hit.

There is also a paper which shows that hyperlexia can survive an ABA program. In this case, the hyperlexic abilities were well established well before the ABA program started, and the ABA program lasted... (working from memory) about 4 years, and failed to make this boy "indistinguishable" (he maintained his diagnosis). Also, you can take a wild guess that his major IQ increases seem to have happened after his ABA program stopped. This is a recent study and I have no idea what kind of ABA program this boy was in, or what its intensity was.

But if you're looking for actual science, remember that behaviour analysts have barely mentioned savant abilities in those fabled hundreds (or is it thousands?) of papers which are routinely claimed to "prove" the "effectiveness" of ABA/IBI. Given the high prevalence of savant abilities in autism, this should give anyone pause as to the presumed objective and accurate observation of behaviour analysts.

None of the existing published controlled or uncontrolled trials of early ABA/IBI (or published descriptions of programs, or manuals) allows for anything like you're describing. I'll never forget Dr Sallows explaining to me that children whose focused interests weren't totally eliminated were taught to hide their interests--their strengths--on the grounds that they were socially unacceptable. This is strongly reflected in an early submitted draft of Sallows & Graupner, where the failure to completely eliminate the "esoteric" interests of some of the children was seen as a shortcoming of the study (compared to Lovaas, 1987).

ABA programs also, as we've discussed elsewhere, ration information and materials, and present information in specific ways--all of which represent the opposite of how autistics who learn exceptionally well (those would be savants) learn.

Michelle Dawson said...

Hi Anonymous,

Science and ethics can be tedious and downright picky. All those statistics, all that, uh, arguing back and forth over methodology and interpretation and so on that's called "peer review", and as you demonstrate, it can be inconvenient to consider the interests or even the existence of others when they are not like you at all. You are saying, who needs the hassle, when you know--as autism advocates like Mr Doherty know--you are totally right about everything?

Unlike autism advocates, I believe autistics are worth the bother--the painstaking hassle of applying high standards of science and ethics. I think autistics deserve the standards of science and ethics that protect and benefit you and Mr Doherty, and that you can take for granted.

Anonymous said...

Hi Michelle,

Thanks for the reply.

I don’t know of anything published that evaluates the kind of ABA/IBI program I described, but from what I’ve encountered, there are more ‘ABA’ or ‘IBI’ practitioners now who are using ABA techniques to teach skills without attempting to use these same techniques to repress autistic characteristics. Whether these practices should still be labeled as ABA or given another name may be a reasonable question.

My daughter is currently in a private IBI program for three hours per day, five mornings per week. This program concentrates on teaching skills, and the program director was quite clear that their goal is not to make my daughter appear non-autistic, and they do not see stimming or other autistic characteristics as something to be repressed. While some of the methods are ABA-related, the program also uses other techniques, depending on what they want the child to learn, including periods during which they follow the child’s lead. FWIW, I gave the local program supervisor a copy of the Gernsbacher paper you linked above - ‘Toward a Behavior of Reciprocity’ – and she quite liked it.

My concern - based on your evidence that ABA can extinguish savant skills - was that that her IBI program might accidentally do this. This is something that we definitely do not want to happen. But as you’ve indicated, since the behavioural analysts tend not to consider savant abilities, the answer to this is probably unknown. As such, it was heartening to learn e.g. that hyperlexia can survive an ABA program, which is a positive sign.

FWIW, I took the ‘More than Words’ course offered by the local autism support agency, and this period corresponded with an explosion in my daughter’s use of gesturing and in her communications and interaction with us. While correlation does not necessarily indicate causality, I would (anecdotally) suggest the possibility of a connection. My daughter does not yet speak (other than ‘babble’, which I’m using in the belief that it is the correct technical term for her verbalizations), but she manages to make herself very well understood.

Anonymous said...

Hi folks,

I am trying to read the Erratum entries on the JoMR site but I am told that I need a subscription to view the article (no abstract like other "full" published papers). I have purchased the article but... is there an alt source for the 2 erratum entries (no "synopsis" is available from the JoMR?

"Erratum in American Journal on Mental Retardation, 105, 508. Erratum in American Journal on Mental Retardation, 106, 208."

Dave.

Anonymous said...

Some "weight" seems to be being placed on the Smith, Groen, & Wynn 2000 study which used randome assignment but produced "less dramatic results" then Lovaas '87... would it be incorrect to wonder if one explanation re the "less dramatic" results might be rooted in the fact that the Smith et. al (2000) paper only provided 25 hours of ABA while Lovaas ('87) provided 40... could this account for the lower then expected outcomes (ie if Smith et al (2000) had had a budget for 40 hours of contact time per week... )?

Dave.

Michelle Dawson said...

Re Dave's first comment, I found those errata (which are listed on PubMed) by going to a university library and looking them up in the AJMR.

Re Dave's second comment, there are many differences between the two UCLA ABA studies. E.g., there's the absence of aversives in Smith, Groen, and Wynn (2000), in contrast to Lovaas (1987). Half the children in Smith et al. (2000) had a diagnosis of PDD-NOS instead of autism; in the absence of the PDD-NOS children, Smith et al. (2000) would have very few results to report (e.g., the increase in IQ was in the PDD-NOS group only). The children in Smith et al. (2000) should have profited from all the claimed great improvements in ABA programs that occurred between 1970 and 1989 (the first intake years of Lovaas, 1987, and Smith et al., 2000, respectively). And so on.

Smith et al. (2000) reports a wide range both of intensity and total quantity of treatment. One child did so poorly in ABA that his hours were kept in the 10-15hr/wk range (this seems not to have been included in the reported range of intensity). Also, treatment was phased out after 18 months for chidren who did poorly. In spite of the wide range of hrs/wk and the huge range of total amount of treatment, the authors do not report any correlation between hrs/wk or total hours and outcome measures.

There are well-established ways to verify if, e.g., intensity or amount of treatment is relevant to outcomes. So far, no autism-ABA controlled trial has reported any correlation between treatment intensity or amount and outcome measures.

Anonymous said...

Ok. Some have suggested that the "less dramatic" results (when comparred to Lovaas '87) for clients in later studies are due to cessation of the use of aversives in studeis that attempted to replicate Lovaas ('87). In essence the assertion has been that results in Dr. Lovaas' '87 research are related to (or an effect of) the use of aversives... I just wondered if, in this case, an alternate explanation might also account for the differences (dramtic outcomes vs. less dramatic)...

Michelle Dawson said...

Hi Anonymous,

See my previous response to Dave. Also, an RCT carries greater weight than a non-randomized trial, because it is a better experimental design.

Anonymous said...

Hi Michelle,

Wow I love stats. (not so much)! In the Sallows & Graupner 2005 paper the authors note: The number of weekly hours of treatment seemed less related to outcomes then did pretreatment variables. In otherwords "Intensity" (hours/wk) was not found to be correlated with outcome measures so "hours per week" would not be a means of predicting success (increased scores) on outcome measures. Right? Was this the point you were trying patiently to make?

But the study did show mean positive outcomes on measures used (the kids as a group did well on outcome scores)...

It also identified several variables on pretreatment scores related to successful outcomes on post program meaures. Children who fit the profile on these variables did well when "treatment" was provdied. Would it be incorrect to conclude, based on this paper, that when Child "A" scores "xyz" on pretreatment tests (those related to outcome measures in this paper) and is placed in a WEAP type program scores on post-program tests will have improved.

The authors also note that the "moderate" learners did make some gains and that they "..were still aquiring new skills...".

Sorry.

Dave.

Michelle Dawson said...

Hi Dave,

I'm not going to repeat again the points I've made re correlation. I've made them clearly and several times (with apologies to other readers) in response to your questions on two different posts.

Uncontrolled trials are of limited use (except as evidence that RCTs should be conducted), particularly when they show no relationship between the treatment (intensity or quality) and outcome measures.

The data show the "moderate learners" in Sallows and Graupner (2005) did very poorly.