Monday, March 12, 2007

Intake IQ in Lovaas (1987)

I'm grateful to Harold Doherty for providing me with a distraction from being pulverized at the Tribunal. Mr Doherty is doing science again, with the usual autism advocacy standards. This time, Mr Doherty takes on a "myth" about the intake IQs of the experimental group in Lovaas (1987).

So let's look at those intake IQs.

I'm not going to launch into a lecture about deviation vs ratio IQs. This is a relevant issue here, but it can largely be circumvented by using the measure Lovaas (1987) used at intake, the PMA (prorated mental age), and its derived ratio IQs. All the sources I cite are peer-reviewed journal articles with Ivar Lovaas in the authorship, and the information I provide can be verified from these articles.

Using the reported PMA in Lovaas (1987), the mean intake IQ in the experimental group is 63. This is an average score in the "mildly retarded" range.

In the "Method" section, Lovaas (1987) also reports that, at intake, 2 experimental group children were in the normal range of intelligence (IQ 70 or more); none was in the "mildly retarded" range (IQ 50-69); 7 were in the "moderately retarded" range (IQ 35-49); and 10 were in the "severely retarded" range (IQ 20-34). None was in the "profoundly retarded" range (IQ of less than 20). Dr Lovaas also reports that this distribution of IQ scores is "identical to that for Control Group 1".

But--even if you assign the normal range children an IQ of 100--I haven't been able to find any report of the range of ratio IQs for the whole experimental group--and also assign all the other children to the highest possible scores in their respective categories (7 children with 49 IQ; 10 children with 34 IQ), this does not even approach a group average of 63. Instead, the average is 46.

It is statistically improbable that all those kids had, at intake, the highest scores within their assigned ranges, and it also looks impossible for the two normal range children to have ratio IQs as high as 100 (this would be inconsistent with reported deviation IQs; McEachin et al., 1993). But even this exercise in trying to produce the highest possible average intake IQ falls well short (17 points) of the reported average.

In the "Results" section, Lovaas (1987) reports that at intake, there were 10 children in the "moderate to severe range", as opposed to the 17 ("moderate" plus "severe" children) reported earlier in this paper.

Smith et al. (1993) report that 5 of the 19 children in Control Group 1 in Lovaas (1987) were "high-functioning", that is, had an IQ of 70 or more at intake. This contradicts the account in Lovaas (1987), where it is reported that only 2 of the 19 children in both the experimental group and Control Group 1 had IQs in the normal range (70 or more). Smith and Lovaas (1997) report a total of 9 children (experimental group and Control Group 1) who have IQs of 75 or more. If this figure is correct, and if 5 of these children were indeed in Control Group 1 (assuming none of the kids with IQs over 70 had IQs under 75...), then 4 must have been in the experimental group. This leaves open the question of whether the experimental group might have included more than 4 children with IQs of 70 or more.

Smith et al. (1997) correctly report that all children with ratio IQs of less than 37 were excluded from Lovaas (1987). That is, none of the experimental group children in Lovaas (1987) was in the "severely retarded" range at intake. This is contrary to what is stated in Lovaas (1987), where it is reported that 10 children--the majority--are in this range.

Lovaas and Smith (1988) add that the average intake IQ of the 9 children who achieved "normal functioning" in Lovaas (1987) is "slightly under 70". In fact it is 70 (69.7), which would be considered (just) in the normal range. And "in the normal range" is another way to say "high-functioning".

Summary?

Lovaas (1987) does not include any autistic children whose intake IQs are in the "severe" and "profound" ranges. The average experimental group intake IQ is in the "mild" range, and the average for those children who achieved normal functioning is (just) in the normal--"high functioning"--range. And the distribution of intake IQ scores is incorrectly (and inconsistently) reported in Lovaas (1987).

These are data I absorbed long ago and carried around for a long time. Only when I saw Mr Doherty's latest foray into science was I reminded to cough them up. Autism advocates who use websites as authoritative sources miss all the fun (and avoid all the hard work) of looking at data from primary sources. As usual, any factual criticisms are welcome--and if anyone can find (I'm sure it's right in front of me somewhere...) a reported range of ratio IQs (not deviation IQs) for the full experimental group in Lovaas (1987), you'll be awarded a trip to that Tribunal hearing (or maybe not).


References:

Lovaas, O.I. (1987). Behavioral treatment and normal educational and intellectual functioning in young autistic children. Journal of Consulting and Clinical Psychology, 55, 3-9.

Lovaas, O.I., and Smith, T. (1988). Intensive behavioral treatment for young autistic children. In B.B. Lahey, and A.E. Kazdin (Eds.), Advances in Clinical Child Psychology, 11. New York: Plenum Press.

McEachin, J.J., Smith, T., and Lovaas, O.I. (1993). Long-term outcome for children with autism who received early intensive behavioral treatment. American Journal on Mental Retardation, 97, 359-72.

Smith, T., & Lovaas, O.I. (1997). The UCLA Young Autism Project: A reply to Gresham and McMillan. Behavioral Disorders, 22, 202– 218.

Smith, T., McEachin, J.J., & Lovaas, O.I. (1993). Comments on replication and evaluation of outcome. American Journal on Mental Retardation, 97, 385–391.

Smith, T., Eikeseth, S., Klevstrand, M., and Lovaas, O.I. (1997). Intensive behavioral treatment for preschoolers with severe mental retardation and pervasive developmental disorder. American Journal on Mental Retardation 103, 238-249.

15 comments:

Ms. Clark said...

Wow. How many m&m's would Lovaas need to be offered in order for him feel it was important enough to clarify all this scattershot reportage?

If you asked Smith to explain the discrepancies do you suppose he'd answer you?

David N. Andrews MEd (Distinction) said...

For what it's worth to you...

Another issue at play here is that Lovaas' paper (if I remember rightly) does not indicate the pre-test verbal IQs or even the performance IQs... just the full-scale IQs. There is a danger in not specifying since replications cannot be said to faithfully match the conditions of the original study (a full scale IQ of 100 can be obtained from VIQ:PIQ pairs such as 40:60 or 30:70 or even 90:10 - even though that would indeed be an extreme example). VIQ and PIQ should be cited for proper matching to the original study.

As for Doherty 'doing science'...

I shall refuse to sully your blog with my opinion of that notion.

jonathan said...

another issue that you don't really cover is the fact that the tests were different pre and post treatment, that the tests given to 3 year old children and younger are very different than the one's given to 7 year olds and older. If the pre-school IQ tests are predictors of the scores of the WISC IQ tests given to children 7 or older, Lovaas neglects to cite any references. Yet the presumption of his study is that the scores on these two different tests are always highly correlated.

Also, Lovaas does not mention if increases in IQ are correlated with successful adult outcomes which as we know he neglected to publish anywhere due to what Tristram Smith tells me were "logistical problems" whatever that means, but that did not prevent them from making the informal presentations at conferences or using cost-benefit analyses with the claims of successful adult outcomes such as Gina Green's et. al.

David N. Andrews MEd (Distinction) said...

In other words, Lovaas' 'study' was not so much a study, but more of a public relations exercise.

Alyric said...

Well then, this ought to satisfy Mr Doherty:)

I'll refer him to here. Ten bucks says he doesn't permit the post.

Michelle Dawson said...

Hi Ms Clark, it sure would be good to have accurate information... but I think it's up to behaviour analysts to decide if, by their standards, autistics deserve accurate reporting.

Hi Jonathan, re pre-post measures, I was writing about problems in accuracy of reporting, as opposed to problems in methodology. I agree re the absence of adult outcomes. This is one area where Dr Green's work (as one of the authors of Jacobson et al., 1998) is conspicuously non-empirical.

jypsy said...
This comment has been removed by the author.
jypsy said...

(trying my comment again with missing words in place and spelling corrected)

Alyric, just noticed that there was a 7th comment over there and thought you might have lost your ten bucks.... should have known better...

Joeymom said...

I just don't get why everyone is in such a hullaballoo about how wonderful ABA is. Trying to find an alternattive to public school for my son here, I just got back from looking over an ABA-based school. I explained my kid is 5, can speak in sentences, knowing his letters, colors, and numbers, and responds to his name 80% of the time. In other words, the poor kid is "high-functioning." However, he can't carry on a conversation, he has several important sensory issues which impact his abilities to focus and complete tasks, bilateral coordination issues, etc. I asked what they could do for him... and the response was a lot of nothing. Why aren't people working on other therapies and supports for autistic people?

Jennifer said...

Michelle,
I think I remember (although I can't find it now) that there were more details given in the McEachin thesis. You might find the facts of the matter there - if you have it available.

Maya M said...

I am much worried by a new research showing IQ increase by ABA (http://www.soton.ac.uk/mediacentre/news/2007/apr/07_59.shtml). I first learned about it from Harold Doherty's blog mentioned in your post. I can't find anything about this research in PubMed, nor among the IMFAR presentations linked by you. The study isn't reported to be in press, either. It looks like it is still pending peer review and hasn't been presented to the scientific community in any way. Instead, it is presented directly to the public and it is sure to have effect on it.
I don't believe that behavioral training can improve cognitive abilities. I think it can improve just the registered IQ scores by making the child more compliant to do the test. Unfortunately, behavior modification influences exactly what can be observed and measured and so the behaviorists' claims, however counterintuitive, may be quite difficult to disprove by the tools of science. E.g. a behaviorist can take patients with personality split, train them not to speak about their alter ego or manifest it in any way and then claim to have cured them. (I hope this idea won't inspire a research, but if it does, I insist to be listed among the authors :). A simple acknowledgement won't do.)

Michelle Dawson said...

Remington et al. (in press), the study Maya M refers to, is one of two UK ABA non-randomized controlled trials that are now in press (at different journals). The other is from Patricia Howlin and colleagues. Data from this other ABA non-randomized controlled trial were presented at IMFAR 2007 (you can, for now at least, find the abstract here).

While I might get round to commenting on Dr Howlin's study as it was presented at IMFAR, Remington et al. (in press, at a journal which does not make its in press articles available) was not presented at IMFAR and is not yet generally available.

Leading autism advocates routinely make definitive pronouncements about data they've never seen. But this isn't something I'm willing to do.

Autism research-related press releases are not always inaccurate, exaggerated, and/or misleading. But many of them are, as are media stories about autism (including about autism treatments). In my view, it's important to look at actual studies and the data they report, rather than the fanfare surrounding them.

Anonymous said...

To you all .... at the end of the day I know that that the 5 year struggle of providing ABA to my son has provided him and the rest of my family with a quality of life that I never thought we would achieve, unlike "booking an institution" upon diagnosis. Debate the research papers and get caught up in the percentages but while your doing that I recommend utilising your time to provide learning opportunities for your children to compensate for their deficits.
Its the hardest job I have ever undertaken and out of the learning theories that i applied to my son ABA was what assisted him to learn.
Best of luck to you all

Regards, Donna

Michelle Dawson said...

Hi Donna,

Vast numbers of parents have written testimonials similar to yours, extolling the great benefits of secretin, chelation, RDI, the Judge Rotenberg Center, etc., and claiming that without these interventions, their autistic child would have been doomed.

Sorry, but my view is that autistics deserve to benefit from and be protected by recognized standards of science and ethics.

Anonymous said...

I have been an ABA provider for the past 8 years. I was able to participate in a replication of the Lovaas study that was done in Arizona (funded by the government). I worked with all of the children at one point or another. I noticed that the children who "recovered" where all very high functioning to begin with. I feel like the company I work for took credit for "recovery" when developmental milestones would have been met regardless of intervention. Having said that, I believe that ABA can make observable changes and is science based. However, I do question the term "recovery" as well as the Lovaas study.