Early autism intervention research carries consequences for all autistics, and for a lot of nonautistics as well. DawsonG et al. (2010) is a newly-published randomized controlled trial of a newly-developed manualized very early autism intervention and as such deserves a close look.
Developed by Sally Rogers and Geraldine Dawson, the Early Start Denver Model has been the subject of a handful of previous papers, none involving anything approaching a major trial. ESDM is reported to combine the Denver Model with Pivotal Response Training, two interventions developed in the 1980s which individually lack good quality evidence for their benefits or harms. If you are interested in ESDM and its possible fall-out, I suggest reading the existing DM, PRT, and ESDM literatures.
A book chapter, possibly written circa 2006, describes the ESDM RCT, apparently before all the results were in (SmithM et al., 2008), and is essential reading, as is the riveting recently-published ESDM manual (Rogers & DawsonG, 2010).
My first exposure to the foundations of ESDM was a 2003 presentation by Sally Rogers. She showed a video of an adult picking up two blocks and banging them together and putting them down. In response a very young nonautistic child automatically picked up the blocks, banged them together, and put them down. A very young autistic child immediately noticed that the blocks were very interesting-shaped (and really they were--these were incredibly interesting blocks), picked them up, and inspected them closely seeing as they were so darn interesting.
You can guess which child's response was deemed wonderful and praiseworthy and "right," and which child's response was deemed defective and unfortunate and totally wrong. With this in mind, here are some comments:
About the ESDM
1. According to current definitions, ESDM is an "eclectic" autism intervention, combining ABA-based with non-ABA-based approaches. The manual specifies that ESDM "most closely resembles" RDI, DIR/Floortime, Hanen, responsiveness training, and SCERTS. ESDM also "has clear ties to" PRT, incidental teaching, and milieu teaching. And ESDM "has in common" some elements of ABA-based interventions as per Lovaas. Speech and occupational therapy are also thrown in. PROMPT too. The wide range of intervention approaches constantly available to children receiving ESDM is emphasized in all its available descriptions.
2. On principle and on the basis of small poor-quality trials, major behaviour analysts have claimed that "eclectic" autism interventions are ineffective; two recent examples here and here. The Association of Professional Behavior Analysts has thoroughly condemned "eclectic" approaches to autism, to uncritical applause by the Association for Behavior Analysis International's autism special interest group.
3. The reported overarching theoretical basis for ESDM is a 1985 book by the well-known psychoanalyst, Daniel Stern. Other psychoanalytic influences have been reported for the DM component of ESDM.
4. ESDM is further premised on autism being (from the manual) "at its heart a social disorder." Autistics' profound (entirely one-sided) failures in displaying proper interpersonal relationships and proper social motivation produce a cascade of purely negative effects disrupting all areas of proper development and learning.
5. The ESDM goal: autistics must become as "normal" (from the manual) as possible. Every waking hour, autistics must work towards the one ESDM-recognized proper way to relate, learn, play, develop, etc.
6. Autistics' tendency to learn in atypical ways is treated as though disordered and harmful--as evidence for our primary social deficits, which in turn deny us all the proper, typical "crucial learning experiences" (e.g., "birthday party games!"--from the ESDM manual).
7. The only proper ESDM way to learn is through typical interpersonal interaction involving proper, typical use of developmentally-proper highly-rationed highly-controlled materials which must only be regarded and manipulated by the child in the one proper socially-related ESDM manner.
8. Keeping information away from autistic children who are persistently at risk of learning from it in non-ESDM-approved ways is a theme throughout the ESDM manual. While it is claimed that ESDM does not require a therapy room, in fact there are many instructions as to how such a room should be set up. This is representative: "Ideally the room should be able to be arranged with nothing in it except a table and chairs and a closed or covered cabinet or shelves." This is called "the natural environment."
9. While a great advantage of ESDM is reported to be that it is "fun to do!" in fact the manual sternly declares, when contemplating autistics who may enjoy learning in the wrong non-ESDM way: "Our job is not to keep children happy; it is to teach them their objectives." And that very limited hierarchical range of objectives must be rigidly achieved in the one rigid proper manner and the one rigid proper order.
About the design of the ESDM RCT (DawsonG et al., 2010)
1. The trial spanned 2003-2008 and was registered more than a year after it started. Many of the reported planned measures (see SmithM et al., 2008), particularly those related to brain activity, go unmentioned in DawsonG et al. (2010).
2. Basic description: 48 autistic children within the age range of 18 to 30 months (actual range is not reported) were randomized to two groups, to recieve either a minimum of 2yrs of ESDM or of services available in the community. Children were evaluated with four main measures (Mullen, Vineland, ADOS, a scale of repetitive behaviours) and according to their diagnostic status, after about one then two years. At the two-year evaluation, children had received an average of 29 months of intervention. Children had to be at least 48 months at the 2-year evaluation; therefore children younger at intake received more intervention than children older at intake.
3. Numerous and strict, but not atypical, exclusion criteria were applied. For example, children considered to fall into the "severe" and "profound" ranges of intellectual disability were excluded, as has been done in many popularly-cited ABA controlled trials.
4. In this trial at least, the ESDM is not an intensive intervention. The authors aimed for 20hrs/wk, but achieved only 15.2 (standard deviation 1.4), which falls into the range of low intensity as currently defined.
5. Parents were expected to apply ESDM principles for an additional 5hrs/wk, but in fact reported (how accurately is unclear) more than triple this amount, indicating great enthusiasm and high expectations. ESDM parents also reported using non-ESDM services for about 5hrs/wk.
6. By the rock-bottom standards of the ambient autism literature, the ESDM trial design has many notable strengths. The intervention is manualized and attempts were made to establish treatment fidelity for the ESDM group. Currently-recognized gold-standard diagnostic instruments were used. By non-autism non-ABA standards, this is a very small trial, but it improves on an abysmal literature rife with even tinier trials. Most important, against the strong trend denying autistics even the possibility of good experimental design, this is an inexcusably rare RCT.
7. However, by very well-established standards in non-autism non-ABA areas, the ESDM RCT is in many respects poorly reported, which makes it difficult to properly asssess its design.
8. The control condition lacks virtually any useful description much less a rationale. Unspecified individuals provided the control group parents with unspecified information (what "resource manuals"? what "reading materials"?) including about unspecified services. The authors briefly trot out a few generic types of available services without indicating whether they were recommended or chosen. Children in this group received on average ~9hrs/wk of unspecified individual therapy plus the same of unspecified group-level therapy. Only mean intensities are reported--no SDs much less ranges. We don't know whether some children received huge amounts of intervention while others received none.
9. In my view, the authors' documented astonishing lack of interest in their own control group speaks to the very human expectations and biases applied to this group, which would be unlikely not to affect their outcomes.
10. Crucial information about who or what generated the randomization sequence, and whether or how allocation was concealed, is missing.
11. Remarkably, 51 autistic children were originally randomized to the two groups. Three disappear off to the side somehow, no explanation given, after random assigment to groups but without showing up in either group. Two assigned to the ESDM group rejected this intervention due to its requirements. The third, of unreported group assignment, was found to be not autistic, but Rett's. But none of these randomly allocated children shows up in their assigned group.
12. Even with the 48 children deemed to be assigned to a group, there is no intention to treat analysis. Data were ultimately lost from 3 children in the control group, no reason given.
13. The pre-post use of the ADOS as a scale of "severity" raises numerous questions. The first preliminary attempt at creating such a scale was not accepted for publication until the end of 2008 (and not epublished until 2009). This scale is further based on the revised ADOS algorithm, which is not cited by DawsonG et al. (2010) and was not accepted for publication until mid-2006.
14. Blinding is notoriously difficult in these kinds of trials. Parents cannot be blinded to intervention group, but some measures rely on parent report. While evaluators are described as "naive to invervention status," they are not described as blind to the nature and purpose of the study or of ESDM. Those involved in providing ESDM would in addition be aware of the study's purpose, importance, and outcome measures.
About the reported results of the ESDM RCT (DawsonG et al., 2010)
1. The ESDM manual provides results claimed to be from DawsonG et al. (2010), which is referred to both as "in press" and as a published 2010 Pediatrics paper. Inexplicably, information and data in the manual reported as being from DawsonG et al. (2010) differs somewhat from what is reported in the paper. There are also contradictions about the RCT within the ESDM manual regarding, for example, number of children in the ESDM group and intensity of the intervention they received.
2. There is an obvious error in the data in DawsonG et al. (2010). I spotted it immediately on first reading, as should anyone--you don't need to be able to do math in your head--who actually looks at the data. For two of the Vineland subscales (daily living and motor skills), some data are apparently reversed from Table 1 to Table 2, such that data in one table does not correspond to data in the other. The persistence of such an obvious error through copy-editing and so on raises questions about whether there are less obvious errors.
3. The main result is in the Mullen composite. On this score, the ESDM group finished at ~79, representing an ~18 point change from baseline but almost no change from the first year score of ~76. There is a just-significant 11-point change score advantage for the ESDM group over their controls at the two-year evaluation, an advantage which diminished from first to second year evaluations. The Mullen ESDM advantage is concentrated in the expressive and receptive language subscales.
4. Most striking re the Mullen composite is the huge increase in the scatter of ESDM scores. First and second year SDs are ~23 and ~24, much higher than baseline (SD ~9) and much higher than in the control group. Regardless, ranges of scores are not provided. But clearly one effect of ESDM is to dramatically increase variability of outcomes at a very young age. The ~11-point year two ESDM group advantage is drowned in their ~24-point SD.
5. While the ESDM group outperformed their controls on the Vineland composite, this results from the control condition reducing Vineland scores. These scores depend on unblinded parent report, which would plausibly reflect the disinterest and low expectations, obvious in how this study is reported, conveyed and applied to the control group. The ESDM group, regardless of their important advantages in this respect, did not experience any improvement in their Vineland scores over two years.
6. In addition, the ESDM group's Vineland scores show a huge increase in scatter, not found in the controls, by year two. The SD balloons from ~6 to ~16 (and from ~8 to ~22 in the communication sub-scale). Again, the ESDM's effect is to greatly increase variability in outcomes.
7. The authors dedicate a lot of space to claims of group differences in subjectively-determined diagnostic status. Those changes were limited to moves between autism and PDD-NOS--none of the children were judged nonautistic. However, on the ADOS, the only reported quantitative measure which offers the possibility of differentiating PDD-NOS from autism, the groups did not differ in outcome. Also, diagnosis was remarkably stable in this study, compared to previous findings regarding children in the 30-month-or-younger age range.
8. Apart from not differing on ultimate ADOS "severity" scores, the groups did not differ on a repetitive behaviours scale. The ESDM did not make autistic children measurably "less severely autistic" by the usual standards.
9. The premise of ESDM is that only through displaying proper typical social interaction and behaviour can autistics begin to learn anything worth learning. In the two reported measures of social abilities (ADOS, Vineland socialization), the ESDM children did not do any better than their controls at either evaluation. They did not improve (by the usual standards) pre-post, with losses in Vineland socialization.
10. The ubiquitous claim that "earlier is better" accompanying the publicity for this study is unjustified. The authors made no attempt to correlate outcomes with their 12-month possible age-range at intake. To the contrary, the authors enacted the view that younger children at intake must receive more intervention in order to be judged alongside children who were older at intake--the contrary position to "earlier is better."
11. The ESDM can only be used up to 60 months, with objectives targeted at children up to 48 months. At the end of this RCT, the ESDM children had a possible age range (actual range not reported) from 48 to 54 months. All were still autistic or PDD-NOS. Given the enormous increase in scatter of Mullen and Vineland scores, some had done extremely poorly by the usual standards. Against the unfounded claims of "earlier is better," questions arise as to how being so judged at such a young age within a highly publicized intervention will affect these children.
12. In reporting the ESDM RCT, the ESDM manual states that "ESDM needs to be independently replicated before it can be considered an empirically supported treatment." But a recent paper whose senior author is Sally Rogers claims that ESDM is "evidence-based" and that its components are "empirically supported."
A few last comments
Eric Fombonne has published a Pediatrics eLetter about the ESDM RCT. He concurs with some points I make, not with others, and he failed to spot the error in the data.
If you rummage around in the ClinicalTrials.gov site you will find an in-progress multi-site RCT of the ESDM where intake age range has been dropped to 12 to 24 months. You will also find a multi-site RCT, targeting 5-12 month old children considered "at risk" for autism, which will likely use ESDM-type intervention in an effort to prevent autism.
As a large body of work, the ESDM manual, RCT, and other related materials, including the formal and informal work of Geraldine Dawson and Sally Rogers, read like a total condemnation of what I've done in the above. I'm not supposed to notice details and errors, contradictions and anomalies--or if I inappropriately do, I should forget about them and concentrate on emitting socially-appropriate responses (pick up the blocks, bang them together, put them down--don't look at them). I'm not supposed to notice other discrepancies, for instance between recognized scientific and ethical standards and those imposed on autistics. I'm supposed to direct all my effort towards the developmentally-appropriate equivalent of "birthday party games!"
The above should be verifiable, through reading the supplied sources. As usual, if I've made any factual errors, always a possibility, please let me know.
References:
Dawson, G., Rogers, S., Munson, J., Smith, M., Winter, J., Greenson, J., Donaldson, A., & Varley, J. (2009). Randomized, Controlled Trial of an Intervention for Toddlers With Autism: The Early Start Denver Model PEDIATRICS, 125 (1) DOI: 10.1542/peds.2009-0958
Rogers, S.J., & Dawson, G. (2010). Early Start Denver Model for Young Children with Autism. New York, NY: Guilford Press.
Smith, M., Rogers, S., & Dawson, G. (2008). The Early Start Denver Model: a comprehensive early intervention approach for toddlers with autism. In: Handleman J.S., Harris S.L., eds., Preschool Education Programs for Children With Autism, 3rd ed. Austin, TX: Pro-Ed, pp. 65–101.
16 comments:
Do I take it that no study which mentions ABA in a positive light even transiently or insubstantially can ever be considered as valid even if it is, according to your epistemology?
When Dawson meets Dawson, then comes the tug of war as they say :)
The 'author' must be new otherwise he would have anticipated Michelle's likely response and not posted that.
Michelle has criticised non-ABA, non-behavioural ideas, standards, research and organisations much more harshly than she ever has ABA or behavioural ideas, standards, research and organisations. But only those who support ABA react in the extraordinary way they do, sending out a message that criticism is not as welcome as it is in other fields.
Or 'the author' could have just read the post properly. There's enough specific criticism there that prejudice is unnecessary for saying this study has problems.
Yes, those blocks were interesting.
I should hope that the person banging them did not make too loud a noise, or I would startle.
It was interesting to notice a loss in Socialisation according to the Vineland, rather than it staying the same or getting better.
Also, the ESDM seems to be (I think) "a jack of all trades and a master of none", a macedoine of interventions, which may or may not work well on their own. We don't know yet.
And in particular, it is based on a psychoanalytic model.
I have had reasons and causes of my own to look at psychoanalytic models.
Have a few comments about points 4 to 8.
In regard to point 1 which talks about the design, how was it that they did not integrate the brain measures? Was it too hard? Not conclusive?
About point three: was the selection to those who needed it the most or those who could potentially get the most out of it? Which is to say, those who were least deviant in the first place?
I saw point 6 in the design. Credit where credit is due.
Points 10 and 14. The design needed tightening up there. And the people who are in it would behave differently.
And wasn't Mahler in the same generation and culture as Kanner and Asperger?
Thank you for taking the time to compile your observations and thoughts on the ESDM work. It's nice to see a sober assessment of this study, as opposed to all the self-congratulatory assessments. (Fombonne's letter to Pediatrics was also helpful in that regard.)
Adelaide Dupont asked, "how was it that they did not integrate the brain measures?"
That's an excellent question. Possibly the reported-as-planned measures of brain activity will appear in later papers? I can only guess.
Re "was the selection to those who needed it the most or those who could potentially get the most out of it," the exclusion criteria are not unusual for trials of early autism interventions (though some trials have fewer). In his eLetter, Eric Fombonne addresses possible associated weaknesses.
Margaret Mahler lived from 1897 to 1985, Kanner from 1894 to 1981, and Asperger from 1906 to 1980, so it looks fair to put them roughly in the same generation.
In response to Alan, I was very happy to see Fombonne's eLetter; by the going standards, it is a good surprise.
The brain measures
All the brain measures I have heard of are electro-encephalo-grams and Magnetic Resource Imaging.
Most recently I have heard and read of microcolumns.
There was an article also about the corpus callosum, which looked promising.
Alan:
It was good to see in Autistic Aphorisms the reference to the study of infant siblings. And your words on the autistic perceptual difference.
Will probably make a comment where appropriate (and possibly when I have time: a few bullet points here and there).
And that's what I really want to know: What makes non-autistic perception non-autistic?
And autistic play activities are functional towards the development of autistic cognition.
From SmithM et al. (2008; see original post for full reference), re the ESDM RCT:
"Children were administered a comprehensive assessment of brain function and development which included electrophysiological responses to social and linguistic stimuli, neuropsychological tasks, structural magnetic resonance imaging (MRI), and MR spectroscopy."
We fully support ABA as a foundation for teaching. As a speech language pathologist I view teaching communication using the terminology 1)form, 2)content and 3)use. ABA is a form of teaching using proven teaching principles to teach new skills. In your review of the ESDM you state that the "ESDM is an eclectic autism intervention." In terms of form of teaching, the ABA foundation is not violated, but is promoted in our reading of the ESDM book. Many misinterpret ABA (form of teaching) as a total program, without giving consideration to the content and use of the overall program. The eclectic part of the ESDM is the fact that the authors draw from an evidence based ABA relative called Pivotal Response Training (PRT). PRT (Koegel & Koegel, 1995), has a foundational teaching strategy using ABA principles needed for teaching, but emphasizes teaching in a natural format,using motivation of the child,teaching language, play and social skills, (content). The natural environment (a place where we all function) is the perfect place to teach new skills and promote socialization and generalization (use). So the ESDM is an eclectic approach in that it embraces new ideas and disciplines while upholding foundational ABA teaching strategies.
In summary, the ESDM upholds the teaching principles of ABA (form of teaching), using research from the PRT and DM models (content), promoting natural, social interactions (use). We treat children with autism with educational training in the fields of speech language pathology, occupational therapy, and ABA. Our form of teaching is always ABA based (form), therapy is drawn from our therapy backgrounds (content), promoting social interactions and learning in a natural environment (use).
The description I provide of ESDM (original post) can be verified by reading the literature I refer to. See for example what the "natural environment" is, according to the ESDM manual.
ESDM is an "eclectic" autism intervention, according to how ESDM is decribed in the ESDM manual and in the ESDM literature to date.
So far, ABA-based autism interventions, including PRT, lack good quality evidence for their benefits and harms. Instead there is a glut of poor quality evidence, combined with a high level of promotion, marketing, and so on.
In contrast, see this post for an idea of the kind of fair test that is still absent from the ABA literature in autism, all ~50 years of it.
Michelle,
I personally think the ESDM is more of a behaviourally based program that uses ABA principles. While it is defined as "eclectic" given current research definitions, all of the ABA principles including reinforcement, shaping, fading, etc. are used.
In response to Autistic Wisdom:
1. The ESDM literature (peer-reviewed papers, book chapter, manual) does not describe ESDM as solely or primarily ABA-based.
2. The ESDM manual describes ESDM as "most closely" resembling a list of non-ABA approaches. These are combined with ABA-based approaches.
3. The ESDM manual includes information about ESDM like the following:
"While many of these activities resemble those found in RDI (Gutstein & Sheely, 2002) or DIR/Floortime (Greenspan et al., 1997), they were created independently and early in the development of the Denver Model (Rogers et al., 1986; Rogers & Lewis, 1989), long before our first exposure to either of these other models."
4. As with ESDM, where the stated major theoretical foundation is a well-known book by a psychoanlyst (see original post), the Denver Model has an explicitly psychoanalytic theoretical foundation.
5. I suggest reading the Denver Model and ESDM literatures (including the ESDM manual).
Michelle, how do you explain your comment that PRT is not evidence-based? Over 200 research articles and being recommended as one of the 10 autism treatments by the National Research Council is not insufficient?
As I wrote, PRT as an autism intervention lacks good quality evidence for its benefits and harms.
PRT has widely been promoted as "evidence based" but then so have numerous other autism interventions that have not yet been fairly tested.
Intervention programs without any peer-reviewed papers reporting any kind of study involving child outcomes were welcomed in the NRC (2001) 10. See pages 166 - 172 for the reported evidence, or lack thereof, for the 10 programs, including PRT.
While I understand this is an evaluation of the technical components of the study itself--let me state that as a mother of a child whom is finishing up two years of participation in this study, that my childs life was 110% without a doubt changed because of it! That may not work for the scientific analysis of the study itself, but changing the path of a childs life is what it is truly all about, and it worked for us!
Dear Michelle,
Have you considered refraining from using the essentialistic term 'autistics' to refer collectively to individuals diagnosed with an autism spectrum disorder?
Who really defines normal? We are all different and you just want everyone to be the same.
Post a Comment