Early autism intervention research carries consequences for all autistics, and for a lot of nonautistics as well. DawsonG et al. (2010) is a newly-published randomized controlled trial of a newly-developed manualized very early autism intervention and as such deserves a close look.
Developed by Sally Rogers and Geraldine Dawson, the Early Start Denver Model has been the subject of a handful of previous papers, none involving anything approaching a major trial. ESDM is reported to combine the Denver Model with Pivotal Response Training, two interventions developed in the 1980s which individually lack good quality evidence for their benefits or harms. If you are interested in ESDM and its possible fall-out, I suggest reading the existing DM, PRT, and ESDM literatures.
A book chapter, possibly written circa 2006, describes the ESDM RCT, apparently before all the results were in (SmithM et al., 2008), and is essential reading, as is the riveting recently-published ESDM manual (Rogers & DawsonG, 2010).
My first exposure to the foundations of ESDM was a 2003 presentation by Sally Rogers. She showed a video of an adult picking up two blocks and banging them together and putting them down. In response a very young nonautistic child automatically picked up the blocks, banged them together, and put them down. A very young autistic child immediately noticed that the blocks were very interesting-shaped (and really they were--these were incredibly interesting blocks), picked them up, and inspected them closely seeing as they were so darn interesting.
You can guess which child's response was deemed wonderful and praiseworthy and "right," and which child's response was deemed defective and unfortunate and totally wrong. With this in mind, here are some comments:
About the ESDM
1. According to current definitions, ESDM is an "eclectic" autism intervention, combining ABA-based with non-ABA-based approaches. The manual specifies that ESDM "most closely resembles" RDI, DIR/Floortime, Hanen, responsiveness training, and SCERTS. ESDM also "has clear ties to" PRT, incidental teaching, and milieu teaching. And ESDM "has in common" some elements of ABA-based interventions as per Lovaas. Speech and occupational therapy are also thrown in. PROMPT too. The wide range of intervention approaches constantly available to children receiving ESDM is emphasized in all its available descriptions.
2. On principle and on the basis of small poor-quality trials, major behaviour analysts have claimed that "eclectic" autism interventions are ineffective; two recent examples here and here. The Association of Professional Behavior Analysts has thoroughly condemned "eclectic" approaches to autism, to uncritical applause by the Association for Behavior Analysis International's autism special interest group.
3. The reported overarching theoretical basis for ESDM is a 1985 book by the well-known psychoanalyst, Daniel Stern. Other psychoanalytic influences have been reported for the DM component of ESDM.
4. ESDM is further premised on autism being (from the manual) "at its heart a social disorder." Autistics' profound (entirely one-sided) failures in displaying proper interpersonal relationships and proper social motivation produce a cascade of purely negative effects disrupting all areas of proper development and learning.
5. The ESDM goal: autistics must become as "normal" (from the manual) as possible. Every waking hour, autistics must work towards the one ESDM-recognized proper way to relate, learn, play, develop, etc.
6. Autistics' tendency to learn in atypical ways is treated as though disordered and harmful--as evidence for our primary social deficits, which in turn deny us all the proper, typical "crucial learning experiences" (e.g., "birthday party games!"--from the ESDM manual).
7. The only proper ESDM way to learn is through typical interpersonal interaction involving proper, typical use of developmentally-proper highly-rationed highly-controlled materials which must only be regarded and manipulated by the child in the one proper socially-related ESDM manner.
8. Keeping information away from autistic children who are persistently at risk of learning from it in non-ESDM-approved ways is a theme throughout the ESDM manual. While it is claimed that ESDM does not require a therapy room, in fact there are many instructions as to how such a room should be set up. This is representative: "Ideally the room should be able to be arranged with nothing in it except a table and chairs and a closed or covered cabinet or shelves." This is called "the natural environment."
9. While a great advantage of ESDM is reported to be that it is "fun to do!" in fact the manual sternly declares, when contemplating autistics who may enjoy learning in the wrong non-ESDM way: "Our job is not to keep children happy; it is to teach them their objectives." And that very limited hierarchical range of objectives must be rigidly achieved in the one rigid proper manner and the one rigid proper order.
About the design of the ESDM RCT (DawsonG et al., 2010)
1. The trial spanned 2003-2008 and was registered more than a year after it started. Many of the reported planned measures (see SmithM et al., 2008), particularly those related to brain activity, go unmentioned in DawsonG et al. (2010).
2. Basic description: 48 autistic children within the age range of 18 to 30 months (actual range is not reported) were randomized to two groups, to recieve either a minimum of 2yrs of ESDM or of services available in the community. Children were evaluated with four main measures (Mullen, Vineland, ADOS, a scale of repetitive behaviours) and according to their diagnostic status, after about one then two years. At the two-year evaluation, children had received an average of 29 months of intervention. Children had to be at least 48 months at the 2-year evaluation; therefore children younger at intake received more intervention than children older at intake.
3. Numerous and strict, but not atypical, exclusion criteria were applied. For example, children considered to fall into the "severe" and "profound" ranges of intellectual disability were excluded, as has been done in many popularly-cited ABA controlled trials.
4. In this trial at least, the ESDM is not an intensive intervention. The authors aimed for 20hrs/wk, but achieved only 15.2 (standard deviation 1.4), which falls into the range of low intensity as currently defined.
5. Parents were expected to apply ESDM principles for an additional 5hrs/wk, but in fact reported (how accurately is unclear) more than triple this amount, indicating great enthusiasm and high expectations. ESDM parents also reported using non-ESDM services for about 5hrs/wk.
6. By the rock-bottom standards of the ambient autism literature, the ESDM trial design has many notable strengths. The intervention is manualized and attempts were made to establish treatment fidelity for the ESDM group. Currently-recognized gold-standard diagnostic instruments were used. By non-autism non-ABA standards, this is a very small trial, but it improves on an abysmal literature rife with even tinier trials. Most important, against the strong trend denying autistics even the possibility of good experimental design, this is an inexcusably rare RCT.
7. However, by very well-established standards in non-autism non-ABA areas, the ESDM RCT is in many respects poorly reported, which makes it difficult to properly asssess its design.
8. The control condition lacks virtually any useful description much less a rationale. Unspecified individuals provided the control group parents with unspecified information (what "resource manuals"? what "reading materials"?) including about unspecified services. The authors briefly trot out a few generic types of available services without indicating whether they were recommended or chosen. Children in this group received on average ~9hrs/wk of unspecified individual therapy plus the same of unspecified group-level therapy. Only mean intensities are reported--no SDs much less ranges. We don't know whether some children received huge amounts of intervention while others received none.
9. In my view, the authors' documented astonishing lack of interest in their own control group speaks to the very human expectations and biases applied to this group, which would be unlikely not to affect their outcomes.
10. Crucial information about who or what generated the randomization sequence, and whether or how allocation was concealed, is missing.
11. Remarkably, 51 autistic children were originally randomized to the two groups. Three disappear off to the side somehow, no explanation given, after random assigment to groups but without showing up in either group. Two assigned to the ESDM group rejected this intervention due to its requirements. The third, of unreported group assignment, was found to be not autistic, but Rett's. But none of these randomly allocated children shows up in their assigned group.
12. Even with the 48 children deemed to be assigned to a group, there is no intention to treat analysis. Data were ultimately lost from 3 children in the control group, no reason given.
13. The pre-post use of the ADOS as a scale of "severity" raises numerous questions. The first preliminary attempt at creating such a scale was not accepted for publication until the end of 2008 (and not epublished until 2009). This scale is further based on the revised ADOS algorithm, which is not cited by DawsonG et al. (2010) and was not accepted for publication until mid-2006.
14. Blinding is notoriously difficult in these kinds of trials. Parents cannot be blinded to intervention group, but some measures rely on parent report. While evaluators are described as "naive to invervention status," they are not described as blind to the nature and purpose of the study or of ESDM. Those involved in providing ESDM would in addition be aware of the study's purpose, importance, and outcome measures.
About the reported results of the ESDM RCT (DawsonG et al., 2010)
1. The ESDM manual provides results claimed to be from DawsonG et al. (2010), which is referred to both as "in press" and as a published 2010 Pediatrics paper. Inexplicably, information and data in the manual reported as being from DawsonG et al. (2010) differs somewhat from what is reported in the paper. There are also contradictions about the RCT within the ESDM manual regarding, for example, number of children in the ESDM group and intensity of the intervention they received.
2. There is an obvious error in the data in DawsonG et al. (2010). I spotted it immediately on first reading, as should anyone--you don't need to be able to do math in your head--who actually looks at the data. For two of the Vineland subscales (daily living and motor skills), some data are apparently reversed from Table 1 to Table 2, such that data in one table does not correspond to data in the other. The persistence of such an obvious error through copy-editing and so on raises questions about whether there are less obvious errors.
3. The main result is in the Mullen composite. On this score, the ESDM group finished at ~79, representing an ~18 point change from baseline but almost no change from the first year score of ~76. There is a just-significant 11-point change score advantage for the ESDM group over their controls at the two-year evaluation, an advantage which diminished from first to second year evaluations. The Mullen ESDM advantage is concentrated in the expressive and receptive language subscales.
4. Most striking re the Mullen composite is the huge increase in the scatter of ESDM scores. First and second year SDs are ~23 and ~24, much higher than baseline (SD ~9) and much higher than in the control group. Regardless, ranges of scores are not provided. But clearly one effect of ESDM is to dramatically increase variability of outcomes at a very young age. The ~11-point year two ESDM group advantage is drowned in their ~24-point SD.
5. While the ESDM group outperformed their controls on the Vineland composite, this results from the control condition reducing Vineland scores. These scores depend on unblinded parent report, which would plausibly reflect the disinterest and low expectations, obvious in how this study is reported, conveyed and applied to the control group. The ESDM group, regardless of their important advantages in this respect, did not experience any improvement in their Vineland scores over two years.
6. In addition, the ESDM group's Vineland scores show a huge increase in scatter, not found in the controls, by year two. The SD balloons from ~6 to ~16 (and from ~8 to ~22 in the communication sub-scale). Again, the ESDM's effect is to greatly increase variability in outcomes.
7. The authors dedicate a lot of space to claims of group differences in subjectively-determined diagnostic status. Those changes were limited to moves between autism and PDD-NOS--none of the children were judged nonautistic. However, on the ADOS, the only reported quantitative measure which offers the possibility of differentiating PDD-NOS from autism, the groups did not differ in outcome. Also, diagnosis was remarkably stable in this study, compared to previous findings regarding children in the 30-month-or-younger age range.
8. Apart from not differing on ultimate ADOS "severity" scores, the groups did not differ on a repetitive behaviours scale. The ESDM did not make autistic children measurably "less severely autistic" by the usual standards.
9. The premise of ESDM is that only through displaying proper typical social interaction and behaviour can autistics begin to learn anything worth learning. In the two reported measures of social abilities (ADOS, Vineland socialization), the ESDM children did not do any better than their controls at either evaluation. They did not improve (by the usual standards) pre-post, with losses in Vineland socialization.
10. The ubiquitous claim that "earlier is better" accompanying the publicity for this study is unjustified. The authors made no attempt to correlate outcomes with their 12-month possible age-range at intake. To the contrary, the authors enacted the view that younger children at intake must receive more intervention in order to be judged alongside children who were older at intake--the contrary position to "earlier is better."
11. The ESDM can only be used up to 60 months, with objectives targeted at children up to 48 months. At the end of this RCT, the ESDM children had a possible age range (actual range not reported) from 48 to 54 months. All were still autistic or PDD-NOS. Given the enormous increase in scatter of Mullen and Vineland scores, some had done extremely poorly by the usual standards. Against the unfounded claims of "earlier is better," questions arise as to how being so judged at such a young age within a highly publicized intervention will affect these children.
12. In reporting the ESDM RCT, the ESDM manual states that "ESDM needs to be independently replicated before it can be considered an empirically supported treatment." But a recent paper whose senior author is Sally Rogers claims that ESDM is "evidence-based" and that its components are "empirically supported."
A few last comments
Eric Fombonne has published a Pediatrics eLetter about the ESDM RCT. He concurs with some points I make, not with others, and he failed to spot the error in the data.
If you rummage around in the ClinicalTrials.gov site you will find an in-progress multi-site RCT of the ESDM where intake age range has been dropped to 12 to 24 months. You will also find a multi-site RCT, targeting 5-12 month old children considered "at risk" for autism, which will likely use ESDM-type intervention in an effort to prevent autism.
As a large body of work, the ESDM manual, RCT, and other related materials, including the formal and informal work of Geraldine Dawson and Sally Rogers, read like a total condemnation of what I've done in the above. I'm not supposed to notice details and errors, contradictions and anomalies--or if I inappropriately do, I should forget about them and concentrate on emitting socially-appropriate responses (pick up the blocks, bang them together, put them down--don't look at them). I'm not supposed to notice other discrepancies, for instance between recognized scientific and ethical standards and those imposed on autistics. I'm supposed to direct all my effort towards the developmentally-appropriate equivalent of "birthday party games!"
The above should be verifiable, through reading the supplied sources. As usual, if I've made any factual errors, always a possibility, please let me know.
References:
Dawson, G., Rogers, S., Munson, J., Smith, M., Winter, J., Greenson, J., Donaldson, A., & Varley, J. (2009). Randomized, Controlled Trial of an Intervention for Toddlers With Autism: The Early Start Denver Model PEDIATRICS, 125 (1) DOI: 10.1542/peds.2009-0958
Rogers, S.J., & Dawson, G. (2010). Early Start Denver Model for Young Children with Autism. New York, NY: Guilford Press.
Smith, M., Rogers, S., & Dawson, G. (2008). The Early Start Denver Model: a comprehensive early intervention approach for toddlers with autism. In: Handleman J.S., Harris S.L., eds., Preschool Education Programs for Children With Autism, 3rd ed. Austin, TX: Pro-Ed, pp. 65–101.
