In this study, 106 autistic children (mean age 8.5yrs, range 3-14.2yrs) were receiving intensive (27-35hrs/wk) one-on-one ABA-based interventions (the ABA school group), while 51 children received "eclectic" non-ABA autism-specialized services (27-30hrs/wk; the ASD unit children).
Some of these children (109) were judged to be in various ranges of intellectual ability, but this information was gathered by questioning individuals who worked with the children and was not verified in any way. Similarly, there is no information about these children's diagnoses, beyond that they were "children with autism spectrum disorder," and for seven children, even their age was unknown.
All 157 autistic children's challenging behaviours were assessed through a version of the Behavior Problems Inventory (Rojahn et al., 2001; Murphy et al., in press, take their description of this instrument nearly verbatim from this earlier paper). This instrument involves interviewing informants (rather than direct observation) and divides challenging behaviours into three categories: stereotypy, self-injury, and aggression.
Regardless of the reasonable sample size, this study's authors chose to organize their data in ways that made many statistical comparisons useless. There is a clear trend, though, if you slog through what information they do provide. This is summarized in the discussion, about comparisons between children in the ABA school group and children in the non-ABA ASD unit group:
With regard to the type of intervention received being a risk factor, findings show that the type of intervention is not related to the prevalence of challenging behavior. However, in relation to participants receiving an ABA intervention, they displayed a higher frequency of stereotypy and also showed higher levels of aggression.If I adhered to autism advocacy standards, I could simply repeat some or all of this excerpt endlessly as some kind of "proof" of what intensive ABA programs accomplish.
But the first sentence in the quote above is not supported by the authors' analyses. The other sentence is accurate in the sense that what is claimed can be traced to statistical comparisons that were actually carried out.
However, these statistical comparisons are within the context of a very poor quality study where the findings, even within the given highly questionable parameters (e.g., definitions of challenging behaviours that, among many other limitations, encompass presumed mental states) are virtually impossible to interpret.
Available data that might assist in interpreting the findings are simply not reported (e.g., demographics of the two different intervention groups). But even were the data better reported, this study is based on an very poorly characterized sample (poor characterization being the standard in the vast ABA literature), and one that according to what scant information is provided, is not representative of autistic children in general.
Indeed, an inventory of the errors (some of them instantly obvious) and major weaknesses in this paper would be lengthy and would include, somewhere down the list, the authors' contention that ABA must have been effective in their study in reducing challenging behaviours in autism because, well, ABA is effective.
This paper was accepted (within two days of being submitted) by an autism journal edited by a famous behaviour analyst. In my view, in its epublished form, this paper contributes nothing reliable or worthwhile to autism research.
What is worthwhile, though, is the reminder that quality and standards matter. The selective reading and promotion of research of overwhelmingly poor quality is central to autism advocacy--is one of its signatures. Autism advocates have successfully insisted that poor quality research is what autistics deserve.
But poor quality research can be selectively invoked to support any position, as I've pointed out here. For example, the low/no standards of science and ethics that autism advocates have imposed on autistics have led to public funding for RDI. But pushing poor standards and poor quality research is not going to lead to better outcomes for autistics. Claiming otherwise, as autism advocates loudly do, harms autistics.
References:
Murphy, O., Healy, O, & Leader, G. (in press). Risk factors for challenging behaviors among 157 children with autism spectrum disorder in Ireland. Research in Autism Spectrum Disorders.
Rojahn, J., Matson, J.L., Lott, D., Esbensen, A.J., & Smalls, Y. (2001). The Behavior Problems Inventory: An instrument for the assessment of self-injury, stereotyped behavior, and aggression/destruction in individuals with developmental disabilities. Journal of Autism and Developmental Disorders, 31, 577-588.