Wednesday, July 18, 2007

About that surge in autism

So are autistics really going to take over the known world? We know there's been a staggering surge in irresponsible autism-related journalism. It's even hit the BMJ.

There's overwhelming evidence that 1 in 58 is not a genuine autism prevalence figure, but the product of shoddy and dishonest reporting (see Ben Goldacre here, looks like he's going to be in the BMJ on Friday, and the Times here). Nothing like irresponsible reporting to waste heaps of time and effort that could otherwise be spent in applying accurate information to help autistic people.

But what about that surge in autism? Is there one? This is not a popular position in our era of autism advocacy--but when in doubt, consult with the peer-reviewed data.

Here are 7 recently reported autism prevalence figures for children of various ages in the US and UK. All figures are for all autistic spectrum diagnoses combined. I've rounded them off to the nearest 5--autism prevalence figures don't come with pin-point precision. Here goes:

1 in 175
1 in 170
1 in 160
1 in 150
[1 in 150]
[1 in 150]
1 in 85
1 in 58

Wow, that looks like a big autism surge, even without that last figure. And those low/no standards for autistics so successfully pushed by autism advocates would demand that the 1 in 58 be tacked on at the end (there it is, in red, from shame). Now it looks like autism is surging even more.

I've also put two of the figures in square parentheses. That's because they're not quite like the other non-red figures. I'll get back to this.

Maybe we should look at when these figures were published in peer-reviewed journals, just in case it's informative.

1 in 175 (2000)*
1 in 160 (2001)**
1 in 150 (2001)
1 in 170 (2005)**
1 in 85 (2006)*
[1 in 150 (2007)]***
[1 in 150 (2007)]***
1 in 58 (not published)

That looks slightly less persuasive, but we could probably still argue that autism is surging, all the more so if we adopt autism advocacy standards for autistics and include the red-faced 1 in 58.

Or we could subject these studies to a bit of scrutiny. With two (non-red) exceptions, these studies meet two criteria: (1) they used DSM-IV or ICD-10 criteria for autism and the other autistic spectrum diagnoses; and (2) at least some of the counted children were directly assessed by the researchers using one or the other or both of the current standardized, quantified gold-standard autism diagnostic instruments.

The two exceptions are in square parentheses. They are US prevalence studies that don't meet my second criterion. They don't involve direct assessment of autistic children, instead relying on less reliable information from educational and/or medical records. But these two are very popular studies. I've included them (in parentheses) because their 1 in 150 has, since these studies were widely publicized early this year, often been reported as the prevalence of autism.

I've also paired up most of the studies. The ones with one asterisk (*) belong with each other. This pair of studies was done in the exact same geographic area. The ones with two asterisks (**) also belong with each other, and also were done in the same geographic area. And the ones with three asterisks (***) belong with each other too, and have some overlap in geographic area.

Regardless of geographic area, and keeping in mind the two criteria for studies I provided above, the pairs are paired in two different ways: studies using the same methodology with different birth year cohorts; and studies using different methodology with the same birth year cohort.

If autism is indeed surging, studies using the same methodology with different birth year cohorts should show autism prevalence increasing over time. And studies of the same birth year cohort using the same diagnostic criteria (DSM-IV or ICD-10) should show the same autism prevalence. If not, then the extent to which differences in methodology contribute to reported differences in prevalence would have to be contemplated--and this might inconvenience the "surging autism" contingent.

So if there really is an autism surge, we should find that the (*) pair represents two studies with the same methodology but different birth year cohorts, where the higher figure (1 in 85) is found in a later birth year cohort than the much lower figure (1 in 175). And we should find that the other two pairings, where the figures are the same (the *** pair, with 1 in 150) or nearly the same (the ** pair, with 1 in 160 and 1 in 170; this is not a significant difference), are studies of the same birth year cohorts done with whatever methodology, provided the same diagnostic criteria are used.

Well, it doesn't quite work out that way. In fact, it's the opposite. That (*) pair represents the same birth year cohort, different methodology. And those other pairs (*** and **) represent different birth year cohorts, same methodology.

That's keeping in mind that apart from the square parentheses (***) pair, and apart from the beet red 1 in 58, all the other studies meet both my own criteria--they use the same current diagnostic criteria, and they involve at least some direct assessment of children with one or the other or both of the current gold-standard diagnostic instruments.

How about ordering all the studies according to the years in which the children being studied were actually born. If that autism surge is autism reality, then we should see that surge, uh, surging right along as birth year cohorts become more recent. It's about time I named the studies, and I've kept the asterisks, just in case anyone's keeping track. Here goes again:

1 in 150 (1988-1995; Bertrand et al., 2001)
1 in 175 (1990-1991; Baird et al., 2000)*
1 in 85 (1990-1991; Baird et al., 2006)*
[1 in 150 (1992; ADDMN, 2007)]***
1 in 160 (1992-1995; Chakrabarti & Fombonne, 2001)**
[1 in 150 (1994; ADDMN, 2007]***
1 in 58 (1993-1997; not published)
1 in 170 (1996-1998; Chakrabarti & Fombonne, 2005)**

That didn't work out too well either. Now we don't have a surge at all, just a bunch of findings--spanning a decade of birth years--that are very close to each other, and (leaving out the crimson 1 in 58) one figure that looks like an outlier. But we can't attribute the 1 in 85 (it is actually 116.1/10,000) in Baird et al. (2006), a figure often rounded off to 1 in 100, to a surge in autism, because much lower prevalence figures have been found in several later birth year cohorts. Never mind that a much lower prevalence figure was found in the same cohort in Baird et al. (2000).

Indeed, what the two Baird et al. studies demonstrate is the dramatic effect methodology can have on reported autism prevalence within the same cohort--even when the same diagnostic criteria are used, even when there is an overlap in the standardized diagnostic instruments used (both Baird studies used the ADI-R), and even when the studies are conducted by an overlapping group of researchers. The two studies differ primarily in that the later study also used the ADOS, two of the diagnosing clinicians changed, and the method of case finding was altered. That was enough to double the reported prevalence within the same cohort.

In contrast, the two Chakrabarti and Fombonne studies show that applying the same methodology to different birth year cohorts results in the same autism prevalence. No surge in sight. The two in-parentheses studies, which are the now-famous CDC prevalence studies, show the same thing but with weaker methodology over a shorter timespan.

Autism advocates are free to seek that recent surge in autism--that catastrophic epidemic--in anecdotes, in education numbers or the CDDS, in sensationalist headlines and so on. This is all in keeping with the rotten standards of science and ethics they've imposed on autistics, and with their own steadfast resistance against verifiable information. But on the off-chance anyone's interested in the published, peer-reviewed data, I thought I'd go fetch some. If anyone finds any factual errors in the information I've presented, I'd greatly appreciate knowing. Accurate information is always good for autistics.

(Edit: Ben Goldacre's now had his say in the BMJ. You can find his column here.)


Autism and Developmental Disabilities Monitoring Network Surveillance Year 2000 Principal Investigators; Centers for Disease Control and Prevention. (2007). Prevalence of autism spectrum disorders--autism and developmental disabilities monitoring network, six sites, United States, 2000. MMWR Surveillance Summaries, 56, 1-11.

Autism and Developmental Disabilities Monitoring Network Surveillance Year 2002 Principal Investigators; Centers for Disease Control and Prevention. (2007). Prevalence of autism spectrum disorders--autism and developmental disabilities monitoring network, 14 sites, United States, 2002. MMWR Surveillance summaries, 56, 12-28.

Baird, G., Charman, T., Baron-Cohen, S., Cox, A., Swettenham, J., Wheelwright, S., & Drew, A. (2000). A screening instrument for autism at 18 months of age: a 6-year follow-up study. Journal of the American Academy of Child and Adolescent Psychiatry, 39, 694-702.

Baird, G., Simonoff, E., Pickles, A., Chandler, S., Loucas, T., Meldrum, D., & Charman, T. (2006). Prevalence of disorders of the autism spectrum in a population cohort of children in South Thames: the Special Needs and Autism Project (SNAP). Lancet, 368, 210-215.

Bertrand, J., Mars, A., Boyle, C., Bove, F., Yeargin-Allsopp, M., & Decoufle P. (2001). Prevalence of autism in a United States population: the Brick Township, New Jersey, investigation. Pediatrics, 108, 1155-61.

Chakrabarti, S., & Fombonne, E. (2001). Pervasive developmental disorders in preschool children. JAMA, 285, 3093-9.

Chakrabarti, S., & Fombonne, E. (2005). Pervasive developmental disorders in preschool children: Confirmation of high prevalence. American Journal of Psychiatry, 162, 1133-41.


srngln said...

Thank you for putting the things in order.

Sharon G.

Bev said...

Thanks very much. This is a most useful way of organizing the information. And so many words for "red", too!

Michelle Dawson said...

You're welcome, Sharon G and Bev.

Bev wrote, 'And so many words for "red", too!'

Now you're making me blush.

Larry Arnold PhD FRSA said...

It all goes to show that there is no such thing as a precise measurement of the phenomenon that is categorised as autism

And before you think I am off on one again, this would be the case for most conditions even those that have clear biological markers.

Can one number how many people are arthritic for instance, that can encompass anything from a problem with one joint to a problem with many joints to a systemic progressive disease encompassing more than joints alone.

But really no-one can tell accurately to the degree of certainty that one can adduce for whether a car is a Ford or a Mazda.

Wait a minute that is not clear either :)

Anonymous said...

I usually agree with Larry about this sort of thing. So to nitpick something in your post Michelle:

"And studies of the same birth year cohort using the same diagnostic criteria (DSM-IV or ICD-10) should show the same autism prevalence."

That assumes that the authors of those studies interpret and apply the DSM-IV and ICD-10 criteria in precisely the same way.

Michelle Dawson said...

Joseph wrote, "That assumes that the authors of those studies interpret and apply the DSM-IV and ICD-10 criteria in precisely the same way."

Yes. My point there was that it's important to look at methodological issues beyond which set of criteria are used, and beyond which gold-standard diagnostic instruments are used (it might be interesting to look at the limits of those instruments too). You have to slog through the published papers to find out what exactly was done.

I made this point by using a pair of studies (the two Baird et al. papers) of the same cohort--where the authors and diagnostic instruments overlapped, where the same criteria were used, but where other aspects of methodology (aspects which don't tend to get a lot of sensational media coverage) differed.

I was addressing an autism advocacy assumption that because the DSM-IV (and ICD-10) criteria have been in place for so long, any currently reported (in the media, as per autism advocacy standards for autistics) increase in prevalence must be evidence of "surging autism". This ignores the effect of other aspects of methodology.

I also provided some evidence of what happens when the same methodology is used across different birth year cohorts. No, these results aren't perfect or even close--this is science after all, not the infallibility of autism advocacy. All studies have limitations. But it's possible to learn from these findings and limitations, and to make better decisions rather than worse decisions.

Anonymous said...

I think someone should try to replicate Lotter (1967). Wouldn't that put the "epidemic" argument to rest at least among non-conspiracy theorists?

I mean, not just replicate the criteria and methodology, but carefully study how Lotter interpreted it. Lorna Wing, for example, explained that Lotter was very strict in who he considered autistic or not.

Anonymous said...

The 1 in 58 figure is from a tool with a known 50% false positives rate but a good specificity (it is good at excluding false negatives). The researchers wrote a report that says: "The CAST is useful as a screening test for autism spectrum conditions in epidemiological research. There is not currently enough evidence to recommend the use of the CAST as a screening test within a public health screening programme in the general population".

There is also a little bit of mention in there about the confidence intervals and why they encompass all of the usual estimates but I'm not going to embarrass myself by even pretending to understand it.

Larry Arnold PhD FRSA said...

SBC at least so far as his diagnostic practice in CLASS goes, claims a faithfullness to DSMIV

however this study, and we won't know until it is published may be experimenting with a number of instruments and criteria, and until we see it all we are not going to know the context.

I don't think SBC believes for one moment that 1 in 58 is an accurate number, the whole point of this study it seems is to look at the variance in order to see whether some coherency can be had from it all.

But ultimatly it goes to show that what you get is a result of what you set as your criteria to begin with and the fact that criteria cannot be set in stone, is ultimately going to bedevil the genetic route aswell because genes in themselves determine nothing until the categories have been decided already, it is putting the chickens before the eggs.

Michelle Dawson said...

A good question is the general one of how useful screening instruments--which have to be short and simple--can be in autism.

The current gold standard diagnostic instruments require professionals who have training in each instrument. They take a long time (hours) to administer. But they are only considered (by existing standards) to be good at identifying the specific diagnosis of autism. They are not good at, e.g., sorting out PDD-NOS or atypical autism from non-autistic-spectrum neurodevelopmental disabilities.

The 116.1/10,000 figure in Baird et al. (2006) includes a majority (~91/10,000) of children who do not meet ADOS and/or ADI-R cut-offs for autism, and only a minority (~25/10,000) who meet both cut-offs.

"Genes don't care about the DSM-IV." That's a recent quote from Thomas Bourgeron (presentation at Ste Justine's in Montreal). It's entirely possible that this view is held by many geneticists, who have decided to pursue autism endophenotypes.

HP said...

I wonder if anyone has even begun to take into account variations in prevalence between different populations, or the effect of things like positive assortative mating. Prevalence alone wouldn't tell you much even if it was accurate.

Still, some autistic people in Canada seem to wish to have genetic counselling:

Makes me wonder whether that's what they'll be given, and what they will gain from it :/

HP said...

oops the link was cut in half

Michelle Dawson said...

Hi Heta,

The limitations of prevalence studies (which can be strongly influenced by availability of services and many other factors) are usually recognized in the autism research community.

These limitations are rarely (if ever) reported in the media.

Unfortunately, autism advocates in Canada insist that autism prevalence and autism incidence are the same thing (see the writings of "autism reality" expert Harold Doherty, e.g.). This goes along with the lousy standards of science and ethics that autism advocates demand for autistics. I don't think autism advocates would ever accept such rotten standards for themselves.

ASC's powerless autistic advisors (they are hand-picked by ASC's members, are told by ASC what they are supposed to discuss, and are supervised) possibly aren't entirely representative of the priorities and values of many autistics in Canada.

Anonymous said...

Yes, it would seem that if autism became the way people were, they'd probably have "special" treatments for all those poor normal folks being left behind.

David N. Andrews M. Ed., C. P. S. E. said...

"Unfortunately, autism advocates in Canada insist that autism prevalence and autism incidence are the same thing (see the writings of "autism reality" expert Harold Doherty, e.g.)."

He's no bleeding expert.....

Andy said...
This comment has been removed by the author.
Andy said...

Thanks for this - interesting and useful. I'd seen the Observer article and thought something was a tad dodgy! The Guardian article you link to is lovely. BTW they seem to be getting interesting people to write for them. For instance check out this article on statistical prediction by Christine Evans-Pughe.

Hope all's good with you. Best wishes from Scotland.


Michelle Dawson said...

Hi Andy,

Good to hear from you. That's a very cool and very useful article from Christine Evans-Pughe, thanks for the heads up.

Meanwhile, the Observer's front page screaming autism scare story has pulled a disappearing act, and is no longer available online, possibly "due to concerns about its accuracy".

Tesseract said...

Being the parent of a set of autistic twin girls, it is nice to see someone putting this in perspective. As a parent it is often hard to see beyond your own children, but sometimes that must be done in order to assist them, again thank you for your site.

Anonymous said...

Michelle, please! Autistics should take over the world.

Look at the horrible mess made by all those short-sighted unthinking rich fools. These people are so dumb and destructive that even the NTs have noticed.

Of course, I don't want to rule the world because I would stuff it up - I would commit some embarrassing blunder or other and have to hide in a cave.

But I would suggest your name, Michelle, without any hesitation.
I almost have faith in you (be extremely flattered).

P Buddery