You are here:
-A A +A

Small Number of Responses in IAN StateStats

Cheryl Cohen
Online Community Director
Interactive Autism Network at Kennedy Krieger Institute
Date Last Revised: 
June 6, 2014

Although many families have contributed data to IAN Research, some states have fewer responses than others, especially when the data are broken down into different categories, such as by diagnosis. This problem of low responses is important because low numbers mean that analysis of the data is much less accurate and exact.

Here is an example from Advanced StateStats:

You would assume from this graph that the median age of diagnosis for children with Asperger's in the District of Columbia is 4.9 years. But if you look closely, you see that the "total responses" for DC is only 7! In fact, if you hover your mouse over the Asperger bar, you can see that the total response is only one person. We cannot assume that these six children with ASD, much less the one respondent with Asperger's, reflect the age of diagnosis of children with Asperger's in the District of Columbia.

Most statistical analyses require total responses of at least 30. We are presenting lower total responses in StateStats because this is the information we have, but please interpret responsibly.

Another example:

A student's grade in Algebra was based on three tests. On one day the student gets a score of 100. On another day, the student, who is coming down with the flu, gets a score of 0. On another day, the student gets a 95. Her final score is an average of the three scores. The unfortunate girl received a 65 and failed Algebra that semester. Her parents complained that the grade was unfair and that the teacher should give more tests so that the true ability of the student is measured. One bad score out of three pulled down her grade significantly and her final grade did not represent her true ability. The next semester, the teacher based grades on 30 different measures including tests, quizes, and homework. Even though the girl forgot her homework once and received a 0, she still received an A for the semester -- a grade that better reflected her abilities. The 0 (sometimes called an outlier) changed her grade less during the second semester than the first because the total number of factors that went into the grade was greater, and one bad grade did not pull down (skew) the semester's grade significantly. So, you can see how a calculation based on a small number of responses can be misleading and not represent the true picture.

Please rate the helpfulness of this article: 
Average: 2 (1 vote)