Understanding Screening: Sample Representativeness

Imagine you are a baker that has made a jar of 100 cookies, where 60% of the cookies are chocolate chip and 40% are sugar cookies. Let’s say you’d like to sell batches of 10 cookies that are representative of the larger jar of cookies. So, ideally you would like your sample of cookies to include 6 chocolate chip cookies and 4 sugar cookies. With this proportion of cookies in your sample you know that you are accurately representing the larger jar of cookies.

The same principles apply when you are evaluating the quality of a screening assessment. If you are trying to determine whether or not the screening tool accurately measures children’s skills, you want to ensure that the sample that is used to validate the tool is representative of your population of interest.

The first goal of evaluating sample representativeness is through a clear understanding of the population that is defined by the screening assessment. A population could be as large as a country, like the United States, or it could be as small as a district or school. The answer to the question of, “Is the sample representative?” lies in a complementary question of, “To whom should children’s scores be compared?”. If a sample’s scores should be compared to a national benchmark, then the demographics of the sample should look like the demographics of the nation. If, instead, a sample’s scores are to be compared to the population of a state, then the samples’ demographics should look like the state and not the nation, a region or district.

Achieving sample representativeness can occur in a number of ways. In this brief, we quickly highlight two mechanisms that the National Center on Intensive Intervention (NCII) uses to assess sample representativeness. First, a sample can be representative by data collection occurring across multiple sites or be one nationally representative sample. Second, a group of samples can be considered representative if the statistical analyses between the samples result in approximately the same results.

Click to Enlarge

Suggested Citation

Pentimonti, J., Petscher, Y., & Stanley, C. (2019). Sample representativeness. Washington, DC: U.S. Department of Education, Office of Elementary and Secondary Education, Office of Special Education Programs, National Center on Improving Literacy. Retrieved fromimprovingliteracy.org.