Evaluating the Content and Quality of Next Generation High School Assessments
The purpose of this study was to evaluate four high school-level assessments – three new, multi-state assessments (ACT Aspire, PARCC, and Smarter Balanced) and a well-regarded existing state assessment (Massachusetts’ state exam [MCAS]) – based on new criteria developed by the Council of Chief State School Officers (CCSSO) for test quality. Three research questions answered in the study were:
Content: Do the assessments place strong emphasis on the most important content of college and career readiness as called for by the Common Core State Standards and other college- and career-readiness standards?
Depth: Do the assessments require all students to demonstrate the range of thinking skills, including higher-order skills, called for by those standards?
Accessibility: Are the assessments accessible to all students, including English learners (ELs) and students with disabilities (SWDs)?
To answer these questions, the study employed a new methodology developed by the National Center for the Improvement of Educational Assessment (NCIEA) based on the CCSSO’s 2014 Criteria for Procuring and Evaluating High-Quality Assessments (see Related Resources below). This methodology evaluates the degree to which test items and supporting program documentation measure content and depth. Additionally, this study evaluated whether the assessments showed evidence of being accessible to all students. The report found that the PARCC and Smarter Balanced tests emphasized the most important content and depth called for by college- and career-ready standards in ELA/Literacy and mathematics. For accessibility, while all of the programs offered a wide range of accessibility features and accommodations, this information was found in test development documents, and further research is needed once actual data is available.
This report adds to the growing evidence that next generation assessments are assessing student knowledge in deeper and more rigorous ways compared to the previous generation of assessments. This study paralleled another study conducted by the Thomas B. Fordham Institute (see Related Resources below), which implemented the same methodology, but for assessments in grades 5 and 8. Together, these reports shed much needed light on the quality – the content, depth, and accessibility – of the four assessments examined. This report is long (200+ pages), and it is somewhat technical in reading because it goes into great detail on the methodology used in the study. Furthermore, the recommendations section focuses on methodological issues. The results from this report can be used widely by stakeholders in schools, districts, and state agencies (see the summary evaluation here). However, the report is especially useful for states that are designing and developing assessments and need to rate the quality of their assessments for college and career readiness.