Evaluating the Content and Quality of Next Generation Assessments
This study’s purpose was to evaluate the quality of four standardized assessments – three new, multi-state assessments (ACT Aspire, PARCC, and Smarter Balanced) and a well-regarded existing state assessment (Massachusetts’ state exam [MCAS]) – to determine whether they met new criteria developed by the Council of Chief State School Officers (CCSSO) for test quality. Three research questions answered in the study were:
Do the assessments reflect strong content?
Are they rigorous?
What are their strengths and areas for improvement?
To answer these questions, the study employed a new methodology developed by the National Center for the Improvement of Educational Assessment (NCIEA) based on the CCSSO’s 2014 Criteria for Procuring and Evaluating High-Quality Assessments (see Related Resources below). This methodology evaluated the degree to which test items and supporting program documentation (e.g., test blueprints and documents describing the item creation process) measured the critical competencies reflected in college and career ready standards, namely the Common Core. Review panels comprised of vetted practitioners, content experts, and assessment specialists who evaluated grades 5 and 8 ELA/Literacy and math assessments based criteria corresponding to the research questions above. Results from this two-year study show that, overall, PARCC and Smarter Balanced assessments had the strongest matches to the CCSSO criteria, and ACT Aspire and MCAS matched well with regard to quality of their items and the depth of knowledge they assessed. The review panel also provided descriptions of strengths and recommendations for improvement for all four assessments.