Center for Standards, Assessment, and Accountability

Spotlight

Assessment Design Toolkit

Overview

The Assessment Design Toolkit (Toolkit) includes videos and supplemental materials to help educators write and select well-designed assessments. Although the primary audience is teachers and principals, district and State leaders can use the Toolkit to design professional development opportunities. The Toolkit was developed to respond to State requests for help to improve assessment literacy among educators. Assessment literacy is important for all teachers, especially for those teachers of non-tested grades and subjects who do not have State assessments to help them measure student growth. The Assessment Design Toolkit includes 13 “modules” divided into four parts:

Orientation: How to Navigate the Modules

The modules address how to plan, write, and select well-designed assessments. Each module includes a summary, a video, and supplemental materials. You can view the videos alone, in teams, or with entire faculties. The run time of the videos is shorter than the time we suggest you take to learn the videos’ content. You can pause the videos whenever you need an extra moment to think about what you have learned, and you can rewind them to revisit key concepts. This module will orient you to how the modules work.

Part I: Key Concepts

Measuring what students know and can do is an essential part of teaching, and, as much of teaching, designing assessments that measure what we want them to measure is sophisticated work. The first three modules provide an introduction to classroom assessment design, focusing on key concepts such as the five elements of assessment design, validity, reliability, purposes of assessment, and assessment items. By completing all of the modules, you will be able to plan, write, and select assessments in which you are confident, and that gives you a clear sense of what your students are learning.

1. Introduction to Assessment Design

By the end of this module, you should be able to define several key terms and concepts that are foundational to this series of modules—including what the modules identify as five elements of assessment design and validity and reliability—and explain why teachers should focus on these five elements of assessment design and not the statistical concepts associated with validity and reliability as researchers and test makers apply them. You will also be able to explain the purpose of two tools that the Toolkit introduces: an assessment blueprint template and an assessment blueprint filled out with an example.

Supplemental Materials:

2. Purposes of Assessment

Now that you’ve had a chance to view the introductory module and learn or get a refresher on some important terms, let’s discuss the multiple purposes of assessment. By the end of this module, you should be able to identify the different purposes of assessment and understand how to use the assessment blueprint to document the primary purpose of an assessment.

Supplemental Materials:

3. Types of Assessment Items

The next module focuses on the types of items you can use when you design assessments, such as selected response, constructed response, and performance tasks. By the end of this module, you should be able to identify, describe, and tell the difference between three different types of assessment items.

Supplemental Materials:

Part II: Five Elements of Assessment Design

In the introductory module, we introduced the concept of five elements of assessment design—alignment, rigor, precision, bias, and scoring—and suggested that if the assessments you write or find elsewhere address these five elements effectively, those assessments stand a great chance of having an appropriate level of validity and reliability for use in your classroom. In this and the next four modules, you will learn how to recognize the effective use of these five elements and be well on your way to writing or selecting well-designed assessments for your students.

4. Alignment

The focus of this module is the first element of assessment design—alignment. By the end of this module, you should be able to define alignment for the purpose of the modules and explain why it is important. You should also be able to explain how to “unpack” a standard to understand its content and use the assessment blueprint to document the skills embedded within it.

Supplemental Materials:

5. Rigor

This module has several goals. By the end of this module, you should be able to (1) define what rigor means for the purpose of these modules, (2) use the verbs in standards and other tools that you have available to you to design assessment items that match the cognitive complexity of the relevant standards, (3) explain why assessments with an appropriate level of rigor also measure for a range of student thinking and understanding, and (4) use the assessment blueprint to document the level of rigor of each skill you are measuring in the assessments you write or select.

Supplemental Materials:

The Rigor video and supplemental materials use Bloom’s Taxonomy as the primary tool to determine rigor level. For those who prefer to use Webb’s Depth of Knowledge (DOK) as a measure of rigor, the video and accompanying supplemental materials linked under the “Alternative Rigor module” header below provide an alternative version of the Rigor module that uses Webb’s DOK levels.

Alternative Rigor module: Webb’s DOK Levels

6. Precision

You’ve now learned about alignment and rigor, the first two of five elements of assessment design, and you are well on your way toward writing or selecting well-designed assessments for your students. The focus of this module is the third element of assessment design—precision. By the end of this module, you should be able to describe what precision means for the purpose of these modules and make an imprecise item more precise. The video in this module is shorter than many other modules. We do, however, provide additional examples of how to design precise selected- and constructed-response items in Part III of this Toolkit.

Supplemental Materials:

7. Bias

You have been working your way through a series of modules to develop the knowledge you need to design great assessments. You’ve already learned about three of the elements—alignment, rigor, and precision. The focus of this module is the fourth element of assessment design—bias. By the end of this module, you should be able to describe what bias means for the purpose of these modules and detect potential bias in assessment items.

Supplemental Materials:

8. Scoring

The fifth component of assessment design—scoring—will likely be the most familiar to you. This module will help make you an even more effective scorer of your students’ work and demonstrate for you how the more effective your scoring practices are, the more likely your assessment results will be reliable. By the end of this module, you should be able to define what scoring means for the purpose of these modules and explain how and why you should use well-designed tools, such as answer keys, scoring guides and rubrics, to score many assessments. You should also be able to explain what distinguishes one tool from another.

Supplemental Materials:

Resources and Examples of Rubrics:

Mastering how to write a well-designed rubric is beyond the scope of this module. We recommend the following resources and examples of well-designed rubrics to help you continue your learning about rubrics.

Part III: Writing & Selecting Assessments

The next series of modules focus on the types of items and assessments you can write or select: selected-response items, constructed-response items, performance tasks, and portfolio assessments. For some of you, this will be a review. Whether you are a novice or experienced assessment developer, we believe there’s enough helpful information here to help you make even more informed decisions about when to use specific types of items and assessments and how to improve the assessments you’ve already been designing so that the results of your assessments truly reflect all that your students are learning.

9. Selected-Response Items

We’re going to start with selected-response items, which, if done well, are a lot more difficult to write than your students probably realize. By the end of this module, you should be able to define what a selected-response item is, identify the benefits and challenges selected-response items present, know the four parts of a well-designed multiple-choice item, and use the assessment blueprint to organize items in an assessment.

Supplemental Materials:

10. Constructed-Response Items

The focus of this module is a different type of item—the constructed-response item. Again, you have probably written many constructed-response items. Our hope is that this module will help you take what you know and can do to the next level and that the completion of this module will be one more step you take on your journey to becoming a master assessment designer for your classroom.

Supplemental Materials:

11. Performance Tasks

You’ve looked at two types of assessment items—selected response and constructed response. Now let’s move on to performance tasks. You likely design simple and complex performance tasks with regularity. They provide you with tremendous flexibility and they allow you to check your students’ higher-level thinking. By the end of this module, you should be able to define what a performance task is, list a variety of performance tasks, identify the benefits and challenges of performance tasks, know that there is a “what-who-how” framework that you can use to design performance tasks, and use the assessment blueprint to design assessment items.

Supplemental Materials:

12. Portfolio Assessments

You have finished watching videos about three different types of assessment items. Next, we will discuss portfolio assessments. Portfolios are not a particular type of assessment item; they are full-bodied assessments, capable of housing a collection of other assessments that come together as a powerful display of student learning. Portfolios should be part of every masterful classroom assessment designer’s toolkit. By the end of this module, you should be able to define a portfolio assessment, distinguish between its two types, identify the benefits and challenges of using portfolio assessments, and know that there is a “what-who-how” framework that you can use to design them.

Supplemental Materials:

Part IV: Reflecting on Assessment Design

The final module focuses on the act of reflecting on assessment design. It provides a checklist that you can use to determine if an assessment appropriately addresses the five elements of assessment design.

13. Reflecting on Assessment Design

You have invested a lot of time in mastering the skills of assessment design. You have learned about the purposes of assessment, five elements of assessment design, the three different types of assessment items, and portfolio assessments. You’ve learned how to use an assessment blueprint to help you design assessments. In this final module, we focus on a tool that you can use to evaluate an assessment you have written or selected. We simply call it the “Assessment Checklist.” It focuses on the five elements of assessment design, which help you design classroom assessments that have an appropriate level of validity and reliability. By the end of this module, you should be able to use the Assessment Checklist to help you determine whether an assessment you design appropriately addresses the five elements of assessment design featured in this series of modules.

Supplemental Materials:

Conclusion

Congratulations are in order! You have officially completed all 13 modules in the Assessment Design Toolkit and mastered most, if not all, of its content. Your hard work has paid off, and you should feel even better prepared than ever to write and select assessments that measure the knowledge and skills you want them to measure. Your students will benefit greatly from your hard work and assessment expertise. Good luck as you continue your career as an educator. Among the many complex skills you will demonstrate in every unit of instruction, count among them the capacity to write and select well-designed assessments.

Repurpose the Toolkit

How Can Educators Use the Toolkit?

The Toolkit is designed so that educators can repurpose its module(s). For example, the modules can be part of a robust professional development plan. State and district leaders can post the modules to online learning platforms for teachers to access independently to fit their schedules. Teams of teachers, Professional Learning Communities, and school departments can use the modules to improve assessment literacy together in groups.

Read Vignettes of How to Use the Toolkit

State, district, and school-level educators will use the modules in different ways. To learn about how the Toolkit might bolster professional development at each of these levels, you can read three vignettes that describe how a State, district, and school leader might use the modules. You can also read a list of ideas for how to use the Toolkit developed by State, district, and school leaders who attended a convening about assessment design.

How Can Educators Repurpose the Toolkit?

We encourage educators to repost the link to the Toolkit and any links within the Toolkit to public forums, websites, or online communities. For example, a district leader leading an in-person professional development session on assessments might include a link to the Toolkit in his or her training resources so that educators can follow up with independent study.

Educators can also embed any one individual video or a subset of videos on a different Web page. To embed a video on a blog or webpage:

  • Hover over the video you wish to embed
  • Right click (for PCs) or CTRL + click (for Macs)
  • Select “Get embed code”
  • Copy the code
  • Paste the code into the new Web page

Download the Supplemental Materials

Download 508 Compliant PDF Supplemental Materials

*For supplemental Rigor materials that use Webb’s Depth of Knowledge levels as the primary tool to determine rigor, please see the Rigor module, and download the alternative materials under the header “Alternative Rigor Module: Webb’s DOK Levels.”