Posted By Doug Peterson
You’ve written good items and had them approved. You’ve assembled them into an assessment. Ready to go? Well, you need to determine what constitutes passing and failing — or maybe basic, proficient and advanced skill levels. You can pick a number out of the air, but that’s not necessarily the best or most fair way to do it, especially for high-stakes exams.
Cutscores: A Manual for Setting Standards of Performance on Educational and Occupational Tests, by Michael J. Zieky, Marianne Perie and Samuel A. Livingston, reviews 18 different methods for determining a cutscore, including three different versions of the popular Angoff method. It provides all the information you need to decide which method is best for you and your situation, and then describes how to carry out a cutscore study and document the results.
I thought this book was going to be a tough technical read, but I felt it offered information with which I needed to familiarize myself, so I prepared to slug my way through it. To my surprise, the book is actually very readable. Don’t get me wrong: It’s no exciting adventure novel, but you don’t have to have an advanced degree in education and statistics to understand it.
I also like the way it’s organized. It starts with “What You Need To Know About Cutscores” and moves on to “What You Have To Do Before You Set Cutscores.” Then the authors do something brilliant: rather than go through each method in detail, they take a chapter to describe each method at a high level and explain when you would use it. Then, in the following chapter, they provide detailed information on carrying out each type of cutscore study. This approach makes it easy to determine which method best meets your needs in one chapter, and then focus only on that method in the next chapter.
The book wraps up by explaining what needs to be done after the cutscore study is complete, and includes examples of standard forms such as Nondisclosure Agreements and various evaluation forms in an appendix.
I recommend this book to anyone involved in the development of high-stakes assessments. This book provides the information you need to set up a fair and legally defensible method for determining the appropriate cutscore for your assessment.