John Kleeman portraitPosted by John Kleeman
Here’s the one piece of advice I’d give above all others to anyone creating quizzes, tests or exams: Test above knowledge.
You may be familiar with Bloom’s taxonomy of learning objectives, which is shown in the diagram below. At the lowest level is Knowledge; questions that test Knowledge ask for simple remembering of facts. At the highest level is Evaluation; questions that test Evaluation require participants to use judgement.

Bloom pyramid

It’s great if you can write questions that assess at the highest levels of Bloom’s Taxonomy, but many organizations have a lot to gain by asking questions at any of the levels  above Knowledge in the pyramid. Although there are times when testing facts can be useful, it’s usually better to test the application or comprehension of those facts.
In today’s world, where facts are easily googleable, it is the use and understanding of those facts in the real world that is genuinely useful. By testing above knowledge, you are checking not just that the participant knows something but that they can apply it in some scenario. This is more valid and more realistic — and for most applications it is also more useful.
Here is a simple example to illustrate the point:

Traffic lightsWhat does a yellow traffic light mean?

  • Stop
  • Go
  • Caution

This is purely a factual, knowledge question.
But here, the question requires that the respondent to apply to meaning of a yellow traffic light to an actual situation:
If you are driving toward an intersection and the light turns from yellow to red, what should you do?

  • Speed up and cross the intersection
  • Stop suddenly
  • Stop gradually

This is a very simple example, but I hope it makes you realize that converting factual questions to scenarios is not very hard.
I’d encourage you to consider using scenarios in your questions: Ask people to apply their knowledge, not just prove that they know some facts. Have your test-takers apply what they know to actual situations.