This is another re-post of a blog post at the i>clicker blog.
Sometimes we can be lucky enough to have access to a great set of clicker questions (see, for example, the list at STEMclickers.colorado.edu). But often a good set of questions for our course doesn’t exist, or another instructor’s questions don’t quite fit. Or, often, we’ll only have what a textbook suggests for multiple choice exam questions – and the problem with such questions is that they are often simple factual recall questions.
Factual recall questions are the most common type of question that you might find yourself stuck with. They are easy to write. But they don’t make a good Peer Instruction question because there is nothing to talk about. You know it or you don’t. Once students determine what they answered, there is no real reason to keep talking. Try it – you’ll hear a little murmur in the room that quickly dies away.
So, how do we take a question like that and turn it into a question that sparks discussion, reasoning, and debate? I use a technique that I call “Bloomifying up” the question.
Bloomifying Your Questions
“Bloomifying” basically takes a lower-level question and uses some handy verbs to get ideas on how to make it a higher-level question that requires more thought. Bloom’s Taxonomy (see picture) is just a classification scheme to talk about different levels of thinking, from simple remembering and factual recall, up to applying information to a new situation, breaking down and analyzing a concept or problem, or even synthesizing something new from what they have learned.
The nice thing is that many people have developed lists of verbs that go along with each level of Bloom’s Taxonomy – and this is a very helpful thing. Here is my favorite list of Bloom’s Verbs. Blooms Verbs (download it now!).
Here is a short list of examples:
Remembering: Choose, describe, identify, list, locate
Understanding: Distinguish, locate, show, summarize
Applying: Calculate, classify, organize, prepare, select, sketch, solve
Analyzing: Distinguish, identify, select, test, categorize
Evaluating: Argue, assess, choose, compare, determine, predict
Creating: Arrange, assemble, construct, develop, produce
You may have noticed that some verbs appear in more than one place: The purpose of a question really depends on its’ context! So here’s how I use these verbs lists – either to Bloomify up a clicker question or a learning objective. I take my question that’s too low level, and “go shopping” for verbs to help me rethink the question. This takes some practice, but is a very good skill to develop!
Here’s an example. The Keeling Curve is that graph of the rise of atmospheric carbon dioxide in the atmosphere. Below is a question that would just test whether students had done the reading to know what it is.
A “Keeling Curve” indicates:
A. how earth’s tilt affects interception of solar output
B. the path of barometric pressure declines across the equator
C. additions of sequested carbon to the atmosphere
D. temperature increases observed from buoys at sea
Credit: McGraw-Hill, http://highered.mcgraw-hill.com/sites/007029416x/student_view0/chapter23/multiple_choice_quiz.html
That wouldn’t really generate much discussion. So, I look on my list of verbs and find the word “predict”. How can I change this to a prediction question? Here’s one possibility.
Predict how might the Keeling Curve change as more deforestation takes place.
A. The seasonal cycle becomes larger
B. The seasonal cycle becomes smaller
C. The increasing trend will become steeper
D. A and C
E. B and C
Credit: An anonymous participant in one of my workshops
This version of the question requires students to really think about what the Keeling Curve means, and how atmospheric carbon dioxide relates to plants and photosynthesis. It’s a much higher-level question, and would generate much more discussion. Note that “prediction” is a real go-to technique of mine when improving clicker questions.
Here’s another example.
What is the definition of velocity?
A. time over distance
B. distance over time
C. distance over time, but with a direction implied
D. acceleration over time
E. more than one
Again, this isn’t a great peer discussion question. So I look on the list of verbs, and see “select” under “analyze” and I think, I could have them select something, analyze something… how about a graph of velocity? To identify where the velocity is greatest? And, after many iterations, I might come up with something like the following:
And one last example, which I am going to leave as an exercise for the reader.
Where does the U.S. rank in the world when it comes to the percentage of women in elected positions?
Peter Newbury writes on his blog about this question, which was penned in great optimism by an instructor there:
Here’s what was supposed to happen…
The students would vote. Then they’d “turn to your neighbors and convince them you’re right.” The students would help each other remember the correct answer, 78th, and then spontaneously launch into a discussion about how that’s surprising because of ABC and interesting because it’s DEF but not GHI and so on, bringing in all the interesting, conceptually challenging ideas that political scientists explore and debate.
That’s not what happened, though. Sure, they voted. The even split across all four answers showed they didn’t know the answer and were guessing. And when she asked them to turn to their neighbors, the room didn’t crescendo with conversation. There was some brief murmuring
“What’d you pick?”?“78th.”?“Huh. I picked 34th.”
Yeah, not quite what was intended. So let’s think about what the main point here was. The instructor wanted students to use political science reasoning to interpret the low number of women officials. Now, I’m not a political scientist so I can’t easily write the question, but perhaps asking students to predict based on political scientist factors? For example.
Between the U.S. and Russia, which is likely to have the highest percentage of women in elected positions?
A. Russia, because it has a larger land mass
B. The U.S., because it is a democracy
C. The U.S., because of affirmative action
D. Russia, because it is a matriarchy
Any other ideas on that one?
– See more at: https://www1.iclicker.com/blogs/how-can-you-make-a-good-clicker-question-great/#sthash.EPLwAaGs.dpuf