Improving the bottom quartile with a metacognitive exercise (#AAPTSM17)

by Stephanie Chasteen on July 25, 2017

I’m in an inspiring session by Charles Atwood (University of Utah) about how they improved the performance of at-risk students in introductory chemistry at the University of Utah.

Abstract:

To improve success rates in large general chemistry sections at the University of Utah, we realized we must improve the bottom two student quartiles performance. We implemented educational research in metacognition as well as the Dunning-Kruger effect into our homework system. In fall semester 2016 we required students in one general chemistry section to predict their scores prior to taking practice tests for each midterm exam and the national exam. Students were given feedback on topics that they did well on as well as topics where they performed poorly. They were required to make a study plan. Comparison between our treatment and control sections shows that all student quartiles improved but for the bottom quartile there was a 15-22% improvement. Using the American Chemical Society nationally normed national exam our treatment section students scored on average at the 82nd percentile (median 89th percentile) while the lowest quartile scored 53rd percentile.

Their data showed that students who received a C in prior courses were at significant risk for failing general chemistry.  Upon finding out about the Dunning-Kruger effect (nicely defined on Wikipedia as where “persons of low ability suffer from illusory superiority”), they realized that this effect could help understand the low performance of these students.  In education, Dunning-Kruger has been shown to result in poor students consistently overestimating their likely performance on exams, thus not studying as much as they should.

They instituted a metacognitive element in their course, where students were asked to reflect on their performance on tests before, and after the test.  After a few iterations, their final structure consisted of:

  1. 30-minute computer graded quiz on previous week’s material (I believe he said he used Madra Learning).
  2. Students predict their score on that quiz (after taking it but before getting feedback)
  3. Students get feedback on their quiz, with suggestions of which topics to work on.
  4. Students make a study plan.

As described in the abstract, this structure resulted in significantly improved performance, especially for at-risk students.

  • Students’ predictions on their quiz results got more accurate over time.  By the end of the course, students underpredict their performance on the final (which is not what you usually see among students).
  • The bottom quartile increased course performance by 17%
  • The bottom quartile increased performance on the national exam by 33%.

These are incredibly strong results, across many measures that were shown.  I want to reiterate that they didn’t get this kind of success overnight; they iterated their approach until it worked well.

You can find more metacognitive strategies on my article on PhysPort about student engagement.

Leave a Comment

Previous post:

Next post: