I’m in an inspiring session by Charles Atwood (University of Utah) about how they improved the performance of at-risk students in introductory chemistry at the University of Utah.

Abstract:

To improve success rates in large general chemistry sections at the University of Utah, we realized we must improve the bottom two student quartiles performance. We implemented educational research in metacognition as well as the Dunning-Kruger effect into our homework system. In fall semester 2016 we required students in one general chemistry section to predict their scores prior to taking practice tests for each midterm exam and the national exam. Students were given feedback on topics that they did well on as well as topics where they performed poorly. They were required to make a study plan. Comparison between our treatment and control sections shows that all student quartiles improved but for the bottom quartile there was a 15-22% improvement. Using the American Chemical Society nationally normed national exam our treatment section students scored on average at the 82nd percentile (median 89th percentile) while the lowest quartile scored 53rd percentile.

Their data showed that students who received a C in prior courses were at significant risk for failing general chemistry.  Upon finding out about the Dunning-Kruger effect (nicely defined on Wikipedia as where “persons of low ability suffer from illusory superiority”), they realized that this effect could help understand the low performance of these students.  In education, Dunning-Kruger has been shown to result in poor students consistently overestimating their likely performance on exams, thus not studying as much as they should.

They instituted a metacognitive element in their course, where students were asked to reflect on their performance on tests before, and after the test.  After a few iterations, their final structure consisted of:

  1. 30-minute computer graded quiz on previous week’s material (I believe he said he used Madra Learning).
  2. Students predict their score on that quiz (after taking it but before getting feedback)
  3. Students get feedback on their quiz, with suggestions of which topics to work on.
  4. Students make a study plan.

As described in the abstract, this structure resulted in significantly improved performance, especially for at-risk students.

  • Students’ predictions on their quiz results got more accurate over time.  By the end of the course, students underpredict their performance on the final (which is not what you usually see among students).
  • The bottom quartile increased course performance by 17%
  • The bottom quartile increased performance on the national exam by 33%.

These are incredibly strong results, across many measures that were shown.  I want to reiterate that they didn’t get this kind of success overnight; they iterated their approach until it worked well.

You can find more metacognitive strategies on my article on PhysPort about student engagement.

{ 0 comments }

Mutual mentoring (liveblogging from #AAPTsm17)

by Stephanie Chasteen on July 25, 2017

I’m now attending a session on Mutual Mentoring for physics faculty, presented by Anne Cox (Eckerd College).

Abstract:

We were part of an NSF ADVANCE grant mutual mentoring project for senior women faculty in chemistry and physics that began in 2007. We have continued our bi-monthly mentoring meetings for the past 10 years (well beyond the initial grant funding) because of the surprising value we found in having peer mentors. is talk will discuss what seems to have made our mentoring group so long-lasting as well as our initial work to spread this approach to others in the physics community through a new NSF-ADVANCE project: eAlliances- Uniting Isolated Physicists and Astronomers, http://ealliances.aapt.org.

While it’s getting better, women are often isolated in their departments: 30% of PhD granting-institutions had 0-1 female faculty members (AIP statistics)… and for those with 0 women, if they increase the number of women in their department, they will likely end up with a single woman in the department.

When looking for peer mentors, then, it can be difficult for women to find peers within their department, and may need to go to peers across institutions.  These horizontal alliances can be very valuable experiences.

The focus of the NSF ADVANCE program is to advance careers of women in STEM in higher education.  Ms. Cox created an ADVANCE program to create horizontal mentoring alliances across institutions.  A set of 5 senior women (chemists and physics) in their departments created an alliance.  As senior faculty (they were all full professors) they were used to mentoring other people, and wondered whether they really needed such mentoring… but found it was very valuable.

How did it work?  They had an initial in-person meeting at a national conference, had meals together, and met every other week to discuss a book (Every Other Thursday: Stories and Strategies from Successful Women Scientists), since a book discussion was safe and the topic was germane.  As they got to know each other, they worked on a project together.  Ten  years later, they still talk every other week, and have met in person 3 times.

They find it valuable to discuss college politics, family, and time management — often encouraging each other to be careful with adding new projects to their life.  They provided accountability to each other, asking each other how a particular commitment is going.

The important parts of the peer mentoring were:

  • In person meeting
  • Frequent conference calls
  • Periodic in person meetings
  • Credibility of the national organization supporting the group
  • Their own commitment to the alliance through the relevance of the work and honest discussions.

A new e-Alliances project is helping to provide such horizontal mentoring electronically — see http://ealliances.aapt.org to sign up as a mentor or mentee.

{ 0 comments }

Promoting innovation and entrepreneurship in physics (liveblogging from #AAPTSm17)

July 24, 2017

I’m at the American Association of Physics Teachers meeting this week, and will blog about a few sessions while I’m here. In a talk by Crystal Bailey (American Physical Society), she argued that we need to more explicitly teach Physics Innovation and Entrepreneurship (PIE) to our students.  I find this a really valuable message; having […]

Read the full article →

Educational change: How systemic thinking helps to push social progress

July 5, 2017

In today’s post I want to share some ponderous thoughts about how educational reforms happen, and how systemic thinking helps to support those reforms.  I am fortunate to be a working group leader in the Accelerating Systemic Change Network (ASCN; ascnhighered.org), and one of the working groups focuses on how theories and models of change can […]

Read the full article →

Changing how universities teach science: The SEI Model

June 21, 2017

We know a lot about how to improve STEM teaching and learning at the college level, and yet these improvements have yet to take hold in a widespread manner.  This is the perennial problem which many of us in STEM education are wrestling with.  The study of institutional change is expanding ever more, including lessons […]

Read the full article →

Data visualization tips

June 14, 2017

Are you trying to tell a story with your data?  This is a big part of my job (as an external evaluator), and I recently attended an excellent webinar on data visualization.  Now, I hate webinars that are trying to sell me a book, but this one was so packed full of great ideas that […]

Read the full article →

You can now embed PhET into Powerpoint!

June 2, 2017

If you’re a PhET user, you’ll be interested in this one.  PhET has a new application that allows you to directly embed the simulations into your Powerpoint.  No more switching back and forth between Powerpoint and the simulation, or awkward pauses while you drag the simulation to your projection screen. Just install their free PhET […]

Read the full article →

A great new book! Teaching and Learning STEM by Felder and Brent

June 2, 2017

I was pleased to be invited to write a review for Physics Today on a new book, Teaching and Learning STEM by Felder and Brent.  I loved this book!  I found it utterly charming, useful, kind, and knowledgable.  I highly recommend it, and am going to be purchasing several copies to be able to give to […]

Read the full article →

Outdoor activities for kids: Big Book of Nature Activities

January 15, 2017

If you’re feeling a little stuck indoors with your kids, here is a resource to get your kids (or students) learning from the outdoors even through the colder months.  Last year I picked up a copy of The Big Book of Nature Activities: A Year-Round Guide to Outdoor Learning.  I found it to be a great […]

Read the full article →

Writing a proposal on institutional change? Here are some great resources!

December 21, 2016

This is a repost of my blog post at the ASCN (Accelerating Systemic Change Network) blog.  Check out and subscribe to their blog for more posts like this! For many of us, it’s proposal writing season. If you are submitting an NSF-IUSE proposal, there are increasing expectations that the proposal will include a theory of […]

Read the full article →

Need to train postdocs or instructors to support teaching improvements?

November 15, 2016

This summer, Carl Wieman will host a workshop at Stanford to train STEM disciplinary experts to become educational specialists working in departments, in a short, intensive version of the postdoc training developed at CU Boulder’s Science Education Initiative and the sister CWSEI at University of British Columbia.  If you need to get someone up to […]

Read the full article →

Theories of Change

November 4, 2016

I have been wanting to blog more, but have been having trouble finding as many “transportable” ideas to share publicly, now that my work is more focused on broader, institutional change.  But one thing that I have been working a lot with, which is becoming more broadly recognized as useful in educational projects, are Theories of […]

Read the full article →

I am such a good physicist….

October 28, 2016

Check it out and you’ll see why I’m posting this video… You have to be patient for about 3 minutes, and you’ll see why I’m posting this.  And no, I had never heard of this guy before!  What a kick!

Read the full article →

“Teacher primers” on physics PhET sims published!

August 5, 2016

Over the past several years, I’ve been working with the PhET Interactive Simulations to determine how best to do video “walk-throughs” of their simulations for teachers, showing the main features and how they can be used in a teaching setting.  We call these short videos “teacher primers,” and I’ve got a set of them that […]

Read the full article →

Faculty perceptions around research based assessment (#AAPTsm16)

July 19, 2016

(liveblogging from the AAPT).  Adrian Madsen shared some work to identify faculty ideas and beliefs around research based assessments in physics, such as concept inventories (think FCI) or non-content instruments (e.g., CLASS).  This work is part of a project by PhysPort.org to collect research-based assessments on a website, to provide a more coherent portal to […]

Read the full article →

eAlliances: Mentoring for women (#AAPTsm16)

July 19, 2016

Another interesting resource from the AAPT conference:  An online mentoring community for female physicists.  The eAlliances project (eAlliances.aapt.org) provides peer mentoring networks so women can get advice, support, and connections to other women physics (and astronomy) faculty.  This sounds awesome, and a real opportunity for providing the kinds of mentoring that women need.  There is […]

Read the full article →

Teaching: The Ultimate State of Happiness (#AAPTsm16)

July 19, 2016

(Liveblogging from the AAPT)  I always find Eugenia Etkina to be inspirational, and today is no exception.  In the session, The Art and Science of Teaching, she got a chance to philosophize about her teaching.  She’s noticed that, no matter how experienced she is as a teacher, she always feels that she runs out of […]

Read the full article →

Make your own interactive video vignettes (#AAPTsm16)

July 18, 2016

(Liveblogging from the AAPT).  I just learned about a really helpful tool for creating short videos, which seems useful for flipped classrooms in multiple disciplines:  The Interactive Video Vignettes.  These vignettes are more than just a video, it includes a simple “wrapper” where students can make predictions about what will happen in the video (including […]

Read the full article →

Postdoctoral positions in STEM education

June 14, 2016

Just a reminder that I’ve been actively updating my ongoing list of postdoctoral positions in STEM education!  Feel free to add your own.

Read the full article →

How to help students engage in active learning?

May 27, 2016

So, about 4 years ago now, Andrew Boudreaux asked me a simple question:  “Hey, wouldn’t it be neat to gather materials to help instructors avoid student pushback to active learning strategies?”   What I thought would be a fun little one month project to archive some strategies has turned into a detailed research project.  Thanks […]

Read the full article →