The burden of proof: What does education research really tell us?

by Stephanie Chasteen on February 25, 2009

UPDATE: I’ve just posted a new article about how educational innovations do (and don’t) spread around.

Do active learning strategies work?

This article — and especially the lively discussion in the comments — argue about why college instructors aren’t using active learning strategies, and whether there is evidence that such strategies work.  He says:

People have known for a long time that college students learn more when they’re actively engaged in learning via hands-on practice and other means. But many professors refuse to adopt these methods, because they don’t want to and they don’t have to.

Am I missing something? To be clear, I’m not advocating for some kind monolithic scripted curriculum. When I put my class together, I made choices about subject matter and methods that suited my expertise and instructional strengths and weaknesses. But it seems to me that the more autonomy faculty are given in the classroom, the greater the burden of proof to demonstrate that their choices are actually working, with that proof being based, in significant part, on some evidence of what students learn. Isn’t that what higher eduction is all about — evidence?

His commenters didn’t necessarily agree.  As taken from Richard Hake’s summary of the discussion:

2. “Chemprof” wrote on 6 Feb 2009:  “Always using hands-on or active learning methods is inefficient. Students like it because it is more fun than lecture, reading, or ordinary group reviews, and administrators like it because it looks exciting and innovative. However, these activities take a long time, and require a significant slowing of the pace (dumbing down) of the class. Making a class easier is usually popular with students because they don’t have to work as hard. It is also popular with administrators because more people pass, so they can brag about ‘student retention’. Of course, students will graduate not knowing very much, and they won’t be able to compete on the world marketplace, but that isn’t the administrator’s problem anymore.. . . . .”

10. Joseph Foster wrote on 9 Feb 2009:  “. . . lecturing is quite efficient, and at least when well prepared and students have attention spans beyond that encouraged by Sesame Street, it is also quite effective. And not just for imparting ‘information'[although our students are certainly lacking in it], but for showing how to develop ideas, arguments, how to tell the difference between facts and data, and how to work from a set of data to significant generalizations.]. . . . .”

22.  ‘Dr. Mike” wrote on 10 Feb 2009: “. . . .Keeping kids ‘involved. in class, in spite of it’s inefficiency, now trumps lecture. It encourages the view that learning is the responsibility of the professor, and that learning occurs only in class. Ironically activities that are labeled as ‘active learning’ in a group setting in class are anything but in a broader sense, as they reinforce many students’ ideas that ‘active learning’ on their part as individuals outside of class is and should be unnecessary. So why do so many professors continue to lecture in spite of the constant railing of educational ‘researchers’ not to? . . . .[Not so – see e.g., Schwartz & Bransford (1998)]. . .
Lecturing is an efficient way to cover the material while still allowing topics to be covered in depth. It has proved to be a very effective mode of learning for hundreds of years. And, I suspect, that when many have experimented with ‘active,’ ‘student-centered,’ or whatever you want to call them, methods out of some sense that they’re not ‘keeping up’ with changing times ( like myself), they’ve found them to be unsatisfactory. When educational ‘research’ and theory actually begin to jibe with personal experience of effectiveness (the ultimate form of evidence) , then they’ll be given more credence, not before.”

Wow.  And it just got more vitriolic from there, including mudslinging at my boss (Nobel Laureate Dr. Wieman, who devoted his Nobel $$ to furthering science education and research).

Bill Goffe wrote in: For a great intro to why physics is moving to interactive approaches to teaching (which is based on reams of data), see  “Why Not Try a Scientific Approach to Science Education?” (PDF) by Dr. Wieman.

Here is a pithy quote from that article, which summarizes quite well what my job is about:

“Our society faces both a demand for improved science education and exciting opportunities for meeting those demands. Taking a more scholarly approach to education-that is, utilizing research on how the brain learns, carrying out careful research on what students are learning, and adjusting our instructional practices accordingly-has great promise. Research clearly shows the failures of traditional methods and the superiority of some new approaches for most students. However, it remains a challenge to insert into every college and university classroom these pedagogical approaches and a mindset that teaching should be pursued with the same rigorous standards of scholarship as scientific research.
Although I am reluctant to offer simple solutions for such a complex problem, perhaps the most effective first step will be to provide sufficient carrots and sticks to convince the faculty members within each department or program to come to a consensus as to their desired learning outcomes at each level (course, program, etc.) and to create rigorous means to measure the actual outcomes. These learning outcomes cannot be vague generalities but rather should be the specific things they want students to be able to do that demonstrate the desired capabilities and mastery and hence can be measured in a relatively straightforward fashion. The methods and instruments for assessing the outcomes must meet certain objective standards of
rigor and also be collectively agreed upon and used in a consistent manner, as is done in scientific research.”

Then on 11 Feb 2009 “Dr. Mike” commented:

Dr. Wieman claims to have educational research to show the superiority of his ideas (although the process by which, and context within which, these were arrived at are not detailed). . . . . . .my training as a scientist and my sense of logic only allows me one conclusion: whatever educational ‘research’ is, its not ‘research’ in the same sense that scientific research is. I suspect that most of us necessarily rely on our own experience more than the pronouncements of others, regardless of their good intentions and credentials. In short, educational research will be given more credence when personal experience can consistently demonstrate its validity.”

I’ve seen this “education research isn’t valid” argument before (in fact, I’ve made it myself).  Certainly the knowledge we gain from education research is of a different flavor than that we get by, say, accelerating protons around a huge track.  But the real test is repeatability.  If we get the same result in different contexts, then we can say that this is something that we accept, as scientists, to be true.  We can write theories to explain it, and then do more measurements to see if it fits the theory.  So far, “active engagement produces learning” seems to fit a great many theories and we have multiple data points to support it.

One comment by John Clement I found particularly interesting:

One of the curious things is that science researchers do not recognize that even scientific articles are designed to convince the reader.  The reader can not look at the original data and analysis and has to take the author’s word on good faith.  There are articles that detail how this type of thing is done.  Actually any scholarly article is designed to do the same thing no
matter what field it is written in.  I suspect that it would be possible to write a completely bogus article for a scientific journal, have it accepted, and have readers think it is interesting.  This has been done in other journals.  Now it may be that some of the traditionalists can not read an
education article because they do not understand the assumptions made by the author.  But I think many would be surprised if they actually read some of the better ones.

And of course, if we want others to accept our arguments and data, we should write them so they make sense outside of our discipline!  (I was happy when a participant at my talk at the AAPT on Monday said that the speakers in that session — on clicker usage — “talked like real people.”).

Dewey Dykstra also argues that clearly our data hasn’t been sufficient to produce disequilibration in faculty — in other words, they’re happy with their current model (students aren’t smart enough) and see no reason to switch to a new model (students need to be engaged to learn).  If the data don’t shake them up, they won’t change their view.  In fact, any time anybody has changed their fundamental views of the world, it’s been because of some mental shake-up like this (cognitive dissonance anyone?).

{ 10 comments… read them below or add one }

Gene Gordon February 26, 2009 at 9:26 pm

I have been fighting this battle for 23 years. The more hands-on and inquiry I do, the more people fight what I do.Thousands of excuses against hands-on boil down to – “I just don’t want to do it that way.” Lecture is way too easy to blame the student so why do it any other way? There is even a current discussion on a physics listserv that teachers are saying that adults learn mainly by reading and not doing. How can people think that?

sciencegeekgirl February 28, 2009 at 6:36 am

this is hard to hear, Gene. All this makes such *sense* to me, and I don’t understand why people argue against it so. Or have I just been indoctrinated into a different doctrine?

Robert March 8, 2009 at 3:25 am

It’s an exceedingly complicated problem, Stephanie. I’ve been trying for the past 2.5 years (ok, that’s an order of magnitude less than Gordon’s 23) to convince colleagues that there might be a better way, and all I can say is that those people who have really been receptive have been those who had previously, on their own, concluded that the current system was not working.

Deirdre March 8, 2009 at 10:30 pm

If you would like to see examples of medical students actively engaged in learning in large classroom settings, see http://medicaleducation.wetpaint.com/page/Active+Engagement.

I attended Dr. Wieman’s lecture in Saskatoon a couple of years ago and he inspired me to take pictures.

sciencegeekgirl March 8, 2009 at 11:22 pm

I attended Dr. Wieman’s lecture in Saskatoon a couple of years ago and he inspired me to take pictures.

What nice pictures, Deirdre, they really capture the difference between a standard classroom and one where the students are engaged. We’re also making professional videos to show what these classrooms look like — the first one is on clickers and is at http://STEMclickers.colorado.edu.

sciencegeekgirl March 8, 2009 at 11:23 pm

all I can say is that those people who have really been receptive have been those who had previously, on their own, concluded that the current system was not working.

I wish I had more time to delve deeply into the institutional change literature. It’s very interesting, and academia can take a lot of tips from business. You can also see my previous post, “Twisting the Ivory Tower”.

sciencegeekgirl March 12, 2009 at 4:37 am

Another recent conversation with a faculty brought this all home to me again; He said that without disequilibration (ie., if their internal story of “what education is” seems to hold) then there’s no reason for them to consider another model.

Just like students, with faculty (and anyone else), to change our ideas of how things work, we need dramatic demonstration that our way of thinking about things isn’t well founded. We need a double-take moment, and we need to *experience* that double-take moment. Just reading this post won’t change anybody’s minds. They have to be in the middle of a situation, and learn by direct experience, that their view of education may be incomplete. Ironic, isn’t it? Direct experience is the only way for us to learn that active engagement is the best way that people learn.

Check out the pingback comment above (at elearnspace) for an alternative view on what this all means for the “lecture vs activity” debate, and my opinion on her opinion. 🙂

Hope this all makes sense, it’s a little late, I’m a little sick.

sciencegeekgirl March 17, 2009 at 5:31 am

Here is a recent post from another blog on using active learning strategies to *engage* students in lecture:
http://teaching-tips-machine.com/blog/active-learning-activities-think-write-discuss-lecture/

April Hayman March 17, 2009 at 3:06 pm

After reading your article, I wondered if those instructors who are so resistant to change just don’t know how to incorporate active learning strategies into their curriculum. Some subjects, like chemistry, lend themselves very well to active learning. However, art history may be a more problematic (although I can think of half a dozen activities off the top of my head). If we are not teaching our instructors how to actively engage students, then how can we expect those most resistant-to-change to adapt to a new mode of teaching and learning?

sciencegeekgirl March 17, 2009 at 3:26 pm

Indeed, I (and other colleagues) have come to the conclusion that we can’t *tell* instructors that the current method isn’t working any more than we can *tell* our students Newton’s Laws (and have them understand it). Instructors need to experience first-hand that the current model isn’t working, and then be motivated to seek solutions. Once that “a-hah” moment has come, then they’ll be more willing to consider active engagement.

But even if they’ve hit that point, then some might just throw up their hands and go back to lecture, because it is, after all, easier in a lot of ways, and it’s the model under which we were taught. Without more professional development in active engagement methods, we risk losing the benefit of those motivated instructors — they just may not know what else they can do, or not have the professional support networks to improve their practice.

Leave a Comment

{ 8 trackbacks }

Previous post:

Next post: