Our education research group here at University of Colorado had a visit and a very interesting talk by Sanjoy Mahajan, director of the teaching and learning laboratory at MIT and former physics professor, last semester. He focuses on understanding and improving students number sense, mostly through use of approximations and estimations. He’s a very provocative fellow. Here are some highlights from his message to us.

There are 26 sheep and 10 goats on a ship. How old is the captain?

That was a question given to 2nd and 3rd graders in France back in the late 1970’s. The answer, of course, is 36. Or at least so stated most of the children who answered it. Here’s an interesting writeup from a researcher who reports on several variations on that original experiment, with odd and disturbing results. Children argued that the number of the flock determined the age of the shepherd, and if members of the flock ran away, then that affected the shepherd’s age! Do click on that link above, it’s very interesting. One of the researchers whose work he discusses said:

The students he interviewed not only failed to note the meaninglessness of the problems as stated but went ahead blithely to combine the numbers given in the problems and produce answers. They could only do so by engaging in what might be called suspension of sense-making – suspending the requirement that the way in which the problems are stated makes sense … There is reason to believe that such suspension of sense-making develops in school, as a result of schooling.

Here is another example, from Sanjoy Mahajan, about a lack of number sense. In a national assessment of mathematics ability, students 13 and 17 years old were asked:

Estimate 3.04 x 5.3

It’s even easier than you think. They were given a set of answer choices:

A) 1.6

B) 16

C) 160

D) 1600

E) No answer

Here are the responses of the students, 13 year olds and 17 year olds

A) 1.6 28% 21%

B) 16 21% 37%C) 160 18% 17%

D) 1600 28% 11%

E) No answer 9% 12%

The conclusion I draw from this? We’re doomed. I mean, the 17-year olds did a little better than the 13 year olds, but not that much. And get this! **70% of those students could correctly do the algorithmical multiplication problem.** This isn’t a problem of multiplication. It’s a deeper problem of not understanding our number system.

Students just wander around in a random walk in solution space, he says, until they get something that looks like an answer. And they put a box around it. We’d rather, of course, that they have a guide, a sort of nose for where they should go in solving the problem. Then a path to the solution will be more direct. But that requires understanding, rather than rote learning. Rote learning, believes Sanjay, is an evil thing to be eradicated from our learning system. Not everyone in our group agreed.

In another example he gave, he demonstrated how much more comfortable students are with algorithmical numerical calculations than with other solution methods. Even when a graph was right in front of them, demonstrating the answer to the question, they ground through the calculation. Sanjay argues this shows a lack of reasoning and understanding. Students have been taugh that numbers are a more valid way of reasoning, and that this is what teachers are looking for in answers, rather than pictures and graphs.

Or, how about this one. You drop a ball on the table and it bounces. What are the forces on the ball at the moment that it’s stationary on the table? Think about it a moment.

Did you answer “mg”? That’s what most students answer. We’re so used to the normal force being equal to mg when items are stationary. So, Sanjay has his students put out their hand on the table. He places a rock on their hand.

Sanjay: What’s the force of the rock on your hand?

Student:

mg

OK, no problem. Now he holds the rock above their hand and makes as if to drop it.

Sanjay: Hey, why are you moving your hand?

He places the rock again on their hand. “That’s what *mg *feels like. Why are you afraid of *mg*?”

OK, so they decide it must not be *mg*. It must be, maybe, 2*mg*. That seems plausible, given all those momentum conservation problems they’ve done. So he puts two rocks on their hand. That’s 2*mg.* That still feels OK. So it must be more than 2 *mg*.

Now that they have that physical intuition, he says, they’re ready to see the symbolic manipulation.

Here’s the answer as he described it. Acceleration goes as the velocity of impact divided by the time of contact. What is the time of contact? The bottom of the ball hits the ground, but the top keeps going until it gets the signal that the bottom has hit, that there’s no more room to move down, and it’s time to start moving up. That happens at the speed of sound.

So

**Conclusion: Ouch**

Sanjay argued that doing this kind of qualitative reasoning is both a diagnostic tool (to see if students have understood you), a treatment (to get students thinking qualitatively) and fun. This gives students a tool to understand and estimate numbers in* any *problem, not just physics. He wants them to have a feel for what’s going on before they start plugging in numbers.

{ 13 comments… read them below or add one }

Love your approach of putting numbers in the broader context of logic. Gonna link you from my blog.

Regarding the impact force: the signal transmits at the speed of sound, but that does not mean the contact time is given by that. Simple inspection of a ball hitting a surface should tell you that the impact time is orders of magnitude longer than 10 microseconds. Model this as a mass on a spring — you want to look at the oscillation and find the half-period. That will tell you the contact time. The force comes from Hooke’s law.

Interesting post – this is a bit related to a question I once got in a job interview: “How many manhole covers are there in Boulder?” It came completely out of nowhere, and I suspect was intended to evaluate my skill of estimation. It certainly didn’t have anything to do with Java programming! This was probably two years ago now, and I *still* am trying to think of a good answer, as well as why on earth anyone would care. Sure, figuring out the math question above makes sense and is relevant. BTW, in a later interview, the question was “How many bears are there in Alaska?” Fortunately, this was a team interview, and the other interviewers said, “What is that supposed to be about?” I think somewhere there was a blog or book about interviewing techniques used by Microsoft.

So is there more to this “gentle art of estimations”? And isn’t the certainty often very low? Low enough to be useless?

There is a ton you can do with estimations… just google “Fermi Problems” and you’ll see a bunch of questions that have no real “answers” but are exercises in estimation.

These are answers without a “real” answer, so I’m not sure what you mean by “certainty”. We can’t see how far off we are in an estimation — the idea is to get a rough sense of the magnitude of scale of an answer. And the main point is the failure of education in this quest. If you ask a student to estimate the size of a car and they come up with something around 10 inches, there is a real problem with their number sense. We get used to plugging numbers into problems to get “the answer.” But do those answers make sense? Having these sorts of heuristics — even if you’re off by a factor of 2 — are extremely valuable when you’re figuring out, say, how much further your car will go before it runs out of gas.

So many people struggle with knowing how big a meter or a kilogram is, they don’t know what a reasonable answer is.

A related post talks about how to use your body to learn the metric system — Body Metrics:

http://blog.sciencegeekgirl.com/2009/02/21/body-metrics-helping-students-learn-the-metric-system/

Neat post. I like the part where you say the conclusion you draw from this is: We’re doomed.

Sigh. We so are. But sometimes I think people have always been this stupid. It’s just we notice it more now that we have standardized tests and universal suffrage.

Jen

http://theartfulamoeba.com

I’m disappointed that you (and Prof. Mahajan?) didn’t mention anything about the latter part of the “How old is the captain?” writeup, which puts a very different spin on the phenomenon! To summarize, the writer-uppers discovered that if students were made aware that the captain’s age problem might not be solvable, they were much less likely to come up with a senseless answer. They conclude that their students had trouble with the captain’s age problem not because they didn’t understand the connection between math and reality, but “because they had learned during their educational socialisation that every mathematical problem has to have a result, which can be definitely determined.”

That’s interesting information, Aaron, thanks for sharing. I did look up a little bit about the study when I wrote the post, but hadn’t found that addendum.

It’s still an interesting problem — the idea that we’re so used to *solvable* problems in school, whereas in real life we get all these unsolvable/wave-your-hands problems we have to make do with our best guess.

No wonder I found school so satisfying. I loved getting an answer and putting a box around it. Where’s my equivalent of that now? I used to get it from balancing my checkbook, but that liminal pleasure evaporated along with my OCD-like attention to detail that made balancing my checkbook possible. 🙁

I couldn’t work out the approximate forces on the rock *feels dumb now* At least I got the multiplication question right – that’s pretty shocking that two thirds of kids 17 years old can’t figure that out.

I remember feeling the same way as the kids in the study in school, that if it’s maths there HAS to be an answer. Highschool at least teaches us that sometimes you don’t have enough information, and you’re allowed to say Captain’s age = x.

The correct answer to the “Age of the capitaine” question exists : it is 39 years old. This is an old french riddle that plays on the double meaning of the word quarantaine which means “quarantine” and “forty years old”. In french the answer is : “Il a 39 ans car il va vers la quarantaine”. Which has two translations : “He will soon be forty” and ” He goes to the quarantine”. ( The correct wording of the question is : A ship transporting 26 sheep and 10 goats is arriving at the harbor. How old is the captain? )

Jean-Michel—

Ha! Awesome! ^_^

I just want to echo Tom’s comment above regarding the miscalculation of the forces involved in the rock-on-hand stuff above.

Even if the rock is infinitely stiff, the hand beneath it isn’t, and will deflect by a few millimeters, not a few microns. This is why one can drop stuff on one’s hand without it shattering.

It’s ironic that an article lamenting the inability of students to use physical reasoning or do order-of-magnitude estimates would use a method of no physical validity to arrive at a numerical answer which is off by orders of magnitude.

P.S. For a fun video of hand vs rock (with the hand deflecting by more than a few millimeters):

http://www.youtube.com/watch?v=mOLp4doE51Q

Thanks for your thoughtful comments, Tom and Anonymous. To clarify, Anonymous, the calculation for the ball is not, I believe, intended to relate directly to the hand. The reasoning for the force on the hand being larger than mg (and larger than 2mg) is the heuristic reasoning earlier in the post. The calculation of the exact force of impact is meant to relate to the ball, not the hand. One aspect I didn’t mention was that the hand is on the table. The ultimate point that the force on the hand (and thus the force on the ball at moment of impact) is larger than mg is unchanged.

So what is the exact force on the ball? You disagree that the time of impact is on the order of microseconds. I took this calculation directly from Dr. Mahajan, and it was rather an afterthought since the point isn’t the symbolic manipulation. I will do some more thinking on the matter. I’m not so sure that his reasoning of the time that it takes for the signal to pass from one side of the ball to the other has “no physical validity”! That seems to be a pretty strong statement for a standard estimation of signal propagation.

Will think more about this tomorrow, gentle readers.

And check out the video posted by Anonymous, it’s pretty neat.

>The calculation of the exact force of impact is meant to relate to the ball, not the hand.

I suspect the forces may be related in some sort of equal-and-opposite fashion. Or am I misunderstanding the problem?

>The ultimate point that the force on the hand (and thus the force on the ball at moment of impact) is larger than mg is unchanged.

I certainly agree with that point.

>I’m not so sure that his reasoning of the time that it takes for the signal to pass from one side of the ball to the other has “no physical validity”! That seems to be a pretty strong statement for a standard estimation of signal propagation.

I apologize for my provocative language, but I stand by the basic point. (As much as an Anonymous Coward can stand by anything.)

I think everyone agrees that we need to calculate some F=ma equation, and to estimate the acceleration we need an estimate of its duration. It seems to me that what Dr. Mahajan has done to estimate this is to pick a number out of a hat (the speed of sound), but it’s not the relevant number for the problem. The important one is the spring constant of the materials involved, as Tom/swansont pointed out above.*

The spring constant is why if I drop a wine glass on a concrete floor it is more likely to break than if I drop it on a trampoline.

If you’re interested in examples of how to estimate accelerations in a variety of situations, I’d point out that Rhett Allain’s “http://blog.dotphys.net/” blog has a variety of nice examples of calculations of this sort.

*Caveat: accurately working out the problem is more complicated than just the static spring constant due to the fact that we’re looking at dynamics and non-point-objects: at high velocities the mass density of the various objects involved will start to become significant. Working out the crossover between the inertia-dominated regime and the spring-constant-dominated regime is left as an exercise for the reader.