Wayman Services, LLC

If I Use Data Like You Say, What Doesn’t Get Done?

A middle school math teacher is teaching a lesson on how to figure percentage. He gives the lesson, then during homework time, he goes around to answer questions and see how the students are doing. As they leave, he thinks, “I don’t think they got this as well as I wanted.” He’d seen some body language during his lesson that indicated confusion and he got some questions during homework time that made him suspicious. After school, he goes next door for 5 minutes to bounce this off of another teacher, who suggests another way to teach percents. He’s not going to redo the entire lesson for tomorrow because it doesn’t fit into his long-term schedule. But he does find a way to work this new way into review time at the start of tomorrow’s class. After that class, he feels better about it – percent seems to make more sense to his class and he feels better about moving on. When the test comes around, the students do ok on the percent portion – not as well as he would have liked, but better than he was worried they would do.

I think most data use experts would be nonplussed by the approach this teacher used. What advice would they offer this teacher? There are many approaches, but maybe the experts might say that he needed to give a more rigorous assessment at the end of the class period. They might recommend that he sits down on his plan period or after school and examines those data for trends – so he knows which students “got it” and which didn’t. They might recommend that he brings those data to that other teacher and has a quick conference about where to go next. Then they might recommend that he groups his students differently tomorrow, enabling those who understood percents to continue to be challenged, while enabling those who didn’t to have more instruction. After tomorrow’s lesson, they might suggest another assessment to make sure the students got it, with a similar process to follow until they do. (Note: we can quibble with the “advice” I describe here. There are a lot of different ways the “experts” might cast this, but that’s not my point. Read on…)

That’s great. Except it probably doesn’t work for the teacher. Why not? Well, ask yourself another question: why did the teacher ignore this second method? Maybe because he didn’t know it, but there’s a very good possibility he does. Instead, I’d argue he went with his method because it worked. Because here’s the deal: the teacher is casting this issue, this use of data, within the other demands of his job. The teacher is, perhaps implicitly, asking himself a question that data use experts often don’t: if I do this second method, what doesn’t get done? The solution offered by the data use experts is great, but it’s much more time-consuming. The teacher can execute his chosen method pretty quickly – he’s already organized his data and interpreted it in his head, and his conversation with the teacher lasts about 5 minutes. He gets a solution and is able to implement it fairly quickly in the frame of what he’s doing during tomorrow’s class. And it doesn’t disrupt his long-term planning. His method works for him. (Note that the teacher actually did use data with his chosen method: he observed body language, he took anecdotal note of their questions during homework time, and he talked to the other teacher. That’s a triangulation of multiple forms, but they’re not rigorous and explicit.)

It’s probably true that his method left some students lacking knowledge, and that’s not good. His method is insufficient, to be sure. But so’s the other method. That second method might look good for that particular instance, but what are the domino effects – how does it look in the bigger picture of what this guy has to do? What other lessons suffer? Which parents can he not call? What other aspects of student learning does he not get to think through?

I’m not saying this guy’s method is great, nor that the other one isn’t sound. I’m just saying that it’s very possible that this guy is a better overall teacher using his method. It’s an area that I think the field of data use has kind of missed the boat on. Research hasn’t typically considered where these other dominoes might fall. Research sees his job in terms of this specific decision, or maybe a group of these decisions, but he sees his job in terms of his overall contribution to teaching. I think it’s a big reason why teachers often don’t use data in the ways that research says they “should.”

So that’s what I think. Thanks for reading.