Wayman Services, LLC

What’s Next: Studying Longitudinal Effects of Teacher Use of a Computer Data System on Student Achievement.

In my previous post, I told you about our recent study.  In it, we looked at teacher use of a data system over two years and correlated that use with student achievement.  Much to our chagrin, we didn’t find consistent correlations over time and across content areas.  I guess somebody could draw the conclusion that this data system – or “data use,” even – isn’t effective.  But we’re not yet ready to go there.  That’s too easy, and it’s shallow thinking.  There’s a lot we still don’t know yet about how data systems fit into teachers’ work, and vice-versa.  That’s what we talked about in our Discussion section and I’ll talk a bit about that here.

One of the things this study made us do is really think a lot about what “use” is: use of a system, use of data, and application of knowledge in general.  In our study, we tried to get really detailed about frequency and consistency of the use of data from this particular system.  In fact, I don’t think anyone other than John Tyler’s 2013 study has gotten this detailed.  But even as detailed as we were, this is just one type of data from one type of system, and there’s probably a lot more to cover in this regard.  For instance, how does the use of this system connect with their use of other systems?  How does the use of these data connect with their use of other data (both formal and informal types)?  How about depth of engagement with these and other data?  And how does a teachers’ skill set affect their use of data and the knowledge they draw?  As we point out in the article, “It is one thing to encounter information in a passing way and another to mull over information or to synthesize insights gleaned from various sources.”

The other thing this made us do is think about how the system was supported.  (If you’re at all familiar with my work, you could have guessed that we’re going there.)  I can tell you that this district did a much better job than most in providing support for the system.  They did good work.  But what’s hard about data use and data systems is that we’re trying to use them to help change teacher practice in positive ways.  The types of supports we provide can’t just be around access and maybe some periodic events that help teachers put this knowledge into practice.  It has to be part of a systemic approach to, as we said in the article, “reshape teacher craft.”  And when we do that, we need to make sure we create structures that allow us to listen to how teachers are using the system and how it fits into their notions of teaching practice.  We need to make sure they have not just proper data literacy, but the pedagogical knowledge to make the best use of the system.  And we need to be cognizant of the professional costs that a new system has on their work – there’s only so much air in the balloon, right? – so we need to be aware of how a new system is going to change the way they do their daily tasks.

Ahhhhhh, this is hard.  There aren’t easy answers.  It’s hard to try to figure this stuff out and I wish it were as easy as just putting a data system in front of teachers.  But I think about this like I think about a great quote from “A League Of Their Own.”  In talking about baseball, Tom Hanks’ character says, “If it wasn’t hard, everyone would do it.  The hard is what makes it great.”  So we’ll keep putting pieces to the puzzle in – like this study – and when we figure it all out, it’ll be great for kids and teachers.  Thanks for reading.