Smart Guys Can Get It Wrong

Exif_JPEG_PICTUREYou know that feeling when one of your friends says something stupid that makes a different friend angry?  And you realize that you agree with the angry friend?  Yeah, so that happened to me a few weeks ago.  Let me explain.

I am a big fan of the FiveThirtyEight blog that Nate Silver runs with his team of statisticians/economists.  Nate has a knack for explaining technical mathematical stuff using everyday examples.  He started in sports and moved to politics (correctly predicting most of the 2014 races), and then ESPN brought his blog back over to their site where it lives now.  When he sticks to those two topics–sports/politics–he is a bastion of logic in a world of opinions.

Lately, though, Silver has been dipping his toes into the realm of educational policy and the ridiculous questionable data that supports some of the recent “reforms”.  In the recent article “The Science of Grading Teachers Gets High Marks“, Silver’s ed dude Andrew Flowers analyzes some of the discussion around the Vergara case.  He discusses the back and forth between statisticians at Harvard, Brown, and Columbia and Jesse Rothstein of Stanford.

While I agree with Flowers that the arguments over methods for analyzing teacher impact are a positive sign that science is working as it should, I side with Valerie Strauss when she writes for the Washington Post that,

“The quality of the underlying standardized  assessment is assumed to be at least adequate — or why use the student scores to evaluate their teachers? — when, in fact, many of them are less than adequate to provide a well-rounded, authentic look at what students have learned and are able to do.”

Flowers provided this throw-away phrase that was guaranteed to make educators angry,

“In order to perfectly isolate the effect of a teacher on a student’s test scores — setting aside whether higher test scores is the right goalstudents would need to be assigned to teachers randomly.”  [emphasis mine]

What?!?  How can you “set aside” the source of all of the data that you are analyzing (or, more accurately, discussing the analysis of)?  That’s like saying, “Setting aside the fact that koalas are not actually bears, observing them is a great way to learn about the bear behavior.” We MUST stop pretending that mathematical analysis can make up for crappy assessments.

Opinions?  You know what to do.


Image: “Friendly Female Koala” by QuartlOwn work. Licensed under CC BY-SA 3.0 via Wikimedia Commons.


Locus of Control

“Better policy would focus on school and teacher inputs. For example, we should agree on a set of clear and specific best teaching practices (with the caveat that they’d have to be sufficiently flexible to allow for different teaching styles) on which to base teacher evaluations. Similarly, college counselors should provide college applicants with guidance about the components of good applications. Football coaches should likewise focus on their players’ decision-making and execution of blocking, tackling, route-running, and other techniques.”

From a recent post by Ben Spielberg (hat tip to Larry Ferlazzo for sharing it) in which he uses quotes from Nate Silver’s “The Signal and The Noise” to destroy the idea that teacher evaluation should be based on short-term data like test scores.  I loved Silver’s book and I enjoyed the connection here.


Wait, Isn’t Norm-Referencing Bad?


I was lucky enough to be in a day-long session last month about assessment and one of the statistical models that is used to measure the value-added by teachers.  Before I start to vent, I want to admit that my statistical background is sketchy at best.  I am also in a particularly frustrated place due to the impact that these models play in the determination of teacher “effectiveness”.

During the presentation, one of the most fundamental aspects of state-wide standardized test analysis was explained to my group and I was floored.  The metric that is used to determine if my students have shown growth (or academic improvement) is not their absolute score on the test.  It is their percentile rank that is expected to improve.  Students who improve at the same rate as the average in the state will have a growth index of zero.

This is akin to lining up all of the students in my state according to their scores.  If this line of students all move five steps (or fifty steps!) forward, they have all made improvement.  By the simplest definition, they have all learned.

But, in the world of high-stakes testing, since none of the students moved “up in line” or improved relative to others, they have not improved.  As a teacher, I have failed them.  Even if they made five hundred steps of forward progress, their growth index is zero.

Perhaps the biggest reason that this frustrates me, is that classroom teachers have been taught for decades that norm-referenced assessments have many weaknesses.  These assessments present students and teachers with a “moving target” since success depends on the performance of others.  Grading on the curve is widely accepted as unfair assessment practice.

Yet, this is exactly what we are doing at the level of standardized tests and teacher effectiveness.  In the interest of continuous improvement, the bar moves from year to year. But, the result is shifting sands beneath our feet that rob educators of the ability to anchor their instructional goals to something concrete.  We are told, “Just keep teaching them the best you can and you’ll do fine.”

But, if large groups of students do better, my evaluation will be negatively affected. In the world of norm-referenced standardized testing, a rising tide can actually sink many ships.  If we were pessimistic about the anti-collaborative effects of standardized testing before this realization, we should be downright fearful that as more educators realize the way that our effectiveness is calculated, corruption will become much more common and teachers will begin to make choices that are not in the best interest of students.

Am I overdoing it on the gloom and doom?  Let me know in the comments.


How Teaching Prepared Me For The Zombie Apocalypse

zombieThese days we live every minute looking over our shoulders for zombies.  The undead are just a fact of life.

When It began, some folks struggled to adapt to the status quo.  They lacked survival skills, or weren’t vigilant enough.  They trusted too easily and tried too hard to work together.  They lacked the ruthlessness that a life on the run demands.  But, not me.

You see, I was a teacher.

In the days before the Zombie Apocalypse, I taught science at a public middle school in North Carolina.  When the federal and state agencies adopted “merit-based teacher assessment”, linking our evaluations to high-stakes standardized tests taken by students once a year, few predicted the effect that it would have.  Sure, many of the “reformers” complained that it would destroy our schools, but no one listened.

They warned that teachers would shed their collaborative nature, and become cutthroat mercenaries.  We would hoard lessons ideas and supplies to keep a leg up on the others in our building.  We would fight tooth and nail to teach the students with the fewest obstacles to learning, reasoning that we can’t help anyone if we don’t have a job.  And you can’t keep your job if you don’t get those scores up.

The reformers predicted that it wouldn’t take long for the cheating to begin.  Under tremendous pressure to show that all of the work that we did in our classrooms had “value”, teachers would start teaching to the test and eventually just feed answers to the students on test day.  No reasonable educator would put her career in the hands of little Johnny and whether he ate breakfast that morning or felt like doing his best on test day.

Days before the virus broke out and brains became a delicacy, an intrepid education blogger posted his “How to Succeed as a Teacher” list.  It was supposed to be a joke.

  1. Don’t share your best ideas with ANYONE!
  2. When forming teams, make sure that there is always someone slower than you.
  3. Hoard and steal.
  4. Trust no one.
  5. Don’t help those who are struggling, as you will end up suffering their fate.
  6. Do whatever it takes to survive.

It turns out that these are the same skills that you need to escape from hungry zombies.  Lucky for me, I was trained to be ruthless as a public school teacher before the epidemic.  I feel bad for those suckers with “21st Century Skills” like collaboration and soft skills like compassion.  They’re just zombie food now.

photo credit: caliopedreams via photopin cc


Response to Satire

I have been following the national debate about teacher evaluation for some time. As an educator, I have mixed feelings about the discussion. On one hand, I agree with the reformers who say that measuring teacher effectiveness is critical to improving teachers and thereby our entire public school system. On the other hand, however, I find fault with most of the current systems for evaluating teachers.

Bob Bowdon’s recent satirical piece on the issue (“Classroom Grading is an Attack on Students“) does much to convince me that attitudes need to change before we’ll find a good solution to this problem. Bowdon uses student grading as a metaphor for teacher evaluation and tries to show that teacher unions are being unreasonable in their focus on the fairness of current systems. While fairness is not the most important factor, it is the one that most separates student grading from teacher evaluation. Students are graded based on multiple measures of performance and with great emphasis on consistency. Teacher evaluations are much more subjective.

Now, I have nothing against honest and accurate measurement of teacher mastery. I’ve written before about the positive changes that have come to the North Carolina Teacher Evaluation Instrument. No one whom I know is arguing that teachers should not be evaluated. On the contrary, most teachers that I speak with welcome evaluation. Fairness is certainly an issue, but it isn’t the biggest one.

The most important parts of this debate center of what constitutes great teaching. Is it the ability to produce high test scores from your students? Is it a knack for helping your students’ scores improve? It is building meaningful relationships and teaching citizenship and character along the way?

If you agree that it is some mixture of all of these characteristics (and more!), then you recognize the absurdity of using only student test scores to determine which teachers are effective.

Just as it would be absurd to measure a student’s mastery based on a single test score.



This is the second post in an occasional series about Why I ♥ My PLN

photo from

One of the most important advantages of blogging is developing an interactive audience. This week, a new commenter really got me thinking about untapped uses for Flip-style pocket video cameras. In response to my post about a unique teacher evaluation system being used in Larry Ferlazzo’s school in Sacramento. Janice writes about a program called in her district “Side by Side” that pairs up novice teachers and experienced mentors,

“With the low cost and ease of use of Flip video cameras, there’s really nothing stopping us from planning a specific instructional strategy focus, videotaping our lessons, and then analyzing them together. In the past, I think teachers felt vulnerable about exposing themselves to consultants who might report weaknesses to administrators, but now that teachers can do it all themselves, I think you’ll see this tool being used more and more often.”

Wow, huh?  Not only is it encouraging to hear about an open-minded district administration willing to push the traditional mode of teacher evaluation, but kudos to them for finding a method that is simple and (probably) more cost-effective.  And, she hits the nail on the head when she identifies the feeling that every educator feels while being judged by an outside party.  This tool allows teachers to tape themselves in a fairly informal way for later reflection and discussion with their mentor.  Or, mentors can do the recording of the rookie to have some concrete “teachable moments” during the post-observation conference.

Best of all, Flip cameras are common enough (and small enough) to be fairly unobtrusive in your classroom, especially if you already use them for student activities.  The toughest part of any of my own recorded lessons was getting my students accustomed to the presence of the camera so that they would begin to forget it was there.

In the days before social networking tools and digital media made it easy for professional educators to share ideas like this one, Larry and Janice’s experiences would have remained locked up in their respective systems.  Now, they are instantly shared and can lead to meaningful improvement in the way that we do what we do.

What’s your take on classroom observation?


Evaluation for the Purpose of Improvement? What an Idea!

flickr user zcreem

One of the most frustrating aspects of the recent teacher-blaming, test-pushing, charter-happy “reform” effort to me is the complete lack of emphasis on improving the work of teachers.  That’s why I was so excited to read Larry Ferlazzo’s recent piece on the videotaping and formative assessment going on at his school.  He writes:

Our school, led by principal Ted Appel, has begun having Kelly Young, an extraordinarily talented consultant on instructional strategies who we have been working with for years, videotape our lessons (I’ve written much about Kelly in this blog). He then meets with us to review an edited version of the tape, with us initially giving our own critique and reflections followed by his comments. This process is entirely outside of the official evaluation process, and is focused on helping teachers improve their craft.

This is mind-blowingly great stuff, isn’t it?  So, what would it take for something similar to happen in your building?  If you are a school leader, what is keeping you from putting this type of tool in the hands of your teachers?