Realizing that an Orwellian future is actually happening now is a sobering thought.

Another great post from Curmudgucation that lays out the dark underbelly of standardization. The one overriding message: It's all about money. Call me a dreamer, but there is more to living a fulfilling life than how much money you have. And kids need to learn more than just how to make it.

Sunday, June 8, 2014

CCSS: Schooling for Wretched People in a Miserable World

The Council of the Great City Schools (yet another group apparently set up to make money by shilling for the Core) has created a marvelous promotional video for the Core. Done in the style of those high-speed marker-drawing videos that the interwebs love, and narrated by a possible-non-caucasian lady narrator, it does a fabulous job of distilling the world view embedded in the Common Core complex.

I'm not going to break it down second by second. Instead, lets look at some of the assumptions built right into the program

Education Is A Single Staircase

The central image of the video is the stairway. The stairway is a single path, always heading upward. It should be exactly the same for everyone-- in fact, every "problem" that the video brings up is visualized as ways in which two separate sets of steps aren't exactly the same. Differences are bad. We feel so strongly about this that we even lie about how the new CCSS staircase is just like the international staircase, because Same is Good.

It's a Dog Eat Dog World

"Like it or not," says the video grimly, the world is all about measuring and competition. We see athletic winners blocks with big piles of money going to the top spot. The world is all about competition. Here is a cartoon American competing with cartoon grads from Shanghai and look-- the Shanghai grads are getting all the money.

So it's clear that not only is life all about competition, but there is only one way of keeping score-- and that's with money.

But, Plateaus

The video highlights one of the oddly self-negating qualities of the Core Complex. Our students need to compete, but we've also got plateaus on the steps where we will gather all the students and make sure they're all on the same step. So it's like a race-- but a race where at the end of every lap, everyone has to sit and wait until all the runners have caught up.

Measures Are For the Fatherland, Not the Child

The video makes it clear that one of our big problems is that the different staircases, different tests and different measures make it hard for the Supervisory Other to see what's going on. I actually appreciate the honesty in this point of view-- it certainly beats the usual line that students and their teachers are somehow too boneheaded to know how the student is doing.

But here it's clear. They don't need to know. It's the bosses, the overlords, the fatherland, the high potentates of business and government-- these are the people who need a good, solid dependable report on how well the education system is doing and what kind of product it's churning out.


This video is worth watching because it captures just how small and meager and bleak is the world envisioned by the reformsters. Our students are to all toil away on exactly the same path, with exactly the same steps, to exactly the same destination, where the only measure of their success or worth in this upward rat race will be how much money they get. And that education, as flat and uninspiring as it is, is not being provided for the benefit of the students, but to benefit the uberclass that runs the schools and the factories and the government, the same uberclass to whom the students and the schools are accountable.

What a miserable joyless de-humanizing version of education. What a sad model of a pointless life for wretched people in a miserable world.

Every discipline has its mythology...even education.

How many of these myths do you subscribe to?

Education Trends Subscribe to RSS

8 Myths That Undermine Educational Effectiveness

May 29, 2014 | Mark Phillips

Image credit: Thinkstock

Certain widely-shared myths and lies about education are destructive for all of us as educators, and destructive for our educational institutions. This is the subject of 50 Myths & Lies That Threaten America’s Public Schools: The Real Crisis in Education (1), a new book by David Berliner and Gene Glass, two of the country’s most highly respected educational researchers. Although the book deserves to be read in its entirety, I want to focus on eight of the myths that I think are relevant to most teachers, administrators, and parents.

Myth #1: Teachers are the Most Important Influence on a Child’s Education

Of course teachers are extremely important. Good teachers make a significant difference in achievement. But research indicates that less than 30 percent of a student's academic success is attributable to schools and teachers. The most significant variable is socioeconomic status, followed by the neighborhood, the psychological quality of the home environment, and the support of physical health provided. There are others, but the bottom line is that teachers have far less power to improve student achievement than do varied outside factors.

Myth #2: Homework Boosts Achievement

There is no evidence that this is true. In Finland, students have higher achievement with little or no homework and shorter school hours. The more important factor is what students experience during the school day. Project-based learning, as one example, places the emphasis on what is done during the day. If students choose to do more after hours, that's their choice. There also may sometimes be other good reasons to assign homework, but there should be no illusion that homework will help increase student achievement.

Myth #3: Class Size Does Not Matter

In an average high school, one teacher is responsible for 100-150 students on any given day. Students inevitably get lost in the shuffle. Research evidence strongly indicates that a decrease in the number of students has a qualitative pedagogical impact. When reductions occur in elementary classrooms, evidence has shown that the extra individualized attention and instruction appear to make it more likely for these students to graduate at higher rates from high school. Affluent families more frequently opt for districts or for private schools with smaller classes. It should come as no surprise that larger class sizes may disproportionally impact the children of the poor. Therefore, reducing class sizes will in fact result in more learning.

Myth #4: A Successful Program Works Everywhere

There is significant evidence against the idea that a program successful in one school or district should be imported elsewhere and expected to work well. Context is the key variable. Programs must be related to the makeup of the school district and/or the specific school. Approaches to education that are marketed for nationwide use may be excellent yet totally inappropriate for some districts. A program has to fit the specific needs of the schools and classrooms in the district, and a careful needs assessment coupled with a thorough examination should determine whether to adopt a program, not the success of the program elsewhere.

Myth #5: Zero-Tolerance Policies are Making Schools Safer

This strikes me as one of the most colossally wrong-headed and destructive of the myths. Berliner and Glass describe numerous examples of this policy being implemented destructively, including one in which two students were suspended because one shared her inhaler with a friend who was having an asthma attack. Most importantly, there is no evidence that zero tolerance policies decrease school violence. To the contrary, the authors note that "suspensions and expulsions have far-reaching implications for a student's academics and can set them up for failure in their personal lives." Zero tolerance policies have resulted in school officials routing record numbers of students through the juvenile justice system, students who are then more likely to also end up in an adult prison later on. And, not surprisingly, all of the unintended effects associated with zero-tolerance policies in schools are multiplied for non-whites.
The authors also give examples of some schools that are learning from this research. As one example, after the tragic events at Sandy Hook Elementary School, teachers, parents, and administrators are focused on crisis preparedness and the politics of the gun debate, not on stricter policing of students.

Myth #6: Money Doesn’t Matter

It's a popular argument that, while we're spending more money than ever, test scores remain stagnant. This is a destructive myth widely shared by those who oppose better funding of our schools. Yet the research is clear. When school districts with sufficient resources are compared with those without, achievement outcomes are definitively higher in the wealthier districts. The authors note that it makes a significant difference in terms of student achievement when higher salaries are used to attract more experienced and better-educated teachers. Schools that serve the poor are more likely to retain well-paid teachers, despite the challenging circumstances they deal with each day. Since class size does matter, as we’ve seen, adequate funding makes it possible to hire more teachers and reduce class sizes. All of these assertions are strongly supported by research. Additionally, the authors cite Linda Darling-Hammond (2)'s report on new research from Finland, Singapore, and other countries that provides "striking evidence that spending more, and targeting that spending at students who come to school with the fewest resources, can have a dramatic positive impact on a nation's overall educational outcomes."
Of course, it is also possible that the school districts spending more money are located in communities in which socioeconomic factors and neighborhood quality play an important role.

Myth #7: College Admissions are Based on Academic Achievement and Test Scores

Berliner and Glass' findings are disturbing. Many colleges and universities practice admissions by category. One example is athletics. The most significant variable at 30 of the most selective universities was discovered to be legacy (whether a family member previously attended the university). Wealthy parents who contribute development funds further increase the likelihood of admission. This doesn't mean that universities don’t pay attention to student achievement in their admissions process. It does mean that there is preferential treatment in admissions that relegates academic accomplishments to a lower priority.

Myth #8: Merit Pay for Teachers Improves Student Performance

The full argument is that merit pay is a good way to increase teacher performance, because teachers should be evaluated on the basis of student performance, and rewarding or punishing schools for student performance will improve our nation's schools. However, evidence suggests that competition between teachers is counterproductive and interferes with collaboration. Measuring teacher effectiveness is very difficult, and no simple measures effectively do this. There is no evidence that merit pay correlates with improved student achievement, but there is strong evidence that basing teacher salaries on student performance is counterproductive and ethically wrong -- it frequently punishes teachers and schools for socioeconomic factors over which they have no control.
Every educator, especially administrators and policymakers, should read 50 Myths & Lies. Based on hard research evidence, this book makes it very clear that we need to do a better job of differentiating myth from reality when we make our educational decisions. All too often, decisions are based on myths and what are essentially lies, not on research evidence.

see more see less

• Mark Phillips's Blog

Why Your Evaluation is Dumb.....but it's not a laughing matter

As you've surmised by now if you read this blog regularly, Curmudgucation is one of my favorite blogs. Here we find an incisive look at why your evaluations are ridiculous, and why you should be very worried about that.....

CURMUDGUCATION

A grumpy old teacher trying to keep up the good classroom fight in the new age of reformy stuff.

 

 

Wednesday, June 4, 2014

Why Your Evaluation Is Dumb

As you contemplate your end of the year evaluation paperwork, you are probably thinking (and not for the first time), "This doesn't make any sense." And you are correct. Current practices in teacher evaluation do not make sense-- if you assume that the purpose of these evals is to actually evaluate teachers accurately and effectively.

A good evaluation system gives the employees clear and useful feedback-- a picture of what they do well, and a plan for what they can improve. A good evaluation system also provides management with a clear picture of their organization's strengths and weaknesses. Current thought in teacher evaluation is not interested in either of these.

Proving What We Already Think We Know

Reformsters are sure that schools are failing, and that they are failing because they are packed floor to ceiling with stinky bad teachers. So evaluations don't need to be created in order to answer the question, "How are we doing?" Reformsters already know how we're doing-- we're failing. What they need is an evaluation system that confirms what we already know.

Hence stack ranking for schools. Stack ranking (ICYMI) is a now-discredited corporate model that involved determining the distribution of rankings before anyone was even evaluated. If there are ten employees in your department, we know before we even start the process that two are excellent, two are poor, and six are fair-to-middlin'.

In teacher eval land, this crops up as statements like "You don't live in excellent/distinguished/super-duper. You just visit." This is not a comment on your actual ability; the system starts with the assumption that there are very few teachers who are really good, and probably only in occasional moments. We are not looking to find excellence, because we already know it is not there.

This is just like deciding, before you even hand out the test, exactly which grades will be given, and the grading the tests by matching each test to one of the pre-determined grades. Whether your students all ace it or all flunk it, the pre-determined grades rule the outcome.

The Illusion of Objectivity

Reformsters think numbers are magical, and that only concrete objects are real (this is one of many reasons that one tends to assume that reformsters have rather sad and shallow inner lives).

If an administrator sees something with his own eyes that he can write down on paper, that must be objective. If an administrator is looking at an artifact that he can touch with his own hands, and he assigns it a number, that must be objective. Because, numbers.

I imagine the people who design these kinds of systems sitting at home evaluating the relationships in their lives. "Well, spousal unit, we performed sexual intercourse a total of two times this month, with an average duration of seventeen minutes. I cross-checked this with the video record of those events and determined that your facial expression shows a 5.7 on the arousal scale, giving us a solid 42 scale intimacy rating for this month. This compares to a 46 rating in April and a 51 in March, by which I must conclude that we are experiencing a significant decrease in marital satisfaction, and -- wait? why are you packing??"

Objectivity in teacher evaluation is an illusion. Because, human beings. If your boss hates you and is out to get you, no system in the world can keep him from finding a way to game your evaluation to hurt you. If she's a decent person who is trying to do the best for her people, no system can keep her from doing so.

Even if we hand evaluators a specific list of behaviors to check off as signifiers of teacher quality, that list is itself a reflection  of the bias of the person who made it, and the observer's own biases will affect what he does or doesn't see. There is no such thing as an objective measure of teacher quality. It does not exist. It has never existed. It will never exist. To present a system and claim that it is objective is in and of itself a demonstration of subjective biases about teaching.

Baloney Out Of Your Control

Depending on your location, you are subject to a bunch of evaluative baloney beyond your control.

This is simply hostage-taking. We want you to take these stupid pointless useless high stakes tests seriously, so we will hold your job rating hostage until you do. We want you to think AP courses are worth spending money on, so we will give you a job rating bump if you give us money.

It's also building in a safety for Reformsters. If we left it up to things that are in your control, it would be harder to get the results that we want. Throwing in some X factors helps guarantee that you won't somehow game the system and keep us from finding the widespread failure that the system exists to "reveal."

Saving for a Rainy Day

In Pennsylvania, we go through a long convoluted process to arrive at a pass-fail grade for teachers. Many other states have also chosen a relatively low-impact approach to evaluating, and so teachers feel relatively unthreatened by the process. Don't be fooled. The data is there, showing a wide range of teacher ability and "proving" that there's a vast pulsating pool of teacherly awfulness. Just because they haven't put the data in your local newspaper yet doesn't mean they won't get around to it.

Teachers Under Attack.....Don't be fooled, they're after us.

The following is an example of a larger game plan that has been simmering for decades, the goal of which is to eliminate public education. The opponents of public education are closer to achieving their goal than ever before. Everyone who is concerned about public education, one of the founding principles of our democracy, needs to be paying attention. If you're not worried, do just a tiny bit of research..... it's happening everywhere.

North Carolina Republicans Offer Nearly $500 Million to Abolish Tenure–This Teacher Says “No”

By dianeravitch

June 3, 2014 //

6

North Carolina’s Republican-dominated State Senate hates teacher tenure. They hate it so much that they are willing to offer nearly $500 million in higher salaries if teachers are willing to abandon their tenure.
Bear in mind that tenure in K-12 education is not a guarantee of lifetime employment; it is a guarantee of due process rights. Also note that until recently, North Carolina was thought to have one of the best school systems in the South. The state has–or had, at last count–more National Board Certified Teachers than any other state in the nation.
Why Republicans hate tenure so passionately is a mystery. There is no reason to believe that principals are itching to fire teachers. North Carolina has had such a large exodus of teachers from the profession and the state that wise policymakers should be worried about holding on to teachers, many of whom are demoralized by years of legislative attacks on them.
Stuart Egan, a National Board Certified Teacher in North Carolina, wrote the following letter in response to this latest move by the State Senate:
“North Carolina’s GOP legislators certainly appear to have paid attention in English class: The motif of “making a deal with the devil” is a common theme in many works of fiction and in anything they write concerning teachers.
“Sen. Phil Berger is championing a bill that would create substantial pay raises for teachers who relinquish “career status” and longevity pay for “professional status.” The salaries of teachers who do not surrender career status would remain frozen in a stagnated schedule. Career status is often referred to as “tenure,” but that is a nebulous term. Career status does not mean teachers are untouchable. The General Assembly has spun this word to make it appear that teachers have the same “tenure” as college professors. Not true. We can still be dismissed for not performing our duties or upholding standards.
“The past 10 years in NC educational policy is enough to tell us where this is going. Under the ABC plan from years ago, teachers in schools that achieved certain growth expectations would get bonuses. That system ran out of money several years before it ended, but the requirements for teachers did not change. The monetary “incentive” simply was taken away.
“When the state budget began experiencing shortfalls, teacher salaries were frozen. Many of us are making the same salary we did years ago, but now we have more students and more classes as well as increases in the cost of living. Consequently, North Carolina has lost many of its best, brightest and potential career educators. Between a lack of financial security and the near-constant disdain in which legislators hold us, there is little reason to stay.
“When the General Assembly tried a few weeks ago to lure teachers into giving up their career status early in exchange for a monetary incentive, the courts struck it down as unconstitutional. But what many in the general public may not know is that the state did not have the funds to finance that incentive past the first year. It would have had to remove the monetary incentive three years early.
“This is exactly what will happen in the proposed legislation introduced this past week. The General Assembly already faces a shortfall for next year, and the salary increase for those who give up their right to due process will be removed because the money does not exist.
“To look at this latest deal another way, it would cause North Carolinians to lose advocates for the public school system. In a time when the state budget siphons off money for a voucher program to promote privatized education and decreases the average amount of money spent per pupil, you need to have teachers speak up for students and schools. Removing the right to due process leads to those same teachers being afraid to do so for fear of reprisal.
“Whether you call it career status or tenure, the concept helps keep public education in the hands of the public. It is so valuable to public schools that Sen. Berger and others are willing to pay more than $400 million to take it away. North Carolinians should take note and wonder why our legislators want teachers hamstrung by either low pay or worry about keeping a job more than they want our students to receive the best education possible to prepare them – and North Carolina – for a modern and innovative job market.
“This teacher will not sell his soul, no matter how attractive the devil tries to make the package. There is too much at stake – for teachers, for students and for North Carolina’s future.”
Stuart Egan, NBCT
West Forsyth High School
English Teacher, Career Status
After Stuart sent the letter above, he added this sad postscript:
Concerning the high teacher salary raises in NC tied to tenure forfeiture, I saw this in my local paper (Winston-Salem Journal) after I sent my previous letter. It seems that to fund these raises, Senator Berger pushed through a budget that “would cut financing for teacher assistants, classrooms teachers, administration and transportation to pay for teacher raises.” Therefore, the county school system would have to request from the county that loss of money to cover the positions lost. But the county commissioners cut the local school budget already. The result would be “the loss of more than 250 early grade teacher assistants and 28 classroom teachers, according to preliminary estimates from the district’s finance department.” That is devastating to the K-3, elementary level.
I have a child with special needs in kindergarten who happens to have Downs Syndrome. If his teacher does not have an assistant, then positive results will not be seen as quickly and effectively in his education. Interestingly enough, if I as a high school teacher (or his regular teacher in elementary school) take the salary increase and make a “deal with the devil,” I may have a direct impact on my own son’s education.
Public education should never be this cruelly ironic.
If this is happening in a place like Winston-Salem, imagine the effect on rural counties in North Carolina.
http://www.journalnow.com/news/local/emory-says-teachers-job-loss-inevitable-under-n-c-senate/article_ac04391e-2aae-58df-8370-8a6c67f1869b.html

Piketty is a sensation at the moment....here's the CliffNotes version.

Piketty for Dummies

bydocmidwestFollow

   

Thomas Piketty's Capital in the Twenty-First Century has taken to U.S. by storm. It's been denounced as warmed-over Marxism, praised for its distinctly un-Marxist analytical coherence, denounced as insidious anti-Marxism, praised for its magisterial use of masses of data, denounced for sloppy and tendentious distortions of data, ... A surprisingly small fraction of the people discussing it seem to have actually read it.
I was lucky enough to pick up what seemed to be one of the last three copies for sale in Berkeley on a recent trip, so here's an attempt to distill the main argument briefly. I'll focus on some key economic facts that Piketty takes from data rather than from analysis, since understanding these facts is likely to be the key to understanding where we're headed and how we might change directions.

The first part of the book, told with numbing repetitiveness, does not directly concern inequality at all. It describes changes in the ratio of total capital to annual production. Piketty makes an overwhelming, unchallenged case that this wealth/income ratio has grown a lot in the last few decades, and is approaching values (say ~6) typical before WWI. The period in which some of us grew up, post WWII, was an anomalous result of the shocks of the wars, the depression, and the Russian revolution. These destroyed a lot of capital and constrained the behavior of capitalists.
It's a little hard initially for a reader to see why that growth of capital should be taken as a bad thing. Isn't it better to have more of our work done with enhanced productivity due to accumulated knowledge and machinery? Why wouldn't we want wealth to grow indefinitely?
Piketty's answer shows up next. Throughout the period for which we have records, wealth has always been much more unevenly distributed than income from labor. That's true even now in the U.S., despite our anomalously large wage inequality. Almost everywhere, the bottom half of the population owns almost nothing. Thus unless the fraction of capital in private hands shrinks or the rate of return (ROI) shrinks, a higher fraction of income will come from capital ownership, and this will be much more unequal than income from work.
Will the growing wealth be held privately? The data show that total public wealth (assets-debts) has converged to around zero, less than +/- a year's income, in modern market economies. So unless we somehow understand and change that general fact, the growing wealth will be held privately.
Will the ROI drop much as the wealth to income ratio grows? Marx thought it would. Classical economics predicts that it will, since when capital is plentiful people shouldn't have to promise much ROI to get some. Many economists have been fond of a simplified relation for that drop, in which the returns on capital stay at a fixed fraction of total income even as the wealth/income ratio increases. Piketty shows that in fact ROI tends to stay up near 5%/year regardless of such predictions. He discusses possible reasons briefly and without making any conclusions. The general impression, however, is that the radical inequality in power that goes with radical wealth inequality coupled with large wealth/income ratios allows a variety of exercises of political and social power to counteract the expected drop in ROI.
Thus putting together these empirical ingredients implies that income inequality will grow as the fraction of income coming from capital grows. Europe shows that pattern. In the U.S., extreme inequality in payment for work currently obscures that pattern except in the highest income brackets.
One stage of the argument is not purely empirical. Piketty refers to analytical work showing how a tail of very large fortunes tends to be generated by normal economic processes. As the quantity (r-g), annual ROI minus annual fractional growth of production, gets large, two things happen. Private wealth gains more rapidly on production, and, within realistically simplified models, the distribution of private wealth tends toward a more unequal steady-state. (For the last model-based argument, Piketty refers to some fairly obscure on-line notes. It is not hard to reproduce the result for very simple models, however.)
Piketty make an additional argument, based on common sense bolstered with some nice data, that ROI is largest for the biggest fortunes. That's just what classical theory would predict, since the biggest fortunes can withstand the highest short-term risk and pay for the most sophisticated information and analysis. They can also obtain the most favorable laws, regulations, and extra-legal maneuvers. Of course, this extra factor just worsens the trend toward greater inequality.
Finally, Piketty concludes with some proposals. The key one is to have some modest international wealth tax. This would be a more systematic way of getting at what Teddy Roosevelt intended with our estate tax- preventing the growth of a hereditary aristocracy.
Several important comments have been published on Piketty. I'll comment on a few of them.
Many people have noted that Piketty's remedy, a wealth tax, sounds reasonable and moderate but might be as hard to attain as a revolution. They have an obvious point, but the alternatives aren't so clear either.
DeLong has noted that the "r" used by Piketty is a short-hand for a collection of r's. One gives bare returns, and is important for directly calculating how much income is coming from a given amount of wealth. Another subtracts from that taxes and typical consumption expected for the rich, to get the quantity that enters into the growth of wealth and the form of the tail of the steady-state distribution. I doubt that Piketty would dispute this obvious point, implicit in some of his arguments, but it is true that most of his presentation mushes this up. (See also.)
Giles in the Financial Times has claimed a variety of data- handling errors, of which one would be consequential. Giles claims that wealth inequality hasn't grown recently in Britain the way that Piketty claims. Piketty has responded that in fact he has used more reliable and consistent data sources than Giles, and that his results are backed by independent analyses. Notice, however, that this dispute concerns only a detail on a detail- whether wealth inequality has already started to increase in one country. The central point, that wealth is already extremely unequal and that the importance of wealth is growing relative to wage income, is entirely untouched by this dispute, as others have noted.
I'm a bit worried about the relatively scanty attention Piketty pays to the growth rate (he guesses it will run about 1.5%) and the natural environment. Given our current effects on the climate, on water supplies, and so forth, it's by no means clear that a full accounting of our current growth would give a positive number. In other words, we're creating an environmental deficit, negative wealth, at a growing rate. Piketty is aware of these issues, which inform his choice of a fairly low expected growth rate, but I wonder if our uncertainty of the magnitude of them might swamp all other issues.
Analytical follow-ups to Piketty's work might focus on several key facts that appear empirically in Piketty's work.
Why does the net public wealth stay near zero, except in post-revolutionary societies, which have not been great economic successes?
Why is private wealth always so unequally distributed? (The simple models tend to describe the high-end tail more than the bulk of the population.)
How does ROI stay high even when there's lots of capital available?
What becomes of all this if environmental factors give a net negative growth?
The answers may in part lie in some fairly depressing behavioral economics. I suspect that the arguments of Robert Frank about how competitive arms races drive luxury consumption and suck resources into that zero-sum or negative-sum sector are relevant. It is interesting that Frank's suggested remedies (taxing consumption, not income) would probably  worsen the problems Piketty addresses. Some sort of unified approach to these problems is needed.

Originally posted to docmidwest on Sat May 31, 2014 at

VAM, bam, thank you ma'am,,,,

Yet more info on the ridiculous nature of value added. What will it take to get rid of this crap?

Consumer Alert: Researchers in New Study Find “Surprisingly Weak” Correlations among VAMs and Other Teacher Quality Indicators

Posted on June 1, 2014 by Audrey Amrein-Beardsley

inShare0

Two weeks ago, an article in the U.S. News and World Report (as well as similar articles in Education Week and linked to from the homepage of the American Educational Research Association [AERA]) highlighted the results of a recent research conducted by University of Southern California’s Morgan Polikoff and University of Pennsylvania’s Andrew Porter. The research article was released online here, in the AERA-based, peer-reviewed, and highly esteemed journal: Education Evaluation and Policy Analysis.

As per the study’s abstract, the researchers found (to which their peer-reviewers apparently agreed) that the extent to which teachers’ instructional alignment was associated with their contributions to student learning and their effectiveness on VAMs, using data from the Bill & Melinda Gates Foundation’s Measures of Effective Teaching (MET) study, were “surprisingly weak,” or as per the aforementioned U.S. News and World Report article, “weak to nonexistent.”

Researchers, specifically, analyzed the (co)relationships among VAM estimates and observational data, student survey data, and other data pertaining to whether teachers aligned their instruction with state standards. They did this using data taken from 327 fourth and eighth grade math and English teachers in six school districts, again as derived via the aforementioned MET study.

Researchers concluded that “there were few if any correlations that large [i.e., greater than r = 0.3] between any of the indicators of pedagogical quality and the VAM scores. Nor were there many correlations of that magnitude in the main MET study. Simply put, the correlations of value-added with observational measures of pedagogical quality, student survey measures, and instructional alignment were small” (Polikoff & Porter, 2014, p. 13).

Interestingly enough, the research I recently conducted with my current doctoral student (see Paufler & Amrein-Beardsley, here), was used to supplement these researchers’ findings. In Education Week, the articles’ author, Holly Yettick, wrote the following:

In addition to raising questions about the sometimes weak correlations between value-added assessments and other teacher-evaluation methods, researchers continue to assess how the models are created, interpreted, and used.

In a study that appears in the current issue of the American Educational Research Journal, Noelle A. Paufler and Audrey Amrein-Beardsley, a doctoral candidate and an associate professor at Arizona State University, respectively, conclude that elementary school students are not randomly distributed into classrooms. That finding is significant because random distribution of students is a technical assumption that underlies some value-added models.

Even when value-added models do account for nonrandom classroom assignment, they typically fail to consider behavior, personality, and other factors that profoundly influenced the classroom-assignment decisions of the 378 Arizona principals surveyed. That, too, can bias value-added results.

Another successful push against VAM.

Tennessee Tied Teachers' Jobs to Standardized Test Scores. Here's How They Pushed Back—And Won.

The teachers found their careers at risk when an erratic statistical tool became a key measure of their success.                 

by Molly Rusk

posted May 30, 2014

Teachers pack the room at an education committee meeting in March 2014. Photo courtesy of the Tennessee Education Association.

What if a surgeon’s medical license could be taken away based on an error-prone statistical formula that ranked his abilities on a scale of 1 to 5, based on the success (or failure) of a small number of the operations he performed? Or imagine if a lawyer could lose her membership to the bar because a statistical estimate of her success predicted that she would lose the majority of her cases next year.

When teachers realized that these erratic scores could put their careers on the line, they started getting worried.

Last year, public school teachers in Tennessee faced precisely that situation, but they didn’t take it lying down. Instead, they started a year of creative actions that led to a decisive change in policy—despite a governor determined to keep an unreliable statistical formula as a key method of evaluating teachers.
Their campaign ended successfully on April 24, when Governor Bill Haslam signed a bill rolling back the use of a statistical instrument known as TVAAS in teacher licensing decisions—and hitting the pause button on an important facet of the testing trend in Tennessee, at least for the moment.
Education experts are divided as to what this development will mean for America’s schoolkids. But many believe that it could spark similar campaigns nationwide.
“The change in Tennessee sends a message about politics,” said Dan Goldhaber, director of the Center for Education Data and Research at the University of Washington. “It will embolden people in other states who think that tests ought not to be used for teacher evaluations to continue the pushback.”
According to Bob Peterson, president of the Milwaukee teachers’ union, the development in Tennessee is just one piece of the puzzle. “The success of the pushback in Tennessee is one part of the larger growing movement for testing reform, against the use of standardized tests to pigeonhole and sort our students, and to scapegoat our public schools and teachers,” Peterson said. “New York, California, Oregon—there’s growing grassroots activity.”

From "chickens and cows" to public education

The story begins in the early 1990s, when the state of Tennessee hired Dr. Bill Sanders, then a statistician at the University of Tennessee’s program in agriculture, to develop a statistical tool that could measure how well teachers were doing their jobs. So Sanders created the Tennessee Value Added Assessment System.
Also known as TVAAS, the state adopted the system in 1993, and school district administrators and board members used it as a diagnostic of how schools and teachers were performing.
While Sanders was a perfectly good statistician, some have cracked wise about his background. “His initial forays into statistical modeling were based on livestock—chickens and cows,” said Jim Wrye, manager of government relations for the Tennessee Education Association, the largest professional association of teachers in the state.

More than 400 teachers  went to Nashville during spring break to personally talk to legislators.

In the context of increasing federal pressure to emphasize testing in public schools, TVAAS was seen as a limited tool. It gathered information about how a given teacher’s students were doing on the state’s standardized test and spat out a number from 1 to 5, with 5 being best. It showed in a general way how well teachers were preparing students for the test. Soon other states began adopting systems based on TVAAS.
But then, beginning in 2010, Tennessee made several policy decisions that changed the role TVAAS played in teacher’s lives.
The reason for the change was a federal initiative called Race to the Top, introduced in 2010, which put states in competition for education funding. The states that came up with the best education reform plans—defined by a set of goals laid out by the federal government—would win a large public education grant. Among the goals of Race to the Top was teacher accountability. To meet that goal, United States Secretary of Education Arne Duncan encouraged school districts to use student test scores to decide whether teachers could receive or renew their teaching licenses, among other things.
In order to win Race to the Top funding—the state was awarded more than $501 million—Tennessee turned TVAAS scores into a major component of teacher evaluations. While 51 percent of a teacher’s total score still came from an observation—in which a school district official would visit a school and watch a teacher in action—TVAAS scores were now worth 35 percent.
In August 2013, the state Board of Education voted to adopt a policy, introduced by State Education Commissioner Kevin Huffman, which placed even more weight on TVAAS scores. Under this new policy, if a teacher scored lower than 2 out of 5 points for three years in a row, she would not be eligible to renew her license.
Before that decision, TVAAS scores didn’t cause much trouble because they were only a diagnostic, with no real consequences. But when teachers realized that these erratic scores could put their careers on the line, they started getting worried.
“The most valuable possession of a teacher is her license,” Wrye said. “A license allows her to do what she loves. It’s worth more than her house, it’s worth more than her car. And when you decide to take it away, you better have a rock-solid reason.”
Teachers saw TVAAS as anything but rock-solid. Like other statistical measurements, it has what’s known as a “standard error.” In other words, a teacher could receive a score of 5 for a given school year but actually deserve only a 3, according to other components of her evaluation. By the same token, a teacher could receive 3 points for a given year, but actually deserve a 5. Errors of this nature are built into statistics, but the likelihood that this would occur was relatively high for TVAAS.
The system was so unpredictable, said Wrye, "It was sometimes viewed as the crazy aunt in the basement."
Wrye also said the system unfairly punished teachers who work in the neediest communities. "Places with high poverty, places with low resources, places that have a high influx of English language learners—there are places where, no matter what the teacher does, they will not have very good test scores or TVAAS scores."
Last year, complaints about TVAAS began trickling in to the TEA. As the association investigated individual cases, they discovered how the system’s standard error, among other kinks, was affecting teachers, according to TEA President Gera Summerford. In one instance, eighth-grade teacher Mark Taylor was denied a bonus because of poor scores, even though the scores only reflected the work of about 16 percent of his students.
These kinds of complaints made it clear that the August 2013 decision—set to take effect in 2015—could cost a teacher her license and perhaps her whole career because of an erratic and often inaccurate statistical calculation. So the TEA launched a campaign to urge the Tennessee legislature to reverse the policy and ensure fairness in teacher evaluations.

Educators from Sequatchie County talking to Rep. Ron Travis. Photo courtesy of the Tennessee Education Association.

 

How they did it

The TEA started by doing what teachers do best: educating. “What we ended up doing is, first and foremost, explaining to the legislature what exactly TVAAS was,” said Jim Wrye, who was among the TEA leaders who spearheaded the campaign.
On January 23, two TEA allies, Senator Mike Bell and House Rep. Mathew Hill, presented an initial bill, the Educator Respect and Accountability Act of 2014, that would completely remove student standardized test scores from teacher evaluations.

The campaign highlights the need for testing data to be used more carefully.

But because of pressure from a minority of legislators who supported the existing policy, the bill was modified to focus specifically on TVAAS. It amended the law so that the state department of education could not revoke a teacher’s license based on her TVAAS scores. It also took away the state Board of Education’s authority to adopt policies on teacher licensing.
Meanwhile, on January 31, the state Board of Education met to finalize new rules for teachers’ licenses. During that meeting, they rescinded the policy they had approved in August 2013.
But that reversal was only the beginning. The modified bill still had to survive a gauntlet of committees, subcommittees, and hearings.
On February 4, Jim Wrye and TEA General Counsel Rick Colbert gave a 15 minute presentation before the state’s House Education Committee, laying out the major problems with TVAAS. Other TEA leaders gave a similar presentation for the Senate Education Committee.

The TEA also created a petition asking the governor to treat teachers as professionals.

Once members of the legislature understood that TVAAS was not a state standardized test but rather an unreliable statistical estimate derived from standardized test scores, the bill gained widespread support. Within two weeks, 88 out of 99 members of the Tennessee legislature, including both House representatives and senators, signed on as co-sponsors.
Despite the growing support for a policy change in the legislature and in the Board of Education, the governor was determined to keep TVAAS as a core component of teacher evaluations. Tennessee Education Commissioner Kevin Huffman was the leading proponent of this policy, and Governor Haslam personally lobbied senators, urging them to support the commissioner’s position.
“One senator said he had spent over 10 years in the general assembly and [had] never been called to the governor’s office on a particular bill, before this one,” Wrye said. “He still voted for us in committee.”
The TEA leaders didn’t do all the talking. The association organized more than 400 teachers who went to Nashville during spring break to personally talk to legislators about TVAAS and other public education issues like charter schools and vouchers.
The TEA also created a petition asking the governor to treat teachers as professionals. After nearly 12,000 people signed the petition, the TEA delivered it to Governor Haslam’s office with comments outlining the key reasons that the TEA’s licensure bill had gained such widespread support in the legislature.
The final twist of the screw came in the form of two lawsuits that the TEA filed against Governor Haslam and Commissioner Huffman. The lawsuits were based on separate complaints that two teachers—one of whom was eighth-grade teacher Mark Taylor, mentioned above—had brought to the association about TVAAS. In both cases, the teachers lost bonuses they believed they deserved, due to poor TVAAS scores that did not represent the full extent of their work.
TEA President Summerford stressed that the lawsuits did not address teacher licensure rules, the issue at the heart of the association’s licensure bill.
However, they “bring into sharp focus ... the fundamental flaws of TVAAS in making high stakes decisions for teachers,” Wrye said.
In the end, there were only six “no” votes on the TEA’s licensure bill in the Tennessee legislature, out of a total of 132 voting members. And when Governor Haslam signed the bill on April 24, teachers across the state celebrated the development as a major policy victory.
“It showed tremendous care and support by the rank and file senators and House members for their teachers,” said Wrye.
Federal pressure to emphasize testing in public schools will continue to affect education policy in Tennessee and in the rest of the country. But the TEA’s campaign highlights the need for testing data to be used more carefully and demonstrates the power of teachers to act effectively when it is not.

---

Molly Rusk wrote this article for YES! Magazine, a national, nonprofit media organization that fuses powerful ideas with practical actions. Molly is a recent graduate of the program in Creative Writing at the University of Washington and an online reporting intern at YES! Follow her on Twitter @mollylynnrusk