Usefulness of Value-Added Models for Gauging Teacher Effectiveness Questioned

USA - English

• Latin America - español

PRINCETON, N.J., Sept. 24, 2013 /PRNewswire-USNewswire/ -- In the midst of the ongoing debate on how to best measure teacher effectiveness, Educational Testing Service (ETS) announced the release today of a presentation given by Stanford University Professor Emeritus Edward Haertel as the 14th William H. Angoff Memorial Lecture. Haertel's lecture, Reliability and Validity of Inferences about Teachers Based on Student Test Scores discusses the use of Value-Added Models (VAMs) as a major part of determining teacher effectiveness and analyzes why their usefulness for doing so has been seriously overstated.

(Logo: http://photos.prnewswire.com/prnh/20120110/DC33419LOGO)

Haertel's lecture drew upon detailed analyses of VAM characteristics — including the exploration of its validity, reliability, predictive powers and potential effects in various situations — and leads to the conclusion that they are not well suited for the job.

"Teacher VAM scores should absolutely not be included as a substantial factor with a fixed weight in consequential teacher personnel decisions," says Haertel. "The information that they provide is simply not good enough to use as a major component for gauging a teacher's effectiveness."

Haertel's presentation and the published lecture released today elaborate on this conclusion by discussing how VAMs, which are essentially a method of evaluating teachers by comparing their students' test scores in a given school year with the scores from a previous one, cannot sufficiently isolate the various factors that influence a student's education to a high enough degree to make VAMs useful. Some of the factors that can cause the discrepancies include a student's out-of-school experiences, individual aptitudes, peer influences, previous education or school-specific academic climates.

"With the wealth of research that has been conducted over the years on the validity and usefulness of Value-Added Models in high-stakes teacher evaluations, Professor Haertel's conclusion that no statistical manipulation of the data can result in an accurate and unbiased comparison of teachers working under extremely different conditions has strong empirical support," says Richard Coley, ETS's Executive Director of the Center for Research on Human Capital and Education.

Despite the limited usefulness that VAMs have when used as a major factor in measuring teacher effectiveness, there are a number of instances in which both students and teachers can benefit from VAMs, according to Haertel.

"VAMs may have a modest place in teacher evaluation systems, but only as an adjunct to other information used in a context where teachers and principals have genuine autonomy in their decisions about using and interpreting teacher effectiveness estimates in local contexts," says Haertel.

Haertel's original presentation and published lecture paper also discuss a number of ways to make VAMs moderately more accurate or useful, including the use of multiple factors, test scores from multiple years, and the presentation of clear and accurate information about any uncertainty that exists regarding the causes of variances in student performance.

About the William H. Angoff Memorial Lecture Series

The William H. Angoff Memorial Lecture Series was created in 1994 to honor the life and work of William Angoff. For more than 50 years, Angoff made major contributions to the fields of educational and psychological measurement. Aligned with his interests, this lecture series is devoted to relatively nontechnical discussions of important public interest issues related to educational measurement.

About ETS

At nonprofit ETS, we advance quality and equity in education for people worldwide by creating assessments based on rigorous research. ETS serves individuals, educational institutions and government agencies by providing customized solutions for teacher certification, English language learning, and elementary, secondary and post-secondary education, as well as conducting education research, analysis and policy studies. Founded in 1947, ETS develops, administers and scores more than 50 million tests annually — including the TOEFL^® and TOEIC^® tests, the GRE^® tests and The Praxis Series™ assessments — in more than 180 countries, at over 9,000 locations worldwide. www.ets.org

SOURCE Educational Testing Service