Policymakers Need to Consider Validity of PreK Observation Systems

USA - English

• Latin America - español

PRINCETON, N.J., April 8, 2014 /PRNewswire-USNewswire/ -- To provide early childhood educators, policymakers and school leaders with information on the reliability and validity challenges that accompany efforts to track and improve the quality of early education programs, Educational Testing Service (ETS) has released a report that examines PreK classroom observation policies across the country.

The report, State-Funded PreK Policies on External Classroom Observations: Issues and Status, was written by Debra J. Ackerman of ETS's Early Childhood Research & Assessment Center. It is the second in a series of early childhood education policy reports that explore issues related to the use of assessment data in programs serving preschool-age children.

"Since early education programs are increasingly being promoted as an integral part of state and federal efforts to ensure that all children enter school ready to learn, it is important to make sure that the efforts to monitor them provide the appropriate and necessary information," says Michael T. Nettles, Senior Vice President of ETS's Policy Evaluation and Research Center.  "Ackerman's report addresses this issue by focusing on the policies of documenting student learning experiences through classroom observation data."

As Ackerman explains in the report, the trend of measuring teacher quality for high-stakes purposes is growing.

"It is only a matter of time before PreK observation score data begins to be used in a greater number of consequential early childhood education decisions that will affect PreK students, teachers and programs," says Ackerman. "That is why it is vital for stakeholders and policymakers to know that they can rely on the quality of the data that is collected."

To help explain the importance of this issue, the report focuses on the three key factors that contribute to the validity and reliability levels of classroom observation score data. These factors include:

• the observation protocol(s) being used;
• the capacity of observers to generate reliable score data; and
• the frequency with which observation data will be collected from classrooms.

Ackerman's report also provides a detailed analysis and description of classroom observation policies for 27 state-funded PreK programs from the 2012–2013 school year. The data for the descriptions were gathered from a survey Ackerman sent to the administrators of 53 PreK programs that were identified in the National Institute for Early Education Research's 2011 Preschool Yearbook. The research questions that the survey aimed to address were:

1. Which PreK programmatic decisions are informed by external observation score data?
2. Which observation protocols are to be used to generate score data?
3. What affiliation, if any, do observers have with the PreK teachers being observed?
4. What training and ongoing reliability supports do observers receive?
5. How frequently are observations to be conducted in any PreK classroom?

"Collecting valid monitoring data will likely remain a high priority for both state and federal government officials as access to early education programs continue to expand over the next few years," adds Ackerman. "We therefore must find ways to meet and overcome some of the unique challenges that exist for generating reliable classroom observation score data as part of the monitoring processes."

Ackerman further explains that given the less-than-robust early childhood literature base that exists regarding potential validity and reliability issues related to policies on classroom observation protocols, observer capacity and frequency of observation data collection, the time may be right for early education stakeholders to include such topics in their best-practice agendas.

"Ackerman's report should prove to be very valuable to users of early childhood classroom observation data, especially those whose decisions have critical consequences about the future of early childhood education programs and can affect the quality of education provided to children," Nettles concludes.

If you wish to receive a copy of State-Funded PreK Policies on External Classroom Observations: Issues and Status, please contact ETS Policy Research Center Administrative Director Jonathan Rochkind at [email protected], or visit www.ets.org/research.

The first report in the series is State Pre-K Assessment Policies: Issues and Status, and a digital copy of the report can be found here.

Logo - http://photos.prnewswire.com/prnh/20120110/DC33419LOGO

About ETS
At ETS, we advance quality and equity in education for people worldwide by creating assessments based on rigorous research. ETS serves individuals, educational institutions and government agencies by providing customized solutions for teacher certification, English language learning, and elementary, secondary and postsecondary education, and by conducting education research, analysis and policy studies. Founded as a nonprofit in 1947, ETS develops, administers and scores more than 50 million tests annually — including the TOEFL^® and TOEIC^® tests, the GRE^® tests and The Praxis Series^® assessments — in more than 180 countries, at over 9,000 locations worldwide. www.ets.org

SOURCE Educational Testing Service