Teacher Rating Scales: Promises and Pitfalls — A Review from #NAGC25

One of the most eye-opening sessions I attended at NAGC25 was Teacher Rating Scales: Promises and Pitfalls presented by Dr. D. Betsy McCoach (Fordham University) and Dr. Karen Rambo-Hernandez (Texas A&M University). Their research, funded through multiple U.S. Department of Education Javits Grants, dives into a tool many of us rely on in gifted identification: teacher rating scales (TRS). And what they revealed was extremely important for gifted coordinators to know.

Big Questions

The presenters opened by discussing tools like the Hope Scale, which includes both academic and social-emotional indicators. They challenged us to think more deeply: If a student demonstrates a particular indicator, what program or service does the indicator actually suggest? What curriculum would meet that student’s needs? In other words, the scale should connect to something concrete—not sit in isolation.

The Teacher Effect: A Huge Source of Variance

One of the most striking findings of their research was the extent to which teachers themselves impact students’ scores.

We all know teachers vary in generosity- some rate more students highly, while others are more conservative. However, the data brought this to life in a sobering way.

In a West Virginia Javits study led by Dr. Rambo-Hernandez, teachers received either live half-day training or asynchronous modules on how to use rating scales. And yet, even with training of differing levels and types, large disparities remained. Teachers simply used the tool differently.

Dr. McCoach described this as Conditional Between-Teacher (Within-School) Variance: even after controlling for student ability and achievement (using the Cognitive Abilities Test- CogAT, the Measures of Academic Progress- MAP, and TRS data), differences between teachers persisted. In fact:

• Ability and achievement barely predicted how students scored on teacher rating scales.
• Teachers themselves accounted for one-third to one-half of the variance in rating scale results.

In plain terms? A student’s score reflected the teacher more than the student.
As someone who has served as both a teacher and a coordinator, I felt that deeply.

How Many Gifted Students Are We Missing?

The most startling finding came from their comparison of cognitive scores with TRS results:

Among students scoring in the top 10% on the CogAT in their district, only 32.4% of them also scored in the top 10% on the TRS.

Let that sink in… we’re missing almost 70% of high-ability students when TRS are used as a gatekeeper.

Even when widening the TRS criteria:

• Top 10% CogAT + top 20% TRS → only a 56.2% overlap
• Top 10% CogAT + top 25% TRS → only a 63.6% overlap
• Top 10% CogAT + top 30% TRS → only a 70.3% overlap

Even in the most generous scenario, we are still missing 30% of students. For anyone committed to equity in gifted education, this is unacceptable.

So What Do We Do Instead?

Thankfully, the session didn’t stop at the problem. Drs. McCoach and Rambo-Hernandez pointed us toward thoughtful, practical solutions.

1. Include Teacher Voice- Just Not as a Gatekeeper

Dr. Joe Renzulli’s “classic” suggestion still holds: Gather your talent pool using multiple measures (ability, achievement, talent-development products, etc.), then ask teachers: “Who is missing?” This reframes teachers as informants, not filters.

2. Provide Annual Rater Training

If you must use TRS:

• Train teachers every year—even a 10-minute refresher matters.
• Clarify what each number means (“What is a 1? What is a 2?”).
• Build a shared frame of reference to improve inter-rater reliability.

This isn’t about perfection. It’s about tuning the instrument.

3. Advocate for Better Policy

If you have influence: push to use TRS as one of multiple data points, like in Virginia (no one score can get a student into a gifted program nor keep a student out of a gifted program). If using TRS, make sure to:

• Train annually (see discussion above)
• Review results for bias
• Ensure no student is denied access to services because of a rating scale alone

The presenters were crystal clear about this point: Do NOT use teacher rating scales as universal screeners. They simply aren’t reliable enough, and we could miss too many students.

Final Thoughts

As a gifted coordinator, I walked away from this session with both urgency and clarity. Teacher rating scales can still play a role in our identification systems, but only when used carefully, responsibly, and alongside multiple measures.

If we truly believe in talent development for all students, then we must question the tools that inadvertently hold so many of them back. This session was a firm reminder that identification is not neutral, and that educators have a responsibility to keep refining our systems for fairness, access, and accuracy.

If you attended this session at #NAGC25 too, I’d love to hear your takeaways in the comments below. Even if you didn’t, but you’re working on improving your division’s identification practices, let’s connect! I love helping schools rethink systems to better spot potential. The more educators can work together to solve identification issues, the better off students will be! ~Ann