Head Start monitoring: what’s different about CLASS in FY2019?
The OHS CLASS Field Guide for FY 2019 was released in September. Although much of the Field Guide remains the same, there are two significant differences in how CLASS observations will be conducted in FY2019 v. FY2018. First, although the observations will still be conducted between the second and third years of the five-year grant cycle, they will not be conducted at the same time as the Focus Area Two monitoring event. Second, OHS will now assign two CLASS observers to each CLASS review instead of one. Even though there will be two observers onsite, each classroom that is selected for observation will still be reviewed by a single CLASS observer who will conduct both observation cycles. The OHS Field Guide provides the following example:
For X grantee with a sample size of 9 classrooms, Reviewer A would be assigned to observe 5 classrooms and Reviewer B would be assigned to observe the remaining 4 classrooms.
What prompted the changes?
The changes were likely prompted by OHS’s attempt to make CLASS reviews more palatable to the Head Start community. CLASS reviews have been controversial because, among other things, there are concerns about the reliability of CLASS. Consider this Teachstone blog post, for example, acknowledging that CLASS isn’t reliable. As the blog post points out, reliability of the CLASS tool hinges on the reliability of CLASS observers. In other words, if observers aren’t reliably coding observations, the results won’t be reliable.
So, how do we ensure that CLASS observers are reliable? Teachstone recommends that when CLASS is used for accountability or high-stakes testing (and DRS would certainly qualify!), CLASS observers should have additional training to ensure reliability in coding. Some suggestions include live double-coding with experts, video double-coding, and calibration (where observers code a sample classroom and receive feedback from a Teachstone master coder) every 2-4 weeks or 10-15 observations.
In the FY2019 Field Guide, OHS says that each CLASS reviewer must be certified annually and must participate in dual coding, but OHS does not specify how often.
Why is reliability important?
CLASS scores are used to trigger Head Start programs for recompetition in two ways. First, programs may be designated for recompetition if their CLASS scores fall below certain minimum thresholds (4 for Emotional Support, 3 for Classroom Organization and 2 for Instructional Support). Second, programs are required to recompete if they have an average CLASS score that is in the lowest 10 percent on any of the three CLASS domains.
Since CLASS scores are used to determine whether programs will have to recompete, it’s important that the scores be reliable. In fact, when Congress mandated that OHS use an observational tool like CLASS to monitor Head Start programs, Congress required that the tool be “valid and reliable.” 42 U.S.C. § 9836A(c)(2)(F). Congress also required that the designation renewal system more generally be “fair, consistent and transparent” and that designations be renewed “in a timely manner.” OHS must periodically evaluate whether DRS is being applied “in a manner that is transparent, reliable and valid.” 42 U.S.C. § 9836(c)(8).
There are serious questions as to whether using CLASS scores to trigger recompetition, particularly using CLASS scores that fall in the lowest 10 percent nationwide, meets this standard. In fact, in 2016 a bipartisan group of 19 Senators sent a letter to OHS criticizing the requirement for recompetition for grantees with CLASS scores in the lowest ten percent. The Senators argued, among other things, that this particular trigger created a moving target, resulted in significant delay and failed to identify low-performing programs. As a result, the Senators argued that the CLASS trigger failed to meet the requirements in the Head Start Act that Head Start monitoring and DRS be timely, transparent, valid and reliable.
In December 2017, OHS invited public comment on the CLASS trigger and, specifically, whether OHS should (1) remove the ‘‘lowest 10 percent’’ trigger, (2) increase the minimum thresholds for the Emotional Support and Classroom Organization domains, and (3) eliminate the minimum threshold for the Instructional Support domain
and authorize the Secretary to establish a minimum threshold each year. In February 2018, OHS invited a second round of public comment, this time on the CLASS trigger, other DRS triggers and the DRS process more broadly. Comments to this second round were due on March 16, 2018, and submitted comments can be found here.
As of October 2018, no changes have been made to how CLASS is used to trigger recompetition.
What should you do to protect your program?
Deviations, however small, from the procedure set forth in the Field Guide have the potential to significantly affect CLASS scores. As a result, we recommend taking detailed notes during your CLASS observation so that you can identify and make a record of any discrepancies. We’ve created this CLASS worksheet to assist you in that process.
If you are triggered for DRS as a result of CLASS scores and you are interested in learning more about your legal options, please contact us for a free consultation.