Sunday, January 15, 2012

Making the MMR Better - A Review

An old boss said to me once, “Bring me solutions, not just issues.”  The administration needs to be advocates for the students, so that issues are understood and the BoE can determine resources (within the budget parameters) to resolve the issues.  The poorly written Management Issues section of the Math Monitoring Report is short changing the students of our schools.

In my last post, I offered a few suggestions about how to make the Math Monitoring Report (“MMR”) better.  Why is this a concern?  The MMR sets the tone for the use of resources within the district.  It also serves (or should serve) as the main source to gain an understanding of the issues and achievements of our students.  If the report is not accurate, or if it downplays failures or overplays successes, we may be ignoring issues or focusing resources in the wrong area.



How Bad Is It?

How well do the recent and previous MMR’s accomplish the task of issue management?  Let’s look at the 2009-10 report first (which now appears on the BoE website in the correct place), then follow through to the draft 2010-11 report. 

The 2009-10 MMR lists six Management Issues:

MI1 - Mathematics Interventions

MI2 - Mathematics Interventions – New Lebanon School

MI3 - Adequate Yearly Progress (AYP) – Secondary

MI4 - Middle School Instructional Time

MI5 - Increasing the Percentage of Middle School Students Reaching Goal or Higher on the CMT

MI6 - Increasing the Percentage of Students Reaching Goal or Higher on the CAPT

The first issue with the above issues, is that some of them are not issues.  The last two really are targets.  The discussion talks about how the Middle School and High School Math Departments will reach the targets.  Neither identifies an issue which needs to be addressed with additional resources, or through some other means.

What’s an issue? 

An issue is a failure to meet a target, or the likelihood that a failure will occur without additional action.   Are targets good to have?  Absolutely, but let’s see all the targets in one place and all the issues in a separate place.

For Management Issues 1-4, I count about nine actual issues.  I say about since one section is written so poorly it is difficult to determine the true meaning.  The issues are:

1.    MI1 - Budget constraint (reduced coaching) may make it so 2009-10 growth will not be duplicated

2.    MI1 - A gap of some sort, probably to the DRG B schools (this is the confusing one)

3.    MI1 - A gap to DRG A schools

4.    MI2 - Lack of sustained growth in Math CMT scores at New Lebanon

5.    MI3 - Not meeting AYP in two subgroups at the High School

6.    MI3 - Not meeting AYP in two subgroups at Western Middle School

7.    MI3 - Struggle to maintain heterogeneously grouped on level classes with the remedial needs of certain subgroups (related to the two preceding items)

8.    MI4 - Insufficient instructional time for Connected Math at the middle schools

9.    MI4 - Difference in instructional time available for Algebra I and Geometry between the middle schools and the High School.

Looking at the 2010-2011 report, you would expect to see some reference to these issues, and whether they had been resolved or were on-going. 

1.    Budget constraint – no specific mention of the impact of the constraint, although there is a discussion of Hamilton Avenue’s 2009-10 success and of North Street’s 2010-11 success due, in part, to coaching. So did the constraint have an impact?  Can’t tell, but this is the perfect place to discuss the continued drop in Dundee’s percentages (Advanced and Goal hit five year lows), and the drop in Hamilton Avenue’s 2010-11 percentages.

2.    A gap of some sort, probably to the DRG B schools – there is some discussion, with similar wording as in 2009-10, about the gap to DRG B schools, and how to close it, with an update on the gap size after the four year trend.

3.    A gap to DRG A schools – a virtual repeat of the wording from 2009-10, with an update on the gap size after the four year trend.

4.    Lack of sustained growth in Math CMT scores at New Lebanon – no mention in the Management Issues, even though the percentages were up, but it was noted in the Executive Summary.

5.    Not meeting AYP in two subgroups at the High School – update provided indicates that the two subgroups still did not make AYP, but the update failed to highlight the fact that the while the Proficient target level went up, the actual results for the subgroups declined.

6.    Not meeting AYP in two subgroups at Western Middle School – update provided that Western had earned Safe Harbor status, but no indication whether the issue is now resolved.

7.    Struggle to maintain heterogeneously grouped on-level classes while trying to meet the remedial needs of certain subgroups (related to point 6) – repeated the issue, with the same verbiage.

8.    Insufficient instructional time for Connected Math at the middle schools – repeated the issue, with the same verbiage.

9.    Difference in instructional time available for Algebra I and Geometry between the middle schools and the High School – repeated the issue, with the same verbiage.

The last two issues were specifically addressed during the BoE Work Session, and the administration is reviewing alternatives to resolve the issues.  From conversation around the table, it appears the issue has been raised for about six years.

Of the nine issues, two (1,4) were not followed up, five (2,3,7,8,9) had updates which repeated the issue with minor changes, and two (5,6) had updates which were less than informative.  No “issue closed.” No “on-going issue.”  No meaningful comparison of progress year-to-year, where appropriate.

For 2010-11, the following additional issues were identified:

1.    Central Middle School not making AYP

2.    Middle schools not likely to hit two targets for 2012-13 (are there any targets for 2011-12?)

3.    Grade six teachers’ concerns regarding grade six students’ preparation for middle school math.

4.    The existing K-8 math material will need to be supplemented in order to meet the change to the Common Core State Standards.

5.    The full impact that the new computer-based assessment system will have on our computer infrastructure is unknown.

Again, the High School “issue” was a restatement of a target.  Credit to those who put together the new middle school points; they seem to get the meaning of “issue.”

How to Make The MMR’s Better

1.    Fix the data

2.    Analyze the corrected data to arrive at meaningful observations

3.    Determine if the observations indicate an issue

4.    State the issue clearly

5.    State the remedial action to be taken, the required resources, and the expected results and time frame

6.    Follow up on the issue – a yearly update should be a minimum until the issue is resolved or the remedial actions are judge to be ineffective

Next up, my summary of the Math Monitoring Report.

No comments:

Post a Comment