Math Monitoring Report Comments - Part 2

This is the second of four parts of my comments on the Math Monitoring Report.  This post highlights the most egregious data error and resulting conclusion.  In comparing Greenwich to other districts (called DRG A and DRG B), the report uses incorrect data and concludes that the results have “shown continuous improvement over the five year period” and as “establishing five year highs in 2011 in the percentage of students at Advanced, Goal…”  This is not the case for our elementary school results.  The report needs to be fixed, and the conclusions redrawn to recognize the drop in performance.  Only then can we start addressing the problems.


____________________________________________________________________________

DATA REVIEW – Elementary and Middle School

Strand Results by Grade 3-5 (pages 7-11)

1.    It is interesting to note the performance on specific strands which might provide positive or negative input to the question of the quality of Everyday Mathematics.  Strand 6 (basic facts) shows consistent performance for grades 4 and 5, but declining performance for grade 3.  Strand 7 (Computation with Whole Numbers and Decimals) shows declining performance as classes move from third to fourth to fifth grades. 

2.    Scores for Strand 8 (Computation with Fractions and Integers) show consistently high scores for grade 5.  This needs to be reconciled to the comments made by middle school teachers (see page 34). 

3.    Strand 11(Estimating Solutions to Problems) and Strand 15 (Approximating Numbers) have some of the lowest scores across all three grades.  One of my concerns with Everyday Math is that it spends so much time on estimating, at the expense (my opinion) of the basic facts and processes.  Apparently even this amount of time is not providing a good knowledge of the estimation process.  Maybe without the basic facts and processes, estimating has no context.

4.    Everyday Math professes to help students apply and explain what they have learned.  Strand 25 (Mathematical Applications) CMT items should measure this skill.  “These items are designed to assess integrated understanding of key mathematical ideas as well as student’s ability to communicate their understanding and demonstrate their reasoning.”   Given the scores in grades 3 and 4 for this strand, Everyday Math appears to be doing as good a job as in the estimation strands.

Elementary DRG Comparisons (page 12)

1.    Chart 7 repeats the same error noted above under Accomplishments/Highlights.  Greenwich 3-5 Elementary Advanced and Goal percentages should coincide with Table 1, but use data from Table 2 (page 38).  The quoted percentages (Advanced – 49.2% and Goal – 81.8%) appear to refer to grades 3-8, not 3-5 (see Table 2).

2.    Given the inconsistencies found between the tables, charts and text, I reverted to source data to reconstruct the figures.  Under the theory that Chart 7 and Table 1 both actually refer to grades 3-5 (as per their title or content), I calculated a pupil weighted average of the percentages for Advanced and Goal percentages across grades 3-5.  My results, compared to Table 1 and Chart 7, appear below.

ADVANCED
	2006-07	2007-08	2008-09	2009-10	2010-11
TABLE 1	42.5	42.8	45.8	44.6	44.9	page 37
CHART 7	42.5	45.0	45.9	45.0	49.2	page 12
My calc.	42.5	43.4	45.9	45.0	44.9	#
GOAL
	2006-07	2007-08	2008-09	2009-10	2010-11
TABLE 1	81.0	77.9	79.8	80.8	80.4	page 37
CHART 7	81.1	78.7	79.7	81.1	81.8	page 12
My  calc.	81.1	78.5	79.7	81.0	80.4	#
# - based on data from state data base Foot note from download: This report does not include ELL-exempt students.  Report name: Performance Level Summary.

3.    While Proficient percentages have been increasing (a satisfying but meaningless result), characterizing the results as having “shown continuous improvement over the five year period” and as “establishing five year highs in 2011 in the percentage of students at Advanced, Goal…” is not accurate.

4.    Placing the Greenwich results in contrast to the DRG A and B results (which I did not verify) paints a much bleaker picture than portrayed in Chart 7and in the accompanying text.  While DRG A and B students achieved gains in Advanced percentages of 9.3% and 6.9%, respectively, Greenwich students gained only 2.4% over five years (using my calculations, which agree in this case to Table 1).  For Goal percentages, DRG A and B students achieved gains of 4.0% and 3.8%, respectively, Greenwich students actually dropped 0.7% over five years (again, consistent with Table 1, except for slight difference in starting point).

CMT4 Results – District (Vertical Scale)

1.    The results from this analysis are encouraging (a possible indication that we could be starting to close the sizable gap to DRG A and B).  It would be helpful if the targets for the year of the quoted gains (2010-2011) are also reported, so that the reader can judge success or failure.