Great Question

During the follow up discussion regarding the Proposed Technology Plan 2012-2015, Board of Education member Jennifer Dayton posed a very searching and profound question: What can technology do to bring about an extraordinary rise in student achievement?

The response was minimal, but this question should form the basis for any investment in technology going forward. 

Are we investing for the sake of making some investment?  Or to have the "best stuff" in our schools?  The bottom line is results.  Are we raising student achievement significantly with our spending?

Great question.

Homework

At the recent Board of Education meeting, there was a discussion regarding homework policy, as part of the larger discussion around the monitoring report for Effective Learning Environment (E040).  The topic had been raised before, seemingly in the context of ensuring the consistency of the application of the homework policy across the district.  See specifically pages 6-7 and pages 12-13.

http://www.greenwichschools.org/uploaded/district/Board_of_Education/meeting_materials/2011-12_meetings/5-17-12_meeting/5-17-12_E040_FINAL.pdf

Pages 12-13 deal with the “Prior Years” management issue around homework, specifically, "the District will be examining research on homework and its relationship to student achievement."  The monitoring report references a book (Visible Learning) which brings together 800 studies around student achievement.  I reproduce the summary contained in the monitoring report below.  I have not read the book (it is not in the Greenwich Library collection).

The recommendation is that the district form a committee of teachers, parents and administrators to explore the homework issue.  From my reading, the research indicates support for less/no homework at the elementary school level, based on the lack of a significant impact on achievement. 

One comment from a Board member was that parents use homework as a means to understand what is going on.  This may indicate a lack of communication, or a lack of informative reports cards (there's a surprise), or both.

Personal note: looking back, I don't recall having any significant or regular homework until seventh grade.  I still remember the excitement when we got our first homework assignments.  Boy, did that end quickly. 

This begs the question, in relation to learning such things as basic math facts: If the current direction is that math facts are to be practiced at home, what would happen if there was no more homework?  Would that drilling return to the schools/teachers?  Would it fall by the side of the road completely?  Perhaps ending the ridiculous Everyday Math homework and letting parents drill their kids in the math facts in the time saved would be an effective answer (and one which appears to be supported by the research - see the seventh bullet point regarding "Effects are highest....").  But what about the parents who are working, or don't get the importance?  Will the achievement gap widen?  Tough questions.

____________________________________________________________

Highlights of Research on Homework

Educational researcher John Hattie spent fifteen years reviewing thousands of studies involving millions of students and teachers on the impact of different influences on student achievement. His findings are summarized in the book Visible Learning: A Synthesis of over 800 Meta-Analyses Relating to Achievement. Effect sizes standardize changes in student achievement allowing us to compare the impact of different factors (an effect size of 1.0 equals one standard deviation). The typical effect size across all the studies Hattie reviewed is d =.40, and the author proposes that this is the level where a strategy or practice begins to noticeably impact student achievement. In Hattie’s words, "The effect size of 0.40 sets a level where the effects of innovation enhance achievement in such a way that we can notice real-world differences and this should be the benchmark of such real-world change."(p. 17) The list below presents highlights from Hattie’s meta-analyses on homework.

• The correlation between time spent on homework and achievement is near zero for elementary students.

• Greater effects for older students vs. younger students.

• Effects of homework are twice as large for high school students as for middle school students. Effects are twice as large for middle school students as for elementary school students.

• Greater effects for high ability students vs. low ability students.

• Higher effects when material was not complex or if it was novel.

• Homework involving higher level conceptual thinking and project based was the least effective.

• Effects are highest when homework involves rote learning, practice, or rehearsal of subject matter.

• Research favored short, frequent homework that was closely monitored by teachers.

• Homework does not help students develop time management skills.

• Direct parental instructional involvement in homework showed a negative relationship with achievement, while parental support for independent homework showed a positive relationship.

• Hattie meta-analyses describes a "Zone of Desired Effects" which ranges from

d= 0.40 to 1.2. The effect for homework is d= 0.29

• Negative impacts of homework:

 Can undermine motivation

 Can cause students to internalize incorrect routines and strategies

 Can reinforce less effective study habits

Hattie, J.A.C. (2009).  Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement. New York:

We are Getting There

I was hoping the school district would have posted the video of the Thursday evening Working Session by now, so that I could provide exact quotes.  Since this is good news though, I don’t want to wait any longer, and will rely on my notes to present the sense of the discussion.

The Board of Education has instructed the administration to scope out the work required to conduct a mathematics curriculum review!

Chairman Leslie Moriarty introduced the topic (as a result of an added item on the agenda) by saying she was interested in obtaining a Board consensus on how to move forward with the transition to Common Core Standards for math.

Peter von Braun immediately answered that we should be doing a “full bore” review.

Barbara O’Neill indicated that it would be “short sighted” to focus on Common Core, and that we should be doing a full review.

Jennifer Dayton said that given the date of the last review, a full review was needed. 

Chairman Moriarty pointed out that there were competing priorities, and that the budget process needed to be considered.

Assistant Superintendent for Curriculum, Instruction and Professional Learning Stacey Gross said that the current plan was to work this summer to start the transition by developing augmented materials and enabling professional learning, with a focus on the elementary level.  She continued that the next Math Monitoring Report was due in November.  She indicated that the curriculum review schedule was “very flexible” and that currently the Science implementation was on track for 2012-13 (first year), the Social Studies review was being expanded, and the English/Language Arts review (scheduled to start next fall I believe) could push out.

Dr. Lulow indicated that his interest in this topic (i.e., he asked for the agenda item) was a result of wanting to know where the new Math Coordinator (to be hired) should focus.  He asked if we should expedite the math curriculum review, and if so, he would take that into account in the coordinator selection.

Peter Sherr asked whether the introduction of the new testing for the Common Core Standards in 2014-15 school year meant that we should be implementing changes in the math curriculum in the 2013-14 school year.  He went on to refer to a Wall Street Journal article questioning whether the Common Core Standards were really an improvement.  In any case, he indicated that he supported a “deeper, sooner, more complete” review.

Jennifer Dayton said that we could move forward with a curriculum review, irrespective of the timing of the implementation, and that she was supportive of a start to the review sooner, picking up on Peter Sherr’s phrase.

Adriana Ospina, referring back to the presentation made by the current Math Coordinator at the 4 April 2012 Work Session, indicated that she supported option #3, which would entail a full curriculum review.  She specifically mentioned Everyday Math (and not in a complementary way).

Seeing that at least five of the Board members were in favor of accelerating the review, Chairman Moriarty summarized by saying this would not be a small change, and then asked the administration to give an indication of the scope of work for a curriculum review. 

Barbara O’Neill mentioned that the first step would be the Math Monitoring Report, scheduled for November (see my comments below).  Peter Sherr asked if we could get outside help for the review, to which Dr. Lulow responded that it was not unprecedented.

I may have missed some of the nuances being expressed (and apologies for misquotes), but the general feeling appeared to me that the review should be a full review and that it should start sooner rather than later.  The tone I sensed was concern that the proposals (options #1 or #2 from the previous Work Session) for transitioning to the Common Core using Everyday Math as a foundation was not going to produce the improvements in performance desired. 

Barbara O’Neill’s comment that the first step would be the Math Monitoring Report (“MMR”) is correct, given the policy for such reviews (the  policy is currently under revision).  But let’s look at the timing that this would impose.  If the MMR is published and accepted in November 2012, starting the review, the 12-18 months stipulated as a maximum time frame would put the first possible implementation in September 2014, at the beginning of the school year leading up to the first testing under the Common Core Standards. A twelve month review would allow 9-10 months for preparation and professional learning, but the start would still be September 2014.  This path would also require spending resources (time and money) on planning and executing a transition program, as it would be unwise to put all the marbles into a big bang implementation.

If the Board decides that this is not a feasible path, the immediate start of a review (as soon as the new Math Coordinator is named or hired) would allow a 12 month (preferably shorter) review, with the remaining months (June-August 2013) for professional learning prior to a September 2013 implementation.  This implementation could be limited to grades 2-5, giving all grades almost two years of the new curriculum prior to the first test in May 2015.

Needless to say, I am a huge supporter of an expedited review.  While it would be too late to provide a better curriculum for our daughter, the benefit to younger students needs to considered.  Even one year less of Everyday Math would be a tremendous improvement!

Are We Talking About the Same Thing?

“Students Make Gains In Testing On Science” - The New York Times, 11 May 2012

 “American eighth graders have made modest gains in national science testing, with Hispanic and black students narrowing the gap between them and their white and Asian peers…”

“Low Scores on Science Test Revive Concerns” - The Wall Street Journal, 11 May 2012

“U.S. eighth graders made modest gains on the latest national science exam, but more than two-thirds still lacked a solid grasp of science facts, according to figures released Thursday that renewed concerns American schools are inadequately preparing children for college and the workforce.

The two newspapers looked at the results on the Science portion of the National Assessment of Educational Progress (“NAEP”) from different perspectives. 

http://nces.ed.gov/nationsreportcard/pdf/main2011/2012465.pdf

The NY Times emphasized the increase from an average of 150 out of 300 (in 2009) to 152 out of 300 (in 2011).  The WSJ focused on the fact that 30% of students earning a Proficient rating in 2009 versus 32% earning that level in 2011. 

Both the headlines and the summary paragraphs are correct, but what really matters?  And what does that have to do with math curriculum in Greenwich Public Schools?

Like it or not, life is not lived in a vacuum, without competition.  We compete not just with our neighbors, but with the whole world now.  And if only one third of our students are Proficient in science, do we really think we are going to be competitive with other states, or with other countries?  Connecticut’s results put us right in the middle of the pack of states.

And think about this: only 2% of the students taking the test (nation-wide and in Connecticut) in 2011 scored at the Advanced level.  This is the most likely pool for the engineers and scientists of the future.  The pool seems rather shallow to me.  Neither the WSJ nor the NYT mentioned this result.

A word on performance levels: A Proficient level result on the NAEP “represents solid academic performance. Students reaching this level have demonstrated competency over challenging subject matter.”  Compare this with Proficient on the CMT’s for math or language arts, which means you are below Proficient (i.e., Basic or even Below Basic) when the results are compared to the NAEP, according to studies.  For long time readers, that is why I have not focused on CMT Proficient percentages, although that is what our state focuses on.

So same results, different interpretations, but same problem: poor results compared to our competitors.  Seems like I have heard this before regarding Greenwich’s CMT math results versus other school districts here in Connecticut.

Common Core or Common Excuse?

I spoke at the Board of Education meeting on 25 April 2012.    The remarks (below) are primarily in response to the comments made by the district’s Math Coordinator at the Board of Education Work Session on 4 April 2012. 

Following are my prepared remarks, which I altered slightly when I read them at the meeting.

During the work session on April fourth, the administration presented three alternatives for transitioning to a mathematics curriculum based on Common Core standards. Dr. Lulow indicated that the administration was outlining options in response to “a public concern raised about the quality of the math curriculum and our willingness to make adjustments in the schedule” of a math curriculum review. 

The first two of these options are based on using the current or a Common Core edition of Everyday Math.  The third option allows for the adoption of a different curriculum to begin in fall 2012.  The points in favor of the first two options revolve around the lower cost for professional development and material, and the minimal change required.  I use minimal change to mean, in Mr. Southworth’s words, that “there is a very close, very close alignment” and that there was “not dramatic change coming up.”

Yet elsewhere in the presentation, Mr. Southworth said the Standards are “organized into domains and…. that’s a significant difference in how we have approached mathematics in the past.”  Later he indicated, in regard to which grades were impacted the most, that “the most significant shifts between what we are currently doing and Common Core are occurring at the elementary school.”  He used fractions and numeracy as examples of the extent and nature of the changes required. 

Peter Sherr asked what major things needed to be taught earlier or later, to which Mr. Southworth replied that that was a difficult question and that “we’re really well aligned.”  Yet, from the very same document that the administration used to claim a 92% alignment, the percentage of standards being taught earlier or later in K-5, is 35%.  For the test years (grades 3-5), the percentage is 44%.

The point here is not to highlight inconsistencies, but to show that any way you slice the pie (that’s a reference to fractions), there is a significant change required. 

Still unaddressed, however, is the question of whether Everyday Math is the proper foundation for this transition.  The lack of progress in raising CMT scores, particularly at the Goal and Advanced levels, the widening of the gap relative to other districts, and the slow progress made in closing subgroup gaps in our district, indicate to me that the curriculum is failing our students.  Do we really want to continue with this program under options one and two, in whatever mutated form it might take?

I proposed a fourth alternative.  Begin the curriculum review now.  Begin implementation in fall 2013 for grades two through five, and in fall 2014 for Kindergarten and first grade.  That will ensure that all grades will have almost two full years of an aligned curriculum before the SBAC tests.  You will have the proposed twelve to eighteen months to do the review and preparation.  You will save the expense for the option one or option two changes, which would likely have been discarded when the scheduled 2014 review was concluded.  And, most importantly, you will have taken the first step toward addressing the impact of this failing program on the lost generation of our students.

Thank you.

________________________________________________________________

Some additional comments and thoughts:

In the presentation made by the administration on the fourth, they promised to post a summary of the options and costs outlined by the Math Coordinator.  This was posted, but then removed (I printed a copy).  The interesting thing is that during the meeting (and you can watch the tape), the third option was to “set aside the current set of curriculum and to adopt a new Common Core edition.”   This was later clarified that the option was to “adopt a K-5 Common Core Edition of a math program” (about 25 minutes into the tape).  This appears to open it up to all curricula, not just Everyday Math.  Yet the summary that was posted (and removed) contained the words “All options assume K-5 Core Math package is Everyday Math.”

The Math Coordinator also made the comment that he thought the administration could switch to a new edition of Everyday Math (i.e., the Common Core edition) without BoE approval, albeit with some additional funding.  Yet if you listen carefully to his rationalization for not switching to another curriculum (about 30 minutes into the tape), the same logic (?) should apply to adopting (actually not adopting) the Common Core edition of Everyday Math.  Talk about doubling down on a losing hand!

During the work session, Board member Peter von Braun asked the Math Coordinator “what are the problems that reduce student achievement in mathematics, what causes those problems, and why does this program (Everyday Math) solve those problems.”  The Math Coordinator’s response was to highlight new students coming into the district with different math backgrounds, students with “other handicaps” such as language difficulties, and the fact that Greenwich does not group students by ability, but differentiates in the classroom.  Don’t other districts also have these factors with which to contend?  So why are we falling behind? 

If you listen to the math Common Core discussion during the work session, and I have several times, the essence is that the administration (while not recommending any particular path) is moving toward making a little bit of change to the current curriculum (don’t forget, it is already close) to get us through to 2014, then do a full scale review.  This would be acceptable IF the current program was not failing, and IF that failure was not jeopardizing the future of our children, and IF the alignment was as good as suggested.  Chairman Moriarty said at the conclusion of the conversation that next steps need to be determined, but no plans were made or information requested (as far as I heard).  We cannot afford to wait for 2014. 

I am betting that at some point a group of parents will realize the impact Everyday Math had on their children (when they drop out of college), and will institute a law suit against the Board of Education and the Greenwich Public Schools for educational malpractice.  This has not worked in the past, but with the focus on accountability, who knows what legal avenue some clever lawyer will find to push the issue.

Another Dimension of Everyday Math

“In Unit 9, children will develop a variety of strategies for multiplying whole numbers.  They will begin by using mental math (computation done by counting fingers, drawing pictures, making diagrams, and computing in one’s head).  Later in this unit, children will be introduced to two specific algorithms, or methods, for multiplication: the partial products algorithm and the lattice method.”

A friend of ours, who has a child in the third grade, passed along this page from the third grade homework book for Everyday Math.  This “Family Letter” is used to introduce parents to what is coming up in the next chapter.  Our friend’s note included the words “troubling” and “irritating” to describe the contents of the Family Letter.

To some extent, I feel like this is an episode out of The Twilight Zone, where I can almost hear Rod Serling say “Imagine if you will, a school, like many schools, except that the third grade children count on their fingers because they don’t know their addition facts yet.  You have just entered (pause), the EDM Zone.”  Key weird music. 

Do we really feel comfortable with a program that encourages third graders to count on their fingers and draw pictures to figure out addition problems?  Shouldn’t they have mastered their basic addition facts in first grade?  Not according to the Teacher’s Reference Manual for Grades 1-3.  On page 196, the table indicates that the “Hard facts” (like 8+7) should be mastered by the end of third grade.  At least Common Core is forcing mastery to an earlier grade.

And is this the best way to teach multiplication?  Are our children really going to learn multiplication by counting on their fingers and drawing pictures?  Now if “computing in one’s head” means recalling number facts automatically, then they are partly right.  But I would interpret that phrase as meaning the students are actually figuring out a multiplication problem like 3x8 by thinking something like, “I know 3x4=12, and twice 4 is 8, so 3x8 must equal 12 +12 or 24.” 

Herein lays the problem with Everyday Math, and its proliferation around the US.  It says all the right things, and any teacher or administrator can point to these statements to defend the curriculum.  For example, who can argue with “Automatically knowing basic number facts is as important to learning mathematics as knowing words by sight is to reading.”  In execution, however, it fails miserably.  And when it fails, the publishers can always claim that the implementation by the school was flawed.

Our friend goes on in the e-mail, “So then I read more of the letter, which notes that soon the kids will learn the lattice method of multiplication.  I only glanced at what that method entails but it looks pretty ridiculous. But what's worse is that the letter tells parents why they are teaching this method and the reasons are like "because it is historically interesting"!  Not because it's a method that leads to fewer errors, or one that is faster or some other sensible reason.” 

The exact quote on the Lattice Method is “Third Grade Everyday Mathematics introduces the lattice method of multiplication for several reasons: This algorithm is historically interesting; it provides practice with multiplication facts and addition of 1-digit numbers; and it is fun.  Also, some children find it easier to use than other methods of multiplication.

Let’s take each one of these reasons. 

1.    Historically interesting: this is another meaningless statement.  I am sure there are very few if any third grade students who care about the history of the method dating back to “Hindu origins in India before A.D. 1100.”

2.    Provides practice: EDM is so desperate to convince parents that it provides practice for multiplication and addition facts that it has to use multiplication problems to support its claim.  Isn’t that a bit like saying we practice spelling by spelling words?  And couldn’t the same thing be said about the traditional algorithm?

3.    It is fun: in the Teacher’s Reference Manual, the authors state that “To our surprise, lattice multiplication has become a favorite of many children.”  Why?  Because they are spending time drawing rather than practicing math.  Fun or results?  No wonder they call this Math Lite.

The Teacher’s Manual also claims that the methodology is very efficient.  I have already dealt with that fallacy in a previous post. 

http://greenwichmath.blogspot.com/2011/11/journey-through-everyday-math.html

The Manual goes on to say you can use lattice multiplication for “problems that are too large for long multiplication (i.e., the traditional algorithm) or for most calculators…”  Since the traditional algorithm is more efficient, the first half of this statement is also fallacious.  And isn’t it interesting that the justification for not teaching the traditional algorithm is that we can always use a calculator, but we should learn to use lattice multiplication because it can handle problems calculators can’t?  My solution: get rid of the calculators and get rid of lattice multiplication (i.e., stop wasting time), and focus on the traditional algorithm.  Better yet, get rid of Everyday Math!  Send it to (pause) The Twilight Zone.

Test Questions to Question

You may have seen articles in the NYT and WSJ (and elsewhere) about a question given on the NYS eighth grade English Language Arts standardized test. 

http://www.nytimes.com/2012/04/21/nyregion/standardized-testing-is-blamed-for-question-about-a-sleeveless-pineapple.html?_r=1&pagewanted=2&pagewanted=all

http://online.wsj.com/article/SB10001424052702303513404577356113609677208.html

You may have also seen a blog posting from The Happy Scientist, a Florida blogger who was preparing questions to aid students taking the science Florida Comprehensive Assessment Test, the FL equivalent of Connecticut’s CMTs.  He presented some interesting findings, and even more interesting feedback from state education officials.

http://thehappyscientist.com/blog/problems-floridas-science-fcat-test

My comment was:

My brother is a former Florida public school teacher, who went on to private, parochial, prison and military schools (the prisoners were the most-well behaved). He had similar concerns about standardized tests.

Two comments:
1. while it is a possible defense to say that the students should not know this or that at the fifth grade level (see comment 2), thereby eliminating a correct response, why not simply come up with three responses that are clearly not (scientifically) correct? Yes, there is always going to be one or two that slip through, or there will be a scientist or mathematician who can posit some reasonable test or answer (as some of your knowledgeable readers have shown), but any reasonable review process by real scientists or good science teachers would at least indicate which answers/definitions need to be examined in more depth before inclusion.

http://www.cbmsweb.org/Forum2/Panel_Presentations/Assessment__Wilfried_S...

2. the state administrator's response that students should not know X, since it is not in the standards for fifth grade, is reflective of what I call the RTTM (Race to the Middle). The thinking is that we only need to get all of our students to the NCLB "Proficient" level (as it is called here in Connecticut), so any effort beyond that (Gifted and Talented, Advanced Learning, self study, good teaching, etc.) is a waste of time and money. Yet everyone is focused on college readiness and on preparing students for the STEM subjects. Proficient does not begin to cut it if one is interested in engineering, especially when many of the state standardized tests actually award Proficient level status for what the NAEP Math and ELA would consider Basic or Below Basic.

Check out the Math and Reading NAEP vs. state test comparisons:

http://nces.ed.gov/nationsreportcard/pdf/studies/2011458.pdf

One question: has anyone actually had experience in using FOIA to get access to the test questions?

So, what’s your point, Brian?  Well, this certainly couldn’t happen here in Connecticut, could it?  Our daughter brought home a practice test for the third grade CMT Mathematics test.  Of the 56 problems on the test, six were estimation problems, three relating to arithmetic operations and three relating to measurement.  Why pick on estimation problems?  This is one of the strands, as mentioned before, with which our students have a difficult time and (coincidentally?) on which Everyday Math places much emphasis.  Not wanting to blame everything on Everyday Math (although my office is still a mess), I wanted to see if there was another explanation for the relatively low scores on this strand.

In examining the three arithmetic operations questions, one was ambiguous (i.e., two of the answers could be correct, depending on how you interpreted the question), one was unambiguous only because the two interpretations yielded the same answer, and one was clearly written.  See if you agree with my thinking.

Question 1: What is the closest estimate of $54 - $38?  Round to the nearest ten.

A.   $10

B.   $20

C.   $30

D.   $90

The answer is A, because you round the 54 to 50 and the 38 to 40, and then subtract.  My daughter put down B, and I agreed with her, since the actual difference of $16 rounds to $20.  I can understand both answers, which is actually the problem.  The wording is poor, as either answer would be defendable. 

Question 2: What is the best estimate of 64 + 77?

A.   130

B.   140

C.   169

D.   180

Only 140 would fit both possible interpretations (round first then add, or add first then round).  Interestingly, no instructions are given for rounding, as in the first problem above.  So my vote is for ambiguous wording.

Question 3: There are 48 pencils and 53 pens in the school supply closet.  Round the numbers to the nearest 10 to estimate the total number of pencils and pens in the closet.

A.   80

B.   90

C.   100

D.   110

Much better wording here, including the clarifying word “numbers” indicating that rounding comes first.  If the first problem had said “Round the numbers to the nearest ten.” most of the ambiguity would have been removed.

I may be off-base on the first problem, as our children are taught (I hope) to round, then subtract or add (that is the useful skill).  This review raises three questions in my mind: First, why make it confusing by having two possible correct answers, and providing insufficient wording to draw a single conclusion?  Second, why is there different wording between problem 1 and problem 2?  And third, if the first problem confuses a Math ALP student, what percent of other students are going to be confused (especially since they are learning using Everyday Math)?

The point is that with only a quick look at a practice test, presumably based on actual test guidelines, I was able to find some ambiguous questions (or answers).  Maybe we have an issue, like Florida and New York.  To be fair, the example tests problems on the CSDE site appear well written.  So this might be a one-off issue with a bad practice test, and the real tests are okay.  But doesn’t that mean that the low strand scores are due to something else, like Everyday Math?

And don’t forget that one of the reasons Everyday Math focuses on estimation is so that the students can check the results of their calculations when they use a calculator.  If we eliminate calculators from K-5 (like Texas wants to: http://www.chron.com/news/houston-texas/article/Early-math-lesson-Put-calculators-aside-3496172.php), not only would we have students who could do their basic calculations better, but we wouldn’t have to spend so much time on estimation.  Sounds like a win-win situation to me.