What do standardized tests really mean?

What do standardized tests really mean?  Can we have every child receive a Proficient score on their standardize tests (Connecticut Mastery Test - "CMT"), and still not understand the math.  Let’s start with the concept that a Proficient score means you are proficient, which is defined by CT as:

 “Generally,…. students who perform at this level demonstrate adequate knowledge of grade-level content. These students demonstrate adequate conceptual understanding, computational skills and problem-solving skills, as well as an ability to solve complex and abstract mathematical problems. Typically, the solutions these students provide to math problems are adequate and include sufficient explanations.”

http://www.cga.ct.gov/2010/rpt/2010-R-0167.htm

Proficient is the middle range on the test, starting from Advanced to Goal to Proficient to Basic to Below Basic.  Goal is the score which Connecticut professes to desire of its students, so Proficient is less than what we want from our students.  On the 2011 CMT, about 85% of Connecticut third to fifth graders scored at or above Proficient; about 93.5% of Greenwich 3-5 graders did.  Pretty good, huh?

So where does that Proficient score on the CMT fit with the National Assessment of Educational Progress (“NAEP”), aka the “Nation’s Report Card?”  The NAEP is given to fourth, eighth and twelfth grade students.  The National Center for Education Statistics compared the rigor of the CMT and the NAEP.  For fourth grade students, it determined that a Proficient score on the CMT was equal to a Basic score on the NAEP (barely – CT’s score was 220, when 214 was the cutoff for Basic and 249 was the cutoff for Proficient on the NAEP).  For an eighth grade student, a Proficient score on the CMT was equal to a Below Basic score on the NAEP.  This information was for the 2007 NAEP and CMT tests. 

http://nces.ed.gov/nationsreportcard/pdf/studies/2010456.pdf

Interestingly, Connecticut lowered its standards between 2005 and 2007. 

The NAEP definition for Basic (at the fourth grade level) is:

“Fourth-grade students performing at the Basic level should show some evidence of understanding the mathematical concepts and procedures in the five NAEP content areas.

Fourth-graders performing at the Basic level should be able to estimate and use basic facts to perform simple computations with whole numbers, show some understanding of fractions and decimals, and solve some simple real-world problems in all NAEP content areas. Students at this level should be able to use—though not always accurately—four-function calculators, rulers, and geometric shapes. Their written responses will often be minimal and presented without supporting information.”

So to answer my question: the CMT may call it Proficient, but it is very close to being not adequate. 

But wait, there’s more!  We can take some solace in the fact that the NAEP is a good and consistent measure of the progress (or lack thereof) we are making, right?  Not so fast. 

Next up: NAEP and NCTM and EDM (MOUSE)