Think Tank Review Project: Lexington Institute's "Portfolios" Report Lacks Research

A lot of reports are being issued to influence the next version of NCLB. The Think Tank Review Project at Arizona State University is keeping an eye on them. Here's their latest press release:

TEMPE, Ariz and BOULDER, Colo. (Sept. 19, 2007)—A Lexington Institute report released earlier this month, and the institute’s subsequent press release of September 17, 2007, criticized the use of student portfolios to assess school performance. A new review of that report, however, finds it is ill-founded and of little value for research or policy development.

The Lexington report, called “Portfolios – A Backward Step in School Accountability,” is intended to influence the debate over the direction of the reauthorization of the No Child Left Behind Act, offering a defense of the current test-based accountability system against the inclusion of “multiple measures.”

The report appears to have been written in anticipation of a “discussion draft” concerning NCLB changes, released by the leadership of the House Education Committee. The draft proposes allowing states to use “multiple indicators” – for example, graduation rates and percent of students taking advanced courses – to assess educational outcomes rather than depend so heavily upon standardized test scores.

The Lexington report was reviewed for the Think Tank Review Project by University of Vermont professor William Mathis. He concludes that the report more closely resembles a political polemic than a research report. It provides no new data, examines only two studies, and includes only results favorable to the report’s conclusions.

As Mathis notes, the House Committee’s summary lists a broad list of various alternative multiple indicators, but portfolio assessment is not on that list. Nonetheless, the Lexington report erroneously generalizes findings about portfolios to all non-test-based multiple indicators. The report also ignores a body of research with findings that present portfolios in a more favorable light.

In fact, Mathis’ review explains that the Lexington report’s focus on portfolios is particularly misguided because portfolios for statewide accountability purposes have received only scant attention since the turn of the millennium. Although Lexington’s press release emphasizes “dusting off” the use of portfolio assessments as a key part of NCLB accountability, the only group seriously discussing the subject appears to be Lexington itself. Given the absence of portfolio assessment from such agendas as well as from the House Committee’s list, it is troubling that the Lexington Institute report offers portfolios as the most notable of what it calls “multiple measures” and then uses that straw man to argue against adopting any instruments other than standardized tests.

In his review of the Lexington report, Mathis concludes that its “selective use of research suggests the author either intentionally slanted the evidence or was unacceptably cursory in his analysis.” The report’s failure to discuss contradictory research undermines its conclusions, and its attempt to “generalize all multiple measures from this questionable base completely discredits [the report].”