Teaching math to the talented: which countries--and states--are producing high-achieving students?

Teaching math to the talented: which countries--and states--are producing high-achieving students? In Vancouver last Winter, the United States proved its competitivespirit by winning more medals--gold, silver, and bronze--at the WinterOlympic Games than any other country, although the German member of ourresearch team insists on pointing out that Canada and Germany both wonmore gold medals than the United States. But if there is some disputeabout which Olympic medals to count, there is no question about Americanmath performance: the United States does not deserve even a paper medal. Maintaining our productivity as a nation depends importantly ondeveloping a highly qualified cadre of scientists, engineers,entrepreneurs, and other professionals. To realize that objectiverequires a system of schooling that produces students with advanced mathand science skills. To see how well schools in the United States do atproducing high-achieving math students, we compared the percentage ofU.S. students in the high-school graduating Class of 2009 with advancedskills in mathematics to percentages of similarly high achievers inother countries. Unfortunately, we found that the percentage of students in the U.S.Class of 2009 who were highly accomplished in math is well below that ofmost countries with which the United States generally compares itself.No fewer than 30 of the 56 other countries that participated in theProgram for International Student Assessment (PISA) math test, includingmost of the world's industrialized nations, had a larger percentageof students who scored at the international equivalent of the advancedlevel on our own National Assessment of Educational Progress (NAEP)tests. Moreover, while the percentage of students scoring at theadvanced level on NAEP varies considerably among the 50 states, not eventhe best state does well in international comparison. A 2005 report fromthe National Academy of Sciences, Rising Above the Gathering Storm,succinctly put the issue into perspective: "Although many peopleassume that the United States will always be a world leader in scienceand technology, this may not continue to be the case inasmuch as greatminds and ideas exist throughout the world." The Demand for High Achievers The gap between the burgeoning business demand for a highlyaccomplished workforce and a lagging education system has steadilywidened. Even as the United States was struggling with a near 10 percentunemployment rate in the summer of 2010, businesses complained that theycould not find workers with needed skills. New York Times writer MotokoRich explained, "The problem ... is a mismatch between the kind ofskilled workers needed and the ranks of the unemployed." Skill shortages have severe consequences for a nation'soverall productivity. Two of the authors of this report have shownelsewhere that countries with students who perform at higher levels inmath and science show larger rates of increase in economic productivitythan do otherwise similar countries with lower-performing students (see"Education and Economic Growth," research, Spring 2008). Public discourse has tended to focus on the need to address lowachievement, particularly among disadvantaged students. Both federalfunding and the accountability elements of No Child Left Behind (NCLB)have stressed the importance of bringing every student up to a minimumlevel of proficiency. As great as this need may be, there is no lessneed to lift more students, no matter their socioeconomic background, tohigh levels of educational accomplishment. In 2006, the Science,Technology, Engineering, and Mathematics (STEM) Education Coalition wasformed to "raise awareness in Congress, the Administration, andother organizations about the critical role that STEM education plays inenabling the U.S. to remain the economic and technological leader of theglobal marketplace for the 21st Century." In the words of aNational Academy of Sciences report that jump-started thecoalition's formation, the nation needs to "increase" its"talent pool by improving K-12 science and mathematicseducation." [ILLUSTRATION OMITTED] A Focus on Math We give special attention to math performance because math appearsto be the subject in which accomplishment in secondary school isparticularly significant for both an individual's and acountry's economic well-being. Existing research, though notconclusive, indicates that math skills better predict future earningsand other economic outcomes than other skills learned in high school.The American Diploma Project estimates that "in 62 percent ofAmerican jobs over the next 10 years, entry-level workers will need tobe proficient in algebra, geometry, data interpretation, probability andstatistics." There is also a technical reason for focusing our analysis on math.This subject is particularly well suited to rigorous comparisons acrosscountries and cultures. There is a fairly clear international consensuson the math concepts and techniques that need to be mastered and on theorder in which those concepts should be introduced into the curriculum.The knowledge to be learned remains the same regardless of the dominantlanguage spoken in a culture. Data and Methodology Our analysis relies on test-score information from NAEP and PISA.NAEP, the National Assessment of Educational Progress, is often calledthe nation's report card. It is a large, nationally representativeassessment of student performance in public and private schools inmathematics, reading, and science that has been administeredperiodically since the early 1970s to U.S. students in 4th grade and 8thgrade, and at the age of 17. PISA, the Program for International StudentAssessment, is an internationally standardized assessment of studentperformance in mathematics, science, and reading established by theOrganisation for Economic Co-operation and Development (OECD). It wasadministered in 2000,2003, and 2006 to representative samples of15-year-olds in all 30 OECD countries (which include the most developedcountries of the world) as well as in many others. We focus on performance of the international equivalent of the U.S.high-school graduating Class of 2009 at the time when this populationwas in the equivalent of U.S. grades 8 and 9. NAEP was administered toU.S. 8th graders in 2005, while PISA 2006 was given one year later tostudents at the age of 15, the year at which most American students arein 9th grade. In 2005, NAEP tested representative samples of 8th-grade public andprivate school students in each of the 50 states in math, science, andreading. For each state, NAEP 2005 calculates the percentage of studentswho meet a set of achievement standards: a "basic" level, a"proficient" level, and an "advanced" level ofachievement. The focus of this report is the top performers, thepercentage of students NAEP found at the advanced level of achievement(subsequently referred to as "advanced"). [ILLUSTRATION OMITTED] Only 6.04 percent of the students in the United States in 8th gradein 2005 scored at the advanced level in math on the NAEP. Some criticsfeel that the standard set by the NAEP governing board is excessivelystringent. However, the 2007 Trends in International Math and ScienceStudy (TIMSS 2007), another international test that has beenadministered to students throughout the world, appears to have set astandard very similar to NAEP 2005, as only 6 percent of U.S. 8thgraders scored at the advanced level on that test as well. We use the NAEP 2005 advanced standard to compare U.S. performancewith that in other countries. Because U.S. students took both NAEP 2005and PISA 2006, it is possible to find the score on PISA that istantamount to scoring at the advanced level on NAEP, i.e., the scorethat will yield the same percentage of students as the percentage of U.S. students who scored at the advanced level on the NAEP. A score on PISA 2006 of 617.1 points is equivalent to the lowestscore attained by anyone in the top 6.04 percent of U.S. students in theClass of 2009. (The PISA assessment has an average score of 500 amongOECD students and a standard deviation of 100.) It is assumed that bothNAEP and PISA tests randomly select questions from a common universe ofmathematics knowledge. Given that assumption, it may be further assumedthat students who scored similarly on the two exams will have similarmath knowledge, i.e., students who scored 617.1 points or better on thePISA test would have been identified at the advanced level had theytaken the NAEP math test. Inasmuch as a score of 617.1 points is morethan one standard deviation above the average student score on the PISA,it is clear that a group of highly accomplished students has beenisolated. (For more methodological details, see sidebar.) Because representative samples of student performance on NAEP 2005are available for each state, it is possible to compare the percentagesof students in the Class of 2009 who were at the advanced level for eachstate to the percentage of equally skilled students in countries fromaround the globe. In short, linking the scores of the Class of 2009 on NAEP 2005 andPISA 2006 provides us with the opportunity to assess from aninternational vantage point how well the country as well as individualstates in the United States are doing at lifting students to high levelsof accomplishment. U. S. Math Performance in World Perspective We begin with an overall assessment of the relative percentages ofyoung adults in the United States and other countries who have reached avery high level of mathematics achievement. It is frequently noted thatthe United States has a very heterogeneous population, with largenumbers of immigrants. Such a diverse population, with students comingto school with varying preparation, may handicap U.S. performancerelative to that of other countries. For this reason, we also examinetwo U.S. subgroups conventionally thought to have better preparation forschool--white students and students from families where at least oneparent is reported to have received a college degree--and compare thepercentages of high-achieving students among them to the (total)populations abroad. Overall results. The percentage of students in the U.S. Class of2009 who were highly accomplished is well below that of most countrieswith which the United States generally compares itself. While just 6percent of U.S. students earned at least 617.1 points on the PISA 2006exam, 28 percent of Taiwanese students did. (See Figure 1 for theseresults as well as for the international rank of each U.S. state.) [FIGURE OMITTED] [FIGURE OMITTED] It is not only Taiwan that did much, much better than the UnitedStates. At least 20 percent of students in Hong Kong, Korea, and Finlandwere similarly highly accomplished. Twelve other countries had more thantwice the percentage of advanced students as the United States: in orderof math excellence, they are Switzerland, Belgium, the Netherlands,Liechtenstein, New Zealand, the Czech Republic, Japan, Canada,Macao-China, Australia, Germany, and Austria. [ILLUSTRATION OMITTED] The remaining countries that educate a greater proportion of theirstudents to a high level are Slovenia, Denmark, Iceland, France,Estonia, Sweden, the United Kingdom, the Slovak Republic, Luxembourg,Hungary, Poland, Norway, Ireland and Lithuania. The 30-country list includes virtually all the advancedindustrialized nations of the world. The only OECD countries producing asmaller percentage of advanced math students than the United States arePortugal, Greece, Turkey, and Mexico. The performance levels of studentsin Spain and Italy are statistically indistinguishable from those ofstudents in the United States, as are those of students in Latvia, whichhas subsequently joined the OECD. State-level performance. The percentage of students scoring at theadvanced level varies among the 50 states. Massachusetts, with over 11percent of its students at the advanced level, does better than anyother state, but its performance trails that of 14 countries. Itsstudents' achievement level is similar to that of Germany andFrance. Minnesota, with more than 10 percent of its students at theadvanced level, ranks second among the 50 states, but it trails 16countries and performs at the level attained by Slovenia and Denmark.New York and Texas each have a percentage of students scoring at theadvanced level that is roughly comparable to the United States as awhole, Lithuania, and the Russian Federation. [ILLUSTRATION OMITTED] Just 4.5 percent of the students in the Silicon Valley state ofCalifornia are performing at a high level, a percentage roughlycomparable to that of Portugal. The lowest-ranking states--WestVirginia, New Mexico, and Mississippi--have a smaller percentage of thehighest-performing students than Serbia or Uruguay, although they doedge out Romania, Brazil, and Kyrgyzstan. In short, the percentages of high-achieving students in the UnitedStates--and in most of its individual states--are shockingly below thoseof many of the world's leading industrialized nations. Results formany states are at a level equal to those of third-world countries. White students. The overall news is sobering. Some might try tocomfort themselves by saying the problem is limited to large numbers ofstudents from immigrant families, or to African American students andothers who have suffered from discrimination. For example, the statementby the STEM Coalition that we "encourage more of our best andbrightest students, especially those from underrepresented ordisadvantaged groups, to study in STEM fields" suggests that thechallenges are concentrated in nonwhite segments of the U.S. population. Without denying that the paucity of high-achieving students withinminority populations is a serious issue, let us consider the performanceof white students for whom the case of discrimination cannot easily bemade. Twenty-four countries have a larger percentage of highlyaccomplished students than the 8 percent achieving at that level amongthe U.S. white student population in the Class of 2009. Looking at justwhite students places the U.S. at a level equivalent to what allstudents are achieving in the United Kingdom, Hungary, and Poland. Sevenpercent of California's white students are advanced, roughly thepercentage for all Lithuanian students. Children of parents with college degrees. Another possibility isthat schools help students reach levels of high accomplishment ifparents are providing the necessary support. To explore thispossibility, we assumed that students who reported that at least oneparent had graduated from college were likely to be given the kind ofsupport that is needed for many to reach high levels of achievement.Approximately 45 percent of all U.S. students reported that at least oneparent had a college degree. The portion of students in the Class of 2009 with acollege-graduate parent who are performing at the advanced level is 10.3percent. When compared to all students in the other PISA countries, thisadvantaged segment of the U.S. population was outranked by students in16 other countries. Nine percent of Illinois students with acollege-educated parent scored at the advanced level, a percentagecomparable to all students in France and the United Kingdom. Thepercentage of highly accomplished students from college-educatedfamilies in Rhode Island is just short of 6 percent, the same percentagefor all students in Spain, Italy, and Latvia. The Previous Rosy Gloss Many casual observers may be surprised by our findings, as twoprevious, highly publicized studies have suggested that--even thoughimprovement was possible--the U.S. was doing all right. This was thepicture from two reports issued by Gary Phillips of the AmericanInstitutes for Research, who compared the average performance in math of8th-grade students in each of the 50 states with the average scores of8th-grade students in other countries. These comparisons used methodsthat are similar to ours to relate 2007 NAEP performance for U.S.students to both TIMSS 2003 and TIMSS 2007. His findings are morefavorable to the United States than those shown by our analyses. Whileour study using the PISA data shows U.S. student performance in math tobe below 30 other countries, Phillips found the average U.S. student tobe performing better than all but 14 other countries in his 2007 reportand all but 8 countries in his 2009 report. (Oddly, the 2007 reporttakes a much more buoyant perspective than the 2009 report, though thedata suggest otherwise.) Phillips also finds that individual states domuch better vis-a-vis other countries than we report. Why do two studies that seem to be employing generally similarmethodologies produce such strikingly different results? The answer to that puzzle is actually quite simple and has littleto do with the fact that Phillips compares average student performancewhile our study focuses on advanced students: many OECD countries,including those that had a high percentage of high-achieving students,participated in PISA 2006 (upon which our analysis is based) but did notparticipate in either TIMSS 2003 or TIMSS 2007, the two surveys includedin the Phillips studies. In fact, 19 countries that outscored the U.S.on the PISA 2006 test did not participate in TIMSS 2003, and 22higher-scoring countries did not participate in TIMSS 2007. As a reportby the U.S. National Center for Education Statistics has explained,"Differences in the set of countries that participate in anassessment can affect how well the United States appears to dointernationally when results are released." Put starkly, if one drops from a survey countries such as Canada,Denmark, Finland, France, Germany, and New Zealand, and includes insteadsuch countries as Botswana, Ghana, Iran, and Lebanon, the averageinternational performance will drop, and the United States will lookbetter relative to the countries with which it is being compared. [ILLUSTRATION OMITTED] Did NCLB shift the focus away from the best and the brightest? Some attribute the comparatively small percentages of studentsperforming at the advanced level to the focus of the 2002 federalaccountability statute, No Child Left Behind, on the educational needsof very low performing students. That law mandates that every student bebrought up to the level a state deems proficient, a standard that moststates set well below NAEP's proficient standard, to say nothing ofthe advanced level that is the focus of this report. In order to comply with the federal law, some assert, schools areconcentrating all available resources on the educationally deprived,leaving advanced students to fend for themselves. If so, then we shouldsee a decline in the percentage of students performing at NAEP'sadvanced level subsequent to the passage of the 2002 federal law. Inmathematics, however, the opposite has happened. The percentageperforming at the advanced level was only 3.7 percent in 1996 and 4.7percent in the year 2000. But the percentage performing at an advancedlevel climbed steadily to the 7.9 percent attained in 2009. Perhaps NCLB's passage in 2002 dampened the prior rate ofgrowth in the achievement of high-performing students. To ascertainwhether that was the case, we compared the rate of change in the NAEPmath scores of the top 10 percent of all 8th graders between 1990 and2003 (before NCLB was fully implemented) with the rate of change afterNCLB had become effective law. Between 1990 and 2003, the scores ofstudents at the 90th percentile rose from 307 to 321, an increment of 14points, or a growth rate of 1.0 points a year. Between 2003 and 2009,the shift upward for the 90th percentile was another 8 points, or achange of 1.3 points a year. Our results are confirmed by a moredetailed study of NCLB's impact on high-performing studentsconducted by economists Brian Jacob and Thomas Dee. [ILLUSTRATION OMITTED] In short, the incapacity of American schools to bring students upto the highest level of accomplishment in mathematics is much moredeepseated than anything induced by recent federal legislation. Conclusions The economic and technological demand for a talented,well-educated, highly skilled population has never been greater. Notonly must everyday workers have a set of technical skills surpassingthose needed in the past, but a cadre of highly talented professionalstrained to the highest level of accomplishment is needed to fosterinnovation and growth. In the words of President Barack Obama,"Whether it's improving our health or harnessing clean energy,protecting our security or succeeding in the global economy, our futuredepends on reaffirming America's role as the world's engine ofscientific discovery and technological innovation. And that leadershiptomorrow depends on how we educate our students today, especially inmath, science, technology, and engineering." Unfortunately, the United States trails other industrializedcountries in bringing a large proportion of its students up to thehighest levels of accomplishment. This is not a story of some statesdoing well but being dragged down by states that perform poorly. Nor isit a story of immigrant or disadvantaged or minority students hiding thestrong performance of better-prepared students. Comparatively smallpercentages of white students are high achievers. Only a smallproportion of the children of our college-educated population isequipped to compete with students in a majority of OECD countries. Major policy initiatives within the United States have in recentyears focused on the educational needs of low-performing students. Suchefforts deserve commendation, but they can leave the impression thatthere is no similar need to enhance the education of those students theSTEM coalition has called "the best and brightest." Yet, withrapidly advancing technologies in an increasingly integrated worldeconomy, no one doubts the extraordinary importance of highlyaccomplished professionals. Admittedly, the United States could simply ignore the needs of itsown young people and continue to import highly skilled scientists andengineers who were prepared by better-performing schools abroad. Buteven such a heartless, irresponsible strategy relies on both the natureof immigration policies and the absence of better opportunities abroad,two things on which we might not want the future to depend. It seemsmuch more prudent to encourage the most capable of our own people toreach high levels of academic accomplishment. Methodology We start with the national share of 8th-grade U.S. public andprivate school students (most of whom are 14 years of age) who reach theadvanced level in math on NAEP 2005: 6.04 percent. These students areassumed to be part of the cohort of 15-year-olds who participated inPISA 2006 one year later. Thus, using the PISA 2006 microdata, we cancalculate the PISA math test score at which the 93.96th percentile(100.00 - 6.04) of the U.S. student population performs. All PISAcalculations use the PISA sampling weights to yield nationallyrepresentative estimates. The PISA scaling methodology returns studentperformance estimates through a range of five plausible values, whichare random draws from the estimated probability distribution for astudent's underlying performance. We perform our analysisseparately for each of the five plausible values provided by PISA 2006.We then average these results. Based on these calculations, we estimatethe PISA score at which the 93.96th percentile of the U.S. studentpopulation performs to be 617.1 PISA points. Next, we calculate from the PISA microdata the share of studentsreaching this cutoff point for each country participating in the PISA2006 test. This provides an estimate of the share of students in eachPISA country who reach the equivalent of the advanced level in 8th-grademath on NAEP 2005. The share of students who reach the advanced level in8th-grade math in each U.S. state is taken from NAEP 2005. Forinformation on the statistical significance of differences amongjurisdictions, see the unabridged version of this study, available ateducationnext.org. Eric A. Hanushek is senior fellow at the Hoover Institution ofStanford University. Paul E. Peterson is the director of Harvard'sProgram on Education Policy and Governance and senior fellow at theHoover Institution. Ludger Woessmann is professor of economics at theUniversity of Munich.