Earlier this month, the Center for International Education Benchmarking introduced a new feature on its web site.  The Study Guide is intended to provide readers with a weekly summary of headlines from the top-performing education systems in the world.  This month’s International Reads highlights some of the most important current issues in the news.

Teachers.  In Japan, changes to teachers’ retirement packages have caused many teachers to think about retiring early before the changes go into effect.  Japan education officials are struggling to figure out a work-around to ensure that students are not left without a teacher for the remainder of the school year, according to The Mainichi.  A Valentine’s Day strike was held by educators in Victoria, Australia in protest of the pay package offered by the state—a modest 2.5 percent a year plus performance-based pay.  The Australian Education Union is demanding a 12 percent raise over three years with no performance-based pay.  But on the issue of improving the quality of the pool entering teaching, the teacher’s union and the government are on the same side—recently, Education Union federal president Angelo Gavrielatos issued a statement supporting the government’s initiative to raise the quality of students entering the teaching profession.  In Ontario, newly elected Education Minister Liz Sandals is facing teacher dissatisfaction.  Last fall, legislation was passed banning teacher strikes in the province.  In January, the new law was used to impose a contract on public secondary school teachers.  In response, teacher unions have asked their members to refrain from supervising extracurricular activities, which they see as outside their regular duties.  Sandals said her first order of business is to ensure that new teacher contracts are the result of negotiation, not legislation.

Early Childhood Education.  In Japan, government officials are considering offering free pre-school to children ages 3-5, in an effort to ease the financial burden on families, according to Inside Japan.  This proposal comes at the same time that President Obama called for free pre-school for all 4-year-olds at or below 200 percent of the poverty line in the United States during his State of the Union address.  Parents in Hong Kong are focused on another concern — that kindergartens are emphasizing grades and tests too much.  In response, they are leading a movement to shift to kindergartens that emphasize learning through play, according to the South China Morning Post.  And in New Zealand, the Pasifika Education Plan 2013-17 aims to lift Pasifika participation in early childhood education from its current rate of 86.8 percent to 98 percent by 2016, according to Radio New Zealand International.

Post-secondary Education.  A February 3rd editorial in the Japan Times calls for major changes in the country’s university entrance exam system, arguing that the current assessments measure knowledge acquired rather than deeper comprehension, aptitude and potential.  Meanwhile, researchers at the Hong Kong Institute of Education (HKIE) say educational inequality is getting worse, despite the increased number of publicly funded university places.  A recent HKIE study found that students from wealthy families are nearly four times more likely to enroll in a university than those living in poverty.  That’s a much wider gap than 20 years ago.  The Netherlands is looking to the liberal arts model to solve some of their higher education challenges related to a lack of differentiation and excellence.  Inside Higher Ed reports liberal art schools (known there as university colleges), “have had an outsized impact on Dutch higher education policies and practices, inspiring the growing movement toward selective admissions and the development of ‘excellence’ programs within a famously egalitarian higher education system.”

Choice and Charters.  The Education Amendment Bill, introduced last year in New Zealand, would create legal recognition of charter schools there.  However, the Treasury has found evidence that school systems using strongly competitive elements do not produce systematically better student outcomes and other critics are arguing that charter schools will take public money but be free from government scrutiny.  While charter schools are not prevalent in Canada, school competition does exist in the form of four separate publicly funded systems catering to the English and French non-religious and Catholic constituencies of Ontario.  With birth rates in that province on the decline, schools are struggling to keep enrollment levels high. But the schools don’t just compete for students in name only: recently, schools have taken to touting their extracurricular programs in advertisements in local media and attending other schools’ open houses in order to gain an edge.  For more on this story, see the article in The Globe and Mail.

International Benchmarking.  And finally, Shanghai is looking forward to the December publication of the 2012 PISA results to show the world that, once again, they are on top of the international education league tables.  According to Shanghai education officials interviewed by the Sydney Morning Herald, “tests recently conducted for the next PISA report…will show Shanghai students have further improved their results and consolidated their lead in the world.”  In the same article, Deputy Director of the OECD Education Division Andreas Schleicher says, “Maybe it’s time to change some of our stereotypes.  What you see today in the school system in Shanghai is what you are going to see in the labour market tomorrow.”  Learn more about the results from the most recent TIMMS and PIRLS international assessments by clicking here.

Check back to our web site on a weekly basis for more education news from the top-performing education systems in the world.

By Marc Tucker

My apologies to Maimonides.  But I would not blame you if you were perplexed about the recent dust-up after the latest PIRLS and TIMSS data came out.  Some of the best-known names in education research worldwide came out with guns blazing, mostly at one-another, in a rapid-fire exchange about what the numbers meant.  I thought some of you might welcome a guide to the shooters and the shots, and a bit of commentary on the profound meaning of it all.

Tom Loveless, the head of the Brown Center at the Brookings Institution jumped on the data to say that they called for a “rethinking of the Finnish miracle success story….If Finland were a state taking the 8^th grade NAEP [the sample survey used in the United States to monitor the progress of American students over time], it would probably score in the middle of the pack.”  Jack Buckley, Commissioner of the National Center for Educational Statistics in the U.S. Department of Education confessed that, “I’ve always been a little puzzled” by the high level of attention paid to Finland.  Well, so much for Finland!

Martin Carnoy and Richard Rothstein wrote an analysis of the data claiming to show that while reading achievement of American students on PISA was growing between 2000 and 2009, it was falling by an even larger amount in Finland.  Similarly, they said, in math, US students from the lowest social class were also gaining substantially, while scores of comparable Finnish students declined.  “This is surprising,” they said, “because the proportion of disadvantaged students in Finland also fell…” And they go on to say that, by their analysis, the achievement gap between the most and the least advantaged students in the United States is actually smaller than in “similar postindustrial countries, and often only slightly larger than gaps in top-scoring nations.”

Ha!  That means that the withering criticism showered on American schools for their poor performance was totally undeserved.  The problem, if there is a problem, lies not in the schools, poor Horatio, which have been doing a much better job than anyone has given them credit for, but in the enormous disparities in family income that have opened up in American society.  And Finland, according to this analysis, hardly deserves its status as the model that the United States should be adopting.

Not so fast, say Paul Peterson, Eric Hanushek and Ludger Woessmann.  Peterson is at Harvard, Hanushek at Stanford and Woessmann at the University of Munich.  The data, they say, don’t show that at all.  What they actually show is that, even if such corrections are made, American students at the top do not perform anywhere near as well as the students in the top performing countries, or, at least, not such a high proportion of them do.  Things are just as bad as they always said they were, and the need to turn up the heat on the schools to perform up to international standards is as great as ever.

But wait a minute, says Andreas Schleicher.  The Carnoy-Rothstein analysis depends, he said, on a challenge to the methods used by OECD-PISA to do its survey research, and that challenge, says Schleicher, just won’t hold up in court.  To which Carnoy and Rothstein said in reply to the reply, Oh yes it will.

So what is going on here?  Why are all these people so exercised about this data?  What are their agendas anyway?  Who is right and who is wrong?  Why does it matter?  And what does it mean?

I know that research is supposed to go where the evidence leads it and the researcher is only there to record the ineluctable result, without fear or favor.  But the reality is that researchers have values to support and reputations to protect, and their conclusions are more often than not influenced by both their values and the reputations they have established as a result of the policy positions they have taken.  So, perhaps it would help to sketch in the positions taken on the relevant issues by the people I have named.

It should surprise no one that spokespeople for the Brookings Institution and the United States National Center for Educational Statistics should be waiting to pounce on Finland and on the people who have used the Finns’ standing in the international league tables to make a case for using the educational strategies the Finns have embraced.  Both Brookings and a series of U.S. Department of Education research executives, some of whom have gone to Brookings when they left the Department of Education, have been deeply skeptical of international education benchmarking and ardent advocates of what they have described as the “gold standard” of education research, meaning the use of experimental research techniques as the only legitimate way to attribute cause in social research.  It is obviously impossible to randomly assign national “treatments” to national populations in the arena of education, so, from their point of view, all statements that this or that set of policies “causes” these or those national outcomes in the arena of education policy are necessarily suspect.

Brookings and the Peterson, Hanushek, Woessmann team are both strong supporters of charters and the introduction of market forces generally as school reform strategies.  Brookings, as well other Washington-based think tanks, are eager to deflate the recent enthusiasm for international education benchmarking in part because they fear that the close examination of the strategies used by the top-performing countries will show little evidence that charters or market strategies in general are effective strategies for raising student achievement at a national scale.

Peterson, Hanushek and Woessmann each have their own views on what is most important in education reform, but all are advocates of charters and reform agendas based on market forces, and all appear to believe that it will take fear of foreign competitors to put this reform agenda over the top in the United States.  They have also done research that they say supports their claim that market strategies do work in the top-performing countries.  Implicitly, then, they believe, unlike their Brookings colleagues, that it is possible to do rigorous research using comparative data gleaned from these international surveys that attributes cause and from which, therefore, it is possible to draw policy conclusions.  This team of researchers has consistently advanced the view, like my own organization, that economic ruin will be the fate of any nation that fails to hold its own in international education competition, though their prescriptions as to the most effective policy agenda are different from our own, based on the study of pretty much the same data.

But Carnoy and Rothstein come from a very different place.  They believe that the relatively poor performance of American students on the international surveys of student achievement is a function of the large and increasing disparity in incomes among Americans, in absolute terms and in relation to other countries.  They are outraged that organizations like my own and researchers like Peterson, Hanushek and Woessmann hold the schools accountable for poor student performance, when they think the fault lies not in the schools and teachers, but rather in a society that tolerates gross and increasing disparities in income among Americans.  They would have us focus on promoting policies that would result in a fairer distribution of income in the United States.

Which puts them in direct conflict not just with Peterson, Hanushek and Woessmann, but also with Andreas Schleicher, the driver of the whole PISA system at the OECD.  Schleicher’s primary framework for the analysis of the PISA data displays the country data on two axes, one for student achievement on the subjects assessed by PISA and the other for equity, the pattern of the distribution of results from the poorest to the best performers within countries.  Countries with short tails in that distribution are described as having high equity; those with long tails are described as having low equity.  Schleicher points out that the United States just barely escapes being among those countries in the worst quartile on both measures.  Another table in Schleicher’s slide deck shows that, when socio-economic status is held constant, the schools of some nations do a much better job than others of reducing achievement disparities among students.  Carnoy and Rothstein would take American teachers off the hook, saying that the performance of poor and minority students is actually improving, the gap is not so large as was thought, and the performance of poor and minority students in the top performing countries is actually declining.  To the extent there is a problem, it is a problem caused by socio-economic status of the students, not the teachers’ performance.  Schleicher would say, no, that is not so.  Even when we look at students from comparable socio-economic backgrounds, American schools do less to close the gap with the students from more favored backgrounds than schools in most other countries.  They cannot both be right.

So it is no wonder that Carnoy and Rothstein go after Schleicher and his data and methods with hammer and tongs.

So who is right and who is wrong here?  All of the people I have named are competent researchers from well-regarded institutions.  Just as each of these people have their own values and established positions on the relevant policy issues, the same is true of me and the organization with which I am associated.  Our analysis of the dynamics of the global economy strongly suggests that high wage countries like the United States will find it increasingly difficult to maintain their standard of living unless they figure out how to provide a kind and quality of education to virtually all their children that they formally thought appropriate only for a few.  And we also believe that the most likely source of good ideas for strategies that will enable them to do that is the countries that have already done it.  We think that whether the source of poor performance is mainly growing disparities of income or relatively poor performance of the education system, the dynamics of the global economy are unforgiving and countries like the United States do not have the option of saying that the educators can do nothing, that the only thing that will save us is income redistribution.  We do not think that the only way to learn what strategies are likely to work is research methods derived from the experimental sciences.  Indeed, we think that the record clearly shows that American business recovered from a devastating assault from Japanese firms in part by inventing and using the very method—industrial benchmarking—that we and others are now using in the field of education.

To me, the most important conclusion to be drawn from the debate whose contours I have just rather roughly outlined is that now, for the first time in the United States, the international surveys of student achievement really matter.  That is a big, big change.  It was not the case before that advocates of the most hotly debated education reforms in the United States felt that they needed to take the data from these surveys seriously, to defend their positions or to advance them.  Clearly, they do now.

The second point is that the data from the international surveys is being used to make points not about peripheral issues, but central issues.  It really matters whether the cause of the United States’ relatively low standing in the international league tables is income disparities among the students’ families or poor education in the schools.  It really matters whether or not countries like Finland have important lessons for the rest of the world.  It matters whether the survey methods being used by the organizations that design and administer them bear up to scientific scrutiny or not.  And, lastly, it also matters whether the methods used by those who do research comparing the effects of different policies and practices on student achievement in multiple countries have enough scientific merit to justify their use by policy makers to make national policy. These are consequential questions.  This is the first time that we have seen a sustained debate by some of America’s leading scholars on these matters.  It is not likely to be the last, and that appears to herald an era in which, for the first time in the United States, international surveys of student achievement are likely to take a prominent place in the public debate about education policy.
You may be wondering where I come out on the welter of claims and counterclaims I described above.  Now that I have laid my analytical framework on the table along with those of the other analysts, you are in a position to apply the same dose of skepticism to my conclusions as I urged you to apply to the others.   My take on the data we now have in hand is more or less as follows.

First, the usual note of caution.  One snapshot does not a movie make.  We should not declare a trend before we have more than one data point.  So we might want to see whether the changes in rankings suggested by the recent PIRLS and TIMSS data hold up over time.

Second, as many have pointed out, TIMSS and PIRLS put the accent on measuring how students do on what amounts to a consensus curriculum.  Did they learn what international experts think they should have been taught in the subjects they assess?  PISA measures the capacity of students to apply what they have learned in the classroom to proxies for real-world problems of the sort they might actually encounter outside the classroom.  I have a strong preference for the latter goal over the first, which mainly comes from an experience I had years ago, when Archie Lapointe, the director at that time of the Young Adult Literacy Survey, told me the following.  The survey asked the young people surveyed to add a column of figures and take a percentage of the result. Almost all could do it.  It also asked the same respondents to take a restaurant check, add up the items, get a total and calculate a tip.  Very few could do it.  Like Alfred North Whitehead, I have very little use for what he called “inert knowledge.”

Third, we need to keep in mind that the fine-grained distinctions in the rankings, for most countries that are near one another, are not statistically significant.  What we should really be paying attention to is the groupings of countries in the rankings, when countries are grouped in such a way that the measured differences among the groups are statistically significant.  If you look at it from this perspective, what we see is the United States still has a long way to go before the vast majority of its students score in the front ranks of performance at many grade or age levels in many subjects, which is how I would define top performers.

Fourth, I think it is pretty clear from the OECD data that smaller proportions of American students score in the higher deciles of performance on the PISA tests, and more in the lower deciles than is the case for students from the top-performing countries.  If that is true, then it cannot also be true that the United States would do as well as the top-performing countries if only the poor, Black and Hispanic students were taken out of the rankings, as many American teachers and some policymakers maintain.  It is also clear from the OECD-PISA analysis, as I pointed out above, that, when the data are corrected for students’ socio-economic status, American schools are less effective than the schools of most of the countries measured at closing the gap between these students and students with higher socio-economic status.

This, of course, is not where Carnoy and Rothstein come out, but I think Andreas Schleicher won the battle between him, on the one hand, and Carnoy and Rothstein on the other.  But don’t take my word for it.  Read the claims and arguments made by both sides carefully.  There is a lot at stake in this conflict.

So, what then are we to make of the fact that, if Massachusetts, North Carolina and Florida were countries, they would have done very well indeed in the most recently released rankings?

The case of Florida, I think, is pretty straightforward.  The Florida Center for Reading Research, administered by Florida State University, is one of the nation’s leading centers for reading research.  Its methods are widely admired throughout the United States.  The state of Florida has managed to leverage this research program and its key figures to produce widespread implementation throughout the state of the methods advocated by the Center.  We can see the results in the PIRLS fourth grade reading results.  The question, of course, is what effect, if any, this will have on student performance in the upper grades as the students who have benefitted from these programs mature through the years.  That story has yet to be told.

In North Carolina, we are looking at a program of education reform that began with Governor Terry Sanford, whose first term as governor began in 1961.  Sanford’s unrelenting emphasis on improving education in the state laid the base for Governor James B. Hunt, Jr., who served as governor from 1977 to 1985 and again from 1993 to 2001, making him the longest serving governor in the state’s history.  Through that whole period, he never lost his focus on education as the key to the state’s economic growth, and, during that period, North Carolina showed more progress on student achievement as measured by the National Assessment of Education Progress than any other state in the United States.  Hunt’s agenda for education reform was profoundly affected by what he was learning about the strategies adopted by the top-performing countries in the world.  Like them, he focused on teacher quality, high quality instructional systems and early childhood education.  North Carolina was among the very first states in the United States to send delegations of key state policy-makers abroad to study the top performers.

Massachusetts is a similar story.  In this case the first phase of the reforms were driven by the business community, organized by Jack Rennie, a very successful businessman who worked hard to organize that community, and Paul Reville a public policy analyst.  They played the key role in pushing the Massachusetts Education Reform Act of 1993 through the legislature.  The Act provided hundreds of millions in new funding for the schools in exchange for explicit performance standards for students, set to international benchmarks and carefully drawn curriculum frameworks, also set to international benchmarks; a new comprehensive assessment system set to the standards and curriculum frameworks; much tougher standards for getting to be a teacher, intended to greatly ratchet up teachers’ command of the subjects they intended to teach, and a system to disclose student performance, school by school, with results reported by student subgroups, so that poor performance by these subgroups would not be hidden in the average scores for the school.  Right after the Act was passed, David Driscoll, until then the Deputy Commissioner of Education, was made Commissioner and remained in that position for ten years.  Under Driscoll’s leadership, Massachusetts, despite a great deal of pressure to do so, never backed off of its decision to set and to maintain internationally benchmarked standards, for both student performance and teacher certification.  After Driscoll left, the new governor created a new position in state government, to provide leadership to all the parts of government concerned primarily with education at all levels.  He filled that position with Paul Reville.  Between them, Driscoll and Reville provided the same kind of strength and continuity of leadership that Governor Hunt provided in North Carolina, and for a very similar agenda, an agenda that is in many respects consistent with our own analysis of the strategies used by the top performing nations to get to the top of the league tables.

You may or may not agree with my analysis of the kerfuffle over the release of the TIMSS and PIRLS results.  You may or may not agree with my explanation for the rise of Florida, Massachusetts and North Carolina on the PIRLS and TIMSS league tables.  But, in any case, I urge you to look at the contending papers, and come to your own conclusions.  All of us could benefit greatly from a long, loud, contentious effort to define what it means to be educated, and to better understand why some nations are more successful than others at educating the vast majority of their young people to whatever standard they choose.

* This is a second version of the original post for this month.  We misstated the conclusions presented by Martin Carnoy and Richard Rothstein in the report described in this newsletter.  We believe we have stated those conclusions accurately here, and apologize to the authors for the error.

For the record, however, the version of the Carnoy-Rothstein conclusions that we based our first statement on was itself based on the version of the report that Carnoy and Rothstein originally released, which claimed that their re-estimate of United States PISA scores would result in the United States ranking 4^th among OECD countries in reading, and 10^th in math, a major revision upwards of the US PISA rankings.  In their most recent version of their report, released last week, Rothstein and Carnoy revised these numbers downward somewhat to 6^th in reading and 13^th in math, but, as the post points out, even these numbers are contested.

In December, the results of the 2011 administration of the Trends in International Mathematics and Science Study (TIMSS) and Progress in International Reading Literacy Study (PIRLS) were published in three separate reports, each examining international performance in reading (at the fourth grade level), math (at the fourth and eighth grade levels) and science (at the fourth and eighth grade levels).  These assessments provide a picture of international student performance in the years before a student reaches the age of 15, which is the age at which students take the OECD’s Programme of International Student Assessment (PISA).  However, there are some central differences between the TIMSS/PIRLS and PISA assessments.  Michael Martin, the Co-Executive Director of the TIMSS and PIRLS Study Center at Boston College, notes that while PISA is intended to measure a student’s general skills in the arenas of reading, math and science, TIMSS and PIRLS are more focused on content mastery.  Additionally, Jack Buckley, the commissioner of the National Center for Education Statistics, has pointed out that the countries participating in both assessments do vary – the TIMSS and PIRLS groups are smaller and represent a mixture of countries at different levels of economic development as compared to the participants in PISA.

Because of the differences between the assessments, the countries that are in the top ten or fifteen of the TIMSS and PIRLS rankings are somewhat different than the top performers on the last incarnation of PISA in 2009.  While league tables of the top countries based on their average scores always garner the most press when the results of international assessments are released, we decided to take a more in-depth look at what level of proficiency students in the top fifteen countries are actually reaching in these subjects.

The IEA has established four “international benchmarks” on their score scale for these assessments.  While the score scale for both PIRLS and TIMSS runs from 0-1000, the vast majority of scores fall between 300 and 700.  The IEA has identified a score of 400 as the “low” international benchmark, indicating that students at this score point have been educated to a “basic” level.  Beyond that, there is a score of 475, or “intermediate;” a score of 550, or “high,” and a score of 625, or “advanced.”  Below, we have plotted the percent of students at each benchmark in the top fifteen countries on the 2011 administration of PIRLS and TIMSS.  This is useful when thinking about the top performers, because it shows, in a clearer way perhaps than the average scale score, what students in each country are really able to do.

In the fourth grade PIRLS reading assessment, a student who reaches the “low” international benchmark is able to “locate and retrieve an explicitly stated detail” in a literary text, and “locate and reproduce explicitly stated information … at the beginning of the text” in an informational text.  By contrast, at the “advanced” international benchmark, students are able to “integrate ideas and evidence across a text,” and “distinguish and interpret complex information from different parts of a text,” among other skills.

The chart above, like the others to follow, is organized from top to bottom in the order of average scale score.  However, the average scale score does not always correlate to the highest percentage of students reaching the “advanced” benchmark in each country.  In this case, it does not, though Hong Kong does have the highest proportion of students meeting either the “high” benchmark or “advanced” benchmark – 67 percent – while in the United States, just 56 percent of students meet those levels.  The tail of students either meeting the “low” benchmark or not meeting a benchmark is also significantly smaller in the top three countries – Hong Kong, the Russian Federation, and Finland (7, 8 and 8 percent, respectively), than in the majority of the other countries.  This more specific data on student performance is useful in terms of thinking about a country’s overall performance, because it gives a clearer sense, potentially, of the equity of the school system, and the ability of the system to educate all students – or any students – to high levels.  It also demonstrates that there are clear differences in student performance between the top handful of countries and the rest of the countries rounding out the top ten or fifteen.

For fourth grade math, in order to reach the “low” benchmark, a student must be able to demonstrate “basic mathematical knowledge,” such as adding and subtracting integers and being able to recognize familiar shapes.  At the “advanced” benchmark, a student must have an understanding of how to apply their knowledge, for example, by solving word problems with multiple steps, and they must show some understanding of more difficult concepts like fractions and decimals.

In the case of TIMSS fourth grade math, the percent of students reaching the “advanced” benchmark does correlate to the country’s average scale score, at least for the top six performers.  This chart indicates very clearly how well the East Asian countries do compared to the rest of the world in instilling advanced-level math skills in their students, even at an early age, with about a third of students or more reaching the “advanced” benchmark in Singapore, Korea, Hong Kong, Taiwan and Japan, and an overwhelming majority reaching either the “advanced” or “high” benchmarks in all cases.  These countries also have the smallest proportions of students who failed to meet the most basic level.  By contrast, starting with Northern Ireland, which is in sixth place in the overall league table in this subject, the other countries have higher proportions of students failing to reach at least the “intermediate” benchmark, and generally much lower proportions of students reaching the “advanced” benchmark.

In eighth grade math, students at the “low” benchmark “have some knowledge of whole numbers and decimals, operations, and basic graphs.”  At the “advanced” level, students are able to demonstrate many mathematical skills, such as solving linear equations, reasoning with geometric figures, and expressing generalizations algebraically.

The pattern in proficiency seen in the TIMSS fourth grade math results is continued in the TIMSS eighth grade math results.  Andreas Schleicher from the OECD and US Education Secretary Arne Duncan have commented on the drop in math and science skills from fourth grade to eighth grade in the United States, and the data bears this out.  In fourth grade, 47 percent of American students met either the “high” or “advanced” benchmarks; in eighth grade, just 30 percent of students did.  Furthermore, twice as many American students – 8 percent – failed to meet any benchmarks in eighth grade than in fourth grade.  In Singapore, however, the number of students meeting the “advanced” or “high” benchmark holds steady at 78 percent in both grades, and the other East Asian countries also do not lose any substantial ground.  Taiwan increases the number of students at the “advanced” level from 30 percent in fourth grade to about half (49 percent) in eighth grade.

In fourth grade science, students at the “low” benchmark “show some elementary knowledge of life, physical and earth sciences,” and “demonstrate knowledge of some simple facts … interpret simple diagrams, complete simple tables, and provide short written responses to questions requiring factual information.”  At the “advanced” benchmark, students can “apply knowledge and understanding of scientific processes … and show some knowledge of the process of scientific inquiry.”  Additionally, “they have a beginning ability to interpret results in the context of a simple experiment, reason and draw conclusions from descriptions and diagrams, and evaluate and support an argument.”

On the TIMSS fourth grade science assessment, the East Asian countries do not dominate in terms of student proficiency at the “advanced” benchmark as completely as they do in math, although perennial top performers South Korea and Singapore still top the list in this measure.  Fewer students overall, across the board, seem to have reached the “advanced” benchmark in science as compared to reading and math.  The United States seems to have a particular problem in this subject, with 19 percent of students either failing to meet any benchmark or only meeting the “low” benchmark.

At the eighth grade level in science, students meeting the “low” benchmark are expected to “recognize some basic facts from the life and physical sciences,” and can display this knowledge by “interpret[ing] simple diagrams, complet[ing] simple tables, and apply[ing] basic knowledge.  Students at the “advanced” level can “communicate an understanding of complex and abstract concepts in biology, chemistry, physics and earth sciences.”  They also “understand basic features of scientific investigation … [and] combine information from several sources to solve problems and draw conclusions, and … provide written explanations to communicate scientific knowledge.”

Like in fourth grade science, overall, there seem to be fewer students who reach the “advanced” benchmark across the board.  The United States sees a 5 percent decline in the number of students reaching the “advanced” benchmark from fourth to eighth grade, and a four percent decline in students reaching the “high” benchmark.  This is compounded by a large jump in the percent of students who either do not meet any benchmarks (7 percent compared to 4 percent) or meet only the “low” benchmark (20 percent compared to 15 percent) – more than a quarter of all US students, in fact.

A separate, but equally interesting, set of data from the 2011 PIRLS results is the level of proficiency of students in two types of reading – literary and informational – as compared to a country’s overall score.  Debates over the value of each type of reading as emphasized in a curriculum have been raging for some time now, and while the PIRLS data does not solve this debate, it does provide interesting new fodder to the discussion.

The chart above depicts the overall average reading score on PIRLS, which is administered to fourth grade students, for the top fifteen systems on that assessment, as well as the average score on the literary reading tasks and on the informational reading tasks.  The top performing countries (Hong Kong, the Russian Federation, Finland and Singapore) all have average informational reading scores that are higher than or equal to their overall reading score, with literary reading scores somewhat lower than or equal to both the overall score and the informational score.  By contrast, the United States, Northern Ireland, Denmark, Ireland, Canada and England all display the opposite trend – literary reading scores that are higher, often statistically significant, than either their informational reading scores or their overall scores.  There is also, in the case of the United States, Ireland and Northern Ireland, a statistical significance in the difference between the lower informational reading score and the overall score.

This suggests that informational reading may, in fact, help aid a student’s overall reading skills, at least as measured by the PIRLS assessment.  It is notable that several East Asian countries, including Singapore, Hong Kong, and Taiwan, all of which traditionally do very well in the math and science assessments, also have students who perform better on informational reading tasks than on literary reading tasks.  In the case of Hong Kong and Singapore, this results in a very high overall score.  In Taiwan, the informational reading score is extremely high compared to the literary reading score, and actually fairly comparable to Singapore’s informational reading score.  However, in this case the literary reading score of Taiwan’s students brings the overall score down, suggesting a need for balance.  In terms of balance, Finland seems to have gotten this just right; the informational, literary and overall scores are indistinguishable from one another, and are all very high.