In December, the results of the 2011 administration of the Trends in International Mathematics and Science Study (TIMSS) and Progress in International Reading Literacy Study (PIRLS) were published in three separate reports, each examining international performance in reading (at the fourth grade level), math (at the fourth and eighth grade levels) and science (at the fourth and eighth grade levels).  These assessments provide a picture of international student performance in the years before a student reaches the age of 15, which is the age at which students take the OECD’s Programme of International Student Assessment (PISA).  However, there are some central differences between the TIMSS/PIRLS and PISA assessments.  Michael Martin, the Co-Executive Director of the TIMSS and PIRLS Study Center at Boston College, notes that while PISA is intended to measure a student’s general skills in the arenas of reading, math and science, TIMSS and PIRLS are more focused on content mastery.  Additionally, Jack Buckley, the commissioner of the National Center for Education Statistics, has pointed out that the countries participating in both assessments do vary – the TIMSS and PIRLS groups are smaller and represent a mixture of countries at different levels of economic development as compared to the participants in PISA.

Because of the differences between the assessments, the countries that are in the top ten or fifteen of the TIMSS and PIRLS rankings are somewhat different than the top performers on the last incarnation of PISA in 2009.  While league tables of the top countries based on their average scores always garner the most press when the results of international assessments are released, we decided to take a more in-depth look at what level of proficiency students in the top fifteen countries are actually reaching in these subjects.

The IEA has established four “international benchmarks” on their score scale for these assessments.  While the score scale for both PIRLS and TIMSS runs from 0-1000, the vast majority of scores fall between 300 and 700.  The IEA has identified a score of 400 as the “low” international benchmark, indicating that students at this score point have been educated to a “basic” level.  Beyond that, there is a score of 475, or “intermediate;” a score of 550, or “high,” and a score of 625, or “advanced.”  Below, we have plotted the percent of students at each benchmark in the top fifteen countries on the 2011 administration of PIRLS and TIMSS.  This is useful when thinking about the top performers, because it shows, in a clearer way perhaps than the average scale score, what students in each country are really able to do.

In the fourth grade PIRLS reading assessment, a student who reaches the “low” international benchmark is able to “locate and retrieve an explicitly stated detail” in a literary text, and “locate and reproduce explicitly stated information … at the beginning of the text” in an informational text.  By contrast, at the “advanced” international benchmark, students are able to “integrate ideas and evidence across a text,” and “distinguish and interpret complex information from different parts of a text,” among other skills.

The chart above, like the others to follow, is organized from top to bottom in the order of average scale score.  However, the average scale score does not always correlate to the highest percentage of students reaching the “advanced” benchmark in each country.  In this case, it does not, though Hong Kong does have the highest proportion of students meeting either the “high” benchmark or “advanced” benchmark – 67 percent – while in the United States, just 56 percent of students meet those levels.  The tail of students either meeting the “low” benchmark or not meeting a benchmark is also significantly smaller in the top three countries – Hong Kong, the Russian Federation, and Finland (7, 8 and 8 percent, respectively), than in the majority of the other countries.  This more specific data on student performance is useful in terms of thinking about a country’s overall performance, because it gives a clearer sense, potentially, of the equity of the school system, and the ability of the system to educate all students – or any students – to high levels.  It also demonstrates that there are clear differences in student performance between the top handful of countries and the rest of the countries rounding out the top ten or fifteen.

For fourth grade math, in order to reach the “low” benchmark, a student must be able to demonstrate “basic mathematical knowledge,” such as adding and subtracting integers and being able to recognize familiar shapes.  At the “advanced” benchmark, a student must have an understanding of how to apply their knowledge, for example, by solving word problems with multiple steps, and they must show some understanding of more difficult concepts like fractions and decimals.

In the case of TIMSS fourth grade math, the percent of students reaching the “advanced” benchmark does correlate to the country’s average scale score, at least for the top six performers.  This chart indicates very clearly how well the East Asian countries do compared to the rest of the world in instilling advanced-level math skills in their students, even at an early age, with about a third of students or more reaching the “advanced” benchmark in Singapore, Korea, Hong Kong, Taiwan and Japan, and an overwhelming majority reaching either the “advanced” or “high” benchmarks in all cases.  These countries also have the smallest proportions of students who failed to meet the most basic level.  By contrast, starting with Northern Ireland, which is in sixth place in the overall league table in this subject, the other countries have higher proportions of students failing to reach at least the “intermediate” benchmark, and generally much lower proportions of students reaching the “advanced” benchmark.

In eighth grade math, students at the “low” benchmark “have some knowledge of whole numbers and decimals, operations, and basic graphs.”  At the “advanced” level, students are able to demonstrate many mathematical skills, such as solving linear equations, reasoning with geometric figures, and expressing generalizations algebraically.

The pattern in proficiency seen in the TIMSS fourth grade math results is continued in the TIMSS eighth grade math results.  Andreas Schleicher from the OECD and US Education Secretary Arne Duncan have commented on the drop in math and science skills from fourth grade to eighth grade in the United States, and the data bears this out.  In fourth grade, 47 percent of American students met either the “high” or “advanced” benchmarks; in eighth grade, just 30 percent of students did.  Furthermore, twice as many American students – 8 percent – failed to meet any benchmarks in eighth grade than in fourth grade.  In Singapore, however, the number of students meeting the “advanced” or “high” benchmark holds steady at 78 percent in both grades, and the other East Asian countries also do not lose any substantial ground.  Taiwan increases the number of students at the “advanced” level from 30 percent in fourth grade to about half (49 percent) in eighth grade.

In fourth grade science, students at the “low” benchmark “show some elementary knowledge of life, physical and earth sciences,” and “demonstrate knowledge of some simple facts … interpret simple diagrams, complete simple tables, and provide short written responses to questions requiring factual information.”  At the “advanced” benchmark, students can “apply knowledge and understanding of scientific processes … and show some knowledge of the process of scientific inquiry.”  Additionally, “they have a beginning ability to interpret results in the context of a simple experiment, reason and draw conclusions from descriptions and diagrams, and evaluate and support an argument.”

On the TIMSS fourth grade science assessment, the East Asian countries do not dominate in terms of student proficiency at the “advanced” benchmark as completely as they do in math, although perennial top performers South Korea and Singapore still top the list in this measure.  Fewer students overall, across the board, seem to have reached the “advanced” benchmark in science as compared to reading and math.  The United States seems to have a particular problem in this subject, with 19 percent of students either failing to meet any benchmark or only meeting the “low” benchmark.

At the eighth grade level in science, students meeting the “low” benchmark are expected to “recognize some basic facts from the life and physical sciences,” and can display this knowledge by “interpret[ing] simple diagrams, complet[ing] simple tables, and apply[ing] basic knowledge.  Students at the “advanced” level can “communicate an understanding of complex and abstract concepts in biology, chemistry, physics and earth sciences.”  They also “understand basic features of scientific investigation … [and] combine information from several sources to solve problems and draw conclusions, and … provide written explanations to communicate scientific knowledge.”

Like in fourth grade science, overall, there seem to be fewer students who reach the “advanced” benchmark across the board.  The United States sees a 5 percent decline in the number of students reaching the “advanced” benchmark from fourth to eighth grade, and a four percent decline in students reaching the “high” benchmark.  This is compounded by a large jump in the percent of students who either do not meet any benchmarks (7 percent compared to 4 percent) or meet only the “low” benchmark (20 percent compared to 15 percent) – more than a quarter of all US students, in fact.

A separate, but equally interesting, set of data from the 2011 PIRLS results is the level of proficiency of students in two types of reading – literary and informational – as compared to a country’s overall score.  Debates over the value of each type of reading as emphasized in a curriculum have been raging for some time now, and while the PIRLS data does not solve this debate, it does provide interesting new fodder to the discussion.

The chart above depicts the overall average reading score on PIRLS, which is administered to fourth grade students, for the top fifteen systems on that assessment, as well as the average score on the literary reading tasks and on the informational reading tasks.  The top performing countries (Hong Kong, the Russian Federation, Finland and Singapore) all have average informational reading scores that are higher than or equal to their overall reading score, with literary reading scores somewhat lower than or equal to both the overall score and the informational score.  By contrast, the United States, Northern Ireland, Denmark, Ireland, Canada and England all display the opposite trend – literary reading scores that are higher, often statistically significant, than either their informational reading scores or their overall scores.  There is also, in the case of the United States, Ireland and Northern Ireland, a statistical significance in the difference between the lower informational reading score and the overall score.

This suggests that informational reading may, in fact, help aid a student’s overall reading skills, at least as measured by the PIRLS assessment.  It is notable that several East Asian countries, including Singapore, Hong Kong, and Taiwan, all of which traditionally do very well in the math and science assessments, also have students who perform better on informational reading tasks than on literary reading tasks.  In the case of Hong Kong and Singapore, this results in a very high overall score.  In Taiwan, the informational reading score is extremely high compared to the literary reading score, and actually fairly comparable to Singapore’s informational reading score.  However, in this case the literary reading score of Taiwan’s students brings the overall score down, suggesting a need for balance.  In terms of balance, Finland seems to have gotten this just right; the informational, literary and overall scores are indistinguishable from one another, and are all very high.

Starting Strong III: A Quality Toolbox for Early Education and Care provides ECE recommendations for countries in five policy areas including:

• Setting out quality goals and regulations;
• Designing and implementing curriculum and standards;
• Improving teacher qualifications, training and working conditions;
• Engaging families and communities; and
• Advancing data collection, research and monitoring.

In conjunction with the Starting Strong series, the OECD has begun publishing a number of country-specific reports on ECE.  Each of these country reports is organized around one of the policy levers identified above, depending on what the country has prioritized in its ECE agenda.  So far the OECD has released country studies for Finland, the Slovak Republic, United Kingdom, the Czech Republic, Korea, Portugal and New Zealand.  We feature Finland, Korea and New Zealand here.  Each is a top performer in the education league tables.

New Zealand and Korea both focused on implementing curriculum standards.  Creating and implementing a common curriculum framework and learning standards is just as important in early childhood education as it is for compulsory education systems.  The framework can ensure quality across different settings and can promote continuity between ECE and primary schooling.  A well laid out curriculum framework with accompanying learning standards can help teachers prepare lessons and give parents direction on how to develop a stimulating home learning environment.

Finland focused on improving qualifications, training and working conditions for ECE professionals.  One of the factors that matters most in early childhood education is the quality of the workforce, measured by their initial education, qualifications and professional development.  The OECD is careful to note that it is not qualifications per se that have an impact on child outcomes, but the ability of better educated staff to create a high-quality learning environment.  Just as in compulsory schools, better working conditions have been shown to improve staff job satisfaction and retention.  The OECD identifies good early childhood education working conditions as high staff-child ratio and low group size, competitive wages and other benefits, reasonable schedules and workloads, a good physical environment and a competent and supportive manager.

So what are the strengths of the early childhood services offered by Korea, Finland and the Netherlands?  New Zealand has created a common, national curriculum framework for early childhood education providers called Te Whāriki in New Zealand.  This document clearly lays out the aims of ECE including what is expected of staff and children at each stage of development along with useful examples.  The curriculum strongly focuses on well-being and learning and emphasizes the importance of tolerance and respect for cultural values and diversity.  The Te Whāriki also provides explicit links to the primary school curriculum, describing what children are expected to do at future levels, how this relates to the experiences in ECE and what activities staff can implement to facilitate a smooth transition. While many countries still use a split system where child care and early education are governed by different ministries or agencies, New Zealand has integrated early childhood education and care under one lead ministry.

Korea, too, chose to focus on curriculum.  This country has created the Standard Child Care Curriculum, which covers children ages zero to five.  Implementation began in 2007 and was revised in 2010 to improve the quality of child care services, extend operating hours to accommodate family needs and strengthen the link between child care and elementary schooling.  In addition to this child care curriculum, Korea has developed a National Kindergarten Curriculum for children ages three to four. Based on research undertaken in 2010, this document provides common standards for organizing and implementing the kindergarten curriculum and places a heavy focus on creativity and character.  In September 2011, the Ministry of Education, Science and Technology and the Ministry of Health and Welfare developed and launched the Nuri Curriculum for all five-year old children participating in early childhood education and care.  It is focused on five distinct objectives: developing basic physical abilities and establishing healthy and safe routines; learning how to communicate in daily life and developing good practices in language use; developing self-respect and learning how to live with others; developing interest in aesthetics, enjoying the arts and learning how to express yourself creatively; and exploring the world with curiosity and enhancing children’s abilities to solve problems by applying math and science in daily life.  Starting in March 2013, the government has plans to extend this curriculum to three- and four-year olds, which is a step towards streamlining the overall ECE curriculum framework in Korea.

Choosing to focus on the early childhood education and care workforce made sense for Finland, a country that puts a strong emphasis on recruiting, hiring and supporting the ECE workforce.  The qualifications for teaching staff, professional development opportunities and favorable working environments make Finland’s ECE workforce one of the best in the world.  Finland requires ECE teaching staff to have at least some post-secondary education as in the case of New Zealand and Sweden.  Professional development is mandatory and individuals do not have to shoulder the full costs as the government and the employer contribute.  The maximum number of children per early childhood professional in Finland is among the most favorable in the OECD with one staff member to four children ages zero- to three and 1:7 for older children in early childhood education or care.  New Zealand has slightly less favorable minimum ratio standards and Korea, at the other end of the spectrum, allows a 1:25 ratio for four-year olds.

In each of the Quality Matters studies, the featured country is evaluated against how it has responded to a number of challenges that commonly arise in the selected policy area of focus.  In enhancing ECE curriculum, those common challenges include defining goals and content; aligning curriculum for continuous child development; implementing effectively; and evaluating systematically.  Korea has made some progress in tackling these challenges.  To develop the Nuri curriculum the government formed a task force, including stakeholders from early childhood education and childcare sectors and ministry officials, charged with collaborating on the design and content of the curriculum.  To help ease implementation efforts, Korea held large-scale public hearings and seminars before and after announcing the revised versions of the National Kindergarten Curriculum and the Standard Childcare Curriculum.  Twenty thousand ECE professionals were trained in 2011 to implement the Nuri Curriculum in 2012.  The OECD suggests that the country could further enhance quality in its ECE agenda by developing one curriculum for children in the whole ECE range and ensuring that assessment practices meet the aspirations of the curriculum.

New Zealand has also made significant headway in facing these common curriculum challenges, most importantly by covering the entire early childhood education and care age range as an integrated system with one national framework.  The Te Whāriki is developed for children from birth to school entry but, to ensure the framework is age-appropriate, the content of the curriculum is divided into three age groups: infants, toddlers and young children.  To answer the evaluation challenge, New Zealand has implemented the Assessment for Learning, which requires teachers to develop effective assessment practices aligned to the curriculum.  The national government offers regular training on this practice.  The Te Whāriki states that “assessment of children’s learning and development should always focus on individual children over a period of time and staff should avoid making comparisons between children”. The OECD suggests that the Te Whāriki place a greater emphasis on strong communication skills for ECE staff so they can effectively work with colleagues on job issues and with parents on child development issues.

As already mentioned, Finland has made several efforts to answer the common workforce challenges highlighted by the OECD report (improving staff qualifications, securing a high-quality workforce supply, retaining the workforce, workforce development and managing the quality of the workforce in private ECE organizations).  Their responses include their efforts to set minimum qualification standards for ECE staff and to encourage professional development.  Additionally, in the mid 1990s, Finland moved kindergarten teacher education to the university level where classroom teacher training was already established.  Once kindergarten and primary teachers were trained, they were better able to support children’s transition from pre-primary to primary school.  The OECD made several suggestions to Finland.  First they observe that the country does not have licensing renewal requirements in place whereas staff in New Zealand must renew their license every three years.  Second, they recommend further developing leadership and computer skills for ECE staff.  And lastly they point out that Finland’s ECE workforce is highly female and the majority is above the age of forty.  An effort to attract more diverse and younger staff to the field is needed.

Additional country reports are expected for Canada, Japan, Norway and Sweden in late September 2012.  Starting Strong III examines ECE through a broad lens and provides a roadmap for anyone with a role to play in developing ECE policy.

In their recent report published in February of this year, the Grattan Institute examined the school systems of several East Asian countries with a view towards drawing policy recommendations from what they learned for Australia. In the report, titled Catching Up: Learning from the best school systems in East Asia (and featured in last month’s International Reads) the authors look to Hong Kong, Shanghai, Korea and Singapore and focus particularly on teacher education, professional development, and approaches to learning. It is the combination of these factors, the authors believe, that produces such impressive results whenever their students are compared to their counterparts in other countries on international assessments such as PISA.

One of the most striking findings in this report is the rate at which students in Singapore, Shanghai, Hong Kong and Korea are learning as compared to their counterparts in the United States, the United Kingdom and, of course, Australia. Using the conversion rates utilized by Thompson et al. in their 2010 ACER publication, Challenges for Australian education: results from PISA 2009: the PISA 2009 assessment of students’ reading, mathematical and scientific literacy (based on OECD analysis of PISA score levels and student competencies), the authors of Catching Up were able to produce a table demonstrating how many months ahead students in Shanghai, Hong Kong, Singapore and Korea were compared to students in the US, the UK, Australia and the EU21. A difference in reading of 39 points represents a full year of difference in learning; the number is similar in math (41 points) and in science (38 points). Thus the difference in student learning between the 2009 PISA top performer (Shanghai) and the bottom performer (Kyrgyzstan) is six to six and a half years of learning in all three subjects.

What emerges from viewing the PISA scores in this way is it shows how far ahead students in Shanghai are, even compared to their top-performing East Asian counterparts. In mathematics, students in Shanghai are more than two and a half years ahead of the average OECD student, with students in Singapore and Hong Kong about a year behind them. Students in the UK and the US, on the other hand, lag a few months behind the average OECD student. In science, again, students in Shanghai have a huge leg up on most others: they are nearly two years ahead of the average OECD student and at least half a year ahead of the other top performers, whereas in the UK, students have just a tiny advantage over the average OECD student, while the United States remains average. Finally, in reading, Shanghai is still the frontrunner by far, but the overall gaps are smaller. Students in Shanghai are “only” about a year and a half ahead of the average OECD student, and again about half a year ahead of the next-best top performer. The United States performs better, by a handful of months, than the average OECD student, while the UK is approximately on par with average.

A common perception about education in Asian countries is that students – and particularly teenagers like those tested in PISA – spend the vast majority of their time either in school or in “cram schools” in order to compete for spots at selective universities. While it is certainly true that the culture of “cram schools” persists in these countries, the authors of Catching Up argue that it is not the extra hours spent studying that lead to these massive gaps in student achievement between East Asian countries and other OECD countries. Instead, these gaps emerge from “effective education strategies that focus on implementation and well-designed programs that continuously improve learning and teaching” (12), which are in place in the top performing countries. As evidence for this statement, they point to Hong Kong, which leaped from 17th place in PIRLS to 2nd place in just five years. Cram schools and Confucian values, they contend, cannot explain that rise.  Neither can system size; although Hong Kong and Singapore are relatively small (Singapore has just under half a million students while Hong Kong has about 700,000), South Korea has 7.2 million students and is also a top performer. Rather, Hong Kong, like Singapore and other rapidly-improving East Asian countries, took it upon itself to implement a series of effective and well thought out reforms with a focus on teacher education, teacher professionalism, and funding equity to get to the top of the pack.

There are many different measures of teacher pay, from a strict comparison of actual salaries in USD PPP, to percentage of per capita GDP, to a comparison of teachers’ salaries with those of workers in other fields with the same level of education in a given country.  Looking at international comparisons of teachers’ salaries and workers in the same country with the same level of education, we find that across the board in top-performing countries, teachers make about what their counterparts with a similar amount of education make.  This is particularly true for upper secondary teachers, and in the Netherlands and Finland, these teachers actually make more than other similarly-educated workers.  Across the OECD as a whole, the proportion is smaller; upper secondary teachers make just over 80 percent of what other workers make, while primary and lower secondary teachers fall shy of the 80 percent mark.  In the United States, by contrast, upper secondary teachers make just over 60 percent of what similarly-educated workers make, while primary and lower secondary teachers make even less.  Other benefits aside, it is clear that providing teachers who could go elsewhere with comparable salaries does help to retain high-quality candidates.

It is difficult to discuss teachers’ salaries without a broader discussion of the overall cost of education systems, particularly because teachers’ salaries tend to represent a majority of the spending in most education systems.  Last month, the OECD posed the question of whether money buys strong performance on PISA in one of their “PISA in Focus” briefs.  What they found was that it is not how much a country spends, but how they spend it, that correlates to higher PISA scores.  They found that once countries spend more than  $35,000 on total student expenses from the ages of 6 to 15, any additional money spent does not seem to pay off in student performance.

One of the highest-spending countries, the United States, has one of the lowest average PISA reading scores, whereas Shanghai and New Zealand, economies which both spend less than half of what the United States spends on individual students, have far better student performance. However, there is clear correlation between investment in teachers’ salaries and PISA performance. Countries in which teachers have higher purchasing power, as measured by their salaries as a proportion of GDP, also tend to have much higher student performance on PISA, as indicated by the second chart.

In many of the countries, and notably in Korea, lower secondary, mid-career teachers are paid, on average, more than the average GDP per capita.  However, while in Korea teachers are paid twice the GDP per capita, they are paid only about 80 percent of what similarly-educated workers in other fields are paid, suggesting that while teachers are paid very well compared to the average worker, their pay is still some distance from that of the highest status professionals.  Overall, the data would suggest that nations that want a first class teaching force need to be prepared to pay enough to take compensation “off the table” as a major consideration for talented young people making career decisions, but need not pay at the top of the professional scale.