Backtracking from a proposal to do away with the General Certificate of Secondary Education (GCSE), the set of examinations that English secondary students take from age 16 on, Education Minister Michael Gove announced February 7 that the government would not establish the EBAC or English Baccalaureate, a plan that had been introduced by him in September.  Under pressure from Ofqual, the body that regulates school qualifications in England, along with teachers unions and school administrators, Gove dropped the plan to allow only one organization to produce exams for each subject instead of allowing many to do so, as has long been the practice.  But he will still push for some of the other elements originally found in his EBAC proposal.  He wants to get rid of coursework (work done on assignments given by teachers and graded by them) and modules (the ability for students to work on small chunks of curriculum and then be tested on those topics).  He wants to give exams at the end of two-years of study in the core subjects instead of at the end of one year, add extension papers in mathematics and science for “the brightest students”, and, most significantly, change the definition of school success when reported to the public through league tables.  The Minister has proposed to get rid of the current league tables where schools are evaluated based on the percentage of students with 5 or more GCSEs scoring at a C or above.  His plan calls for schools to be evaluated on two measures: the percentage of students passing English and mathematics and a report on the progress that students make from year to year in eight GCSE subjects.

In addition to the teachers unions, Ofqual and the school administrators association, the Education Select Committee in Parliament also weighed in against Gove’s proposal.  The Conservative PM that heads the committee argued that if the country were to institute the qualifications system Gove envisioned, which would have given students who did not complete their EBAC a “statement of achievement” rather than a qualification, those students would be given “a badge of failure” hurting less able students rather than helping them.

For more information on EBacc, visit the Department for Education website.  Gove’s latest proposal, which includes a new national curriculum emphasizing the traditional subjects taught in a more structured way, would go into effect in schools in 2015 with the first examinations to be given in 2017.

By Marc Tucker

My apologies to Maimonides.  But I would not blame you if you were perplexed about the recent dust-up after the latest PIRLS and TIMSS data came out.  Some of the best-known names in education research worldwide came out with guns blazing, mostly at one-another, in a rapid-fire exchange about what the numbers meant.  I thought some of you might welcome a guide to the shooters and the shots, and a bit of commentary on the profound meaning of it all.

Tom Loveless, the head of the Brown Center at the Brookings Institution jumped on the data to say that they called for a “rethinking of the Finnish miracle success story….If Finland were a state taking the 8^th grade NAEP [the sample survey used in the United States to monitor the progress of American students over time], it would probably score in the middle of the pack.”  Jack Buckley, Commissioner of the National Center for Educational Statistics in the U.S. Department of Education confessed that, “I’ve always been a little puzzled” by the high level of attention paid to Finland.  Well, so much for Finland!

Martin Carnoy and Richard Rothstein wrote an analysis of the data claiming to show that while reading achievement of American students on PISA was growing between 2000 and 2009, it was falling by an even larger amount in Finland.  Similarly, they said, in math, US students from the lowest social class were also gaining substantially, while scores of comparable Finnish students declined.  “This is surprising,” they said, “because the proportion of disadvantaged students in Finland also fell…” And they go on to say that, by their analysis, the achievement gap between the most and the least advantaged students in the United States is actually smaller than in “similar postindustrial countries, and often only slightly larger than gaps in top-scoring nations.”

Ha!  That means that the withering criticism showered on American schools for their poor performance was totally undeserved.  The problem, if there is a problem, lies not in the schools, poor Horatio, which have been doing a much better job than anyone has given them credit for, but in the enormous disparities in family income that have opened up in American society.  And Finland, according to this analysis, hardly deserves its status as the model that the United States should be adopting.

Not so fast, say Paul Peterson, Eric Hanushek and Ludger Woessmann.  Peterson is at Harvard, Hanushek at Stanford and Woessmann at the University of Munich.  The data, they say, don’t show that at all.  What they actually show is that, even if such corrections are made, American students at the top do not perform anywhere near as well as the students in the top performing countries, or, at least, not such a high proportion of them do.  Things are just as bad as they always said they were, and the need to turn up the heat on the schools to perform up to international standards is as great as ever.

But wait a minute, says Andreas Schleicher.  The Carnoy-Rothstein analysis depends, he said, on a challenge to the methods used by OECD-PISA to do its survey research, and that challenge, says Schleicher, just won’t hold up in court.  To which Carnoy and Rothstein said in reply to the reply, Oh yes it will.

So what is going on here?  Why are all these people so exercised about this data?  What are their agendas anyway?  Who is right and who is wrong?  Why does it matter?  And what does it mean?

I know that research is supposed to go where the evidence leads it and the researcher is only there to record the ineluctable result, without fear or favor.  But the reality is that researchers have values to support and reputations to protect, and their conclusions are more often than not influenced by both their values and the reputations they have established as a result of the policy positions they have taken.  So, perhaps it would help to sketch in the positions taken on the relevant issues by the people I have named.

It should surprise no one that spokespeople for the Brookings Institution and the United States National Center for Educational Statistics should be waiting to pounce on Finland and on the people who have used the Finns’ standing in the international league tables to make a case for using the educational strategies the Finns have embraced.  Both Brookings and a series of U.S. Department of Education research executives, some of whom have gone to Brookings when they left the Department of Education, have been deeply skeptical of international education benchmarking and ardent advocates of what they have described as the “gold standard” of education research, meaning the use of experimental research techniques as the only legitimate way to attribute cause in social research.  It is obviously impossible to randomly assign national “treatments” to national populations in the arena of education, so, from their point of view, all statements that this or that set of policies “causes” these or those national outcomes in the arena of education policy are necessarily suspect.

Brookings and the Peterson, Hanushek, Woessmann team are both strong supporters of charters and the introduction of market forces generally as school reform strategies.  Brookings, as well other Washington-based think tanks, are eager to deflate the recent enthusiasm for international education benchmarking in part because they fear that the close examination of the strategies used by the top-performing countries will show little evidence that charters or market strategies in general are effective strategies for raising student achievement at a national scale.

Peterson, Hanushek and Woessmann each have their own views on what is most important in education reform, but all are advocates of charters and reform agendas based on market forces, and all appear to believe that it will take fear of foreign competitors to put this reform agenda over the top in the United States.  They have also done research that they say supports their claim that market strategies do work in the top-performing countries.  Implicitly, then, they believe, unlike their Brookings colleagues, that it is possible to do rigorous research using comparative data gleaned from these international surveys that attributes cause and from which, therefore, it is possible to draw policy conclusions.  This team of researchers has consistently advanced the view, like my own organization, that economic ruin will be the fate of any nation that fails to hold its own in international education competition, though their prescriptions as to the most effective policy agenda are different from our own, based on the study of pretty much the same data.

But Carnoy and Rothstein come from a very different place.  They believe that the relatively poor performance of American students on the international surveys of student achievement is a function of the large and increasing disparity in incomes among Americans, in absolute terms and in relation to other countries.  They are outraged that organizations like my own and researchers like Peterson, Hanushek and Woessmann hold the schools accountable for poor student performance, when they think the fault lies not in the schools and teachers, but rather in a society that tolerates gross and increasing disparities in income among Americans.  They would have us focus on promoting policies that would result in a fairer distribution of income in the United States.

Which puts them in direct conflict not just with Peterson, Hanushek and Woessmann, but also with Andreas Schleicher, the driver of the whole PISA system at the OECD.  Schleicher’s primary framework for the analysis of the PISA data displays the country data on two axes, one for student achievement on the subjects assessed by PISA and the other for equity, the pattern of the distribution of results from the poorest to the best performers within countries.  Countries with short tails in that distribution are described as having high equity; those with long tails are described as having low equity.  Schleicher points out that the United States just barely escapes being among those countries in the worst quartile on both measures.  Another table in Schleicher’s slide deck shows that, when socio-economic status is held constant, the schools of some nations do a much better job than others of reducing achievement disparities among students.  Carnoy and Rothstein would take American teachers off the hook, saying that the performance of poor and minority students is actually improving, the gap is not so large as was thought, and the performance of poor and minority students in the top performing countries is actually declining.  To the extent there is a problem, it is a problem caused by socio-economic status of the students, not the teachers’ performance.  Schleicher would say, no, that is not so.  Even when we look at students from comparable socio-economic backgrounds, American schools do less to close the gap with the students from more favored backgrounds than schools in most other countries.  They cannot both be right.

So it is no wonder that Carnoy and Rothstein go after Schleicher and his data and methods with hammer and tongs.

So who is right and who is wrong here?  All of the people I have named are competent researchers from well-regarded institutions.  Just as each of these people have their own values and established positions on the relevant policy issues, the same is true of me and the organization with which I am associated.  Our analysis of the dynamics of the global economy strongly suggests that high wage countries like the United States will find it increasingly difficult to maintain their standard of living unless they figure out how to provide a kind and quality of education to virtually all their children that they formally thought appropriate only for a few.  And we also believe that the most likely source of good ideas for strategies that will enable them to do that is the countries that have already done it.  We think that whether the source of poor performance is mainly growing disparities of income or relatively poor performance of the education system, the dynamics of the global economy are unforgiving and countries like the United States do not have the option of saying that the educators can do nothing, that the only thing that will save us is income redistribution.  We do not think that the only way to learn what strategies are likely to work is research methods derived from the experimental sciences.  Indeed, we think that the record clearly shows that American business recovered from a devastating assault from Japanese firms in part by inventing and using the very method—industrial benchmarking—that we and others are now using in the field of education.

To me, the most important conclusion to be drawn from the debate whose contours I have just rather roughly outlined is that now, for the first time in the United States, the international surveys of student achievement really matter.  That is a big, big change.  It was not the case before that advocates of the most hotly debated education reforms in the United States felt that they needed to take the data from these surveys seriously, to defend their positions or to advance them.  Clearly, they do now.

The second point is that the data from the international surveys is being used to make points not about peripheral issues, but central issues.  It really matters whether the cause of the United States’ relatively low standing in the international league tables is income disparities among the students’ families or poor education in the schools.  It really matters whether or not countries like Finland have important lessons for the rest of the world.  It matters whether the survey methods being used by the organizations that design and administer them bear up to scientific scrutiny or not.  And, lastly, it also matters whether the methods used by those who do research comparing the effects of different policies and practices on student achievement in multiple countries have enough scientific merit to justify their use by policy makers to make national policy. These are consequential questions.  This is the first time that we have seen a sustained debate by some of America’s leading scholars on these matters.  It is not likely to be the last, and that appears to herald an era in which, for the first time in the United States, international surveys of student achievement are likely to take a prominent place in the public debate about education policy.
You may be wondering where I come out on the welter of claims and counterclaims I described above.  Now that I have laid my analytical framework on the table along with those of the other analysts, you are in a position to apply the same dose of skepticism to my conclusions as I urged you to apply to the others.   My take on the data we now have in hand is more or less as follows.

First, the usual note of caution.  One snapshot does not a movie make.  We should not declare a trend before we have more than one data point.  So we might want to see whether the changes in rankings suggested by the recent PIRLS and TIMSS data hold up over time.

Second, as many have pointed out, TIMSS and PIRLS put the accent on measuring how students do on what amounts to a consensus curriculum.  Did they learn what international experts think they should have been taught in the subjects they assess?  PISA measures the capacity of students to apply what they have learned in the classroom to proxies for real-world problems of the sort they might actually encounter outside the classroom.  I have a strong preference for the latter goal over the first, which mainly comes from an experience I had years ago, when Archie Lapointe, the director at that time of the Young Adult Literacy Survey, told me the following.  The survey asked the young people surveyed to add a column of figures and take a percentage of the result. Almost all could do it.  It also asked the same respondents to take a restaurant check, add up the items, get a total and calculate a tip.  Very few could do it.  Like Alfred North Whitehead, I have very little use for what he called “inert knowledge.”

Third, we need to keep in mind that the fine-grained distinctions in the rankings, for most countries that are near one another, are not statistically significant.  What we should really be paying attention to is the groupings of countries in the rankings, when countries are grouped in such a way that the measured differences among the groups are statistically significant.  If you look at it from this perspective, what we see is the United States still has a long way to go before the vast majority of its students score in the front ranks of performance at many grade or age levels in many subjects, which is how I would define top performers.

Fourth, I think it is pretty clear from the OECD data that smaller proportions of American students score in the higher deciles of performance on the PISA tests, and more in the lower deciles than is the case for students from the top-performing countries.  If that is true, then it cannot also be true that the United States would do as well as the top-performing countries if only the poor, Black and Hispanic students were taken out of the rankings, as many American teachers and some policymakers maintain.  It is also clear from the OECD-PISA analysis, as I pointed out above, that, when the data are corrected for students’ socio-economic status, American schools are less effective than the schools of most of the countries measured at closing the gap between these students and students with higher socio-economic status.

This, of course, is not where Carnoy and Rothstein come out, but I think Andreas Schleicher won the battle between him, on the one hand, and Carnoy and Rothstein on the other.  But don’t take my word for it.  Read the claims and arguments made by both sides carefully.  There is a lot at stake in this conflict.

So, what then are we to make of the fact that, if Massachusetts, North Carolina and Florida were countries, they would have done very well indeed in the most recently released rankings?

The case of Florida, I think, is pretty straightforward.  The Florida Center for Reading Research, administered by Florida State University, is one of the nation’s leading centers for reading research.  Its methods are widely admired throughout the United States.  The state of Florida has managed to leverage this research program and its key figures to produce widespread implementation throughout the state of the methods advocated by the Center.  We can see the results in the PIRLS fourth grade reading results.  The question, of course, is what effect, if any, this will have on student performance in the upper grades as the students who have benefitted from these programs mature through the years.  That story has yet to be told.

In North Carolina, we are looking at a program of education reform that began with Governor Terry Sanford, whose first term as governor began in 1961.  Sanford’s unrelenting emphasis on improving education in the state laid the base for Governor James B. Hunt, Jr., who served as governor from 1977 to 1985 and again from 1993 to 2001, making him the longest serving governor in the state’s history.  Through that whole period, he never lost his focus on education as the key to the state’s economic growth, and, during that period, North Carolina showed more progress on student achievement as measured by the National Assessment of Education Progress than any other state in the United States.  Hunt’s agenda for education reform was profoundly affected by what he was learning about the strategies adopted by the top-performing countries in the world.  Like them, he focused on teacher quality, high quality instructional systems and early childhood education.  North Carolina was among the very first states in the United States to send delegations of key state policy-makers abroad to study the top performers.

Massachusetts is a similar story.  In this case the first phase of the reforms were driven by the business community, organized by Jack Rennie, a very successful businessman who worked hard to organize that community, and Paul Reville a public policy analyst.  They played the key role in pushing the Massachusetts Education Reform Act of 1993 through the legislature.  The Act provided hundreds of millions in new funding for the schools in exchange for explicit performance standards for students, set to international benchmarks and carefully drawn curriculum frameworks, also set to international benchmarks; a new comprehensive assessment system set to the standards and curriculum frameworks; much tougher standards for getting to be a teacher, intended to greatly ratchet up teachers’ command of the subjects they intended to teach, and a system to disclose student performance, school by school, with results reported by student subgroups, so that poor performance by these subgroups would not be hidden in the average scores for the school.  Right after the Act was passed, David Driscoll, until then the Deputy Commissioner of Education, was made Commissioner and remained in that position for ten years.  Under Driscoll’s leadership, Massachusetts, despite a great deal of pressure to do so, never backed off of its decision to set and to maintain internationally benchmarked standards, for both student performance and teacher certification.  After Driscoll left, the new governor created a new position in state government, to provide leadership to all the parts of government concerned primarily with education at all levels.  He filled that position with Paul Reville.  Between them, Driscoll and Reville provided the same kind of strength and continuity of leadership that Governor Hunt provided in North Carolina, and for a very similar agenda, an agenda that is in many respects consistent with our own analysis of the strategies used by the top performing nations to get to the top of the league tables.

You may or may not agree with my analysis of the kerfuffle over the release of the TIMSS and PIRLS results.  You may or may not agree with my explanation for the rise of Florida, Massachusetts and North Carolina on the PIRLS and TIMSS league tables.  But, in any case, I urge you to look at the contending papers, and come to your own conclusions.  All of us could benefit greatly from a long, loud, contentious effort to define what it means to be educated, and to better understand why some nations are more successful than others at educating the vast majority of their young people to whatever standard they choose.

* This is a second version of the original post for this month.  We misstated the conclusions presented by Martin Carnoy and Richard Rothstein in the report described in this newsletter.  We believe we have stated those conclusions accurately here, and apologize to the authors for the error.

For the record, however, the version of the Carnoy-Rothstein conclusions that we based our first statement on was itself based on the version of the report that Carnoy and Rothstein originally released, which claimed that their re-estimate of United States PISA scores would result in the United States ranking 4^th among OECD countries in reading, and 10^th in math, a major revision upwards of the US PISA rankings.  In their most recent version of their report, released last week, Rothstein and Carnoy revised these numbers downward somewhat to 6^th in reading and 13^th in math, but, as the post points out, even these numbers are contested.

Some highlights from the conversation follow:

Currently the United Kingdom has a coalition government, which is important context for understanding the education policy environment they find themselves in.  The coalition is made up of Conservatives and Liberal Democrats.  According to Sanders, the Conservatives support rigorous standards accompanied by more traditional teaching methods, while the Liberal Democrats are focusing in on social mobility, the achievement differences among different groups of students, why certain cohorts underperform, and how they can close the attainment gap.  With both parties forming a coalition in Parliament, they have worked to develop consensus toward a number of education reforms.

The major reform that the government has recently proposed is abolishing the GCSE (General Certificate of Secondary Education), the qualification that students in England work toward in school from ages eleven to sixteen.  Sanders mentioned that for some time now, there has been a debate about grades on GCSEs going up each year for more and more students.  There has been a question about whether this is because of grade inflation or if these improvements really reflect changes in teaching and outcomes.  England has a number of examination boards, or organizations that deliver the syllabi, curriculum frameworks, examinations and scoring of GCSE examinations to the schools.  They compete with each other by subject and schools can choose which examination board they want to use.  Many in government believe that the competing examination boards are pushing grade scales down and giving schools too much help and leading schools and teachers to choose the easiest “Board” in any given subject for their school so that they can meet their performance goals and do well on league tables.  The government, in order to tackle what it believes is grade inflation, the dumbing down of courses and exams, and to compete with the world’s best, has proposed to reform its examination system.

The government’s solution is to replace the GCSEs with what it is calling the English Baccalaureate Certificate (EBAC).  The new system was announced in September by Education Secretary Michael Gove.  Here are some highlights:

Beginning in the 2015 school year, students will begin new programs of study and then take examinations in 2017 in English, math and science.  These include exams in 7 subtopics: English Language, English Literature, pure math or applied math, biology, chemistry and physics.  Beginning in 2016 new courses will be offered in history, geography and foreign languages with the first exams given in 2018.  To get an EBAC qualification, students will have to succeed in the six core subjects: English, math, two sciences, a foreign language, history or geography.

All students in England will take the new exams, although some students who are struggling may be able to delay taking them until they are 17 or 18 years old and will receive a Certificate of Achievement if they do not meet EBAC standards.  The current A* to G grading system will also be revised and there will no longer be two tiers of exam papers, only one that measures the full ability range.  Coursework, or commonly assessed tasks under controlled conditions, will be phased out in most subjects.  After competing through an open competition, each subject will be provided by one examination board for an initial five- year period.

The new EBAC qualification will be different from the current GCSE in a number of other ways as well.  These include moving away from modular courses, where some courses currently are broken down into smaller units of study that students can take over again if they do not do well, into exams taken only at the end of two years of study.  Tests will be much longer at approximately three hours per exam rather than the current ninety-minute GCSE tests.  There will be a strong focus on grading for spelling, grammar and punctuation.  There will be more emphasis on algebra in math exams and more full-length essays in English.  Six hundred thousand students will be impacted by this new system.

Critics of the government’s EBAC proposal worry that the new system will be a return to the two-tiered system represented by the old O- and A-levels, providing little chance for students that need more help to get it.  They also argue that the move “back to basics” does not reflect the needs of a 21st century economy.  A case in point is that the required EBAC subjects leave out the arts.  Critics say that this will erode England’s creative economy.

Since announcing the new EBAC proposal in September, the government has released a consultation paper laying out, in detail, the proposed changes and is asking for comments from the public.  The consultation paper is available through December 10, 2012 after which some changes may be made to the original proposal.  Not all political parties in England support the move away from GCSEs to EBACs including the Labour Party.  Ultimately, if there is a change in government in England, EBAC plans may not go ahead, but the examination boards are already gearing up for the competitive process to determine who wins each subject.

Though the GCSE’s are used only in the UK, there is an international version of the GCSE’s called the International General Certificate of Education (IGCSE) that is used by high schools all over the world, including, recently, the United States.  Some countries used customized versions of the IGCSE as their national examination system.  Although the changes just described to the GCSE need not change the IGCSE, they may lead to such changes and some countries may choose to make changes in their own system based on the changes in the GCSEs.

So Top of the Class will keep on eye on the progress of the EBAC, its impact on schools in England and continue to report on changes to the EBAC curriculum and assessments that may have implications for other countries.

Vivien Stewart, Senior Advisor for Education at Asia Society and CIEB International Advisory Board Member, reports on the proceedings of the inaugural meeting of the Global Cities Education Network, held in Hong Kong.

The 21st century will be the century of cities, according to some observers.  Today, half of all humanity lives in cities.  Massive migration from rural areas and internationally has made cities increasingly diverse, typically including multiple languages, ethnic and/or religious groups.  With rapidly growing populations of poor, often unskilled residents, aging populations to take care of, and overtaxed public services, large cities are the sites of societies’ greatest challenges.  But they also possess significant advantages in terms of wealth and of cultural and social opportunities.  They are the creative hubs of economies and societies, the dominant drivers of both U.S. and global economic growth.

As knowledge- and innovation-based economies become more dominant, a critical factor in determining cities’ future economic success will be the skills and talent of their workforces.  In a world that is increasingly interconnected, the opportunities for success will also require both individuals and cities to be able to compete and cooperate on a global scale.

It was these new challenges that brought cities from Asia, Australia and North America to Hong Kong for the inaugural meeting of the Global Cities Education Network.  Founded and convened by Asia Society, an international, non-profit educational organization, the Global Cities Education Network seeks to act as a mechanism for collaborative learning and problem solving between large urban schools systems.

In recent years, as the role of education in driving economic and social development has become ever more apparent, international benchmarking of educational best practices has become an increasingly valuable tool for policymaking.  However, these international education comparisons have hitherto been made primarily at the national level.  But while education policies are usually set at the national or state level, it is in cities that such policies are actually implemented in real schools and with real students.

So, teams of policymakers, practitioners and researchers from Chicago, Denver, Hong Kong, Melbourne, Seoul, Shanghai, Singapore, Seattle, Toronto and the charter network Ed Visions came together in the Global Schools Education Network to discuss the critical challenges they face, and to identify ways to learn from each other and from the world’s best practices.  This first meeting was, in a sense, an experiment.  Although it is clear that good ideas travel across cultures, these cities are very disparate.  The context in Seoul is not the same as in Chicago – would they be able to find common ground?

The participating cities discussed two critical sets of issues — achieving quality education for all students and retooling their education systems to develop the knowledge and skills needed in the 21st century.

ACHIEVING EQUITY AND QUALITY

The highest performing education systems are those that combine quality with equity.  In these systems, the vast majority of students have the opportunity to attain high levels of skills, regardless of their own personal and socio-economic circumstances.  Yet even in the highest performing systems, a significant number of students fail to achieve a minimum level of education.

Every city in the Global Schools Education Network is working to provide greater equity in its education system, some with more success than others.

A particular focus of the discussion among the participants at the meeting was on the increasing diversity of cities.  In Toronto more than 20 percent of the population have been born outside of Canada (and are referred to as “new Canadians”).  And despite the overall increase in student performance and secondary school graduation, there are still groups that are falling behind, especially black males, native Canadians, and students who have come from Latin America and the Middle East.  In Melbourne, 24 percent of students have one parent born overseas and 20 percent speak a language other than English at home.  In Shanghai and Hong Kong, massive migration from poor rural and inland areas poses challenges to the traditional schools.  And while Seoul’s diversity is small in scale (2 percent) compared to other cities, it is nevertheless challenges the traditional processes of the education system.

Most cities give more resources to schools serving disadvantaged students but quantity of resources may not be as important as the ability to have the best teachers working in these schools.  Recognizing that the quality of the teacher is the single biggest in-school factor affecting student achievement, the discussion also focused on how to get enough high-quality people to go into teaching and how to ensure that the neediest students have access to the highest quality teaching.  Some cities such as Singapore have done extensive work on developing a high-quality teaching profession; others have worked on specific aspects of the issue such as Shanghai’s efforts to get the best teachers into the weakest schools.  These efforts and others could be used to inform other cities.

Another trend in most of the cities was towards the greater provision of choice and options for different types of schools.  Singapore, for example, is developing portfolios of schools.  Melbourne has government, Catholic and independent schools.  In the United States, charter schools, like the Education Visions network, are increasingly part of the city mix.  Seattle has pushed a great deal of decision making to the school level, which has stimulated innovation but exacerbated inconsistent results.  In all the participating cities, the trend is towards greater decentralization of authority to the school level with just broad policies set at the city or district level.  However, choice and decentralization can lead to greater inequities if not designed with equity in mind.  So the challenge in running an effective urban system of schools is — what needs to be consistent across schools and where can flexibility be allowed?

Despite their challenges, cities also have many advantages.  Often the broader cultural and economic environment for education is more favorable.  And particular approaches such as choice among schools or professional learning communities among teachers are easier to implement in a city.  Indeed, an analysis conducted by OECD showed that in many parts of the world, cities outperform non-urban parts of their countries.

TRANSFORMING LEARNING: KNOWLEDGE AND SKILLS FOR THE 21st CENTURY

Around the world and certainly in each of the participating cities, there is a sense that the aims and processes of education in the 21st century need to be fundamentally different from those in the 20th.  No longer is providing basic literacy skills for the majority of students and higher order skills for a few an adequate goal.

Every participating city is engaged in or contemplating wide-ranging reforms of curriculum, instruction, and assessment to prepare students for the increasingly complex demands of life and work in the 21st century.

While there was real agreement among the cities on the general direction in which education needs to go, there are tremendous challenges of implementation and cities approach the task with different strengths and limitations.  Asian cities have developed highly effective systems for knowledge transmission, where all the elements of the system are aligned and produce high performance, but their pedagogy is more traditional.  Western cities, on the other hand, have a more developed tradition of constructivist pedagogy and more freewheeling societies.  They have schools that are renowned as “peaks of excellence” but they have been less effective in developing systems to get all students to high levels of achievement.

The rapid changes in knowledge today are also putting a greater premium on investing in lifelong learning, raising new questions not just about the goals and focus of schooling but also about how to distribute learning resources over the lifecycle.  Every city it seems faces critical challenges in trying to reduce the enormous gap between what modern societies and economies demand and what education systems currently deliver.

COMMON PRIORITIES

In the final sessions of the meeting, participating cities agreed on a number of key common priorities of policy and practice where international benchmarking efforts through the Global Cities Education Network would be particularly helpful.

1.    Developing High-Quality Teachers and School Leaders
Cities want to know how to improve their efforts to attract, hire, develop, evaluate and retain high-quality teachers – and to ensure that the most disadvantaged students have highly capable teachers.  Since cities differ in the degree of influence they have over certain aspects like teacher training and teacher distribution among schools, a range of strategies for improving quality and distribution need to be identified.

2.    Improving Achievement of Low-Achieving and Linguistically and Culturally Diverse Students
In every city, some groups of students still lag significantly behind.  And the increasing scale and complexity of diversity facing large cities makes improving policies and practices in this area an urgent priority.  Bringing together the best available international research with a comparative analysis of the approaches of selected cities could shed important light on how the achievement of these students can be improved and how cities can make their increasing diversity an asset.

3.    Implementation and Assessment of 21st Century Skills
Every city is trying to varying degrees to modernize the content, methods and outcomes of their education systems towards 21st century skills and learning environments.  Perhaps most strategic in terms of moving systems in this direction is the need to craft ways to better assess these skills.  An analysis of what different systems around the world are doing to measure different aspects of 21st century skills together with an examination of ideas from the world’s best research on measurement would be an important contribution to helping cities transform their systems in this direction.

4.    Effective Systems Design: Centralization, Decentralization and Choice
All the cities are moving away from top-down management, with its emphasis on tight prescription and uniformity of educational practice, to giving more autonomy to individual schools.  They are encouraging portfolios of different types of schools and providing more choices of educational paths to students, especially at the secondary level.  What needs to be centralized and what should be decentralized to address these challenges is a major issue of system design, one that every city is grappling with to varying degrees and would be another fruitful area for comparative work.

This is a shortened version of a longer piece initially published on the Asia Society website.  Click here to read the full report and to learn more about the Global Cities Education Network.

Indeed, economists know that deep recessions are typically occasions for transformations in national economies, and it looks as though this one is no exception.  Companies facing stunted demand and plenty of cash have used the opportunity to make major investments in automation.  That means that many of the jobs requiring relatively routine skills are never coming back and many of the jobs that are created as demand comes back will call for considerably higher skills.  This was the general direction before the Great Recession, but that trend, it seems, as been greatly accelerated by those events, ratcheting up skills requirements considerably.

The 2012 issue of Education at a Glance focuses, in particular, on the relationship of compulsory and higher education investments by nations to their economic outcomes noting that, despite the financial constraints on governments because of the recession, spending on education (both public and private) has, in many cases, increased across OECD countries.  The OECD considers secondary education to be the “baseline” qualification needed in today’s economy, with many of their indicators measuring the proportion of the population who hope to achieve or have achieved education beyond this baseline.

There are also a host of new indicators included in this year’s Education at a Glance.  Two are specifically directed at the relationship between the global economy and the education level of a population: how education influences economic growth, labor costs and earning power; and the extent of social mobility in each country studied.  The former supports the growing body of evidence about the importance of workers having some post-secondary education:  the OECD found that labor income growth among highly educated people has contributed to more than half of GDP growth in OECD countries, whereas workers with less than a secondary education actually serve as a drag on labor income growth.

As policymakers around the world have turned their attention to increasing post-secondary education attainment rates, students, too, seem to grasp the increasing importance of higher education in most OECD countries, with educational attainment levels on the rise across the board.  Across all OECD countries, an average of 31 percent of adults have completed a post-secondary education.  Canada is the only country where more than half of all adults (people aged 25-64) have some higher education.  While many countries have steadily increased their percentage of adults with a college degree, some have increased at a much faster rate.  From 2000 to 2010, Canada’s average annual growth rate was 2.4 percent, the United States’ was 1.3 percent, and Finland’s was 1.8 percent.  But, during the same time frame, Ireland, Luxembourg, and Poland experienced growth rates ranging from 6.9 to 7.3 percent. The OECD average growth rate is 3.7 percent.  South Korea is also moving up fast with an average annual growth rate of 5.2 percent and they lead the world in higher education attainment rates for their young adult population, with 65 percent of their people aged 25-34 years completing post-secondary education.  In other high performing countries, like Japan, students who hope to enter higher education are hobbled by high tuition and low student supports.

Another new indicator in this year’s edition of Education at a Glance asks to what extent does parents’ education influence access to tertiary education?  Not surprisingly, the data shows that if at least one parent has completed post-secondary education, students are more than twice as likely than the average student to attend higher education themselves, while students whose parents did not complete upper secondary education have a 44 percent chance of attending post-secondary education.  Some countries are better than others in creating a pipeline to post secondary education even for students whose parents did not complete upper secondary school. Students whose parents have low education levels have greater chances of attending higher education in Iceland, Turkey, Portugal, Ireland, the United Kingdom, Denmark and Sweden, where the odds of attending are greater than 50 percent.  While not all students in this group who attend higher education actually graduate, some of these countries are also adept at ensuring fairly high rates of completion.  In Australia, the tertiary attainment rate of students without highly educated parents is more than 40 percent; in Ireland, Iceland, Sweden, and Finland, about 30 percent of students from this group attain tertiary degrees.  The OECD average for this group is just 20 percent.  Many of these countries employ a number of strategies to strengthen the educational pipeline for students whose parents are less educated including providing equal access to high-quality K-12 educational experiences, offering robust student support systems such as college and career counseling, and maintaining reasonable college and university tuition costs.  Australia, perhaps the most successful country when judged by the metrics of both the participation and attainment of students whose parents have low levels of education, has annual tuition fees at public institutions of $4200 US per year (fairly high compared to other OECD countries, though not as high as in the United States or the United Kingdom), but more than 75 percent of students receive financial aid.  Sweden, which also has high rates of tertiary participation among students whose parents have low education levels, and a fairly high rate of completion among this group, takes a different tack, not charging tuition fees at all.

In the United States, the odds of a student going on to college if his or her parents have not finished high school is just 29 percent.  The odds are lower in just two countries, Canada and New Zealand (figure 2).  However, the figures may not be as dire as they seem at first reading.  In his webinar presentation prior to the launch of this year’s report, Andreas Schleicher, Deputy Director for Education and Special Advisor on Education Policy to the Secretary-General of the OECD, points out that the OECD did not count associate’s degrees in their analysis. When the definition of higher education is expanded, the United States most certainly will improve in this category; however, this still brings into question the quality of the post-secondary credentials that many young people in the United States acquire compared to those in other OECD countries.

Schleicher also mentioned that OECD researchers had initially hypothesized that high tuition would be a barrier to disadvantaged students attending higher education.  However, the evidence shows this is not always the case.  In countries with high tuition rates and strong student support systems (for example, widespread access to loans and grants and manageable debt repayment plans), high tuition does not generally seem to prevent students from pursuing higher education.  Schleicher highlighted the system in the United Kingdom, which bases student loan repayments on salaries.  If former students do not meet a certain income threshold, they are not required to repay their loans.  Therefore, pursuing higher education is less of a gamble for low-income students; when students do not have to worry about repaying their loans if they cannot find a job, more students are willing to continue their studies.  This is a notable lesson for other countries with high tuition rates and less forgiving repayment programs – Schleicher stated that the UK government has found that the social returns of investing in getting more students into higher education are worth the risk.

Other new indicators ask:

• What is the difference between the career aspirations of boys and girls and the fields of study they pursue as young adults?
• How well do immigrant students perform in school?
• How do early childhood education systems differ around the world?
• Who makes key decisions in education systems?
• What are the pathways and gateways to secondary and tertiary education?

A few highlights from these new indicators tell us that key decisions in education systems are least commonly made at the intermediate level of governance; 16 of 36 countries allow schools to make key decisions (and half of those decisions are made within a framework created by a more centralized structure), and 12 countries fall at the other end of the spectrum, with key decisions made at the central level.  The Netherlands is the most autonomous system in the OECD, with 85 percent of decisions made at the school level.  However, school autonomy does not necessarily predict student performance; the top performers are scattered across the spectrum.

Another indicator looks at secondary and post-secondary gateways in education systems.  Twenty-three of the 36 OECD countries require students to take examinations at the upper secondary level and 22 make those examinations a requirement for passing a grade, graduating from school or earning a certificate.  Very few, however, require national examinations in elementary school, with the United States, Indonesia and Turkey being the only exceptions.  This suggests that most OECD countries value mastery of content and skills at the upper secondary level as an essential component of their education systems.

Some of the top-performing countries in primary and secondary education seem to be slightly behind the curve when it comes to early childhood education – at least early childhood education supported by government expenditures – a point which was driven home in three of the four country notes.  Three countries rely on private investment to drive their systems.  In South Korea, for example, more than 80 percent of four-year-olds are enrolled in a preschool program, but the vast majority (79 percent) attend private preschools, whereas across the OECD, just under 16 percent of students attend private ECE programs.  In Japan, preschool attendance for four-year-olds is nearly universal (97.2 percent), but spending on ECE is among the lowest in the OECD, and household spending makes up nearly 40 percent of all spending on ECE.  In Australia, both enrollment and spending lag; enrollment is the fourth lowest in the OECD at just 52 percent, and spending on ECE as a proportion of GDP is also among the lowest.

The annual publication of Education at a Glance once again serves as the go-to source for up-to-date information on the measurable features of education systems in OECD countries.  New indicators on higher education certainly make this year’s issue the most comprehensive yet.

Photo from the 2011 International Summit on the Teaching Profession

I left the second International Summit on the Teaching Profession not at all sure of what I had learned.  But, after a few days to sort it out, there is quite a lot.  Here it goes —

1.    Swiftly broadening goals

I was struck by the way many of the top-performing countries talked about their goals for their students.  Singapore is a good example.  Lee Sing Kong, the Director of Singapore’s National Institute of Education and a member of the Center on International Education Benchmarking (CIEB) Advisory Board, explained that the small country’s vision statement for its education system was completely overhauled recently.  They reminded themselves that what they do in education is for the learner, their needs, their interests, and not simply to cover the content.  They said they wanted to help their students achieve understanding of essential concepts and ideas, not just dispense information.  They want to prepare their students for the test of life, not just for tests.  They said they want to focus on teaching the whole child, on nurturing them holistically across domains, not on the subjects per se.  They want to teach their students the values, attitudes and mindsets that will serve them well in life, and not only how to score good grades on exams.

Shinichi Yamanaka, Deputy Minister of the Japanese Ministry of Education, Culture, Sports, Science and Technology, said that thirty years ago, when the economy shifted from mass production to high-value-added manufacturing and customized production, Japan decided that it had to overhaul its education system away from rote learning and toward the growth and development of the autonomous individual.  He said this was an enormous undertaking and, thirty years later, Japan is still figuring out how to accomplish it.

Zhang Minxuan, a key leader of Shanghai’s drive to the top of the education league tables, said that for more than a millennium, the Chinese people believed that if you could recite 300 famous poems, you could be a poet.  This led to the Chinese commitment to a regime of exams based on memorization and rote learning.  But, he said, they do not believe that anymore.  Their priorities are on thinking, problem-solving, preparing Chinese students to live in a highly integrated global environment and cultivating individual talent.

Mrs. Cherry Tse, the Permanent Secretary for Education for the Hong Kong Special Administrative Region, said that young people are now being exposed to masses of data on a scale not imagined by their parents and much now depends on being able to help them sort out the real information from “the crap.”  They need, as never before, to be “discerning,” to live and work effectively in a state of constant flux.  She worried that educators live in a sort of cocoon that will make it hard for them to prepare students for such a world.  Part of this worry comes from her belief that it is more important than ever for students in Hong Kong and elsewhere to develop a strong sense of empathy for people in other parts of the world, especially for those who are less fortunate.  The schools, she said, are not changing as fast as society, and that must be fixed.

Note that there is no mention in this litany of the importance of learning to read, to write and to have basic mathematical literacy.  This much is assumed.  Which is to say that the aim of education in the world’s leading mass education systems is no longer simply to provide basic literacy.  As these statements witness, the aim has now gone far beyond that to embrace what were, through the 20th century, the aims for only the elite.

2.    A focus on the distance between rhetoric and reality—or—between the political leaders and the teachers in the classrooms

I can hear you now dismissing the sentiments just expressed as the usual talk of politicians—vague aspirations that have little impact on what actually happens in classrooms.  Which leads to my second observation.  The very same people who made the observations just reported also worried about the distance between rhetoric and reality, which had the effect of making me believe that these people were expressing genuine ambitions for their education systems, ambitions they intended to make good on, to the best of their ability.  Some examples:

A representative of Norway pointed out that “we say we value 21st century skills but we test basic skills.  We are not testing what we say is most important.  I understand that the politicians need data for their purposes, but we need to be careful that their needs do not distort and distract from what is best for our students.”

A representative from New Zealand said, “We want our students to be capable, confident, creative and innovative, but all we measure is the first of these.  Some nations around this table put high stakes only on the first of these and some put high stakes only on language and math.”

Please note this last.  The international meetings I have gone to over the years have been models of politeness.  Statements of this sort are never made.  Because they are never made, the underlying issues are never raised, much less addressed.  But it did not end there.

3.    A focus on implementation

Mrs. Cherry Tse, from Hong Kong, said that it is now crucially important to align the goals of the system designers and managers [the government] with the goals of all the other participants, not least the teachers.  She mused about the difficulty of knowing whether government is doing the right (that is the moral) thing, and said that this is why it is so important to have a very inclusive discussion involving many stakeholders about the purposes of education to create as broad a consensus on that point as possible.

Kai-ming Cheng, one of the conference rapporteurs and a member of the CIEB Advisory Board, who is from Hong Kong, noted that the distance between the goals driving the system managers and the teachers tended to moderate in well-managed systems and be much larger and more problematic in systems that are not so well managed.

Ben Jensen, Director of the school education program at the Grattan Institute in Australia, observed that the expressed goals of many poor-performing national education systems are often very like those of successful systems.  He wondered whether what distinguishes the successful systems from the less successful systems might be the care and planning they put into implementation of their policies.  That is, he wondered whether it is execution, not intention, that separates the successful from the unsuccessful.

I am a veteran of many, many years of meetings at which senior representatives of national education systems have droned on and on, hour after hour about the virtues of their education systems and the wisdom of their plans.  This meeting was very different.  The room was full of people for whom their goals were not just rhetorical expressions of windy aspirations but statements of aims they knew to be very difficult to achieve that they were nevertheless working overtime to turn into reality.  They were quick to acknowledge their frustrations and concerns about their own plans.  They knew and quickly acknowledged the distance between their rhetoric and the reality on the ground.  They recognized that the only way they could bridge that gap was by paying far more attention in the future than they had in the past to the importance of execution, of making real changes happen on a very large scale on the ground in their schools.  And they were determined to pull that off.  They knew it would take a long time.  They came to this meeting to learn everything they could from their colleagues in other countries that would help them achieve their goals back home.

That was exhilarating.  And gave me more hope than I have had in some time.

I leave my readers around the world to ask themselves how their country fits into this account of the conversation at the second International Summit on the Teaching Profession.  How broad is your discussion of goals for students?  Can you assume that they will get the basic skills they need?  Or is that still an issue in your country?  Has your country really made the commitment to provide to all students the skills formerly thought appropriate only for a small elite?  Is your country’s education system still held captive to a high stakes accountability system driven by high stakes tests of the basic skills?  Is there a broad and deep consensus on a real 21st century conception of the goals of education?  Does your country acknowledge the distance between the aims of the designers and managers of the system and its teachers?  How large is that distance?  Does your country put as much energy and commitment into designing and carefully executing sound plans to implement your reform agenda as it does into its development, or does the old rhetoric fade into obscurity as the new rhetoric arrives to take its place?

In this section of our newsletter, we provide a summary of the key findings of the OECD report.  In our Statistic of the Month section, you can find graphic displays taken from this report and another recent OECD report that neatly summarize some of the most important findings and show how a number of the countries surveyed for the report sort themselves out along the dimensions of interest.  This article should be read in conjunction with those graphics.

Unsurprisingly, the report finds that certain investments in educational equity pay off.  They argue for early government investment in education, and maintaining a strong investment in the system through upper secondary school, with specific, targeted investments in and policies for low-performing schools. The authors recommend elimination of system practices that hinder education, among them grade repetition, early tracking, school-choice schemes that do not actually offer “choice” to all students, and “dead-end” upper secondary programs as opposed to academically demanding vocational tracks in upper secondary school that ensure that all students have a high quality pathway to the job market.  The authors also propose several broadly outlined strategies for improving low-performing schools with a particular emphasis on strong school leadership, highly-qualified teachers and strong links to the community.

The report points to countries that have been unusually successful at minimizing the effects of socio-economic status on school performance. Finland, in particular, and to a somewhat lesser extent Canada, South Korea and Japan, have all been able to help a high proportion of their students from low socio-economic backgrounds achieve at high levels, as measured by the 2009 PISA scores in reading. The OECD average reading score is far lower than it is in any of the countries just listed, while the average percent of variance in reading achievement among students from various social backgrounds is nearly twice as high as it is in Finland. Other countries, like the United States and New Zealand, have higher average PISA scores than the OECD average, but also a higher percent of variance in those scores – showing that those systems are both lower achieving overall and more unequal within their countries.

Other Recent Reports of Note

• The measurement of educational inequality: achievement and opportunity, The World Bank (publication date: November, 2011). The authors of this working paper examine educational inequality and measurement issues in international standardized assessments like PISA. They use these measurement issues to calculate inequality indices for 57 countries and provide results along with an analysis of whether education inequality correlates with factors like GDP, school spending and student tracking.
• Student Standardised Testing: Current Practices in OECD Countries and a Literature Review, OECD (publication date: October 11, 2011). This report discusses the most relevant issues concerning student standardised testing in which there are no-stakes for students through a literature review and a review of the trends in standardised testing in OECD countries. It provides an overview of the standardised testing typology in the no-stakes context, including identifying the driving trends behind the gradual increase in standardised testing in OECD countries and the different purposes of standardised tests.
• Teachers’ and School Heads’ Salaries and Allowances in Europe, 2009/10, Eurydice (publication date: October 4, 2011). This Eurydice data collection and comparative study on teachers’ and school heads’ salaries and allowances covers full-time, fully qualified teachers and school heads at pre-primary, primary, lower secondary and upper secondary education levels for the 2009/10 school year. The cross-country comparative analysis focuses on comparing the decision-making levels that are responsible for setting teachers’ and school heads’ statutory salaries. The minimum and maximum statutory salaries are presented relative to the GDP per capita in each country, with an indication of salary progression and its relation to professional experience. The latest increase/decrease in the purchasing power of personnel employed in education in relation to the impact of the economic crisis since 2008 is also analyzed. Finally, the different types of allowances that teachers may receive are presented as well as the decision-making levels responsible for their allocation and their levels.

NCEE: Can you give us an overview of the history of Australia’s National Assessment and curriculum efforts and what spurred on their development and the decision to create ACARA?

Barry McGaw:Australia has been involved in international studies of student performance since they began in the 1950s; however, the first national surveys of literacy and numeracy occurred in 1976.  The evaluations were sample-based only, a bit like NAEP in the United States.  And then various Australian states introduced sample-based surveys in other curriculum areas.

Barry McGaw

The first assessment of all students across the country in literacy and numeracy occurred in 1990 in New South Wales, which is the largest state in Australia. It was very controversial at the time; particularly with the teachers unions. So, New South Wales began testing all of their students in literacy and numeracy, and then Victoria, the second-largest Australian state, followed, and then other states gradually joined in. Western Australia participated with a really creative set of sample-based surveys that covered their entire curriculum, but only moved to test the entire student cohort when the then-federal Minister of Education made it a condition of federal funding.

By the mid-2000s, the state Ministers of Education decided it would be good to get results expressed in a way that made them comparable across the country. They instituted a process of creating a common scale allowing each state to continue administering its own assessments, but calibrated onto a common scale. After a couple of years, there was some concern about how good the calibration was, and they said, if we’re all testing in order to get the results on a common scale, why don’t we all use the same tests? And so from 2008, the six states and two territories in Australia have all used the same literacy and numeracy tests nationwide, also known as the National Assessment Program – Literacy and Numeracy (NAPLAN).

The next major event occurred in 2007.  During the federal election campaign, the opposition announced that it supported the establishment of a national curriculum in English, Mathematics, Science and History. They won the election at the end of 2007. At the beginning of 2008, they set up an interim national curriculum board. I was appointed chair, with the goal of developing national curriculum in English, Science, Mathematics and History. The government added geography and languages other than English to the curriculum development plans in mid-2008, and then the arts were added later.

At this point, the intention of the federal government was to discuss with the state governments what kind of governance arrangement would be instituted in the longer term for this body. There was some debate about whether it would be set up as a not-for-profit that the ministers collectively owned or a more formal statutory authority of the Australian parliament. The federal minister won out, suggesting that the Council of Ministers (the federal minister and the six state and two territory ministers) would be the policy board for this new body. In late 2008, Australia established the new Australian Curriculum, Assessment and Reporting Authority (ACARA) with responsibility not only for national curriculum development, the national literacy and numeracy assessments, but also the sample-based assessments already underway on a three-yearly cycle for science, civics and citizenship and ICT literacy. Finally the task of public reporting on schools was added to the portfolio.  That work has resulted in the creation of the My School website.

NCEE: You mentioned that these national assessments were controversial with teachers unions. Can you talk a little bit about what their concerns were, and how ACARA was able to address them?

Barry McGaw: The teachers unions were unhappy about the public nature of the reporting and this kind of external assessment. They typically argued that these types of assessments don’t tell us anything we don’t already know about our students. The response was, “you know how your students are doing in relation to one another, but you don’t know how your students stand compared to students across the country.” What typically wins the day in this debate is parents. Parents value information that shows them the bigger picture.  When the My School website came along, suddenly parents are not just getting reports on their own children, but they’re able to see, collectively, for all the students in their school how they’re doing in comparison with other schools. This information can be controversial, but to help on this front, we include information about the circumstances of the school.

Some other countries do what they call value-added analysis. Here is our approach. We obtain measures of students’ background, in particular on their parents’ education and their parents’ occupation. We’re not trying to measure socioeconomic status, we’re trying to measure socio-educational status – what are the benefits that kids get from the occupation and education of their parents as they come to school. We use this information to create an Index of Community Socio-educational Advantage. And then for every school, we look at just 60 schools – 30 schools immediately above it and the 30 immediately below it on that index. These are schools that have got essentially the same kinds of kids. These are the schools that you can learn from, that can challenge you. Now we can show a school another school with kids exactly like theirs that is doing much better. That school can then ask themselves, what policies they might consider that are different from the ones currently being used? What practices could the school adopt that might reproduce the kind of superior performance that you see in the higher performing schools? So that’s essentially the strategy of the My School website. Not only can it assist parents in their choice of schools, but it underpins attempts at school improvement.

NCEE: How are schools and teachers using the data to improve performance?

Barry McGaw: Well, it’s only been produced twice, so we know that within some of the state systems, they’re using it to help schools make these comparisons, but we don’t have much data ourselves on it yet. We know that huge numbers of people look at the site.  States do bring together small groups of schools to look at the data and analyze it. Every school in the country, or ten thousand schools, is in the My School database.

My School website

In terms of test results, we make sure that we only report results for students who were in any given school for each administration of a test for each year it was given. We drop any grade three kids that have gone somewhere else, and we won’t count any grade five kids that have joined you since then, and we’ll compare you with the students in your comparison group of schools with a similar social background, but only in those cases where all of those other schools also have students who were in each school on both occasions. This year, we’re going to provide growth comparisons, which are very interesting and useful, because it begins to give you even further information on what value the schools are adding.

NCEE: Do you plan eventually to use the NAPLAN results to evaluate individual teacher performance in addition to school performance?

Barry McGaw: No. NAPLAN assessments are given only in Grades 3, 5, 7 and 9. It would be very difficult to allocate responsibility for students’ performances or improvements to individual teachers quite apart from the question of how those teaching in grades not tested would be evaluated.

NCEE: Do you see teacher performance as ever becoming a component of the data on the My School website?

Barry McGaw: There are serious discussions going on about how to recognize and reward high-performing teachers but no consideration is being given to reporting on teachers on the My School site.

NCEE: The OECD has recently published a report on evaluation and assessment in Australia (available as a free download from OECD) as part of their international study on these issues. A team of experts visited Australia and observed your system; they made a number of policy observations and suggestions. What did you think about the recommendations that they made for Australia’s system, and are there any that you think the government definitely should implement as you move forward with developing the program?

Barry McGaw: I think it’s a good report. The big thing that we are doing now, as the report pointed out, is developing a strategy for formative assessment. But let me explain where we are first.

The final version of the national curriculum in English, Math, Science and History for kindergarten to grade 10 was adopted last Friday (October 14, 2011), and is now up on the website. It’s quite a historic moment, actually. Already the curriculum is being implemented in the Australian Capital Territory, which is like Washington, DC, because they agreed to the content a year ago. Queensland and South Australia and the Northern Territory will implement next year beginning in January – our school year is the calendar year – and Victoria will have a major pilot in a couple of hundred schools; New South Wales and Western Australia will start in 2013.

What we now have to clarify is the achievement standards. For example, the curriculum states, that, in grade five, in mathematics, these are the things students should have an opportunity to learn. We see our curriculum as a kind of statement of student entitlement. What they should have an opportunity to learn is knowledge, understanding and skills, not just factual stuff.  Then we declare in the achievement standards, if a student has satisfactorily learned this, what will a student be able to do? Those statements can be difficult to interpret in any kind of precise way, so what we are doing now, is putting on the website actual samples of students’ work, produced in response to real classroom tasks with annotations to say this student work meets the standards and why. What we will have up by the end of this year, that is by December 2011, for every achievement standard, is some samples of student work. But then next year, while the curriculum is actually being implemented, we’ll be obtaining a richer set of samples illustrating different levels of achievement at the A, B, C, D and E levels.  The samples of student work will be annotated, for the first time, by teachers across the country, so that we’ll have nationally annotated samples of student work that can move in the direction of getting consistent use of formative assessments across the country.

NCEE: Will that all be available online?

Barry McGaw: Yes, by the end of next year. And the federal government has just put up funds as well to produce some online assessment resources for teachers.

NCEE: Can you expand on why it’s important to have examples of student work when presenting the new curriculum to educators?

Barry McGaw: You will see on the website, that there are statements of achievement standards to give teachers an idea of what students can do, given the opportunity to develop the knowledge, understanding and skills, set out in a particular part of the curriculum. We think that it is difficult to write such statements in a way that is unambiguous for teachers and that it is much more helpful to also provide samples of real student work in response to real tasks created by teachers, but then assessed by a group of teachers from across the country and annotated to provide an explanation for the judgments they make.

Under the previous federal government there was a requirement introduced that all schools report student performance to parents on an A-E (or equivalent) scale. Our annotated samples of students’ work will illustrate performance for each score, A to E, for each subject, each year. We have collected quite a few this year from schools involved in piloting the K-10 English, Mathematics, Science and History curricula, but will collect more during 2012 as some of the states will have already begun full implementation.

The Council of Education Ministers recently approved the K-10 curricula for English, Mathematics, Science and History on October 14th. You can see details of the implementation plans on our website here.

NCEE: You mentioned the necessity of aligning achievement standards with the national curriculum moving forward. Can you clarify where Australia stands with regard to the link between achievement standards and curriculum content? Were national achievement standards developed before the national curriculum?

Barry McGaw: We think that the curriculum should come first as the expression of the goals of education in terms of the learning entitlements of students.  Assessment should follow, shaped by the expectations of student learning.

I recognize that there is a place for ‘assessment-led reform’ where the availability of new forms of assessment can show teachers how to assess learnings that are important but to which they might not attach sufficient significance if they cannot see how to assess them.  In that case, it is still the curriculum and its expectations that come first.

NCEE: While building NAPLAN and the national curriculum, what lessons did you draw from other countries? Are there any countries in particular that you used as a model, and in what ways? What do you see as distinctive about the Australian system?

Barry McGaw: NAPLAN grew out of state-based assessments of literacy and numeracy that began in New South Wales in its then Basic Skills Testing Program in 1990.  The other states followed over the years.  While I was in Paris at the OECD, the Ministers for Education decided that the results should all be expressed on a common scale across the country. The separate tests were equated to achieve this, but then the Ministers decided that it would be better to use common tests.  NAPLAN was the result and the first NAPLAN tests were introduced in 2008. Interestingly, there was no common curriculum behind NAPLAN.  The new test reflected the separate tests that it replaced.

As part of the development of the Australian Curriculum, ACARA was directed also to develop literacy and numeracy continua and then to review and revise NAPLAN as necessary to reflect those continua.  We will time this change on the basis of implementation of the new curriculum with a revised NAPLAN probably to come in 2014.

In our curriculum, we paid attention to practices elsewhere.  Our mathematics curriculum, for example, has been increased in difficulty particularly at the elementary school level on the basis of our analysis of mathematics curricula in Singapore and Finland, two countries that outperform Australia in the international comparisons offered by programs such as OECD’s PISA.

NCEE: We know that the NAPLAN assessments are a combination of multiple choice and short answer questions, and are scored electronically and by trained, independent markers.  How did you arrive at this system – why are they structured in this way?

Barry McGaw: The form of the test was established before responsibility for it was passed to ACARA.  There is a preference in Australia for constructed response questions balanced by cost considerations in favour of machine scoreable responses.  As in PISA, the final choice is based on the two considerations.

NCEE: Many people these days think that it is important to measure creativity and the capacity for innovation. Do you agree? If so, how does NAPLAN (or the other sample tests) measure these things? Are these considerations reflected in the national curriculum?

Barry McGaw: They are important but NAPLAN does not measure them.  They are in our curriculum, embedded in the subject content not as add-on equivalents to additional subjects.  If a teacher wants to focus on creativity, for example, the teacher can apply a filter to the curriculum that will highlight the opportunities that the curriculum in each subject for the school grades of interest to the teacher provides for a focus on creativity. The teacher could use this, for example, in developing an integrating theme through which all the relevant subjects are drawn on.  Such a theme could be followed for some days or weeks.

The opportunities will be expanded as we add additional subjects to the Australian Curriculum.  Development is now well advanced for Geography and the Arts and the rest are following.

NCEE: What are the lessons that other countries trying to build a national assessment system can draw from Australia?

Barry McGaw: First and foremost is to tie assessment to the curriculum. We’ll probably end up making some adjustments to the literacy and numeracy assessments now that the national curriculum has been adopted. What happened historically in Australia was that each of the states developed its own literacy and numeracy assessments, as I said, but did it in relation to their own curriculum. Then they adopted common assessment practices without having adopted a common curriculum. Now we’ve got the common curriculum as well; we just need to make sure that’s aligned, and the developmental sequences are right. One of the big problems is – and I think this is a legitimate criticism of these kinds of assessment programs – that they can narrow the curriculum.  Particularly if you make it really high stakes.  And you can’t make it any more high stakes than putting it on a public website like the My School site. So you start to worry about people gaming the system, encouraging poor performing students to stay at home on the day of the assessment, those kinds of things.

To deal with this, we publish right alongside the school’s performance the proportion of students that were in school on the day assessments were given. So, if there’s any obvious manipulation, or indeed, even if there’s not manipulation, if there’s a low participation rate, that’s evident. There is also the question of whether the system can be gamed by narrowing the school’s teaching focus to what you think might prepare students for a particular form of test. Our view is that the research shows that coaching for tests is effective if what it’s doing is making sure the students are familiar with the test’s format. But it is also the case that if you want to prepare your kids’ literacy skills, the way to do it is through a rich curriculum. Kids learn language in history. They learn language in social science studies. They learn numeracy skills with data representations in geography and other areas of social science as well as in math. They learn it in science. So the best way to develop literacy and numeracy is to have a full and rich curriculum. In Hong Kong they use different forms with different kids in the same class. That’s where we’re heading.

NCEE: Is there anything else you would like to talk about with regard to the report, or the direction the system is going in?

McGaw: I’d like to say something about the curriculum itself, rather than the assessment system. When we got started, we were calling what we did the development of content standards.  I found out from talking to an American journalist that we borrowed that term from you.  I also learned that in the United States you couldn’t talk about national or state curriculum, so you used these words.  What we are doing now is saying that we are developing curriculum or the learning entitlements. We say to schools that by whatever means you teach, this is the knowledge, understanding and skills that your kids are entitled to have the opportunity to acquire. You’ve got to get around the constitutional arrangements in order to do the right thing. Australia has strong constitutional arrangements that say that education is the responsibility of the states, not the commonwealth, not the federal government. So how did we get there? We got there by making it a collaborative arrangement. All of this is decided not by the federal minister; all of this is decided by the six states, two territories and the one federal minister sitting at the table together.

Recent Reports of Note

“Quality Counts 2012: The Global Challenge—Education in a Competitive World,” Editorial Projects in Education, Jan. 12, 2012
This report takes a critical look at the nation’s place among the world’s public education systems, with an eye toward providing policymakers with perspective on the extent to which high-profile international assessments can provide valid comparisons and lessons. It examines effective reform strategies in the US and abroad that have gained traction and may be replicable. And, the report highlights the political and social challenges policymakers will face in improving American education to meet the demands of a 21st-century work force. Learn more here.

Andreas Schleicher, “Chinese Lessons,” OECD Education Today Blog, Oct. 14, 2011
Andreas Schleicher, Special Advisor on Education Policy to the OECD’s Secretary General and Head of the Indicators and Analysis Division in the Directorate of Education, recently visited China to launch the OECD’s first-ever Chinese edition of Education at a Glance. He blogs about his visit to an experimental school in Shanghai, China’s particularly successful educational Petri dish where potential nationwide reforms are developed and piloted.  Read the full blog post here.

Miller, David C. and Laura K. Warren, “Comparative Indicators of Education in the United States and Other G-8 Countries: 2011,” NCES, October, 2011    
Every two years, the National Center for Education Statistics (NCES) releases a compendium of statistics intended to enable comparison between the US and the seven other G-8 countries: Canada, France, Germany, Italy, Japan, the Russian Federation and the United Kingdom. This report focuses on five topical areas – population and school enrollment, academic performance, contexts for learning, expenditures for education and educational attainment and learning. The statistics are drawn from the OECD’s Programme for International Student Assessment (PISA) and their Indicators of Education Systems (INES). To read the full report, click here.

“Education Indicators in Canada: An International Perspective,” Tourism and the Centre for Education Statistics Division, Sept. 2011
This report is intended to be read alongside the OECD’s Education at a Glance 2011 as an in-depth look at the state of Canadian education. Readers interested in Canada’s education system should note the report’s amendments to OECD data; the report points out, for example, that although the OECD statistics show a smaller gap between teachers’ starting and top of scale salaries in Canada, Canadian teachers actually reach the top of the pay scale in half the time of other OECD countries, suggesting a different interpretation of the OECD data. Another notable statistic is the small correlation between students’ reading performance and socioeconomic status; this correlation is far below the OECD average, perhaps indicating particularly successful management of student class disparity in Canada. To read the full report, click here.

“Thematic Probe: Curriculum specification in seven countries,” International Review of Curriculum and Assessment Frameworks, April 2011
INCA’s Thematic Probe provides curriculum and standards information for Australia, Canada, Finland, New Zealand, Scotland, Singapore and South Africa. The information is organized around several questions, as follows: How is the curriculum specified? Are there national standards/expected outcomes? Are curriculum and standards specified and articulated separately or together? Who is responsible for specifying the curriculum? Who is responsible for specifying the standards? How is the curriculum published? Are curriculum components specified locally or nationally? Linked statutory testing – what, when, why? The responses are organized into tables, and provide insight into the links between government control, curricula and standards in some of the top performing countries. To read the full report, click here.