Advertisement

HomeImprove Your SchoolInternational Comparisons

The learning tower of PISA

Every three years, parents around the world get a reality check when they learn how their country's schools stack up against the competition. Are the latest results reason for Americans to panic -- or not all they're cracked up to be?

By Carol Lloyd

It was bad news for the U.S. when. PISA (or the Programme for International Student Assessment), announced the results of its 2009 assessments of 5,000 randomly chosen 15-year-olds in 65 countries. The United States, the superpower still waiting for super-heroic education reform, didn't fare well. We ranked as average in reading and science (similar to countries like France and Hungary) and below average in mathematics (along with Spain, Turkey, and Croatia).

The super-learners included the usual suspects: South Korea and Finland led the pack of sovereign nations, followed by Singapore, Canada, New Zealand, and Japan. A couple of standout Chinese cities also topped the list: Newcomer Shanghai pummeled the competition by ranking first in all categories, and Hong Kong, which has participated in the past and always performs well, ranked near the top too.

What do PISA results mean for U.S. parents?

On the most rudimentary level, the news is not comforting. Education Secretary Arne Duncan declared that the results are "a wake-up call,” revealing the "brutal truth" that our students are being “out-educated.” Especially in the face of top-performing Shanghai, the test offered more evidence that our supremacy is on the wane while other (not-so-long-ago desperately poor and uneducated) world powers are rising. Let little inconspicuous Finland be first in education, sure. But China? Them’s fightin' words.

Even the blurb announcing the results on PISA's website seemed carefully calibrated in anticipation of a panicked response to Shanghai's performance: "Korea and Finland top OECD’s latest PISA survey of education performance. The next strongest performances were from Hong Kong-China, Singapore, Canada, New Zealand and Japan. The municipality of Shanghai also tops the rankings."

"Also tops"? In fact, Shanghai outperformed the second-place countries by 17 points in reading, 38 points in math, and 21 points in science. The inclusion of the 20-million-strong modern metropolis marked the first time that PISA had tried to assess China’s mainland educational system. Of course, extrapolating much about a massive and diverse country from its most modern, well-educated city may not produce the most accurate picture of the nation as a whole. On the other hand, who cares? Could any metropolitan area of our country come close to Shanghai’s performance? Not likely. Even affluent, well-educated Massachusetts, the crown jewel of U.S. education, wouldn't have made it into PISA's top echelon.

Untangling significance from statistics

Yet parsing these results is easier said than done. Dive into the number-crunching debate about what these numbers really mean, and you’re likely to emerge psychically bloodied and thoroughly confused. To misquote w.shakes, that prototypical blogger: "Oh what a tangled web we weave when first we attempt to quantify high school academic performance across countries and languages!"

Ranking the rankings

On the one hand, PISA ranks and displays countries individually for each subject matter. For instance, the United States' science score of 502 just edges out the Czech Republic’s 500 but falls short of Hungary’s 503. These numbers may provide fodder for interesting cocktail banter, but statistical uncertainties make such detailed rankings of little value. When GreatSchools worked with analysts to create the Education Nation Scorecard, our educational statisticians cautioned against displaying individual rankings at all.

Big generalities

PISA breaks up the results into broad categories: below average, average (always around 500), and above average. Acknowledging that differences across languages, curricula, and cultures can play a role in how students perform on tests, PISA queries participating countries to choose which questions would most accurately reflect the knowledge of its students, and calculates two kinds of results for each country — the overall result on all questions (the rankings it publishes) and the result based on the preferred questions. Presumably, if there were a huge distinction between the two, you could deduce that the general test was culturally biased against the students of a particular country. In the case of the United States, however, PISA reported that the preferred-questions results didn’t significantly differ from the general results.

Does PISA topple under the weight of scrutiny?

Critics of standardized testing question the whole endeavor: Such tests, by definition, always entail a reductionist approach to education. School systems like China's and South Korea's are always going to perform better than countries that don't "teach to the test." But PISA claims to be testing students' ability to apply their knowledge to real-life problems that require ingenuity and creative problem solving. Sounds good, right? After sampling a few of the math, reading, and science questions, however, I marveled at the creative thinking of PISA's marketing machine rather than the creative thinking required of test takers. It's a smartly designed standardized test but no more. How it can claim to test creative problem solving, I have no idea.

Still, it's not worth dismissing PISA out of hand. It's not that we want our students to be test-taking machines. But we do want them to be able to read a short, easy passage about a worldly topic — like global warming or how running shoes are designed (to name two from the reading samples) — and understand it well enough to answer a few questions.

What gets less press but may be far more valuable than its test results is PISA's analysis of countries' achievement gap, immigration rates, and socioeconomic status in relation to their educational performance. Not surprisingly, socioeconomic status was linked to higher test scores, but many school systems — like Shanghai's and Singapore's — don't allow socioeconomic diversity to translate into low expectations.

Probably the most interesting document for U.S. parents is a 259-page report buried on the PISA website, "Lessons From PISA for the United States," which analyzes data from Canada, Asia, and other regions through an American prism. Before we decide this is the crisis to trigger the next "Sputnik moment" (as President Barack Obama has suggested) or a test with little merit, it's worth stepping back from the hype and turning this into a learning moment — then bracing for the results of the next PISA assessment, due out in December, 2013.

is the executive editor of GreatSchools and mother to two raucous daughters, ages 9 and 13.

Comments from GreatSchools.org readers

02/7/2011:
"Money is not the problem folks, it's the teachers and the parents. Teachers need to teach more academics and less kumbaya. They need to be qualified in math and science. Parents need to be involved in their child's education daily! The countries that are out educating us spend less by percent of GDP than we do, look it up."
02/2/2011:
"I don't doubt that America is on a downward spiral it seems like it is on a downward spiral with everything. Why are we on a downward spiral in education??? Well it is the no child left behind act. Wow, what a bright idea no child left behind. If the child is not comprehending what is being taught lets pass him/her on to the next grade or the next unit so they can continue to fail. What a brilliant idea, seems like the American government is full of them. "
02/1/2011:
"The article and analysis lack substance. As well, I am embarrassed by the lack of sophistication and understanding. Several comments by teachers (I know), provide all the evidence needed to understand the problems in education. The 'great' teachers represent 25% of the teachers in a school. When education is dictated by politics, it is a formula for given failure. There is no correlation between money spent and achievement. The empirical facts distinctly prove this. This is not arguable. Yet we have a multitude of faculty that swear by this false edict. We do not teach the difference between ideas and knowledge. Our standards are pathetic. To learn by rote memory does not prohibit the lessons of critical thinking. We indoctrinate, we do not teach. "
02/1/2011:
"The article and analysis lack substance. As well, I am embarrassed by the lack of sophistication and understanding. Several comments by teachers (I know), provide all the evidence needed to understand the problems in education. The great teachers represents 25% of the teachers in a school. There is no correlation between money spent and achievement. The empirical facts distinctly prove this. This is not arguable. Yet we have a multitude of faculty that swear by this false edict. We do not teach the difference between ideas and knowledge. Our standards are pathetic. To learn by rote memory does not prohibit the lessons of critical thinking. We indoctrinate, we do not teach. "
12/21/2010:
"I teach middle school US History. For the 10 week Progress Reports 24 of my 192 students failed my class. On each of the report cards to parents of students who failed, I added the comment: 'Call teacher for appointment.' Two called! Perhaps 1/3 of my students have reading skills below what's necessary to read effectively an 8th grade textbook. (I'm not saying 1/3 are below grade level. I'm saying their levels aer below what's necessary to understand well enough what they are reading.) My school district quit holding back students about 10 years ago. Since then, 75% of my 'D' student don't do enough work to earn so they get 'F's', half my 'C' student work enough to only get 'D's' etc. Of course I get blamed for this ... like I'm told my pension benefits will be too high - even though I have required deducted contributions to the system, but my state fails to make their contributions - even though when the 1986 Tax Reform Act put everyone else in the SOcial Security System, my state kept us out so they didn't have to make the additional employer matching contribution! My principal tells us we cannot have homework worth more than 20% of a students grade because if its higher than that students fail because they don't do the homework. I'm sure that's my fault too. America is getting what it deserves because ti has been taking advantage of teachers for my entire professional life - since 1978! Joe"
12/21/2010:
"The chief method of education in Asian countries , like Korea China and its SAR district, HK, is rote memory; and since tests, by and large, test memory, students from these countries will do better than ours. Learning and understanding, as taught in the West, is a complex process (Bloom's Taxonomy of Learning is one of many ) that allows flexibility while stressing problem solving and analysis. Education is not 'Jeopardy Game'. The USA has always been on the right track with better trained teachers. Problems in learning always have root in parental interest in progress (but not micro-managing every detail in the classroom),cooperation and appreciation for the school. Parents that require their child to 'do your best'; parents that show an interest in and follow up on lessons, parents that provide a place, a daily routine, and an expectation for homework completion––usually have kids with the best grades. This is how Asian Americans do it. They teach their kids neve! r to expect an entitlement. Achievement follows involvement. In the past, parents let the teachers teach, and there was cooperation. Not now. And that is the big problem."
12/21/2010:
"These results are eye opening but not surprising. Unfortunately there is no easy answer and the solution is a combination of many factors that cited are below. Among the most important are: parent involvement, outstanding teachers, community commitment, high expectations and an openness to change. Easy to say, but extremely difficult to implement and coordinate. Parents need to step up and become involved in the schools and hold their children accountable for their work and behavior. We need to encourage and reward outstanding teachers. We need to either improve under-performing teachers or remove them from the system. Tenure is an anachronism that can be replaced with due process in evaluations. The community must place a prority on education. We must expect and demand excellence on all levels. We must be open to change -- making informed decisions based on facts for the good of our children and not based on fiction, fads, opinions or self-interest. If you think th! e above are hard, getting everyone on the same page to cooperate is the hardest, but can be achieved with open and honest communications among all parties over a long period of time with the parties behaving consistently and fairly. I know this can work at the local level, but I am not as sanguine at the state or national levels because of all the interest groups. Perhaps the tools of social networking can be applied at the grassroots level for the national good. One man's opinion."
12/15/2010:
"One word or one Phrase might very well explain much of our educational slide 'Bush' 'No Child Left Behind' and other factors include: countries invest real money into their education system as we, the US, seem to be draining our educational funding to pay banks, the pentagon and the big busines multinationals that produce highly over priced weaponry that the US Gov. sells to other countries for the multinationals for huge profits and the US gov buys the lions share of these weapons to perpetuate it constant War State and has done so for over 50 years! "
12/15/2010:
"Wow, are we seriously surprised that countries like China, Japan and Korea beat us? I always knew their students were more educated then ours, even when I was a student in grade school myself. Yes we do need to do something about our academics, but not because of our standing against other countries. If anything we need to adapt whatever it is that they're doing into our schools, since they've obviously been doing something right for so long that we still haven't got a clue on yet."
12/15/2010:
"Sad, very sad. This is not about not enough money for schools, this is about how they are used. Our school district spends on average student ~9K/year. Pretty good money to spend on quality private school. I have deep feeling that public school system just does not work at all, and not because of lack of money, but because somebody ( sorry, majority) does not have any idea how to work. I gave up. My son now goes to private school where he receives some help with language, however does not have any other problems. He was special education student and was treated as idiot at public school. He had altered curriculum and still was below grade level. His teacher told me, he needed medz to help him learn, because of her style of teaching she could not provide him structured predictable environment as was required by his IEP. Two public school years were never ending nightmare for me. My son's IQ score is in high superior range. I am not surprised with the results. More over I suspect that very basic knowledge were tested by PISA. "
12/15/2010:
"I am a teacher and have been one for 30+ years. I have a BA in Special Ed/ Elementary Ed, an MS in Early Childhood Ed, and I am National Board Certified in Early Childhood as well. Last year I was in the final 8 for Teacher of the Year in my state. I said all this only to show that I have a little knowledge about education in general and Early Childhood in particular. We will have to change a few things in order to make a turn around and they are simple. All these countries start formal education at age 7. Finland has preschool which is for ages 4,5, and 6 and emphasize social-emotional education. Social emotional intelligence has been determined to be more important to academic success than any of these tests or even I.Q. They teach children basic manners, behavior, and personal responsibility. When they start school they know the behaviors necessary to be a success. The US on the other hand decided to throw out any of this teaching and move academics down to kindergarten. ! In other words Kindergarten teaches what First Grade used to teach, and First Grade teaches what second once taught etc, etc. It does not work. Child Development experts will tell you that it is like demanding all children learn to walk at 6 months. The brain is not developed enough for this and our so many of our children are not learning social skills at home so this is self defeating in this way also. Another important thing is that none of these countries try to education the whole range of society in one classroom. We have everything from emotionally disturbed, autistic, sociopathic, mentally disabled, non-English speaking, to gifted in one classroom. Our teachers do not have the range of expertise needed to teach all these children well, nor is there enough time in the day to do so if they could. So everyone is left behind. All children have the right to an education, but in the appropriate setting for them. These two areas will have to be 'Fixed' before anything else will work."
12/15/2010:
"We would all do much better if, we as parents stepped up to the plate and disciplined our kids or at least allowed the schools to reign in ill behaved kids. Stop the excuses. Really, how much learning can our kids get in school if teachers spend the whole day just trying to control the classroom."
12/15/2010:
"Finland, really? What are they doing with all that knowledge? Trying to think of one non-obsure contribution they've made to modern society. The US on the other hand leaves them in the dust. The conclusion I draw from this is that they just aren't very industrious."
12/15/2010:
"There should be some correlation between the PISA scores and innovation/ betterment in the overall quality of the society. Test scores reflect effort and congratulations to the high performers. However, education's objective in building productive, innovative and progressive societies/ citizenry should not be lost out on."
12/15/2010:
"You get out of education what you put into education. I hate the rigor of the curriculm is rather scary but its harder to find brighter kids in the mass now day because society allows it. Not every community puts education first. Its what we allow, close our eyes to, believe or forget brainwashing is just that simple. When we allow one citizen to fail, we all fail. I hate to read it, but not going to happen in my house."
12/15/2010:
"As a student, this disappoints me highly. It saddens me to think that this country has regressed so much as from being so advanced, basically at the top, to our students getting below average as test scores. We are not a third world country that has high rates of illiteracy, every child has a right to an education, so why students don't take an advantage of this is beyond me, of course this is probably because they don't realize the importance of education and how much it can benefit them. I only hope that drastic changes will occur in our education system, however I'm not getting my hopes up."
12/15/2010:
"Regardless of the 'intensity' of the problem, the problem is real. Question is how we are going to solve it without enough resources (budget deficit) and enough involvement (parents out of the picture)"
12/15/2010:
"Good golly. What about the parents? I'm working with a high school to write grants for additional funding. The parents are supportive of the grant idea but almost none read the grant applications. When they do contribute text, things like 'it's' and 'its' get confused in their writing. Don't pay attention to this article's conclusion that the PISA may be biased. The PISA is correct. The high school I'm working with spends $5,800/year/student. Excuse me? $5,800??? In contrast, the county spends about $14,000 on special-needs kids. That should be reversed, and spend $14,000 on the brightest and $5,800 on the 'special.' We have to transfer money from Afghanistan and DoD to Education and our brightest."
12/15/2010:
"Chinese schools typically have 50 to 90 students in a class and only one teacher and no aides. The students are expected to clean the schools even during vacations and holidays. The day starts at 7:30 and ends about 5:00. In China teachers are important, in America, teacher are regarded as a nuisance. Misbehaving children even just two or three destroy the academic environment because the teacher needs to spend most of their time disciplining not teaching. Get rid of them. Send them permanently to schools catering to disciple problem children. Establish a national test, make students who do not pass stay behind. At some point a national test should also be given to see if the student is working toward being college ready or should be sent to a vocational training school. Most other countries do this. As a certified teacher and also a substitute I WOULD NEVER, EVER send my children to public school (at least in Texas). In Texas there was a court case with the most mi! nd boggling subject matter: should a student who does not work on an assignment and turns in a blank paper get a 50%!!!!!!! The court said 'No, they should get a zero.' duh, no kidding but several school districts wanted to appeal the decision...I think they must have had a Texas public school education and I use the term education loosely ."
12/15/2010:
"This should not really surprise anyone. The U.S. school system has not progressed much over the last many decades. The approach, the method, the entire system is lacking. One can not rely on our schools to educate our children with valuable tools and information. It IS a parents job."
12/15/2010:
"Are all children in all of the participating countries taken from the same/similar pools? Some children in other countries never get the chance for education, i.e.,those that bloom later. Is this a specially selected random pool or is it a true random selection from all children?"
12/14/2010:
"I marvel at the idea that our country does not 'teach to the test.' We do, and our public education system rarely rewards a child for high marks. It seems like it is super to be 'special' and you get your very own aide to escort you, and it is good to be 'standard/average,' but if you are gifted or above average the solution is 'after you do this mind numbing work that is far too easy for you have some more work that agian will not access your real apptitude.' It is no wonder that the real wealth of our county (the brillinat and gifted ones) hide their 'candle under a bushel.' We have taught our society to step in line from the elelmentary school on up, and that my fellow American's are why our children have been thrown udner the education bus."
12/14/2010:
"This is sad for our children, but not surprising. I live in California and with each passing day our schools become a bigger joke. There is simply no money left for the schools. Classes have 40-45 kids in them. Teachers are tired and can't keep up with the mayhem. My son, who has been in private school his whole life, decided he wanted to go to public high school. Half way through his first semester, he begged me to go back to his former school. He hated public school. His teachers cussed, didn't show up for class, and the kids openly cheated during exams. The entire adventure was an experience of non-chalance. Don't get me wrong: I am NOT blaming the teachers. Rather, the state for spending so much money on other things that they have screwed up the school system. We need smaller classes, more teachers and better systems of punishing the kids who can't adhere or mind the rules. Instead, it is every man for himself. Sad."
12/14/2010:
"It's approaching high time for both policy makers as well as parents in the US to accept the fact that for many of our children, we no longer provide nor demand a first class education. Education matters, and collectively we are failing to develop in our children the skills they will need to be competitive in the increasingly globalized labor market."
12/14/2010:
"School systems like China's and South Korea's are always going to perform better than countries that don't 'teach to the test.' But PISA claims to be testing students' ability to apply their knowledge to real-life problems that require ingenuity and creative problem solving. Sounds good, right? After sampling a few of the math, reading, and 'science questions, however, I marveled at the creative thinking of PISA's marketing machine rather than the creative thinking required of test takers.It's a smartly designed standardized test but no more. How it can claim to test creative problem solving, I have no idea.' I believe the skillful in testing is when you can apply your regular learning same as creative problem solving. Creative thinking is just creating 'dreamer' not in reality.I believe those top education countries, they have gone through lots of difficulties in decades and they study for living, not for dream.In American, kids are training to be creative, but not quite capable to live the knowledge out.Kids are too pampered in their growing stages, they are not likely be able to survive,as those top high rank countries, when they face reality, the real life."
ADVERTISEMENT

More in International Comparisons

ADVERTISEMENT