Money matters: PISA tests financial literacy

PISA financial literacyYou’d expect stakeholders in a highly-competitive, high-profit, high-risk, globalised industry to have a clear financial vision, but when the heroes of Irving Welsh’s Trainspotting are discussing what they’d do with the expected profits from a drug deal, only Spud seems to have thought it through. But although “Buy somethin’ for my Ma ” is a lovely reply, it suggests that young Murphy lacks the Financial Literacy Skills for the 21st Century.

If you think you’re smarter than the average 15 year-old, try the test they sat as part of the latest PISA round. Nearly 30,000 students in 18 OECD and other countries took the test, representing around nine million 15-year-olds. The survey was carried out because “Shrinking welfare systems, shifting demographics, and the increased sophistication and expansion of financial services have all contributed to a greater awareness of the importance of ensuring that citizens and consumers of all ages are financially literate.”

The idea was to assess “knowledge and understanding of financial concepts and risks, and the skills, motivation and confidence to apply such knowledge and understanding in order to make effective decisions across a range of financial contexts, to improve the financial well-being of individuals and society, and to enable participation in economic life”.

Of course the 15 year-olds taking the tests are already participating in economic life as consumers and (in far smaller numbers) as savers. Many of them already have a bank card, and some cards are aimed at children as young as 8. This Indian bank shows a complex mixture of nationalism, naivety and opportunism on its website, claiming that: “We strongly believe that Indian kids are the smartest in the world and will use their hard earned savings in the best possible manner.”

Indian students took part in the PISA 2012 tests, but not on financial literacy, which may help explain why the top performers in the results published today are from China. Students from Shanghai score the highest in financial literacy, on average, with a mean score of 603 points, 103 points above the OECD average. There are wide differences in average performance between the highest- and lowest-performing countries and economies: more than 75 points (a full PISA proficiency level) among OECD countries and economies, and more than 225 points across all participants.

Only one in ten students in the OECD area scores at the highest financial literacy proficiency level – Level 5. That means they can solve non-routine financial problems such as calculating the balance on a bank statement, taking into account such factors as transfer fees, and understand the wider financial landscape, including the implications of income-tax brackets.

At the other end of the scale, 15% of students, on average, score below the baseline level of performance. In describing these results, the report makes what looks like one of the most radical claims you’ll ever see in an OECD publication: “At best, these students can recognise the difference between needs and wants.” In my opinion, if we could all do that, whole sections of the economy would collapse. In fact some companies have business models based on trying to convince people with thinking difficulties that there’s no difference between the two.

What Financial Literacy Skills means though is that these students can “make simple decisions about everyday spending, recognise the purpose of common financial documents, such as an invoice, and apply single and basic numerical operations (addition, subtraction or multiplication) in contexts that they are likely to have encountered personally.”

Given that being able to count and read is important for understanding bank statements, bills and so on, it may come as a surprise that high proficiency in mathematics and reading does not necessarily signal high performance in financial literacy. Students in some countries score higher in financial literacy than their performance in mathematics and reading would predict, while students in other countries perform worse in financial literacy than you’d predict from their performance in mathematics and reading.

And while PISA has consistently shown a gender gap in mathematics and reading performance, no such difference is observed between boys’ and girls’ average scores in financial literacy in 17 out of the 18 countries and economies that took part in the survey.

Family background is important though. A “more socio-economically advantaged” student scores 41 points higher in financial literacy – the equivalent of more than half a proficiency level – than a less-advantaged student. In Shanghai, family wealth is more strongly associated with financial literacy than with mathematics performance. In Israel, New Zealand, Shanghai and Spain, family wealth is more strongly related to financial literacy than to reading performance.

You can find full details here about PISA Volume VI and the other V volumes (Fibonacci may have introduced Hindu-Arabic numerals to facilitate double-entry bookkeeping at the start of the 13th century, but we’ll stick to Roman for financial literacy in the 21st).

Useful links

What do teens know about money? By Andreas Schleicher, Director for Education and Skills on the educationtoday blog

PISA, we have a problem …

Any mystified adult trying to figure out the settings of a mobile phone knows there’s only one thing to do – find someone younger. Roused from their slumbers, even sleepy-eyed teens seem instinctively to know how to set up Wi-Fi, program the dishwasher (not that they ever would) and connect that cable whatsit to the TV’s thingamajig.

But are some teens better at these tasks than others? The most recent round of the OECD’s PISA student assessments set out to investigate how well the world’s 15-year-olds do when it comes to tackling real-life, interactive problems – “creative problem solving” – so demonstrating their capacity to reason outside the classroom. Results from the assessments are released today.

If you followed the first set of results from PISA 2012 late last year, you won’t be surprised to learn that, once again, youngsters in East Asia have done very well. Top of the heap is Singapore, followed by Korea and Japan. Chinese-speaking cities and economies fill out the other top seven places. (But note the usual health warnings with these country rankings. PISA is a survey, so there are margins of error in the results; country rankings may be based on differences that are not statistically significant.)

What sort of problems were the students asked to solve? Some weren’t too dissimilar from the challenges mentioned above. Among the tasks were figuring out the fastest route on a map, operating an air-conditioner and buying subway tickets from a vending machine (click on the links or here to take the tests yourself).

Students took the tests on computers, which meant that the problems could be designed to be interactive. That allowed students to receive feedback on their efforts, which, say the PISA people, meant they had to be “open to novelty, tolerate doubt and uncertainty and dare to use intuitions”. Those sorts of attitudes and skills, it’s generally agreed, are increasingly in demand in the workplace. According to the OECD’s adult skills survey, 10% of workers have to deal every day with complex problems that require at least 30 minutes to solve.

There are some interesting contrasts between these latest findings on creative problem-solving and the previous results from PISA released late last year. In general, and not too surprisingly, students who did well in problem solving also did well in mathematics, reading and science. But, in some countries, for example the Unites States, Italy and Australia, students did rather better than might have been expected from the earlier results. This may be evidence that schools are not making the most of students’ potential in core subjects.

Another group also did better than might have been expected: students from disadvantaged families. Although they didn’t match the performance of better-off students in problem solving, they weren’t as far behind as in the traditional PISA subjects. One reason for this may be that – regardless of family background – all young people have opportunities to use and develop practical, problem-solving skills outside the classroom.

As for differences between the sexes, boys generally did better than girls, especially among the top-performing students, where on average there were three boys for every two girls.

Useful links

PISA 2012 Results: Creative Problem Solving (Vol. V) (OECD, 2014)

PISA 2012

OECD educationtoday blog

Follow PISA on Twitter

Are the Chinese cheating in PISA or are we cheating ourselves?


Today’s post, courtesy of educationtoday, is by Andreas Schleicher, in charge of the OECD PISA Programme and Deputy Director for Education and Skills as well as Special Advisor on Education Policy to the OECD’s Secretary-General.

Whenever an American or European wins an Olympic gold medal, we cheer them as heroes. When a Chinese does, the first reflex seems to be that they must have been doping; or if that’s taking it too far, that it must have been the result of inhumane training.

There seem to be parallels to this in education. Only hours after results from the latest PISA assessment showed Shanghai’s school system leading the field, Time magazine concluded the Chinese must have been cheating. They didn’t bother to read the PISA technical report, which shows there was no cheating, whatsoever, involved. Nor did they speak with the experts who had drawn the samples or with the international auditors who had carefully reviewed and validated the sample for Shanghai and those of other countries.

Others were quick to suggest that resident internal migrants might not be covered by Shanghai’s PISA sample, because years ago those migrants wouldn’t have had access to Shanghai’s schools. But, like many things in China, that has long changed and, as described by PISA, resident migrants were covered by the PISA samples in exactly the way they are covered in other countries and education systems. Still, it seems to be easier to cling to old stereotypes than keep up with changes on the ground (or to read the PISA report).

True, like other emerging economies, Shanghai is still building its education system and not every 15-year-old makes it yet to high school. As a result of this and other factors, the PISA 2012 sample covers only 79% of the 15-year-olds in Shanghai. But that is far from unique. Even the United States, the country with the longest track record of universal high-school education, covered less than 90% of its 15-year-olds in PISA – and it didn’t include Puerto Rico in its PISA sample, a territory that is unlikely to have pulled up US average performance.

International comparisons are never easy and they are never perfect. But anyone who takes a serious look at the facts and figures will concede that the samples used for PISA result in robust and internationally comparable data. They have been carefully designed and validated to be fit for purpose in collaboration with the World’s leading experts, and the tests are administered under strict and internationally comparable conditions. Anyone who really wants to find out can review the underlying data.

Short of arguments about methodology, some people turn to dismissing Shanghai’s strong performance by saying that Shanghai’s students are only good on the kind of tasks that are easy to teach and easy to test, and that those things are losing in relevance because they are also the kind of things that are easy to digitise, automate and outsource. But while the latter is true, the former is not. Consider this: Only 2% of American 15-year-olds and 3% of European ones reach the highest level of math performance in PISA, demonstrating that they can conceptualise, generalise and use math based on their investigations and apply their knowledge in novel contexts. In Shanghai it is over 30%. Educators in Shanghai have simply understood that the world economy will pay an ever-rising premium on excellence and no longer value people for what they know, but for what they can do with what they know.

PISA didn’t just test what 15-year-olds know in maths, it also asked them what they believe makes them succeed. In many countries, students were quick to blame everyone but themselves: More than three-quarters of the students in France, an average performer on the PISA test, said the course material was simply too hard, two-thirds said the teacher did not get students interested in the material, and half said their teacher did not explain the concepts well or they were just unlucky. The results are very different for Shanghai. Students there believe they will succeed if they try hard and they trust their teachers to help them succeed. That tells us a lot about school education. And guess which of these two countries keeps improving and which is not? The fact that students in some countries consistently believe that achievement is mainly a product of hard work, rather than inherited intelligence, suggests that education and its social context can make a difference in instilling the values that foster success in education.

And even those who claim that the relative standing of countries in PISA mainly reflects social and cultural factors must concede that educational improvement is possible: In mathematics, countries like Brazil, Turkey, Mexico or Tunisia rose from the bottom; Italy, Portugal and the Russian Federation have advanced to the average of the industrialised world or close to it; Germany and Poland rose from average to good, and Shanghai and Singapore have moved from good to great. Indeed, of the 65 participating countries, 45 saw improvement in at least one subject area. These countries didn’t change their culture, or the composition of their population, nor did they fire their teachers. They changed their educational policies and practices. Learning from these countries should be our focus. We will be cheating ourselves and the children in our schools if we miss that chance.

International comparisons are never easy and they aren’t perfect. But PISA shows what is possible in education, it takes away excuses from those who are complacent, and it helps countries see themselves in the mirror of the educational results and educational opportunities delivered by the world’s educational leaders. The world has become indifferent to tradition and past reputations, unforgiving of frailty and ignorant of custom or practice. Success will go to those individuals, institutions and countries which are swift to adapt, slow to complain and open to change. And the task for governments is to help citizens rise to this challenge. PISA can help to make that happen.

Useful links

PISA 2012

How the world reported PISA

Main findings on student perforamnce
Main findings on student performance

Much of the media coverage around PISA focused on the strong performance of Asia’s students. But regular followers of the OECD’s assessments of worldwide student performance had something else on their minds: where’s Finland?

They weren’t the only ones feeling troubled: “The golden days are over,” lamented Finnbay, a Finnish media outlet, as it reported that Finland, long a PISA poster-child, had “dropped from the top of the league.” Yle, the national broadcaster, even wondered if the country’s education system might be at risk of sharing “the fate of another fallen national champion – Nokia.” Numerous explanations were offered for the Nordic country’s performance, including that it may be a victim of its own PISA success.

“Some argue that complacency and focus on explaining the past to thousands of education tourists have shifted attention away from developing Finland’s own school system,” wrote Pasi Sahlberg, who works on education reform in Finland. In response, Finland’s education minister, Krista Kiuru, promised “strong action,” according to Finnbay. There was handwringing, too, across the border: “No other country has fallen so abruptly as Sweden in maths over a ten-year span,” announced TheLocal, a Swedish news outlet.

Tremors from the Nordic countries’ PISA showing were felt in the United Kingdom, where the Financial Times reported that the performance of the country’s students was “stagnating”. That, combined with the results of Finland and Sweden in PISA, put the education minister, Michael Gove, “under pressure to defend his Scandinavian-inspired education reforms,” according to the FT.  In response, it wrote, Mr Gove said “a ‘more rigorous system of accountability’ was also needed to make sure that poor performance did not go unchecked.”

As well as looking at national results, the media also examined PISA itself. In The Guardian, Mona Chalabi discussed “the basic claim of the assessment – that it is able to accurately capture the full range of students’ abilities and compare them across the world.” (You can find some FAQs about PISA’s methodology and approach here.) There was discussion, too, of the usefulness of international comparisons in education. Writing in a Washington Post blog before the PISA release, Valerie Strauss warned about the “fetishization of international test scores”. As The Economist noted, however, “critics will point out the rankings have imperfections. But it gives us a clue to how successful our classrooms are—and that is hard to ignore.”

Away from making international comparisons, PISA also looks into how factors outside the classroom affect student performance, most notably, perhaps, social background. The importance of some of these seemed to receive particular recognition in France, where Libération (in French) described the country as the “world champion in education inequalities”.

Writing on The Huffington Post, Jacques Attali, a leading economist and public intellectual in France, warned that the relatively weak performance of disadvantaged children in France’s schools posed an existential crisis for his country: “Unless this changes, millions of talented individuals … will be lost, and they will leave, frustrated, to go to other countries or underground. France will lose her soul in this. And her future.”

What about the “winners” – the Chinese city of Shanghai as well as OECD Members like Japan and Korea, among other Asian economies, that topped the PISA tables? While their achievements in education were widely recognized, there was concern, too, about the price being paid by the region’s hard-working students.

Korea’s Chosun Ilbo newspaper noted that while Korean students performed well in PISA, the results also demonstrated that “they showed a lack of interest in the subjects.” This is sometimes attributed to the relentless pressure on Korean kids to succeed, which sees many follow an intensive eight-hour day in school with another three or four hours in an evening cram-school.

“For South Korean teenagers a double shift of school, every week day, is just a way of life,” the BBC’s Reeta Chakrabarti reported. But there are signs of a shift in attitudes, she added, quoting the country’s education minister, Seo Namsoo: “We still have a long way to go,” he said, “but we are doing some soul-searching in our society, and our goals now are about how to make our people happier.”

Useful links

PISA 2012

Asia’s students storm PISA 2012

Main findings on student perforamnce
Click for main findings on student performance

Students in Asia have topped the rankings in the latest round of the OECD’s PISA programme of international student assessment. The results, released this morning, show countries and economies in Asia grabbing seven of the top 10 slots in mathematics, the focus subject of the PISA 2012 round.

Students in the Chinese city of Shanghai continued their impressive showing from PISA 2009, taking first place not just in mathematics but also reading and science, the two other core subjects tested by PISA.

The strength of their performance in mathematics, especially, is striking: In effect, the city’s 15-year-olds scored the equivalent of nearly three years of schooling above students in most OECD countries. Not only that, more than half of them – or around 55% – performed at the highest levels in maths, compared to an average in OECD countries of around one in eight students.

Of course, Shanghai is not China – a distinction that tends to get lost in coverage of the PISA results. The city is the wealthiest in China, and the life experience and education of its young residents are a long way removed of those of young Chinese in, say, rural areas. (That said, unpublished results from the PISA 2009 assessments in other parts of China showed a  “remarkable performance,” according to the OECD’s Andreas Schleicher, who runs PISA.)

Shanghai’s students have also benefited from the city’s bold efforts to reform its schools. The result, according to The New York Times’ Tom Friedman, “is a relentless focus on all the basics that we know make for high-performing schools … These are: a deep commitment to teacher training, peer-to-peer learning and constant professional development, a deep involvement of parents in their children’s learning, an insistence by the school’s leadership on the highest standards and a culture that prizes education and respects teachers.”

But while Shanghai should not be read as representative of China, its strong performance in maths is in line with that of many of the other Asian economies in PISA. Undoubtedly, some will turn to wider cultural explanations for this phenomenon. However, the impressive performance of Asian economies like Singapore and Korea across the range of subjects examined in PISA suggests other factors may be at work within education systems.

Chief among these, according to the OECD, is how these countries and economies approach teaching: “Top performers, notably in Asia, place great emphasis on selecting and training teachers, encourage them to work together and prioritise investment in teacher quality, not classroom sizes,” the organisation said in a press statement. “They also set clear targets and give teachers autonomy in the classroom to achieve them.”

Useful links

Today sees the release of a special data tool designed to let users explore the performance of individual countries as well as many of the key issues examined by PISA. These include how boys and girls do, the impact of social background, and the role of students’ self-belief and motivation.

Here’s a sampling of some of the ways in which you can explore PISA 2012.

  • Main findings, country-related content and video streams can be accessed at PISA 2012.
  • You can also find a useful overview (pdf) of the main results as well as a snapshot (pdf) of how countries performed.
  • On the Internet, there’s a Facebook page for parents and a special PISA Day website and, later on Tuesday, a webinar.
  • You can also follow PISA 2012 on Twitter.
  • And you can read Andreas Schleicher’s thoughts at the OECD educationtoday blog.
  • Full results and analysis from PISA 2012 are presented in the following reports (with more to come next year): Vol. I presents the main findings on student performance; Vol. II looks at issues of equity and fairness in education; Vol. III looks at student engagement and self-beliefs; and Vol. IV looks at the role of how schools are run and resourced. And there’s a special report on what the United States can learn from PISA.

But how would he do on PISA?

Here's the other one
Here’s the other one

We couldn’t resist this video by Aria Shahrokhshahi, a young British lad who’s been struggling with his sums.

“One year before this video I was at a grade F in maths,” Aria explained. “In England you need a C (pass) to basically do anything with your life. […] Neither me or my dad or my teachers thought I was going to get my C.”

But, as the video shows, Aria surprised everyone – especially his dad – by getting that C.

Soon, it won’t just be Mr Shahrokhshahi who’s getting excited about mathematics scores. For on Tuesday the OECD will release the latest round of results from its PISA programme of international student assessments, or, as the BBC calls it, “the world’s most important exam”.

This latest release of PISA results covers the most recent round of the three-yearly assessments, which took place in 2012. As usual, it covered three main topics – mathematics, reading and science. Also as usual, it placed a special focus on just one of these subjects – in 2012, it was mathematics. In total, more than half a million 15-year-old students took part in the assessments, representing 64 countries and economies.

So, what to watch out for this week? Given the media deluge that typically follows a new PISA release, it might be easier to talk about what you shouldn’t watch out for. Chief among these, perhaps, are the famous league tables, which “rank” the performances of students in each participating country and economy. These tend to receive the lion’s share of coverage, with lots of excitable talk about how one country’s students are three places higher than another’s. How does the OECD itself feel about these tables? “We don’t attach that much importance to them,” PISA’s Andreas Schleicher told The Guardian last week, “but people want to see comparisons.”

Of more interest, perhaps, is the broader picture of whether each country is above, around or below the average and whether its performance is improving or declining. And, of course, there will be usual data and analysis on a whole range of other issues that PISA investigates, such as how students from poorer families do compared to better-off kids, how girls do compared to boys and the importance of students’ attitudes.

If you want to follow the PISA launch, coverage begins at the PISA website tomorrow (3 Dec.) at 1000 GMT (that’s 5am in New York, 10am in London, 11am in Berlin and Paris and 7pm in Tokyo). If you’re following on Twitter, the hashtag is #OECDPISA. Later on Tuesday, you can join a “webinar” with Andreas Schleicher. You might also like to go along to PISA Day, a special website created by the Alliance for Excellent Education and partners. And, if you’re a parent, there’s a Facebook page that might interest you –

Useful links

OECD work on education

OECD educationtoday blog