What Works for What?

“If you’re not using evidence, you must be using prejudice …”
Kevan Collins, EEF

In our brave new world of research based this and evidence informed that, I am very much swimming against the tide, because I don’t think we should try to scientise education. While almost everyone else is focused on neuroscience, randomised controlled trials and the EEF, I’ve just written a book about education as an art form. Yes, educational research can be interesting and valuable, and it certainly adds to the store of knowledge we have on the subject of how we might educate our children, but there are a significant number of problems with basing our practice on what evidence ‘says’, and some of those advocating evidence based practice seem happy to gloss over these. Commentators will often talk (at great length) about the kinds of bias to which other human beings are subject, but they completely fail to see how biased they are themselves. I sometimes feel like I’m shouting “but the emperor has no clothes!” while people tell me that he is, in fact, wearing Armani.

The first problem with evidence based anything, is that in education we don’t have a common set of values to which we want to work – we haven’t even agreed what education is for. The Education Select Committee did hold an inquiry to try and figure this out, but they didn’t seem to come up with much in the way of an answer, and it’s all gone very quiet on that front now. Meanwhile, on social media, one group of people believes that the main goal of schools is to get as many facts into our kids as possible so they get ‘cleverer’. Another group of people thinks that the main purpose of school is to make kids ‘socially mobile’ and to end inequality. (There is quite a bit of cross over between these first two groups.) However, another group of people believes that school is about preparing children for the future, equipping them with skills that will help them do jobs, some of which don’t even exist yet. And yet another group thinks school should be about agency and oracy for young people. (Again, there is lots of crossover between these two.) Then there is a group of people (often politicians) who think that school is mainly there to create a well trained and compliant future workforce. And finally, if we’re totally honest with ourselves, some parents see school as a free source of childcare that allows them to go out to work. But if we can’t even agree what we’re doing school for, how on earth can we agree about what works for doing it?

Building on the problem of what education is for, it’s also crucial to consider what it is not for, and to take into account some pretty serious questions about ethics. If we could prove that Intervention X would lead to better outcomes, at least in the short term, should we always use it? Let’s say Intervention X involved hitting children with a stick. Would we still be happy to do the intervention? Clearly not, since that would be illegal, although it is not so very long ago that schools were perfectly happy to use this method. Now let’s say Intervention X involved the public ranking of children according to their test results. Would we still be happy to do the intervention? You might or might not think that this is a perfectly reasonable thing to do. But put it this way: if you thought it was okay to use this method on my children, I would not be happy to send them to your school.

The next problem with basing education on research results is that schools tend to be messy places, where all kinds of big and small people meet to work and slosh around together. There’s nothing at all wrong with this – it’s part of the reason why I love schools so much. However, this is nothing like the controlled, clinical environment where medical randomised controlled trials take place. As it happens, my best mate is the manager of a Clinical Research Facility. The CRF she runs cost millions of pounds to set up. She has access to cutting edge resources, top end equipment, highly qualified staff and a clinically controlled atmosphere in which trials can be run at great expense. Whereas in education we have Class 3E in a mobile classroom on a wet Friday afternoon doing a new scheme with Mrs Jones who has been teaching for twenty five years and is, to be frank, fed up of new schemes. You might be able to spot the slight flaw in this bit of the plan.

Yet another problem with research in education is that we have to find a way to measure the outcomes of the research we do, and education being education, we like nothing better than a nice test to sum up how we’re doing. The EEF are currently funding a trial into Philosophy for Children. Nothing wrong with that, surely? A bit of philosophy sounds like a lovely idea. In fact, I don’t know why I didn’t think of it before! But if you read about the trial you eventually get to the bit on the website that tells you that “The primary outcome measure will be attainment measured by combined Key Stage 2 maths and English scores.” We are measuring the value of philosophy for children by seeing what it does to high stakes test results. Clearly it’s not the children who need the philosophy lessons.

The next thing to bear in mind is that all these trials cost money to fund (the EEF was set up with a DfE grant of £125 million). At a time when budgets are being slashed, and school staff are being cut, we are still spending millions of pounds trialing new interventions.  The EEF was set up with the aim of doing research to narrow the gap between outcomes for children from advantaged and disadvantaged homes. While this is an admirable aim, despite all the talk about how we can do this, the gap stays persistently wide and more children are living in poverty, not less. And at the same time as the EEF are funding research into things that might close the gap, we have a government who have decided to ignore what research says and fund things that we are told are likely to widen it. How can anyone stomach the DfE calling for evidence based education when they see fit to fund new grammar schools?

Another very interesting problem about education research, which hadn’t occurred to me until I read an article about it recently, is that we seem to have completely overlooked the possibility that interventions can have negative side effects. When the doctor gives you a prescription for some new pills, she will warn you that there might be side effects. The leaflet inside the box of pills outlines exactly what the potential side effects are. But in education, we have become so focused on what research tells us about the benefits of certain approaches, that we have completely forgotten to think about the downsides. I’ve mostly given up commenting on phonics, because every time I do, I get angry messages from people who produce SSP schemes. But a few years ago I wrote this blog, listing many of my concerns. When people are deeply invested in a piece of research being proved ‘correct’, there can be a distinct reticence to discuss the potential downsides. And history tells us a story about the issues this has caused in the field of medicine.

And finally, call me a picky, middle class, sharp elbowed parent if you like, but those in favour of research in education seem to be building the system on the assumption that it is okay to use school children as guinea pigs. And this might happen to include my children. When my friend recruits for her trials, she has to follow very careful procedures to select suitable test subjects, and the people involved in the trials get paid. While I’m sure there are careful procedures in education, you can’t exactly remove your child from school if you don’t want them taking part in a trial. I’m fine for my children’s class teachers to try out new approaches that they think will suit my child; I’m not fine for my children to be used as guinea pigs for trials that could make money for someone’s company. I’m happy for my children’s teachers to make decisions based on experience and intuition; I’m not happy for my children to be involved in someone else’s expensive experiment. No, I’m not using evidence, but I’m not using prejudice either. I am exercising my professional judgement. And we used to set great store in that.