Finding the middle ground between reflection and inquiry

When I first started writing this blog, I didn’t think it would last very long. Its name, ‘Reflecting English’, was a bit of an afterthought, a quick admin decision. Over recent months I have become increasingly uncomfortable with the word ‘reflecting’. Intuitively, practitioner reflection has a vital role in the iterative process of teaching – plan, teach, reflect … plan, teach, reflect – yet it is also subject to a number of weaknesses.

I am not intending to argue against reflection. In fact, I think it is impossible to be a half-decent teacher without it. It’s just that I have come to understand its limitations more clearly. Let’s say a student misbehaves during my lesson: I reflect afterwards and decide to move him to the front where his behaviour dramatically improves over the next few weeks. Perhaps my students are struggling to answer a particular exam question: I reflect and decide to spend a lesson carefully modelling an answer step-by-step and their responses significantly improve in an end of term test. In both scenarios, I can tentatively conclude that my action has had the desired impact over time – and, even if I am wrong about the cause of the change, things have improved so there is little need to look back.

It is when we share our reflections that we need to be more careful, especially when we use them to make claims about ‘what works’ or, even more so, when we use them to help us to reach judgements about the quality of other teachers. We are subject to a huge range of biases and exemplary classroom teachers can overlook the strong effects of the automatic, unconscious elements of their practice. This can easily lead to attributing success to an intervention, when other unnoticed factors are at play. In the complex social environment of the classroom, cause and effect are hard to isolate.

An alternative to reflection lies in more rigorous inquiry. This might come under a number of guises. At last weekend’s excellent Research Leads event at the University of Brighton a number of models were shared.

I discussed the action research model we are currently working on in partnership with Dr Brian Marsh: three teachers are running inquiry projects based on areas of interest that match with school priorities.

James Mannion shared his revolutionary Praxis model, an online platform which, to paraphrase Freire, encourages teachers to reflect and act upon their classrooms in order to transform them. Praxis provides a structure for inquiry and evaluation beyond traditional reflection.

Another presenter, Nick Rose, shared the inquiry tools he has been building to support his coaching work with teachers: these include coaching logs, focused and formative observations and classroom climate logs.

In many schools, lesson study triads have been set up as vehicles for inquiry.

However, as exciting as these ideas and innovations are, just how much reach do they have? Sadly, teachers remain hugely hard-pressed for time and, even if these models become embedded, will they meaningfully change the culture of schools? My personal view at the moment is that involvement in an inquiry project should be optional for an experienced and successful teacher. That does not mean, however, that this teacher should not engage with evidence in any form.

So, is there a third way, a middle-ground between reflection and disciplined inquiry? Some are suggesting that the growing interest in research is really a growing interest in more rigorous evaluation and informed decision-making. To bring these about, we need to provide more checks and balances at various levels in our schools so that we avoid some of the obvious biases and flaws of thinking.

The suggestions below follow two simple principles: 1) they  lead to more robust evaluation and decision-making and 2) they do not lead to increased workload.

Share ideas before meetings. Groupthink is the psychological phenomenon where desire for conformity within a group leads to lack of creativity and poor decision-making. A simple solution is to ensure that before meetings – especially those that will evaluate past interventions or decide on future ones – ideas are independently submitted and compiled so that they can be shared. (Cut 15 minutes from the meeting time to allow this to happen.) A broader range of ideas is likely to be on the table and it is less easy to suppress dissenting or quieter voices.

Allow time for questions. At meetings, teach meets, 15-minute forums, etc, ring-fence time for questions. Be brave and actively invite tough questions. Everyone will learn a lot from them and you may refine your actions as a result. Make sure you provide an option for people to privately share their thoughts too: sometimes we need to chew the fat for a while or would prefer not to question publicly.

Appoint people to take the role of devil’s advocate. Every leadership team and department could benefit from  someone who takes this role. (You might want to call it the critical friend role.) The aim is not to annihilate every new innovation; it is to test that its potential strengths outweigh its potential weaknesses. Alex Quigley has written an excellent post on the importance of this role. Similarly, I love the idea of the premortem – when teams imagine a project has failed in the future and they work out the causes.

Encourage tentative language. Quite simply, school culture is rooted in language. A simple and subtle shift is to encourage educators to choose their wording carefully – does/did work and will/has work(ed) become seemed to work, might work, could work, etc. This is not a call for relativism, just for more careful skepticism. If modelled by leaders, this change would be relatively easy to implement.

Encourage a broader engagement with theory/research/ideas/other schools/other classrooms. We need to broaden perspectives. Often we do not ask questions of our own practice because we do not realise that there are questions to be asked. Introducing staff to the world of blogs and twitter are easy ways in to help teachers to consider their practice in new ways. Circulating a blog of the week and a research bulletin, or encouraging teachers to video themselves or providing them with the cover to watch another teacher in action can lead to more rigorous reflection.

Anyway, there are a few ideas for you. I might be wrong, but I think there is a gap between day-to-day reflection and more extensive inquiry that could be bridged by a simple whole-school approach.

Please take the time to comment. These ideas are in their embryonic stages.

Thanks for reading.

This post by Gary Jones on groupthink and bias has informed some of my suggestions.

Related posts:

The art of teaching clearly: on why we should treat intuition with care

Creating a research-rich climate: our first steps

Advertisements

Sequencing lessons in the run up to exams

The weeks preceding an exam always feel like a mad rush, at times chaotic. There’s so much our students need to know and be able to do in such a short space of time that it can be hard to identify what to cover and what to leave out. These final lessons should pinpoint the  critical content of our subjects, but sometimes in our desperation to throw everything at students we can create more problems than we solve, leading to confusion and inertia.

My year 10s will be sitting their AQA, GCSE literature exams towards the end of May. One paper will ask them to write extended responses to two literary texts: Of Mice and Men and An Inspector Calls. The other will ask them to compare two poems on the theme of conflict – of a group of 15 – and respond to an ‘unseen’ poem (i.e. one they have not read before). Barring two poems, we have now thankfully covered everything else on the syllabus. There is still, however, much to do. Not only do my students need to revise, extend and connect – and perhaps even critique -what they already know, they also need to practise writing exam answers in a fluently academic style. How do I fit it all in?

Two recent posts by English teachers have inspired me: Joe Kirby’s on ‘knowledge organisers’ and Phil Stock’s on how and why we should teach students to memorise key quotations. Joe’s post argues that by creating a simple one-page organiser we can a) meticulously clarify the key knowledge in every topic and b) provide a memory tool for students to revisit and test themselves with. Phil’s post looks philosophically at how learning a quotation might lead to greater appreciation of literature. He also argues that “from a practical point standpoint, a well-selected quotation can stimulate deeper language analysis or act as a cue for links to other ideas and interpretations across a text.” Phil then shares a number of practical memory strategies to help students learn them accurately.

I have created knowledge organisers for An Inspector Calls (link) and Of Mice of Men (link), complete with key quotes.

These were more tricky to design than I had expected. I agonised over what I was leaving out as much as what I left in; I imagine many other English teachers will quibble with my choices. Still, I see these as cues and points of reference to inspire richer, fuller interpretations rather than ends in their own right. I decided not to create a poetry organiser for two reasons: 1) it is easy to find quotes in the exam as students are given complete poems and 2) I will use these first two as experiments – I do not want to over-complicate things at this stage.

So, the next question was: how should I plan the sequence of lessons leading to the exams? Here, I have looked at the evidence from learning science shared in Make it Stick by Brown et al and How We Learn by Benedict Carey. This is also nicely summarised by Dunlosky et al (2013) here.

This is my revision programme:

Lessons (apart from extended exam practice) will:

1) Begin with a ‘memory platform‘ where they will be tested on key knowledge/quotes from the knowledge organisers using a wide range of  quizzing methods. Students need to be fluent in this knowledge and to do this they will need to cover it repeatedly. They will be encouraged to elaborate on these points of knowledge, exploring their wider significance.

2) Revise and extend a key area. OMAM and AIC – the green lessons – will be taught side-by-side, whereas the poems will be revised in pairs.

3) Lessons will end with 15 minutes of deliberate writing practice where students will be expected to hone the finer parts of their analytical style by writing a paragraph at A/A* standard – this is a top set – based on the content of the lesson (modelling and scaffolding will feature here too).

Rather than tell students to ‘go home and revise’ I will give them specific revision tasks for the first few weeks before the cramming inevitably ensues. These will involve self-testing using the knowledge organisers, further extended writing practice and planning exam answers. Fortunately, we do have more than 12 lessons until the exams – I have given myself some wiggle room and a chance to respond to any other pressing needs that arise.

So why do I think that this has a reasonable chance of working effectively?

The evidence suggests that distributed practice is more effective than massed practice. I have avoided blocks (two weeks of OMAM, two weeks of AIC and two weeks of poetry). This scheme breaks this into smaller chunks that sit side-by-side. Another way of looking at it is that students will be constantly shifting across an interconnected body of knowledge, practising it as they go, rather than the more traditional approach of studying three isolated units of knowledge.

The evidence suggests that regular retrieval practice aids long-term retention – it is much more effective than re-reading notes. The ‘memory  platforms’ allow for regular quizzing – teacher, self and peer. Before we revise each poem, they will read it and annotate themselves; this will ultimately be a retrieval task (although, of course, they may usefully generate new ideas too). The extended writing opportunities will be a chance for students to both extend and apply their knowledge. Lastly, lessons will be peppered with retrieval questions: e.g. ‘Give us a quote/event that supports that argument?’ or ‘Let’s have two adjectives to describe Slim.’

The evidence suggests that getting students to elaborate – the process of finding additional layers of meaning in new material – can further aid retention. Rather than simply recalling information and quotes, students will constantly be asked to explain the broader significance of this, linking it within and across texts.

The evidence suggests that through interleaving – alternating between different problems rather than focusing entirely on one – not only is the material more likely to be retained, but it will improve  chances of success in a future test. As you will have noticed, the lessons alternate between poetry and drama/prose all the way through. Teaching  literary texts side-by-side is really exciting: students must consider the finer, subtle differences between them. Which writer provides the most positive portrayal of women? Both Priestley and Steinbeck took a socialist stance: how do they differ? Which text has the greater emotional impact? In other words, students might well gain a more nuanced understanding of the texts by considering them in relief rather than in isolation.

Caveats

As a research lead at my school, it is incumbent upon me to present evidence-informed ideas in a cautionary light. I think my sequence has a strong chance of working, but…

1) While the research into the learning methods presented above is robust, it has yet to be tested in my classroom. Will these findings extend to the study of English literature in my real-world context or not?

2) The success or failure of my plan will, as always, hinge as much on quality of my teaching along with the motivation of the students. We will all need to work hard outside of lessons.

3) While Phil Stock’s ideas about learning quotations appeals to my confirmation bias, I have yet to see/read about conclusive proof that this method produces better extended writing. It is great fodder for an action research project.

4) With so many variables in the mix, it will be hard to evaluate the effectiveness of this approach. Without a control group, it would be unwise to jump to assumptions based on exam grades. Looking at student work for evidence of impact would be a safer option – as would buying back one or two final exam scripts. I ‘could’ interview students on their views. However, the research into ‘desirable difficulties’ – as the strategies I have shared are sometimes known – suggests that even though people may have learnt more through exposure to these approaches, they often feel that they have not.

Your thoughts would be appreciated.

Thank you for reading.

You win some, you lose some

Image: @jasonramasami

In the lower sixth I became a slacker. I devoted a great deal of time to the exciting distractions that the threshold of adulthood brings. I’d never really had such freedom before –  and I capitalised on it. This, you will agree, was pretty normal for a 17 year old. However, there was one subject I was particularly enjoying at this point and that was A-Level psychology. Milgram’s classic study of obedience gripped me; even now I remember learning the meaning of terms like ‘diffusion of responsibility’ and ‘extrapolation’.

Nevertheless, my interest and work-rate were unevenly matched. At the start of the year I had written some promising pieces, but gradually laziness took over. It all came to a head on the day my teacher handed me back a marked project. It had been awarded (if that’s the right word for it) a U. I promptly ripped it in half and fell into a dark sulk. My teacher kept me behind and asked me, knowingly, why I had torn it. I had let myself down, I responded; I had not done as well as I could. If I recall correctly, she said very little. She didn’t need to; I could say it all myself.

I was reminded of this moment when I sat down this week to watch Richard Linklater’s wonderful coming-of-age movie, Boyhood. If you’ve not seen it yet, you really must (beware – spoilers coming!). Filmed in time-lapse over twelve-years, it gently and languidly follows Mason and his journey from childhood towards maturity. We see him grow physically, intellectually amd emotionally before our eyes. His dysfunctional family life is a typical one: good and bad events interweave in predictable fashion. The film dips into his school life from time to time. The usual clichés are there – his peers ridicule his new haircut; bullies threaten him in the toilets; a note is passed to him from a pretty girl – but they do not feel banal.

Mason excels in photography. He has developed a passion and a talent for the subject but devotes too much time to the dark room when he should be undertaking the assignments necessary to complete the course. Eventually he has a showdown with his teacher who shares some choice wisdom (I forget the exact wording). There are plenty of good photographers out there in the real world, but only those who have been prepared to match their talent with hard work have found success. Happily, the film later shows that Mason is awarded a photography scholarship at college.

However, the film is too subtle and well-made to settle for simple cause and effect. We never learn whether his teacher’s wise words and actions were enough to turn Mason around or whether the motivation ultimately came from within. This is wise filmmaking: when you step back to take a longitudinal view of life and education, it is an error to look for single causes to complex outcomes. It’s all so much more complicated than that.

All this got me reflecting on my own story. Was that frustrated moment at the end of class really the catalyst for my eventual A grade in psychology?

Much of how we interpret an educational outcome depends upon our vantage point. We turn our lives into easy, simple stories, finding coherence by knitting together jumbled-up and unrelated moments. In hindsight, everything seems clear, the arbitrary complexity of life filtered down into narrative simplicity. The story of my A-grade in psychology is just that: a story. Other participants – my teacher, my parents, my friends – might tell it very differently. In a hundred parallel universes, my success might be attributed to a hundred different causes. (Or perhaps I just lucked-out with the exam questions? Who knows?)

As teachers we face a timeless paradox: our individual influence can be huge and profound, yet often we are no more than a tiny cog in the idiosyncratic engine of a child’s life. Sometimes a choice word in an ear changes a future; at other times, ten-thousand words lead to little alteration. Therein lies the sheer frustration of this job.

We know, for instance, that genetics provide a more accurate indication of future GCSE results than environmental factors. It would be easy to hear this and find solace in fatalism. Yet there are individual children, classes and, indeed, whole schools that buck this trend – see Doug Lemov’s work on ‘champion teachers’ for example. Sadly, however much we dress it up, education is designed to separate the ‘winners’ from the ‘losers’. Statistically, some kids will ‘pass’, others will ‘fail’. Some teachers will get great exam results, others will not. Even if the quality of education were to improve beyond all measure, this uncomfortable circumstance would remain stable.

So, let’s give ourselves a break. Let’s aim to find out what has the best chance of working well, do it as best we can, learn from it… and then move on. However hard and well we work, failure, success and everything in between will still happen to us. Not all factors are within our sphere of control. We are humans, not gods. Life is complex: the impact of our actions and words might not be felt by the child for many years or in the way we expect it to be felt.

Yes, you win some, you lose some, but this is composed of many factors, not all within the realm of your influence.

*

Related posts:

Reflections on a successful student

GCSE results and the stories we tell ourselves

In celebration of failure

The art of teaching clearly: on why we should treat intuition with care

Image: @jasonramasami

I have been reading Rolf Dobelli’s book The Art of Thinking Clearly on systematic cognitive errors. It is a thoroughly readable romp (Dobelli is a novelist by trade) through many of the flaws in decision-making and judgement that human beings are prone to. It is not that we are completely irrational – just that we are not as rational and clear thinking as we like to believe we are. Like many, I first encountered ideas about cognitive biases and heuristics – common mental shortcuts used to side-step thinking about difficult problems – in Daniel Kahneman’s seminal work, Thinking: Fast and Slow. It is a must-read.

The routine cognitive errors shared in both books are largely inescapable. They are wired into our thinking, probably as evolutionary relics passed down genetically from our hunter-gatherer ancestors.  Through careful self-control we can all get better at managing our own decisions, and because decision-making based on a complex range of variables is part-and-parcel of teaching and school management, I think a broader awareness of cognitive biases could prove particularly useful.

What follows are descriptions of three common cognitive errors, along with real and hypothetical school-based scenarios to exemplify their significance. For each, I also offer some tentative solutions.

1. Availability bias is the way we are susceptible to making evaluations based on the evidence that comes most immediately to mind.

Here’s an example. I have taught at my current school for nine years and, without fail, every few months I hear somebody say that they think that behaviour in the school is steadily deteriorating. I tend not commit to this conversation, mainly because I have judged the complete opposite to be true: I think behaviour has improved immeasurably.

Both our judgements, however, are likely to be tainted by availability bias. Perhaps my colleague has recently been the victim of several examples of poor behaviour in her lessons. These memories loom so large that she automatically links them together to form her judgement, forgetting about ninety-nine per cent of her pupils who have been consistently impeccable. Similarly, examples of good behaviour are more available to me because of two factors: 1) the longer I have remained at the school, the more established and confident I have become and 2) I have been extremely lucky with the behaviour profile of my classes over the last two years.

So, how do we find an answer much closer to the truth? We could look at school exclusion figures first. These would give a general picture but they might not fully tell us what we they think they do. Do lower exclusion rates mean improved behaviour? Or do they instead indicate higher tolerance levels of bad behaviour, or indeed better on-site school provision for the behaviourally challenged? It would be advisable to look for other sources of evidence, too. If you have an on-call ‘duty’ system like we do, could you collect data on how often senior staff are called to incidents of poor behaviour? Or could you survey the students on a termly basis by asking ‘how many times were your lessons disrupted by bad behaviour yesterday?’? Even better – do both. To avoid the availability bias, therefore, we need to find our answers in a wider base of evidence – and preferably evidence from a range of sources.

2. Self-serving bias is the way we tend to attribute our successes to ourselves and our failures to external factors beyond our control.

Never is the self-serving bias more evident in schools than on exam results day. Let’s take two hypothetical students, Dan and Tom, to exemplify this phenomenon. Now Dan and Tom were in my GCSE class and were statistically similar: white, working-class, with lower than average attendance and below average prior attainment. However, there was one difference on that August afternoon: Dan achieved an E in English, Tom got a B.

How do I attribute these differences? Well, Dan was hardly ever at school, and he arrived in year 7 with a level 3 KS3 score and, to be honest, white working-class boys achieved terribly nationally anyway. As for Tom? Well (!), we built up a great relationship right from the start, he seemed to find my lessons particularly engaging and I did spend an awful lot of time giving my students careful feedback. In other words: I have blamed Dan’s failure on external factors and accredited Tom’s success to the enduring prowess of my classroom practice. The self-serving bias caught red-handed.

Although the self-serving bias is natural (and, to some extent, provides us with a useful self-protecting mechanism), the education system currently incentivises such disregard for objective truth. If you want honest accounts of success and failure then you first need to remove data targets and performance-related pay. Even if we discount out-of-school factors (genetics and family background), there are a huge number of in-school factors outside the control of the average classroom teacher that influence both good and bad final exam results: curriculum design, choice of exam board, subject timetabling, class profile, how subject knowledge and skills are reinforced by the whole-school curriculum, how much progress the child made with KS3 teachers…the list is endless. To truly begin to understand the complex ways students succeed or fail we need to see the teacher’s contribution for what it is – one part of a much bigger picture. Once we take this rational view, we can begin building up an evidence base – perhaps through long-term case studies – that might help us to better understand the interaction between the complex variables we work with.

3. The regression to the mean delusion is a common fallacy. If a variable is extreme on its first measurement, it is likely to be much closer to average the second time it is measured.

Let’s use behaviour in class as an example. Imagine one Tuesday your class are particularly badly behaved – the worst you have seen them. The next lesson you try something different: perhaps a more engaging starter, or a new seating plan, or bawling at them like a sergeant major or threatening them all with a lunch-time detention. That lesson the class are better behaved, so you believe your newly adopted strategy to be an out-and-out success.

Now perhaps it really was an out-and-out success in its own right, but the improvement in the class’ behaviour may only have been a regression to the mean. When you have witnessed extreme, beyond-the-ordinary behaviour from a class, it is statistically more likely that behaviour will be better the next lesson – i.e they return back towards the norm. In the same way, if a problematic class are better behaved than usual for one lesson, be prepared for them to return back to their old tricks in the subsequent one. In both cases, they have regressed to the mean.

The takeaway here – applicable to all strategies and interventions undertaken in a school – is to be careful not to jump to assumptions too quickly. When you witness extremes of success or failure out of kilter with the norm, be vigilant not to immediately attribute this to the intervention itself. Until you have more robust evidence, be careful not to jump to conclusions too quickly. Try out the intervention a few more times and if you do choose to share it with colleagues before you have strong evidence, do so tentatively.

***

There are many, many more biases worth knowing about – I cannot share them all here. What is most critical, however, is that we question, test and evaluate our assumptions and intuitions, whatever our position in the educational system – from trainee teacher to Secretary of State. Unless the evidence is incontrovertible (and it almost never is) always avoid the simplistic answer, however seductive it sounds.

So, does this mean that we should adopt a nihilistic, iconoclastic standpoint, always seeking to pour scorn on tradition or the latest initiative (or both)? Absolutely not. A few things work very well in education; many work partially well; some do not work at all.  Sometimes our intuitions are absolutely right, sometimes they are not. We should not embrace uncertainty, but instead accept it as the starting point on a journey that might lead us towards greater certainty. More robust judgements and decisions allow us to tweak and modify our actions more confidently. However, these are only possible in a climate of enquiry, honesty and humility. The system has a lot to do until we reach that point.

Further reading:

I loved this post by Harry Fletcher-Wood on ways of using evidence in education.