arrow_back Back to Teachers

Recommended Educational Research Papers for Teachers to Read

My interest in educational research only really began when I spoke to the likes of Dylan Wiliam, Daisy Christodoulou, Will Emney, Mark McCourt, Bruno Reddy, Greg Ashman and Tom Bennett for my Mr Barton Maths Podcast. Up until that point I guess had had been blindly going about my business, teaching based largely on gut instinct and the snippets of advice that had been popular throughout the first few years of my teaching career (learning styles, to name but one!). I had never really questioned whether what I was doing was the best for my students... until now (notice the pause for dramatic effect).

This page contains the academic research I have read to date that has influenced the way I plan and teach mathematics. It is meant as a complement to my Recommended Books for Teachers to Read page. It will be continually updated. I have divided it up into broad categories, but these are somewhat arbitrary due to the many interconnections between the papers. The recurring theme throughout everything reviewed on this page is trying to improve students' learning (i.e. their understanding, storage, retrieval and ability to transfer) using evidence based research. As well as linking to the original paper, I have also tried to summarise my main takeaway from it, explaining the practical ways it has changed the way I plan, teach and help my students.

Care must be taken when approaching educational research. There is just so much of it out there, and you can pretty much find a study or a piece of research to support any view you might hold. The problem is that any research conducted in laboratories may have questionable application to classroom environments, whereas any research conducted in classrooms is, by their very nature, prone to influence by many extraneous factors. Greg Ashman provides a very useful guide here explaining what he looks for when evaluating a piece of research. On this page I have tried to follow these guidelines wherever possible, but please be aware that I am as prone to bias as anyone!

I have also written two workshop based on these findings, and my subsequent experience trying my takeaways from them with my students, called "The many ways Educational Research has changed the way I teach Mathematics" and "How Cognitive Load Theory changed my life" For more information, please visit the Mr Barton for Hire page.

I really hope you find this selection of research useful.

Contents

Cognitive Sciencekeyboard_arrow_up
Back to Top

Cognitive Science (along with Cognitive Psychology - often the two are hard to separate) is concerned with how people think, what motivates them to think, the importance of what they think about, and what conditions foster effective thinking. For Daniel Willingham, memory is the residue of thought (I love that phrase!), so if we can help students think, then we can help them remember and learn.

Research Paper Title:
The Science of Learning
Author(s): Deans for Impact
My Takeaway:
This paper was a game-changer for me, and is a superb way to begin any voyage into the world of educational research. It is a summary of existing cognitive science research about how students learn. Each of the cognitive principles is stated clearly, and is presented alongside classroom implications, making it both practical and incredibly teacher-friendly. All of the principles identified are fascinating (and references are provided if you wish to dig into each one deeper), but what struck me in particular were the common misconceptions about how students think and learn: Students do not have different “learning styles”; Humans do not use only 10% of their brains; People are not preferentially “right-brained” or “left brained” in the use of their brains; Novices and experts cannot think in all the same ways; and Cognitive development does not progress via a fixed progression of age-related stages. How many of those have you heard? How many have you had training on? As I say, this was the paper that first opened my eyes.
My favourite quote:
The Science of Learning does not encompass everything that new teachers should know or be able to do, but we believe it is part of an important — and evidence-based — core of what educators should know about learning. 

Research Paper Title:
Why don't students like school? Because the mind is not designed for thinking.
Author(s): Daniel T. Willingham
My Takeaway:
This is an excellent summary of one of my all-time favourite books, Why Don't Students Like School. There is so much to takeaway from this, but the biggest thing for me is about how students think. Students (like all of us) are naturally curious and enjoy thinking, but that curiosity is fragile, and if the conditions are not right then students will avoid thinking. For Willingham, memory is the residue of thought, so unless students are thinking, there is little chance they will remember or learn. Willingham stresses that successful thinking requires three things:
1) information from the environment
2) facts and procedures stored in long term memory
3) space in working memory.
When students are given new information, they hold it in working memory as they connect it to other new information and experiences and evaluate it against known concepts. The problem is that working memory capacity is limited, and gets filled when students need to carry out basic procedures or search for facts. Hence, when faced with complex problems, if those facts and procedures are missing from long term memory, working memory gets filled up, thinking becomes hard, students stop doing it, and no learning takes place. Here is the key point: experts and novices think differently. Experts rely on retrieving whole schema (connected items of information) from long-term memory to get around the limits on our fragile working memories. Novices (i.e. most students) don’t have these memorised schema to rely on so attempt to hold too much information in their working memories which leads to cognitive overload. Therefore, we can make thinking easier for students by ensuring they have access to sufficient knowledge and procedures stored in long term memory, which allows them to develop these schema, which frees up capacity in working memory, which can then be used for problem solving. As teachers we can help students acquire this knowledge and these procedures via Explicit Instruction, emphasising the value of deliberate practice to improve, not taking study skills for granted, and praising effort not ability. Furthermore, once embedded, we can help students retain this knowledge and these procedures using the work discussed in the three Memory sections. It's as easy as that ;-)
My favourite quote:
Teachers often seek to draw students in to a lesson by presenting a problem that they believe interests students, or by conducting a demonstration or presenting a fact that they think students will find surprising. In either case, the goal is to puzzle students, to make them curious. This is a useful technique, but it’s worth considering whether these strategies might also be used not at the beginning of a lesson, but after the basic concepts have been learned.

Research Paper Title: Is Working Memory Training Effective? A Meta-Analytic Review
Author(s): Monica Melby-Lervåg and Charles Hulme
My Takeaway:
A question that might arise from reading Willingham's work is "can we increase working memory capacity?", because if we can then thinking will be a lot easier for students. In recent years, the there has been an increase in the popularity of brain training games and strategies, which claim to boost this all important working memory capacity. Unfortunately, the findings of this comprehensive meta-analysis of existing research suggest that any benefits are short-term, and crucially do not transfer to other situations. So, whilst you may be able to train your brain to hold more of a specific type of written or oral information, this will not transfer across to enable you to solve more complex maths problems. However, all is not lost, because we can get around the limits of working memory using the power of knowledge, as the next two papers in this section discuss.
My favourite quote:
Currently available working memory training programs have been investigated in a wide range of studies involving typically
developing children, children with cognitive impairments (particularly ADHD), and healthy adults. Our meta-analyses show clearly that these training programs give only near-transfer effects, and there is no convincing evidence that even such near-transfer effects are durable. The absence of transfer to tasks that are unlike the training tasks shows that there is no evidence these programs are suitable as methods of treatment for children with developmental cognitive disorders or as ways of effecting general improvements in adults’ or children’s cognitive skills or scholastic attainments.


Research Paper Title: A Simple Theory of Complex Cognition
Author(s): John R. Anderson
My Takeaway:
I am not entirely sure that "simple" is the best description of this theory, at least not for me anyway! But the main message is a crucial one: to be able to think, we need knowledge. Long-term memory is capable of storing thousands of facts, and when we have memorised thousands of facts on a specific topic, these facts together form what is known as a schema - connected items of information. When we think about that topic, we use that schema. When we meet new facts about that topic, we assimilate them into that schema. In other words, when students are given new information, they hold it in working memory as they connect it to other new information and experiences and evaluate it against known concepts. Crucially, if we already have a lot of facts in that particular schema, it is much easier for us to learn new facts about that topic. New knowledge builds on existing knowledge. And how best to learn and retain these facts? Well, the sections on Explicit Instruction, Cognitive Load Theory, and Memory should provide some answers.
My favourite quote:
All that there is to intelligence is the simple accrual and tuning of many small units of knowledge that in total produce complex cognition. The whole is no more than the sum of its parts, but it has a lot of parts.

Research Paper Title: How Knowledge Helps
Author(s): Daniel T. Willingham
My Takeaway:
This is a great paper, with the same powerful message as the one above by Anderson: knowledge does much more than just help students hone their thinking skills: it actually makes learning easier. Willingham argues - providing links to research along the way - that knowledge helps students take in more information, think about new information, and remember new information. What will perhaps be of most interest to teachers is that knowledge helps students solve problems. This sounds ridiculously obvious, but I don't think it is - at least not to me, anyway! One teaching strategy that I have been guilty of over my career is assuming that exposing students to lots of problems will help them become good problem solvers. The simple answer is that it won't. Without existing knowledge, the student cannot recognise familiar patterns (the ability to "chunk" information is crucial here and is related to the concepts of schema above), and hence attempts to solve problems individually, which inevitably overburdens working memory. This issue is discussed more in the Problem Solving section.
My favourite quote:
Those with a rich base of factual knowledge find it easier to learn more—the rich get richer. In addition, factual knowledge enhances cognitive processes like problem solving and reasoning. The richer the knowledge base, the more smoothly and effectively these cognitive processes—the very ones that teachers target—operate. So, the more knowledge students accumulate, the smarter they become.

Research Paper Title: Categorization and Representation of Physics Problems by Experts and Novices
Author(s): Michelene T H Chi, Paul J Feltovich and Robert Glaser
My Takeaway:
I found this study fascinating. We have seen in the paper above that knowledge is crucial to aid thinking, and that an obvious distinguishing feature between experts and novices is that experts have more knowledge than novices. Here we have clear evidence of just how big an advantage that extra knowledge is - it actually enables exerts to see problems differently. The authors asked physics novices and experts (all undergraduate students) to sort physics problems into categories. The novices sorted by the surface features of a problem—whether the problem described springs, an inclined plane, and so on. The experts, however, sorted the problems based on the physical law needed to solve it (e.g., conservation of energy). The advantage of the latter approach was that the exerts could see the deeper structure of the problem, which meant they could more readily access the appropriate schema to solve the problem, which aided transfer. Novices, on the other hand, focused on the surface structure and hence did not make the necessary connections between problems. Experts don't just know more than novices—they actually see problems differently. An immediate implication for me here is going through past papers with students. How often do we find that students focus on the surface structure of a problem? A question about lowest common multiple set in the context of bus timings on one paper becomes a question all about buses, and no connection is made between a question with essentially the same deep structure but set in the context of boxes of cakes and donuts on another paper. They are treated as two completely separate problems. Without strong, domain-specific knowledge, students are unlikely to recognise the deep structure of problems, and hence will be unlikely to transfer the knowledge successfully to different situations. Going through past papers, covering loads of different topics, before learners are secure i each of those topics, is likely to be a waste of time. It is perhaps better to focus on domain-specific knowledge (i.e. teach the basics of lowest common multiple so the calculations become fluent) and then carefully expose students to problems in different contexts but with the same deep structure, guiding them by means of example-pairs, clearly articulating your thought processes. These techniques are covered in detail in the Cognitive Load Theory and Problem Solving sections.
My favourite quote:
Our research goal has been to ultimately understand the difference between experts and novices in solving physics problems. A general difference often found in the literature and also in our own study is that experts engage in qualitative analysis of the problem prior to working with the appropriate equations. We speculate that this method of solution for the experts occurs because the early phase of problem solving (the qualitative analysis) involves the activation and confirmation of an appropriate principle-oriented knowledge structure, a schema. The initial activation of this schema can occur as a data-driven response to some fragmentary cue in the problem. Once activated, the schema itself specifies further (schema-driven) tests for its appropriateness. When the schema is confirmed, that is, the expert has decided that a particular principle is appropriate, the knowledge contained in the schema provides the general form that specific equations to be used for solution will take.

Research Paper Title:
Top 20 Principles from Psychology for PreK-12 Teaching and Learning
Author(s): American Psychological Association
My Takeaway:
This is a wonderful collection of key principles from psychology that have direct implications for teaching and learning, all of which are backed up by research. Many of these have directly influenced how I plan and deliver my lessons, but for the sake of brevity, I will focus on just three here:
1)  What students already know affects their learning. This so ridiculously obvious, but is also ridiculously easy to overlook - at least for me, anyway. The authors explain that learning consists of either adding to existing student knowledge (conceptual growth), or transforming or revising student knowledge (conceptual change). Conceptual growth cannot occur is their existing knowledge is incomparable with the new knowledge, or incomplete, and strategies such as Formative Assessment may be needed to establish this. Likewise, conceptual change can be tricky to achieve if students hold misconceptions, and simply telling students they need to think differently will generally not lead to substantial change in student thinking. Ways to deal with this are covered in the Explicit Instruction section. 
2) Students tend to enjoy learning and to do better when they are more intrinsically rather than extrinsically motivated to achieve. This is directly related to research on Motivation discussed later on this page. As students develop increasing competence, the knowledge and skills that have been developed provide a foundation to support the more complex tasks, which become less effortful and more enjoyable. When students have reached this point, learning often becomes its own intrinsic reward. This is obviously preferable to being externally motivated, as students may disengage once the external rewards are no longer provided. However, one point I feel is worth mentioning is that some (or even all) students may need a certain degree of external motivation during the initial stages of learning a topic, before mastery has a chance to become its own reward.
3) Students persist in the face of challenging tasks and process information more deeply when they adopt mastery goals rather than performance goals. This is very much related to the work on student Mindset, but has some very practical applications. Students can engage in achievement activities for two very different reasons: They may strive to develop competence by learning as much as they can (mastery goals), or they may strive to display their competence by trying to outperform others (performance goals). Performance goals can lead to students’ avoiding challenges if they are overly concerned about performing as well as other students. In typical classroom situations, when students encounter challenging materials, mastery goals are generally more useful than performance goals. How do we help develop these mastery goals? The authors advise strategies such as avoiding social comparisons, avoiding non-task-specific feedback such as "brilliant", conduct student evaluations in private, and focus on improvement over achievement.
My favourite quote:
Psychological science has much to contribute to enhancing teaching and learning in the classroom. Teaching and learning are intricately linked to social and behavioral factors of human development, including cognition, motivation, social interaction, and communication. Psychological science can also provide key insights on effective instruction, classroom environments that promote learning, and appropriate use of assessment, including data, tests, and measurement, as well as research methods that inform practice. We present here the most important principles from psychology—the “Top 20”—that would be of greatest use in the context of preK–12 classroom teaching and learning, as well as the implications of each as applied to classroom practice
.

Research Paper Title:
When More Pain is preferred to Less: Adding a Better End
Author(s): Daniel Kahneman, Barbara L. Fredrickson, Charles A. Schreiber, and Donald A. Redelmeier
My Takeaway:
Often I would end my lessons with a tricky question - maybe a past exam question. This would be the most difficult question I had asked all lesson, and its purpose would for both myself and my students to see how far they had come. I would usually build it up: "okay, this is as tough as it gets. Can you do it?". And often the result was that some could and some couldn't, and I was okay with that as it was meant as an extension question. But having read this paper, I now look at it from the perspectives of the students who could not do the question. What is their  impression of that lesson? What will they remember? Will it be the 45 minutes of success they enjoyed at the start, or the 5 minutes of "failure" at the end. Cognitive psychology suggests the latter. This fascinating paper looks at how people's memory of an experience is often dominated by the feelings of pain and discomfort during the final moments, as opposed to what happened during the rest of the experience. Applying this to my students, when thinking about their maths lesson, many of them would have judged it as a failure because of that final, tricky problem. This negative emotion - potentially swirling around their heads the couple of days until their next lesson - could lead to a subsequent lack of confidence and a lack of engagement in mathematics, neither of which are conducive to learning. Hence, I now always end my lesson with a question that is of mid-range difficulty, or maybe even easier. The majority of the learning has happened in the first 45 minutes - my objective in those last 5 minutes is for my students to feel good about themselves ready for their next maths lesson.
My favourite quote:
The results add to other evidence suggesting that duration plays a small role in retrospective evaluations of aversive experiences; such evaluations are often dominated by the discomfort at the worst and at the final moments of episodes.

Research Paper Title: Organizing Instruction and Study to Improve Student Learning
Author(s): Harold Pashler et al
My Takeaway:
Much like The Science of Learning above, this is an outstanding overview of key research findings from cognitive psychology that have direct implications for the classroom. A wonderful feature of this paper is that it also addresses potential roadblocks to implementing the finding in the classroom and suggests possible strategies to overcome them. The recommendations, each of which are covered in a comprehensive and yet easy to follow way, are: Space learning over time, Interleave worked example solutions and problem-solving exercises, Combine graphics with verbal descriptions, Connect and integrate abstract and concrete representations of concepts, Use quizzing to promote learning, Help students allocate study time efficiently, Help students build explanations by asking and answering deep questions. These recommendations have implications for planning, teaching, and helping students formulate effective revision strategies.
My favourite quote:
We recommend a set of actions that teachers can take that reflect the process of teaching and learning, and that recognizes the ways in which instruction must respond to the state of the learner. It also reflects our central organizing principle that learning depends upon memory, and that memory of skills and concepts can be strengthened by relatively concrete—and in some cases quite nonobvious - strategies

Research Paper Title: Do Visual, Auditory, and Kinesthetic Learners Need Visual, Auditory, and Kinesthetic Instruction?
Author(s): Daniel T. Willingham
My Takeaway:
When I first started teaching 12 years ago, Visual, Auditory and Kinesthetic learning (or the VAK model) was all the rage. My students had to fill out surveys which indicated what type of learner they were, and I had to adapt my lessons accordingly. Needless to say, it was chaos (how exactly do you make solving quadratic equations appropriate for a kinesthetic learner? Maybe do a dance about them?). This wonderful paper presents a clear summary of the evidence on learning types, along with some key implications:
1) Some memories are stored as visual and auditory representations—but most memories are stored in terms of meaning. How you first learn something may be in a visual or auditory way, but how you remember it is tied to what it actually means. When I think about how to add two fractions together (as I regularly do), that information is neither stored in an visual or auditory way. I just know it.
2) The different visual, auditory, and meaning-based representations in our minds cannot serve as substitutes for one another. This is crucial. Our minds have these different types of representations for a reason: Different representations are more or less effective for storing different types of information. The particular shade of green of a frozen pea would be stored visually because the information is inherently visual, whereas the sound of my wife's voice shouting at me is stored in auditory form. These cannot be swapped around.
3) Children probably do differ in how good their visual and auditory memories are, but in most situations, it makes little difference in the classroom. This is the big one. It's likely that some students should have a relatively better visual memory or auditory memory, but that doesn't mean we should always teach to it. The key is that teachers should focus on the content's best modality—not the student's. I teach geometry topics in a visual way, regardless of the preferred learning styles of my students, because even a so-called auditory learner will understand it better that way.
My big takeaway is that it is far more important to carefully consider the best form of presentation for a concept than to worry about catering for each and every child's differing preferences.
My favourite quote:
Experiences in different modalities simply for the sake of including different modalities should not be the goal. Material should be presented auditorily or visually because the information that the teacher wants students to understand is best conveyed in that modality. There is no benefit to students in teachers' attempting to find auditory presentations of the Mayan pyramids for the students who have good auditory memory. Everyone should see the picture. The important idea from this column is that modality matters in the same way for all students.

Research Paper Title: Are Sleepy Students Learning?
Author(s): Daniel T. Willingham
My Takeaway:
Some of my Year 11 lads yawn more than they speak, so I was fascinated by this paper. I was fascinated to find out that children undergo a biological change (a change in chronotype) in their teenage years that gives them a preference to staying up late versus getting up early. This effect is compounded if teenagers use electronic devices (phones, tablets, consoles watching tv etc) late into the evening as exposure to the back-lit screens makes them wakeful. And what effect does this lack of sleep have? In sum, sleep deprivation influences many (but not all) aspects of children's mood, cognition, and behaviour. Lack of sleep is associated with poorer school performance as rated by students themselves and by teachers. Restricted sleep is also associated with lower grades in studies in the United States, a finding replicated in Norway and Korea. So, what can we teachers do to help? Unfortunately, not a great deal. The two things that would likely make the most difference are out of our control - the time students go to bed, and the time lessons start in the morning. In terms of the latter, studies suggest the later the better, and interesting implications for school administrators are discussed. That finding is interesting, because I always have a preference for teaching morning lessons versus those in the afternoon - trying to get anything out of some classes during the last period of the day can be torture! But it seems likely that Period 1 might be difficult as well! My takeaway is to make the most of the lessons later in the morning, and crucially to pass these findings onto students and their parents. In terms of the work they do outside of the lesson, if your students can combine a good night's sleep with the strategies advocated in the Revision session, then... well, maybe you might even end up with a good night's sleep yourself :-)
My favourite quote:
Inadequate sleep represents a challenge to educators that is in one sense overt—teachers see students drowsy in class every day—and in another sense subtle, because it seems like a common nuisance rather than a real threat to education. And indeed, the problem should not be overstated, at least insofar as it affects education. The impact of typical levels of inadequate sleep on student learning is quite real, but it is not devastating. All the same, its impact lasts for years, and there is every reason to think that it is cumulative.

Explicit Instructionkeyboard_arrow_up
Back to Top

Explicit instruction may be thought of as teacher-led instruction. It is more interactive than simply lecturing, involving questioning and responsive teaching, but a key characteristic is that the teacher dictates the content and structure of the lesson, in contrast to more student-centered approaches. I often think of Explicit Instruction as comprising of four elements: explaining, modeling, scaffolding and practising. A great deal of research suggests this is the most effective way to help children learn, and despite having moved away from it over the course of my career, it is now the central structure upon which I build my lessons. These papers discuss the advantage and techniques involved in effective explicit instruction, which are then examined further in the section on Cognitive Load Theory.

Research Paper Title: Follow Through Evaluation (and for a lovely online summary of Project Follow Through and the work of Siegfried Engelmann click here)
Author(s): Siegfried Engelmann
My Takeaway:
When I interviewed Dylan Wiliam for my podcast, he described Project Follow Through as the most important piece of educational research ever conducted - and yet I had never heard of it. Project Follow Through was the most extensive educational experiment ever conducted. Beginning in 1968 under the sponsorship of the US government, it was charged with determining the best way of teaching at-risk children from kindergarten through to grade 3. Over 200,000 children in 178 communities were included in the study, and 22 different models of instruction were compared. The results were startling.  Siegfried Englemann's model of Direct Instruction outperformed all other models in basic skills, and indeed all but two of the other models returned negative numbers, which means they were soundly outperformed by children of the same demographic strata who did not go through Project Follow Through. But what is perhaps most interesting is that Direct Instruction also outperformed all other models on DI was not expected to outperform the other models on cognitive skills, which require higher-order thinking. Indeed, it was the only model to return positive scores across reading, maths concepts and and maths problem solving. And if that wasn't enough, students who followed the Direct Instruction Model also had higher self esteem and self-confidence. This image presents these findings clearly. The thing that struck me most was the effect Direct Instruction had on problem solving skills. I had always assumed that you use instruction to get the basics sorted, and then things like investigations, puzzles and rich tasks to develop students problem solving. But, as we shall see, it is a lot more complicated that than, and models of direct and explicit instruction are key to developing the problem solving skills that we all want our students to have.
My favourite quote:
The basic problem we face is that the most popular models in education today (those based on open classrooms, Piagetian ideas, language experience, and individualized instruction) failed in Follow Through. As a result there are many forces in the educational establishment seeking to hide the fact that Direct Instruction, developed by a guy who doesn’t even have a doctorate or a degree in education, actually did the job. To keep those promoting popular approaches from hiding very important outcomes to save their own preconceptions will take formidable help from persons like yourself. We hope it is not too late.

Research Paper Title:
Why Minimal Guidance During Instruction Does Not Work (there is also a shorter, easier to digest version Putting Students on the Path to Learning, along with an amazing sketchnote summary from Oliver Caviglioli)
Author(s): Paul A. Kirschner and John Sweller
My Takeaway:
This article, possibly more than any other on this page, changed my way of thinking. It provides evidence that for everyone apart from experts, partial guidance during instruction (the kind used in inquiry-based learning or project work) is less effective than full guidance (direct or explicit instruction). This suggests that practice (often encouraged by Ofsted and senior management alike) of encouraging students to "discover" learning, and for the teacher to be the "facilitator" of learning and not the expert imparting their knowledge, is detrimental to students' learning and long term development. Without facts and procedures stored in long term memory, students cannot become the problem solvers we all want them to be. Solving the kind of problems the students encounter during projects, inquiries or even multi-mark exam questions requires knowledge and procedures stored in long term memory. Without these - and crucially without guidance from the teacher - the student's fragile working memory is faced with trying to process too much information: What is the question asking? What knowledge do I need? What procedures do I need? Now how do I actually carry out out that procedure? This can lead to Cognitive Overload, which will be covered in detail later. I have four major takeaways from this:
1) Students can look like they are working hard on complex problems, and not actually be learning anything - this is the concept of problem-solving search, and will be addressed in the Cognitive Overload section.
2) You cannot teach students to become problem solvers by simply giving them loads of different problems to solve. They need to be expertly supported by the teacher, and the problems selected carefully. These concepts will be covered in more detail in the Problem Solving section.
3) It is quite likely that throughout the course of minimal guided instruction, students will develop misconceptions, or incomplete and disorganised knowledge. This is because there is necessarily less control over the direction of their learning - which for many years I assumed to be a good thing!
4) I love an inquiry or a rich tasks, but now when considering one, I think about the opportunity cost. I ask myself: is there a more efficient way of my students acquiring the knowledge I want them to? When asking myself this these days, more often than not the answer is "yes". If I can tell students the knowledge - having clearly and carefully planned out my explanation - and then get them to practice applying that knowledge to begin the process of committing it to long term memory, as opposed to hoping they discover it, and then having to correct and reteach any areas that are missing, can I really justify not doing so?
Finally, this paper is also careful to point out that full guidance may not be the most suitable instructional technique for students once they reach mastery, or become experts - but all too often I have assumed students are at that level before they actually are.
My favourite quote:
After a half-century of advocacy associated with instruction using minimal guidance, it appears that there is no body of research supporting the technique. In so far as there is any evidence from controlled studies, it almost uniformly supports direct, strong instructional guidance rather than constructivist-based minimal guidance during the instruction of novice to intermediate learners. Even for students with considerable prior knowledge, strong guidance while learning is most often found to be equally effective as unguided approaches. Not only is unguided instruction normally less effective; there is also evidence that it may have negative results when students acquire misconceptions or incomplete or disorganized knowledge.

Research Paper Title: Principles of Instruction
Author(s): Barak Rosenshine
My Takeaway:
I absolutely love this paper. It presents 10 research-based principles from cognitive science and studies of master teachers, together with practical strategies for classroom implantation. The 10 principles are all based around the model of explicit instruction, and sound so simple when you see them written down:
1) Begin a lesson with a short review of previous learning.
2) Present new material in small steps with student practice after each step
3) Ask a large number of questions and check the responses of all students
4) Provide models.
5) Guide students practice.
6) Check for student understanding.
7) Obtain a high success rate.
8) Provide scaffolds for difficult tasks.
9) Require and monitor independent practice.
10) Engage students in weekly and monthly review.
For me, this is the structure of a very well planned lesson, encompassing aspects of spacing, retrieval, worked examples, modelling, formative assessment and problem solving, all of which will be discussed in greater detail further on this page. For each element there are links to further research to support the author's claim.
My favourite quote:
Education involves helping a novice develop strong, readily accessible background knowledge. It's important that background knowledge be readily accessible, and this occurs when knowledge is well rehearsed and tied to other knowledge. The most effective teachers ensured that their students efficiently acquired, rehearsed, and connected background knowledge by providing a good deal of instructional support. They provided this support by teaching new material in manageable amounts, modeling, guiding student practice, helping students when they made errors, and providing for sufficient practice and review. Many of these teachers also went on to experiential, hands-on activities, but they always did the experiential activities after, not before, the basic material was learned.

Research Paper Title: What makes great teaching? Review of the underpinning research
Author(s): Robert Coe, Cesare Aloisi, Steve Higgins and Lee Elliot Major
My Takeaway:
This report from CEM, Durham University and the Sutton Trust seeks to answer the simple, but unsurprisingly complex, question of "What makes good teaching?". The authors outline six common components of good teaching and rank them by how strong the evidence is in showing that focusing on them can improve student outcomes. The six components are:
1) Content knowledge (Strong evidence of impact on student outcomes)
2) Quality of instruction (Strong evidence of impact on student outcomes)
3) Classroom climate (Moderate evidence of impact on student outcomes)
4) Classroom management (Moderate evidence of impact on student outcomes)
5) Teacher beliefs (Some evidence of impact on student outcomes)
6) Professional behaviours (Some evidence of impact on student outcomes)
Each of these is worthy of further discussion, but I want to focus on the first two. There are two aspects to content knowledge. Firstly, teacher need to have a strong, connected understanding of the material being taught. But in addition to this, teachers must also understand the ways students think about the content, be able to evaluate the thinking behind non-standard methods, and identify typical misconceptions students have. We have all seen great mathematicians who do not make good teachers as they lack the second of these aspects, but likewise there is no getting around the fact that to be a good maths teacher you need to know your maths. Quality of Instruction concerns the best way to communicate knowledge to students. I can do little better than to quote the report itself: Quality of instruction is at the heart of all frameworks of teaching effectiveness. Key elements such as effective questioning and use of assessment are found in all of them. Specific practices like the need to review previous learning, provide models for the kinds of responses students are required to produce, provide adequate time for practice to embed skills securely and scaffold new learning are also elements of high quality instruction. To me, this seems like a model of effective explicit instruction.
My favourite quote:
There is some evidence that an understanding of what constitutes effective pedagogy – the method and practice of teaching – may not be so widely shared, and even where it is widely shared it may not actually be right. Hence it is necessary to clarify what is known about effective pedagogy before we can think about how to promote it. Unless we do that there is a real danger that we end up promoting teaching practices that are no more – and perhaps less – effective than those currently used.

Research Paper Title: The Role of Deliberate Practice in the Acquisition of Expert Performance (for an easier to read introduction to Deliberate Practice that covers all the key principles, see The Making of an Expert)
Author(s): K. Anders Ericsson, Ralf Th. Krampe, and Clemens Tesch-Romer
My Takeaway:
To become an expert at something, you must practice in the right way. This paper (along with the excellent book, Peak), outlines a model of Deliberate Practice. A key feature of this model is that you breakdown a complex process, isolate an individual skill and work on it, receiving regular and specific feedback so you can improve you performance. Crucially (and this was a game-changer for me), the skill you are practising may not look like the final thing you are working towards. It is why young Spanish footballers start on tiny pitches, playing 5-aside, working on specific drills - you learn far more from that than you ever would in a big 11-aside game where you hardly touch the ball, and there are so many other factors out of your control. It is why professional musicians practice scales over and over again instead of constantly playing full pieces. The implications for teaching are huge. When we teach a topic, say adding fractions, perhaps we need to break the skills down into their most minute components. If we want students to have success in tricky exam questions (e.g. those carrying several marks, or those of a more problem-solving nature)  then it might not be the case that we should present these questions in their final forms. Instead we should break them up, isolate individual skills (such as basic numeracy, or identifying exactly what the question is asking), and practice these in isolation until students have mastered them before even going near an exam. The activities students do in practice may look nothing like what they will be expected to do in the final exam, but that is absolutely fine. Moreover - and for me this is just as important - students need immediate feedback on their practice. Think about a tennis player hitting hundreds of balls during a service drill. Each time they hit a serve, they immediately see the result and can adjust their technique to improve. How can we ensure students can do the same during their practice, so errors are identified and not compounded? It could be as simple as giving them immediate access to the answers. This is why two teachers I interviewed for my Mr Barton Maths Podcast always supply students with the answers - John Corbett during classwork, and Greg Ashman for homework. After all - and this is one of my alll-time favourite phrases - practice doesn't make perfect, practice makes permanent.
My favourite quote:
During a 3-hr baseball game, a batter may get only 5-15 pitches (perhaps one or two relevant to a particular weakness), whereas during optimal practice of the same duration, a batter working with a dedicated pitcher has several hundred batting opportunities, where this weakness can be systematically explored

Research Paper Title: Educating the Evolved Mind: Conceptual Foundations for an Evolutionary Educational Psychology
Author(s): David C. Geary
My Takeaway:
This paper is brilliant. The author argues that there are two types of knowledge and ability: those that are biologically primary and emerge instinctively by virtue of our evolved cognitive structures, and those that are biologically secondary and exclusively cultural, acquired through formal or informal instruction or training. Evolution over millions of years has led to us having brains that eagerly and rapidly acquire those things that are biologically primary, whereas the brain has simply not has enough time to adapt to make biologically secondary knowledge and ability as easy to acquire. Much of what students learn in school is biologically secondary. For me, this has three major implications for teaching. The first is it goes someway to explain why students seem to enjoy speaking to their friends more than engaging in my lovely discussion of how to solve a quadratic equation. The first is biologically primary (folk psychology - interest in people), whereas the second is not. Secondly, it explains why subject such as mathematics should be explicitly taught and not "discovered" through minimal guided instruction - learning maths is simply not natural for humans and hence is too difficult on your own without the help of an expert. Thirdly, it does open up a potential limitation to the model of explicit instruction - we assume students will believe what they are told. When that involves something like how to calculate the percentage of an amount, or how to use trigonometry to find missing sides in right-angled triangles, this is probably fine as students are unlikely to hold any intuitive (i.e. biologically primary), contradictory beliefs. However, I am not convinced that is true for all topics. Take something like probability. Probability is filled with so many counter-intuitive results. The birthday paradox, and the Monty Hall problem are two classics that catch out everyone, but we've all seen students hold erroneous beliefs about the probability of getting a Head and a Tail when tossing two coins, or rolling a "lucky" 6 on a dice. Such results directly contradict the natural intuition held by students. Geary tells us that such biologically primary knowledge is not easy to shake off, so that simply being told you are wrong and what the correct answer is may not be enough. For topics such as these, students may well need to be convinced of the answer - for it is only when they are convinced that their deeply geld intuitions can be changed. Is this also true of misconceptions? Take a skill such as adding two fractions together. By instinct, students are likely to simply add the numerators and denominators. Now, this is perhaps not representative of a biologically primary piece of knowledge, but such misconceptions are so prevalent among students (how many Year 11s have you seen making that mistake despite 6 years of being taught it?), that once again they may need convincing as opposed to just being instructed. How do we convince them? It is not easy, but in short, we need to create cognitive conflict - a concept that will be discussed further in the next paper. Students need to see and believe the real results for themselves. We have traditional tools such as demonstration and mathematical proof, as well as more modern approaches such as Dan Meyer's 3 Act Math structure.
My favourite quote:
From an evolutionary perspective there are several key points: First, secondary learning is predicted to be heavily dependent on teacher- and curriculum-driven selection of content, given that this content may change across and often within lifetimes. Second, for biologically primary domains, there are evolved brain and perceptual systems that automatically focus children’s attention on relevant features (e.g., eyes) and result in a sequence of attentional shifts (e.g., face scanning) that provide goalrelated information, as needed, for example, to recognize other people. Secondary abilities do not have these advantages and thus a much heavier dependence on the explicit, conscious psychological mechanisms of the motivation-to-control model— Ackerman’s (1988) cognitive stage of learning—is predicted to be needed for the associated learning. Third, children’s inherent motivational biases and conative preferences are linked to biologically primary folk domains and function to guide children’s fleshing out of the corresponding primary abilities (see Figure 1.2). In many cases, these biases and preferences are likely to conflict with the activities needed for secondary learning.

Research Paper Title: How Do I Get My Students Over Their Alternative Conceptions (Misconceptions) for Learning?
Author(s): Joan Lucariello and David Naff
My Takeaway:
In the paper above we were introduced to the potential difficulties of relying on explicit instruction to convince students to go against their biologically primary knowledge and intuition. This paper takes this further by looking at practical ways we may help students change the erroneous misconceptions they hold. The author provides a handy list of effective strategies for changing students' misconceptions, including asking them to write down their existing preconceptions of a topic before teaching it, and using diverse instruction to present a few examples that challenge multiple assumptions, rather than a larger number of examples that challenge just one assumption. These are all covered in detail, with practical strategies and supporting research. However, my main takeaway is related to concept of creating cognitive conflict. Cognitive conflict can lead to conceptual change or the accommodation of current cognitive concepts. The authors suggest two strategies for bringing about this cognitive conflict, both of which have direct relevance to mathematics:
1) Present students with anomalous data. This is data that does not accord with their misconception. An example from maths might be to get students to type in 1/3 + 1/4 on their calculator and let them see for themselves that the answer is not 2/7. If they have built up sufficient trust in their calculator over the years, this should be enough to induce cognitive conflict which will make them more amenable to hearing alternative approaches.
2) Present students with refutational texts. A refutational text introduces a common misconception, refutes it, and offers a new (alternative) theory that proves to be more satisfactory. An example might be: "some students think you can just add the numerators and the denominators when adding fractions together. You cannot. If you could, then 1/4 + 1/4 would be 2/8, which is just 1/4. We know 1/4 + 1/4 = 2/4, which shows us that the denominators do not get added together". What I like about this is it confronts the misconception head on, dismisses it with a reason, and then offers a more viable solution. This had really influenced the notes I give my students - no longer just explaining the answer, but confronting wrong answers as well. I feel this is important, because can you fully understand a concept unless you also know the possible misconceptions associated with it? The matter of misconceptions and how to deal with them is discussed further in the Formative Assessment section.
My favourite quote:
Alternative conceptions (misconceptions) interfere with learning for several reasons. Students use these erroneous understandings to interpret new experiences, thereby interfering with correctly grasping the new experiences. Moreover, misconceptions can be entrenched and tend to be very resistant to instruction. Hence, for concepts or theories in the curriculum where students typically have misconceptions, learning is more challenging. It is a matter of accommodation. Instead of simply adding to student knowledge, learning is a matter of radically reorganizing or replacing student knowledge. Conceptual change or accommodation has to occur for learning to happen.Teachers will need to bring about this conceptual change.

Research Paper Title: The Privileged Status of Story
Author(s): Daniel Willingham
My Takeaway:
This lovely paper is directly related to Geary's argument about biologically primary and secondary knowledge and skills above. I include it as a potential model for explicit instruction, as well as to show the explicit instruction is not simply lecturing to students in a dull, uninspiring way.  Here, Willingham argues that planning a lesson around the structure of a story is a good way not just to engage students but to help them learn and retain the content. He argues that stories are both easier to comprehend and easier to remember than if that information was presented another way. Why should this work? Well, to return to Geary, story-telling has existed in human societies since the evolution of language, whereas communicating in written form is a relatively new development, as is sitting in the whole school environment for that matter. Learning from stories may be viewed as more biologically primary than other forms of instruction. With this in mind, Willingham suggests the following key components of a story: causality, conflict, complications and character. How easy is it to structure a maths lesson around these components without it feeling forced, false and downright unbelievable to the students? Well, not as difficult as you might think. Causality is as simple as showing one concepts follows directly from another. Conflict can be presented as a problem we need to solve. Complications exist when our current mathematical tools let us down, or we encounter a surprising result. Character is perhaps a little more difficult, but searching the rich history of maths to see where the great ideas started from can breathe humanity into otherwise abstract ideas. Once again I return to Dan Meyer's 3 Act Math structure, and his amazing aspirin-headache approach, as possible tools to incorporate these key components, but I am convinced that it doesn't even need to be that complicated. Simply considering the structure of a story when planning lessons, possibly incorporating one or two of these components, may just lead to an experience that is easier to understand and more memorable for your students - and that, after all, is the goal of teaching.
My favourite quote:
Screenwriters know that the most important of the four Cs is the conflict. If the audience is not compelled by the problem that the main characters face, they will never be interested in the story. Movies seldom begin with the main conflict that will drive the plot. That conflict is typically introduced about 20 minutes into the movie. For example, the main conflict in Star Wars is whether Luke will succeed in destroying the death star, but the movie begins with the empire's attack on a rebel ship and the escape of the two droids. All James Bond movies begin with an action sequence, but it is always related to some other case. Agent 007's main mission for the movie is introduced about 20 minutes into the film. Screenwriters use the first 20 minutes—about 20 percent of the running time—to pique the audience's interest in the characters and their situation. Teachers might consider using 10 or 15 minutes of class time to generate interest in a problem (i.e., conflict), the solution of which is the material to be learned.

Research Paper Title: Cognitive Supports for Analogies in the Mathematics Classroom
Author(s): Lindsey E. Richland, Osnat Zur and Keith J. Holyoak
My Takeaway:
Related to the importance of a story structure to lessons discussed above comes the finding described in this paper that students respond particularly well to the use of analogies during instruction. The work by Daniel Willingham in the Cognitive Science section outlines how students connect new knowledge to existing knowledge in the formation of schema and how this is crucial to long-term learning, and hence it would appear to be good practice to use anaoliges wherever possible during instruction. The authors identified Six strategies involving analogies after studying maths lessons in the US, Hong Kong and Japan. Teachers:
(A) used a familiar source analog to compare to the target analog being taught;
(B) presented the source analog visually
(C) kept the source visible to learners during comparison with the target
(D) used spatial cues to highlight the alignment between corresponding elements of the source and target (e.g., diagramming a scale below the equals sign of an equation)
(E) used hand or arm gestures that signaled an intended comparison (e.g., pointing back and forth between a scale and an equation);
(F) used mental imagery or visualizations (e.g., “picture a scale when you balance an equation”)
Fascinatingly, each of these strategies were used less by teachers in the US than by teachers in the other two (higher performing) regions. The strategies seem obvious, and I guess I have always been doing parts of them, but knowing these strategies has made me focus on them more explicitly in my delivery of concepts. They all provide that extra support that might be crucial in the early stages of concept development. The researchers offer an explanation (see quote) as to why analogies are successful in maths instruction, and this supports the findings from Cognitive Load Theory. This may go some way to explaining the recent popularity of visual approaches such as Bar Modelling, which could potentially tap into more than one of the strategies outlined above. A word of caution though: unsuccessful analogies may produce misunderstanding that can even lead to nasty, lingering misconceptions. Choose your analogies wisely!
My favourite quote:
If the source analog is not familiar and not visible, then students may struggle with processing. First, students will need to perform a taxing memory search to understand the source. Then, assuming that memory retrieval is successful, lack of visual availability will place further burdens on working memory during production of the relational comparison. Finally, lack of supporting cues to guide the comparison itself may result in the student learning much less than, or something quite different from, the new relational concept the teacher means to convey.

Research Paper Title: Inflexible Knowledge: The First Step to Expertise
Author(s): Daniel T. Willingham
My Takeaway:
We have seen in the Cognitive Science section that experts and novices think in fundamentally different ways due to the more developed set of schema that the former possess. This allows experts to see the deeper structure of problems, and to make the sort of connections and provide the sort of the solutions that are out of the novices' grasp. This lovely paper by Daniel Willingham argues that in order to transform novices to experts we must first accept that they need to first develop inflexible knowledge. Willingham distinguishes between flexible,inflexible and rote knowledge. Knowledge is flexible when it can be accessed out of the context in which it was learned and applied in new contexts. Inflexible knowledge is meaningful, but narrow; it's narrow in that it is tied to the concept's surface structure, and the deep structure of the concept is not easily accessed. Rote knowledge is devoid of meaning. Willingahm's point is that we as teachers often think rote knowledge is bad (which it is), but most of what we consider as rote knowledge may in fact be inflexible knowledge, and the development of inflexible knowledge is a necessary step long the path to expertise. Take something like "angles on a straight line add to 180 degrees". The first time students encounter this, they are unlikely to understand why, and will probably only be able to answer questions that are presented in a straight-forward manner (focusing on the surface structure). Their knowledge still has meaning, albeit limited, hence their knowledge is inflexible not rote. However, the more questions they encounter, and the more situations they see this concept appear in (different shapes, algebraic, etc), then the more they will begin to appreciate the deeper structure and their knowledge becomes more flexible. For me, there are a few interesting takeaways from this:
1) It is fine - and indeed necessary - for students to develop inflexible knowledge of concepts
2) We aid the path to expertise by ensuring this knowledge is secure and then showing it in lots of different scenarios via carefully chosen examples
3) Sometimes it may be sensible to teach the "how" before the "why". This is somewhat controversial, and I first thought about this in my conversation with Dani Quinn on my podcast, but I think it makes sense. Go back to my angles example. Is there any point trying to convince students why the angles on a straight line add to 180 degrees when they first encounter the topic? Is it not better to get them fluent at the calculations, competent at answering questions with the same surface structure, confident in their abilities, and then later on return to the why when they have a greater appreciation of the concept? Relating this to Cognitive Loads Theory, my concern introducing the why and the how at the same time is that it is too much for the students to take in and they end up learning nothing.
My favourite quote:
There is a broad middle-ground of understanding between rote knowledge and expertise. It is this middle-ground that most students will initially reach and they will reach it in ever larger domains of knowledge (from knowing how to use area formulas fluently to mastering increasingly difficult aspects of geometry). These increasingly large stores of facts and examples are an important stepping stone to mastery. For example, your knowledge of calculating the area of rectangles may have once been relatively inflexible; you knew a limited number of situations in which the formula was applicable, and your understanding of why the formula worked was not all that clear. But with increasing experience, you were able to apply this knowledge more flexibly and you better understood what lay behind it. Similarly, it is probably expecting too much to think that students should immediately grasp the deep structure beneath what we teach them. As students work with the knowledge we teach, their store of knowledge will become larger and increasingly flexible, although not immediately.


Cognitive Load Theory keyboard_arrow_up
Back to Top

Dylan Wiliam described Cognitive Load Theory as "the single most important thing for teachers to know", and I have to agree. I find it provides rationale and practical strategies for the principles of Explicit Instruction discussed in the previous section, and it has transformed the way I teach. Indeed, I have felt compelled (much to my wife's dismay) to write a workshop entitled "How Cognitive Load Theory changed my life" - click here for more details. Cognitive Load Theory is focussed upon the conditions that make learning easier for students, and it is fascinating to look at this alongside Bjork's work in the Memory section which extols the advantages of making learning harder. For me, the matter is resolved by using the principles of Cognitive Load Theory to aid the early knowledge acquisition phase, and then utilising concepts such as spacing, interleaving, testing, variation and other desirable difficulties later in the learning process. In this section we look at key "effects" of the Cognitive Load Theory relating to worked examples, presentation of information, goal-free problems and the development of problem solving skills, and the practical implications they have for the classroom.

Before we dive into the research papers, Oliver Caviglioli has produced some amazing sketchnote summaries of the 2011 Cognitive Load Tory book, which really help illustrate the key ideas:
Chapters 1 and 2
Chapters 3 and 4
Chapters 5 and 6
Chapters 7 and 8

Research Paper Title: Cognitive Architecture and Instructional Design (there is also a short review of the development of Cognitive Load Theory that gives a good summary of the key points: Story of a Research Programme)
Author(s): John Sweller, Jeroen J. G. van Merrienboer,  and Fred G. W. C. Paas
My Takeaway:
It is no exaggeration to say that Cognitive Load Theory has changed my life. Well, my teaching life, at least. It explains so many of the struggles and mistakes I have seen my students making over the years, and I am just fuming that it has taken me 12 years to discover it. But better late than never! This paper is an outstanding overview of Cognitive Load Theory, together with its related effects and implications for instructional design. Cognitive Load Theory is concerned with how cognitive resources are focussed and used during problem solving and learning. The key principle is that our working memories are severely limited, and that once a student's working memory reaches "cognitive load", no learning takes place. Indeed, it was a real eye-opener for me to discover that students could be working hard, and yet not actually be learning anything. What can be done about this? Well, as teachers we need to help reduce both the intrinsic and extraneous loads:

Intrinsic load - this is determined by the interaction between the nature of the learning tasks and the expertise of the learner. It depends on the amount of element interactivity in the tasks that must be learned. Something like learning the names of polygons has a relatively low element interactivity, as knowing the name of a pentagon is not necessary to know the name of an octagon - hence they can be learned separately. However, something like solving an equation has a relatively high element interactivity as you need to combine many dependent skills together to arrive at the answer. The degree of element interactivity also depends on the expertise of the learner, because what are numerous elements for a low-expertise learner may be only one or a few elements (i.e. chunks) for a high-expertise learner. As we have seen in previous sections, working memory capacity is severely limited (possibly being as low as 3 to 5 items), but the intrinsic load can be reduced by:
  • By ensuring students have sufficient knowledge, which can be organised in their long-term memory as schema. This allows students to work with a sizeable "chunk" of information (or schema) as if it was one item, which frees up capacity in working memory. As we have seen from the papers by Anderson and Willingham in the Cognitive Science section, knowledge matters!
  • Presenting complementary visual and auditory information together
Extraneous load - this is load that is not necessary for learning (indeed, it is unhelpful), and it typically results from badly designed instruction and the way material is presented to the learner. This extraneous load can be reduced by:
  • The use of goal free problems
  • Making use of worked examples
  • Carrying out completion tasks
  • The careful integration of related information
  • Reducing redundant information
In short, as teacher we need to take steps to focus students' limited working memory capacity on the things that matter, removing everything else, whilst at the same time ensuring students have sufficient background knowledge to better deal with the limitations of their fragile working memories. For an in-depth discussion with a maths teacher who regularly practices the lessons from Cognitive Load Theory, you can listen to my interview with Greg Ashman. The papers that follow in this section look in detail at the concepts introduced above and their implications for explicit instruction.

My favourite quote:
The design of practice and the organization and presentation of information is the domain of instructional designers. Although there are many factors that a designer may consider, the major thesis of this paper is that the cognitive load imposed by instructional designs should be the pre-eminent consideration when determining design structures. Limited working memory is one of the defining aspects of human cognitive architecture and, accordingly, all instructional designs should be analyzed from a cognitive load perspective. We argue that many commonly used instructional designs and procedures, because they were designed without reference to working memory limitations, are inadequate.

Research Paper Title: Cognitive Load Theory, Learning Difficulty and Instructional Design
Author(s): John Sweller
My Takeaway:
As well as providing an excellent overview of Cognitive Load Theory, this paper highlights the contrasting problem solving strategies of novices and experts, and thus introduces one of my favourite concepts: the Goal Free Effect. This has a big implication for teaching, especially when introducing new concepts to students. Goal-specific problems are those in which the top-level goals (i.e. final answer) can only be achieved by successfully completing the sub-goals (i.e. the steps leading up to it). When faced with goal-specific problems, novice learners tend to embark upon a means-end analysis. In essence, this means the student tries to juggle all the possible sub-steps that minimize the difference between the current state (the problem) and the end state (the goal). This is a backwards-thinking approach, and it is no surprise that it quickly overloads working memory. I know I have said this before, but it is worth repeating - even though students are working hard in such a scenario, their cognitive overload means they are not actually learning anything. Just as important is the fact that this means-end approach is incompatible with the development of schema, meaning that whilst the students may solve the specific problem at hand, they are unlikely to actually be learning. Learning is a change in long term memory, not just solving one specific problem. A relatively simple solution is to instead present students with ‘goal-free’ problems, which focuses attention upon working forward from the information present one step at a time, rather than trying to hold multiple possible steps in mind at once. A classic example in maths is instead of giving students a complex, multi-step question whose goal is to find the size of "angle x", instead ask them to find the size of as many angles as they can. Similar things can be done to trigonometry questions, or even when working with cumulative frequency diagrams. This breaks the process down and reduces the burden on novices' working memories. As will be discussed in the remaining papers in this section, once students become expert enough in a given topic, they are likely to have sufficient working memory capacity to dedicate to solving goal-specific problems. The key (and difficulty!) is judging when to introduce them, but in early skill acquisition, goal-free problems seem to be the way to go.
My favourite quote:
This means-ends procedure is a highly efficient technique for attaining the problem-goal. It is designed solely for this purpose. It is not intended as a learning technique and bears little relation to schemas or schema acquisition. In order to acquire an appropriate problem solving schema, students must learn to recognize each problem state according to its relevant moves. Using a means-ends strategy, much more must be done. Relations between a problem state and the goal state must be established; differences between them must be extracted; problem operators that impact favourably on those differences must be found. All this must be done essentially simultaneously and repeated for each move keeping in mind any subgoals. Furthermore, for novices, none of the problem states or operators are likely to be automated and so must be carefully considered. According to cognitive load theory, engaging in complex activities such as these that impose a heavy cognitive load and are irrelevant to schema acquisition will interfere with learning. 


Research Paper Title:
Learning from Examples: Instructional Principles from the Worked Examples Research
Author(s):
Robert K. Atkinson , Sharon J. Derry , Alexander Renkl , Donald Wortham
My Takeaway:
A key component of Cognitive Load Theory is the Worked Example Effect, whereby learners who study worked examples perform better on test problems than learners who solve the same problems themselves. When I first came across this, I didn't believe it - surely you learn more from trying to solve problems than by reading the solution. But the key is in the "trying to solve problems". The rather counter-intuitive result is due to the fact that studying worked examples focuses all attention on the correct solution and procedure, reducing the extraneous load compared to a means-end analysis that the learner may embark upon if left to their own devices. This is likely to lead to the development and acquisition of all-important schema which will help the students transfer their knowledge to related contexts. There are so many fascinating concepts discussed throughout this paper extolling the virtues of worked examples, and when they are most effective. Concepts such as the split-attention effect, and variability of problem types are also discussed, and we will cover these in later papers. The quote I have chosen for this paper summarises the main findings nicely, and serves as an excellent set of guidelines for the use of worked examples during lessons. I am going to focus on one that has directly changed the way I teach - Example-Problem Pairs. I used to do a load of worked examples at the start of the lesson, and then give the students a set of problems to work on for the remainder of the lesson. The problem (and it seems so obvious now) was that by the time I had done the 3rd worked example, students had forgotten the first as they had not had the opportunity to practice it for themselves. Hence, they would start working on the questions, get stuck, and I would need to go through it all again. This paper recommends interleaving worked examples with related questions for students to solve alone. When doing a worked example in class, I now split my board in two, having the worked example on the left, and a mathematically similar example for my students to try themselves immediately afterwards on the right. It sounds so simple, but the positive effects have been quite startling. Greg Ashman discussed this strategy when describing how he plans his lessons when I interviewed him for my podcast, and he has written a blog post all about his use of Example-Problem Pairs here. The more I read about examples, the more obsessed I became, so I have created a separate Examples section.
My favourite quote:
First, transfer is enhanced when there are at least two examples presented for each type of problem taught. Second, varying problem sub-types within an instructional sequence is beneficial, but only if that lesson is designed using worked examples or another format that minimizes cognitive load. Third, lessons involving multiple problem types should be written so that each problem type is represented by examples with a finite set of different cover stories and that this same set of cover stories should be used across the various problem types. Finally, lessons that pair each worked example with a practice problem and intersperse examples throughout practice will produce better outcomes than lessons in which a blocked series of examples is followed by a blocked series of practice problems.

Research Paper Title: Cognitive Architecture and Instructional Design
Author(s): John Sweller, Jeroen J. G. van Merrienboer,  and Fred G. W. C. Paas
My Takeaway:
I include this wonderful paper again as I find it the best for discussing the Completion Problem Effect. There is an obvious danger when just presenting students with worked examples, as opposed to problems to solve, that they do not fully engage with the examples. As all the work has been done for them, where is there incentive to think? And, as we have seen, if students are not thinking then they cannot be learning. I have seen this many times myself - some students will pay little attention during thr worked examples, and then (surprisingly!) get stuck when they are set to work oh their own. This is where the simple idea of requiring students to complete various pieces of information and steps within a worked example comes into play. This could be as simple as creating a full worked solution and then Tipexing or deleting key sections for the students to complete, or even pausing whilst going through a worked example on the board, and asking students to predict (maybe using mini-whiteboards) what comes next. Completion problems provide a bridge between worked examples and conventional problems. The authors put it like thus: worked examples are completion problems with a full solution, and conventional problems are completion problems with no solution. A good progression might be to start with completion problems that provide almost complete solutions, and gradually work to completion problems for which all or most of the solution must be generated by the learners.
My favourite quote:
There is considerable evidence that, compared to conventional problems, they decrease extraneous cognitive load, facilitate the construction of schemas, and lead to better transfer performance. In short duration studies, results indicated that completion problems are equally effective as worked examples intermixed with conventional problems. In studies of a longer duration, completion problems may better help learners to maintain motivation and focus their attention on useful solution steps that are available in the partial examples.

Research Paper Title:
Reducing Cognitive Load by Mixing Auditory and Visual Presentation Modes
Author(s): Seyed Yaghoub Mousavi, Renae Low, and John Sweller
My Takeaway:
This paper introduces both the Split-Attention Effect and the Modality Effect, both of which have significantly changed my teaching.

Split Attention Effect
We have seen that the worked-example effect occurs because worked examples reduce extraneous cognitive load, but there can be no guarantee that all worked examples appreciably reduce cognitive load. This is especially true if the worked example consists of a diagram, and a separate worked solution, neither of which are intelligible without the other. The learner must split their attention between the two forms of presentation, which increases the cognitive load. Hence, when presenting students with worked examples where a diagram is involved (for example, most geometry topics), the text solution should be carefully integrated within the diagram, and not separate from it. This will prevent students from having to deal with these two forms separately and reduce the extraneous cognitive load. Similarly, keeping all aspects of a question visible at the same time (i.e. without having to constantly turn the page over) will also help reduce this unnecessary load.

Modality Effect
To understand the importance of the modality effect we need to understand the components of working memory. The Central Executive acts a bit like a supervisor, The Phonological Loop deals with speech and sometimes other kinds of auditory information, the Visuo-Spatial Sketchpad holds visual information and the spatial relationships between objects, and the Episodic Buffer integrates new information with information already stored in long-term memory. They key point here is that working memory gets overloaded if too much information flows into one of these components, but we can use different components to aid processing. Hence, the capacity of working memory may be determined by the modality (auditory or visual) of presentation, and the effective size of working memory may be increased by presenting information in a mixed (auditory and visual mode) rather than in a single mode. There are two key implications of this for me.
1) Make use of complementary representations of concepts. This will be discussed more in the next paper.
2) When students are presented with written information, talking over the top of it can cause overload as both are processed by the auditory component. So, the simple practice of putting up a slide of text and allowing the students the opportunity to read it in silence BEFORE reading it aloud or discussing it can reduce the split-attention effect by decreasing the extraneous load. For me, the biggest change this has made is encouraging me to shut up a little more. Previously I would start my students off on some problems, and then feel an incessant need to provide a running commentary over the top - "read the question carefully", "remember to show your working", etc. I thought I was being helpful, but in fact I was overloading my students' finite working memories as they were having to process my oral ramblings together with reading the text of the problems they were solving. In short, I was inhibiting their learning.

My favourite quote:
We began by indicating that basic research into the characteristics of working memory has suggested that this processing system is divided into at least two partially independent subprocessors: an auditory system devoted heavily to language and a visual system for handling images, including writing. Because both systems can be used simultaneously, limited working memory capacity might be effectively increased if information that must be stored or simultaneously processed is presented in a manner that permits it to be divided between the two systems, rather than processed in one system alone. As a consequence, informationally equivalent material that may be difficult to process in a purely visual manner may be more easily handled if it can be presented partially in both modalities.

Research Paper Title: The Instructive Animation: Helping Students Build Connections Between Words and Pictures in Multimedia Learning
Author(s): Richard E. Mayer and Richard B. Anderson
My Takeaway:
This paper further discusses the Modality Effect in the context of a dual-coding model. The authors conducted an experiment in which students studied an animation depicting the operation of a bicycle tire pump or an automobile braking system, along with concurrent oral narration of the steps in the process, and tested their performance on both a retention test and a problem solving test against groups who had the animation alone, narration alone, or no instruction. On both tests, the group who had the animation alongside the oral narration performed the best. The authors conclude that these results are consistent with a dual-coding model in which retention requires the construction of "representational connections" and problem solving requires the construction of "representational and referential connections". The obvious implication for teaching from this paper is that pictures and words together can be more effective than pictures or words alone. If the oral description is there to support the comprehension of the animation, then it should benefit students by easing the strain placed on their working memories. Hence, I now make more use of carefully selected diagrams, as well as GIFs, Geogebra demonstrations and interactive Desmos graphs. For example, introducing circle theorems using something like this has really helped my students grasp the key concepts. But it is obviously a fine balance. In the Real Life Maths section, I discuss a paper where the use of video to support maths comprehension had a negative effect on performance. For what it is worth, having carefully read the last two papers, my advice is as follows. For occasions where either text alone or a diagram alone will not adequately describe the situation to students (such as a geometric problem), then carefully integrate the two together. Where one or the other may suffice, I would advise presenting them separately and then together. For example, if I was demonstrating the lines of symmetry of a rectangle, I would show the diagram, give the students chance to process it, and then provide an oral description whilst the diagram was still on the screen. Hopefully that would have the effect of reducing the cognitive load on working memory, whilst also taking advantage of the concurrent presentation of information outlined in this paper. 
My favourite quote:
What makes an instructive animation? The results presented in this article demonstrate that animation per se does not necessarily improve students' understanding of how a pump or a brake works, as measured by creative problem solving performance. For example, in both experiments, students who received animation before or after narration were able to solve transfer problems no better than students who had received no instruction. In contrast, when animation was presented concurrently with narration, students demonstrated large improvements in problem-solving transfer over the no-instruction group. We conclude that one important characteristic of an instructive animation is temporal contiguity between animation and narration. We hypothesize that contiguity of words and pictures during instruction encourages learners to build connections between their verbal and visual representations of incoming information, which in turn supports problem-solving transfer.

Research Paper Title: Cognitive Architecture and Instructional Design
Author(s): John Sweller, Jeroen J. G. van Merrienboer,  and Fred G. W. C. Paas
My Takeaway:
I include this wonderful paper again as I find it the best for discussing the Redundancy Effect. In the past I had assumed that simply repeating the same information twice, but in a different form, would at worst have a neutral effect on learning. After all, what harm can it do? Moreover, surely it is good to be told something twice - giving students two opportunities to get it? But if that extra information is redundant - i.e. if students can infer all they need for the initial presentation - then I am likely to be imposing an unnecessary load upon students working memories. We must be careful to distinguish this from the split attention effect. Split-attention occurs when learners are faced with multiple sources of information that must be integrated before they can be understood. The individual sources of information cannot be used by learners if considered in isolation, hence the need for integration. The redundancy effect occurs when multiple sources of information are self-contained and can be used without reference to each other. This is because that redundant information is very difficult to ignore, and hence must be processed in students' limited working memories. The message for me is clear: if a concept cannot be understood without a second piece of information, then carefully integrate the two pieces of information together. If the second piece of information is not needed, then leave it out!
My favourite quote:
Redundancy is a major effect that should be considered seriously by instructional designers. A large range of experimental results indicate the negative consequences of including redundant material when designing instruction. We know of no experimental work demonstrating advantages of redundancy, and we suspect that such a result only could be obtained under conditions where one set of instructional materials was so poor that any redundant alternative would inevitably confer benefits.

Research Paper Title:
Teaching Complex Rather Than Simple Tasks: Balancing Intrinsic and Germane Load to Enhance Transfer of Learning
Author(s): Jeron J G Van Merrienboer, Liesbeth Kster and Fred Paas
My Takeaway:
So far the focus of Cognitive Load Theory has been on making thinking as easy for students in the sense that all their limited working memory capacity should be focused entirely on the thing we want them to think about to prevent cognitive overload occurring. In other words, as teachers we should try to:
1) eliminate unhelpful extraneous load via worked examples, the careful presentation of information and the use of goal free problems
2) reduce intrinsic load by helping students acquire the background knowledge necessary so that sub-components of a complex task are automated.
However, what if we reduce these two types of loads so much that thinking actually becomes too easy? This fascinating (and controversial!) paper addresses this issue by introducing the concept of Germane Load. This can be viewed as "good cognitive load", in that it directly contributes to learning. It does this by aiding the construction of cognitive structures and processes that improve performance. The authors of this paper found that whilst load reducing extraneous load is effective in producing high retention of the material, these techniques hinder the transfer of learning. They argue that there is a need to vary the conditions of practice and only give limited guidance and feedback in order to induce germane cognitive load and improve transfer. In other words, in order to improve learning (in particular the transfer of skills to new contexts), we need to make learning more difficult... but difficult in the right way! This is a concept similar to Bjork's fascinating idea of "desirable difficulties" that will be discussed at length in the Memory section.
The reason this paper is controversial is that one of the originators of Cognitive Load Theory, John Sweller, has distanced himself from the concept of germane load as he believes it makes his theory impossible to falsify. For example, assuming that the overall load is kept constant, a decrease in performance will be attributed to a rise in extraneous load that impairs germane cognitive processes. Conversely, if the performance increases it will be attributed to a germane load enhancement made possible by a drop in extraneous load.
What is my takeaway from all this? Well, I'll be honest - I am not 100% sure! Building in the concept of germane load might be making Cognitive Load Theory unnecessarily complicated. My take is this: during initial skill acquisition we need to ensure thinking is as focused and easy for the student as possible using all the principles we have discussed in the papers above. But, we need to ensure that thinking is not too easy. If students are cruising through lessons on autopilot, then their learning is unlikely to be deep, and learning without the ability to transfer it to new situations is not really learning at all. Of course, this is a fine balance, and will be covered in far more detail in the Memory sections.
My favourite quote:
In general, well-designed instruction should decrease extraneous load and optimise germane load, within the limits of total available capacity in order to prevent cognitive overload. However, this article is mainly about the situation that even after the removal of all sources of extraneous cognitive load, the element interactivity of the complex tasks is still too high to allow for efficient learning. Thus, it is about balancing intrinsic load, which is caused by dealing with the element interactivity in the tasks, and germane load, which is caused by genuine learning processes. The structure of our argument is as follows. First, we discuss research findings indicating that germane-load inducing instructional methods used for practicing simple tasks are not used for practicing complex tasks, at the cost of transfer of learning. Second, we argue that the element interactivity of learning tasks should be limited early in training to decrease their intrinsic load, so that germane-load inducing methods might be used right from the start of the training program.


Research Paper Title:
Cognitive Load during Problem Solving: Effects on Learning
Author(s): John Sweller
My Takeaway:
One huge question from all we have seen so far on Explicit Instruction and Cognitive Load Theory is: "how do we get our students to become good problem solvers?". This paper offers the first clue. For students to become good problem solvers they need to form mental schema from domain-specific knowledge which they can then apply to different situations. Unlike experts, novices lack the appropriate schema to recognise and memorise problem configurations. They set about solving problems by focusing on the detail and ignoring structure, embarking upon a means-end analysis. However, problem solving itself is not an effective way of developing these crucial mental schema. Why? Well, because trying to solve problems (problem-solving search) overloads working memory, and hence the mental schema are not developed. My key takeaway from this is that students do not learn from problem solving. Problem solving must come at the end of the process, after the necessary domain specific knowledge has been learned. Taking this further, I believe there is little point going through lots of difficult exam questions in the hope students understand them and make connections between related questions. I have been there myself - going through a series of unrelated problems with a class of 30 Year 11s, successfully answering my prompt-filled questions, and happily nodding along when I ask them if they get it. But any apparent success from teaching problem solving to novices is likely to be just mimicry. The skills will not be transferred. This will be covered further in the Problem Solving section, but the key point is that if basic skills are not in place, then problem-solving search suggests that there will simply not be enough capacity in working memory for students to develop the mental schema necessary to learn and transfer.
My favourite quote:
Most mathematics and mathematics-based curricula place a heavy emphasis on conventional problem solving as a learning device. Once basic principles have been explained and a limited number of worked examples demonstrated, students are normally required to solve substantial numbers of problems. Much time tends to be devoted to problem solving and as a consequence, considerable learning probably occurs during this period. The emphasis on problem solving is nevertheless, based more on tradition than on research findings. There seems to be no clear evidence that conventional problem solving is an efficient learning device and considerable evidence that it is not. If, as suggested here, conventional problems impose a heavy cognitive load which does not assist in learning, they may be better replaced by nonspecific goal problems or worked examples. The use of conventional problems should be reserved for tests and perhaps as a motivational device.

Research Paper Title: The Expertise Reversal Effect
Author(s): Alexander Renkl and Robert K. Atkinson
My Takeaway:
So far all the talk has been of using Explicit Instruction and the key features of Cognitive Load Theory as the best method of teaching students to become fluent in the facts and procedures they will need to learn more complex skills. Specifically, in early skill acquisition, learning from worked examples is very advantageous, and learning by solving problems is not. However, as this paper describes, instructional techniques that are highly effective with inexperienced learners (novices) can lose their effectiveness and even have negative consequences when used with more experienced learners (experts), hence the Expertise Reversal Effect. The argument is that worked examples contain information that is easily determined by the more experienced learners themselves and, therefore, can be considered redundant. As we have seen via the Redundancy Effect, devoting working memory to redundant information effectively takes away a portion of the learners’ limited cognitive capacity that could be devoted to the more useful germane load. Moreover, this redundant information may even interfere with the schemas constructed by experienced learners, preventing them from seeing the deeper connections in problems that are essential for transfer. For example, what if students have solved a problem differently to how I have presented it in thew worked example? At this stage of development, working through complex problems independently is likely to be more beneficial for long-term learning than studying worked examples. Of course, one major difficulty of this is recognising when students have made the transition from novice to expert and hence can start to be exposed to more complex problems. It is a delicate balancing act! What I have started doing is making worked examples "optional" for students once we have covered the basics of a topic. That way they can judge themselves whether they are at the stage where using worked examples will help or hinder them.
My favourite quote:
When a problem can be solved relatively effortlessly, analyzing a redundant worked example and integrating it with previously acquired schemas in working memory may impose a greater cognitive load than problem solving. Under these circumstances, practice in problem solving may result in more effective learning than studying worked examples because solving problems may adequately facilitate further schema construction and automation

Research Paper Title: Structuring the Transition From Example Study to Problem Solving in Cognitive Skill Acquisition: A Cognitive Load Perspective
Author(s): Alexander Renkl and Robert K. Atkinson
My Takeaway:
This paper offers a slightly different take on problem solving than purported via the Expertise Reversal Effect above, but still under the umbrella of Cognitive Load Theory. The authors propose a fading procedure, in which problem-solving elements are successively integrated into the study of worked examples until the learners are expected to solve problems on their own. The crucial difference here is that students do not need to become "experts" before being exposed to problem solving - it is something that is gradually introduced alongside the acquisition of knowledge. Just like the paper above, we do not start with the problem solving questions. But at the same time the authors caution against introducing problem solving too late.
My favourite quote:
First, a complete example is presented (model). Second, an example is given in which one single solution step is omitted (coached problem solving). Then, the number of blanks is increased step by step until just the problem formulation is left, that is, a to-be-solved problem (independent problem solving). In this way, a smooth transition from modeling (complete example) over coached problem solving (incomplete example) to independent problem solving is implemented.


Exampleskeyboard_arrow_up
Back to Top

Cognitive Load Theory really brought home to me the importance of examples in teaching. This view was reinforced in my podcast interview with Daisy Christodoulou who described a lesson from her past that didn't go as planned because students had not understood her explanation of a concept. This made me realise I had been making a mistake in my teaching - I put the emphasis of my planning on the explanations I would give my students, with the examples I chose playing a secondary role. I have now come to realise that the choice of example are more important than anything else. This section is my attempt to explain why.

Research Paper Title: Exemplification in Mathematics Education
Author(s): Liz, Bills, Tommy Dreyfus, John Mason, Pessia Tsamir, Anne Watson, Orit Zaslavsky
My Takeaway:
This is a brilliant summary of relevant research into the use of examples in the teaching and learning of mathematics. There are many things I found fascinating, but here are a few of my key takeaways:
1) The concept of "varied examples" discussed by Matron can be an effective way to encounter concepts. The authors notes that what is needed is variation in a few different aspects closely juxtaposed in time so that the learner is aware of that variation as variation. This for me is similar to the concept of minimally different examples favoured by the likes of Bruno Reddy and Kris Boulton, whereby just one aspect of a particular example is changed each time. This should mean the student is more likely to notice key features of examples, and allows the teacher to have more control over the discussion. Bruno explained it to me in terms of a science experiment, whereby you are looking to isolate the effect of one variable, therefore you only change one thing at a time.
2) A teacher's poor choice of examples can have a determinantal effect on learning by making it more likely students will jump to the wrong conclusions. A study by Rowland documents this for novice teachers in a primary setting, where the unintentionally ‘special’ nature of an example can mislead learners. I have been there myself. When teaching the effect of squaring a number, I have been known to choose "2", hence potentially reinforcing the major confusion between squaring and doubling. Or asking which is bigger, 2.7 or 2.85, which learners may get right without understanding place value (asking them to choose between 2.7 and 2.65 would be a better example).
3) A study by Wilson reports that learners can be distracted by irrelevant aspects of examples, so the presence of non-examples provides more information about what is, and is not, included in a definition. This is crucial. I used to assume that learners would understand the definitions I gave. However, you can argue that to understand a definition you need to understand the concept that is being defined, and hence definitions are perhaps not the best way to introduce new concepts. This is where examples come into play, and crucially the explicit use of non-examples to illustrate to students what does and does not fit into the definition. Something as simple as showing students carefully selected images of quadrilaterals and non-quadrilaterals will enable them to build up their own understanding (and subsequent definition of what a quadrilateral is far more effectively than presenting them with a definition of something they know little about.
My favourite quote:
Examples play a crucial role in learning about mathematical concepts, techniques, reasoning, and in the development of mathematical competence. However, learners may not perceive and use examples in the ways intended by teachers or textbooks especially if underlying generalities and reasoning are not made explicit. The relationship between examples, pedagogy and learning is under-researched, but it is known that learners can make inappropriate generalisations from sets of examples, or fail to make any conceptual inferences at all if the focus is only on performance of techniques. The nature and sequence of examples, non-examples and counterexamples has a critical influence of what opportunities learners are afforded, but even more critical are the practices into which learners are inducted for working with and on examples.

Research Paper Title:
Shedding Light On and With Example Spaces (and a very useful summary version here)
Author(s): Paul Goldenberg & John Mason
My Takeaway:
This paper (and the excellent summary) builds nicely on the previous paper, and contains some beautifully written points that have really influenced the examples I now give my students:
1) Well chosen examples can make-up for limited vocabulary and conceptual understanding of students that can render definitions alone pretty useless. To quote: Examples might be thought of as bits of context—ways to give information other than “saying what the word means”—allowing vocabulary-learning in school to grow at least slightly closer to the natural language learning at which children are so adept. Examples allow teachers to use a word communicatively until students are able to use it as well. Teachers can use the word rather than explaining it because the example provides the context and carries the meaning. Only then, when the students already have a rough meaning from communicative use in context can one effectively clarify the meaning formally with other words, through discussion and/or definition.
2) Including non-examples can give a less ambiguous signal of understanding. To quote: Asking a student to circle all the parallelograms in a collection of figures that includes non-parallelograms, prototype parallelograms, and various special-case parallelograms that are often thought not to be parallelograms because they have their own special names, we take each special-case figure that the student does not circle as evidence that the student’s understanding of parallelogram is incomplete. But if we ask a student to draw a parallelogram, we expect not to see the special cases that we’d hope the student would circle in the previous example. And, in fact, if we do get them as responses, we might well take that as evidence of error or incomplete understanding.
3) The definition is still important for clarity and confirmation. This avoids you having to present a potentially infinite number of examples and non-examples. This definition and needs to come after exposure to carefully chosen examples and non-examples once students have a rough idea of what is going on. This avoids you having to present The lovely section on "smaglings" at the end of the summary paper conveys this point brilliantly.
My favourite quote:
Without belaboring it, here’s the point: just as definitions without examples are generally insufficient to convey meaning, so are examples without definitions. No matter how numerous and varied our examples and non-examples are, unless they are exhaustive (i.e., the set of smanglings is finite, and we have encountered every one of them as either an example or non-example), examples alone are insufficient to allow us to decide all cases, because they provide no way of knowing whether or not some perverse exception lurks among the cases that have not been seen. But the examples—and especially the task of trying to choose among the unknowns and then defend that choice—make it much easier to perceive the dimensions of possible variation and the range of permissible change. One advantage for students of encountering this meta-mathematical idea is that it helps motivate what otherwise often seems like bizarre over-particularity in the wording of definitions. There is a lot we must say to define a smangling in a way that allows us to decide, definitively and without question, which of the unknowns is and isn’t a smangling.

Research Paper Title:
Getting Students to Create Boundary Examples
Author(s): Anne Watson and John Mason
My Takeaway:
Building on the previous papers, here we are introduced to the interesting concept of Boundary Examples. Boundary examples distinguish between having and not having a specified property. The authors assert that if students are only offered well-behaved examples, or examples which have additional, but irrelevant, features, then the reason for careful statements of conditions to a theorem or definition might pass them by, and they may well develop the idea that it is possible to have ambiguous or undecided cases. The authors offer the example of sequences. If students are only shown increasing or decreasing linear sequences, they may focus on the fact that sequences are either increasing or decreasing, and be oblivious to the fact that some sequences go up by different amounts (e.g. 1, 1, 2, 3, 5, 8...), some have limiting values (e.g. 8, 4, 2, 1, 0.5...), and some have the same terms throughout (e.g. 1, 1, 1, 1...). If students leave a lesson thinking that all sequences have constant differences and the only thing that distinguishes them is whether they go up and down, then their understanding is incomplete. Elsewhere the authors explain "We use the word ‘boundary’ because we see students’ experiences of examples in terms of spaces: families of related objects which collectively satisfy a particular situation, or answer a particular mathematics question, or deserve the same label. Such spaces appear to cluster around dominant central images". The point is, that by not explicitly addressing examples "on the boundary" there is the danger that key features will be lost at the expense of others. The authors make the bold claim: "If you cannot construct boundary examples for a theorem or a technique, then you do not fully appreciate or understand it". My big takeaway from all this is to ensure that I do not always provide the "obvious" worked examples in class - instead being aware of unusual examples that still fit into the topic that I am teaching. Polygons with convex angles, straight lines in the form x =, linear equations like 4 - 2x = 10, the mean of algebraic terms, and quadratic expressions that do not factorise, are all possibilities that spring to mind. The key point is that if these types of examples are left out, it may be possible for students to appear as though they have understood a topic, whereas in fact they only have a surface level of understanding. Furthermore, challenging students to construct a particular example, then a peculiar example (eg. one which no-one else in the class is likely to think of), and then (if appropriate) a general, or at least maximally general example, seems a really useful practice to develop, all whilst considering the burden on students' working memories. The authors acknowledge that the process of creating boundary examples is likely to be tricky for students at first. Their advice is to give students time, work with them over a period of weeks and months, using the concept of boundary examples wherever appropriate. Eventually students will get better at it, and their understanding of mathematics will improve. As the authors say: "we cannot afford not to invest the time needed in order to enable students to appreciate the ideas to which they are being exposed". I discuss the importance of examples in my podcast interview with Daisy Christodoulou.
My favourite quote:
We have found that many students do not appreciate the range or scope of choice of objects which are permitted by a theorem. Most theorems can be seen as a description of something which is invariant-amidst-change, and the theorem states the scope and range of change that are permitted. But if students have not tried to construct examples for themselves, have not probed the role of various conditions in making a theorem or technique work, then they are unlikely to use it appropriately, and probably unlikely to think of using it at all!

Research Paper Title: Learning from Worked Examples: What happens if mistakes are included?
Author(s): Cornelia S Grosse and Alexander Renkl
My Takeaway:
I am a huge fan of "spot the mistake" style activities, whereby I show my students a complete solution to a problem and they must identify, explain and correct the mistakes. However, following my reading into Cognitive Load Theory, I wanted to ensure that evidence suggested this was a sound technique. This paper provides the answer. Learning from worked examples where errors are included can enhance learning and transfer, but only if students have good prior knowledge of the topic. This makes perfect sense. Firstly, if students do not understand the topic, then how are they to spot the mistakes? Secondly, if their understanding is not secure, then their working memories are likely to become overloaded whilst searching for the right and wrong answers simultaneously. As a result, I will be more careful with my use of Spot the Mistake in future.
My favourite quote:
Learning with incorrect examples poses challenging demands on the learners. They have to represent not only the correct solution in their working memory, but also the incorrect step with an explanation why it is wrong. Learners with low prior knowledge who cannot form larger chunks for information coding can easily be overtaxed.


Problem Solvingkeyboard_arrow_up
Back to Top

For me, Cognitive Load Theory provides a sound argument for why students cannot learn to be problem solvers before they have sufficient knowledge and procedures stored in long term memory, and Explicit Instruction provides a model for students acquiring those knowledge and procedures. But how do we then help our students take that next step towards developing those key problem solving skills that we all want them to have? We have already seen some answers at the end of the Cognitive Load Theory section with the suggestion of a phrased approach, and noting the importance of the expertise-reversal effect. Here we seek to see if there is such a thing as a set of problem solving skills that we can help our students develop.

Research Paper Title:
Critical thinking: why is it so hard to teach?
Author(s): Daniel T Willingham
My Takeaway:
This paper does a really good job of summarising Willingham's thoughts on problem solving. We all want our students to become better problem solvers, and to think like mathematicians, but the finding here is that unlike, say, how to factorise a quadratic equation or how to add two fractions together, problem solving per se cannot be taught - at least not to novice learners. Many times I have witnessed students able to solve one problem, but then being unable to transfer those skills to another problem that is clearly related. The key is that the two problems are only clearly related to me. Experts (and I am by no means saying I am an expert here, but relative to the students I am) and novices approach problems differently. When faced with, for example, a worded maths problem in context, typically students are focusing on the scenario that the word problem describes (its surface structure) instead of on the mathematics required to solve it (its deep structure). So even though students have been taught how to solve a particular type of word problem, when the scenario changes, students still struggle to apply the solution because they don’t recognise that the problems are mathematically the same. "Sir, you've not taught us about paint", is a particularly painful post GCSE exam memory for me. Anyway, for Willingham, there are two ways of addressing this issue, but both have important limitations:
1) Students need to be familiar with a problem's deep structure. Exposure to lots of variations of problems that have the same underlying deep structure certainly helps students to look beyond the surface elements of each problem and focus on the relevant mathematics underlying it. So, lots of problems that revolve around the concept of Lowest Common Multiple should help students better spot those. The problem? Well, think how many concepts there are in maths! And then, think about what happens when those concepts are combined! It soon becomes impractical to exposure students to variations of problems covering every single concept in maths.
2) Students need to know they are looking for a deep structure. This is known as meta-cognition. If students are given a problem, they should be prompted to think: "okay, this is a maths problem, it must be related to something I have done before, now can I figure out what it is?". Whilst this strategy undoubtedly helps students avoid the temptation to focus all their efforts on the surface structure, there is a big problem: if they then lack the knowledge to then answer the question, all their efforts are in vain.
This paper has had a profound effect on me. Firstly, when I am happy that students have a sound knowledge of a given topic, I then select a group of related problem solving question on that topic, but crucially I explicitly discuss how they are connected. I force myself and my students to do this, to really emphasise the cues and signals, to try and get around the undo attention give to the surface structure. Sure, I cannot do this for every sub-topic and every combination of topics, but it is a start. And what I no longer do is con myself into thinking that giving students lots of different problems with different underlying deep structures will somehow enable them to develop the skill of "problem solving". Without the knowledge of the topic in place, and without purposeful practice looking at connected problems, this is likely to be a futile and frustrating exercise for all involved.
My favourite quote:
What do all these studies boil down to? First, critical thinking (as well as scientific thinking and other domain-based thinking) is not a skill. There is not a set of critical thinking skills that can be acquired and deployed regardless of context. Second, there are metacognitive strategies that, once learned, make critical thinking more likely. Third, the ability to think critically (to actually do what the metacognitive strategies call for) depends on domain knowledge and practice. For teachers, the situation is not hopeless, but no one should underestimate the difficulty of teaching students to think critically.

Research Paper Title: Domain-Specific Knowledge and Why Teaching Generic Skills Does not Work
Author(s): André Tricot and John Sweller
My Takeaway:
This brilliant paper offers a complementary view to the Willingham paper above and follows directly from the work we looked at concerning Cognitive Load Theory. The authors argue that there is an assumption that there exists such a thing as domain-general cognitive knowledge (i.e. generic skills such as problem solving and critical thinking), and this can be used to explain student achievement. This assumption then leads to the related assumption that we can teach these generic skills. The authors argue, however, that in fact domain-specific knowledge held in long-term memory provides a better explanation for the acquisition of skills. Moreover, they offer up several explanations of why we as teachers are so keen to embrace the idea of a set of teachable generic skills: "At any given time, we are unaware of the huge amount of domain specific knowledge held in long-term memory. The only knowledge that we have direct access to and are conscious of must be held in working memory. Knowledge held in working memory tends to be an insignificant fraction of our total knowledge base. With access to so little of our knowledge base at any given time, it is easy to assume that domain-specific knowledge is relatively unimportant to performance. It may be difficult to comprehend the unimaginable amounts of organised information that can be held in long-term memory precisely because such a large amount of information is unimaginable. If we are unaware of the large amounts of information held in long-term memory, we are likely to search for alternative explanations of knowledge-based performance"
My favourite quote:
We have argued that expertise based on biologically secondary, domain-specific knowledge held in long-term memory is by far the best explanation of performance in any cognitive area. Furthermore, in contrast to domain-general cognitive knowledge, there is no dispute that domain-specific knowledge and expertise can be readily taught and learned. Indeed, providing novice learners with knowledge is the main role of schools. We might guess that most school teachers in most schools continue to emphasise the domain-specific knowledge that always has been central, making little attempt to teach domain-general knowledge. Based on our argument, they should continue to do so. At school, children acquire knowledge that overcomes the need to engage in inefficient problem solving search and other cognitive processes. That knowledge allows people to function in a wide variety of tasks outside of school
.

Research Paper Title:
Research-Based Strategies for Problem-Solving in Mathematics
Author(s): Florida Department of Education
My Takeaway:
This is a good summary of various problem solving strategies, and their applications in the classroom. The paper is structured around Polya's classic four-step approach to problem solving:
1) understanding the problem
2) devising a plan to solve the problem,
3) implementing the plan
4) reflecting on the problem
Each of these four stages in analysed in great detail, with helpful suggestions for strategies to use in the classroom, together with examples and links to relevant research. I particularly like the emphasis on the first part of this process - understanding the problem. What should be the simplest part of this process if often the most difficult, but suggested techniques such as The Paraphrasing Strategy and Visualisation may help. However, what struck me most about this paper is the sheer variety of strategies outlined. There is no single "problem solving strategy" that students can learn and then apply to any problem. Topic-specific knowledge, together with an understanding of deep structure of problems that only knowledge can bring, is needed to correctly select the most appropriate strategy for a given problem and apply it successfully.
My favourite quote:
The first step in the Polya model is to understand the problem. As simple as that sounds, this is often the most overlooked step in the problem-solving process. This may seem like an obvious step that doesn’t need mentioning, but in order for a problem-solver to find a solution, they must first understand what they are being asked to find out.
Polya suggested that teachers should ask students questions similar to the ones listed below:
    Do you understand all the words used in stating the problem?
    What are you asked to find or show?
    Can you restate the problem in your own words?
    Can you think of a picture or a diagram that might help you understand the problem?
    Is there enough information to enable you to find a solution?


Research Paper Title: Problem-Solving Strategies: Research Findings from Mathematics Olympiads
Author(s): Cheung Pak-Hong
My Takeaway:
This paper takes Polya's approach to problem solving and tries to apply it to tricky Mathematical Olympiad questions. A variety of problems are presented, together with a discussion around which problem solving strategies could be used to tackle them. Those sitting this paper are among the very best student mathematicians in the world, and hence their approach to solving problems should shed some valuable light onto the most effective strategies. The conclusion of the authors is of crucial importance: the most effective problem-solving strategies are topic-specific. There is no generic problem-solving strategy. To be successful at these most challenging of problems, students must know their topics inside and out. This once again gives credence to the view expressed by Cognitive Load Theory that you cannot teach problem solving merely by exposure to problems. Topic-specific knowledge must be acquired in order for appropriate mental schema to develop, which allows students to have the best chance of solving problems.
My favourite quote:
This pilot study revealed that, for problems at Olympiad level, while heuristics suggested by Polya are useful in analysing the problems and in exploring feasible solutions, most of the more effective strategies are topic oriented. Olympiad problems in geometry are almost excluded from this report because although common strategies for solving them do exist, such as expressing quantities in terms of areas of triangles, they are confined to geometry.



Inquiries keyboard_arrow_up
Back to Top

With minimal guided instruction coming under fire in the Explicit Instruction section, you may not think there is a place for inquiries in mathematics. But if done carefully, I believe there still is, with potential benefits in terms of motivation and problem solving.

Research Paper Title:
Inquiry Teaching
Author(s): Andrew Blair
My Takeaway:
Inquiry teaching is often misunderstood as "learning by discovery". This article provides a good summary of the key distinction, together with the perceived benefits of inquiry teaching. The amount of structure or guidance offered by the teacher is determined by students' ability to think independently and critically. A key feature of inquiry lessons is "negotiation" between the teacher and the students over the structure and direction of the lesson. It is worth noting, however, that proponents of the direct instruction approach will likely balk at statements such as "Students not only meet concepts and skills in a meaningful pursuit of answering their own or peers’ questions, they also participate in a debate about how to learn and why to learn in a certain way" and "(the teacher must) redistribute authority to students". But I think that if the inquiries are well structured, and students only participate in them when they are ready (i.e. they are not used to teach concepts, just to apply them), then they have a valid place in the maths classroom.
My favourite quote:
In teacher directed strategy lessons, there is little guarantee that a method permits students to reach an ‘ideal’ understanding. In the investigative classroom, the learning experience is rigidly structured, leaving students the unenviable responsibility to discover a conceptual relationship. It is only the inquiry classroom that offers students a mechanism to harmonise conceptual learning with the method of learning.

Research Paper Title: Extending example spaces as a learning/teaching strategy in mathematics
Author(s): Anne Watson and John Mason
My Takeaway:
This is a fascinating paper which looks at the importance of examples for the understanding of mathematics. We have seen in the Cognitive Load Theory section the importance of worked examples for early skill acquisition under a model of Explicit Instruction, but here we look at something different - the use of student generated examples. The paper cites several studies in which student generated examples have aided concept development. Asking students to "find an equation of a straight line that has two intersection points with the parabola y = x2+ 4x+ 5." gives them opportunities to test their core coordinate geometry skills, but also opens the door for further investigation and generalisation - why do some lines cross a parabola twice, whereas others do not cross it at all, for example? Hence, we have examples leading to inquiry. Having read the Cognitive Load Theory section, there is the obvious danger that such questions are too much for students' fragile working memories to deal with during early skill acquisition. In the absence of well formed schema, novice learners may engage in inefficient problem-solving search, and hence could actually be thinking hard but not learning anything. However, the authors point to a study involving bottom-set year 9 students explaining the definition of a prime number by generating their own counter-examples, suggesting that student generated examples may in fact aid the important encoding process. For me, there is little doubt that framing a question in terms of asking students to generate their own examples is a great way to promote inquiry, foster motivation and to provide opportunities to make deep connections. Without wishing to sound like a broken record, I feel we just need to be careful to use these strategies when students' understanding of the basic concepts are relatively secure.
My favourite quote:
We have illustrated how seeing teaching/learning mathematics as creation and extension of personal example spaces can inform the construction of tasks in which students can work directly on their own mathematical structures and relationships. Achieving competence in mathematics can be seen as the development of complex, interconnected, accessible example spaces.



Memory: Encodingkeyboard_arrow_up
Back to Top

How many times have you seen students seemingly understand something one lesson, and then act as though they have never seen it before only a day later? Learning without the ability to retain and retrieve is not really learning at all. Remembering things involves three processes: encoding information (learning it, by perceiving it and relating it to past knowledge, so that it is in a format which it can then be stored in memory), storing it (maintaining it over time), and then retrieving it (accessing the information when needed). The next few sections are going to look at the fascinating concepts of retrieval and storage, including how making life more difficult for students might be the key for forming long-lasting memories. However, without successful encoding, there is little point worrying about the rest. So, how do we help students successfully encode information? Well, we have seen already that the principles of Explicit Instruction and Cognitive Load Theory (in particular the careful presentation of information and the use of worked examples) should help focus students' fragile, limited working memories on the things that really matter, and discussed the story structure of lessons and the use of analogies to make information more meaningful to students. In short, effective encoding relies upon the attention of the learner and on the associative nature of the brain. By making new learning personally relevant and linked to prior learning in long-term memory you can increase the associativity of the new learning and increase the effectiveness of the encoding. So, we have  done most of the work in previous sections! However, there are a couple of papers here that I found particularly interesting.

Research Paper Title: Students Remember ... What They Think About
Author(s): Daniel T Willingham
My Takeaway:
This article (along with Willingham's outstanding book Why Don't Students Like School) has had a profound effect on how I plan my lessons. Memory is the residue of though. Quite simply: students will remember what they are thinking about, so I need to plan my lessons accordingly. When I present my students with new material, it is not just the presentation of that material that is important, it is what the students are actually thinking about. Two related problems can emerge here if we are not careful:
1) Shallow Knowledge. Students may grasp onto one message from the lesson, but they may fail to grasp the underlying meaning that supports it. Take a lesson on adding fractions. Students may remember that denominators have to be different, but if they do not understand why, or understand how to make the denominators the same, then that shallow knowledge isn't going to be much use to them.
2) Thinking about the wrong things. If my students are revising quadratic equations by making a poster, will they be thinking about the subtleties of the algebra, or the colour of the highlighters they are using? If they are revising the laws of fractions by putting together a PowerPoint presentation, will they be thinking about fractions or PowerPoint animations?
Number 2) seems relatively easy to fix - I just need to plan activities like this carefully, bearing in mind that any increase in engagement that these different activities bring must be weighed up against a likely decrease in learning. But what about the issue of shallow knowledge? Willingham provides some useful, practical strategies:
1) Anticipate what your lesson will lead students to think about. When planning my lessons, the main question I ask myself during each part of the lesson is "what will my students actually be thinking about at this point?". And if it is not the key concept I need to get across, then I need to change my plan.
2) Use discovery learning carefully. We have already seen arguments against discovery learning in the Explicit Instruction section, but during the encoding process it seems even more important that teachers guide students carefully to try to ensure they are thinking about the correct concepts and information. I try to avoid bold statements, but I am going to make one here: discovery learning definitely should not be used during the encoding process.
3) Design reading assignments that require students to actively process the text. You may not think this has any relevance to maths, but when I interviewed Dani Quinn for my podcast she described how at Michaela the students take part in whole-class reading, combined with regular rests of knowledge to ensure students are focussed and not just drifting along.
4) Design tests that lead students to think about and integrate the most important material. This is a really interesting one for me. My students will often ask what is going to be on an upcoming test, and I will be purposefully vague. However, Willingham argues that if I am more specific in what I tell students (there will be two questions on adding fractions, one of which involves a negative number), then it will lead students to think more deeply about these topics during revision instead of a shallow glance across a number of topics. Of course, you need to ensure that tests across the year cover all the key concepts, but I thought this was an interesting approach.
My favourite quote:
In summary, in the early stages of learning, students may display "shallow" learning. These students have acquired bits of knowledge that aren't well-integrated into a larger picture. Research tells us that deep, connected knowledge can be encouraged by getting students to think about the interrelation of the various pieces of knowledge that they have acquired. Cognitive science has not progressed to the point that it can issue prescriptions of exactly how that can be achieved—that job is very much in the hands of experienced teachers. But in considering how to encourage students to acquire meaningful knowledge, teachers will do well to keep the "memory is as thinking does" principle in mind.

Research Paper Title: Levels of Processing: A Framework for Memory Research
Author(s): Fergus I M Craik and Robert S Lockhart
My Takeaway:
This classic paper was essentially a precursor to the work on schema for memories that we discussed in the Cognitive Science section. The authors argue that stimulus information is processed at multiple levels simultaneously (not serially) depending on characteristics, attention and meaningfulness. This supports the arguments discussed in the Willingham paper above. New information does not have to enter in any specific order, and it does not have to pass through a prescribed channel. The authors further contend that the more deeply information is processed, the more that will be remembered. For me, the most important point of the whole paper was this: the more connections to a single idea or concept, the more likely it is to be  remembered. The message is clear: as students encounter new information they should try to relate it to information they already know. Knowledge builds upon knowledge. One of the most important differences between novices and experts is the structure and organization of domain-specific knowledge. Experts have existing schema which allows them to better assimilate new information, making connections and thus processing it more deeply. If we wish our students to successfully encode new information, we must first ensure they have the sufficient domain-specific knowledge present, and then try our best to help students see the connections between their existing knowledge and this new information. This could be through the use of analogies, a story structure, or just effective explicit instruction.
My favourite quote:
That is, items are kept in consciousness or in primary memory by continuing to rehearse them at a fixed level of processing. The nature of the items will depend upon the encoding dimension and the level within that dimension. At deeper levels the subject can make more use of learned cognitive structures so that the item will become more complex and semantic. The depth at which primary memory operates will depend both upon the usefulness to the subject of continuing to process at that level and also upon the amenability of the material to deeper processing. Thus, if the subject's task is merely to reproduce a few words seconds after hearing them, he need not hold them at a level deeper than phonemic analysis. If the words form a meaningful sentence, however, they are compatible with deeper learned structures and larger units may be dealt with. It seems that primary memory deals at any level with units or "chunks" rather than with information


Memory: Forgetting, Spacing and Interleavingkeyboard_arrow_up
Back to Top

Once students have successfully Encoded information, we need to ensure that those memories are stored and can be retrieved. The papers in this section dig into this absolutely fascinating area, specifically with regard to forgetting, spacing and interleaving. The power of Testing was such a revelation to me that it is a separate section. There are also profound implications for students' revision, which will also be dealt with explicitly in the Revision section. The underlying message running throughout these papers blows my mind - in order to improve learning, we need to make it more difficult. This may appear in direct opposition to the work outlined in the Cognitive Load Theory section, which described conditions needed to make learning as easy as possible. For me, the matter is resolved by using the principles of Cognitive Load Theory in early skill inquisition and during the encoding process, and then adopting the recommendations of Bjork and others for increasing the challenge. Bjork's paper on Desirable Difficulties brings this all together beautifully.

Research Paper Title: Learning Versus Performance: An Integrative Review
Author(s): Nicholas C. Soderstrom and Robert A. Bjork
My Takeaway:
It is no exaggeration to say that this paper has changed my whole approach to teaching. The main premise is one of those things that sounds so obvious when you say it out loud, but I had been missing it for the first 12 years of my career: learning and performance are two different things. The ultimate goal of instruction should be to facilitate long-term learning—that is, to create relatively permanent changes in comprehension, understanding, and skills of the types that will support long-term retention and transfer. During the instruction or training process, however, all we can observe and measure is performance. That would all be fine, apart from the rather inconvenient fact that current performance can be a highly unreliable guide to whether learning has happened. During lessons, teachers, consciously or otherwise, provide cues and prompts to elicit correct answers. Students are also pretty good at mimicking, which allows them to say what they think the teacher wants to hear. All of this boosts performance, but may have no impact on learning. Hence, we are at risk of being fooled by current performance, which can lead us as teachers to choose less effective conditions of learning over more effective conditions - i.e. use strategies that lead to short-terms gains in performance, such as teaching to a particular test. Moreover, it can lead students to prefer certain revision techniques, such as cramming, over others as they can see short-term improvements in performance. Here is the ultimate kicker - high performance may lead to lower learning! This next point is so important that i I am going to put it in bold: when we design lessons to boost pupils performance, we may actually hinder their long term learning. Likewise, conditions that induce the most errors during acquisition are often the conditions that lead to the most learning. The reasons behind this incredible finding are discussed in the paper On the Symbiosis of Learning and Forgetting below. The crucial distinction between learning and performance has huge implications for teaching, revision and lesson observation are huge, and is directly to related to concepts such as spacing, interleaving, and overlearning which will be discussed further in this section. 
My favourite quote:
Given that the goal of instruction and practice— whether in the classroom or on the field—should be to facilitate learning, instructors and students need to appreciate the distinction between learning and performance and understand that expediting acquisition performance today does not necessarily translate into the type of learning that will be evident tomorrow. On the contrary, conditions that slow or induce more errors during instruction often lead to better long-term learning outcomes, and thus instructors and students, however disinclined to do so, should consider abandoning the path of least resistance with respect to their own teaching and study strategies. After all, educational interventions should be based on evidence, not on historical use or intuition.


Research Paper Title:
On the Symbiosis of Learning and Forgetting
Author(s): Robert A. Bjork
My Takeaway:
The work of Robert Bjork and Elizabeth Bjork has blown my mind. Their "Theory of Dissuse" outlines the importance and - wait for it - the benefits of forgetting. They explain that each item in memory has a storage strength and a retrieval strength. Storage indicates how well an item is embedded in long-term memory and retrieval indicates how easily an item can be brought to mind when needed. Ideally, we want our students to have both. But here is the key point - attempts to increase retrieval strength improve performance in the short term but very quickly fade. Worse still, trying to retrieve something from memory too quickly can interfere with our ability to store it more strongly. Cramming, cues, and teaching to a test, all serve to make it easier to retrieve that information, but do not give it chance to develop a high storage strength and hence become embedded in long-term memory. This all sounds pretty bad for teaching - but there is good news: if we wait until we’ve started to forget something, retrieval practice increases our ability to recall it in the long term. Therefore, the best way to increase storage strength is to allow memories to fade before trying to retrieve them. The conclusion is both simple and profound: the best way to remember is to forget. This has huge implications for both teaching and revision. Firstly, it sets up nicely the work on interleaving and spacing that we will cover later in this section. But it also calls into question the idea of giving students loads of the same type of question to do following the teaching of a new concept. Sure, they can all do the questions by the end of the lesson (high retrieval strength), but then when asked to do a similar question next lesson, or next week, students often cannot (low storage strength). How do we rectify this? Well, maybe we give students less questions to answer on a given topic (to borrow a phrase from my past as an Economics student, there is a diminishing marginal utility to each one), allow them to forget it, and then cover the material again at a later date. This cycle of practising, forgetting, and then practicing again, as opposed to keep practising the same thing, allows the storage strength of these skills to increase, which is needed for long-term learning. Of course, this will be a frustrating experience for students, as well as us teachers, and crucially performance in the short-term may decrease. So, both students and teachers may need informing of the power of forgetting, and the crucial distinction between learning and performance.
My favourite quote:
Perhaps the prime example of forgetting enhancing learning is the spacing effect, one of the most robust and general effects from the entire history of experimental psychology When a second study opportunity is provided after a delay following a first study opportunity, rather than being presented with little or no delay, long-term recall is enhanced, often very significantly. Again, though, were the studied material to be tested following a short delay or a long delay, we would observe that the longer delay results in poorer recall of the studied material—that is, more forgetting. Similarly, interleaving, rather than blocking, the learning trials on separate to-be-learned tasks produces more forgetting between trials on a given task during the learning phase, but tends to enhance long-term retention and transfer.

Research Paper Title: Making Things Hard on Yourself, But in a Good Way: Creating Desirable Difficulties to Enhance Learning
Author(s): Elizabeth L. Bjork and Robert Bjork
My Takeaway:
This is paper is one of the best things I have ever read. It is directly related to the issue of distinguishing between learning and performance, and argues the seemingly counter-intuitive point that you improve learning by making it more (desirably!) difficult.  Bjork defines a difficulty as desirable if it makes retrieval practice harder in the short term but acts to increase retention and transfer. Therefore, these deliberate difficulties will likely lead to a reduction in short-term performance, but an improvement in long-term learning. The implications for lessons (and of course lesson observations!) are huge. The authors suggest the following strategies to induce desirable difficulties:
1) allowing students to forget some of the material covered before it’s reintroduced (spacing);
2) mixing up different content in order to prevent students developing the illusion of knowledge (interleaving);
3) asking questions about material which has already been covered rather than restudying it in order to prevent students developing a false sense of familiarity and fluency (retrieval practice);
4) varying the conditions in which instruction takes place in order to prevent contextual cues from building up and making it harder for students to transfer what they’ve learned to new contexts (variation);
5) progressively reducing the frequency and quantity of feedback given in order to prevent students from becoming dependent on external sources of expertise (see the Marking and Feedback section for a discussion on this amazing final strategy).
I fully agree with all of this, and we will look at some of Bjork's describable difficulties in the papers that follow, but I will add one thought - if things are always too difficult for students, then motivation quickly disappears, and no teaching strategy can be effective if your students have switched off and have zero desire to engage in the learning process. Indeed, the principle of describable difficulty does seem in contrast to Dan Willingham's view of thinking discussed on the Cognitive Science section - if thinking is too hard, students stop thinking - as well as Cognitive Load Theory, which seems to me to be a model based on making thinking as easy and focused as possible. Much like David Didau (in this wonderful blog post on the same paper), I feel students need to taste success first in order to be motivated (see the Zimmerman article on the importance of self-efficacy in the Motivation and Praise section for more on this), then try to ensure that key knowledge and processes are successfully encoded via a process of explaining, modeling, scaffolding and practising as suggested by Cognitive Load Theory, before finally things can become deliberately difficult. Didau proposes what seems a very sensible three stage plan to achieve this: Encode success, Promote internalisation, Increase challenge.
My favourite quote:
The basic problem learners confront is that we can easily be misled as to whether we are learning effectively and have or have not achieved a level of learning and comprehension that will support our subsequent access to information or skills we are trying to learn. We can be misled by our subjective impressions. Rereading a chapter a second time, for example, can provide a sense of familiarity or perceptual fluency that we interpret as understanding or comprehension, but may actually be a product of low-level perceptual priming. Similarly, information coming readily to mind can be interpreted as evidence of learning, but could instead be a product of cues that are present in the study situation, but that are unlikely to be present at a later time. We can also be misled by our current performance. Conditions of learning that make performance improve rapidly often fail to support long-term retention and transfer, whereas conditions that create challenges and slow the rate of apparent learning often optimize long-term retention and transfer.

Research Paper Title:
Spacing and Interleaving of Study and Practice
Author(s): Shana K. Carpenter
My Takeaway:
This is a really nice introduction to the concepts of spacing and interleaving, which are two of Bjork's "desirable difficulties" described above. Best of all, both of these concepts are relatively easy to implement. The Spacing Effect refers to the findings that learning is better when two or more exposures to information are separated in time (i.e. spaced apart) than when the same number of exposures occurs back-to-back in immediate succession. The Interleaving Effect contrasts a "blocking" approach, whereby students study the same type of material over and over again before moving on to a different type of material, against an "interleaving" approach, where students practice all of the problems in an order that is more random and less predictable. It is quite easy to get the two concepts mixed up (for me, anyway!), and the two are inextricably linked as interleaving produces spacing, but a good way to think about it might be that spacing describes the scheduling of exposures to a single concept (A), and interleaving describes the scheduling of exposures to multiple concepts (A, B, and C). The general findings are that both spacing and interleaving can produce significant benefits with regard to memory and learning, and these will be discussed further in the papers in this section. Spacing is relatively easy to implement. You just need to ensure you routinely revisit material at fixed intervals throughout the year - and a paper in this section provides guidance for what those fixed intervals should be. Similarly, tapping into the benefits of interleaving could be as simple as adapting how you present questions in class and modifying your existing homeworks and assignments slightly, as will be discussed in the papers in this section. Before we move on, a key point that also needs raising is that blocked practice might be more appropriate when a skill is first being learned. After all, spacing and interleaving work by inducing students to retrieve concepts from long-term memory. If concepts are not there in the first place, then there is nothing to retrieve!
My favourite quote:
Students and instructors are faced with these decisions on a daily basis. Research on human cognition has shown that learning can be significantly affected by the way in which repetitions are scheduled. This research can help inform the decisions that students must make concerning when to study information in order to maximize learning.

Research Paper Title:
Using Spacing to Enhance Diverse Forms of Learning
Author(s): Shana K. Carpenter, Nicholas J Cepeda,  Doug Rohrer, Sean H. K. Kang & Harold Pashler
My Takeaway:
The concept behind spacing is relatively simple, and its effects can be profound. This paper finds that performance on final tests of learning is improved if multiple study sessions are separated (spaced apart) in time rather than massed in immediate succession. The optimal length of the spacing depends on when the material is going to be tested. In general, longer spacings are more beneficial when the test is a long way away, whereas shorter spacing are more beneficial for tests in the near future. These longer periods may be weeks or even months. Three practical recommendations for teachers:
1) Teachers should dedicate part of each lesson to reviewing concepts learned several weeks earlier. I often do this during the starter, either by means of a low-stakes quiz (see the Testing section), or using something like Corbett Maths 5-a-day. The key is careful planning to ensure that as much previously taught material as possible comes up again.
2) Homework assignments should be used to re-expose students to important information they have learned previously. We have a Revision section at the start of every homework, but as will be discussed in the Interleaving papers that follow, there may be a way to make this even more effective.
3) Teachers should give exams and quizzes that are cumulative. This again suggests the benefits of low stakes quizzes, which not only tap into the benefits of The Spacing Effect, but which also utilise the Retrieval Effect that will be discussed in the Testing section
This last point even has an added benefit - as well as re-exposing students to information that they have previously learned, cumulative exams and quizzes also provide students with a good reason to review information on their own, hence hopefully improving their revision strategies. Revision is covered in a later section.
My favourite quote:
The key criterion is that information should be reviewed after a period of time has passed since the initial learning. Particularly if the goal is long-term retention, the findings from Cepeda et al. (2008) suggest that the ideal time to review information may be several weeks or months after it was initially learned.

Research Paper Title: Spacing effects in learning: A temporal ridgeline of optimal retention
Author(s): Nicholas J Cepeda,  Edward Vul, Doug Rohrer, John T Wixted & Harold Pashler
My Takeaway:
After the last paper you may be thinking: "wouldn't it be great if someone could tell me the optimal time to revisit content with my students depending on when their test is?". Well, your wish is my command. This paper offers a rough guide:
Time to test: 1 week, Optimal interval between study sessions: 1– 2 days
Time to test: 1 month, Optimal interval between study sessions: 1 week
Time to test: 2 months, Optimal interval between study sessions: 2 weeks
Time to test: 6 months, Optimal interval between study sessions: 3 weeks
Time to test: 1 year, Optimal interval between study sessions: 4 weeks
My favourite quote:
To put it simply, if you want to know the optimal distribution of study time, you need to decide how long you wish to remember something. Although this poses challenges for practical application, certain conclusions can nonetheless be drawn. If a person wishes to retain information for several years, a delayed review of at least several months seems likely to produce a highly favorable return on a time investment— potentially doubling the amount ultimately remembered, holding study time constant—as compared to less temporally distributed study.

Research Paper Title: Interleaving Helps Students Distinguish among Similar Concepts
Author(s): Doug Rohrer
My Takeaway:
Students confusing related concepts is the single biggest cause of errors on my Diagnostic Questions website. Classics include confusing area and perimeter, the rules for adding and multiplying fractions, mean and median, the laws of algebra - the list goes on! This wonderful paper which summarises the key findings from a wide range of interleaving research which suggests that interleaved mathematics practice helps students learn to distinguish between different kinds of problems. This is obviously a critical skill because solving a mathematics problem requires that students first identify what kind of problem it is, which means that they must identify those features of a problem that indicate which concept or procedure is appropriate. Identifying the kind of problem is not always easy. Take solving equations. The familiar instruction, “Solve for x” does not indicate which one of several solving strategies is appropriate. For instance, students must use the quadratic formula to solve some equations (e.g., x2 − x − 100 = 0), and they must factorise to solve others (e.g., x3 − x = 0). Then there is completing the square, not to mention linear equations, cubic equations, equations involving fractions, and so on! Solving a mathematics problem requires students to know which strategy is appropriate and not only how to execute the strategy. The key point here is that with blocked practice, students need not identify an appropriate strategy because every problem in the assignment can be solved by the same strategy. In essence, blocking provides scaffolding. This might be useful when students meet a new concept, but students who receive only blocked assignments do not have the opportunity to practice without this crutch, and studies quoted here suggest this has negative consequences for long-term learning. So, what are we to take from this? Firstly, it seems clear that interleaving practice problems on homeworks and tests is likely to have a significant advantage on long-term learning, and this will be discussed in the papers that follows. But what are the implications for the actual teaching of related concepts? For me, there are two extremes:
1) Teach related concepts together and block practice each. For example, teach the mean, then practice the mean. Then teach the median, and practice the median, and so on, until all four averages have been covered. What happens with that approach - students confuse those related concepts as they have not had sufficient opportunity to distinguish between them.
2) Teach related concepts completely separately. So, teach the mean, then teach adding fractions, then area of a triangle, before returning to teaching the median. I have not been able to find specific research on this approach, but I would be fascinated by it. Whilst it undoubtedly taps into the benefits of interleaving, my concern is that it does not allow students to compare and contrast the related concepts, and to pick up on the connections between them. Take averages. In the scenario described here, how do we teach students about the appropriateness of using each average, or solve complex problems involving more than one of them, when they are taught so far apart? It would also have huge implications on school's existing schemes of work and assessments.
For me, there is an optimal middle ground. It would go something like this: teach the mean (following the principles of Explicit Instruction and Cognitive Load Theory), practice the mean. Then teach the median, but practice both the mean and the median. Then teach the mode, and practice the mean, median and mode. Here, students get to distinguish between related concepts as practice is not blocked, but hopefully still benefit from seeing the connections between the concepts. Finally, as highlighted in the quote, there is little doubt that interleaving makes learning more difficult and is likely to lead to a short term dip in performance. Both students and teachers need to be aware of this.
My favourite quote:
Students might balk because interleaving increases the difficulty of a question or problem. A group of questions or problems are easier when all relate to the same topic or concept. By contrast, answering a set of interleaved biology questions might require students to consult material presented in previous chapters, and an interleaved mathematics assignment prevents students from simply repeating the same procedure throughout the assignment. Students will therefore make more errors, and work more slowly, when assignments are interleaved. This in itself is not problematic because the aim of classroom instruction is ultimate mastery, not error-free learning. Still, some students might be unwilling to make the extra effort. In this scenario, interleaving is like bad-tasting cough syrup—ineffective because children refuse to use it.

Research Paper Title:
Interleaved Practice Improves Mathematics Learning
Author(s): Doug Rohrer, Robert F. Dedrick, and Sandra Stershic
My Takeaway:
This study is fascinating. Previously I had thought interleaving had to involve completely changing the order lessons are taught. However, that is not the case. Seventh-grade students saw their teachers’ usual maths lessons and received regular homework assignments. Every student received the exact same problems, but the scheduling of the problems was altered so that students received blocked or interleaved practice. Later, students received a review of all the content, followed 1 or 30 days later by an unannounced test. Students following the interleaving program performed significantly better on both tests, with a greater difference in performance found on the test taken 30 days later, suggesting a significant impact on retention. Now, the "blocked practice" approach is one I have used regularly. Teach students fractions, give them an assignment on fractions. Then teach them equations, given them an assignment on equations. Then it is time for angles and averages, then let's have a review of all four topics, and then a half term test. Our assignments do consist of short revision questions at the start which are taken from a variety of questions throughout the year, but the main focus of each assignment is undoubtedly on the topic that has just been covered. This paper suggest that approach is not quite good enough. The assignments should consist of a mixture of questions from previous topics, and not just the quick-fire revision ones. Moreover, each assignment should contain a majority of questions from other topics, not a minority. For example, the kind of fractions problems that would have appeared as questions 4, 6 and 9 on our first assignment should be included in the second assignment instead, alongside questions on equations and any other topics previously taught, and so on. I particularly like the fact that there are no implications for changing the way or order topics are taught, or the content of assignments - it is simply a case of varying the composition of the assignments themselves. This seems like a very quick win.
My favourite quote:
Benefits of interleaved practice have been consistently observed with a variety of mathematics skills and with students in elementary school, middle school, and college. As argued here, these benefits arise because interleaved practice provides students with an opportunity to learn how to choose an appropriate strategy (or learn that they cannot do it). In short, interleaved practice simply provides students with an opportunity to practice the very skill they are expected to learn.

Research Paper Title: The Benefit of Interleaved Mathematics Practice is not limited to Superficially Similar kinds of Problems
Author(s): Doug Rohrer, Robert F. Dedrick & Kaleena Burgess
My Takeaway:
This paper discusses the point raised in the previous paper, but then takes it further. The correct solution to most mathematical problems involves two steps: identify the strategy needed to solve the problem, and then successfully carry out that strategy. The authors argue that most mathematical assignments deny students the opportunity to practice that first step - identifying the strategy. For example, if a lesson on the Pythagorean theorem is followed by a group of problems requiring the Pythagorean theorem, students know the appropriate strategy before they read each problem. Similarly, blocked practice fundamentally changes the pedagogy of worded problems, where often the majorly difficulty is not carrying out the strategy required to solve the problem, but actually identifying what that strategy should be. An alternative approach is interleaving, where a majority of the problems within each assignment are drawn from previous lessons, so that no two consecutive problems require the same strategy. With this approach, students must choose an appropriate strategy and not only execute it, just as they must choose an appropriate strategy when they encounter a problem during a cumulative exam or high-stakes test. Such interleaved practice also ensures that problems are spaced, which can tap into the benefits related to The Spacing Effect. Interestingly, unlike the previous paper, the authors do not attribute the benefit of interleaving to merely enabling students to better discriminate between problems. They argue solving a mathematics problem requires students not only to discriminate between different kinds of problems, but also to associate each kind of problem with an appropriate strategy. Blocked assignments often allow students to ignore the features of a problem that indicate which strategy is appropriate, which precludes the learning of the association between the problem and the strategy. Interleaved practice can help the development of both of these crucial skills. However, I must point out that many of Bjork's "desirable difficulties" seem inappropriate in early skill acquisition phrase. A degree of blocked practice may be necessary in lessons when a topic is being introduced, followed by interleaved practice for homework.
My favourite quote:
Although it might seem surprising that a mere reordering of problems can nearly double test scores, it must be remembered that interleaving alters the pedagogical demand of a mathematics problem. As was detailed in the introduction, interleaved practice requires that students choose an appropriate strategy for each problem and not only execute the strategy, whereas
blocked practice allows students to safely assume that each problem will require the same strategy as the previous problem.


Research Paper Title: Effect of Overlearning on Retention
Author(s): Jaems E Driskell, Ruth P Willis and Carolyn Cooper
My Takeaway:
The important distinction between learning and performance naturally raises the question: when do I know if my students have practiced enough? A score of 10/10 is certainly a good performance, but how much of that is down to mimicry, and how much has been retained in long-term memory? That leads to the concept of overlearning - or, as I like to think of it, practising beyond mastery. This meta-analysis finds a moderately positive effect on long-term retention of overlearning. In short, learning can still occur after performance has seemingly peaked. I took two things away from this paper. Firstly, I will no longer settle for a single good performance on, say, a fractions test. Students need to continue practising, even if they do not see a visible improvement in performance. Concepts such as spacing in lessons and interleaving in homeworks will help with this. Secondly, students need to be informed of this concept, otherwise there is the danger they will moan at the fact you are making them do something they can already do. 
My favourite quote:
One question that organizations and training practitioners face is the question Of how much training is enough. One approach is to provide the level Of training that is estimated to  meet the requirement of the average trainee. That is, if the average trainee can achieve proficiency in 5 sessions, then this becomes the level Of training provided to all. A second approach is to provide training to meet some set criterion, such as one errorless performance of the task. In this case, each individual is trained until he or she reaches the criterion level. A third approach, overlearning. requires that training continue for a period past this initial mastery level. The results of this meta-analysis document the effectiveness of overlearning and show that retention is enhanced when learning proceeds beyond initial mastery

Research Paper Title: The Effects of Overlearning and Distributed Practise on the Retention of Mathematics Knowledge
Author(s): Doug Rohrer and Kelli Taylor
My Takeaway:
The final paper on interleaving from my Doug Rohrer collection, and this has equally big implications. In two experiments, 216 college students learned to solve one kind of mathematics problem before completing one of various practice schedules. In Experiment 1, students either massed 10 problems in a single session or distributed these 10 problems across two sessions separated by 1 week. The benefit of distributed practice was nil among students who were tested 1 week later but extremely large among students tested 4 weeks later. In Experiment 2, students completed three or nine practice problems in one session. The additional six problems constituted a strategy known as overlearning, but this extra effort had no effect on test scores 1 or 4 weeks later. A few things struck me. Firstly, the improvement in performance from distributed practice was not seen immediately - this is important for both students and teachers to know. Secondly, as has been pointed out in the previous studies, most mathematics textbooks rely on a format that emphasises overlearning and minimises distributed practice, as do most of the worksheets and sets of questions I give my students to do in class. Finally, with regard to overlearning, the authors do not dismiss it (and indeed, with the problem of distinguishing between learning and performance discussed earlier in this section, it is dangerous to assume a student has learned something based purely on a good test score), but instead suggest a better use of students' finite time. The authors sum this up very nicely: "we suggest that assignments should err slightly in the direction of too much practise, perhaps by including three or four problems relating to each new concept in the most recent lesson (in addition to any examples given in the written lesson or class lecture). However, beyond these first three or four problems, the present data suggest that the completion of additional problems of the same type is a terribly inefficient use of study time. Instead, our findings suggest that the student should devote the remainder of the practise session to problems drawn from earlier lessons in order to reap the benefits of distributed practise"
My favourite quote:
With this distributed practise format, each lesson is followed by the usual number of practise problems, but only a few of these problems relate to the immediately preceding lesson. Additional problems of the same type might also appear once or twice in each of the next dozen assignments and once again after every fifth or tenth assignment thereafter. In brief, the number of practise problems relating to a given topic is no greater than that of typical mathematics textbooks, but the temporal distribution of these problems is increased dramatically.

Research Paper Title: Why interleaving enhances inductive learning
Author(s):
Monica S. Birnbaum, Nate Kornell, Elizabeth Ligon Bjork & Robert A. Bjork
My Takeaway:
A fascinating paper to end our discussion on interleaving (for now!). An inductive approach to teaching involves showing examples of how a concept is used in the hope that students will ‘notice’ how the concept works, as opposed to a deductive approach, whereby the teacher provides the material pupils need to think about and reduces the quantity of information they are required to hold in working memory. It would seem sensible to think that a deductive approach works best, and indeed when learning basic content that does seem to be the case. However, this research suggests that students tend to understand and remember more when learning occurs inductively, but only if this is combined with an interleaved approach to studying content. In short - students best learn a rule by seeing examples containing the rule mixed up with examples that don’t. Why does this work? Well, frequent alternation of categories (interleaving) has the advantage of highlighting features that serve to distinguish categories. Conversely, infrequent alternation of categories (blocking) has the advantage of highlighting information that remains constant across the members within a category. For me, it depends what you want to achieve. If you wish students to become fluent in a particular skill or concept, then a blocking approach is likely to be best. However, if you want students to be able to distinguish between problems, recognising key features and selecting the most appropriate method to use (as is often needed for more wordy questions, questions where the required method is not obvious, or on high-stakes tests), then interleaving is necessary. The research we looked at in the Explicit Instruction section suggests that students need key knowledge and processes to be stored in long term memory before they can become effective problem solvers, so my takeaway is that blocked practice may be optimal when students first encounter a skill, but interleaving is needed for students to develop their learning further. On a related note - learners (and teachers!) are likely to find the process of interleaving more difficult than blocking. This is largely because they have to think more, and may see a short term dip in performance. Crucially, they may feel they are not learning as much and begin to lose motivation. This relates directly to Bjork's "desirable difficulties" discussed earlier in this section.
My favourite quote:
The great majority of the  participants in the present study, as well as those in prior  studies, judged that they had learned  more effectively with blocked than with interleaved study. Thus, a bit of practical advice to learners and educators seems  warranted: If your intuition tells you to block, you should  probably interleave.

Research Paper Title:
Environmental context and human memory
Author(s): Steven M Smith, Arthur Glenberg and Robert A Bjork
My Takeaway:
Another of Bjork's "desirable difficulties" concerns varying the location of where learning and retrieval take place. In this fascinating study, the authors gave participants a lost of words to learn and had them study either in the same room twice or in two separate rooms (one a cluttered basement, the other a windowed office). Three hours later, the students were moved into a third ‘neutral room’ and asked to recall as many of the words as they could. Those who studied in the same room managed an average of 16 out of 40 whereas those who studied in two different rooms recalled 24 words. The experiment showed strong recall improvements with variation of environmental contexts.This has become known as the Variation Effect and since then there are been many other studies which have replicated this findings. Why does varying conditions work? Well, it may enhance long-term learning because the material becomes associated with a greater range of memory cues that serve to facilitate access to that material later. Whilst it is clear that this experiment is concerned with the recall of words - and so we must be very careful in making any wild claims about solving mathematical problems - the fact that recall in mathematics plays an important part suggests that we should take note of these results. For me, there are two clear implications. Firstly, wherever possible vary where students in school learn and practice retrieval. My students sit in the same seats, in the same classrooms, most of the time. And yet, when it comes to taking their exams they are suddenly thrust into a brand new environment. As a result of this paper, I now routinely change my seating plans, and where possible try to take advantages of room changing opportunities. There is the potential cost of disruption and daft behaviour that this "new environment" brings, but the potential benefit far outweighs these. Secondly, a simple piece of advice to pass onto students when revising is to vary where they do their revision. Try a different room, a different time of day, etc. Anything that varies the environmental conditions is likely to lead to better retrieval.
My favourite quote:
Another type of explanation of the results can be found in Tulving and Watkins' (1973) continuity hypothesis, which states that recall and recognition require essentially the same retrieval process(es), but that free recall involves such processes to a greater extent than does recognition. In free recall, where the experimenter provides few explicit retrieval cues, the subject must make use of varied sources of cues to provide access to the stimuli. Environmental context could be one such source of cues. In cued recall there is more specific retrieval information, and in recognition memory the subject is supplied with highly effective, explicit cues, thereby reducing his dependence on other cue sources.



Memory: Testingkeyboard_arrow_up
Back to Top

One of the golden rules of school life is that students hate tests. But I would like to refine this rule to say students hate "high-stakes" tests. The papers in this section will outline a finding that has possibly transformed my lessons more than any other: the power of regular, low-stakes testing. By low-stakes, I mean short tests (typically 3 to 5 questions) that the students take at the start of every single lesson based on content from earlier in the year (to take advantage of the Spacing and Interleaving effects), they mark them themselves, the marks are not recorded anywhere, I don't know their scores, students don't know each others' scores, and I go through the answers straight away so students have immediate feedback. You will not believe me when I say this, but the students love them. More importantly, the effects on their retention have been profound. The biggest impact this has had on my teaching is that I now see tests as "learning events", as opposed to just a means of assessment. When I interviewed Dani Quinn for my podcast, she explained how Michaela Community School makes extensive use of regular weekly tests. In this section I attempt to get to the bottom of the power of testing.

Research Paper Title: Ten Benefits of Testing and their Application to Educational Practice
Author(s): Henry L. Roediger III, Adam L. Putnam and Megan A. Smith
My Takeaway:
This paper is stunning. It details (as the title suggests) ten benefits of testing, citing research to support each claim. Each benefit is worth discussion here, but I am going to limit myself to just three. 1) The Testing Effect: retrieval aids later retention. A fascinating study described here explains that students who followed a pattern of study-study-study-test for a topic performed better on the final test than a group who followed study-test-test-test. But - and here is the key - when tested a week later, the exact opposite had occurred, even though the latter group had much less exposure to the material. Testing led to better long-term retention. The Testing Effect is examined further later in this section. 2) Testing causes students to learn more from the next learning episode. When students take a test and then restudy material, they learn more from the presentation than they would if they restudied without taking a test, hence it helps with revision. 3) Testing improves transfer of knowledge to new contexts. This is the holy grail! We have seen in the Cognitive Psychology section that the inability to transfer knowledge to new situation is one of the characteristics that defines novice learners and inhibits their learning. It seems that testing can help with that, possibly by enabling learners to form and strengthen schemas that they can later draw upon in new contexts. Add to all of this the benefit that testing can give you the teacher information about gaps in students' knowledge which you can resolve in class, and it promotes the benefits of testing to students so that they may use it in their revision, and you have possibly one of the most important teaching tools ever.
My favourite quote:
We have reviewed 10 reasons why increased testing in educational settings is beneficial to learning and memory, as a self-study strategy for students or as a classroom tactic. The benefits can be indirect—students study more and attend more fully if they expect a test – but we have emphasized the direct effects of testing. Retrieval practice from testing provides a potent boost to future retention. Retrieval practice provides a relatively straightforward method of enhancing learning and retention in educational settings.

Research Paper Title: The Critical Importance of Retrieval for Learning
Author(s): Jeffrey D. Karpicke, et al
My Takeaway:
For years I have underestimated just how powerful actively practising retrieval can be, relative to other more common revision techniques. Three findings in this paper particularly stood out to me. Firstly, repeated studying after learning had no effect on delayed recall, but repeated testing produced a large positive effect. I have seen this myself - students think they have cracked a topic like adding fractions because they have got questions correct in the past, and so during revision may merely glance over their notes on fractions (studying) instead of trying questions out (retrieval). Such revision feels easy, and hence has little effect on long term retention - as we have seen from the work of Bjork in the Memory section above, learning needs to be difficult. Secondly, students’ predictions of their performance were uncorrelated with actual performance. The students in the study were not aware of the benefits of practising recall, even after they had done it! This is both fascinating and worrying, and suggests that students need to be convinced of the power of this strategy long before they start their revision. Regular low-stakes testing in the classroom seems a sensible way of achieving this. Finally - and I have saved the best to last - if retrieval is so important for learning (as you will see in the next paper, it can actually cause learning), then is it a good idea to just do a load of past papers for exam preparation, as I have done for many years? NO! Why? Well, exam papers are designed to cater to a wide variety of abilities. Hence, they do not always encourage retrieval from long term memory, but instead test elements of problem solving (specifically, elements not stored in long term memory). We have seen in the Cognitive Load Theory section that problem solving does not always lead to learning, and hence students can work through exam papers and not actually learn anything. I conclude that until students have covered the full course, regular, low-stakes tests on topics that students have already studied is the best way to profit from the positive effect of retrieval.
My favourite quote:
The conventional wisdom shared among students and educators is that if information can be recalled from memory, it has been learned and can be dropped from further practice, so students can focus their effort on other material. Research on students’ use of self-testing as a learning strategy shows that students do tend to drop facts from further practice once they can recall them. However, the present research shows that the conventional wisdom existing in education and expressed in many study guides is wrong. Even after items can be recalled from memory, eliminating those items from repeated retrieval practice greatly reduces long-term retention. Repeated retrieval induced through testing (and not repeated encoding during additional study) produces large positive effects on long-term retention.
 

Research Paper Title:
Retrieval Practice Produces More Learning than Elaborative Studying with Concept Mapping
Author(s): Jeffrey D. Karpicke, et al
My Takeaway:
This paper is by the same team of authors as the previous one, but makes two key additional points. Firstly, it makes explicit the finding that practising retrieval is more effective on long term retention than attempting to encode information during the learning process - possibly using strategies such as mind-mapping, and making notes. Practising retrieval was found to be more effective than more traditional revision strategies when students were later tested on both factual recall and more problem solving questions. However, possibly the more interesting point is that the researchers found that retrieval practice can actually produce learning (as opposed to being neutral). That is, tests act not only as passive assessments of what is stored in memory (as is often the traditional perspective in education) but also as vehicles that modify what is stored in memory. This startling finding is called the Testing Effect (or the Retrieval Effect) and is possibly due to the cognitive strain experienced when trying to reconstruct knowledge, which is related to the fascinating paper on Desirable Difficulties discussed later in this section. Once again, for me this emphasises how important it is that students are aware of the power of self-testing during revision (and not necessarily of complete exam papers, as discussed in the paper above), and the importance of low-stakes tests in the classroom.
My favourite quote:
Research on retrieval practice suggests a view of how the human mind works that differs from everyday intuition. Retrieval is not merely a read out of the knowledge stored in one’s mind; the act of reconstructing knowledge itself enhances learning. This dynamic perspective on the human mind can pave the way for the design of new educational activities based on consideration of retrieval processes.

Research Paper Title: Both Multiple-Choice and Short-Answer Quizzes Enhance Later Exam Performance in Middle and High School Classes
Author(s): Kathleen B. McDermott, Pooja K. Agarwal, Laura D’Antonio, Henry L. Roediger, III, and Mark A. McDaniel
My Takeaway:
This paper is key for proponents of low-stakes testing. There is little surprise that the authors found that practicing retrieval of recently studied information enhances the likelihood of the learner retrieving that information in the future, as we have seen the benefits of testing for retrieval throughout this paper. But the key difference with this paper is that  the format of the quiz (multiple-choice or short-answer) did not need to match the format of the critical test (e.g. end of unit exam) for this benefit to emerge. This supports the research related to deliberate practice, whereby the activities involved in practice do not need to exactly replicate the final performance for the practice to be effective. The authors also find that frequent classroom quizzing with feedback improves student learning and retention, and multiple-choice quizzing is as effective as short-answer quizzing for this purpose. There is more discussion of the merits of multiple choice questioning in the Formative Assessment section, but a key takeaway for me here is that both multiple-choice questioning and short form skill-based questions are ideal to use in the classroom for regular low-stakes testing.
My favourite quote:
First, we consistently observed that the format of the quizzes did not have to match the format of the unit exam for the quizzing benefits to occur. This is the most important finding of the present report, in that it is novel, was unanticipated from the laboratory literature, and is a critically important practical point for teachers. Even quick, easily administered multiple-choice quizzes aid student learning, as measured by unit exams (either in multiple-choice or short-answer format). Further, the benefits were long lasting: Robust effects were seen on the end-of-semester exams in Experiments 1a, 1b, and 2; that is, both multiple-choice and short-answer quizzing enhanced performance on end-of-semester class exams (again, in both multiple-choice and short-answer formats).

Research Paper Title: The Pretesting Effect: Do Unsuccessful Retrieval Attempts Enhance Learning?
Author(s): Lindsey E. Richland,  Nate Kornell,  Liche Sean Kao
My Takeaway:
So far we have seen the power of testing to enhance retrieval and long-term learning. If that wasn't impressive enough, the next two papers in this section highlight another, rather surprising, benefit of testing - the power of the Pretest.  Firstly, unlikely as it sounds, generating a wrong answer increases our chances of learning the right answer. In this study, one group of students was given the text on which they would be tested – passages with key facts marked, a second group was given the opportunity to memorise the questions they would be asked, a third group was given an extended study period and a fourth group was given a pretest. Even though they got almost every pretest question wrong, students in the pretesting group out-performed all other groups on a final test, including those students who had been allowed to memorise the  test questions. It would seem that the act of unsuccessfully attempting to answer questions has a greater effect on learning than studying the questions on which you are to be tested. The simple takeaway here is that it doesn't matter if students get things wrong - it is the act of attempting to retrieve the answer that is the key. However, we are once again faced with the same Learning v Performance dilemma - a dip in short-term performance (students getting the initial questions wrong) is the price to be paid for an increase in long-term learning. This, of course, needs communicating to teachers and students, and is expressed beautifully in the quote below.
My favourite quote:
When a learner makes an unsuccessful attempt to answer a question, both learners and educators often view the test as a failure, and assume that poor test performance is a signal that learning is not progressing. Thus, compared with presenting information to students, which is not associated with poor performance, tests can seem counterproductive. Tests are rarely thought of as learning events, and most educators would probably assume that giving students a test on material before they had learned it would have little impact on student learning beyond providing teachers with insight into their students’ knowledge base. In terms of long-term learning, however, unsuccessful tests fall into the same category as a number of other effective learning phenomena - providing challenges for learners leads to low initial test performance, thereby alienating learners and educators, while simultaneously enhancing long-term learning. The current research suggests that tests can be valuable learning events, even if learners cannot answer test questions correctly, as long as the tested material has educational value and is followed by instruction that provides answers to the tested questions.

Research Paper Title: Unsuccessful Retrieval Attempts Enhance Subsequent Learning
Author(s): Nate Kornell, Matthew Jensen Hays, and Robert A. Bjork
My Takeaway:
The previous paper outlined the potential benefit to long-term learning of the Pretest Effect, but surely being given a test on something students have never studied before is a complete waste of time? Surely students will just end up guessing, and we all know that guessing is pointless. Well, apparently not! This study sought to answer this question: does an unsuccessful retrieval attempt impede future learning or enhance it? The authors examined this question  using materials that ensured that retrieval attempts would be unsuccessful. They found that as long as students are given appropriate feedback, they will still see a testing benefit even if they get the answer wrong on a pretest. So, here we have a crucial consideration - the benefits of the Pretesting Effect are only realised if immediate feedback is given. Students need to know they are wrong, and what the right answer is.  Often I give my students some form of baseline test before starting a new topic. More often than not, as discussed in the Formative Assessment section, this involves asking a series of Diagnostic Questions at the start of my lesson, listening to their answers an explanations, and adapting my teaching accordingly. The benefit of this approach (or so I had assumed) was purely so I could get a sense of the current levels of understanding in the class, address any misconceptions, and move on to the new topic when I deemed the class ready. Amazingly, it seems that the students were also benefiting from this approach, even if they were being tested on material they have never encountered before. So, a written baseline test before teaching a new topic is likely to have a positive effect, so long as you go through the answers immediately with students. But, if you combine it in a formative assessment setting, then the benefits are likely to be even greater. One word of caution: students may become demoralised if you give them a test on material they cannot do (and indeed should not be expected to do if they have not studied it before). My advice here is to explain, open and honestly to them, that this test is purely for you, their teacher, to find out where they are at so you can better help them, and that the mere act of them attempting to answer the questions is actually helping them learn.
My favourite quote:
The current findings support Izawa’s (1970) argument that tests potentiate the learning that occurs when an answer is presentedafter a test, even if the test is unsuccessful. The results also suggest that, in situations where tests and study opportunities are interleaved or testing is followed by feedback, the benefits of testing go beyond the benefits attributable to the learning that happens on successful tests. With respect to theoretical explanations of the testing effect, this finding is important because it demonstrates that the benefits of testing are not limited to the benefits of successful retrieval; rather, for a theory to fully explain the benefits of tests, it needs to explain the benefits of retrieval failure as well as the benefits of retrieval success. Successful tests obviously play a role, and perhaps a unique role—the findings do not imply that unsuccessful tests and successful tests are equally effective or that they are necessarily effective for the same reasons—but unsuccessful tests can also have a positive effect on long-term retention.

Research Paper Title: The generation effect: A meta-analytic review
Author(s): Sharon Bertsch, Bryan J Pesta, Richard Wiscott and Michael A McDaniel
My Takeaway:
Closely related to the Testing Effect comes the Generation Effect. In a typical generation experiment, participants are asked to either generate the to-be-learned items themselves—for example, by producing opposites when presented with a word (e.g., hot–???)—or to simply read the items (e.g., long–short). A later retention test is then administered, which usually consists of presenting the cues (hot–???, long–???) and asking participants to recall their corresponding targets (cold, short). The finding from this extensive meta-analysis, is that when people learn material by generating components of it themselves, the effect on long term retention is far greater than simply reading them. Once again, we are faced with the issue that most of the experiments done are not in the realms of mathematics, but the amount of rules that need to be learned in maths suggest that a strategy such as this which improves recall could prove very useful. Something as simple as providing students with a list of notes (on the rule of fractions, for example) and leaving out key words for them to fill in themselves is likely to be far more benefical then a set of notes without the gaps, as may be found in a revision guide.
My favourite quote:
The current findings support Izawa’s (1970) argument that tests potentiate the learning that occurs when an answer is presentedafter a test, even if the test is unsuccessful. The results also suggest that, in situations where tests and study opportunities are interleaved or testing is followed by feedback, the benefits of testing go beyond the benefits attributable to the learning that happens on successful tests. With respect to theoretical explanations of the testing effect, this finding is important because it demonstrates that the benefits of testing are not limited to the benefits of successful retrieval; rather, for a theory to fully explain the benefits of tests, it needs to explain the benefits of retrieval failure as well as the benefits of retrieval success. Successful tests obviously play a role, and perhaps a unique role—the findings do not imply that unsuccessful tests and successful tests are equally effective or that they are necessarily effective for the same reasons—but unsuccessful tests can also have a positive effect on long-term retention.

Memory: Revisionkeyboard_arrow_up
Back to Top

A game-changer for me when reading research into revision was just how ineffective some of the most common revision strategies that students common use really are. Making students aware of this is crucial. Alongside the research presented in this section, I would strongly advise reading the papers in Memory: Forgetting, Spacing and Interleaving, and Memory: Testing especially with regard to the importance of testing and spacing. They are two very simple techniques that are likely to significantly improve a student's ability to store and recall information. Two final things I tell my students as a result of this research. Firstly, I emphasise the difference between learning and performance is key, because many of the most effective revision techniques will not be as effective in the short-term on performance, but are likely to lead to long-term learning. Secondly, I tell them that revision should not be easy. If it feels easy (like re-reading and highlighting does), then they are not learning. 

Just before we get into the research, LearningSciences.org have produced a wonderful series of revision materials based on research. These "Six Strategies for Effective Learning" are available as posters or PowerPoints, and are ideal to share with your students to help their revision be as effective as possible. The six strategies are:
1. Spaced Practice: Poster, PowerPoint
2. Retrieval Practice: Poster, PowerPoint
3. Elaboration: Poster, PowerPoint
4. Interleaving: Poster, PowerPoint
5. Concrete Examples: Poster, PowerPoint
6. Dual Coding: Poster, PowerPoint


Research Paper Title: Improving Students’ Learning With Effective Learning Techniques
Author(s): John Dunlosky, Katherine A. Rawson, Elizabeth J. Marsh, Mitchell J. Nathan, and Daniel T. Willingham
My Takeaway:
This paper outlines 10 learning techniques and discusses the relative merits of each one, in terms of their impact on learning and whether this is also true given different student characteristics. The two techniques that come out on top are Practice Testing - which can include something as simple as recalling information from flashcards, or working through questions from a textbook/worksheet, and Distributed Practice - spreading the learning/revision of a topic over different time periods instead of cramming. Interestingly, some of the techniques found to have little positive impact on learning include: highlighting, re-reading and use of imagery for text descriptions. This all fits in nicely with my mantra to students: the only way to get good at maths is to do maths. Note, that simply doing exam papers may not be as worthwhile as focused practice on a specific topic as it is often hard to isolate areas for improvement. The paper on Deliberate Practice in the Explicit Instruction section deals with this in more detail.
My favourite quote:
Concerning students’ commitment to ineffective strategies, recent surveys have indicated that students most often endorse the use of rereading and highlighting, two strategies that we found to have relatively low utility.

Research Paper Title:
What makes distributed practice effective?
Author(s): Aaron S. Benjamin and Jonathan Tullis
My Takeaway:
This paper seeks to explain the benefits of distributed practice during students' revision in terms of a theory of reminding. As well as confirming the benefits of distributed practice that we have seen in the paper above, and in all the work on spacing in the first Memory section, the authors of this paper make three additional points that I found particularly interesting. Firstly, they point out that distributed practice is a bit of a balancing act. We have seen in the work of Robert Bjork the importance of forgetting for learning. Students must have the opportunity to forget in order that when they practice the material again the storage strength of their memories are increased. Therefore, the timing of distributed practice is crucial - too much forgetting leads to unlikely reminding, and too little forgetting leads to impotent reminding. This will differ from student to student and topic to topic, and the best way for students to tell is probably through repeated self-testing to find the perfect balance. Secondly, because reminding slow does forgetting, the authors suggest that expanding interval retrieval schedules should prove superior for long-term retention than constant-interval schedules. So, students may revisit (specifically, retest themselves) on fractions after one week, then a further two weeks later, then a further three weeks, and so on. This requires careful planning on the student's part. Finally, the authors explain who reminding enhances the memory of the initial learning of the event, not the subsequent retrievals of it, so it is of utmost importance that the initial encoding is sound. This is expanded on in the quote below, and obviously has implications for us teachers in terms of the importance of how we initial present ideas and concepts to students.
My favourite quote:
A second implication of the reminding view presented here is that the memory enhancement occurs for the original event, not for the reminding event. Applied to skill learning, this view suggests that it is the quality of the original encoding that is particularly important for successful acquisition. If one initially learns a poor golf swing, for example, then the many later practice opportunities will serve to reinforce those bad habits, not correct them. How feedback plays a role in such corrections remains unexplored: it may be that corrective feedback can decrease reminding on an undesirable original memory, or that it serves to “tune” that memory by enhancing reminding of the original event and the corrective information.

Research Paper Title:
What will improve a student's memory?
Author(s): Daniel Willingham
My Takeaway:
This is a superb paper, which contains three really useful, practical pieces of advice to help students with their revision:
1) If you want to remember what things mean, you must select a mental task that will ensure that you think about their meaning, and if the task has little meaning, then use a mnemonic. We have seen from the Encoding section that students remember what they think about. This message needs spreading loud and clear to students. They must be engaged and thinking about the material. For example, if they can think about the meaning behind, say, the different forms of a quadratic expression and the turning and crossing points of its graph, then they are more likely to remember it. The problem is, quite a bit of maths appears to have no meaning, especially to students, and thus simply needs remembering. In these circumstances, the key is to make sure the memory itself has meaning to the student. So, if students are struggling with the meaning of a topic, then mnemonics can help. "Keep, flip, change" springs to mind for dividing fractions, and there are numerous slightly/highly inappropriate ways students have devised to remember SOH CAH TOA, which instill meaning on it for them. If it works for them - and they don't repeat them when my Headteacher is in the room - then I am happy.
2) Memories are lost mostly due to missing or ambiguous cues, so make your memories distinctive, distribute your studying over time, and plan for forgetting by continuing to study even after you know the material. Once again, mnemonics can help here. So long as they are distinctive, students should be able to associate them with the relevant memory and hence be more likely to successfully retrieve it. Encouraging students to come up with their own mnemonic that have meaning to them is a good idea. And we have of course seen may times in the Memory section the importance of spaced practice. This is possibly the single best piece of advice I give my students: test yourself, leave it a while, and then test yourself again.
3) Individuals’ assessments of their own knowledge are fallible, so don’t use an internal feeling to gauge whether you have studied enough. Test yourself, and do so using the same type of test you’ll take in class. Once again we see the benefits of testing, and also a call back to the fact that students (just like any of us) are not the best at judging how much they know a topic. This directly relates to the principle that if revision feels easy (strategies like rereading and highlight certainly give that comforting impression), then it probably isn't effective revision. Revision should require thought and feel difficult, and the best way to achieve that  -and indeed the only way to discover if you truly do know so something - is to test yourself.
Each of these principles is elaborated in greater detail, together with practical strategies, making it an ideal document to share with your students.
My favourite quote:
Many of my students also tell me that they reviewed their notes and were quite surprised when they did not do well on the test. I’ve found that these students typically know little about how their memories work and, as a result, do not know how to study effectively.

Research Paper Title: A Shield against Distraction from Environmental Noise
Author(s): Niklas Halin
My Takeaway:
This paper is fascinating - and potentially dangerous if not read carefully!  Participants were given four passages to read about fictitious cultures, two in an easy to read font (Times New Roman) and two in a difficult to read font (Haettenschweiler). While reading two of the passages, they heard someone describing another fictitious culture, which they were told to ignore. They then took multiple-choice tests over the passages to assess their learning. The results we as follows: when looking at the easy font, the passage with the background noise was remembered much more poorly than the passage read in silence. However, when looking at the hard font, the passage with background noise was remembered a little bit better than the passage read in silence! What is going on there? Well, it seems that when people really need to concentrate, they are good at blocking out environmental noise. However, when they consider something easy, then environmental noise can be distracting - although students may not notice this - and leads to poorer longer term recall. This is related to working memory - if you are really concentrating then there is little room in working memory for distracting noise to fit in, whereas when you are not concentrating as hard, there is plenty of spare working memory capacity for a distractor such as noise to work its way in there. My main takeaway - and indeed, what I told my students as soon as I read this paper - was that whilst they may do really hard study in silence, they they may be tempted to study easy material with their friends around, or music on, and so on. If they choose to do this, then they are probably going to remember less and/or it’s going to take a lot longer to process that information. There is also an interesting parallel with the recurring theme running throughout the Memory section of the importance of making learning difficult. Perhaps one slightly radical implication from this study is to make students revision material hard to read so that they concentrate more, which will aid memory.
My favourite quote:
The current thesis tested whether the shielding effect of higher focal-task engagement would also hold in a more applied context, with sounds that are more common in the built environment compared to tones (e.g., background speech, road-traffic noise, and aircraft noise), and with tasks that resemble those that people work with in offices and schools (i.e., proofreading and memory for text). The results of this thesis show that one way to attenuate distraction to environmental noise on office-related tasks is to promote focal-task engagement. A simple way to achieve higher focal task engagement is by merely changing the font of the text to one that is harder to read. This may not only be a practical solution to reduce the impairment of environmental noise on cognitive performance in work and school environments. It may also be an intervention that can reduce error rates in health care situations, but also be of aid to specific populations of individuals that generally have poorer attention control, like individuals with ADHD, children, or elderly people. Possibilities like these are something for future research to investigate further.

Research Paper Title: Expecting to teach enhances learning and organization of knowledge in free recall of text passages
Author(s): John F. Nestojko, Dung C. Bui, Nate Kornell & Elizabeth Ligon Bjork
My Takeaway:
This is a nice revision strategy to use that has been shown to be effective. Simply telling students that they will teach something to another student changes their mindset so much that even if they don’t actually teach the information, they remember it better later on when tested. The effect on their retention was even greater than if they were just told to learn the material for a test. This suggests that a pretty easy way of improving a student's revision programme is to regularly have them "teach" other students, as the preparation for this teaching could have significant benefits.
My favourite quote:
Expecting to teach appears to encourage effective learning strategies such as seeking out key points and organizing information into a coherent structure... Students seem to have a toolbox of effective study strategies that, unless prodded to do so, they do not use.

Research Paper Title: Why Students Think They Understand—When They Don't and How to Help Students See When Their Knowledge Is Superficial or Incomplete
Author(s): Daniel T. Willingham
My Takeaway:
Two fascinating papers by Daniel Willingham that seek to explain something I have noticed regularly over the last 12 years - often students think they know something better than they actually do. Willingham put this down to issues of being fooled by  Familiarity (specifically mistaking it for the ability to recollect) and Partial Access (quick retrieval of partial information leads to the assumption that you could retrieve all the information). Willingham goes on to identify three common ways these features can manifest themselves in school:
1) Rereading - students glance over their notes for Fractions and think "that seems familiar, I must know that)
2) Shallow Processing - related to the key principle from Cognitive Science that students remember what they think about, if they are not engaging with the core principle of a lesson (the fundamental laws of algebra), but instead focusing on some less pertinent aspect (the fact that letters are written in alphabetical order), then they could well leave the lesson believing they have understood everything.
3) Recollecting Related Information - students know a lot of information related to the target topic, and that makes them feel as though they know the target information. So students know a bit about straight line graphs, so assume they can work out the gradient, find the equation of perpendicular lines, etc.
Willingham then goes on to suggest strategies to help students avoid these traps. These include:
  • Make it clear to students that the standard of "knowing" is the "ability to explain to others," not "understanding when explained by others."
  • Require students to articulate what they know in writing or orally, thereby making what they know and don't know explicit, and therefore easier to evaluate, and easier to build on or revise.
  • Begin each day (or selected days) with a written self test.
  • Ask students to do self tests at home or in preparing for examinations.
Notice how these utilise the Retrieval Effect discussed in length in the first Memory section, and also tap into the recurring theme throughout this whole page - if learning feels easy, it probably isn't learning. Finally, an important implication of this paper is that decisions of a class' understanding of a topic that are based on students' own self-assessment (for example, asking students to indicate how confident in a topic they are, or asking them to place a sticker next to their "level" of understanding) are likely to be pretty dodgy indeed.
My favourite quote:
Cognitive science research confirms teachers' impressions that students do not always know what they think they know. It also shows where this false sense of knowledge comes from and helps us imagine the kinds of teaching and learning activities that could minimize this problem. In particular, teachers can help students test their own knowledge in ways that provide more accurate assessments of what they really know—which enables students to better judge when they have mastered material and when (and where) more work is required.


Formative Assessment and Questioning keyboard_arrow_up
Back to Top

Dylan Wiliam (interviewed on my podcast here) likes to think of formative assessment as "responsive teaching", and it is a fundamental part of every lesson I teach. When done well it allows me to get an accurate snap-shot of my class' understanding of a concept at a given time and adapt my teaching accordingly. Now, there are critics of formative assessment, such as David Didau, who argue that because of the distinction between Learning and Performance (see Memory section), formative assessment can only capture performance, which is an unreliable measure of learning. Whilst I see some validity in that claim, I would argue that the main purpose I use formative assessment is to identify and understand misconceptions, and for that there is no more effective tool. Needless to say, I am slightly biased when it comes to advocating the use of multiple choice questions for formative assessment, having created Diagnostic Questions, but John Mason's wonderful paper in this section provides an incredible guide to the different types of questions we teachers can ask. This section will be my attempt to survey the literature for the most effective formative assessment practices around.

Research Paper Title: Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments
Author(s): Justin Kruger and David Dunning
My Takeaway:
Before we get into the concept of formative assessment, we first need to establish if there is a need for it. Why can't we simply say to students "do you understand this topic?", or "what topics do you need help on?" and base our teaching decisions on that? Well, to put it simply, because students (like most novices) are not particularly good judges of their own abilities. This lovely paper describes this phenomenon as the Dunning Kruger Effect. In a series of tests of humour, grammar and logic, the researchers found that people tend to hold overly favorable views of their abilities in many social and intellectual domains. The authors suggest that this overestimation occurs, in part, because people who are unskilled in these domains suffer a dual burden: not only do these people reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the meta-cognitive ability to realise it. What are the implications for teaching? Well, if we assume that most students are novices in terms of their understanding of mathematics, then it is likely that they overestimate their abilities in mathematics. Hence, relying on their own judgement of their abilities is likely to be flawed, and thus we need another, more reliable measure. In the long term, this could be the results from a range of sumamtive assessments, but in the here and now of the busy classroom, our best effort is likely to come from a well designed question. This also calls into question the practices of traffic lights, thumbs-up, sticking postit notes on level descriptors, and other types of common self assessment strategies I have seen (and used!). How can we help students become better judges of their own abilities? Well, paradoxically the authors found that improving the skills of participants, and thus increasing their meta-cognitive competence, helped them recognise the limitations of their abilities. Once again, helping our students acquire more knowledge seems to be the key.
My favourite quote:
In sum, we present this article as an exploration into why people tend to hold overly optimistic and miscalibrated views about themselves. We propose that those with limited knowledge in a domain suffer a dual burden: Not only do they reach mistaken conclusions and make regrettable errors, but their incompetence robs them of the ability to realize it. Although we feel we have done a competent job in making a strong case for this analysis, studying it empirically, and drawing out relevant implications, our thesis leaves us with one haunting worry that we cannot vanquish. That worry is that this article may contain faulty logic, methodological errors, or poor communication. Let us assure our readers that to the extent this article is imperfect, it is not a sin we have committed knowingly.

Research Paper Title:
Inside the Black Box. Raising Standards Through Classroom Assessment and Mathematics Inside the Black Box: Assessment for Learning in the Mathematics Classroom
Author(s):
Paul Black, Dylan Wiliam and Jeremy Hodgen
My Takeaway:
This is the bible of formative assessment, full of practical steps that are easy to implement and can make a huge difference. The key finding is that formative assessment raises standards of learning, but there is still plenty of room for improvement. Practical strategies that classroom teachers can use to improve students' learning include: feedback to any pupil should be about the particular qualities of his or her work, with advice on what he or she can do to improve, and should avoid comparisons with other pupils; for formative assessment to be productive, pupils should be trained in self-assessment so that they can understand the main purposes of their learning and thereby grasp what they need to do to achieve; and opportunities for pupils to express their understanding should be designed into any piece of teaching, for this will initiate the interaction whereby formative assessment aids learning. Two points in particular stood out for me:
1) Students can (and should) be trained in self-assessment. But this is difficult to do. We have seen from the paper above that novices find it difficult to judge their own abilities. We can try to help by providing things like "pupil friendly level descriptors", but these can be flawed, as Daisy Cristodoulou so eloquently explained when I interviewed her. Perhaps the best thing we can do to help students develop good self-assessment skills is to provide them with the necessary knowledge to understand the given domain, and provide the kind of constructive, useful feedback that we will discuss in the Feedback section.
2) Formative Assessment is (or at least should be) responsive teaching. It provides an amazing opportunity to find out what your students do, or do not understand, and react accordingly. As a result of reading this paper, together with the papers that follow, I ask three diagnostic questions at the start of every lesson, and adapt the lesson depending on the students' responses.
My favourite quote:
The main plank of our argument is that standards are raised only by changes which are put into direct effect by teachers and pupils in classrooms. There is a body of firm evidence that formative assessment is an essential feature of classroom work and that development of it can raise standards. We know of no other way of raising standards for which such a strong prima facie case can be made on the basis of evidence of such large learning gains.

Research Paper Title: Five Key Strategies for Effective Formative Assessment
Author(s): Dylan Wiliam for the National Council of Teachers of Mathematics
My Takeaway:
A wonderful, practical guide that discusses five key classroom practices that make the processes of formative assessment as effective as possible. Each is backed up by research evidence and illustrated with examples, and I would strongly advise reading the full paper as I will not do it justice here. The five strategies are:
1) Clarifying, sharing and understanding goals for learning and criteria for success with learners. A key strategy I took form this was sharing examples of other students' work is a great way to clearly convey the kind of work you are expecting in a way students can relate to more than abstract, confusing learning objectives.
2) Engineering effective classroom discussion, questions, activities, and tasks that elicit evidence of students' learning. This is all about planning effective questions. These might not necessarily be the kind of open-ended, probing questions that teachers are often advised to ask, but instead carefully planned, multiple-choice diagnostic questions. These have the advantage of being quick to ask, quick to collect information, and incorrect answers reveal the specific misconceptions students have. Needless to say, I love a Diagnostic Question
3) Providing feedback that moves learning forward. This is directly relevant to the Marking and Feedback section. The key point is that feedback should be more work for the student than the teacher, and the student should have time to read, respond to and act upon that feedback.
4) Activating students as owners of their own learning. This is fascinating. Dylan suggests that traffic-lights are a good way of encouraging this, which is a strategy that I have routinely dismissed as I have found students tend to just show green so I will leave them alone! However, Dylan adds the advice to ask students who demonstrate green cards to explain the concept to those with red cards. This gives students a stronger incentive to be honest.
5) Activating students as learning resources for each other. Two interesting findings here. Firstly, feedback students give to each other is likely to be more direct and hard-hitting than a teacher would give. Even more interestingly, the person providing the feedback benefits just as much as the recipient because she or he is forced to internalise the learning intentions and success criteria in the context of someone else k work, which is less emotionally charged than doing it in the context of one's own work. I would add an important caveat to this - that only works if the feedback is correct. Providing students with answers is an obvious way around this, but we must ensure students arrive at these correct answers in the right way.
My favourite quote:
The available research evidence suggests that considerable enhancements in student achievement are possible when  teachers use assessment, minute-by-minute and day-by-day, to adjust their instruction to meet their students' learning needs. However, it is also clear that making such changes is much more than just adding a few routines to one's normal practice. It involves a change of focus from what the teacher  is putting into the process and to what the learner is getting out of it, and the radical nature of the changes means that the support of colleagues is essential. Nevertheless, our experiences to date suggest that the investment of effort in these changes is amply rewarded. Students are more engaged in class, achieve higher standards, and teachers find their work more professionally fulfilling. As one teacher said, "I'm not babysitting any more."

Research Paper Title: Mathematics Inside the Black Box: Assessment for Learning in the Mathematics Classroom
Author(s):
Jeremy Hodgen and Dylan Wiliam
My Takeaway:
This is a wonderful, mathematically specific paper that follows on nicely from the generic research that kick-started this section. Indeed, this paper is so good it makes a second appearance in the Marking and Feedback section. The authors discuss five principles of learning, which sound so obvious, but whose effects can be powerful. I will not do the principles justice here, so please read it in full:
1) Start from where the learner is - this is obvious, but difficult. The majority of maths teaching (in my experience, anyway) involves teaching something that the students have already encountered before, or which relies on pre-existing knowledge. Attempting to discover where students are at before you attempt to teach them new knowledge - and in particular which misconceptions they hold - is one of the most important parts of teaching
2) Students must be active in the learning process - we have seen throughout these papers that students remember what they think about, and sometimes learning needs to be difficult. If students are passive bystanders in lesson - and by that I mean if they are not thinking about the content of the lesson - they are less likely to learn
3) Students need to talk about their ideas - this is a tricky one, as students often find it difficult to articulate their thinking. The authors provide some really useful prompts and frameworks that can be used straight away, such as Can you put Amy’s idea into your own words?, What can we add to Saheera’s answer? Well, if you’re confused you need to ask Jack a question.
Which parts of Suzie’s answer would you agree with? Can someone improve on Simon’s answer?
4) Students must understand the learning intention. This requires an understanding of two key things: what would
count as a good quality work (success criteria), and where they stand in relation to that target. Knowing these things allows students to steer their own learning in the right direction, and is known as meta-cognition. Effective peer and self assessment are key to this, and practical tips for making both effective are provided.
5) Feedback should tell pupils how to improve - this will be covered in depth in the Marking and Feedback section
Practical ideas on how to achieve all of these are provided throughout this wonderful paper.
My favourite quote:
The changes in practice recommended here are not easily made. They require changes in the ways teachers work with students, which may seem risky, and which will certainly be challenging. The work we have done with teachers suggests that the teachers who are most successful are those who change their practice slowly, by focusing on only one or two aspects at a time. As they become skilled with these new ideas, and incorporate them into their natural practice, they can then turn their attention to new ideas. Teachers who try to change many things about their practice at the same time are unlikely to be successful.

Research Paper Title: Effective Questioning and Responding in the Mathematics Classroom
Author(s): John Mason
My Takeaway:
This is, quite simply, the best thing I have ever read on questioning in mathematics. There is so much good advice in this paper by John Mason that the best thing I could probably do is copy and paste the entire paper and have that as my Takeaway. However, I will resist and instead focus on a couple of the main things that have changed my teaching. The first is Mason's point that the distinction between open and closed questions is rather a pointless one. My work on Diagnostic Questions has shown me the value in a well-designed closed question in term of quickly finding out information about a class' understanding of a concept, as well as provoking fruitful discussion about misconceptions. As Mason so beautifully explains: "Questions are just words with a question mark: the notion of openness and closedness is more to do with how the question is interpreted than with the question itself". Then there is the section on using question effectively. A really important point is that I had not considered was to avoid using questions for controlling purposes, and instead make use of direction instruction or statements. This ensures that questioning in the classroom is used only to promote mathematical thinking, and not behaviour management. I also find that asking students questions such as "why exactly do you feel it is appropriate to do that?" always leaves open the possibility of an unwelcome, and public, discussion taking place, whereas a simple "stop that" might allow the focus of the lesson to return to mathematics. Next comes "funneling" - something I am regularly guilty of - whereby you give your students so many clues in the desperate hope they will begin to figure out what is in your mind. Mason's advice in that instance is to simply tell them, and then enquire as to why they had not thought of it themselves. This is likely to be much quicker and more fruitful than kidding yourself that students have actually understood something because you essentially directed them straight to the answer. Finally, Mason emphasises the importance of being genuinely interested in not only in what learners are thinking, but in how they are thinking, in what connections they are making and not making. Techniques such as giving students more time to think, and asking for methods instead of answers can make a huge difference to the atmosphere in the classroom. I could go on, but I better stop there. This is a truly wonderful paper.
My favourite quote:
Although a very common activity, question asking is at best problematic and at worst an intrusion into other people’s thinking. By catching yourself expecting a particular response you can avoid being caught in a funnelling sequence of ‘guess what is in my mind’. By being explicit at first, then increasingly indirect in your prompts, you can assist learners to internalise useful questions which they can use for themselves to help them engage in effective and productive mathematical thinking. Above all, the types of questions you ask will quickly inform your learners of what you expect of them, and covertly, of your enacted philosophy of teaching. The key to effective questioning lies in rarely using norming and controlling questions, in using focusing questions sparingly and reflectively, and using genuine enquiry-questions as much as possible. This means being genuinely interested in the answers you receive as insight into learners’ thinking, and it means choosing the form and format of questions in order to assist learners to internalise them for their own use (using meta-questions reflectively). The kinds of questions you ask learners indicates the scope and breadth of your concern for and interest in them, as well as the scope, aims, and purposes of mathematics and the types of questions that mathematics addresses.

Research Paper Title:
Is there value in just one?
Author(s): Caroline Wylie and Dylan Wiliam
My Takeaway:
This paper deals with answering a key question a teacher regularly needs to ask themselves: "when are my students ready for me to move on?", and was very influential when I was developing Diagnostic Questions. It makes the point that waiting for test data to confirm this is often unsuitable due to the time lag and the abundance of information, and relying on student judgement of their own readiness is flawed. With teachers needing to make this decision "on the fly" mid-lesson, the authors suggest that one well-written question, can do the job. The question should have three characteristics:
1) designed for easy collection of information (I use one finger for A, two for B, etc);
2) Incorrect responses assist the teacher diagnose what students do not understand, and, ideally, provide ideas about what to do about this (this is all comes down to the choice of good distractors);
3) Correct responses support a reasonable inference that students understand the concept being assessed (students should not be able to get the question correct whilst still holding key misconceptions).
Whilst the information gleaned from asking this one question is not going to be perfect, there is no quicker, more accurate way of a teacher doing it mid-lesson. I ask three of these questions each lesson. If the responses of my students reveal misconceptions, then I respond accordingly - even if this takes up more of the lesson than I had intended. For my take on the use of such questions, see my Pedagogy videos.
My favourite quote:
The issue of “readiness to move on” is a familiar one to teachers, and the decision not to move on will initiate routines that are also familiar: to engage in whole class remediation, to pull aside a small group or individual students for additional assistance, to construct alternative learning opportunities that will assist students in their learning, and so forth, so that the teacher can later reassess the situation and proceed in the learning sequence.

Research Paper Title: Using Diagnostic Classroom Assessment: One Question at a Time
Author(s): Joseph F. Ciofalo and Caroline Wylie
My Takeaway:
This paper also looks at the use of diagnostic questioning in the classroom. There is a nice focus on the characteristics of good questions with some nice examples provided that illustrate these characteristics clearly. My favourite part is the description of how these questions might be used in the classroom, providing a suggestion with how teachers might deal with different response scenarios from students. I have recorded a series of videos about how I use these questions in the classroom each day here.
My favourite quote:
The goal is to help teachers better utilize questioning and discussion to improve student learning, a key strategy within formative assessment. By starting with the instructional decisions that teachers make, the intention is to create both a resource and a habit of mind that is of obvious value to teachers. Lesson planning is a natural part of every teacher’s daily activities. Embedding a focus on questioning strategies—through carefully constructed diagnostic items—supports teachers in improving instruction by providing them with a natural extension to their practice, rather than requiring a major shift in mind-set.

Research Paper Title: The memorial consequences of multiple-choice testing
Author(s): Elizabeth J Marsh, Henry L Roediger III, Robert A Bjork, Elizabeth L Bjork
My Takeaway:
We have seen in the section on Memory and Revision the existence of the Testing Effect (or Retrieval Effect), whereby testing can actually enhance learning. This paper attempts to show that whilst multiple choice questions are often used to measure understanding (via formative assessment), they can also have a similarly positive impact on learning as other forms of assessment via the Testing Effect. Interestingly, these benefits are not limited to simple definition or fact-recall multiple choice questions, but extend to those that promote higher-order thinking. The authors also address something a common criticism of multiple choice questions - the fear that the choice of wrong answers (the distractors) can cause students to learn false facts. The authors find that such persistence appears due to faulty reasoning rather than to an increase in the familiarity of the distractors. There is no doubt, however, that good distractors have the potential to cause students to learn faulty information. The authors put it like this: "multiple-choice lures may become integrated into subjects’ more general knowledge and lead to erroneous reasoning about concepts." However before we make all multiple choice questions illegal, three things need saying. 1) The benefits of using multiple choice questions in terms of eliciting information about students' understanding in an efficient way, and positive effects on their learning via the Testing Effect far outweigh the costs.
2) The issue of learning false facts can be overcome via immediate feedback. This could be a discussion in a lesson when using a multiple choice question for formative assessment purposes, automated marking when using a system such as Diagnostic Questions, or through task-specific written feedback.
3) I am firmly of the beleif that you cannot fully understand a topic or concept unless you are also aware of the misconceptions that accompany it. These need to be confronted head-on (as discussed in the Joan Lucariello and David Naff paper in the Explicit Instruction section). I guess it boils down to when you introduce these misconceptions, and here I think there is an argument for waiting until students have grasped and had some success with the basics. They need to know what is right before they confront what is wrong.
My favourite quote:
More generally, the prevailing societal emphasis on testing as assessment is unfortunate, because it obscures the critical pedagogical aspects of testing. Tests, optimally constructed, can enhance later performance, provide feedback to the learner on what has and has not been learned, and potentiate the efficiency of subsequent study opportunities.

Research Paper Title: Multiple-Choice Tests Exonerated, at Least of Some Charges: Fostering Test-Induced Learning and Avoiding Test-Induced Forgetting
Author(s): Jeri L. Little, Elizabeth Ligon Bjork, Robert A. Bjork, and Genna Angello
My Takeaway:
A common criticism of multiple choice questions is that because students know the correct answer is one of the listed options, they can get it correct by recognition as opposed to the retrieval process from long term memory that we know can aid learning. This paper addresses that criticism by testing whether multiple-choice tests could trigger productive retrieval processes—provided the alternatives (distractors) were made plausible enough to enable test takers to retrieve both why the correct alternatives were correct and why the incorrect alternatives were incorrect. In two experiments, they found not only that properly constructed multiple-choice tests can indeed trigger productive retrieval processes, but also that they had one potentially important advantage over cued-recall tests. Both testing formats fostered retention of previously tested information, but multiple-choice tests also facilitated recall of information pertaining to incorrect alternatives, whereas cued-recall tests did not. In other words, so long as the multiple choice question is a good one (see papers earlier in this section for the rules of a good multiple choice question), then when searching for the answer students thoughts turn to reasons why the distractors are incorrect, and hence they exercise the very retrieval processes they have been accused of bypassing. This is huge, because it suggests that whilst multiple choice questions have all the advantages of speed of information gathering discussed earlier, they have an extra advantage of encouraging the students to think harder than if the question was presented without any options to choose from.
My favourite quote:
The present findings vindicate multiple-choice tests, at least of charges regarding their use as practice tests. In fact, our findings suggest that when multiple-choice tests are used as practice tests, they can provide a win-win situation: Specifically, they can foster test-induced learning not only of previously tested information, but also of information pertaining to the initially incorrect alternatives. This latter advantage is especially important because, typically, few if any practice-test items are repeated verbatim on the subsequent real test. From that standpoint, the advantage of initial multiple-choice testing over initial cued-recall testing is a truly significant one.


Marking and Feedbackkeyboard_arrow_up
Back to Top

Whether we like it or not, most of us have to mark and give feedback to students. For many (including me), it is one of the worst parts of being a teacher. The best advice I ever heard on feedback was from Dylan Wiliam, who said "the only good feedback is that which is acted on". You could write the best feedback in the world, but if the your students neither have the time nor the inclination to do it, or if they simply do not understand what you are asking them to do, then it is an absolute waste of time. A second guiding principle that I now try to stick to is "feedback should make students think". All too often my feedback has been so carefully constructed, and full of so many hints, that my students simply have to say "yeah, yeah, I get it now, sir", make a few changes, and then move on, without having learned a single thing, and having cost me a good few hours of my life. For years I have been doing far more work than my students when it comes to marking and feedback. Here we look at what the evidence has to say about marking and feedback, both in terms of the type and the immediacy, when feedback can have a negative effect, and when less is definitely more!

Please note: papers concerning marking and feedback, but that are more explicitly to do with praise can be found in the Motivation and Praise section.


Research Paper Title:
A marked improvement. A review of the evidence on written marking
Author(s): Education Endowment Foundation and Oxford University
My Takeaway:
Lots of practical, easy to implement strategies to improve the effectiveness of marking. My favourites include: dealing with careless mistakes and misconceptions differently when marking; making feedback as specific and actionable as possible so the students actually understand it; "acknowledgement" marking (i.e. a tick to say you have seen the work) is a waste of time, and hence a good rule is mark less but mark better; specific time must be dedicated to students to respond to marking. And then the big one for me: awarding grades/levels for each piece of work means students focus upon these marks at the expense on the formative comments the teacher has written, and probably took ages!
My favourite quote:
It also appears worthwhile to caution against elements of dialogic or triple impact marking that do not follow the wider principles of effective marking that are underpinned by relatively stronger evidence summarised elsewhere in this review. For example, there is no strong evidence that ‘acknowledgment’ steps in either dialogic or triple impact marking will promote learning.

Research Paper Title: The Effects of Feedback Interventions on Performance: A Historical Review, a Meta-Analysis, and a Preliminary Feedback Intervention Theory
Author(s): Avraham N. Kluger and Angelo DeNisi
My Takeaway:
When I interviewed Dylan Wiliam, he cited this paper as the most surprising piece of research he had ever encountered. The authors found over 3,000 research studies published between 1905 and 1995, but found that only 131 of the studies were well-enough designed for their results to be taken seriously. The 131 studies reported 607 effect sizes, which showed that, on average, feedback did increase achievement. But—in what could possibly be one of the most counter-intuitive results in all of psychology—231 of the 607 reported effect sizes were negative. In almost two out of every five studies, feedback lowered performance. What are we to take from this? Well, the authors suggest that the further feedback moves away from the task itself and towards the individual student, the less effective it is - even going as far as to have a negative effect. If teachers are going to dedicate a significant proportion of their time to giving feedback, we must make it task-focused. I will leave the final words to Dylan himself: If there’s a single principle teachers need to digest about classroom feedback, it’s this: The only thing that matters is what students do with it. No matter how well the feedback is designed, if students do not use the feedback to move their own learning forward, it’s a waste of time. We can debate about whether feedback should be descriptive or evaluative, but it is absolutely essential that feedback is productive.
My favourite quote:
The central assumption of Feedback Intervention Theory is that Feedback Interventions change the locus of attention among 3 general and hierarchically organized levels of control: task learning, task motivation, and meta-tasks (including self-related) processes. The results suggest that feedback intervention effectiveness decreases as attention moves up the hierarchy closer to the self and away from the task.

Research Paper Title: The Power of Feedback
Author(s): John Hattie and Helen Timperley
My Takeaway:
The first line of this paper hooked me in: "Feedback is one of the most powerful influences on learning and achievement, but this impact can be either positive or negative." This was a revelation to me. Surely, if I was spending so much time doing something, it has to be beneficial for my students? This is a fascinating, comprehensive paper that addresses all the major issues surrounding feedback. I would strongly recommend reading the part on page 98 about the issues surrounding positive and negative feedback. This really made me realise that some students benefit from praise, for some it does them more harm than good, and some students need a metaphorical kick up the arse. Judging this is difficult, but ultimately comes down to a teacher's relationship with and knowledge of their students. My key takeaway, however, is the fact that feedback is likely to be ineffective if the student simply does not have the relevant knowledge to solve the problem in the first place. I know that sounds obvious, but it wasn't to me. Feedback can only build upon knowledge. You can give all the task-focused prompts in the world, but if the students lack the knowledge to do the work, then feedback is likely to be pretty useless. I have done this myself, literally spending hours making corrections, carefully writing prompts, inventing follow-up questions for students to do, and then one of two things happens - either the students still cannot do it, or I have ended up doing that vast majority of the work, so they can make corrections without thinking. In that instance, I believe it is far better to simply re-teach the students. I write a quick note along the lines of "we will cover this in class", and then I do a a series of worked examples following the principles of Explicit Instruction, and then ask the students to correct their work. This approach has the onus of putting the emphasis back onto the students, saves me hours, and also allows my students to benefit from the positive effects of spacing covered in the Memory section. Of course, determining whether a student could not do the work, or could not be bothered to do the work, can be difficult, and once again comes down to knowledge and relationships.
My favourite quote:
Feedback, however, is not “the answer”; rather, it is but one powerful answer. With inefficient learners, it is better for a teacher to provide elaborations through instruction than to provide feedback on poorly understood concepts. If feedback is directed at the right level, it can assist students to comprehend, engage, or develop effective strategies to process the information intended to be learned. To be effective, feedback needs to be clear, purposeful, meaningful, and compatible with students’ prior knowledge and to provide logical connections. It also needs to prompt active information processing on the part of learners, have low task complexity, relate to specific and clear goals, and provide little threat to the person at the self level. The major discriminator is whether it is clearly directed to the task, processes, and/or regulation and not to the self level. These conditions highlight the importance of classroom climates that foster peer and self-assessment and allow for learning from mistakes.

Research Paper Title: Learning versus Performance
Author(s): Nicholas C. Soderstrom and Robert A. Bjork 
My Takeaway:
This is the second appearance of this lovely paper following its debut in the Memory section, and once again its effect on me has been huge. This paper seeks to question the common assumption that immediate feedback is the best kind of feedback - and leads us on to the final of Bjork's "desirable difficulties". The logic makes perfect sense - if a learner gets something wrong, surely they need to know that they have got it wrong immediately, and why they have got it wrong, in order to correct their mistake? And indeed, during the early phases of knowledge acquisition, immediate feedback may be beneficial to ensure mistakes do not become embedded. However - and wait for this because it is a classic - research presented in this paper suggests that delaying or even reducing feedback can have a long term benefit to student's learning. Why? Well, because regular, immediate feedback can cause learners to become overly dependent upon it (almost seeing it as a crutch to their learning). To relay this to themes covered in other sections of this page, immediate feedback prevents students from thinking hard, and also having the opportunity to forget. I have seen this myself - you give students their work back with detailed feedback and corrections to make, and they look at it and go "yeah, yeah", and make the necessary corrections. But are they actually thinking? Ironically (or perhaps, unfortunately) the better the feedback, the less they need to think. Once again we have the issue of distinguishing between learning and performance. The authors suggest that delaying or reducing feedback is likely to have a determinantal effect on short-term performance, but a positive effect on long-term learning.  What are the implications for us teachers, especially given that the Hattie paper above stresses the potential positive benefits of feedback? Well, one idea I had reading this paper was when initially marking a piece of homework to give no feedback whatsoever, simply to provide ticks and crosses. Then give this work back to the students and see if they could identify the source of their errors and correct them. When I next take the books in, this is when I would give feedback. In my view, there are two advantages of such an approach. Firstly, it will force students to think about why they got a question wrong, instead of just blindly following my step-by-step prompts, and hence become less reliant on me. Secondly, it may enable me to better identify actual gaps in their knowledge. As discussed in the paper above, feedback when students do not have the knowledge to begin with is likely to be a waste of time. However, if indicating where students have gone wrong allows students to have a better stab at those questions, then I can then better distinguish between gaps in their knowledge versus the kind of mistakes that well constructed, task-focused feedback is likely to help resolve. The knowledge gaps may then need addressing again via instruction.
My favourite quote:
Delaying feedback until after the task was completed yielded greater long-term learning than concurrent feedback. Performance gains during acquisition, however, were made more rapidly by those receiving concurrent feedback.
 
Research Paper Title: Formative Assessment: Practical Ideas for Improving the Efficiency and Effectiveness of Feedback to Students
Author(s): Geraldine O’Neill
My Takeaway:
A really good summary of effective feedback practice, together with the value of this idea to either the teacher or the students, all referenced to relevant research-based evidence. There are many interesting practices cited here, but the ones that stood out to me were:
1) Consider feedback in different media/formats, such as on-line, audio-feedback, verbal class feedback, use of ‘clickers’ in large class contexts. This may not only save the teacher time, but may make feedback more permanent and easily accessible to students.
2) Student Requested Feedback -  ask students to submit specific requests for areas for feedback. As this is student-focused feedback it is more inclined to motivate students to act. This also encourages students to take some responsibility in the process.
3) In class peer and self assess feedback activities in terms of discussing and giving feedback to annonymised work. This will help students engage in the feedback process and make them more aware of the desired standard in relation to their own work.
4) Comment in actionable language. This relates to the recurring theme throughout this section that feedback should be task-focussed, but also reminds us that if the students cannot understand what the feedback is on about, it is likely to be a complete waste of time.
My favourite quote:
One of the key themes emerging to address this dilemma is to develop students own self-monitoring skills in order to help them narrow the gap between their performance and the standards expected of them. The timing, type and specification of feedback can also improve student ability to self-monitor. In addition, good feedback should feed into some specific actions that can be used in the next assessment. Feedback need not always be from the academic staff, students themselves are a good resource to each other when given guidance on how to do this. New technologies also open up some efficient feedback opportunities.

Research Paper Title: Mathematics Inside the Black Box: Assessment for Learning in the Mathematics Classroom
Author(s): Jeremy Hodgen and Dylan Wiliam
My Takeaway:
This is a wonderful paper that first made an appearance in the Formative Assessment section, but it is with regard to its advice on marking and feedback that I want to focus on here. Often I find it difficult to write good comments in students' books. Comments such as "show your working" and "correct your mistakes" have tended to have little impact in my experience. The main piece of advice I took from this paper was that the comments I write in students' books such match the kind of questions I would ask the students in class. I have long been an advocate of planning questions as opposed to planning resources, and so thinking of feedback in these terms really made sense to me. It also fits in very well with the general principles that feedback should make students think. The authors kindly provide a suggestion of the types of comments/questions you could write. My favourites include:
1) Enabling pupils to identify the errors for themselves:
- There are five answers here that are incorrect. Find them and fix them.
- The answer to this question is [. . .] Can you find a way to work it out?
2) Identifying where pupils use and extend their existing knowledge:
- You’ve used substitution to solve all these simultaneous equations. Can you use elimination?
- You seem to be having difficulty adding some of these fractions and not others. In question 2 you used equivalent fractions; could you use this on question 4?
3) Encouraging pupils to reflect:
- You used two different methods to solve these problems. What are the advantages and disadvantages of each?
- You have understood [. . .] well. Can you make up your own more difficult problems?
4) Suggesting pupils discuss their ideas with other pupils:
- You seem to be confusing sine and cosine. Talk to Katie about how to work out the difference.
- Compare your work with Ali and write some advice to another student talking this topic for the first time.
5) Helping pupils to show their working:
- The way in which you are presenting graphs is much clearer. Look back at your last work on graphs in February. What advice would you give on how to draw graphs?
- Your solutions are all correct, but they are a bit brief. Look at the examination marking criteria. Work with Leo to produce model answers that would convince the examiner to award you all the marks.
I love this feedback because it is written in a language students understand, it makes students think, it gives them strategies to use if they are struggling, and it should prove to be more work for the students than the teacher. Of course, these comments take time to write, and it is worth bearing in mind that students need to be given dedicated time to act upon this feedback in lessons.
My favourite quote:
The content of effective written comments, of course, varies according to the activity and mathematical content. Often, advice to pupils will be very similar to the kinds of interventions and questions that a teacher uses with the whole class, although it is an opportunity for the teacher to give personalized feedback
 
Research Paper Title: Task-Involving and Ego-Involving Properties of Evaluation: Effects of Different Feedback Conditions on Motivational Perceptions, Interest, and Performance
Author(s): Ruth Butler
My Takeaway:
This is a really interesting study into the effectiveness, and indeed the effects, of different types of feedback. A total of 200 fifth- and sixth-grade students with high or low school achievement were given takes to complete. Individual comments, numerical grades, standardized praise, or no feedback were received after Sessions 1 and 2. The results following a post-test were that interest, performance, and attributions of effort were highest at both levels of achievement after receipt of comments. Indeed, post-test scores were one stand deviation higher for students in the Comments group, with no significant differences between scorers in the other groups. Similarly, ego-involved attributions were highest after receipt of grades and praise. In other words, grades and praise had no effect on performance, and served only to increase ego-involvement. It is perhaps of little surprise that comments were the most successful in generating the kind of response we would want from students. However, I was taken aback by the fact that grades had the same impact as praise in generating ego-involving responses. My conclusion, based on my own experiences with my students, is that grades cause students to think about themselves - eliciting emotions ranging from joy to despair - and this prevent them from focussing on the task itself. This will be discussed further in the next study.
My favourite quote:
The present results further confirmed that individual comments yielded higher task-involved perceptions and lower ego involved ones than either grades or praise and that no feedback yielded perceptions of both kinds of factors as being relatively non-determinative of both effort and outcome. The similar and ego-involved perceptions induced by grades and praise seem particularly significant. Both anecdotal evidence and some research findings suggest that grades are perceived as potent sources of control over learning.

Research Paper Title: Enhancing and undermining Intrinsic motivation; the effects of task-involving and ego-involving evaluation on interest and performance
Author(s): Ruth Butler
My Takeaway:
The last paper showed that task-focussed comments were preferable to grades in terms of subsequent performance on tasks, as well as general involvement and interest in the task. This begs the question: what if comments and grades are combined? This paper provides the answer. 48 eleven-year old Israeli students were selected from the upper and lower quartiles of attainment from 12 classes in 4 schools and worked in pairs over three sessions on two tasks (one testing convergent thinking and the other, divergent). After each session, each student was given written feedback on the work they had done in the session in one of three forms: A) individualised comments on the extent of the match of their work with the assessment criteria that had been explained to each class at the beginning of the experiment; B) grades, based on the quality of their work in the previous session; C) both grades and comments. Students given comments showed a 30% increase in scores over the course of the experiment, and the interest of all students in the work was high. Students given only grades showed no overall improvement in their scores, and the interest of those who had high scores was positive, while those who had received low scores show low interest in the work. Perhaps most surprisingly, the students given both grades and comments performed similarly to those given grades alone—no overall improvement in scores, and interest strongly correlated with scores—and the researchers themselves describe how students given both grades and comments ignored the comments, and spent their time comparing their grades with those achieved by their peers. The message is simple: If you are going to grade or mark a piece of work, you are wasting your time writing careful diagnostic comments.
My favourite quote:
The results confirm the importance of distinguishing between task involvement and ego-involvement when investigating intrinsic task motivation. As hypothesized, both high and low achievers who received comments continued to express high interest both on Session 2 when they anticipated further comments, and at post-test, when they did not.

Research Paper Title: Rank as an Incentive
Author(s): Anh Tran and Richard Zeckhauser
My Takeaway:
The papers above have made clear that if you give students both a grade/mark and a formative piece of feedback, all they will look at is the grade/mark. Here we consider a different type of feedback - instead of telling students their grade, tell them their rank in the class. Whenever I give students back their tests, they are always desperate to find out how everyone else got on, and although I never tell them, they pretty quickly - over whispers, gestures and paper waving - work out their rough ranking in the class. When I interviewed Dani Quinn, the Head of Maths at Michaela School, she explained how she not only ranks her class after every assessment, but puts these rankings on public display on the notice board in the corridor. What does the research have to say about such an approach? Well, this paper makes interesting reading. In one experiment, the researchers gave a class of Vietnamese college students some reading material to prepare for a test that would be given in ten days. Each student received a participation fee. They randomly assigned the students into one control group (with no extra incentive) and three treatment groups (one with rank incentive, one with financial incentive and one with both). They found that: (i) the group that knew that their rankings would be publicised outperformed the control group; (ii) the group that both earned cash for correct answers and knew that their rankings would be publicised outperformed the group that merely earned cash for correct answers. The paper also quotes other research that replicates the general finding that competition improves performance. Before making any sweeping generalisations, we must be aware that the sample size is small and the students are of university age in Vietnam. However, it has made me think. The knowledge that students are going to be ranked may well provide a good incentive for them to put extra effort into their homework or test. However, this must be traded against the possible repercussions of that ranking. Will students who are in the top 3 in the class get complacent? Will students in the bottom 3 in the class give up? As we have seen with much of the research on feedback, a lot of this comes down to the relationship a teacher has with their students, and the school culture as a whole. Michaela's justification is that they believe performance on tests is highly correlated with effort, and so exposing students who perform badly is a way of highlighting the consequences of poor effort. That is fine so long as the correlation hold true. If performance is instead linked to ability, then it will be pretty demoarilsing for those students who may find themselves permanently ranked near the bottom. And what happens if everyone improves? How is that reflected in the rankings? Making students aware of ranking seems to be to be a zero-sum game with a lot of potential risks, but it is something that has certainly caused me to think.
My favourite quote:
Economists admire competition because it promotes efficiency and enables the market system to work efficiently. Rank incentives may be net beneficial in some circumstances, as they encourage all to perform better. They may be detrimental in others. But whatever the net report card, the record is clear. Humans care considerably about their rank, and economic models that seek descriptive relevance must attend to that incentive.

Research Paper Title: The Impact of Self- and Peer-Grading on Student Learning
Author(s): Philip M. Sadler and Eddie Good
My Takeaway:
With so much marking to do, I often like to ask my students to either mark their own work or mark their neighbour's. Indeed, self or peer marking in a fundamental component of my practice of regular low stake tests. But that all begs the question, which is the most useful: self or peer assessment. The findings from this paper are fascinating. Firstly, for either strategy to work students must be familiar with how to mark accurately. This suggests strategies such as exposing students to mark schemes and showing them examples of other students' work and then discussing them are likely to be successful in achieving this aim. Secondly, there are indications of bias in the students' marks: when grading others, students
awarded lower grades to the best performing students than their teacher did, but when grading themselves, lower performing students tended to inflate their own low scores. However, what is most interesting to me is the impact on learning. When students were given a follow-up test based on the same content that they had been marking, students who graded their peers’ tests did not gain significantly more than a control group of students, but those students who corrected their own tests improved dramatically. This implies that self-assessment is more beneficial to learning than peer assessment, and I have put this finding into practice by allowing students to mark their own low-stakes tests. My students also seem to prefer this, and it has led to less sounds of "what does this say?" that often accompanies peer assessment. One thing to bear in mind, however, is you lose the benefits of peer-to-peer discussion and feedback that has been identified in some of the papers in this section.
My favourite quote:
Student-grading is best thought of not as an isolated educational practice, but as a part of the system of learning and assessment carried out in a teacher’s classroom. It involves sharing with students some of the power traditionally held by the teacher, the power to grade. Such power should not be exercised recklessly or unchecked. Such power in their own hands or in those of their peers can make students uncomfortable or wary. Teachers should train their students in the skills entailed in accurate grading and should monitor students for accuracy in self- or peer-grading. When used responsibly student-grading can be highly accurate and reliable, saving teachers’ time. In this study, self-grading appears to further student understanding of the subject matter being taught.



Motivation and Praisekeyboard_arrow_up
Back to Top

I had always been motivated to learn mathematics, so it was quite the shock when I started teaching that not every student's idea of a good way to spend Friday night was solving quadratic equations. Until relatively recently in my career, I was convinced that the best way to motivate the majority of students was to give them praise (and that was only because my wallet could not quite stretch to the confectionery-related demands of my students). However, I always had nagging questions. Was there a better way to motivate my students? Was there a danger that I was doing more harm than good with my use of praise? This collection of research is my attempt to answer those questions.

Please Note: papers concerning motivation and praise, but that are more explicitly to do with marking and feedback can be found in the Marking and Feedback section. It is also worth re-reading Top 20 Principles from Psychology that was covered in the Cognitive Science section as that has several key things to say about motivation.

 
Research Paper Title:
How Praise Can Motivate—or Stifle
Author(s): Daniel J Willingham
My Takeaway:
Praise is often a cornerstone of feedback, both written and in lessons, and this paper from Daniel Willingham seeks to answer when it can be a good or bad thing. Willingham summarises things really nicely in the introduction: A rule of thumb that can summarise this complex research literature is that if you try to use praise for your own ends or even in a conscious attempt to help the student, it is likely to go wrong. If, on the other hand, praise is an honest expression meant to congratulate the student, it will likely be at least neutral or even helpful to the student; even under these circumstances, however, care must be taken in what is praised. This shows you what a thorny issue praise an be. For Willingham, the guiding principles are:
1) Praise should be sincere. if praise is dishonest, controlling or unearned, it is likely to have negative consequences. I have struggled in particular with the last one of these. A student who never hands in their homework finally hands something in, and it is a load of rubbish. Do I praise the fact that at least he handed it in, or does that send a signal that such an effort is acceptable? For Willingham, the answer is to say/write something like "It's great that you finished the assignment, but I'm a little disappointed in the quality of this work because I know you can do better"
2) Praise should emphasise process not ability. Praising ability may lead students to have a fixed view of ability, which may be detrimental to their long-term development (this issues is addressed further in the Mindset section). However, simply praising effort has complications as well - often it is socially more acceptable for students to put as little effort in as possible, and being told "you tried really hard" my be interpreted by a student as "you are thick, but nice try". Willingham's solution is to praise the product of the process - "that is a brilliant solution", as opposed to "you worked really hard on that solution"
3) Praise should be immediate and unexpected. Praise obviously loses much of its informational and motivational impact if the teacher praises a child for having shown good effort two weeks ago. However, this should not be confused with the potential benefits of delayed feedback, which are discussed in the Bjork paper later in this section. Making praise unpredictable is hard to do, but can be of huge benefit. The goal is not simply to get the child to stop asking for praise; it is to help the child to think of their work differently—as something that is done for the student's own satisfaction, not to garner praise from the teacher. This needs to be part of a long term strategy with your students.
My favourite quote:
It likely comes as no surprise that praise is neither an automatic expander of self-esteem, nor the ruin of a child's self-efficacy. Praise can take so many forms that its effects are inevitably complex. Still, some useful generalizations can be made. Praise should be sincere, meaning that the child has done something praiseworthy. The content of the praise should express congratulations (rather than express a wish of something else the child should do). The target of the praise should be not an attribute of the child, but rather an attribute of the child's behavior. Parents and teachers are familiar with the admonition "criticize the behavior, not the child." For similar reasons, the same applies to praise—praising the child carries the message that the attribute praised is fixed and immutable. Praising the process the child used encourages the child to consider praiseworthy behaviors as under his or her control.

Research Paper Title: Self-Efficacy: An Essential Motive to Learn
Author(s): Barry J. Zimmerman
My Takeaway:
I think a lot about how to motivate my students. I have tried attempting to make the maths we study more relevant to their lives, tried the all-singing, all-dancing lessons, used videos, technology and more. These strategies have had mixed success, but even the best have not proved sustainable in the long run. This article has convinced me what I think I have always known - students are motivated by their own success. If you can convince students that they are successful at maths (self-efficacy is defined as "as one's belief in one's ability to succeed in specific situations or accomplish a task"), then they will be motivated to learn more. A similar principle is covered in Daniel Pink's excellent book Drive, where he argues that mastery must come first to make something enjoyable - in other words it is not motivation that leads to success, but success that leads to motivation. This view is further reinforced in the paper Intrinsic Motivation and Achievement in Mathematics in Elementary School (you need to pay a fee to read the full text) which found "Contrary to the hypothesis that motivation and achievement are reciprocally associated over time, our results point to a directional association from prior achievement to subsequent intrinsic motivation". Likewise, the Top 20 Principles from Psychology paper in the Cognitive Science section argues that as students develop increasing competence, the knowledge and skills that have been developed provide a foundation to support the more complex tasks, which become less effortful and more enjoyable. When students have reached this point, learning often becomes its own intrinsic reward. So, the evidence seems remarkably clear - achievement and students' belief in their own ability provide motivation, not the other way around. How you help bring about that success and positive belief in ability is up to you. For some it will be a Mindset approach, whereas for others it will be Explicit Instruction to ensure students are equipped with the fundamentals needed for more complex thinking, and hence further success. 
My favourite quote:
This empirical evidence of its role as a potent mediator of students’ learning and motivation confirms the historic wisdom of educators that students’ self-beliefs about academic capabilities do play an essential role in their motivation to achieve.

Research Paper Title: Motivation for Achievement in Mathematics: Findings, Generalizations, and Criticisms of the Research
Author(s): James A. Middleton and Photini A. Spanias, Arizona State Universit
My Takeaway:
This is a comprehensive review of recent research into motivation in mathematics, which reaches five interesting conclusions:
1) Findings across theoretical orientations indicate that students' perceptions of success in mathematics are highly influential in forming their motivational attitudes. This is directly related to the findings from the paper above. Students need a relatively high degree of success in mathematics for engagement in mathematics to be perceived as worthwhile. This is not the same as saying "only high ability students will be motivated at maths", as through careful instruction and choice of tasks, every student can enjoy success, and hence enjoy the resulting motivation.
2) Motivations toward mathematics are developed early, are highly stable over time, and are influenced greatly by teacher actions and attitudes. This is interesting. Jo Boaler often stresses the importance of children's early experiences with mathematics, and how one bad experience can put you off the subject for life. For me, the takeaway here is that as teachers we need to be good role models. We need to constantly present a positive attitude towards mathematics, showing it as a fun, challenging, inspiring, wonderful subject where it is absolutely fine to make mistakes on the path to understanding. Crucially, we need to even more explicit with these messages with the youngest students we teach.
3) Providing opportunities for students to develop intrinsic motivation in mathematics is generally superior to providing extrinsic incentives for achievement. Students need to want to do mathematics, not because they expect and extrinsic reward, but because it is a subject they value and enjoy. Now, this is easier said than done, and extrinsic rewards may play a part in motivation during the early stages of encountering a tricky, new topic. But emphasising the importance of maths, and the pleasure that can be had from solving problems, is crucial to develop intrinsic rewards.
4) Inequities exist in the ways in which some groups of students in mathematics classes have been taught to view mathematics. Interestingly, the authors cite evidence that is girls who are influenced through gender-role stereotyping, teacher expectations, and peer pressure to view themselves negatively with respect to mathematics motivation. I do not have any quick-fix solutions to this (certainly Jo Boaler's wonderful paper When Do Girls Prefer Football to Fashion shows that it is not as easy as simply shoehorning in supposedly "female friendly" contexts into lessons,) but I now pay special attention to ensuring I do all I can to raise the expectations and self-belief of all my students.
5) Achievement motivation in mathematics, though stable, can be affected through careful instructional design. This all comes down to careful planning. Planning how you introduce topics to make them seem worthwhile. Planning on how you explain concepts, providing worked examples and scaffolding, to support students and help them believe they can understand it. Providing interesting work for them to do. Using Formative Assessment to identify and misconceptions and resolve them early. Being very careful with praise. Basically, all the things we have looked at on this page that constitute good teaching.
My favourite quote:
Motivation to achieve in mathematics is not solely a product of mathematics ability nor is it so stable that intervention programs cannot be designed to improve it. Instead, achievement motivation in mathematics is highly influenced by instructional practices, and if appropriate practices are consistent over a long period of time, children can and do learn to enjoy and value mathematics


Research Paper Title:
Classroom Applications of Cognitive Theories of Motivation
Author(s): Nona Tollefson
My Takeaway:
This is a fascinating paper that examines cognitive theories of motivation and their application to classroom experiences of students and teachers. There is so much to take away from this paper, but I have limited myself to two things directly relevant to this section and my teaching experiences:
1) "Expectancy x value theory" postulates that the degree to which students will expend effort on a task is a function of (a) their expectation they will be able to perform the task successfully and by so doing obtain the rewards associated with successful completion of the task and (b) the value they place on the rewards associated with successful completion of the task. We can help with (a) by following many of the principles described in the paper above - namely providing good explanations, modelling examples, setting appropriate tasks, and careful use of praise. (b) is perhaps the trickier one. External rewards have the problem that students become over-reliant on them, and they beg the question of what happens when those rewards disappear? Helping students value mathematics for the sake of doing mathematics is difficult, but achievable, and for me much of it comes down to the messages we convey as teachers and role models.
2) In our look so far at both praise and feedback, a resounding message is that we should praise effort and not ability. That makes logical sense, as the former appears more readily changeable (in the eyes of students) than the latter. But this paper points to a potential problem with this. I can explain this no better than in the words of the authors themselves: attributing either success or failure to effort is a ‘‘double-edged sword.’’ On one hand, expending effort and being successful brings a sense of accomplishment and pride. However, having to expend extraordinary effort to be successful implies that one has lower ability than persons who can successfully complete the task with limited or moderate effort expenditures. Students who believe they lack the ability to complete academic tasks successfully may not expend effort because failure would be a public admission of low ability. Covington and Omelich explain that not trying and failing is ‘‘not really failing,’’ because ‘‘true failure’’ occurs only in the case where an individual tries hard to accomplish a task and fails to do so. They also explain failure resulting from lack of effort as an attempt to protect and preserve a sense of self-worth. Wow! So, what are we to do about this? Perhaps we need to be careful to praise not effort, but the outcome of that effort. It sounds a pointless distinction, but praising the solution to a tricky multi-mark exam question ("that is a great solution") as opposed to praising effort ("you have worked really hard on that solution") may make a subtle but key difference.
My favourite quote:
Because the relationship between effort expenditure, success, and feelings of pride is complex, teachers and parents need to recognize that telling students to ‘‘try harder’’ and rewarding them for expending effort will not necessarily encourage students to expend additional effort. The task demands, the value of the rewards associated with the task, students’ outcome and efficacy expectations, goal orientations, levels of task involvement, age, and attributions for success and failure on school-related tasks all interact to explain why some students are willing to expend effort and others are not.

Research Paper Title: “That’s not Just Beautiful—That’s Incredibly Beautiful!” - The Adverse Impact of Inflated Praise on Children with Low Self-Esteem
Author(s): Eddie Brummelman, Sander Thomaes, Bram Orobio de Castro, Geertjan Overbeek and Brad J. Bushman
My Takeaway:
I love this paper. Often as a teacher I over-inflate my praise for students who I know have a particularly low self-esteem in mathematics. I might describe an answer to a relatively straight-forward angles question as "incredible", or a written solution to a linear equation as "amazing". Surely this can do no harm, and hopefully give the student a much needed boost? Well, maybe not. The first finding from this paper is that it is not just me - people (in particular parents and teachers) are more likely to give "over the top " praise to children they perceive as having low self-esteem. In order to figure out whether this actually mattered or not, the authors looked at how being given praise impacted on one particular aspect of children’s behaviour – challenge seeking. The students first completed a questionnaire to assess their level of self-esteem, and then were asked to draw a copy of van Gogh’s Wild Roses. The children were told that a professional painter would then assess their drawing, and tell them what he thought of it. In reality, the painter didn’t exist, and children were simply given inflated praise, non-inflated praise, or no praise at all. Afterwards, the children were shown four complex and four easy pictures, and asked to have a go at reproducing some of them. Crucially, they were told that if they picked the difficult picture, they might make a lot of mistakes, but they might also learn lots - in other words, the number of difficult pictures the children chose to draw was taken as a measure of challenge seeking. The authors found that if children with lower self-esteem were given overly-inflated praise, they were less inclined to seek a challenge in the second task – they would go for easy drawings over the harder ones, and therefore miss out on the chance for a new learning experience. On the other hand, children with high self-esteem were more likely to seek a challenge after being given inflated praise. Interestingly, the only difference between the inflated and non-inflated praise was a single word – incredible (“you made an incredibly beautiful drawing!” versus “you made a beautiful drawing!”). The authors suggest that inflated praise might set the bar very high for children in the future, and so inadvertently activates a self-protection mechanism in those with low self-esteem. This once again suggests that positive praise isn’t necessarily good for all children in all circumstances. For children with low self-esteem, although we might feel the need to shower them in adulation, this might end up having precisely the opposite effect. Even words like incredible can end up having a huge unintended impact. So, when I'm telling my students they have done an amazing job, I will choose my words more carefully.
My favourite quote:
In current Western society, everyday life is replete with instances of inflated praise—like “Perfect!” or “That’s incredibly beautiful!” Our research represents the first empirical study of inflated praise. Our findings show that adults are inclined to give inflated praise to children with low self-esteem. Unfortunately, inflated praise may cause children with low self-esteem to avoid challenges that might lead to failure. These findings show that inflated praise, although well-intended, may cause children with low self-esteem to avoid crucial learning experiences.



Differentiationkeyboard_arrow_up
Back to Top

I find differentiation to be one of the most difficult parts of teaching, and is the thing I am asked most about by teachers of all ages and experiences. Before we dive into the literature, it is worth recapping a few important points we have learned from previous sections:
1) Learning styles are not as important as many of us have been led to believe, and what is more important is that material is presented to students in its most appropriate form (see Cognitive Science)
2) Students are not good at judging what they understand, and so any reliance on student self-assessment as a guide to what they know is not likely to be accurate (see Cognitive Science)
3) Learning and Performance are very different things, and we should be very careful about inferring anything about learning from a student's performance (see Memory)
4) Overlearning can be beneficial (see Memory)
These all suggest that we should not worry too much about differentiation. We should instead focus on teaching all the students as well as we can. However, then we have the Expertise-Reversal Effect (see Cognitive Load Theory), which suggests that once students have mastered a concept, their learning can be inhibited by studying worked examples (a key tool of Explicit Instruction), and they would instead benefit more from solving problems. Hence, perhaps we cannot present material in the same way to all students. Deary me, teaching is hard! So, bearing all that in mind, this is my attempt to survey the literature to investigate the concept of differentiation.

Research Paper Title: What Is Developmentally Appropriate Practice?
Author(s): Daniel T. Willingham
My Takeaway:
I love this paper. Willingham begins by providing evidence against Piaget's proposed stages of development, and instead argues two key points. Firstly, development does not occur in discrete, pervasive stages. Anyone aware of this fact is likely to feel the same way I do when we are expected to show students making progress every single time data is reported. Secondly, children’s cognitive abilities vary by task and day, not just by age and individual developmental pace. And that causes a huge problem when planning for differentiation in your lesson. You simply cannot know how well students will respond to a particular task or topic until you do it. Simply assuming some students will not be up to the work denies them the opportunity to prove you wrong. Likewise, assuming other students do not require as much support and guidance risks leaving them with an incomplete understanding. With this in mind, Willingham gives four pieces of advice to teachers with regard to differentiation:
1) Use information about principles, but not in the absolute. It is useful knowing what students should know - and age related expectations are pretty good for this. They may even give you insight into how students may think and their likely misconceptions. But they should act only as a guide, and should not be taken as absolute.
2) Think about the effectiveness of tasks. Often it may not be the concept that is the problem, but the task itself. For example, I have had success introducing key concepts in algebra to Year 7s using The Border problem, whereas other tasks have failed to provide the same level of understand to older, higher achieving students. Finding good tasks, explanations or analogies for key concepts and always learning from each time you use them is crucial.
3) Think about why students do not understand. Related to the point above, if a child seems to not understand something, the issue may not be with the concept itself, but a feature of the task, or  - and I increasingly find this is a case - a missing bit of background knowledge. Effective use of Formative Assessment to assess background knowledge and resolve any misconceptions may be the key to this.
4) Recognise that no content is inherently developmentally inappropriate. This is a big claim! Willingham gives the example of probability: "the notion of probability is embedded in games that children play using dice, and this understanding can be expanded to include the notion of a distribution. Thus, one approach is to help the child gain an intuitive appreciation of a complex principle long before she is prepared to learn the formal description of it. Without trivializing them, complex ideas can be introduced by making them concrete and through reference to children’s experience." Whilst I agree to a certain extent with this, I still feel there are certain topics that should be left until students are older and more mathematically experience. Take trigonometry, for example. I would make it illegal to study trigonometry before Year 10. But this is not just because I feel younger students are not cognitively developed enough to cope with the demands of that content, it is also because I feel it is important to leave some surprises, some topics that have not yet been covered, for students' final years of high school mathematics.
All of this does not make differentiation any easier, but it does suggest to me that giving all students the opportunity to achieve is better than putting them into categories before the lesson. If misconceptions do become apparent in the course of teaching (exposed via the techniques of Formative Assessment), then of course differentiation will be needed. But the point here is that - perhaps unfortunately - we may only be able to deduce this within the lesson itself.
My favourite quote:
If a child, or even the whole class, does not understand something, you should not assume that the task you posed was not developmentally appropriate. Maybe the students are missing the necessary background knowledge. Or maybe a different presentation of the same material would make it easier to understand.

Research Paper Title: How Do Teachers' Expectations Affect Student Learning
Author(s): D. Stipek
My Takeaway:
One aspect of differentiation is planning different materials to give to students, and the potential problems with this have been outlined in the paper above. However, perhaps a more common form of differentiation is the help we give to different students during lessons. In particclar, I have been inclined to offer more help to students who I perceived as lower ability, or who I thought might struggle with a particular concept. This fascinating paper offers up a stark warning against that. It surveys research into the ways in which teachers' beliefs about students affect their behavior toward students. One thing that struck me more than anything way this: by simply offering up help, we may be communicating to the student in question that we have low expectations of them. That makes perfect sense. Many times I will set my class off on some work and immediately ask a student who is prone to struggling something like "do you need any help?", or "give us a shout if you get stuck on anything". I will do this quietly so other students do not hear, and I am genuinely doing it with the best of intentions, but what message is it sending to that student? As we have seen in the Praise and Motivation section, if students do not believe themselves that they can achieve, motivation and subsequent achievement will likely diminish, and we as teachers are key to helping them believe in themselves. My major takeaway from reading this is often the best intentions of teachers can lead to students believing we have low-exceptions of them. In future, I will think more carefully before I act and, as difficult as it may be for all parties involved, let all students struggle for a while.
My favourite quote:
Helping behavior can also give students a message that they are perceived as low in ability, and it can undermine the positive achievement-related emotions associated with success. Meyer (1982) describes a study by Conty in which the experimenter offered unrequested help either to the subject or to another individual in the room working on the same task. Subjects who were offered help claimed to feel negative emotions (incompetence, anger, worry, disappointment, distress, anxiety) more, and positive emotions (confidence, joy, pride, superiority, satisfaction) less than subjects who observed another person being helped. Graham and Barker (1990) report that children as young as six years rated a student they observed being offered help as lower in ability than another student who was not offered help.

Group Work and Cooperative Learningkeyboard_arrow_up
Back to Top

Group work has always been a thorny issue for me - probably not helps by a previous school's obsession with Kagan training. Whilst I see the benefits of having students work together, talk about mathematics, and learn from each other, I am also acutely aware of the possibilities of students being off-task, and relying on others in the group to do the work for them. I guess it comes down to one question: do students learn more being in a group, or working individually? And of course, cooperative learning extends to those occasions where you might say "discuss this with your partner for 10 seconds". Here is my attempt to use educational research to find out the best practices for group work and cooperative learning.

Research Paper Title:
Cooperative Learning and Achievement: Theory and Research
Author(s): Robert E. Slavin, Eric A. Hurley and Anne Chmberlain
My Takeaway:
This is a fascinating review on the research into cooperative learning, which comes to a positive conclusion about its  potential effect on achievement. Alongside reviewing contrasting theories of cooperative learning, the paper identifies two key facts that are necessary for successful cooperative learning. The first is the importance of structuring group interactions. The concept of reciprocal teaching is discussed, with emphasis placed on the importance of the teacher modeling these interactions. Not assuming students instinctively know how to effectively work together will be a key theme of this section. Secondly, there is the issue of group goals and individual accountability. The key finding is this: cooperative learning is most consistently effective when groups are recognised or rewarded based on individual learning of their members. Without this, there is the danger that members of the group will free-ride on others' work. This is obviously tricky to do, and a suggestion from the authors is found in "my favourite quote" below. In short, groups work best when there are:
1) group goals (so pupils are working as a group and not just in a group)
2) individual accountability (so any pupil falling down on the job harms the entire group's work).
One other part of this paper I found fascinating was the section on which ability of student benefits most from cooperative learning. You could argue that high achievers could be held back by having to explain material to their low-achieving group mates. However, it would be equally possible to argue that because students who give elaborated explanations typically learn more than do those who receive them, high achievers should be the students who benefit most from cooperative learning because they most frequently give elaborated explanations. Interestingly, research has failed to provide a conclusive answer to this, and the authors remain in support of cooperative learning for all.
My favourite quote:
If students can only do as well as the group and the group can succeed only by ensuring that all group members have learned the material, then group members will be motivated to teach each other. Studies of behaviors within groups that relate most to achievement gains consistently show that students who give each other explanations (and less consistently, those who receive such explanations) are the students who learn the most in cooperative learning. Giving or receiving answers without explanation has generally been found to reduce achievement (Webb, 1989, 1992). At least in theory, group goals and individual accountability should motivate students to engage in the behaviors that increase achievement and avoid those that reduce it. If a group member wants her group to be successful, she must teach her group mates (and learn the material herself). If she simply tells her group mates the answers, they will fail the quiz that they must take individually. If she ignores a group mate who does not understand the material, the group mate will fail, and the group will fail as well.

Research Paper Title: Group Work for the Good
Author(s): Tom Bennett
My Takeaway:
Tom Bennett reviews the research in favour of group work and teaches a different conclusion. He argues that much of the research derives from the field that can be broadly termed Constructivism - the idea that students are active participants in the process of learning, and not passive recipients of experience and factual knowledge. This area has been covered in detail in the Explicit Instruction section of this page. Bennett argues that once you question the validity of constructivism, then the arguments in favour of group work start to lose their power. Bennett goes on to outline what he sees as four main drawbacks of group work:
1) Disguised inactivity - if you give a task to three or four people, one or two may realise it's time to freeze, because others will carry the burden of the task, and in the meantime, they can coast under the guise of "research" or "running the group."
2) Unequal loading - Related to this is the problem that while ever student might participate, the participation might be profoundly uneven.
3) Inappropriate Socialisation - students may end up competing to see who can discuss the task the least.
4) Unfair assessment - When a teachers praises a pupil, it's a clear one-to-one relationship, whereas in grading groups, we often must give collective grades.
For me, the key point that the author raises is the opportunity cost of group work - what else could the students be doing with their time? If Explicit Instruction could achieve the same result in 5 minutes that a group could achieve in 30 minutes, then how can we justify it? But if the activity (and the class) lends itself well to a group work activity, then the benefits of having students share ideas, learn from each other, and even something as simple as to vary the type of classroom activity to reengage students, may well be worth it.
My favourite quote:
Here's my parting advice: use group work when you feel it is appropriate to the task you want your students to achieve, and at no other time. The irony of the advocates' position is that while it correctly identifies the many benefits to using group work, their error is made when group work is preferred over other strategies because of some imagined potency, or when it is fetishized as a method imbued with miraculous properties. It isn't dogma, it isn't a panacea, and it isn't the messiah. It's one strategy among many. And it's a perfectly reasonable part of a teacher's arsenal of strategies. Not because pseudo-research has settled the matter, but because the teacher feels it appropriate at that time, for that lesson, with those children. And not before.

Research Paper Title: Group work and whole-class teaching with 11- to 14-year-olds compared
Author(s): Maurice Galtona, Linda Hargreaves and Tony Pell
My Takeaway:
This findings of this paper provide somewhat of a contrast to the one above. Here researchers compared the academic performance and classroom behaviour of pupils when taught new concepts or engaged in problem solving in sessions organised either as cooperative group work or whole class, teacher directed instruction. Comparisons of attainment were made in classes of pupils aged 11 to 14 years (Key Stage 3) in English, mathematics and science. The attainment results suggest that a grouping approach is as effective, and in some cases more effective, than when whole class teaching is used. Classroom observation indicated that there were more sustained, higher cognitive level interactions when pupils worked in groups than during whole class discussions. Crucially, the researchers argue that the group work results could be improved still further if teachers gave more attention to training pupils to work in groups and if more time was given to debriefing after group work. For me these last two points are crucial. We cannot assume students intrinsically know how to work in groups, and without training then the drawbacks that Bennett identifies in the paper above may well be realised. This possibly even extends to students talking in pairs. The next paper in this section provides a suggestion for a simple form of training. Secondly, a debrief may go some way towards providing accountability to group members. By discussing what went well and what didn't throughout the group work, and drawing attention explicitly to groups that worked well, the teacher can emphasise the point that group work is to be taken seriously, and offer directions for students to improve.
My favourite quote:
There are a number of reasons for claiming that the group work could have been more effective, to do mainly with the context in which teachers had to operate when taking part in this study. First, as part of training pupils to work effectively in groups it is vital that teachers brief and debrief the class so that they can begin to gain metacognitive awareness of what it means to be part of a group. Debriefing sessions therefore are particularly important because they not only evaluate how individuals responded in the groups but they also call for participants to make suggestions about suitable strategies for improving the situation on future occasions. After each session, observers completed a lesson overview schedule which recorded, amongst other things, whether or not briefing or debriefing had taken place. It was noticeable, however, particularly in science, that teachers rarely found time for these debriefing sessions. It was rare, for example, to observe a science lesson where the teacher with, say, five minutes of the period left preferred to keep discussion of the results over until the next lesson and instead engaged in a debriefing exercise. More often teachers preferred to use an evaluation sheet which they handed to pupils as they left the class. Thus the exercise tended to take the form of an additional homework task rather than generate a debate on the consequences of the previous classroom activity.

Research Paper Title: Teaching children how to use language to solve maths problems
Author(s): Neil Mercer and Claire Sams
My Takeaway:
This paper cites observational research which suggests that primary school children often do not work productively in group-based classroom activities, with the implication that they lack the necessary skills to manage their joint activity. To counteract this, the authors explored the role of the teacher in guiding the development of children’s skills in using language as a tool for reasoning. It involved an interventional teaching programme called Thinking Together, designed to enable children to talk and reason together effectively. The results obtained indicate that children can be enabled to use talk more effectively as a tool for reasoning; and that talk-based group activities can help the development of individuals’ mathematical reasoning, understanding and problem-solving. The important part of this study for me is that the students were explicitly shown how to effectively work in groups, with special emphasis on the language they should use, instead of assuming that they would automatically know how to do so. The lessons preceding the group work were explicitly focussed on making students aware of the need and benefit to work together. They were encouraged to discuss things and ask questions, include everyone’s ideas , ask what people think and what their reasons are , listen to each other, and so on, all of which was supported by clear modelling from the teacher. If we are going to embark upon group work (or even paired work), then such a structured approach before the task itself may well be a sensible step. However, one be aware of the relatively small sample size used in the study before drawing any significant conclusions.
My favourite quote:
More generally, our results enhance the validity of a sociocultural theory of education by providing empirical support for the Vygotskian claim that language-based, social interaction (intermental activity) has a developmental influence on individual thinking (intramental activity). More precisely, we have shown how the quality of dialogue between teachers and learners, and amongst learners, is of crucial importance if it is to have a significant influence on learning and educational attainment.

Mindsetkeyboard_arrow_up
Back to Top

Carol Dweck's work on mindsets has proven both popular and influential across schools over the last decade. The general thrust of her work is students with a growth mindset believe that their intelligence is not fixed, embrace mistakes as learning opportunities, and strive to improve through practice. This can be most readily seen in her book, Mindset. Whilst I see the advantage of having a growth mindset, I am yet to be fully convinced of how to practically develop such a way of thinking in my students. I share some more thoughts in my Takeaways below. 

Research Paper Title:
Boosting Achievement with Messages that Motivate
Author(s): Carol S. Dweck
My Takeaway:
A great introduction to Dweck's work on Mindset. She outlines her belief that students with a fixed mindset follow the cardinal rule "look smart at all costs", which leads to their desire to not work hard, not make mistakes, and if you do make mistakes to not try to repair them. In contrast, those with a growth mindset follow the rule "it is much more important to learn than get good grades", which leads them to take on challenges, work hard, and correct any mistakes. Studies quoted show that teaching students a growth mindset results in increased motivation, better grades, and higher achievement test scores
My favourite quote:
Many teachers see evidence for a fixed mindset every year. The students who start out at the top of their class end up at the top, and the students who start out at the bottom end up there. Research by Falko Rheinberg shows that when teachers believe in fixed intelligence, this is exactly what happens. It is a self-fulfilling prophecy. However, when teachers hold a growth mindset, many students who start out lower in the class blossom during the year and join the higher achievers.

Research Paper Title: Is it true that some people just can't do math?
Author(s): Daniel J Willingham
My Takeaway:
This is a really sensible discussion, backed by research, into the effect of student's ability and mindset when it comes to high school mathematics. Willingham argues that whilst it is true that some people are better at maths than  others - just like some are better than others at writing or building cabinets or anything else it - is also true that the vast majority of people are fully capable of learning the levels of mathematics they need for high school. We have seen in the Cognitive Science section that learning mathematics is Biologically Secondary knowledge, and so does not come as naturally as learning to speak, but our brains do have the necessary equipment. So, learning maths is somewhat like learning to read: we can do it, but it takes time and effort, and requires mastering increasingly complex skills and content. I love this bit: "Just about everyone will get to the point where they can read a serious newspaper, and just about everyone will get to the point where they can do high school level algebra and geometry even if not everyone wants to reach the point of comprehending James  Joyce's Ulysses or solving partial differential equations." For Willingham, in order to be successful in maths, students need the following three types of knowledge:
1) Factual - this is knowledge of times tables and number bonds which must be automated as discussed in the Fluency section. As we have seen throughout the research studies on this page, without this knowledge students' working memories are likely to become overloaded when attempting to solve more complex problems.
2) Procedural - this is a sequence of steps by which a frequently encountered problem may be solved. This may involve using the gird method for long multiplication, or SOHCAHTOA for trigonometry questions.
3) Conceptual - this refers to an understanding of meaning; knowing that multiplying two negative numbers yields a positive result is not the same thing as understanding why it is true.
These latter two types of knowledge form the basis for the conflicts between progressives and traditionalist views of maths education. Conceptual knowledge is probably the most difficult type for students to acquire, but it is of fundamental importance as new mathematical concepts almost always build on previous ones (for example, a clear understanding of the equals sign is needed to solve linear equations). Without this conceptual knowledge, students are forced to rely on algorithms to carry out procedures, but given the amount of procedures in mathematics, not to mention to difficulty of spotting which procedure is required, means this is simply unsustainable. What does all of this mean for maths teaching, and for the issue of mindset, I hear you ask? Well, Willingham has three pieces of advice for maths teachers, each of which is expanded upon in the paper:
1) Think carefully about how to cultivate conceptual knowledge, and find an analogy that can be used across topics
2) In cultivating greater conceptual knowledge, don't sacrifice procedural or factual knowledge
3) In teaching procedural and factual knowledge, ensure that students get to automaticity
4) Choose a curriculum that supports conceptual knowledge
5) Don't let it pass when a student says "I am no good at maths"
And it is the final point that brings us back to mindset. For Willingham, as the quote below illustrates, maths is something that can be learned through hard work and good teaching. As I said in my introduction, I firmly believe the concept of a growth mindset is important, but we need to help the students see that their achievement in maths can be changed for the better.
My favourite quote:
We hear it a lot, but it's very seldom true. It may be true that the  student finds math more difficult than other subjects, but with some persistence and hard work, the student can learn math and as he learns more, it will get easier. By attributing the difficulty to an unchanging quality within himself, the student is saying that he's powerless to succeed.

Research Paper Title:
Academic Tenacity: Mindsets and Skills that Promote Long-Term Learning
Author(s): Carol S. Dweck, Gregory M. Walton, Geoffrey L. Cohen
My Takeaway:
This particular paper presents some compelling findings and practical strategies to help foster a growth mindset and develop academic tenacity. Two findings that stood out to me were:
1) Students who received effort praise chose challenging tasks that could help them learn, while students who received intelligence praise were more likely to choose tasks in their comfort zone that they could perform well on;
2) Classrooms that encourage competition and individualistic goals may be particularly ill suited to minority students, who are more likely to be reared in cultural contexts that emphasize the importance of communal and cooperative goals over individualistic or competitive goals.
My favourite quote:
Research shows that students’ belief in their ability to learn and perform well in school—their self-efficacy—can predict their level of academic performance above and beyond their measured level of ability and prior performance

Research Paper Title:
Does mindset affect children’s ability, school achievement, or response to challenge? Three failures to replicate.
Author(s): Yue Li & Timothy C. Bates
My Takeaway:
I include this paper for balance as it was the most empirically rigorous one I could find that questions Dweck's findings on mindset. Specifically, the authors find praise for intelligence failed to harm cognitive performance and children’s mindsets had no relationship to their IQ or school grades. Finally, believing ability to be malleable was not linked to improvement of grades across the year. Two points I would raise here: the authors do not find that promoting mindset beliefs to have a negative effect, and more importantly work by the likes of Dan Willingham and others discussed above strongly believe that effort and practice can significantly increase achievement. Hence, I personally have no problem promoting a growth mindset, both in students and teachers. However, a growth mindset without the success that will most likely follow from an Explicit Instruction approach may not be sustainable in the long run, in my opinion.
My favourite quote:
We find no support for the idea that fixed beliefs about basic ability are harmful, or that implicit theories of intelligence play any significant role in development of cognitive ability, response to challenge, or educational attainment.




Fluency with Maths Factskeyboard_arrow_up
Back to Top

Research from the Cognitive Science, Explicit Instruction and Cognitive Load Theory sections of this page stress the importance of students having key mathematical facts ready to be retrieved from long term memory so they do not take up valuable space in working memory. For want of a better expression, I will refer to this as developing fluency with maths facts. Over the last few years, I have tried to get students to develop this fluency via number talks, and more recently through drills/rote learning. This section is my attempt to find research-based evidence on the merits of these, as well as to address the issue of whether fluency is important when students have calculators and mobile phones!

For an overview of how Number Talks work, I would recommend this article, together with this video from Jo Boaler.

For an overview of drilling, I would recommend reading the following two blog posts by teachers at Michaela Community School: Dani Quinn and Hin-Tai Ting.

Research Paper Title: Automaticity in Computation and Student Success in Introductory Physical Science Courses
Author(s): JudithAnn R. Hartman, Eric A. Nelson
My Takeaway:
Students (and parents!) often say to me something along the lines of: "why do I need to work that out when I can just bang it into my calculator/smart phone?". To be honest, my answers over the years have not been great. Fortunately, with the findings of this fascinating paper, I now have some evidence up my sleeve. This paper looks at the effect the move away from the practice of key mathematical skills in US high schools has had on students taking science degrees. The authors find that between 1984 and 2011, the percentage of US bachelor’s degrees awarded in physics declined by 25%, in chemistry declined by 33%, and overall in physical sciences and engineering fell 40%. Data suggest that these declines are correlated to a K-12 (kindergarten to the end of high school) de-emphasis in most states of practicing computation skills in mathematics. The authors cite recent studies in cognitive science that have found that to solve well-structured problems in the sciences, students must first memorize fundamental facts and procedures in mathematics and science until they can be recalled “with automaticity,” then practice applying those skills in a variety of distinctive contexts. Even with access to a calculator, students working memories can become overloaded, which can prevent them being able to solve more complex problem and hence inhibit learning. To explain this further, I can do no better than to quote from the paper itself:
My favourite quote:
As one example, if as part of a calculation “8 times 7” cannot be recalled, the calculator answer of 56 must be stored in working memory so that it can be transferred to where the calculation is being written. On a problem of any complexity, that storage may bump out of working memory an element that is needed to solve the problem. An answer from a calculator takes up limited working memory space; an answer recalled from long term memory does not.

If arithmetic and algebraic fundamentals are automated, when examples are based on simple ratios or equations, room is available in novel WM for the context that builds conceptual understanding, and problem solving builds an intuitive, fluent understanding of when to apply facts and procedures (Willingham 2006). Conversely, if a student lacks “mental math” automaticity, conceptual explanations based on proportional reasoning or “simple whole-number-mole ratios” will likely not be simple. If a student must slowly reason their way through steps of algebra that could be performed quickly if automated, the “30 seconds or less” limit on holding the goal, steps, and data elements of the problem in working memory ticks away.

In recent years, the internet has facilitated the finding of facts and procedures, but new information occupies the limited space in novel WM that is needed to process the unique elements of a problem. Unless new information is moved into long term memory by repeated practice at recall, during future problem solving that new information will again need to be sought, and when found, it will again restrict cognitive processing (Willingham 2004, 2006).


Research Paper Title:
Fluency without Fear
Author(s): Jo Bolaer
My Takeaway:
Jo Boaler agrees on the importance of knowing mathematical facts, but argues that the memorisation of math facts through times table repetition, practice and timed testing is unnecessary and damaging. Boaler goes further to argue that when teachers emphasise the memorisation of facts, and give tests to measure number facts, students suffer in two important ways. Firstly, for about one third of students the onset of timed testing is the beginning of math anxiety, which can block working memory and prevent learning taking place. Secondly, they can put students off mathematics for life. Boaler advocates a move away from seeped and memorisation, and towards encouraging students to work with, explore and discuss numbers. This will allow them to commit important facts to memory, but in a fun and engaging context. She then goers on to describe Number Talks, as well as some other strategies for developing a sense of number. My concern with this is founded in personal experience of using and observing Number Talks extensively over the last three years - whilst it is supposed to be the lowest achieving students who gain the most from them, I have found they are often held back from fully participating by their lack of knowledge of key maths facts. It ends up being the teacher suggesting strategies, students copying them down and seemingly understanding then, only for them to be unable to transfer them to a new calculation in the next Number Talk. Contrast this to higher achieving students, who can happily break apart and put back together numbers in wonderfully efficient ways, and really seem to get a lot out of the Number Talks, sharing and discussing each others' approaches. I believe all students can (and should) get to this level, but I don't think this can be done through Number Talks alone. Students cannot develop fluency with numbers without these facts in long term memory. I have found that the more comfortable students get with their times tables and number bonds, the more readily they can think of, and successfully carry out the kind of efficient strategies that Number Talks promote. I draw the analogy with problem solving - you cannot teach problem solving by just showing students how to solve problems. Likewise, I believe you cannot teach fluency with numbers simply by showing students how to be fluent.
My favourite quote:
High achieving students use number sense and it is critical that lower achieving students, instead of working on drill and memorization, also learn to use numbers flexibly and conceptually. Memorization and timed testing stand in the way of number sense, giving students the impression that sense making is not important. We need to urgently reorient our teaching of early number and number sense in our mathematics teaching in the UK and the US. If we do not, then failure and drop out rates - already at record highs in both countries - will escalate.

Research Paper Title: Developing Fluency with Basic Number Facts: Intervention for Students with Learning Disabilities
Author(s): Katherine Garnett
My Takeaway:
This paper provides a fascinating insight into attempts to develop fluency with students with learning difficulties. The authors cite that on timed assessments, 5th grade students with learning disabilities completed only one-third as many multiplication fact problems as their non disabled counterparts. Interestingly, the students with learning disabilities were very much slower, but not significantly less accurate, than their non-disabled peers. Additionally, they demonstrated basic conceptual understanding of the basic maths operations. Thus, many students with learning disabilities establish basic understanding of the number relations involved in basic facts, but continue using circuitous strategies long after their non-disabled peers have developed fluent performance. And as we have seen from Cognitive Load Theory, it is the inability to recall facts from long term memory that will hinder such students in solving more complex problems as their working memories will become overloaded. However, the author points out that becoming fluent is not simply a case of remembering a load of facts - it is about forming a well-developed network of number relationships, easily activated counting and linking strategies, and well-practiced navigational rules for when to apply which maneuver". The author goes on to argue that the only way to develop these skills is though several years of frequent and varied number experiences and practice, and drilling is not enough. The author recommends presenting Challenge Problems and discussing strategies, much in the same way as Number Talks, and a really useful collection of prompt questions is provided. I am still not convinced that strategy comes before knowledge, but I am convinced that simply knowing facts is not enough given the immeasurable number of combinations of facts that would be required to answer every single maths problem!
My favourite quote:
In investigating the effects of challenge problems, many of the guidelines offered here would be useful, especially the emphasis on interactive, oral work. Regularly including challenge problems in student/teacher interactive math work could well promote the "mental math" prowess needed by so many students with learning disabilities who cling to number lines and paper-pencil routines.

Research Paper Title: Mastering Maths Facts: Research and Results
Author(s): Otter Creek Institute
My Takeaway:
This authors provide a really good summary of the key findings from this paper: "Learning math facts proceeds through three stages: I) procedural knowledge of  figuring out facts; 2) strategies for remembering facts based on relationships; 3) automaticity in maths facts—declarative knowledge. Students achieve automaticity with math facts when they can directly retrieve the correct answer, without any intervening thought process. The development of automaticity is critical so students can concentrate on higher order thinking in maths. Students who are automatic with math facts answer in less than one second, or write between 40 to 60 answers per minute, if they can write that quickly. Research shows that math facts practice that effectively moves students towards automaticity proceeds with small sets of no more than 2 —4 facts at a time. During practice, the answers must be remembered rather than derived. Practice must limit response times and give correct answers immediately if response time is slow. Automaticity must be achieved with each small set of facts, and maintained with the facts previously mastered, before more facts are introduced. Suggestions for doing this with flashcards or with worksheets are offered." So, this paper stresses the importance of the automaticity of maths facts, and advocates the use of timed drills to achieve this. I found the suggested order to be interesting: strategies comes before automaticity. This did not seem to fit in with my experiences described in my Takeaway on the Boaler article above. However, digging a little deeper, the authors suggest that strategy comes first for addition and subtraction facts, but memorisation is needed for multiplication facts. As most Number Talks require an element of times table knowledge, this certainly fits in with my experiences. The paper also makes one point that I feel is of paramount importance: if students are relying on a counting strategy to solve basic maths facts (such as counting on fingers for multiplication), then no amount of drilling will help them transfer these facts to long term memory. For students relying on these strategies, drills can become a painful process. Students need to be moved away from inefficient strategies as soon as possible, and the authors suggest that timed drills might be a way to achieve this. Of course, we need to bear in mind Boaler's important point about the dangers of maths anxiety - but there are plenty of ways to make this kind of drilling fun and non-threatening, as the Michaela blogs demonstrate.
My favourite quote:
What is required for students to develop automaticity is a particular kind of practice focused on small sets of facts, practiced under limited response times, where the focus is on remembering the answer quickly rather than figuring it out. The introduction of additional new facts should be withheld until students can demonstrate automaticity with all previously introduced facts. Under these circumstances students are successful and enjoy graphing their progress on regular timed tests. Using an efficient method for bringing math facts to automaticity has the added value of freeing up more class time to spend in higher level mathematical thinking.

Research Paper Title: Mental calculation methods used by 11-year-olds in different attainment bands
Author(s): Derek Foxman
My Takeaway:
This is fascinating. A sample of 247 eleven year old children was divided into three bands of attainment as measured independently by their scores on a written test of concepts and skills.They were then given a series of mental arithmetic questions to answer during one-ton-one interviews, and crucially asked to explain how they arrived at their answer. An example of one of the questions children were given is "I buy fish and chips for £1.46. How much change should I get from £5 ?". Their responses were classified as either being Complete (e.g. £5 − £1 − 46p), or Split (e.g. £5 − £1; £1 − 46p). There were three main findings:
1) Complete number methods were far more successful than Split number methods, even more so in the two lower attainment bands than in the top band.
2) For all three questions, Complete number strategy use declined from the Top to the Bottom attainment band, while Split number strategy use increased from Top to Bottom.
3) Complete number strategies were used far more frequently than either Split methods or the Algorithm for working out the in context questions.
I was surprised by these results - for me the Split strategy seems more efficient, and is how I approach the problems. But, then again, I can see how it is prone to error. Relating this to Cognitive Load Theory - if a students does not have facts and processes stored in long term memory, then think of the cognitive demands placed on working memory when trying to process a problem, split it up, work out the individual components, and then put it back together again. Without such facts and procedures in long term memory, the complete strategy was always going to be the most successful.
My favourite quote:
The main significance of these findings is that the two mental computation strategies represent different attitudes towards numbers. The Split strategies suggests that numbers up to 100 are viewed as consisting of tens and units and children using them attempt to deal with these values separately. Such strategies can frequently lead to the sort of errors that occur when using the written standard algorithm. By contrast, Complete number strategies treat numbers as wholes. Furthermore, the calculation steps are sequential so that subtotals are operated on as they occur and do not have to be stored separately in memory.

Research Paper Title: Developing Automaticity in Multiplication Facts: Integrating Strategy Instruction with Timed Practice Drills
Author(s): John Woodward
My Takeaway:
This study seeks to establish whether it is better to use a strategy of timed drills to teach multiplication facts, or a combination of timed drills with activities that promote the use of strategies to teach these facts. The strategies involved were those similar found in Number Talks, for example the multiplication fact 6 x 7 was shown, through discussion, to be equivalent to 6 x 6 + 6. Groups of students were taught using either Drills or and Integrated approach for 4 weeks. They were then given three types of test: Computation, Extended Facts and Approximations, followed by an Attitude Towards Maths survey. Both groups performed equally well on Computations, with the Integrated group performing better on the Extended Facts and Approximations tests. Both groups reported the same level of happiness in the attitudes survey, which may surprise those who fear the Drill and Kill strategy. My only reservation about recommending an integrated approach based upon these findings is that we do not have longitudinal data to see the levels of retention. The students in the Drill approach got through far more computations, and hence there is a chance that their degree of retention will be higher than the Integrated group.This study supports my view that an integrated approach can work - but the key knowledge and facts must be in students' long term memories before attempting to develop and discuss these strategies with numbers, and drilling may be the best way to achieve that. Then, hopefully, you can end up with the best of both worlds.
My favourite quote:
Results from this study indicate that an integrated approach and timed practice drills are comparable in their effectiveness at helping students move toward automaticity in basic facts. If educators were only considering facts as a foundation for traditional algorithm proficiency, either method would probably suffice. Yet, the educationally significant differences between groups found on the extended facts and approximations tests should encourage special educators to consider how strategy instruction can benefit students’ development of number sense.


Real Life Mathskeyboard_arrow_up
Back to Top

"Sir, when will we ever use this in real life?", is a question I never tire of hearing... And for years, I have not had a good reply. My response has often been to try to shoehorn a supposed "real-life" context into my lesson, and believe me "shoehorn" is the correct way of describing it. Did David Beckham really consider the properties of quadratic graphs when lining up a free kick? Advocates of real life problems argue that they have a motivating factor, but as we will see in some of the papers that follow, this must be weighed up against the potential for misinterpretation. More recently, I have given up. I simply tell the students we are doing mathematics for the love, challenge and beauty of doing mathematics, and everyone has gotten on a whole lot better ever since. So, it was with some interest that I sought to see what educational research had to say about the use of real life contexts in mathematics.

Research Paper Title: Relevance as MacGuffin in Mathematics Education
Author(s): Dylan Wiliam
My Takeaway:
In this brilliant paper, Dylan Wiliam looks at the use of "real life" maths problems in the classroom. Wiliam shows examples of where the context adds nothing at all to the question: " Alan drank 5/8 , of his pint of beer. What fraction was left?", as well as examples where the context can actually lead students to get the "wrong" answer: "a disc jockey plays two records. The first lasts 2 5/8 minutes and the second lasts 3 1/4 minutes. Find the total time taken to play both records" - the point being that DJs are likely to talk or play jingles between songs, and hence students may (sensibly) choose to account for this in their answer. There is a fascinating insight into how, when presented with a real-life problem, many girls seek to relate the problem to their existing knowledge, supplying any information that they feel is missing from their own experience, while boys are often content to tackle the problem in isolation from their previous experience. Finally, Wiliam suggests three criteria that should be applied to ensure that the contexts used in problems are useful:
1) Commonality - metaphors and contexts can be really useful to aid understanding, but they must be commonly shared by all students, otherwise they could lead to confusion
2) Match - does the task (and possible interpretations of the task) match the core mathematical activities you want to convey? Will students need to suppress/ignore attributes to engage in the mathematical activity you intended?
3) Range - how far does the model take you along your journey to understanding a topic? Wiliam cites the example of negative numbers: "Envisaging negative numbers as temperatures above or below zero may be very useful for understanding ordering of integers, and may even provide good pictures for adding directed number. However, contriving plausible scenarios where temperatures below zero are subtracted from each other, or are multiplied is rather more difficult!"
My main takeaway from this was to think really carefully about what the context is adding to the question. If the effect is neutral (as in the pint of beer question), then why bother with it - indeed we have learned from the Redundancy Effect in Cognitive Load Theory that pointless information imposes an unnecessary load onto fragile working memory? If it is negative (such as with the DJ, or if it is likely to disadvantage some students), then remove it. But if a metaphor or a framing can be used to impose extra clarity on the problem, without altering its underlying structure, then this would be valid. I will certainly be checking my examples more carefully in the future.
My favourite quote:
In the vast majority of classrooms, relevance is a MacGuffin—a device to motivate learners; to convince them that the activities they are given are somehow of the real world, even though they do not appear to be connected to it. Students know this, and as a result their thinking in mathematics lessons becomes divorced even from their thinking elsewhere in school, let alone the world outside school. Mathematics lessons thus become literally mindless—an activity in which students come to believe that thinking is not helpful.

Research Paper Title: The effect of using real world contexts in post-16 mathematics questions
Author(s): Chris Little and Keith Jones
My Takeaway:
Before we get into the study itself, the authors have a really nice way of summing up the dilemma of using real world contexts: "On the one hand, by making a connection between the abstract world of mathematics and  everyday, or scientific, contexts, we are reinforcing the utility of mathematics as a  language for explaining the patterns and symmetries of the ‘real’ world. On the other  hand, if we manipulate and ‘sanitise’ real-world experiences to enable them to be  modeled by a pre-ordained set of mathematical techniques, then the result can appear  to be artificial and contrived". I like that! Anyway, in this interesting study, alternative versions of the same questions (all on sequences) were presented in explicit, algebraic, word and pattern contexts, and set to  a sample of 594 Year 13 students (aged 17-18) in a one-hour test. The results showed that that setting sequence questions in real-world contexts does indeed add to the overall demand, though a context can on occasions provide ‘mental scaffolding’ to help the solver to use context-specific heuristic strategies. My key takeaway is that we need to think carefully what our aim is. If it is to ensure our students are comfortable with the basics, then there is little point setting questions in a real-world context. Interpreting the question and deciding what strategy to use all impose additional cognitive load to the student's working memory, and as we have seen in the sections on Cognitive Load Theory and Cognitive Science, that may mean that no learning actually takes place. However, the researchers did find that some real-world contexts provided additional support for students in guiding them towards the correct method, so it may not be as clear cut as we would like. Of course, if your aim is assess if students can apply their knowledge and skills to different contexts, then using real-world questions may be a way to do that. I guess we just need to be aware how we interpret students' results in that case - do they struggle because the fundamentals are not in place, because they do not understand what the question is asking, or because they have been misled by the context itself?
My favourite quote:
The potency of algebraic formulae lies in their universality and blindness to individual contexts, and, in resorting to context-specific thinking to solve these questions, students are avoiding the necessity to transfer and abstract from context to mathematical model, which is, arguably, the heuristic strategy intended by the questions.

Research Paper Title: Do realistic contexts and graphical representations always have a beneficial impact on students’ performance?
Author(s): D. De Bock, L. Verschaffel, D. Janssens, W. Van Dooren, K. Claes
My Takeaway:
This is an amazing study which looks at the effect of using video to aid mathematical explanation. One hundred and fifty-two eighth graders (13–14-year olds) and 161 tenth graders (15– 16-year olds) participated. They were give a paper-and-pencil test about the relationships among the lengths, areas and volumes of different types of rectilinear and non-rectilinear figures. The problems were administered in or out of an authentic context and setting, and either with or without an integrated drawing instruction, leading to four combinations of groupings. The "authentic context" involved students watching clips of a screen version of ‘Gulliver’s Travels’, a world in which all lengths are 12 times as small as in our (and Gulliver’s) world. In the drawing instruction groups, students were provided with a drawing of the (geometrical) object introduced in the problem, and they were asked to complete the drawing by making a reduced copy using the given scale factor. How did the groups perform on the test? Students who watched Gulliver’s Travels and who received the video-related items performed significantly worse than the students from the other groups. Surprisingly, students who had to make a drawing performed significantly worse than students from the non-drawing groups, although the authors admit this is probably due to the nature of the linear scale factor task itself. To explain the video results, one possibility suggested really resonated with me: students perceive video as a less difficult medium than written materials, and therefore are inclined to invest less mental effort in working with information transmitted by this easy medium as compared to media that are perceived as difficult. As we have seen in the Cognitive Science and Memory sections, students remember what they think about, and sometimes thinking needs to be difficult. When watching a video (or indeed any other form of novel media we might use in a lesson, such as a website, app, or music) are students thinking about the task itself or the form of media? Any increase in engagement and motivation must be weighed up against this.
My favourite quote:
Some of the qualitative findings indicate that the reason why these two new forms of help did not yield the expected positive effect, was that they conflicted with students’ implicit norms, expectations and beliefs about doing mathematics, especially about their appreciation of formal and informal strategies and of drawings as a valuable modelling tool. From that point of view, classroom interventions that are only partial and instantaneous, and that are unable to influence or alter these more fundamental attitudes and beliefs, have little chance of success. Most likely, only a long-term classroom intervention, not only acting upon students’ deep conceptual understanding of proportional reasoning in a modelling context, but also taking into account the social, cultural and emotional context for learning, can produce a positive effect in defeating the illusion of linearity.

Research Paper Title: The role of context in linear equation questions: utility or futility?
Author(s): Chris Little
My Takeaway:
Four linear equations questions were presented, each set in a context. Two of these questions were : 1) In 18 years time, Halley will be five times as old as he was 2 years ago. How old is he now?  4) The largest angle of a triangle is six times as big as the smallest. The third angle is 75°. Find the size of the three angles. The author argues that these questions may be regarded as ‘applications’ of linear equations, in the sense that they involve formulating and then solving them. However, they have no practical utility value - none of the results provide significant information about the context. So, what is the point in the contexts? They are certainly not motivating. But they are unlikely to introduce any of the misinterpretations highlighted in previous papers in this section. The author makes a really interesting point: whilst the underlying algebraic structure of all four problems is the same, the contexts may actually encourage students to try different methods to solve them - trial and improvement, for example. I have seen this a lot with my students - often the context prevents students from spotting the topic that the question intends to test, and as a result the students end up trying something different. My takeaway here is similar to the paper on post-16 students - think about what we want our students to achieve. Do we want them to practise solving equations? If so, then maybe it is best to cut out the context and just have the equations. If we want them to practice recognising when it might be appropriate to formulate a question in terms of a linear equation, then we have seen in the sections on Cognitive Science and Cognitive Load Theory that students will only be in a position to do that - and crucially, to learn from it - when the basic knowledge of how to solve linear equations is secure in long term memory. Otherwise, the context may inhibit their learning.
My favourite quote:
Some researchers into word problems (Greer, 1997) have advocated introducing more elements of realism into classroom tasks, for example by  adding irrelevant information, which then has to be discounted by the solver.  However, expecting students to engage in genuine mathematical modelling activity  before they serve an apprenticeship in formulating algebraic equations, and learning  abstract, analytical methods for solving them, is perhaps itself unrealistic. Many  students find the process of translating real-life numerical concepts into algebraic variables demanding enough, without being deflected by realistic ‘noise’.

Research Paper Title: Intellectual Need and Problem-Free Activity in the Mathematics Classroom
Author(s): Evan Fuller, Jeffrey M. Rabin, Guershon Harel
My Takeaway:
I discovered this interesting paper following my interview with Dan Meyer on my podcast. The authors argue that intellectual need is necessary for significant learning to occur. Among other concepts, it introduces the need for computation. This is defined as the need to find more efficient computational methods, such as one might need to extend computations to larger numbers in a reasonable “running time.” The key here is that in order for the need for computation to be met, students need to experience the “longcut” before they learn the shortcut, for only then will they appreciate the power of that shortcut. Otherwise the techniques we teach seem like just another trick in the endless series of tricks students call maths class. This forms the basis of Dan Meyer's "Headach and Aspirin" series, where he asks the question: If Math Is The Aspirin, Then How Do You Create The Headache?. Dan argues that real world applications of many maths skills we teach in school are a lie. So if our theory is “maths is interesting only it’s real world,” then we will struggle to find interest in many of the things we teach. Instead, we should ask ourselves, “Why did mathematicians think this skill was worth even a little bit of our time? If the ability to factorise quadratic expressions is aspirin for a mathematician, then how do we create the headache?”. This approach has been incredibly influential in my teaching, and has enabled me to offer approaches into topics that I would never have thought of. I would strongly recommend checking out Dan's factorising quadratic expressions blog post to see how the headache and aspirin approach works. I will never introduce factorising quadratics in the same way again.
My favourite quote:
The need for computation is not a student’s psychological motivation to solve drill exercises on algorithms, but her intellectual recognition that realistic and compelling problems require the development of efficient computational methods for their solution


Improving Teachingkeyboard_arrow_up
Back to Top

To conclude this page of research, I want to take a look at developing the skill of being a teacher. So far we have seen recommendations for improving the delivery of material to students, and plenty of ways to help students encode, store, retrieve and apply knowledge. But what about improving our own teaching, and that of the teachers we mentor and work with?
1) Improving our own teaching. I am always looking to improve as a teacher, but there is a problem (and it is the same problem that makes research carried out in classrooms so difficult). There are just so many factors at play. Say I decide to try something new with my Year 7s - maybe I have come up with a new way of introducing algebra. How do I know how successful it has been? Sure, I can compare their performance in an algebra test with last Year's class, but that was a different class, with different students, taught in different circumstances and (lest we forgot) that given the distinction between learning and performance identified in the Memory section, how can I even judge if they have really learned algebra any better? So, if it is difficult to evaluate our own teaching in an effort to improve, can others help us through observations? Having read research on the matter, I believe they can. But - and this is a big but - only if we move away from the high-stakes, once-a-year observation that seems to be a key part of the performance management cycle these days. I believe supportive observations, with clearly defined and measurable goals in mind, can help develop teachers of all experiences.
2) Helping novice teachers develop. I have been extremely fortunate to work with many inexperienced teachers over the years in a bid to help them improve and develop. It is only now, having read the research cited on this page about skill development and the difference between how experts and novices think, that I see what a complex task this is. Whilst it is clear that some people have a natural ability to teach - just as some students have a natural ability to do mathematics - good teachers can be made. I believe we need to follow the principles of Explicit Instruction, as opposed to simply hoping inexperienced teachers will magically discover their way to becoming experts, which implies that novice teachers may require more support than they currently receive
.
Research Paper Title: The Development of Expertise in Pedagogy
Author(s): David C. Berliner
My Takeaway:
This is a truly wonderful paper that has lots to say on the transition from novice to expert in any field, but with particular relevance to teaching. The authors identify key areas where novices and experts differ, providing examples and supporting research for each:
1) Interpreting classroom phenomena
2) Discerning the importance of events
3) Using routines
4) Predicting classroom phenomena
5) Judging typical and atypical events
6) Evaluating performance
A few things struck me about this. Firstly, how many of these areas can only be improved by experience? Almost all of them, I think. And how many could I describe in great detail? Not many at all. For example, if a student teacher watches me teach and afterwards asks me why I said what I said to a group of students, or didn't say something to another, or ignored a certain situation but acted on another, or how I developed my classroom routines, or knew to adjust my explanation, or deviate from my lesson plan, or pause before moving on, I will find it hard to explain. I cannot nail down the numerous visual and audio cues to cause me to act the way I do, it is just instinct and experience. Like any (so-called!) "expert", I am on autopilot dealing with many things in the classroom, which enables my limited working memory to focus on more complex pedagogical issues. So, how do we help novice teachers develop these skills? Well, the authors have a few recommendations, but two of which stood out to me more than the rest:
1) If expert teachers have so many of these crucial routines on autopilot, and because the very fact they have been automatised makes them hard to articulate to novices, is this an argument for mentoring of novice teachers to be done by "competent" teachers as opposed to "experts"? I guess this is a similar to the argument that great mathematicians sometimes do not make the best teachers - they simply cannot relate to the difficulties students have in grasping basic concepts. Is it better to have a mentor who can remember what it was like to be a novice - can relate to the struggles, can remember how they coped with them - instead of an expert who flies through lessons on autopilot and may struggle to give novice teachers to explicit guidance they need?
2) The novices' relative inexperience in a complex environment allows a good case to be made for the importance of teaching them standard lesson forms and scripts. Now, this is controversial! Two of my podcast guests, Dani Quinn and Greg Ashman have both advocated centrally planned lessons, with the justification that these are lessons that have been planned by experienced teachers and are known to work. Novice teachers may benefit from following these "scripts" in class, enabling them to focus on other areas of teaching (e.g. behaviour, questioning, interactions with students) as they develop. This may also free up some of those hours that are spent searching for PowerPoints and worksheets, and allow them instead to spend their time going over the lesson plan in detail, rehearsing questions and responses, and ensuring they are familiar with the mathematical content. A key point here is that this is not the same as simply giving a novice teacher a PowerPoint on, say, straight-line graphs that worked well for you. It is not just the resource that is important, but the delivery, pace, questions, possible misconceptions, and so on. Essentially this is the same argument for Explicit Instruction versus discovery learning for novice students - in early skill acquisition, novice teachers are likely to benefit from more guidance from experts rather than less.
My favourite quote:
I am suggesting that our extensive knowledge base about teaching and teachers be thought of as more or less appropriate to people in different stages of their development. I am also suggesting that pre-service education may not be the most appropriate place to teach some things, and therefore we have to extend our programs of teacher education for some time after our students have entered practice. I am suggesting as well that the forms of evaluation for experienced and beginning teachers may have to differ. And I am suggesting that experts, revered as they may be, may not always make the best teachers of novices. I am arguing that the development of competence out of ignorance and expertise out of competence may take a long time in a profession as complicated as teaching. We may be unable to shorten the trip very much because extensive experience is fundamental to development, but we certainly ought to help nurture those willing to undertake the journey by providing training and evaluation appropriate for their level of development.

Research Paper Title: Practice with Purpose: The Emerging Science of Teacher Expertise
Author(s): Deans for Impact
My Takeaway:
In the Explicit Instruction section we looked at the fascinating concept of deliberate practice and how it could be applied to student learning. Here we turn our attention to using Deliberate Practice to improve teaching. Five key principles are identified and discussed with respect to teaching:
1) Deliberate practice requires presenting challenges that push novices just beyond their current abilities
2) Deliberate practice requires setting goals that are well-defined, specific, and measurable
3) Deliberate practice requires a significant level of focus; the practice involves conscious effort on the part of the novice in order to improve.
4) Deliberate practice requires providing high-quality feedback to the novice and adjustment by the novice in response to that feedback.
5) Deliberate practice both produces and relies on mental models and mental representations to guide decisions.These models allow practitioners to self-monitor performance to improve their performance.
Each of these are worthy of discussion, but a couple in particular stood out to me. The setting of specific goals was one. I remember early on in my career when I was being observed, I would rarely have specific goals in mind. I would just want to teach a good lesson. I would have been much better focusing on specific aspects, such as trying to give only task-focused feedback, or reducing the number of controlling questions I asked. Where possible, these goals should be measurable, so you have something concrete and objective to reflect upon. The second point is directly related - feedback. We have seen how important feedback can be for student learning, and how it can have both a positive and negative impact. The same is true for teacher development. We need to know if what we are doing is successful, but that is incredibly difficult when (at best) all we can observe is the performance of our students and, as we have seen in the Memory section, performance is a poor indicator of learning. That is where a colleague, observing in a supporting manner, complete with a predefined set of measurable goals, can help. And this cannot be confined to a once-a-year experience. Regular feedback is the key to deliberate practice, making tweaks where needed and seeing the outcome. I'll be honest - in my opinion developing teaching does not lend itself as well to the principles of deliberate practice as, say, learning to add fractions, but there are certainly key elements that can be used to great effect.
My favourite quote:
The principles of deliberate practice have the promise to improve the quality of teacher education. There will inevitably be challenges with this work: for teacher-educators learning new techniques; for institutions that need to change incentive structures in order to encourage faculty to own collectively the success of every teacher-candidate; and for teacher-candidates and novice teachers who will be pushed beyond their comfort zones. This work will not be easy, but we believe that it is both possible and necessary if we are to advance the field of teacher preparation and prepare effective teachers to serve every student.

Research Paper Title: Does Teaching Experience Increase Teacher Effectiveness?
Author(s): Tara Kini and Anne Podolsky
My Takeaway:
There are four key findings from this paper, three of which once again extol the benefit of experience, and on which was a game-changer for me:
1) Teaching experience is associated with increased student achievement gains throughout a teacher’s career
2) As teachers gain experience, their students are more likely to do better on other measures of success beyond test scores, such as school attendance
3 More experienced teachers confer benefits to their colleagues and to the school as a whole, as well as to their own students. Again, experience counts. Although the authors do point out that some highly experienced teachers are not particularly effective or have retired on the job, and some novice teachers are dynamic and effective. This is related to the principles of deliberate practice discussed in the paper above - it can be quite easy to coast along in teaching, relying on your experience to get you through. Likewise, as teachers progress in their careers, they often have other responsibilities within schools that take their time away from both planning and teaching. It can be difficult to improve past a certain point, but if a teacher wants to then the principles of deliberate practice can help.
4) Teachers make greater gains in their effectiveness when they teach in a supportive and collegial working environment, or accumulate experience in the same grade level, subject, or district. This is the big one for me. I have always assumed it is good to teach as wide a range of abilities and ages of  students as possible - that way you build up your set of teaching tools that can be used in any situation. But findings detailed here suggest it is best to specialise - in other words focus in on a particular ability of student (eg top-set or bottom-set) or age-range (e.g. Year 11). The more I think about this, the more it makes sense. In every school I have worked in there have been specialists - the "GCSE C/D borderline specialist", the "top-set specialist", or the "Statistics specialist" at A Level. Indeed, if I am honest, I find it easier stimulating and extending top-sets than bottom sets, but each year I take a few bottom-sets as I assumed it was the best thing to do for my teaching as a whole. But maybe it is not. Maybe specialising in one specific area of maths teaching allows teachers to develop and refine skills tailor-made to getting the most out of those students. Of course, there are issues with this. How do you know what your area of expertise is if you haven't taught lots of different classes? Sometimes "competition" between teachers teaching parallel classes can be a good thing in terms of stimulating new ideas and giving each other an incentive to strive to improve. Then there are political concerns - who gets the top sets? But I do think there is an argument for specialising, and possibly a strong argument for specialising in your first year of teaching so you can hone your craft on a narrower range of classes and students.
My favourite quote:
The common refrain that teaching experience does not matter after the first few years in the classroom is no longer supported by the preponderance of the research. Based on an extensive research base, it is clear that teachers’ effectiveness rises sharply in the first few years of their careers, and this upward trajectory continues well into the second and often third decade of teaching. The overwhelming majority of the 30 studies reviewed here (93 percent)—and 100 percent of the 18 studies using the teacher fixed effects methods—reach this conclusion. The effects of teaching experience on student achievement are significant, and the compounded positive effect of having a series of accomplished, experienced teachers for several years in a row offers the opportunity to reduce or close the achievement gap for low-income students and students of color.123 Given this knowledge, policymakers should direct renewed attention to developing a teacher workforce composed of high-ability teachers who enjoy long careers in supportive and collegial schools.

Research Paper Title: What makes great teaching? Review of the underpinning research
Author(s): Robert Coe, Cesare Aloisi, Steve Higgins and Lee Elliot Major
My Takeaway:
This is the second appearance of this paper, having been previously discussed in the Explicit Instruction section. I reference it again here simply because one of the finding that good pedagogical content knowledge has a strong impact on student achievement. Knowledge of the mathematical content itself is directly under the control of a teacher. I remember when I first taught the Decision 1 A Level module for the first time - I literally had no idea what I was doing. I had neither studied nor seen any of the material before in my life. And so I did the only thing I could do - I studied and studied, reading the notes in the textbook, completing all the exercises, and doing all the past papers (incidentally, I found doing past papers without notes by far the most useful way to learn the material, possibly due to the Testing Effect identified in the Memory section). How did my teaching of the unit go? Pretty average, to be honest. Because one part of subject knowledge is not directly under a teacher's control - the pedagogical side. I did not really know how to communicate the ideas clearly in a way that students understood. Why was this? Well, I simply lacked the experience. All I had to go on to identify the potential stumbling blocks and misconceptions where my own experiences of the material, and these often proved different to those of my students. It was only having taught the module twice through that I began to get a sense of the pace I needed to deliver the material, the misconceptions I needed to anticipate and address, and the alternative explanations I needed to offer up. So, there was no way I could have taught that module without good knowledge of the mathematics, but that was not sufficient to teach it well. There is no fast-track to gaining the pedagogical experience needed, but I could have made life easier for myself by watching more experienced teachers in action.
My favourite quote:
The most effective teachers have deep knowledge of the subjects they teach, and when teachers’ knowledge falls below a certain level it is a significant impediment to students’ learning. As well as a strong understanding of the material being taught, teachers must also understand the ways students think about the content, be able to evaluate the thinking behind students’ own methods, and identify students’ common misconceptions.