arrow_back Back to Teachers

Recommended Educational Research Papers for Teachers to Read

My interest in educational research only really began when I spoke to the likes of Dylan Wiliam, Daisy Christodoulou, Will Emney, Mark McCourt, Bruno Reddy, Greg Ashman and Tom Bennett for my Mr Barton Maths Podcast. Up until that point I guess had had been blindly going about my business, teaching based largely on gut instinct and the snippets of advice that had been popular throughout the first few years of my teaching career (learning styles, to name but one!). I had never really questioned whether what I was doing was the best for my students... until now (notice the pause for dramatic effect).

This page contains the academic research I have read to date that has influenced the way I plan and teach mathematics. It is meant as a complement to my Recommended Books for Teachers to Read page. It will be continually updated. I have divided it up into broad categories, but these are somewhat arbitrary due to the many interconnections between the papers. The recurring theme throughout everything reviewed on this page is trying to improve students' learning (defined as "a change in long-term memory" by Kirschner, Sweller and Clarke) using evidence based research. As well as linking to the original paper, I have also tried to summarise my main takeaway from it, explaining the practical ways it has changed the way I plan, teach and help my students.

Care must be taken when approaching educational research. There is just so much of it out there, and you can pretty much find a study or a piece of research to support any view you might hold. The problem is that any research conducted in laboratories may have questionable application to classroom environments, whereas any research conducted in classrooms is, by their very nature, prone to influence by many extraneous factors. Greg Ashman provides a very useful guide here explaining what he looks for when evaluating a piece of research. On this page I have tried to follow these guidelines wherever possible, but please be aware that I am as prone to bias as anyone!

I have also written a workshop based on these findings, and my subsequent experience trying my takeaways from them with my students, called "How I wished I'd taught maths". For more information, please visit the Mr Barton for Hire page.

I really hope you find this selection of research useful.

Contents

Cognitive Sciencekeyboard_arrow_up
Back to Top

Cognitive Science (along with Cognitive Psychology - often the two are hard to separate) is concerned with how people think, what motivates them to think, the importance of what they think about, and what conditions foster effective thinking. For Daniel Willingham, memory is the residue of thought (I love that phrase!), so if we can help students think, then we can help them remember and learn.

Research Paper Title:
The Science of Learning
Author(s): Deans for Impact
My Takeaway:
This paper was a game-changer for me, and is a superb way to begin any voyage into the world of educational research. It is a summary of existing cognitive science research about how students learn. The questions addressed are:
1. How do students understand new ideas?
2. How do students learn and retain new information?
3. How do students solve problems?
4. How does learning transfer to new 4 situations in or outside of the classroom?
5. What motivates students to learn?
6. What are common misconceptions 6 about how students think and learn?
Each of the cognitive principles is stated clearly, and is presented alongside classroom implications, making it both practical and incredibly teacher-friendly. All of the principles identified are fascinating (and references are provided if you wish to dig into each one deeper) and most will be covered at length on this page. However, what struck me in particular were the common misconceptions about how students think and learn: Students do not have different “learning styles”; Humans do not use only 10% of their brains; People are not preferentially “right-brained” or “left brained” in the use of their brains; Novices and experts cannot think in all the same ways; and Cognitive development does not progress via a fixed progression of age-related stages. How many of those have you heard? How many have you had training on? As I say, this was the paper that first opened my eyes. It is an essential read for all teachers.
My favourite quote:
The Science of Learning does not encompass everything that new teachers should know or be able to do, but we believe it is part of an important — and evidence-based — core of what educators should know about learning. 

Research Paper Title: Do Learners Really Know Best? Urban Legends in Education
Author(s): Paul A. Kirschner and Jeroen J. G. van Merrienboer
My Takeaway:
Before we get into what Cognitive Science does recommend as good practice for improving student learning, I feel it is worth considering one further paper to debunk a few incredibly pervasive educational myths. In this fascinating paper, the authors tackle three "urban legends", all of which I have heard (and attending training on!) in recent years. For each one they provide examples of the myth, together with supporting research to debunk it. If you enjoy this, then you will absolutely love Daisy Christodoulou's wonderful book Seven Myths about Education, which investigates these issues and more. The urban legends tackled in this paper are:
1. Learners as digital natives who form a generation of students knowing by nature how to learn from new media, and for whom “old” media and methods used in teaching/learning no longer work.
2. Learners have specific learning styles and that education should be individualized to the extent that the pedagogy of teaching/learning is matched to the preferred style of the learner.
3. Learners ought to be seen as self-educators who should be given maximum control over what they are learning and their learning trajectory. This latter point has been an absolute game-changer for me and will be discussed in detail in the Explicit Instruction section.
My favourite quote:
Our analysis of three urban legends in teaching and education clearly shows that, although widespread, widely believed, and even widely implemented as well-meaning educational techniques or innovations, they are not supported by scientific evidence. It should be clear by now that students are really not the best managers of their own learning with respect to navigating through and learning in the digitalworld, choosing the best way inwhich to study and learn (i.e., learning styles), or gathering useful information from the Internet. However, a continuum of available evidence exists for refuting these and other legends. At one extreme, are urban legends for which there is a tiny bit of incomplete support—but the legend itself is false or at least a severe overgeneralization (e.g., the claim that giving learners full control over the learning process will have positive effects on learning). At the other extreme of the continuum are urban legends for which there is strong empirical evidence for the opposite, showing that they are totally counterproductive in education (e.g., the claim that children are capable of effective multitasking). Finally, there are urban legends for which researchers claim that there is evidence, and for which there are even empirical studies purporting to support the legend, but the research itself or the body of research is flawed. This was demonstrated for the learning styles hypothesis but also, for example, by Lalley and Miller (2007) with respect to the learning pyramid and by Kirschner, Sweller, and Clark (2006) with respect to minimal guidance during instruction. It is yet true for all legends that they are primarily based on beliefs and convictions, not on scientific theories supported by empirical findings.

Research Paper Title: Working Memory: Theories, Models, and Controversies
Author(s): Alan Baddeley
My Takeaway:
Many models of memory are based upon the model proposed by Alan Baddeley, and this paper provides a wonderful introduction into the development of the model. It explain the multi-component aspects of working memory (divided up into the phonological loop, the visuo-spatial sketchpad, the episodic buffer, all governed by the central executive), and later how this interacts with long term memory. This model provides the basis for much of the work on this page, and you could very well make the argument that trying to understand working memory - and specifically the limits of working memory - is just about the most important thing a teacher can do.
My favourite quote:
What then are the essentials of the broad theory? The basis is the assumption that it is useful to postulate a hypothetical limited capacity system that provides the temporary storage and manipulation of information that is necessary for performing a wide range of cognitive activities. A second assumption is that this system is not unitary but can be split into an executive component and at least two temporary storage systems, one concerning speech and sound while the other is visuo-spatial. These three components could be regarded as modules in the sense that they comprise processes and storage systems that are tightly interlinked within the module and more loosely linked across modules, with somewhat more remote connections to other systems such as perception and LTM

Research Paper Title: Executive Attention, Working Memory Capacity, and a Two-Factor Theory of Cognitive Control
Author(s): Randall W Engle and Michael J. Kane
My Takeaway:
This comprehensive paper aims to describe the nature of working memory capacity (WMC), specifically with regard to its limitations, the effects of these limitations on higher order cognitive tasks, their relationship to attention control and general fluid intelligence, and their neurological substrates. I'll be honest with you, things got a little too technical for me, but the key finding was a clear one: higher order cognitive tasks are easier if you have a higher working memory capacity. To quote the authors: One of the most robust and, we believe, interesting, important findings in research on working memory is that WMC span measures strongly predict a very broad range of higher-order cognitive capabilities, including language comprehension, reasoning, and even general intelligence. This makes perfect logical sense when we look at the models of how students think later in this section, and especially when we consider the findings from Cognitive Load Theory. Unfortunately, as we will also see, it may not be possible to expand the capacity of students' (or indeed anyone's) working memories, at least not in a general way that can be transferred to all cognitive situations.
My favourite quote:
We proposed a two-factor model by which individual differences in WMC or executive attention leads to performance differences; We argued that executive attention is important for maintaining information in active memory and secondly is important in the resolution of conflict resulting from competition between task-appropriate responses and prepotent but inappropriate responses. The conflict might also arise from stimulus representations of competing strength. This two-factor model fits with current thinking about the role of two brain structures: the prefrontal cortex as important to the maintenance of information in an active and easily accessible state and the anterior cingulate as important to the detection and resolution of conflict.


Research Paper Title:
Why don't students like school? Because the mind is not designed for thinking.
Author(s): Daniel T. Willingham
My Takeaway:
This is an excellent summary of one of my all-time favourite books, Why Don't Students Like School. There is so much to takeaway from this, but the biggest thing for me is about how students think. Students (like all of us) are naturally curious and enjoy thinking, but that curiosity is fragile, and if the conditions are not right then students will avoid thinking. For Willingham, memory is the residue of thought, so unless students are thinking, there is little chance they will remember or learn. Willingham stresses that successful thinking requires three things:
1) information from the environment
2) facts and procedures stored in long term memory
3) space in working memory.
When students are given new information, they hold it in working memory as they connect it to other new information and experiences and evaluate it against known concepts. The problem is that working memory capacity is limited, and gets filled when students need to carry out basic procedures or search for facts. Hence, when faced with complex problems, if those facts and procedures are missing from long term memory, working memory gets filled up, thinking becomes hard, students stop doing it, and no learning takes place. Here is the key point: experts and novices think differently. Experts rely on retrieving whole schema (connected items of information) from long-term memory to get around the limits on our fragile working memories. Novices (i.e. most students) don’t have these memorised schema to rely on so attempt to hold too much information in their working memories which leads to cognitive overload. Therefore, we can make thinking easier for students by ensuring they have access to sufficient knowledge and procedures stored in long term memory, which allows them to develop these schema, which frees up capacity in working memory, which can then be used for problem solving. As teachers we can help students acquire this knowledge and these procedures via Explicit Instruction, emphasising the value of deliberate practice to improve, not taking study skills for granted, and praising effort not ability. Furthermore, once embedded, we can help students retain this knowledge and these procedures using the work discussed in the three Memory sections. It's as easy as that ;-)
My favourite quote:
Teachers often seek to draw students in to a lesson by presenting a problem that they believe interests students, or by conducting a demonstration or presenting a fact that they think students will find surprising. In either case, the goal is to puzzle students, to make them curious. This is a useful technique, but it’s worth considering whether these strategies might also be used not at the beginning of a lesson, but after the basic concepts have been learned.

Research Paper Title: Is Working Memory Training Effective? A Meta-Analytic Review
Author(s): Monica Melby-Lervåg and Charles Hulme
My Takeaway:
A question that might arise from reading Willingham's work is "can we increase working memory capacity?", because if we can then thinking will be a lot easier for students. In recent years, the there has been an increase in the popularity of brain training games and strategies, which claim to boost this all important working memory capacity. Unfortunately, the findings of this comprehensive meta-analysis of existing research suggest that any benefits are short-term, and crucially do not transfer to other situations. So, whilst you may be able to train your brain to hold more of a specific type of written or oral information, this will not transfer across to enable you to solve more complex maths problems. However, all is not lost, because we can get around the limits of working memory using the power of knowledge, as the next two papers in this section discuss.
My favourite quote:
Currently available working memory training programs have been investigated in a wide range of studies involving typically
developing children, children with cognitive impairments (particularly ADHD), and healthy adults. Our meta-analyses show clearly that these training programs give only near-transfer effects, and there is no convincing evidence that even such near-transfer effects are durable. The absence of transfer to tasks that are unlike the training tasks shows that there is no evidence these programs are suitable as methods of treatment for children with developmental cognitive disorders or as ways of effecting general improvements in adults’ or children’s cognitive skills or scholastic attainments.


Research Paper Title: A Simple Theory of Complex Cognition
Author(s): John R. Anderson
My Takeaway:
I am not entirely sure that "simple" is the best description of this theory, at least not for me anyway! In the Adaptive Character of Thought (ACT—R) theory, complex cognition arises from an interaction of procedural and declarative knowledge. Declarative knowledge is represented as an associative memory network which contains the facts known by the system. It is represented in units called "chunks". Procedural knowledge is represented as a production system and enables the system to apply its knowledge and execute behaviour to achieve its goals. It is represented in units called production rules. A great many such knowledge units underlie human cognition. From this large database, the appropriate units are selected for a particular context by activation processes that  are tuned to the statistical structure of the environment. Declarative knowledge can be acquired quickly from direct encoding of the environment, while procedural knowledge takes longer and must be compiled from declarative knowledge through practice. After a certain amount of practice, the path or production becomes stable and procedural learning has occurred. The conditions under which we learn procedures, therefore, are determined by existing declarative knowledge. According to the ACT-R theory, the power of human cognition depends on the amount of knowledge encoded and the effective deployment of the encoded knowledge. The main message is a crucial one: to be able to think, we need knowledge, and the more knowledge the better. Long-term memory is capable of storing thousands of facts, and when we have memorised thousands of facts on a specific topic, these facts together form what is known as a schema - connected items of information. When we think about that topic, we use that schema. When we meet new facts about that topic, we assimilate them into that schema. In other words, when students are given new information, they hold it in working memory as they connect it to other new information and experiences and evaluate it against known concepts. Crucially, if we already have a lot of facts in that particular schema, it is much easier for us to learn new facts about that topic. New knowledge builds on existing knowledge. And how best to learn and retain these facts? Well, the sections on Explicit Instruction, Cognitive Load Theory, and Memory should provide some answers.
My favourite quote:
All that there is to intelligence is the simple accrual and tuning of many small units of knowledge that in total produce complex cognition. The whole is no more than the sum of its parts, but it has a lot of parts.

Research Paper Title: Brain Changes in the Development of Expertise: Neuroanatomical and Neurophysiological Evidence about Skill-Based Adaptations
Author(s): Nicole M. Hill & Walter Schneider
My Takeaway:
This paper proposes that experts (those with high levels of domain-specific knowledge) don't just know more than novices, they actually think in a fundamentally different way. They explain that experts differ from novices in terms of their knowledge, effort, recognition, analysis, strategy, memory use, and monitoring, and that these differences are due to the structure of their long-term memories. As we learn, our brain architecture changes and thoughts are processed differently. This means that as we move to mastery of a given skill or concept, our brains form different links in long-term memories, and it is actually possible to observe different activation patterns during problem solving.  They conclude that in addition to processing efficiency, enriched representations, and structural expansions, experts can flexibly use strategies, by recruiting the associated brain regions, to solve a range of problems, whereas novice performers can not. The fact that experts and novices think in a fundamentally different way, and that knowledge is at the route of this, will have huge implications throughout this page, but for for Problem Solving in particular.
My favourite quote:
The specific nature of the representational areas suggests that both training and performance will be sensitive to the strategy and nature of the training. What is learned is based on which representational areas are active during training. Typically, as practice develops, activity decreases, and there are rarely new areas that develop in laboratory studies of skill acquisition. This suggests that training causes local changes in the specific representational areas that support skilled performance. In studies of extensive training, there is ample evidence for changes in cognitive processing as well as structural changes in the nervous system.

Research Paper
Title: How Knowledge Helps
Author(s): Daniel T. Willingham
My Takeaway:
This is a great paper, with the same powerful message as the one above by Anderson: knowledge does much more than just help students hone their thinking skills: it actually makes learning easier. Willingham argues - providing links to research along the way - that knowledge helps students take in more information, think about new information, and remember new information. What will perhaps be of most interest to teachers is that knowledge helps students solve problems. This sounds ridiculously obvious, but I don't think it is - at least not to me, anyway! One teaching strategy that I have been guilty of over my career is assuming that exposing students to lots of problems will help them become good problem solvers. The simple answer is that it won't. Without existing knowledge, the student cannot recognise familiar patterns (the ability to "chunk" information is crucial here and is related to the concepts of schema above), and hence attempts to solve problems individually, which inevitably overburdens working memory. This issue is discussed more in the Problem Solving section.
My favourite quote:
Those with a rich base of factual knowledge find it easier to learn more—the rich get richer. In addition, factual knowledge enhances cognitive processes like problem solving and reasoning. The richer the knowledge base, the more smoothly and effectively these cognitive processes—the very ones that teachers target—operate. So, the more knowledge students accumulate, the smarter they become.

Research Paper Title: Categorization and Representation of Physics Problems by Experts and Novices
Author(s): Michelene T H Chi, Paul J Feltovich and Robert Glaser
My Takeaway:
I found this study fascinating. We have seen in the paper above that knowledge is crucial to aid thinking, and that an obvious distinguishing feature between experts and novices is that experts have more knowledge than novices. Here we have clear evidence of just how big an advantage that extra knowledge is - it actually enables exerts to see problems differently. The authors asked physics novices (undergraduates) and experts (PhD students) to sort physics problems into categories. The novices sorted by the surface features of a problem—whether the problem described springs, an inclined plane, and so on. The experts, however, sorted the problems based on the physical law needed to solve it (e.g., conservation of energy). The advantage of the latter approach was that the experts could see the deeper structure of the problem, which meant they could more readily access the appropriate schema to solve the problem, which aided transfer. Novices, on the other hand, focused on the surface structure and hence did not make the necessary connections between problems. experts (those with strong subject knowledge) don't just know more than novices - they actually see problems differently. It is not the use of an inefficient means-ends strategy that allows experts to solve problems so readily. Instead, experts have richly organised knowledge (analogous to problem schemas) that allow them to represent the problems in such a way that the solutions became transparent. Indeed, experts are likely expanding less mental energy solving problems than novices because they have the relevant schemas in place. Because of their extensive domain-specific knowledge, experts quickly recognise the problem’s deep structure, relate it to similar problems they have met in the past, and embark upon a (usually successful) strategy to solve it.
An immediate implication for me here is going through past papers with students. How often do we find that students focus on the surface structure of a problem? A question about lowest common multiple set in the context of bus timings on one paper becomes a question all about buses, and no connection is made between a question with essentially the same deep structure but set in the context of boxes of cakes and donuts on another paper. They are treated as two completely separate problems. Without strong, domain-specific knowledge, students are unlikely to recognise the deep structure of problems, and hence will be unlikely to transfer the knowledge successfully to different situations. Going through past papers, covering loads of different topics, before learners are secure i each of those topics, is likely to be a waste of time. It is perhaps better to focus on domain-specific knowledge (i.e. teach the basics of lowest common multiple so the calculations become fluent) and then carefully expose students to problems in different contexts but with the same deep structure, guiding them by means of example-pairs, clearly articulating your thought processes. These techniques are covered in detail in the Cognitive Load Theory and Problem Solving sections.
My favourite quote:
Our research goal has been to ultimately understand the difference between experts and novices in solving physics problems. A general difference often found in the literature and also in our own study is that experts engage in qualitative analysis of the problem prior to working with the appropriate equations. We speculate that this method of solution for the experts occurs because the early phase of problem solving (the qualitative analysis) involves the activation and confirmation of an appropriate principle-oriented knowledge structure, a schema. The initial activation of this schema can occur as a data-driven response to some fragmentary cue in the problem. Once activated, the schema itself specifies further (schema-driven) tests for its appropriateness. When the schema is confirmed, that is, the expert has decided that a particular principle is appropriate, the knowledge contained in the schema provides the general form that specific equations to be used for solution will take.

Research Paper Title:
Top 20 Principles from Psychology for PreK-12 Teaching and Learning
Author(s): American Psychological Association
My Takeaway:
This is a wonderful collection of key principles from psychology that have direct implications for teaching and learning, all of which are backed up by research. Many of these have directly influenced how I plan and deliver my lessons, but for the sake of brevity, I will focus on just three here:
1)  What students already know affects their learning. This so ridiculously obvious, but is also ridiculously easy to overlook - at least for me, anyway. The authors explain that learning consists of either adding to existing student knowledge (conceptual growth), or transforming or revising student knowledge (conceptual change). Conceptual growth cannot occur is their existing knowledge is incomparable with the new knowledge, or incomplete, and strategies such as Formative Assessment may be needed to establish this. Likewise, conceptual change can be tricky to achieve if students hold misconceptions, and simply telling students they need to think differently will generally not lead to substantial change in student thinking. Ways to deal with this are covered in the Explicit Instruction section. 
2) Students tend to enjoy learning and to do better when they are more intrinsically rather than extrinsically motivated to achieve. This is directly related to research on Motivation discussed later on this page. As students develop increasing competence, the knowledge and skills that have been developed provide a foundation to support the more complex tasks, which become less effortful and more enjoyable. When students have reached this point, learning often becomes its own intrinsic reward. This is obviously preferable to being externally motivated, as students may disengage once the external rewards are no longer provided. However, one point I feel is worth mentioning is that some (or even all) students may need a certain degree of external motivation during the initial stages of learning a topic, before mastery has a chance to become its own reward.
3) Students persist in the face of challenging tasks and process information more deeply when they adopt mastery goals rather than performance goals. This is very much related to the work on student Mindset, but has some very practical applications. Students can engage in achievement activities for two very different reasons: They may strive to develop competence by learning as much as they can (mastery goals), or they may strive to display their competence by trying to outperform others (performance goals). Performance goals can lead to students’ avoiding challenges if they are overly concerned about performing as well as other students. In typical classroom situations, when students encounter challenging materials, mastery goals are generally more useful than performance goals. How do we help develop these mastery goals? The authors advise strategies such as avoiding social comparisons, avoiding non-task-specific feedback such as "brilliant", conduct student evaluations in private, and focus on improvement over achievement.
My favourite quote:
Psychological science has much to contribute to enhancing teaching and learning in the classroom. Teaching and learning are intricately linked to social and behavioral factors of human development, including cognition, motivation, social interaction, and communication. Psychological science can also provide key insights on effective instruction, classroom environments that promote learning, and appropriate use of assessment, including data, tests, and measurement, as well as research methods that inform practice. We present here the most important principles from psychology—the “Top 20”—that would be of greatest use in the context of preK–12 classroom teaching and learning, as well as the implications of each as applied to classroom practice
.

Research Paper Title:
When More Pain is preferred to Less: Adding a Better End
Author(s): Daniel Kahneman, Barbara L. Fredrickson, Charles A. Schreiber, and Donald A. Redelmeier
My Takeaway:
Often I would end my lessons with a tricky question - maybe a past exam question. This would be the most difficult question I had asked all lesson, and its purpose would for both myself and my students to see how far they had come. I would usually build it up: "okay, this is as tough as it gets. Can you do it?". And often the result was that some could and some couldn't, and I was okay with that as it was meant as an extension question. But having read this paper, I now look at it from the perspectives of the students who could not do the question. What is their  impression of that lesson? What will they remember? Will it be the 45 minutes of success they enjoyed at the start, or the 5 minutes of "failure" at the end. Cognitive psychology suggests the latter. This fascinating paper looks at how people's memory of an experience is often dominated by the feelings of pain and discomfort during the final moments, as opposed to what happened during the rest of the experience. Applying this to my students, when thinking about their maths lesson, many of them would have judged it as a failure because of that final, tricky problem. This negative emotion - potentially swirling around their heads the couple of days until their next lesson - could lead to a subsequent lack of confidence and a lack of engagement in mathematics, neither of which are conducive to learning. Hence, I now always end my lesson with a question that is of mid-range difficulty, or maybe even easier. More often than not, it will be a diagnostic multiple choice question (see the Assessment for Learning section). The majority of the learning has happened in the first 45 minutes - my objectives in those last 5 minutes are for me to identify any key misconceptions that will inform my future planning, and for my students to feel good about themselves ready for their next maths lesson.
My favourite quote:
The results add to other evidence suggesting that duration plays a small role in retrospective evaluations of aversive experiences; such evaluations are often dominated by the discomfort at the worst and at the final moments of episodes.

Research Paper Title: Organizing Instruction and Study to Improve Student Learning
Author(s): Harold Pashler et al
My Takeaway:
Much like The Science of Learning above, this is an outstanding overview of key research findings from cognitive psychology that have direct implications for the classroom. A wonderful feature of this paper is that it also addresses potential roadblocks to implementing the finding in the classroom and suggests possible strategies to overcome them. The recommendations, each of which are covered in a comprehensive and yet easy to follow way, are: Space learning over time, Interleave worked example solutions and problem-solving exercises, Combine graphics with verbal descriptions, Connect and integrate abstract and concrete representations of concepts, Use quizzing to promote learning, Help students allocate study time efficiently, Help students build explanations by asking and answering deep questions. These recommendations have implications for planning, teaching, and helping students formulate effective revision strategies.
My favourite quote:
We recommend a set of actions that teachers can take that reflect the process of teaching and learning, and that recognizes the ways in which instruction must respond to the state of the learner. It also reflects our central organizing principle that learning depends upon memory, and that memory of skills and concepts can be strengthened by relatively concrete—and in some cases quite nonobvious - strategies

Research Paper Title: Do Visual, Auditory, and Kinesthetic Learners Need Visual, Auditory, and Kinesthetic Instruction?
Author(s): Daniel T. Willingham
My Takeaway:
When I first started teaching 12 years ago, Visual, Auditory and Kinesthetic learning (or the VAK model) was all the rage. My students had to fill out surveys which indicated what type of learner they were, and I had to adapt my lessons accordingly. Needless to say, it was chaos (how exactly do you make solving quadratic equations appropriate for a kinesthetic learner? Maybe do a dance about them?). This wonderful paper presents a clear summary of the evidence on learning types, along with some key implications:
1) Some memories are stored as visual and auditory representations—but most memories are stored in terms of meaning. How you first learn something may be in a visual or auditory way, but how you remember it is tied to what it actually means. When I think about how to add two fractions together (as I regularly do), that information is neither stored in an visual or auditory way. I just know it.
2) The different visual, auditory, and meaning-based representations in our minds cannot serve as substitutes for one another. This is crucial. Our minds have these different types of representations for a reason: Different representations are more or less effective for storing different types of information. The particular shade of green of a frozen pea would be stored visually because the information is inherently visual, whereas the sound of my wife's voice shouting at me is stored in auditory form. These cannot be swapped around.
3) Children probably do differ in how good their visual and auditory memories are, but in most situations, it makes little difference in the classroom. This is the big one. It's likely that some students should have a relatively better visual memory or auditory memory, but that doesn't mean we should always teach to it. The key is that teachers should focus on the content's best modality—not the student's. I teach geometry topics in a visual way, regardless of the preferred learning styles of my students, because even a so-called auditory learner will understand it better that way.
My big takeaway is that it is far more important to carefully consider the best form of presentation for a concept than to worry about catering for each and every child's differing preferences.
My favourite quote:
Experiences in different modalities simply for the sake of including different modalities should not be the goal. Material should be presented auditorily or visually because the information that the teacher wants students to understand is best conveyed in that modality. There is no benefit to students in teachers' attempting to find auditory presentations of the Mayan pyramids for the students who have good auditory memory. Everyone should see the picture. The important idea from this column is that modality matters in the same way for all students.

Research Paper Title: Learning Styles: Concepts and Evidence
Author(s): Harold Pashler, Mark McDaniel, Doug Rohrer, and Robert Bjork
My Takeaway:
For me, this is the final nail n the coffin for learning styles. A comprehensive review of the evidence by respected researchers, which reaches a clear conclusion. I can do no better than directly quote the authors themselves:
My favourite quote:
Our review of the learning-styles literature led us to define a particular type of evidence that we see as a minimum precondition for validating the use of a learning-style assessment in an instructional setting. As described earlier, we have been unable to find any evidence that clearly meets this standard…
…The contrast between the enormous popularity of the learning-styles approach within education and the lack of credible evidence for its utility is, in our opinion, striking and disturbing. If classification of students’ learning styles has practical utility, it remains to be demonstrated


Research Paper Title: Improving Education: A Triumph of Hope ever Experience
Author(s): Robert Coe
My Takeaway:
This is the transcript from a lecture that Rob Coe made in Durham in 2013 and it caused a big outcry on Twitter. The theme of the paper is that much of the evidence that attainment in schools has risen over the last 30 years is questionable, and the entire paper is worth a read. However, the part I want to focus on here are the "Poor Proxies for Learning" as identified by the author. They are:
1. Students are busy: lots of work is done (especially written work)
2. Students are engaged, interested, motivated
3. Students are getting attention: feedback, explanations
4. Classroom is ordered, calm, under control
5. Curriculum has been ‘covered’ (ie presented to students in some form)
6. (At least some) students have supplied correct answers (whether or not they really understood them or could reproduce them independently)
It is worth reminding ourselves what this means. Coe is not saying that any of these things prohibit learning taking place, nor is he dismissing the possibility that they can occur alongside learning. The point Coe is making is that because learning is invisible and hence we can only observe proxies of learning, and the items in this list are not particularly good proxies. If we observe any of them alone, without further evidence, then we should be extremely careful in concluding that learning is taking place. Perhaps the most controversial one of these is engagement. For many years that was the number one thing I strived for in my own lessons, and also one of the main things I looked for when observing others. But when you think about it, it makes sense. Just because a student (or a group of students) is engaged, it does not necessarily mean they are learning. We will see in the Encoding section that students remember what they think about, so if students are engaged in the wrong thing (such as the colour they will choose for the next bit of the revision poster they are working on), then they are unlikely to be learning. Likewise, we have all witnessed the quiet, passive student who does not take part in class discussion, appears away with the fairies, and yet performs really well on homeworks and assessments. They do not appear "engaged", but they are learning. So, what are we to take from this? Well, for me it is all about not assuming learning is taking place by relying on easily observable things like those listed above. Instead, we need evidence. And where should we get this evidence? The most obvious way is from test performance, hence my obsession with low-stakes quizzes that I will discuss in the Testing section.
My favourite quote:
If it is true that teaching is sometimes not focussed on learning, how can we make them better aligned? One answer is that it may help to clarify exactly what we think learning is and how it happens, so that we can move beyond the proxies. I have come up with a simple formulation: Learning happens when people have to think hard.
Obviously, this is over-simplistic, vague and not original. But if it helps teachers to ask questions like, ‘Where in this lesson will students have to think hard?’ it may be useful.


Research Paper Title:
Are Sleepy Students Learning?
Author(s): Daniel T. Willingham
My Takeaway:
Some of my Year 11 lads yawn more than they speak, so I was fascinated by this paper, and its relation to the one above. I was fascinated to find out that children undergo a biological change (a change in chronotype) in their teenage years that gives them a preference to staying up late versus getting up early. This effect is compounded if teenagers use electronic devices (phones, tablets, consoles watching tv etc) late into the evening as exposure to the back-lit screens makes them wakeful. And what effect does this lack of sleep have? In sum, sleep deprivation influences many (but not all) aspects of children's mood, cognition, and behaviour. Lack of sleep is associated with poorer school performance as rated by students themselves and by teachers. Restricted sleep is also associated with lower grades in studies in the United States, a finding replicated in Norway and Korea. So, what can we teachers do to help? Unfortunately, not a great deal. The two things that would likely make the most difference are out of our control - the time students go to bed, and the time lessons start in the morning. In terms of the latter, studies suggest the later the better, and interesting implications for school administrators are discussed. That finding is interesting, because I always have a preference for teaching morning lessons versus those in the afternoon - trying to get anything out of some classes during the last period of the day can be torture! But it seems likely that Period 1 might be difficult as well! My takeaway is to make the most of the lessons later in the morning, and crucially to pass these findings onto students and their parents. In terms of the work they do outside of the lesson, if your students can combine a good night's sleep with the strategies advocated in the Revision session, then... well, maybe you might even end up with a good night's sleep yourself :-)
My favourite quote:
Inadequate sleep represents a challenge to educators that is in one sense overt—teachers see students drowsy in class every day—and in another sense subtle, because it seems like a common nuisance rather than a real threat to education. And indeed, the problem should not be overstated, at least insofar as it affects education. The impact of typical levels of inadequate sleep on student learning is quite real, but it is not devastating. All the same, its impact lasts for years, and there is every reason to think that it is cumulative.

Effective Teachingkeyboard_arrow_up
Back to Top

Trying to deduce what makes effective teaching may seem like an impossible job. For years I was convinced that there was no "right" way, and it was very much down to the discretion of the individual teacher. However, whilst I believe there are no definitive answers to questions such as "what is the best way to introduce fractions", I am starting to believe that there are several key principles that make the teaching of maths in general effective, based around what we have learned about how students think in the Cognitive Science section.

Research Paper Title: Principles of Instruction
Author(s): Barak Rosenshine
My Takeaway:
I absolutely love this paper. It presents 10 research-based principles from cognitive science and studies of master teachers, together with practical strategies for classroom implementation. The 10 principles are all based around the model of Explicit Instruction that will be the focus of the next section, and sound so simple when you see them written down:
1) Begin a lesson with a short review of previous learning.
2) Present new material in small steps with student practice after each step
3) Ask a large number of questions and check the responses of all students
4) Provide models.
5) Guide students practice.
6) Check for student understanding.
7) Obtain a high success rate.
8) Provide scaffolds for difficult tasks.
9) Require and monitor independent practice.
10) Engage students in weekly and monthly review.
For me, this is the structure of a very well planned lesson, encompassing aspects of spacing, retrieval, worked examples, modelling, formative assessment and problem solving, all of which will be discussed in greater detail further on this page. For each element there are links to further research to support the author's claim.
My favourite quote:
Education involves helping a novice develop strong, readily accessible background knowledge. It's important that background knowledge be readily accessible, and this occurs when knowledge is well rehearsed and tied to other knowledge. The most effective teachers ensured that their students efficiently acquired, rehearsed, and connected background knowledge by providing a good deal of instructional support. They provided this support by teaching new material in manageable amounts, modeling, guiding student practice, helping students when they made errors, and providing for sufficient practice and review. Many of these teachers also went on to experiential, hands-on activities, but they always did the experiential activities after, not before, the basic material was learned.

Research Paper Title: What makes great teaching? Review of the underpinning research
Author(s): Robert Coe, Cesare Aloisi, Steve Higgins and Lee Elliot Major
My Takeaway:
This report from CEM, Durham University and the Sutton Trust seeks to answer the simple, but unsurprisingly complex, question of "What makes good teaching?". The authors outline six common components of good teaching and rank them by how strong the evidence is in showing that focusing on them can improve student outcomes. The six components are:
1) Content knowledge (Strong evidence of impact on student outcomes)
2) Quality of instruction (Strong evidence of impact on student outcomes)
3) Classroom climate (Moderate evidence of impact on student outcomes)
4) Classroom management (Moderate evidence of impact on student outcomes)
5) Teacher beliefs (Some evidence of impact on student outcomes)
6) Professional behaviours (Some evidence of impact on student outcomes)
Each of these is worthy of further discussion, but I want to focus on the first two. There are two aspects to content knowledge. Firstly, teacher need to have a strong, connected understanding of the material being taught. But in addition to this, teachers must also understand the ways students think about the content, be able to evaluate the thinking behind non-standard methods, and identify typical misconceptions students have. We have all seen great mathematicians who do not make good teachers as they lack the second of these aspects, but likewise there is no getting around the fact that to be a good maths teacher you need to know your maths. Quality of Instruction concerns the best way to communicate knowledge to students. I can do little better than to quote the report itself: Quality of instruction is at the heart of all frameworks of teaching effectiveness. Key elements such as effective questioning and use of assessment are found in all of them. Specific practices like the need to review previous learning, provide models for the kinds of responses students are required to produce, provide adequate time for practice to embed skills securely and scaffold new learning are also elements of high quality instruction. To me, this seems like a model of effective explicit instruction.
My favourite quote:
There is some evidence that an understanding of what constitutes effective pedagogy – the method and practice of teaching – may not be so widely shared, and even where it is widely shared it may not actually be right. Hence it is necessary to clarify what is known about effective pedagogy before we can think about how to promote it. Unless we do that there is a real danger that we end up promoting teaching practices that are no more – and perhaps less – effective than those currently used.

Research Paper Title: What works best: Evidence-based practices to help improve NSW student performance
Author(s): Centre for Education Statistics and Evaluation
My Takeaway:
Another superb attempt to categorise the practices that are likely to improve student learning. For each one we get a summary of why it matters, what the evidence says, and practical; implications for teachers. The seven areas identified are:
1. High expectations
2. Explicit teaching
3. Effective feedback
4. Use of data to inform practice
5. Classroom management
6. Wellbeing
7. Collaboration
Again, it is interesting to note the inclusion of explicit teaching, but no mention of discovery/inquiry based learning.
My favourite quote:
These themes offer helpful ways of thinking about aspects of teaching practice but they are not discrete. Rather, they overlap and connect with one another in complex ways. For example, providing timely and effective feedback to students is another element of explicit teaching – two of the more effective types of feedback direct students’ attention to the task at hand and to the way in which they are processing that task. Similarly, being explicit about the learning goals of a lesson and the criteria for success gives high expectations a concrete form, which students can understand and aim for. Wellbeing and quality teaching are mutually reinforcing – if students with high levels of general wellbeing are more likely to be engaged productively with learning, it is also true that improving intellectual engagement can improve wellbeing


Explicit Instructionkeyboard_arrow_up
Back to Top

Explicit instruction may be thought of as teacher-led instruction. It is more interactive than simply lecturing, involving questioning and responsive teaching, but a key characteristic is that the teacher dictates the content and structure of the lesson, in contrast to more student-centered approaches. I often think of Explicit Instruction as comprising of four elements: explaining, modeling, scaffolding and practising. A great deal of research suggests this is the most effective way to help children learn, and despite having moved away from it over the course of my career, it is now the central structure upon which I build my lessons. These papers discuss the advantage and techniques involved in effective explicit instruction, which are then examined further in the section on Cognitive Load Theory.

Research Paper Title: Follow Through Evaluation (and for a lovely online summary of Project Follow Through and the work of Siegfried Engelmann click here)
Author(s): Siegfried Engelmann
My Takeaway:
When I interviewed Dylan Wiliam for my podcast, he described Project Follow Through as the most important piece of educational research ever conducted - and yet I had never heard of it. Project Follow Through was the most extensive educational experiment ever conducted. Beginning in 1968 under the sponsorship of the US government, it was charged with determining the best way of teaching at-risk children from kindergarten through to grade 3. Over 200,000 children in 178 communities were included in the study, and 22 different models of instruction were compared. The results were startling.  Siegfried Englemann's model of Direct Instruction outperformed all other models in basic skills, and indeed all but two of the other models returned negative numbers, which means they were soundly outperformed by children of the same demographic strata who did not go through Project Follow Through. But what is perhaps most interesting is that Direct Instruction also outperformed all other models on DI was not expected to outperform the other models on cognitive skills, which require higher-order thinking. Indeed, it was the only model to return positive scores across reading, maths concepts and and maths problem solving. And if that wasn't enough, students who followed the Direct Instruction Model also had higher self esteem and self-confidence. This image presents these findings clearly. The thing that struck me most was the effect Direct Instruction had on problem solving skills. I had always assumed that you use instruction to get the basics sorted, and then things like investigations, puzzles and rich tasks to develop students problem solving. But, as we shall see, it is a lot more complicated that than, and models of direct and explicit instruction are key to developing the problem solving skills that we all want our students to have.
My favourite quote:
The basic problem we face is that the most popular models in education today (those based on open classrooms, Piagetian ideas, language experience, and individualized instruction) failed in Follow Through. As a result there are many forces in the educational establishment seeking to hide the fact that Direct Instruction, developed by a guy who doesn’t even have a doctorate or a degree in education, actually did the job. To keep those promoting popular approaches from hiding very important outcomes to save their own preconceptions will take formidable help from persons like yourself. We hope it is not too late.

Research Paper Title:
Why Minimal Guidance During Instruction Does Not Work (there is also a shorter, easier to digest version Putting Students on the Path to Learning, along with an amazing sketchnote summary from Oliver Caviglioli)
Author(s): Paul A. Kirschner, John Sweller, Richard E. Clarke
My Takeaway:
This article, possibly more than any other on this page, changed my way of thinking. It provides evidence that for everyone apart from experts, partial guidance during instruction (the kind used in inquiry-based learning or project work) is less effective than full guidance (direct or explicit instruction). This suggests that the practice (often encouraged by previous Ofsted inspection guidelines and senior management alike) of encouraging students to take more ownership of their learning, "discover" key concepts, and for the teacher to be the "facilitator" of learning and not the expert imparting their knowledge, is detrimental to students' learning and long term development. Without facts and procedures stored in long term memory, students cannot become the problem solvers we all want them to be. Solving the kind of problems the students encounter during projects, inquiries or even multi-mark exam questions requires knowledge and procedures stored in long term memory. Without these - and crucially without guidance from the teacher - the student's fragile working memory is faced with trying to process too much information: What is the question asking? What knowledge do I need? What procedures do I need? Now how do I actually carry out out that procedure? This can lead to Cognitive Overload, which will be covered in detail later. I have four major takeaways from this:
1) Students can look like they are working hard on complex problems, and not actually be learning anything - this is the concept of problem-solving search via means-end analysis, and will be addressed in the Cognitive Load Theory section.
2) You cannot teach students to become problem solvers by simply giving them loads of different problems to solve. They need to be expertly supported by the teacher, and the problems selected carefully. These concepts will be covered in more detail in the Problem Solving section.
3) It is quite likely that throughout the course of minimal guided instruction, students will develop misconceptions, or incomplete and disorganised knowledge. This is because there is necessarily less control over the direction of their learning - which for many years I assumed to be a good thing!
4) I love an inquiry or a rich tasks, but now when considering one, I think about the opportunity cost. I ask myself: is there a more efficient way of my students acquiring the knowledge I want them to? When asking myself this these days, more often than not the answer is "yes". If I can tell students the knowledge - having clearly and carefully planned out my explanation - and then get them to practice applying that knowledge to begin the process of committing it to long term memory, as opposed to hoping they discover it, and then having to correct and reteach any areas that are missing, can I really justify not doing so?
Finally, this paper is also careful to point out that full guidance may not be the most suitable instructional technique for students once they reach mastery in a given domain (i.e. become experts) - but all too often I have assumed students are at that level before they actually are.
My favourite quote:
After a half-century of advocacy associated with instruction using minimal guidance, it appears that there is no body of research supporting the technique. In so far as there is any evidence from controlled studies, it almost uniformly supports direct, strong instructional guidance rather than constructivist-based minimal guidance during the instruction of novice to intermediate learners. Even for students with considerable prior knowledge, strong guidance while learning is most often found to be equally effective as unguided approaches. Not only is unguided instruction normally less effective; there is also evidence that it may have negative results when students acquire misconceptions or incomplete or disorganized knowledge.

Research Paper Title: Assisting Students Struggling with Mathematics
Author(s): Institute of Education Sciences
My Takeaway:
This is a fascinating paper on the most effective practices for assisting students who struggle with mathematics that would be of great value to any teacher who teaches lower achieving students or students with special educational needs. I am not afraid to admit that this is a particular area of weakness in my own teaching. The paper provides eight recommendations, some of which are aimed more at a senior leadership of over governmental level. However, one recommendation stood out to me in particular, and is directly relevant to our discussion in this section:
Recommendation 3: Instruction during the intervention should be explicit and systematic. This includes providing models of proficient problem solving, verbalization of thought processes, guided practice, corrective feedback, and frequent cumulative review. This recommendation is then followed by supporting evidence and practical strategies to carry it out.
In the past I have been guilty of changing my teaching approach for students who struggled with mathematics. I often tried to make it more about discovery, less teacher led, all with an aim of making mathematics less daunting for these students. However, the research suggests I should have been doing the opposite. As we have seen in the papers above, the only people likely to benefit from minimally guided instruction are experts, and hence the students most likely to benefit from a more teacher-led approach, together with careful use of examples, explanation, modelling and practice, are the students who find maths the most difficult. I will not be making that mistake again.
My favourite quote:
Our panel judged the level of evidence supporting this recommendation to be strong. This recommendation is based on six randomized controlled trials that met WWC standards or met standards with reservations and that examined the effectiveness of explicit and systematic instruction in mathematics interventions. These studies have shown that explicit and systematic instruction can significantly improve proficiency in word problem solving and operations across grade levels and diverse student populations.

Research Paper Title:
The Role of Deliberate Practice in the Acquisition of Expert Performance (for an easier to read introduction to Deliberate Practice that covers all the key principles, see The Making of an Expert)
Author(s): K. Anders Ericsson, Ralf Th. Krampe, and Clemens Tesch-Romer
My Takeaway:
To become an expert at something, you must practice in the right way. This paper (along with the excellent book, Peak), outlines a model of Deliberate Practice. A key feature of this model is that you breakdown a complex process, isolate an individual skill and work on it, receiving regular and specific feedback so you can improve you performance. Crucially (and this was a game-changer for me), the skill you are practising may not look like the final thing you are working towards. It is why young Spanish footballers start on tiny pitches, playing 5-aside, working on specific drills - you learn far more from that than you ever would in a big 11-aside game where you hardly touch the ball, and there are so many other factors out of your control. It is why professional musicians practice scales over and over again instead of constantly playing full pieces. The implications for teaching are huge. When we teach a topic, say adding fractions, perhaps we need to break the skills down into their most minute components. If we want students to have success in tricky exam questions (e.g. those carrying several marks, or those of a more problem-solving nature)  then it might not be the case that we should present these questions in their final forms. Instead we should break them up, isolate individual skills (such as basic numeracy, or identifying exactly what the question is asking), and practice these in isolation until students have mastered them before even going near an exam. The activities students do in practice may look nothing like what they will be expected to do in the final exam, but that is absolutely fine. Moreover - and for me this is just as important - students need immediate feedback on their practice. Think about a tennis player hitting hundreds of balls during a service drill. Each time they hit a serve, they immediately see the result and can adjust their technique to improve. How can we ensure students can do the same during their practice, so errors are identified and not compounded? It could be as simple as giving them immediate access to the answers. This is why two teachers I interviewed for my Mr Barton Maths Podcast always supply students with the answers - John Corbett during classwork, and Greg Ashman for homework. After all - and this is one of my alll-time favourite phrases - practice doesn't make perfect, practice makes permanent.
My favourite quote:
During a 3-hr baseball game, a batter may get only 5-15 pitches (perhaps one or two relevant to a particular weakness), whereas during optimal practice of the same duration, a batter working with a dedicated pitcher has several hundred batting opportunities, where this weakness can be systematically explored

Research Paper Title: Deliberate Practice and Performance in Music, Games, Sports, Education, and Professions: A Meta-Analysis (and for a critical view of deliberate practice, I would recommend this article from the New Yorker)
Author(s): Brooke N. Macnamara, David Z. Hambrick, and Frederick L. Oswald
My Takeaway:
Just when I was beginning to thin that Deliberate Practice was the answer to all my problems, I came across this paper. It is a meta-analysis covering all major domains in which deliberate practice has been investigated. The authors found that deliberate practice explained 26% of the variance in performance for games, 21% for music, 18% for sports, and less than 1% for professions. What was the figure for education?... wait for it... 4%! The authors conclude that deliberate practice is important, but not as important as has been argued. So, what are we to make of this? Firstly - and perhaps unsurprisingly - deliberate practice cannot explain all difference in performance between students. Factors such as their class teacher, school culture, measures of ability, and others are bound to play a role. But why is the effect size of deliberate practice so low? Well, the authors themselves offer a plausible suggestion - deliberate practice is not well defined in education. As far as I could tell from following up the papers mentioned in this meta-analysis, no-one has studied deliberate practice properly in education. By that I mean comparing two cohorts of students, with one tackling a topic such a simultaneous equations by working through questions as a whole, whilst the other works at individual components (identifying whether you add or subtract, manipulating algebraic expressions, solving linear equations, etc), practicing each component many times, getting immediate feedback, nailing each one before moving on to the next component, before finally putting it all together. For me, that is effective deliberate practice. That is more like the deliberate practice that is seen in sports, where the effect size is far greater. Until I see research related to that form of deliberate practice, I am not giving up on it just yet!
My favourite quote:
Why were the effect sizes for education and professions so much smaller? One possibility is that deliberate practice
is less well defined in these domains. It could also be that in some of the studies, participants differed in amount of prestudy expertise (e.g., amount of domain knowledge before taking an academic course or accepting a job) and thus in the amount of deliberate practice they needed to achieve a given level of performance.


Research Paper Title:
Educating the Evolved Mind: Conceptual Foundations for an Evolutionary Educational Psychology
Author(s): David C. Geary
My Takeaway:
This paper is brilliant. The author argues that there are two types of knowledge and ability: those that are biologically primary and emerge instinctively by virtue of our evolved cognitive structures, and those that are biologically secondary and exclusively cultural, acquired through formal or informal instruction or training. Evolution over millions of years has led to us having brains that eagerly and rapidly acquire those things that are biologically primary, whereas the brain has simply not has enough time to adapt to make biologically secondary knowledge and ability as easy to acquire. Much of what students learn in school is biologically secondary. For me, this has three major implications for teaching. The first is it goes someway to explain why students seem to enjoy speaking to their friends more than engaging in my lovely discussion of how to solve a quadratic equation. The first is biologically primary (folk psychology - interest in people), whereas the second is not. Secondly, it explains why subject such as mathematics should be explicitly taught and not "discovered" through minimal guided instruction - learning maths is simply not natural for humans and hence is too difficult on your own without the help of an expert. Thirdly, it does open up a potential limitation to the model of explicit instruction - we assume students will believe what they are told. When that involves something like how to calculate the percentage of an amount, or how to use trigonometry to find missing sides in right-angled triangles, this is probably fine as students are unlikely to hold any intuitive (i.e. biologically primary), contradictory beliefs. However, I am not convinced that is true for all topics. Take something like probability. Probability is filled with so many counter-intuitive results. The birthday paradox, and the Monty Hall problem are two classics that catch out everyone, but we've all seen students hold erroneous beliefs about the probability of getting a Head and a Tail when tossing two coins, or rolling a "lucky" 6 on a dice. Such results directly contradict the natural intuition held by students. Geary tells us that such biologically primary knowledge is not easy to shake off, so that simply being told you are wrong and what the correct answer is may not be enough. For topics such as these, students may well need to be convinced of the answer - for it is only when they are convinced that their deeply geld intuitions can be changed. Is this also true of misconceptions? Take a skill such as adding two fractions together. By instinct, students are likely to simply add the numerators and denominators. Now, this is perhaps not representative of a biologically primary piece of knowledge, but such misconceptions are so prevalent among students (how many Year 11s have you seen making that mistake despite 6 years of being taught it?), that once again they may need convincing as opposed to just being instructed. How do we convince them? It is not easy, but in short, we need to create cognitive conflict - a concept that will be discussed further in the next paper. Students need to see and believe the real results for themselves. We have traditional tools such as demonstration and mathematical proof, as well as more modern approaches such as Dan Meyer's 3 Act Math structure.
My favourite quote:
From an evolutionary perspective there are several key points: First, secondary learning is predicted to be heavily dependent on teacher- and curriculum-driven selection of content, given that this content may change across and often within lifetimes. Second, for biologically primary domains, there are evolved brain and perceptual systems that automatically focus children’s attention on relevant features (e.g., eyes) and result in a sequence of attentional shifts (e.g., face scanning) that provide goalrelated information, as needed, for example, to recognize other people. Secondary abilities do not have these advantages and thus a much heavier dependence on the explicit, conscious psychological mechanisms of the motivation-to-control model— Ackerman’s (1988) cognitive stage of learning—is predicted to be needed for the associated learning. Third, children’s inherent motivational biases and conative preferences are linked to biologically primary folk domains and function to guide children’s fleshing out of the corresponding primary abilities. In many cases, these biases and preferences are likely to conflict with the activities needed for secondary learning.

Research Paper Title: Numerical Cognition: Age-Related Differences in the Speed of Executing Biologically Primary and Biologically Secondary Processes
Author(s): David C. Geary and Jennifer Lin
My Takeaway:
Since reading the Geary paper above, the distinction between what is biologically primary and secondary with regard to mathematical skills has fascinated me. As well as providing an interesting study on the effect of age on the speed of mathematical computations, this paper addresses maths specific biologically primary and secondary skills.
1) Enumeration: Human infants and animals from many other species are able to enumerate or quantify the number of objects in sets of three to four items. This process that is termed subitizing and is defined as the ability to quickly and automatically quantify small sets of items without counting.  For the enumeration of larger set sizes, adults typically count the items or guess This likely reflects a combination of primary and secondary competencies. Primary features include an implicit understanding of counting, and secondary features include learning the quantities associated with numbers beyond the subitizing range. Thus, subitizing appears to represent a more pure primary enumeration process than does counting.
2) Magnitude Comparison: The speed of determining which of two numbers is smaller or larger becomes slower as the magnitude of the numbers increases, but becomes faster and less error prone as the distance between the two numbers increases. The most conservative approach is to assume that the limit of these innate representations is quantities associated with 1 to 3, inclusive.
3) Subtraction: It appears that human infants, pre-verbal children, and even the common chimpanzee (Pan troglodytes) are able to add and subtract items from sets of up to three (sometimes four) items. Although this primary knowledge almost certainly provides the initial framework for the school-based learning of simple addition and subtraction, most of the formal arithmetic skills learned in school appear to be biologically secondary. For example, effective borrowing is dependent on a conceptual understanding of the base-10 structure of the Arabic number system and on school-taught procedures.
Finally, it is fascinating to note the findings from this paper: Componential analyses of solution times suggested that younger adults are faster than older adults in the execution of biologically primary processes. For biologically secondary competencies, a pattern of no age-related differences or an advantage for older adults in speed of processing was found. In other words, the younger you are the quicker you are at biologically primary knowledge, but that relationship is not seen (or may even reverse) when it comes to biologically secondary. One conclusion from this might be to highlight the importance of how students are taught the biologically secondary skills.
My favourite quote:
The primary–secondary distinction also provides a means of considering the extent to which experiences in childhood, such as schooling, might influence cognitive performance in old age. The result of this and other studies (e.g., Bahrick & Hall, 1991) suggest that the development of strong academic competencies, that is, secondary abilities, in childhood can have important mitigating effects on any more general declines in cognitive performance with adult aging, perhaps by facilitating their use throughout the life span. This latter implication leads us to wonder about the arithmetical competencies of the current generation of younger American adults as they age.

Research Paper Title: An Evolutionary Upgrade of Cognitive Load Theory
Author(s): Fred Paas & John Sweller
My Takeaway:
We still have the delights of Cognitive Load Theory awaiting us in the next section, but I include this paper here as it adds an extra dimension to Geary's interpretation of biologically primary and secondary knowledge and the implications for teaching. Here the authors argue that working memory limitations may be critical only when acquiring novel information based on culturally important knowledge that we have not specifically evolved to acquire. Cultural knowledge is known as biologically secondary information, and as we have seen includes most of the maths students are taught at school. Working memory limitations may have reduced significance when acquiring novel information that the human brain specifically has evolved to process, known as biologically primary information. If biologically primary information is less affected by working memory limitations than biologically secondary information, it may be advantageous to use primary information to assist in the acquisition of secondary information. One interesting application to maths is “mindful movement”, defined as the use of body movements, for instance children forming a circle, for the purpose of learning about the properties of a circle. The use of mindful movement was expected to be particularly effective for children who are able to cooperate but are not yet capable of high-level verbal interaction. Research quoted found that, compared with the conventionally taught control group, the experimental group using mindful movement in cooperative learning obtained better results. Also, an explanation of the modality effect (see next section) is proposed: we may have evolved to listen to someone discussing an object while looking at it, hence why the process of describing an image or an animation is effective. We certainly have not evolved to read about an object while looking at it because reading itself requires biologically secondary knowledge.
My favourite quote:
The major purpose of this paper has been to indicate that biologically primary knowledge that makes minimal demands on working memory resources can be used to assist in the acquisition of the biologically secondary knowledge that provides the content of most instruction and that imposes a high working memory load. Evidence for this suggestion can be found in the collective working memory effect, the human movement effect and in embodied cognition through the use of gestures and object manipulation. The collective working memory effect indicates that our primary skill in communicating with others can be used to reduce individual cognitive load when acquiring secondary knowledge. The human movement effect demonstrates that we are able to overcome transience and the resultant cognitive load of animations if those animations deal with human motor movement because we may have evolved to readily acquire motor movement knowledge as a primary skill. The use of gestures and object manipulation are primary skills that do not need to be explicitly taught but can be used to acquire the secondary skills associated with instructional content.

Research Paper Title:
How Do I Get My Students Over Their Alternative Conceptions (Misconceptions) for Learning?
Author(s): Joan Lucariello and David Naff
My Takeaway:
In the papers above we were introduced to the potential difficulties of relying on explicit instruction to convince students to go against their biologically primary knowledge and intuition. This paper takes this further by looking at practical ways we may help students change the erroneous misconceptions they hold. The author provides a handy list of effective strategies for changing students' misconceptions, including asking them to write down their existing preconceptions of a topic before teaching it, and using diverse instruction to present a few examples that challenge multiple assumptions, rather than a larger number of examples that challenge just one assumption. These are all covered in detail, with practical strategies and supporting research. However, my main takeaway is related to concept of creating cognitive conflict. Cognitive conflict can lead to conceptual change or the accommodation of current cognitive concepts. The authors suggest two strategies for bringing about this cognitive conflict, both of which have direct relevance to mathematics:
1) Present students with anomalous data. This is data that does not accord with their misconception. An example from maths might be to get students to type in 1/3 + 1/4 on their calculator and let them see for themselves that the answer is not 2/7. If they have built up sufficient trust in their calculator over the years, this should be enough to induce cognitive conflict which will make them more amenable to hearing alternative approaches.
2) Present students with refutational texts. A refutational text introduces a common misconception, refutes it, and offers a new (alternative) theory that proves to be more satisfactory. An example might be: "some students think you can just add the numerators and the denominators when adding fractions together. You cannot. If you could, then 1/4 + 1/4 would be 2/8, which is just 1/4. We know 1/4 + 1/4 = 2/4, which shows us that the denominators do not get added together". What I like about this is it confronts the misconception head on, dismisses it with a reason, and then offers a more viable solution. This had really influenced the notes I give my students - no longer just explaining the answer, but confronting wrong answers as well. I feel this is important, because can you fully understand a concept unless you also know the possible misconceptions associated with it? The matter of misconceptions and how to deal with them is discussed further in the Formative Assessment section.
My favourite quote:
Alternative conceptions (misconceptions) interfere with learning for several reasons. Students use these erroneous understandings to interpret new experiences, thereby interfering with correctly grasping the new experiences. Moreover, misconceptions can be entrenched and tend to be very resistant to instruction. Hence, for concepts or theories in the curriculum where students typically have misconceptions, learning is more challenging. It is a matter of accommodation. Instead of simply adding to student knowledge, learning is a matter of radically reorganizing or replacing student knowledge. Conceptual change or accommodation has to occur for learning to happen.Teachers will need to bring about this conceptual change.

Research Paper Title: Cognitive conflict, direct teaching and student's academic level
Author(s): Anat Zohar and Simcha- Aharon Kravets
My Takeaway:
The suitability of cognitive conflict to overcome misconceptions is far from universally accepted, and research findings into its effectiveness are mixed. This piece of research suggests a reason for that, and with it an important - and logical - condition on when cognitive conflict is likely to be a useful way of helping students overcome their misconceptions. The study sought to compare the effectiveness of two teaching methods - Inducing a Cognitive Conflict (ICC) versus Direct Teaching (DT) for students of two academic levels (low versus high). 121 students who learned in a heterogeneous school were divided into four experimental groups in a 2X2 design. The key finding was that the ICC teaching method was more effective for high level students following a test of retention 5 months later, while the DT method was more effective for low level students. Specifically, high level students benefited from the ICC teaching method while the DT method delayed their progress. In contrast, low -level students benefited from the DT method while the ICC teaching method delayed their progress. Why would this be the case? Well two common reasons given for the apparent failure to demonstrate the effectiveness of cognitive conflict in the classroom are:
1) students often fail to reach a stage of meaningful conflict that requires a certain degree of both prior knowledge and reasoning abilities
2) students may not have an appropriate degree of motivation (goals, values and self - efficacy) that are potential mediators in the process of conceptual change
The authors speculate that students with low academic aptitudes and achievements tend to have a lower degree of prior knowledge, less advanced reasoning abilities and a lower degree of motivation than students with high academic aptitudes and achievements. These explanations thus suggest that as a group,students with low academic aptitudes and achievements will tend to benefit from instruction using cognitive conflict less then students with high academic aptitudes and achievements. Perhaps such instruction even obstructs the learning of low-achieving students compared to other teaching methods such as direct teaching. If this is correct, then it implies that academic ability/achievement is key determinant of the success of cognitive conflict. This makes logical sense if you consider the point that students cannot experience cognitive conflict without first understanding the related domain specific concepts involved in the method being demonstrated.
My favourite quote:
The findings confirm our hypothesis that inconclusive findings regarding the effectiveness of teaching with cognitive conflict may be caused by an interaction effect between students' academic level and teaching method.

Research Paper Title:
The Privileged Status of Story
Author(s): Daniel Willingham
My Takeaway:
This lovely paper is directly related to Geary's argument about biologically primary and secondary knowledge and skills above. I include it as a potential model for explicit instruction, as well as to show the explicit instruction is not simply lecturing to students in a dull, uninspiring way.  Here, Willingham argues that planning a lesson around the structure of a story is a good way not just to engage students but to help them learn and retain the content. He argues that stories are both easier to comprehend and easier to remember than if that information was presented another way. Why should this work? Well, to return to Geary, story-telling has existed in human societies since the evolution of language, whereas communicating in written form is a relatively new development, as is sitting in the whole school environment for that matter. Learning from stories may be viewed as more biologically primary than other forms of instruction. With this in mind, Willingham suggests the following key components of a story: causality, conflict, complications and character. How easy is it to structure a maths lesson around these components without it feeling forced, false and downright unbelievable to the students? Well, not as difficult as you might think. Causality is as simple as showing one concepts follows directly from another. Conflict can be presented as a problem we need to solve. Complications exist when our current mathematical tools let us down, or we encounter a surprising result. Character is perhaps a little more difficult, but searching the rich history of maths to see where the great ideas started from can breathe humanity into otherwise abstract ideas. Once again I return to Dan Meyer's 3 Act Math structure, and his amazing aspirin-headache approach, as possible tools to incorporate these key components, but I am convinced that it doesn't even need to be that complicated. Simply considering the structure of a story when planning lessons, possibly incorporating one or two of these components, may just lead to an experience that is easier to understand and more memorable for your students - and that, after all, is the goal of teaching.
My favourite quote:
Screenwriters know that the most important of the four Cs is the conflict. If the audience is not compelled by the problem that the main characters face, they will never be interested in the story. Movies seldom begin with the main conflict that will drive the plot. That conflict is typically introduced about 20 minutes into the movie. For example, the main conflict in Star Wars is whether Luke will succeed in destroying the death star, but the movie begins with the empire's attack on a rebel ship and the escape of the two droids. All James Bond movies begin with an action sequence, but it is always related to some other case. Agent 007's main mission for the movie is introduced about 20 minutes into the film. Screenwriters use the first 20 minutes—about 20 percent of the running time—to pique the audience's interest in the characters and their situation. Teachers might consider using 10 or 15 minutes of class time to generate interest in a problem (i.e., conflict), the solution of which is the material to be learned.

Research Paper Title: Cognitive Supports for Analogies in the Mathematics Classroom
Author(s): Lindsey E. Richland, Osnat Zur and Keith J. Holyoak
My Takeaway:
Related to the importance of a story structure to lessons discussed above comes the finding described in this paper that students respond particularly well to the use of analogies during instruction. The work by Daniel Willingham in the Cognitive Science section outlines how students connect new knowledge to existing knowledge in the formation of schema and how this is crucial to long-term learning, and hence it would appear to be good practice to use anaoliges wherever possible during instruction. The authors identified Six strategies involving analogies after studying maths lessons in the US, Hong Kong and Japan. Teachers:
(A) used a familiar source analog to compare to the target analog being taught;
(B) presented the source analog visually
(C) kept the source visible to learners during comparison with the target
(D) used spatial cues to highlight the alignment between corresponding elements of the source and target (e.g., diagramming a scale below the equals sign of an equation)
(E) used hand or arm gestures that signaled an intended comparison (e.g., pointing back and forth between a scale and an equation);
(F) used mental imagery or visualizations (e.g., “picture a scale when you balance an equation”)
Fascinatingly, each of these strategies were used less by teachers in the US than by teachers in the other two (higher performing) regions. The strategies seem obvious, and I guess I have always been doing parts of them, but knowing these strategies has made me focus on them more explicitly in my delivery of concepts. They all provide that extra support that might be crucial in the early stages of concept development. The researchers offer an explanation (see quote) as to why analogies are successful in maths instruction, and this supports the findings from Cognitive Load Theory. This may go some way to explaining the recent popularity of visual approaches such as Bar Modelling, which could potentially tap into more than one of the strategies outlined above. A word of caution though: unsuccessful analogies may produce misunderstanding that can even lead to nasty, lingering misconceptions. Choose your analogies wisely!
My favourite quote:
If the source analog is not familiar and not visible, then students may struggle with processing. First, students will need to perform a taxing memory search to understand the source. Then, assuming that memory retrieval is successful, lack of visual availability will place further burdens on working memory during production of the relational comparison. Finally, lack of supporting cues to guide the comparison itself may result in the student learning much less than, or something quite different from, the new relational concept the teacher means to convey.

Research Paper Title: Inflexible Knowledge: The First Step to Expertise
Author(s): Daniel T. Willingham
My Takeaway:
We have seen in the Cognitive Science section that experts and novices think in fundamentally different ways due to the more developed set of schema that the former possess. This allows experts to see the deeper structure of problems, and to make the sort of connections and provide the sort of the solutions that are out of the novices' grasp. This lovely paper by Daniel Willingham argues that in order to transform novices to experts we must first accept that they need to first develop inflexible knowledge. Willingham distinguishes between flexible,inflexible and rote knowledge. Knowledge is flexible when it can be accessed out of the context in which it was learned and applied in new contexts. Inflexible knowledge is meaningful, but narrow; it's narrow in that it is tied to the concept's surface structure, and the deep structure of the concept is not easily accessed. Rote knowledge is devoid of meaning. Willingahm's point is that we as teachers often think rote knowledge is bad (which it is), but most of what we consider as rote knowledge may in fact be inflexible knowledge, and the development of inflexible knowledge is a necessary step long the path to expertise. Take something like "angles on a straight line add to 180 degrees". The first time students encounter this, they are unlikely to understand why, and will probably only be able to answer questions that are presented in a straight-forward manner (focusing on the surface structure). Their knowledge still has meaning, albeit limited, hence their knowledge is inflexible not rote. However, the more questions they encounter, and the more situations they see this concept appear in (different shapes, algebraic, etc), then the more they will begin to appreciate the deeper structure and their knowledge becomes more flexible. For me, there are a few interesting takeaways from this:
1) It is fine - and indeed necessary - for students to develop inflexible knowledge of concepts
2) We aid the path to expertise by ensuring this knowledge is secure and then showing it in lots of different scenarios via carefully chosen examples
3) Sometimes it may be sensible to teach the "how" before the "why". This is somewhat controversial, and I first thought about this in my conversation with Dani Quinn on my podcast, but I think it makes sense. Go back to my angles example. Is there any point trying to convince students why the angles on a straight line add to 180 degrees when they first encounter the topic? Is it not better to get them fluent at the calculations, competent at answering questions with the same surface structure, confident in their abilities, and then later on return to the why when they have a greater appreciation of the concept? Relating this to Cognitive Loads Theory, my concern introducing the why and the how at the same time is that it is too much for the students to take in and they end up learning nothing.
My favourite quote:
There is a broad middle-ground of understanding between rote knowledge and expertise. It is this middle-ground that most students will initially reach and they will reach it in ever larger domains of knowledge (from knowing how to use area formulas fluently to mastering increasingly difficult aspects of geometry). These increasingly large stores of facts and examples are an important stepping stone to mastery. For example, your knowledge of calculating the area of rectangles may have once been relatively inflexible; you knew a limited number of situations in which the formula was applicable, and your understanding of why the formula worked was not all that clear. But with increasing experience, you were able to apply this knowledge more flexibly and you better understood what lay behind it. Similarly, it is probably expecting too much to think that students should immediately grasp the deep structure beneath what we teach them. As students work with the knowledge we teach, their store of knowledge will become larger and increasingly flexible, although not immediately.

Research Paper Title: Antagonism Between Achievement and Enjoyment in ATI Studies
Author(s): Richard Clark
My Takeaway:
This fascinating paper reaches an incredibly important conclusion: students often report enjoying the method from which they learn the least.  It appears that students make inaccurate judgments about the amount of effort they will have to expend to achieve maximum learning outcomes. Specifically, low ability students typically report liking more permissive instructional methods (such as inquiry based learning), apparently because they allow them to maintain a "low profile" so that their failures are not as visible. However, as we have seen in this section and will see again in the section concerned with Cognitive Load Theory, in order to experience maximum achievement low ability students actually require less permissive methods (such as explicit instruction) which lower the information processing load on them. Conversely, high ability students like more structured methods which they believe will make their efforts more efficient, and yet these lower load methods seem often to interfere with their learning, as we will see when we consider the expertise-reversal effect. However, high ability students actually seem to learn more from more permissive approaches (such as problem solving independently) which allow them to bring their own considerable skills to bear on learning tasks. As well as highlighting the differing needs of experts and novices that is a recurring theme throughout this page for me, this is another argument against letting students choose the type of activity that they life - their preferences are likely to be influenced by their levels of enjoyment, which are poor indicators of how much they are learning.
My favourite quote:
The reason for this antagonism between achievement and enjoyment may stem from a situation where students seem to enjoy investing less effort to achieve and inaccurately assess the effect of investing less effort on their subsequent achievement. They appear to make judgments based on their perceived efficiency. They will report enjoying methods which appear to them
to bring maximum achievement with less investment of time and work. The decision that one method is superior in efficiency may come from a mistaken judgment that they are familiar enough with a method to profit from it. It is the methods which students perceive to be more familiar, however, for which Berlyne (1964) would predict the greatest enjoyment scores. But again it is not dear whether students accurately assess the extent to which they actually are familiar with a method. It is possible that the manifest and nominal characteristics of a method may only seem familiar to students. Berlyne (1964) offers evidence that objectively complex or demanding stimuli are often perceived as subjectively simple by subjects who locate familiar features in a complex display and categorize the whole display as "familiar".




Cognitive Load Theory keyboard_arrow_up
Back to Top

Dylan Wiliam described Cognitive Load Theory as "the single most important thing for teachers to know", and I have to agree. I find it provides rationale and practical strategies for the principles of Explicit Instruction discussed in the previous section, and it has transformed the way I teach. Indeed, I have felt compelled (much to my wife's dismay) to write a talk entitled "How Cognitive Load Theory changed my life" as part of my "How I wished I'd taught maths" workshop - click here for more details. Cognitive Load Theory is focussed upon the conditions that make learning easier for students, and it is fascinating to look at this alongside Bjork's work in the Memory section which extols the advantages of making learning harder. For me, the matter is resolved by using the principles of Cognitive Load Theory to aid the early knowledge acquisition phase, removing all unnecessary load on working memory, and then utilising concepts such as spacing, interleaving, testing, variation and other desirable difficulties later in the learning process. In this section we look at key "effects" of the Cognitive Load Theory relating to worked examples, presentation of information, goal-free problems and the development of problem solving skills, and the practical implications they have for the classroom.

Before we dive into the research papers, Oliver Caviglioli has produced some amazing sketchnote summaries of the 2011 Cognitive Load Theory book, which really help illustrate the key ideas:
Chapters 1 and 2
Chapters 3 and 4
Chapters 5 and 6
Chapters 7 and 8
Chapters 9 and 10
Chapters 11 to 18
Complete Download

Also, this report from the New South Wales Centre for Education Statistics and Evaluation is just about the most clear and concise summary of Cognitive Load Theory that I have seen

Research Paper Title: Cognitive Architecture and Instructional Design (there is also a short review of the development of Cognitive Load Theory that gives a good summary of the key points: Story of a Research Programme)
Author(s): John Sweller, Jeroen J. G. van Merrienboer,  and Fred G. W. C. Paas
My Takeaway:
It is no exaggeration to say that Cognitive Load Theory has changed my life. Well, my teaching life, at least. It explains so many of the struggles and mistakes I have seen my students making over the years, and I am just fuming that it has taken me 12 years to discover it. But better late than never! This paper is an outstanding overview of Cognitive Load Theory, together with its related effects and implications for instructional design. Cognitive Load Theory is concerned with how cognitive resources are focussed and used during problem solving and learning. The key principle is that our working memories are severely limited, and that once a student's working memory reaches "cognitive load", no learning takes place. Indeed, it was a real eye-opener for me to discover that students could be working hard, and yet not actually be learning anything. What can be done about this? Well, as teachers we need to help reduce both the intrinsic and extraneous loads:

Intrinsic load - this is determined by the interaction between the nature of the learning tasks and the expertise of the learner. It depends on the amount of element interactivity in the tasks that must be learned. Something like learning the names of polygons has a relatively low element interactivity, as knowing the name of a pentagon is not necessary to know the name of an octagon - hence they can be learned separately. However, something like solving an equation has a relatively high element interactivity as you need to combine many dependent skills together to arrive at the answer. The degree of element interactivity also depends on the expertise of the learner, because what are numerous elements for a low-expertise learner may be only one or a few elements (i.e. chunks) for a high-expertise learner. As we have seen in previous sections, working memory capacity is severely limited (possibly being as low as 3 to 5 items), but the intrinsic load can be reduced by:
  • By ensuring students have sufficient knowledge, which can be organised in their long-term memory as schema. This allows students to work with a sizeable "chunk" of information (or schema) as if it was one item, which frees up capacity in working memory. As we have seen from the papers by Anderson and Willingham in the Cognitive Science section, knowledge matters!
  • Presenting complementary visual and auditory information together
Extraneous load - this is load that is not necessary for learning (indeed, it is unhelpful), and it typically results from badly designed instruction and the way material is presented to the learner. This extraneous load can be reduced by:
  • The use of goal free problems
  • Making use of worked examples
  • Carrying out completion tasks
  • The careful integration of related information
  • Reducing redundant information
In short, as teacher we need to take steps to focus students' limited working memory capacity on the things that matter, removing everything else, whilst at the same time ensuring students have sufficient background knowledge to better deal with the limitations of their fragile working memories. For an in-depth discussion with a maths teacher who regularly practices the lessons from Cognitive Load Theory, you can listen to my interview with Greg Ashman. The papers that follow in this section look in detail at the concepts introduced above and their implications for explicit instruction.

My favourite quote:
The design of practice and the organization and presentation of information is the domain of instructional designers. Although there are many factors that a designer may consider, the major thesis of this paper is that the cognitive load imposed by instructional designs should be the pre-eminent consideration when determining design structures. Limited working memory is one of the defining aspects of human cognitive architecture and, accordingly, all instructional designs should be analyzed from a cognitive load perspective. We argue that many commonly used instructional designs and procedures, because they were designed without reference to working memory limitations, are inadequate.

Research Paper Title: Cognitive Load Theory, Learning Difficulty and Instructional Design
Author(s): John Sweller
My Takeaway:
As well as providing an excellent overview of Cognitive Load Theory, this paper highlights the contrasting problem solving strategies of novices and experts, and thus introduces one of my favourite concepts: the Goal Free Effect. This has a big implication for teaching, especially when introducing new concepts to students. Goal-specific problems are those in which the top-level goals (i.e. final answer) can only be achieved by successfully completing the sub-goals (i.e. the steps leading up to it). When faced with goal-specific problems, novice learners tend to embark upon a means-end analysis. In essence, this means the student tries to juggle all the possible sub-steps that minimize the difference between the current state (the problem) and the end state (the goal). This is a backwards-thinking approach, and it is no surprise that it quickly overloads working memory. I know I have said this before, but it is worth repeating - even though students are working hard in such a scenario, their cognitive overload means they are not actually learning anything. Just as important is the fact that this means-end approach is incompatible with the development of schema, meaning that whilst the students may solve the specific problem at hand, they are unlikely to actually be learning. Learning is a change in long term memory, not just solving one specific problem. A relatively simple solution is to instead present students with ‘goal-free’ problems, which focuses attention upon working forward from the information present one step at a time, rather than trying to hold multiple possible steps in mind at once. A classic example in maths is instead of giving students a complex, multi-step question whose goal is to find the size of "angle x", instead ask them to find the size of as many angles as they can. Similar things can be done to trigonometry questions, or even when working with cumulative frequency diagrams. This breaks the process down and reduces the burden on novices' working memories. As will be discussed in the remaining papers in this section, once students become expert enough in a given topic, they are likely to have sufficient working memory capacity to dedicate to solving goal-specific problems. The key (and difficulty!) is judging when to introduce them, but in early skill acquisition, goal-free problems seem to be the way to go.
My favourite quote:
This means-ends procedure is a highly efficient technique for attaining the problem-goal. It is designed solely for this purpose. It is not intended as a learning technique and bears little relation to schemas or schema acquisition. In order to acquire an appropriate problem solving schema, students must learn to recognize each problem state according to its relevant moves. Using a means-ends strategy, much more must be done. Relations between a problem state and the goal state must be established; differences between them must be extracted; problem operators that impact favourably on those differences must be found. All this must be done essentially simultaneously and repeated for each move keeping in mind any subgoals. Furthermore, for novices, none of the problem states or operators are likely to be automated and so must be carefully considered. According to cognitive load theory, engaging in complex activities such as these that impose a heavy cognitive load and are irrelevant to schema acquisition will interfere with learning. 


Research Paper Title: Cognitive Architecture and Instructional Design
Author(s): John Sweller, Jeroen J. G. van Merrienboer,  and Fred G. W. C. Paas
My Takeaway:
I include this wonderful paper again as I find it the best for discussing the Worked Example Effect and the Completion Problem Effect.
 
Worked Example Effect
The Worked Example Effect, perhaps more than any other identified by Cognitive Load Theory, has had the most significant impact on my teaching. In short, the Worked Example effect attempts to explain the finding whereby learners who study worked examples perform better on test problems than learners who solve the same problems themselves. When I first came across this, I didn't believe it - surely you learn more from trying to solve problems than by reading the solution. But the key is in the "trying to solve problems". The rather counter-intuitive result is due to the fact that studying worked examples focuses all attention on the correct solution and procedure, reducing the extraneous load compared to a means-end analysis that the learner may embark upon if left to their own devices. This is likely to lead to the development and acquisition of all-important schema which will help the students transfer their knowledge to related contexts. I look at worked examples in greater detail in the sections on Making the most of Worked Examples and The Importance of the Choice of Examples.
 
Completion Problem Effect
There is an obvious danger when just presenting students with worked examples, as opposed to problems to solve, that they do not fully engage with the examples. As all the work has been done for them, where is there incentive to think? And, as we have seen, if students are not thinking then they cannot be learning. I have seen this many times myself - some students will pay little attention during the worked examples, and then (surprisingly!) get stuck when they are set to work on their own. This is where the simple idea of requiring students to complete various pieces of information and steps within a worked example comes into play. This could be as simple as creating a full worked solution and then Tipexing or deleting key sections for the students to complete, or even pausing whilst going through a worked example on the board, and asking students to predict (maybe using mini-whiteboards) what comes next. Completion problems provide a bridge between worked examples and conventional problems. The authors put it like thus: worked examples are completion problems with a full solution, and conventional problems are completion problems with no solution. A good progression might be to start with completion problems that provide almost complete solutions, and gradually work to completion problems for which all or most of the solution must be generated by the learners. This is something that will be addressed further in the Making the most of Worked Examples section.

My favourite quote:
There is considerable evidence that, compared to conventional problems, they decrease extraneous cognitive load, facilitate the construction of schemas, and lead to better transfer performance. In short duration studies, results indicated that completion problems are equally effective as worked examples intermixed with conventional problems. In studies of a longer duration, completion problems may better help learners to maintain motivation and focus their attention on useful solution steps that are available in the partial examples.

Research Paper Title:
Reducing Cognitive Load by Mixing Auditory and Visual Presentation Modes
Author(s): Seyed Yaghoub Mousavi, Renae Low, and John Sweller
My Takeaway:
This paper introduces both the Split-Attention Effect and the Modality Effect, both of which have significantly changed my teaching.

Split Attention Effect
We have seen that the worked-example effect occurs because worked examples reduce extraneous cognitive load, but there can be no guarantee that all worked examples appreciably reduce cognitive load. This is especially true if the worked example consists of a diagram, and a separate worked solution, neither of which are intelligible without the other. The learner must split their attention between the two forms of presentation, which increases the cognitive load. Hence, when presenting students with worked examples where a diagram is involved (for example, most geometry topics), the text solution should be carefully integrated within the diagram, and not separate from it. This will prevent students from having to deal with these two forms separately and reduce the extraneous cognitive load. Similarly, keeping all aspects of a question visible at the same time (i.e. without having to constantly turn the page over) will also help reduce this unnecessary load.

Modality Effect
To understand the importance of the modality effect we need to understand the components of working memory. The Central Executive acts a bit like a supervisor, The Phonological Loop deals with speech and sometimes other kinds of auditory information, the Visuo-Spatial Sketchpad holds visual information and the spatial relationships between objects, and the Episodic Buffer integrates new information with information already stored in long-term memory. They key point here is that working memory gets overloaded if too much information flows into one of these components, but we can use different components to aid processing. Hence, the capacity of working memory may be determined by the modality (auditory or visual) of presentation, and the effective size of working memory may be increased by presenting information in a mixed (auditory and visual mode) rather than in a single mode. There are two key implications of this for me.
1) Make use of complementary representations of concepts. This will be discussed more in the next paper.
2) When students are presented with written information, talking over the top of it can cause overload as both are processed by the auditory component (after text is read, it is processed as if it is being heard). The problem is made even worse if the text on the slide and the words I am saying out loud are the same, as here we have redundant information, and hence an example of the Redundancy Effect, which will be discussed further later in this section. So, the simple practice of putting up a slide of text and allowing the students the opportunity to read it in silence BEFORE reading it aloud or discussing it can reduce the modality effect by decreasing the extraneous load. For me, the biggest change this has made is encouraging me to shut up a little more. Previously I would start my students off on some problems, and then feel an incessant need to provide a running commentary over the top - "read the question carefully", "remember to show your working", etc. I thought I was being helpful, but in fact I was overloading my students' finite working memories as they were having to process my oral ramblings together with reading the text of the problems they were solving. In short, I was inhibiting their learning.

My favourite quote:
We began by indicating that basic research into the characteristics of working memory has suggested that this processing system is divided into at least two partially independent subprocessors: an auditory system devoted heavily to language and a visual system for handling images, including writing. Because both systems can be used simultaneously, limited working memory capacity might be effectively increased if information that must be stored or simultaneously processed is presented in a manner that permits it to be divided between the two systems, rather than processed in one system alone. As a consequence, informationally equivalent material that may be difficult to process in a purely visual manner may be more easily handled if it can be presented partially in both modalities.

Research Paper Title: The Instructive Animation: Helping Students Build Connections Between Words and Pictures in Multimedia Learning
Author(s): Richard E. Mayer and Richard B. Anderson
My Takeaway:
This paper further discusses the Modality Effect, but this time in the context of a dual-coding model, which will be discussed in greater detail in the next paper. The authors conducted an experiment in which students studied an animation depicting the operation of a bicycle tire pump or an automobile braking system, along with concurrent oral narration of the steps in the process, and tested their performance on both a retention test and a problem solving test against groups who had the animation alone, narration alone, or no instruction. On both tests, the group who had the animation alongside the oral narration performed the best. The authors conclude that these results are consistent with a dual-coding model in which retention requires the construction of "representational connections" and problem solving requires the construction of "representational and referential connections". The obvious implication for teaching from this paper is that pictures and words together can be more effective than pictures or words alone. If the oral description is there to support the comprehension of the animation, then it should benefit students by easing the strain placed on their working memories. Hence, I now make more use of carefully selected diagrams, as well as GIFs, Geogebra demonstrations and interactive Desmos graphs. For example, introducing circle theorems using something like this, combined with my oral narration over the top, has really helped my students grasp the key concepts. But it can be a fine balance, and we should be careful not to simply use a different form a multi-media for the sake of it. For example, in the Real Life Maths section, I discuss a paper where the use of video to support maths comprehension had a negative effect on performance as students assumed the medium was easier to understand and hence put less effort into their thinking. 
My favourite quote:
What makes an instructive animation? The results presented in this article demonstrate that animation per se does not necessarily improve students' understanding of how a pump or a brake works, as measured by creative problem solving performance. For example, in both experiments, students who received animation before or after narration were able to solve transfer problems no better than students who had received no instruction. In contrast, when animation was presented concurrently with narration, students demonstrated large improvements in problem-solving transfer over the no-instruction group. We conclude that one important characteristic of an instructive animation is temporal contiguity between animation and narration. We hypothesize that contiguity of words and pictures during instruction encourages learners to build connections between their verbal and visual representations of incoming information, which in turn supports problem-solving transfer.

Research Paper Title: Research‐Based Principles for Designing Multimedia Instruction
Author(s): Richard E. Mayer
My Takeaway:
The paper above offered just a small taste of the Cognitive Theory of Multimedia Learning, which has been developed over many years by Richard Mayer, and others. There are clear parallels to be drawn with Cognitive Load Theory in its representation of working memory and emphasis on the importance of reducing extraneous load, but is primarily focussed upon the design of instruction materials. The theory is based upon three key assumptions: 
1) dual channel assumption—people have separate channels for processing visual and verbal material
2) limited capacity assumption—people can process only a limited amount of material in a channel at any one time
3) active processing assumption—meaningful learning occurs when learners select relevant material, organize it into a coherent structure, and integrate it with relevant prior knowledge
The key principle in the Cognitive Theory of Multimedia Learning is The Multimedia Principle - people learn more deeply from words and graphics than from words alone, which was examined in the paper above. However, this is just the beginning, and this paper provides an excellent introduction into the other principles of this theory, all of which have direct practical relevance to the design and presentation of worked examples, demonstrations, worksheets, etc. Many are related to the effects of Cognitive Load Theory, but two that particularly stood out to me were:
The Signalling principle - People learn more deeply from a multimedia message when cues are added that highlight the organization of the essential material. Hence, finding ways to focus students' attention on the parts of examples that really matter is crucial.
The Coherence Principle - People learn more deeply from a multimedia message when extraneous material is excluded rather than included. The non-essential "fluff" I tend to put around examples (usually pathetic jokes) is only doing my students harm. Likewise, this also calls into question fun murder mystery style investigations. After all, we know from Willingham in the Cognitive Science section that students remember what they think about.
This whole paper is a fascinating read, and the theory itself provides a nice complement to the findings and recommendations from Cognitive Load Theory.
My favourite quote:
What makes an instructive animation? The results presented in this article demonstrate that animation per se does not necessarily improve students' understanding of how a pump or a brake works, as measured by creative problem solving performance. For example, in both experiments, students who received animation before or after narration were able to solve transfer problems no better than students who had received no instruction. In contrast, when animation was presented concurrently with narration, students demonstrated large improvements in problem-solving transfer over the no-instruction group. We conclude that one important characteristic of an instructive animation is temporal contiguity between animation and narration. We hypothesize that contiguity of words and pictures during instruction encourages learners to build connections between their verbal and visual representations of incoming information, which in turn supports problem-solving transfer.

Research Paper Title: Cognitive Architecture and Instructional Design
Author(s): John Sweller, Jeroen J. G. van Merrienboer,  and Fred G. W. C. Paas
My Takeaway:
I include this wonderful paper again as I find it the best for discussing the Redundancy Effect. In the past I had assumed that simply repeating the same information twice, but in a different form, would at worst have a neutral effect on learning. After all, what harm can it do? Moreover, surely it is good to be told something twice - giving students two opportunities to get it? But if that extra information is redundant - i.e. if students can infer all they need for the initial presentation - then I am likely to be imposing an unnecessary load upon students working memories. We must be careful to distinguish this from the split attention effect. Split-attention occurs when learners are faced with multiple sources of information that must be integrated before they can be understood. The individual sources of information cannot be used by learners if considered in isolation, hence the need for integration. The redundancy effect occurs when multiple sources of information are self-contained and can be used without reference to each other. This is because that redundant information is very difficult to ignore, and hence must be processed in students' limited working memories. The message for me is clear: if a concept cannot be understood without a second piece of information, then carefully integrate the two pieces of information together. If the second piece of information is not needed, then leave it out! This is especially true when information is presented in written form on a slide, and yet I feel the incessant need to also read it out to students. Redundant, inhibiting and incredibly annoying!
My favourite quote:
Redundancy is a major effect that should be considered seriously by instructional designers. A large range of experimental results indicate the negative consequences of including redundant material when designing instruction. We know of no experimental work demonstrating advantages of redundancy, and we suspect that such a result only could be obtained under conditions where one set of instructional materials was so poor that any redundant alternative would inevitably confer benefits.

Research Paper Title:
Teaching Complex Rather Than Simple Tasks: Balancing Intrinsic and Germane Load to Enhance Transfer of Learning
Author(s): Jeron J G Van Merrienboer, Liesbeth Kster and Fred Paas
My Takeaway:
So far the focus of Cognitive Load Theory has been on making thinking as easy for students in the sense that all their limited working memory capacity should be focused entirely on the thing we want them to think about to prevent cognitive overload occurring. In other words, as teachers we should try to:
1) eliminate unhelpful extraneous load via worked examples, the careful presentation of information and the use of goal free problems
2) reduce intrinsic load by helping students acquire the background knowledge necessary so that sub-components of a complex task are automated.
However, what if we reduce these two types of loads so much that thinking actually becomes too easy? This fascinating (and controversial!) paper addresses this issue by introducing the concept of Germane Load. This can be viewed as "good cognitive load", in that it directly contributes to learning. It does this by aiding the construction of cognitive structures and processes that improve performance. The authors of this paper found that whilst load reducing extraneous load is effective in producing high retention of the material, these techniques hinder the transfer of learning. They argue that there is a need to vary the conditions of practice and only give limited guidance and feedback in order to induce germane cognitive load and improve transfer. In other words, in order to improve learning (in particular the transfer of skills to new contexts), we need to make learning more difficult... but difficult in the right way! This is a concept similar to Bjork's fascinating idea of "desirable difficulties" that will be discussed at length in the Memory section.
The reason this paper is controversial is that one of the originators of Cognitive Load Theory, John Sweller, has distanced himself from the concept of germane load as he believes it makes his theory impossible to falsify. For example, assuming that the overall load is kept constant, a decrease in performance will be attributed to a rise in extraneous load that impairs germane cognitive processes. Conversely, if the performance increases it will be attributed to a germane load enhancement made possible by a drop in extraneous load.
What is my takeaway from all this? Well, I'll be honest - I am not 100% sure! Building in the concept of germane load might be making Cognitive Load Theory unnecessarily complicated. My take is this: during initial skill acquisition we need to ensure thinking is as focused and easy for the student as possible using all the principles we have discussed in the papers above. But, we need to ensure that thinking is not too easy. If students are cruising through lessons on autopilot, then their learning is unlikely to be deep, and learning without the ability to transfer it to new situations is not really learning at all. Of course, this is a fine balance, and will be covered in far more detail in the Memory sections.
My favourite quote:
In general, well-designed instruction should decrease extraneous load and optimise germane load, within the limits of total available capacity in order to prevent cognitive overload. However, this article is mainly about the situation that even after the removal of all sources of extraneous cognitive load, the element interactivity of the complex tasks is still too high to allow for efficient learning. Thus, it is about balancing intrinsic load, which is caused by dealing with the element interactivity in the tasks, and germane load, which is caused by genuine learning processes. The structure of our argument is as follows. First, we discuss research findings indicating that germane-load inducing instructional methods used for practicing simple tasks are not used for practicing complex tasks, at the cost of transfer of learning. Second, we argue that the element interactivity of learning tasks should be limited early in training to decrease their intrinsic load, so that germane-load inducing methods might be used right from the start of the training program.


Research Paper Title:
Cognitive Load during Problem Solving: Effects on Learning
Author(s): John Sweller
My Takeaway:
One huge question from all we have seen so far on Explicit Instruction and Cognitive Load Theory is: "how do we get our students to become good problem solvers?". This paper offers the first clue. Two of the main strategies involved in problem solving are:
1) Schema acquisition. This involves recognising similarities between novel and previously solved problems, and calling upon knowledge stored in long term memory to apply to the new situation.
2) Means-end analysis. This is a generic problem-solving strategy that we all possess, and it involves measuring your current state, evaluating how far you are from the solution state and then deciding which moves may get you closer.
For students to become good problem solvers they need to form mental schema from domain-specific knowledge which they can then apply to different situations. Unlike experts, novices lack the appropriate schema to recognise and memorise problem configurations. They set about solving problems by focusing on the detail and ignoring structure, embarking upon a means-end analysis. This would all be fine, apart from the claim in this paper that solving problems via such a strategy is itself is not an effective way for novices to develop these crucial mental schema. Why? Well, because trying to solve problems in this manner (problem-solving search via means-end analysis) is cognitively demanding. the learner has to maintain the following aspects of the problem in his or her mind: current problem state, goal state, differences between these two states, operators that reduce the differences between the goal state and the present state, and subgoals. This overloads working memory, and hence the mental schema are not developed. Indeed, even if you manage to solve the problem, you might not recall the solution method, and hence you might not actually learn anything from the process. In other words, during a means-end approach to solving a problem, local goals and relationships may swamp the more global relationships. My key takeaway from this is a rather big one - students may not learn key knowledge and procedures from problem solving. Sure, they mauy solve the problem, but they are unlikely to be able to solve a related one (and almost certainly not an unrelated one, because the idea of a generic "Problem Solving skill" is flawed, as we shall see in the Problem Solving section), and it is an ineffective way of teaching the fundamental skills and procedures required to solve the problem. Problem solving is not a learning device. Problem solving must come at the end of the process, after the necessary domain specific knowledge has been learned. So, what are the implications for the classroom? Well, firstly, students should not be exposed to complex problem too early in the learning process. Secondly, I believe there is little point going through lots of difficult exam questions in the hope students understand them and make connections between related questions. I have been there myself - going through a series of unrelated problems with a class of 30 Year 11s, successfully answering my prompt-filled questions, and happily nodding along when I ask them if they get it. But any apparent success from teaching problem solving to novices is likely to be just mimicry. The skills will not be transferred. This will be covered further in the Problem Solving section, but the key point is that if basic skills are not in place, then problem-solving search via means-end analysis suggests that there will simply not be enough capacity in working memory for students to develop the mental schema necessary to learn and transfer.
My favourite quote:
Most mathematics and mathematics-based curricula place a heavy emphasis on conventional problem solving as a learning device. Once basic principles have been explained and a limited number of worked examples demonstrated, students are normally required to solve substantial numbers of problems. Much time tends to be devoted to problem solving and as a consequence, considerable learning probably occurs during this period. The emphasis on problem solving is nevertheless, based more on tradition than on research findings. There seems to be no clear evidence that conventional problem solving is an efficient learning device and considerable evidence that it is not. If, as suggested here, conventional problems impose a heavy cognitive load which does not assist in learning, they may be better replaced by nonspecific goal problems or worked examples. The use of conventional problems should be reserved for tests and perhaps as a motivational device.

Research Paper Title: The Expertise Reversal Effect
Author(s): Sweller, J., Ayres, P. L., Kalyuga, S. & Chandler, P. A. 
My Takeaway:
So far all the talk has been of using Explicit Instruction and the key features of Cognitive Load Theory as the best method of teaching students to become fluent in the facts and procedures they will need to learn more complex skills. Specifically, in early skill acquisition, learning from worked examples is very advantageous, and learning by solving problems is not. However, as this paper describes, instructional techniques that are highly effective with inexperienced learners (novices) can lose their effectiveness and even have negative consequences when used with more experienced learners (experts), hence the Expertise Reversal Effect. The argument is that worked examples contain information that is easily determined by the more experienced learners themselves and, therefore, can be considered redundant. As we have seen via the Redundancy Effect, devoting working memory to redundant information effectively takes away a portion of the learners’ limited cognitive capacity that could be devoted to the more useful germane load. Moreover, this redundant information may even interfere with the schemas constructed by experienced learners, preventing them from seeing the deeper connections in problems that are essential for transfer. For example, what if students have solved a problem differently to how I have presented it in thew worked example? At this stage of development, working through complex problems independently is likely to be more beneficial for long-term learning than studying worked examples. Of course, one major difficulty of this is recognising when students have made the transition from novice to expert and hence can start to be exposed to more complex problems. It is a delicate balancing act! What I have started doing is making worked examples "optional" for students once we have covered the basics of a topic. That way they can judge themselves whether they are at the stage where using worked examples will help or hinder them.
My favourite quote:
When a problem can be solved relatively effortlessly, analyzing a redundant worked example and integrating it with previously acquired schemas in working memory may impose a greater cognitive load than problem solving. Under these circumstances, practice in problem solving may result in more effective learning than studying worked examples because solving problems may adequately facilitate further schema construction and automation


Student Self-Explanationskeyboard_arrow_up
Back to Top

Previously, I thought the main benefit for getting students to explain their thinking was for me, so I could get a better sense of their levels of understanding of a given topic or concept. However, it seems it is so much more than this. According to Chi (see first paper), the term self-explaining refers to “the activity of generating explanations to oneself, usually in the context of learning from an expository text. It is somewhat analogous to elaborating, except that the goal is to make sense of what one is reading or learning, and not merely to memorize the materials (as is often the case when subjects in laboratory experiments are asked to elaborate). In this sense, self-explaining is a knowledge-building activity that is generated by and directed to oneself”. Hence, we have something similar to the Testing Effect, whereby self-explaining can actually cause learning, and is not just an indication of it. Likewise, in the language of Cognitive Load Theory, the act of self-explaining is likely to increase cognitive load (as it is cognitively demanding), but in a way that contributes to schema acquisition, and hence may be described as Germane Load.  this section I survey the research into student self-explanations and look at practical ways we can make use of it in the classroom.

Research Paper Title: Self-explaining: The dual processes of generating inferences and repairing mental models
Author(s): Michelene T.H. Chi
My Takeaway:
Whilst not maths specific like some of the other papers in this section, this paper is an excellent starting point into the world of student self-explanations.  According to Chi, the self-explanation effect is a dual process - involving two key elements:
1) Generating inferences - this involves the learner inferring information that is missing from a text passage or an example’s solution
2) Repairing the learner’s own mental model - here it is assumed that the learner engages in the self-explanation process if he or she perceives a divergence between his or her own mental representation and the model conveyed by the text passage or example’s solution.
According to Chi  “each student may hold a naive model that may be unique in some ways, so that each student is really customizing his or her self-explanations to his or her own mental model”. Chi is careful to point out that self-explaining is different from talking to or explaining something to others, which is something I had not considered before. The focus in self-explaining is simply to understand or make sense of something, while the purpose of talking or explaining to others is to convey information to them. Talking or explaining to others adds the requirement to the learner of monitoring the listener's comprehension, which might prevent the learner from acquiring the knowledge if cognitive load becomes a problem. It is reasonable to assume that the cognitive capacity that is taxed through talking may hinder the learner from engaging in critical self-explaining behaviours. This has made me much more selective in my use of "convince the person next to you", suggesting this should only be used one students have engaged in a period of self-explanation to convince themselves. As well as providing plenty of evidence in support of the power of student self-explanations (findings which are matched by other papers in this section), this paper also confronts a questions I have long had: what if students’ self explanations are wrong? Obviously, assessing a class-full of answers alone is quicker and easier than assessing explanations, so if it is not practical to assess explanations, and if incorrect explanations are damaging to learning, then we should not bother with them. Interestingly, Chi argues that if anything incorrect student self-explanations may actually be beneficial to learning! The reason: because they are likely to provide an opportunity for cognitive conflict later in the learning process.  I am a little dubious about this, and I always strive to elicit and discuss student explanations via the process of Formative Assessment. However, it is somewhat reassuring that incorrect explanations may not be as damaging as one might think, and the benefits appear to significantly outweigh the costs.
My favourite quote:
In conclusion, self-explaining seems to be an effective domain-general learning activity. If psychologists and educators had heeded Ben Franklin’s wise remarks, we would not have had to waste our time studying learning (as measured by remembering and forgetting) in the context of telling and teaching. We should have known that we needed to focus on involvement, a form of which is self-explaining, in order to achieve learning. However, perhaps Franklin could have gone one step further, and added, “Challenge me and I understand.”

 
Research Paper Title: Microgenetic studies of self-explanation
Author(s): Robert Siegler
My Takeaway:
This paper gives a lovely introduction to Siegler’s Overlapping Waves Theory of learning, relating it to the importance of student self-explanation. In short, the overlapping waves theory states that individuals know and use a variety of strategies which compete with each other for use in any given situation. With improved or increased knowledge, good strategies gradually replace ineffective ones. However, for more efficient change to occur, learners must reject their ineffective strategies, which can only happen if they understand both that the procedure is wrong and why it is wrong (i.e. which problem features make the strategy inappropriate). Indeed, the author suggests one contributing factor to the high levels of maths performance of Japaneses students compared to their US and English counterparts is the emphasis on on generating explanations for why mathematical algorithms work. In Japanese classrooms, both teachers and students spend  considerable time trying to explain why solution procedures that differ
superficially generate the same answer, and why seemingly plausible approaches yield incorrect answers. Encouraging children to explain why  the procedures work appears to promote deeper understanding of them  than simply describing the procedures, providing examples of how they  work, and encouraging students to practice them — the typical approach to mathematics instruction in the US.
The paper provides comprehensive research to reach six fascinating conclusions related to the importance of self-explanation:
1) Encouragement to explain other people's statements is causally related  to learning;
2) Five-year-olds as well as older children can benefit from encouragement to explain;
3) Explaining other people's answers is more useful than explaining your own, at least when the other people's answers are consistently correct and your own answers include incorrect ones;
4) Variability of initial reasoning is positively related to learning;
5) Explaining why correct answers are correct and why incorrect answers are incorrect yields greater learning than only explaining why correct answers are correct;
6) The mechanisms through which explaining other people's reasoning exercises its effects include increasing the probability of trying to explain observed phenomena; searching more deeply for explanations when such efforts are made; increasing the accessibility of effective strategies relative to ineffective ones; and increasing the degree of engagement with the task.
My favourite quote:
One way in which encouragement to explain exercises its effects is  to increase the probability of the learner seeking an explanation at all.  When people are told that an answer is wrong, they often simply accept  the fact without thinking about why it is wrong or how they might generate correct answers in the future. The number conservation data provide evidence regarding this source of effectiveness. Children who were told that their answer was wrong and which answer was right, but who were not asked to explain why the correct answer was correct, did not increase the accuracy of their answers over the course of the four sessions. In contrast, children who received the same feedback but who also were asked to explain how the experimenter had generated the correct answer, did increase their accuracy. Further, those children who showed the largest increases in successfully explaining the experimenter's reasoning also showed the largest increases in generating correct answers on their own. Thus, encouragement to generate self-explanation seems to work partially through encouraging children to try to explain observed outcomes.
 
Research Paper Title: Promoting Transfer: Effects of Self-Explanation and Direct Instruction
Author(s): Bethany Rittle-Johnson
My Takeaway:
This study looked at third- through fifth-grade students (ages 8 – 11) learning about mathematical equivalence under one of four conditions varying in (a) instruction on versus invention of a procedure and (b) self-explanation versus no explanation. The results found that both self-explanation and instruction helped children learn and remember a correct procedure, and self-explanation promoted transfer regardless of instructional condition. Neither manipulation promoted greater improvements on an independent measure of conceptual knowledge. Analysing the results of their study, the authors suggest the following benefits of self-explanation:
1) Self-explanation aided invention of new problem-solving approaches. Children who self-explained in the intervention condition were more likely to invent at least one correct procedure, and those in the instruction condition were more likely to invent a second correct procedure, compared with children who did not self-explain.
2) Self-explanation broadened the range of problems to which children accurately applied correct procedures. This is crucial. When people learn new ideas, they often use them on an overly narrow range of problems. Recognising
the range of problems to which an approach applies, regardless of changes in surface features, is a critical component of learning and development
3) Self-explanation supported the adaptation of procedures to solve novel problems that did not allow rote application of the procedure. Solving some of the transfer problems required considerable insight into the rationale behind the procedure, and self-explanation supported such flexible adaptations
4) Self-explanation supported retention of correct procedures over a 2-week delay. There were no effects of test time in any analysis, indicating that the benefits of self-explanation observed immediately after the intervention were maintained on the delayed posttest
Hence, once more we have support for getting students to explain their answers, and an extra notch on rhe bed-post for direct instruction versus inquiry.
My favourite quote:
Prompts to self-explain seem to facilitate transfer equally well under conditions of invention or instruction,and these benefits persist over a delay. A growing body of research indicates that there is indeed a time for telling;invention is not necessary for children to be productive and adaptive. What may be necessary is for people to engage in effective cognitive processes, such as generating self-explanations.
 
Research Paper Title: An effective metacognitive strategy: learning by doing and explaining with a computer-based Cognitive Tutor
Author(s): Vincent A.W.M.M. Aleven and Kenneth R. Koedinger
My Takeaway:
This is another study which supports the view that students’ self-explanations are a key component to improving learning, specifically looking at its application for mathematics and the use of technology to support this. The authors investigated whether self-explanation can be scaffolded effectively in a classroom environment using a Cognitive Tutor, which is intelligent instructional software that supports guided learning by doing. Basically, students answer a question and then explain their problem-solving steps by selecting from a menu the name of the problem-solving principle that justifies the step. In two classroom experiments, the authors found that students who explained their steps during problem-solving practice with a Cognitive Tutor learned with greater understanding compared to students who did not explain steps. Students who were promot to give self-explanations better explained their solutions steps and were more successful on transfer problems. The authors interpret these results as follows: “by engaging in explanation, students acquired better-integrated visual and verbal declarative knowledge and acquired less shallow procedural knowledge”. Of course, this study relied on computer-based software (and indeed, reading this paper was one of the key reasons I brought the student explanation aspect into my Diagnostic Questions website), but the principle of getting students to explain each of their solution steps can be done on paper as well as it can on a screen. For those students who struggle to come up with explanations, or provide shallow ones, a key principle from this study could be borrowed - a list of relevant explanations could be placed on the bard for students to choose from. These could be topic specific, such as angle facts (“corresponding angles are equal”), or steps for solving equations (“divide both sides by…“). As well as supporting students, this would have the added advantage of making the explanations far easier to mark and assess.
My favourite quote:
A surprising finding is the fact that self-explanation is effective even when students are asked only to name the problem-solving principles that were involved in each step, but not to state the problem-solving principle or to elaborate how it applies to a problem. Perhaps equally surprising is the fact that self-explanation was scaffolded effectively by a computer tutor. Thus, other aspects of self-explanation that have been hypothesized to be crucial, such as the presence of a human instructor or the fact that students explain in natural language (as opposed to a structured computer interface) are not a necessary condition for effective support. It is an interesting question whether these aspects are even relevant. For example, will students learn more effectively when they are asked to provide more complete explanations or state explanations in their own words? This is a focus of our on-going research.
 
Research Paper Title: Is self-explanation worth the time? A comparison to additional practice
Author(s): Katherine L. McEldoon, Kelley L. Durkin, and Bethany RittleJohnson
My Takeaway:
We have seen the benefits of students self-explanation throughout the papers in this section, but there is no doubt that it takes longer to explain and answer a question than just to answer it. When you combine this observation with the clear benefits of regular practice, is begs the obvious question: Is self-explanation worth the time, or should we just get our students to practice more instead? This paper seeks to provide an answer. The authors compared the effectiveness of self-explanation prompts to the effectiveness of solving additional practice problems (to equate for time on task) and to solving the same number of problems (to equate for problem-solving experience). The authors found that compared to the control condition, self-explanation prompts promoted conceptual and procedural knowledge, hence once again confirming the benefits of student self-explanation. However, compared to the additional-practice condition, the benefits of self-explanation were more modest and only apparent on some subscales. The findings suggest that self-explanation prompts have some small unique learning benefits, but that greater attention needs to be paid to how much self-explanation offers advantages over alternative uses of time. So, we are left without a clear answer, so I must conclude that a balance is needed. We cannot afford to miss out on the benefits of self-explanation, but we also need to ensure students gain sufficient practice. Perhaps the answer is to focus on self-explanation in class time, where the teacher has a more important role in evaluating the students’ explanations, teasing out the meaning and correcting where needed. Whereas, depending on the nature of the problems, practice could be done at home with the students being supplied with the answers so misconceptions do not get reinforced.
My favourite quote:
The findings suggest some small, unique benefits of self-explanation relative to an alternative use of time. At the same time, it is important to consider potential benefits of this alternative - supporting additional practice, particularly on unfamiliar problems. Both activities are constructive learning activities, as they each require responses that go beyond what is provided in the original material. Both the self-explanation prompts and additional practice provided more opportunities for thinking about correct procedures (describing or implementing them) than the control condition. In turn, this should strengthen a procedure’s memory trace and related relevant knowledge, increasing the likelihood that the procedure will be selected in the future. Consequently, self-explanation prompts and additional practice can both provide opportunities for students to improve their knowledge, although there may be some benefits specific to self-explanation prompts.
 
Research Paper Title: Learning from Worked-Out Examples: A Study on Individual Differences
Author(s): Alexander Renkl
My Takeaway:
This is a key paper both to end our look at self-explanations and to tie this in with how we can really make the most of worked examples that will be the focus of the next section. Renkl’s research found two types of successful self-explaining strategies:
1) Anticipative Reasoners. These are learners who tended to self-explain by anticipating the next step in an example solution, then checking to determine whether the predicted step corresponded to the actual step. Crucially, these learners tend to have high levels of relevant prior knowledge.
2) Principle-based explainers. These learners tended to identify the essential meaning of a problem by attempting to articulate its goal structure—including the application of operators—while also elaborating on the principle that the example was intended to convey. Learners who adopted this strategy tended to have low prior knowledge.
However, perhaps the most important finding was that the majority of learners do not spontaneously engage in successful self-explanation strategies. This is crucial, as there is a danger in assuming that any spare working memory capacity will be used up for things that contribute towards learning (i.e. germane load, in the language of Cognitive Load Theory). If this is not the case, and students do not naturally engage in self-explanations, then their learning will not be as effective as it could be. This has huge implications for the classroom, and suggests that we should prompt students to provide self-explanations during instruction. Taking this together with Renkl’s other findings implies that these prompts should differ depending on the prior knowledge level of the students we are working with. Hence we should attempt to elicit principle-based explanations to learners with low prior knowledge while encouraging anticipative reasoning to learners with high prior knowledge. This important finding will be discussed further in the section on Making the most out of Worked Examples.
My favourite quote:
The finding that more than half of the subjects had to be assigned to the group of unsuccessful learners, reaffirms research findings that learners, left to their own devices, typically fail to show effective learning behaviors when no external support (e.g., teacher guidance or scaffolding) is present



Worked Examples: Making the most of themkeyboard_arrow_up
Back to Top

It was my reading of Cognitive Load Theory that first prompted my appreciation of the power of worked examples. Understanding that studying worked examples could be more effective for learning than solving problems blew my mind and prompted me to read all I could about how to make the worked examples I used in class as effective as possible. This section provides a summary of what I found.

Research Paper Title: The Contributions of Studying Examples and Solving Problems to Skill Acquisition
Author(s): J. Gregory Trafton and Brian J. Reiser
My Takeaway:
The first paper in this section has two major findings which, whilst they may seem obvious, are both of crucial importance. The researchers conducted experiments designed to answer two key questions they had about the most effective way to study from worked examples.
Does separating source examples from target problems hamper learning?
The researchers conducted an experiment comparing participants who solved a target problem immediately after the source example (Alternating Example) versus those  who studied a block of source examples followed by a block of solving target problems (Blocked Example). Subjects who solved problems interleaved with examples took less time on the target problems than subjects who studied a block of source examples and a block of target problems. Crucially, participants in the Alternating Example condition also submitted more accurate solutions than subjects receiving blocked examples.
Is solving sources better than studying examples if the examples are not accessible during subsequent problem solving?
This is a key question. There is a danger that we can get too caught up in the power of examples, and think that students do not need to solve any problems at all. However, the researchers found that participants who attempted to solve problems performed better than those who merely studied worked examples. As we shall see in the sections on Testing, it is this retrieval process induced when answering a question that is so important to learning, and if students are never compelled to access their memories of those examples, then they will never benefit from, what we will come to call the Testing Effect. Hence, we can conclude that subsequent problem solving appears to be required to derive the full benet of studying examples
My favourite quote:
In summary, studying examples is clearly a very effective method to improve learning. In order for an
example to be most effective, however, the knowledge gained from the example must be applied to solving a new problem. The most efficient way to present material to acquire a skill is to present an example, and then a similar problem immediately following. We hypothesize that this presentation method allows subjects to construct rules that are general enough to work for both the example and the rule. Although the extra practice solving sources may speed target problem solving, apparently more effective problem solving rules are formed when target problem solving can be guided by an accessible source example.

 
Research Paper Title: Learning from Examples: Instructional Principles from the Worked Examples Research
Author(s): Robert K. Atkinson , Sharon J. Derry , Alexander Renkl , Donald Wortham
My Takeaway:
There are so many fascinating concepts discussed throughout this paper extolling the virtues of worked examples, and when they are most effective. Concepts such as the split-attention effect, and variability of problem types are also discussed, which we have covered in the Cognitive Load Theory section. The quote I have chosen for this paper summarises the main findings nicely, and serves as an excellent set of guidelines for the use of worked examples during lessons. I am going to focus on one that has directly changed the way I teach, and which is directly related to the finding from the paper above: Example-Problem Pairs. I used to do a load of worked examples at the start of the lesson, and then give the students a set of problems to work on for the remainder of the lesson. The problem (and it seems so obvious now) was that by the time I had done the 3rd worked example, students had forgotten the first as they had not had the opportunity to practice it for themselves. Hence, they would start working on the questions, get stuck, and I would need to go through it all again. This paper recommends interleaving worked examples with related questions for students to solve alone. When doing a worked example in class, I now split my board in two, having the worked example on the left, and a mathematically similar example for my students to try themselves immediately afterwards on the right. It sounds so simple, but the positive effects have been quite startling. Greg Ashman discussed this strategy when describing how he plans his lessons when I interviewed him for my podcast, and he has written a blog post all about his use of Example-Problem Pairs here.
My favourite quote:
First, transfer is enhanced when there are at least two examples presented for each type of problem taught. Second, varying problem sub-types within an instructional sequence is beneficial, but only if that lesson is designed using worked examples or another format that minimizes cognitive load. Third, lessons involving multiple problem types should be written so that each problem type is represented by examples with a finite set of different cover stories and that this same set of cover stories should be used across the various problem types. Finally, lessons that pair each worked example with a practice problem and intersperse examples throughout practice will produce better outcomes than lessons in which a blocked series of examples is followed by a blocked series of practice problems.
 
Research Paper Title: Design-Based Research Within the Constraints of Practice: AlgebraByExample
Author(s): Julie L. Booth, Laura A. Cooper, M. Suzanne Donovan, Alexandra Huyghe, Kenneth R. Koedinger & E. Juliana Paré-Blagoev
My Takeaway:
We have seen the benefits of learning by worked example in our study of Cognitive Load Theory, in particular with the example-problem pairs approach. We have also seen the benefits of students explaining their thinking in our look at Student Self-Explanations. So, what happens when you combine these two powerful findings together? This paper seeks to demonstrate. The study in question came about from a challenge to identify an approach to narrowing the minority student achievement gap in Algebra 1 without isolating minority students for intervention. They attempted to do this by designing and testing 42 Algebra 1 assignments with interleaved worked examples that targeted common misconceptions and errors. The worked examples contained three parts:
1) the worked example
2) a section to reflect on what certain aspects of the worked meant, and how and why they were carried out that way
3) a related problem to complete
Hence, it is the middle section - the opportunity for self-explanation - that differentiates this approach from the example-problem pair approach. Notice also how the students are compelled to self-explain, which seems important given the findings from the Renkl paper in the Self-Explanations section that the majortiy of studetns do not spontaneously use self-explanation strategies. The results were impressive. The approach  provides a boost in performance, with the greatest impact on students at the lower end of the performance distribution. On the researcher-designed assessment of conceptual understanding, treatment students in the lower half of the performance distribution outscored comparable control students by approximately 10 percentage points. Treatment students overall scored 7 percentage points higher on a test composed entirely of released items from the state standardized tests, and 5 percentage points higher on the conceptual posttest. Procedural posttest scores were also 4 percentage points higher in the treatment group, even though control students had twice the practice solving problems on the assignments. As a result of this, together with the related research into the power of self-explanations, I have added that third element to my example-problem pair approach.
My favourite quote:
Of equal significance, AlgebraByExample achieved these gains with an intervention that meets all the constraints imposed by the districts: it targets all students, it can be used with any existing Algebra 1 curriculum, and it is an asset that teachers can use with minimum disruption to their practice. Moreover, study teachers reported that students using AlgebraByExample required less teacher support to complete assignments than those using control assignments, and reported rethinking their own practice in response to students’ positive experiences with worked examples.
 
Research Paper Title: Using example problems to improve student learning in algebra: Differentiating between correct and incorrect examples
Author(s): Julie L. Booth, Karin E. Lange, Kenneth R. Koedinger, Kristie J. Newton
My Takeaway:
This study takes worked examples with student self-explanations to the next level by considering the inclusion of incorrect examples. This is something I have always been wary of. I can see the benefits of non-examples when it comes to understanding factual definitions in maths (such as “what is a polygon?”), but is there a danger that exposing students to non-procedures (i.e. incorrect worked examples) will lead to them developing misconceptions? The authors cite findings from previous research that outline two reasons why the inclusion of incorrect examples might be of benefit to learning. First, they can help students to recognise and accept when they have chosen incorrect procedures, leading to improved procedural knowledge over practice alone or correct examples plus practice. Second, and perhaps more important, it can draw students’ attention to the particular features in a problem that make the procedure inappropriate. Therefore a combination of correct and incorrect examples is beneficial because the incorrect examples help to weaken faulty knowledge and force students to attend to critical problem features (which helps them not only to detect and correct errors, but also to consider correct concepts), while the correct examples provide support for constructing correct concepts and procedures, beyond that embedded in traditional instruction. In their study, the authors conducted two experiments, both focussed on the Algebra 1 course take in high-schools in the US. They sought to answer two questions:
1) Do worked examples with self-explanation improve student learning in Algebra when combined with scaffolded practice solving problems? The results were a resounding “yes”.
2) Are there differential effects on learning when students explain correct examples, incorrect examples, or a combination thereof? Results indicated that students performed best after explaining incorrect examples; in particular, students in the Combined condition gained more knowledge than those in the Correct only condition about the conceptual features in the equation, while students who studied only incorrect examples displayed improved encoding of conceptual features in the equations compared with those who only received correct examples.
The authors have a really nice way of summarising this key finding: “This finding is especially important to
note because when examples are used in classrooms and in textbooks, they are most frequently correctly solved examples. In fact, in our experience, teachers generally seem uncomfortable with the idea of presenting incorrect examples, as they are concerned their students would be confused by them and/or would adopt the demonstrated incorrect strategies for solving problems. Our results strongly suggest that this is not the case, and that students should work with incorrect examples as part of their classroom activities.”
My favourite quote:
Our results do not suggest, however, that students can learn solely from explaining incorrect examples. It is important to note that all students saw correct examples, regardless of condition, not only because they are regularly included in textbooks and classroom instruction, but because the correctly completed problems the students produced with the help of the Cognitive Tutor could also be considered correct examples of sorts. We maintain that students clearly need support for building correct knowledge, however, if that support is coming from another source (e.g., guided practice with feedback), spending additional time on correct examples may not be as important as exposing students to incorrect examples.
 
Research Paper Title: Learning from Worked Examples: What happens if mistakes are included?
Author(s): Cornelia S Grosse and Alexander Renkl
My Takeaway:
I include this paper as a word of caution before getting too carried away with exposing students to incorrect worked examples. The authors found that learning from worked examples where errors are included can enhance learning and transfer, but only if students have good prior knowledge of the topic. This makes perfect sense. Firstly, if students do not understand the topic, then how are they to spot the mistakes? Secondly, in the context of Cognitive Load Theory, if their understanding is not secure, then their working memories are likely to become overloaded whilst searching for the right and wrong answers simultaneously. This finding is supported by a paper from Große, C. S., & Renkl, A (behind a pay-wall) which found that relatively novice learners cannot benefit from incorrect examples when they are expected to locate and identify the error in the example themselves. Again, It makes sense that novice students would have difficulty with this component, given that they likely make many of the mistakes themselves and may not recognize them as incorrect. So what are we to make of this? It seems clear that students need a certain amount of knowledge of a topic in order to benefit from exposure to mistakes in worked examples. If they do not know what is right, how can they know and explain what is wrong? Hence, I would be leaning towards correct worked examples combined with student explanation in initial skill acquisition, moving onto clearly labelled incorrect worked examples combined with student explanation once students begin to get familiar with the concepts. 
My favourite quote:
Learning with incorrect examples poses challenging demands on the learners. They have to represent not only the correct solution in their working memory, but also the incorrect step with an explanation why it is wrong. Learners with low prior knowledge who cannot form larger chunks for information coding can easily be overtaxed.
 
Research Paper Title: Structuring the Transition From Example Study to Problem Solving in Cognitive Skill Acquisition: A Cognitive Load Perspective
Author(s): Alexander Renkl and Robert K. Atkinson
My Takeaway:
This paper provides a challenge to the Example-Problem Pair approach heralded by the papers above. The authors propose that instead of providing students with a complete worked example, followed by a problem for them to solve, followed by a complete worked example, followed by  problem to solve, etc (i.e. the Example-Problem Pair approach), instead a fading procedure is more effective. This involves providing a complete worked example, and then following this up with an almost complete worked example but with one step removed that students need to complete. There are obvious parallels to be drawn here with the Completion Effect noted in the Cognitive Load Theory section. The researchers two things that are of particular relevance to this section:
1) The fading procedure produced reliable effects on near-transfer items but not on far-transfer items. Near-transfer problems have the same deep structure but different surface structures. In other words, the strategy to solve them is exactly the same, but the context is different. Far-transfer problems have both a different context and a different deep structure. In other words near transfer is the acquisition of relatively simple rules, whereas far transfer is more a measure of understanding.
2) It was more advantageous to fade out worked- out solution steps using a backward approach by omitting the last solution steps first instead of omitting the initial solution steps first (i.e., a forward approach).
So, we have an alternative to the example-problem pair approach, which has been shown to outperform its rival, both in this study and the one that follows. This begs the obvious question: when do we use each one? For me, it depends on the complexity of the problems and the prior knowledge of the class. Relatively simple problems probably lend themselves better to an example-problem approach. Likewise, with students with higher prior knowledge, or students who have seen the topic before (i.e. in a revision lesson), I would favour the example-problem approach. However, for more complex problems or with students who are struggling, the fading approach seems very appropriate.
My favourite quote:
First, a complete example is presented (model). Second, an example is given in which one single solution step is omitted (coached problem solving). Then, the number of blanks is increased step by step until just the problem formulation is left, that is, a to-be-solved problem (independent problem solving). In this way, a smooth transition from modeling (complete example) over coached problem solving (incomplete example) to independent problem solving is implemented.
 
Research Paper Title: Transitioning From Studying Examples to Solving Problems: Effects of Self-Explanation Prompts and Fading Worked-Out Steps
Author(s): Robert K. Atkinson, Alexander Renkl and Mary Margaret Merrill
My Takeaway:
This paper aims to plug the one remaining gap in the fading procedure for worked examples proposed in the paper above - how to improve performance on far-transfer problems? Well, just like we saw in the Booth papers above, we can improve the effectiveness of worked examples by prompting student explanations. Here the researchers designed experiments which combined fading with the introduction of prompts designed to encourage learners to identify the underlying principle illustrated in each worked-out solution step. They set up four conditions: Example-Problem Pair (EP), Example-Problem Pair Plus (EP+) which contained the self-explanation prompts, Backwards Fading (BF) and Backwards Fading Plus (BF+). Crucially, the experiment was designed in a way so that participants had to complete the exact same number of solution steps in each condition - it was just in the EP approach they all came together, whereas in the BF model they were introduced more gradually. The key findings were as follows:
1) the BF condition was associated with a higher solution rate of near-transfer problems than EP
2) a simple prompting procedure can substantially foster both near and far transfer. Hence, the acquisition not only of relatively simple rules (i.e., near transfer) but also of understanding (i.e., far transfer) can be fostered by this instructional procedure. It is also notable that the advantage of prompting could be achieved without significantly increasing learning time
3) there was no evidence of an interaction between the use of fading and the use of self-explanation prompts on any of the measures. This may be regarded as a positive finding from an educational point of view because both instructional means produced at least medium effects on learning outcomes and were combined without causing any decrement in performance.
For me, there are two key takeaways here. Firstly, the fading procedure must be taken seriously. Secondly, prompting students to self-explain is effective no matter how the worked examples are presented, and hence is something of a no-brainer when it comes to teaching and modelling.
My favourite quote:
One may ask whether it is practical to use the instructional procedures analyzed in this article for teaching skills in well- structured domains. Overall, the use of prompts that encourage the learners to figure out the principle that underlies a certain solution step can be recommended for several reasons, including the fol- lowing: (a) it produces medium to high effects on transfer performance, (b) these effects are consistent across different age levels (university and high school), (c) it does not interfere with fading, (d) it is very easy to implement (even without the help of computer technology), and (e) it requires no additional instructional time.



Worked Examples: The Importance of Choicekeyboard_arrow_up
Back to Top

Cognitive Load Theory really brought home to me the importance of examples in teaching. This view was reinforced in my podcast interview with Daisy Christodoulou who described a lesson from her past that didn't go as planned because students had not understood her explanation of a concept. This made me realise I had been making a mistake in my teaching - I put the emphasis of my planning on the explanations I would give my students, with the examples I chose playing a secondary role. I have now come to realise that the choice of example are more important than anything else. This section is my attempt to explain why.

Research Paper Title: Exemplification in Mathematics Education
Author(s): Liz, Bills, Tommy Dreyfus, John Mason, Pessia Tsamir, Anne Watson, Orit Zaslavsky
My Takeaway:
This is a brilliant summary of relevant research into the use of examples in the teaching and learning of mathematics. There are many things I found fascinating, but here are a few of my key takeaways:
1) The concept of "varied examples" discussed by Matron can be an effective way to encounter concepts. The authors notes that what is needed is variation in a few different aspects closely juxtaposed in time so that the learner is aware of that variation as variation. This for me is similar to the concept of minimally different examples favoured by the likes of Bruno Reddy and Kris Boulton, whereby just one aspect of a particular example is changed each time. This should mean the student is more likely to notice key features of examples, and allows the teacher to have more control over the discussion. Bruno explained it to me in terms of a science experiment, whereby you are looking to isolate the effect of one variable, therefore you only change one thing at a time.
2) A teacher's poor choice of examples can have a detrimental effect on learning by making it more likely students will jump to the wrong conclusions. A study by Rowland documents this for novice teachers in a primary setting, where the unintentionally ‘special’ nature of an example can mislead learners. I have been there myself. When teaching the effect of squaring a number, I have been known to choose "2", hence potentially reinforcing the major confusion between squaring and doubling. Or asking which is bigger, 2.7 or 2.85, which learners may get right without understanding place value (asking them to choose between 2.7 and 2.65 would be a better example).
3) A study by Wilson reports that learners can be distracted by irrelevant aspects of examples, so the presence of non-examples provides more information about what is, and is not, included in a definition. This is crucial. I used to assume that learners would understand the definitions I gave. However, you can argue that to understand a definition you need to understand the concept that is being defined, and hence definitions are perhaps not the best way to introduce new concepts. This is where examples come into play, and crucially the explicit use of non-examples to illustrate to students what does and does not fit into the definition. Something as simple as showing students carefully selected images of quadrilaterals and non-quadrilaterals will enable them to build up their own understanding (and subsequent definition of what a quadrilateral is far more effectively than presenting them with a definition of something they know little about.
My favourite quote:
Examples play a crucial role in learning about mathematical concepts, techniques, reasoning, and in the development of mathematical competence. However, learners may not perceive and use examples in the ways intended by teachers or textbooks especially if underlying generalities and reasoning are not made explicit. The relationship between examples, pedagogy and learning is under-researched, but it is known that learners can make inappropriate generalisations from sets of examples, or fail to make any conceptual inferences at all if the focus is only on performance of techniques. The nature and sequence of examples, non-examples and counterexamples has a critical influence of what opportunities learners are afforded, but even more critical are the practices into which learners are inducted for working with and on examples.

Research Paper Title: The "Curse of Knowledge" or Why Intuition About Teaching Often Fails
Author(s): Carl Wieman
My Takeaway:
The Curse of Knowledge refers to the idea that when you know something, it is extremely difficult to think about it from the perspective of someone who does not know it. It is a finding that has been demonstrated across a number of studies, several of which are referenced in this paper. The Curse of Knowledge clearly has major implications for teaching, because teachers (we hope anyway) know more than our students. It brings into focus the important distinction between subject knowledge and pedagogical knowledge - you can be the best mathematician in the world, but unless you can find ways to communicate that knowledge effectively to students, then you are unlikely to be a good teacher. That is where knowledge of misconceptions is crucial, and will be discussed further in the Formative Assessment section. However, this paper made me consider another implication of the Curse of Knowledge for teachers - the importance of our choice of examples. If the suitability of our explanations is called into question by the Curse of Knowledge (perhaps we pitch them at too high a level, go too fast, use sophisticated language, or refer to concepts students do not have a complete grasp of yet), then perhaps we can avoid some of the damage of this via an increased emphasis on explanations. A running theme through this section is that explanations may be more important than examples, and for me the Curse of Knowledge provides another justification for that argument.   
My favourite quote:
This “curse of knowledge" means is that it is dangerous, and often profoundly incorrect to think about student learning based on what appears best to faculty members, as opposed to what has been verified with students. However, the former approach tends to dominate discussions on how to improve physics education. There are great debates in faculty meetings as to what order to present material, or different approaches for introducing quantum mechanics or other topics, all based on how the faculty now think about the subject. Evaluations of teaching are often based upon how a senior faculty member perceives the organization, complexity, and pace of a junior faculty member's lecture

Research Paper Title:
Shedding Light On and With Example Spaces (and a very useful summary version here)
Author(s): Paul Goldenberg & John Mason
My Takeaway:
This paper (and the excellent summary) builds nicely on the previous paper, and contains some beautifully written points that have really influenced the examples I now give my students:
1) Well chosen examples can make-up for limited vocabulary and conceptual understanding of students that can render definitions alone pretty useless. To quote: Examples might be thought of as bits of context—ways to give information other than “saying what the word means”—allowing vocabulary-learning in school to grow at least slightly closer to the natural language learning at which children are so adept. Examples allow teachers to use a word communicatively until students are able to use it as well. Teachers can use the word rather than explaining it because the example provides the context and carries the meaning. Only then, when the students already have a rough meaning from communicative use in context can one effectively clarify the meaning formally with other words, through discussion and/or definition.
2) Including non-examples can give a less ambiguous signal of understanding. To quote: Asking a student to circle all the parallelograms in a collection of figures that includes non-parallelograms, prototype parallelograms, and various special-case parallelograms that are often thought not to be parallelograms because they have their own special names, we take each special-case figure that the student does not circle as evidence that the student’s understanding of parallelogram is incomplete. But if we ask a student to draw a parallelogram, we expect not to see the special cases that we’d hope the student would circle in the previous example. And, in fact, if we do get them as responses, we might well take that as evidence of error or incomplete understanding.
3) The definition is still important for clarity and confirmation. This avoids you having to present a potentially infinite number of examples and non-examples. This definition and needs to come after exposure to carefully chosen examples and non-examples once students have a rough idea of what is going on. This avoids you having to present The lovely section on "smaglings" at the end of the summary paper conveys this point brilliantly.
My favourite quote:
Without belaboring it, here’s the point: just as definitions without examples are generally insufficient to convey meaning, so are examples without definitions. No matter how numerous and varied our examples and non-examples are, unless they are exhaustive (i.e., the set of smanglings is finite, and we have encountered every one of them as either an example or non-example), examples alone are insufficient to allow us to decide all cases, because they provide no way of knowing whether or not some perverse exception lurks among the cases that have not been seen. But the examples—and especially the task of trying to choose among the unknowns and then defend that choice—make it much easier to perceive the dimensions of possible variation and the range of permissible change. One advantage for students of encountering this meta-mathematical idea is that it helps motivate what otherwise often seems like bizarre over-particularity in the wording of definitions. There is a lot we must say to define a smangling in a way that allows us to decide, definitively and without question, which of the unknowns is and isn’t a smangling.

Research Paper Title:
Getting Students to Create Boundary Examples
Author(s): Anne Watson and John Mason
My Takeaway:
Building on the previous papers, here we are introduced to the interesting concept of Boundary Examples. Boundary examples distinguish between having and not having a specified property. The authors assert that if students are only offered well-behaved examples, or examples which have additional, but irrelevant, features, then the reason for careful statements of conditions to a theorem or definition might pass them by, and they may well develop the idea that it is possible to have ambiguous or undecided cases. The authors offer the example of sequences. If students are only shown increasing or decreasing linear sequences, they may focus on the fact that sequences are either increasing or decreasing, and be oblivious to the fact that some sequences go up by different amounts (e.g. 1, 1, 2, 3, 5, 8...), some have limiting values (e.g. 8, 4, 2, 1, 0.5...), and some have the same terms throughout (e.g. 1, 1, 1, 1...). If students leave a lesson thinking that all sequences have constant differences and the only thing that distinguishes them is whether they go up and down, then their understanding is incomplete. Elsewhere the authors explain "We use the word ‘boundary’ because we see students’ experiences of examples in terms of spaces: families of related objects which collectively satisfy a particular situation, or answer a particular mathematics question, or deserve the same label. Such spaces appear to cluster around dominant central images". The point is, that by not explicitly addressing examples "on the boundary" there is the danger that key features will be lost at the expense of others. The authors make the bold claim: "If you cannot construct boundary examples for a theorem or a technique, then you do not fully appreciate or understand it". My big takeaway from all this is to ensure that I do not always provide the "obvious" worked examples in class - instead being aware of unusual examples that still fit into the topic that I am teaching. Polygons with convex angles, straight lines in the form x =, linear equations like 4 - 2x = 10, the mean of algebraic terms, and quadratic expressions that do not factorise, are all possibilities that spring to mind. The key point is that if these types of examples are left out, it may be possible for students to appear as though they have understood a topic, whereas in fact they only have a surface level of understanding. Furthermore, challenging students to construct a particular example, then a peculiar example (eg. one which no-one else in the class is likely to think of), and then (if appropriate) a general, or at least maximally general example, seems a really useful practice to develop, all whilst considering the burden on students' working memories. The authors acknowledge that the process of creating boundary examples is likely to be tricky for students at first. Their advice is to give students time, work with them over a period of weeks and months, using the concept of boundary examples wherever appropriate. Eventually students will get better at it, and their understanding of mathematics will improve. As the authors say: "we cannot afford not to invest the time needed in order to enable students to appreciate the ideas to which they are being exposed". I discuss the importance of examples in my podcast interview with Daisy Christodoulou.
My favourite quote:
We have found that many students do not appreciate the range or scope of choice of objects which are permitted by a theorem. Most theorems can be seen as a description of something which is invariant-amidst-change, and the theorem states the scope and range of change that are permitted. But if students have not tried to construct examples for themselves, have not probed the role of various conditions in making a theorem or technique work, then they are unlikely to use it appropriately, and probably unlikely to think of using it at all!

Research Paper Title: Basic skills versus conceptual understanding: A Bogus Dichotomy in Mathematics Education
Author(s): H. Wu
My Takeaway:
This is a wonderful paper which attempts to tear down the myth that you can either teach students basic skills with little understanding, or give them conceptual understanding but without the basic skills underpinning this. For Wu, it is possible to do both, and an important component of that is the examples we choose. Wu makes the bold statement that: we should not make students feel that the only problems they can do are those they can visualize. To illustrate this point, he takes a common way of developing conceptual understanding of how to divide fractions, which is to use nice fractions that students can visualise. For example, when attempting to explain 2 divided by 1/4, we can either show a 2 litre jug filling up cups with a capacity of 1/4 litre, or ask "how many quarters in two wholes". According to Wu, this is all well and good, but then how does that help when dealing with numbers that are not quite so nice, such as 3/7 divided by 2/5, not to mention to introduction of mixed number fractions? Are we to ask students to take a leap of faith, saying "well, you saw how you could conceptualise the nice numbers, and how the written algorithm worked for them, right? So now, just trust me that it will also work for these numbers that are not so nice?". Wu argues that a natural consequence of such an approach is that children develop a sense of extreme insecurity upon the sight of any fraction other than the simplest possible. For Wu the answer is to introduce meaning behind standard algorithms. So, in this case students would be shown exactly why the method of "keep, flip change", or "inverted multiplication" works for dividing fractions, which they can then apply to any problem, no matter how complex. I fully agree with the notion that only using nice examples can lead to big issues when things get more complicated. However, I have a slight issue with the proposed solution, and it is related to my belief that sometimes it is better to teach the how before the why. You only need to read the paper to see how complex it can be to explain why the division of fractions algorithm works. How much conceptual knowledge do students need to have in place about the multiplication of fractions, algebra and generalisation in order to understand it? And are they likely to have this in place at the stage of their mathematical development when they first encounter the division of a fraction? The issue is highlighted even more starkly with the use of the algorithm for written addition later in the paper. I 100% agree that teaching the understanding behind why a robust algorithm works is essential, but I am just not convinced that this needs to come before students have used it. After all, what do we do with the student who cannot understand why the algorithm for dividing fractions or adding number works? Ban them from using it? Is there anything wrong with teaching students the algorithm well, getting them confident with it, and then revisiting the algorithm later in their mathematical development when they are in a better position to understand exactly why it works?
There is a second point in this paper that I feel is key to our discussion - the idea of creating a need or a purpose. This is something Dan Meyer discusses in his wonderful headache-aspirin series. Wu does a fantastic job of creating a purpose for the written algorithm for adding and multiplying by showing how painstakingly slow life would be without them. For me, this creation of need is far more powerful than any contrived real-life context would ever be.
My favourite quote:
Finally, we call attention to the breathtaking simplicity of the multiplication algorithm itself despite the tediousness of its derivation. The conceptual understanding hidden in the algorithm is the kind that students eventually need in order to prepare for algebra. In short, this algorithm is a shining example of elementary mathematics at its finest and is fully deserving to be learned by every student. If there is any so-called harmful effect in learning the algorithms, it could only be because they are not taught properly.

Research Paper Title: Relational Understanding and Instrumental Understanding
Author(s): Richard R. Skemp
My Takeaway:
I wasn't too sure whereabouts on this page to include this classic article by Richard Skemp, but I have opted for the section on the importance of the choice of examples, for a reason that will hopefully become clear shortly. This paper discusses an important distinction between two different types of understanding:
1) Relational understanding - knowing both what to do and why it is done that way
2) Instrumental understanding - the ability to be able to do something without really understanding why (rules without reasons)
Obviously the former is the most desirable, but the author argues that most of school mathematics involves the first, with students being taught maths via a set of rules which enable them to successfully answer questions across a narrow domain (usually with an exam in mind) without really having a clue what is going on. In an attempt to support his view of the importance of relational understanding, the author plays devil's advocate and lists what he sees as the most commonly stated benefits of instrumental understanding:
1) Within its own context, instrumental mathematics is usually easier to understand; sometimes much easier
2) So the rewards are more immediate, and more apparent. It is nice to get a page of right answers, and we must not underrate the importance of the feeling of success which pupils get from this
3) Just because less knowledge is involved, one can often get the right answer more quickly and reliably by instrumental thinking than relational
Then he discusses what he sees as the benefits of relational understanding:
1) It is more adaptable to new tasks
2) It is easier to remember
3) Relational knowledge can be effective as a goal in itself.
4) Relational schemas are organic in quality
But here's my question: who is to say that relational understanding needs to come before instrumental understanding? The author makes the point several times that relational understanding is harder than instrumental (often requiring knowledge of other areas of maths, like the example of circumference of a circle), and that we should not underestimate the importance of students' feelings of success and achievement. Why not teach instrumental understanding really well, via example-problem pairs using examples that cover the entire domain of the topic (hence negating the argument that students with instrumental understanding can only answer a narrow range of questions, and once again highlighting the importance of the choice of examples), and then return to relational understanding once students have tasted success and developed more mathematically? This could be at the end of the topic, or more likely the following term or year, bringing in the related topics necessary to achieve relational understanding, hence tapping into the positive effects of spacing and interleaving. Students revisiting the topic will do so with a feeling of success, and perhaps even have their interests piqued as to why the methods they have learned and used successfully work, as opposed to attempting to teach the why first, when students may lack the skills to understand it, and have no context in which to understand it either. Now, I am not saying that the How should always come first, nor indeed that relational understanding should not be the ultimate goal of teaching. But for topics where relational understanding is likely to be difficult, then it is seriously worth questioning whether we would not serve our students better by teaching the how first. 
My favourite quote:
Suppose that a teacher reminds a class that the area of a rectangle is given by A = L x B. A pupil who has been away says he does not understand, so the teacher gives him an explanation along these lines. “The formula tells you that to get the area of a rectangle, you multiply the length by the breadth.” “Oh, I see,” says the child, and gets on with the exercise. If we were now to say to him (in effect) “You may think you understand, but you don’t really,” he would not agree. “Of course I do. Look; I’ve got all these answers right.” Nor would he be pleased at our devaluing of his achievement. And with his meaning of the word, he does understand.


Problem Solvingkeyboard_arrow_up
Back to Top

For me, Cognitive Load Theory (particularly the paper Cognitive Load during Problem Solving: Effects on Learning discussed in the Cognitive Load Theory section) provides a sound argument for why students cannot learn to be problem solvers before they have sufficient knowledge and procedures stored in long term memory, and Explicit Instruction provides a model for students acquiring those knowledge and procedures. But how do we then help our students take that next step towards developing those key problem solving skills that we all want them to have? We have already seen some answers at the end of the Cognitive Load Theory section with the suggestion of a phased approach, and noting the importance of the expertise-reversal effect. Here we seek to see if there is such a thing as a set of problem solving skills that we can help our students develop.


Research Paper Title:
Learning to think mathematically: problem solving, metacognition and sense-making in mathematics
Author(s): Alan H. Schoenfeld
My Takeaway:
This is a fantastic summary of problem solving in mathematics. There were several points that caught my eye:
1) There are two rather contrasting definitions of a "problem":
Definition 1: "In mathematics, anything required to be done, or requiring the doing of something."
Definition 2: "A question... that is perplexing or difficult."
I had always thought of "problems" as being more of the second variety, but it is worth bearing in mind that when much of the literature refers to "problems", they mean the first type. And whilst we are on the subject, I think a useful way to think about problems is in terms of GCSE Maths Assessment Objectives:
AO1 - Use and apply standard techniques
AO2 - Reason, interpret and communicate mathematically
AO3 - Solve problems within mathematics and in other contexts
It is AO2 and AO3 that I think of when I hear the terms "problem" or "problem solving". It seems to me that AO1 style questions can be taught using the techniques of example-problem pairs, deliberate practice and careful use of desirable difficulties, whereas to help students to answer A02 and AO3 questions you maybe need something more.
2) The authors present a framework for thinking mathematically and solving problems. They argue that all of these categories - core knowledge, problem solving strategies, effective use of one's resources, having a mathematical perspective, and engagement in mathematical practices -- are fundamental aspects of thinking mathematically. I strongly recomend reading each of these sections.
3) The authors make clear the importance of the knowledge base for solving problems. They sum this up nicely as follows: In sum, the findings of work in domains such as chess and mathematics point strongly to the importance and influence of the knowledge base. First, it is argued that expertise in various domains depends of having access to some 50,000 chunks of knowledge in LTM. Since it takes some time (perhaps 10 seconds of rehearsal for the simplest items) for each chunk to become embedded in LTM, and longer for knowledge connections to be made, that is one reason expertise takes as long as it does to develop. Second, a lot of what appears to be strategy use is in fact reliance on well-developed knowledge chunks of the type "in this well-recognized situation, do the following.". For me it is the second point that is crucial - what is often seen as strategy is in fact knowledge. This will be a recurring theme throughout this section.
4) Strategy is still important: Nonetheless, it is important not to overplay the roles of these knowledge schemata, for they do play the role of vocabulary -- the basis for routine performance in familiar territory. Chess players, when playing at the limit of their own abilities, do rely automatically on their vocabularies of chess positions, but also do significant strategizing. Similarly, mathematicians have immediate access to large amounts of knowledge, but also employ a wide range of strategies when confronted with problems beyond the routine
5) The acquisition of knowledge must come before any such problem solving strategies are developed, otherwise the following may happen: a reliance on schemata in crude form -- "when you see these features in a problem, use this procedure" -- may produce surface manifestations of competent behavior. However, that performance may, if not grounded in an understanding of the principles that led to the procedure, be error-prone and easily forgotten.
6) Teaching the classic problem solving strategies as suggested by Poyla may not be that easy: A substantial amount of effort has gone into attempts to find out what strategies students use in attempting to solve mathematical problems... No clear-cut directions for mathematics education are provided by the findings of these studies. In fact, there are enough indications that problem solving strategies are both problem- and student-specific often enough to suggest that finding one (or few) strategies which should be taught to all (or most) students are far too simplistic.
7) Novices and Experts approach problems in different ways. This finding is related to the work of Cognitive Load Theory on the issues of a means-end problem solving strategy. The author presents a fascinating graph of how studetns and mathematicians typically approach problems, showing how their time is split across the areas of Read, Analyze, Explore, Plan, Implement and Verify. Following a quick read, students typically spend the majority of time exploring. In Schoenfeld's collection of (more than a hundred) videotapes of college and high school students working unfamiliar problems, roughly sixty percent of the solution attempts are of the "read, make a decision quickly, and pursue that direction come hell or high water" variety. And that first, quick, wrong decision, if not reconsidered and reversed, guarantees failure. This is in stark contrast to the graph of the mathematician. The mathematician spent more than half of his allotted time trying to make sense of the problem. Rather than committing himself to any one particular direction, he did a significant amount of analyzing and (structured) exploring -- not spending time in unstructured exploration or moving into implementation until he was sure he was working in the right direction. Second, each of the small inverted triangles in Figure 4 represents an explicit comment on the state of his problem solution, for example "Hmm. I don't know exactly where to start here" (followed by two minutes of analyzing the problem) or "OK. All I need to be able to do is [a particular technique] and I'm done" (followed by the straightforward implementation of his problem solution).
8) Teaching students metacnognitvie strategies for problem solving is possible to help make the the students' graphs more closely resemble those of the expert. The author explains: However, it is the case that such skills such can be learned as a result of explicit instruction that focuses on metacognitive aspects of mathematical thinking. That instruction takes the form of "coaching," with active interventions as students work on problems. Roughly a third of the time in Schoenfeld's problem solving classes is spent with the students working problems in small groups. The class divides into groups of three or four students and works on problems that have been distributed, while the instructor circulates through the room as "roving consultant." As he moves through the room he reserves the right to ask the following three questions at any time:
What (exactly) are you doing? (Can you describe it precisely?)
Why are you doing it? (How does it fit into the solution?)
How does it help you? (What will you do with the outcome when you obtain it?)
He begins asking these questions early in the term. When he does so the students are generally at a loss regarding how to answer them. With the recognition that, despite their uncomfortableness, he is going to continue asking those questions, the students begin to defend themselves against them by discussing the answers to them in advance. By the end of the term this behavior has become habitual.

9) A related approach is suggested by Lester, Garofalo & Kroll on page 65 and is definitely worth reading. Three important  conclusions come from this study in particular:
  • Metacognition instruction is most effective when it takes place in a domain specific context.
  • Problem-solving instruction, metacognition instruction in particular, is likely to be most effective when it is provided in a systematically organized manner under the direction of the teacher.
  • It is difficult for the teacher to maintain the roles of monitor, facilitator, and model in the face of classroom reality, especially when the students are having trouble with basic subject matter.

Hence we see the importance of modelling and teacher-led instruction, together with the necessity for knowledge before problem solving.

My favourite quote:
This chapter has focused on an emerging conceptualization of mathematical thinking based on an alternative epistemology in which the traditional conception of domain knowledge plays an altered and diminished role, even when it is expanded to include problem solving strategies. In this emerging view metacognition, belief, and mathematical practices are considered critical aspects of thinking mathematically. But there is more. The person who thinks mathematically has a particular way of seeing the world, of representing it, of analyzing it. Only within that overarching context do the pieces -- the knowledge base, strategies, control, beliefs, and practices -- fit together coherently

Research Paper Title: Critical thinking: why is it so hard to teach?
Author(s): Daniel T Willingham
My Takeaway:
This paper does a really good job of summarising Willingham's thoughts on problem solving. We all want our students to become better problem solvers, and to think like mathematicians, but the finding here is that unlike, say, how to factorise a quadratic equation or how to add two fractions together, problem solving per se cannot be taught - at least not to novice learners. Many times I have witnessed students able to solve one problem, but then being unable to transfer those skills to another problem that is clearly related. The key is that the two problems are only clearly related to me. Experts (and I am by no means saying I am an expert here, but relative to the students I am) and novices approach problems differently. When faced with, for example, a worded maths problem in context, typically students are focusing on the scenario that the word problem describes (its surface structure) instead of on the mathematics required to solve it (its deep structure). So even though students have been taught how to solve a particular type of word problem, when the scenario changes, students still struggle to apply the solution because they don’t recognise that the problems are mathematically the same. "Sir, you've not taught us about paint", is a particularly painful post GCSE exam memory for me. Anyway, for Willingham, there are two ways of addressing this issue, but both have important limitations:
1) Students need to be familiar with a problem's deep structure. Exposure to lots of variations of problems that have the same underlying deep structure certainly helps students to look beyond the surface elements of each problem and focus on the relevant mathematics underlying it. So, lots of problems that revolve around the concept of Lowest Common Multiple should help students better spot those. The problem? Well, think how many concepts there are in maths! And then, think about what happens when those concepts are combined! It soon becomes impractical to exposure students to variations of problems covering every single concept in maths.
2) Students need to know they are looking for a deep structure. This is known as meta-cognition. If students are given a problem, they should be prompted to think: "okay, this is a maths problem, it must be related to something I have done before, now can I figure out what it is?". Whilst this strategy undoubtedly helps students avoid the temptation to focus all their efforts on the surface structure, there is a big problem: if they then lack the knowledge to then answer the question, all their efforts are in vain.
This paper has had a profound effect on me. Firstly, when I am happy that students have a sound knowledge of a given topic, I then select a group of related problem solving question on that topic, but crucially I explicitly discuss how they are connected. I force myself and my students to do this, to really emphasise the cues and signals, to try and get around the undo attention give to the surface structure. Sure, I cannot do this for every sub-topic and every combination of topics, but it is a start. And what I no longer do is con myself into thinking that giving students lots of different problems with different underlying deep structures will somehow enable them to develop the skill of "problem solving". Without the knowledge of the topic in place, and without purposeful practice looking at connected problems, this is likely to be a futile and frustrating exercise for all involved.
My favourite quote:
What do all these studies boil down to? First, critical thinking (as well as scientific thinking and other domain-based thinking) is not a skill. There is not a set of critical thinking skills that can be acquired and deployed regardless of context. Second, there are metacognitive strategies that, once learned, make critical thinking more likely. Third, the ability to think critically (to actually do what the metacognitive strategies call for) depends on domain knowledge and practice. For teachers, the situation is not hopeless, but no one should underestimate the difficulty of teaching students to think critically.

Research Paper Title:
Analogical Problem Solving and Schema Induction and Analogical Transfer
Author(s): Mary L Glick and Keith Holyoak
My Takeaway:
The use of analogies in conveying concepts is prevalent in subjects like History in English, but not much in maths (it is quite hard to come up with a story to explain how to find the turning point on a quadratic graph, for example). However, I have come to realise that analogies play a huge role in many of the problem-solving questions we give our students to do. Indeed, any in-context problem is surely an analogy, with the surface structure being used as a way to convey - or, as is often the case, disguise - the deep structure that lies below. Hence, work related to analogies and the issues of transfer are directly relevant to mathematics. These two classic papers clearly demonstrate the issues Willingham is describing above, as well as offering a solution. The first paper describes a famous example of a lack of transfer, whereby participants are told a story about a general trying to overtake a fortress ruled by a dictator and challenged to come up with solutions. They were then given a problem involving a doctor faced with a patient who has a malignant tumor in his stomach. The key point is that the two stories have identical solutions. In other words, they have different surface structures, but the same deep structures. However, very few participants made the connection. Indeed, the experimenters found that unless they prompted students to use the initial story, most ignored it and tried to come up with entirely new solutions to the tumor problem. Only 20% of students who attempted to solve the problem gave the correct answer without that prompt. This is clearly related to the finding in the Cognitive Science section that novices and experts think differently, with the former focusing on the surface structure of the problem. So, what is the solution? Well, the second paper provides a possibility. This time the researchers gave two different analogous problems to students (different surface structure but same deep structure) before presenting them with the problem to solve. When students were given these two example stories, with different surface details, and then given the problem, they were much more likely to give the analogous solution. Fifty-two percent of students – a substantial increase over the 20% from before – were able to give the analogous solution. How does this help us as maths teachers? Well, a simple takeaway is that two examples are better than one, especially when it comes to wordy problem-solving questions where the method to use is not entirely obvious. Notice that prompts were also important to help students notice the connection between the two problems. So, if we want students to go beyond the surface features of the problem, we need to present them with groups of problems with the same deep structure, together with our guided support, so that they can begin to make sense of the connections. The use of unrelated problems to solve here and there is unlikely to promote the ability to transfer knowledge to different situations that we need our students to have.
My favourite quote:
The experiments in Part I attempted to foster the abstraction of a problem schema from a single story analog by means of summarization instructions, or else either verbal or visual statements of the underlying principle. We found no evidence that any of these devices yielded more abstract representations of the story, nor did any consistently facilitate analogical transfer. In contrast, the results obtained in Part II were dramatically more positive. Once two prior analogs were given, subjects often derived an approximation to the convergence schema as the incidental product of describing the similarities of the analogs; furthermore, the quality of the induced schema was highly predictive of subsequent transfer performance. In addition, the same verbal statements and diagrams that had failed to influence transfer from a single analog proved highly beneficial when paired with two.

Research Paper Title: Domain-Specific Knowledge and Why Teaching Generic Skills Does not Work
Author(s): André Tricot and John Sweller
My Takeaway:
This brilliant paper offers a complementary view to the Willingham paper above and follows directly from the work we looked at concerning Cognitive Load Theory. The authors argue that there is an assumption that there exists such a thing as domain-general cognitive knowledge (i.e. generic skills such as problem solving and critical thinking), and this can be used to explain student achievement. This assumption then leads to the related assumption that we can teach these generic skills. The authors argue, however, that in fact domain-specific knowledge held in long-term memory provides a better explanation for the acquisition of skills. Moreover, they offer up several explanations of why we as teachers are so keen to embrace the idea of a set of teachable generic skills: "At any given time, we are unaware of the huge amount of domain specific knowledge held in long-term memory. The only knowledge that we have direct access to and are conscious of must be held in working memory. Knowledge held in working memory tends to be an insignificant fraction of our total knowledge base. With access to so little of our knowledge base at any given time, it is easy to assume that domain-specific knowledge is relatively unimportant to performance. It may be difficult to comprehend the unimaginable amounts of organised information that can be held in long-term memory precisely because such a large amount of information is unimaginable. If we are unaware of the large amounts of information held in long-term memory, we are likely to search for alternative explanations of knowledge-based performance"
My favourite quote:
We have argued that expertise based on biologically secondary, domain-specific knowledge held in long-term memory is by far the best explanation of performance in any cognitive area. Furthermore, in contrast to domain-general cognitive knowledge, there is no dispute that domain-specific knowledge and expertise can be readily taught and learned. Indeed, providing novice learners with knowledge is the main role of schools. We might guess that most school teachers in most schools continue to emphasise the domain-specific knowledge that always has been central, making little attempt to teach domain-general knowledge. Based on our argument, they should continue to do so. At school, children acquire knowledge that overcomes the need to engage in inefficient problem solving search and other cognitive processes. That knowledge allows people to function in a wide variety of tasks outside of school
.

Research Paper Title:
Classroom Cognitive and Meta-Cognitive Strategies for Teachers
Author(s): Florida Department of Education
My Takeaway:
This is a good summary of various problem solving strategies, and their applications in the classroom. The paper is structured around Polya's classic four-step approach to problem solving:
1) understanding the problem
2) devising a plan to solve the problem,
3) implementing the plan
4) reflecting on the problem
Each of these four stages in analysed in great detail, with helpful suggestions for strategies to use in the classroom, together with examples and links to relevant research. I particularly like the emphasis on the first part of this process - understanding the problem. What should be the simplest part of this process if often the most difficult, but suggested techniques such as The Paraphrasing Strategy and Visualisation may help. However, what struck me most about this paper is the sheer variety of strategies outlined. There is no single "problem solving strategy" that students can learn and then apply to any problem. Topic-specific knowledge, together with an understanding of deep structure of problems that only knowledge can bring, is needed to correctly select the most appropriate strategy for a given problem and apply it successfully.
My favourite quote:
The first step in the Polya model is to understand the problem. As simple as that sounds, this is often the most overlooked step in the problem-solving process. This may seem like an obvious step that doesn’t need mentioning, but in order for a problem-solver to find a solution, they must first understand what they are being asked to find out.
Polya suggested that teachers should ask students questions similar to the ones listed below:
    Do you understand all the words used in stating the problem?
    What are you asked to find or show?
    Can you restate the problem in your own words?
    Can you think of a picture or a diagram that might help you understand the problem?
    Is there enough information to enable you to find a solution?


Research Paper Title: The Subgoal Learning Model: Creating Better Examples So That Students Can Solve Novel Problems
Author(s): Richard Catrambone
My Takeaway:
This paper is important for problem solving, Making the most of Examples and Self Explanations. The author points out that learners have great difficulty solving problems requiring changes to solutions demonstrated in examples. This is obviously incredibly important because it is impossible to construct examples that cover every eventuality, despite how careful we can be with concepts such as Boundary Examples. However, the authors found that if the worked example learners study are organised by sub-goals (in other words, a meaningful conceptual piece of an overall solution procedure), then the learners are more successful. In other words for multi-step procedures, teachers can encourage students to identify and label the substeps required for solving a problem. This practice makes students more likely to recognise the underlying structure of the problem and to apply the problem-solving steps to other problems. Sub-goal learning is hypothesised to be aided by cues in example solutions that indicate that certain steps go together. These cues may induce a learner to attempt to self-explain the purpose of the steps, resulting in the formation of a sub-goal. Across 4 experiments it was found that a label for a group of steps in examples helped participants form sub-goals as assessed by measures such as problem-solving performance and talk aloud protocols. That is no real surprise - labeling parts of worked solutions to more complex problems seems a sensible idea to draw students' attention to the features of that problem and why it is important in the grand scheme of things. As the authors point out, and as we know from our look at Student Self Explanations, compelling students to pause and consider the purpose of individual steps in a solution is likely to be important for their overall understanding. However, what I found particularly interesting is that not all labels were equally as effective. Abstract labels (in this case a symbol to stand for the total when calculating the mean) were more likely than superficial labels (labels tied to the surface structure of the particular problem, such as "total number of suitcases") to lead participants to form sub-goals with fewer ties to surface features. The issue, of course, with focusing on the surface structure is that students are less likely to be able to transfer such solutions to contexts with the same deep structure but different surface structures. However, before we are tempted to jump to the conclusion "abstract labels are best for everyone", there is a problem as explained in the quote below. Hence, we once again see the importance of knowledge, not only in allowing learners to circumvent the distracting features of the surface structure in the first place, but from fully benefiting from the power of sub-goals and labels.
My favourite quote:
Unfortunately, a label that is related to surface features of a problem will be more likely to lead a learner to form a solution procedure that is tied to those features. An abstract label is less likely to lead a learner to make this mistake, although the learner must have relevant background knowledge in order to take advantage of an abstract label. These results suggest that cues such as labels can play a strong role in the formation of solution procedures. Because of this, care must be taken to construct cues in a way to aid the formation of structured solution procedures. For learners with weaker backgrounds these cues might need to be tied at least partially to example features despite the danger that this may lead the learner to form representations that have erroneous surface ties. However, for learners with stronger backgrounds, the cues can be constructed more abstractly, thus helping them to form appropriate subgoals.

Research Paper Title:
Problem-Solving Strategies: Research Findings from Mathematics Olympiads
Author(s): Cheung Pak-Hong
My Takeaway:
This paper takes Polya's approach to problem solving and tries to apply it to tricky Mathematical Olympiad questions. A variety of problems are presented, together with a discussion around which problem solving strategies could be used to tackle them. Those sitting this paper are among the very best student mathematicians in the world, and hence their approach to solving problems should shed some valuable light onto the most effective strategies. The conclusion of the authors is of crucial importance: the most effective problem-solving strategies are topic-specific. There is no generic problem-solving strategy. To be successful at these most challenging of problems, students must know their topics inside and out. This once again gives credence to the view expressed by Cognitive Load Theory that you cannot teach problem solving merely by exposure to problems. Topic-specific knowledge must be acquired in order for appropriate mental schema to develop, which allows students to have the best chance of solving problems.
My favourite quote:
This pilot study revealed that, for problems at Olympiad level, while heuristics suggested by Polya are useful in analysing the problems and in exploring feasible solutions, most of the more effective strategies are topic oriented. Olympiad problems in geometry are almost excluded from this report because although common strategies for solving them do exist, such as expressing quantities in terms of areas of triangles, they are confined to geometry.



Inquiries keyboard_arrow_up
Back to Top

With minimal guided instruction coming under fire in the Explicit Instruction section, you may not think there is a place for inquiries in mathematics. But if done carefully, I believe there still is, with potential benefits in terms of motivation and problem solving.

Research Paper Title:
Inquiry Teaching
Author(s): Andrew Blair
My Takeaway:
Inquiry teaching is often misunderstood as "learning by discovery". This article provides a good summary of the key distinction, together with the perceived benefits of inquiry teaching. The amount of structure or guidance offered by the teacher is determined by students' ability to think independently and critically. A key feature of inquiry lessons is "negotiation" between the teacher and the students over the structure and direction of the lesson. It is worth noting, however, that proponents of the direct instruction approach will likely balk at statements such as "Students not only meet concepts and skills in a meaningful pursuit of answering their own or peers’ questions, they also participate in a debate about how to learn and why to learn in a certain way" and "(the teacher must) redistribute authority to students". But I think that if the inquiries are well structured, and students only participate in them when they are ready (i.e. they are not used to teach concepts, just to apply them), then they have a valid place in the maths classroom.
My favourite quote:
In teacher directed strategy lessons, there is little guarantee that a method permits students to reach an ‘ideal’ understanding. In the investigative classroom, the learning experience is rigidly structured, leaving students the unenviable responsibility to discover a conceptual relationship. It is only the inquiry classroom that offers students a mechanism to harmonise conceptual learning with the method of learning.

Research Paper Title: Extending example spaces as a learning/teaching strategy in mathematics
Author(s): Anne Watson and John Mason
My Takeaway:
This is a fascinating paper which looks at the importance of examples for the understanding of mathematics. We have seen in the Cognitive Load Theory section the importance of worked examples for early skill acquisition under a model of Explicit Instruction, but here we look at something different - the use of student generated examples. The paper cites several studies in which student generated examples have aided concept development. Asking students to "find an equation of a straight line that has two intersection points with the parabola y = x2+ 4x+ 5." gives them opportunities to test their core coordinate geometry skills, but also opens the door for further investigation and generalisation - why do some lines cross a parabola twice, whereas others do not cross it at all, for example? Hence, we have examples leading to inquiry. Having read the Cognitive Load Theory section, there is the obvious danger that such questions are too much for students' fragile working memories to deal with during early skill acquisition. In the absence of well formed schema, novice learners may engage in inefficient problem-solving search via means-end analysis, and hence could actually be thinking hard but not learning anything. However, the authors point to a study involving bottom-set year 9 students explaining the definition of a prime number by generating their own counter-examples, suggesting that student generated examples may in fact aid the important encoding process. For me, there is little doubt that framing a question in terms of asking students to generate their own examples is a great way to promote inquiry, foster motivation and to provide opportunities to make deep connections. Without wishing to sound like a broken record, I feel we just need to be careful to use these strategies when students' understanding of the basic concepts are relatively secure.
My favourite quote:
We have illustrated how seeing teaching/learning mathematics as creation and extension of personal example spaces can inform the construction of tasks in which students can work directly on their own mathematical structures and relationships. Achieving competence in mathematics can be seen as the development of complex, interconnected, accessible example spaces.



Memory: Encodingkeyboard_arrow_up
Back to Top

How many times have you seen students seemingly understand something one lesson, and then act as though they have never seen it before only a day later? Learning without the ability to retain and retrieve is not really learning at all. Remembering things involves three processes: encoding information (learning it, by perceiving it and relating it to past knowledge, so that it is in a format which it can then be stored in memory), storing it (maintaining it over time), and then retrieving it (accessing the information when needed). The next few sections are going to look at the fascinating concepts of retrieval and storage, including how making life more difficult for students might be the key for forming long-lasting memories. However, without successful encoding, there is little point worrying about the rest. So, how do we help students successfully encode information? Well, we have seen already that the principles of Explicit Instruction and Cognitive Load Theory (in particular the careful presentation of information and the use of worked examples) should help focus students' fragile, limited working memories on the things that really matter, and discussed the story structure of lessons and the use of analogies to make information more meaningful to students. In short, effective encoding relies upon the attention of the learner and on the associative nature of the brain. By making new learning personally relevant and linked to prior learning in long-term memory you can increase the associativity of the new learning and increase the effectiveness of the encoding. So, we have  done most of the work in previous sections! However, there are a couple of papers here that I found particularly interesting.

Research Paper Title: Students Remember ... What They Think About
Author(s): Daniel T Willingham
My Takeaway:
This article (along with Willingham's outstanding book Why Don't Students Like School) has had a profound effect on how I plan my lessons. Memory is the residue of though. Quite simply: students will remember what they are thinking about, so I need to plan my lessons accordingly. When I present my students with new material, it is not just the presentation of that material that is important, it is what the students are actually thinking about. Two related problems can emerge here if we are not careful:
1) Shallow Knowledge. Students may grasp onto one message from the lesson, but they may fail to grasp the underlying meaning that supports it. Take a lesson on adding fractions. Students may remember that denominators have to be different, but if they do not understand why, or understand how to make the denominators the same, then that shallow knowledge isn't going to be much use to them.
2) Thinking about the wrong things. If my students are revising quadratic equations by making a poster, will they be thinking about the subtleties of the algebra, or the colour of the highlighters they are using? If they are revising the laws of fractions by putting together a PowerPoint presentation, will they be thinking about fractions or PowerPoint animations?
Number 2) seems relatively easy to fix - I just need to plan activities like this carefully, bearing in mind that any increase in engagement that these different activities bring must be weighed up against a likely decrease in learning. But what about the issue of shallow knowledge? Willingham provides some useful, practical strategies:
1) Anticipate what your lesson will lead students to think about. When planning my lessons, the main question I ask myself during each part of the lesson is "what will my students actually be thinking about at this point?". And if it is not the key concept I need to get across, then I need to change my plan.
2) Use discovery learning carefully. We have already seen arguments against discovery learning in the Explicit Instruction section, but during the encoding process it seems even more important that teachers guide students carefully to try to ensure they are thinking about the correct concepts and information. I try to avoid bold statements, but I am going to make one here: discovery learning definitely should not be used during the encoding process.
3) Design reading assignments that require students to actively process the text. You may not think this has any relevance to maths, but when I interviewed Dani Quinn for my podcast she described how at Michaela the students take part in whole-class reading, combined with regular rests of knowledge to ensure students are focussed and not just drifting along.
4) Design tests that lead students to think about and integrate the most important material. This is a really interesting one for me. My students will often ask what is going to be on an upcoming test, and I will be purposefully vague. However, Willingham argues that if I am more specific in what I tell students (there will be two questions on adding fractions, one of which involves a negative number), then it will lead students to think more deeply about these topics during revision instead of a shallow glance across a number of topics. Of course, you need to ensure that tests across the year cover all the key concepts, but I thought this was an interesting approach.
My favourite quote:
In summary, in the early stages of learning, students may display "shallow" learning. These students have acquired bits of knowledge that aren't well-integrated into a larger picture. Research tells us that deep, connected knowledge can be encouraged by getting students to think about the interrelation of the various pieces of knowledge that they have acquired. Cognitive science has not progressed to the point that it can issue prescriptions of exactly how that can be achieved—that job is very much in the hands of experienced teachers. But in considering how to encourage students to acquire meaningful knowledge, teachers will do well to keep the "memory is as thinking does" principle in mind.

Research Paper Title: Levels of Processing: A Framework for Memory Research
Author(s): Fergus I M Craik and Robert S Lockhart
My Takeaway:
This classic paper was essentially a precursor to the work on schema for memories that we discussed in the Cognitive Science section. The authors argue that stimulus information is processed at multiple levels simultaneously (not serially) depending on characteristics, attention and meaningfulness. This supports the arguments discussed in the Willingham paper above. New information does not have to enter in any specific order, and it does not have to pass through a prescribed channel. The authors further contend that the more deeply information is processed, the more that will be remembered. For me, the most important point of the whole paper was this: the more connections to a single idea or concept, the more likely it is to be  remembered. The message is clear: as students encounter new information they should try to relate it to information they already know. Knowledge builds upon knowledge. One of the most important differences between novices and experts is the structure and organization of domain-specific knowledge. Experts have existing schema which allows them to better assimilate new information, making connections and thus processing it more deeply. If we wish our students to successfully encode new information, we must first ensure they have the sufficient domain-specific knowledge present, and then try our best to help students see the connections between their existing knowledge and this new information. This could be through the use of analogies, a story structure, or just effective explicit instruction.
My favourite quote:
That is, items are kept in consciousness or in primary memory by continuing to rehearse them at a fixed level of processing. The nature of the items will depend upon the encoding dimension and the level within that dimension. At deeper levels the subject can make more use of learned cognitive structures so that the item will become more complex and semantic. The depth at which primary memory operates will depend both upon the usefulness to the subject of continuing to process at that level and also upon the amenability of the material to deeper processing. Thus, if the subject's task is merely to reproduce a few words seconds after hearing them, he need not hold them at a level deeper than phonemic analysis. If the words form a meaningful sentence, however, they are compatible with deeper learned structures and larger units may be dealt with. It seems that primary memory deals at any level with units or "chunks" rather than with information


Memory: Forgetting, Spacing and Interleavingkeyboard_arrow_up
Back to Top

Once students have successfully Encoded information, we need to ensure that those memories are stored and can be retrieved. The papers in this section dig into this absolutely fascinating area, specifically with regard to forgetting, spacing and interleaving. The power of Testing was such a revelation to me that it is a separate section. There are also profound implications for students' revision, which will also be dealt with explicitly in the Revision section. The underlying message running throughout these papers blows my mind - in order to improve learning, we need to make it more difficult. This may appear in direct opposition to the work outlined in the Cognitive Load Theory section, which described conditions needed to make learning as easy as possible. For me, the matter is resolved by using the principles of Cognitive Load Theory in early skill inquisition and during the encoding process, removing all unnecessary load on working memory, and then adopting the recommendations of Bjork and others for increasing the challenge. Bjork's paper on Desirable Difficulties brings this all together beautifully.

Research Paper Title: Learning Versus Performance: An Integrative Review
Author(s): Nicholas C. Soderstrom and Robert A. Bjork
My Takeaway:
It is no exaggeration to say that this paper has changed my whole approach to teaching. The main premise is one of those things that sounds so obvious when you say it out loud, but I had been missing it for the first 12 years of my career: learning and performance are two different things. The ultimate goal of instruction should be to facilitate long-term learning—that is, to create relatively permanent changes in comprehension, understanding, and skills of the types that will support long-term retention and transfer. During the instruction or training process, however, all we can observe and measure is performance. That would all be fine, apart from the rather inconvenient fact that current performance can be a highly unreliable guide to whether learning has happened. During lessons, teachers, consciously or otherwise, provide cues and prompts to elicit correct answers. Students are also pretty good at mimicking, which allows them to say what they think the teacher wants to hear. All of this boosts performance, but may have no impact on learning. Hence, we are at risk of being fooled by current performance, which can lead us as teachers to choose less effective conditions of learning over more effective conditions - i.e. use strategies that lead to short-terms gains in performance, such as teaching to a particular test. Moreover, it can lead students to prefer certain revision techniques, such as cramming, over others as they can see short-term improvements in performance. Here is the ultimate kicker - high performance may lead to lower learning! This next point is so important that i I am going to put it in bold: when we design lessons to boost pupils performance, we may actually hinder their long term learning. Likewise, conditions that induce the most errors during acquisition are often the conditions that lead to the most learning. The reasons behind this incredible finding are discussed in the paper On the Symbiosis of Learning and Forgetting below. The crucial distinction between learning and performance has huge implications for teaching, revision and lesson observation are huge, and is directly to related to concepts such as spacing, interleaving, and overlearning which will be discussed further in this section. 
My favourite quote:
Given that the goal of instruction and practice— whether in the classroom or on the field—should be to facilitate learning, instructors and students need to appreciate the distinction between learning and performance and understand that expediting acquisition performance today does not necessarily translate into the type of learning that will be evident tomorrow. On the contrary, conditions that slow or induce more errors during instruction often lead to better long-term learning outcomes, and thus instructors and students, however disinclined to do so, should consider abandoning the path of least resistance with respect to their own teaching and study strategies. After all, educational interventions should be based on evidence, not on historical use or intuition.


Research Paper Title:
On the Symbiosis of Learning and Forgetting
Author(s): Robert A. Bjork
My Takeaway:
The work of Robert Bjork and Elizabeth Bjork has blown my mind. Their "Theory of Dissuse" outlines the importance and - wait for it - the benefits of forgetting. They explain that each item in memory has a storage strength and a retrieval strength. Storage indicates how well an item is embedded in long-term memory and retrieval indicates how easily an item can be brought to mind when needed. Ideally, we want our students to have both. But here is the key point - attempts to increase retrieval strength improve performance in the short term but very quickly fade. Worse still, trying to retrieve something from memory too quickly can interfere with our ability to store it more strongly. Cramming, cues, and teaching to a test, all serve to make it easier to retrieve that information, but do not give it chance to develop a high storage strength and hence become embedded in long-term memory. This all sounds pretty bad for teaching - but there is good news: if we wait until we’ve started to forget something, retrieval practice increases our ability to recall it in the long term. Therefore, the best way to increase storage strength is to allow memories to fade before trying to retrieve them. The conclusion is both simple and profound: the best way to remember is to forget. This has huge implications for both teaching and revision. Firstly, it sets up nicely the work on interleaving and spacing that we will cover later in this section. But it also calls into question the idea of giving students loads of the same type of question to do following the teaching of a new concept. Sure, they can all do the questions by the end of the lesson (high retrieval strength), but then when asked to do a similar question next lesson, or next week, students often cannot (low storage strength). How do we rectify this? Well, maybe we give students less questions to answer on a given topic (to borrow a phrase from my past as an Economics student, there is a diminishing marginal utility to each one), allow them to forget it, and then cover the material again at a later date. This cycle of practising, forgetting, and then practicing again, as opposed to keep practising the same thing, allows the storage strength of these skills to increase, which is needed for long-term learning. Of course, this will be a frustrating experience for students, as well as us teachers, and crucially performance in the short-term may decrease. So, both students and teachers may need informing of the power of forgetting, and the crucial distinction between learning and performance.
My favourite quote:
Perhaps the prime example of forgetting enhancing learning is the spacing effect, one of the most robust and general effects from the entire history of experimental psychology When a second study opportunity is provided after a delay following a first study opportunity, rather than being presented with little or no delay, long-term recall is enhanced, often very significantly. Again, though, were the studied material to be tested following a short delay or a long delay, we would observe that the longer delay results in poorer recall of the studied material—that is, more forgetting. Similarly, interleaving, rather than blocking, the learning trials on separate to-be-learned tasks produces more forgetting between trials on a given task during the learning phase, but tends to enhance long-term retention and transfer.

Research Paper Title: Making Things Hard on Yourself, But in a Good Way: Creating Desirable Difficulties to Enhance Learning
Author(s): Elizabeth L. Bjork and Robert Bjork
My Takeaway:
This is paper is one of the best things I have ever read. It is directly related to the issue of distinguishing between learning and performance, and argues the seemingly counter-intuitive point that you improve learning by making it more (desirably!) difficult.  Bjork defines a difficulty as desirable if it makes retrieval practice harder in the short term but acts to increase retention and transfer. Therefore, these deliberate difficulties will likely lead to a reduction in short-term performance, but an improvement in long-term learning. The implications for lessons (and of course lesson observations!) are huge. The authors suggest the following strategies to induce desirable difficulties:
1) allowing students to forget some of the material covered before it’s reintroduced (spacing);
2) mixing up different content in order to prevent students developing the illusion of knowledge (interleaving);
3) asking questions about material which has already been covered rather than restudying it in order to prevent students developing a false sense of familiarity and fluency (retrieval practice);
4) varying the conditions in which instruction takes place in order to prevent contextual cues from building up and making it harder for students to transfer what they’ve learned to new contexts (variation);
5) progressively reducing the frequency and quantity of feedback given in order to prevent students from becoming dependent on external sources of expertise (see the Marking and Feedback section for a discussion on this amazing final strategy).
I fully agree with all of this, and we will look at some of Bjork's describable difficulties in the papers that follow, but I will add one thought - if things are always too difficult for students, then motivation quickly disappears, and no teaching strategy can be effective if your students have switched off and have zero desire to engage in the learning process. Indeed, the principle of describable difficulty does seem in contrast to Dan Willingham's view of thinking discussed on the Cognitive Science section - if thinking is too hard, students stop thinking - as well as Cognitive Load Theory, which seems to me to be a model based on making thinking as easy and focused as possible. Much like David Didau (in this wonderful blog post on the same paper), I feel students need to taste success first in order to be motivated (see the Zimmerman article on the importance of self-efficacy in the Motivation and Praise section for more on this), then try to ensure that key knowledge and processes are successfully encoded via a process of explaining, modeling, scaffolding and practising as suggested by Cognitive Load Theory, before finally things can become deliberately difficult. Didau proposes what seems a very sensible three stage plan to achieve this: Encode success, Promote internalisation, Increase challenge.
My favourite quote:
The basic problem learners confront is that we can easily be misled as to whether we are learning effectively and have or have not achieved a level of learning and comprehension that will support our subsequent access to information or skills we are trying to learn. We can be misled by our subjective impressions. Rereading a chapter a second time, for example, can provide a sense of familiarity or perceptual fluency that we interpret as understanding or comprehension, but may actually be a product of low-level perceptual priming. Similarly, information coming readily to mind can be interpreted as evidence of learning, but could instead be a product of cues that are present in the study situation, but that are unlikely to be present at a later time. We can also be misled by our current performance. Conditions of learning that make performance improve rapidly often fail to support long-term retention and transfer, whereas conditions that create challenges and slow the rate of apparent learning often optimize long-term retention and transfer.

Research Paper Title:
Spacing and Interleaving of Study and Practice
Author(s): Shana K. Carpenter
My Takeaway:
This is a really nice introduction to the concepts of spacing and interleaving, which are two of Bjork's "desirable difficulties" described above. Best of all, both of these concepts are relatively easy to implement. The Spacing Effect refers to the findings that learning is better when two or more exposures to information are separated in time (i.e. spaced apart) than when the same number of exposures occurs back-to-back in immediate succession. The Interleaving Effect contrasts a "blocking" approach, whereby students study the same type of material over and over again before moving on to a different type of material, against an "interleaving" approach, where students practice all of the problems in an order that is more random and less predictable. It is quite easy to get the two concepts mixed up (for me, anyway!), and the two are inextricably linked as interleaving produces spacing, but a good way to think about it might be that spacing describes the scheduling of exposures to a single concept (A), and interleaving describes the scheduling of exposures to multiple concepts (A, B, and C). The general findings are that both spacing and interleaving can produce significant benefits with regard to memory and learning, and these will be discussed further in the papers in this section. Spacing is relatively easy to implement. You just need to ensure you routinely revisit material at fixed intervals throughout the year - and a paper in this section provides guidance for what those fixed intervals should be. Similarly, tapping into the benefits of interleaving could be as simple as adapting how you present questions in class and modifying your existing homeworks and assignments slightly, as will be discussed in the papers in this section. Before we move on, a key point that also needs raising is that blocked practice might be more appropriate when a skill is first being learned. After all, spacing and interleaving work by inducing students to retrieve concepts from long-term memory. If concepts are not there in the first place, then there is nothing to retrieve!
My favourite quote:
Students and instructors are faced with these decisions on a daily basis. Research on human cognition has shown that learning can be significantly affected by the way in which repetitions are scheduled. This research can help inform the decisions that students must make concerning when to study information in order to maximize learning.

Research Paper Title:
Using Spacing to Enhance Diverse Forms of Learning
Author(s): Shana K. Carpenter, Nicholas J Cepeda,  Doug Rohrer, Sean H. K. Kang & Harold Pashler
My Takeaway:
The concept behind spacing is relatively simple, and its effects can be profound. This paper finds that performance on final tests of learning is improved if multiple study sessions are separated (spaced apart) in time rather than massed in immediate succession. The optimal length of the spacing depends on when the material is going to be tested. In general, longer spacings are more beneficial when the test is a long way away, whereas shorter spacing are more beneficial for tests in the near future. These longer periods may be weeks or even months. Three practical recommendations for teachers:
1) Teachers should dedicate part of each lesson to reviewing concepts learned several weeks earlier. I often do this during the starter, either by means of a low-stakes quiz (see the Testing section), or using something like Corbett Maths 5-a-day. The key is careful planning to ensure that as much previously taught material as possible comes up again.
2) Homework assignments should be used to re-expose students to important information they have learned previously. We have a Revision section at the start of every homework, but as will be discussed in the Interleaving papers that follow, there may be a way to make this even more effective.
3) Teachers should give exams and quizzes that are cumulative. This again suggests the benefits of low stakes quizzes, which not only tap into the benefits of The Spacing Effect, but which also utilise the Retrieval Effect that will be discussed in the Testing section
This last point even has an added benefit - as well as re-exposing students to information that they have previously learned, cumulative exams and quizzes also provide students with a good reason to review information on their own, hence hopefully improving their revision strategies. Revision is covered in a later section.
My favourite quote:
The key criterion is that information should be reviewed after a period of time has passed since the initial learning. Particularly if the goal is long-term retention, the findings from Cepeda et al. (2008) suggest that the ideal time to review information may be several weeks or months after it was initially learned.

Research Paper Title: Spacing effects in learning: A temporal ridgeline of optimal retention
Author(s): Nicholas J Cepeda,  Edward Vul, Doug Rohrer, John T Wixted & Harold Pashler
My Takeaway:
After the last paper you may be thinking: "wouldn't it be great if someone could tell me the optimal time to revisit content with my students depending on when their test is?". Well, your wish is my command. This study offers a rough guide. The research looked to trial a number of different gaps with a number of  different retention intervals. The gap + the RI = the total time between initial study and test. They did not find any magic ratio between the gap and the retention interval. As predicted, as the retention interval increased, so too did the optimal gap for retesting, but not in a constant fashion Indeed, the ratio of retention interval to gap falls. Mr Benney has written a wonderful blog post that contains a graph plotting optimal spacing intervals based on the data. For example, if you finish a topic today and the test is in 60 days then (perhaps) your optimum gap before restudy would be 10 days which leaves an RI of roughly 50 days (a 1:5 ratio or gap being 20% of RI), whereas If your class finishes a topic today and the test is in 114 days then (perhaps) your optimum gap before restudy would be 14 days which leaves an RI of 100 days (a 7:50 ratio or gap being 14% of RI). This is not surprising if we assume that the optimal time to retest someone is when they are on the verge of forgetting. As Mr Benney explains: It seems that it is preferable to have the restudy session within a shortish time of the original study despite the fact it gives a very long RI. This is surely because if the gap was any bigger too much of the original study material would have been forgotten and retrieval strength would be practically zero. It is better to keep a relatively short gap and trade off with a very long RI. I imagine a spacing session much later than the optimum gap would be more like a reteaching lesson rather than a restudy/recall lesson.
However, as the authors of the research point out, there is a danger in going too far the other way. The compression of learning into a too-short period is likely to produce misleadingly high levels of immediate mastery that will not survive the passage of substantial periods of time. Moreover, while there are costs to using a gap that is longer than the optimal value, these costs are much smaller than the costs of using too small a gap value, as evidenced by the fact that, as gap increases, accuracy increases steeply and then declines much more gradually. Hence, if in doubt, leave the intervals between tests of recall longer than shorter. Students need time to be on the verge of forgetting in order for the Testing Effect to display its full power.
My favourite quote:
To put it simply, if you want to know the optimal distribution of study time, you need to decide how long you wish to remember something. Although this poses challenges for practical application, certain conclusions can nonetheless be drawn. If a person wishes to retain information for several years, a delayed review of at least several months seems likely to produce a highly favorable return on a time investment— potentially doubling the amount ultimately remembered, holding study time constant—as compared to less temporally distributed study.

Research Paper Title: Interleaving Helps Students Distinguish among Similar Concepts
Author(s): Doug Rohrer
My Takeaway:
Students confusing related concepts is the single biggest cause of errors on my Diagnostic Questions website. Classics include confusing area and perimeter, the rules for adding and multiplying fractions, mean and median, the laws of algebra - the list goes on! This wonderful paper which summarises the key findings from a wide range of interleaving research which suggests that interleaved mathematics practice helps students learn to distinguish between different kinds of problems. This is obviously a critical skill because solving a mathematics problem requires that students first identify what kind of problem it is, which means that they must identify those features of a problem that indicate which concept or procedure is appropriate. Identifying the kind of problem is not always easy. Take solving equations. The familiar instruction, “Solve for x” does not indicate which one of several solving strategies is appropriate. For instance, students must use the quadratic formula to solve some equations (e.g., x2 − x − 100 = 0), and they must factorise to solve others (e.g., x3 − x = 0). Then there is completing the square, not to mention linear equations, cubic equations, equations involving fractions, and so on! Solving a mathematics problem requires students to know which strategy is appropriate and not only how to execute the strategy. The key point here is that with blocked practice, students need not identify an appropriate strategy because every problem in the assignment can be solved by the same strategy. In essence, blocking provides scaffolding. This might be useful when students meet a new concept, but students who receive only blocked assignments do not have the opportunity to practice without this crutch, and studies quoted here suggest this has negative consequences for long-term learning. So, what are we to take from this? Firstly, it seems clear that interleaving practice problems on homeworks and tests is likely to have a significant advantage on long-term learning, and this will be discussed in the papers that follows. But what are the implications for the actual teaching of related concepts? For me, there are two extremes:
1) Teach related concepts together and block practice each. For example, teach the mean, then practice the mean. Then teach the median, and practice the median, and so on, until all four averages have been covered. What happens with that approach - students confuse those related concepts as they have not had sufficient opportunity to distinguish between them.
2) Teach related concepts completely separately. So, teach the mean, then teach adding fractions, then area of a triangle, before returning to teaching the median. I have not been able to find specific research on this approach, but I would be fascinated by it. Whilst it undoubtedly taps into the benefits of interleaving, my concern is that it does not allow students to compare and contrast the related concepts, and to pick up on the connections between them. Take averages. In the scenario described here, how do we teach students about the appropriateness of using each average, or solve complex problems involving more than one of them, when they are taught so far apart?
I think the solution is to teach related concepts in isolation in early skill acquisition phrase, using the principles of Explicit Instruction and Cognitive Load Theory, and then carefully bring them together later on. So, introduce the mean in Year 7 term 1, perhaps as part of a unit on number operations, or using a calculator. Then introduce the median at the start of Year 8, and the mode sometime towards the end of the year. Continually revisit these concepts throughout the two years via starters, low-stakes quizzes and homeworks. Then, in Year 9 it is time to bring them all together in a unit on statistical inferences. At this stage, with students fluent in each of them, similarities and differences between the related concepts can be made explicit. Of course, this would have big implications for a school's existing schemes of work, homeworks and assessments, but the effort put into getting this right could have huge payoffs.
Finally, as highlighted in the quote, there is little doubt that interleaving makes learning more difficult and is likely to lead to a short term dip in performance. Both students and teachers need to be aware of this.
My favourite quote:
Students might balk because interleaving increases the difficulty of a question or problem. A group of questions or problems are easier when all relate to the same topic or concept. By contrast, answering a set of interleaved biology questions might require students to consult material presented in previous chapters, and an interleaved mathematics assignment prevents students from simply repeating the same procedure throughout the assignment. Students will therefore make more errors, and work more slowly, when assignments are interleaved. This in itself is not problematic because the aim of classroom instruction is ultimate mastery, not error-free learning. Still, some students might be unwilling to make the extra effort. In this scenario, interleaving is like bad-tasting cough syrup—ineffective because children refuse to use it.

Research Paper Title:
The Effects of Cumulative Practice on Mathematics Problem Solving
Author(s): Kristin H Mayfield and Philip N Chase
My Takeaway:
This study is discussed in the paper above, but deserves its own place here, not least because it is mathematics specific. College students in need of mathematics intervention attended dozens of sessions over a period of several summer months in which they solved problems using five algebraic rules about the laws of indices. One group of students learned the rules
through a procedure akin to blocking, in which each rule was learned and then practiced extensively before moving on to the next rule. Another group learned the same rules through a procedure akin to interleaving, which involved continuous practice of previously-learned skills. For example, after learning two types of skills (e.g., order of operations and multiplying indices), students practiced a mixture of problems involving these two skills. A third skill was then introduced (e.g., finding the roots of indices) and this skill was then added to the practice set along with the two previously-learned skills, such that participants practiced a mixture of problems tapping all three skills. A fourth skill was then added, followed by a mixture of problems tapping all four skills, and so on, until all five skills had been learned and practiced. Subjects were tested 1 or 2 days after the last practice session, and they returned for a second test between 4 and 12 weeks later, depending on their availability. On both tests, the interleaved practice group outscored the blocked practice group by factor of at least 1.3, both on skill-based questions and crucially also on problem solving. The next few papers that we will look at extol the benefits of interleaving in terms of the the content of homework and assessments. However, I feel this paper is more about how topics are taught. Previously I would have taught a topic in a block (say fractions), assessed it, and then moved on. The lesson here is that once I have taught fractions, I need to ensure fractions regularly appear in the study of other topics. This could be as simple as ensuring they are parts of homework and starters, but better still they should be integrated within the new topics. So, fractions become part of negative numbers, calculating the mean, and so on. Such an approach goes by many different (often incorrectly applied) labels, such as Mastery or Shanghai. But one thing is clear, it has significant implications for the planning of curriculum and schemes of work.
My favourite quote:
In summary, the results from the current study suggest that incorporating cumulative practice into training procedures will lead to high levels of performance on novel, untrained skills. More specifically, what are typically thought of as advanced mathematics skills, such as applying individually trained rules in a novel situation and synthesizing rules into novel combinations, can be facilitated through a cumulative practice training procedure. Neither providing extra practice on each component rule nor incorporating individual reviews of previously trained rules proved to be adequate to produce similar results, particularly on problem-solving skills.

Research Paper Title: Interleaved Practice Improves Mathematics Learning
Author(s): Doug Rohrer, Robert F. Dedrick, and Sandra Stershic
My Takeaway:
This study is fascinating. Previously I had thought interleaving had to involve completely changing the order lessons are taught. However, that is not the case. Seventh-grade students saw their teachers’ usual maths lessons and received regular homework assignments. Every student received the exact same problems, but the scheduling of the problems was altered so that students received blocked or interleaved practice. Later, students received a review of all the content, followed 1 or 30 days later by an unannounced test. Students following the interleaving program performed significantly better on both tests, with a greater difference in performance found on the test taken 30 days later, suggesting a significant impact on retention. Now, the "blocked practice" approach is one I have used regularly. Teach students fractions, give them an assignment on fractions. Then teach them equations, given them an assignment on equations. Then it is time for angles and averages, then let's have a review of all four topics, and then a half term test. Our assignments do consist of short revision questions at the start which are taken from a variety of questions throughout the year, but the main focus of each assignment is undoubtedly on the topic that has just been covered. This paper suggest that approach is not quite good enough. The assignments should consist of a mixture of questions from previous topics, and not just the quick-fire revision ones. Moreover, each assignment should contain a majority of questions from other topics, not a minority. For example, the kind of fractions problems that would have appeared as questions 4, 6 and 9 on our first assignment should be included in the second assignment instead, alongside questions on equations and any other topics previously taught, and so on. I particularly like the fact that there are no implications for changing the way or order topics are taught, or the content of assignments - it is simply a case of varying the composition of the assignments themselves. This seems like a very quick win.
My favourite quote:
Benefits of interleaved practice have been consistently observed with a variety of mathematics skills and with students in elementary school, middle school, and college. As argued here, these benefits arise because interleaved practice provides students with an opportunity to learn how to choose an appropriate strategy (or learn that they cannot do it). In short, interleaved practice simply provides students with an opportunity to practice the very skill they are expected to learn.

Research Paper Title: The Benefit of Interleaved Mathematics Practice is not limited to Superficially Similar kinds of Problems
Author(s): Doug Rohrer, Robert F. Dedrick & Kaleena Burgess
My Takeaway:
This paper discusses the point raised in the previous paper, but then takes it further. The correct solution to most mathematical problems involves two steps: identify the strategy needed to solve the problem, and then successfully carry out that strategy. The authors argue that most mathematical assignments deny students the opportunity to practice that first step - identifying the strategy. For example, if a lesson on the Pythagorean theorem is followed by a group of problems requiring the Pythagorean theorem, students know the appropriate strategy before they read each problem. Similarly, blocked practice fundamentally changes the pedagogy of worded problems, where often the majorly difficulty is not carrying out the strategy required to solve the problem, but actually identifying what that strategy should be. An alternative approach is interleaving, where a majority of the problems within each assignment are drawn from previous lessons, so that no two consecutive problems require the same strategy. With this approach, students must choose an appropriate strategy and not only execute it, just as they must choose an appropriate strategy when they encounter a problem during a cumulative exam or high-stakes test. Such interleaved practice also ensures that problems are spaced, which can tap into the benefits related to The Spacing Effect. Interestingly, unlike the previous paper, the authors do not attribute the benefit of interleaving to merely enabling students to better discriminate between problems. They argue solving a mathematics problem requires students not only to discriminate between different kinds of problems, but also to associate each kind of problem with an appropriate strategy. Blocked assignments often allow students to ignore the features of a problem that indicate which strategy is appropriate, which precludes the learning of the association between the problem and the strategy. Interleaved practice can help the development of both of these crucial skills. However, I must point out that many of Bjork's "desirable difficulties" seem inappropriate in early skill acquisition phrase. A degree of blocked practice may be necessary in lessons when a topic is being introduced, followed by interleaved practice for homework.
My favourite quote:
Although it might seem surprising that a mere reordering of problems can nearly double test scores, it must be remembered that interleaving alters the pedagogical demand of a mathematics problem. As was detailed in the introduction, interleaved practice requires that students choose an appropriate strategy for each problem and not only execute the strategy, whereas
blocked practice allows students to safely assume that each problem will require the same strategy as the previous problem.


Research Paper Title: Effect of Overlearning on Retention
Author(s): Jaems E Driskell, Ruth P Willis and Carolyn Cooper
My Takeaway:
The important distinction between learning and performance naturally raises the question: when do I know if my students have practiced enough? A score of 10/10 is certainly a good performance, but how much of that is down to mimicry, and how much has been retained in long-term memory? That leads to the concept of overlearning - or, as I like to think of it, practising beyond mastery. This meta-analysis finds a moderately positive effect on long-term retention of overlearning. In short, learning can still occur after performance has seemingly peaked. I took two things away from this paper. Firstly, I will no longer settle for a single good performance on, say, a fractions test. Students need to continue practising, even if they do not see a visible improvement in performance. Concepts such as spacing in lessons and interleaving in homeworks will help with this. Secondly, students need to be informed of this concept, otherwise there is the danger they will moan at the fact you are making them do something they can already do. 
My favourite quote:
One question that organizations and training practitioners face is the question Of how much training is enough. One approach is to provide the level Of training that is estimated to  meet the requirement of the average trainee. That is, if the average trainee can achieve proficiency in 5 sessions, then this becomes the level Of training provided to all. A second approach is to provide training to meet some set criterion, such as one errorless performance of the task. In this case, each individual is trained until he or she reaches the criterion level. A third approach, overlearning. requires that training continue for a period past this initial mastery level. The results of this meta-analysis document the effectiveness of overlearning and show that retention is enhanced when learning proceeds beyond initial mastery

Research Paper Title: The Effects of Overlearning and Distributed Practise on the Retention of Mathematics Knowledge
Author(s): Doug Rohrer and Kelli Taylor
My Takeaway:
The final paper on interleaving from my Doug Rohrer collection, and this has equally big implications. In two experiments, 216 college students learned to solve one kind of mathematics problem before completing one of various practice schedules. In Experiment 1, students either massed 10 problems in a single session or distributed these 10 problems across two sessions separated by 1 week. The benefit of distributed practice was nil among students who were tested 1 week later but extremely large among students tested 4 weeks later. In Experiment 2, students completed three or nine practice problems in one session. The additional six problems constituted a strategy known as overlearning, but this extra effort had no effect on test scores 1 or 4 weeks later. A few things struck me. Firstly, the improvement in performance from distributed practice was not seen immediately - this is important for both students and teachers to know. Secondly, as has been pointed out in the previous studies, most mathematics textbooks rely on a format that emphasises overlearning and minimises distributed practice, as do most of the worksheets and sets of questions I give my students to do in class. Finally, with regard to overlearning, the authors do not dismiss it (and indeed, with the problem of distinguishing between learning and performance discussed earlier in this section, it is dangerous to assume a student has learned something based purely on a good test score), but instead suggest a better use of students' finite time. The authors sum this up very nicely: "we suggest that assignments should err slightly in the direction of too much practise, perhaps by including three or four problems relating to each new concept in the most recent lesson (in addition to any examples given in the written lesson or class lecture). However, beyond these first three or four problems, the present data suggest that the completion of additional problems of the same type is a terribly inefficient use of study time. Instead, our findings suggest that the student should devote the remainder of the practise session to problems drawn from earlier lessons in order to reap the benefits of distributed practise"
My favourite quote:
With this distributed practise format, each lesson is followed by the usual number of practise problems, but only a few of these problems relate to the immediately preceding lesson. Additional problems of the same type might also appear once or twice in each of the next dozen assignments and once again after every fifth or tenth assignment thereafter. In brief, the number of practise problems relating to a given topic is no greater than that of typical mathematics textbooks, but the temporal distribution of these problems is increased dramatically.

Research Paper Title: Why interleaving enhances inductive learning
Author(s):
Monica S. Birnbaum, Nate Kornell, Elizabeth Ligon Bjork & Robert A. Bjork
My Takeaway:
A fascinating paper to end our discussion on interleaving (for now!). An inductive approach to teaching involves showing examples of how a concept is used in the hope that students will ‘notice’ how the concept works, as opposed to a deductive approach, whereby the teacher provides the material pupils need to think about and reduces the quantity of information they are required to hold in working memory. It would seem sensible to think that a deductive approach works best, and indeed when learning basic content that does seem to be the case. However, this research suggests that students tend to understand and remember more when learning occurs inductively, but only if this is combined with an interleaved approach to studying content. In short - students best learn a rule by seeing examples containing the rule mixed up with examples that don’t. Why does this work? Well, frequent alternation of categories (interleaving) has the advantage of highlighting features that serve to distinguish categories. Conversely, infrequent alternation of categories (blocking) has the advantage of highlighting information that remains constant across the members within a category. For me, it depends what you want to achieve. If you wish students to become fluent in a particular skill or concept, then a blocking approach is likely to be best. However, if you want students to be able to distinguish between problems, recognising key features and selecting the most appropriate method to use (as is often needed for more wordy questions, questions where the required method is not obvious, or on high-stakes tests), then interleaving is necessary. The research we looked at in the Explicit Instruction section suggests that students need key knowledge and processes to be stored in long term memory before they can become effective problem solvers, so my takeaway is that blocked practice may be optimal when students first encounter a skill, but interleaving is needed for students to develop their learning further. On a related note - learners (and teachers!) are likely to find the process of interleaving more difficult than blocking. This is largely because they have to think more, and may see a short term dip in performance. Crucially, they may feel they are not learning as much and begin to lose motivation. This relates directly to Bjork's "desirable difficulties" discussed earlier in this section.
My favourite quote:
The great majority of the  participants in the present study, as well as those in prior  studies, judged that they had learned  more effectively with blocked than with interleaved study. Thus, a bit of practical advice to learners and educators seems  warranted: If your intuition tells you to block, you should  probably interleave.

Research Paper Title:
Environmental context and human memory
Author(s): Steven M Smith, Arthur Glenberg and Robert A Bjork
My Takeaway:
Another of Bjork's "desirable difficulties" concerns varying the location of where learning and retrieval take place. In this fascinating study, the authors gave participants a lost of words to learn and had them study either in the same room twice or in two separate rooms (one a cluttered basement, the other a windowed office). Three hours later, the students were moved into a third ‘neutral room’ and asked to recall as many of the words as they could. Those who studied in the same room managed an average of 16 out of 40 whereas those who studied in two different rooms recalled 24 words. The experiment showed strong recall improvements with variation of environmental contexts.This has become known as the Variation Effect and since then there are been many other studies which have replicated this findings. Why does varying conditions work? Well, it may enhance long-term learning because the material becomes associated with a greater range of memory cues that serve to facilitate access to that material later. Whilst it is clear that this experiment is concerned with the recall of words - and so we must be very careful in making any wild claims about solving mathematical problems - the fact that recall in mathematics plays an important part suggests that we should take note of these results. For me, there are two clear implications. Firstly, wherever possible vary where students in school learn and practice retrieval. My students sit in the same seats, in the same classrooms, most of the time. And yet, when it comes to taking their exams they are suddenly thrust into a brand new environment. As a result of this paper, I now routinely change my seating plans, and where possible try to take advantages of room changing opportunities. There is the potential cost of disruption and daft behaviour that this "new environment" brings, but the potential benefit far outweighs these. Secondly, a simple piece of advice to pass onto students when revising is to vary where they do their revision. Try a different room, a different time of day, etc. Anything that varies the environmental conditions is likely to lead to better retrieval.
My favourite quote:
Another type of explanation of the results can be found in Tulving and Watkins' (1973) continuity hypothesis, which states that recall and recognition require essentially the same retrieval process(es), but that free recall involves such processes to a greater extent than does recognition. In free recall, where the experimenter provides few explicit retrieval cues, the subject must make use of varied sources of cues to provide access to the stimuli. Environmental context could be one such source of cues. In cued recall there is more specific retrieval information, and in recognition memory the subject is supplied with highly effective, explicit cues, thereby reducing his dependence on other cue sources.



Memory: Testingkeyboard_arrow_up
Back to Top

One of the golden rules of school life is that students hate tests. But I would like to refine this rule to say students hate "high-stakes" tests. The papers in this section will outline a finding that has possibly transformed my lessons more than any other: the power of regular, low-stakes testing. By low-stakes, I mean short tests (typically 3 to 5 questions) that the students take at the start of every single lesson based on content from earlier in the year (to take advantage of the Spacing and Interleaving effects), they mark them themselves, the marks are not recorded anywhere, I don't know their scores, students don't know each others' scores, and I go through the answers straight away so students have immediate feedback. You will not believe me when I say this, but the students love them. More importantly, the effects on their retention have been profound. The biggest impact this has had on my teaching is that I now see tests as "learning events", as opposed to just a means of assessment. When I interviewed Dani Quinn for my podcast, she explained how Michaela Community School makes extensive use of regular weekly tests. In this section I attempt to get to the bottom of the power of testing.

Research Paper Title: Ten Benefits of Testing and their Application to Educational Practice
Author(s): Henry L. Roediger III, Adam L. Putnam and Megan A. Smith
My Takeaway:
This paper is stunning. It details (as the title suggests) ten benefits of testing, citing research to support each claim. Each benefit is worth discussion here, but I am going to limit myself to just three.
1) The Testing Effect: retrieval aids later retention. A fascinating study described here explains that students who followed a pattern of study-study-study-test for a topic performed better on the final test than a group who followed study-test-test-test. But - and here is the key - when tested a week later, the exact opposite had occurred, even though the latter group had much less exposure to the material. Testing led to better long-term retention. The Testing Effect is examined further later in this section.
2) Testing causes students to learn more from the next learning episode. When students take a test and then restudy material, they learn more from the presentation than they would if they restudied without taking a test, hence it helps with revision.
3) Testing improves transfer of knowledge to new contexts. This is the holy grail! We have seen in the Cognitive Science section that the inability to transfer knowledge to new situation is one of the characteristics that defines novice learners and inhibits their learning. It seems that testing can help with that, possibly by enabling learners to form and strengthen schemas that they can later draw upon in new contexts.
Add to all of this the benefit that testing can give you the teacher information about gaps in students' knowledge which you can resolve in class, and it promotes the benefits of testing to students so that they may use it in their revision, and you have possibly one of the most important teaching tools ever.
My favourite quote:
We have reviewed 10 reasons why increased testing in educational settings is beneficial to learning and memory, as a self-study strategy for students or as a classroom tactic. The benefits can be indirect—students study more and attend more fully if they expect a test – but we have emphasized the direct effects of testing. Retrieval practice from testing provides a potent boost to future retention. Retrieval practice provides a relatively straightforward method of enhancing learning and retention in educational settings.

Research Paper Title: The Critical Importance of Retrieval for Learning
Author(s): Jeffrey D. Karpicke, et al
My Takeaway:
For years I have underestimated just how powerful actively practising retrieval can be, relative to other more common revision techniques. Three findings in this paper particularly stood out to me.
1) Repeated studying after learning had no effect on delayed recall, but repeated testing produced a large positive effect. I have seen this myself - students think they have cracked a topic like adding fractions because they have got questions correct in the past, and so during revision may merely glance over their notes on fractions (studying) instead of trying questions out (retrieval). Such revision feels easy, and hence has little effect on long term retention - as we have seen from the work of Bjork in the Memory section above, learning needs to be difficult.
2) Students’ predictions of their performance were uncorrelated with actual performance. The students in the study were not aware of the benefits of practising recall, even after they had done it! This is both fascinating and worrying, and suggests that students need to be convinced of the power of this strategy long before they start their revision. Regular low-stakes testing in the classroom seems a sensible way of achieving this.
3) Finally - and I have saved the best to last - if retrieval is so important for learning (as you will see in the next paper, it can actually cause learning), then is it a good idea to just do a load of past papers for exam preparation, as I have done for many years? NO! Why? Well, exam papers are designed to cater to a wide variety of abilities. Hence, they do not always encourage retrieval from long term memory, but instead test elements of problem solving (specifically, elements not stored in long term memory). We have seen in the Cognitive Load Theory section that problem solving does not always lead to learning, and hence students can work through exam papers and not actually learn anything. I conclude that until students have covered the full course, regular, low-stakes tests on topics that students have already studied is the best way to profit from the positive effect of retrieval.
My favourite quote:
The conventional wisdom shared among students and educators is that if information can be recalled from memory, it has been learned and can be dropped from further practice, so students can focus their effort on other material. Research on students’ use of self-testing as a learning strategy shows that students do tend to drop facts from further practice once they can recall them. However, the present research shows that the conventional wisdom existing in education and expressed in many study guides is wrong. Even after items can be recalled from memory, eliminating those items from repeated retrieval practice greatly reduces long-term retention. Repeated retrieval induced through testing (and not repeated encoding during additional study) produces large positive effects on long-term retention.
 

Research Paper Title:
Retrieval Practice Produces More Learning than Elaborative Studying with Concept Mapping
Author(s): Jeffrey D. Karpicke, et al
My Takeaway:
This paper is by the same team of authors as the previous one, but makes two key additional points.
1) It makes explicit the finding that practising retrieval is more effective on long term retention than attempting to encode information during the learning process - possibly using strategies such as mind-mapping, and making notes. Practising retrieval was found to be more effective than more traditional revision strategies when students were later tested on both factual recall and more problem solving questions.
2) However, possibly the more interesting point is that the researchers found that retrieval practice can actually produce learning (as opposed to being neutral). That is, tests act not only as passive assessments of what is stored in memory (as is often the traditional perspective in education) but also as vehicles that modify what is stored in memory. This startling finding is called the Testing Effect (or the Retrieval Effect) and is possibly due to the cognitive strain experienced when trying to reconstruct knowledge, which is related to the fascinating paper on Desirable Difficulties discussed later in this section. Once again, for me this emphasises how important it is that students are aware of the power of self-testing during revision (and not necessarily of complete exam papers, as discussed in the paper above), and the importance of low-stakes tests in the classroom.
My favourite quote:
Research on retrieval practice suggests a view of how the human mind works that differs from everyday intuition. Retrieval is not merely a read out of the knowledge stored in one’s mind; the act of reconstructing knowledge itself enhances learning. This dynamic perspective on the human mind can pave the way for the design of new educational activities based on consideration of retrieval processes.

Research Paper Title: Test anxiety in UK schoolchildren: Prevalence and demographic patterns
Author(s): David W. Putwain
My Takeaway:
Test Anxiety is a concept I have observed many times in many of my students over the years, but never attributed this particular label to it. It can be defined as a psychological condition in which people experience extreme distress and anxiety in testing situations. I notice this most in the build up to high-stakes exams, such as GCSEs or A Levels. Students who have been calm and composed throughout the rest of the year begin to fall apart - they cannot sleep, they look anxious in class, they start getting things wrong that they never previously had a problem with. This is a huge issue, and one I have had many a long conversation with concerned parents about. This fascinating paper sheds some light on the issue. The author was specifically looking at UK students in their final two years of compulsory schooling (i.e. Key Stage 4), which he describes as being “of critical importance to the future life trajectory of the student”. Firstly, it was clear that tests induce anxiety in students, and that this anxiety is likely to inhibit performance. One explanation purported for this is that the "worry" component of anxiety takes up valuable working memory space, which makes the processing of the kind of complex tasks you are likely to find on a high-stakes exam more difficult. However, contrary to his predictions, Putwain did not find that the higher stakes exams produced the most anxiety; instead he found the exam with the lowest stakes produced the highest anxiety. But what did he mean by that? Well, Putwain labeled a mock exam as his lowest stakes variable. It is possible that mock exams could be highly anxiety producing for students if their perceptions of these exams were that current performance will predict future performance. In this situation the way in which the “mock” exams were presented to the students would be very important - "I got an E on this mock, that is what I am going to get in the real thing, my life is officially over!" Also, in Putwain’s research all self-reported anxiety measures were administered after taking the exam in question. Student’s taking mock exams may realise how much more they need to prepare which may have caused the higher self-reported anxiety levels after the mock exams in his study. So, what are we to make of all this? Well, firstly it has made me acutely aware that test anxiety is a real and serious thing, with implications both for students' psychological well-being and their performance on tests. Secondly, the finding that "lower-stakes" tests seemingly produce more anxiety than their high-stakes cousins has in fact made me more convinced of the validity and importance of an increase in low-stakes testing! Let me try to explain. Students in the study exhibited the most anxiety from, mock exams. Why? Well, possibly because they were not used to taking exams, and this was the first time their ability to retrieve knowledge was put to the test. We have seen how reading notes makes it far easier to convince yourself you know something versus explicitly testing retrieval. So, it is no surprise that when students struggled with retrieval in the mocks, got a rubbish mark that they never saw coming ("but I knew it all when I read it last night, sir") and then realised their actual exams were a matter of weeks away, that panic ensued. How to combat this? For me, it is regular low-stakes tests. These have all the benefits explained earlier on in this section (identifying areas of weakness, making future study more effective, and even causing learning), with the added bonus that they are likely to get students more comfortable with testing as a concept, and hence hopefully reduce the overall level of anxiety. We are never likely to reduce test anxiety completely, and possibly nor should we strive to. It is important students take high-stakes tests seriously, we just do not want their performance to be inhibited by something that we can possibly control.
My favourite quote:
Later models such as the ‘processing efficiency’ theory (Eysenck & Calvo, 1992) suggest that the additional demands on working memory resources made by task-irrelevant worry cognitions may reduce processing efficiency, but not necessarily the effectiveness. A highly test anxious student could maintain effectiveness on tasks requiring low working memory demands with extra effort to compensate for lowered efficiency. A decline in processing effectiveness would only be predicted on assessment demands making heavy demands on working memory resources (e.g. difficult questions, high memory load, tasks involving coordinative complexity, etc.). Only under these conditions, would a decline in task performance manifest.


Research Paper Title:
Both Multiple-Choice and Short-Answer Quizzes Enhance Later Exam Performance in Middle and High School Classes
Author(s): Kathleen B. McDermott, Pooja K. Agarwal, Laura D’Antonio, Henry L. Roediger, III, and Mark A. McDaniel
My Takeaway:
This paper is key for proponents of low-stakes testing. There is little surprise that the authors found that practicing retrieval of recently studied information enhances the likelihood of the learner retrieving that information in the future, as we have seen the benefits of testing for retrieval throughout this paper. But the key difference with this paper is that  the format of the quiz (multiple-choice or short-answer) did not need to match the format of the critical test (e.g. end of unit exam) for this benefit to emerge. This supports the research related to deliberate practice, whereby the activities involved in practice do not need to exactly replicate the final performance for the practice to be effective. The authors also find that frequent classroom quizzing with feedback improves student learning and retention, and multiple-choice quizzing is as effective as short-answer quizzing for this purpose. There is more discussion of the merits of multiple choice questioning in the Formative Assessment section, but a key takeaway for me here is that both multiple-choice questioning and short form skill-based questions are ideal to use in the classroom for regular low-stakes testing.
My favourite quote:
First, we consistently observed that the format of the quizzes did not have to match the format of the unit exam for the quizzing benefits to occur. This is the most important finding of the present report, in that it is novel, was unanticipated from the laboratory literature, and is a critically important practical point for teachers. Even quick, easily administered multiple-choice quizzes aid student learning, as measured by unit exams (either in multiple-choice or short-answer format). Further, the benefits were long lasting: Robust effects were seen on the end-of-semester exams in Experiments 1a, 1b, and 2; that is, both multiple-choice and short-answer quizzing enhanced performance on end-of-semester class exams (again, in both multiple-choice and short-answer formats).

Research Paper Title: The Pretesting Effect: Do Unsuccessful Retrieval Attempts Enhance Learning?
Author(s): Lindsey E. Richland,  Nate Kornell,  Liche Sean Kao
My Takeaway:
So far we have seen the power of testing to enhance retrieval and long-term learning. If that wasn't impressive enough, the next two papers in this section highlight another, rather surprising, benefit of testing - the power of the Pretest.  Firstly, unlikely as it sounds, generating a wrong answer increases our chances of learning the right answer. In this study, one group of students was given the text on which they would be tested – passages with key facts marked, a second group was given the opportunity to memorise the questions they would be asked, a third group was given an extended study period and a fourth group was given a pretest. Even though they got almost every pretest question wrong, students in the pretesting group out-performed all other groups on a final test, including those students who had been allowed to memorise the  test questions. It would seem that the act of unsuccessfully attempting to answer questions has a greater effect on learning than studying the questions on which you are to be tested. The simple takeaway here is that it doesn't matter if students get things wrong - it is the act of attempting to retrieve the answer that is the key. However, we are once again faced with the same Learning v Performance dilemma - a dip in short-term performance (students getting the initial questions wrong) is the price to be paid for an increase in long-term learning. This, of course, needs communicating to teachers and students, and is expressed beautifully in the quote below.
My favourite quote:
When a learner makes an unsuccessful attempt to answer a question, both learners and educators often view the test as a failure, and assume that poor test performance is a signal that learning is not progressing. Thus, compared with presenting information to students, which is not associated with poor performance, tests can seem counterproductive. Tests are rarely thought of as learning events, and most educators would probably assume that giving students a test on material before they had learned it would have little impact on student learning beyond providing teachers with insight into their students’ knowledge base. In terms of long-term learning, however, unsuccessful tests fall into the same category as a number of other effective learning phenomena - providing challenges for learners leads to low initial test performance, thereby alienating learners and educators, while simultaneously enhancing long-term learning. The current research suggests that tests can be valuable learning events, even if learners cannot answer test questions correctly, as long as the tested material has educational value and is followed by instruction that provides answers to the tested questions.

Research Paper Title: Unsuccessful Retrieval Attempts Enhance Subsequent Learning
Author(s): Nate Kornell, Matthew Jensen Hays, and Robert A. Bjork
My Takeaway:
The previous paper outlined the potential benefit to long-term learning of the Pretest Effect, but surely being given a test on something students have never studied before is a complete waste of time? Surely students will just end up guessing, and we all know that guessing is pointless. Well, apparently not! This study sought to answer this question: does an unsuccessful retrieval attempt impede future learning or enhance it? The authors examined this question  using materials that ensured that retrieval attempts would be unsuccessful. They found that as long as students are given appropriate feedback, they will still see a testing benefit even if they get the answer wrong on a pretest. So, here we have a crucial consideration - the benefits of the Pretesting Effect are only realised if immediate feedback is given. Students need to know they are wrong, and what the right answer is.  Often I give my students some form of baseline test before starting a new topic. More often than not, as discussed in the Formative Assessment section, this involves asking a series of Diagnostic Questions at the start of my lesson, listening to their answers an explanations, and adapting my teaching accordingly. The benefit of this approach (or so I had assumed) was purely so I could get a sense of the current levels of understanding in the class, address any misconceptions, and move on to the new topic when I deemed the class ready. Amazingly, it seems that the students were also benefiting from this approach, even if they were being tested on material they have never encountered before. So, a written baseline test before teaching a new topic is likely to have a positive effect, so long as you go through the answers immediately with students. But, if you combine it in a formative assessment setting, then the benefits are likely to be even greater. One word of caution: students may become demoralised if you give them a test on material they cannot do (and indeed should not be expected to do if they have not studied it before). My advice here is to explain, open and honestly to them, that this test is purely for you, their teacher, to find out where they are at so you can better help them, and that the mere act of them attempting to answer the questions is actually helping them learn.
My favourite quote:
The current findings support Izawa’s (1970) argument that tests potentiate the learning that occurs when an answer is presentedafter a test, even if the test is unsuccessful. The results also suggest that, in situations where tests and study opportunities are interleaved or testing is followed by feedback, the benefits of testing go beyond the benefits attributable to the learning that happens on successful tests. With respect to theoretical explanations of the testing effect, this finding is important because it demonstrates that the benefits of testing are not limited to the benefits of successful retrieval; rather, for a theory to fully explain the benefits of tests, it needs to explain the benefits of retrieval failure as well as the benefits of retrieval success. Successful tests obviously play a role, and perhaps a unique role—the findings do not imply that unsuccessful tests and successful tests are equally effective or that they are necessarily effective for the same reasons—but unsuccessful tests can also have a positive effect on long-term retention.

Research Paper Title: The generation effect: A meta-analytic review
Author(s): Sharon Bertsch, Bryan J Pesta, Richard Wiscott and Michael A McDaniel
My Takeaway:
Closely related to the Testing Effect comes the Generation Effect. In a typical generation experiment, participants are asked to either generate the to-be-learned items themselves—for example, by producing opposites when presented with a word (e.g., hot–???)—or to simply read the items (e.g., long–short). A later retention test is then administered, which usually consists of presenting the cues (hot–???, long–???) and asking participants to recall their corresponding targets (cold, short). The finding from this extensive meta-analysis, is that when people learn material by generating components of it themselves, the effect on long term retention is far greater than simply reading them. Once again, we are faced with the issue that most of the experiments done are not in the realms of mathematics, but the amount of rules that need to be learned in maths suggest that a strategy such as this which improves recall could prove very useful. Something as simple as providing students with a list of notes (on the rule of fractions, for example) and leaving out key words for them to fill in themselves is likely to be far more benefical then a set of notes without the gaps, as may be found in a revision guide.
My favourite quote:
The current findings support Izawa’s (1970) argument that tests potentiate the learning that occurs when an answer is presentedafter a test, even if the test is unsuccessful. The results also suggest that, in situations where tests and study opportunities are interleaved or testing is followed by feedback, the benefits of testing go beyond the benefits attributable to the learning that happens on successful tests. With respect to theoretical explanations of the testing effect, this finding is important because it demonstrates that the benefits of testing are not limited to the benefits of successful retrieval; rather, for a theory to fully explain the benefits of tests, it needs to explain the benefits of retrieval failure as well as the benefits of retrieval success. Successful tests obviously play a role, and perhaps a unique role—the findings do not imply that unsuccessful tests and successful tests are equally effective or that they are necessarily effective for the same reasons—but unsuccessful tests can also have a positive effect on long-term retention.

Memory: Revisionkeyboard_arrow_up
Back to Top

A game-changer for me when reading research into revision was just how ineffective some of the most common revision strategies that students common use really are. Making students aware of this is crucial. Alongside the research presented in this section, I would strongly advise reading the papers in Memory: Forgetting, Spacing and Interleaving, and Memory: Testing especially with regard to the importance of testing and spacing. They are two very simple techniques that are likely to significantly improve a student's ability to store and recall information. Two final things I tell my students as a result of this research. Firstly, I emphasise the difference between learning and performance is key, because many of the most effective revision techniques will not be as effective in the short-term on performance, but are likely to lead to long-term learning. Secondly, I tell them that revision should not be easy. If it feels easy (like re-reading and highlighting does), then they are not learning. 

Just before we get into the research, LearningSciences.org have produced a wonderful series of revision materials based on research. These "Six Strategies for Effective Learning" are available as posters or PowerPoints, and are ideal to share with your students to help their revision be as effective as possible. The six strategies are:
1. Spaced Practice: Poster, PowerPoint
2. Retrieval Practice: Poster, PowerPoint
3. Elaboration: Poster, PowerPoint
4. Interleaving: Poster, PowerPoint
5. Concrete Examples: Poster, PowerPoint
6. Dual Coding: Poster, PowerPoint


Research Paper Title: Improving Students’ Learning With Effective Learning Techniques and Strengthening the Student Toolbox
Author(s):
John Dunlosky, Katherine A. Rawson, Elizabeth J. Marsh, Mitchell J. Nathan, and Daniel T. Willingham
My Takeaway:
These two complimentary papers outline 10 learning techniques and discusses the relative merits of each one, in terms of their impact on learning and whether this is also true given different student characteristics. The two techniques that come out on top are Practice Testing - which can include something as simple as recalling information from flashcards, or working through questions from a textbook/worksheet, and Distributed Practice - spreading the learning/revision of a topic over different time periods instead of cramming. Interestingly, some of the techniques found to have little positive impact on learning include: highlighting, re-reading and use of imagery for text descriptions. This all fits in nicely with my mantra to students: the only way to get good at maths is to do maths. Note, that simply doing exam papers may not be as worthwhile as focused practice on a specific topic as it is often hard to isolate areas for improvement. The paper on Deliberate Practice in the Explicit Instruction section deals with this in more detail.
My favourite quote:
Concerning students’ commitment to ineffective strategies, recent surveys have indicated that students most often endorse the use of rereading and highlighting, two strategies that we found to have relatively low utility.

Research Paper Title:
What makes distributed practice effective?
Author(s): Aaron S. Benjamin and Jonathan Tullis
My Takeaway:
This paper seeks to explain the benefits of distributed practice during students' revision in terms of a theory of reminding. As well as confirming the benefits of distributed practice that we have seen in the paper above, and in all the work on spacing in the first Memory section, the authors of this paper make three additional points that I found particularly interesting. Firstly, they point out that distributed practice is a bit of a balancing act. We have seen in the work of Robert Bjork the importance of forgetting for learning. Students must have the opportunity to forget in order that when they practice the material again the storage strength of their memories are increased. Therefore, the timing of distributed practice is crucial - too much forgetting leads to unlikely reminding, and too little forgetting leads to impotent reminding. This will differ from student to student and topic to topic, and the best way for students to tell is probably through repeated self-testing to find the perfect balance. Secondly, because reminding slow does forgetting, the authors suggest that expanding interval retrieval schedules should prove superior for long-term retention than constant-interval schedules. So, students may revisit (specifically, retest themselves) on fractions after one week, then a further two weeks later, then a further three weeks, and so on. This requires careful planning on the student's part. Finally, the authors explain who reminding enhances the memory of the initial learning of the event, not the subsequent retrievals of it, so it is of utmost importance that the initial encoding is sound. This is expanded on in the quote below, and obviously has implications for us teachers in terms of the importance of how we initial present ideas and concepts to students.
My favourite quote:
A second implication of the reminding view presented here is that the memory enhancement occurs for the original event, not for the reminding event. Applied to skill learning, this view suggests that it is the quality of the original encoding that is particularly important for successful acquisition. If one initially learns a poor golf swing, for example, then the many later practice opportunities will serve to reinforce those bad habits, not correct them. How feedback plays a role in such corrections remains unexplored: it may be that corrective feedback can decrease reminding on an undesirable original memory, or that it serves to “tune” that memory by enhancing reminding of the original event and the corrective information.

Research Paper Title:
What will improve a student's memory?
Author(s): Daniel Willingham
My Takeaway:
This is a superb paper, which contains three really useful, practical pieces of advice to help students with their revision:
1) If you want to remember what things mean, you must select a mental task that will ensure that you think about their meaning, and if the task has little meaning, then use a mnemonic. We have seen from the Encoding section that students remember what they think about. This message needs spreading loud and clear to students. They must be engaged and thinking about the material. For example, if they can think about the meaning behind, say, the different forms of a quadratic expression and the turning and crossing points of its graph, then they are more likely to remember it. The problem is, quite a bit of maths appears to have no meaning, especially to students, and thus simply needs remembering. In these circumstances, the key is to make sure the memory itself has meaning to the student. So, if students are struggling with the meaning of a topic, then mnemonics can help. "Keep, flip, change" springs to mind for dividing fractions, and there are numerous slightly/highly inappropriate ways students have devised to remember SOH CAH TOA, which instill meaning on it for them. If it works for them - and they don't repeat them when my Headteacher is in the room - then I am happy.
2) Memories are lost mostly due to missing or ambiguous cues, so make your memories distinctive, distribute your studying over time, and plan for forgetting by continuing to study even after you know the material. Once again, mnemonics can help here. So long as they are distinctive, students should be able to associate them with the relevant memory and hence be more likely to successfully retrieve it. Encouraging students to come up with their own mnemonic that have meaning to them is a good idea. And we have of course seen may times in the Memory section the importance of spaced practice. This is possibly the single best piece of advice I give my students: test yourself, leave it a while, and then test yourself again.
3) Individuals’ assessments of their own knowledge are fallible, so don’t use an internal feeling to gauge whether you have studied enough. Test yourself, and do so using the same type of test you’ll take in class. Once again we see the benefits of testing, and also a call back to the fact that students (just like any of us) are not the best at judging how much they know a topic. This directly relates to the principle that if revision feels easy (strategies like rereading and highlight certainly give that comforting impression), then it probably isn't effective revision. Revision should require thought and feel difficult, and the best way to achieve that  -and indeed the only way to discover if you truly do know so something - is to test yourself.
Each of these principles is elaborated in greater detail, together with practical strategies, making it an ideal document to share with your students.
My favourite quote:
Many of my students also tell me that they reviewed their notes and were quite surprised when they did not do well on the test. I’ve found that these students typically know little about how their memories work and, as a result, do not know how to study effectively.

Research Paper Title: A Shield against Distraction from Environmental Noise
Author(s): Niklas Halin
My Takeaway:
This paper is fascinating - and potentially dangerous if not read carefully!  Participants were given four passages to read about fictitious cultures, two in an easy to read font (Times New Roman) and two in a difficult to read font (Haettenschweiler). While reading two of the passages, they heard someone describing another fictitious culture, which they were told to ignore. They then took multiple-choice tests over the passages to assess their learning. The results we as follows: when looking at the easy font, the passage with the background noise was remembered much more poorly than the passage read in silence. However, when looking at the hard font, the passage with background noise was remembered a little bit better than the passage read in silence! What is going on there? Well, it seems that when people really need to concentrate, they are good at blocking out environmental noise. However, when they consider something easy, then environmental noise can be distracting - although students may not notice this - and leads to poorer longer term recall. This is related to working memory - if you are really concentrating then there is little room in working memory for distracting noise to fit in, whereas when you are not concentrating as hard, there is plenty of spare working memory capacity for a distractor such as noise to work its way in there. My main takeaway - and indeed, what I told my students as soon as I read this paper - was that whilst they may do really hard study in silence, they they may be tempted to study easy material with their friends around, or music on, and so on. If they choose to do this, then they are probably going to remember less and/or it’s going to take a lot longer to process that information. There is also an interesting parallel with the recurring theme running throughout the Memory section of the importance of making learning difficult. Perhaps one slightly radical implication from this study is to make students revision material hard to read so that they concentrate more, which will aid memory.
My favourite quote:
The current thesis tested whether the shielding effect of higher focal-task engagement would also hold in a more applied context, with sounds that are more common in the built environment compared to tones (e.g., background speech, road-traffic noise, and aircraft noise), and with tasks that resemble those that people work with in offices and schools (i.e., proofreading and memory for text). The results of this thesis show that one way to attenuate distraction to environmental noise on office-related tasks is to promote focal-task engagement. A simple way to achieve higher focal task engagement is by merely changing the font of the text to one that is harder to read. This may not only be a practical solution to reduce the impairment of environmental noise on cognitive performance in work and school environments. It may also be an intervention that can reduce error rates in health care situations, but also be of aid to specific populations of individuals that generally have poorer attention control, like individuals with ADHD, children, or elderly people. Possibilities like these are something for future research to investigate further.

Research Paper Title: Expecting to teach enhances learning and organization of knowledge in free recall of text passages
Author(s): John F. Nestojko, Dung C. Bui, Nate Kornell & Elizabeth Ligon Bjork
My Takeaway:
This is a nice revision strategy to use that has been shown to be effective. Simply telling students that they will teach something to another student changes their mindset so much that even if they don’t actually teach the information, they remember it better later on when tested. The effect on their retention was even greater than if they were just told to learn the material for a test. This suggests that a pretty easy way of improving a student's revision programme is to regularly have them "teach" other students, as the preparation for this teaching could have significant benefits.
My favourite quote:
Expecting to teach appears to encourage effective learning strategies such as seeking out key points and organizing information into a coherent structure... Students seem to have a toolbox of effective study strategies that, unless prodded to do so, they do not use.

Research Paper Title: Why Students Think They Understand—When They Don't and How to Help Students See When Their Knowledge Is Superficial or Incomplete
Author(s): Daniel T. Willingham
My Takeaway:
Two fascinating papers by Daniel Willingham that seek to explain something I have noticed regularly over the last 12 years - often students think they know something better than they actually do. Willingham put this down to issues of being fooled by  Familiarity (specifically mistaking it for the ability to recollect) and Partial Access (quick retrieval of partial information leads to the assumption that you could retrieve all the information). Willingham goes on to identify three common ways these features can manifest themselves in school:
1) Rereading - students glance over their notes for Fractions and think "that seems familiar, I must know that)
2) Shallow Processing - related to the key principle from Cognitive Science that students remember what they think about, if they are not engaging with the core principle of a lesson (the fundamental laws of algebra), but instead focusing on some less pertinent aspect (the fact that letters are written in alphabetical order), then they could well leave the lesson believing they have understood everything.
3) Recollecting Related Information - students know a lot of information related to the target topic, and that makes them feel as though they know the target information. So students know a bit about straight line graphs, so assume they can work out the gradient, find the equation of perpendicular lines, etc.
Willingham then goes on to suggest strategies to help students avoid these traps. These include:
  • Make it clear to students that the standard of "knowing" is the "ability to explain to others," not "understanding when explained by others."
  • Require students to articulate what they know in writing or orally, thereby making what they know and don't know explicit, and therefore easier to evaluate, and easier to build on or revise.
  • Begin each day (or selected days) with a written self test.
  • Ask students to do self tests at home or in preparing for examinations.
Notice how these utilise the Retrieval Effect discussed in length in the first Memory section, and also tap into the recurring theme throughout this whole page - if learning feels easy, it probably isn't learning. Finally, an important implication of this paper is that decisions of a class' understanding of a topic that are based on students' own self-assessment (for example, asking students to indicate how confident in a topic they are, or asking them to place a sticker next to their "level" of understanding) are likely to be pretty dodgy indeed.
My favourite quote:
Cognitive science research confirms teachers' impressions that students do not always know what they think they know. It also shows where this false sense of knowledge comes from and helps us imagine the kinds of teaching and learning activities that could minimize this problem. In particular, teachers can help students test their own knowledge in ways that provide more accurate assessments of what they really know—which enables students to better judge when they have mastered material and when (and where) more work is required.


Formative Assessment and Questioning keyboard_arrow_up
Back to Top

Dylan Wiliam (interviewed on my podcast here) likes to think of formative assessment as "responsive teaching", and it is a fundamental part of every lesson I teach. When done well it allows me to get an accurate snap-shot of my class' understanding of a concept at a given time and adapt my teaching accordingly. Now, there are critics of formative assessment, such as David Didau, who argue that because of the distinction between Learning and Performance (see Memory section), formative assessment can only capture performance, which is an unreliable measure of learning. Whilst I see some validity in that claim, I would argue that the main purpose I use formative assessment is to identify and understand misconceptions, and for that there is no more effective tool. Needless to say, I am slightly biased when it comes to advocating the use of multiple choice questions for formative assessment, having created Diagnostic Questions, but John Mason's wonderful paper in this section provides an incredible guide to the different types of questions we teachers can ask. This section will be my attempt to survey the literature for the most effective formative assessment practices around.

Research Paper Title: Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments
Author(s): Justin Kruger and David Dunning
My Takeaway:
Before we get into the concept of formative assessment, we first need to establish if there is a need for it. Why can't we simply say to students "do you understand this topic?", or "what topics do you need help on?" and base our teaching decisions on that? Well, to put it simply, because students (like most novices) are not particularly good judges of their own abilities. This lovely paper describes this phenomenon as the Dunning Kruger Effect. In a series of tests of humour, grammar and logic, the researchers found that people tend to hold overly favorable views of their abilities in many social and intellectual domains. The authors suggest that this overestimation occurs, in part, because people who are unskilled in these domains suffer a dual burden: not only do these people reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the meta-cognitive ability to realise it. What are the implications for teaching? Well, if we assume that most students are novices in terms of their understanding of mathematics, then it is likely that they overestimate their abilities in mathematics. Hence, relying on their own judgement of their abilities is likely to be flawed, and thus we need another, more reliable measure. In the long term, this could be the results from a range of sumamtive assessments, but in the here and now of the busy classroom, our best effort is likely to come from a well designed question. This also calls into question the practices of traffic lights, thumbs-up, sticking postit notes on level descriptors, and other types of common self assessment strategies I have seen (and used!). How can we help students become better judges of their own abilities? Well, paradoxically the authors found that improving the skills of participants, and thus increasing their meta-cognitive competence, helped them recognise the limitations of their abilities. Once again, helping our students acquire more knowledge seems to be the key.
My favourite quote:
In sum, we present this article as an exploration into why people tend to hold overly optimistic and miscalibrated views about themselves. We propose that those with limited knowledge in a domain suffer a dual burden: Not only do they reach mistaken conclusions and make regrettable errors, but their incompetence robs them of the ability to realize it. Although we feel we have done a competent job in making a strong case for this analysis, studying it empirically, and drawing out relevant implications, our thesis leaves us with one haunting worry that we cannot vanquish. That worry is that this article may contain faulty logic, methodological errors, or poor communication. Let us assure our readers that to the extent this article is imperfect, it is not a sin we have committed knowingly.

Research Paper Title:
Inside the Black Box. Raising Standards Through Classroom Assessment
Author(s):
Paul Black and Dylan Wiliam
My Takeaway:
This is the bible of formative assessment, full of practical steps that are easy to implement and can make a huge difference. The key finding is that formative assessment raises standards of learning, but there is still plenty of room for improvement. Practical strategies that classroom teachers can use to improve students' learning include: feedback to any pupil should be about the particular qualities of his or her work, with advice on what he or she can do to improve, and should avoid comparisons with other pupils; for formative assessment to be productive, pupils should be trained in self-assessment so that they can understand the main purposes of their learning and thereby grasp what they need to do to achieve; and opportunities for pupils to express their understanding should be designed into any piece of teaching, for this will initiate the interaction whereby formative assessment aids learning. Two points in particular stood out for me:
1) Students can (and should) be trained in self-assessment. But this is difficult to do. We have seen from the paper above that novices find it difficult to judge their own abilities. We can try to help by providing things like "pupil friendly level descriptors", but these can be flawed, as Daisy Cristodoulou so eloquently explained when I interviewed her. Perhaps the best thing we can do to help students develop good self-assessment skills is to provide them with the necessary knowledge to understand the given domain, and provide the kind of constructive, useful feedback that we will discuss in the Feedback section.
2) Formative Assessment is (or at least should be) responsive teaching. It provides an amazing opportunity to find out what your students do, or do not understand, and react accordingly. As a result of reading this paper, together with the papers that follow, I ask three diagnostic questions at the start of every lesson, and adapt the lesson depending on the students' responses.
My favourite quote:
The main plank of our argument is that standards are raised only by changes which are put into direct effect by teachers and pupils in classrooms. There is a body of firm evidence that formative assessment is an essential feature of classroom work and that development of it can raise standards. We know of no other way of raising standards for which such a strong prima facie case can be made on the basis of evidence of such large learning gains.

Research Paper Title: Five Key Strategies for Effective Formative Assessment
Author(s): Dylan Wiliam for the National Council of Teachers of Mathematics
My Takeaway:
A wonderful, practical guide that discusses five key classroom practices that make the processes of formative assessment as effective as possible. Each is backed up by research evidence and illustrated with examples, and I would strongly advise reading the full paper as I will not do it justice here. The five strategies are:
1) Clarifying, sharing and understanding goals for learning and criteria for success with learners. A key strategy I took form this was sharing examples of other students' work is a great way to clearly convey the kind of work you are expecting in a way students can relate to more than abstract, confusing learning objectives.
2) Engineering effective classroom discussion, questions, activities, and tasks that elicit evidence of students' learning. This is all about planning effective questions. These might not necessarily be the kind of open-ended, probing questions that teachers are often advised to ask, but instead carefully planned, multiple-choice diagnostic questions. These have the advantage of being quick to ask, quick to collect information, and incorrect answers reveal the specific misconceptions students have. Needless to say, I love a Diagnostic Question
3) Providing feedback that moves learning forward. This is directly relevant to the Marking and Feedback section. The key point is that feedback should be more work for the student than the teacher, and the student should have time to read, respond to and act upon that feedback.
4) Activating students as owners of their own learning. This is fascinating. Dylan suggests that traffic-lights are a good way of encouraging this, which is a strategy that I have routinely dismissed as I have found students tend to just show green so I will leave them alone! However, Dylan adds the advice to ask students who demonstrate green cards to explain the concept to those with red cards. This gives students a stronger incentive to be honest.
5) Activating students as learning resources for each other. Two interesting findings here. Firstly, feedback students give to each other is likely to be more direct and hard-hitting than a teacher would give. Even more interestingly, the person providing the feedback benefits just as much as the recipient because she or he is forced to internalise the learning intentions and success criteria in the context of someone else's work, which is less emotionally charged than doing it in the context of one's own work. I would add an important caveat to this - that only works if the feedback is correct. Providing students with answers is an obvious way around this, but we must ensure students arrive at these correct answers in the right way.
My favourite quote:
The available research evidence suggests that considerable enhancements in student achievement are possible when  teachers use assessment, minute-by-minute and day-by-day, to adjust their instruction to meet their students' learning needs. However, it is also clear that making such changes is much more than just adding a few routines to one's normal practice. It involves a change of focus from what the teacher  is putting into the process and to what the learner is getting out of it, and the radical nature of the changes means that the support of colleagues is essential. Nevertheless, our experiences to date suggest that the investment of effort in these changes is amply rewarded. Students are more engaged in class, achieve higher standards, and teachers find their work more professionally fulfilling. As one teacher said, "I'm not babysitting any more."

Research Paper Title: Mathematics Inside the Black Box: Assessment for Learning in the Mathematics Classroom
Author(s):
Jeremy Hodgen and Dylan Wiliam
My Takeaway:
This is a wonderful, mathematically specific paper that follows on nicely from the generic research that kick-started this section. Indeed, this paper is so good it makes a second appearance in the Marking and Feedback section. The authors discuss five principles of learning, which sound so obvious, but whose effects can be powerful. I will not do the principles justice here, so please read it in full:
1) Start from where the learner is - this is obvious, but difficult. The majority of maths teaching (in my experience, anyway) involves teaching something that the students have already encountered before, or which relies on pre-existing knowledge. Attempting to discover where students are at before you attempt to teach them new knowledge - and in particular which misconceptions they hold - is one of the most important parts of teaching
2) Students must be active in the learning process - we have seen throughout these papers that students remember what they think about, and sometimes learning needs to be difficult. If students are passive bystanders in lesson - and by that I mean if they are not thinking about the content of the lesson - they are less likely to learn
3) Students need to talk about their ideas - this is a tricky one, as students often find it difficult to articulate their thinking. The authors provide some really useful prompts and frameworks that can be used straight away, such as Can you put Amy’s idea into your own words?, What can we add to Saheera’s answer? Well, if you’re confused you need to ask Jack a question.
Which parts of Suzie’s answer would you agree with? Can someone improve on Simon’s answer?
4) Students must understand the learning intention. This requires an understanding of two key things: what would
count as a good quality work (success criteria), and where they stand in relation to that target. Knowing these things allows students to steer their own learning in the right direction, and is known as meta-cognition. Effective peer and self assessment are key to this, and practical tips for making both effective are provided.
5) Feedback should tell pupils how to improve - this will be covered in depth in the Marking and Feedback section
Practical ideas on how to achieve all of these are provided throughout this wonderful paper.
My favourite quote:
The changes in practice recommended here are not easily made. They require changes in the ways teachers work with students, which may seem risky, and which will certainly be challenging. The work we have done with teachers suggests that the teachers who are most successful are those who change their practice slowly, by focusing on only one or two aspects at a time. As they become skilled with these new ideas, and incorporate them into their natural practice, they can then turn their attention to new ideas. Teachers who try to change many things about their practice at the same time are unlikely to be successful.

Research Paper Title: Effective Questioning and Responding in the Mathematics Classroom
Author(s): John Mason
My Takeaway:
This is, quite simply, the best thing I have ever read on questioning in mathematics. There is so much good advice in this paper by John Mason that the best thing I could probably do is copy and paste the entire paper and have that as my Takeaway. However, I will resist and instead focus on a couple of the main things that have changed my teaching. The first is Mason's point that the distinction between open and closed questions is rather a pointless one. My work on Diagnostic Questions has shown me the value in a well-designed closed question in term of quickly finding out information about a class' understanding of a concept, as well as provoking fruitful discussion about misconceptions. As Mason so beautifully explains: "Questions are just words with a question mark: the notion of openness and closedness is more to do with how the question is interpreted than with the question itself". Then there is the section on using question effectively. A really important point that I had not considered was to avoid using questions for controlling purposes, and instead make use of direction instruction or statements. This ensures that questioning in the classroom is used only to promote mathematical thinking, and not behaviour management. I also find that asking students questions such as "why exactly do you feel it is appropriate to do that?" always leaves open the possibility of an unwelcome, and public, discussion taking place, whereas a simple "stop that" might allow the focus of the lesson to return to mathematics. Next comes "funneling" - something I am regularly guilty of - whereby you give your students so many clues in the desperate hope they will begin to figure out what is in your mind. Mason's advice in that instance is to simply tell them, and then enquire as to why they had not thought of it themselves. This is likely to be much quicker and more fruitful than kidding yourself that students have actually understood something because you essentially directed them straight to the answer. Finally, Mason emphasises the importance of being genuinely interested in not only in what learners are thinking, but in how they are thinking, in what connections they are making and not making. Techniques such as giving students more time to think, and asking for methods instead of answers can make a huge difference to the atmosphere in the classroom. I could go on, but I better stop there. This is a truly wonderful paper.
My favourite quote:
Although a very common activity, question asking is at best problematic and at worst an intrusion into other people’s thinking. By catching yourself expecting a particular response you can avoid being caught in a funnelling sequence of ‘guess what is in my mind’. By being explicit at first, then increasingly indirect in your prompts, you can assist learners to internalise useful questions which they can use for themselves to help them engage in effective and productive mathematical thinking. Above all, the types of questions you ask will quickly inform your learners of what you expect of them, and covertly, of your enacted philosophy of teaching. The key to effective questioning lies in rarely using norming and controlling questions, in using focusing questions sparingly and reflectively, and using genuine enquiry-questions as much as possible. This means being genuinely interested in the answers you receive as insight into learners’ thinking, and it means choosing the form and format of questions in order to assist learners to internalise them for their own use (using meta-questions reflectively). The kinds of questions you ask learners indicates the scope and breadth of your concern for and interest in them, as well as the scope, aims, and purposes of mathematics and the types of questions that mathematics addresses.

Research Paper Title:
Is there value in just one?
Author(s): Caroline Wylie and Dylan Wiliam
My Takeaway:
This paper deals with answering a key question a teacher regularly needs to ask themselves: "when are my students ready for me to move on?", and was very influential when I was developing Diagnostic Questions. It makes the point that waiting for test data to confirm this is often unsuitable due to the time lag and the abundance of information, and relying on student judgement of their own readiness is flawed. With teachers needing to make this decision "on the fly" mid-lesson, the authors suggest that one well-written question, can do the job. The question should have three characteristics:
1) Designed for easy collection of information (I use one finger for A, two for B, etc);
2) Incorrect responses assist the teacher diagnose what students do not understand, and, ideally, provide ideas about what to do about this (this is all comes down to the choice of good distractors);
3) Correct responses support a reasonable inference that students understand the concept being assessed (students should not be able to get the question correct whilst still holding key misconceptions).
Whilst the information gleaned from asking this one question is not going to be perfect, there is no quicker, more accurate way of a teacher doing it mid-lesson. I ask three of these questions each lesson. If the responses of my students reveal misconceptions, then I respond accordingly - even if this takes up more of the lesson than I had intended. For my take on the use of such questions, see my Pedagogy videos.
My favourite quote:
The issue of “readiness to move on” is a familiar one to teachers, and the decision not to move on will initiate routines that are also familiar: to engage in whole class remediation, to pull aside a small group or individual students for additional assistance, to construct alternative learning opportunities that will assist students in their learning, and so forth, so that the teacher can later reassess the situation and proceed in the learning sequence.

Research Paper Title: Using Diagnostic Classroom Assessment: One Question at a Time
Author(s): Joseph F. Ciofalo and Caroline Wylie
My Takeaway:
This paper also looks at the use of diagnostic questioning in the classroom. There is a nice focus on the characteristics of good questions with some nice examples provided that illustrate these characteristics clearly. My favourite part is the description of how these questions might be used in the classroom, providing a suggestion with how teachers might deal with different response scenarios from students. I have recorded a series of videos about how I use these questions in the classroom each day here.
My favourite quote:
The goal is to help teachers better utilize questioning and discussion to improve student learning, a key strategy within formative assessment. By starting with the instructional decisions that teachers make, the intention is to create both a resource and a habit of mind that is of obvious value to teachers. Lesson planning is a natural part of every teacher’s daily activities. Embedding a focus on questioning strategies—through carefully constructed diagnostic items—supports teachers in improving instruction by providing them with a natural extension to their practice, rather than requiring a major shift in mind-set.

Research Paper Title: The memorial consequences of multiple-choice testing
Author(s): Elizabeth J Marsh, Henry L Roediger III, Robert A Bjork, Elizabeth L Bjork
My Takeaway:
We have seen in the first section on Memory the existence of the Testing Effect (or Retrieval Effect), whereby testing can actually enhance learning. This paper attempts to show that whilst multiple choice questions are often used to measure understanding (via formative assessment), they can also have a similarly positive impact on learning as other forms of assessment via the Testing Effect. Interestingly, these benefits are not limited to simple definition or fact-recall multiple choice questions, but extend to those that promote higher-order thinking. The authors also address something a common criticism of multiple choice questions - the fear that the choice of wrong answers (the distractors) can cause students to learn false facts. The authors find that such persistence appears due to faulty reasoning rather than to an increase in the familiarity of the distractors. There is no doubt, however, that good distractors have the potential to cause students to learn faulty information. The authors put it like this: "multiple-choice lures may become integrated into subjects’ more general knowledge and lead to erroneous reasoning about concepts." However before we make all multiple choice questions illegal, three things need saying.
1) The benefits of using multiple choice questions in terms of eliciting information about students' understanding in an efficient way, and positive effects on their learning via the Testing Effect far outweigh the costs.
2) The issue of learning false facts can be overcome via immediate feedback. This could be a discussion in a lesson when using a multiple choice question for formative assessment purposes, automated marking when using a system such as Diagnostic Questions, or through task-specific written feedback.
3) I am firmly of the beleif that you cannot fully understand a topic or concept unless you are also aware of the misconceptions that accompany it. These need to be confronted head-on (as discussed in the Joan Lucariello and David Naff paper in the Explicit Instruction section). I guess it boils down to when you introduce these misconceptions, and here I think there is an argument for waiting until students have grasped and had some success with the basics. They need to know what is right before they confront what is wrong.
My favourite quote:
More generally, the prevailing societal emphasis on testing as assessment is unfortunate, because it obscures the critical pedagogical aspects of testing. Tests, optimally constructed, can enhance later performance, provide feedback to the learner on what has and has not been learned, and potentiate the efficiency of subsequent study opportunities.

Research Paper Title: Feedback enhances the positive effects and reduces the negative effects of multiple-choice testing
Author(s): Andrew Bulter and Henry L Roediger III
My Takeaway:
This study develops the second point made in the paper above - namely that feedback is crucial both to boost the positive impact of multiple choice questions (increased retention via the Testing Effect) and reduce negative effects (acquiring misinformation via the lures or distractors). Subjects studied passages and then received a multiple choice test with immediate feedback, delayed feedback, or no feedback. In comparison with the no-feedback condition, both immediate and delayed feedback increased the proportion of correct responses and reduced the proportion of intrusions (i.e., lure responses from the initial multiple-choice test) on a delayed cued recall test. The advice for teachers is very simple - if using multiple choice tests, ensure students are given feedback. It is interesting to note that both immediate feedback and delayed feedback had the same beneficial effect. The authors point out that their version of delayed feedback was not synonymous with what may happen in a classroom, with students taking a test and potentially not receiving feedback for a week or two - delayed feedback in this study simply involved not being told the answer immediately after the question. However, this opens up an interesting possibility. We will see in the Marking and Feedback section that Bjork recommends delaying feedback as one of his desirable difficulties. This way feedback stops being a crutch for students, and also taps into Dylan Wiliam's key point that feedback must make students think. Perhaps the important thing here is not necessarily the time delay between the answer and the feedback, but what happens in between. If students answer a question incorrectly, do not know they are incorrect, and then subsequently answer another 100 questions in that matter, then that misconception is likely to be difficult to remove - after all, practice makes permanent. However, if they answer that question and no more like it before receiving feedback, any negative effects may be reduced to the same extent as they would be after immediate feedback, with the added bonus that students need to think more. This is pure speculation, but it makes sense combining the findings of several papers.
Educators should provide feedback when using multiple-choice tests
My favourite quote:
A pragmatic solution to the possible negative effects of multiple-choice tests is to ensure that students always receive feedback after testing. Feedback enhances the positive effects of taking a test and helps students correct their errors, thereby reducing the acquisition of misinformation. The latter outcome is especially important when the same questions and alternatives from a first test are reused on a later test, because the production of misinformation often increases the chance that it will be produced again on a later test

Research Paper Title:
Multiple-Choice Tests Exonerated, at Least of Some Charges: Fostering Test-Induced Learning and Avoiding Test-Induced Forgetting
Author(s): Jeri L. Little, Elizabeth Ligon Bjork, Robert A. Bjork, and Genna Angello
My Takeaway:
A common criticism of multiple choice questions is that because students know the correct answer is one of the listed options, they can get it correct by recognition as opposed to the retrieval process from long term memory that we know can aid learning. This paper addresses that criticism by testing whether multiple-choice tests could trigger productive retrieval processes—provided the alternatives (distractors) were made plausible enough to enable test takers to retrieve both why the correct alternatives were correct and why the incorrect alternatives were incorrect. In two experiments, they found not only that properly constructed multiple-choice tests can indeed trigger productive retrieval processes, but also that they had one potentially important advantage over cued-recall tests. Both testing formats fostered retention of previously tested information, but multiple-choice tests also facilitated recall of information pertaining to incorrect alternatives, whereas cued-recall tests did not. In other words, so long as the multiple choice question is a good one (see papers earlier in this section for the rules of a good multiple choice question), then when searching for the answer students thoughts turn to reasons why the distractors are incorrect, and hence they exercise the very retrieval processes they have been accused of bypassing. This is huge, because it suggests that whilst multiple choice questions have all the advantages of speed of information gathering discussed earlier, they have an extra advantage of encouraging the students to think harder than if the question was presented without any options to choose from.
My favourite quote:
The present findings vindicate multiple-choice tests, at least of charges regarding their use as practice tests. In fact, our findings suggest that when multiple-choice tests are used as practice tests, they can provide a win-win situation: Specifically, they can foster test-induced learning not only of previously tested information, but also of information pertaining to the initially incorrect alternatives. This latter advantage is especially important because, typically, few if any practice-test items are repeated verbatim on the subsequent real test. From that standpoint, the advantage of initial multiple-choice testing over initial cued-recall testing is a truly significant one.


Marking and Feedbackkeyboard_arrow_up
Back to Top

Whether we like it or not, most of us have to mark and give feedback to students. For many (including me), it is one of the worst parts of being a teacher. The best advice I ever heard on feedback was from Dylan Wiliam, who said "the only good feedback is that which is acted on". You could write the best feedback in the world, but if the your students neither have the time nor the inclination to do it, or if they simply do not understand what you are asking them to do, then it is an absolute waste of time. A second guiding principle that I now try to stick to is "feedback should make students think". All too often my feedback has been so carefully constructed, and full of so many hints, that my students simply have to say "yeah, yeah, I get it now, sir", make a few changes, and then move on, without having learned a single thing, and having cost me a good few hours of my life. For years I have been doing far more work than my students when it comes to marking and feedback. Here we look at what the evidence has to say about marking and feedback, both in terms of the type and the immediacy, when feedback can have a negative effect, and when less is definitely more!

Please note: papers concerning marking and feedback, but that are more explicitly to do with praise can be found in the Motivation and Praise section.


Research Paper Title:
A marked improvement. A review of the evidence on written marking
Author(s): Education Endowment Foundation and Oxford University
My Takeaway:
Lots of practical, easy to implement strategies to improve the effectiveness of marking, each one evaluated against available research. My favourites include:
1) Dealing with careless mistakes and misconceptions differently when marking;
2) Making feedback as specific and actionable as possible so the students actually understand it; "acknowledgement" marking (i.e. a tick to say you have seen the work) is a waste of time, and hence a good rule is mark less but mark better; specific time must be dedicated to students to respond to marking.
3) And then the big one for me: awarding grades/levels for each piece of work means students focus upon these marks at the expense on the formative comments the teacher has written, and probably took ages!
My favourite quote:
It also appears worthwhile to caution against elements of dialogic or triple impact marking that do not follow the wider principles of effective marking that are underpinned by relatively stronger evidence summarised elsewhere in this review. For example, there is no strong evidence that ‘acknowledgment’ steps in either dialogic or triple impact marking will promote learning.

Research Paper Title: A marked improvement. A review of the evidence on written marking
Author(s): Education Endowment Foundation and Oxford University
My Takeaway:
Lots of practical, easy to implement strategies to improve the effectiveness of marking, each one evaluated against available research. My favourites include:
1) Dealing with careless mistakes and misconceptions differently when marking;
2) Making feedback as specific and actionable as possible so the students actually understand it; "acknowledgement" marking (i.e. a tick to say you have seen the work) is a waste of time, and hence a good rule is mark less but mark better; specific time must be dedicated to students to respond to marking.
3) And then the big one for me: awarding grades/levels for each piece of work means students focus upon these marks at the expense on the formative comments the teacher has written, and probably took ages!
My favourite quote:
It also appears worthwhile to caution against elements of dialogic or triple impact marking that do not follow the wider principles of effective marking that are underpinned by relatively stronger evidence summarised elsewhere in this review. For example, there is no strong evidence that ‘acknowledgment’ steps in either dialogic or triple impact marking will promote learning.

Research Paper Title:
The Effects of Feedback Interventions on Performance: A Historical Review, a Meta-Analysis, and a Preliminary Feedback Intervention Theory
Author(s): Avraham N. Kluger and Angelo DeNisi
My Takeaway:
When I interviewed Dylan Wiliam, he cited this paper as the most surprising piece of research he had ever encountered. The authors found over 3,000 research studies published between 1905 and 1995, but found that only 131 of the studies were well-enough designed for their results to be taken seriously. The 131 studies reported 607 effect sizes, which showed that, on average, feedback did increase achievement. But—in what could possibly be one of the most counter-intuitive results in all of psychology—231 of the 607 reported effect sizes were negative. In almost two out of every five studies, feedback lowered performance. What are we to take from this? Well, the authors suggest that the further feedback moves away from the task itself and towards the individual student, the less effective it is - even going as far as to have a negative effect. If teachers are going to dedicate a significant proportion of their time to giving feedback, we must make it task-focused. I will leave the final words to Dylan himself: If there’s a single principle teachers need to digest about classroom feedback, it’s this: The only thing that matters is what students do with it. No matter how well the feedback is designed, if students do not use the feedback to move their own learning forward, it’s a waste of time. We can debate about whether feedback should be descriptive or evaluative, but it is absolutely essential that feedback is productive.
My favourite quote:
The central assumption of Feedback Intervention Theory is that Feedback Interventions change the locus of attention among 3 general and hierarchically organized levels of control: task learning, task motivation, and meta-tasks (including self-related) processes. The results suggest that feedback intervention effectiveness decreases as attention moves up the hierarchy closer to the self and away from the task.

Research Paper Title: The Power of Feedback
Author(s): John Hattie and Helen Timperley
My Takeaway:
The first line of this paper hooked me in: "Feedback is one of the most powerful influences on learning and achievement, but this impact can be either positive or negative." This was a revelation to me. Surely, if I was spending so much time doing something, it has to be beneficial for my students? This is a fascinating, comprehensive paper that addresses all the major issues surrounding feedback. I would strongly recommend reading the part on page 98 about the issues surrounding positive and negative feedback. This really made me realise that some students benefit from praise, for some it does them more harm than good, and some students need a metaphorical kick up the arse. Judging this is difficult, but ultimately comes down to a teacher's relationship with and knowledge of their students. My key takeaway, however, is the fact that feedback is likely to be ineffective if the student simply does not have the relevant knowledge to solve the problem in the first place. I know that sounds obvious, but it wasn't to me. Feedback can only build upon knowledge. You can give all the task-focused prompts in the world, but if the students lack the knowledge to do the work, then feedback is likely to be pretty useless. I have done this myself, literally spending hours making corrections, carefully writing prompts, inventing follow-up questions for students to do, and then one of two things happens - either the students still cannot do it, or I have ended up doing that vast majority of the work, so they can make corrections without thinking. In that instance, I believe it is far better to simply re-teach the students. I write a quick note along the lines of "we will cover this in class", and then I do a a series of worked examples following the principles of Explicit Instruction, and then ask the students to correct their work. This approach has the onus of putting the emphasis back onto the students, saves me hours, and also allows my students to benefit from the positive effects of spacing covered in the Memory section. Of course, determining whether a student could not do the work, or could not be bothered to do the work, can be difficult, and once again comes down to knowledge and relationships.
My favourite quote:
Feedback, however, is not “the answer”; rather, it is but one powerful answer. With inefficient learners, it is better for a teacher to provide elaborations through instruction than to provide feedback on poorly understood concepts. If feedback is directed at the right level, it can assist students to comprehend, engage, or develop effective strategies to process the information intended to be learned. To be effective, feedback needs to be clear, purposeful, meaningful, and compatible with students’ prior knowledge and to provide logical connections. It also needs to prompt active information processing on the part of learners, have low task complexity, relate to specific and clear goals, and provide little threat to the person at the self level. The major discriminator is whether it is clearly directed to the task, processes, and/or regulation and not to the self level. These conditions highlight the importance of classroom climates that foster peer and self-assessment and allow for learning from mistakes.

Research Paper Title: The Secret of Effective Feedback
Author(s): Dylan Wiliam
My Takeaway:
I make no secret of the fact that Dylan Wiliam has - perhaps more than any other individual - had the greatest influence on my teaching. We have already seen his work in the Assessment for Learning section, and now it is time to read his advice when it comes to making feedback effective. This article from 2016 is an excellent summary of Wiliam's views on feedback that he has developed over many years of studying the topic. The opening sentence summarises the main thrust of his argument: feedback is only successful if students use it to improve their performance. Wiliam goes on to outline a set of general principles for making feedback as effective as possible:
1. Keeping Purpose in Mind. Wiliam explains that most of the time the student work we're looking at is not important in and of itself, but rather for what it can tell us about students—what they can do now, what they might be able to do in the future, or what they need to do next. Looking at student work is essentially an assessment process. We give our students tasks, and from their responses we draw conclusions about the students and their learning needs. When we realize that most of the time the focus of feedback should be on changing the student rather than changing the work, we can give much more purposeful feedback. If our feedback doesn't change the student in some way, it has probably been a waste of time.
2. Giving Feedback They Can Use. We need to start from where the learner is, not where we would like the learner to be. We need to use the information we obtain from looking at the student's work—even through that information may be less than perfect—and give feedback that will move the student's learning forward.
3. Assign Tasks That Illuminate Students' Thinking. Looking at students' work often tells us only that they didn't do it very well and they need to do it again, but better. Designing tasks that, in Ritchhart and Perkins's (2008) phrase, "make thinking visible" takes time, but front-loading the work in this way makes it much more likely that we'll provide useful feedback.
4. Make Feedback into Detective Work. Consider a maths teacher who provides feedback on 20 solved equations. Rather than telling the student which equations are incorrect, the teacher can instead say, "Five of these are incorrect. Find them and fix them." Such practices ensure that students, the recipients of feedback, do as much work as the teacher who provides the feedback. Making feedback into detective work encourages students to look at the feedback more closely and to think about their original work more analytically.
5. Build Students' Capacity for Self-Assessment. The amount of feedback we can give our students is limited. In the longer term, the most productive strategy is to develop our students' ability to give themselves feedback. Most teachers seem to believe that students make most of their progress when the teacher is present, with homework as a kind of optional add-on. It's important, therefore, to develop students' capacity for self-assessment. Wiliam suggests an approach called "plus, minus, interesting,". At the end of a task, ask students to identify something they found easy about the task, something they found challenging or difficult, and something they found interesting. Such reflection develops language skills and helps the students become clear about what areas they need to work on.
6. A Trusting Relationship for Feedback. In the end, it all comes down to the relationship between the teacher and the student. To give effective feedback, the teacher needs to know the student—to understand what feedback the student needs right now. And to receive feedback in a meaningful way, the student needs to trust the teacher—to believe that the teacher knows what he or she is talking about and has the student's best interests at heart. Without this trust, the student is unlikely to invest the time and effort needed to absorb and use the feedback.
My favourite quote:
The only thing that matters is what the student does with the feedback. If the feedback you're giving your students is producing more of what you want, it's probably good feedback. But if your feedback is getting you less of what you want, it probably needs to change. Finally, talk to your students. Ask them, "How are you using the feedback I'm giving to help you learn better?" If they can give you a good answer to that question, then your feedback is probably effective. And if they can't, ask them what they would find useful. After all, they're the clients.

Research Paper Title:
Learning versus Performance
Author(s): Nicholas C. Soderstrom and Robert A. Bjork 
My Takeaway:
This is the second appearance of this lovely paper following its debut in the Memory section, and once again its effect on me has been huge. This paper seeks to question the common assumption that immediate feedback is the best kind of feedback - and leads us on to the final of Bjork's "desirable difficulties". The logic makes perfect sense - if a learner gets something wrong, surely they need to know that they have got it wrong immediately, and why they have got it wrong, in order to correct their mistake? And indeed, during the early phases of knowledge acquisition, immediate feedback may be beneficial to ensure mistakes do not become embedded. However - and wait for this because it is a classic - research presented in this paper suggests that delaying or even reducing feedback can have a long term benefit to student's learning. Why? Well, because regular, immediate feedback can cause learners to become overly dependent upon it (almost seeing it as a crutch to their learning). To relay this to themes covered in other sections of this page, immediate feedback prevents students from thinking hard, and also having the opportunity to forget. I have seen this myself - you give students their work back with detailed feedback and corrections to make, and they look at it and go "yeah, yeah", and make the necessary corrections. But are they actually thinking? Ironically (or perhaps, unfortunately) the better the feedback, the less they need to think. Once again we have the issue of distinguishing between learning and performance. The authors suggest that delaying or reducing feedback is likely to have a determinantal effect on short-term performance, but a positive effect on long-term learning.  What are the implications for us teachers, especially given that the Hattie paper above stresses the potential positive benefits of feedback? Well, one idea I had reading this paper was when initially marking a piece of homework to give no feedback whatsoever, simply to provide ticks and crosses. Then give this work back to the students and see if they could identify the source of their errors and correct them. When I next take the books in, this is when I would give feedback. In my view, there are two advantages of such an approach. Firstly, it will force students to think about why they got a question wrong, instead of just blindly following my step-by-step prompts, and hence become less reliant on me. Secondly, it may enable me to better identify actual gaps in their knowledge. As discussed in the paper above, feedback when students do not have the knowledge to begin with is likely to be a waste of time. However, if indicating where students have gone wrong allows students to have a better stab at those questions, then I can then better distinguish between gaps in their knowledge versus the kind of mistakes that well constructed, task-focused feedback is likely to help resolve. The knowledge gaps may then need addressing again via instruction.
My favourite quote:
Delaying feedback until after the task was completed yielded greater long-term learning than concurrent feedback. Performance gains during acquisition, however, were made more rapidly by those receiving concurrent feedback.
 
Research Paper Title: Formative Assessment: Practical Ideas for Improving the Efficiency and Effectiveness of Feedback to Students
Author(s): Geraldine O’Neill
My Takeaway:
A really good summary of effective feedback practice, together with the value of this idea to either the teacher or the students, all referenced to relevant research-based evidence. There are many interesting practices cited here, but the ones that stood out to me were:
1) Consider feedback in different media/formats, such as on-line, audio-feedback, verbal class feedback, use of ‘clickers’ in large class contexts. This may not only save the teacher time, but may make feedback more permanent and easily accessible to students.
2) Student Requested Feedback -  ask students to submit specific requests for areas for feedback. As this is student-focused feedback it is more inclined to motivate students to act. This also encourages students to take some responsibility in the process.
3) In class peer and self assess feedback activities in terms of discussing and giving feedback to annonymised work. This will help students engage in the feedback process and make them more aware of the desired standard in relation to their own work.
4) Comment in actionable language. This relates to the recurring theme throughout this section that feedback should be task-focussed, but also reminds us that if the students cannot understand what the feedback is on about, it is likely to be a complete waste of time.
My favourite quote:
One of the key themes emerging to address this dilemma is to develop students own self-monitoring skills in order to help them narrow the gap between their performance and the standards expected of them. The timing, type and specification of feedback can also improve student ability to self-monitor. In addition, good feedback should feed into some specific actions that can be used in the next assessment. Feedback need not always be from the academic staff, students themselves are a good resource to each other when given guidance on how to do this. New technologies also open up some efficient feedback opportunities.

Research Paper Title: Mathematics Inside the Black Box: Assessment for Learning in the Mathematics Classroom
Author(s): Jeremy Hodgen and Dylan Wiliam
My Takeaway:
This is a wonderful paper that first made an appearance in the Formative Assessment section, but it is with regard to its advice on marking and feedback that I want to focus on here. Often I find it difficult to write good comments in students' books. Comments such as "show your working" and "correct your mistakes" have tended to have little impact in my experience. The main piece of advice I took from this paper was that the comments I write in students' books such match the kind of questions I would ask the students in class. I have long been an advocate of planning questions as opposed to planning resources, and so thinking of feedback in these terms really made sense to me. It also fits in very well with the general principles that feedback should make students think. The authors kindly provide a suggestion of the types of comments/questions you could write. My favourites include:
1) Enabling pupils to identify the errors for themselves:
- There are five answers here that are incorrect. Find them and fix them.
- The answer to this question is [. . .] Can you find a way to work it out?
2) Identifying where pupils use and extend their existing knowledge:
- You’ve used substitution to solve all these simultaneous equations. Can you use elimination?
- You seem to be having difficulty adding some of these fractions and not others. In question 2 you used equivalent fractions; could you use this on question 4?
3) Encouraging pupils to reflect:
- You used two different methods to solve these problems. What are the advantages and disadvantages of each?
- You have understood [. . .] well. Can you make up your own more difficult problems?
4) Suggesting pupils discuss their ideas with other pupils:
- You seem to be confusing sine and cosine. Talk to Katie about how to work out the difference.
- Compare your work with Ali and write some advice to another student talking this topic for the first time.
5) Helping pupils to show their working:
- The way in which you are presenting graphs is much clearer. Look back at your last work on graphs in February. What advice would you give on how to draw graphs?
- Your solutions are all correct, but they are a bit brief. Look at the examination marking criteria. Work with Leo to produce model answers that would convince the examiner to award you all the marks.
I love this feedback because it is written in a language students understand, it makes students think, it gives them strategies to use if they are struggling, and it should prove to be more work for the students than the teacher. Of course, these comments take time to write, and it is worth bearing in mind that students need to be given dedicated time to act upon this feedback in lessons.
My favourite quote:
The content of effective written comments, of course, varies according to the activity and mathematical content. Often, advice to pupils will be very similar to the kinds of interventions and questions that a teacher uses with the whole class, although it is an opportunity for the teacher to give personalized feedback
 
Research Paper Title: Task-Involving and Ego-Involving Properties of Evaluation: Effects of Different Feedback Conditions on Motivational Perceptions, Interest, and Performance
Author(s): Ruth Butler
My Takeaway:
This is a really interesting study into the effectiveness, and indeed the effects, of different types of feedback. A total of 200 fifth- and sixth-grade students with high or low school achievement were given takes to complete. Individual comments, numerical grades, standardized praise, or no feedback were received after Sessions 1 and 2. The results following a post-test were that interest, performance, and attributions of effort were highest at both levels of achievement after receipt of comments. Indeed, post-test scores were one stand deviation higher for students in the Comments group, with no significant differences between scorers in the other groups. Similarly, ego-involved attributions were highest after receipt of grades and praise. In other words, grades and praise had no effect on performance, and served only to increase ego-involvement. It is perhaps of little surprise that comments were the most successful in generating the kind of response we would want from students. However, I was taken aback by the fact that grades had the same impact as praise in generating ego-involving responses. My conclusion, based on my own experiences with my students, is that grades cause students to think about themselves - eliciting emotions ranging from joy to despair - and this prevent them from focussing on the task itself. This will be discussed further in the next study.
My favourite quote:
The present results further confirmed that individual comments yielded higher task-involved perceptions and lower ego involved ones than either grades or praise and that no feedback yielded perceptions of both kinds of factors as being relatively non-determinative of both effort and outcome. The similar and ego-involved perceptions induced by grades and praise seem particularly significant. Both anecdotal evidence and some research findings suggest that grades are perceived as potent sources of control over learning.

Research Paper Title: Enhancing and undermining Intrinsic motivation; the effects of task-involving and ego-involving evaluation on interest and performance
Author(s): Ruth Butler
My Takeaway:
The last paper showed that task-focussed comments were preferable to grades in terms of subsequent performance on tasks, as well as general involvement and interest in the task. This begs the question: what if comments and grades are combined? This paper provides the answer. 48 eleven-year old Israeli students were selected from the upper and lower quartiles of attainment from 12 classes in 4 schools and worked in pairs over three sessions on two tasks (one testing convergent thinking and the other, divergent). After each session, each student was given written feedback on the work they had done in the session in one of three forms: A) individualised comments on the extent of the match of their work with the assessment criteria that had been explained to each class at the beginning of the experiment; B) grades, based on the quality of their work in the previous session; C) both grades and comments. Students given comments showed a 30% increase in scores over the course of the experiment, and the interest of all students in the work was high. Students given only grades showed no overall improvement in their scores, and the interest of those who had high scores was positive, while those who had received low scores show low interest in the work. Perhaps most surprisingly, the students given both grades and comments performed similarly to those given grades alone—no overall improvement in scores, and interest strongly correlated with scores—and the researchers themselves describe how students given both grades and comments ignored the comments, and spent their time comparing their grades with those achieved by their peers. The message is simple: If you are going to grade or mark a piece of work, you are wasting your time writing careful diagnostic comments.
My favourite quote:
The results confirm the importance of distinguishing between task involvement and ego-involvement when investigating intrinsic task motivation. As hypothesized, both high and low achievers who received comments continued to express high interest both on Session 2 when they anticipated further comments, and at post-test, when they did not.

Research Paper Title: Rank as an Incentive
Author(s): Anh Tran and Richard Zeckhauser
My Takeaway:
The papers above have made clear that if you give students both a grade/mark and a formative piece of feedback, all they will look at is the grade/mark. Here we consider a different type of feedback - instead of telling students their grade, tell them their rank in the class. Whenever I give students back their tests, they are always desperate to find out how everyone else got on, and although I never tell them, they pretty quickly - over whispers, gestures and paper waving - work out their rough ranking in the class. When I interviewed Dani Quinn, the Head of Maths at Michaela School, she explained how she not only ranks her class after every assessment, but puts these rankings on public display on the notice board in the corridor. What does the research have to say about such an approach? Well, this paper makes interesting reading. In one experiment, the researchers gave a class of Vietnamese college students some reading material to prepare for a test that would be given in ten days. Each student received a participation fee. They randomly assigned the students into one control group (with no extra incentive) and three treatment groups (one with rank incentive, one with financial incentive and one with both). They found that: (i) the group that knew that their rankings would be publicised outperformed the control group; (ii) the group that both earned cash for correct answers and knew that their rankings would be publicised outperformed the group that merely earned cash for correct answers. The paper also quotes other research that replicates the general finding that competition improves performance. Before making any sweeping generalisations, we must be aware that the sample size is small and the students are of university age in Vietnam. However, it has made me think. The knowledge that students are going to be ranked may well provide a good incentive for them to put extra effort into their homework or test. However, this must be traded against the possible repercussions of that ranking. Will students who are in the top 3 in the class get complacent? Will students in the bottom 3 in the class give up? As we have seen with much of the research on feedback, a lot of this comes down to the relationship a teacher has with their students, and the school culture as a whole. Michaela's justification is that they believe performance on tests is highly correlated with effort, and so exposing students who perform badly is a way of highlighting the consequences of poor effort. That is fine so long as the correlation hold true. If performance is instead linked to ability, then it will be pretty demoarilsing for those students who may find themselves permanently ranked near the bottom. And what happens if everyone improves? How is that reflected in the rankings? Making students aware of ranking seems to be to be a zero-sum game with a lot of potential risks, but it is something that has certainly caused me to think.
My favourite quote:
Economists admire competition because it promotes efficiency and enables the market system to work efficiently. Rank incentives may be net beneficial in some circumstances, as they encourage all to perform better. They may be detrimental in others. But whatever the net report card, the record is clear. Humans care considerably about their rank, and economic models that seek descriptive relevance must attend to that incentive.

Research Paper Title: The Impact of Self- and Peer-Grading on Student Learning
Author(s): Philip M. Sadler and Eddie Good
My Takeaway:
With so much marking to do, I often like to ask my students to either mark their own work or mark their neighbour's. Indeed, self or peer marking is a fundamental component of my practice of regular low stake tests. But that all begs the question, which is the most useful: self or peer assessment. The findings from this paper are fascinating.
1) For either strategy to work students must be familiar with how to mark accurately. This suggests strategies such as exposing students to mark schemes and showing them examples of other students' work and then discussing them are likely to be successful in achieving this aim. The authors sum this up with the following: "For optimal student-grading, we suggest training, blind grading, incentives for accuracy, and checks on accuracy compared to teacher grades".
2) There are indications of bias in the students' marks: when grading others, students awarded lower grades to the best performing students than their teacher did, but when grading themselves, lower performing students tended to inflate their own low scores.
3) However, what is most interesting to me is the impact on learning. When students were given a follow-up test based on the same content that they had been marking, students who graded their peers’ tests did not gain significantly more than a control group of students, but those students who corrected their own tests improved dramatically. This implies that self-assessment is more beneficial to learning than peer assessment, and I have put this finding into practice by allowing students to mark their own low-stakes tests. My students also seem to prefer this, and it has led to less sounds of "what does this say?" that often accompanies peer assessment. One thing to bear in mind, however, is you lose the benefits of peer-to-peer discussion and feedback that has been identified in some of the papers in this section.
My favourite quote:
Student-grading is best thought of not as an isolated educational practice, but as a part of the system of learning and assessment carried out in a teacher’s classroom. It involves sharing with students some of the power traditionally held by the teacher, the power to grade. Such power should not be exercised recklessly or unchecked. Such power in their own hands or in those of their peers can make students uncomfortable or wary. Teachers should train their students in the skills entailed in accurate grading and should monitor students for accuracy in self- or peer-grading. When used responsibly student-grading can be highly accurate and reliable, saving teachers’ time. In this study, self-grading appears to further student understanding of the subject matter being taught.



Motivation and Praisekeyboard_arrow_up
Back to Top

I had always been motivated to learn mathematics, so it was quite the shock when I started teaching to discover that not every student's idea of a good way to spend Friday night was solving quadratic equations. Until relatively recently in my career, I was convinced that the best way to motivate the majority of students was to give them praise (and that was only because my wallet could not quite stretch to the confectionery-related demands of my students). However, I always had nagging questions. Was there a better way to motivate my students? Was there a danger that I was doing more harm than good with my use of praise? This collection of research is my attempt to answer those questions.

Please Note: papers concerning motivation and praise, but that are more explicitly to do with marking and feedback can be found in the Marking and Feedback section. It is also worth re-reading Top 20 Principles from Psychology that was covered in the Cognitive Science section as that has several key things to say about motivation.

 
Research Paper Title:
How Praise Can Motivate—or Stifle
Author(s): Daniel J Willingham
My Takeaway:
Praise is often a cornerstone of feedback, both written and in lessons, and this paper from Daniel Willingham seeks to answer when it can be a good or bad thing. Willingham summarises things really nicely in the introduction: A rule of thumb that can summarise this complex research literature is that if you try to use praise for your own ends or even in a conscious attempt to help the student, it is likely to go wrong. If, on the other hand, praise is an honest expression meant to congratulate the student, it will likely be at least neutral or even helpful to the student; even under these circumstances, however, care must be taken in what is praised. This shows you what a thorny issue praise an be. For Willingham, the guiding principles are:
1) Praise should be sincere. if praise is dishonest, controlling or unearned, it is likely to have negative consequences. I have struggled in particular with the last one of these. A student who never hands in their homework finally hands something in, and it is a load of rubbish. Do I praise the fact that at least he handed it in, or does that send a signal that such an effort is acceptable? For Willingham, the answer is to say/write something like "It's great that you finished the assignment, but I'm a little disappointed in the quality of this work because I know you can do better"
2) Praise should emphasise process not ability. Praising ability may lead students to have a fixed view of ability, which may be detrimental to their long-term development (this issues is addressed further in the Mindset section). However, simply praising effort has complications as well - often it is socially more acceptable for students to put as little effort in as possible, and being told "you tried really hard" my be interpreted by a student as "you are thick, but nice try". Willingham's solution is to praise the product of the process - "that is a brilliant solution", as opposed to "you worked really hard on that solution"
3) Praise should be immediate and unexpected. Praise obviously loses much of its informational and motivational impact if the teacher praises a child for having shown good effort two weeks ago. However, this should not be confused with the potential benefits of delayed feedback, which are discussed in the Bjork paper later in this section. Making praise unpredictable is hard to do, but can be of huge benefit. The goal is not simply to get the child to stop asking for praise; it is to help the child to think of their work differently—as something that is done for the student's own satisfaction, not to garner praise from the teacher. This needs to be part of a long term strategy with your students.
My favourite quote:
It likely comes as no surprise that praise is neither an automatic expander of self-esteem, nor the ruin of a child's self-efficacy. Praise can take so many forms that its effects are inevitably complex. Still, some useful generalizations can be made. Praise should be sincere, meaning that the child has done something praiseworthy. The content of the praise should express congratulations (rather than express a wish of something else the child should do). The target of the praise should be not an attribute of the child, but rather an attribute of the child's behavior. Parents and teachers are familiar with the admonition "criticize the behavior, not the child." For similar reasons, the same applies to praise—praising the child carries the message that the attribute praised is fixed and immutable. Praising the process the child used encourages the child to consider praiseworthy behaviors as under his or her control.

Research Paper Title: Self-Efficacy: An Essential Motive to Learn
Author(s): Barry J. Zimmerman
My Takeaway:
I think a lot about how to motivate my students. I have tried attempting to make the maths we study more relevant to their lives, tried the all-singing, all-dancing lessons, used videos, technology and more. These strategies have had mixed success, but even the best have not proved sustainable in the long run. This article has convinced me what I think I have always known - students are motivated by their own success. If you can convince students that they are successful at maths (self-efficacy is defined as "as one's belief in one's ability to succeed in specific situations or accomplish a task"), then they will be motivated to learn more. A similar principle is covered in Daniel Pink's excellent book Drive, where he argues that mastery must come first to make something enjoyable - in other words it is not motivation that leads to success, but success that leads to motivation. This view is further reinforced in the paper Intrinsic Motivation and Achievement in Mathematics in Elementary School (you need to pay a fee to read the full text) which found "Contrary to the hypothesis that motivation and achievement are reciprocally associated over time, our results point to a directional association from prior achievement to subsequent intrinsic motivation". Likewise, the Top 20 Principles from Psychology paper in the Cognitive Science section argues that as students develop increasing competence, the knowledge and skills that have been developed provide a foundation to support the more complex tasks, which become less effortful and more enjoyable. When students have reached this point, learning often becomes its own intrinsic reward. So, the evidence seems remarkably clear - achievement and students' belief in their own ability provide motivation, not the other way around. How you help bring about that success and positive belief in ability is up to you. For some it will be a Mindset approach, whereas for others it will be Explicit Instruction to ensure students are equipped with the fundamentals needed for more complex thinking, and hence further success. 
My favourite quote:
This empirical evidence of its role as a potent mediator of students’ learning and motivation confirms the historic wisdom of educators that students’ self-beliefs about academic capabilities do play an essential role in their motivation to achieve.

Research Paper Title: Academic Self-Concept and Academic Achievement: Developmental Perspectives on Their Causal Ordering
Author(s): Frederic Guay, Michel Boivin and Herbert W. Marsh
My Takeaway:
This interesting study adds further support to the findings of the Zimmerman paper above. The authors test theoretical and developmental models of the causal ordering between academic self-concept and academic achievement. They focused on younger children, looking at students across three age groups (Grades 2, 3, and 4 from 10 elementary schools) in an attempt to identify the direction of the relationship between self-concept and achievement - i.e. achievement has an effect on self-concept (skill-development model) or that academic self-concept has an effect on achievement (self-enhancement model). The authors found support for a reciprocal effects model, where the effects run both ways - a virtual cycle. They also found this was consistent across the three age groups. Perhaps most interesting finding of all was that while there is a strong correlation between self perception and achievement and we tend to think of it in that order, the actual effect of achievement on self perception is stronger than the other way round. Once again, the the most effective way to motivate our students may well be to help them achieve success. This will then feed motivation, which will feed further success, and before you know it, everyone is happy :-)
My favourite quote:
In conclusion, we began by arguing that the critical question in self-concept research is whether there exists a causal link from
prior academic self-concept to subsequent achievement. Although there is increasing evidence in support of this effect for older students in middle and high schools, there is a very limited body of strong research and no consistent pattern of results for young students in the early primary school years. This is indeed unfortunate because, as many researchers and practitioners alike argue, this is a critical time for young children to develop positive self-concepts of themselves as students. In contrast to all previous research, we offer a methodologically strong study that provides clear support for this link that is consistent across comparisons based on different age cohorts of young students and different waves within each cohort. In summary, the results of our study provide stronger support for the generality over preadolescent ages of this important link between prior self-concept and subsequent achievement.


Research Paper Title:
Motivation for Achievement in Mathematics: Findings, Generalizations, and Criticisms of the Research
Author(s): James A. Middleton and Photini A. Spanias
My Takeaway:
This is a comprehensive review of recent research into motivation in mathematics, which reaches five interesting conclusions:
1) Findings across theoretical orientations indicate that students' perceptions of success in mathematics are highly influential in forming their motivational attitudes. This is directly related to the findings from the paper above. Students need a relatively high degree of success in mathematics for engagement in mathematics to be perceived as worthwhile. This is not the same as saying "only high ability students will be motivated at maths", as through careful instruction and choice of tasks, every student can enjoy success, and hence enjoy the resulting motivation.
2) Motivations toward mathematics are developed early, are highly stable over time, and are influenced greatly by teacher actions and attitudes. This is interesting. Jo Boaler often stresses the importance of children's early experiences with mathematics, and how one bad experience can put you off the subject for life. For me, the takeaway here is that as teachers we need to be good role models. We need to constantly present a positive attitude towards mathematics, showing it as a fun, challenging, inspiring, wonderful subject where it is absolutely fine to make mistakes on the path to understanding. Crucially, we need to even more explicit with these messages with the youngest students we teach.
3) Providing opportunities for students to develop intrinsic motivation in mathematics is generally superior to providing extrinsic incentives for achievement. Students need to want to do mathematics, not because they expect and extrinsic reward, but because it is a subject they value and enjoy. Now, this is easier said than done, and extrinsic rewards may play a part in motivation during the early stages of encountering a tricky, new topic. But emphasising the importance of maths, and the pleasure that can be had from solving problems, is crucial to develop intrinsic rewards.
4) Inequities exist in the ways in which some groups of students in mathematics classes have been taught to view mathematics. Interestingly, the authors cite evidence that is girls who are influenced through gender-role stereotyping, teacher expectations, and peer pressure to view themselves negatively with respect to mathematics motivation. I do not have any quick-fix solutions to this (certainly Jo Boaler's wonderful paper When Do Girls Prefer Football to Fashion shows that it is not as easy as simply shoehorning in supposedly "female friendly" contexts into lessons,) but I now pay special attention to ensuring I do all I can to raise the expectations and self-belief of all my students.
5) Achievement motivation in mathematics, though stable, can be affected through careful instructional design. This all comes down to careful planning. Planning how you introduce topics to make them seem worthwhile. Planning on how you explain concepts, providing worked examples and scaffolding, to support students and help them believe they can understand it. Providing interesting work for them to do. Using Formative Assessment to identify and misconceptions and resolve them early. Being very careful with praise. Basically, all the things we have looked at on this page that constitute good teaching.
My favourite quote:
Motivation to achieve in mathematics is not solely a product of mathematics ability nor is it so stable that intervention programs cannot be designed to improve it. Instead, achievement motivation in mathematics is highly influenced by instructional practices, and if appropriate practices are consistent over a long period of time, children can and do learn to enjoy and value mathematics

Research Paper Title: Extrinsic Rewards and Intrinsic Motivation in Education: Reconsidered Once Again
Author(s): Edward L. Deci, Richard Koestner and Richard M. Ryan
My Takeaway:
The effects of external rewards on intrinsic motivation is addressed in depth by the Deci, Koestner and Ryan meta-analysis. Their work is based around Cognitive Evaluation Theory (CET). CET proposes that underlying intrinsic motivation are the innate psychological needs for competence and self-determination. According to the theory, the effects on intrinsic motivation of external events such as the offering of rewards, the delivery of evaluations, the setting of deadlines, and other motivational inputs are a function of how these events influence a person's perceptions of competence and self-determination. The key findings from this meta-analysis were as follows:
1. Verbal rewards (i.e., positive, feedback) tend to have an enhancing effect on intrinsic motivation
2. However, verbal rewards are less likely to have a positive effect for younger children (up to the age of 16) than for older individuals.
3. Verbal rewards can even have a negative effect on intrinsic motivation if the interpersonal context within which they are administered is controlling rather than informational.
4. Tangible rewards decrease intrinsic motivation, because tangible rewards are frequently used to persuade people to do things they would not otherwise do, that is, to control their behaviour
5. Tangible rewards may control immediate behaviors, they have negative consequences for subsequent interest, persistence, and preference for challenge, especially for children
6. Unexpected rewards are not be detrimental to intrinsic motivation, whereas expected rewards are. The reasoning is that if people are not doing a task in order to get a reward, they are not likely to experience their task behaviour as being controlled by the reward.
7. Engagement-contingent rewards (those offered explicitly for engaging in an activity, regardless of the outcome of that activity) and completion-contingent rewards (those given for completion of an activity, again regardless of the outcome) significantly diminish intrinsic motivation
8. Performance-contingent rewards (defined as rewards given explicitly for doing well at a task or for performing
up to a specified standard)  can maintain or enhance intrinsic motivation if the receiver of the reward interprets it informationally, as an affirmation of competence. Yet, because performance-contingent rewards are often used as a vehicle to control not only what the person does but how well he or she does it, such rewards can easily be experienced as very controlling, thus undermining intrinsic motivation. Interestingly, the researchers reported the situation whereby in at least some participants got less than the maximum possible rewards was associated with the largest undermining effect on intrinsic motivation of any category used in the entire meta-analysis.
My favourite quote:
To sunmnarize, results of the meta-analysis make clear that the undermining of intrinsic motivation by tangible rewards is indeed a significant issue. Whereas verbal rewards tended to enhance intrinsic motivation (although not for children and not when the rewards were given controlling) and neither unexpected tangible rewards nor task-noncontingent tangible rewards affected intrinsic motivation, expected tangible rewards did significantly and substantially undermine intrinsic motivation, and this effect was quite robust. Furthermore, the undermining was especially strong for children. Tangible rewards-both material rewards, such as pizza patsies for reading books, and symbolic rewards, such as good student. awards- --are widely advocated by many educators and are used in many classrooms, yet the evidence suggests that these rewards tend to undermine intrinsic motivation for the rewarded activity. Because the undermining of intrinsic motivation by tangible rewards was especially strong for school-aged children, and because studies have linked intrinsic motivation to high-quality learning and adjustment, the findings from this meta-analysis are of particular import for primary and secondary school educators

Research Paper Title: Using Load Reduction Instruction (LRI) to boost motivation and engagement
Author(s): Andrew J. Martin
My Takeaway:
This paper introduces the concept of Load reduction instruction (LRI), which the author explains is an umbrella term referring to instructional approaches that seek to reduce and/or manage cognitive load in order to optimize students’ learning and achievement. As such, it is clearly related to the work we have look at with regard to Cognitive Load Theory, and indeed it provides a really comprehensive model of a direct instruction approach from taking learners from novices to experts. However, what is of most interest to us in this section is the how the development of fluency and automaticity within this framework also have implications for students’ motivation and engagement. The author introduces The Motivation and Engagement wheel, which identifies three key components of positive motivation:
  • Self-efficacy is students’ belief and confidence in their ability to understand or to do well in schoolwork, to meet challenges they face, and to perform to the best of their ability.
  • Valuing is how much students believe what they learn at school is useful, relevant, meaningful, and important.
  • Mastery orientation refers to students’ interest in and focus on learning, developing new skills, improving, understanding, and doing a good job for its own sake and not just for rewards or the marks they will get for their efforts.

Interestingly, the author also identifies three factors that may reduce motivation:

  • Anxiety has two parts: feeling nervous and worrying. Feeling nervous is the uneasy or sick feeling students get when
    they think about or do their schoolwork, assignments, or tests. Worrying refers to fearful thoughts
    about schoolwork, assignments, or tests.
  • Uncertain control reflects students’ uncertain or low sense of control, typically when they are unsure how to do well or how to avoid doing poorly.
  • Failure avoidance refers to a motivation to do one’s schoolwork in order to avoid doing poorly, to avoid being seen to do poorly, or to avoid the negative consequences of poor performance.

The author sees good teaching, following a model of direct instruction in novice learners, moving towards more structured discovery as expertise develops, as a means of fostering motivation, and practical ways for all the given factors are discussed. Crucially, as the quote below shows, there is a reciprocal relationship between the development of motivation and the students' achievement, whereby a virtuous cycle could allow both achievement and motivation to be boosted.
My favourite quote:

The present review has identified the potential for LRI approaches to foster and facilitate students’ motivation and engagement. Of course, this connection is not static. Research shows there is a cycle that operates such that learning (‘skill’) fosters subsequent motivation and engagement (‘will’). For example, self-efficacy is likely to be enhanced (or sustained) through the academic knowledge and skill that explicit instruction is shown to develop. Similarly, self-efficacy is associated
with enhanced academic knowledge and academic skill (Schunk & Miller, 2002). Students who are high in self-efficacy generate alternative courses of action when at first they do not succeed, invest greater effort and persistence, and are better at adapting to problem situations (Bandura, 1997). Accordingly, they tend to achieve more highly. There is thus a reciprocal relationship between students’ academic motivation and engagement on the one hand, and their academic learning and achievement on the other hand.


Research Paper Title:
Classroom Applications of Cognitive Theories of Motivation
Author(s): Nona Tollefson
My Takeaway:
This is a fascinating paper that examines cognitive theories of motivation and their application to classroom experiences of students and teachers. There is so much to take away from this paper, but I have limited myself to two things directly relevant to this section and my teaching experiences:
1) "Expectancy x value theory" postulates that the degree to which students will expend effort on a task is a function of (a) their expectation they will be able to perform the task successfully and by so doing obtain the rewards associated with successful completion of the task and (b) the value they place on the rewards associated with successful completion of the task. We can help with (a) by following many of the principles described in the paper above - namely providing good explanations, modelling examples, setting appropriate tasks, and careful use of praise. (b) is perhaps the trickier one. External rewards have the problem that students become over-reliant on them, and they beg the question of what happens when those rewards disappear? Helping students value mathematics for the sake of doing mathematics is difficult, but achievable, and for me much of it comes down to the messages we convey as teachers and role models.
2) In our look so far at both praise and feedback, a resounding message is that we should praise effort and not ability. That makes logical sense, as the former appears more readily changeable (in the eyes of students) than the latter. But this paper points to a potential problem with this. I can explain this no better than in the words of the authors themselves: attributing either success or failure to effort is a ‘‘double-edged sword.’’ On one hand, expending effort and being successful brings a sense of accomplishment and pride. However, having to expend extraordinary effort to be successful implies that one has lower ability than persons who can successfully complete the task with limited or moderate effort expenditures. Students who believe they lack the ability to complete academic tasks successfully may not expend effort because failure would be a public admission of low ability. Covington and Omelich explain that not trying and failing is ‘‘not really failing,’’ because ‘‘true failure’’ occurs only in the case where an individual tries hard to accomplish a task and fails to do so. They also explain failure resulting from lack of effort as an attempt to protect and preserve a sense of self-worth. Wow! So, what are we to do about this? Perhaps we need to be careful to praise not effort, but the outcome of that effort. It sounds a pointless distinction, but praising the solution to a tricky multi-mark exam question ("that is a great solution") as opposed to praising effort ("you have worked really hard on that solution") may make a subtle but key difference.
My favourite quote:
Because the relationship between effort expenditure, success, and feelings of pride is complex, teachers and parents need to recognize that telling students to ‘‘try harder’’ and rewarding them for expending effort will not necessarily encourage students to expend additional effort. The task demands, the value of the rewards associated with the task, students’ outcome and efficacy expectations, goal orientations, levels of task involvement, age, and attributions for success and failure on school-related tasks all interact to explain why some students are willing to expend effort and others are not.

Research Paper Title: “That’s not Just Beautiful—That’s Incredibly Beautiful!” - The Adverse Impact of Inflated Praise on Children with Low Self-Esteem
Author(s): Eddie Brummelman, Sander Thomaes, Bram Orobio de Castro, Geertjan Overbeek and Brad J. Bushman
My Takeaway:
I love this paper. Often as a teacher I over-inflate my praise for students who I know have a particularly low self-esteem in mathematics. I might describe an answer to a relatively straight-forward angles question as "incredible", or a written solution to a linear equation as "amazing". Surely this can do no harm, and hopefully give the student a much needed boost? Well, maybe not. The first finding from this paper is that it is not just me - people (in particular parents and teachers) are more likely to give "over the top " praise to children they perceive as having low self-esteem. In order to figure out whether this actually mattered or not, the authors looked at how being given praise impacted on one particular aspect of children’s behaviour – challenge seeking. The students first completed a questionnaire to assess their level of self-esteem, and then were asked to draw a copy of van Gogh’s Wild Roses. The children were told that a professional painter would then assess their drawing, and tell them what he thought of it. In reality, the painter didn’t exist, and children were simply given inflated praise, non-inflated praise, or no praise at all. Afterwards, the children were shown four complex and four easy pictures, and asked to have a go at reproducing some of them. Crucially, they were told that if they picked the difficult picture, they might make a lot of mistakes, but they might also learn lots - in other words, the number of difficult pictures the children chose to draw was taken as a measure of challenge seeking. The authors found that if children with lower self-esteem were given overly-inflated praise, they were less inclined to seek a challenge in the second task – they would go for easy drawings over the harder ones, and therefore miss out on the chance for a new learning experience. On the other hand, children with high self-esteem were more likely to seek a challenge after being given inflated praise. Interestingly, the only difference between the inflated and non-inflated praise was a single word – incredible (“you made an incredibly beautiful drawing!” versus “you made a beautiful drawing!”). The authors suggest that inflated praise might set the bar very high for children in the future, and so inadvertently activates a self-protection mechanism in those with low self-esteem. This once again suggests that positive praise isn’t necessarily good for all children in all circumstances. For children with low self-esteem, although we might feel the need to shower them in adulation, this might end up having precisely the opposite effect. Even words like incredible can end up having a huge unintended impact. So, when I'm telling my students they have done an amazing job, I will choose my words more carefully.
My favourite quote:
In current Western society, everyday life is replete with instances of inflated praise—like “Perfect!” or “That’s incredibly beautiful!” Our research represents the first empirical study of inflated praise. Our findings show that adults are inclined to give inflated praise to children with low self-esteem. Unfortunately, inflated praise may cause children with low self-esteem to avoid challenges that might lead to failure. These findings show that inflated praise, although well-intended, may cause children with low self-esteem to avoid crucial learning experiences.



Differentiationkeyboard_arrow_up
Back to Top

I find differentiation to be one of the most difficult parts of teaching, and is the thing I am asked most about by teachers of all ages and experiences. Before we dive into the literature, it is worth recapping a few important points we have learned from previous sections:
1) Learning styles are not as important as many of us have been led to believe, and what is more important is that material is presented to students in its most appropriate form (see Cognitive Science)
2) Students are not good at judging what they understand, and so any reliance on student self-assessment as a guide to what they know is not likely to be accurate (see Cognitive Science)
3) Learning and Performance are very different things, and we should be very careful about inferring anything about learning from a student's performance (see Memory)
4) Overlearning can be beneficial (see Memory)
These all imply that we should not worry too much about differentiation in the sense of preparing lots of different materials for different students in advance of a lesson. We should instead focus on teaching all the students as well as we can, making judgements and interventions during the lesson using the principles of Formative Assessment. However, what does research specifically about differentiation have to say on the matter?

Research Paper Title:
The Feasibility of High-end Learning in a Diverse Middle School
Author(s): Catherine M. Brighton et al
My Takeaway:
I include this paper at the start of our discussion of differentiation to call into question the almost universally accepted idea that differentiation is beneficial to student learning. It is a large scale study carried out across many states in the US, and one of the authors (Carol Ann Tomlinson) is a huge proponent of differentiation. The study looked at the effectiveness of differentiated instruction and differentiated assessment. The surprising conclusion from this detailed paper is that the researchers could not get differentiation to work. This finding supports the conclusion from a large meta analysis entitled The effectiveness of universal design for learning (payment needed to access the full paper), which states with reference to differentiated instruction: "the impact on educational outcomes has not been demonstrated". Why on earth would this be the case? Well, there are two possibilities:
1) Differentiation is hard to do correctly. This is the conclusion of the authors, as can be seen in the quote below. And indeed, differentiation is incredibly difficult. It is time consuming preparing a plethora of different resources before the lesson, and it can often lead to chaotic lessons simply trying to manage, monitor and assess students working on different pieces of work.
2) Differentiation itself is not effective. This of course is a controversial view, but for the four reasons I outlined in my introduction to this section, it is certainly worth considering. I am not gong as far as to say all students should be doing all of the same work all of the time, but I think starting from an assumption before a lesson that different students will require different work is dangerous. As we will see from the paper below, it is incredibly difficult to predict how students will respond to a given concept, and once a decision is made it often takes a lot to deviate from that path, especially if you have taken time preparing the resources. I am leaning towards clear instruction for all students following the principles of Explicit Instruction and Cognitive Load Theory, followed by making decisions in the lesson using the principles of Formative Assessment. Having a well chosen, domain-specific extension activity for students who successfully complete the core work is important, and follows the findings related to Expertise Reversal Effect identified in Cognitive Load Theory and from the papers in Problem Solving section. Likewise, having supporting resources ready when needed is likely to be beneficial. Once again, following the benefits outlined in the Cognitive Load Theory section, these are likely to consist of example-problem pairs for the students to study and attempt.
My favourite quote:
Results suggest that differentiation of instruction and assessment are complex endeavors requiring extended time and concentrated effort to master. Add to these complexity current realities of school such as large class sizes, limited resource materials, lack of planning time, lack of structures in place to allow collaboration with colleagues, and ever-increasing numbers of teacher responsibilities, and the tasks become even more daunting.

Research Paper Title: Teaching to What Students Have in Common
Author(s): Daniel Willingham and David Daniel
My Takeaway:
This is a fascinating review of relevant research that reaches the clear conclusion that teaching geared to common learning characteristics can be more effective than instruction focused on individual differences. The rationale behind this is that students are in fact more similar than they are different. The authors identify two varieties of cognitive characteristics that all students share:
(1) things that the cognitive system needs to operate effectively
  • Factual knowledge
  • Practice
  • Feedback from a knowledgeable source
(2) methods that seem to work well to help most kids meet those needs
  • Distribute study over time
  • Practice recalling facts
  • Cycle between the concrete and the abstract.
The point here is that these are characteristics that all students need to learn effectively, and it is clear (to me, at least) that the three characteristics in (1) are far more easily achieved within a framework of Explicit Instruction. Sure, there will be individual differences concerning how quickly certain students comprehend a concept, or the amount of practice certain students need. But, as the authors point out, "the observation that not every student can do everything the exact same way at the exact same time should not lead to the overreaction of hyper-individualizing the curriculum". One final point really resonated with me - the most of failure. We know that students share many common cognitive characteristics, and we know how best to teach to those. But we don't know nearly as much about their cognitive differences and the best ways of teaching to those. And, as we shall see in the next paper, the issue is made even more complex given the fact that students cognitive abilities vary task by task, and day by day. So, teaching to individual differences, as well as being impractical in many large classes, may also not be as effective as we might hope. Students are more similar than they are different, and gearing teaching towards these common characteristics is likely to be more effective than an attempt to over-individualise.
My favourite quote:
Of course, students will differ with regard to how they respond to and benefit from any single instructional strategy in a given lesson. There is no doubt that students have individual differences that are both situational and preferential. And there is no doubt that effective teachers address these differences using their own experience as a guide. But when it comes to applying research to the classroom, it seems inadvisable to categorize students into more and more specialized groups on the basis of peripheral differences when education and cognitive sciences have made significant progress in describing the core competencies all students share. Teachers can make great strides in improving student achievement by leveraging this body of research and teaching to commonalities, not differences.

Research Paper Title:
What Is Developmentally Appropriate Practice?
Author(s): Daniel T. Willingham
My Takeaway:
I love this paper. Willingham begins by providing evidence against Piaget's proposed stages of development, and instead argues two key points. Firstly, development does not occur in discrete, pervasive stages. Anyone aware of this fact is likely to feel the same way I do when we are expected to show students making progress every single time data is reported. Secondly, children’s cognitive abilities vary by task and day, not just by age and individual developmental pace. And that causes a huge problem when planning for differentiation in your lesson. You simply cannot know how well students will respond to a particular task or topic until you do it. Simply assuming some students will not be up to the work denies them the opportunity to prove you wrong. Likewise, assuming other students do not require as much support and guidance risks leaving them with an incomplete understanding. With this in mind, Willingham gives four pieces of advice to teachers with regard to differentiation:
1) Use information about principles, but not in the absolute. It is useful knowing what students should know - and age related expectations are pretty good for this. They may even give you insight into how students may think and their likely misconceptions. But they should act only as a guide, and should not be taken as absolute.
2) Think about the effectiveness of tasks. Often it may not be the concept that is the problem, but the task itself. For example, I have had success introducing key concepts in algebra to Year 7s using The Border problem, whereas other tasks have failed to provide the same level of understand to older, higher achieving students. Finding good tasks, explanations or analogies for key concepts and always learning from each time you use them is crucial.
3) Think about why students do not understand. Related to the point above, if a child seems to not understand something, the issue may not be with the concept itself, but a feature of the task, or  - and I increasingly find this is a case - a missing bit of background knowledge. Effective use of Formative Assessment to assess background knowledge and resolve any misconceptions may be the key to this.
4) Recognise that no content is inherently developmentally inappropriate. This is a big claim! Willingham gives the example of probability: "the notion of probability is embedded in games that children play using dice, and this understanding can be expanded to include the notion of a distribution. Thus, one approach is to help the child gain an intuitive appreciation of a complex principle long before she is prepared to learn the formal description of it. Without trivializing them, complex ideas can be introduced by making them concrete and through reference to children’s experience." Whilst I agree to a certain extent with this, I still feel there are certain topics that should be left until students are older and more mathematically experience. Take trigonometry, for example. I would make it illegal to study trigonometry before Year 10. But this is not just because I feel younger students are not cognitively developed enough to cope with the demands of that content, it is also because I feel it is important to leave some surprises, some topics that have not yet been covered, for students' final years of high school mathematics.
All of this does not make differentiation any easier, but it does suggest to me that giving all students the opportunity to achieve is better than putting them into categories before the lesson. If misconceptions do become apparent in the course of teaching (exposed via the techniques of Formative Assessment), then of course differentiation will be needed. But the point here is that - perhaps unfortunately - we may only be able to deduce this within the lesson itself.
My favourite quote:
If a child, or even the whole class, does not understand something, you should not assume that the task you posed was not developmentally appropriate. Maybe the students are missing the necessary background knowledge. Or maybe a different presentation of the same material would make it easier to understand.

Research Paper Title: How Do Teachers' Expectations Affect Student Learning
Author(s): D. Stipek
My Takeaway:
One aspect of differentiation is planning different materials to give to students, and the potential problems with this have been outlined in the paper above. However, perhaps a more common form of differentiation is the help we give to different students during lessons. In particclar, I have been inclined to offer more help to students who I perceived as lower ability, or who I thought might struggle with a particular concept. This fascinating paper offers up a stark warning against that. It surveys research into the ways in which teachers' beliefs about students affect their behavior toward students. One thing that struck me more than anything way this: by simply offering up help, we may be communicating to the student in question that we have low expectations of them. That makes perfect sense. Many times I will set my class off on some work and immediately ask a student who is prone to struggling something like "do you need any help?", or "give us a shout if you get stuck on anything". I will do this quietly so other students do not hear, and I am genuinely doing it with the best of intentions, but what message is it sending to that student? As we have seen in the Praise and Motivation section, if students do not believe themselves that they can achieve, motivation and subsequent achievement will likely diminish, and we as teachers are key to helping them believe in themselves. My major takeaway from reading this is often the best intentions of teachers can lead to students believing we have low-exceptions of them. In future, I will think more carefully before I act and, as difficult as it may be for all parties involved, let all students struggle for a while.
My favourite quote:
Helping behavior can also give students a message that they are perceived as low in ability, and it can undermine the positive achievement-related emotions associated with success. Meyer (1982) describes a study by Conty in which the experimenter offered unrequested help either to the subject or to another individual in the room working on the same task. Subjects who were offered help claimed to feel negative emotions (incompetence, anger, worry, disappointment, distress, anxiety) more, and positive emotions (confidence, joy, pride, superiority, satisfaction) less than subjects who observed another person being helped. Graham and Barker (1990) report that children as young as six years rated a student they observed being offered help as lower in ability than another student who was not offered help.

Group Work and Cooperative Learningkeyboard_arrow_up
Back to Top

Group work has always been a thorny issue for me - probably not helps by a previous school's obsession with Kagan training. Whilst I see the benefits of having students work together, talk about mathematics, and learn from each other, I am also acutely aware of the possibilities of students being off-task, and relying on others in the group to do the work for them. I guess it comes down to one question: do students learn more being in a group, or working individually? And of course, cooperative learning extends to those occasions where you might say "discuss this with your partner for 10 seconds". Here is my attempt to use educational research to find out the best practices for group work and cooperative learning.

Research Paper Title:
Cooperative Learning and Achievement: Theory and Research
Author(s): Robert E. Slavin, Eric A. Hurley and Anne Chmberlain
My Takeaway:
This is a fascinating review on the research into cooperative learning, which comes to a positive conclusion about its  potential effect on achievement. Alongside reviewing contrasting theories of cooperative learning, the paper identifies two key facts that are necessary for successful cooperative learning. The first is the importance of structuring group interactions. The concept of reciprocal teaching is discussed, with emphasis placed on the importance of the teacher modeling these interactions. Not assuming students instinctively know how to effectively work together will be a key theme of this section. Secondly, there is the issue of group goals and individual accountability. The key finding is this: cooperative learning is most consistently effective when groups are recognised or rewarded based on individual learning of their members. Without this, there is the danger that members of the group will free-ride on others' work. This is obviously tricky to do, and a suggestion from the authors is found in "my favourite quote" below. In short, groups work best when there are:
1) group goals (so pupils are working as a group and not just in a group)
2) individual accountability (so any pupil falling down on the job harms the entire group's work).
One other part of this paper I found fascinating was the section on which ability of student benefits most from cooperative learning. You could argue that high achievers could be held back by having to explain material to their low-achieving group mates. However, it would be equally possible to argue that because students who give elaborated explanations typically learn more than do those who receive them, high achievers should be the students who benefit most from cooperative learning because they most frequently give elaborated explanations. Interestingly, research has failed to provide a conclusive answer to this, and the authors remain in support of cooperative learning for all.
My favourite quote:
If students can only do as well as the group and the group can succeed only by ensuring that all group members have learned the material, then group members will be motivated to teach each other. Studies of behaviors within groups that relate most to achievement gains consistently show that students who give each other explanations (and less consistently, those who receive such explanations) are the students who learn the most in cooperative learning. Giving or receiving answers without explanation has generally been found to reduce achievement (Webb, 1989, 1992). At least in theory, group goals and individual accountability should motivate students to engage in the behaviors that increase achievement and avoid those that reduce it. If a group member wants her group to be successful, she must teach her group mates (and learn the material herself). If she simply tells her group mates the answers, they will fail the quiz that they must take individually. If she ignores a group mate who does not understand the material, the group mate will fail, and the group will fail as well.

Research Paper Title: Group Work for the Good
Author(s): Tom Bennett
My Takeaway:
Tom Bennett reviews the research in favour of group work and teaches a different conclusion. He argues that much of the research derives from the field that can be broadly termed Constructivism - the idea that students are active participants in the process of learning, and not passive recipients of experience and factual knowledge. This area has been covered in detail in the Explicit Instruction section of this page. Bennett argues that once you question the validity of constructivism, then the arguments in favour of group work start to lose their power. Bennett goes on to outline what he sees as four main drawbacks of group work:
1) Disguised inactivity - if you give a task to three or four people, one or two may realise it's time to freeze, because others will carry the burden of the task, and in the meantime, they can coast under the guise of "research" or "running the group."
2) Unequal loading - Related to this is the problem that while ever student might participate, the participation might be profoundly uneven.
3) Inappropriate Socialisation - students may end up competing to see who can discuss the task the least.
4) Unfair assessment - When a teachers praises a pupil, it's a clear one-to-one relationship, whereas in grading groups, we often must give collective grades.
For me, the key point that the author raises is the opportunity cost of group work - what else could the students be doing with their time? If Explicit Instruction could achieve the same result in 5 minutes that a group could achieve in 30 minutes, then how can we justify it? But if the activity (and the class) lends itself well to a group work activity, then the benefits of having students share ideas, learn from each other, and even something as simple as to vary the type of classroom activity to reengage students, may well be worth it.
My favourite quote:
Here's my parting advice: use group work when you feel it is appropriate to the task you want your students to achieve, and at no other time. The irony of the advocates' position is that while it correctly identifies the many benefits to using group work, their error is made when group work is preferred over other strategies because of some imagined potency, or when it is fetishized as a method imbued with miraculous properties. It isn't dogma, it isn't a panacea, and it isn't the messiah. It's one strategy among many. And it's a perfectly reasonable part of a teacher's arsenal of strategies. Not because pseudo-research has settled the matter, but because the teacher feels it appropriate at that time, for that lesson, with those children. And not before.

Research Paper Title: Group work and whole-class teaching with 11- to 14-year-olds compared
Author(s): Maurice Galtona, Linda Hargreaves and Tony Pell
My Takeaway:
This findings of this paper provide somewhat of a contrast to the one above. Here researchers compared the academic performance and classroom behaviour of pupils when taught new concepts or engaged in problem solving in sessions organised either as cooperative group work or whole class, teacher directed instruction. Comparisons of attainment were made in classes of pupils aged 11 to 14 years (Key Stage 3) in English, mathematics and science. The attainment results suggest that a grouping approach is as effective, and in some cases more effective, than when whole class teaching is used. Classroom observation indicated that there were more sustained, higher cognitive level interactions when pupils worked in groups than during whole class discussions. Crucially, the researchers argue that the group work results could be improved still further if teachers gave more attention to training pupils to work in groups and if more time was given to debriefing after group work. For me these last two points are crucial. We cannot assume students intrinsically know how to work in groups, and without training then the drawbacks that Bennett identifies in the paper above may well be realised. This possibly even extends to students talking in pairs. The next paper in this section provides a suggestion for a simple form of training. Secondly, a debrief may go some way towards providing accountability to group members. By discussing what went well and what didn't throughout the group work, and drawing attention explicitly to groups that worked well, the teacher can emphasise the point that group work is to be taken seriously, and offer directions for students to improve.
My favourite quote:
There are a number of reasons for claiming that the group work could have been more effective, to do mainly with the context in which teachers had to operate when taking part in this study. First, as part of training pupils to work effectively in groups it is vital that teachers brief and debrief the class so that they can begin to gain metacognitive awareness of what it means to be part of a group. Debriefing sessions therefore are particularly important because they not only evaluate how individuals responded in the groups but they also call for participants to make suggestions about suitable strategies for improving the situation on future occasions. After each session, observers completed a lesson overview schedule which recorded, amongst other things, whether or not briefing or debriefing had taken place. It was noticeable, however, particularly in science, that teachers rarely found time for these debriefing sessions. It was rare, for example, to observe a science lesson where the teacher with, say, five minutes of the period left preferred to keep discussion of the results over until the next lesson and instead engaged in a debriefing exercise. More often teachers preferred to use an evaluation sheet which they handed to pupils as they left the class. Thus the exercise tended to take the form of an additional homework task rather than generate a debate on the consequences of the previous classroom activity.

Research Paper Title: Teaching children how to use language to solve maths problems
Author(s): Neil Mercer and Claire Sams
My Takeaway:
This paper cites observational research which suggests that primary school children often do not work productively in group-based classroom activities, with the implication that they lack the necessary skills to manage their joint activity. To counteract this, the authors explored the role of the teacher in guiding the development of children’s skills in using language as a tool for reasoning. It involved an interventional teaching programme called Thinking Together, designed to enable children to talk and reason together effectively. The results obtained indicate that children can be enabled to use talk more effectively as a tool for reasoning; and that talk-based group activities can help the development of individuals’ mathematical reasoning, understanding and problem-solving. The important part of this study for me is that the students were explicitly shown how to effectively work in groups, with special emphasis on the language they should use, instead of assuming that they would automatically know how to do so. The lessons preceding the group work were explicitly focussed on making students aware of the need and benefit to work together. They were encouraged to discuss things and ask questions, include everyone’s ideas , ask what people think and what their reasons are , listen to each other, and so on, all of which was supported by clear modelling from the teacher. If we are going to embark upon group work (or even paired work), then such a structured approach before the task itself may well be a sensible step. However, one be aware of the relatively small sample size used in the study before drawing any significant conclusions.
My favourite quote:
More generally, our results enhance the validity of a sociocultural theory of education by providing empirical support for the Vygotskian claim that language-based, social interaction (intermental activity) has a developmental influence on individual thinking (intramental activity). More precisely, we have shown how the quality of dialogue between teachers and learners, and amongst learners, is of crucial importance if it is to have a significant influence on learning and educational attainment.

Mindsetkeyboard_arrow_up
Back to Top

Carol Dweck's work on mindsets has proven both popular and influential across schools over the last decade. The general thrust of her work is students with a growth mindset believe that their intelligence is not fixed, embrace mistakes as learning opportunities, and strive to improve through practice. This can be most readily seen in her book, Mindset. Whilst I see the advantage of having a growth mindset, I am yet to be fully convinced of how to practically develop such a way of thinking in my students. I share some more thoughts in my Takeaways below. 

Research Paper Title:
Boosting Achievement with Messages that Motivate
Author(s): Carol S. Dweck
My Takeaway:
A great introduction to Dweck's work on Mindset. She outlines her belief that students with a fixed mindset follow the cardinal rule "look smart at all costs", which leads to their desire to not work hard, not make mistakes, and if you do make mistakes to not try to repair them. In contrast, those with a growth mindset follow the rule "it is much more important to learn than get good grades", which leads them to take on challenges, work hard, and correct any mistakes. Studies quoted show that teaching students a growth mindset results in increased motivation, better grades, and higher achievement test scores
My favourite quote:
Many teachers see evidence for a fixed mindset every year. The students who start out at the top of their class end up at the top, and the students who start out at the bottom end up there. Research by Falko Rheinberg shows that when teachers believe in fixed intelligence, this is exactly what happens. It is a self-fulfilling prophecy. However, when teachers hold a growth mindset, many students who start out lower in the class blossom during the year and join the higher achievers.

Research Paper Title: Is it true that some people just can't do math?
Author(s): Daniel J Willingham
My Takeaway:
This is a really sensible discussion, backed by research, into the effect of student's ability and mindset when it comes to high school mathematics. Willingham argues that whilst it is true that some people are better at maths than  others - just like some are better than others at writing or building cabinets or anything else it - is also true that the vast majority of people are fully capable of learning the levels of mathematics they need for high school. We have seen in the Cognitive Science section that learning mathematics is Biologically Secondary knowledge, and so does not come as naturally as learning to speak, but our brains do have the necessary equipment. So, learning maths is somewhat like learning to read: we can do it, but it takes time and effort, and requires mastering increasingly complex skills and content. I love this bit: "Just about everyone will get to the point where they can read a serious newspaper, and just about everyone will get to the point where they can do high school level algebra and geometry even if not everyone wants to reach the point of comprehending James  Joyce's Ulysses or solving partial differential equations." For Willingham, in order to be successful in maths, students need the following three types of knowledge:
1) Factual - this is knowledge of times tables and number bonds which must be automated as discussed in the Fluency section. As we have seen throughout the research studies on this page, without this knowledge students' working memories are likely to become overloaded when attempting to solve more complex problems.
2) Procedural - this is a sequence of steps by which a frequently encountered problem may be solved. This may involve using the gird method for long multiplication, or SOHCAHTOA for trigonometry questions.
3) Conceptual - this refers to an understanding of meaning; knowing that multiplying two negative numbers yields a positive result is not the same thing as understanding why it is true.
These latter two types of knowledge form the basis for the conflicts between progressives and traditionalist views of maths education. Conceptual knowledge is probably the most difficult type for students to acquire, but it is of fundamental importance as new mathematical concepts almost always build on previous ones (for example, a clear understanding of the equals sign is needed to solve linear equations). Without this conceptual knowledge, students are forced to rely on algorithms to carry out procedures, but given the amount of procedures in mathematics, not to mention to difficulty of spotting which procedure is required, means this is simply unsustainable. What does all of this mean for maths teaching, and for the issue of mindset, I hear you ask? Well, Willingham has five pieces of advice for maths teachers, each of which is expanded upon in the paper:
1) Think carefully about how to cultivate conceptual knowledge, and find an analogy that can be used across topics
2) In cultivating greater conceptual knowledge, don't sacrifice procedural or factual knowledge
3) In teaching procedural and factual knowledge, ensure that students get to automaticity
4) Choose a curriculum that supports conceptual knowledge
5) Don't let it pass when a student says "I am no good at maths"
And it is the final point that brings us back to mindset. For Willingham, as the quote below illustrates, maths is something that can be learned through hard work and good teaching. As I said in my introduction, I firmly believe the concept of a growth mindset is important, but we need to help the students see that their achievement in maths can be changed for the better.
My favourite quote:
We hear it a lot, but it's very seldom true. It may be true that the  student finds math more difficult than other subjects, but with some persistence and hard work, the student can learn math and as he learns more, it will get easier. By attributing the difficulty to an unchanging quality within himself, the student is saying that he's powerless to succeed.

Research Paper Title:
Academic Tenacity: Mindsets and Skills that Promote Long-Term Learning
Author(s): Carol S. Dweck, Gregory M. Walton, Geoffrey L. Cohen
My Takeaway:
This particular paper presents some compelling findings and practical strategies to help foster a growth mindset and develop academic tenacity. Two findings that stood out to me were:
1) Students who received effort praise chose challenging tasks that could help them learn, while students who received intelligence praise were more likely to choose tasks in their comfort zone that they could perform well on;
2) Classrooms that encourage competition and individualistic goals may be particularly ill suited to minority students, who are more likely to be reared in cultural contexts that emphasize the importance of communal and cooperative goals over individualistic or competitive goals.
My favourite quote:
Research shows that students’ belief in their ability to learn and perform well in school—their self-efficacy—can predict their level of academic performance above and beyond their measured level of ability and prior performance

Research Paper Title:
Does mindset affect children’s ability, school achievement, or response to challenge? Three failures to replicate.
Author(s): Yue Li & Timothy C. Bates
My Takeaway:
I include this paper for balance as it was the most empirically rigorous one I could find that questions Dweck's findings on mindset. Specifically, the authors find praise for intelligence failed to harm cognitive performance and children’s mindsets had no relationship to their IQ or school grades. Finally, believing ability to be malleable was not linked to improvement of grades across the year. Two points I would raise here: the authors do not find that promoting mindset beliefs to have a negative effect, and more importantly work by the likes of Dan Willingham and others discussed above strongly believe that effort and practice can significantly increase achievement. Hence, I personally have no problem promoting a growth mindset, both in students and teachers. However, a growth mindset without the success that will most likely follow from an Explicit Instruction approach may not be sustainable in the long run, in my opinion.
My favourite quote:
We find no support for the idea that fixed beliefs about basic ability are harmful, or that implicit theories of intelligence play any significant role in development of cognitive ability, response to challenge, or educational attainment.




Maths Anxietykeyboard_arrow_up
Back to Top

Until I began taking a keen interest in educational research, I must confess that I did not quite realise just how significant a problem maths anxiety is. Ashcraft defines maths anxiety as a feeling of tension, apprehension, or fear that interferes with maths performance. And that is one of the really sad things about maths anxiety - it stops students reaching their potential. In this section I will discuss a number of papers that have had a profound effect on me, and look at their practical implications for the classroom and beyond.


Research Paper Title: Math Anxiety: Personal, Educational, and Cognitive Consequences
Author(s): Mark H. Ashcraft
My Takeaway:
This article is a summary of Ashcraft's 30 years of work into the study of maths anxiety. There are a number of key points raised in the paper, but here are a selection that caught my eye:
1) Highly maths-anxious individuals avoid maths. They take fewer elective maths courses, both in high school and in college, than people with low maths anxiety. And when they take maths, they receive lower grades. Highly math-anxious people also espouse negative attitudes toward maths, and hold negative self-perceptions about their math abilities. The correlations between math anxiety and variables such as motivation and self-confidence in math are strongly negative. Now, of course our secondary school students cannot formally opt-out of studding maths until A Level, but they can informally do so. They can study less at home, take part less in class, and talk themselves out of even so much as attempting a problem. And with practice in maths being so vital to success, we end up in a vicious cycle with avoidance leading to poorer performance, which inevitably adds to the existing anxiety.
2) Maths anxiety is only weakly related to overall intelligence. Moreover, the small correlation of 0.17 between maths anxiety and intelligence is probably inflated because IQ tests include quantitative items, on which individuals with maths anxiety perform more poorly than those without maths anxiety. This was a surprise to me. I had assumed a strong relationship between intelligence and maths anxiety. But when I think about it, I can recall many students I have taught over the years who were highly able mathematically, and yet had (what seemed to that stupid, uninformed me) an irrational fear of the subject. The next paper addresses this crucial point further.
3) Timed tests seem to cause anxiety. The researchers found no anxiety effects on whole-number arithmetic problems when participants were tested using a pencil and- paper format. But when participants were tested on-line (i.e., when they were timed as they solved the problems mentally under time pressure in the lab), there were substantial anxiety effects on the same problems. This may seem obvious,. but it in fact poses us with two major problems. Firstly, every major exam that students encounter has a timed element to it. Secondly, as we will see in the Fluency section, without the time pressure students are unlikely to develop the kind of automatic knowledge of key number facts that they need to free up capacity in working memory to solve more complex problems. Perhaps the key is to introduce the timed element slowly and carefully, in a supportive atmosphere, together with a policy of never collecting in or announcing students' scores.
4) Maths anxiety lowers performance because it takes up vital space in working memory. Anxious individuals devote attention to their intrusive thoughts and worries, rather than the task at hand. In the case of maths anxiety, such thoughts probably involve preoccupation with one’s dislike or fear of math, one’s low self-confidence, and the like. Maths anxiety lowers math performance because paying attention to these intrusive thoughts acts like a secondary task, distracting attention from the math task. It follows that cognitive performance is disrupted to the degree that the maths task depends on working memory.
5) Related to this is the finding that maths anxiety to does lower performance in all areas of maths - just the more cognitively demanding ones. Routine arithmetic processes like retrieval of simple facts require little in the way of working memory processing, and therefore show only minimal effects of math anxiety. But problems involving carrying, borrowing, and keeping track in a sequence of operations (e.g., long division) do rely on working memory, and so should show considerable maths anxiety effects. Higher-level math (e.g., algebra) probably relies even more heavily on working memory, so may show a far greater impact of math anxiety. There are two things we can do to help students with this. The first is to try to reduce maths anxiety, and the final paper in this section looks at strategies for this. The second is to ensure students have the relevant knowledge and procedures stored in long-term memory to free up capacity in their working memories. This, of course, is easier said than done, going back to Point 3).
6) Finally, the authors make what I feel is a key point: note how difficult it will be, when investigating high-level math topics, to distinguish clearly between the effects of high math anxiety and low math competence.
My favourite quote:
Math anxiety is a bona fide anxiety reaction, a phobia, with both immediate cognitive and long-term educational implications.

Research Paper Title: Math Anxiety, Working Memory, and Math Achievement in Early Elementary School
Author(s): Gerardo Ramirez, Elizabeth A. Gunderson, Susan C. Levine, and Sian L. Beilock
My Takeaway:
The key finding from this paper is both fascinating and worrying: Students with the highest level of working memory capacity show the most pronounced negative relation between maths anxiety and math achievement. How can this be? We have seen that maths anxiety depresses maths performance because it eats up working memory space. So, wouldn't these students have spare working memory capacity, so anxiety would have less of an impact? The authors explain that there is no definite explanation for this, but that one possibility is that students with the most working memory (WM) capacity tend to rely on more advanced problem-solving strategies. High-WM children, for example, are more likely to use direct retrieval as opposed to finger counting when solving math problems, and retrieval efficiency is particularly disrupted by interference. In contrast, low-WM children’s maths achievement may remain relatively unaffected by maths anxiety precisely because they use less sophisticated (and less WM-demanding) problem-solving strategies. Hence, the association between math anxiety and math achievement may be present among high-WM (but not low-WM) children because math anxiety disrupts the resources that high-WM children rely on to retrieve basic facts from long-term memory and to inhibit competing answers. Even more concerning is the suggestion that maths anxiety-induced disruption of WM leads high-WM children to switch to less successful problem-solving strategies as a means of circumventing the burden of maths anxiety on WM. Ironically, something that usually helps students in maths — large working memory capacity— becomes vulnerable to disruption when they are anxious.
My favourite quote:
In conclusion, our results highlight the potential of math anxiety to negatively impact children’s math achievement as early as first and second grade. The finding that children who are higher in WM may be most susceptible to the deleterious effects of math anxiety is particularly worrisome because these students arguably have the greatest potential for high achievement in math. Investigating the development of math anxiety from the earliest grades will not only increase our understanding of the relation between math anxiety and math performance across the school years but is also a critical first step in developing interventions designed to ameliorate these anxieties and increase math achievement.

Research Paper Title: Math Anxiety: can teachers help students reduce it?
Author(s): Sian L. Beilock and Daniel T. Willingham
My Takeaway:
This is a wonderful overview of the research into maths anxiety, including some shocking statistics as to how widespread it is (see quote below). What I found most useful was the authors offering up four practical strategies teachers can employ to help reduce maths anxiety in their students:
1) Ensure fundamental skills. Once again we see the importance of knowledge, not just for thinking but for reducing anxiety. The authors state that enhancing both numerical and spatial processing may help guard against the development of maths anxiety in younger students.
2) Focus on teacher training. The is based on the finding that a teacher's anxiety about maths can be transferred to their students. Ensuring teachers are confident both in their knowledge of the subject and effective ways to communicate content may help with this. I would argue that well-planned lessons following a model of explicit instruction, with worked examples and structured purposeful practice, are easier to deliver and "control" than more inquiry based lessons.
3) Change the assessment. We have seen the maths anxiety is more strongly linked to poor performance when students take a timed test due to the burden it places on working memory. So removing the time element is likely to be beneficial by reducing worries and giving students time and space to consider their answers
4) Use a writing exercises. I must admit,l I had not considered this one. Giving students around 10 minutes to write freely about their emotions concerning an upcoming event (such as an exam) can alleviate the burden negative emotions place on working memory and hence boost test performance.
5) Think carefully what to say when students struggle. Often consolation in the form of "it's okay, not everyone can be good at these kind of problems", validates students' view that they are not good at maths. Better to focus on how you are convinced that hard work will help them get better, crucially following this up with concrete, effective study strategies like those outlined in the Revision section.
My favourite quote:
Math anxiety is not limited to a minority of individuals nor to  one country. International comparisons of high school students  show that some students in every country are anxious about  math. It is perhaps unsurprising that there is an inverse relationship between anxiety and efficacy: countries where kids are less  proficient in math (as measured by the Program for International  Student Assessment, or PISA) tend to have higher levels of math anxiety. In the United States, an estimated 25 percent of four-year college students and up to  80 percent of community college students suffer from a moderate  to high degree of math anxiety. Most students report having at least one negative experience with math at some point during their schooling.



Fluency with Maths Factskeyboard_arrow_up
Back to Top

Research from the Cognitive Science, Explicit Instruction and Cognitive Load Theory sections of this page stress the importance of students having key mathematical facts ready to be retrieved from long term memory so they do not take up valuable space in working memory. For want of a better expression, I will refer to this as developing fluency with maths facts. Over the last few years, I have tried to get students to develop this fluency via number talks, and more recently through drills/rote learning. This section is my attempt to find research-based evidence on the merits of these, as well as to address the issue of whether fluency is important when students have calculators and mobile phones!

For an overview of how Number Talks work, I would recommend this article, together with this video from Jo Boaler.

For an overview of drilling, I would recommend reading the following two blog posts by teachers at Michaela Community School: Dani Quinn and Hin-Tai Ting.

Research Paper Title: Assisting Students Struggling with Mathematics
Author(s): Institute of Education Sciences
My Takeaway:
The second appearance of this brilliant paper which is full of research-based evidence and practical strategies for assisting students who struggle with mathematics. This time we turn to a recommendation that is directly relevant to this section:
Recommendation 6 - Interventions at all grade levels should devote about 10 minutes in each session to building fluent retrieval of basic arithmetic facts.
The paper then goes on to suggest three practical strategies to aid with this:
1. Provide about 10 minutes per session of instruction to build quick retrieval of basic arithmetic facts. Consider using technology, flash cards, and other materials for extensive practice to facilitate automatic retrieval.
2. For students in kindergarten through grade 2, explicitly teach strategies for efficient counting to improve the retrieval of mathematics facts.
3. Teach students in grades 2 through 8 how to use their knowledge of properties, such as commutative, associative, and distributive law, to derive facts in their heads.
For me, strategies 1 and 2 are all about drilling, whereas strategy 3 lends itself well to the concepts of Number Talks which will be covered later on in this session. The overall conclusion is clear - a knowledge of basic mathematical facts is necessary in order for students to achieve success in mathematics, and as teachers we should help students gain this by making a focus on fluency a regular part of each lesson.
My favourite quote:
These studies reveal a series of small but positive effects on measures of fact fluency128 and procedural knowledge for diverse student populations in the elementary grades.In some cases, fact fluency instruction was one of several components in the intervention, and it is difficult to judge the impact of the fact fluency component alone. However, because numerous research teams independently produced similar findings, we consider this practice worthy of serious consideration. Although the research is limited to the elementary school grades, in the panel’s view, building fact fluency is also important for middle school students when used appropriately.

Research Paper Title: Improving Basic Multiplication Fact Recall for Primary School Students
Author(s): Monica Wong and David Evans
My Takeaway:
This is a great overview into the importance of times tables, and the strategies students employ. There are a number of key points addressed:
1) Whilst conceptual understanding of the times tables is important, it is not sufficient. Automaticity must be achieved in orer to free up space in working memory to solve more complex problems (I love this quote: The  importance  of  automaticity  becomes  apparent  when  it  is  absent. Lessons may stall as students look up facts they should recall from memory. Thus conceptual  understanding  is  necessary,  but  insufficient  for  mathematical proficiency)
2) Basic  multiplication  facts  are  considered  to  be  foundational  for  further advancement  in  mathematics.  They  form  the  basis  for  learning  multi-digit multiplication,  fractions, ratios, division, and decimals  Many tasks across all domains of  mathematics  and  across  many  subject  areas  call  upon  the  recall  of  basic multiplication  facts  as  a  lower-order  component of  the overall  task. To  enable students  to  focus  on  more  sophisticated  tasks  such  as  problem  solving, proficiency  in basic  facts and  skills  is an advantage. Without  procedural  fluency  and  the ability to recall facts from memory, the student’s focus during problem solving will be on basic skills rather than the task at hand, thus drawing attention away from  the  learning objectives of  the  task. If  the  student cannot perform  these basic  calculations without  the need  to use  calculators or other  aids,  higher-order  processing  in  problem  solving  will  be  impeded
3)  When answers are predominantly recalled  from memory,  the student should be able  to answer approximately 40 basic mathematics  questions  correctly  in  one minute
4) The order in which facts are introduced and sequenced can assist students to become proficient in learning and recalling basic multiplication facts. Facts that can be learned easily should be presented first during practice (e.g., 0, 1, 10, 2, 5, 9), then they should be followed by the more demanding multiplication sequences (e.g., 4, 7, 3, 8, and 6). Students should also be proficient at counting from 1 to 100 and be able to skip count.
5) To  improve speed  of  fact  recall,  students  should  be  given  a  specific  time  to  respond  to  a question or a constant time delay, typically starting at five seconds and gradually reducing  to  one  and  a  half  seconds.  Reducing  the  response  time  forces  the student  to  abandon  inefficient  counting  strategies  and  attempt  to  retrieve  the answer from memory
6) Practice  on  computers  is  said  to  afford  some  advantages  over  more traditional delivery modes. Students can progress at their own rate and practise using  varying  representations  (horizontal  or  vertical).  Feedback  is  immediate and scoring systems automatically monitor progress. Students who used computers as part of  their usual  instruction generally  learn more  in less  time and retain  the  information  for  longer
7) A  constant  time  delay  approach  is effective for developing fluency in students with learning difficulties. for The students were then taught 15 unknown multiplication facts using computer-assisted instruction based on a five second constant time delay procedure.  The results indicated that the constant time delay procedure was an effective method of teaching multiplication facts to those students. 
8) Interspersion of known and unknown facts in each practice session increases the speed at which facts are committed to, maintained in, and retrieved from long- term memory. It also assists in the remediation of errors from previous sessions and improves the speed of retrieval of known facts from long-term memory.
9) Procedural and conceptual misunderstandings need addressing. Some students consistently calculated the A x 0 multiplication  fact  incorrectly  by  writing  the  value  of  the  A,  whereas  its commuted counterpart, 0 x A was answered as 0. The revision of concepts and procedures  needs  to  be  included  in  the  practice  sessions  to  ensure  students possess the necessary pre-skills to answer questions accurately. This can be achieved through building conceptual knowledge though the use of manipulatives, through building ‘generalisable’ rules, and through providing practice in the use  of  virtual manipulatives.  Virtual manipulatives  allow  the  introduction  of concepts, and provide practice and remediation
10) While proficiency  in multiplication  facts  is  important,  there are also other basic  facts  that  require  practice  to  maintain  ongoing  development  of mathematical proficiency, such as addition and subtraction facts.  Therefore  practising  other  basic  arithmetic  skills could  be  included  in  practice  sessions  like  those  used  in  this  study.  Initially, multiplication  facts may  be  practised  separately  to  promote  proficiency;  later they could be mixed with other facts to allow students to become more proficient in selecting from and discriminating between operations. Although it is a more
My favourite quote:
In  summary,  the  belief  that  the  development  of  basic multiplication  fact recall  is  enhanced  by  practice  has  been  supported. Results  generalised  to  the study group have shown that a systematic practice of basic multiplication facts by  interspersing known and unknown  facts  improved students’  recall of  these facts for all but a few.  For these few students, revisiting multiplication concepts may be necessary. Without  this  improved  recall of basic multiplication  facts, working memory  is consumed  by  the most  fundamental  of problems. Releasing working memory capacity  allows  students  to  tackle  more  difficult  tasks  such  as  multi-step problems or questions demanding higher-order thinking. 


Research Paper Title:
Developing Multiplication Fact Fluency
Author(s): Jonathan Brendefur, S. Strother, K. Thiede and S. Appleton
My Takeaway:
This paper investigates if presenting times tables with different representations can help students develop fluency in them more effectively than drilling. The researchers used third, fourth and fifth grade students, splitting them up into two groups. the Strategy group received instruction based on cognitive and social-interactional framework for fluency development; whereas, the  Drill group received fluency instruction for basic multiplication facts using an approach  emphasizing memorization and rehearsal techniques typically practiced in schools. The techniques used in the Strategy group were Based around Bruner's theory of Modes of Representation, and used the following procedure:
1. Strategy group students began by building arrays with physical models (e.g. tiles) and  finding arrays in pictures as well as the surrounding environment. Students then drew  diagrams of the arrays on either grid paper (to structure the drawings) or freehand.
2. Students then transitioned from arrays to using a 12x12 blank grid as a multiplication  table as both a way to list facts they knew but also as an example of an array. Students  overlaid their derived facts strategies (e.g. 8x5 + 8x1 to recall 8x6) on these 12x12  multiplication grids.
3. Eventually, students' materials were removed and they were engaged in what was  called "fluency-talks". Students sat on the floor with no writing materials or  manipulatives available and were presented with various facts. They had to discuss as a
class how they might use related facts to solve the unknown facts presented on the  board.
4. To culminate the strategy group's fact development, pairs of students created sets of  strategy cards, which were essentially multiplication flash cards (with a fact on the front  of the card), but strategy cards included derived facts strategies the pair preferred  written on the back of the card. The pairs would alternate describing two or three facts strategies for each card's fact.
The results were that sstudents receiving instruction through drill and rehearsal gained on  average 0.79 facts per minute over the five weeks, with positive gains of 2 facts per minute in  fourth grade 2.36 facts per minute in fifth grade. Students receiving instruction grounded in the social-interactional approach demonstrated an average gain of 6.08 facts per minute, with the highest gains in fourth grade of 6.65 facts per minute. This suggests a different approach is needed to introduce students to times tables at a younger age, or possibly to help those older students who do not have a firm grasp of times tables develop the fluency they need.
My favourite quote:
The evidence from this study also demonstrates that students receiving instruction grounded  in a framework built upon Bruner's Modes of Representation combined with social-  interactional elements significantly outperform students who receive instruction grounded in a  behavioristic theory of learning. These instructional activities designed for the Strategy group  emphasized strategic thinking and mathematical relationships between multiplication facts  and created greater and more consistent gains in fact fluency than activities emphasizing  memorization and repetition. Although Brownell (1935) had similar conjectures and findings,  more recently Russell (2000) found that students build an understanding of multiplication  facts through problem solving and, then, sharing and examining their own strategies.  One of the implications to this finding involves memory: one might presume that students will  remember any piece of information (including multiplication facts) more accurately if the  information is connected to already easily-remembered information (Hiebert & Carpenter,  1992). In the case of multiplication facts, this would mean that students who know 8x5 would  ideally spend instructional time learning to use that knowledge to solve 8x4, 8x6, and 8x7 and  so forth. Time spent building flexibility with facts would in turn produce fluency with facts as  students would have related strategies to refer to should memory fail them. This would  contrast with a similar amount of time being spent by students trying to commit 8x4, 8x6, 8x7,  etc. to memory.


Research Paper Title: Automaticity in Computation and Student Success in Introductory Physical Science Courses
Author(s): JudithAnn R. Hartman, Eric A. Nelson
My Takeaway:
Students (and parents!) often say to me something along the lines of: "why do I need to work that out when I can just bang it into my calculator/smart phone?". To be honest, my answers over the years have not been great. Fortunately, with the findings of this fascinating paper, I now have some evidence up my sleeve. This paper looks at the effect the move away from the practice of key mathematical skills in US high schools has had on students taking science degrees. The authors find that between 1984 and 2011, the percentage of US bachelor’s degrees awarded in physics declined by 25%, in chemistry declined by 33%, and overall in physical sciences and engineering fell 40%. Data suggest that these declines are correlated to a K-12 (kindergarten to the end of high school) de-emphasis in most states of practicing computation skills in mathematics. The authors cite recent studies in cognitive science that have found that to solve well-structured problems in the sciences, students must first memorize fundamental facts and procedures in mathematics and science until they can be recalled “with automaticity,” then practice applying those skills in a variety of distinctive contexts. Even with access to a calculator, students working memories can become overloaded, which can prevent them being able to solve more complex problem and hence inhibit learning. To explain this further, I can do no better than to quote from the paper itself:
My favourite quote:
As one example, if as part of a calculation “8 times 7” cannot be recalled, the calculator answer of 56 must be stored in working memory so that it can be transferred to where the calculation is being written. On a problem of any complexity, that storage may bump out of working memory an element that is needed to solve the problem. An answer from a calculator takes up limited working memory space; an answer recalled from long term memory does not.

If arithmetic and algebraic fundamentals are automated, when examples are based on simple ratios or equations, room is available in novel WM for the context that builds conceptual understanding, and problem solving builds an intuitive, fluent understanding of when to apply facts and procedures (Willingham 2006). Conversely, if a student lacks “mental math” automaticity, conceptual explanations based on proportional reasoning or “simple whole-number-mole ratios” will likely not be simple. If a student must slowly reason their way through steps of algebra that could be performed quickly if automated, the “30 seconds or less” limit on holding the goal, steps, and data elements of the problem in working memory ticks away.

In recent years, the internet has facilitated the finding of facts and procedures, but new information occupies the limited space in novel WM that is needed to process the unique elements of a problem. Unless new information is moved into long term memory by repeated practice at recall, during future problem solving that new information will again need to be sought, and when found, it will again restrict cognitive processing (Willingham 2004, 2006).


Research Paper Title:
Fluency without Fear
Author(s): Jo Bolaer
My Takeaway:
Jo Boaler agrees on the importance of knowing mathematical facts, but argues that the memorisation of math facts through times table repetition, practice and timed testing is unnecessary and damaging. Boaler goes further to argue that when teachers emphasise the memorisation of facts, and give tests to measure number facts, students suffer in two important ways. Firstly, for about one third of students the onset of timed testing is the beginning of math anxiety, which can block working memory and prevent learning taking place. Secondly, they can put students off mathematics for life. Boaler advocates a move away from seeped and memorisation, and towards encouraging students to work with, explore and discuss numbers. This will allow them to commit important facts to memory, but in a fun and engaging context. She then goers on to describe Number Talks, as well as some other strategies for developing a sense of number. My concern with this is founded in personal experience of using and observing Number Talks extensively over the last three years - whilst it is supposed to be the lowest achieving students who gain the most from them, I have found they are often held back from fully participating by their lack of knowledge of key maths facts. It ends up being the teacher suggesting strategies, students copying them down and seemingly understanding then, only for them to be unable to transfer them to a new calculation in the next Number Talk. Contrast this to higher achieving students, who can happily break apart and put back together numbers in wonderfully efficient ways, and really seem to get a lot out of the Number Talks, sharing and discussing each others' approaches. I believe all students can (and should) get to this level, but I don't think this can be done through Number Talks alone. Students cannot develop fluency with numbers without these facts in long term memory. I have found that the more comfortable students get with their times tables and number bonds, the more readily they can think of, and successfully carry out the kind of efficient strategies that Number Talks promote. I draw the analogy with problem solving - you cannot teach problem solving by just showing students how to solve problems. Likewise, I believe you cannot teach fluency with numbers simply by showing students how to be fluent.
My favourite quote:
High achieving students use number sense and it is critical that lower achieving students, instead of working on drill and memorization, also learn to use numbers flexibly and conceptually. Memorization and timed testing stand in the way of number sense, giving students the impression that sense making is not important. We need to urgently reorient our teaching of early number and number sense in our mathematics teaching in the UK and the US. If we do not, then failure and drop out rates - already at record highs in both countries - will escalate.

Research Paper Title: Developing Fluency with Basic Number Facts: Intervention for Students with Learning Disabilities
Author(s): Katherine Garnett
My Takeaway:
This paper provides a fascinating insight into attempts to develop fluency with students with learning difficulties. The authors cite that on timed assessments, 5th grade students with learning disabilities completed only one-third as many multiplication fact problems as their non disabled counterparts. Interestingly, the students with learning disabilities were very much slower, but not significantly less accurate, than their non-disabled peers. Additionally, they demonstrated basic conceptual understanding of the basic maths operations. Thus, many students with learning disabilities establish basic understanding of the number relations involved in basic facts, but continue using circuitous strategies long after their non-disabled peers have developed fluent performance. And as we have seen from Cognitive Load Theory, it is the inability to recall facts from long term memory that will hinder such students in solving more complex problems as their working memories will become overloaded. However, the author points out that becoming fluent is not simply a case of remembering a load of facts - it is about forming a well-developed network of number relationships, easily activated counting and linking strategies, and well-practiced navigational rules for when to apply which maneuver". The author goes on to argue that the only way to develop these skills is though several years of frequent and varied number experiences and practice, and drilling is not enough. The author recommends presenting Challenge Problems and discussing strategies, much in the same way as Number Talks, and a really useful collection of prompt questions is provided. I am still not convinced that strategy comes before knowledge, but I am convinced that simply knowing facts is not enough given the immeasurable number of combinations of facts that would be required to answer every single maths problem!
My favourite quote:
In investigating the effects of challenge problems, many of the guidelines offered here would be useful, especially the emphasis on interactive, oral work. Regularly including challenge problems in student/teacher interactive math work could well promote the "mental math" prowess needed by so many students with learning disabilities who cling to number lines and paper-pencil routines.

Research Paper Title: Mastering Maths Facts: Research and Results
Author(s): Otter Creek Institute
My Takeaway:
This authors provide a really good summary of the key findings from this paper: "Learning math facts proceeds through three stages: I) procedural knowledge of  figuring out facts; 2) strategies for remembering facts based on relationships; 3) automaticity in maths facts—declarative knowledge. Students achieve automaticity with math facts when they can directly retrieve the correct answer, without any intervening thought process. The development of automaticity is critical so students can concentrate on higher order thinking in maths. Students who are automatic with math facts answer in less than one second, or write between 40 to 60 answers per minute, if they can write that quickly. Research shows that math facts practice that effectively moves students towards automaticity proceeds with small sets of no more than 2 —4 facts at a time. During practice, the answers must be remembered rather than derived. Practice must limit response times and give correct answers immediately if response time is slow. Automaticity must be achieved with each small set of facts, and maintained with the facts previously mastered, before more facts are introduced. Suggestions for doing this with flashcards or with worksheets are offered." So, this paper stresses the importance of the automaticity of maths facts, and advocates the use of timed drills to achieve this. I found the suggested order to be interesting: strategies comes before automaticity. This did not seem to fit in with my experiences described in my Takeaway on the Boaler article above. However, digging a little deeper, the authors suggest that strategy comes first for addition and subtraction facts, but memorisation is needed for multiplication facts. As most Number Talks require an element of times table knowledge, this certainly fits in with my experiences. The paper also makes one point that I feel is of paramount importance: if students are relying on a counting strategy to solve basic maths facts (such as counting on fingers for multiplication), then no amount of drilling will help them transfer these facts to long term memory. For students relying on these strategies, drills can become a painful process. Students need to be moved away from inefficient strategies as soon as possible, and the authors suggest that timed drills might be a way to achieve this. Of course, we need to bear in mind Boaler's important point about the dangers of maths anxiety (see Anxiety section for more) - but there are plenty of ways to make this kind of drilling fun and non-threatening, as the Michaela blogs demonstrate.
My favourite quote:
What is required for students to develop automaticity is a particular kind of practice focused on small sets of facts, practiced under limited response times, where the focus is on remembering the answer quickly rather than figuring it out. The introduction of additional new facts should be withheld until students can demonstrate automaticity with all previously introduced facts. Under these circumstances students are successful and enjoy graphing their progress on regular timed tests. Using an efficient method for bringing math facts to automaticity has the added value of freeing up more class time to spend in higher level mathematical thinking.

Research Paper Title: Mental calculation methods used by 11-year-olds in different attainment bands
Author(s): Derek Foxman
My Takeaway:
This is fascinating. A sample of 247 eleven year old children was divided into three bands of attainment as measured independently by their scores on a written test of concepts and skills.They were then given a series of mental arithmetic questions to answer during one-ton-one interviews, and crucially asked to explain how they arrived at their answer. An example of one of the questions children were given is "I buy fish and chips for £1.46. How much change should I get from £5 ?". Their responses were classified as either being Complete (e.g. £5 − £1 − 46p), or Split (e.g. £5 − £1; £1 − 46p). There were three main findings:
1) Complete number methods were far more successful than Split number methods, even more so in the two lower attainment bands than in the top band.
2) For all three questions, Complete number strategy use declined from the Top to the Bottom attainment band, while Split number strategy use increased from Top to Bottom.
3) Complete number strategies were used far more frequently than either Split methods or the Algorithm for working out the in context questions.
I was surprised by these results - for me the Split strategy seems more efficient, and is how I approach the problems. But, then again, I can see how it is prone to error. Relating this to Cognitive Load Theory - if a students does not have facts and processes stored in long term memory, then think of the cognitive demands placed on working memory when trying to process a problem, split it up, work out the individual components, and then put it back together again. Without such facts and procedures in long term memory, the complete strategy was always going to be the most successful.
My favourite quote:
The main significance of these findings is that the two mental computation strategies represent different attitudes towards numbers. The Split strategies suggests that numbers up to 100 are viewed as consisting of tens and units and children using them attempt to deal with these values separately. Such strategies can frequently lead to the sort of errors that occur when using the written standard algorithm. By contrast, Complete number strategies treat numbers as wholes. Furthermore, the calculation steps are sequential so that subtotals are operated on as they occur and do not have to be stored separately in memory.

Research Paper Title: Developing Automaticity in Multiplication Facts: Integrating Strategy Instruction with Timed Practice Drills
Author(s): John Woodward
My Takeaway:
This study seeks to establish whether it is better to use a strategy of timed drills to teach multiplication facts, or a combination of timed drills with activities that promote the use of strategies to teach these facts. The strategies involved were those similar found in Number Talks, for example the multiplication fact 6 x 7 was shown, through discussion, to be equivalent to 6 x 6 + 6. Groups of students were taught using either Drills or and Integrated approach for 4 weeks. They were then given three types of test: Computation, Extended Facts and Approximations, followed by an Attitude Towards Maths survey. Both groups performed equally well on Computations, with the Integrated group performing better on the Extended Facts and Approximations tests. Both groups reported the same level of happiness in the attitudes survey, which may surprise those who fear the Drill and Kill strategy. My only reservation about recommending an integrated approach based upon these findings is that we do not have longitudinal data to see the levels of retention. The students in the Drill approach got through far more computations, and hence there is a chance that their degree of retention will be higher than the Integrated group.This study supports my view that an integrated approach can work - but the key knowledge and facts must be in students' long term memories before attempting to develop and discuss these strategies with numbers, and drilling may be the best way to achieve that. Then, hopefully, you can end up with the best of both worlds.
My favourite quote:
Results from this study indicate that an integrated approach and timed practice drills are comparable in their effectiveness at helping students move toward automaticity in basic facts. If educators were only considering facts as a foundation for traditional algorithm proficiency, either method would probably suffice. Yet, the educationally significant differences between groups found on the extended facts and approximations tests should encourage special educators to consider how strategy instruction can benefit students’ development of number sense.


Real Life Mathskeyboard_arrow_up
Back to Top

"Sir, when will we ever use this in real life?", is a question I never tire of hearing... And for years, I have not had a good reply. My response has often been to try to shoehorn a supposed "real-life" context into my lesson, and believe me "shoehorn" is the correct way of describing it. Did David Beckham really consider the properties of quadratic graphs when lining up a free kick? Advocates of real life problems argue that they have a motivating factor, but as we will see in some of the papers that follow, this must be weighed up against the potential for misinterpretation. More recently, I have given up. I simply tell the students we are doing mathematics for the love, challenge and beauty of doing mathematics, and everyone has gotten on a whole lot better ever since. So, it was with some interest that I sought to see what educational research had to say about the use of real life contexts in mathematics.

Research Paper Title: Relevance as MacGuffin in Mathematics Education
Author(s): Dylan Wiliam
My Takeaway:
In this brilliant paper, Dylan Wiliam looks at the use of "real life" maths problems in the classroom. Wiliam shows examples of where the context adds nothing at all to the question: " Alan drank 5/8 , of his pint of beer. What fraction was left?", as well as examples where the context can actually lead students to get the "wrong" answer: "a disc jockey plays two records. The first lasts 2 5/8 minutes and the second lasts 3 1/4 minutes. Find the total time taken to play both records" - the point being that DJs are likely to talk or play jingles between songs, and hence students may (sensibly) choose to account for this in their answer. There is a fascinating insight into how, when presented with a real-life problem, many girls seek to relate the problem to their existing knowledge, supplying any information that they feel is missing from their own experience, while boys are often content to tackle the problem in isolation from their previous experience. Finally, Wiliam suggests three criteria that should be applied to ensure that the contexts used in problems are useful:
1) Commonality - metaphors and contexts can be really useful to aid understanding, but they must be commonly shared by all students, otherwise they could lead to confusion
2) Match - does the task (and possible interpretations of the task) match the core mathematical activities you want to convey? Will students need to suppress/ignore attributes to engage in the mathematical activity you intended?
3) Range - how far does the model take you along your journey to understanding a topic? Wiliam cites the example of negative numbers: "Envisaging negative numbers as temperatures above or below zero may be very useful for understanding ordering of integers, and may even provide good pictures for adding directed number. However, contriving plausible scenarios where temperatures below zero are subtracted from each other, or are multiplied is rather more difficult!"
My main takeaway from this was to think really carefully about what the context is adding to the question. If the effect is neutral (as in the pint of beer question), then why bother with it - indeed we have learned from the Redundancy Effect in Cognitive Load Theory that pointless information imposes an unnecessary load onto fragile working memory? If it is negative (such as with the DJ, or if it is likely to disadvantage some students), then remove it. But if a metaphor or a framing can be used to impose extra clarity on the problem, without altering its underlying structure, then this would be valid. I will certainly be checking my examples more carefully in the future.
My favourite quote:
In the vast majority of classrooms, relevance is a MacGuffin—a device to motivate learners; to convince them that the activities they are given are somehow of the real world, even though they do not appear to be connected to it. Students know this, and as a result their thinking in mathematics lessons becomes divorced even from their thinking elsewhere in school, let alone the world outside school. Mathematics lessons thus become literally mindless—an activity in which students come to believe that thinking is not helpful.

Research Paper Title: The effect of using real world contexts in post-16 mathematics questions
Author(s): Chris Little and Keith Jones
My Takeaway:
Before we get into the study itself, the authors have a really nice way of summing up the dilemma of using real world contexts: "On the one hand, by making a connection between the abstract world of mathematics and  everyday, or scientific, contexts, we are reinforcing the utility of mathematics as a  language for explaining the patterns and symmetries of the ‘real’ world. On the other  hand, if we manipulate and ‘sanitise’ real-world experiences to enable them to be  modeled by a pre-ordained set of mathematical techniques, then the result can appear  to be artificial and contrived". I like that! Anyway, in this interesting study, alternative versions of the same questions (all on sequences) were presented in explicit, algebraic, word and pattern contexts, and set to  a sample of 594 Year 13 students (aged 17-18) in a one-hour test. The results showed that that setting sequence questions in real-world contexts does indeed add to the overall demand, though a context can on occasions provide ‘mental scaffolding’ to help the solver to use context-specific heuristic strategies. My key takeaway is that we need to think carefully what our aim is. If it is to ensure our students are comfortable with the basics, then there is little point setting questions in a real-world context. Interpreting the question and deciding what strategy to use all impose additional cognitive load to the student's working memory, and as we have seen in the sections on Cognitive Load Theory and Cognitive Science, that may mean that no learning actually takes place. However, the researchers did find that some real-world contexts provided additional support for students in guiding them towards the correct method, so it may not be as clear cut as we would like. Of course, if your aim is assess if students can apply their knowledge and skills to different contexts, then using real-world questions may be a way to do that. I guess we just need to be aware how we interpret students' results in that case - do they struggle because the fundamentals are not in place, because they do not understand what the question is asking, or because they have been misled by the context itself?
My favourite quote:
The potency of algebraic formulae lies in their universality and blindness to individual contexts, and, in resorting to context-specific thinking to solve these questions, students are avoiding the necessity to transfer and abstract from context to mathematical model, which is, arguably, the heuristic strategy intended by the questions.

Research Paper Title: Do realistic contexts and graphical representations always have a beneficial impact on students’ performance?
Author(s): D. De Bock, L. Verschaffel, D. Janssens, W. Van Dooren, K. Claes
My Takeaway:
This is an amazing study which looks at the effect of using video to aid mathematical explanation. One hundred and fifty-two eighth graders (13–14-year olds) and 161 tenth graders (15– 16-year olds) participated. They were give a paper-and-pencil test about the relationships among the lengths, areas and volumes of different types of rectilinear and non-rectilinear figures. The problems were administered in or out of an authentic context and setting, and either with or without an integrated drawing instruction, leading to four combinations of groupings. The "authentic context" involved students watching clips of a screen version of ‘Gulliver’s Travels’, a world in which all lengths are 12 times as small as in our (and Gulliver’s) world. In the drawing instruction groups, students were provided with a drawing of the (geometrical) object introduced in the problem, and they were asked to complete the drawing by making a reduced copy using the given scale factor. How did the groups perform on the test? Students who watched Gulliver’s Travels and who received the video-related items performed significantly worse than the students from the other groups. Surprisingly, students who had to make a drawing performed significantly worse than students from the non-drawing groups, although the authors admit this is probably due to the nature of the linear scale factor task itself. To explain the video results, one possibility suggested really resonated with me: students perceive video as a less difficult medium than written materials, and therefore are inclined to invest less mental effort in working with information transmitted by this easy medium as compared to media that are perceived as difficult. As we have seen in the Cognitive Science and Memory sections, students remember what they think about, and sometimes thinking needs to be difficult. When watching a video (or indeed any other form of novel media we might use in a lesson, such as a website, app, or music) are students thinking about the task itself or the form of media? Any increase in engagement and motivation must be weighed up against this.
My favourite quote:
Some of the qualitative findings indicate that the reason why these two new forms of help did not yield the expected positive effect, was that they conflicted with students’ implicit norms, expectations and beliefs about doing mathematics, especially about their appreciation of formal and informal strategies and of drawings as a valuable modelling tool. From that point of view, classroom interventions that are only partial and instantaneous, and that are unable to influence or alter these more fundamental attitudes and beliefs, have little chance of success. Most likely, only a long-term classroom intervention, not only acting upon students’ deep conceptual understanding of proportional reasoning in a modelling context, but also taking into account the social, cultural and emotional context for learning, can produce a positive effect in defeating the illusion of linearity.

Research Paper Title: The role of context in linear equation questions: utility or futility?
Author(s): Chris Little
My Takeaway:
Four linear equations questions were presented, each set in a context. Two of these questions were : 1) In 18 years time, Halley will be five times as old as he was 2 years ago. How old is he now?  4) The largest angle of a triangle is six times as big as the smallest. The third angle is 75°. Find the size of the three angles. The author argues that these questions may be regarded as ‘applications’ of linear equations, in the sense that they involve formulating and then solving them. However, they have no practical utility value - none of the results provide significant information about the context. So, what is the point in the contexts? They are certainly not motivating. But they are unlikely to introduce any of the misinterpretations highlighted in previous papers in this section. The author makes a really interesting point: whilst the underlying algebraic structure of all four problems is the same, the contexts may actually encourage students to try different methods to solve them - trial and improvement, for example. I have seen this a lot with my students - often the context prevents students from spotting the topic that the question intends to test, and as a result the students end up trying something different. My takeaway here is similar to the paper on post-16 students - think about what we want our students to achieve. Do we want them to practise solving equations? If so, then maybe it is best to cut out the context and just have the equations. If we want them to practice recognising when it might be appropriate to formulate a question in terms of a linear equation, then we have seen in the sections on Cognitive Science and Cognitive Load Theory that students will only be in a position to do that - and crucially, to learn from it - when the basic knowledge of how to solve linear equations is secure in long term memory. Otherwise, the context may inhibit their learning.
My favourite quote:
Some researchers into word problems (Greer, 1997) have advocated introducing more elements of realism into classroom tasks, for example by  adding irrelevant information, which then has to be discounted by the solver.  However, expecting students to engage in genuine mathematical modelling activity  before they serve an apprenticeship in formulating algebraic equations, and learning  abstract, analytical methods for solving them, is perhaps itself unrealistic. Many  students find the process of translating real-life numerical concepts into algebraic variables demanding enough, without being deflected by realistic ‘noise’.

Research Paper Title: Intellectual Need and Problem-Free Activity in the Mathematics Classroom
Author(s): Evan Fuller, Jeffrey M. Rabin, Guershon Harel
My Takeaway:
I discovered this interesting paper following my interview with Dan Meyer on my podcast. The authors argue that intellectual need is necessary for significant learning to occur. Among other concepts, it introduces the need for computation. This is defined as the need to find more efficient computational methods, such as one might need to extend computations to larger numbers in a reasonable “running time.” The key here is that in order for the need for computation to be met, students need to experience the “longcut” before they learn the shortcut, for only then will they appreciate the power of that shortcut. Otherwise the techniques we teach seem like just another trick in the endless series of tricks students call maths class. This forms the basis of Dan Meyer's "Headach and Aspirin" series, where he asks the question: If Math Is The Aspirin, Then How Do You Create The Headache?. Dan argues that real world applications of many maths skills we teach in school are a lie. So if our theory is “maths is interesting only it’s real world,” then we will struggle to find interest in many of the things we teach. Instead, we should ask ourselves, “Why did mathematicians think this skill was worth even a little bit of our time? If the ability to factorise quadratic expressions is aspirin for a mathematician, then how do we create the headache?”. This approach has been incredibly influential in my teaching, and has enabled me to offer approaches into topics that I would never have thought of. I would strongly recommend checking out Dan's factorising quadratic expressions blog post to see how the headache and aspirin approach works. I will never introduce factorising quadratics in the same way again.
My favourite quote:
The need for computation is not a student’s psychological motivation to solve drill exercises on algorithms, but her intellectual recognition that realistic and compelling problems require the development of efficient computational methods for their solution


Improving Teachingkeyboard_arrow_up
Back to Top

To conclude this page of research, I want to take a look at developing the skill of being a teacher. So far we have seen recommendations for improving the delivery of material to students, and plenty of ways to help students encode, store, retrieve and apply knowledge. But what about improving our own teaching, and that of the teachers we mentor and work with?
1) Improving our own teaching. I am always looking to improve as a teacher, but there is a problem (and it is the same problem that makes research carried out in classrooms so difficult). There are just so many factors at play. Say I decide to try something new with my Year 7s - maybe I have come up with a new way of introducing algebra. How do I know how successful it has been? Sure, I can compare their performance in an algebra test with last Year's class, but that was a different class, with different students, taught in different circumstances and (lest we forgot) that given the distinction between learning and performance identified in the Memory section, how can I even judge if they have really learned algebra any better? So, if it is difficult to evaluate our own teaching in an effort to improve, can others help us through observations? Having read research on the matter, I believe they can. But - and this is a big but - only if we move away from the high-stakes, once-a-year observation that seems to be a key part of the performance management cycle these days. I believe supportive observations, with clearly defined and measurable goals in mind, can help develop teachers of all experiences.
2) Helping novice teachers develop. I have been extremely fortunate to work with many inexperienced teachers over the years in a bid to help them improve and develop. It is only now, having read the research cited on this page about skill development and the difference between how experts and novices think, that I see what a complex task this is. Whilst it is clear that some people have a natural ability to teach - just as some students have a natural ability to do mathematics - good teachers can be made. I believe we need to follow the principles of Explicit Instruction, as opposed to simply hoping inexperienced teachers will magically discover their way to becoming experts, which implies that novice teachers may require more support than they currently receive
.
Research Paper Title: The Development of Expertise in Pedagogy
Author(s): David C. Berliner
My Takeaway:
This is a truly wonderful paper that has lots to say on the transition from novice to expert in any field, but with particular relevance to teaching. The authors identify key areas where novices and experts differ, providing examples and supporting research for each:
1) Interpreting classroom phenomena
2) Discerning the importance of events
3) Using routines
4) Predicting classroom phenomena
5) Judging typical and atypical events
6) Evaluating performance
A few things struck me about this. Firstly, how many of these areas can only be improved by experience? Almost all of them, I think. And how many could I describe in great detail? Not many at all. For example, if a student teacher watches me teach and afterwards asks me why I said what I said to a group of students, or didn't say something to another, or ignored a certain situation but acted on another, or how I developed my classroom routines, or knew to adjust my explanation, or deviate from my lesson plan, or pause before moving on, I will find it hard to explain. I cannot nail down the numerous visual and audio cues to cause me to act the way I do, it is just instinct and experience. Like any (so-called!) "expert", I am on autopilot dealing with many things in the classroom, which enables my limited working memory to focus on more complex pedagogical issues. So, how do we help novice teachers develop these skills? Well, the authors have a few recommendations, but two of which stood out to me more than the rest:
1) If expert teachers have so many of these crucial routines on autopilot, and because the very fact they have been automatised makes them hard to articulate to novices, is this an argument for mentoring of novice teachers to be done by "competent" teachers as opposed to "experts"? I guess this is a similar to the argument that great mathematicians sometimes do not make the best teachers - they simply cannot relate to the difficulties students have in grasping basic concepts. Is it better to have a mentor who can remember what it was like to be a novice - can relate to the struggles, can remember how they coped with them - instead of an expert who flies through lessons on autopilot and may struggle to give novice teachers to explicit guidance they need?
2) The novices' relative inexperience in a complex environment allows a good case to be made for the importance of teaching them standard lesson forms and scripts. Now, this is controversial! Two of my podcast guests, Dani Quinn and Greg Ashman have both advocated centrally planned lessons, with the justification that these are lessons that have been planned by experienced teachers and are known to work. Novice teachers may benefit from following these "scripts" in class, enabling them to focus on other areas of teaching (e.g. behaviour, questioning, interactions with students) as they develop. This may also free up some of those hours that are spent searching for PowerPoints and worksheets, and allow them instead to spend their time going over the lesson plan in detail, rehearsing questions and responses, and ensuring they are familiar with the mathematical content. A key point here is that this is not the same as simply giving a novice teacher a PowerPoint on, say, straight-line graphs that worked well for you. It is not just the resource that is important, but the delivery, pace, questions, possible misconceptions, and so on. Essentially this is the same argument for Explicit Instruction versus discovery learning for novice students - in early skill acquisition, novice teachers are likely to benefit from more guidance from experts rather than less.
My favourite quote:
I am suggesting that our extensive knowledge base about teaching and teachers be thought of as more or less appropriate to people in different stages of their development. I am also suggesting that pre-service education may not be the most appropriate place to teach some things, and therefore we have to extend our programs of teacher education for some time after our students have entered practice. I am suggesting as well that the forms of evaluation for experienced and beginning teachers may have to differ. And I am suggesting that experts, revered as they may be, may not always make the best teachers of novices. I am arguing that the development of competence out of ignorance and expertise out of competence may take a long time in a profession as complicated as teaching. We may be unable to shorten the trip very much because extensive experience is fundamental to development, but we certainly ought to help nurture those willing to undertake the journey by providing training and evaluation appropriate for their level of development.

Research Paper Title: Practice with Purpose: The Emerging Science of Teacher Expertise
Author(s): Deans for Impact
My Takeaway:
In the Explicit Instruction section we looked at the fascinating concept of deliberate practice and how it could be applied to student learning. Here we turn our attention to using Deliberate Practice to improve teaching. Five key principles are identified and discussed with respect to teaching:
1) Deliberate practice requires presenting challenges that push novices just beyond their current abilities
2) Deliberate practice requires setting goals that are well-defined, specific, and measurable
3) Deliberate practice requires a significant level of focus; the practice involves conscious effort on the part of the novice in order to improve.
4) Deliberate practice requires providing high-quality feedback to the novice and adjustment by the novice in response to that feedback.
5) Deliberate practice both produces and relies on mental models and mental representations to guide decisions.These models allow practitioners to self-monitor performance to improve their performance.
Each of these are worthy of discussion, but a couple in particular stood out to me. The setting of specific goals was one. I remember early on in my career when I was being observed, I would rarely have specific goals in mind. I would just want to teach a good lesson. I would have been much better focusing on specific aspects, such as trying to give only task-focused feedback, or reducing the number of controlling questions I asked. Where possible, these goals should be measurable, so you have something concrete and objective to reflect upon. The second point is directly related - feedback. We have seen how important feedback can be for student learning, and how it can have both a positive and negative impact. The same is true for teacher development. We need to know if what we are doing is successful, but that is incredibly difficult when (at best) all we can observe is the performance of our students and, as we have seen in the Memory section, performance is a poor indicator of learning. That is where a colleague, observing in a supporting manner, complete with a predefined set of measurable goals, can help. And this cannot be confined to a once-a-year experience. Regular feedback is the key to deliberate practice, making tweaks where needed and seeing the outcome. I'll be honest - in my opinion developing teaching does not lend itself as well to the principles of deliberate practice as, say, learning to add fractions, but there are certainly key elements that can be used to great effect.
My favourite quote:
The principles of deliberate practice have the promise to improve the quality of teacher education. There will inevitably be challenges with this work: for teacher-educators learning new techniques; for institutions that need to change incentive structures in order to encourage faculty to own collectively the success of every teacher-candidate; and for teacher-candidates and novice teachers who will be pushed beyond their comfort zones. This work will not be easy, but we believe that it is both possible and necessary if we are to advance the field of teacher preparation and prepare effective teachers to serve every student.

Research Paper Title: Does Teaching Experience Increase Teacher Effectiveness?
Author(s): Tara Kini and Anne Podolsky
My Takeaway:
There are four key findings from this paper, three of which once again extol the benefit of experience, and on which was a game-changer for me:
1) Teaching experience is associated with increased student achievement gains throughout a teacher’s career
2) As teachers gain experience, their students are more likely to do better on other measures of success beyond test scores, such as school attendance
3 More experienced teachers confer benefits to their colleagues and to the school as a whole, as well as to their own students. Again, experience counts. Although the authors do point out that some highly experienced teachers are not particularly effective or have retired on the job, and some novice teachers are dynamic and effective. This is related to the principles of deliberate practice discussed in the paper above - it can be quite easy to coast along in teaching, relying on your experience to get you through. Likewise, as teachers progress in their careers, they often have other responsibilities within schools that take their time away from both planning and teaching. It can be difficult to improve past a certain point, but if a teacher wants to then the principles of deliberate practice can help.
4) Teachers make greater gains in their effectiveness when they teach in a supportive and collegial working environment, or accumulate experience in the same grade level, subject, or district. This is the big one for me. I have always assumed it is good to teach as wide a range of abilities and ages of  students as possible - that way you build up your set of teaching tools that can be used in any situation. But findings detailed here suggest it is best to specialise - in other words focus in on a particular ability of student (eg top-set or bottom-set) or age-range (e.g. Year 11). The more I think about this, the more it makes sense. In every school I have worked in there have been specialists - the "GCSE C/D borderline specialist", the "top-set specialist", or the "Statistics specialist" at A Level. Indeed, if I am honest, I find it easier stimulating and extending top-sets than bottom sets, but each year I take a few bottom-sets as I assumed it was the best thing to do for my teaching as a whole. But maybe it is not. Maybe specialising in one specific area of maths teaching allows teachers to develop and refine skills tailor-made to getting the most out of those students. Of course, there are issues with this. How do you know what your area of expertise is if you haven't taught lots of different classes? Sometimes "competition" between teachers teaching parallel classes can be a good thing in terms of stimulating new ideas and giving each other an incentive to strive to improve. Then there are political concerns - who gets the top sets? But I do think there is an argument for specialising, and possibly a strong argument for specialising in your first year of teaching so you can hone your craft on a narrower range of classes and students.
My favourite quote:
The common refrain that teaching experience does not matter after the first few years in the classroom is no longer supported by the preponderance of the research. Based on an extensive research base, it is clear that teachers’ effectiveness rises sharply in the first few years of their careers, and this upward trajectory continues well into the second and often third decade of teaching. The overwhelming majority of the 30 studies reviewed here (93 percent)—and 100 percent of the 18 studies using the teacher fixed effects methods—reach this conclusion. The effects of teaching experience on student achievement are significant, and the compounded positive effect of having a series of accomplished, experienced teachers for several years in a row offers the opportunity to reduce or close the achievement gap for low-income students and students of color.123 Given this knowledge, policymakers should direct renewed attention to developing a teacher workforce composed of high-ability teachers who enjoy long careers in supportive and collegial schools.

Research Paper Title: What makes great teaching? Review of the underpinning research
Author(s): Robert Coe, Cesare Aloisi, Steve Higgins and Lee Elliot Major
My Takeaway:
This is the second appearance of this paper, having been previously discussed in the Explicit Instruction section. I reference it again here simply because one of the finding that good pedagogical content knowledge has a strong impact on student achievement. Knowledge of the mathematical content itself is directly under the control of a teacher. I remember when I first taught the Decision 1 A Level module for the first time - I literally had no idea what I was doing. I had neither studied nor seen any of the material before in my life. And so I did the only thing I could do - I studied and studied, reading the notes in the textbook, completing all the exercises, and doing all the past papers (incidentally, I found doing past papers without notes by far the most useful way to learn the material, possibly due to the Testing Effect identified in the Memory section). How did my teaching of the unit go? Pretty average, to be honest. Because one part of subject knowledge is not directly under a teacher's control - the pedagogical side. I did not really know how to communicate the ideas clearly in a way that students understood. Why was this? Well, I simply lacked the experience. All I had to go on to identify the potential stumbling blocks and misconceptions where my own experiences of the material, and these often proved different to those of my students. It was only having taught the module twice through that I began to get a sense of the pace I needed to deliver the material, the misconceptions I needed to anticipate and address, and the alternative explanations I needed to offer up. So, there was no way I could have taught that module without good knowledge of the mathematics, but that was not sufficient to teach it well. There is no fast-track to gaining the pedagogical experience needed, but I could have made life easier for myself by watching more experienced teachers in action.
My favourite quote:
The most effective teachers have deep knowledge of the subjects they teach, and when teachers’ knowledge falls below a certain level it is a significant impediment to students’ learning. As well as a strong understanding of the material being taught, teachers must also understand the ways students think about the content, be able to evaluate the thinking behind students’ own methods, and identify students’ common misconceptions.