Worked Examples: Making the most of them
It was my reading of Cognitive Load Theory that first prompted my appreciation of the power of worked examples. Understanding that studying worked examples could be more effective for learning than solving problems blew my mind and prompted me to read all I could about how to make the worked examples I used in class as effective as possible. This section provides a summary of what I found.Research Paper Title: The Contributions of Studying Examples and Solving Problems to Skill Acquisition
Author(s): J. Gregory Trafton and Brian J. Reiser
My Takeaway:
The first paper in this section has two major findings which, whilst they may seem obvious, are both of crucial importance. The researchers conducted experiments designed to answer two key questions they had about the most effective way to study from worked examples.
Does separating source examples from target problems hamper learning?
The researchers conducted an experiment comparing participants who solved a target problem immediately after the source example (Alternating Example) versus those who studied a block of source examples followed by a block of solving target problems (Blocked Example). Subjects who solved problems interleaved with examples took less time on the target problems than subjects who studied a block of source examples and a block of target problems. Crucially, participants in the Alternating Example condition also submitted more accurate solutions than subjects receiving blocked examples.
Is solving sources better than studying examples if the examples are not accessible during subsequent problem solving?
This is a key question. There is a danger that we can get too caught up in the power of examples, and think that students do not need to solve any problems at all. However, the researchers found that participants who attempted to solve problems performed better than those who merely studied worked examples. As we shall see in the sections on Testing, it is this retrieval process induced when answering a question that is so important to learning, and if students are never compelled to access their memories of those examples, then they will never benefit from, what we will come to call the Testing Effect. Hence, we can conclude that subsequent problem solving appears to be required to derive the full benet of studying examples
My favourite quote:
In summary, studying examples is clearly a very effective method to improve learning. In order for an
example to be most effective, however, the knowledge gained from the example must be applied to solving a new problem. The most efficient way to present material to acquire a skill is to present an example, and then a similar problem immediately following. We hypothesize that this presentation method allows subjects to construct rules that are general enough to work for both the example and the rule. Although the extra practice solving sources may speed target problem solving, apparently more effective problem solving rules are formed when target problem solving can be guided by an accessible source example.
Research Paper Title: Learning from Examples: Instructional Principles from the Worked Examples Research
Author(s): Robert K. Atkinson , Sharon J. Derry , Alexander Renkl , Donald Wortham
My Takeaway:
There are so many fascinating concepts discussed throughout this paper extolling the virtues of worked examples, and when they are most effective. Concepts such as the split-attention effect, and variability of problem types are also discussed, which we have covered in the Cognitive Load Theory section. The quote I have chosen for this paper summarises the main findings nicely, and serves as an excellent set of guidelines for the use of worked examples during lessons. I am going to focus on one that has directly changed the way I teach, and which is directly related to the finding from the paper above: Example-Problem Pairs. I used to do a load of worked examples at the start of the lesson, and then give the students a set of problems to work on for the remainder of the lesson. The problem (and it seems so obvious now) was that by the time I had done the 3rd worked example, students had forgotten the first as they had not had the opportunity to practice it for themselves. Hence, they would start working on the questions, get stuck, and I would need to go through it all again. This paper recommends interleaving worked examples with related questions for students to solve alone. When doing a worked example in class, I now split my board in two, having the worked example on the left, and a mathematically similar example for my students to try themselves immediately afterwards on the right. It sounds so simple, but the positive effects have been quite startling. Greg Ashman discussed this strategy when describing how he plans his lessons when I interviewed him for my podcast, and he has written a blog post all about his use of Example-Problem Pairs here.
My favourite quote:
First, transfer is enhanced when there are at least two examples presented for each type of problem taught. Second, varying problem sub-types within an instructional sequence is beneficial, but only if that lesson is designed using worked examples or another format that minimizes cognitive load. Third, lessons involving multiple problem types should be written so that each problem type is represented by examples with a finite set of different cover stories and that this same set of cover stories should be used across the various problem types. Finally, lessons that pair each worked example with a practice problem and intersperse examples throughout practice will produce better outcomes than lessons in which a blocked series of examples is followed by a blocked series of practice problems.
Research Paper Title: Design-Based Research Within the Constraints of Practice: AlgebraByExample
Author(s): Julie L. Booth, Laura A. Cooper, M. Suzanne Donovan, Alexandra Huyghe, Kenneth R. Koedinger & E. Juliana Paré-Blagoev
My Takeaway:
We have seen the benefits of learning by worked example in our study of Cognitive Load Theory, in particular with the example-problem pairs approach. We have also seen the benefits of students explaining their thinking in our look at Student Self-Explanations. So, what happens when you combine these two powerful findings together? This paper seeks to demonstrate. The study in question came about from a challenge to identify an approach to narrowing the minority student achievement gap in Algebra 1 without isolating minority students for intervention. They attempted to do this by designing and testing 42 Algebra 1 assignments with interleaved worked examples that targeted common misconceptions and errors. The worked examples contained three parts:
1) the worked example
2) a section to reflect on what certain aspects of the worked meant, and how and why they were carried out that way
3) a related problem to complete
Hence, it is the middle section - the opportunity for self-explanation - that differentiates this approach from the example-problem pair approach. Notice also how the students are compelled to self-explain, which seems important given the findings from the Renkl paper in the Self-Explanations section that the majortiy of studetns do not spontaneously use self-explanation strategies. The results were impressive. The approach provides a boost in performance, with the greatest impact on students at the lower end of the performance distribution. On the researcher-designed assessment of conceptual understanding, treatment students in the lower half of the performance distribution outscored comparable control students by approximately 10 percentage points. Treatment students overall scored 7 percentage points higher on a test composed entirely of released items from the state standardized tests, and 5 percentage points higher on the conceptual posttest. Procedural posttest scores were also 4 percentage points higher in the treatment group, even though control students had twice the practice solving problems on the assignments. As a result of this, together with the related research into the power of self-explanations, I have added that third element to my example-problem pair approach.
My favourite quote:
Of equal significance, AlgebraByExample achieved these gains with an intervention that meets all the constraints imposed by the districts: it targets all students, it can be used with any existing Algebra 1 curriculum, and it is an asset that teachers can use with minimum disruption to their practice. Moreover, study teachers reported that students using AlgebraByExample required less teacher support to complete assignments than those using control assignments, and reported rethinking their own practice in response to students’ positive experiences with worked examples.
Research Paper Title: Using example problems to improve student learning in algebra: Differentiating between correct and incorrect examples
Author(s): Julie L. Booth, Karin E. Lange, Kenneth R. Koedinger, Kristie J. Newton
My Takeaway:
This study takes worked examples with student self-explanations to the next level by considering the inclusion of incorrect examples. This is something I have always been wary of. I can see the benefits of non-examples when it comes to understanding factual definitions in maths (such as “what is a polygon?”), but is there a danger that exposing students to non-procedures (i.e. incorrect worked examples) will lead to them developing misconceptions? The authors cite findings from previous research that outline two reasons why the inclusion of incorrect examples might be of benefit to learning. First, they can help students to recognise and accept when they have chosen incorrect procedures, leading to improved procedural knowledge over practice alone or correct examples plus practice. Second, and perhaps more important, it can draw students’ attention to the particular features in a problem that make the procedure inappropriate. Therefore a combination of correct and incorrect examples is beneficial because the incorrect examples help to weaken faulty knowledge and force students to attend to critical problem features (which helps them not only to detect and correct errors, but also to consider correct concepts), while the correct examples provide support for constructing correct concepts and procedures, beyond that embedded in traditional instruction. In their study, the authors conducted two experiments, both focussed on the Algebra 1 course take in high-schools in the US. They sought to answer two questions:
1) Do worked examples with self-explanation improve student learning in Algebra when combined with scaffolded practice solving problems? The results were a resounding “yes”.
2) Are there differential effects on learning when students explain correct examples, incorrect examples, or a combination thereof? Results indicated that students performed best after explaining incorrect examples; in particular, students in the Combined condition gained more knowledge than those in the Correct only condition about the conceptual features in the equation, while students who studied only incorrect examples displayed improved encoding of conceptual features in the equations compared with those who only received correct examples.
The authors have a really nice way of summarising this key finding: “This finding is especially important to
note because when examples are used in classrooms and in textbooks, they are most frequently correctly solved examples. In fact, in our experience, teachers generally seem uncomfortable with the idea of presenting incorrect examples, as they are concerned their students would be confused by them and/or would adopt the demonstrated incorrect strategies for solving problems. Our results strongly suggest that this is not the case, and that students should work with incorrect examples as part of their classroom activities.”
My favourite quote:
Our results do not suggest, however, that students can learn solely from explaining incorrect examples. It is important to note that all students saw correct examples, regardless of condition, not only because they are regularly included in textbooks and classroom instruction, but because the correctly completed problems the students produced with the help of the Cognitive Tutor could also be considered correct examples of sorts. We maintain that students clearly need support for building correct knowledge, however, if that support is coming from another source (e.g., guided practice with feedback), spending additional time on correct examples may not be as important as exposing students to incorrect examples.
Research Paper Title: Learning from Worked Examples: What happens if mistakes are included?
Author(s): Cornelia S Grosse and Alexander Renkl
My Takeaway:
I include this paper as a word of caution before getting too carried away with exposing students to incorrect worked examples. The authors found that learning from worked examples where errors are included can enhance learning and transfer, but only if students have good prior knowledge of the topic. This makes perfect sense. Firstly, if students do not understand the topic, then how are they to spot the mistakes? Secondly, in the context of Cognitive Load Theory, if their understanding is not secure, then their working memories are likely to become overloaded whilst searching for the right and wrong answers simultaneously. This finding is supported by a paper from Große, C. S., & Renkl, A (behind a pay-wall) which found that relatively novice learners cannot benefit from incorrect examples when they are expected to locate and identify the error in the example themselves. Again, It makes sense that novice students would have difficulty with this component, given that they likely make many of the mistakes themselves and may not recognize them as incorrect. So what are we to make of this? It seems clear that students need a certain amount of knowledge of a topic in order to benefit from exposure to mistakes in worked examples. If they do not know what is right, how can they know and explain what is wrong? Hence, I would be leaning towards correct worked examples combined with student explanation in initial skill acquisition, moving onto clearly labelled incorrect worked examples combined with student explanation once students begin to get familiar with the concepts.
My favourite quote:
Learning with incorrect examples poses challenging demands on the learners. They have to represent not only the correct solution in their working memory, but also the incorrect step with an explanation why it is wrong. Learners with low prior knowledge who cannot form larger chunks for information coding can easily be overtaxed.
Research Paper Title: Structuring the Transition From Example Study to Problem Solving in Cognitive Skill Acquisition: A Cognitive Load Perspective
Author(s): Alexander Renkl and Robert K. Atkinson
My Takeaway:
This paper provides a challenge to the Example-Problem Pair approach heralded by the papers above. The authors propose that instead of providing students with a complete worked example, followed by a problem for them to solve, followed by a complete worked example, followed by problem to solve, etc (i.e. the Example-Problem Pair approach), instead a fading procedure is more effective. This involves providing a complete worked example, and then following this up with an almost complete worked example but with one step removed that students need to complete. There are obvious parallels to be drawn here with the Completion Effect noted in the Cognitive Load Theory section. The researchers two things that are of particular relevance to this section:
1) The fading procedure produced reliable effects on near-transfer items but not on far-transfer items. Near-transfer problems have the same deep structure but different surface structures. In other words, the strategy to solve them is exactly the same, but the context is different. Far-transfer problems have both a different context and a different deep structure. In other words near transfer is the acquisition of relatively simple rules, whereas far transfer is more a measure of understanding.
2) It was more advantageous to fade out worked- out solution steps using a backward approach by omitting the last solution steps first instead of omitting the initial solution steps first (i.e., a forward approach).
So, we have an alternative to the example-problem pair approach, which has been shown to outperform its rival, both in this study and the one that follows. This begs the obvious question: when do we use each one? For me, it depends on the complexity of the problems and the prior knowledge of the class. Relatively simple problems probably lend themselves better to an example-problem approach. Likewise, with students with higher prior knowledge, or students who have seen the topic before (i.e. in a revision lesson), I would favour the example-problem approach. However, for more complex problems or with students who are struggling, the fading approach seems very appropriate.
My favourite quote:
First, a complete example is presented (model). Second, an example is given in which one single solution step is omitted (coached problem solving). Then, the number of blanks is increased step by step until just the problem formulation is left, that is, a to-be-solved problem (independent problem solving). In this way, a smooth transition from modeling (complete example) over coached problem solving (incomplete example) to independent problem solving is implemented.
Research Paper Title: Transitioning From Studying Examples to Solving Problems: Effects of Self-Explanation Prompts and Fading Worked-Out Steps
Author(s): Robert K. Atkinson, Alexander Renkl and Mary Margaret Merrill
My Takeaway:
This paper aims to plug the one remaining gap in the fading procedure for worked examples proposed in the paper above - how to improve performance on far-transfer problems? Well, just like we saw in the Booth papers above, we can improve the effectiveness of worked examples by prompting student explanations. Here the researchers designed experiments which combined fading with the introduction of prompts designed to encourage learners to identify the underlying principle illustrated in each worked-out solution step. They set up four conditions: Example-Problem Pair (EP), Example-Problem Pair Plus (EP+) which contained the self-explanation prompts, Backwards Fading (BF) and Backwards Fading Plus (BF+). Crucially, the experiment was designed in a way so that participants had to complete the exact same number of solution steps in each condition - it was just in the EP approach they all came together, whereas in the BF model they were introduced more gradually. The key findings were as follows:
1) the BF condition was associated with a higher solution rate of near-transfer problems than EP
2) a simple prompting procedure can substantially foster both near and far transfer. Hence, the acquisition not only of relatively simple rules (i.e., near transfer) but also of understanding (i.e., far transfer) can be fostered by this instructional procedure. It is also notable that the advantage of prompting could be achieved without significantly increasing learning time
3) there was no evidence of an interaction between the use of fading and the use of self-explanation prompts on any of the measures. This may be regarded as a positive finding from an educational point of view because both instructional means produced at least medium effects on learning outcomes and were combined without causing any decrement in performance.
For me, there are two key takeaways here. Firstly, the fading procedure must be taken seriously. Secondly, prompting students to self-explain is effective no matter how the worked examples are presented, and hence is something of a no-brainer when it comes to teaching and modelling.
My favourite quote:
One may ask whether it is practical to use the instructional procedures analyzed in this article for teaching skills in well- structured domains. Overall, the use of prompts that encourage the learners to figure out the principle that underlies a certain solution step can be recommended for several reasons, including the fol- lowing: (a) it produces medium to high effects on transfer performance, (b) these effects are consistent across different age levels (university and high school), (c) it does not interfere with fading, (d) it is very easy to implement (even without the help of computer technology), and (e) it requires no additional instructional time.