Marking and Feedback
Whether we like it or not, most of us have to mark and give feedback to students. For many (including me), it is one of the worst parts of being a teacher. The best advice I ever heard on feedback was from Dylan Wiliam, who said "the only good feedback is that which is acted on". You could write the best feedback in the world, but if the your students neither have the time nor the inclination to do it, or if they simply do not understand what you are asking them to do, then it is an absolute waste of time. A second guiding principle that I now try to stick to is "feedback should make students think". All too often my feedback has been so carefully constructed, and full of so many hints, that my students simply have to say "yeah, yeah, I get it now, sir", make a few changes, and then move on, without having learned a single thing, and having cost me a good few hours of my life. For years I have been doing far more work than my students when it comes to marking and feedback. Here we look at what the evidence has to say about marking and feedback, both in terms of the type and the immediacy, when feedback can have a negative effect, and when less is definitely more!Please note: papers concerning marking and feedback, but that are more explicitly to do with praise can be found in the Motivation and Praise section.
Research Paper Title: A marked improvement. A review of the evidence on written marking
Author(s): Education Endowment Foundation and Oxford University
My Takeaway:
Lots of practical, easy to implement strategies to improve the effectiveness of marking, each one evaluated against available research. My favourites include:
1) Dealing with careless mistakes and misconceptions differently when marking;
2) Making feedback as specific and actionable as possible so the students actually understand it; "acknowledgement" marking (i.e. a tick to say you have seen the work) is a waste of time, and hence a good rule is mark less but mark better; specific time must be dedicated to students to respond to marking.
3) And then the big one for me: awarding grades/levels for each piece of work means students focus upon these marks at the expense on the formative comments the teacher has written, and probably took ages!
My favourite quote:
It also appears worthwhile to caution against elements of dialogic or triple impact marking that do not follow the wider principles of effective marking that are underpinned by relatively stronger evidence summarised elsewhere in this review. For example, there is no strong evidence that ‘acknowledgment’ steps in either dialogic or triple impact marking will promote learning.
Research Paper Title: A marked improvement. A review of the evidence on written marking
Author(s): Education Endowment Foundation and Oxford University
My Takeaway:
Lots of practical, easy to implement strategies to improve the effectiveness of marking, each one evaluated against available research. My favourites include:
1) Dealing with careless mistakes and misconceptions differently when marking;
2) Making feedback as specific and actionable as possible so the students actually understand it; "acknowledgement" marking (i.e. a tick to say you have seen the work) is a waste of time, and hence a good rule is mark less but mark better; specific time must be dedicated to students to respond to marking.
3) And then the big one for me: awarding grades/levels for each piece of work means students focus upon these marks at the expense on the formative comments the teacher has written, and probably took ages!
My favourite quote:
It also appears worthwhile to caution against elements of dialogic or triple impact marking that do not follow the wider principles of effective marking that are underpinned by relatively stronger evidence summarised elsewhere in this review. For example, there is no strong evidence that ‘acknowledgment’ steps in either dialogic or triple impact marking will promote learning.
Research Paper Title: The Effects of Feedback Interventions on Performance: A Historical Review, a Meta-Analysis, and a Preliminary Feedback Intervention Theory
Author(s): Avraham N. Kluger and Angelo DeNisi
My Takeaway:
When I interviewed Dylan Wiliam, he cited this paper as the most surprising piece of research he had ever encountered. The authors found over 3,000 research studies published between 1905 and 1995, but found that only 131 of the studies were well-enough designed for their results to be taken seriously. The 131 studies reported 607 effect sizes, which showed that, on average, feedback did increase achievement. But—in what could possibly be one of the most counter-intuitive results in all of psychology—231 of the 607 reported effect sizes were negative. In almost two out of every five studies, feedback lowered performance. What are we to take from this? Well, the authors suggest that the further feedback moves away from the task itself and towards the individual student, the less effective it is - even going as far as to have a negative effect. If teachers are going to dedicate a significant proportion of their time to giving feedback, we must make it task-focused. I will leave the final words to Dylan himself: If there’s a single principle teachers need to digest about classroom feedback, it’s this: The only thing that matters is what students do with it. No matter how well the feedback is designed, if students do not use the feedback to move their own learning forward, it’s a waste of time. We can debate about whether feedback should be descriptive or evaluative, but it is absolutely essential that feedback is productive.
My favourite quote:
The central assumption of Feedback Intervention Theory is that Feedback Interventions change the locus of attention among 3 general and hierarchically organized levels of control: task learning, task motivation, and meta-tasks (including self-related) processes. The results suggest that feedback intervention effectiveness decreases as attention moves up the hierarchy closer to the self and away from the task.
Research Paper Title: The Power of Feedback
Author(s): John Hattie and Helen Timperley
My Takeaway:
The first line of this paper hooked me in: "Feedback is one of the most powerful influences on learning and achievement, but this impact can be either positive or negative." This was a revelation to me. Surely, if I was spending so much time doing something, it has to be beneficial for my students? This is a fascinating, comprehensive paper that addresses all the major issues surrounding feedback. I would strongly recommend reading the part on page 98 about the issues surrounding positive and negative feedback. This really made me realise that some students benefit from praise, for some it does them more harm than good, and some students need a metaphorical kick up the arse. Judging this is difficult, but ultimately comes down to a teacher's relationship with and knowledge of their students. My key takeaway, however, is the fact that feedback is likely to be ineffective if the student simply does not have the relevant knowledge to solve the problem in the first place. I know that sounds obvious, but it wasn't to me. Feedback can only build upon knowledge. You can give all the task-focused prompts in the world, but if the students lack the knowledge to do the work, then feedback is likely to be pretty useless. I have done this myself, literally spending hours making corrections, carefully writing prompts, inventing follow-up questions for students to do, and then one of two things happens - either the students still cannot do it, or I have ended up doing that vast majority of the work, so they can make corrections without thinking. In that instance, I believe it is far better to simply re-teach the students. I write a quick note along the lines of "we will cover this in class", and then I do a a series of worked examples following the principles of Explicit Instruction, and then ask the students to correct their work. This approach has the onus of putting the emphasis back onto the students, saves me hours, and also allows my students to benefit from the positive effects of spacing covered in the Memory section. Of course, determining whether a student could not do the work, or could not be bothered to do the work, can be difficult, and once again comes down to knowledge and relationships.
My favourite quote:
Feedback, however, is not “the answer”; rather, it is but one powerful answer. With inefficient learners, it is better for a teacher to provide elaborations through instruction than to provide feedback on poorly understood concepts. If feedback is directed at the right level, it can assist students to comprehend, engage, or develop effective strategies to process the information intended to be learned. To be effective, feedback needs to be clear, purposeful, meaningful, and compatible with students’ prior knowledge and to provide logical connections. It also needs to prompt active information processing on the part of learners, have low task complexity, relate to specific and clear goals, and provide little threat to the person at the self level. The major discriminator is whether it is clearly directed to the task, processes, and/or regulation and not to the self level. These conditions highlight the importance of classroom climates that foster peer and self-assessment and allow for learning from mistakes.
Research Paper Title: The Secret of Effective Feedback
Author(s): Dylan Wiliam
My Takeaway:
I make no secret of the fact that Dylan Wiliam has - perhaps more than any other individual - had the greatest influence on my teaching. We have already seen his work in the Assessment for Learning section, and now it is time to read his advice when it comes to making feedback effective. This article from 2016 is an excellent summary of Wiliam's views on feedback that he has developed over many years of studying the topic. The opening sentence summarises the main thrust of his argument: feedback is only successful if students use it to improve their performance. Wiliam goes on to outline a set of general principles for making feedback as effective as possible:
1. Keeping Purpose in Mind. Wiliam explains that most of the time the student work we're looking at is not important in and of itself, but rather for what it can tell us about students—what they can do now, what they might be able to do in the future, or what they need to do next. Looking at student work is essentially an assessment process. We give our students tasks, and from their responses we draw conclusions about the students and their learning needs. When we realize that most of the time the focus of feedback should be on changing the student rather than changing the work, we can give much more purposeful feedback. If our feedback doesn't change the student in some way, it has probably been a waste of time.
2. Giving Feedback They Can Use. We need to start from where the learner is, not where we would like the learner to be. We need to use the information we obtain from looking at the student's work—even through that information may be less than perfect—and give feedback that will move the student's learning forward.
3. Assign Tasks That Illuminate Students' Thinking. Looking at students' work often tells us only that they didn't do it very well and they need to do it again, but better. Designing tasks that, in Ritchhart and Perkins's (2008) phrase, "make thinking visible" takes time, but front-loading the work in this way makes it much more likely that we'll provide useful feedback.
4. Make Feedback into Detective Work. Consider a maths teacher who provides feedback on 20 solved equations. Rather than telling the student which equations are incorrect, the teacher can instead say, "Five of these are incorrect. Find them and fix them." Such practices ensure that students, the recipients of feedback, do as much work as the teacher who provides the feedback. Making feedback into detective work encourages students to look at the feedback more closely and to think about their original work more analytically.
5. Build Students' Capacity for Self-Assessment. The amount of feedback we can give our students is limited. In the longer term, the most productive strategy is to develop our students' ability to give themselves feedback. Most teachers seem to believe that students make most of their progress when the teacher is present, with homework as a kind of optional add-on. It's important, therefore, to develop students' capacity for self-assessment. Wiliam suggests an approach called "plus, minus, interesting,". At the end of a task, ask students to identify something they found easy about the task, something they found challenging or difficult, and something they found interesting. Such reflection develops language skills and helps the students become clear about what areas they need to work on.
6. A Trusting Relationship for Feedback. In the end, it all comes down to the relationship between the teacher and the student. To give effective feedback, the teacher needs to know the student—to understand what feedback the student needs right now. And to receive feedback in a meaningful way, the student needs to trust the teacher—to believe that the teacher knows what he or she is talking about and has the student's best interests at heart. Without this trust, the student is unlikely to invest the time and effort needed to absorb and use the feedback.
My favourite quote:
The only thing that matters is what the student does with the feedback. If the feedback you're giving your students is producing more of what you want, it's probably good feedback. But if your feedback is getting you less of what you want, it probably needs to change. Finally, talk to your students. Ask them, "How are you using the feedback I'm giving to help you learn better?" If they can give you a good answer to that question, then your feedback is probably effective. And if they can't, ask them what they would find useful. After all, they're the clients.
Research Paper Title: Learning versus Performance
Author(s): Nicholas C. Soderstrom and Robert A. Bjork
My Takeaway:
This is the second appearance of this lovely paper following its debut in the Memory section, and once again its effect on me has been huge. This paper seeks to question the common assumption that immediate feedback is the best kind of feedback - and leads us on to the final of Bjork's "desirable difficulties". The logic makes perfect sense - if a learner gets something wrong, surely they need to know that they have got it wrong immediately, and why they have got it wrong, in order to correct their mistake? And indeed, during the early phases of knowledge acquisition, immediate feedback may be beneficial to ensure mistakes do not become embedded. However - and wait for this because it is a classic - research presented in this paper suggests that delaying or even reducing feedback can have a long term benefit to student's learning. Why? Well, because regular, immediate feedback can cause learners to become overly dependent upon it (almost seeing it as a crutch to their learning). To relay this to themes covered in other sections of this page, immediate feedback prevents students from thinking hard, and also having the opportunity to forget. I have seen this myself - you give students their work back with detailed feedback and corrections to make, and they look at it and go "yeah, yeah", and make the necessary corrections. But are they actually thinking? Ironically (or perhaps, unfortunately) the better the feedback, the less they need to think. Once again we have the issue of distinguishing between learning and performance. The authors suggest that delaying or reducing feedback is likely to have a determinantal effect on short-term performance, but a positive effect on long-term learning. What are the implications for us teachers, especially given that the Hattie paper above stresses the potential positive benefits of feedback? Well, one idea I had reading this paper was when initially marking a piece of homework to give no feedback whatsoever, simply to provide ticks and crosses. Then give this work back to the students and see if they could identify the source of their errors and correct them. When I next take the books in, this is when I would give feedback. In my view, there are two advantages of such an approach. Firstly, it will force students to think about why they got a question wrong, instead of just blindly following my step-by-step prompts, and hence become less reliant on me. Secondly, it may enable me to better identify actual gaps in their knowledge. As discussed in the paper above, feedback when students do not have the knowledge to begin with is likely to be a waste of time. However, if indicating where students have gone wrong allows students to have a better stab at those questions, then I can then better distinguish between gaps in their knowledge versus the kind of mistakes that well constructed, task-focused feedback is likely to help resolve. The knowledge gaps may then need addressing again via instruction.
My favourite quote:
Delaying feedback until after the task was completed yielded greater long-term learning than concurrent feedback. Performance gains during acquisition, however, were made more rapidly by those receiving concurrent feedback.
Research Paper Title: Formative Assessment: Practical Ideas for Improving the Efficiency and Effectiveness of Feedback to Students
Author(s): Geraldine O’Neill
My Takeaway:
A really good summary of effective feedback practice, together with the value of this idea to either the teacher or the students, all referenced to relevant research-based evidence. There are many interesting practices cited here, but the ones that stood out to me were:
1) Consider feedback in different media/formats, such as on-line, audio-feedback, verbal class feedback, use of ‘clickers’ in large class contexts. This may not only save the teacher time, but may make feedback more permanent and easily accessible to students.
2) Student Requested Feedback - ask students to submit specific requests for areas for feedback. As this is student-focused feedback it is more inclined to motivate students to act. This also encourages students to take some responsibility in the process.
3) In class peer and self assess feedback activities in terms of discussing and giving feedback to annonymised work. This will help students engage in the feedback process and make them more aware of the desired standard in relation to their own work.
4) Comment in actionable language. This relates to the recurring theme throughout this section that feedback should be task-focussed, but also reminds us that if the students cannot understand what the feedback is on about, it is likely to be a complete waste of time.
My favourite quote:
One of the key themes emerging to address this dilemma is to develop students own self-monitoring skills in order to help them narrow the gap between their performance and the standards expected of them. The timing, type and specification of feedback can also improve student ability to self-monitor. In addition, good feedback should feed into some specific actions that can be used in the next assessment. Feedback need not always be from the academic staff, students themselves are a good resource to each other when given guidance on how to do this. New technologies also open up some efficient feedback opportunities.
Research Paper Title: Mathematics Inside the Black Box: Assessment for Learning in the Mathematics Classroom
Author(s): Jeremy Hodgen and Dylan Wiliam
My Takeaway:
This is a wonderful paper that first made an appearance in the Formative Assessment section, but it is with regard to its advice on marking and feedback that I want to focus on here. Often I find it difficult to write good comments in students' books. Comments such as "show your working" and "correct your mistakes" have tended to have little impact in my experience. The main piece of advice I took from this paper was that the comments I write in students' books such match the kind of questions I would ask the students in class. I have long been an advocate of planning questions as opposed to planning resources, and so thinking of feedback in these terms really made sense to me. It also fits in very well with the general principles that feedback should make students think. The authors kindly provide a suggestion of the types of comments/questions you could write. My favourites include:
1) Enabling pupils to identify the errors for themselves:
- There are five answers here that are incorrect. Find them and fix them.
- The answer to this question is [. . .] Can you find a way to work it out?
2) Identifying where pupils use and extend their existing knowledge:
- You’ve used substitution to solve all these simultaneous equations. Can you use elimination?
- You seem to be having difficulty adding some of these fractions and not others. In question 2 you used equivalent fractions; could you use this on question 4?
3) Encouraging pupils to reflect:
- You used two different methods to solve these problems. What are the advantages and disadvantages of each?
- You have understood [. . .] well. Can you make up your own more difficult problems?
4) Suggesting pupils discuss their ideas with other pupils:
- You seem to be confusing sine and cosine. Talk to Katie about how to work out the difference.
- Compare your work with Ali and write some advice to another student talking this topic for the first time.
5) Helping pupils to show their working:
- The way in which you are presenting graphs is much clearer. Look back at your last work on graphs in February. What advice would you give on how to draw graphs?
- Your solutions are all correct, but they are a bit brief. Look at the examination marking criteria. Work with Leo to produce model answers that would convince the examiner to award you all the marks.
I love this feedback because it is written in a language students understand, it makes students think, it gives them strategies to use if they are struggling, and it should prove to be more work for the students than the teacher. Of course, these comments take time to write, and it is worth bearing in mind that students need to be given dedicated time to act upon this feedback in lessons.
My favourite quote:
The content of effective written comments, of course, varies according to the activity and mathematical content. Often, advice to pupils will be very similar to the kinds of interventions and questions that a teacher uses with the whole class, although it is an opportunity for the teacher to give personalized feedback
Research Paper Title: Task-Involving and Ego-Involving Properties of Evaluation: Effects of Different Feedback Conditions on Motivational Perceptions, Interest, and Performance
Author(s): Ruth Butler
My Takeaway:
This is a really interesting study into the effectiveness, and indeed the effects, of different types of feedback. A total of 200 fifth- and sixth-grade students with high or low school achievement were given takes to complete. Individual comments, numerical grades, standardized praise, or no feedback were received after Sessions 1 and 2. The results following a post-test were that interest, performance, and attributions of effort were highest at both levels of achievement after receipt of comments. Indeed, post-test scores were one stand deviation higher for students in the Comments group, with no significant differences between scorers in the other groups. Similarly, ego-involved attributions were highest after receipt of grades and praise. In other words, grades and praise had no effect on performance, and served only to increase ego-involvement. It is perhaps of little surprise that comments were the most successful in generating the kind of response we would want from students. However, I was taken aback by the fact that grades had the same impact as praise in generating ego-involving responses. My conclusion, based on my own experiences with my students, is that grades cause students to think about themselves - eliciting emotions ranging from joy to despair - and this prevent them from focussing on the task itself. This will be discussed further in the next study.
My favourite quote:
The present results further confirmed that individual comments yielded higher task-involved perceptions and lower ego involved ones than either grades or praise and that no feedback yielded perceptions of both kinds of factors as being relatively non-determinative of both effort and outcome. The similar and ego-involved perceptions induced by grades and praise seem particularly significant. Both anecdotal evidence and some research findings suggest that grades are perceived as potent sources of control over learning.
Research Paper Title: Enhancing and undermining Intrinsic motivation; the effects of task-involving and ego-involving evaluation on interest and performance
Author(s): Ruth Butler
My Takeaway:
The last paper showed that task-focussed comments were preferable to grades in terms of subsequent performance on tasks, as well as general involvement and interest in the task. This begs the question: what if comments and grades are combined? This paper provides the answer. 48 eleven-year old Israeli students were selected from the upper and lower quartiles of attainment from 12 classes in 4 schools and worked in pairs over three sessions on two tasks (one testing convergent thinking and the other, divergent). After each session, each student was given written feedback on the work they had done in the session in one of three forms: A) individualised comments on the extent of the match of their work with the assessment criteria that had been explained to each class at the beginning of the experiment; B) grades, based on the quality of their work in the previous session; C) both grades and comments. Students given comments showed a 30% increase in scores over the course of the experiment, and the interest of all students in the work was high. Students given only grades showed no overall improvement in their scores, and the interest of those who had high scores was positive, while those who had received low scores show low interest in the work. Perhaps most surprisingly, the students given both grades and comments performed similarly to those given grades alone—no overall improvement in scores, and interest strongly correlated with scores—and the researchers themselves describe how students given both grades and comments ignored the comments, and spent their time comparing their grades with those achieved by their peers. The message is simple: If you are going to grade or mark a piece of work, you are wasting your time writing careful diagnostic comments.
My favourite quote:
The results confirm the importance of distinguishing between task involvement and ego-involvement when investigating intrinsic task motivation. As hypothesized, both high and low achievers who received comments continued to express high interest both on Session 2 when they anticipated further comments, and at post-test, when they did not.
Research Paper Title: Rank as an Incentive
Author(s): Anh Tran and Richard Zeckhauser
My Takeaway:
The papers above have made clear that if you give students both a grade/mark and a formative piece of feedback, all they will look at is the grade/mark. Here we consider a different type of feedback - instead of telling students their grade, tell them their rank in the class. Whenever I give students back their tests, they are always desperate to find out how everyone else got on, and although I never tell them, they pretty quickly - over whispers, gestures and paper waving - work out their rough ranking in the class. When I interviewed Dani Quinn, the Head of Maths at Michaela School, she explained how she not only ranks her class after every assessment, but puts these rankings on public display on the notice board in the corridor. What does the research have to say about such an approach? Well, this paper makes interesting reading. In one experiment, the researchers gave a class of Vietnamese college students some reading material to prepare for a test that would be given in ten days. Each student received a participation fee. They randomly assigned the students into one control group (with no extra incentive) and three treatment groups (one with rank incentive, one with financial incentive and one with both). They found that: (i) the group that knew that their rankings would be publicised outperformed the control group; (ii) the group that both earned cash for correct answers and knew that their rankings would be publicised outperformed the group that merely earned cash for correct answers. The paper also quotes other research that replicates the general finding that competition improves performance. Before making any sweeping generalisations, we must be aware that the sample size is small and the students are of university age in Vietnam. However, it has made me think. The knowledge that students are going to be ranked may well provide a good incentive for them to put extra effort into their homework or test. However, this must be traded against the possible repercussions of that ranking. Will students who are in the top 3 in the class get complacent? Will students in the bottom 3 in the class give up? As we have seen with much of the research on feedback, a lot of this comes down to the relationship a teacher has with their students, and the school culture as a whole. Michaela's justification is that they believe performance on tests is highly correlated with effort, and so exposing students who perform badly is a way of highlighting the consequences of poor effort. That is fine so long as the correlation hold true. If performance is instead linked to ability, then it will be pretty demoarilsing for those students who may find themselves permanently ranked near the bottom. And what happens if everyone improves? How is that reflected in the rankings? Making students aware of ranking seems to be to be a zero-sum game with a lot of potential risks, but it is something that has certainly caused me to think.
My favourite quote:
Economists admire competition because it promotes efficiency and enables the market system to work efficiently. Rank incentives may be net beneficial in some circumstances, as they encourage all to perform better. They may be detrimental in others. But whatever the net report card, the record is clear. Humans care considerably about their rank, and economic models that seek descriptive relevance must attend to that incentive.
Research Paper Title: The Impact of Self- and Peer-Grading on Student Learning
Author(s): Philip M. Sadler and Eddie Good
My Takeaway:
With so much marking to do, I often like to ask my students to either mark their own work or mark their neighbour's. Indeed, self or peer marking is a fundamental component of my practice of regular low stake tests. But that all begs the question, which is the most useful: self or peer assessment. The findings from this paper are fascinating.
1) For either strategy to work students must be familiar with how to mark accurately. This suggests strategies such as exposing students to mark schemes and showing them examples of other students' work and then discussing them are likely to be successful in achieving this aim. The authors sum this up with the following: "For optimal student-grading, we suggest training, blind grading, incentives for accuracy, and checks on accuracy compared to teacher grades".
2) There are indications of bias in the students' marks: when grading others, students awarded lower grades to the best performing students than their teacher did, but when grading themselves, lower performing students tended to inflate their own low scores.
3) However, what is most interesting to me is the impact on learning. When students were given a follow-up test based on the same content that they had been marking, students who graded their peers’ tests did not gain significantly more than a control group of students, but those students who corrected their own tests improved dramatically. This implies that self-assessment is more beneficial to learning than peer assessment, and I have put this finding into practice by allowing students to mark their own low-stakes tests. My students also seem to prefer this, and it has led to less sounds of "what does this say?" that often accompanies peer assessment. One thing to bear in mind, however, is you lose the benefits of peer-to-peer discussion and feedback that has been identified in some of the papers in this section.
My favourite quote:
Student-grading is best thought of not as an isolated educational practice, but as a part of the system of learning and assessment carried out in a teacher’s classroom. It involves sharing with students some of the power traditionally held by the teacher, the power to grade. Such power should not be exercised recklessly or unchecked. Such power in their own hands or in those of their peers can make students uncomfortable or wary. Teachers should train their students in the skills entailed in accurate grading and should monitor students for accuracy in self- or peer-grading. When used responsibly student-grading can be highly accurate and reliable, saving teachers’ time. In this study, self-grading appears to further student understanding of the subject matter being taught.