Using error analysis sheets to improve learning in higher education

We developed and evaluated the effectiveness of a new teaching strategy aimed at improving students' learning in a Research Design degree module with high mathematical content. The strategy involves presenting students with case studies based on published research and a set of questions to answer, following which they are given error analysis sheets detailing the correct answer for each question, the most common incorrect answers, and an explanation of why these answers are wrong. These sheets were created based on students' answers to the case-study questions and were used with the aim of helping them learn from their mistakes. A high proportion of students perceived the error analysis sheets to have been useful for improving their learning and understanding, and this was especially the case for those with a higher level of math anxiety. In addition, students' final grades were positively related to their attendance at classes in which error analysis sheets were used.


Introduction
Many students find subjects with mathematical content difficult, and hence it is essential to develop innovative ways of supporting their learning (Núñez-Peña et al., 2013a;Núñez-Peña et al., 2015).Indeed, it is important not only to increase students' motivation and interest for these subjects (Garfield, 1995) but also to help them approach assessment or examinations with confidence (Nagi et al., 2010).By doing so, students will be more likely to acquire a good level of knowledge and skill and be able to demonstrate what they have learned.Given that emotional factors also play a role in the learning process (Cassady, 2010;Núñez-Peña & Bono, 2019;Thomas et al., 2017), it is important, in the case of subjects with mathematical content, to consider the possible impact of math anxiety.Math anxiety is defined as feelings of tension and apprehension that interfere with the ordinary manipulation of numbers and the solving of mathematical problems (Ashcraft & Faust, 1994).It has been observed among students from elementary school through to higher education (Beilock & Willingham, 2014), and it negatively affects academic achievement in subjects involving mathematics (Ashcraft & Krause 2007;Zhang et al., 2019).
In this study we describe a new teaching strategy implemented in the context of a psychology degree module with mathematical content: Research Design.The strategy involved: 1) producing a series of case studies based on published research in different areas of psychology (clinical, educational, social, etc.), 2) devising questions for each case study that students had to answer, and 3) creating an error analysis sheet for each case study based on students' answers to the questions.The case studies were chosen to reflect the syllabus of the Research Design module and also addressed topics that students had encountered in other course modules.Error analysis sheets detailed the correct answer for each question, the most common incorrect answers given by students, and an explanation of why these answers were wrong.The goal of this new teaching strategy was to help students analyze and learn from their own mistakes, and ultimately to become more confident in the subject.This would, we hoped, allow them to feel better prepared and to approach assessment with more confidence and less anxiety.Our expectation was that error analysis sheets would be particularly useful for highly math-anxious students.
The effectiveness of this new teaching strategy was evaluated by exploring students' perceptions regarding the extent to which the use of error analysis sheets had helped to improve their learning and understanding.We also examined the relationship between math anxiety and the perceived usefulness of error analysis sheets.

Participants
Participants were 66 students (68.9% female, 31.1% male; age range 19-56 years) who were enrolled during the 2018-2019 academic year in the Research Design module of the Degree in Psychology offered by the University of Barcelona.All students signed informed consent before participating.

Material
Case studies.Eight case studies were created based on studies published in different areas of psychology.Research papers were summarized and adapted to the syllabus requirements of the Research Design module.A database was also simulated for each case study so that the results of the original studies could be reproduced.Therefore, each case study included a summary of the original research, a database, and a series of questions for students to answer.This material is available in the OMADO collection, the University of Barcelona's digital repository of teaching materials.
Error analysis sheets.During the 2018-2019 academic year, students enrolled in the Research Design module were presented with the case studies and asked to answer the corresponding questions during a practical class.They also had to send their answers to teaching staff via one of the university's digital platforms (Virtual Campus or Learning Management System).Teachers reviewed students' answers for each case study and drew up the corresponding error analysis sheet showing the correct answer, the most common incorrect answers, and an explanation of why these answers were wrong.
Questionnaire to assess the usefulness of error analysis sheets.A questionnaire was developed ad hoc to evaluate the perceived usefulness of error analysis sheets.The seven items, each rated on a Likert-type scale from 1 (totally agree) to 5 (totally disagree), were as follows: (1) Error sheet classes have helped me to learn the subject, (2) Error sheet classes have improved my learning, (3) Error sheet classes have given me a better understanding of the assessment criteria, (4) Error sheet classes have allowed me to self-assess, (5) Error sheet classes have helped me to feel less anxious when preparing for exams, (6) In general, I think the error sheet classes have been very useful, and (7) I would like there to be error sheet classes for all the degree modules.This questionnaire also recorded students' frequency of attendance at different types of classes (theory, practical, and those involving error analysis sheets).Núñez-Peña et al., 2013b).The sMARS comprises 25 items, each of which describes a situation that may generate math anxiety (e.g., "Entering a math class").For each item, respondents indicate using a 5-point Likert-type scale (1, no anxiety; 5, a lot of anxiety) how much anxiety they would feel in that situation.The sMARS includes three scales: math test anxiety (score range: 15-75), numerical test anxiety (score range: 5-25), and math course anxiety (score range: 5-25).

Shortened Math Anxiety Rating Scale, sMARS (Spanish version by
Assessment for the Research Design module.At the end of the module, students were given a test comprising 15 theory/practice questions.

Procedure
During the 2018-2019 academic year the case studies were used in conjunction with the error analysis sheets in teaching the Research Design module.Each week, students were given one of the case studies and asked to answer the associated questions.In the following class they were given the corresponding error analysis sheet, thus providing them with feedback on their performance and giving them the opportunity to learn from their mistakes (hereinafter, error sheet classes).At the beginning of the Research Design module, all students completed the sMARS in their usual classroom setting; one of the researchers was present to answer any queries and to ensure that their responses were individual.At the end of the module, students were asked to complete the questionnaire about the perceived usefulness of error analysis sheets and their frequency of attendance at different types of classes.They also sat the 15question exam to assess their learning.

Results
The proportion of students who always or regularly attended the different types of classes was 97% for theory classes, 97% for practical classes, and 92.4% for error sheet classes.As regards the perceived usefulness of error sheet classes, over 80% of students agreed or strongly agreed that these classes had helped them to learn the subject, to improve their learning, to understand the assessment criteria, and to self-assess.
We then calculated Spearman correlations between responses to the items exploring the perceived usefulness of error sheet classes and scores on the sMARS (math anxiety).Responses to item 3 ("Error sheet classes have given me a better understanding of the assessment criteria") were positively correlated with sMARS total scores (r = .299,p = .030)and with scores on its math test anxiety (r = .287,p = .037)and math course anxiety scales (r = .382,p = .005).A positive correlation was also observed between responses to item 6 ("In general, I think the error sheet classes have been very useful") and scores on math course anxiety (r = .287,p = .037).There were no correlations between responses to any of the remaining items and scores on the sMARS.

Conclusion
Our primary aim in this study was to develop a new approach to teaching a Research Design module with high mathematical content.This involved presenting students with case studies based on published research and a set of accompanying questions to answer, following which they were given an error analysis sheet.After implementing this approach, we explored students' perceptions regarding the extent to which the use of error analysis sheets had enhanced their learning.We also examined the relationship between the perceived usefulness of error analysis sheets and math anxiety.
A high proportion of students considered that the error sheet classes had helped them to learn the subject, to improve their learning, to understand the assessment criteria, and to self-assess, indicating satisfaction with this new approach.Importantly, students' final grades were associated not only with their attendance at theory and practical classes but also, and more strongly, with their attendance at error sheet classes.Previous studies have shown the usefulness of providing students with tools that can help them to feel better prepared and more self-confident when it comes to assessment.For instance, Bono and Núñez-Peña (2019) and Nuñez-Peña et al. (2015) demonstrated the effectiveness of a formative assessment system based on feedback and rubrics in improving the performance of undergraduates.
As for the relationship between the perceived usefulness of error analysis sheets and math anxiety, students with higher levels of math anxiety were those who considered that the error sheet classes had been most useful and had given them a better understanding of the assessment criteria.This suggests that the use of error analysis sheets could help highly mathanxious students to gain confidence in their ability to learn subjects of this kind, thereby reducing the negative impact of math anxiety on their performance.
In conclusion, the results suggest that this new teaching strategy based on the use of error analysis sheets could help to improve students' learning of subjects with high mathematical content, especially in the case of highly math-anxious students.Importantly, students' final grades were positively related to their attendance at error sheet classes.