Examining university student satisfaction and barriers to taking online remote exams

Recent years have seen a surge in the popularity of online exams at universities, due to the greater convenience and flexibility they offer both students and institutions. Driven by the dearth of empirical data on distance learning students' satisfaction levels and the difficulties they face when taking online exams, a survey with 562 students at The Open University (UK) was conducted to gain insights into their experiences with this type of exam. Satisfaction was reported with the environment and exams, while work commitments and technical difficulties presented the greatest barriers. Gender, race and disability were also associated with different levels of satisfaction and barriers. This study adds to the increasing number of studies into online exams, demonstrating how this type of exam can still have a substantial effect on students experienced in online learning systems and technologies.


Introduction
Online exams have become increasingly popular in universities in recent years, as they offer students and institutions greater flexibility and convenience. Despite the growing popularity, there is still limited research that addresses student satisfaction with online exams, and in particular students in distance-learning universities. This paper examines the evidence on student satisfaction and barriers to online exams at universities and explores how students in distance learning universities engage with this new exams model.
Recent studies suggest that students are generally satisfied with online exams, as they allow for more flexibility and convenience. For example, Bashir et al. (2019) found that students are more satisfied with online exams as they are allowed more time to prepare, and the exams were more accessible. However, the increased use of online exams in universities has raised questions about students' potential challenges when taking such tests. Technological issues are a major barrier for many students taking online exams. These include slow internet connection, lack of access to the necessary technology, and lack of technical knowledge (Barrot et al., 2021). In addition to technical challenges, students face psychological difficulties, such as stress and anxiety, when taking online exams. For instance, Almossa (2021) noted that students felt overwhelmed and anxious during online exams due to the lack of face-to-face interaction with the instructor.
Recent studies have shown that online exams are more beneficial to certain student groups than others. Dikmen (2022) examined the effect of online exams on female medical students and found that they were more likely to experience anxiety and lower self-efficacy when taking online exams than their male counterparts. This was because they felt they had less control over the exams and less access to resources. Further, Tai et al. (2022) note that with online exams, disabled students can take the exam in a location of their choosing, adjust the font size and background colour, and have more time to read, write and review answers. However, the literature also highlights some drawbacks of online exams for disabled students, such as feelings of isolation (Tai et al., 2022). This can be particularly problematic for those with disabilities that require more support. Finally, Tran and Reilly (2019) suggest that online exams can create challenges for Black, Asian or Minority Ethnicity (BAME) students due to their lower access to technology, less access to tutoring or other academic supports, and lower levels of comfort and confidence with the technology.
This study explores students' satisfaction and barriers to taking online exams at The Open University (OU), an institution with a long tradition of distance learning and mature student enrolment in the UK. The OU delivers its courses via virtual learning environments, online tutorials and tutor groups. Pre-pandemic, 24% of the OU courses ended with a face-to-face exam, which has now been replaced with mainly remote open book style exams (e.g., multiple choice questionnaires, essays, equations and numerical workings). These exam interactions range from timed exams (2-4.5 hours) to 7-day submission windows. Motivated by the current lack of an empirical basis for insights into distance learning students' satisfaction and barriers to taking online exams, we explored the following research questions (RQs): 1. To what extent are distance learning students' satisfied with the different elements of online exams? 2. What are distance learning students' barriers to taking online exams? 3. Are there any significant differences among student groups (gender, race, disability) in their satisfaction and barriers to taking online exams?

Development of survey instruments
The instruments used in this study consist of the 'online exam satisfaction' and 'barriers to taking online exams', hereby mentioned as 'satisfaction' and 'barrier' instruments, respectively. The 'satisfaction' instrument was developed as part of a bigger assessment project by the Student Experience of Feedback, Assessment and Revision (SEFAR) team at the OU (Cross, Whitelock & Mittelmeier, 2016) and was validated through an institutional survey administered in 2015 (n = 281) and thirteen interviews. The instrument includes items developed from constructs tested previously by others (e.g., Vattøy, Gamlem & Rogne, 2021) and, in part, composed of newly created items designed to probe additional themes such as anxiety (Falchikov & Boud, 2007) and exam preparedness (Payne & Brown, 2011).
The 'barrier' instrument was developed as part of a Covid-19 evaluation project at the OU (Aristeidou & Cross, 2021). It consists of items identified in the literature as reasons for students' disrupted studies during the pandemic. Some examples of these items include a lack of technical equipment and great demand for devices and the internet in the household (Barrot et al., 2021), childcare and other caring responsibilities (Chirikov et al., 2020). Both instruments were in the form of 5-item Likert scales, ranging from 1 (strongly disagree) to 5 (strongly agree), with an extra 'not applicable' (N/A) option.

Data collection
Data collected include undergraduate students self-reports on the online exams 'satisfaction' and 'barriers' survey instruments. Demographics and other student characteristics were retrieved from the university database, such as gender (female or male), declared disability (yes/no), and BAME (yes/no). The dataset was anonymised on the 30th of May 2022, prior to initiating the process of data analysis. Ethical approval was obtained from the authors' university ethics committee. The participants provided us with a written consent.

Data analysis
Visualisations of the Likert scales were used to describe students's satisfaction with different aspects of online exams (RQ1) and the barriers to taking online exams (RQ2). Percentages of participants were presented for each frequency selection, and items were presented in order of negative impact on the frequency of agreeing to a particular statement. Data presented in RQ1 include only the students who had an experience of taking online exams (n = 190), while data presented in RQ2 includes all survey participants (n = 562).
Then, to determine how gender, disability, race, previous qualification and qualification intention relate to each 'satisfaction' or 'barrier' statement (RQ3), chi-square tests were performed. For the tests, dichotomous variables were used for each statement, in which option 1 included all the 'agree' and 'strongly agree' responses to the statement and 0 all the 'disagree' and 'strongly disagree'. N/A and 'neither agree nor disagree' were excluded from the chi-square test analysis. An alpha level of .05 was used for all the analyses.

Satisfaction with online exams
The survey respondents who had an experience with online exams (n = 190) were mainly found to agree that they were able to find a quiet space to take the exam (86%), they were satisfied with the quality of that space (85%), the type of exam allowed them to demonstrate their learning (78%), thought that the exam questions were clear (77%), felt a sense of achievement upon completion of the exam (76%), were satisfied with the mark received (72%), felt prepared just before starting the exam (61%), felt anxious during the exam (58%), thought that the exam was harder than they expected (48%), and enjoyed the exam (31%) (Figure 1).

Figure 1. Student satisfaction with different areas of online exams, presented in ascending order of agreement with each statement
Findings from examining student experience during and with the online exams and among students in different groups indicated that BAME students were less likely to feel prepared immediately before starting the exam (χ 2 = 4.53, df = 1, p = 0.03) than those of a white background. There were also borderline differences between students who declared disability and those who did not (χ 2 = 3.74, df =1 , p = 0.053). Gender-wise, female students were less likely to report that the exam questions allowed them to demonstrate what they had learnt (χ 2 = 4.32, df = 1, p = 0.04). Moreover, female students (χ 2 = 6.48, df = 1, p = 0.01) and students with declared disabilities (χ 2 = 5.53, df =1 , p = 0.02) were less likely to be satisfied with the mark they received. Further, students with declared disabilities (χ 2 = 8.72, df =1, p = 0.003) were less likely to be satisfied with the quality of the home space they used.
There were also no significant differences among different student groups as to whether (a) the exam questions were clear, (b) the exam to be harder than they expected, (c) they felt anxious when doing the exam, (d) they felt a sense of achievement when completing the exam and (e) they were able to find a quiet space to take the exam.

Barriers to taking online exams
The survey respondents, both those with and without an experience with online exams (n = 562), selected employment commitments as the most significant barrier to taking online exams at home -chosen by about one in three students (30%) (Figure 2). This barrier was followed by a lack of confidence in dealing with technical difficulties, expressed by 29% of the survey respondents, when dealing with potential technical difficulties during the online assessment. The third most chosen challenge, selected by 28% of the respondents, was the reliability or quality of the internet connection. Other less selected barriers involved a lack of quiet working space (20%), childcare responsibilities (17%), mental health issues (13%),

5
Examining university student satisfaction and barriers to taking online remote exams confidence in setting up and using technologies (13%), access to an internet connection (10%), and other caring responsibilities (10%).

Figure 2. Student barriers to taking online exams, presented in ascending order of agreement with each statement
Findings from examining barriers to taking online exams among students in different groups indicated that female students are more likely to report a lack of confidence in setting up and using technology (χ 2 = 10.00, df = 1, p < 0.01) and dealing with potential technical difficulties during assessment (χ 2 = 11.07, df = 1, p < 0.01). In addition, female students (χ 2 = 4.51, df = 1, p = 0.03) were more likely to choose mental health issues as a barrier to taking online exams at home. Students with declared disabilities (χ 2 = 9.01, df = 1, p < 0.01) are more likely to choose other caring responsibilities as a barrier to taking online exams at home. Finally, students with declared disabilities (χ 2 = 35.32, df = 1, p < 0.01) were more likely to choose physical disability as a barrier to taking online exams at home.
There were no differences among student groups in the following statements: employment commitments or constraints, reliability or quality of internet connections, lack of quiet working space, childcare responsibilities, access to an internet connection, and household competition for access to devices. Furthermore, there were no differences between students of different races and faculties.

Discussion
Distance learning university students at the OU demonstrated, like traditional university students (e.g., Bashir et al., 2019), high levels of satisfaction with the environment in which they took the exam and the exam procedure, including the type of exams and the clarity of the questions. However, similarly to other studies (e.g., Almossa, 2021), they felt anxious during the exam and thought it was harder than expected. Barriers to taking exams involved various areas, with the most affected being employment commitments -expected in a university with students of a higher average age. Likewise other studies (e.g., Barrot et al., 2021), technical difficulties also ranked high, with barriers related to dealing with unexpected software and internet issues. Other important barriers related to finding a suitable environment for the exam and arranging childcare -which can relate again to the fact the students have different responsibilities compared to the average university student.
Our findings align well with previous studies wherein online exams are less beneficial for specific student groups than others. In line with Dikmen (2022), female students were found to experience more issues than their male counterparts, including more lack of confidence in technology, and mental health issues. They also thought the type of exam impacted their performance. We have also verified that students with declared disabilities face some drawbacks when taking online exams, including lack of access to assistive technologies or mental health issues (e.g., Tai et al., 2022). However, our results go beyond previous reports, showing how their environment and caring responsibilities may also be barriers. A similar conclusion to Tran and Reilly (2019)'s was reached on BAME's students' confidence with online exams, with findings in the current study concerning their feelings of unpreparedness.
Our work adds to a growing corpus of research on online exams, showing how this type of exam can still impact students familiar with online learning formats and technologies. Overall, the literature and our findings suggest that online exams can offer several advantages over traditional exams. However, it is also important for universities to recognise the potential drawbacks, and carefully consider the needs of students when planning online exams, ensuring they are adapted to meet the different student groups' requirements.
In addition to the greater convenience and flexibility, online exams have also provided a pragmatic and workable solution to challenges as a result from crises such as the recent global pandemic. In this context, online exams can, as indeed they did during 2020 and 2021, provide means to secure academic business continuity and student success. The findings from this and similar studies can give direction to universities about which processes and policies they need to put in place. The barriers to student satisfaction identified here seem, from an institutional perspective, perfectly manageable and do not require unaffordable or not yet existing high tech solutions: acknowledging employment commitments can be addressed through assessment load and scheduling, anxieties about technology can be lowered through training and responsive support services. There is, in fact, little in these findings that could not be implemented at speed: it would therefore all the sooner address the barriers, secure confidence in the robustness of assessment, and offer a better service overall to students.