A case study of hybrid learning in higher education using quantitative research design

There is currently limited work on synchronous forms of hybrid education although it has become the norm in university education. In addition, there is a gap on empirical studies that are focusing on quantitative measures (such as student performance data) in this particular setting. To compensate for this gap in the literature, the paper presents a case study of hybrid teaching in the context of higher education that revolves around a quasi-experimental design with student performance data. The results were analysed quantitatively and they indicate that there was no significant statistical difference between the two groups in their performance in the final exams (with a large size effect). That finding is somewhat surprising since there is research indicating that the learning experience and performance of the students that attend a hybrid learning course remotely are not equally good as in the case of the students that participate in-class/on-site.


Introduction
Hybrid teaching and learning is the main learning mode in higher education nowadays (Mavroudi & Gynnild, 2021), yet more research focusing especially on synchronous forms of it is still needed (Raes et al., 2020).In the context of this case study, hybrid learning is defined as synchronous blended learning "in which both onsite and remote students can simultaneously attend learning activities" (Raes er al., 2020, p. 1).Most of the existing literature in synchronous forms of hybrid teaching and learning in higher education is exploratory and qualitative in nature focusing on parameters such as students' learning experiences and perceptions of the learning environment, whereas empirical studies that fall within the quantitative research paradigm taking into account data associated to the students' outcomes have only begun to emerge (Raes et al., 2020).Yet, research in some cases has shown that there is an achievement gap between students taking courses exclusively offered online versus those enrolled in face-to-face classes in higher education (Lightner & Lightner-Laws, 2016).The Covid19 pandemic intensified the adoption of hybrid forms of education since higher education institutions all over the world had to abruptly shift to online learning.A question that is posed by Matta and Palvia (2021) and many others is: which pedagogical innovations should we keep when the situation returns back to normal?One answer is that we could consider these ones for which there is evidence that they can effectively support student learning outcomes.
Taking into account the lack of empirical evidence on synchronous hybrid university learning and the question posed by Matta and Pavlia (2021), this case study collected and analysed student performance data from students that participated in the same course via different learning modes.The difference refers to following a significant part of the same course via different modes of participation: online versus face-to-face.Except for this difference, the remaining parts of the course were the same for both student groups.
The remainder of the paper is structured as follows: the Background section provides a view on relevant recent literature, while the Context section is providing a description of a) the teaching context and b) the course design.The following section revolves around the analysis of the collected student data and the results.Finally, the last section interprets the results, concludes on implications, presents limitations and suggests future research.

Background
The authors of this paper scanned the relevant research literature searching for similar research works like the one described herein following this method: for papers before year 2020, they used as point of reference the systematic review of Raes et al. (2020), whilst for papers with publication year 2020 or later they searched using appropriate keywords and the google scholar engine.Regarding the former mode, three papers were elicited from the systematic review of Raes et.al (2020) with selection criterion that they present and discuss some form of empirical research on synrchonous hybrid university education.From these three studies only one was focusing on student performance, the study of Lightner and Lightner-Laws (2016).This paper is an empirical study comparing course delivery modes: (online, remote and traditional) and investigates its impact on students grades.More specifically, it revolves around a blended course model for statistics and quantitative methods that allowed students to choose between three different course delivery modes: online, remote (via interactive television), and traditional course delivery.The study collected students grades along the three delivery modes and analysed them quantitively.It concluded that there was no signitificant difference on student academic achievement for the students that used this particular blended course model.They did noticed though in their preliminary analysis (i.e.before introducing this particular model) that there was a performance gap in the students achievement between the traditional face-to-face course delivery and the online coure delivery in their university.
The manual search of revevant and recent literature (2020 or later) revealed a few relevant articles which are briefly described in the remainder of this section.The purpose of the study of Kustiawan et al. (2021) was to determine the effect of hybrid education to the training on pedagogic competence of kindergarden educators.The authors used an experimental research method on the pedagogical competence variable based on the learning method factor (i.e.hybrid format and conventional format).The analysis of their results indicate that there are significant differences in the development of pedagogical competences of the two groups of educators in favour of the group that was trained using the hybrid format.In another study, Denton (2020) compared learning outcomes between two groups of students: those following a hybrid blended model and those following a traditional classroom model for the same course (a musculoskeletal spine assessment course) in a doctor of physical therapy program.The study examined differences in student safisfaction, student academic achievement, and competency skills.The results indicate no significant difference between the two modes with respect to student satisfaction as well as academic achievement.However, findings suggest a significant difference between the two groups with respect to competency skills in favour of the student group that followed the hybrid learning program.Finally, the study of Rhoads (2020) included a purposive sample of students following several undergraduate courses offered in the traditional and non-traditional programs of a private college in California over the course of five academic semesters.Statistical findings on student performance/learning ganins did not reveal a significant difference between course delivery modalities focusing on the final grade average.

Teaching context
The case stude involves a course in a renewable energy program for bachelor students offered by the Norwegian University of Science and Technology to students affiliated to different campuses: the main campus located in the city of Trondheim and peripheral campuses located in two different cities in Norway (namely, Ålesund and Gjøvik).The course revolves around a methodology to evaluate the environmental impacts of products and processes, with emphasis on energy systems.The first part of the course consists of the theoretical foundation of modelling and methods for evaluating the environmental impacts.The second part is focused on the use of a dedicated software for the impact assessment of case studies.The course is following the project-based learning paradigm, that is, the students are working in groups with a project that revolves around the use of the dedicated software.The tutor of the course is physically located in one of the peripheral campuses (in Gjøvik).The course is mandatory for the students based in the Gjøvik campus, while it is elective for the other two campuses (Ålesund, Trondheim).The student population registered in spring 2021 was derived from two campuses (Gjøvik, Trondheim).
The first two weeks of the course took place fully online for all students due to covid-19 restrictions.After teaching online for a period of two weeks, the covid-19 restrictions were lifted only in Gjøvik, so that the students could participate by being physically present inclass, whilst students in Trondheim continued to participate online.That is, the course followed a hybrid learning approach with both groups of students participating.For the students participating online (i.e. the students that were registered with the Trondheim campus) the lesson was streamed in real-time using a dedicated commercial platform provided by the university to the faculty members.These students could have the opportunity to ask questions orally by using their microphones or in a written format by using the chat of the webconference tool.This learning situation lasted from week 3 to week 8. From week 9 up until near the end of the semester the students were working in groups with their projects, and they received guidance and support from the tutor on an as-needed basis via online group meetings.Each group consisted of students that were registered in the same campus (i.e.either Trondheim campus or Gjøvik campus).Near the end of the semester, all participant students took the same final exam.The final exam was a summative assessment designed to assess to what extent the students have achieved the course outcomes.The assessment schema was following a grading scale from A to F, where grades were assigned using percentage points suggested by the university for all course following a 6-level grading scale (A: 89-100 points, B: 77-88 points, C: 65-76 points, D: 53-64 points, E: 41-52 points, F: 0-40 points).

Course design
The learning design of the course can be described using a Teaching-Research (T-R) nexus framework (Healey, 2005).The framework maps different activities along two dimensions: the content of learning (what students learn) and the process of learning (how students are involved in learning).The content of learning can either be focused on research results or research processes and skills, and students either be involved as participants or as an audience.Based on these two dimensions, four distinct approaches establishing the T-R nexus (figure 1) can be discerned: research-led, where students learn about current research; research-tutored, where students are engaged in research discussions; research-based, where students undertake research and inquiry; and research-oriented, where students learn about research methods and techniques.According to the Healy's T-R nexus framework, the course herein can be described as research-tutored, where students are engaged in research groups working with a dedicated software using a project-based learning approach.The fact that the course activities were mapped to this framework in this particular way had implications on its learning design, including the sequence of its main learning activities.For instance, the tutor was offering possibilities to the students to learn about the use of the dedicated software at the beginning of the course and not after the theory.Pedagogically speaking, the rationale is that the students would benefit from being involved in the research activity at the very early stage of the learning process.Student support on how to develop the project and guidance from the lectures was highly demanded in the first phase of the course to start the main student project.

Student data analysis and results
Regarding some basic student demographics, the majority of the students was bachelor students in their second year of study.They had almost the same age (around 22 years old).
A case study of hybrid learning in higher education using quantitative research design Regarding the distribution of the students sample in terms of gender, it was similar in both groups, that is equivalent to a ratio of 4 males to 3 females.The sample size was comprised of 27 students in Gjøvik and 7 students in Trondheim.
Figures 2a and 2b depict the distribution of the final grades for the course where the students could attend in presence (in Gjøvik) and where the students were attending online (those that were registered in Trondheim), respectively.An independent-samples t-test was conducted to compare the student performance between the two student groups (where the highest score of "A" was mapped to "6", score "B" to "5", …, score "F" to 1).The results (t(32) =0,41, p =0.680) indicated that there was no significant difference in the student performance scores for students participating online (M=5.11,SD=0.641) and students participating on-site in Gjøvik campus (M=5.00,SD=0.577).These results suggest that student performance in the course was not affected by the mode of participation (online or on-site).To calculate the effect size of this result, the Hedge's G coefficient was calculated.This particular coefficient was selected due to the fact that the two groups have almost similar standard deviations and different sizes, which are rather small.The Hegde's G coefficient had a value of 0,804 (>0,8) indicating a large effect size.In effect, this value means that the samples sizes were adequate (Sullivan & Feinn, 2012), even though they were rather small.

Conclusion and discussion
A recent review on hybrid teaching and learning revealed that only a limited number of studies have involved empirical research to assess differences in outcomes between students who attend online versus in-person courses.There is currently limited work that 1) provides insights on synchronous forms of hybrid education and 2) employs empirical research using a quasi-experimental design focusing on student performance data.Furthermore, there is a fundamental question posed by several researchers recenty asking what kind of pedagogical innovations related to online learning is worth considering in the context of higher education after the end of the pandemic.To compensate for this gap in the literature and to contribute to the debate around the aforementioned question, the paper presents a case study of hybrid teaching and learning in the context of higher education.Current research is rather inconclusive on whether the distance education aspect in the hybrid learning settings might by associated with lower student academic achievement and outcomes.In this case study, student performance data were collected and statistically analysed.
The findings of the statistical analysis herein indicate that the students' performance was not affected by the distance, since there was no significant difference between the two groups (with a large size effect of more than 0.80).One possible interpretation is that it might be due to the nature on this course (based on the use of the dedicated software) and also due to the constant involvement of the lecturer in providing support and guidance to the different student groups.Taking into account these findings, we conclude that hybrid synchronous learning might be one of the approaches that the community could consider after the end of the pandemic.
This study adds on to the literature on hybrid synchronous university learning, but not without limitations.Limitations of this study pertain to the fact that the size of the groups is relatively small, but the large size effect value indicates that it is appropriate.Furthermore, the quasiexperimental research design employed herein cannot determine causes (i.e., why did both student groups performed equally well in the final exams) as other variables, both known and unknown, could still affect the outcome.There are several external threats in this research and interviewing the students could eliminate their negative effects.Thus, more research is needed to establish a cause-and-effect relationship.Future research could focus on a mixed method approach that involves combined results of quantitative analysis of performance grades and qualitative analysis of students' interviews.In general, it has emered from reviewing the literature that there is a lack in studies taking into account in a mixed method research approach student performance data focusing on a comparison regarding learning gains in the different modes along the spectrum of blended learning.