Improving the quality of learning in a blended learning environment for first-year biology

Increased class sizes and rapid advancement of information technology has prompted institutions to move toward blended learning. The effectiveness of the instructional design of the blended learning courses has not been studied extensively in large classes. This study aims to interrogate the effectiveness of the instructional design of a large first year biology class with the aim of providing the most effective blend for all students but focusing on the at-risk and murky middle students. This was done by firstly investigating which of the different learning opportunities contributed most to the success of the students and secondly by investigating student engagement with the learning opportunities provided to them. The results show that small, face-to-face tutorial classes and online formative assessments contributed the most to student success. The results also show that at-risk and murky middle students tend to make use of learning opportunities less after the first summative assessment, possibly putting them at risk of failing.


Introduction
It is widely acknowledged that higher education worldwide faces the challenge of growing student numbers, increased diversity of students and reduced resources (Cash, Letargo, Graether, & Jacobs, 2017).This is particularly acute in South Africa where students numbers have almost doubled since 1994 but the per capita funding has decreased (Badat, 2014).The increase in class size and the rapid advancement of information technology have prompted institutions to move to blended learning.Although blended learning has been investigated (Gleadow, Macfarlan, & Honeydew, 2015) the effectiveness of the blend, especially in large classes, has not been studied extensively.In order to study the effectiveness of the blend one should investigate student engagement with the learning opportunities provided to them and the impact that the different activities have on student performance.Thus, student engagement can be used as a theoretical lens to study the instructional design of the course.

Theoretical framework
In the early 1980's student engagement was defined as a student's "time-on-task" with educational activities.The current definition highlights the obligation of both the institution and students to take responsibility for engagement.Zepke & Leach (2010) suggest that engagement can act as a proxy for quality in education.Thus facilitating student learning is an institutional responsibility which is acted out by the lecturers involved in instructional design of the courses they present.Siemens and Long (2011) defined learning analytics (LA) as the use of learner produced intelligent data and analysis models to uncover information and to predict and advise on learning.With LA, student learning, academic progress and teaching practice can be analyzed which, in turn, can be used to inform instructional design.
In principle each cohort of students can be divided into three groups; students that are likely to pass with relative ease, students that are likely to fail the course in the absence of substantial interventions (termed at-risk) and students for whom the prediction of outcomes is difficult.The Student Success Collaborative in the USA (Student Success Collaborative, 2014) coined the term the "murky middle" (MM) for these students for whom the academic outcome is difficult to predict.It was demonstrated that a lot of resources are allocated to the at-risk group and comparatively little to the MM students, while an investment of resources to support this group of students will deliver a high return on investment (Student Success Collaborative, 2014).In this study we want to analyse the engagement patterns of each of the three groups in anticipation that each group could potentially benefit from a different combination of learning opportunities.We are also particularly interested in the engagement patterns of the MM and the blend of learning opportunities that could promote their chances of success.

Rationale
Many higher education institutions have moved toward a blended learning approach for teaching and learning, providing learning opportunities both in and outside of the classroom.Given the cost of higher education it is important to monitor the effectiveness and uptake of learning opportunities provided to the students for two reasons; firstly the human resource implications of its setup and maintenance and secondly, the cost involved for students to access online resources.The effectiveness of the instructional design of the course is seldom investigated.One would ideally like to provide the most students with the best possible interventions given limited resources.Thus using the premise of the MM, the study will investigate which learning opportunities are used most by the at-risk and MM students and which of these have the biggest impact on success for these two groups of students.

Aim of the research
The aim of this research is to interrogate the design of the blended learning environment to ensure that it is optimal to advance student success.Thus, the research questions that we would like to explore are: RQ 1. Which of the activities in the blended learning environment are associated most strongly with success for each of the three groups?RQ 2. What is the difference in the uptake of learning opportunities by the three different groups of students?

Methodology
For this study students were categorized based on their performance in the Grade 12 final examination for the subject physical sciences.The at-risk group were classified as students with a physical sciences mark below 72%, the MM as students with a mark between 72% and 81% and the LTP students had a mark of 81% or more (Kritzinger, Lemmens and Potgieter, under review).The first objective of this study was to determine if there was a difference in engagement patterns between the at-risk, MM and likely-to-pass (LTP) students.Previously it was shown that the summative assessments (semester tests 1 and 2) were the best predictors of success for the course (Kritzinger, Lemmens and Potgieter, under review).The summative assessments were spaced at regular intervals in the timeline of the course, which allowed for the data to be analysed to detect a shift in the use of learning opportunities over time.Chi-square Automatic Interaction Detector (CHAID) was chosen as the method of analysis to determine which activity contributed the most to success overall.This analysis was performed for the total student population as well as the Improving the quality of learning in a blended learning environment for first-year biology three subgroups with the first and second summative assessment and the exam mark as outcome variables, respectively.Success is defined as either a fail or pass mark for the respective summative assessments.Analysis of Variance (ANOVA) and Tukey post hoc tests were chosen as the methods of analysis to determine if there was any difference in use of the learning opportunities by the subgroups.

Context of the module
Molecular and Cell Biology (MLB 111) is a first year module in the Faculty of Natural and Agricultural Sciences at the University of Pretoria, South Africa.A large number of students annually enroll for this course (ca.1500 students).The module is a blended learning course with both face-to-face and online components.Data for this study were obtained from the students enrolled during the first semester of 2015 for MLB 111.A total of 1084 student records were used for the study.The sample comprised 730 females (67%) and 354 males.The engagement data for the study were obtained from the Grade Centre of the Learning Management System (LMS).
The instructional design for the course is summarised in the activities described below.The description below indicates how activity engagement were recorded and provides a motivation for inclusion of these activities in the instructional design.

Compulsory activities
There were three compulsory activities that students had to participate in to be granted access to the examination at the end of the semester; namely class participation, tutorials and Connect quizzes.
A. Class attendance and participation in class have been shown to correlate well with achievement as reported in the meta-analysis by Credé, Roch, and Kieszczynka (2010) and were thus included as a compulsory activity for this module.Students had to attend four theory classes a week during which at least one peer learning activity was conducted and recorded.Responses were logged using clickers.B. Face-to-face tutorials in small classes have been part of the course design for a number of years and are considered essential for the success of the students.Tutorials are presented every week and are led by senior post-graduate students that were trained beforehand by the lecturers.The tutorials are structured around peer learning activities (Lasry, Mazur, & Watkins, 2008) and students participated in the tutorials by using clickers.C. Online testing has become commonplace in the last few years as part of the blended learning initiative at the university.A review of the literature by Gikandi, Morrow and Davis (2011) shows that it is mostly used for immediate feedback, engagement with critical learning processes and promoting equitable education by addressing diverse student needs and has a positive effect on student engagement (Angus & Watson, 2009).Online quizzes were used as low stakes tests at the end of each study unit.

Voluntary activities
Some optional learning opportunities were also provided.Lieu, Wong, Asefirad and Shaffer (2017) showed that students do not read their textbooks unless an incentive is provided, and that pre-reading in first year biology has been correlated with improved exam performance.Learnsmart is an adaptive learning system that is part of the online suite of the prescribed textbook for the course.Learnsmart was introduced as a voluntary activity for the module with the intention to help students prepare for the upcoming lectures and to prompt them to read the textbook.In the case of MLB 111, the student were given time before the revelant section of the work started to complete the Learnsmart assignments in order for them to prepare before the class.Virtual tutorials were conducted outside of class time by senior post-graduate students who also presented face-to-face tutorial classes.Students could participate on a voluntary basis and participation was logged into the LMS.

Prelminary findings
The analysis of engagement data generated a rich description of differences between subgroups as well as shifts in engagement during the course of the semester.These finding are reported in Table 1 and Figure 1 and only the most important aspects are described below.
A CHAID analysis of the complete dataset using the exam as outcome variable showed that the biggest contribution to success for all students was participation and performance in tutorials.The results show that the likelihood of students to pass the exam if they had below 82% for their tutorials was 23% (node 1).These students' chances increased to 37% if they also participated in the online quizzes (labelled Online tests) and scored more than 81% in these tests, but dropped to 17% if they achieved less than 81% average for the online quizzes.The second node showed that if students obtained between 82% and 86% for the tutorials they had a likelihood of 49% to pass the exam, however they increased their likelihood of passing the exam to 65% if they gained above 81% for the online quizzes and decreased their likelihood to a mere 33% if they did not get 81% for the online quizzes.Nodes three and four showed that students who had 86% and more for their tutorial classes had a likelihood of 76% or more of passing the exam.Detailed CHAID analysis (not shown here) for each group of students (at-risk, MM and LTP) using the summative assessments as outcome variables was also performed and confirmed the results of the CHAID showing that tutorials were the most important predictor of success (RQ 1).The analysis of the MM subgroup showed that over the duration of the course different learning opportunities contributed differently to student success.
Improving the quality of learning in a blended learning environment for first-year biology Following the CHAID analysis we investigated the uptake of learning opportunities by the three subgroups before semester test 1 (period 1) and between semester tests 1 and 2 (period 2) (Table 1).Table 1 lists the three compulsory activities with a statistical analysis of the uptake of these activities by the three subgroups for the two time periods (voluntary activities not included).
In pairwise comparisons there were a significant difference between all groups for all three compulsory activities, expect for class participation where the MM resembled the LTP group in period one.In period two a shift occurred and all three groups participated less, however two shifts were notable; for the peer led tutorial classes the significant difference present between group one (at-risk) and two (MM), in period one, was not present in period two anymore.For class participation the resemblance between group two (MM) and three (LTP) disappeared.Both shifts are reseason for concern.Uptake of the voluntary activities was low overall and was not included in Table 1 due to space constraints.online quizzes and tutorial classes contributed to the success of the MM students (Fig 1).The research suggests that the most important element of the course design should be well planned face-to-face interactions in smaller groups with well-trained tutors.Regular, formative assessments also helps students, especially the MM, as it provides real time feedback on their learning throughout the semester.The findings of the study has implications for instructional design of this course.Since voluntary activities did not contribute significantly to the outcomes for any of the subgroups, scarce resources can be directed elsewhere.

Figure 1 :
Figure 1: CHAID with exam as outcome variable using the complete data set.