Students experience of blended teaching formats in pre-calculus courses

This work aims to investigate the difference (if any) between two blended teaching formats in a pre-calculus course for undergraduate engineering students. In 2022 students could choose among two settings for the preparatory course: a face-to-face and an online blended format, both based on a Pre-Calculus MOOC. At the very end of the course, we collected data from a survey to which 129 students answered. The analysis of the responses shows that the main differences between face-to-face and online blended formats concern intrinsic characteristics of the two settings. On the other hand, the data show that an accurate planning of the online format allows students’ engagement even more than in face-to-face blended setting. Moreover, the findings shed a light on the potentiality of online learning in making the transition from high school to university smoother.


Introduction
One of the main obstacles for students enrolled in first year STEM programs is the Calculus course. Different researchers in Mathematics Education (Tall, 2004) suggest that the main difficulties are created by a different approach to mathematics with respect to the one seen in high school, since there is a shift of the focus from procedural aspects to conceptual understanding (Clark and Lovric, 2008). Moreover, learning resources are perceived in a different way (Kock, Brunetto & Pepin, 2019) and students are supposed to study and work with more independence with respect to high school. In order to smoothen the transition between secondary and tertiary education, universities usually offer preparatory courses to first-year students. As Gamer and Gamer (2001) pointed out, student-directed learning promotes conceptual learning more effectively if compared to teacher-directed approaches. One possibility to encourage student directed learning is given by blended learning formats. In this context, Massive Open Online Course (MOOC) is a significant resource that can be exploited in a blended course: students have to watch videos making sense of their content without the teachers' guidance; they have the possibility to self-evaluate themselves with exercises and to resort online forums and other sources clarifying their doubts. All this engages students in different mathematical activities that generate a personal production of meanings and mathematical knowledge.
Blended learning formats recently became even more preeminent and relevant, since universities are trying to adapt to the "new normal" after the COVID-19 pandemic forced them to do remote teaching (Bakker et al., 2021). It is well documented that emergency remote teaching and online learning are quite different (Hodges et al., 2020), but the exceptional situation of 2020 created room for new teaching experimentations with online learning, since both students and teachers gained familiarity with new technological tools as services for web conferences (as Zoom, Webex, Microsoft Teams), software for sharing boards and contents (as Jamboard, padlet, Google Drive) and so on.
In this work we want to investigate how students experience online and face-to-face teaching in terms of quality and usefulness. We will focus on a pre-calculus preparatory course, delivered in 2022 for first year students at Politecnico di Milano, in which students were given the possibility to attend face-to-face (f2f) classes or online classes.

Methodology
The preparatory math course is delivered every year at the Politecnico di Milano before the beginning of the first semester. Usually, the course is structured in two parts: a MOOC that students are invited to attend during the summer and eight classes delivered in the two weeks before the beginning of the academic year. The pre-calculus MOOC is delivered on the POK platform (www.pok.polimi.it), where students can watch videos about theory explanations and resolutions of exercises, test their basic knowledge in mathematics through quizzes and interact with other students and tutors through a forum. The MOOC course is structured in six chapters: arithmetic, algebra, geometry, logics, functions, probability. The eight classes of four hours each, follow the syllabus of the MOOC course.
In the 2022-23 academic year the attendance part of the preparatory course was organized in two different formats, that we will call face-to-face (f2f) and online blended classroom. In the face-to-face format the eight classes were delivered in a traditional classroom setting, that is tutors and students met in a physical learning space. Each class started with a "warm-up" activity that was carried out in small groups, followed by the correction of the tasks and the discussion of other examples and exercises by the tutor. At the end of the class, a formative assessment test was left to students in order to solidify their knowledge and skills. In the online format there was not a physical room: the tutor and the students were interacting using an online platform (Webex). The online class started with asynchronous activities, in the sense that students were not connected live with the tutor. Students received instructions about which part of the pre-calculus MOOC had to be studied (videos and quizzes) and, afterwards, they were prompted to solve the task of the "warm-up" activity, posting the solutions and questions on the MOOC forum. Later, there was a synchronous part of the class, delivered through the online platform, in which the tutor discussed the solution of the warm-up activity, addressed the questions, and deepened the topic. At the end of this part students were given some time to answer the formative-assessment test (the same the faceto-face students were doing). We underline that the two formats are equivalent in the sense that they refer to the same topics and use the same educational resources, but they differ in terms of educational environment and timing. The MOOC was available for both groups, students received instructions by email when they enrolled to the preparatory course and they were reminded about all educational resources daily by verbal communication by the tutor.
The preparatory course, which was not mandatory, was attended by 380 students, on average. Among them, almost the 15% of students attended the online format, held by one single tutor, while the huge majority attended the f2f format held by 7 tutors in 7 physical classrooms. More precisely, two classes had about 20 students while the other 5 had about 60-70 students. At the end of the course, namely at the end of day 8, students were asked to respond to a satisfaction survey voluntarily. We collected 129 responses (about the 40% of the students who attended the last lesson). The survey is composed of 4 parts, we focus on the questions about the course, and open comments and suggestions. The former is formulated as three (0-2) likert-scale questions. Table 1 reports the 16 items (i1, i2, i3, …) to which students answered using 0-"disagree" / "not at all", 1-"agree"/ "enough", 2-"strongly agree"/ "fully". We classified the 16 items a-posteriori according to the two dimensions we wanted to investigate: D1) the quality of the course in terms of tutor practice (i12-i16), topics (i5), and students' interaction with each other and with the tutor (i10, i11), and D2) the usefulness, in terms of overcoming the transition issues, such as fostering conceptual-mathematics (i6-i9), prompt autonomy (i1-i4).

Quality Usefulness
i5 -During the classes we revised the most important topics we studied in high school.
i10 -During the classes, we worked on our own.
i11 -During the classes, we discussed and cooperated.
i12 -During the classes, the tutor fostered students' collaboration.
i13 -During the classes, the tutor involved students.
i14 -During the classes, the tutor's explanations were clear.
i15 -During the classes, the tutor precisely answered our questions.
i16 -During the classes, the tutor positively influenced the independent study.
i1 -Before the beginning of the refresher course I attended the pre-calculus MOOC.
i2 -During the refresher course I attended the pre-calculus MOOC.
i3 -During the refresher course I was involved in the Discussion forum on the pre-calculus MOOC.
i4 -During the refresher course, I did the formative-assessment quiz.
i6 -During the classes, I found a new language to do mathematics.
i7 -During the classes, I found new mathematics topics.
i8 -During the classes, we focused a lot on formulas and algorithms to solve exercises.
i9 -During the classes, we analyzed definitions, theorems, and general properties.
In this work we aim at addressing the following research question: "How do students experience online and face-to-face teaching in terms of quality and usefulness?". To that end, the students' responses were analyzed both quantitatively and qualitatively. For each item, we resort to descriptive statistics (mean, standard deviation, and histograms) distinguishing between two groups of students, namely the f2f students' responses (102) and online students' responses (27). Moreover, we employ the one-way analysis of variance (ANOVA) to determine if data from the two groups have a common mean. We recall that the ANOVA helps determine if the mean of the two (or more) groups are all the same. The null hypothesis H0 is that, for each item, the means are the same (Wu & Hamada, 2000). Once the test is performed, the H0 is rejected if the returned p-value is less than a specific threshold (0.05 by default). Indeed, the p-value consists of the probability of making a mistake rejecting the H0.
Finally, we analyzed the students' comments looking for the indicators of quality and usefulness, to deepen our analysis and to better interpret the previous results. It is worth noticing that among 127 students, only 90 provided a comment. More precisely, almost all the online students (23 out of 27) left a comment, whilst only 67 (out of 102) of the f2f students did.

Data analysis and discussion
In this section we report data analysis of the 16 items related to the quality and usefulness, enriched by some of the students' comments. Table 2 summarizes the results of the statistical analysis. We observe that there are only two questions (i2, i14) whose responses are statistically different (p-value < 0.05, marked with * in Table 2). Nevertheless, some questions (i5, i10, i11) have slightly different results with respect to the two groups (0.05 < p-value < 0.15, marked ** in Table 2). We observe that the most relevant difference between online and f2f students concerns the use of online resources, i.e., the Pre-Calculus MOOC, during the two weeks of the preparatory course (see i2 in Table 2). More precisely, the percentage of f2f students who declared not to use the MOOC is greater than four-times the percentage of online students; on the other hand, more than 60% of the online students declared to study the MOOC resources during the course (see Figure 1). We stress that such a difference depends on the structure of the online course, in which students were recalled to study the videos and to do activities on MOOC on a daily basis. It is important to underline that the MOOC was available to all the students and both formats were designed according to the MOOC structure. This allows us to claim that students resort to online resources only if they are directly instructed about it. Furthermore, a student of the online course commented: "I really appreciated the forum. It really helped me to see different strategies to solve some of the exercises [warm-up tasks]". This is an indicator that the guided use of the online resources also allows students to exploit the potentiality of new tools, such as the forum. The item i10 regards the self-regulation of the students, namely if they perceived they worked on their own during the classes. The two groups are slightly different (1.33 for online and 1.12 for f2f) because online students reported more work on their own than f2f students. This could be explained by the fact that half of the online class was explicitly devoted to asynchronous and autonomous activities (such as the warm-up exercises). To that end, we report the following comment by an online student: "[I appreciated] the possibility of managing on my own the study moments and reviewing the scheduled topics". The comment underlines two aspects: the autonomy dimension and the scheduled topics. The student was able to self-organize their study thanks to the shared planning of the classes. Therefore, we argue that the majority of first-year students may lack autonomy when they are not guided as they were used to in high school.
The item i14, whose answers are statistically different, concerns the clarity of tutor explanations. However, from an educational point of view the difference of the two means (1.57 for online and 1.80 for f2f) are not relevant, because the values are high. We infer that the different results depend more on the tutor-factor than on the format-factor. Indeed, looking at the f2f format, the 7 tutors were perceived (by students) positively in different ways, namely the means associated to each tutor ranges from 1.5 to 2. Moreover, students of both groups commented with positive feedback about tutors, for instance a f2f student reported "I do appreciate the clarity of the tutor explanation" and an online student wrote: "the tutor is very competent, clear and precise".
We underline that i14 is one of the 5 questions related to the tutor's influence on the course, so that any possible differences between the two groups' answers may be affected by the tutor-factor. However, the means of those questions are not so different. In particular, we focus on i12 and i13 which aimed at investigating tutor's influence in fostering students' collaboration and involving them during classes, respectively. The means of the online classes (1.74 for i12 and 1.78 for i13) are slightly higher than the f2f classes (1.66 for i12 and 1.64 for i13). Despite the common idea that online classes are not a good setting for engaging students, this result shows that students may have an active role even in online classes. Another result that supports such a claim comes from the answers to item i11, in which students were asked if in classes they discussed and cooperated. The two groups' means are 1.81 (for online) and 1.62 (for f2f) with a slightly statistical difference (p-value=0.11). Moreover, the analysis of the students' comments shows that only one of the f2f students provided a comment about the students' collaboration: "I liked to work with my course mates to solve the tasks", while online students gave more comments in such direction, for instance: "I really appreciated the possibility of learning not only from the tutor but from my course mates' ideas and advices" and "I liked the continuous discussion and interaction among mates". Somehow, students felt they discussed and interacted more online than students did in the classroom.

Conclusion
The spreading of innovative learning and teaching experiences raises questions about their efficacy and impact with respect to "traditional" experiences. In this work, we do not dwell on the definitions of such terms, but we are interested in investigating how students perceive differences between two formats of the same course. To that end, we focus on a preparatory course for first-year students in a STEM program, addressing the following research question: how students perceive online and f2f teaching in terms of usefulness and quality.
The usefulness and quality dimensions are meant in the frame of transition between high school and university mathematics.
Overall, the data analysis shows that no perceived differences were reported by the students for the two formats, which were perceived both as good courses. This was also supported by a final question in the survey: "do you recommend this course to future first-year students?", to which none of the students answered "no". Nevertheless, we underline that the few differences can be explained according to the unavoidable characteristics of the course, such as the tutors who taught the course, and the different instructions the students received.
Concerning the latter aspect, we stress that students need to be guided step by step in the use of educational resources. Despite having the same topics and tasks of the f2f format, the online format was not just a migration to a remote teaching method, but it was carefully planned according to the potentiality and constraints of the online environment. In this way also online students were able to interact, discuss and cooperate gaining autonomy and being guided during the transition to higher education.