Peruse this! Use of an educational social platform for a Global Entrepreneurship flipped class

The COVID-19 pandemic made all of our universities to move to a full-online teaching scheme this March. Online teaching presents different drawbacks compared to classroom teaching, but probably the main one is the difficulty to keep the interactivity with the students throughout the online sessions. This is why a flipped classroom scheme becomes even more relevant in the new normal. A key aspect for a flipped class is that the information transfer, traditionally performed through lectures, is moved out of the classroom to different types of pre-class assignments. Students are expected to fulfill those assignments before the class allowing the instructor to engage in more interactive activities with the students in the classroom. Nevertheless, getting students to read the proposed contents before the class is a very important problem in higher education. In this work, we show the results of the implementation of pre-class reading assignments in a Global Entrepreneurship course through an educational social platform that allows students to discuss the reading online with their classmates: PERUSALL. We use the platform to understand how students are reading and understanding the different materials, their reading behaviors and how the instructor can take advantage of all that information. Considering the type of group (50 students, 65% of them international students from exchange programmes we have also performed a survey to assess the platform and evaluate the perception of the students of their learning process.


Introduction
The key principle of a flipped classroom model is to give the opportunity to students to have access to the instructor when they need it most.Moving activities like problem solving or group assignments to class improves student learning as it provides them the opportunity to actively engage with the instructor and each other (Herreid and Schiller, 2013).But by doing that, the instructor delivering the lecture must be moved out of the classroom to a pre-class assignment that students are expected to complete before coming to the classroom session.Different studies have shown the advantages this method presents.(Schwartz and Bransford, 1998;Marcell, 2008).
Perusall is an online, educational social platform designed to promote high pre-class reading compliance, engagement, and conceptual understanding (Miller et al., 2018).Once the instructor creates the online course and the reading assignments, students can asynchronously annotate the readings by posting, commenting and questioning the text as in a social network.
Perusall also provides the instructor constant feedback on how the students are engaging with the reading assignments and, through an algorithm, creates the so-called confusion report: an automatically generated report which summarizes the top areas of student confusion.
Nowadays Perusall is a platform serving more than 500,000 students around the Globe from more than 2,000 educational institutions in more that 75 countries.One of the most interesting features of Perusall is that it is suitable for very different courses: from its origin for Introductory Physics (Miller et al., 2018) to more recent examples in Property Law (McFarlin, 2020), Biochemistry and Molecular Biology (Procko et al., 2020) or Global Entrepreneurship, as explained in this paper.This transversal need for new reading tools is crucial given the essential role of reading in shaping knowledge in higher education which makes more important than ever the need of new approaches to engage our students in reading practice (Nguyen and Henderson, 2020).

Teaching experience using the platform 2.1.1. The course
The course chosen to implement Perusall was Global Entrepreneurship, a course from the fourth year of the degree in International Business at ESIC Business & Marketing School (Valencia campus).The course is taught in English, had 50 students enrolled with 35% of national students and 65% students from different countries and studying in the campus in Valencia , Spain, most of them from theErasmus Program.This was the third year the course is available for enrollment.
The course is structured in 4 hours per week class distributed in two 90 minutes sessions and one 60-minute session every week during the first semester.The contents of the course focus on introducing the students to the entrepreneurial ecosystem from a global perspective.The students also learn how to define a business model, the financial structure of start-ups and the different agents present in the entrepreneurial ecosystem.Strategic internationalization for start-ups and global market positioning is also studied under this course.The main learning outcomes are to recognize the main characteristics of entrepreneurship and start-ups, to identify key concepts of the business model generation, to use relevant data in the financial analysis of start-ups, to recognize and select the main financial sources for start-ups, to cite the key aspects of entrepreneurial related legislation, to discuss the convenience of the different types of accelerators and incubators, to investigate the internationalization strategy of different start-ups and to recognize the positioning strategies of start-ups when operating globally.
The assignment criteria are based on the importance of group activities and practical implementation of the theoretical concepts developed among the course.Hence the written test accounts only for 30% of the final grade, whereas the group project weighs 60% and the active participation 10% of the final grade.It is in this 10% of active participation where the grades from Perusall have been used.Five different assigments have been scheduled in the course in Perusall.Table 1 shows details of the assigments.

Font: Prepared by the authors
As seen in Table 1, different types of reading materials have been used through the course to make the students work on different topics.The materials were made available in twoweek intervals and the students had one week to read and comment them.

Descriptive information available to the instructor
The Perusall platform gives comprehensive information on the performance of the students and their degree of implication on the reading task.Table 2 shows the main statistical indicators given by the platform.As shown in Table 2, the instructor gets a full report of the activity of the students for a given assignment.The instructor can monitor each one of the individual comments and questions made in the different parts of the text (as it appears highlighted and linked to the comment) or just the reports generated by the platform.
Figure 1 shows and example of the descriptive information obtained for the activity on a particular assignment.The instructor gets detailed information about the most commented and voted comments, the most active students, the time map of when the students have been working on the assignment and the pages that have drawn more attention.
All these details give the instructor very useful information in real time on how the students are working on the contents and can predict their level of readiness for the classroom session regarding the contents included in the assignment: the contents that have drawn more attention are clear (comments from the students, votes and page views) and the instructors gets a time map so they objectively know whether the students have read the materials.

Other important information available to the instructor
In addition to that information, the instructor gets another important information: the confusion report.Perusall's algorithm analyzes student's comments and produce a report about those parts of the reading material that have given more problems to the students.This is a very useful tool, as it gives the instructor the information needed to prepare the classroom session to focus on those contents.Figure 2 shows and example of a confusion report.In this case, the majority of students showed problems regarding three topics of the scientific paper.Hence, the classroom session can focus on those contents and it is not necessary to spend extra time on other contents that have already being collectively commented and have raised no important questions to the class.
Perusall can also analyze the students' comments and assign a grade to each student depending on the quality, depth and interest generated of the comments provided.The rubric can be adjusted for each course and the instructor can manually change the grade given to one of the different comments.Figure 3 shows an example of the 0-2 scale for the comments of the students.

Learning experience using the platform
In order to assess the learning experience for the students, a questionnaire was sent to the students once the semester was finished.The questionnaire was answered by 50% of the students, from which 68% where females and also 68% of the students who answered were Erasmus students.All of them used Perusall for the first time.
First of all, the user experience for the student was analyzed using a zero to ten scale.76% of the students rated 8 or higher the process for joining Perusall, 88% rated 8 or higher the easiness to access the materials and 60% rated 8 or higher the process for adding comments and questions to the assignments.
Regarding the learning experience, 72% rated 8 or higher the usefulness of Perusall for reading and assimilating new contents, 62% of the students rated 8 or higher the usefulness of viewing and commenting other's questions and comments and 68% of the students rated the usefulness of Perusall for learning 8 or higher.Finally, 80% of the students would like to use Perusall again and 88% of them rated Perusall overall with 7 or a higher score.Figure 4 shows the answers of the students to the questionnaire.
Finally, when asked for giving further comments, the students said: "It was a nice way to see the thoughts of other students about a subject and it brought me more knowledge!""It is one of the best learning tools I have ever used" "I did like Perusall, because more students would read the articles uploaded as it was mandatory.However, I do not think that the grading algorithm is calibrated that well.Some comments made were just .... and irritated when reading and also the grades did not completely reflect the work done by the students.""Great App, really brings the organic way of functioning and interaction between fellow students giving them the ability to express their thoughts and participate in a useful discussion.""The marks were inappropriate sometimes".

Conclusions
When implementing a flipped class the instructor needs to move part of the information transfer out of the classroom to different types of pre-class assignments.In order to tackle the problems arising from students not fulfilling those reading assignments before the class, Peruse this! Use of an educational social platform for a Global Entrepreneurship flipped class Editorial Universitat Politècnica de València an educational social platform has been used in a Global Entrepreneurship course from the first semester of the fourth year of the degree in International Business at ESIC Business & Marketing School (Valencia campus).The course was taught in English, had 50 students enrolled with 35% of national students and 65% students from different countries and studying in our Campus under de Erasmus Program.This was the third year the course is available for enrollment but the first year using the Perusall platform.
For the instructor, it has been a really good tool as Perusall has given them a way to understand how students are reading and understanding the different materials.The reading behaviors of the students and their main doubts and concerns allowed the instructor to use all that information to prepare the classroom sessions in a more effective way .It was not necessary to go over the concepts that were clear and the instructor could focus on those aspects that had brought a greater debate on the platform.The algorithm for the confusion report was useful to prepare new materials to work on those matters that were not understood and the marking algorithm was a good tool for evaluating the implication of the students with the course.
For the students, Perusall has been a very good tool also.From the user experience point of view, Perusall is an easy to use platform and the students had no problem joining the course and starting using all the different functionalities.From an educational perspective, the students found useful and interesting being able to interact with the rest of the students.The "social reading" was viewed as a powerful tool to know other's point of view.It is interesting to see that they also recognized the up side on getting a way to effectively know who was reading the pre-assignment materials.The comments through the course were also consistent with those expressed in the questionnaire.
It is also interesting to see that the students do not only complain for the grades given by the instructor but also to those given by the algorithm.This clearly demonstrate that there is not a proper way to assign a grade under the student's point of view.The teaching and the learning experience have been highly positive.The instructors will use Perusall once again next course.

Font:
Fig. 1 Detail of the information given to the instructor for a given assignment in Perusall with the (a) most upvoted annotations, (b) most active students, (c) annotation submission time map and (d) page view report Font: The author's Perusall course

Fig. 2
Fig.2 Detail of the confusion report for a given assignment in PerusallFont: The author's Perusall course