Use of Software Tools for Real-time Monitoring of Learning Processes : Application to Compilers subject

The effective implementation of the Higher European Education Area has meant a change regarding the focus of the learning process, being now the student at its very center. This shift of focus requires a strong involvement and fluent communication between teachers and students to succeed. Considering the difficulties associated to motivate students to take a more active role in the learning process, we explore how the use of a software tool can help both actors to improve the learning experience. We present a tool that can help students to obtain instantaneous feedback with respect to their progress in the subject as well as providing teachers with useful information about the evolution of knowledge acquisition with respect to each of the subject areas. We compare the performance achieved by students in two academic years: results show an improvement in overall performance which, after observing graphs provided by our tool, can be associated to an increase in students interest in the subject.


Introduction and Motivation
During the last decade, and thanks to the implementation of the Higher European Education Area, there has been a shift in terms of focus in the learning process being the student at its center.As part of this change, some actions have been undertaken such as decreasing the number of students in each class to ease student-teacher interaction as well as changing the balance between theoretical and practical sessions.As a result of these changes, teachers have also had to adapt the way students are evaluated towards a more continuous evaluation.
The continuous proposal and evaluation of learning activities is a high time consuming task, which also needs of strong student motivation to take part in the different proposed activities.As students have to divide their efforts among the different subjects, it is difficult to pursue them to work continuously on each of them as they tend to focus on the most inmediate assignment deadline.This, along the low attendance to classroom activities, Bukoye (2017), makes it difficult teachers to have continuous information about evolution of the learning process.This only allows us teachers to correct potential knowledge gaps in specific moments in the semester, mainly as a result of evaluation activities.
To overcome this, some alternatives have been proposed such as rewarding the students for their attendance to classroom activities Bukoye (2017), involving students in the evaluation Valero (2010) Conde ( 2017) Harland (2017) or, more recently, to include gamification in the learning process Kapp (2012) Su (2015) Mauricio (2017).
As a use-case, we show how we have adapted the subject we teach to the new learning process focus Valero (2010) García-Peñalvo (2014).Our subject is part of a Computer Sciences degree and requires students to learn the basics of compiler building theories.One big part of the subject involves students to build their own compiler; this task is supported by explanations during theorical and seminar activities.As a result of our experience over years, we have observed the following problems associated to the practical part of the subject: 1) low students attendance and performance and 2) big performance gap between practicum exam and practicum assignments.Students work in pairs and are evaluated individually del Canto (2015) at the end of the semester to verify that each of them have actually taking part in the practicum assignment.We associate differences between assignment and exam marks to individual students taking charge of a group assignment, excessive help among students and practicum copying, as all students had the same assignment.These reasons might come as a result of low student motivation in the subject, which can be caused by the appearance of difficulties in the learning process that the student is not able to solve and, as they are not known by the teacher, they are difficult to solve.
We study in this paper the role of that a software tool can have to support students learning process.The tool proposed incorporates evaluation and monitoring capabilities so the teacher can know in real-time the level of assessment of the different concepts at a glance without requiring additional information to students.We study the benefits associated to the use of the tool by comparing students performance over two consecutive academic years.

Learning Process Monitoring Tool
We present in this section our learning process support tool.To ease readers understanding, we use as example a real assignment from our subject.The task students have to undertake is to add new functionalities over a basic compiler.

Assignment Preparation
At the beginning of the semester, the teachers define the different additional functionalities that will be incorporated to the compiler.For each functionality, several variations are explored aiming to cover all possible different scenarios that the compiler might face (an example is shown in Fig. 1).Each variation is given a difficulty score by the teachers, as a result of both personal experience and students observation during the previous academic year.To assign the funcionaltiy variations each student has to work with, we use the assignment preparation tool.This tool works under the following rules: 1) all assignments have to be different, 2) all the assignments should have a similar difficulty, and 3) all assignment should have one variation from each functionality.This is achieved by the use of a backtracking algorithm Priestley (1994).The teacher can incorporate additional constraints such as imposing compulsory functionality variation to tackle.The use of this tool naturally prevents students from copying, as none of them has all the same functionality variations to add.

Assignment Evaluation
In order to assess that these functionalities have been correctly incorporated by students, two different types of tests are designed.Public tests aim to assess if the student has acquired the basic knowledges of the subject whereas private ones are focused to explore if students have gone beyond the minimum requirements in order to build a more robust solution.Private tests do not require additional theorical explanations but a careful thought about the solution that is being prepared.As an example, a public test will check if the power operation between integers provide the expected results whereas the private one will explore whether the combination of some variable types is allowed (i.e., the compiler should not allow the power between an integer and a character).The content of the public tests is known by the students in advance, and they should be all overcome in order to pass the subject.Private tests are not known by the students and they are used to modulate the mark between 5 and 10.
Students can upload their solution to the assignment using a dedicated website.Every time a new delivery is uploaded, the assignment evaluation tool checks whether the tests associated to each of the student-specific functionality variations are overcome.This tool provides instantaneous feedback to the student by generating a report summarizing the level of assessment of the different proposed tasks.This is an evolution over what was done in previous years, as students have to ask the teacher to test their solution which might delay the obtention of the feedback as well as having information about the progress of the learning activities.
For the case of private tests, we only inform students about the percentage of private tests that have been overcome as a way to encourage them to try harder in order to achieve the maximum mark, inspired by gamification theories.By doing this, we aim to transform knowledge acquisition into a discovering experience that can motivate students to gain interest in the subject, as they are 'battling' against the unkwnown.

Performance monitoring
As students upload the solutions to their assignment, the performance monitoring tool also generates a text file that, conveniently processed by common software tools, can provide teachers with useful information with respect to the evolution learning process.For instance, we can easily obtain the following information: 1) number of deliveries and its evolution over time per group or class, 2) percentage of functionalities, variations and test that have been overcome by each group or class, and its time evolution.By this information, teachers can have information about the level of interest of a group (number of deliveries) or difficulties associated to specific functionalities (low number of tests overcome with respect to the number of deliveries).As the system allows teachers to have this information in real time, learning actions can be implemented to solve knowledge gaps during theoretical and seminar activities and, by this, improve the level of students assessment of the different key concepts.

Results
We show in Table 1 a comparison of students performance in two consecutive years: 2015-2016 and 2016-2017.During the latter, the learning process monitoring tool was used.The difficulty level of the practicum was equivalent.The main result of this study is a general increase in student performance, which is specially observed with respect to the practicum exam where the percentage of students that surpass the minimum mark is almost doubled.This improvement in the practicum marks is also reflected in the ratio of students that pass the subject, doubled from previous year.We associate improvements in students performance to them being more engaged to the subject.In order to check the validity of this conclusion, we present next some graphs extracted by the use of our performance monitoring tool.
Fig. 2 shows the evolution in the number of mean deliveries per group and the mean mark during the period the assignment is active.We can observe how students interact continuously with the tool (though peaks can be observed coinciding with practicum classroom activities).We can also see how the majority of the students achieve the assigned task before delivery date and how, as a result of their interest in the subject, they keep improving their solution which results in an increase in the final mean mark.Our tool also allows teachers to observe which of the tasks presented more difficulties to students.Fig. 3 shows the dependence between the functionality and the number of times the student has tried to overcome the different tests associated to it.We can observe how some functionalities (Parameters, Operators) needed of less effort than others, especially Initialization.This information can be used to reinforce the theoretical explanation of some concepts to reduce the effort needed.Our tool also allows us to observe in detail the performance related to each functionality.Fig. 4 shows how a very small reduced number of students overcome Object recursion, indicating an area in which to apply a learning action.Finally, Fig. 5 shows the global results obtained by the students in all the different functionalities variations that were studied.This graph allows us to determine which of them were easier for the students (higher percentage of overall sucesss and higher percentage of hidden test overcome) as well as to observe which of the sub-functionalities required more effort by the students to obtain the minimum required mark.This

Conclusions and future work
Keeping a hig level of interest of the student in a given subject is key to a positive result of the learning process.In this paper we have proposed a software tool to observe students learning process.Our tool incorporates assignment preparation and evaluation as well as monitoring capabilities.
Our tool allows students to have inmediate feedback of their performance and also allows teachers to have real time information about students progress.This information can be used to correct knowledge gaps during the present course or to plan improvements in the learning activities of the subject for a posterior year.
A comparison study between two academic years shows promising results associated to the use of the monitoring tool, which suggest that the improvement in overall performance can be associated to an increase in students interest.
Future work should consist of incorporating a control panel which allows the teacher to have direct access to student-specific graphs.We also plan to generate student-specific reports indicating them areas in which they have to improve as well as suggesting supporting material to study.Finally we would like to study the use of mobile applications , either to adapt the ones we propose or use already existing ones such as Kahoot Wang (2016) or Plickers Wood (2017), to obtain real-time information of students learning assessment in order to extend our proposal to lecture classroom activities.

Figure 1 .
Figure 1.Examples of how variations are defined from a single functionality.

Figure 2 .
Figure2.Evolution of the number of deliveries and mean mark over the assignment period.

Figure 3 .
Figure 3. Effort associated to each of the functionalities proposed to the students.

Figure 4 .
Figure 4. Effort associated to each of the private tests associated to Initialization functionality.
information can be used to prepare the assignments for a new academic year, as teachers have powerful to better balance between assignments.

Figure 5 .
Figure 5. Effort associated to each of the functionalities variations proposed to students.