Context-aware assessment in out-of-classroom activities by means of mobile technologies

This paper proposes a framework for context-aware assessment of out-ofclassroom subjects with the support of mobile technologies. The paper discusses: (a) the principles and implications of out-of-classroom activities, e.g. internships, outdoors practicals of topography, of agricultural or marine engineering, virtual university courses, etc.; and (b) context-aware assessment, in which some context elements (location, date and time, environmental circumstances) may be incorporated into the assessment by means of mobile technology. The aim of this framework is to provide the tools that will allow for an enhanced evaluation, by taking into account the context of the activities being carried out. Given the characteristics of the activities, the assessment will be based on submissions prepared by the students, which will automatically incorporate context information that can be relevant for the assessment. The framework will be tested with several groups of students participating in internships and/or international mobility programmes, from different universities (public, private, semi-public).


Introduction
Context-aware assessment of learning activities opens a new range of possibilities for better and more accurate evaluation of students' achievements (Poikela, 2004), by making explicit use of environmental information that is usually not considered in traditional assessment methods.
Out-of-classroom learning also plays an important role in education as a complement to formal learning at school.The constructivist theory (Kolb, 1984) describes how new knowledge based on real life experience is constructed and integrated into existing knowledge after a reflective thinking of the activity being carried out.The current proliferation of mobile devices like smartphones or tablets adds new opportunities to enhance not only the student experience in this type of activities, but also the monitoring and assessment of the learning process (Coulby et al., 2011).
After discussing the fundamentals of out-of-classroom learning and context-aware assessment, this paper presents work in progress around a project whose goal is to define a framework aimed at supporting and facilitating the development of context-aware assessment applications based on mobile technologies specially adapted to out-ofclassroom activities.

Out-of-classroom activities
The goal of this project is to propose a framework for context-aware assessment of out-ofclassroom activities, using mobile technologies.
In this work, out-of-classroom activities are understood as learning activities that make up a regular course, or a fundamental part of it, and that are developed in one or more locations different from the traditional classroom in the academic institution.These activities are carried out more or less continuously over a specific period, as opposed to other types of sporadic activities which can also be carried out away from the school, e.g.occasional outdoor activities for a specific purpose, such as a visit to a botanic park or an exhibition, which are included in a formal subject taught in the classroom (Santos et al., 2014).Examples of out-of-classroom activities are open university courses, internships, practicals of certain disciplines like topography, agricultural or marine engineering, exchange programmes, etc.
Out-of-classroom activities can be carried out indoors or outdoors.For the purposes of collecting context information, as will be seen below, this fact can make a significant difference, but in other respects they can be considered as the same type of activity Perramon, X.; Alemany, J.; Torres-Kompen, R.; Kennett, C. regardless of the specific location.It is also possible for an out-of-classroom activity to be performed en route, i.e. in an itinerant fashion, and even that part of it is indoors and part outdoors.

Context-aware assessment
The use of context information for the assessment of learning has been the subject of various studies.In general, a context-based assessment requires careful consideration of situational and contextual factors of the learning processes which, according to Poikela (2004), can be classified as social (assessing and understanding), reflective (assessing and developing), cognitive (remembering and understanding) and operational (doing and acting), in contrast to traditional assessment.This theory, based on constructivist principles, states that the assessment is more productive when it uses expression of the individual, shared and organisational contexts of learning, which in the case of an out-ofclassroom environment integrates the action characterised by situation, time and place.
A new field opens up if the context details can be gathered automatically by means of the appropriate devices.In the case of mobile-based learning, more context items can be associated with the activities to be assessed.Hwang et al. (2008) explain the relationship amongst u-learning (learning anywhere and anytime), mobile learning (learning with mobile devices and wireless communication), u-computing in learning (learning with ubiquitous computing technology), and the newly defined "context-aware u-learning", i.e. learning with mobile devices, wireless communications and sensor technologies to better understand the learner behaviour and the timely environmental parameters in the real world, such as the location and behaviour of the learner, as well as specific physical features of the context.Some studies have been published around context-aware m-learning in specific areas and disciplines (Hwang and Wu, 2014;Perera et al., 2014), such as primary and secondary school (Santos et al., 2014), natural sciences, nursing or archaeology.These works focus primarily on the use of mobile devices in the learning process, but their use in the assessment process and the relationship to the assessed activity is not deeply analysed.Back in 2008, Hwang et al. pointed out the potential for additional research in this field.

Mobile technologies
Mobile devices are more and more popular everyday, and they are reaching all social layers and communities, including also students.A student may very easily have access to a smartphone or a tablet and use it for their learning activities.

Context-aware assessment in out-of-classroom activities by means of mobile technologies
A mobile device usually hosts a number of sensors that can provide wide-ranging data; from the general context-aware assessment perspective the most relevant items may be those related to date and time, location, and possibly motion, but for some specific assessment applications other types of context information can also be of interest (such as orientation, speed, acceleration, gravity, proximity to other objects, temperature, pressure, humidity, illumination, magnetic field, signals from various antennas: telephony, Wi-Fi, Bluetooth, NFC, GPS, images from a camera, sounds from a microphone, etc.).
Sensor outcomes may depend on whether the device is outdoors or indoors, especially in the location information.If the device is outdoors and it is equipped with a satellite navigation sensor (typically GPS), it will be able to determine its global location.Otherwise, this information can be obtained more or less accurately through other means such as Wi-Fi, Bluetooth or QR code scanning.

Proposed study
Based on the principles presented above, an exploratory study is outlined in order to determine the type and quantities of data that could be collected during out-of-classroom activities.In an effort to obtain data that will allow us to propose a framework that supports the development of context-aware assessment applications for out-of-classroom learning, we have decided to take a user-driven approach by involving a sample of the final users during the design phase.The goal is to define which variables and metrics will be most useful in the design of u-learning activities to be assessed with context information automatically gathered by mobile devices, and in the end propose a framework that can be applied in a diversity of situations and scenarios.
The main questions that are to be approached during the first phase refer to the variables and metrics to be gathered, and also the means by which they will be collected.
As mentioned earlier in this paper, variables such as date, time, location and motion are provided by all smartphones; the question would be, in this case, which other variables could be measured that may be used in terms of assessment for an out-of-classroom activity?
In terms of the technology used, although smartphones probably are the best-known example of mobile technologies, there are other alternatives to be considered."Nearables" such as Estimote (http://estimote.com/) or Tile (https://www.thetileapp.com/),for example, have more precision than GPS systems and could be used in specific situations.Wearables, such as the iWatch, could also be considered.Other types of sensors, such as RFID, may provide data that is not easy to obtain with a smartphone.Perramon, X.; Alemany, J.; Torres-Kompen, R.; Kennett, C.
The first phase of the study would involve the design of questionnaires for the three main actors involved in the out-of-classroom activities: the students, the academic tutors and the in-field supervisors.A series of focus-group/brainstorm meetings will follow, in order to compare the results and prioritise the metrics in terms of relevance and impact on the assessment.
During the second phase, a pilot test will be run, in order to verify the feasibility of the data-collection process and the relevance of the data collected.
In the third phase, the data collected and the analysis from the second phase will be used to propose a framework, which will be tested with two groups of students during the fourth phase of the study.
With this framework, the learning activity designer can decide which context information is of interest for the assessment of an out-of-classroom activity.

Methodology
The first phase is underway at the time of this writing, and it involves the design of questionnaires and surveys, whose target audience will be the main three groups of stakeholders that have been identified in the preliminary stages of this project: the students, the academic tutors and the in-field supervisors.The working scenario for the design of these instruments is an upcoming field trip in which one of the researchers is participating, but with the aim of extending it to other scenarios, such as those described in the introduction.As planned, phases two and three are to be executed during the second quarter of 2016, in order to finalise a framework proposal by the end of the academic year (September) 2015-2016.
In order to explore the potential data collection opportunities with educators, an action research approach is proposed.The first iteration of the research would involve a series of three workshops with the following aims and structures: • Workshop 1: creativity and brainstorming.
Aims: understand the current practices used in out-of-class activity organisation in the educational context; explore existing data collection initiatives among educators; identify opportunities to collect data and potential restrictions.
Aims: design an out-of-class activity that would enable data collection to occur including: learning objectives; technology requirements; specific activities to be undertaken; evaluation criteria; stakeholder feedback tools (pre and post); calendar and timings.

Context-aware assessment in out-of-classroom activities by means of mobile technologies
• Workshop 3: evaluation and reflection.
Aims: post activity session to bring together feedback from stakeholders (students, professors, supervisors) and critically analyse the activity in terms of design, implementation and evaluation; identify areas for improvement for a second iteration.
The stakeholder feedback tools will include semi-structured interviews and pre/post activity questionnaires.The three workshops will be recorded, transcribed and analysed.
After the completion of the first iteration, depending on the results obtained, a second iteration will be implemented with the aim of improving the effectiveness of the activity and to gather comparative data.
The type of data to be collected will most likely fall into one of these categories (as proposed by Feng et al., 2004) Social environment: traffic jams, gathering, meeting, surrounding people.
Computational environment: surrounding devices.
Probably the most obvious type of data to be collected is location.Some studies have looked at location data as a way of determining potential for interaction (for example, whether a person is close to one of their coworkers or team members); video and audio could also be used to share an individual's situation and availability for starting a conversation, for example.
Other types of data that may be of interest include: time and duration of activities, routes and paths chosen when visiting specific locations, etc. Perramon, X.; Alemany, J.; Torres-Kompen, R.; Kennett, C.

Preliminary results
At the time of writing, a first set of questionnaires has been proposed (see project Resources, section 8) to be applied to a sample of 54 users: 50 representing students, and 4 representing professors who will play the dual role of academic tutors and in-field supervisors, with an action research approach.The data collected will be both qualitative and quantitative, as can be seen from the proposed questionnaires and will be used in the subsequent stages of the project (see project Resources).Some of the demographic data of the sample is shown on Table 1.A questionnaire has also been developed with the aim of analysing the working scenario in terms of the description of the stakeholder groups and technological capabilities.
This, we expect, will shed some light on the potential usefulness of the data to be collected, from the point of view of learning design.

Implications
Although context-aware data shows great potential, there are some implications that have to be taken into account, such as: • Privacy: obtaining context-aware data usually happens without the users being actively involved.Who owns these data and what are the limitations to their use?• Complexity: the quantities of data that may be collected could pose a challenge both in terms of storage and analysis.

Potential contributions
The main contribution of this research would be the development of a framework for the use of context-aware data in learning design.By involving all stakeholders, the collection of data can be prioritised in terms of potential usefulness and application in the creation of While in face-to-face learning the instructor implicitly acquires contextual knowledge about the students and the group, in out-of-classroom activities this context information is not directly available.It is expected that the framework will contribute to a better and more accurate assessment of activities carried out by students out of the standard academic environment, by taking into account new contextual factors that have not been considered up to now.
This work is licensed under a Creative Commons License CC BY-NC-ND 4.0 Editorial Universitat Politècnica de València Context-aware assessment in out-of-classroom activities by means of mobile technologies learning activities, assessment of the learning process and identification of possible need for modifications or improvements.