Content review using Kahoot ! with aeronautical engineering students

In the framework of the subject “Analysis of aeronautical structural components” included in the Master of Science of Aeronautical Engineering at the Universitat Politècnica de València, the application Kahoot! is employed in order to review the technical content and obtain feedback of the level of knowledge acquired by the students. Kahoot! allows developing multiple choice interactive quizzes that are solved in the classroom. Using this type of tools enhances the attention of the students and helps to create interesting discussions making the students be part of the teachinglearning process. Besides, the platform allows analyzing the results of the technical content reviewed which, in turn, permits the lecturer to adapt the didactic material to a real scenario. Furthermore, the results obtained from the assessment survey show the high level of satisfaction of the students with an activity which allows them to learn in a fun way. In conclusion, the application of Kahoot! for content review helps to perfect the teachinglearning process and improves academic performance in an attractive and engaging environment.


Introduction
In these days, as a result of the European Higher Education Area (EHEA) harmonization process, more flexibility on the learning-teaching process is required.This is necessarily linked to the development of new methodologies to both improve the continuous progress of students and, not less important, enhance the motivation and professionality of teachers.These goals can be accomplished with the integration of electronic learning and mobile learning strategies inside the educational scenario (Jin, 2009).
Although educational professionals have traditionally shown a great inertia against the change from traditional teaching methodologies (as classical theory lessons) to new classroom techniques, in the last decades, the use of the Information and Communication Technologies (ICT) in higher education has increased and currently they constitute new channels of information transmission (Fructuoso, 2015, Serrano y Fajardo 2017).
New generations of students have already interiorized in their daily life the dynamic of digital games which are characterized, among others, by the speed on making decisions or the immediacy of response.The incorporation of this technique based on game dynamics is known as gamification and has already proved to have positive effects in the development of the courses (Labrador et al. 2016, Sánchez 2015).
A very useful format for the gamification of content review sessions is the use of real time quizzes.The advances in technology have simplified the way in which quizzes can be applied in the classroom and implementing these techniques is even simpler when students can use personal devices capable of connecting to the web such as laptops, tablets or smartphones.Besides, students will feel more comfortable and confident with their own electronic device when dealing with a learning task.Also the fact that no clickers have to be purchased in order to carry out the activity facilitates its management and preparation, and at the same time, makes this technique a really economical option (Shea 2016, Lee et al. 2013, Grinias 2017).
One of the online systems which allows the possibility of performing these quizzes at the classroom is Kahoot!(2013).This application enables, in an easy and quick manner, the creation of real time quizzes based on speed scoring and board rankings.It helps to gamify the activity since the quiz is solved in an interactive way.On the screen, the answers given by the students are displayed immediately and this real time feedback may encourage them to improve in the next questions of the quiz.After each question, the application creates a leaderboard classification.Some works can be found describing the positive influence on the student's motivation of showing the scores of the classroom games in the same way that it is done in video games (Mauricio 2015, San Miguel et al. 2017).
In the same line, there are other online applications, such as Socrative (2013), used by lecturers for the implementation of quizzes in the classroom.In this case, Socrative permits also the preparation of open short answer questions contrarily to Kahoot!, where only the classical multiple choice questions can be implemented.Although this fact multiplies the design possibilities of quizzes, in this case of study, Kahoot! was chosen in order enhance the dynamism of the activity.Besides, it must be highlighted that at Universitat Politècnica de València already exists a platform available for students and lecturers called PoliformaT which also allows the implementation of tests.However, the application where these tests are develop obliges the students to answer all the questions of the quiz in a given period of time, being possible only to know the correct answers once the test is finished.This dynamic does not permit to generate a discussion with the students after each question.
Thus, Kahoot! is a very intuitive application which can be used in the classroom and has important advantages, as the real time feedback.Besides, it permits to evaluate previous knowledge if the quiz is performed at the beginning of a lesson or even at the opening of the course, so the lecturer can have information about the initial level of knowledge of the students.Also, since it is an activity quick and easy to be developed, it can be considered a good tool to use for the continuous evaluation of the subject along the academic year.This paper presents a new experience of the implementation of Kahoot! in higher education designed for students of the Master of Science of Aeronautical Engineering at the Universitat Politècnica de València (Valencia, Spain).The methodology is applied in the framework of the subject "Analysis of aeronautical structural components".In this case, the activity consists of solving a series of questions of a quiz developed in order to review the contents already taught in previous lessons of the subject and prior to the final standardized exam.Besides, the results of the quiz can be used not only for the students to review those aspects where they found more difficult but also for the lecturers to review their educational material and adapt it for the next lessons to the level and progression of the classroom.
The main aspects of the experience developed and its assessment are described in the next sections.They can be used in the design of a similar activity, but, since the project presented has been performed in the framework of a Master of Science of Aeronautical Engineering, some issues may need to be adapted for another different context.

Preparation of the activity
The development of the activity starts previously to the opening of the course, when the lecturer is defining the learning outcomes of the subject.Focused on the evaluation of the student's achievement of these outcomes, some quizzes are previously prepared by lecturer.They are going to be used at the end of some lessons to evaluate in an attractive environment the students learning progress.Therefore, these quizzes should be short, clear and developed in a very direct way.
As it was explained above the online platform Kahoot!provides an excellent web-based context to develop this assessment methodology.It uses colourful graphics and interactive displays to transform temporally the classroom in a gaming show.In order to facilitate the quiz creation, a specific Kahoot!web site is available for lecturers.Thus, multiple choice quizzes can be created very easily and additional options as time limit, award points or attached media file (image or video) can be included.
However, before implementing the activity, it is necessary to take into account that the students know how to use the application.In order to avoid problems during the quizzes, some proofs were done previously.Besides, due to the necessity of having a permanent and good internet connections, it was really important to assure that all the participants could have access to the internet and so, to the application.

Context
The assessment methodology described in this paper was developed during the academic year 16/17 for the subject "Analysis of aeronautical structural components" of the first course of Master of Science of Aeronautical Engineering at the Universitat Politècnica de València.The students enrolled in this course have previously obtained a Bachelor Degree in Aerospace Engineering.However, it has been noticed during the last years that they come from a variety of different Aerospace Engineering Schools from several cities as León, Madrid or Valencia, among others.Therefore, the background knowledge of these students about aeronautical structural component analysis is not exactly the same.This is one of the most important reasons why the methodology explained in this paper for the formative assessment of the student's outcome progress was implemented.In previous courses, the lecturers found out that students had different learning progressions and they realized that it would be necessary to have real time feedback to adapt the didactic material to the classroom real scenario.
The teaching plan of this course is divided into 9 general units, which match with the learning objectives of the subject.Each of these units is developed through two or three lessons.The first lesson of each unit consist of an initial classical theory lesson while the second and third one are based on problem solving lessons where key problems are proposed to the students to be solved with the lecturer guidance.The Kahoot! quizzes prepared previously by lectures are conducted at the end of the first lesson of each unit, just after the theory explanation.It takes only a few minutes and provides important information to the lecturer about the student's learning outcome achievement.Additionally, this assessment methodology encourage them to promote their learning progress in a gaming context.

Lecturer role
In this student-centered activity, the lecturer assumed a secondary role.Specifically, the lecturer acts as the gaming show host while the students are the competitors.The activity is introduced by a brief description of the Kahoot!platform and all students are invited to join the quiz session through their laptops, class computers or mobile devices.The platform flexibility allows that all students can be ready to start the assessment activity in only a few seconds.
Before the quiz starts, to encourage the student participation, the lecturer describes some important aspects about the dynamic of the quiz as the time limit and score of each question.Thus, the students are introduced in a competition context.During the quiz, the lecturer should be acting as a controller because each question should be submitted step by step from the main computer, taking into account that all scores are processed.
After each question, the lecturer has the chance of involving the students in a discussion.Besides, the most usual mistakes are reviewed and the lecturer has feedback in order to adapt the material.Therefore, in the next lessons more emphasis can be put on the explanation of those concepts which appeared to be the most difficult.

Activity progress
In the activity, each question is launched by the lecturer and the students have 30 seconds to answer.In this case, multiple choice questions were created with four possible answers.For the sake of clarity, each question can be accompanied of figures or pictures.The students should answer the question through their smartphones.The platform shows a very colorful and clear answer screen to encourage competition, see Fig. 2.
When the question time is finished, and previously to launch the following question, the students receive an automatic personal feedback to know if the answer was correct or wrong.In addition, the correct answer and the partial score are provided to the students.
The system considered how fast the question is answered, increasing the score of those with the fastest rates.This procedure allows a motivational and formative assessment.At the end of the quiz, the student nickname with higher score is displayed in the main screen.Besides, the students can send to the lecturer a blind rate, in a 1-5 rating, about the activity.This last rating provides proper feedback about the opinion of the students.The results of this survey will be analyzed in the next section.

Results
Once the activity is finished, the results can be downloaded in a very detailed format from Kahoot! platform.This information can be processed to provide conclusions.In this experience, a diagram was prepared for each unit, where percentage of correct and wrong answer is displayed for each question.In Fig. 3, the results of two of the quizzes are shown.Through these diagrams it can be observed, for instance, that Q2 in unit 9 had a very low percentage of correct answers.It means that the learning outcome "Warping of open thinwalled sections" has not been reached properly.Therefore, this content is deeply prepared in the following problem solving sessions in order to fix this deficiency.Besides, this indicator will permit, at the end of the course, to improve both the didactic material and the lecturer explanations regarding this concept.

Assessment
Additionally, Kahoot! permits the student to express their satisfaction with the activity by means of a brief survey.This survey consists of two simple questions, one focuses on the engaging character of the technique and the other on the level of knowledge acquire or reinforced with the activity.Thus, the lecturer has immediate feedback and can act to make more dynamic the session.In table 1, the mean rating for the quizzes of all units is shown.

Id Value
How fun was it?4.53 (1-5 rating) Did you learn something?0.90 (0-1 rating) As can be observed, the results were satisfactory.In fact, the response of the students when new technique is introduced in the classroom tends to be positive.In this case, a mark of 4.35 over 5 was obtained related with the implementation of the activity, which encourage the lecturer to carry out the same type of game in next academic years.Also it is important to note that most of the students (90%) consider that they have learnt through these quizzes, which demonstrates the value of the tool in the teaching-learning process.

Conclusions
In the framework of the subject "Analysis of aeronautical structural components" of the Master of Science of Aeronautical Engineering, the application Kahoot! was employed for content review by means of multiple choice interactive quizzes.This tool helps to make the students be part of the teaching-learning process by creating interesting discussions and enhancing their attention.Besides, the lecturer can adapt the didactic material to a real scenario since the application allows analyzing the results of the technical content reviewed.Furthermore, the results obtained from the assessment survey show the high level of satisfaction of the students with this activity which allows them to learn in a fun way.
In conclusion, the application of Kahoot! for content review helps to perfect the teachinglearning process and improves academic performance in an attractive and engaging environment.However, it is really important and necessary to explain carefully to the students which is their role in the activity and the mission of the game.As other authors have pointed out (Kapp, 2012) overusing this type of activities in the framework of a university classroom may have the opposite effect.Gamification must be used with moderation so the objective of the subject and the goal of learning and achieving certain competences and skills is never missed.