Use of scoring rubrics for evaluating oral presentations in aerospace engineering education

Under the frame of Bologna system, alternative assessment methodologies gain significance in the evaluation process of the students. This article analyses the use of scoring rubrics for evaluating oral presentations of two different courses in aerospace engineering education, namely Aircraft Design and Aerodynamics II, at Universitat Politècnica de València. The score obtained at the presentation represents a significant percentage of the final grade for both courses. It has been observed that the students find difficulties to keep timing limitations overall. In addition, rubrics have proved to be a powerful tool to enhance some skills of the students, such as critical thinking and selfevaluation, although they may lead to unfairly high grades nonetheless.


Introduction
UNESCO (Delors, 1996), through the International Commission on Education for the XXI century, points out that the main theme of any educational project should be 'teach to know to do, to educate'.In other words, the students should learn by applying the knowledge, but also determining under which conditions they have the capability to apply it, for instance, enclosed and summarized in a single assignment.Therefore, education must not allow to comprise the understanding just from one point of view.
Engineering education in Europe is nowadays experiencing important changes (Joshi (2009)).In particular, aerospace engineering education is being affected by two main reasons.Firstly, the problem of educating aerospace engineers is worldwide, given the strong competition that USA is facing from Europe and Japan nowadays in this industry (Niosi and Zhegu, 2005).Secondly, the Bologna declaration and the European Higher Education Area (UK HE Educational Unit, 2015) make it necessary to improve the teaching-learning process.Assessments become more dynamic and short-term based.Therefore the students need to mature the knowledge in a faster way, which is not possible only through exams.New initiatives, projects and methodologies are needed in order to provide the future engineers with a comprehensive view of the practical aspects of engineering, forming not just scientists and designers, but professionals that could deal with multidisciplinary problems including design, manufacturing, finances and business plans, among others.At this point, teaching activities and facilities that bring the students closer to real life situations in aerospace systems are essential in order to progress in the teachinglearning procedure and contribute to stimulate the student to acquire knowledge.As pointed out above, aerospace is one of the most dynamic and competitive of the technical fields (Lapins, 1997).Therefore, it is important to determine whether the students have acquired critical knowledge and skills.With this purpose, it is convenient to stimulate the self-evaluation when the students perform an engineering work and focus a portion of their thinking to a critical process through an objective foundation represented by a professional character.Exams are the most common way to evaluate the students.They are a good instrument when the evaluation is individual and its purpose is to make the student demonstrate his knowledge (Crooks, 1988).Through this action however, the students do not get the capability to determine their capacity and demonstrate their attitudes to carry out engineering work by themselves.
In other words, they do not develop critical knowledge.New approaches to assessment, often known as "authentic" or "performance" assessment, are needed to make the students develop important transversal skills like public speaking or the aforementioned self-evaluation.Project courses and oral presentations, in combination with others, are good evaluation methodologies for doing so (Palomba, 1999).Nonetheless, the information interchange between the professors and the students has a strong impact on the success of these methodologies.On one side, the students autoevaluate the information and they can improve the theoretical lessons offered by the professor, and on the other side the professors are responsible for improving the critical spirit created in the student.
Courses in topics such as aircraft design or aerodynamics play an essential role in any Aerospace Engineering Degree.Their contents are fundamental to reach further detailed knowledge of aircraft systems.Hence, it is needed to provide the students with a suitable balance between conceptual and real world background, so that a successful future professional career can be ensured.In that sense, integration of theoretical and practical work is desirable.Theoretical concepts are necessary in topics such as design, in which complex morphology (airfoil, engine location, pressure distribution, fuselage shape, etc.) is found in real systems.To accomplish these goals, Aircraft Design and Aerodynamics II courses at Universitat Politècnica de València include a public oral defense of the course projects as part of their evaluation process.In these projects, which are described in the next section, the students put into practice the theory explained during the theoretical and practical classes and compare their results with real aerospace vehicles that fly or flew on the skies.

Evaluation of oral presentations
The common way to evaluate the quality of an oral presentation is to employ a scoring rubric.In education terminology, this means "a standard of performance for a defined population" (NCSESA, 1996).Rubrics are standardized ratings linked to learning objectives which theoretically support student self-reflection and selfassessment as well as communication between an assessor (the professor) and those being assessed (the students).They also allow professors, in combination with students, to review the employed criteria, which can be complex and subjective.
Since more than 30 years ago, many scoring rubrics have been presented in a graphic format, mainly as tables, by many universities, associations and companies.
According to Herman et al. (1992), all of them present the same elements: • One or more traits of dimensions that serve as the basis for judging the student response.

• Definitions and examples to clarify the meaning of each trait or dimension.
• A scale of values on which to rate each dimension.One of the main advantages of this type of evaluation is that the criteria employed to prove the quality of the performance (what in real worlds may be a product, process, etc.) can be known beforehand by the student.Nonetheless, this feature may imply a negative effect on the overall performance.The students, obviously, focus on the traits which are evaluated and forget other aspects of an oral presentation that, although not evaluated, may be relevant (for example, the dressing code).This work analyzes the students' outcome with and without knowing in advance the assessment criteria.
Another issue of scoring rubrics is that each level of fulfillment covers a wide range of marks.In addition, a small change in the evaluation on the rubric may lead to an unfair change in the numerical grade.Therefore, one of the subjects presents numerical scoring determined by ranges according to levels of fulfillment.
Multidimensional rubrics allow students to hide and compensate for a lack of ability in one trait by improving another one.However, if the traits are weighted properly, rubrics are a very effective way to improve the student weaknesses.

Description of the courses
Aircraft Design and Aerodynamics II courses are taught simultaneously during the first semester of the 4 th year of the Aerospace Engineering Bachelor Degree at Universitat Politècnica de València.At that point, the students already have the basic foundations in Mathematics, Physics, Structures and Loads, Solid Mechanics, Fluid Mechanics, Aerodynamics, Control Systems and Aerospace Technology.The teaching-learning methodology is similar for both courses.In both evaluation methodologies, a significant percentage of the total grade of the course is obtained through the assessment of an oral public presentation.The presentation deals with the defense of their ideas and the work performed during the semester.With this kind of system, not only knowledge and critical thinking are evaluated, but also transversal skills such as teamwork and communication.

Aircraft Design
The Aircraft Design course is structured as a project-based course in which the students are required to apply their knowledge of different topics in order to perform a course project about the conceptual and preliminary design of an aircraft that fulfils the requirements in all different areas, taking into account general requirements such as performance, applications, green design, sustainability and economy.
Its evaluation is divided in three different parts: the evaluation of the course project, a oral defense of it, and a traditional exam.The project accounts for the 50% of the student grade, while the oral exposition and the exam weight 25% each.The course project consists in pre-designing an aircraft following certain specifications given by the professors, such as maximum cruise speed, range or propulsion technology.This project is carried out by groups of 3-4 students.
In this course, the scoring rubric of the oral exposition (see Table 1) is not known in advance by the students.Thus, they must use their general knowledge (or personal research) to figure out which abilities are ranked in the evaluation process.Professors who form the evaluating tribunal are also asked to give a global mark to the presentation without taking into account the traits of the rubric, just by their general opinion.   1 shows that each criterions is qualified with a score from 0 to 10.As previously said, different levels of fulfilment determine scoring ranges, providing clear and uniform criteria for the different professors.Due to manuscript length limitations, the levels of fulfilment for each dimension are not included.

Aerodynamics II
The Aerodynamics II course is a traditional lecture-based course with several computational laboratory sessions.The main topic of this course is fundamentals of high speed aerodynamics, for example potential flow over supersonic wings or lift force of slender bodies.
The global evaluation of this course consists of three exams, a course project and an oral presentation of the project.Exams represent 70% of the grade, the course project 20% and the presentation 10%.The course project and also the oral presentation comprises four laboratory sessions in which the students analyze different aircraft geometries (a missile, a slender wing…) under different high speed (subsonic and supersonic) flight conditions.In this case, the students know the scoring rubric of the presentation (shown in Table 2) since the beginning of the course.In addition, they are provided with a brief guide about "how to" perform an oral presentation.This rubric could be used by the students to guess the aspects evaluated in the Aircraft Design course, but the tribunal is not the same.For this reason rubrics and mainly rank criteria are quite different.In this course, the score for each dimension can only present 5 different values, from 0 to 2.5 points (see table 2), corresponding to the different levels of fulfilment.Again, for the sake of brevity, descriptors are omitted for the scoring rubric described in Table 2.

Results and Discussion
In this section, the scores obtained by the students as well as the personal opinion of professors and students alike are analyzed.In order to simplify and extract general trends, averaged values and standard deviations are given.The same 40 students (academic year 2014-2015) were ranked in both courses to minimize uncertainties.The time gap between the two evaluation events is less than two weeks, so the experience of the students in oral presentations is more or less the same.

Case 1: Aircraft Design
Table 3 shows the Aircraft Design presentation score.Relatively high scores were obtained by all the students.The low standard deviation found in each item remarks that the groups performed in a quite homogeneous way.The highest average grade was obtained for the "Time" criterion, i.e., the ability of the students to defend their project in no more nor less than a predefined time (in this case, 15 minutes).However, it has the highest standard deviation.This is due to the fact that a couple of groups exceeded the time limitations in a really important way.
On the other side, the "Innovation" dimension was the aspect in which the students performed in a poorer way.This criterion awards the groups of students which design an innovative aircraft in terms of aerodynamics, operability, target mission… instead of a straightforward one.The authors noticed that this is a trend that changes over the years in the Aircraft Design course: when students from one year really try to innovate in their design in order to improve their marks in this sense, the students from following years show a lack in innovation in order to enhance other aspects of the project and go further in their calculations.The authors suspect that this is due to the fact that older students prevent the next generation ones from introducing complicated innovations in their designs due to the added difficulty.
Another important result is that the global marks given by the professors without considering the scoring rubric matches almost perfectly the average of the scoring rubric criteria, with just a slightly higher standard deviation.This could be explained since the evaluators' experience probably leads them to assess a global mark based on the same items considered in the rubric in an unconscious way.What is more, the marks they globally give to the presentations might be inferred by the marks they just gave to each of the criteria, since the professors internally try to respect these marks.

Case 2: Aerodynamics II
Table 4 presents the Aerodynamics II course presentation marks.Generally, high scores were obtained with high difference between teams.Low homogeneity is observed in the group of students.Some of them took profit of the documentation given to them and performed very well with oral presentations, whilst some other did not defend their work in a proper manner.It is interesting to see that "Organization" is the aspect with the highest average score and lowest deviation.This means that Aerodynamics II students are able to sort the data correctly and logically.Again, "Time" is the criterion with the the highest deviations.However, in this subject this dimension presents the lowest average mark.Students find it very hard to stick to a given time when they feel that they have done a high amount of work and want to present all of it.This means that they have to improve the transversal competence related to summarization of information.

General results
Score of presentations is generally high.In fact, no one (out of 40) failed.In average, there is a difference of about 15 points when this score is compared to the mark obtained in other evaluation activities, i.e. exams.As stated in the "Methods" section, rubrics allow the students to hide some of their weaknesses.This does not mean rubrics are not useful.The students are aware of their downsides and that is why they are able to hide them.Thus, the auto-evaluation skill has been enhanced indeed.Nevertheless, score presentations then seem kind of unfair.
When both studied cases are compared, it is noted that the influence of knowing in advance the scoring rubrics leads the student to perform better in the different items that are evaluated, obtaining scores about 6 points higher than the ones obtained without this information.When preparing their presentation, students tend to focus on the particular items that will be assessed, rather than trying to introduce additional features to their presentation.For instance, it was noticed by the professors that most students stuck to a formal dress code in the Aircraft Design presentations, whereas they did not give any importance to this fact in the Aerodynamics II ones.

Conclusions
The use of scoring rubrics for the evaluation of oral presentations in an Aerospace Engineering Degree has been analyzed.Their advantages and drawbacks are well known by the community, and they have also been observed in this work.However, interesting results have been obtained.Overall, the students find difficulties to keep timing limitations even when they are warned to do so by giving them the information about the scoring rubric in advance.
Given the relevance of timing in engineering projects, exercises to improve the summarizing skill should be proposed to the students.The use of scoring rubrics is hence a good tool to identify the performance of the students in this transverse competences.
In addition, this work pointed out that the fact that the students know the scoring rubric in advance leads them to focus exclusively in the stipulated criteria, forgetting about other features that could also improve the overall quality of their presentations.
In order to get the best of both worlds, the criteria might be given in advance, but an additional "overall impression" dimension should be included.
However, in general terms, the objective of enhancing self-evaluation and critical thinking is fulfilled with the methodology proposed in this paper.Thus, it is recommended to keep the use of scoring rubrics for the evaluation of next generations.

1st
International Conference on Higher Education Advances, HEAd´15 This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).

1st
International Conference on Higher Education Advances, HEAd´15 This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).

1st
International Conference on Higher Education Advances, HEAd´15 This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).

1st
International Conference on Higher Education Advances, HEAd´15 This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).

Table 1 .
Scoring rubric employed in the Aircraft Design course

Table 2 .
Scoring rubric employed in the Aerodynamics II course

Table 3 .
Grade obtained by the students in the oral presentation of the Aircraft Design course This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).

Table 4 .
Grade obtained by the students in the oral presentation of the Aerodynamics II course