Strategies to assess generic skills for different types of students

The Universitat Politècnica de València (UPV) has synthesized a profile to be acquired by all the students based on 13 generic skills. For its assessment, the UPV has also developed a rubric for every skill depending on the level of the course. In this research, we develop an educational innovation for validating the rubrics for 3 of the 13 generic skills specified by the UPV. The chosen skills are: “Ability to think practically and apply knowledge in practical situations”, “Innovation, creativity and entrepreneurship ability” and “Teamwork and leadership ability”. To do this, we develop the same methodology in two groups (Morning/English) of the same course (Marketing Research of the Degree of Business Administration and Management of the Faculty of Business Administration and Management at the UPV) with significantly different student profiles. The assessment results of the skills reveal that there are no significant differences between groups. In conclusion, we could say that the rubrics developed by the UPV are adequate to assess all types of students: Erasmus or non-erasmus, working or having worked in the last 2 years or without work experience, and regardless of their satisfaction with the course.


Introduction
The Universitat Politècnica de València (UPV), taking into account the most important standards and guidelines as well as national and international scientific literature, has listed 13 generic skills.These skills synthesize a profile that has to be acquired by all the students at the UPV.For its assessment, the UPV has also developed a rubric for every skill depending on the level of the course (Universitat Politècnica de València, 2016).
Competency-based learning supposes a completely different way of curricular organization and teaching and learning methods (Lasnier, 2000).The challenge we face now is how to develop assessment strategies for these 13 generic skills according to Bologna Process (European Ministers Responsible for Higher Education, 1999).These strategies should be focused on the students for a better learning (Biggs and Tang, 2011).
In this research, we develop an educational innovation for validating the rubrics for 3 of the 13 generic skills specified by the UPV.The chosen skills are: "Ability to think practically and apply knowledge in practical situations", "Innovation, creativity and entrepreneurship ability" and "Teamwork and leadership ability".The specific objectives of the research are: (i) measure the achievement of every generic skill considered, and (ii) check for significant differences in the achievement of students depending on their profile.

Theoretical framework
Assessment is a necessary byproduct of the current emphasis in higher education on accountability and learning outcomes.Assessment activities can identify learning outcomes for student success and improvement over time in student learning (Dunn, 2002).Multiple approaches are needed in the assessment of learning, many of which require the use of a rubric (Dunbar, Brooks and Kubicka-Miller, 2006).
Rubrics are documents that articulate the expectations for an assignment, or a set of assignments, by listing the assessment criteria and by describing levels of quality in relation to each of these criteria (Reddy and Andrade, 2010).In a student-centered approach, the rubric could be shared with the students in order to support student learning.The main reason for this potential lies in the fact that rubrics make expectations and criteria explicit, which also facilitates feedback and self-assessment (Jonsson and Svingby, 2007).In the case of the UPV, both the 13 generic skills and the corresponding rubrics are published on their own website for the students' knowledge.
However, there are a number of factors identified that may moderate the effects of using rubrics formatively, as well as factors that need further investigation.Panadero and Jonsson Baviera-Puig, A.;Escriba-Perez, C.;Buitrago-Vera, J. (2013) point out Gender and Educational level among others.In this research, we consider Gender and other variables in order to define the students' profile in higher education.

Methodology
The course where we develop this educational innovation is "Marketing Research".It is taught in the first semester of the third year of the Degree of Business Administration and Management of the Faculty of Business Administration and Management at the UPV.The educational innovation is carried out in the Morning Group and in the English Group.
First of all, students were divided into groups of 2-4 people.In total, 32 groups of students were formed: 20 in the Morning Group and 12 in the English Group.Once organized, they were asked to propose a new product / service for a supermarket.To do this, they had to elaborate a concept test in order to determine the suitability of the new product / service.They have worked on this concept test both in theory classes (to sketch and generate ideas) and in lab sessions (for the completion of the questionnaire and analysis of the answers obtained from it).Based on their work, they have drafted one report per group.This group report is the one we are going to use for the assessment of "Ability to think practically and apply knowledge in practical situations" and "Innovation, creativity and entrepreneurship ability".It is evaluated by the lecturer.Instead, to evaluate "Teamwork and leadership ability", we use self-assessment and co-evaluation among members of the same group.For both the report and the self-assessment and co-evaluation, we use the rubrics developed by the UPV.Following the institutional project, 4 different categories are established: A. Excellent / exemplary; B. Good / adequate; C. In development; D.Not reached.In order to facilitate an average per group, these 4 categories have been translated into numbers according to the following scale: A = 4, B = 3, C = 2 and D = 1.

Group profile
In the total sample, there are 110 students: 65 in the Morning Group and 45 in the English Group.To determine the profile of every group, a cross-tabulation of frequencies between the data of the Groups with the different variables considered (Gender, Erasmus, Working and Satisfaction with the course) was conducted.As a statistical test, Pearson's Chi-square test (χ2) was performed (Santesmases, 2009).From the results obtained, it can be observed that there are not significant differences between the Morning Group and the English Group regarding the Gender variable (Table 1).However, there are significant differences with respect to the other variables.As for the Erasmus variable (p <1%), 44.44% of the students in the English Group are Erasmus Exchange students and in the Morning Group only 3.13% (Table 2).In the English Group, 71.11% are working or have worked in the last 2 years, while in the Morning Group this percentage drops to 41.38% (p <1%) (Table 3).Finally, in the Morning Group, students who consider the course interesting or very interesting are 88.33%, rising to 95.56% in the English Group (p <5%) (Table 4).In short, we can state that, despite having analyzed a single course, the profiles of the two groups are different in terms of the number of Erasmus Exchange students, the number of students who are working or have worked in the last 2 years and their satisfaction with the course.

Comparison of skills assessment by group
In order to show the results of the assessment of the 3 generic skills, the average of each of the indicators considered in the rubrics was calculated by Group (Morning / English) and for the total of the sample.To verify if there are significant differences between the results of each group, a cross-tabulation of mean values was performed.As a statistical test, the Snedecor F test was used (Santesmases, 2009).
There are not significant differences by indicator between both groups for the "Ability to think practically and apply knowledge in practical situations" (Table 5).The indicator that has the greatest average in each group and in the total of the sample is "Sets specific targets in relation to the situations that are presented".In contrast, the indicator that has the lowest average in the total sample and in the English Group is "Evaluates the quality of information available for application".The Morning Group obtains the lowest average in "Proposes control indicators for monitoring the plan".The greatest and lowest means are underlined in Table 5.For "Innovation, creativity and entrepreneurship ability" (Table 6), there are not significant differences per indicator between both groups except for "Controls Results" (p <5%).In this case, the Morning Group obtains its lowest average (2.55) in this indicator, while the English Group obtains a higher score (3.25).The indicator that has the lowest average in the English Group is "Uses creative strategies and / or techniques to shape ideas and solutions in a formal way".The total sample obtains its lowest means (2.81) in both cases.

Strategies to assess generic skills for different types of students
On the contrary, the indicator that has the highest average in each group and in the total of the sample is "Identifies opportunities and / or improvement aspects".The greatest and lowest means are underlined in Table 6, as well as the significant probability at 5%.Baviera-Puig, A.; Escriba-Perez, C.; Buitrago-Vera, J.
There are not significant differences per indicator between the two groups for "Teamwork and leadership ability" (Table 7).The indicator that has the highest average in each group and in the total of the sample is "Participates in the planning of the objectives", while the indicator with the lowest mean is "Acts to deal with the conflicts of the team".The greatest and lowest means are underlined in Table 7.It should be pointed out that this generic skill has greater averages (all of them are greater than 3.55) in comparison with the other two skills.This might be due to the self-assessment and co-evaluation conducted by the students.

Conclusions
The educational innovation is developed in the Morning and English Groups of the course "Marketing Research" of the Degree of Business Administration and Management of the Faculty of Business Administration and Management at the UPV.The total sample analyzed is 110 students (65 in the Morning Group and 45 in the English Group).In spite of having analyzed a single course, the profiles of the two groups are significantly different in terms of the number of Erasmus Exchange students, the number of students who are working or have worked in the last 2 years and their satisfaction with the course.
Nevertheless, despite this difference in profile, no significant differences are found in the assessment of the 3 generic skills.Each of the indicators considered in the rubrics has been evaluated and no significant differences in the mean value of each group have been obtained.We only have one significant difference in the indicator "Controls results" of "Innovation, creativity and entrepreneurship ability".
As there are no significant differences between the Morning Group and the English Group in the assessment of skills, we could say that the rubrics developed by the UPV are Strategies to assess generic skills for different types of students adequate to evaluate all types of students: Erasmus or non-erasmus, working or having worked in the last 2 years or without work experience, and regardless of their satisfaction with the course.
Having evaluated each generic skill with a different methodology, we observe how students are more generous in self-evaluation and in evaluating their peers in the "Teamwork and leadership ability".On the contrary, the lecturers of the course are more rigorous when evaluating "Ability to think practically and apply knowledge in practical situations" and "Innovation, creativity and entrepreneurship ability" from the report elaborated by the group.
As future lines of research, this educational innovation could be conducted in different degrees of the UPV to validate these rubrics from a multidisciplinary approach.

Table 5 . Assessment of "Ability to think practically and apply knowledge in practical situations".
Source: Authors.

Table 7 . Assessment of "Teamwork and leadership ability".
Source: Authors.