EPIC in action, measuring entrepreneurial competencies in higher education

Increasingly university programmes are introducing a range of experiential learning based programmes to support students to develop their entrepreneurial competencies during their time at university. This paper describes how the University of Galway is utilising the Entrepreneurial Potential and Innovation Competences Tool (EPIC) to track the changes in student self-reported competencies having participated in one of its flagship student entrepreneurial programmes. Based in Ideaslab, the university’s Human Centre Design Studio, the approach used is experiential, design centric and informed by D.School, Stanford. Initial findings from the EPIC surveys completed by 23 students are reported. These data are part of a university wide initiative and further data will be collected over the next three semesters. In so doing, we hope to add to the body of knowledge concerning the utility of this approach to measuring the changes in entrepreneurial competencies following participating in a university based entrepreneurial learning activity.


Introduction
framework for entrepreneurship education distinguished between education about enterprise, education for enterprise and education in enterprise. The core educational approach taken by the Human Centred Design studio in this case study is located within both the "education for" and "education in " approaches. In particular, the team utilise "Design Thinking" as a methodology across their programmes. They focus on fostering design thinking as a basis for practical engagement in developing innovative solutions to real world problems (Brown, 2008). Using the Stanford d.school1 approach, students in Ideaslab learn to apply active problem solving as they work on real life challenges. Working within this methodology, IdeasLab are part of an international movement to use design thinking approaches to enhance the provision of entrepreneurial education (Linton & Klinton, 2019;Mueller & Thoring, 2012;Sarooghi et al., 2019).
Within the University of Galway, the programmes offered by Ideaslab are currently being enhanced and developed as part of the Designing Futures Initiative, a suite of complementary and connected initiatives and modules, focused on developing the attributes, dispositions and skills required of graduates, to best support them in the challenges they face in the world of today. This overarching project is being tracked using a formative evaluation approach (Patton, 2010). Specifically, the evaluation team decided to deploy the EPIC tool (https://www.heinnovate.com) to explore whether participation in IdeasLab fosters the development of the entrepreneurial competencies of the students who take part.
The Entrecomp framework was developed by the Joint Research Centre (JRC) of the European Commission with the aim of building "a bridge between the worlds of education and work, by contributing to a better understanding and promotion of entrepreneurship competence in Europe" (Bacigalupo et. al. 2016, p.7). The framework consists of 15 competencies and detailed learning outcomes across three thematic areas, Ideas and Opportunities, Resources and Into Action. In 2018, the European Commission established the EEEPHEIC project (Evaluation of Entreprenuership Education Program in Higher Education Institutions and Centres) to facilitate the development of common evaluation and measurement frameworks for entrepreneurial education (Baggen & Kaffta, 2022). One specific outcome was the publication of the EPIC tool (Entrepreneurial Potential and Innovative Competences), based on the Entrecomp framework. It is a course assessment tool that can be used by education providers to assess the effectiveness of the training provided. A series of thematic statements are provided across five domains of entrepreneurial competencies, entrepreneurial intentions and attitudes, enterprising behaviours, entrepreneurial strategies and education effects. The EPIC tool is freely available on the HEInnovate website. It is a design that it can be used flexibly and modified according to the cohort of students or learning situation. In this instance, the evaluation developed an online survey to be circulated to students participating in two Ideaslab programmes across one semester. This paper describes the programmes provided, the survey tool deployed and some initial findings from the data collection.

Programme Description
Two types of programmes were included in this case study. Firstly, 38 students participated in an enterprise challenge that was delivered through a combination of class room activity, team based work in students own time and mentoring sessions with external enterprise partners. Table 1 below provides an overview of the classroom content delivered to challenge participants. At the outset of the programme, each student was assigned to one of eight separate teams. These teams are multidisciplinary and span students from first year undergraduate to Phd level. The challenges consisted of four personas detailing a real life situation for the enterprise partners. Each partners provided 4 sessions of mentoring to individual teams over the course of the challenge. These were to guide and support the students in their problem solving and idea development. Ideaslab staff facilitated the teams to make contact with their mentors at the arranged times and issued reminders about mentoring sessions at the weekly content workshops. In addition, the enterprise partners gave their time and expertise to attend the final team pitches and the selection of the winning ideas. These pitch sessions lasted approximately 2 hours, with four teams presenting to each of the two partners at two separate sessions. The challenge ended in week 8 of the semester with the selection of a winning pitch based on the effectiveness of the proposed solution in meeting the challenge set out by each of the enterprise partners.
The second activity in this case study is the Ideaslab internship programme is a paid placement for University of Galway students to work on projects with the Ideaslab team over 10 weeks in semester. Students are encouraged to work up to 4 hours per week on assigned tasks and are supported by a developmental scholarship for their participation. Students were recruited through an open call through Ideaslab and the college social media channels early in the semester. They applied to Ideaslab using a CV and a personal statement. A number were then selected for interview and 11 students were successful in becoming an Ideaslab intern. The internship programme offered a range of opportunities for students to get involved in Ideaslab projects over the course of the semester. At the outset, students were asked to write a short biography and publish it to the Intern team SharePoint site as an aid to building a community across the team. They were asked to update their LinkedIn profile and engage with the Ideaslab social media channels. Students were also asked to begin and maintain a reflective practice log. In this log, they were asked to identify their goals for the placement, to make weekly updates on their progress and complete a short reflection of their experience at the end of the internship. Interns were also able to attend sessions in Ideaslab on Design Thinking during the placement as well as engage with the Student Success Coaching2 service.

Survey Tool
The evaluation team reviewed the online EPIC tool and identified a sub set of 20 statements across three of the five domains that best matched the course content. A number of demographic items and open ended items were included. The survey was circulated to all participants at the end of each of the programmes using Qualtrics. Ethical approval for the research was secured from the university's research ethics committee.

Programme Participation
Across both programmes, a total of 49 students participated. The profile of this group of participants indicate a high level of representation of both international students (55.1%) and postgraduate students (53.1%). The Ideaslab team reported that this semester's participants was an outliner in terms of international and post graduate students and plan to track participation levels to monitor this issue next term to establish if this is a recurring trend.

Results
The survey was completed by a total of 23 students across both programmes, a return rate of 46.9%. Figure 1 compares the profile of survey returns to the overall profile of programme participants, illustrating a representative sample. Table 2 below provides a breakdown of the EPIC scores, illustrating the initial mean score across the group, the final mean score and provides the calculated difference. Notwithstanding the small sample size, a number of initial points can be made. Firstly, the area with the most overall increase in scores is in the "Ideas and Opportunities" area. This finding makes intuitive sense given the design centric focus of the programme. It is likely that this trend is reflected in the gains in the sub items on "looking for new opportunities" and "want to solve problems in new ways". Secondly, there is also a positive trend in a number of the items dealing with team engagement and working with others, "convince others to engage" "actively participate in teamwork". Although as will be noted below the feedback on team engagement was developed further by considering comments made in the open-ended items included in the survey. Finally, the least development in scores occurred in the item dealing with "estimated budgets". This finding does reflect less engagement on this topic during this iteration of the programme.

Discussion
The EPIC tool did indicate positive impacts of participation on self-reported scores. However, as noted earlier, this particular sample was quite small (n=23). It will be possible to collate EPIC scores into SPSS for each of the next three semesters to track score changes across a larger sample, checking to see if there are different scoring patterns according to number of events attended or demographics etc.
The tool currently uses a device to allow participants to complete the survey at the end of the programme with a sliding scale to score each competency at their level both when they started and also at the end of the programme. A number of students in this cohort reported that it was difficult to be consistent using this approach and to estimate their level at the start of the event. As a result, the research team have modified the approach so that next semester students will rate their competencies on two different surveys, one at the start and one at the end of the programme.
While the items on the EPIC tool did indicate positive impacts on team engagement, a number of students indicated tensions with managing the time commitment, maintaining contact across the team and promoting team engagement in the open-ended survey items. Based on this feedback, the Ideaslab team have decided to provide an increase focus on team building on the start of the programmes to promote the positive development of team cohesion. In addition, as the semester progresses, they will regularly remind participants to link back for support and advice if challenges with team engagement persist.
There was also interest in including an item in the survey to get feedback from participants on how they would share a "bonus" with team members as a proxy measure of team effectiveness. The evaluator will source some options on this issue for the team to consider and include in the next semester's data collection.

Conclusion
Baggan and Kaffka (2022) recommended that "the capability of European citizens to engage with the unknown and deal with uncertainty should be developed from childhood. Experiential or challenge-based learning programmes address this need and its offer should be ensured via progression lines from primary education to higher education and beyond, in the form of lifelong learning (both formal and informal) of adults" (p.10). This paper has described the deployment of the EPIC tool to measure the effectiveness of participation in two experiential learning opportunities aimed at increasing entrepreneurial competencies in a Higher Education setting. The initial data from this first cohort indicates that this measure has potential to track these changes in students' self-reports. In particular, the tool seems to be sensitive to changes in those specific areas of design and ideation focus central to these particular initiatives. Some amendments have been suggested to the deployment of the tool in the next round of data collection in this project. Further analysis will be carried out to consider the suitability of the tool with the total cohort of students over the course of the project.

Funding Acknowledgement
Designing Futures (DF) is a flagship programme which received €7.57m funding through Human Capital Initiative, Pillar 3 by Ireland's Higher Education Authority (HEA, 2020).