Creating the “examination kiosk”: blended assessment to foster self-directed learning in higher education

The field of higher education is continually evolving; with it, examination strategy must also adapt. Especially self-directed learning will be a focus. We are developing an innovative solution to address the changing needs of university students and their exams. We found current challenges and new requirements for examination strategies based on extensive research analysis of relevant literature in the field and student-based feedback. The solution is a didactical methods in the form of an examination kiosk designed specifically for portfolio examinations. The purpose of the examination kiosk is to promote self-directed learning. It will allow for greater flexibility and individuality in the assessment process. We present the examination kiosk in detail and provide insights into its creation and potential applications.


Introduction
Given the constantly changing conditions of the labor market and the development of technology (e.g. Chatgpt), university graduates need a variety of hard and soft skills. Therefore, it is important to adapt the daily routine in university teaching and the associated examination culture to test in a competence-oriented manner (Dede, 2010;Ayaz and Gök, 2020). We pay particular attention to self-direction, which refers to students' ability to set and pursue goals, make decisions, and manage their learning and development, independent of external guidance or direction (Morris, 2019), which graduates will need in future professional careers. Accordingly, the design of a holistic exam strategy is of high relevance.
In the recently established research project "Portfolio Profis" at the faculty of psychology at Technische Universität Dresden in Germany (TUD), we will design, implement, and evaluate new portfolio strategies for higher education in an iterative process. The search for new solutions became necessary to meet the requirements of the new Psychotherapy Act for the study program in psychology. The conversion of this highly demanded degree program with 120 students each year required the adaptation of the study and examination regulations. Fifteen out of the 16 modules in the study program now have the portfolio as the form of examination. Each of the modules is completed by a graded portfolio based on 30 to 60 hours of work. Portfolio work was chosen as the form of exam to give the teaching staff a high degree of flexibility in designing the exams. It is also intended to reduce the huge amount of feedback and grading required at the end of the semester by assessing performances. It is intended to support students in continuously designing their learning process. However, the basic understanding of portfolio exams at the faculty of psychology differs greatly from that of portfolio work. To improve the quality of teaching, it is necessary to develop and test concepts that combine portfolio work with the requirements of portfolio examinations. However, so far there are no proven concepts for graded portfolio examinations for large groups of students, especially if they aim at a reflective mapping of self-directed learning processes. The current paper, therefore, addresses the following question:

How to develop the concept of portfolio examinations to support self-directed learning processes based on fundamental aspects of portfolio work?
To answer this question, the following sections compare portfolio exams and portfolio work concerning various aspects. Subsequently, a new concept for the implementation of portfolio exams is designed from the comparison and discussed with students of the bachelor's program in psychology at TUD.

Comparison of Portfolio Work and Portfolio Examinations
An extensive literature search enabled us to compare portfolio work with portfolio examinations regarding their structure, use, and potential. We focused the search on publications from German-speaking countries to achieve the highest possible fit for synthesis with the portfolio examinations. Additionally, we filtered based on availability online and date of publication after the year 2000. We found many different types and implementations of portfolios, making a single, universally applicable definition difficult (this is described in Knauf, Behrend, & Knutzen (2020) and Keplinger (2014), among others). To answer our research question and to further develop portfolio examinations towards a more reflective design supporting self-directed learning, it does not seem necessary to differentiate the concept of the portfolio work to different criteria but to elaborate on the basic elements. Therefore, the definition by Hornung-Prähauser et al. (2007) serves as the basis for the present description: A portfolio is "a (digital) collection of "skillfully made works" (=lat. artifacts) of a person who thereby wants to document and illustrate the product (learning outcomes) and the process (learning path/growth) of his/her competence development in a certain time and for certain purposes. The person in question has independently selected the artifacts and organized them about the learning objective. She (he), as the owner, has complete control over who, when, and how much information from the portfolio may be viewed." Based on this definition, the following section describes aspects of portfolio work and contrasts them with the current conditions of portfolio examinations. The result is a collection of aspects that -as the diversity of portfolio use suggests -are not all addressed in each of the individual publications but are derived from a summative analysis (Barrett, 2011;Bauer & Baumgartner, 2012;Bräuer, 2016).
To contrast portfolio work and portfolio examinations we need also a description of portfolio examinations. In terms of the corresponding examination regulations, portfolio examinations are defined in the examination regulation of the bachelor's program in psychology (Technische Universität Dresden, 2021): Portfolios serve, using a compilation of similar or dissimilar individual performances, to demonstrate the ability to place the aspects of professional, scientific action determined by the respective task in a larger context. This includes the ability to work in a team if the respective task requires it. Portfolios can include both face-to-face and distant work, and the result is an objective, e.g. written work. The time required for portfolios is specified in each case in the module descriptions and may not exceed 300 hours. Derived from this, the deadline for the submission of individual performances, the duration of individual performances, and the deadline for the submission of the entire portfolio are to be determined within the framework of the respective assignment.
The aspects of portfolio work and portfolio examination are contrasted in Table 1. The presentation in Table 1 reveals differences in almost all aspects between the basic ideas of portfolio work and the portfolio examinations as they are currently used. The largest degree of similarity can be found in the list of individual components, which labeled differently as either artifacts or individual achievements but the components are similar in terms of form and medium.

Didactic Development of Portfolio Examinations
Next we will identify the aspects of portfolio examinations that require further adaptation. It is therefore necessary to characterize differences to portfolio work but also to take into account to what extent the conditions given by the examination regulations described above allow for an adaptation. Further developments are required and possible with regard to the following aspects: Objectives: Fundamental for an improvement of the teaching quality is that the strengthening of the student's ability to reflect and their self-competence as well as the support of selfdirected learning are added to the goals of the use of the portfolio examinations. While these are the goals of the Bachelor's program in psychology at TUD they are not yet addressed by the current practice of portfolio examinations.
Who decides: It also appears to be quite essential that students are given a greater opportunity to influence the design of their learning and thus the examination process. In a first step, this can concern the form, content, and/or number of individual performances. Therefore, teaching staff should provide a framework -in accordance with the examination regulations -that provide various opportunities for the students. However, the different opportunities wihin a portfolio have to be comparable but also the individual and overall workload has to be in line with the examination regulations that are defined for the respective module. In this context, it could become critical that students are overwhelmed with the possibilities to design their learning and examination process.

Dimensions and Content:
To strengthen the student's ability to reflect on their own level of knowledge, they need to have opportunities to do so, which also includes to reflect on their learning process, experiences and strategies. Therefore, the portfolio assessment should be expanded to include a process dimension as well as reflective content. However, assessing the reflective component in the context of portfolio exams seems difficult. Therefore, this point should also be discussed with the students during the focus group. To ensure general acceptance for a further developed version of the portfolio examinations also requires to consider the supervision effort on the side of the teaching staff. The higher this effort will be, the lower will be the general acceptance and the willingness for implementation.

Student-based Feedback:
To evaluate current portfolio implementations a focus group discussion was conducted with students in the Bachelor psychology program. Eleven students discussed within the framework of a world café suggestions and ideas for the didactic iteration of the portfolio examinations. The following results for further consideration emerged: Students show dissatisfaction with the current practice of portfolio examinations and expressed the need for a development towards a stronger consideration of self-directed and reflective learning. Students do not see any danger of being overwhelmed or unsettled by greater freedom to design their own learning and examination process. One central idea suggests that the teaching staff provides a selection of portfolio assignments to be completed (e.g., 10 individual assignments) and students select and complete a certain number (e.g., 5 individual assignments). This suggestion was very much appreciated. It is important, that students do not have to specify at the beginning of the semester which tasks they want to complete, so that they have enough flexibility for trial and error. Further, they see the importance of including the process dimension and reflection in the portfolio assessment.

Creating the "Examination Kiosk"
The results of the comparison between portfolio work and the current practice of portfolio examinations as well as the information from focus group discussions were used to further develop the concept of portfolio examination. We call the concept the "examination kiosk" and define it as follows: The examination kiosk is a type of graded portfolio examinations. In this concept, students compose their individual assignment combination and thereby design their learning and examination process. Each student selects their own subtasks from the available options. The sum of all subtasks creates the portfolio examination.
For a successful implementation, a smooth transition seems appropriate, therefore we will describe a "start-up version" of the examination kiosk together with prospects for the further development.

Figure 1. "Examination Kiosk"
Objectives: The examination kiosk is used as a form of examination to check the achievement of the module objectives laid down in the study regulations. It also supports self-directed learning and the promotion of the students' ability to reflect and their self-(learning) and action competence. Lectures and students will benefit from a more equal distribution of examinations across the semester as well as from greater flexibility in the design of the module examination.

Dimensions:
The exam kiosk primarily addresses the product dimension, but also considers the process component. This is achieved through an ungraded reflection of the learning process and outcome. In perspective, the reflection may be given bonus points and/or graded.
Structure: At the beginning of the semester, the procedure is transparently presented to students with Video, FAQ, PDF and Miro Board as well as a weekly consultation hour throughout the semester. Thereby, the teaching staff informs the students about the (i) number of available subtasks, (ii) the required number of subtasks that need to be completed by the students, and the scope in work units of each. In order to give students sufficient choice, we recommend that at least six subtasks are offered, of which students may choose four. The content, form, and latest submission date will also be specified by the teaching staff. Students decide during the semester which subtasks they would like to work on and hand in. All submitted individual performances will be evaluated with points. The points will be summed up to a grade at the end of the semester. In an accompanying, ungraded reflection, students answer predetermined questions that address, among other things, the choice of individual performances and the learning process. In the future, students can be given even more decision-making options, e.g. by allowing them to determine the content, number and form of the individual assignments themselves on the basis of their own learning objectives.

Components:
The kiosk consists of different subtasks, which are arranged by the students to their personal Portfolio. The concrete tasks for the subtasks and the expected form are specified by the teaching staff. The forms of the subtasks depend on the respective learning objective that is to be examined. For example, tests, one-minute papers, essays, steps of a research project, forum contributions, memes, videos or mind maps can be subtasks. Ideally, these are products that are naturally created by students as part of their learning process.

Conclusion
The examination kiosk will be used for the first time in the summer semester of 2023. Its use will be closely accompanied by a scientific study so that the knowledge gained can be incorporated into a continuous development process. By using the examination kiosk, the teaching/learning practice and thus the learning and teaching culture can develop further. In the process, the teaching staff gradually change their role to that of learning facilitator. The students are involved in a participatory way and are trained with regard to their self-learning competence. Our long-term goal is to raise the quality of university teaching and its assessment strategies and to train learners to become mature, independent and reflective graduates through complex examination systems to create a participatory learning culture. Furthermore, we aim to support teaching staff to implement complex assessment strategies by educating them how to mix assessment formats, for example by using the Blended Assessment Cube (Jantos and Langesee, 2023).