Evaluating Learner Engagement in Arts Education : Perspectives from Music and Drama in Education

In this paper we aim to investigate learner engagement and how it can be evaluated, in the context of higher education research. Specifically we consider learner engagement evaluation in Arts Education, where the educational focus is on the process, rather than the product – drawing on music and drama in education research and practice. First, we position the notion of evaluation as opposed to assessment, with attention to its etymological roots. Second, we discuss the multifaceted notion of engagement as process, exploring the nature of learner engagement and a number of possible engagement indicators. We then synthesize these categories into descriptors which, we argue, can be useful to evaluate learners’ engagement in arts education practices. Third, we ground theory into practice by offering two examples drawn from the authors’ PhD case study research, respectively in music education and drama in education. We conclude that engagement is a multifaceted construct, which we frame as a mutual exercise of agency – whereby the teacher and the students act in a partnership as co-artists.


Introduction
In this paper we set out to investigate the nature of learner engagement and how engagement can be evaluated in the context of higher education research.The discussion on engagement and its evaluation is situated within the context of Arts Education, more specifically within music and drama in education, where the educational focus is on the process, rather than the product.In drama-based pedagogy, also known as process drama (O'Toole, 1992), the focus is on meaning-making to experience a collective 'dramatic elsewhere' (a process), rather than rehearsing and performing for an audience (a product).Similarly, in process music the focus is on practicing and listening to explore meaning transactions and possibilities (process) rather than music reading and performing skills to create recital events (product).In these contexts, as well as other arts-based practices, where the nature of the artistic form summons a process-based pedagogy, the need to evaluate engagement as/within the process clashes with the psychometric requirements of the academic assessment system.This paradox opens up a number of questions, paved by key educational theorists like Dewey (1934), and more recently Kress (2009) and Fautley (2010), related to the effectiveness of contemporary assessment.In the field of Arts Education, key scholars have long pondered on such matters, reflecting on the importance of framing the arts in education in terms of evaluation, rather than assessment (Abbs, 1989;Eisner 1985).
Before proceeding any further, it is important to define the terminology that will be used throughout the paper, particularly the difference between 'evaluation' and 'assessment'.On an etymological level, the term 'evaluation' comes from the French noun évaluation, with the verb évaluer meaning 'to find the value of'.'Assessment', on the other hand, comes from the Latin assessus, sitting-by, past participle of assidere, 'to sit beside'.In its Medieval Latin origin, 'assessment', from assessare was related to tax collection, whereby a judge's assistant would 'sit beside' a person to estimate the amount of a fine or a tax.Evidently, the terms 'evaluation' and 'assessment' have very different roots, and have been taken up by contemporary society to mean different things, respectively looking at the 'how' (process) and the 'what' (product) in students' engagement.In this paper we subscribe to a 'process' vision of evaluation, within the construct of learner engagement, as illustrated below.

Engagement as Process
Learner engagement is an elusive concept, which cannot be easily pinned down.Any experienced educator would agree that it is somehow connected to learner motivation, emotion and autonomy; yet when it comes to capturing the elements that inform this notion, these parameters become less tangible.In the last three decades, several key educational theorists have attempted to define it.As Svalberg (2009) argues, "researchers and practitioners alike are currently trying to come to grips with 'engagement' as the place where learning happens" (p.243).Interestingly, in Kress' (2009) definition of learning, 'engagement' is encompassed within the learning process.In fact, he defines 'learning' as "the result of a semiotic/conceptual/meaning-making engagement with an aspect of the world" (p.19).Kress (2009) subsequently defines 'engagement' as: "The meaning-maker's 'interested', energetic and sustained involvement with a framed segment of the world, which is at issue in an interaction" (p.38).
Within this general definition of engagement, our research focuses on a particular aspect of learner engagement within the arts, and that is, engagement with languagewhere language is intended in its verbal and non-verbal domains.To this respect, while Author's (2014) research deals with understanding the acquisition of a second language by using drama in education, and the nature of learner engagement resulting from such intervention, Author's (2014) research involves looking at engagement as intrinsic/extrinsic motivation in music education.The problem with attempting to define learner engagement is that it can afford a dual, at times contradictory interpretation, both as a process and as a product.Svalberg (2009) acknowledges this issue by offering a definition of engagement with language that encompasses both: "Engagement is a cognitive, and/or affective, and/or social state and a process in which the learner is the agent and the language is the object and may be the vehicle (means of communication)".(p.244) Svalberg's (2009) definition positions engagement as a process and a state, across three axis: cognitive, and/or affective, and/or social.In our vision, in line with a Deweyan philosophy of education (Dewey, 1934), "and/or" in the definition above can be safely replaced with 'simultaneously'that is, engagement within the arts in education is simultaneously cognitive, affective and social.Moreover, we argue that another fundamental dimension of learner engagement is aesthetic engagement, indented as learners perceiving through the senses, internalizing and re-elaborating form through symbolic and metaphoric means, to express their own meaning (Abbs, 1989;Vygotsky, 2004).

Learner Engagement as Agency
Van Lier (2008) advocates 'agency' as key to the engagement and learning process.If seen as a perceptual process of attention, engagement with learning requires an 'active perception' or "perception-in-action", a phrase coined by Van Lier (2004, p. 97) drawing on Vygotsky and Luria (1994), who define learning as a "newly born unity of perception, speech and action" (p.109).Van Lier (2008) frames agency as "something that learners do, rather than something learners possess; in other words, it is behaviour, rather than a property (p.171).Van Lier identifies three qualities of agency: 1. Agency involves initiative or self-regulation by the learner (or group); 2. Agency is interdependent, that is, it mediates and is mediated by the sociocultural context; 3. Agency includes an awareness of the responsibility for one's own actions vis-à-vis the environment, including affected others.(2008, p. 172) We endorse Van Lier's definition and extend it further, with attention to the relationship between the students and the teacher.In Van Lier's third point, we argue, 'affected others' does not only refer to other students, but can also refer to the teacher.As a matter of fact, in process drama and process music agency significantly affects, and is affected by, the pedagogical relationship between the students and the teacher.Here we draw on Sen's (1984) discourse of agency as freedom and well-being, what the "person is free to do and achieve in pursuit of whatever goals or values he or she regards as important" (p.203), and proceed to frame agency in terms of the active exchange of students' and teachers' ideas, resulting in a 'flourishing' (Snow, 2015) and an ongoing challenge to opinions, attitudes and, ultimately, identity.

Indicators of Learner Engagement as Agency
What indicators can be used to evaluate such engagement process?In our research, described in the section below, we identify a number of indicators, manifested as a change in the meaning-makers' cognitive, emotional, social and imaginative perception-in-action, occurring within verbal and non-verbal planes.These indicators have been synthesized into three descriptors: 1) Agency as autonomy, intended as: a sense of ownership of the work; a willingness to create; willingness to self-assess; 2) Agency as risk-taking, manifested as: willingness to share one's creations; making discoveries; making choices; opening up one's vulnerabilities to the group and to the teacher; making an emotional investment in the work; 3) Agency as playfulness, intended as: willingness to question reality, including self, others, the teacher and the curriculum; exploring different meanings; injecting irony through symbols and metaphors.In other words, we define the student-teacher agentic relationship as a meaning transaction requiring a degrees of autonomy, risk-taking and playfulness towards the self, others, the teacher and the curriculum.

Case Studies
In the section below we offer two examples to ground theory into practice, drawing on our research in music and drama in education.Within the scope of this paper, we limit ourselves to giving an overall portrait of the research design, and including one significant anecdote that can illuminate an aspect of the discussion.

Case Study One: Music Education
This case study is part of a wider doctoral study on music education, undertaken at Ca' Foscari University (Italy).The data collection, grounded in a mixed methods paradigm, took place in 2014, in two music-oriented Middle Schools in Rome (experimental and a control group).The research design involved designing and facilitating an 18-week long process music intervention with the experimental group (seven 10 year-old students).Research methods included: filming the process music and control group lessons, interviewing the control group's music teacher, and administering four pre-and postintervention tests: Advanced Measures of Music Auditions (AMMA) Edwin E. Gordon; Torrance Test of Creative Thinking (TTCT Figural); Thinking Creatively with Sounds and Words (Level 1 TCSW Torrance, Katena-Cunningrom); Cornell Critical Thinking (CCTT Level x).The data analysis was conducted using MAXQDA video analysis software, cross referencing the video recording and transcription data with the researcher journal, control teacher's interview, pre-and post tests, students' self-evaluation and music compositions.
For the purpose of this paper, the discussion zooms on one 10 year-old student, Giorgio, and his evolution throughout the intervention.This analysis is focused on a series of 11 one-to-one guitar lessons (40 minutes each) between Giorgio and the teacher-researcher.Before the intervention, Giorgio had studied guitar for three months in a traditional, transmissive way based on reading and practicing.He had basic technical skills (First position, beginner right hand technique and music reading) and in the first meeting he appeared uninterested about doing music classes again.He also seemed inclined to find shortcuts to gain the maximum results for minimum effort.But even so in his first class, upon composing his first piece, he started to develop a sense of ownership to the work, making choices about his music writing: No. [He observes carefully his composition for about two minutes] C is not right here!Or: [To the teacher-researcher playing out his composition] Could you play from here?I want to hear my piece clearly.His personal mind-set to music making emerged in the third lesson: There is something I want to do.I'd like to compose a piece for that [showing a piece of paper] During our Italian class we read some poetry and I want to write something based on that.His risk-taking and emotional investment to music making became clear in the fifth lesson.He wrote a piece because he, confessed, was infatuated with a girl, and asked: Could you come with me tomorrow?I want to play this piece and I'll need some help to play perfectly on time.At this point his Evaluating Learner Engagement in Arts Education composition was becoming more personal.He started to come up with his own technical solutions when he did not know how to write something, inventing his own music writing.He also started to correct the teacher-researcher, in terms of errors in transcribing his composition on the computer, as well as how his compositions were played.Over the course of the 11 lessons, something changed in the way he listened other pieces of music.One time, the teacher-researcher asked him to listen to the 4th movement, 'Allegro Pizzicato' from the Bartok's String Quartet No.4 in C major, to show Giorgio certain similarities with his composition.After listening, he stated: They play wrong here [Points to one section of the score] This was meant to be played piano (p) [he plays on his guitar].It's a piano-pianissimo (ppp).In my next composition I also want to use ppp.Four classes before the end of the intervention, the other students decided to attend a music competition to play their compositions in front of an audience.At this possibility Giorgio said: I'm not coming.What for?How many other schools will be there?Yet, after two weeks, he completely changed his mind and decided not only to participate in the competition, but also to conduct his own piece, which he had written for a guitar ensemble.
From this brief account of the data collection, learner engagement in the process manifested as autonomy (ownership of the work; willingness to create) risk-taking (sharing creations; making choices; emotional investment) and playfulness (willingness to question reality; exploring different meanings), with the teacher playing a key role in this mutual exchange.

Case Study Two: Drama in Education
The case study below is part of a wider doctoral study on drama in education, undertaken at Griffith University (QLD, Australia).The aim of the research was to explore the constructs of learner engagement and teacher artistry, when using process drama to teach Italian as a Second Language (L2).The data collection took place in three L2 schools (one University and two private schools) of adult international students in Milan, Italy.The research design saw the teacher-researcher create and facilitate 45-hours of process drama, spread across three case studies, with 60 participants: 46 international students, who took part in the drama workshops, and 16 teachers, who observed the workshops to take notes on learners' engagement.Data collection methods included: filming the process drama lessons; interviewing the student-participants; interviewing the teacher-participants (before and after the observations); analyzing teacher-participants' observation notes; administering a selfevaluation engagement questionnaire to the student-participants after each drama workshop; conducting a final focus group with all participants, using Video-Stimulated Recall (VSR) and other embodied approaches such as using props and image theatre to trigger reflection on their engagement.The data analysis was informed by a microgenetic approach and conducted using NVIVO 10 analysis software.
For the purpose of this paper, the discussion briefly focuses on one 27 year-old Russian student, Yelena, who seemed to have connected with an idea beyond the drama, related to 'time being an illusion'.At the end of her fourth workshop, she approached the teacherresearcher privately, visibly thrilled, whispering she just had an insight.Yelena disclosed a strong connection related to a discussion on the relativity of time and space.This insight affected her profoundly during her involvement with the drama.In the focus group, Yelena pointed out how her identification with the drama was so strong that, after the drama, for two days, she was deeply engaged with the drama: I was still inside the situation, with my head, with my thoughts, with my senses.Yelena's self-evaluation score for her engagement in workshop 4 was 10/10; she also wrote some additional notes in her questionnaire: "Emotional; I had goose bumps at the end of the class".Interestingly, during that specific workshop, her intense connection created an internal conflict with the group.In fact, as they were improvising, in role, using a strategy they had previously agreed upon, Yelena changed tactic of her own accord, causing confusion and frustration in the others.When prompted to comment, Yelena replied: When [the main character in the drama] came in… nobody made a suggestion, so… I took the initiative in my hands and I changed topic and nobody really understood this.After the improvisation, this incident created a heated discussion in the group, who openly confronted her about it.This instance was framed by Linda, a teacher-participant, as "evidence of high engagement" (p.9:22-23), in terms of the group emotionally investing in the dramatic situation.Thus, paradoxically, Yelena's strong connection with the work triggered a sense of autonomy that initially disconnected her from the group.Eventually Yelena apologized to her classmates and the group bonded even more.The teacher-researcher channeled this episode, and the reflection that followed, to heighten the dramatic engagement in the drama.This incident, and the discussion that followed, heightened Yelena's awareness of her own attitude during team work: I learnt that the group can be like a person, like for example the sun and its rays [mimes the rays]this is a group, but all come in the centre.Yelena's experience suggests that by exercising agency as autonomy, willingness to become vulnerable and emotional investment in the group, she became fully engaged, deepening her understanding of herself.Yelena's engagement here, manifesting as agency, combined with meta-reflection, resulted in a shift of opinions, attitudes, in a kind of 'flourishing' (Snow, 2015) that may have impacted on her sense of identity.By looking at both case studies, it is obvious that the meaning-makers' engagement manifested as simultaneously cognitive, affective, social and aesthetic.

Conclusion
Autonomy, risk taking and playfulness.Three descriptors that may help arts education researchers to evaluate engagement.The question is: for what kind of education?We have argued that engagement is closely related with the notion of agency.We suggest that placing agency at the centre of the curriculum also means to highlight the students' uniqueness, subjectivity and talents.Accordingly, we wish to consider the possibility of extending this further to reframe Arts Education following a new "humanistic approach" (UNESCO, 2016).Future research could focus on teacher education and pedagogical practices and policies, with agency at the heart of a teacher-student relationship, in a mutual partnership as co-artists.In other words, shifting paradigms from engagement as aesthetic judgement, to engagement as co-creation of meaning.