Conceptual Framework for Assessing Process Variables Salient for Service-Learning Experience

Service-learning is an established pedagogy which integrates experiential learning with community service. It has been widely adopted in higher education around the world including in Hong Kong, yet the key ingredients that determine its successful impacts for its stakeholders have not been fully assessed. This study reviewed the past literature, which indicates the key ingredients that may be found in successful service-learning programmes. We identify six key ingredients: students provide meaningful service; the community partner representative plays a positive role; effective preparation and support for students; effective reflection by students; effective integration of service-learning within the course design; and stakeholder synergy in terms of collaboration, communication and co-ownership. In order to obtain an inter-subjectively fair and trustworthy data set, reflecting the extent to which those key ingredients are perceived to have been achieved, we propose a multi-stakeholder approach for data collection, involving students, instructors and community partner representatives.


The Need to Assess Key Ingredients That Drive Benefits in Service-Learning
Service-learning, a form of experiential learning which aims to enable and empower students to apply knowledge learnt in class to serve people and or organizations in a community setting, has been adopted by a number of higher educational institutions in Hong Kong over the past decade (Snell & Lau, 2020). In adopting service-learning pedagogy, these institutions appear to adhere to the original objective of service-learning, defined as "a form of experiential education, in which students engage in activities that address human and community needs together with structured opportunities for reflection, intentionally designed to promote student learning and development" (Jacoby, 1996, p. 5).
A broad range of intended developmental outcomes for students have been invoked. For example, Lingnan University's service-learning programme aims to enhance seven graduate attributes, including problem-solving skills, critical thinking skills and social competency (Ma & Chan, 2013). The Hong Kong Polytechnic University launched its service-learning programme in 2010 to facilitate four developmental outcomes: a) knowledge and skill application; b) empathy, civic engagement and responsibility; c) becoming professional and responsible citizens; and d) connecting between the academic content and the need of society (Chan & Ngai, 2014).
As a rigorous pedagogy that has received much attention and development over the last two decades, the learning outcomes of service-learning have been extensively researched and documented (e.g. Celio et al., 2011;Conway et al., 2009;Warren, 2012;Yorio & Ye, 2012). However, the key ingredients regarding how to implement successful servicelearning projects, which lead to positive outcomes, have not been as systematically investigated as the outcomes themselves, particularly in Hong Kong. The current paper therefore builds an analytical framework based on the past literature, and goes on to propose a measurement instrument based on the key ingredients associated with successful service-learning projects, so as to complete a missing link in service-learning research in Hong Kong and beyond.

The Key Ingredients Proposed for Successful Service-Learning in Past Literature
In his book, "Service-Learning in Higher Education", Jacoby (1996) listed a series of good practices in service-learning. These comprise five ingredients: a) make sure that community voices are included in developing the service-learning programme; b) sufficient orientation and training for students to engage in and learn from their service-learning experience; c) meaningful action, which means that the service components of the service-learning are necessary and valuable for the community; d) effective reflective activities for students to consolidate their learning after service-learning experience; and e) effective evaluation that measures the impacts of the service-learning on students and on the community. Jacoby (1996) also stressed other important principles, derived from previous studies, including: continuous improvement of service-learning programmes; provision of sufficient support and coordinating mechanisms; minimizing the distinction between the student's learning roles in the community and in the classroom; maximizing the orientation of the respective courses toward encouraging responsibility for the community; not compromising academic rigor for service contribution; and crediting students for demonstrating their learning and not for providing the service.
In addition, Jacoby (1996) proposed a service-learning 'kaleidoscope' as a means to assess the extent to which the above principles have been put into practice for a given servicelearning project. The kaleidoscope encompasses three aspects of stakeholder collaboration. The first aspect involves common goals and purposes, including shared responsibility and authority, sharing of resources, regular exchange of information, mutual adjustment, mutual capacity enhancement, and mutual trust. The second aspect is reciprocity, under which every stakeholder functions as both learner and educator, thereby avoiding exploitation. The third aspect is diversity in the service-learning context, which enables participants to appreciate and respect human differences and not be confined by one's own perspective.
Around the same time, Eyler & Giles (1999) reported findings from a research study that gathered data from interviews with students regarding what made their service-learning effective. Eyler & Giles (1999) distinguished five programme characteristics that are predictors of effective service-learning outcomes. They are: a) placement (or service) quality; b) quality of knowledge application; c) inducement of effective student reflection; d) exposure for students to diverse groups; and e) influence of community voices.
Although Eyler & Giles (1999) found that the above five characteristics were in most cases positive predictors of students' development, there were sometimes exceptions regarding diversity and community voices. Sometimes these were negative predictors, due to tensions arising from differences between the expectations and interests of different stakeholders. Furthermore, Eyler & Giles (1999) invoked the Five C's principles for effective servicelearning reflection: Connection, Continuity, Context, Challenge, and Coaching. Godfrey et al. (2005) identified three critical ingredients for successful service-learning and have characterized them as the 3 Rs. The first of these is Reality, in terms of how servicelearning can connect real situations with academic content. The second is Reflection, which enables students to think deeply about their service-learning experience and its personal impact on themselves. The third is Reciprocity, which involves the students and the community contributing and combining their different bodies of knowledge and working together so that synergy occurs as both parties gain from their collaboration.
Hong Kong based researchers, Chen et al. (2018) identified three interrelated variables in addition to the above-mentioned 3 Rs, which help distinguish between service-learning project experience and non-service-learning project experience. The first variable is project experiences. These are primarily set up by a centralized office of service-learning, in order that there is an effective communication network that connects the salient university and community stakeholders. The second variable comprises partner organization representative responsiveness, i.e. POR responsiveness, the extent to which PORs offer open access for students to people, resources and information that are essential to furthering the project. The third variable is project efficacy belief, which represents the students' perceptions that their projects will make a positive difference, and derives from the 3Rs, plus a sense of mastery experience, perceptions of enacting effective social persuasion, and positive role modeling by other students engaged in the same service-learning project.
The findings of Chen at al. (2018) resonate with an earlier paper by Snell et al. (2015), which derived a set of ten principles for service-learning based on a qualitative study in Hong Kong. The first of these is that service-learning projects should address authentic problems or needs. Second, PORs should commit to their ongoing availability for consultation. Third, students should receive a complete orientation to service-learning. Fourth, project themes should align with the course curriculum. Fifth, there should be a foundation of inter-institutional commitment and trust. Sixth, there should be initial site visits by students prior to the main project phase. Seventh, there should be in-class project consultations during the service-learning phase. Eighth, there should be shared and supportive leadership within the student project team. Ninth, instructors should provide templates to help students to reflect on the processes and outcomes of their personal development. Tenth, project reports and reflective reports should be included among the graded coursework assessment requirements.

Operational Process Variables for Successful Service-Learning Experiences
Based on the above discussion about the past literature, the key ingredients that contribute to successful service-learning experiences and outcomes can be analyzed into six overall ingredients. The first of these is meaningful service, comprising significant action, inclusion of community voices in establishing service needs, and the experience for students of encountering social diversity during the service. The second is that the POR plays a constructive role by, for example, being readily available for and responsive to questions. The third is that the students receive effective preparation and support, including domainspecific training, orientation, consultation during the service-learning project, and logistical support. The fourth is engagement in effective reflection as a means for connecting the student's experiential learning during service to the course curriculum. The fifth is effective course design, such that project themes are closely linked to the course curriculum, and that student's project reports and reflections an integral part of the assessment requirements for the course. The sixth is stakeholder synergy, based on effective collaboration, coownership, and communication, along with reciprocity in terms of resource commitments and derived benefits. Table 1 summarizes how the six key ingredients proposed in the current paper correspond to the literature reviewed in the previous section.  (Jacoby, 1996) b. Addressing authentic problems (Snell et al., 2015) c. Reality (Godfrey et al., 2005) d. Project efficacy belief (Chen et al., 2018) e. Community voices are included (Jacoby, 1996) f. Influence of community voices (Eyler & Giles, 1999) g. Service quality, diversity in service (Eyler & Giles, 1999) 2. POR plays a constructive role a. POR responsiveness (Chen et al., 2018) b. POR commitment (Snell et al., 2015) 3. Effective preparation and support a. Sufficient support, coordination, orientation & training (Jacoby, 1996) b. Project experiences (Chen et al., 2018) c. In-class project consultation (Snell et al., 2015) 4. Effective reflection a. Effective reflective activities (Jacoby, 1996) b. Effective student reflection (Eyler & Giles, 1999) c. Reflection (Godfrey et al., 2005) d. Measures to enhance student reflection (Snell et al., 2015) 5. Effective course design a. Service aligning with course curriculum (Snell et al., 2015) b. Quality of knowledge application (Eyler & Giles, 1999) c. Effective evaluation (Jacoby, 1996) d. Crediting students for demonstrating their learning (Jacoby, 1996) e. Grading service project results (Snell et al., 2015) 6. Stakeholder synergy a. Common goals, purposes, responsibility & resources (Jacoby, 1996) b. Reciprocity (Godfrey et al., 2005) c. Inter-institutional commitment and trust (Snell et al., 2015) 57

A Multi-stakeholder Approach for Assessing the Key Ingredients
Success in service-learning requires collaboration between the stakeholders, who comprise students, instructors, PORs, end-beneficiaries of the service, and the coordinating centre, which is typically an office of service-learning (Wade, 1997). Accordingly, for assessing the extent to which the aforementioned key ingredients are present, we propose a multistakeholder approach, using mainly quantitative data collection methods, supplemented where appropriate by qualitative methods, with questions about particular key ingredients addressed to the salient stakeholders. For example, ingredients related to the constructive role of the POR will be answered by instructors and students, whereas whether students engage in effective reflection will be assessed by instructors and PORs. Although the experiences of end-beneficiaries are important in service-learning, for various reasons (e.g., time availability, contactability) it may not be feasible to collect data directly from them about their perceptions. As a pragmatic approach, we suggest that the perceptions of endbeneficiaries can be reflected through the observations of the POR as a proxy. Table 2 lists the proposed items for measuring key ingredients in the process of servicelearning that were synthesized from the literature presented in the previous chapter and according to the above multi-stakeholder framework.

Process Aspect/ Variable
Sample Items Answered by* I S C 1. Meaningful service a. Significant action i) The service was aligned with the real needs of our service recipients ii) The service providers (students) were able to address the concerns of our service recipients iii) The service providers (students) were able to make a positive contribution for our service recipients iv) The service providers (students) were able to help the partner organization to improve its service Conceptual Framework for Assessing Process Variables Salient for Service-Learning Experience v) All stakeholders were willing to go the "extra mile" for the project vi) All stakeholders were able to think "outside the box," and apply questioning insight b. Reciprocity i) During the service, the end-beneficiary or beneficiaries was/were able to learn from other stakeholders, such as the student, the instructor and the community partner(s) ii) During the service, the students were able to learn from other stakeholders, such as the end-beneficiaries, the instructor and the community partner(s) iii) During the service, the instructor was able to learn from other stakeholders, such as the students, the endbeneficiaries and the community partner(s) iv) During the service, the community partner(s) was/were able to learn from other stakeholders, such as the student, the end-beneficiaries and the instructor Y Y Y Remark * I: Instructors; S: Students; C: PORs.
In addition to gathering data about the above process variables, we also consider that it is important to obtain the POR's perspective on three broad community impact domains, namely, perceived capacity enhancement; perceived benefits from furthering the mission and values of the partner organization; and new operational insights for the POR.

Conclusions
Based on a review of prior literature, this paper has identified the importance of assessing the extent to which six key ingredients of successful service-learning projects are present. Among these six ingredients, it is important that students provide meaningful service, and that the POR plays a constructive role. The preparation and support provided to students, the reflection activities in which they engage, and the integration of the service-learning project within the course design should all be effective. In addition, there should be stakeholder synergy in terms of collaboration, communication and co-ownership. For data collection, it is proposed to solicit the perceptions of students, instructors and PORs about salient ingredients, treating the POR as a proxy for the end-beneficiaries. We have proposed some sample survey items for the key ingredients.