Students ’ surveys and involvement in educational activities within virtual environments are related to students ’ satisfaction in e-learning graduate programs

This paper evaluates associations across the dimensions included in students’ surveys and virtual activities for applying educational models within e-learning graduate programs. Firstly, we focus on the outcomes derived from teachers’ and students’ participation in virtual classroom forums and, secondly, on the determinants of students’ satisfaction in the graduate program. Data analyses show that teachers’ and students’ participation in forums as dynamic educational activities are positively correlated with students’ general satisfaction. To study the determinants of students’ satisfaction, we perform a regression analysis that considers as explanatory variables educational planning, teaching qualifications and development of learning, as well as of a number of controls related to the virtual learning environment and participation in the academic program. According to the obtained results, teaching skills and learning environments are associated with higher students’ satisfaction in a virtual learning environment. This type of analysis is of great interest in a social environment characterized by increasing communication via electronic networking. We find that dynamic educational activities and dimensions taken on board on students’ surveys are related to students’ satisfaction in e-learning graduate programs.


Introduction
The potentiality of virtual learning environments (VLEs) to develop higher education students' competencies and knowledge might be associated to the changes that are being experienced worldwide with the development of the so-called network society (see Castells, 2010 and2016;and Márquez-Ramos & Mourelle, 2017).As it occurs in traditional learning environments, evaluation systems in VLEs involving lecturers' performance are a tool for universities to evaluate the degree of compliance of their lecturers.These evaluation systems are included in more general evaluation of educational models applied in universities.Evaluation models generally include some of the questions or dimensions included in students' surveys, based on the idea that students learn more from highly rated lecturers (Uttl, White, Wong-Gonzalez, 2016).This fact determines that models that support management teaching quality systems have their base on the satisfaction of requirements and expectations of actual and potential users (e.g., students...) (Duart & Martínez, 2001).
In this study, we analyze the relationship between the organization of the teaching process in virtual conditions and the dimensions of the assessment carried out by the students.To do so, we focus on the DOCENTIA-UJI model that involves four dimensions included in the students' survey as an evaluation system for lecturers' performances (see http://www.uji.es/serveis/opaq/base/gestio-qualitat/aval-doc/docentia/ for a detailed description).In VLEs contexts, lecturers are not a content transmitter, but a learning environment designer.They have to join students, who are the protagonists of the learning process, creating a climate of interest and giving autonomy in all this process (Bautista, 2011).The student has to maintain an active attitude in her interaction with others (lecturer or other students) being competent in a relational level (Sancho & Borges, 2011).
As Laurillard (2002) reported, the VLEs can support different media forms of learning technologies that could alter the educational environment: narrative media for attending and apprehending, interactive media for investigating and exploring, communicative media for discussing and debating, adaptive media for experimenting and practicing, and productive media for expressing and presenting.However, VLEs are principally focused in narrative and interactive media (Lameras et al., 2012), but not other media that involve different degrees of dynamism (as chats or forums).Thus, there are a lot of studies that show contradictory results regarding student satisfaction and performance in VLEs (Stöhr, Demazière, Adawi, 2016).This study may serve to provide empirical evidence in an elearning graduate program, which can be interpreted in the context of the effectiveness of the teaching process.The aim of our study was to study how students´ survey dimensions on lecturers compliance were related to communicative media in a VLE.Barrós-Loscertales, A.; Marquez, L.; Climent, O.; Bueichekú, E.; Bustamante, J.C.

Methodology
In this study we recruited the Lectures Evaluation system for students from an e-learning graduate program at the Universitat Jaume I (UJI).In total, 12 different subjects and 25 different teachers were evaluated for a mean sample of 10.41 (SD±1.84)evaluations per subject.Mean number of students per subject was 33.73 (SD±0.84).
Lectures Evaluation System involves four indexes used for the DOCENTIA-UJI model, which has been referred in the Introduction section.These indexes are called: D1, Educational Planning; D2, Teaching Qualifications; D3, Development of Learning; D4, General Satisfaction.Moreover, we extracted a fifth index called Degree of Compliance, D5.Each of these indexes are evaluated by asking from four to one different questions in a 5 point-Likert scale.Table 1 summarizes the questions for each index.

D1
Educational Planning Q1 At the beginning of the course or program I have a clear guide to the subject Q2 In this subject I always know what the assessment criteria are going to be used Q3 The course materials (books, notes, etc.) included in the program or the subject are adequate Q4 The teacher meets the subject's program

D2
Teaching Qualifications Q1 The teacher is clear in the explanations Q2 The teacher achieves students' participation Q3 The teacher's communication skills facilitates my learning process Q4 The teacher maintains an adequate learning climate

Q1
The amount of work required is proportional to the credits of the course Q2 The teacher applies appropriate methodologies and resources Q3 I find the aid received in tutorials (in-person and/or virtual) effective to learn Q4 Continuous assessment activities are adequate

Q1
In general, I am satisfied with the teacher of this subject

Q1
The teacher has facilitated my learning, broadening my knowledge and improving my skills Q2 I have improved my skills according to the expected competencies detailed in the program of the subject

Students´ surveys and involvement in e-learning graduate program relate to their satisfaction
The development of communicative media within the virtual classroom were measured as different variables related with the development of forums.Forums in e-learning allow students to connect and collaborate in virtual environments, making them an ideal addition to both asynchronous and synchronous instructional strategies.We measured four different variables related to lecturers' forum activity: number of open forums; number of lecturers' participations in forums; total number of characters entries in forums; and total number of forums entries including students and lecturers (see Table 2).Furthermore, we controlled for the number of times and total time lecturers connected to the virtual classroom (see Table 2).It should be noticed that the number of participants for each lecturers' evaluation ranged from 14 to 7 students within subjects, in which between 33 to 31 students were enrolled and, therefore, students' participation in survey ranged between 45 to 22 percent per lecturer.

Analyses and Results
As we can see in table 2, the means of the dimensions score-related are above 4.47 points.
In the analysis of the association between the dimensions of the educational model and the participation in forums we excluded three lecturers that were not involved in forums development at all.Then, a bivariate correlation analysis showed a positive correlation between the total number of forums' entries including lecturers and students, and the D4 General Satisfaction (r(26)=0.46,two-sided p=0.02).However, number of open forums for each lecturer or their individual participation in each forum in terms of entries or characters were not significantly correlated with any educational model dimension.Likewise, lecturers' connections to the virtual classroom in terms of entries or time in seconds did not correlate significantly to any model dimension, or any other variable.
Furthermore, the association between the total number of forum participations and D4 General Satisfaction remained significant (r(22)=0.42;two-sided p=0.04) after partial correlation with the number of open forums by each lecturer.In our second analysis, we focused on what other dimension included in students surveys related to the general satisfaction of the students including all lecturers.For the analysis of these associations, we performed a regression analysis and looked into the determinants of the indicator D4 (in logs), i.e., the indicator that proxies for general satisfaction.On the other hand, we considered as explanatory variables the perceived values and experience of lecturers, which we proxy alternatively by using the previously introduced indicators D1, D2 and D3.In this regression analysis, we included D1, D2 and D3 (in logs) separately in every regression to avoid multicollinearity problems.Secondly, we included as explanatory variables the controls related to the communicative media in the virtual environment and academic program´s variables such as number of ECTS.Specifically, these controls were 1) the total number of forums per subject, 2) the number of forums per lecturer, 3) the number of times that the lecturer participates in forums, 4) the number of characters in lecturers' participations, 5) the total number of entries in the forums by lecturers and students, 6) the number of connections by the lecturers, 7) the lecturer's connected time, 8) the number of students, and 9) the number of credits (ECTS; European Credit Transfer and Accumulation System) that the lecturer teaches in the virtual environment.
The goodness of fit of the three estimated models was good, as it varied between 66 and 71.2 per cent.The obtained results1 showed that the estimated coefficients for D1, D2 and D3 were positive signed and statistically significant and then, when these indicators increase, also do so the general satisfaction of the students enrolled in the academic program.
This result is complemented by looking at beta coefficients, which are used to compare the magnitude of the estimated coefficients obtained in the regression analysis of those variables measured using different scales.The highest beta coefficients obtained are for Teaching Qualifications, D2, followed by Development of Learning, D3.These results show that all dimensions included in educational models are related to general satisfaction to a different degree.Particularly, teacher skills and learning environments strongly matter for students' general satisfaction in a virtual learning environment.

Discussion
In this study we analyzed the association between educational model dimensions and lecturers' development of communicative media of dynamic forums in a virtual classroom for an e-learning graduate program.Interestingly, model dimension D4 General Satisfaction was directly related to a measure that sums up lecturers and students participation in virtual forums.Moreover, Teaching Qualifications and Development of Learning dimensions are related to general satisfaction of students.Therefore, we can conclude that general satisfaction with e-learning graduate programs are related to lecturers and students involvement in their dynamic educational activities, as well as surveyed teacher skills, and learning environments are directly related to general satisfaction.
It is important to create virtual environments where students can construct by themselves their own knowledge and lecturers provide guidance.It is fundamental to think about how to provide students the tools to develop the scaffold that supports learning (Stöhr et al., 2016).Nevertheless, as previous studies report (Maarop and Embi, 2016), despite lecturers could be able to develop good designs, they need institutional support to find the right balance between student autonomy and lecturer-student interaction (considering the ratio presentiality/virtuality: classroom, online and blended teaching).
Interestingly, our results may be considered under the recent meta-analysis published by Uttl et al. (2016), in which authors show that faculty teaching effectiveness is not related to ratings from student survey of teaching.Thus, we showed that general satisfaction in Barrós-Loscertales, A.; Marquez, L.; Climent, O.; Bueichekú, E.; Bustamante, J.C. student surveys were related to other evaluated dimensions linked to teachers´ skills and learning environment, as well as to student and teachers involvement in communicative media within the virtual environment.However, all these measures have nothing to say related to lecturers' effectiveness on students learning or career success.Our results restrict to students' learning experience and involvement on virtual environments.Therefore, the involvement of students in dynamic activities, such as forums within virtual environments, favor a higher general satisfaction with lecturers.Furthermore, we observed that the general satisfaction of students were related to their subjective perceptions of lecturers' skills, and the learning activities developed by lecturers.In this sense, we may question whether any of these dimensions are independent or they are reflecting the same effect on students' satisfaction.
Finally, our research scope is bounded by study limitations.First, the sample size that completed the survey was small and, therefore, results should be cautiously considered given that they are underpowered.Second, the questions that evaluate each dimension and the dimensions themselves may be adjusted to the DOCENTIA-UJI model.However, this model is addressed to the evaluation of both virtual and present educational environments, and this model needs to be validated for a virtual environment.It is worth mentioning that, although the research has been carried out on a small sample and the results should be drawn carefully, this study provides a solid basis for further research.

Conclusion
The aim of our study was to test the association of different variables and students' survey dimensions related to lecturers' compliance in a virtual environment during the development of an e-learning graduate program.We were interested in showing whether dynamic activities, such as lecturers' forums activity, were related to the dimensions evaluated within the educational model of the university (students' satisfaction principally).Moreover, we wanted to see the association between the dimensions included in the educational model, independently of teachers and students involvement in the virtual environment.For this purpose we performed multivariate analyses.We found that both students and lecturers involvement in educational activities, as well as students subjective perception of teacher skills and educational environment, relate to the dimension of students' general satisfaction.We suggest the development of specific students' surveys on lecturers' educational abilities in order to validate the evaluation of e-learning activities in virtual classrooms.Objective measures of students and teachers' involvement in dynamic activities may serve as indicators of general satisfaction too.

TABLE 2 : Mean (SD) and range interval for dimension and lecturers' forum activity-related variables
*European Credit Transfer and Accumulation System