Learning analytics dashboard to support instructors: a literature review

The purpose of this study is to conduct an evaluation of the literature on learning analytics dashboards in order to address the following research questions: "How do instructors use learning analytics dashboards to send notifications?" and "How can a learning analytics dashboard help students get reminded of the due dates for assignments, quizzes, and exams?". A total of 20 papers were analyzed. Although the majority of them discussed how students utilize dashboards to track their progress or compare their progress with their peers, none of them mentioned how dashboards may be utilized to automatically notify students or send reminders, reducing the amount of work instructors have to do. This can be taken into account when developing a dashboard for learning analytics in the future.


Introduction
Learning Analytics (LA) is a growing field that draws on many different fields of study, such as Data Analytics (DA) and Artificial Intelligence (AI), to support and enhance education (Nouri, 2019).Among the several uses of LA, including supporting learners self-regulated process (Winne et al., 2019), prediction of students' success (Chen et al., 2020), and analysis of students study behaviors (Sher et al., 2022), there are works on Learning Analytics Dashboards (LADs).
LADs can automatically analyze large amounts of data and deliver the results to relevant stakeholders.It consists of "a single display that aggregates several indicators regarding the learner(s), learning process(es), and/or learning context(s) into one or multiple visualizations" (Schwendimann et al., 2016).The stakeholders in this context could be instructors, students, and/or academic advisors.We are particularly interested in understanding how instructors could use LADs to improve feedback and communication with their students.Previous research has demonstrated that instructor feedback positively impacts students' performance both in-person (Baranczyk & Best, 2020) as well as in online classes (Lee, 2020).Considering that, this study aims to answer the following research questions: RQ1: "How do instructors use learning analytics dashboards to send notifications?" RQ2: "How can a learning analytics dashboard help students get reminded of the due dates for assignments, quizzes, and exams?" To answer those research questions, we conducted a literature review of the last five years on learning analytics and learning analytics dashboards.

Methodology
We conducted a literature study by searching for and reviewing recent five-year (2018 -2022) published articles written in English in order to gather the most recent work and identify any limits or prospective future research from those articles.The databases that we used were IEEE Xplore, ACM Digital Library, Wiley Online Library and Google Scholar.These sources were selected because they are known for publishing works related to the learning analytics area.In particular, the ACM Digital Library is the current publisher of the Learning Analytics and Knowledge Conference proceedings.
The keywords used for the search were: "learning analytics" and "learning analytics dashboards".In some cases, the search resulted in a large number of results; the number of total distinct results is not reported because searching different sources returned many repeated results across all searches.
For the search carried on IEEE Xplore, there are 8,073 search results for "learning analytics" and 106 for "learning analytics dashboard."We skimmed the abstract of those 106 papers and selected five papers which were related to LAD for teachers or students.Most of the other papers were related to other dashboards.
There are over 150,000 results for "learning analytics dashboard" in the ACM digital library.However, among those there were non-learning analytics-related papers because the search also included "learning," "analytics," and "dashboard".We manually screened papers which were relavant to LAD.We went through titles of first 1000 latest results and found that none of them were related to LAD.We have gone through 100 relavant papers because after 100 results there were no papers for LAD.Among the relavant papers, we read 5 papers and skimmed through 10 papers based on the titles.The 10 papers were not included in the total papers we have analyzed because they were not related with LAD for teachers and/or students.
We looked at the first 50 results for the term "learning analytics" in the Wiley Online Library and read the papers and literature reviews among them that were published after 2018 (the previous five years).After that, we read the first 20 results in the Wiley online library for the keyword "learning analytics dashboard".The majority of the results were related to general dashboards and not specifically dashboards related to learning aspects.
When we searched in Google Scholar using keywords "learning analytics" and "learning analytics dashboard" most of the results were the same as which we have already read from IEEE Xplore, ACM Digital Library, Wiley Online Library.So we searched for journals using the phrase "International Journals of Learning Analytics dashboards" over the range 2018 -2023.We got 16,700 results.From those, we read the first three journal papers presented in the first page results, since we skimmed through the remainder papers in the first page and they weren't related to learning dashboards.
Besides the database searches, we used the snowball technique and read four papers from the previous papers found.We read all papers and summarized their purposes in order to look for information that would respond our research questions.

Results
A total of 20 papers were analyzed.From those, three are literature review papers and 17 were either conferences or journal papers.One literature review paper is in the area of LA and the other two are related to LAD.Ahmad et al. (2022) looked at 161 learning publications to investigate the function of learning design in learning analytics.By offering a reference framework where they suggest potential linkages between learning analytics and learning design, they sought to better align learning design with learning analytics and to demonstrate the prior study and applications of LA indicators and metrics.They discovered that a few learning analytics articles did, in fact, take into account learning design activities for gathering user data.They also identified an ongoing rise in the quantity and quality of indicators, as well as an evolution of these indicators through time.

Previous Literature Review on LA and LAD
By leveraging theoretical foundations including human cognition and perception, context awareness, and technology, designers need to have a thorough understanding of how learners see and think in order for LA to be effective (Few, 2009).The degree to which the target objectives of LAD are consistent with the domain metrics used to assess their effectiveness is investigated in a Valle et al. (2021) literature study.Additionally, it states that regardless of the intended target audience, the dashboard's primary purpose is to support learners as end users.
In order to evaluate the influence on teaching and learning, a thorough evaluation of the literature was done on the LADs research that presents empirical evidence.Self-regulated learning has been cited in a number of earlier research reviews as the main area of interest for LADs.The literature on self-regulated learning and how self-regulated learning is supported has, however, received considerably less attention than it deserves.They reviewed empirical data on LADs using Winne and Hadwin's well-known model of self-regulated learning (Winne & Hadwin, 1998) in order to solve this limitation.The findings demonstrated that existing LADs have significant shortcomings in how their evaluation is carried out and reported, are rarely based in learning theory, cannot be proposed to support metacognition, do not provide any information about efficient learning tactics and strategies, and cannot be suggested to support metacognition (Matcha et al., 2020).

Previous Papers on LA and LAD
From the 17 papers, four were related exclusively to learning analytics.Those will be described first in this section, followed by the description of the 13 papers related to LAD.
In order to comprehend and improve learning, learning analytics makes extensive use of data on learner interactions in digital learning environments.Although measurement is a key component of learning analytics, there hasn't been much study looking at how learning analytics and assessment are related.In order to fully utilize the potential of the connections between learning analytics and assessment, interdisciplinary teams will be required to address task design, analysis of learning progressions, trustworthiness, and fairness, as well as other pressing scientific and practical challenges and opportunities (Gašević et al., 2022).Concerns about fairness and bias may be advanced by integrating learning analytics and assessment.The amount of study on fairness and bias in learning analytics, much alone in assessment using analytics, is, nevertheless, lacking (Gašević et al., 2022).
The application of learning analytics could help curriculum committees reflect on their curricula.According to Chou et al. (2018), competency-based learning analytics are suggested as a way to analyze competency-based curricula and graduates' academic records of coursework in order to provide systematic evaluation information that could help curriculum committees reflect on curricula, faculty teaching, and student learning.According to Whitelock et al. (2019), in order for future implementations of LA to be successful, it is essential to gauge student expectations of LA services.The authors presented a descriptive instrument to assess students' ideal and anticipated expectations of LA services.They created and validated a descriptive questionnaire that offers a robust and methodologically sound solution to measuring student expectations of LA services using the identified expectation themes (ethics and privacy, agency, intervention, and meaningfulness) and expectation types (ideal and predicted).They found that while some characteristics of LA services may be useful (such as the implementation of early alert systems), they may not be the features that students anticipate (such as LA services created to enhance academic abilities like selfregulated learning).
Ten of 13 LAD papers mentioned giving students feedback to track their development or compare it to peers.Those LADs display data that students need to track their progress, such as grades and how they compare to others, number of times they've read the course materials, and amount of hours spent watching lecture videos.
A paper discussed about the student's ability to get acquainted with the LAD's and focused on helping students practice self-regulated learning (SRL) and to keep track of the learning progress with the help of technologies (Park et al., 2022).Another paper that addressed queries raised by students in virtual breakout rooms did so by creating a dashboard (Stanislav et al., 2021) .The design approach consists of: interviews with teachers in which questions were asked to teachers about their current practices while monitoring groups of students.Based on open answers from teachers, an inductive analysis is performed to map the critical challenges they face with the evidence that can be used to address them.Codes emerged through the analysis are grouped thematically.Based on the teachers' questions identified in the previous steps, the most highly ranked questions (i.e., those that address the concerns of the majority of the participants) are explicitly added to the dashboard or LA user interface (Echeverria et al., 2018).
In a study by Wang and Han (2020), LADs are created using process-oriented feedback, providing both visual and textual feedback for various learning processes.A virtual learning system called iTutor was used for the study.They upgraded iTutor by including LAD to give students process-oriented feedback based on the performance and behavioral data that are automatically recorded during students' learning processes.Students used those visual feedbacks to compare their performance to that of other students.Rienties et al. (2018) used an Analytics4Action (A4AW) workshop to examine how instructors in a two-hour training tried to make sense of LAD and whether their technology use affected their satisfaction with A4AW.The study found that 89% of participants were pleased with the A4AW program.68% of participants thought open university LA tools were useful.(i.e., the degree to which teachers believe the use of LAD and visualizations will, for example, improve the quality of their teaching or increase academic retention).Jaramillo et al. (2022), proposed a dashboard to gather information about students who are suspected of academic dishonesty.The dashboard is composed of information about how students have interacted with the course materials and videos, how many forum posts have been made, how many tests have been turned in, and what was the learner's behavior on a specific day or during a specific lesson.

Discussion and Conclusion
We examined learning analytics and learning analytics dashboard papers from the past five years to see whether any answered this paper's research questions.We found out that no papers addressed the research questions.We want to solve this literature gap by designing and implementing a LAD to help instructors improve feedback and communication with students.
According to Bouchrika (2022), learning management systems (LMS) are considered mandatory in higher education and institutions are using them to display course material, assignments, and quizzes in their courses.Professors generally create an announcements section, which will be used by them from time to time to send out announcements.Announcements can be used to convey a wide range of information, including reminders for impending tests, projects, exams, and assignments, as well as information on altered class times.The time spent by professors in keeping track of their students is huge, especially if we consider courses with a large number of students (Hernández-García et al. 2015).Even if the professors use the learning analytics already provided by the LMS there is a considerable number of selections that needs to be done in order for the professor to know if the students did or not did the assignment and which messages to send in each case.By introducing a LAD, which can automatically gather information regarding due dates from assignments and display this information to professors along with a tool to personalize deliver alerts about such deadlines, we aim to reduce the amount of time professors spend on doing such feedback.Through this automation, professors will benefit from the time and effort savings.This literature review helped us to identify that there is no proposals for a LAD which can help teachers reduce their burden by automating and personalizing student's feedback.So we plan to design a LAD that can send notifications to students automatically by taking the data from the assignments, quizzes, or any other sections included in the course which would address our research questions.