Symbiosis between Learning Analytics and digital transformation

. Abstract Higher Education Institutes (HEIs) are undergoing a profound and widespread digital transformation with the introduction of digital technologies in education. The introduction of digitised educational environments produced huge data repositories that could serve learning analytics, for the purposes of understanding and optimising learning and the environments in which it occurs. Where the digital transformation of education made learning analytics better and more qualitative, learning analytics leverages the digital transformation of HEIs by taking better informed actions. This paper elaborates on the process of developing a conceptual learning analytics platform at KU Leuven (Belgium). Where the project started off as a pure data project, gradually several preconditions emerged, which caused a lot of side-tracking. The introduction of learning analytics needs a digital transformation of education. At the same time the digital transformation of education needs qualitative and quantitative learning analytics to take it to the next level. As if Learning Analytics and digital transformation live in symbiosis with each other.


Introduction
Due to the impact of digital technologies, society is undergoing a profound and widespread digital transformation. This causes disruptive innovation in many different sectors, including higher education. No Higher Education Institute (HEI) will be able to afford ignoring the evidence from the data, without losing competitiveness in the future (Hidalgo, 2018). Hereby, the digital transformation of HEIs is not a goal as such but must serve a purpose. However, most HEIs are focused more on digitisation and digitalisation than actual digital transformation, triggered by the introduction of Learning Management Systems (LMS). To mature towards a complete digital transformation a coordinated pedagogical and technological shift is needed (Brooks & McCormack, 2020). Yet, the biggest challenge in this change process is not the technology as this is merely related to budgets and resources, allocating these in the most optimal way. The pedagogical shift is much harder since it involves people and the culture of the HEI. Even the Covid-19 pandemic, regularly referred to as the so-called burning platform that compelled HEIs to use technology and quickly ramp up their capacity for online teaching and learning, was not enough to induce a full digital transformation (Pelletier, 2021). Many lecturers that had a bad experience with the forced crisis-change, revert to the known recipe of traditional classroom education and assessment, complicating digital transformation. Nevertheless, the changing role of universities stimulates the urge to change anyway (Barnett, 2010).
Learning analytics plays an important role in this necessary transformation of HEIs and can be defined as "the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs" (SoLAR, 2022). The creation of huge data repositories by the introduction of digitised educational environments such as LMS, MOOCs or other virtual environments indeed leads to new areas of research and techniques.
The goal is to optimise the learning process (Khalil & Martin, 2016) but at the same time these new kinds of analytics ask us to reflect deeply on what kinds of learning we value. Learning analytics do not passively describe sociotechnical reality, they begin to shape it. (Buckingham Shum, 2014). Using 'The learning analytics cycle' (Clow, 2012) Buckingham Shum (2014) describes how the implementation of Learning Analytics triggers very fundamental questions. By answering these questions, the university takes decisions on what kind of education they want to offer, how they want to measure this and what actions they want to take to improve learning.  (Buckingham Shum, 2014) based on 'The learning analytics cycle' (Clow, 2012).
Already in 2012, Greller & Drachsler proposed a generic design framework that can act as a useful guide for setting up Learning Analytics services in support of educational practice and learner guidance, in quality assurance, curriculum development, and in improving teacher effectiveness and efficiency (Greller & Drachsler, 2012).
While interest in learning analytics has grown rapidly among HEIs, the maturity levels of HEIs in terms of being 'student data informed' are only at early stages. The SHEILA (Supporting Higher Education to Integrate Learning Analytics) project aimed to build a policy development framework that supports systematic, sustainable and responsible adoption of LA at an institutional level (Tsai et al., 2018). Both frameworks underline the importance of the involvement of all stakeholders in a LA project, with a special emphasis on their privacy.
This inspired the Science, Engineering and Technology Group (SET Group) of KU Leuven to initiate the project 'Learning Analytics & Dashboards: in function of the efficiency and effectiveness of blended learning' to investigate how existing data on students' digital study behaviour can be used to give motivating and action-oriented feedback to didactic teams. During this project it became clear that Learning Analytics was more than just unlocking some data and producing fancy dashboards. Although many specific issues that arose during the project were already described in detail in other documents, the project did not move forward by just pasting together the proposed partial solutions.
This paper elaborates on typical pitfalls and matching insights on the symbiosis between Learning Analytics and the Digital Transformation. The authors hope to inspire Higher Education Institutes (HEIs) to implement Learning Analytics and use it as a tool to leverage the digital transformation in their HEI.

Methodology
The project 'Learning Analytics & Dashboards' was set up in an agile way, with 'planning periods' of about 10 weeks and associated sub-goals. At the end of each planning period, a steering committee discusses the results and defines sub-goals for the next iteration. Based on the SHEILA framework we worked with colleagues collectively to determine the priorities for the project. With standard data modelling techniques, we've created a dimensional data model that would cover the needs for those learning processes needing learning analytics the most.

From conceptual learning analytics platform to Proof of Concept by involving stakeholders.
During the project specific feedback sessions were organised with stakeholders whose opinions were important for the further progress. Depending on the subgoal, participants were selected from a very broad range of roles and functions within the university. These sessions allowed qualitative analysis of problem statements, alignment with the needs of the stakeholders and thus their support of the proposed solutions. They also generated input from involved lecturers, so that several demo-dashboards could be built, and a conceptual learning analytics platform was developed.
By involving the different institutional stakeholders in the development, testing, deployment, and assessment phase of learning analytics tools, most of the resistance to change might already be mitigated in an early stage of the project. It will be of critical importance for its acceptance that the development of learning analytics takes a bottom-up approach focused on the interests of the learners as the main driving force. (Greller & Drachsler, 2012) However, to scale-up these solutions, and generate proof for the concept, a more structured and targeted approach is required to get a clear view of the present operational needs that exist in terms of study data and learning analytics. Through semi-structured 1-on-1 interviews with innovators and early adopters in Learning Analytics the conceptual learning analytics platform is surveyed.

Finding unity and scalability towards a Minimum Viable Product
Where a bottom-up approach helps to increase the support of the stakeholders, there is a risk that proposed solutions diverge from each other giving a scattered landscape of tools and dashboards for learning analytics. To create a uniform learning analytics platform, there is a need for organizational information governance encompassing the processes, standards, rules, and practices an organization follows. The findings of the project contributed considerably to the recently initiated KU Leuven working group 'learning data policy framework' to set out the boundaries and conditions for a centralised learning analytics data platform.
Simultaneously, the side-project E-COOL (Effective and Consistently Organized Online Learning environment) sought guidelines to bring systematics in terms of structure, layout, and communication on the LMS, considering the differences in didactic work forms and learning objectives. A systematisation in the structure of a course is crucial to create unambiguous and useful learning analytics. Interpreting digital traces requires knowledge of course structure and how learning materials are integrated into the learning approach. Even more importantly when dashboard present cross-curricular data where the information user can not have a detailed insight in how each course is structured.

Results and discussion
Where the learning analytics project started off as a pure data project, the present scope is much wider. Gradually, several preconditions emerged, which caused a lot of side-tracking: privacy, data quality, course & curriculum design, structuring the LMS, external tools & LTI couplings, ... In addition, the context in which the project was set up turned out to be very complex: the momentum of the pandemic, discrepancy between learning objectives and learning content assessment, the rollout of formative testing & blended learning, migration to a new LMS, ...
Nevertheless, a conceptual learning analytics platform and proof of concept could be developed based on the feedback sessions and interviews. This allowed the generation of concept dashboards, which fed the feedback sessions and interviews (Figure 2). At the same time, many didactic teams appeared to question the usefulness of Learning Analytics. A huge gap in engagement for learning analytics between the Innovators & Early Adopters on the one side and the Early Majority on the other side became very clear. To tackle this growing resistance to change the focus on the support of the stakeholders became even more important, together with scalability and sustainability of the proposed solutions. Where the Early Adopter can still live with teething problems, bugs and beta models, the Early majority are pragmatists who only use things that are 'Complete' and somehow solve their problems (Peeters, Grommen & Tubbax 2023). To gain wider support for learning analytics to justify the deployment of a full learning analytics platform at the university, the gap between the Early Adopters and Early Majority of learning analytics needs to be bridged.
Additionally, when inventorying other ongoing Learning Analytics investigations at KU Leuven, typically executed by innovators at a small scale, we noticed that most projects had a double complexity. Not only were they investigating new technologies or applications to capture, process and/or visualise study data. They were also looking at innovating the educational processes that were more suited for blended education. Because innovators do not always have scalability in mind, but rather seek an ad hoc solution to their own specific problem, most of these projects were not viable. Even more reason to find a solution and build a centralised learning analytics data platform that still satisfies them, but which is scalable.
Building a centralised learning analytics data platform is an essential step towards scalable analytical capabilities. This requires good information management capabilities to integrate, extract, transform, and access transactional data (Davenport & Harris, 2007). Whether we are speaking about the Knowledge Discovery in Databases (KDD) process in datamining (Fayyad et al., 1996) (Figure 3) or of the extract, transform and load (ETL) process in Business Intelligence & Analytics (Vassiliadis, 2009), the base of qualitative analytics is materialised by the analytical database on which visualisations, reports, Machine Learning Training, etc. for the data consumers can be built. Figure 3 Overview of the steps constituting the KDD process (Fayyad, Piatetsky-Shapiro et al. 1996).
The building of such an analytical database can only be achieved when the processes that would fill the source databases are digitalized sufficiently, so that the database is rich enough to draw well-founded conclusions and to take data-driven action. This is why the working group 'learning data policy framework' has put effort in defining a typology and inventory of which study data should be integrated in learning analytics. Furthermore, through the SHEILA framework priorities on learning analytics policy were agreed on.
The results of this seed project will be used as a blueprint for a scale-up project in which the learning analytics strategy and system are implemented and deployed. The partnership in the current project with people of the Educational Policy Units and ICTS is crucial for the continuation toward the scale-up project.

Conclusions
Building a learning analytics strategy for HEIs is so much more than merely aggregating some data and building generic dashboards. The maturing of the learning analytics and the evolution from digitisation towards digital transformation clearly go hand in hand. As HEIs are maturing in their digital transformation, more qualitative learning information is generated, and more accurate learning analytics become feasible. By providing more and better information about the study progress of a student, the HEI can improve their digital transformation in education, on every level.
This sounds like a positive spiral every HEI would want to enter. However, there are some conditions and pitfalls. For instance, the streamlining of these processes and standardising the information in a central learning data store, is essential to transform the institution's operations, strategic directions, and value proposition. To develop this centralised learning analytics platform, involvement of all stakeholders seemed necessary, in an iterative process.
Moreover, the implementation of learning analytics and the digital transformation of the HEI must be managed in sync with each other so that both exist in symbiosis. Otherwise, the positive spiral turns in to a negative vicious circle, with one change running ahead from the other, bringing unused solutions. A challenging mission in a challenging world that keeps HEIs competitive for the future.