The Digital Learning Laboratory Model to Catalyze Change in University Teaching and Learning

This paper outlines a unique model catalyzing change in teaching and learning known as the Digital Learning Laboratory (DLL) model that a large research university in the northeastern United States currently employs. We focus here on the MOOC work that the individuals in the DLL lead that have spread to improvements in teaching practices and learning experiences across departments beyond MOOCs. We discuss the MOOC development process and the ways in which this process can differ greatly from the development of an in-person course creating the initial and continued need for the DLL. Then, we describe the Digital Learning Laboratory, a community of practice of academics with advanced degrees in their field of specialization and housed in the relevant departments across our university. Finally, we discuss potential advantages of this model, including having a person with subject-matter expertise leading MOOC and hybrid projects and not requiring a different tenure-track faculty member to learn MOOC development skills for each new course.


Introduction
It has been over seven years since the New York Times declared 2012 the "Year of the MOOC" (Pappano, 2012). In this time, Massive Open Online Courses have matured. Though the initial hype associated with MOOCs has died down, these online courses have started to take a central role in a variety of higher educational opportunities. Through the MicroMasters program on edX, learners can complete graduate-level coursework, add to the credentials of their admission application portfolios, and earn credit from a partner university accepting that student (Barbosa De Almeida Cabral, Ribeiro Jorge & van Valkenburg, 2017). Fully online Master's degree programs at scale are increasing educational access while decreasing degree cost (Goodman, Melkers & Pallais, 2019), and MOOCs are expanding how we think about hybrid education (Pérez-Sangustín et al., 2017) and continuing education (Zipper, 2016). This paper outlines a unique model to transform teaching and learning in higher education known as the Digital Learning Laboratory (DLL) model that emerged as a result of the initial excitement over MOOCs and sustained due to the success and broader impact in departments and across a large research university in the northeastern United States. We begin with a general overview of the MOOC development process, and then describe the DLL approach and the reasons that this approach is particularly well-suited for facilitating change in teaching and learning in the university.

Why MOOCs Initiated a Need for Embedded Cross-Disciplinary Academics in Departments
MOOC development frequently remains a time-and resource-intensive process. Though exceptions certainly exist (Nissenson & Shih, 2015), MOOC development budgets are generally high and may not include costs from the time spent by faculty or other existing university staff on the project. The following section outlines a number of considerations that one must address when developing a MOOC, identifying areas where MOOC development differs substantially from in-person course development.

An Overview of MOOC Development
There are a variety of different steps and considerations to address when developing and running a MOOC. The following list outlines many of the steps in course development, though not exhaustive:  Determine the goals and learning objectives of your course and develop an overarching course plan aligned with those goals and objectives  Make design choices for your course that incorporate evidence-based best practices.  Develop video resources from storyboarding to recording, editing, and animating.  Develop text-based resources.  Create original figures/diagrams/artwork.  Secure intellectual property rights to use third-party charts/images/readings.  Incorporate universal design for accessibility.  Plan how to assess student learning.  Write assessments that align to the objectives.  Create interactive demonstrations, simulations, and visualization tools for the learners.  Beta-test all course materials before making them available to the general public.  Monitor discussion forums and provide learner support while the course is running.  Analyze course data after the course run is complete.  Revise the course during and after the course run to incorporate learner feedback and data for future runs of the course.  Manage a team and complex project of many parts on time and within budget.

How does MOOC development differ from the development of an in-person course?
A number of the steps of MOOC development are not wholly or partially necessary when developing an in-person course, but many steps should also happen for an in-person course. However, there are a number of ways that MOOC development differs significantly from the development of an in-person course due to the public nature of the final products, scale of access, and self-imposed standards for publication-level quality that are not often to the same level for in-person teaching.
Many of these differences relate simply to the scale of MOOCs. The challenge of developing a learning experience where learners have their needs met and feel connected to the wider course community is a larger challenge at scale. MOOC instructors and institutes must determine how they will provide support for a large number of learners, ensuring that these learners have resources available when they have questions about or struggles with the course material.
Assessment is a particular challenge when considering education at scale. The easiest approach to online assessment-multiple choice questions-is also the most challenging at developing and assessing higher-level cognitive skills. Looked at through the lens of Bloom's taxonomy, multiple choice questions do a good job of assessing skills relating to knowledge and comprehension, but alternative assessment techniques need to be considered 277 The Digital Learning Laboratory Model to Catalyze Change in University Teaching and Learning to develop and assess skills relating to application, analysis, synthesis, and evaluation. Instructors often turn to peer-evaluated grading to assess and assign grades to MOOC learners, and there is limited research that supports the ability of peers to relatively accurately assign grades to their classmates (Peich, 2013). Questions, however, of providing meaningful, expert feedback to MOOC learners remain.
In addition to preparing for a large number of learners, MOOCs also tend to consist of a particularly diverse set of learners. This diversity can prove to be one of the exciting benefits of MOOCs; learners from around the world can learn together and support one another on course forums and social networking sites related to the MOOC. However, to meet the needs of our learners, when developing and running a MOOC, it is important to consider that learners 1) are of a wide variety of ages, 2) have varying educational backgrounds, 3) have a variety of levels of comfort with the language of instruction, 4) may have physical or learning disabilities, and 5) come from a variety of different cultural backgrounds. This is a much different situation than a typical university classroom, where students have a smaller range for demographics like age or backgrounds and represent diversity from one or two of these categories at a time. MOOC design has to include the diverse needs of the learners from all five of these categories that fall within a wider range in the same course at the same time.
Finally, managing the use and reuse of third-party materials in MOOCs matters much more than in-person classrooms. Instructors preparing for an in-person course show video and images in class or assign readings from copyrighted sources under fair use guidelines, benefiting from university library resources, or aligned with the historical practice of the burden of purchasing a textbook being on the student. Requiring learners to purchase course materials like textbooks or access to a research paper behind a paywall diminishes the no-and low-cost benefit of MOOCs and adds a barrier to access. This means that instructors need to either 1) secure the rights to third-party resources, 2) search for open source alternatives to these resources or 3) take the time to develop a course that is selfcontained and can stand alone without the incorporation of any third-party materials. All of these alternatives tend to consume significantly more time and/or resources than assigning a chapter in a textbook or assigning students to read a journal article available through the university.
The simple educational potential of this online environment can also complicate the process of course development. An online learning environment can provide students with a rich variety of resources, such as virtual field trips (Quintanilla-Terminel, Pec, and Jagoutz, 2019), three-dimensional video animations, and interactive online demonstrations. Furthermore, an online learning environment can provide an extensive amount of data to course instructors, allowing them to revise and update course materials to best serve the learners in the course. All of these examples represent exciting opportunities to provide a rich experience for online learners; however, they also represent a significant time investment and require particular expertise on the part of the course developers. Given the demands on tenure-track faculty time already, the ability to prioritize or dedicate time to learning new skills specific to MOOC or hybrid initiatives were unlikely. So instead, individuals rose to taking on these responsibilities in close collaboration with those faculty when MOOCs started.

An Overview of the Digital Learning Laboratory Model
The Digital Learning Laboratory is a community of practice consisting of academics who are housed in and reporting to different departments and take responsibility for MOOC development, hybrid learning initiatives, and research related to these projects. These individuals hold advanced degrees in their field of specialization; for example, a DLL member in the Department of Physics will hold an advanced degree in that discipline, typically a doctorate. The DLL members lead the production of MOOCs in their department, and they work closely with the faculty to develop these courses. The specific professional tasks of each DLL members vary by department and is deliberately kept flexible so that the position can be customized to their department's needs; however, all DLL members devote either all or a significant portion of their time to digital learning initiatives. The members of the Digital Learning Laboratory meet on a regular basis to discuss best practices in digital education and to support each other with course development. Members of the Digital Learning Laboratory are divided into two main categories: Digital Learning Scientists: The Digital Leaning Scientists are responsible for the MOOC development efforts of an entire academic department. They are employed only in departments that have a substantial MOOC development program, and typically hold an appointment as Lecturer within their departments. The DLL scientists are responsible for overseeing the execution and helping to develop the digital learning strategy of their academic department. They manage teams and projects to coordinate course development of a variety of different MOOCs and hybrid learning efforts of their departments on campus, as well. Digital Learning Scientists also conduct research to better understand engagement and learning design in MOOC development.
Digital Learning Fellows: Digital Learning Fellows are most often postdoctoral associates who also have a strong background and interest in the field of education. The role of the Fellows tends to be somewhat more focused than the role of the scientists, leading the development of a single course or course series, for example. Like the Scientists, the Fellows also support residential education at our university, and many conduct educational 279 The Digital Learning Laboratory Model to Catalyze Change in University Teaching and Learning research as well. In departments with established online learning programs, the Fellows tend to work under a Digital Learning Scientist, while in departments with smaller programs, they work independently.

Alternatives to the DLL model of course development
To our knowledge, the DLL approach to MOOC development is unique to our university. There are a number of alternative methods for approaching MOOC development, including 1) having faculty take the lead, reaching out to specialists as-needed in the course development process and 2) coordinating course development through a centralized MOOC/technology/teaching and learning center that includes instructional designers and technology experts, but no subject-matter experts.

Advantages of the Digital Learning Laboratory model
We believe that the Digital Learning Laboratory model has several important advantages to alternative models of MOOC development or efforts to improve teaching and learning.
Most importantly, the DLL academic who is responsible for overseeing course development has expertise in the field in which they are developing a course and a strong background and interest in teaching, which provides a number of advantages over a centralized system with no such experts:  Because faculty collaboration is essential in this process, it is very useful to have a colleague who "speaks the same language" as the faculty and who the faculty can trust to understand the critical learning objectives and teaching approach of the course.
 The DLL academic has a strong understanding of the material that they are charged with teaching, and they understand how that material fits more broadly into their field of specialization. They can make course design decisions that reflect the learning and ideas that are most important to their field of practice.
 The DLL academic can independently develop assessments, videos, and other course materials with only minimal input from the faculty. This lessens the obligation and time of the faculty to develop all of the course materials on their own.
 The DLL academic can directly support the learning objectives of the course and serve as the instructor running the MOOC when live.
 The DLL academic serves as the resident expert on topics outside of the discipline that faculty feel receptive to getting advice on for their teaching. These topics include educational technology, best practices for pedagogy, and the research behind the learning sciences to implement in digital projects.
 The DLL academic prioritizes teaching and learning without the pressures of writing for research funding or publications compared to their tenure-track colleagues.
 The DLL model organically developed out of solving needs and problems within departments rather than setting up a structure first for others to fit into.
If instead, we compare the Digital Learning Laboratory model to an individual faculty member directing MOOC development, we see a different set of advantages emerge:  DLL academics possess or quickly develop a diverse set of knowledge and skills necessary for MOOC development. DLL members are not degree-holding experts in the following fields, but they have a strong working knowledge of the following areas is necessary for successful MOOC and hybrid learning projects: 1) best practices in learning experience design, 2) learning management system platform knowledge, 3) video production and editing, 4) intellectual property regulations, 5) best practices to create accessible course materials, 6) emerging trends in ed tech for higher ed, and 7) data analytics skills to study and evaluate courses once complete.
 This specialized MOOC or hybrid project development knowledge becomes integrated within departments, and individual faculty no longer need to learn these skills to develop MOOCs.

Conclusions
We have presented the Digital Learning Laboratory model as a catalyst for transforming teaching and learning via MOOC and hybrid learning initiatives. In this model, academic departments employ advanced degree holders in their fields of study to oversee MOOC and hybrid initiatives within their departments. These academics are in turn part of a community of practice known as the Digital Learning Laboratory, where they can work with and learn from other digital learning practitioners throughout the university. This model for improving teaching practices and student learning experiences has several advantages, including the benefit of a person with subject-matter expertise leading course development and the benefit of not requiring a faculty learn skills outside of their area of research expertise every time a department wants to create a new MOOC or hybrid learning experience.

281
The Digital Learning Laboratory Model to Catalyze Change in University Teaching and Learning