A mixed reality laboratory for developing competencies in control engineering

This work presents a novel mixed reality laboratory for the development of competences in control engineering. This laboratory simulates various control systems that may interact with different control devices. The mixed reality lab has three components: (i) a virtual environment, (ii) a virtual-electronic interface, and (iii) a control unit. The virtual environment displays a virtual representation of the system under control. The electronic interface uses a microcontroller to solve the differential equations that models the simulated system. The values solved by this interface are sent to the virtual environment for driving its animation. Moreover, input and output signals are connected to the control unit for the implementation of a control law. The functionally of the proposed mixed reality laboratory is showcased by modeling and controlling a Planar Vertical Take-off and Landing (PVTOL) system. Furthermore, the motivation of a group of students, after experimenting with the laboratory, is reported.


Introduction
In the engineering education community, there is a consensus that laboratory experiences are fundamental to learning processes. Through these experiences, students develop relevant procedural and disciplinary competences since they offer an interactive learning strategy. In recent years, universities worldwide seek innovative solutions to reconfigure or succeed traditional laboratories. The area of control and automation engineering has been an essential field for the implementation of innovative solutions to overcome the limitations of conventional laboratories. In general, control and automation theory is abstract and difficult to understand, since it consists mainly of theoretical models. Although traditional hands-on laboratories allow students to associate theoretical concepts with physical phenomena, they have some limitations (Hernández-de-Menéndez et al., 2019). The most common limitations are: (1) equipment for control engineering experiments is often expensive; (2) equipment needs to be shared by students in groups; (3) both laboratory opening hours and space are limited; (4) experimental sessions are always detached from theoretical courses; and (5) experimentation with real systems could be risky.
The rise of virtual and remote laboratories is removing the limitations of traditional handson laboratories (Li et al., 2020;Macías García et al., 2020;Morales-Menendez et al., 2019;Tejado et al., 2021). Among these laboratories, those that use mixed reality and augmented reality laboratories have demonstrated several advantages over others because they combine the benefits of physical and virtual components (Liang & Liu, 2018;Zata et al., 2017). In particular, mixed reality laboratories have been successfully applied to train technicians in different fields, such as medical procedures (Aebersold et al., 2018;Barsom et al., 2016), industrial maintenance and assembly (Gavish et al., 2015), among others. Regarding control engineering, mixed reality laboratories have shown significantly improves for learning specific concepts (Frank & Kapila, 2017).
This work presents a mixed reality laboratory that aims to develop the procedural and disciplinary competences of engineering students taking control engineering courses. Engineering competencies are important skills and abilities required to solve modern challenges (Guajardo-Cuéllar et al., 2020, 2022. Control engineering is not exempt from these modern challenges. The proposed laboratory has three elements: (i) a virtual environment that simulates the system to be controlled, (ii) a control unit that uses a physical electronic device to control the simulated system, and (iii) an electronic interface that communicates the virtual environment and the control unit. A Planar Vertical Take-Off and Landing (PVTOL) system is used to showcase the functionality of the proposed laboratory. Furthermore, seventeen students from the Tecnologico de Monterrey used the laboratory and reported their experience in a survey. The students found the mixed reality laboratory very useful, easy to use, and realistic.

Virtual Environment
This section describes the virtual environment. Electro-mechanical systems are designed using a CAD software, then they are imported to a game engine for creating the virtual environment. For the development, the systems must be firstly designed using a CAD software. Two examples are shown in Figure 1: (left) a Planar Vertical Take-Off and Landing (PVTOL) system and (right) an inverted pendulum. The PVTOL system consists of a beam that rotates about its central axis, whose movement is induced by the thrust generated by the rotors located at both ends of the beam. This system is a simplified version of a quad-rotor, in which engineering students can apply control concepts. The second system is an inverted pendulum on a cart driven by a direct current (DC) motor. The center of mass of the pendulum is above the pivot point. Therefore, the pendulum is unstable without a control algorithm. Through this system, engineering students can understand and apply stability concepts that arise in applications such as rocket guidance.
The CAD models of the two systems are then exported to Unity, a well-known game engine, where they are integrated with different assets in a virtual environment. Afterward, Unity generates an application containing the virtual environment, which can be executed in a personal computer, tablet, or smartphone (a link to a video is provided in the next section).

Electronic Interface
This section shows how the differential equations of a system can be implemented using a microcontroller. A numerical method for solving the equations of the system is codified on an Arduino board. It is enabled for sending the data to the application via serial communication, it also writes these values on the electrical outputs of the microcontroller.

Mathematical model implementation
The system's dynamics is solved in the electronic interface. In detail, the model of the PVTOL system is: where is the angle of the beam, is the viscous friction coefficient, is the distance from the center of the beam to the rotors, is moment of inertia of the system, and " and ! are forces on both ends of the beam. " and ! depend on the rotor speeds as follows: where # is the speed of the -th rotor, and is a proportional constant that relates the force and speed of the motor. Equation (1) is solved and implemented in an Arduino Mega board that is included in the electronic interface.

Connection with the virtual environment
In order to animate the simulated system, it is necessary to send the system variables computed by the microcontroller to the virtual environment.
The virtual environment was programmed for receiving the data via serial communication and display it. Moreover, the values are used to animate the simulated system. In the case of the PVTOL, the application obtains the values of aforementioned variables: θ, ω, u " , and u ! . Moreover, the virtual beam is rotated according to the angle θ, received by the application.

Connection with the control unit
In order to interact with electronic controllers, the interface microcontroller must read input signals from certain analog inputs, and it must provide output signals through analog outputs.
For the inputs, the microcontroller reads ports attached to Analog to Digital Converters (ADCs), obtaining signals in the range from 0 to 5 Volts. The input signals can be provided by a control unit (microcontrollers, PLCs, DSPs, FPGAs, etc.) or other electronic devices, such as potentiometers and operational amplifiers.
On the other hand, the interface provides analog signals representing values of state variables (e.g., angles, velocities and forces). These values are provided by the microcontroller as pulse width modulation (PWM) signals. Then, an electronic circuit transforms the PWM signals into analog ones, by using Resistor-Capacitor (RC) filters. Thus, these electrical signals can be read by the control unit, emulating electronic sensors.

Mixed reality lab demonstration
The following link opens a video about a simple simulation of the PVTOL system using the virtual environment and potentiometers as external signals: https://youtu.be/RNu05flYITs The video shows the application running in a personal computer. First, the application is initialized and the "Balancín" option is selected to open the virtual classroom containing the PVTOL system. Then, the user walks around the classroom using the keys 'a', 's', 'd', 'w', and the mouse. Later, the serial port "COM14" of the computer is connected with the application, this is the port where the Arduino board is also connected. At this moment, the sound of two propellers starts because the application is now receiving the data corresponding to the rotor speeds from the microcontroller. These rotor speeds are defined by the value of two potentiometers that can be observed in the second screen (displayed at the bottom right of the video). Then the value of left potentiometer (left rotor) is increased, this produces a negative torque on the beam, and consequently it rotates until it reaches -45° (this is a limit value imposed in the code). After that, the value of the right potentiometer (right rotor) is increased. When it overpasses the left potentiometer value, it produces a positive torque on the beam, and consequently the beam rotates until it reaches 45°. Finally, the user tries to balance the beam by regulating the speed of the right rotor; however, this is a very difficult task for a human being because the system is unstable and the user fails to balance the beam.
The virtual environment and the electronic interface comprise the Mixed Reality Labs proposed in this work. The former provides two significant advantages: it is possible to design and simulate several engineering systems, and it provides a visualization of the system behavior. The latter allows to interact physically with the simulated system, which can be controlled using a wide variety of control devices.  Figure 2 shows a scheme with the connections between the mixed reality lab and an external microcontroller, which in this particular case is an Arduino Uno board. For the PVTOL system, the outputs are the beam angle θ and its velocity , their electrical signals are sent from the mixed reality lab to the external controller. In the same way, the inputs of the PVTOL system are the rotor speeds " and ! , which are driven by the electrical signals received from the external microcontroller.

Control Unit
The output variables of the system are used by a Proportional-Integral-Derivative (PID) control algorithm for computing an actuator signal effort, necessary for maintaining a outputs inputs

Mixed Reality Lab
External Controller 207 required reference value. This signal is provided as an input to the system. The PID algorithm is running in an Arduino Uno board, for maintaining the angle of the beam at a required value. The following link is a video showing an implementation of this controller, it maintains the beam angle in different required positions: https://youtu.be/WDKJmbDNWWM First, the user opens the application and connects the Arduino Mega that simulates the PVTOL. Then, the user uploads the PID controller code in an Arduino Uno board and launches a plotter for visualizing the required angle and the current angle of the PVTOL system. During the experiment, the video shows the virtual environment and the plotter at the same time. In the plotter, the red line represents the required angle for the beam, the values are -20°, -10°, 0°, and 10°. The blue line represents the current angle of the PVTOL system. This behavior can be appreciated in the virtual environment, where the angle of the PVTOL system is maintained. The main characteristic of the control unit that commands the behavior of the virtual system is its versatility.

Implementation with students
The mixed reality laboratory was used in a group of undergraduate engineering students from the Tecnologico de Monterrey, in order to asses their motivation when working with this platform. The group included seventeen students of the Mechatronics Engineering program. This project has the approval of the experimental protocol submitted in December 2022 and approved in January 2023. It is important to note that the sample used for this study previously signed an informed consent. After the experimentation with the mixed reality laboratory, a survey was applied to the students to measure their motivation regarding this laboratory. The formulation of the survey questionnaire was based on similar work on remote control laboratories (Achuthan et al., 2021;Liang & Liu, 2018;Tejado et al., 2021). The reliability of the questionnaire used was validated in previous studies (Achuthan et al., 2021), using the Cronbach's Alpha method and the instrument was found to be reliable ( > 0.79).
The survey questionnaire consisted of eleven positive claims, regarding the impact of the laboratory in three aspects: (1) learning motivation; (2) comprehension without supervision; and (3) realistic experimentation. Seventeen students answered the survey, Figure 3 shows the scores averages grouped by aspects.

208
Manuel Navarro-Gutiérrez et al. The results of the survey demonstrated that, through the use of the mixed reality laboratory, the students were motivated to learn about the implementation of control strategies on hardware.

Conclusions
A mixed reality laboratory for control engineering courses was described in this work, which combines simulations of control systems in a virtual environment and an electronic device that functions as interface with external control units. This scheme allows students to practice the implementation of any kind of controller, in any kind of electronic device. Moreover, the motivation of students working with the laboratory was assessed. The measurement of competencies development, regarding control engineering, is left as future work.