Teaching and learning with AR/VR

Photo: None

In recent years, immense progress has been made in the field of virtual reality, not least through new types of head-mounted displays and input devices. At the chair, the potential of this new technology for education is the main focus. No other digital medium currently offers the possibility to deal with the learning subject as directly and intensively as is the case with VR. Our research focuses primarily on the possible forms of interaction and presence experience. Of particular interest for us are the effects on teaching and learning. A special form of VR is Augmented Reality, in which the focus of the user remains in the physical reality, but this reality is extended by virtual artifacts and information. Here, too, exciting projects have already been developed at the chair to explore the didactic possibilities of AR.

The chair members Raphael Zender and Matthias Weise are also initiators and spokespersons of the VR/AR Learning working group, which is assigned to the Bildungstechnologien and virtual reality and augmented reality departments of the Gesellschaft für Informatik. The working group deals with current developments, challenges and trends in teaching and learning scenarios with virtual and augmented reality technologies.

Photo: None

Projects and publications

Selection of interaction techniques for virtual reality applications

When developing virtual reality applications, the selection of the appropriate interaction techniques is a fundamental task. In VR, for example, one can maneuver through the world by flying, teleportation or physical locomotion, or select or manipulate objects by grasping them directly or using a light beam. The number of available techniques is large and depending on the application scenario, these have advantages and disadvantages for the users of the application. A doctoral project at the chair should support the decision making process. For this purpose, interaction techniques are examined and compared with regard to previously defined criteria. With the help of a tool it should then be possible to define requirements depending on the application scenario and thus select the right interaction techniques.

Publication:

Weise, M.; Zender, R.; Lucke, U.: A Comprehensive Classification of 3D Selection and Manipulation Techniques. In: Florian Alt, Andreas Bulling und Tanja Döring (Hg.): Proceedings of Mensch und Computer 2019 on - MuC'19. Mensch und Computer 2019. Hamburg, 2019. New York, New York, USA: ACM Press, S. 321–332.
Weise, M.; Zender, R.: A Systematic Literature Review on Independent Variables Used in User Studies on 3D Selection and Manipulation Techniques. In (Grimm, P.; Jung, Y.; Dörner R.; Geiger, C. Hrsg.): Virtuelle und Erweiterte Realität - 16. Workshop der GI-Fachgruppe VR/AR, Fulda, September, 2019.
Weise, M.: Auswahl von Interaktionstechniken für Virtual Reality-Anwendungen, in: Mensch und Computer 2018 - Tagungsband. Regensburg: Gesellschaft für Informatik e.V.

Editor: Matthias Weise

VR Negotiation Training

Successful negotiation is a key factor for corporate success. However, training it properly can be both time-consuming and costly, as it ideally requires repeated personal training with professional negotiators or agents. The Virtual Reality Negotiation Training creates a realistic simulation in VR within a spatially authentic training situation. Furthermore, a linguistically interacting dialogue system serves as an automated, virtual negotiation agent. This agent was trained with interaction data from a negotiation study and thus offers training persons an effective training partner for VR negotiation training.

Publication:

Klaue, K.; Oehlschläger, P.; Zender, R.; Siebert, E.; Lucke, U; Herbst, U.; Hefner, M.: Automatisierung im Verhandlungstraining durch den Einsatz intelligenter Dialogsysteme und Virtual Reality. In: Procededings der 15. Internationale Tagung Wirtschaftsinformatik (WI2020), GITO mbH Verlag, Potsdam, 2020. doi.org/10.30844/wi_2020_t3-klaue

Editor: Raphael Zender

Opportunities and challenges in learning and teaching with VR/AR technologies

Virtual and Augmented Reality are currently strongly discussed technologies. In particular, they are considered to have a high potential to improve teaching and learning and to realize novel teaching/learning scenarios. But what added value do these technologies actually offer for educational settings? An article for the e-Learning Fachtagung Informatik (DeLFI) compiles an excerpt from the current state of research as a position paper of the VR/AR-Learning working group of the Gesellschaft für Informatik (GI). Within the framework of a collaborative online brainstorming session, experts were asked to identify the various opportunities and challenges of VR/AR for teaching and learning. The result was substantiated and evaluated by means of a literature research.

Publication: Zender, R.; Weise, M.: von der Hyde, M.; Söbke, H.: "Lehren und Lernen mit VR und AR – Was wird erwartet? Was funktioniert?", in: Die 16. e-Learning Fachtagung Informatik (DeLFI), 2018.

Assessment of natural hazards with the help of Virtual Reality

The analysis and assessment of natural hazards requires skills that can usually only be acquired in the study of geosciences through practical learning units in the field. This poses a financial challenge for students and teachers and is associated with high costs. Virtual Reality has the potential to virtually recreate similar experiences through the possibility of immersive three-dimensional simulation and can thus overcome the obstacles in the teaching of natural hazards. In the context of a master thesis, potential tasks were considered that can be used in a learning scenario to teach basic skills for the analysis and evaluation of natural hazards. A selection of these tasks was implemented in a VR learning application based on three-dimensional models of the region near the volcano Chaitén. These were generated from images of a drone flight and freely accessible geodata.

Publication: Richter, A.; Weise, M.: "Bewertung von Naturgefahren mit Hilfe von Virtual Reality", in: Die 16. e-Learning Fachtagung Informatik (DeLFI), 2018.

Editor: Andreas Richter

Photo: own representation

Augmented reality learning scenario in the automotive sector

Training in the automotive industry is characterized by a close interlocking of theory and practice. Using the Microsoft Augmented Reality glasses Hololens, a learning scenario was designed that enables a context-based support of the learner during a real transmission assembly in the field of vision by holographic insertions. In addition, the implementation of a framework for the creation of learning scenarios paved the way for experts of the respective knowledge domain to access the learning scenario. In spite of the increased complexity of the technologies used in this medium, the effort for the realization of new learning scenarios should increase only slightly compared to conventional teaching methods. The master's thesis was developed in cooperation with the training centre affiliated to Mercedes-Benz Ludwigsfelde GmbH.

Publication: Tallig, G.; Zender, R.; Runge, M.: Frameworkbasiertes Augmented Reality-Lernszenario in der Kfz-Ausbildung. In: Workshopband der 15. e-Learning Fachtagung Informatik (DeLFI), 2017.

Editor: Gregor Tallig

Hard disk change in VR

Virtual learning environments can provide learners with the opportunity to safely apply theoretical content in a simulated practical situation. In the context of a master thesis, a prototype was developed that simulates hard disk replacement in a Storage Aria Network (SAN) and enables students and trainees of computer science related disciplines to experience theoretical learning content on net-based storage systems in practice. For the implementation a didactic concept according to the constructivist learning approach was developed and an efficient operating concept based on a Natural User Interface was implemented, which was realized by the CORPUS framework.

Publication: Lutze, F.; Zender, R.; Lucke, U.: "HardDrive Exchange – Eine VR-Lernanwendung zur Durchführung von Festplattenwechseln in Speichersystemen", in: Die 14. e-Learning Fachtagung Informatik (DeLFI), 2016.

Editor: Fabian Lutze

Photo: own representation

CORPUS - Framework for systematic gesture acquisition and processing

Sensors that can detect the movements of the human body play an increasingly important role in human-computer interaction. They allow the derivation of gestures that enable the more intuitive control of novel applications, such as computer-aided design or virtual reality applications. In a master thesis the framework CORPUS was developed, which combines the data of several sensors and provides client applications with a uniform body model. The model is accessed via a RESTful Webservice or a WebSocket connection. By abstracting from the sensors, the framework thus enables an easier development of applications that meet the increasing requirements for communication between humans and computers. In order to prove the functionality of the framework, a VR application was implemented, which enables the user to learn the basics of the piano.

Publication: Weise, M.; Zender, R.; Lucke, U.: "Systematic Fusion of Gesture Sensors for Practical Learning in Virtual Environments", in: Proc. 15th IEEE International Conference on Advanced Learning Technologies (ICALT 2015), IEEE, 2015.

Editor: Matthias Weise