Using AR/MR/VR/XR to evaluate virtual prototypes

VR model of a car radio
VR model of a car radio

In the development of technical devices and machines, besides functionality, the user interface is central. An attractive design, a good user experience, and usability are all aims in this process. To do this, user interfaces must be evaluated in the development process by future users. Often costly prototypes are individually manufactured for this purpose. Virtual prototypes in AR/MR/VR/XR can also be used, which results in a reduction in associated costs. However, producing virtual prototypes is currently still complex and requires correspondent programming skills. We are working to simplify this process and to advance evaluation methods with virtual prototypes. Our efforts include: 

Remote UX and usability testing for technical devices

AR model of a toaster on the smartphone
AR model of a toaster on the smartphone

The use of virtual prototypes in the development of technical devices and machines as well as the evaluation of user interfaces provides new evaluation opportunities. Material prototypes enable only local testing by the manufacturer. In contrast, virtual prototypes can reach users, where they are, to undergo testing. This process is called remote testing and is used intensely for web sites and mobile apps. We are working to establish this approach for virtual prototypes and are investigating various objectives in this effort:

  • How must suitable apps be designed for remote testing? It is important that the apps themselves do not influence the quality of the evaluation to realize valid results.
  • Are there differences in remote testing as related to AR, MR, or VR?
  • How can additional questionnaires be integrated into the evaluations to obtain additional user information?
  • How can additionally collected interaction data be evaluated to obtain further indications about problems with the user interface of a virtual prototype? This crosses over with the next area of research?

AI-based interaction and interface evaulation

Interface of AI-based interaction evaluation using the example of a user test for a coffee machine.
Interface of AI-based interaction evaluation using the example of a user test for a coffee machine.

Artificial intelligence (AI) is currently a hot topic in many areas. It has played a large role for some time in evaluating user interfaces. AI enables rule-based testing of web sites, for example, for certain structural problems. This is called static analysis. But user data from interfaces can also be evaluated and checked for conspicuous patterns. These patters, so-called usability smells, allow us to draw conclusions or inferences about user interaction problems. We are advancing the recognition of usability smells with our research in this area. We are focused on newer interface technologies, such as AR/MR/VR/XR, and on the evaluation of interaction with virtual prototypes (see the section above). We are also investigating the potential of static analysis of user interfaces of technical devices and machines based on virtual prototypes. In the future, machine learning algorithms will play role in this area, which previously have been used rarely in this topic. Previous results of our work in this field of research include: