This chapter is from the book
This chapter is from the book
This chapter is from the book
9.2 System Control Issues
In this section, we discuss higher-level characteristics of the human and of the system that influence the design and user experience of system control interfaces.
9.2.1 Human Factors
3D UI designers have to deal with a number of perception issues that can limit the user experience of a system control interface. Issues include visibility, focus switching, and the choice of feedback modalities.
Visibility is probably the most prevalent issue in 3D applications. In both VR and AR, system control elements (such as menus or buttons) can occlude the content of the environment. Scaling down system control elements could be an option if supported by the screen resolution, but this may result in (further) reduction of legibility. Another approach is to use semitransparent system control elements. While this may make content more visible, it may come at the cost of reduced visibility and readability of the system control element itself. Visibility issues particularly effect graphical widgets (see section 9.5).
Focus switching occurs when a system control element is visually decoupled from the area where the main interaction is performed. For example, if you are using a tablet to display menus while modeling in an immersive environment, you will need to switch focus between the tablet screen and the object you are working on. The visual system may literally have to adjust its focus (accommodation) when using the separate displays. In addition, if system control actions occur frequently, this may interrupt the flow of action. And especially when multiple displays are used, the sequence of actions in a task may influence performance (Kruijff et al. 2003; McMahan 2007). Collocated menus (see section 9.5.1) are one way to avoid this sort of switching.
In a 2D desktop GUI, mode change feedback is often a combination of haptic, auditory, and visual events: the user presses the mouse or touchpad and may feel a click (haptic), hear a click generated by the input device or operating system (auditory), and observe how the appearance of, for example, a button may change (visual). While it may seem straightforward to port the same feedback to a 3D application, this is often not the case. Visual button press feedback may easily remain unnoticed, while auditory feedback may be drowned out by ambient noise. Multimodal feedback is often a good choice to deal with these issues; we will take a closer look at this type of interface in section 9.9.
The main cognitive issue in system control is the functional breadth and depth of the system. Depending on the complexity of an application, a varying number of options (functions) may be available. These options are likely structured in different categories (breadth). Each category may have several options that may be further structured in subcategories (depth). Complex functional structures may cause users to be cognitively challenged to understand which options are available and where they can be found, and designers should be careful to create understandable classifications of functions and to hide rarely used functionality.
Ergonomic issues in system control interfaces include control placement, and the pose, grip, and motion types a particular device is used with. Shape, size, and location of controls can highly affect system control. For example, the button to trigger a system control action may not be easily reachable while a device is held with a particular grip, and thus pressing the button may require regrasping the device in a grip that is not optimal for other tasks. This is often the case in handheld AR setups: holding the device for accessing system controls with the thumbs, for example, is not compatible with the grasp used for viewing the augmented scene. Designers should investigate the relationship between different grips and the accessibility of buttons. For more information, refer to Veas and Kruijff (2008, 2010), as well as standard industrial reference guides (Cuffaro 2013).
The motion required to perform a specific system control action is another ergonomic issue. For example, for a 1-DOF menu (see section 9.4.1), we need to consider how a user can rotate his wrist, as well as the placement and size of the menu items that will lead to comfortable and efficient selection.
Finally, when multiple devices are used for system control and other actions, designers should consider how switching between devices is accomplished, where devices may get stored (“pick up and put away”), and how the tasks are matched to specific devices. For example, while using a tablet with a pen might improve the control of detailed menus within an immersive environment, the flow of action can be disrupted when users need to switch frequently between the tablet and, for example, a 3D mouse for other tasks.
9.2.2 System Factors
As we noted above, system control is often the glue that holds together the application. High-level characteristics of the system are important to consider when designing system control interfaces. System characteristics can even dictate specific choices for system control. The main issues are the visual display devices, the input devices, and ambient factors.
Visual displays will impose specific perceptual boundaries, such as resolution, size, and luminance, that will affect what system control methods may be used and how well they will perform. For example, more complex applications can force the use of a secondary screen (such as a tablet) for system control, as system control would otherwise clutter and occlude the main screen.
The choice of input devices defines the possibilities for control mappings. General-purpose devices such as a stylus or joystick may work with a number of system control techniques. In other cases, you may need to introduce a secondary device or physically extend the primary input device to provide controls that are well suited for system control methods. Extending an existing device or designing a completely new device is not trivial, but it is certainly possible because of the wide availability of DIY approaches (see Chapter 6, “3D User Interface Input Hardware,” section 6.6).
Ambient system factors such as noise, device constraints (e.g., no tethered devices possible, nowhere to place additional devices), or the motion range of a user may also affect the design of system control techniques and system control task performance. For example, ambient noise may cause recognition errors in speech interfaces, and such errors can reduce user performance and perceived usability.