Should You Build a Multimodal Interface for Your Web Site?
- Does the New Input Mode Add Value to the Web Application?
- Does the Application Leverage the Strengths of the New Mode and Avoid Its Weaknesses?
- Does the User Have Access to the Required Hardware and Software Required by the New Mode?
- Where Will Multimodal Applications Be Used?
- Conclusion
Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
Multimodal interfaces promise to enable users to enter requests and information using any of several input modes such as a mouse and keyboard, handwriting, and speech. Multimodal interfaces are the current hot technology. Everyone is talking about multimodal interfaces—how natural they are, and how users like them. It seems as if multimodal interfaces offer solutions to many of our user interface problems, as well as enabling new classes of applications.
But wait. While multimodal user interfaces promise to improve the naturalness and usability of Web applications, the incorrect application of a new mode might make a user interface cumbersome and awkward.
Three general questions will help you decide whether a new mode of input should be added to your application:
Does the new input mode should add value to the Web application?
Does the application leverage the strengths of the new mode and avoid its weaknesses?
Does the user have access to the required hardware and software required by the new mode?
Each of these questions will be discussed in detail in the following sections.
Does the New Input Mode Add Value to the Web Application?
Using a new mode just because it is available does not make a useful or even usable user interface. Each additional mode must add its own value to the Web application. Here are examples of new input modes that offer additional value:
Offload overloaded existing modes. A new mode offloads content from existing modes. For example, a keyboard and mouse are traditionally used to manage a display. Managing the location and visibility of multiple windows within a display can be offloaded from the mouse and keyboard to verbal commands, freeing the keyboard and mouse to manage the application content within the windows.
Replace traditional modes that are not practical in specific situations. In-car computing has practical constraints for the driver, whose eyes are busy watching the road; and whose hands are busy manipulating the steering wheel, gear shift, and controls for lights, horn, and windshield wipers. Speech seems a reasonable mode to replace the mouse and keyboard for drivers. Another example is that if it is too noisy for automatic speech recognition to recognize what you say, then use a keyboard or handwriting.
Increase bandwidth by using modes in parallel. New modes present new opportunities to increase the rate of communication between the user and the computer. For example, voice commands can be used as a "third hand" in games that require both hands to manipulate game controls. Voice also can be used in a drawing application to control the parameters of drawing tools. An example is using voice to change the line width and color of a line drawing tool while it is being used.
Use the most expressive mode. While the keyboard can be used to enter text, voice can be used to speak text and convey emotion. Drawing a map often conveys better accuracy than listening to verbal directions.
Use mixed modes. Some requests are easier to express using multiple modes rather than single modes. For example, the following combined verbal and pointing request:
"Put that" (point to the third icon in the fourth row) "there" (point to the second container from the left edge of the display).
is easier than saying:
"Put the third icon in the fourth row into the second container from the left edge of the display."
Each new mode should be used to provide additional value. If not, then the new mode will just get in the way of the user performing desired tasks.