Technology has bestowed us with many comforts and luxuries, and made life way easier for us today. In order to use technology and its various inventions, it is vital we know how to use, accept and adopt it. Human interaction with technology takes place through desktop computers, cameras, mobile phones, internet etc. The interface between humans and system may become problematic at one point when the systems are less useful and usable. An advanced move towards building an enhanced human interaction calls for a close understanding of humans, information and various interactions. Providing the users with multiple modes of interaction with the system and offering various benefits over the conventional unimodal interaction is the step ahead. This is termed as Multimodal Interaction as it refers to the different modes available to the user for interacting. Multimodal interaction has created engaging experiences for users across different systems with multiple modalities ensuring an enhanced communication between humans and systems. Humans interact via a wide variety of modalities for communication in daily life; like speech, gestures, facial expressions, touch etc. The multimodal interaction is an effort towards humanizing the human-computer interface. Smartphone is one universally used tool that offers users with such multimodal interaction experience.
Smart phones and its technology have come a long way since its inception. Today smart phones can be found in almost everyone’s pockets and the use of a mobile phone has gone far beyond just calling. There is ample in a smartphone that a user can walk around besides the usual functions of calling and messaging. Smart phone manufacturers like the Apple, Samsung, HTC, Nokia, Sony and Micromax (of India) have equipped cellphones with sensors, microphones, touch screens, vibrators, accelerometers and gyroscopes.
With these technologies it has become possible to interact with the phones not just through sight but through touch, vibrations, movement, speech and various other multimodal interactions. The ability to interact with software using multiple senses provides designers and software developers with a fair opportunity to create accessible, immersive, easy to use and understandable software. By modality, we mean the channel of communication used to put across or obtain information. With such an interface, we provide humans with multiple input modalities. Software applications and computing systems that have multiple modalities of input and output are called multimodal.
By adding sound and touch to software, the use of visual aids can be minimized; this helps the people who are visually or physically impaired. Users can have an easy interaction, without having to look at the screen, with speech acting as the input while driving, riding, running etc. They can easily interact even when their hands and eyes are engaged elsewhere. Such an interaction is well adapted to fulfill the needs of various users in different circumstances. With several senses engaged in an application the experience becomes richer and more immersive. Multimodal interaction is more engaging and facilitates easier, faster and an efficient flow of information. It increases the accessibility of the applications as multimodal interfaces can be accessed in various manners by the users. It then serves to a wide category of users and hence gains a larger acceptability.
Few applications and even fewer phone user interfaces on the market take full advantage of these capabilities. This feature might help users a great deal while using e-commerce and location finder applications.
Mostly sound, touch and vibrations are treated as separate entities or combined without much thought. It seems hard to move away from today’s more visual approach as multiple senses mean increased complexity and it demands more in terms of both design and technology. Since there is no standard approach to multimodal interaction in smart phone, it becomes more cumbersome to use normal visual interaction. The benefits however are many; more senses mean more involvement, more information and more possibilities.
Finally Multimodal interfaces offer users an opportunity to choose the most natural interaction pattern depending on the actual task to be accomplished, the context, and according to their own preferences and abilities. While this also gives developers an opportunity to embed multiple modes in their user interface depending on the type of device. This concept creates a prospect for developing applications that provide a multimodal User Experience. Hence, the mobile multimodal user interfaces have the potential to fulfill the requirements and guidelines for Universal Design.