The Evolution of Human-Computer Interaction
Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
The Emergence of Usability
Human-Computer Interaction (HCI) is the study and the practice of usability. It is about understanding and creating software and other technology that people will want to use, will be able to use, and will find effective when used. The concept of usability, and the methods and tools to encourage it, achieve it, and measure it are now touchstones in the culture of computing.
Through the past two decades, HCI emerged as a focal area of both computer science research and development and of applied social and behavioral science. Some of the reasons for its success are straightforwardly technical: HCI evoked many difficult problems and elegant solutions in the recent history of computing—for example, in work on direct manipulation interfaces, user interface management systems, task-oriented help and instruction, and computer-supported collaborative work. Other reasons are broadly cultural: The province of HCI is the view the nonspecialist public has of computer and information technology and the impact that technology has on their lives in the sense that it is the visible part of computer science and technology. The most recent reasons are commercial: As the underlying technologies of computing become commodities, inscribed on generic chips, the noncommodity value of computer products and services resides in applications and user interfaces—that is, in HCI.
The beginning of HCI is sometimes traced to the March 1982 (U.S.) National Bureau of Standards conference, "Human Factors in Computer Systems," though related conferences and workshops were conducted throughout the world at about that time. It is surely true that after the Bureau of Standards conference, HCI experienced meteoric growth. However, four—largely independent—threads of technical development from the 1960s and 1970s provided the foundation that allowed this interdisciplinary program to gel so rapidly in the early 1980s.
These four threads were prototyping and iterative development from software engineering; software psychology and human factors of computing systems; user interface software from computer graphics; and models, theories, and frameworks from cognitive science. It is interesting to remember these four roots of HCI, since the concerns that evoked them and that brought them together are still underlying forces in HCI today.
Prototyping and Iterative Development
In the 1960s, advances in computer hardware enabled new applications requiring software systems of far greater scale and complexity than before. But these greater possibilities exacerbated problems of software development: cost overruns, late delivery, and ineffective and unreliable systems that were difficult to maintain. This was termed the "software crisis." It led to the emergence of software engineering as a professional discipline.
The software crisis was never resolved per se. Rather, it helped to establish design and development methods as a central topic in computing. Early approaches emphasized structured decomposition and representation of requirements and specifications, and a disciplined workflow of stages and hand-offs called the "waterfall." Indeed, this was part of a broad movement toward more formal design methods during the 1960s (Jones 1970).
However, empirical studies of the design process and practical experience in system development raised questions about the new design methods. A prominent case was Brooks's (1975/1995) analysis of the development of the IBM 360 Operating System, one of the largest and most scrupulously planned software design projects of its era. Brooks, the project manager, observed that critical requirements often emerge during system development and cannot be anticipated. He concluded that software designers should always "plan to throw one away."
This was a striking lesson and one that continues to inspire studies of design. Design is now seen as opportunistic, concrete, and necessarily iterative. Designers typically work toward partial solutions for subsets of requirements, using prototypes to evoke further requirements and, indeed, to reformulate the goals and constraints of the problem. By providing techniques to quickly construct, evaluate, and change partial solutions, prototyping has become a fulcrum for system development.
Software Psychology and Human Factors
The software crisis intensified interest in programming as a human activity. It heightened the need for more programmers, for better-trained programmers, for more productive programmers. The development of time sharing and interactive computing allowed new styles of programming and made the dynamics of individual programmer activity more salient. Programming became recognized as an area of psychology involving problem solving and symbol manipulation (Weinberg 1971).
Through the 1970s, a behavioral approach to understanding software design, programming, and the use of interactive systems developed rapidly. This work addressed a wide assortment of questions about what people experience and how they perform when they interact with computers. It studied how system response time affects productivity; how people specify and refine queries; how syntactic constructions in programming languages are more or less difficult; and how aids like mnemonic variable names, in-line program comments, and flowcharts support programming. By the end of that decade, a software psychology research community had formed (Shneiderman 1980).
This work inspired many industrial human factors groups to expand the scope of their responsibilities toward support for programming groups and the usability of software. During the latter 1970s, several extensive compilations of research-based guidelines appeared, and most computer manufacturers (there were no exclusively-software companies at that time) established usability laboratories, whose scope of responsibility steadily expanded.
New User Interface Software
Before the 1960s, the notion of "user interface" was completely unarticulated. The focus of computing was literally on computations, not on intelligibly presenting the results of computations. This is why the early visions of personal, desktop access to massive information stores (Bush 1945), graphical and gestural user interfaces (Sutherland 1963), and synchronous collaboration through direct pointing and shared windows (Engelbart and English 1968) are historically so significant.
Through the 1970s, advances in workstation computers and bit-mapped displays allowed these early visions to be consolidated. A prominent example is work at the Xerox Palo Alto Research Center on the Alto computer and the Smalltalk-72 environment. It is striking that the essential concepts of desktop computing that guided the next 20 years of research and development emerged during this early period.
Models, Theories, and Frameworks
During the latter 1970s, cognitive science had coalesced as a multidisciplinary project encompassing linguistics, anthropology, philosophy, psychology, and computer science. One principle of cognitive science was that an effective multidisciplinary science should be capable of supporting application to real problems and to benefit from it. Many domains were investigated, including mechanics, radiology, and algebra. HCI became one the original cognitive science domains.
The initial vision of HCI as applied science was to bring cognitive science methods and theories to bear on software development. Most ambitiously, it was hoped that cognitive science theory could provide substantive guidance at very early stages of the software development process. This guidance would come from general principles of perception and motor activity, problem-solving and language, communication and group behavior, and so on. It would also include developing a domain theory, or theories, of HCI.
A prominent early example was the Goals, Operators, Methods, and Selection (GOMS) rules model for analyzing routine human-computer interactions (Card, Moran, and Newell 1983). This was an advance on prior human factors modeling, which did not address the cognitive structures underlying manifest behavior. But it was also an advance on the cognitive psychology of the time: It explicitly integrated many components of skilled performance to produce predictions about real tasks. The GOMS model is important because it set a standard for scientific and theoretical rigor and innovation that became a defining characteristic of HCI.
The foundations of HCI remain an active focus of research. The first group of papers in this volume shows how these foundations are continuing to expand the disciplinary scope and relevance of HCI models, theories, and frameworks to practitioners.
User-Centered System Development
These four starting points converged in the early 1980s through organizational initiatives including ACM's Special Interest Group in Computer-Human Interaction (SIGCHI) and IFIP's Task Group on Human-Computer Interaction (later, Technical Committee 13). Initially, HCI had two foci, methods and software, and a major theme was the integration of the two in a framework called user-centered system development.
The methods focus was on techniques to achieve better usability. This entailed explicating the concept of usability with respect to learning, skilled performance, and subjective experiences, like satisfaction and fun. It involved the development and empirical validation of models and theories. It involved laboratory studies of people learning and using systems, and of techniques for evaluating systems. And it involved working within development organizations to understand how to involve usability professionals earlier and more effectively in software development. The methods focus became known as usability engineering.
The software focus of HCI was concerned with inventing and refining graphical user interface concepts and techniques to make systems more powerful, more useful, and more usable. An important consequent objective was to make new user interface concepts and techniques easier for developers to employ. This entailed the development of toolkits and software frameworks. The software focus became known as user interface software and tools.
The method and software foci often cross-leverage one another. For example, new user interface metaphors are developed from theory and from user studies, refined and implemented in prototype systems, evaluated in further user studies, and then incorporated into toolkits. Through the past two decades, these original focus areas have continued to expand and diversify, though their synergistic relationship remains a cornerstone of HCI. Other focal areas have developed, such as groupware/cooperative activity and media/information.
Usability Engineering Methods and Concepts
The early focus of usability engineering was evaluation: gauging the success of implemented software and systems with respect to measurable criteria. It was patterned on the laboratory-oriented human factors paradigm in telecommunications. However, the ascendance of prototyping and iterative development in software and the ambition of engaging cognitive science as a foundation for human-computer interaction pushed the focus of evaluation work upstream in the system development process.
Prototyping and iterative development place a high premium on formative, rather than summative, evaluation (Scriven 1967)—that is, on evaluation that is carried out within the development process and that can guide redesign, rather than merely gauge attributes of a design result. Formative evaluation methods are often qualitative; a typical method involves having people "think aloud" as they perform a task. Theory-based models and tools took an even more ambitious position, seeking to enable analytic evaluations of designs before they were implemented even as prototypes. Usability engineering has remained a core concern of the ACM SIGCHI community, and its CHI Conference, but subcommunities have also developed for usability methods (the Usability Professional's Association) and for theory-based models (the Conference on User Modeling).
The objective of providing guidance earlier in the system development process entrained fundamental changes in usability engineering. An example is the recognition that the earliest point for impact is requirements analysis. The sociologists and anthropologists who had come to HCI through its connection to cognitive science showed through field studies of work practices that people do their work and use their tools in surprisingly creative ways (Suchman 1987). But their work practices are often not easy to anticipate without direct study or direct user participation in the development process. A subcommunity addressing these themes has formed around the Participatory Design Conference and the ACM Conference on Computer-Supported Cooperative Work (CSCW).
A second example is the growing focus on design methods: Usability can be designed as well as evaluated, but a design-time usability process entails coordination with graphical and interaction designers that is still just beginning. This design thread has led to another subcommunity formed around the ACM Symposium on Designing Interactive Systems (DIS), a conference series started in 1995.
Complementary to moving usability work further upstream in design and development is a focus in usability engineering on cost-benefit tradeoffs in methods (Bias and Mayhew 1994). The most evident manifestation of this theme has been widespread effort at developing "low-cost" inspection and walkthrough methods. But cost-benefit is a complex issue. Different methods have different goals, producing different types of benefits. Some continuing questions for usability engineering are how methods can leverage foundations in science and theory; how methods can be evaluated; and how different types of methods, like laboratory studies, field studies, walkthroughs, and analytic models, can be integrated with one another and with other methods and processes of system development.
The second group of papers in this volume shows how usability engineering is developing. It is addressing an ever-greater variety of types of systems and usability phenomena, such as worker adaptation. It is developing new approaches, such as the integration of user interface management systems with cognitive models. But it is also focusing on better consolidation and utilization of results and concepts in hand.
User Interface Software and Tools
User interface software and tools is concerned with user interface concepts and metaphors, display and interaction techniques, and software development methods. This is surely the most visibly successful area of HCI. The user interfaces that more or less everyone encounters more or less every day—the mouse and bitmap display, the desktop metaphor and window management, display widgets like scroll-bars, menus, and dialog boxes—emerge from this research and development work. This area is a distinct professional subcommunity within HCI, with its own ACM conference, User Interface Software and Tools, since 1987.
A continuing research thread in this area is architectures for user interface software. An early objective was separation of the user interface and application functionality into distinct layers. This approach modularized the user interface in user interface management systems, encouraging iterative redesign (for example, Tanner and Buxton 1985). However, layering entrained limitations on the granularity of user interface interactions. It also proved to be an obstacle to incremental development methods, because it presupposed top-down decomposition with respect to what was user interface and what was application functionality. Current approaches favor developing user interfaces and functionality in the same language, either in new languages invented for this purpose, like Visual Basic, or through extensions to standard languages for implementing functionality, such as libraries and toolkits for C++ or Java.
A key goal in this area has always been to ease the development of interactive systems. Through time, this goal has become more challenging because the skills of application developers have become more diverse. For example, this motivated a family of prototyping tools based on the premise that user interface software could be directly created "by demonstration."
The third group of papers summarizes the progress to date in user interface software and tools and identifies some of the key challenges for the future. The key questions for user interface software and tools are what models and techniques will be most appropriate for creating interfaces with controls and displays quite different from those of the graphical user interface paradigm of the past 20 years. For example, what are suitable architectures and tools for interfaces with voice, gesture, or position-sensing controls and immersive or wearable displays?
Groupware and Cooperative Activity
The early vision that interactive computing would enable human collaboration was already borne out in the 1970s by ARPA Net e-mail and Usenet newsgroups. Before HCI even existed, there had been considerable research on teleconferencing and other networked collaboration (Hiltz and Turoff 1978/1993). In the 1980s, possibilities for computer-supported cooperative work (CSCW), as well as the scope human-oriented issues considered, became more diverse. E-mail became a universal communication tool. Videoconferencing, electronic meeting rooms, and workflow support systems became common. Several very visible experiments with media spaces and shared virtual environments were carried out. Through the 1990s, networked collaboration became richer and more accessible—for example, through the World Wide Web.
CSCW, as a subcommunity in HCI, formed around the ACM CSCW Conference series starting in 1986. At first, the focus was on collaborative systems (groupware) and human-computer interactions with collaborative systems. But the more significant impact of CSCW, one that goes beyond the conference, is the recognition that all systems are used in a social context. In this sense, CSCW has become more a view of HCI than a subcommunity with it. CSCW has served as a conduit for the expansion of the science foundation of HCI to incorporate activity theory, ethnomethodology, and conversation analysis, among others.
Studies of work practices and of technology in the workplace have radically altered the scope of HCI concerns. Subtle features in the organization of work are often critical but easily missed in standard requirements processes. Some of the powerful features may be idiosyncratic to a particular type of work context, such as air traffic control rooms, or even to individual workplaces and groups. Even when technology appropriately addresses workplace needs, adopting new technology ineluctably disrupts work, at least temporarily. The effects of technology are not uniform across an organization; some workers benefit, but others may lose. And technology can be a double-edged sword: Making someone's work easier reduces the skill required to perform the work. In a given organizational context, it may reduce status, pay, and even job security.
Much of the focus of CSCW continues to be on new groupware systems. In the past decade, this thrust of CSCW has emphasized Internet systems. For example, there are many current reworkings of the classic multi-user domain (MUD) paradigm but bundling suites of communication tools, such as chats, multi-user whiteboards, videoconferencing, and shared applications, with graphical Web-clients.
Recently, work on groupware systems and cooperative activity has converged around the concept of "community." Examples of communities are varied, including dispersed groups of collaborating scientists, people subscribed to given newsgroups or other online services, and towns or neighborhoods with local networking infrastructures. On the one hand, this work investigates how a variety of interactions can be coherently supported to foster the qualities of community in participants. On the other hand, it investigates the social and psychosocial consequences of participation in such communities.
The fourth group of papers in this volume examines the history, current approaches, and challenges of groupware and cooperative activity.
Media and Information
In the early 1980s, possibilities for media other than formatted text were quite limited. Toward the end of the 1980s, this had changed dramatically: Hypertext was everywhere. In 1987, Apple began including HyperCard with all of its personal computers. At the same time, standard image formats were making it easier to create and share graphics and visualizations. Relatively good quality synthetic speech became available on personal computers. In the 1990s, these trends accelerated: The World Wide Web made hypertext a standard information design, Web-pages and single-click e-mail attachments made sharing images as easy as sharing text, and surprisingly good speech recognition became available on personal computers.
Where CSCW was initially a subcommunity of HCI that ended up providing a new perspective on what HCI is about, multimedia and hypermedia were initially a small collection of enhancements to user interface presentations but have continued to develop and diversify to produce half a dozen new subareas of HCI. Each of these areas is redefining the scope of HCI. Some examples are digital libraries, visualization and virtual environments, spoken-language interfaces, and agent-based and recommender systems.
The World Wide Web is a vast collection of information, but it is not a library. Finding things in the Web may be hit or miss, but it has helped to focus research and development interest on the human aspects of building and accessing large hypertext databases. Digital libraries were originally conceived of as an integration of database systems, multimedia, and information retrieval. However, the rapid growth of interest in digital libraries and the consequent need to make them accessible to a wider range of people has transformed digital libraries into a subarea of HCI as well.
Graphics and visualization techniques were central to the development of the contemporary user interface paradigm. However, continuing advances in hardware speed and other underlying technologies now allow large-scale graphical animations. Users do not merely display and inspect static visualizations; they can view animated sequences of visualizations and navigate through visualized spaces. In a limiting case, users are surrounded by wall-sized displays, viewed through depth-enhancing goggles, perceptually immersed in their data.
Twenty years ago, sound in the user interface meant warning beeps. Sound got attention in circumstances where flashing text boxes sometimes did not. During the 1980s, HCI incorporated a wider range of nonspeech sounds into the user interface (sometimes called "earcons") and made progress enhancing the quality of synthetic speech and applying it in telephone-based information systems. More recently, advances in speech recognition and natural language processing, and in underlying hardware and software technologies, have allowed remarkable progress in speech input, particularly in dictation applications.
Artificial intelligence has always played a role in HCI. Intelligent help and tutoring systems have been heavily researched through the past two decades. One of the obstacles to widely deploying such techniques in user interfaces is the amount of knowledge engineering required. This is one reason that intelligent tutoring systems tend to be developed for relatively closed domains of knowledge, like elementary mathematics. More recent approaches are exploring self-organizing agents and recommendation systems that aggregate the decisions and actions of many people to provide information resources.
The hypertext conference has continued, but the digital library community now has its own ACM conference series. There is also an ACM conference on multimedia. The fifth group of papers in this volume presents a diverse view of ongoing work in this area. It seems this "subarea" of HCI is already too rich and too diverse to cohere. In the new millennium, media and information will surely become several distinct areas. For example, the sixth group of papers addresses the integration of computation with real environments, a focus that has become distinctive only recently and promises to be become far more important in the future.
The last group, "HCI and Society," includes discussions of challenges and possibilities for education, community-building, and the development of social capital more generally, as facilitated, or at least modulated, by information technology. In the new millennium these considerations may yet become mainstream as foundations for HCI and as criteria for usability engineering methods.
Toward the New Millennium
At the inception of HCI, the notion that computer systems and software should be designed and developed with explicit consideration of the needs, abilities, and preferences of their ultimate users was not a dominant view. Most writings about computing from the mid-1970s are stunningly dismissive of usability and rather patronizing of users. After only a decade, the computer industry and the discipline of computer science were transformed. The case had been made for a user-centered system development process, a process in which usability was a primary goal. People began to distinguish sharply between technology-driven exploratory development, which is now often accompanied by explicit disclaimers about usability, and real system development, in which empirically verified usability is the final arbiter.
With the advent of the 1990s, HCI research had become relatively well integrated in computer science. A 1988 Association for Computing Machinery (ACM) task force enumerated HCI as one of nine core areas of the computer science discipline (Denning et al. 1989). A joint curriculum task force of the ACM and the IEEE (Institute of Electrical and Electronic Engineers) recommended the inclusion of HCI as a common requirement in computer science programs (Tucker and Turner 1991). And HCI was included as one of ten major sections of the first Handbook of Computer Science and Engineering (Tucker 1997). In the 1990s, computer science students and the corporations that hire them demanded HCI courses in university curricula. Several major computer science departments have designated HCI as a research focus, and several comprehensive undergraduate texts have appeared.
In the computing industry, HCI practitioners have become well integrated in system development. HCI specialists have moved into a great variety of roles beyond human factors assurance. They have been routinely included in customer/user interactions to understand the need for new products, product planning and specification; in the development and evaluation of prototypes and systems; in the design of documentation and training; and in installation and user support. There has been an obvious trend for HCI specialists to be promoted into project management.
HCI remains an emerging area in computer science. Its four roots from the 1960s and 1970s—software engineering, software human factors, computer graphics, and cognitive science—have grown and intertwined. New influences and strands have been incorporated—broader social science, networking, media, information management, and artificial intelligence. HCI has become the focus for a new view of what computing is about. The future promises to be far more exciting than the quite exciting recent past. HCI only existed for the final quarter-century of the second millennium—time enough to irrevocably shake up the discipline of computer science and the trajectory of computing technology, but not a lot of time. HCI in the new millennium gives us much to look forward to.
Acknowledgment
This is a much revised and broadened descendant of a survey paper that appeared in Annual Review of Psychology, Volume 48, Palo Alto, CA: Annual Reviews, 1997, pages 501–522. That earlier version develops some of the points here in more detail and with more complete citations. I am grateful to Andrew Dillon, Brad Myers, Sharon Oviatt, Ben Shneiderman, Alistair Sutcliffe, Terry Winograd, and an anonymous reviewer for guidance, discussion, and comments on this essay.
References
Bias, R.G., and Mayhew, D.J., Eds. (1994). Cost-justifying usability. Boston: Academic Press.
Brooks, F.P. 1975. (1995). The mythical man-month: Essays on software engineering. (1995 Anniversary Edition). Reading, MA: Addison-Wesley.
Bush, V. (1945). As we may think. Atlantic Monthly, 176, 1, 105–108.
Card, S.K., Moran, T.P., and Newell, A. (1983). The psychology of human-computer interaction. Hillsdale, NJ: Erlbaum.
Denning, P.J., Comer, D.E., Gries, D., Mulder, M.C., Tucker, A.B., Turner, A.J., and Young, P.R. (1989). Computing as a discipline. Communications of the ACM, 32, 9–23.
Engelbart, D.C., and English, W.K. (1968). A research center for augmenting human intellect. AFIPS Proceedings of the Fall Joint Computer Conference, 33, 395–410.
Hiltz, S.R., and Turoff, M. (1978). The Network Nation: Human communication via computer. Reading, MA: Addison-Wesley. (Revised edition, 1993 by MIT Press, Cambridge, MA.)
Jones, J.C. (1970). Design methods: Seeds of human futures. New York: John Wiley & Sons.
Kay, A., and Goldberg, A. (1977). Personal dynamic media. IEEE Computer, 10, 3, 31–41.
Scriven, M. (1967). The methodology of evaluation. In R. Tyler, R. Gagne, and M. Scriven (Eds.), Perspectives of Curriculum Evaluation, 39–83. Chicago: Rand McNally.
Shneiderman, B. (1980). Software psychology: Human factors in computer and information systems. Cambridge, MA: Winthrop.
Suchman, L.A. (1987). Plans and situated actions: The problem of human-machine communication. New York: Cambridge University Press.
Sutherland, I. (1963). Sketchpad, a man-machine graphical communications system. Proceedings of Spring Joint Computer Conference, 329–346. New York: Spartan Books.
Tanner, P.P., and Buxton, W.A.S. (1985). Some issues in future user interface management system (UIMS) development. In G.E. Pfaff (Ed.), User interface management systems: Proceedings of the workshop on user interface management systems, held in Seehiem, FRG, November 1–3, 1983, 67–79. New York: Springer Verlag.
Tucker, A.B. (Ed.) (1997). The handbook of computer science and engineering. Boca Raton, FL: CRC Press.
Tucker, A.B., and Turner, A.J. (1991). A summary of the ACM/IEEE-CS Joint Curriculum Task Force Report: Computing Curricula 1991. Communications of the ACM, 34, 68–84.
Van Dam, A. (1966). Computer driven displays and their use in man/machine interaction, In F.L. Alt and M. Rubinoff (Eds.), Advances in Computers, 7, 239-290. New York: Academic Press.
Weinberg, G.M. (1971). The psychology of computer programming. New York: Van Nostrand Reinhold.