PROJECT DESCRIPTION
CLUSTER 'USER INTERFACES AND BEHAVIOUR'
PREPARED FOR TELEMATICS INSTITUTE
Editors:
Dr. M.A. Neerincx and Dr. J.H. Kerstholt
TNO Human Factors Research Institute
P.O. Box 23
3769 ZG Soesterberg
Dr. E.M.A.G. van Dijk and Prof. dr. ir. A. Nijholt
CTIT, University Twente
Department of Computer Science (DCS)
P.O. box 217
7500 AE Enschede
S. Pemberton
CWI, Research Centre for Mathematics and Computer Science
P.O. Box 94079
1090 GB Amsterdam
Dr. G.P.J. Spenkelink
CTIT, University Twente
Ergonomics Department (ED)
P.O. box 217
7500 AE Enschede
Participating research institutes:
Participating companies:
Coordination Telematics Institute:
Web-based Services for Information and Commerce: User-Centred Design Principles, Methods and Applications
Acronym: U-WISH (Usability of Web-based Information Services for Hypermedia)
General design principles will be developed to attune Web-based user interfaces to the user tasks and abilities. These principles will be based on experiments addressing the information transfer for three different applications: a theatre booking service, a public counter ('overheidsloket 2000'), and a company Web-site. Subsequently, the principles will be integrated into a design method consisting of techniques and tools for the specification and assessment of Web-environments. Finally, the method will be applied and evaluated for new applications.
This project distinguishes two phases. Phase I will provide general design principles for attuning Web-based user interfaces to the user tasks and abilities, and an overview of the state of the art regarding user-centred design and evaluation techniques and tools. Specific attention will be paid to human information processing abilities of the elderly. To derive the principles, experiments will be conducted in three different test environments: theatre booking services, public counters, and company web-sites. To establish the other elements of a user-centred design method, techniques for assessing usability in the process of software development, a specification technique for multi-modal dialogues, and dedicated tools for the design of Web-support functions will be further developed. In phase II, the results of phase I are integrated into a concise and coherent method for the design of Web-based services for information and commerce. The method will be applied to develop new applications and the results will be evaluated. Finally, it will be provided as an electronic (Web-based) handbook.
Important applications of telematics are Web-based services for information and commerce. They appear in a wide variety of domains. Examples are electronic public counters for government information, electronic publications of publishers, theatre-booking agencies, travel information services, telemarketing, teleshopping, telebanking, and company Web-sites to communicate with interested parties. Notwithstanding the diversity, there are some main generalities across such applications with respect to information provision, user interface, user tasks and usability bottlenecks. The information sources or data bases are situated at several locations and interconnected via several paths, while the data base content is timely and dynamic. The network user interfaces contain hyperlinks to corresponding information providing information in several formats, such as text, graphics, photos, audio and video, and these interfaces can provide various communication functions.
The user tasks, in the Human-Computer Interaction (HCI)-specific technical meaning of the word, of Web-based services consist of monitoring or exploring whether possible important information is present, assessing the meaning and relevance of the information given the current situation and task objectives, and, if needed, making decisions based on this assessment such as further exploration or buying a product. In current Web-based systems, bottlenecks appear such as inefficient navigation, disorientation and loss of overview (Nielsen, 1995; Sutcliffe, 1995; Passenier & Neerincx, 1996; Neerincx & Houttuin, 1996). As a consequence users often do not find the information and services they need.
The user interface of a service, though often seen in lay circles as only the surface structure of an application, such as the use of menus and buttons, actually covers both a broad and a deep region. Apart from the supplier side of the application, such as the required functionality, and the (to-be-perceived) structure of the data, the role of the user (such as a potential customer 'voluntarily' browsing an online shop, or an employee trying to accomplish some required task), their abilities (inexperienced, experienced), and physical and psychological makeup (colour-blind, visually restricted, which may include people operating machinery or using an application to repair a machine), to name but a few factors, all have an impact on the required design of an application.
The number and type of users of Web-based services is growing enormously, and it should be remarked, that while the details of how Web applications are implemented may change in the coming years, the underlying model of platform-independent delivery of data and applications to a wide public will remain. Consequently, the need for adapting the interface to the specific user abilities, knowledge and experience is growing too. This project aims at increasing the accessibility and usability of telematics systems and services for all potential users. In this respect, special attention will be paid to the specific abilities and corresponding interface support needs of the elderly. The elderly will be an important target group in the future, not only because of the proportional increase of the ageing population but also because they can and will live on their own longer. Research showed that the elderly will only accept telematics services and systems that are really easy-to-use (Cjaza, Clark, Weber & Nachbar, 1990).
Taken together, the combination of users' tasks, abilities and the interface support determine the actual user performance or, in other words, the effectiveness of Web-based services. For specific tasks and abilities, a specific type of user support will be required. The general question is:
How to attune the Web-based support and interaction style to the users' goals and abilities?
Sub-questions to be answered are concerned with establishing the main success and failure factors and the interactions between these factors that appear in the four components of the human-telematics system:
So the main beneficiaries of this work are companies and organisations designing Web-based applications and services who need to understand the parameters for application design (e.g. stakeholders), and need a methodology based on well-founded and tested principles for designing such systems (e.g. software companies), and in the longer term for the users of such systems, who will have easier to use and easier to understand systems at their disposal. This project will provide the general design principles for attuning Web-based user interfaces to the user tasks and abilities. It is important to note that the designer's perspective of users, user tasks and the corresponding optimal dialogue can be incomplete and incorrect. In order to be able to realise user-friendly Web-based services, more empirical data will have to be gathered, translated into guidelines, and made available to designers. The goal of the present study is therefore to derive usability guidelines, recommendations and principles from experiments in which the user abilities, tasks and user interface support functions are systematically varied. Subsequently, we will develop user-centred techniques and tools for the design and evaluation of telematics applications. These techniques and tools will be tested: a prototype of a telematics application will be designed and evaluated.
This project comprises a number of new aspects in relation with ongoing research. The first aspect is integration. Human Factors principles and guidelines, techniques for HCI-specification and assessment, and tools are developed in a coherent design framework, bringing these elements together. A major added value of this project is the coming together of researchers from several different angles of attack within the HCI field: from experimental cognitive, formal cognitive, and software engineering areas.
Second, this project will provide an empirical foundation of the user-centred method, a major need in the field of HCI-research (see section 1.6, the state of the art). This method contributes to the challenge of developing a 'design for all' approach, i.e., to establish a design for a large range of users (including the elderly) and their goals or tasks.
Third, the research approach provides results that are of interest for HCI-theory. The approach of conducting experiments by systematically manipulation of factors in a realistic task environment provides a new basis of behaviour and cognitive theory (cf. 'practical theory' Silverman, 1991; 'activity theory' Nardi, 1996; 'naturalistic decision making' Zsambok & Klein, 1997). This will result in a theoretical framework to describe and evaluate user strategies and styles. Techniques for the formal specification of multi-modal dialogues and dialogue strategies and of different interaction styles will be developed. Syntactic, semantic and pragmatic aspects of human-computer interaction will be distinguished. From all three points of view models of (multi-modal) human-computer interaction will be developed and specification languages will be defined that allow for the definition of different interaction styles. One of the main questions of interest is how these interaction styles are reflected in the syntactic/semantic/pragmatic framework underlying the specification. An approach will be chosen that enables the description of interaction profiles for different groups of users and the use of these profiles for design and evaluation.
The proposed project complies with the aims and objectives of the Telematics Institute as it will provide design guidelines that will ensure efficient interaction between people and telematics systems. Specifically, because a human centred approach is used, these guidelines will facilitate the introduction to and acceptance by users and support them in obtaining their goals efficiently.
The guidelines are based on research investigating the behaviour of users interacting with telematics systems and services. The approach used focuses on human information processing and design methods from different research areas (HCI, language engineering, software engineering). Theories of human task performance will be further developed, translated and validated in the context of Web-environments. Further progress is foreseen in the integration of theory and methods of the disciplines of HCI, language engineering and software engineering, resulting in improved techniques for specifying and assessing cognitive task demands, support functions and dialogues.
Web-based information and commerce services are regarded as an important economic factor in the near future. A central concern is the adaptation of the information sources to all kind of users, so that telematics systems will become available to a wide range of users. Special attention will be paid to elderly people, a user group of growing importance.
This project proposal is submitted to the cluster "User Interfaces and Behaviour" focusing on the first and third objective of this cluster, i.e., [1] to develop and test a cognitive engineering method comprising a set of guidelines for user-centred design and [3] to improve the usability of telematics systems and services in general, and in particular for elderly people. In line with the rational of this horizontal cluster, this project will yield and apply knowledge related to various projects in other clusters, in particular to the clusters "Electronic Commerce" and "Data- and Knowledge Engineering (centring around the theme knowledge management)". The present project will focus on Web-based information and commerce services. However, because the guidelines are based on general characteristics of human information processing they will be useful to a wider range of applications.
In the 80's, the Graphical User Interface (GUI) became familiar to more and more users who did not have specific computer expertise. These interfaces were based on the 'WIMP'-concept (Windows, Icons, Menus and Pointer). In particular the direct manipulation, provided by WIMP-interfaces, was shown to improve the usability for non-expert users. In the 90's, GUI's are becoming widespread and a standard 'look-and-feel' has emerged for user interfaces for different platforms (Mac, Motif, PM and Windows). Furthermore, usability guidelines and style guides have been developed for these interfaces. Many of these guidelines are of a more or less overlapping nature (see, for instance, Williges et al., 1987 for a classification scheme or Smith & Mosier, 1986 for an extensive review of guidelines for software interface design).
Currently user interface technology is developing fast. The development of the World Wide Web has had a major impact on this development. So-called Web browsers can be used to search for multimedia information in databases via a network. These browsers are more-and-more an integrated part of the user interface, involving a new 'look-and-feel' for which standards are being set. For example, users can navigate in a 3D-world, can interact with 3D objects and can be informed about process changes from remote systems by intelligent agents. The future Network User Interfaces (NUIs) offer a consistent, browser-like interface for navigating through local and remote file systems. They can display 'Java applets' and other dynamic Web content on their desktops without a Web browser. They automatically update dynamic content using 'push' and 'pull' Webcasting technologies. And they will blur the increasingly irrelevant distinctions between native/cross-platform and local/remote applications (Halfhill, 1997). Since this interface technology is relatively new, empirical-founded guidelines and style guides are hardly available.
The current practice of Web-design is characterised by local solutions with - and this should be emphasised - hardly any systematic design or evaluation. Consequently, to date, little is known about the success and failure factors underlying the overall performance of users interacting with telematics systems, although much has been remarked about the poor state of Web application design. However, recent research shows that general usability principles can be derived from Human-Computer Interaction (HCI) research and that the application of such principles in various domains results in general support functions for user tasks. For example, a consistent header in the Web pages of a company site on the World Wide Web proved to provide insight in the Web-site content thereby optimising navigation. Furthermore, a menu consisting of a button matrix led to more explorative search than a "classic" menu list (Neerincx & Ruijsendaal, in preparation). The support functions for Web-site navigation and search were based on general principles for interface design and, consequently, could also be included in a different type of application: the user interfaces for controlling and supervising experiments in future space laboratories (Neerincx & Ruijsendaal, 1997). Next to attuning user interfaces to user actions like navigation and search, it has been observed that the interface should be adapted to user abilities. In particular "less abled" users may need extra support functions. For example, users with poor spatial ability have more problems with navigation in Web-sites, requiring extra search support (Chen & Rada, 1996).
In different research areas (HCI, language engineering, software engineering, and cognitive science) theories, methods and techniques have been developed that are of relevance for designing telematics systems. However, empirically founded design principles are scarce and a concise design method that integrates such principles into a coherent set of techniques from the different research areas is lacking.
One of the subgoals of this project is to further develop assessment techniques incorporating methods used in software and language engineering. Presently almost no theoretical framework to describe and evaluate strategies and styles is available. There is a wide range of specification formalisms in software engineering. Recent research on bridging the gap between software engineering and human-computer interaction has provided some general frameworks, but there are still many gaps, for instance for the design of multi-modal dialogues. There is a gap between human-computer interaction in general and human-computer interaction which involves speech and language: the restricted amount of theory that has been developed so far is even less if we look at speech and language as possible modalities of interaction; and no research has been conducted regarding different user characteristics.
Akker, R. op den, Doest H. ter, Moll M. & Nijholt, A. 1995). Parsing in dialogue systems using typed feature structures. In: Proceedings 3rd International Workshop on Parsing Technologies (IWPT'95), Prague, Karlovy Vary, 1995.
Andernach, T., Doest H. ter, Akker R. op den, Schaake J., Hoeven G.F. van der, Nijholt, A. & Burgt, S.P. van de (1995). Language analysis for dialogue management in a theatre information and booking system. Language Engineering '95, AI'95, 15th International Conference, Montpellier, June 1995, 351-362.
Andernach, T., Poel, M.& Salomons, E. (1997). Finding Classes of Dialogue Utterances with Kohonen Networks. In: Proceedings NLP workshop of the European Conference on Machine Learning (ECML), Prague, April 1997.
Attema, J., & van der Veer, G.C.(1992) Design of the currency exchange interface: Task action grammar used to check consistency. Zeitschrift für Psychology, 200, 121-134
Barfield, L., Boeve, E. and Pemberton, S. (1991). The Views User-Interface System; CHI 91, teaching through technology.
Berk, M. van den (1997). Bioscoop Reservering Systeem (Eerste Ontwerp). Rapport i.o.v. TeleCats B.V. Parlevink Onderzoeksgroep. December 1997.
Boeve, E., Barfield, L. Pemberton, S. (1993). WYSIWYG Editors: And What Now?, In L. Bass, J. Gornostaev, C. Unger, Human-Computer Interaction, Springer Verlag, 1993.
Boeve, E. Barfield, L. Pemberton, S. (1993). Objects, Invarients and Treacle: Animation in the Views System, Proc. HCI 93, Loughborough, UK
Bouwman, A.G.G. & Hulstijn, J. (1998). Dialogue Strategy (Re-)Design with Reliability Measures. Submitted for publication (LREC).
Braat, K. & Oortwijn, H.-J.(1997). Een generiek reserveringssysteem. Rapport i.o.v. VoiceRes (Coopers & Lybrand). Parlevink Onderzoeksgroep.
Cohen, A.M.; L.G.L.T. Meertens, and S. Pemberton (1996). Interactive Mathematical Books, Images of SMC Research.
Collis, B., Andernach, T. & Diepen, N. van (1997). Web environments for group-based project work in higher education. International Journal of Educational Telecommunications 3 (2/3), 109-130.
De Greef, H.P. & Neerincx, M.A. (1995). Cognitive support: designing aiding to supplement human knowledge, Int. J. Human-Computer Studies, 42, 531-571.
De Greef, H.P., Neerincx, M.A. & Hurts, C.M.M. (1993). Integrating human-computer interaction with software engineering. In: van der Veer G.C., White, T.N. & Arnold, A.G. (eds.), Interacting with computers: preparing for the nineties, pages 129-139, SIC.
De Vries, A.P., van der Veer, G.C., & Blanken, H.M. (1997) Let's talk about it: dialogues with multimedia databases - database support for human activity. CTIT Technical Report Series, 97-13, ISSN 1381-3635, 1-11
Doest, H. ter, Moll, M., Bos, R., Burgt, S.P. van de, & Nijholt, A.(1996). Language Engineering in Dialogue Systems. Computers in Engineering Symposium. Session on Natural Language in Human-Computer Interfaces. Volume I, Book VI, Houston, Texas, Februari 1996, 68-79.
Hoeven, G.F. v.d., Andernach, J.A., Burgt, S.P. v.d., Kruijff, G-J.M., Schaake, J., Nijholt, A.& Jong, F.M.G. de. (1995). SCHISMA: A Natural Language Accessible Theatre Information and Booking System. Proceedings First Workshop on Applications of Natural Language to Data Bases, Versailles, France, June 1995, 271-285.
Hulstijn, J., Steetskamp, R., Doest, H. ter, Burgt, S.P. van de & Nijholt, A. (1997). Dialogues in a theatre information and booking system. In: Memorias. V Symposio Internacional de Comunicación Social. Santiago de Cuba, Centro de Lingüística Aplicada, 1997, 87-99.
Hulstijn, J. (1997). Structured information states - raising and resolving issues. In: Proceedings of Mundial'97, Formal Semantics and Pragmatics of Dialogue. G. Jäger and A. Benz (eds.), University of Munich [Also: CTIT Technical Report series, No. 97-18, ISSN 1381 - 3625].
Hulstijn, J. (1997). Goals and Information States. In: Proceedings of Accolade97, Dutch Graduate School in Logic. Joeri Engelfried and Marco Hollenberg and Eva Hoogland and Esther Kraak and Renata Wasserman (eds), 1997, OTS, Utrecht University.
Instone, K., Pemberton, S. (1996) HCI issues of the World-Wide Web. Human Factors in Computing Systems Conference Companion, CHI 96 Common Ground.
Kerstholt, J.H. (1992). Information search and choice accuracy as a function of task complexity and task structure. Acta Psychologica, 80, 185-197
Kerstholt, J.H. (1994). The effect of time pressure on decision making behaviour in a dynamic task environment. Acta Psychologica, 86, 89-104.
Kerstholt, J.H. (1995). Decision making in a dynamic situation: The effect of false alarms and time pressure. Journal of Behavioral Decision Making, 8, 181-200.
Kerstholt, J.H., Passenier, P.O., Houttuin, K. and Schuffel, H. (1996). The effect of a priori probability and complexity on decision making in a supervisory control tasks. Human Factors, 38, 65-78.
Kerstholt, J.H. & Passenier, P.O. (1996). ATOMOS II: General guidelines for efficient supervisory control. TNO Report TM-96-B016, Soesterberg: TNO Human Factors Research Institute.
Kerstholt, J.H. (1996). The effect of information costs on strategy selection in dynamic tasks. Acta Psychologica, 94, 273-290.
Kerstholt, J.H. & Passenier, P.O. (1997). Design and evaluation of decision support for integrated ship operation. In S. Bagnare, E. Hollnagel, M. Mariani & L. Norros (Eds.). Time and space in process control. Proceedings CAPS '97. Roma: CNR.
Kruijff, G.-J.M.& Schaake, J. (1997). Discerning relevant information in discourses using TFA. In: Recent Advances in Natural Language Processing. Selected papers from RANLP'95. John Benjamins Publishing Company, Amsterdam/Philadelphia, 259-270.
Leenes, R., Schaake, J., Coelman, B., Roelofs, S. & Fehse, K. (1997). Systeemconcept. Overheidsloket Enschede 2000. Gemeente Enschede / Universiteit Twente, maart 1997.
Leenes, R. & Schaake, J. (1997). OLE2000: A study in questions and demand. Presented at the IFIP WG 8.5 ('Information Systems in Public Administration') workshop 'Empowering Citizens through IT -Innovation in Governmental and Municipal Information Systems', Stockholm, May 5-6, 1997. Also, Information Infrastructure and Policy. An international journal on the development, adoption, use and effects of information technology (to appear).
Lie, D.H., Hulstijn, J., Doest, H. ter & Nijholt, A. (1997). Language analysis in Schisma. In: International Workshop on Parsing Technologies 1997 (IWPT '97), MIT, Boston, USA, 243-244.
Loo, R. van, Roelofs, S., Rommes, E. & Schaake, J. (1997). Evaluatie Overheidsloket Enschede 2000: Gebruikersparticipatie. Gemeente Enschede / Universiteit Twente, december 1997.
Luperfoy, S., Nijholt, A. & Veldhuizen van Zanten, G. (eds.) (1996). Dialogue Management in Natural Language Systems. Proceedings Twente Workshop on Language Technology 11.
Meertens, L., Pemberton, S. The Ergonomics of Computer Interfaces - Designing a System for Human Use; CWI Report CS-R9258,
Neerincx, M.A. (1996). Mapping cognitive support to human needs. Proceedings XV European Annual Conf. Human Decision Making and Manual Control, pages 4.1-1--4.1-6, TNO-HFRI, Soesterberg, the Netherlands.
Neerincx, M.A. (1997). Employing and extending human knowledge and processing capacities. In: Harris (ed.), Engineering Psychology and Cognitive Ergonomics, Volume 1, Transportation Systems, Chapter 46, pages 413-419. Aldershot, etc.: Ashgate.
Neerincx, M.A. & de Greef, H.P. (1993). How to aid non-experts. In Ashlund, S., Mullet, K., Henderson, A., Hollnagel, E. & White, T., editors, Proceedings of INTERCHI'93, 165-171, New York, ACM.
Neerincx, M.A. & de Greef, H.P. (1998). Cognitive support: extending human knowledge and processing capacities. Human-Computer Interaction, Vol 13 (nr 1), pages 73-106.
Neerincx, M.A. & van Doorne, H. (1997). Cognitive Task Analysis for User Interface Development in Naval Ship Control Centres. TNO-report TM 1997 A-079. Soesterberg, The Netherlands: TNO Human Factors Research Institute.
Neerincx, M.A. & Griffioen, E. (1996). Cognitive task analysis: harmonizing tasks to human capacities. Ergonomics, 39(4), 543-561.
Neerincx, M.A. & Houttuin, K. (1996). Een gebruiksvriendelijke Web-site ontwerpen [Designing a user-friendly Web-site] (Report TM-96-C055 (in Dutch)). Soesterberg, The Netherlands: TNO Human Factors Research Institute.
Neerincx, M.A. & Ruijsendaal, M. (1997). Advanced Crew Terminal usability in the Software Development Process. Technical report, deliverable ESA/CONTRACT 11547/95/NL/JG, W.O. 11.
Neerincx, M.A. & Ruijsendaal, M. (in preparation). How to realize Web-site usability. TNO-report TM 1998 B-... Soesterberg, The Netherlands: TNO Human Factors Research Institute.
Nijholt, A. (1997). Human-computer interaction. Area description of the CTIT (Center for Telematics and Information Technology), July 1997.
Nijholt A. & Scollo, G. (eds.) (1998). Algebraic Methods in Language Processing. Special Issue of Theoretical Computer Science, Volume 197/1-2, May 1998.
Nijholt, A. & Hulstijn, J. (eds.) (to appear). Formal Semantics and Pragmatics of Dialogue. Proceedings Twendial'98 (TWLT13), Universty of Twente, The Netherlands, May 1998.
Pemberton, S. The Ergonomics of Software Porting. CWI Report CS-R9266,
Pemberton, S. (1992). Programming Aspects of an Open Architecture Environment; Proceedings Advanced Visual Interfaces, AVI 92, Rome.
Pemberton , S. (editor) (1997). Looking to the Future: Proceedings of CHI 97, Human Factors in Computing Systems, ACM.
Roelofs, S., Leenes, R., Schaake, V. & Coelman, B. (1997). Vraagpatronenonderzoek Overheidsloket Enschede 2000. Gemeente Enschede / Universiteit Twente, april 1997.
Van der Veer, G.C. (1994) Groupware task analysis: modeling complex reality. In: R. Oppermann, S. Bagnara, D. Benyon (eds), Human-computer interaction: from individuals to groups in work, Leisure, and Everyday Life. ISBN: 3-88457-233-4, ISSN: 0170-8120. GMD, Sankt Augustin, GMD-Studien, 233, 341-351
Van der Veer, G.C. (1994) Design methods for human-computer interfaces. In: K. Brunnstein, E. Raubold (eds), Applications and impacts - information processing "94. ISBN: 0-444-81987-8, ISSN: 0926-5473. North-Holland, Amsterdam, IFIP Transactions, A-52, 188-195
Van der Veer, G.C. & van Vliet , J.C. (1994) Team design for groupware - a view on structure. In: M.J. Tauber, R. Traunmueller, S. Kaplan (eds), Interdisciplinary Approaches to System Analysis and Design - conference proceedings. Inst. fuer Informatik, Joh. Kepler Universitaet, Linz
Van der Veer, G.C. (1995) User Interface design from the viewpoint of human learning. In: C. Zucchermaglio, S. Bagnara, S.U. Stucky (eds), Organizational Learning and Technological Change, ISBN 3-540-58917-1. Springer, Berlin, NATO ASI Series F, 141, 138-164
Van der Veer, G.C., van Vliet, J.C., & Lenting, B.F.(1995) Designing complex systems - a structured activity. In: G.M. Olson, S. Schuon (eds), DIS '95 Symposium on Designing Interactive Systems, ISBN 0-89791-673-5. ACM Press, New York, NY, 207-217
Van der Veer, G.C., Lenting, B.F., & Bergevoet, B.A.J. (1996) GTA: Groupware Task Analysis - Modeling Complexity. Acta Psychologica, 91, 297-322
Van der Veer, G.C., Hoeve, M., Lenting, B.F.(1996) Modeling complex worksysystems - method meets reality. In: T.R.G. Green, J.J. Canas, C.P Warren (eds), Cognition and the worksystem- 8th European Conference on Cognitive Ergonomics. EACE, INRIA, Le Chesnay cedex, 115-120
Van Setten, M., van der Veer, G.C. & Brinkkemper, S. (1997) Comparing interaction design techniques: a method for objective comparison to find the conceptual basis for interaction design. In: G.C. van der Veer, A. Henderson, S. Coles (eds), Designing interactive systems: processes, preactices, methods, and techniques. ACM, ISBN 0-89791-863-0, New York, NY, 349-357
Van der Veer, G.C., van Welie, M., & Thorborg, D. (1997) Modeling complex processes in GTA. In: S. Bagnara, E. Hollnagel, M.Mariani, L. Norros (eds), Time and space in process control. CNR, ISBN 88-85059-08-2, Roma, 87-93
Vertegaal, R., Velichkovsky, B., van der Veer, G.C.(1997) Catching the eye: Management of joint attention in cooperative work. SIGCHI Bulletin, 29(4)
Other references
Carroll, J.M. (1984). Minimalist design for active users. In B. Shackle (editor), Human-Computer Interaction, Interact'84, pages 219-225, Amsterdam: North-Holland.
Carroll, J.M. (editor) (1995). Scenario-based design: envisioning work and technology in system development. New York, etc.: Wiley & Sons Inc.
Chen, C. & Rada, R. (1996). Interacting with hypertext: a meta-analysis of experimental studies. Human-Computer Interaction, vol. 11, 125-156.
Cockburn, A. & Jones, S. (1996). Which way now? Analysing and easing inadequacies in WWW navigation. International Journal of Human-Computer Studies, vol. 45, 105-129.
Cjaza, S.J., Clark, M.C., Weber, R.A. & Nachbar, D. (1990). Computer communication among older adults. Proceedings of the Human Factors Society 34th Annual Meeting.
Fisk, A.D. & Rogers, W.A. (ed.) (1997). Handbook of human factors and the older adult. San Diego, etc.: Academic Press.
Halfhill, T.R. (1997). Good-Bye, GUI; Hello NUI: Network-centric user interfaces are coming to PCs as well as to network computers. Byte, July 1997, pp. 60-72
Lightner, N.J., Bose, I. & Salvendy, G. (1996). What is wrong with the World-Wide Web?: a diagnosis of some problems and prescription of some remedies. Ergonomics, Vol. 39, No.8.
McDonald, S. & Stevenson, R.J. (1996). Disorientation in hypertext: the effects of three text structures on navigation performance. Applied Ergonomics, Vol. 27, No. 1.
Najjar, L.J. (1995). A review of the fundamental effects of multimedia information presentation on learning. Technical Report GIT-GVU-95-20. Atlanta: Georgia Institute of Technology.
Nardi, B.A. (ed.) (1996). Context and consciousness: activity theory and human-computer interaction. Cambridge, Massachusetts: the MIT Press.
Nielsen, J. (1995). Multimedia and hypertext, the Internet en beyond. Cambridge, MA: Academic Press professional, ISBN 0-12-518408-5.
Preece, J., Rogers, Y., Sharp, H., Benyon, D., Holland, S. & Carey, T . (1994). Human-computer interaction. New York, NY: Addison-Wesley Publishing Company, ISBN 0-201-62769-8.
Silverman, B.G. (1991). Expert critics: operationalizing the judgement/decisionmaking literature as a theory of "bugs" and repair strategies, Knowledge Acquisition, 3, 175-214.
Smith, S.L. & Mosier, J.N. (1986). Guidelines for Designing User Interface Software, MITRE-report MTR10090, MITRE Corp., Bedford, MA.
Sutcliffe, A.G. (1995). Human-computer interface design. Second edition. London: MacMillan Press.
Thüring, M., Hannemann, J. & Haake, J.M. (1995). Hypermedia and cognition: designing for comprehension. Communications of the ACM, Vol. 38, No.8, 57-66.
Williges, R.C., Williges, B.H. & Elkerton, J. (1987). Software Interface Design. In: Handbook of Human Factors, Salvendy, G. (ed.), John Wiley & Sons, New York.
Zsambok, C.E. & Klein, G. (eds.) (1997). Naturalistic decision making. Mahwah, New Jersey: Lawrence Erlbaum Ass.
This project distinguishes two phases of one and a half year each. In the first phase, current applications and design solutions for Network User Interfaces (NUIs) are investigated to show the success and failure factors of "state of the art" designs and to derive general usability principles for future Web-based services. Due to the availability of the general cognitive engineering framework and the test environments, the Web-design and -evaluation context is already established allowing for a quick start in obtaining experimental results. In phase I, the companies involved will mainly serve as a sounding board to assure the tuning to practice and to establish a fast and direct knowledge transfer between the participating research institutes and companies. Phase I provides principles, techniques for assessment and specification, and tools for Web-based services for information and commerce. In phase II, these elements are integrated into a coherent design method. Further, the companies and research institutes will choose an application for this method that incorporates the success and failure factors for a demonstration and evaluation of the method. Thus, in phase II, companies have a major role in establishing the application domain and providing relevant domain knowledge.
Before specific design guidelines and principles can be provided, basic questions about the relation between human behaviour and characteristics of telematics systems and services need to be answered:
Data will be collected by studying the behaviour of various users who interact with prototypes of telematics systems (performing the tasks of monitoring, assessing and decision making). The goal of these experiments is to identify the interaction behaviour of users and the problems they meet when interacting with the systems. Especially factors that underlie the identified user behaviour and problems are of interest. The experimental findings will be used to deduce principles and guidelines for the design of Web-based services for commerce and information that incorporate the needs of specific groups of users (with special attention for elderly people). For this purpose, the project needs test environments that, in sum, comprise the diversity of potential users and tasks, i.e. professional and occasional users who want to acquire general information ('exploring'), buy a product or get specific information objects ('goal-directed search'). Developing new environments may cost a substantial amount of resources. Therefore, use will be made of prototypes that have been developed (or are under development) by the research institutes: a theatre information and booking service, a public counter for government information and a company Web-site. Taken together, these environments fulfil the general requirements of this project (however, some adaptations have to be made to meet the specific requirements of this project). Whereas, these three prototypes were developed for specific 'local' objectives---an architecture for natural language dialogues, development of a public counter and optimising a specific Web-site---this project uses them to identify general patterns in user behaviour in the context of Web-based services to derive general design principles. The prototype systems and their domains, as well as their specific value for the experiments will be described briefly below.
The first existing prototype system that will be used in the experiments is named SCHISMA (acronym of Dutch SCHouwburg Informatie Systeem): a dialogue system for theatre information and booking. This system has been developed to serve as a testbed to investigate possible architectures and techniques for building natural language dialogue systems. The SCHISMA project started as a joint research project of KPN Research and the Parlevink group of the University of Twente, and the Twentse Schouwburg provided the necessary information to fill the database. The reason for choosing a theatre information and booking service is twofold: [1] it involves some sort of transactions, as the user may reserve tickets for a particular performance, and [2] the domain is richer then the usual train- or airline information services, as it, for example, allows both goal-directed search and exploration (it also includes additional background information). Especially the innovative techniques used in the design of this system (e.g. natural language dialogues, speech interaction, Virtual Reality environment), added to the diversity of the potential users, make this prototype system to be of value for use in the experiments: The impact of the use of these new technologies on the behaviour and satisfaction of different types of users can be studied.
A significant portion of their target group does not consult many public services, which is partly due to fragmentation of service provision. The national program "Public Counter 2000" ('Overheidsloket 2000') aims at creating user-oriented public counters for municipal and government information and services, that make use of information technology. The public counters aimed at in this project, as well as the information and services they provide, should be organised according to the questions of potential users, and not according to the municipal organisation (as physical counters often are now). This can give rise to an integrated clustering of services and to the re-organisation of (virtual and even physical) counters. In the design of these public counters, the focus is on a user-oriented organisation of the information and the services (at least in the way it is presented to the user), according to 'demand patterns' (i.e., frequent or logical co- occurrences of questions). The University of Twente is involved in the project "Teleloket" which provides a contribution to the Public Counter pilot in the city of Enschede and in several research projects co-ordinated by the Executive Bureau of the Public Counter program.
With respect to the Enschede pilot the concept of 'user orientation' has been worked out towards an Internet application. A main research topic is to develop a user interface that will support the user to construct a question. This construction will be in stages. At each stage the user adds a piece or a phrase to the question. This 'question' can also be a description to the user's problem or situation.
The main value of this prototype system for the experiments is that it allows the examination of the effects (on user behaviour, problems and user satisfaction) of organising a multitude of information and services in a user-oriented way. The potential users of the system are all citizens, but the system is the more interesting because it is developed with the aim to even reach potential users that are inexperienced with computers and not very apt to use computers. In the experiments this group of users will be of special interest.
The TNO Human Factors Research Institute developed a small set of design principles for a company web-site. Application of these principles resulted in a first prototype Web-site with a coherent interface structure derived from envisioned use scenarios and support facilities for orientation, navigation and search. Subsequently, tools were developed to evaluate the usability of alternative Web-site designs with users performing several types of tasks. The results of a first experiment showed the advantages of some support functions (in particular the general interface structure, the menu layout, and the headers), but also some remaining problems for users (in particular choosing the optimal search strategy). The Web-site test-bed will be further developed and used for experiments in the present project.
The experiments provide therefore insight into user behaviour, in relation to both the task demands, how does user behaviour depend on the task required of the user, and user characteristics, of which age will be of primary importance. These results will provide input to the requirements for the user interface, particularly with regard to the support specific user groups will need.
A major advantage of electronic information systems is the possibility to use several presentation modes; i.e. information may either be presented as text, video, graphics etc. Multi-media present numerous opportunities to support users in dealing with their task. If, for example, it is concluded from the experiments that older people are less able to remember large amounts of textual information, it might be considered to provide the information through a video-film that closely corresponds to information already stored in memory. As it is foreseen that multi-modal dialogues provide large opportunities to support users in their search for information, particular attention will be paid to this topic.
In designing information systems it is not only necessary to have insight into the cognitive demands of the users, and potential solutions for support, but it is also necessary to know how to acquire this information and how to realise the appropriate design specification. The knowledge that is acquired from the experiments and the multi-modal specification techniques have a general character, but in order to apply this knowledge to a particular application, one needs to analyse the specific tasks required of the user for that application and one needs to know how to translate the requirements into a Web design. The present study will therefore also deal with tools and techniques that can be used for the design process, in particular task analysis techniques and tools for Web-design.
In general, the first phase will therefore address three topics: user behaviour (through experimental studies), techniques and tools for user-centred design and multi-modal specification techniques. Together, the results contain a set of principles, techniques and tools for user centred design, but the temporal dimension is still lacking, i.e., how to proceed from the initial task analyses to the final implementation. This question will be addressed in phase II.
Whereas in phase I the elements of the user-centred approach are established, in phase II the process of the user-centred design method is established. First, the dependencies between these elements, as they appear in the process, are determined, resulting in a description of the complete design method. Then, the design method is applied to develop new Web-based applications, to be chosen in careful consideration with the industrial partners. A detailed plan for phase II will be formulated during phase I. Currently it is not yet possible, first, because it depends on the results of phase I (i.e., the exact elements of the design approach) and, second, because a substantial involvement of the companies is foreseen that has to be worked out in phase I. The general structure is to acquire information requirements and user profiles by task analysis for the applications and specify the basic interface. Then extra support will be designed and integrated into this interface. The resulting prototype will be evaluated and redesigned. This design and test process will be guided by the guidelines formulated in the first part of the project. The user-centred design method will be useful for application designers, as well as for assessment of applications.
Be ahead of the description of workpackages in the following section; Figure 1 shows the general structure of the work approach en the dependencies between the activities.
Figure 1: General structure of work approach.
Figure 1 shows the general strucure of the workpackages and the dependencies. In phase I, design principles and techniques are developed. In phase II, the principles and techniques are integrated into the process of software development to develop new applications.
This workpackage provides results about how to attune the interface dialogue and support to the tasks and users. The objective is to establish principles, guidelines and criteria for the design and evaluation of Network User Interfaces (NUI's) and their support, based on general characteristics of users' interaction behaviour that appear in the context of such interfaces for telematics services. These principles, guidelines and criteria will be an integrated part of the design method to be established in phase II of this project
For the experiments, the three existing prototypes discussed above are used. Factors that will be investigated are:
For the evaluation of the effectiveness of design solutions for the users and their tasks, three measures are distinguished:
To explain and to derive predictions from the results, important aspects of user behaviour will be identified and related to these three measures (such as search strategies and task switching). Methods that will be used to register these aspects are video and audio recordings, interaction logging, questionnaires and the think-aloud method. The importance of each of the aspects mentioned above will vary for different types of tasks and different types of users. For a user conducting a goal directed search, for instance, efficiency will be more important than for someone engaged in exploration. For the exploring users, on the other hand, attractiveness of the interface is likely to be important. In this case it can be favourable that user and system take their time. This is of importance for information and commerce: because of its attractiveness the 'customer' keeps surfing and is persuaded to buy something or to make a reservation for a performance. Or in the domain of governmental information and service the users might encounter relevant information increasing the number of well-informed persons of a target group. An interface that is meant to be used by both these types of users will have to support both kinds of use.
The experiments for the three domains are performed in parallel, following the same time-line (to be concluded within one year):
Roles of participants:
Deliverables:
1. Report with a description of the experiments. TNO-HFRI is the main editor. Dates:
The experiments of WP 1.1 provide results about how to attune the interface dialogue and support to the tasks and users. In parallel, in WP 1.2 it will be investigated how to do the task analysis and synthesis for designing Web-based services, i.e., how to establish the tasks of (future) Web-users, and how to specify their implementation in a manner that is comprehensible for both system designers and engineers. Broadly speaking, the design cycle for web-based systems include two essential phases, which are iterated throughout a design project:
A. Analysis phase. This provides analytical information about the current functionality of a complex (work) situation. It includes analysis of existing situations, including the evaluation of prototype implementations or designs.
B. Design or Synthesis phase. In this phase, analytical information is transformed by the designer in a qualitative (i.e., non- or semi-formal) fashion into a design specification of user functionality of a system (or system image). Since this typically includes the filling in of higher-level analytical concepts from the analysis phase into a description of the user's functional communication with the system, this phase is best characterised as being synthesis-oriented.
The main objective of this work package is to provide guidelines for support of phase B, the synthesis of results from a task analysis phase into a description of a concrete system design. However, guidelines for both the analysis and synthesis phases will be integrated into a design method to be established in phase II of this project.
Ad A) Task Analysis
As an information acquisition technique, the existing Groupware Task Analysis (GTA) technique will be used. Although this technique provides a framework for the whole design cycle of a project, it is currently aimed mostly at support of the task analysis phase. Task analysis includes the following steps:
Although the GTA technique is still under development, its method for analysing and modelling current and future situations is well defined. However, GTA currently lacks a high-level specification language bridging the gap between the analysis phase and the implementation phase. Therefore, the emphasis within this project will be on the development of techniques to support the Design/Synthesis phase B. This is interfaced with GTA by means of step 2 and 3 in the above procedure.
Ad B) Task Synthesis Techniques
A technique is developed that enables web designers to implement a user centred design process in which the application of general interface design principles (e.g. the results of WP 1.1) can be handled.
The practice learns that there is still a gap between current information acquisition and task analysis techniques on the one hand and the technical design process on the other hand. This gap concerns the task synthesis, which is a crucial step in order to arrive at user centred (and thus task centred) interface design solutions. In this work package, a design specification language is developed that provides the missing link, i.e. the task synthesis, between task analysis and technical design. Furthermore, the task analysis (GTA) and synthesis methods will be interfaced for combined application. Our design specification language will be complementary to that of WP 1.3 in that it will concentrate on the functional level and on the more non-verbal aspects. The synthesis method provides the semantics and syntax to complete this important design step and is developed from a semiotic stance. This latter is a logical extension of task analysis techniques such as GTA, in which the situational nature of goal setting and actual behaviour is recognised explicitly, e.g. by focussing on the different roles that can be assumed by users.
In many IC systems the information conveyed (visually, orally) by the interface is the user's one and only window on the reality that lies beyond. Often it appears to be a very small window compared to the vast dimensions of the system it represents. The selective and distorting cognitive viewport often realises system images that convey the understanding of the designer, which is at best a partial understanding of the users and their tasks, and always a specific understanding of the user-machine system. The representation of the system and its functionality, however, should facilitate the tasks of the user, such as maintaining accurate knowledge about the state the work object is in, knowing what the machine is doing, finding specific information, or the operational interaction with the represented system. In order to achieve that, the user needs and situational factors regarding task execution are modelled by the task analysis technique GTA. However, a thorough analysis of a given task situation is only a first step towards the design of the system image. If the knowledge of the task situation is not communicated to users in an integral and coherent fashion, usability problems will remain. Therefore, a design specification method is developed that offers designers the semantics and syntax to perform the task synthesis such that task related functions and situational factors are reflected. This method, as well as the practical means by which the design process can be supported is investigated in a qualitative study of the web design process. Design requires a basic structure of elements to be modelled (tasks, people, objects, roles, events) and their relations. Many different representations can be developed for different types of expertise in design (human factors, engineering, graphical design, evaluation), for different phases in the design process, e.g. workflow representations, task hierarchies, role descriptions, specifications of functionality and dialogues (e.g. sketches, video clips, audio tracks, formal representations). The different representations need to relate to the basic repository of design entities, which asks for the means and tools to describe and communicate the shared design space.
Roles of participants:
Deliverables:
While WP1.2 involves general techniques and tools for the design of Web environments, WP1.3 focuses on the details of specific aspects of interaction in Web-based information and transaction environments, i.e. the effects of different interaction styles, and possible schemes for allocating and integrating modalities in multi-modal dialogues. The expressive power of existing specification formalisms to support description and evaluation of these aspects are studied. Such formalisms may include formalisms from software engineering (e.g. logical specification formalisms like Z and CSP) and human-computer interaction (dialogue specification languages like DDL/SDL, HDDL, 3DT, task specification formalisms like GOMS and task flow modeling, and also cognitive simulation languages like DenK and SOAR). A new formalism is developed by extending and combining concepts of existing ones to fit the special needs of the project, in particular speech, language, and multi-modality.
The work plan for WP1.3 is as follows:
Roles of participants:
Deliverables:
Report by CTIT-CS with an overview of existing specification techniques studied and a description of a new technique (adapted from existing ones) that is suitable to describe multi-modal dialogues. The first draft of this report, to be reviewed by the knowledge institutes, will be ready on 1 June 1999. The report will be finished on 1 July 1999.
The results of all work packages in phase I will be integrated into an index of usability principles, techniques and tools from two perspectives.
From a design perspective, detailed guidelines are formulated for all phases in software development, interrelating the guidelines with user activities (user goals, use scenarios and use contexts), user abilities (such as spatial ability) and the corresponding adaptation functions that the user interface should provide. "State of the art" techniques and tools for the specification and assessement of Web-environments will be described. The resulting document will provide guidance for user interface designers of web-based services.
From a requirements perspective, a coherent set of general usability principles will be formulated that apply to Web-based services. These principles can be used to derive requirements specifications for specific development projects and to assess the (interim-)products. The resulting document will provide guidance for project managers showing the usability aspects that should be addressed in the development process.
Interim products of the work package will be assessed. In a workshop the first results will be discussed and possible applications for phase II will be explored. Taking this discussion into account, concept reports are drafted and forwarded for review to experts in the research field. In co-operation with the companies, a detailed work plan for phase II will be produced.
Roles of participants:
Deliverables:
1. Report with an index of the usability guidelines, user activities, user abilities, interface adaptation functions, and techniques and tools for the specification and assessment of Web-environments (focussing on the design process). TNO-HFRI is the editor. Dates:
Next to these TI-deliverables, the research institutes aim at a paper presentation at an international conference (e.g. CHI) and/or a publication in an international journal.
The process of the user-centred design method is established. For this, the dependencies between the design principles, techniques and tools, as they appear in the process, are determined. Thus, it is formulated how to move from one design stage to the other, to move from one technique/tool to the other and how to make use of the design principles in this process. The result will be a description of the complete, coherent design method.
The task analysis and task synthesis methods developed in WP 1.2, as integrated in WP 1.4, are applied to the design of services that will be selected in co-operation with the industrial partners. In this phase the method is finalisten and the process will be described so that it can be implemented in WP 2.5. The result of this WP is a specification of the task interface. The result functions as input for WP 2.3.
This work package consists of the application of principles and tools for the design of general support functions for communication, search, orientation and navigation to be incorporated in the example application. For specific tasks supportive agents may be developed (e.g. a 'front-office' co-worker function) and for specific users extra dialogue support (e.g. facial displays for elderly). For this part use will be made of the index of WP 1.4.
Subsequently, a prototype information service will be developed, according to the design method. For this part, use will be made of the task analysis, suggested guidelines and support facilities from previous workpackages. In particular the interface and support specification of WP 2.2 is input for this second part of WP 2.3.
Scenario-based evaluation techniques will be employed to evaluate the Web-prototype, where both experts, potential users, and clients may be needed to participate.
The handbook will describe