Department of Applied Computing, University of Dundee, Dundee DD1 4HN
The design and development of any product should be carried out with the end-users in mind, in particular in software development, users should be involved throughout. Standards for this process are specified in ISO 13407. However, determining the users' requirements and their qualitative satisfaction with the developing product are two major problems which arise, often due to the different backgrounds and perspectives of the different stakeholders, including the software engineer, the domain expert and the end-user.
Computer-based interviewing has been found to a valuable tool in eliciting information, which can also enhance interpersonal communication afterwards. When used in applications of complex information exchange, a computer-based interview can enable a user to explore ideas, and pass on information that they did not know they knew.
The proposed research will build on the applicants' previous investigations into the computer simulation of interpersonal communication. The project will identify good practice techniques from usability engineering and computer-based interviewing, and will simulate these in computer software. The package developed will be evaluated with industrial human factors experts, and a usability engineering process which includes the use of this package will be developed.
This will facilitate the gathering of user requirements and conducting user evaluations.
The Department of Applied Computing at the University of Dundee achieved a rating of 5A in the 1996 Research Assessment Exercise. It has an international reputation in human-computer interaction (HCI) studies, the use of computers in rehabilitation and therapy, health informatics research and medical image analysis techniques. It contains one of the three largest research groups in the world investigating the application of computer systems for the disabled. The multi-disciplinary staff collaborate closely with many local, national and international partners. In addition, the department has strong links with industrial software engineers, including NCR and the European Space Agency. These links have led to many systems being licensed to the commercial sector for exploitation and to products being sold world-wide.
The Department offers undergraduate and postgraduate degrees in Applied Computing in a unique programme where the learning of HCI and usability engineering techniques is integral throughout the courses.
Dr Ramanee Peiris completed her PhD titled Computer Interviews: Enhancing their Effectiveness by Simulating Interpersonal Techniques in 1997, funded by a University of Dundee scholarship. She is a lecturer in the Department of Applied Computing and has published 12 journal and conference papers. Current and completed projects under her supervision include using computer interview techniques to enhance conversation prediction (Nuffield Undergraduate Research Bursary) and the use of computer-based interventions in alcohol misuse and rape crisis counselling situations (University of Dundee Research Initiative funding). She graduated from the University of York with a Mathematics degree, and has five years industrial experience as a software engineer.
Dr Peter Gregor has worked in the field of computer-based interviewing for over 10 years. He pioneered the work on developing computer interviews which were based on an analysis of the processes and techniques which expert interviewers adopt (Gregor, 1991). His software has been applied to social work, educational psychology and forensic psychiatry settings, and has been marketed by a commercial company as ChatterBoxä , for use in school guidance work. He lectures in usability engineering at undergraduate and postgraduate level. He also runs external training courses for industrial clients, and received the 1999 NCR Training Award for this work. He currently holds grants to the value of £370,000 and previous research funding of the order of £500,000. He has extensive experience of research project supervision over a period of 10 years and has over 40 research publications.
Professor Alan Newell is Head of the Department of Applied Computing at the University of Dundee, and established its forerunner, the MicroCentre research group, in 1984. His main research interest is the development of computer systems in areas of high social impact, and he has been researching into computer systems to assist people with disabilities for nearly thirty years. He developed the concept of the parallel between "ordinary" people operating in "extra-ordinary" environments (such as stress) and "extra-ordinary" people (such as those with disabilities) operating in "ordinary" environments. He has published over 200 papers, including 80 journal papers, and 16 book chapters, and holds 12 patents.
The applicants are members of the Computer Interviewing Research Group (CIRG) which includes Applied Computing staff, a research nurse and a Professor of Social Work. The group has been investigating computer based interviews and interventions in a wide variety of settings. These include taking a behavioural history from parents of children attending a child psychiatry clinic (Morton and Alm, 1990), anger assessment of forensic psychiatric patients (Buchanan et al., 1994), a simulation of a confrontational situation for adolescents (Gregor and Newell, 1994), assisting engagement in interviews between children and social workers (Connolly, 1995) and empowering people who have suffered disabling trauma (Williams et al., 1999). The CIRG will act as a steering group for this project.
There is a consistent body of evidence that in certain situations when people interact with computers, an unexpectedly high degree of engagement takes place. This occurs particularly in situations in which the computer is in some sense a conversation partner. In addition, research over the past three decades has demonstrated that computer-conducted interviews are often more effective than interviews carried out by skilled professionals (Slack et al., 1966, Dove et al., 1977, Millstein, 1987 and Locke et al., 1992). It is particularly interesting to note that these observations hold true even when the interviewee knows that a transcript of the interview will be discussed later with a human interviewer.
The processes and techniques involved in interpersonal interviewing provide a model for purposeful interaction between human beings where there is a specific agenda. This agenda may be highly focused, such as an employment interview. Or, it may have a looser focus, such as a counselling session, enabling the interviewee to communicate complex information and feelings which the interviewer doesn't know about, and which the interviewee may be unaware of at the start of the interaction.
The proposed project draws on previous work involving the use of computers to interview people which has identified qualities which are particular to the human-computer interview situation, and differ from those observed in interpersonal interaction. These include:
Research by Peiris et al. (1995) has analysed the techniques and processes adopted by expert human interviewers, and simulated these techniques and processes in software. New prototype software using fuzzy logic principles is able to simulate these techniques, including aspects of more complex interviewing situations where the interviewer aims to explore ideas with the interviewee with a view to constructing a network of interconnecting ideas. This approach includes interviews where the client is encouraged to tell their story in an exploratory way which leads to insights on the part of the client about their own situation (Peiris, 1997).
The research has developed a computer-based system which helps the user to externalise their knowledge, thoughts and feelings through a process of building a network of interconnecting ideas. The system engine contains an encapsulation of various interviewing techniques, and starts with a (designer supplied) weighted index of topic areas. It also contains specific questions and optionally, an agenda of topics/questions which must be covered during the course of any given interaction. As the interaction proceeds, the system uses its interviewing expertise to build a picture based on what the user requires and the information which the user is giving.
The interaction is currently text-based and works through the system asking questions, responses to which are logged and analysed, using simple pattern matching techniques. The program's analysis of the user's responses alters the weighting of components within the particular interaction design and response set. These weightings dynamically determine the direction in which the interaction will flow. This is achieved through the use of fuzzy logic techniques to alter the weighting of topic domains so that the interaction flows naturally from the user's point of view and elicits the required information from the designer's point of view.
There are many situations of complex information exchange which these techniques can be applied to, in particular those for which the interviewee does not have clear thoughts and ideas beforehand. This proposal addresses the situation of human-centred software design and seeks to produce tools for usability engineering which utilise the computer-based interviewing techniques described above.
The design and development of any product should be carried out with the end users in mind. In particular, in software development, users should be involved throughout. ISO 13407 specifies standards for this process. The users who should be involved include the people requesting the product, and those who will ultimately use it. Working with users gives the developer great insight into what is truly required, and also lets the developer see how the users actually work.
In practice, however, this can be difficult. Typically the user is less technically minded than the developer, and both have their own area of expertise and vocabulary of jargon. At an initial face to face meeting, the user may not know what they actually want from a new product or system, nor be aware of how they operate at the moment. They may also be unaware of the potential of a new system. The user may feel inhibited or intimidated as they do not have the same level of technical knowledge as the developer, and may feel uneasy and reticent, as they do not wish to appear foolish or naïve. The developer may experience unease until they become aware of the level of technical knowledge the user has, and also they field the user is working in. Also, the developer may not be trained in interviewing, so may not extract the information required (Isaacs, 1997), and may be uneasy about appearing ignorant of the user's area of expertise.
These communication difficulties can lead to an incomplete set of requirements being gathered, and subsequent prototypes and working systems may be based on an inaccurate starting point.
There are established formal methods for software engineering and usability engineering, such as MUSE (Lim and Long, 1994). These methods use a variety of techniques and tools to enhance the requirements gathering phase, such as visualisation to enhance interviewing (Fulton, 1994) and message sequence charts (Holzman et al., 1997). However, few of these empower the user, nor do they give the user the chance to brainstorm or prepare without the developer being present.
This project will develop computer-based interviewing software to conduct requirements gathering interviews with users. This will allow the user to spend time thinking about their requirements, without the pressure of the developer's presence, and the interview techniques used should elicit thoughts and ideas which they were not aware of. A subsequent face to face meeting between developer and user will use the interview transcript, or a summary of it, as a starting point, meaning both are better prepared.
A further phase of usability engineering is the evaluation of prototypes and software, and there are many techniques for carrying out evaluations. One useful technique is to ask what the user thought of a prototype. This can range from a formal, quantitative paper questionnaire, to an informal chat. Both have merits and drawbacks, with regards to the quality and quantity of information imparted. A more serious issue is that of social politeness, where the user does not want to upset or offend the developer, and will respond favourably to the prototype, irrespective of what they really feel (Preece, 1993).
One solution has been to deliver questionnaires using a computer, though care is needed to use a different computer to the one being evaluated, as there is the risk of users still answering politely if asked questions by the same computer (Nass et al., 1994).
This project will also develop computer-based interviews to run evaluations incorporating a formal, quantitative, multiple-choice section, and an informal, user-driven, section.
The final product will be a set of tools for use in usability engineering (UsE-IT: Usability Engineering Interview Tools), and a human-centred development process which includes the user of these tools.
The product will be developed using human-centred design processes and practices (Gould, 1988). In particular, the project will involve potential end-users (the user team) and software engineers (the usability team). The teams will be recruited from local companies - the user team will include people who have expertise in product specification, in a sense they are experts at being users; the software engineers in the usability team will have user-centred design experience. The teams will act as advisers to the project and will provide formative feedback and evaluative data as the project progresses.
The project team will base project progression and management on the Hix and Hartson (1993) model for ensuring usability. In this way, the project will progress in an iterative manner, with evaluation of the various stages at the centre of the management process. As the project involves significant software development, adopting these processes will have a beneficial effect on the usability of the final project. Software development will be based on evolutionary prototyping using a Rapid Application Development system, which is an efficient method of incorporating evaluative findings into an evolving software package.
Requirements for the UsE-IT package will be gathered from the user team and the usability team. That is, both requirements for the design of the software and, from the usability team only, requirements for the content of an effective requirements gathering interview and an effective evaluation interview. A series of prototypes will be developed and evaluated by both the user team and the usability team. Methods will include interviews, domain expert heuristic evaluation of prototypes, questionnaires and focus groups.
The aim of final evaluations is to establish whether the UsE-IT package for requirements gathering and user evaluations is effective. It also aims to establish whether UsE-IT does indeed elicit deeper and more accurate requirements, and conduct better and more thorough evaluations than traditional methods.
The final evaluations will involve members of the user team each specifying two products which members of the usability team will prototype. The development of one product will use the UsE-IT tools, whilst the other will use a conventional approach. The requirements report for each product will be assessed by independent software engineer judges and domain experts for the following: depth of understanding of the problem, number of important details, accuracy of findings and adequacy for proceeding to the design stage - i.e. completeness and suitability.
The usability teams will then produce prototypes and conduct evaluations either using UsE-IT or conventional methods. Again, the resulting reports will be compared, and the judges will look for completeness, and will assess whether the UsE-IT evaluations gathered more accurate, and honest feedback.
With the increasing growth of the software market and the growing emphasis on the usability of software products, there is a need to find better methods for eliciting user requirements and conducting user evaluations accurately and comprehensively. The traditional approaches are both time consuming, over-reliant on the expertise of individuals and do not provide the best environment for the end user. The proposed UsE-IT package will provide a reliable, economical and comprehensive mechanism for the in-depth elicitation of requirements and evaluation feedback from users.
The application of computer-based interviewing techniques which aim to simulate human interviewing skills to the field of usability engineering is an innovative development which has the potential to revolutionise the early stages of system design.
The project objectives will be addressed by the work packages below:
Work Package 1: Requirements Gathering
Work Package 2: Initial Prototype Development
Work Package 3: Second Prototype Development
Work Package 4: Main Product Development
Work Package 5: Final Evaluations
Work Package 6: Project Write-up
One postgraduate research assistant will be employed for 18 person-months to carry out the research in this proposal.
The applicants will act as the project supervisors, and will hold weekly project meetings with the RA. The Computer Interviewing Research Group will act as a steering group for the project, and will meet with the RA at three monthly intervals. An initial meeting will be held with the user team and the usability team to brief them on the project, and others will be held at appropriate stages of the project. A final meeting will take place with both teams and the Computer Interviewing Research Group.
The research will result in a new human-centred approach to requirements analysis and user evaluation in software engineering. This will help to overcome the universally recognised problems of eliciting genuine, valid and useful requirements data from potential users of new software and hardware systems. Designers and software engineers will thus be provided at an early stage in their development cycle, with higher quality information from potential users. This will include a better likelihood of eliciting knowledge which users do not realise they have. Designers will also be able to gain more accurate information from evaluation interview sessions.
This research aims to extrapolate the understanding of the human computer interaction process during computer based interviews to produce an optimal environment for other complex information exchange situations, such as requirements analysis. The UsE-IT package and the evaluations to be carried out will enable the researchers to study human-computer interaction in a unique way, by focusing on the actual interaction, to develop and evaluate a facilitative environment for the communication of complex information from users to designers.
This work will be presented at national and international HCI conferences, such as HCI or Interact. The final results will be published in an academic journal such as the International Journal for Human-Computer Studies or Human-Computer Interaction.
Commercial exploitation of the product will be discussed with the University's Research and Innovation Service. It is anticipated that an early version may be made available via shareware, and that feedback from those users may influence the final development.
Staff: One postgraduate RA with software development experience and training in human-centred design will be employed to carry out this work.
Equipment: Development hardware and associated software is requested to carry out the work described, including a desktop PC for the RA, and a laptop for prototyping and evaluation purposes.
Travel: Funding is requested for travel to meet usability experts and end users, and also for attendance at two conferences.
Consumables: Subject fees have been requested to cover travel costs for the members of the user and usability teams.
Host Organisation: The host organisation will provide accommodation for the project, plus secretarial and technical support, including access to computer networks, and departmental hardware resources.
British Standard ISO 13407. Human-centred design processes for interactive systems.
Buchanan, I., Renwick, S., Duncan, J., Rooney, H., Harrison, D., Gregor, P. and Alm, N. (1994). An evaluation of the applicability of computer-based interviewing in the assessment of forensic psychiatric patients Papers presented at a Seminar on Computer-Aided Interviewing, University of Dundee, 14-15.
Connolly, P. (1995).The artistic appliance of micro-science. Professional Social Work March 1995, 8.
Dove, G.A.W., Wigg, P., Clarke, J.H.C., Constantinidou, M., Royappa, B.A., Evans, C.R., Milne, J., Goss, C., Gordon, M. and de Wardener, H.E. (1977). The therapeutic effect of taking a patient's history by computer. Journal of the Royal College of General Practitioners vol. 27 477-481.
Fulton, D. (1994). The Use of Visual Indexing as an Interview Support Technique. In Cockton, G., Draper, S.W. and Weir, G.R.S. (Eds.) People and Computers IX Proceedings of the HCI '94 Conference 37-51.
Gould, J.D. (1988). How to design usable systems. In Helander (Ed), Handbook of Human Computer Interaction, North-Holland, Amsterdam.
Gregor, P. (1991). Computer-based interviewing: steps towards the development of a generic system for serious and sensitive information. Unpublished PhD Thesis, University of Dundee.
Gregor, P. and Newell, A.F. (1994) Can computer aid the process of interviewing young people? British Journal of Educational Technology vol. 25 228-230.
Hix, D. and Hartson, H.R. (1993). Developing user interfaces, pub. Wiley, New York.
Holzmann, G.J., Peled, D.A. and Redberg, M.H. (1997). Design tools for requirements engineering. Bell Labs Technical Journal, vol. 2 86-95.
Isaacs, E.A. (1997). Interviewing Customers: Discovering what they can't tell you. In Looking to the Future Proceedings of CHI '97 vol. 2 180-181.
Lim, K.Y. and Long, J. (1994). The MUSE Method for Usability Engineering, pub. Cambridge University Press.
Locke, S.E., Kowaloff, H.B., Hoff, R.G., Safran, C., Popovsky, M.A., Cotton, D.J., Kinkelstein, D.M., Page, P.L. and Slack, W.V. (1992). Computer-Based Interview for Screening Blood Donors for Risk of HIV Transmission. Journal of the American Medical Association vol. 268 1301-1305.
Millstein, S.G. (1987). Acceptability and reliability of sensitive information collected via computer interview. Educational and Psychological Measurement vol. 47 523-533.
Morton, H.G. and Alm, N. (1990). Computer-aided interviewing of parents in a child psychiatric clinic. Journal of Microcomputer Applications vol. 13 273-280.
Nass, C., Steuer, J. and Tauber, E.R. (1994). Computers are social actors. In Celebrating Independence Proceedings of CHI '94 72-78.
Peiris, D.R., Alm, N. and Gregor, P. (1995). Computer interviewing: an initial investigation using free text responses. In Kirby, M.A.R., Dix, A.J. and Finlay, J.E. (Eds.) People and Computers X Proceedings of the HCI '95 Conference 281-288.
Peiris, D.R. (1997). Computer Interviews: Enhancing their Effectiveness by Simulating Interpersonal Techniques. Unpublished PhD Thesis, University of Dundee.
Preece, J. (Ed.) (1993). A guide to usability - human factors in computing, pub. Addison-Wesley, Wokingham.
Slack, W.V., Hicks, G.P., Reed, C.E. and van Cura, L.J. (1966). A computer-based medical-history system. New England Journal of Medicine vol. 274 194-198.
Williams, B., Peiris, D.R., Gregor, P., Alm, N., Cumming, S., Flockhart, G. and Groundwater, M. (1999). Computer-based interventions for assisting people who have suffered disabling trauma. New Technology in the Human Services vol. 12 69-75.
Diagrammatic Workplan