%0 Journal Article
%T A Comparison of Field-Based and Lab-Based Experiments to Evaluate User Experience of Personalised Mobile Devices
%A Xu Sun
%A Andrew May
%J Advances in Human-Computer Interaction
%D 2013
%I Hindawi Publishing Corporation
%R 10.1155/2013/619767
%X There is a growing debate in the literature regarding the tradeoffs between lab and field evaluation of mobile devices. This paper presents a comparison of field-based and lab-based experiments to evaluate user experience of personalised mobile devices at large sports events. A lab experiment is recommended when the testing focus is on the user interface and application-oriented usability related issues. However, the results suggest that a field experiment is more suitable for investigating a wider range of factors affecting the overall acceptability of the designed mobile service. Such factors include the system function and effects of actual usage contexts aspects. Where open and relaxed communication is important (e.g., where participant groups are naturally reticent to communicate), this is more readily promoted by the use of a field study. 1. Introduction Usability analysis of systems involving stationary computers has grown to be an established discipline within human-computer interaction. Established concepts, methodologies, and approaches in HCI are being challenged by the increasing focus on mobile applications. Real-world ethnographic studies have received relatively little attention within the HCI literature, and little specific effort has been spent on delivering solid design methodologies for mobile applications [1]. Researchers and practitioners have been encouraged to investigate further the criteria, methods, and data collection techniques for usability evaluation of mobile applications [2]. Lab-based experiments and field-based experiments are the methods most discussed in relation to evaluating a mobile application [2¨C4]. There has been considerable debate over whether interactions with mobile systems should be investigated in the field or in the more traditional laboratory environment. There seems to be an implicit assumption that the usability of a mobile application can only be properly evaluated in the field, for example, Kjeldskov and Stage [5]. Some argue that it is important that mobile applications are tested in realistic settings, since testing in a conventional usability lab is unlikely to find all problems that would occur in real mobile usage (e.g., [2, 6, 7]). For example, Christensen et al. [6] presented a study of how ethnographic fieldwork can be used to study childrenĄ¯s mobility patterns via mobile phones. Authors consider that field studies make it possible to carry out analysis that can broaden and deepen understanding of peoplesĄ¯ everyday life. However, some authors have highlighted how ethnographic field
%U http://www.hindawi.com/journals/ahci/2013/619767/