A Qualitative Review of Empirical Mobile Usability Studies
Constantinos Coursaris
Sign up for access to the world's latest research
checkGet notified about relevant papers
checkSave papers to use in your research
checkJoin the discussion with peers
checkTrack your impact
Abstract
The turn of this century marked an increased focus on mobile usability studies for research in the field of Human Computer Interaction. Such studies offer practitioners the needed insight to deliver usable mobile products and services adopted by consumers at increasing rates contributing to a $20 billion industry. Scholars also benefit by identifying new questions that need to be addressed, thereby enriching our understanding of this dynamic domain within HCI. A challenge for both of these groups exists in that many scholars define and operationalize usability differently. This paper presents a roadmap for future usability research that consists of two parts. First, a framework is adapted for the taxonomy of empirical mobile usability studies. Second, results of the qualitative review of 45 empirical mobile usability studies include: i) the contextual factors studied; ii) the core and peripheral usability dimensions measured; and iii) key findings. Expected contributions of the completed research are also outlined.
Key takeaways
- Therefore, usability is a more important issue for mobile technology than for other areas, since many mobile applications remain difficult to use, lack flexibility and robustness.
- Based on the literature review, a qualitative review framework for empirical mobile usability studies is presented next.
- These four variables (i.e. user, task, environment, technology) will be used for the presentation of the review of previous empirical research that relates to the usability assessment of mobile applications and/or mobile devices.
- This qualitative review began with the search for empirical mobile usability studies literature.
- Hence, attitude (as defined in these usability studies) may be collapsed in the single measure of satisfaction.
Related papers
A Meta-Analytical Review of Empirical Mobile Usability Studies
Journal of Usability Studies, 2011
In this paper we present an adapted usability evaluation framework to the context of a mobile computing environment. Using this framework, we conducted a qualitative meta-analytical review of more than 100 empirical mobile usability studies. The results of the qualitative review include (a) the contextual factors studied; (b) the core and peripheral usability dimensions measured; and (c) key findings in the form of a research agenda for future mobile usability research, including open and unstructured tasks are underutilized, interaction effects between interactivity and complexity warrant further investigation, increasing research on accessibility may improve the usability of products and services for often overlooked audiences, studying novel technology and environmental factors will deepen contextual mobile usability knowledge, understanding which hedonic factors impact the aesthetic appeal of a mobile device or service and in turn usability, and a high potential for neuroscience research in mobile usability. Numerous additional findings and takeaways for practitioners are also discussed.
Usability of mobile applications: literature review and rationale for a new usability model
Journal of Interaction Science, 2013
The usefulness of mobile devices has increased greatly in recent years allowing users to perform more tasks in a mobile context. This increase in usefulness has come at the expense of the usability of these devices in some contexts. We conducted a small review of mobile usability models and found that usability is usually measured in terms of three attributes; effectiveness, efficiency and satisfaction. Other attributes, such as cognitive load, tend to be overlooked in the usability models that are most prominent despite their likely impact on the success or failure of an application. To remedy this we introduces the PACMAD (People At the Centre of Mobile Application Development) usability model which was designed to address the limitations of existing usability models when applied to mobile devices. PACMAD brings together significant attributes from different usability models in order to create a more comprehensive model. None of the attributes that it includes are new, but the existing prominent usability models ignore one or more of them. This could lead to an incomplete usability evaluation. We performed a literature search to compile a collection of studies that evaluate mobile applications and then evaluated the studies using our model.
Conceptual framework and models for identifying and organizing usability impact factors of mobile phones
2006
Usability has been regarded as a critical factor affecting the quality of mobile phones. Many studies have examined usability impact factors of mobile phones on the basis of software usability concepts. However, considering mobile phones as multi-media and information appliances, a new usability concept and associated factors should be developed. This paper proposes a conceptual framework which has five views to reflect different aspect of interactions between users and mobile phones, and from which various usability impact factor models can be derived. Five views include user view, product view, interaction view, dynamic view, and execution view. Furthermore, we developed a hierarchical model which organizes usability factors in terms of goalmeans relations. Through two case studies, we could verify the usefulness of the framework and model. Lastly, we developed a set of checklists that are helpful to measure the usability of mobile phones, thereby increasing the practicality of the framework and model.
Usability evaluation of mobile applications; where do we stand
The range and availability of mobile applications is expanding rapidly. With the increased processing power available on portable devices, developers are increasing the range of services by embracing smartphones in their extensive and diverse practices. While usability testing and evaluations of mobile applications have not yet touched the accuracy level of other web based applications. The existing usability models do not adequately capture the complexities of interacting with applications on a mobile platform. Therefore, this study aims to presents review on existing usability models for mobile applications. These models are in their infancy but with time and more research they may eventually be adopted. Moreover, different categories of mobile apps (medical, entertainment, education) possess different functional and non-functional requirements thus customized models are required for diverse mobile applications.
Mobile Application Usability: Conceptualization and Instrument Development
MIS Quarterly, 2015
This paper presents a mobile application usability conceptualization and survey instrument following the 10step procedure recommended by MacKenzie et al. (2011). Specifically, we adapted Apple's user experience guidelines to develop our conceptualization of mobile application usability that we then developed into 19 first-order constructs that formed 6 second-order constructs. To achieve our objective, we collected 4 datasets: content validity (n=318), pre-test (n=440), validation (n=408), and cross-validation (n=412). The nomological validity of this instrument was established by examining its impact on two outcomes: continued intention to use and mobile application loyalty. We found that the constructs that represented our mobile application usability conceptualization were good predictors of both outcomes and compared favorably to an existing instrument based on Microsoft's usability guidelines. In addition to being an exemplar of the recent procedure of MacKenzie et al. to validate an instrument, this work provides a rich conceptualization of an instrument for mobile application usability that can serve as a springboard for future work to understand the impacts of mobile application usability and can be used as a guide to design effective mobile applications.
On measuring usability of mobile applications
2008
ABSTRACT In this paper we discuss challenges of usability evaluation of mobile applications. We outline some key aspects of mobile applications and the special characteristics of their usability evaluation that have recently lead to the laboratory vs field discussion. Then we review the current trend and practices. We provide an example of a usability evaluation study from our own background. We conclude with discussion of some open issues of usability evaluation of mobile applications.
Usability issues in mobile-wireless information systems
Issues in Informing Science and Information …, 2009
Usability is one of the characteristics that compose the multi-dimension concept of information systems' quality. Technology improvements enable the building of information systems to be used "any place, any time", through mobile handheld devices and wireless networks. Usability is threatened by the use of mobile-wireless information systems, especially because of the use of small devices with tiny screens, the difficulty to operate and the use during mobility. Therefore, there is a need to measure the usability characteristic, in order to define the level of quality of such information systems. This paper introduces the definitions of usability, and presents the definition of new metrics, defined in order to measure the quality level of usability of mobilewireless information systems. These metrics were empirically validated using four different experiments, based on handheld devices.
Mobile Usability Report
Göttingen: BusinessVillage, 2001
Concerning the classical, stationary internet it is by now widely accepted that factors like relevance, speed and reliability are not the only variables that determine a long-term customer-binding and a general success in the e-business. Usability and its relevance for successful e-business got more and more recognized. With this study we transferred the usability-question to the mobile internet. What factors determine the acceptance of this medium? Is it comparable to the stationary internet? Our aim was to provide a style guide full of recommendations especially for managers and developers who are engaged in the development of mobile data services and, moreover, to demonstrate in particular the importance of usability as a determining factor of success also in the mobile internet. Based on this motivation, an extensive usability test with 36 representative users was carried out. We included four WAP (Wireless Application Protocol) portals of leading German mobile operators as well as 23 selected WAP services which were intensively investigated. The usability testing was based on three WAP-enabled mobile phones by Siemens, Nokia and Motorola. Using task scenarios which represented a typical everyday use-situation, participants were asked to explore different WAP services. The results are straight forward: Only user-friendly mobile services enjoy a high potential of user acceptance. The main factor of general acceptance is described by the experienced utility and usability of the WAP services. In other terms, successful services are especially characterized by limited functionality and a clear focus on the specific mobile situation of the user. This clearly demonstrates the users requests towards the mobile internet compared to the stationary internet. Despite the underlying restrictions of the mobile internet, our data prove that consumers generally show interest in mobile services as they see the advantages of independence of time and space. On the other hand, our results demonstrate the necessity of a considerable improvement of the usability and the quality of information presentation. The results will be specifically discussed to create usability recommendations for further developments.
Smartphone applications usability evaluation: a hybrid model and its implementation - Submitted Version
Abstract. Evaluating the usability of smartphone applications is crucial for their success, so developers can learn how to adapt them considering the dynamicity of mobile scenarios. The HCI community recommends considering different requirements when evaluating those applications, such as quantitative data (metrics), subjective evaluation (users’ impressions) and context data (e.g. environment and devices conditions). We observed a lack in the literature of approaches that support those three requirements combined into a single experiment; generally one or a pair of them is used. Besides, performing usability evaluation on real mobile scenarios is hard to achieve and most proposals are based on laboratory-controlled experiments. In this paper, we present our proposal for a hybrid usability evaluation of smartphone applications, which is composed by a model and an infrastructure that implements it. The model describes how to automatically monitor and collect context data and usability metrics, how those data can be processed for analysis support and how users’ impressions can be collected. An infrastructure is provided to implement the model allowing it to be plugged into any smartphone Android-based application. To evaluate our proposal, we performed a field experiment, with 21 users using three mobile applications during a 6-month period, in their day-to-day scenarios.
An Empirical Study of the Use Contexts and Usability Problems in Mobile Internet
2002
Mobile Internet, which is a combination of the Internet with mobile devices, has become popular recently. Mobile Internet is primarily different from stationary Internet in that it may be used in various contexts, whereas stationary Internet is mostly used in predetermined environments. However, it is far from clear in what places and situations Mobile Internet has been used frequently and what the impact of the contexts has been on the ease of use. This paper proposes a framework for studying the use context relevant to Mobile Internet. It then presents the results of an empirical study of the use context and service usability for Mobile Internet by using monitoring methods. The results indicate the use contexts of Mobile Internet are of a concentrated type rather than being widely diverse. Moreover, the different contexts present unique usability problems. The paper concludes by discussing the theoretical and practical implications of the results.
Related papers
Usability Analysis and Evaluation of Mobile ICT Systems
hft.org
There is a need for new usability methods tailored to suit the development of mobile ICT. In response to this, three methods that have been developed to a prototype level are presented.The methods are designed to be cost-effective and provide information on (1) user problems as experienced in users’ everyday lives, (2) contexts of use, and (3) integration of technology in users’ patterns of activity. The methods have been tried out in a small-scale evaluation in conjunction with the introduction of PDAs and mobile telephones with PDAs in an organisation.
Challenges, Methodologies, and Issues in the Usability Testing of Mobile Applications
International Journal of Human-Computer Interaction, 2005
Usability testing of software applications developed for mobile devices is an emerging research area that faces a variety of challenges due to unique features of mobile devices, limited bandwidth, unreliability of wireless networks, as well as the changing context (environmental factors). Traditional guidelines and methods used in usability testing of desktop applications may not be directly applicable to a mobile environment. Therefore, it is essential to develop and adopt appropriate research methodologies that can evaluate the usability of mobile applications. The contribution of this paper is to propose a generic framework for conducting usability tests for mobile applications through discussing research questions, methodologies, and usability attributes. The paper provides an overview of existing usability studies and discusses major research questions that have been investigated. Then, it proposes a generic framework and provides detailed guidelines on how to conduct such usability studies.
Usability evaluation for mobile device: a comparison of laboratory and field tests
… of the 8th conference on Human- …, 2006
A framework for evaluating the usability of mobile phones based on multi-level, hierarchical model of usability factors
Interacting with Computers, 2009
As a mobile phone has various advanced functionalities or features, usability issues are increasingly challenging. Due to the particular characteristics of a mobile phone, typical usability evaluation methods and heuristics, most of which are relevant to a software system, might not effectively be applied to a mobile phone. Another point to consider is that usability evaluation activities should help designers find usability problems easily and produce better design solutions. To support usability practitioners of the mobile phone industry, we propose a framework for evaluating the usability of a mobile phone, based on a multilevel, hierarchical model of usability factors, in an analytic way. The model was developed on the basis of a set of collected usability problems and our previous study on a conceptual framework for identifying usability impact factors. It has multi-abstraction levels, each of which considers the usability of a mobile phone from a particular perspective. As there are goal-means relationships between adjacent levels, a range of usability issues can be interpreted in a holistic as well as diagnostic way. Another advantage is that it supports two different types of evaluation approaches: task-based and interface-based. To support both evaluation approaches, we developed four sets of checklists, each of which is concerned, respectively, with task-based evaluation and three different interface types: Logical User Interface (LUI), Physical User Interface (PUI) and Graphical User Interface (GUI). The proposed framework specifies an approach to quantifying usability so that several usability aspects are collectively measured to give a single score with the use of the checklists. A small case study was conducted in order to examine the applicability of the framework and to identify the aspects of the framework to be improved. It showed that it could be a useful tool for evaluating the usability of a mobile phone. Based on the case study, we improved the framework in order that usability practitioners can use it more easily and consistently.
Leveraging Microsoft’s Mobile Usability Guidelines: Conceptualizing and Developing Scales for Mobile Application Usability
International Journal of Human-Computer Studies, 2016
This research conceptualizes mobile application usability and develops and validates an instrument to measure the same. Mobile application usability has attracted widespread attention in the field of human-computer interaction because well-designed applications can enhance user experiences. To conceptualize mobile application usability, we analyzed Microsoft's mobile usability guidelines and defined 10 constructs representing mobile application usability. Next, we conducted a pilot study followed by a quantitative assessment of the content validity of the scales. We then sequentially applied exploratory factor analysis and confirmatory factor analysis to two samples (n=404; n=501) consisting of German consumers using mobile social media applications on their smartphones. To evaluate the confirmatory factor model, we followed a step-by-step process assessing unidimensionality, discriminant validity and reliability. To assess the nomological validity of our instrument, we examined the impact of mobile application usability on two outcomes: continued intention to use and brand loyalty. The results confirmed that mobile application usability was a good predictor of both outcomes. The constructs and scales associated with mobile application usability validated in this paper can be used to guide future research in human-computer interaction and aid in the effective design of mobile applications.
Usability Dimensions for Mobile Applications-A Review
2013
Abstract: Usability has been increasingly recognized as a significant quality dimension to determine the success of mobile applications. Due to its importance, a number of usability guidelines have been proposed to direct the design of usable applications. The guidelines are intended particularly for desktop and web-based applications. Mobile applications on the other hand are different in many ways from those applications due to the mobility nature of mobile devices. To date, the usability guidelines for mobile applications are very limited. They in fact are isolated, which makes usability evaluation for mobile devices more difficult. This study aims to address this issue by proposing a set of usability dimensions that should be considered for designing and evaluating mobile applications. The dimensions are illustrated as a model that considers four contextual factors: user, environment, technology and task/activity. The model was proposed based on the reviews of previous related s...
Framework and Model of Usability Factors of Mobile Phones
Handbook of Research on User Interface Design and Evaluation for Mobile Technology
This chapter aims at developing a framework and model for identifying and organizing usability factors of mobile phones. Although some studies have been made on evaluating the factors, there is no systematic framework for identifying and categorizing them. This chapter proposes a conceptual framework which has multiple views to explain different aspects of the interaction between users and mobile phones, and
Evaluation of Usability Dimensions of Smartphone Applications
International Journal of Advanced Computer Science and Applications
This study analyses different techniques used for evaluation of various usability dimensions of software applications (apps) being used on the smartphones. The scope of this study is to evaluate various aspects of the usability techniques employed in the domain of smartphone apps. Usability assessment methodologies are evaluated for different types of applications running on different operating systems like Android, Blackberry and iOS, etc. Usability evaluation techniques and methodologies with respect to usability heuristics like field experiments, laboratory experiments models and usability standards are discussed in detail. The issues for evaluation of usability of smartphone apps are identified by considering limitations and areas of improvement outlined in the contemporary literature. A conceptual framework for usability evaluation of smartphone apps is also designed which would be validated through experimentation in the thesis work. This study is particularly useful to comprehend usability issues and their likely remedies to produce high quality smartphone apps.
A Set of Usability Heuristics for Mobile Applications
Human-Computer Interaction. Perspectives on Design, 2019
The innovations proposed by the cell phone market have grown steadily in recent years, as well as the evolution of the complexity of operating systems, hardware and applications available. With these changes and changes, new challenges and usability-related quirks emerge and need to be considered during the development process of these applications, which incorporate new user-application interactions, increasingly changing the behavior of smartphone users. It is known that usability is an important factor when choosing the use of these technologies. Usability depends on factors such as the User, their characteristics and abilities, the Task which the user intends to achieve and also the Context of the application's use. This work will lead to a Systematic Review of Literature with the objective of identifying the heuristics and usability metrics used in the literature and/or industry, and based on the results obtained, it is intended to propose a set of usability heuristics focused for the context of mobile applications on smartphone, considering the User, Task and Context, as usability factors and Cognitive Load as an important attribute of usability. Furthermore, an empirical validation of the proposal will be performed with usability specialists and improvements can be incorporated into the proposed model after this validation.
Loading Preview
Sorry, preview is currently unavailable. You can download the paper by clicking the button above.