A metric-based evaluation model for applications on mobile phones

2013

Abstract: Usability has been increasingly recognized as a significant quality dimension to determine the success of mobile applications. Due to its importance, a number of usability guidelines have been proposed to direct the design of usable applications. The guidelines are intended particularly for desktop and web-based applications. Mobile applications on the other hand are different in many ways from those applications due to the mobility nature of mobile devices. To date, the usability guidelines for mobile applications are very limited. They in fact are isolated, which makes usability evaluation for mobile devices more difficult. This study aims to address this issue by proposing a set of usability dimensions that should be considered for designing and evaluating mobile applications. The dimensions are illustrated as a model that considers four contextual factors: user, environment, technology and task/activity. The model was proposed based on the reviews of previous related s...

IEEE Access

The innovations proposed by the cell phone market have grown steadily in recent years, along with the increasing complexity of the hardware, operating systems, and applications available in this market. These changes bring new challenges related to usability that need to be considered during the development process of these applications since the new forms of user-application interactions increasingly require adapting the behavior of smartphone users. In this situation, usability is an important issue that depends on factors such as the Users, their characteristics and abilities, the Task which the users intend to achieve and also the application usage Context. This work presents a systematic literature review with the objective of identifying the heuristics and usability metrics used in the literature and/or industry. Based on the review results, this work presents another contribution with a proposal of a set of usability heuristics focused in mobile applications on smartphones, considering the User, Task and Context as usability factors and Cognitive Load as an important attribute of usability. The components of this set are detailed in a model intended to be used in empirical validations allowing to dynamically incorporate improvements to the proposal.

Jurnal Teknologi, 2015

Nowadays, mobile phones provide not just voice call and messaging services, but plethora of other services. Those computational capabilities allow mobile phones to serve people in various areas including education, banking, commerce, travelling, and other daily life aspects. Meanwhile, the number of mobile phone users has increased dramatically in the last decade. On the other hand, the usability of an application can usually be verified through the user interface. Therefore, this paper aims to design a measurement tool to evaluate the usability of mobile applications based on the usability attributes and dimensions that must be considered in the interface. To obtain the appropriate attributes, a Systematic Literature Review (SLR) has been conducted and the Goal Question Metric (GQM) has been used to design the tool. From 261 related works only 18 most relevant ones were selected, through four SLR. 25 dimensions were found through the SLR, but some of these dimensions are synonymous or a part of other dimensions. Consequently, three dimensions must be included in any usability evaluation instrument, which is broken down into ten sub dimensions.

Interacting with Computers, 2009

As a mobile phone has various advanced functionalities or features, usability issues are increasingly challenging. Due to the particular characteristics of a mobile phone, typical usability evaluation methods and heuristics, most of which are relevant to a software system, might not effectively be applied to a mobile phone. Another point to consider is that usability evaluation activities should help designers find usability problems easily and produce better design solutions. To support usability practitioners of the mobile phone industry, we propose a framework for evaluating the usability of a mobile phone, based on a multilevel, hierarchical model of usability factors, in an analytic way. The model was developed on the basis of a set of collected usability problems and our previous study on a conceptual framework for identifying usability impact factors. It has multi-abstraction levels, each of which considers the usability of a mobile phone from a particular perspective. As there are goal-means relationships between adjacent levels, a range of usability issues can be interpreted in a holistic as well as diagnostic way. Another advantage is that it supports two different types of evaluation approaches: task-based and interface-based. To support both evaluation approaches, we developed four sets of checklists, each of which is concerned, respectively, with task-based evaluation and three different interface types: Logical User Interface (LUI), Physical User Interface (PUI) and Graphical User Interface (GUI). The proposed framework specifies an approach to quantifying usability so that several usability aspects are collectively measured to give a single score with the use of the checklists. A small case study was conducted in order to examine the applicability of the framework and to identify the aspects of the framework to be improved. It showed that it could be a useful tool for evaluating the usability of a mobile phone. Based on the case study, we improved the framework in order that usability practitioners can use it more easily and consistently.

Journal of Interaction Science, 2013

The usefulness of mobile devices has increased greatly in recent years allowing users to perform more tasks in a mobile context. This increase in usefulness has come at the expense of the usability of these devices in some contexts. We conducted a small review of mobile usability models and found that usability is usually measured in terms of three attributes; effectiveness, efficiency and satisfaction. Other attributes, such as cognitive load, tend to be overlooked in the usability models that are most prominent despite their likely impact on the success or failure of an application. To remedy this we introduces the PACMAD (People At the Centre of Mobile Application Development) usability model which was designed to address the limitations of existing usability models when applied to mobile devices. PACMAD brings together significant attributes from different usability models in order to create a more comprehensive model. None of the attributes that it includes are new, but the existing prominent usability models ignore one or more of them. This could lead to an incomplete usability evaluation. We performed a literature search to compile a collection of studies that evaluate mobile applications and then evaluated the studies using our model.

The consumers' market of mobile devices ranging from mobile phones, laptops, PDAs, Tablets, iPad amongst others is gaining momentum due to the increase in demand by the users, with the main interest of meeting their respective needs. The diverse needs of the users especially as it affects the purpose of the individual interest make developing mobile applications not only a challenging one, and also tasking, and in this light, usability characteristics of mobile applications have been indispensable when developing a useable mobile application. This paper aims to highlight the expected quality characteristics of a mobile application with a detailed reviewed discussion mostly on the usability characteristics, being one of the external characteristics of mobile applications according to ISO 9126.

There are several methods to evaluate mobile applications. The aim is to ensure that the applications are usable and satisfying users. Among the methods include lab test and field test. However, not many have focused on the quality characteristics should be used during the test. This study aims to develop subjective metrics for evaluation of mobile application using Goal Question Metric (GQM). To ensure the metrics we develop are reliable and useful, we conduct usability study that used the metrics we developed using GQM approach. The result indicates that the metrics can be used to evaluate mobile application in term of user satisfaction. Four test cases with different platforms employed in this study shows that the metrics are reliable to be used in mobile evaluation.

International journal of engineering research and technology, 2018

Mobile devices are evolving and becoming more complex with a variety of features and functionalities. Many applications that were originally deployed as desktop applications or web applications are now being ported to mobile devices. Due to this the usefulness of mobile devices has increased greatly in recent years allowing users to perform more tasks in a mobile context. This increase in usefulness makes it compulsory to evaluate the usability of mobile application and mobiles as well. Usability evaluation is an important step in software development in order to improve certain aspects of the system. However, it is often a challenge especially when it comes to evaluating applications running on mobile devices because of the restrictions posed by the device and the lack of supporting tools and software available to collect the necessary usability data.

2009

Abstract Usability analysis is an important step in software development in order to improve certain aspects of the system. However, it is often a challenge especially when it comes to evaluating applications running on mobile devices because of the restrictions posed by the device and the lack of supporting tools and software available to collect the necessary usability data. This paper proposes a methodology and framework to aid developers in preparing the mobile system for usability analysis.

Canadian Conference on Electrical and Computer Engineering

Fatih Nayebi, Jean-Marc Desharnais, Alain Abran, ‘The State of the Art of Mobile Application Usability Evaluation’, 25th IEEE Canadian Conference on Electrical and Computer Engineering (IEEE CD: 978-1-4673-6/12) Montreal, April 29-May 2, 2012.

MIS Quarterly, 2015

This paper presents a mobile application usability conceptualization and survey instrument following the 10step procedure recommended by MacKenzie et al. (2011). Specifically, we adapted Apple's user experience guidelines to develop our conceptualization of mobile application usability that we then developed into 19 first-order constructs that formed 6 second-order constructs. To achieve our objective, we collected 4 datasets: content validity (n=318), pre-test (n=440), validation (n=408), and cross-validation (n=412). The nomological validity of this instrument was established by examining its impact on two outcomes: continued intention to use and mobile application loyalty. We found that the constructs that represented our mobile application usability conceptualization were good predictors of both outcomes and compared favorably to an existing instrument based on Microsoft's usability guidelines. In addition to being an exemplar of the recent procedure of MacKenzie et al. to validate an instrument, this work provides a rich conceptualization of an instrument for mobile application usability that can serve as a springboard for future work to understand the impacts of mobile application usability and can be used as a guide to design effective mobile applications.

Testing is one of the critical stages in the software development life cycle (SDLC). Usability testing is a very important field that helps the applications be usable and easy for the users. Because of the importance of usability testing, a metrics has been developed to help in measuring the usability through converting the main qualitative usability attributes in ISO to quantitative steps that provide the developer a framework to follow in developing to achieve usability of their applications and helps the tester with a checklist and a tool to measure the usability percentage of their application. The framework provides a set of steps to achieve the usability attributes and answers the question of how you could measure this attribute with the defined steps. The framework results in a 95% average accuracy in the high-rate application and a 59% average accuracy in the low-rate application. Finally, the framework is programmed in a tool to measure the usability percentage of the appli...

SpringerPlus, 2016

This paper presents an empirical study based on a set of measures to evaluate the usability of mobile applications running on different mobile operating systems, including Android, iOS and Symbian. The aim is to evaluate empirically a framework that we have developed on the use of the Software Quality Standard ISO 9126 in mobile environments, especially the usability characteristic. To do that, 32 users had participated in the experiment and we have used ISO 25062 and ISO 9241 standards for objective measures by working with two widely used mobile applications: Google Apps and Google Maps. The QUIS 7.0 questionnaire have been used to collect measures assessing the users' level of satisfaction when using these two mobile applications. By analyzing the results we highlighted a set of mobile usability issues that are related to the hardware as well as to the software and that need to be taken into account by designers and developers in order to improve the usability of mobile applications.

Increasing trend for mobile devices raised the question of usability of mobile applications. A device is defined as mobile when it has a small size and a light weight. Hence a mobile device would have a small screen, which in turn requires small interfaces for mobile applications. Input and output processes become difficult in small interfaces. Therefore, new technologies are required to increase usability of mobile applications. In this study, the studies done and the methods proposed to increase usability of mobile devices are investigated. This study is aimed to be a reference for future studies on usability of mobile applications.