Divya Madhavan Usability Test Plan for Introduction to Retail Industry Course Website 1 Document Overview

Page 1 of 5

This document describes a test plan for conducting a usability test during the development phase of Introduction to Retail Industry course website. The goal of usability testing is to identify potential design and content presentation concerns that need to be addressed in order to improve the efficiency, productivity, and end-user satisfaction. The usability test objectives are to: determine design and content inconsistencies within the user interface implement the web site under controlled and remote test conditions with representative users determine user-satisfaction levels of the user interface The website will be launched to a team of 8 India-based software engineers on May 6, 2013. The usability test for the website will be conducted during the early phase of the interface development, between April 11, 2013 and April 17, 2013. There will be two user Groups A and B with two software engineers in each group who will participate in the usability test. Group A will participate from different physical location and Group B will participate in controlled test conditions from the same physical location in Boston. 2 Executive Summary

The specific functions that will be evaluated through usability test of this website are 1) navigation such as locating functions, number of keystrokes to complete a function, following recommended screen flow, 2) content presentation such as locating and properly acting upon desired information, labeling ambiguities and 3) control usage such as correct entry fields. 3 Methodology

There will be a total of four participants in two Groups A and B, who will evaluate the interface. The measures that will be collected in the usability test are user-interface experience of the design, user satisfaction and suggestions for improvement. User Group A will be provided access to the website along with general guidelines and questionnaires for the test. This remote usability test will capture their experiences of using the interface through subjective evaluation. They will respond to questionnaires consisting of entrance questions, task scenarios and exit questions. User Group B will be invited to usability lab that will be set up in a mutually convenient location. Besides subjective evaluation, their experience will also be recorded by the facilitator through think aloud technique and observation of their interaction with the interface. 3.1 Participants

The user Groups A and B are recruited based on their location and availability for participation, number of years of association with the organization, educational background, skill-set, work experience, access to internet and familiarity with using online media and technologies. The two engineers in Group A, who are the representatives of the intended learner group, will participate in remote usability test from their respective locations in India. The two engineers in Group B, who are not the representatives of intended learner group but have similar professional skills/ backgrounds as the intended learners and work for the same organization in Boston, will participate in a traditional usability test facilitated by the designer. The participants' responsibilities will be to attempt to complete a set of task scenarios presented to them in as efficient and timely manner as possible, and provide feedback regarding the usability and acceptability of the user interface. The participants will be directed to provide honest opinions regarding the usability of the application. 3.2 Training

The participants will receive an overview of the procedure and purpose of the usability test. While the website will be completely developed before the launch, only few sections of the website will be fully functional for usability testing. The participants will receive information about those features and functions that may not be functional during the test. 3.3 Procedure

The participants in the remote testing will be seated at their workstation in their work environment. They will receive guidance over phone or email, depending on their preference. The participants in the traditional testing will be seated in the usability lab and think-aloud to allow the facilitator to observe and enter user behavior, comments, and task actions. All the participants will receive directions about the procedure be informed that their experience of the website will be evaluated rather than their performance complete the consent form be timed the entire experience including time-on-task not receive much explanation of individual tasks outside the task flow until after completion read aloud the task scenario description from the given document and begin the task record the post-task experience and satisfaction in the questionnaire and provide suggestions to improve Sample of the consent form, questionnaire and task scenarios are presented below. 3.3.1 Consent form Please read the following Usability Test Participation Agreement, print your name/date and email it to [email protected].

I agree to participate in the usability study conducted by Divya Madhavan. I grant permission to take notes of my response during the usability test for Introduction to Retail Industry course website. I understand that I may be quoted directly in usability test report, with or without reference to my name. I understand that my usability response, name and other personal details will not be used for any purpose other than developing the usability test report. This consent will not be made the basis of a future claim of any kind against Divya Madhavan. Date: ___________________ Please print your name: ______________________________ Thank you! I appreciate your participation. 3.3.2 TASK SCENARIOS You can either enter your comments in the online questionnaire as you perform the following scenarios on the Introduction to Retail Industry course website or save your comments to be filled later till you complete the scenarios. Please find the questionnaire at https://docs.google.com/spreadsheet/viewform?formkey=dDlYYTZFWGdtWm1QV212NU dlODVIWEE6MQ i. ii. iii. iv. v. vi. vii. viii. Scan through the webpage. Are the characters on the screen legible? Read any paragraph on the webpage. Is the language understandable? Read Lesson 1. Focus on the terminologies. Are the terminologies consistent? Navigate through few pages to see the amount of information on each page. Do you think they are clustered into small chunks to allow easy reading? Navigate through few pages and read the information on each page. What do you think about the organization and sequence of information? Look through few pages and notice the different colors used. Are the color codes consistent? Locate menu options. Are they ordered logically? Look for menu options, navigation buttons, exit buttons, data entry fields and content in different pages. Are they consistently labeled? Is the labeling format conventional? Are they positioned appropriately? Click on videos. See images, text and other graphics across few pages. Are they compatible in your browser? Do they load without errors? Are they appropriately used for the course? Explore the website as you wish. What do you think about the ease of learning the website? Since you have already explored the website now, choose a task you would like to perform on a different page. Can you accomplish the task successfully from the current page within the fewest number of steps? Was this easy? Can you remember the actions involved in executing the task so that you can repeat them quickly? Did you find information about progress towards accomplishing the desired task helpful? Did you land in a different page? Was there ambiguity in tracing the correct page? Were you quickly able to identify that you were in the wrong page and compete the task within few steps? Look at Lesson 1 and the topics for other lessons. What do you think of the relevance of the topics to the course? From your experience, will you be able to continue taking the course through this website?

ix. x. xi.

xii. xiii.

Roles

The different roles involved in the usability test are test participants and the designer. The designer will play multiple roles as below: trainer to provide training overview prior to usability testing facilitator to provide overview of study to participants, define usability and purpose of usability testing to participants, assist in conducting the test and respond to participant's requests for assistance data logger to record participants actions and comments test observer to observe, take notes and facilitate data logging in identifying problems and concerns 4.1 Ethics

Individual participant's name, personal details and test responses will not be used for purposes other than the usability test study of this website. 5 Usability Tasks

Though the website will not be fully functional as it will be still under construction during usability testing, the test scenarios will be developed such that they reflect a range of typical tasks to be performed on the website by the actual learners. 6 Usability Metrics

Success rates of task completion, error rates while performing tasks, time to learn task actions and complete scenarios, retention of task actions and subjective evaluations will be used as metrics to evaluate against the goals. 7 Usability Goals

The usability goals for the Introduction to Retail Industry course website are 8 minimum time to learn functions task completion rate of 100% task error-free rate of 100% speedy task performance easy retention of task actions high end-user satisfaction Problem Severity

The user experience data will be used to prioritize recommendations for revising the design. Based on the degree of impact of the problem and the frequency of users who experienced the

problem when performing the task, a degree of severity for each problem will be assigned to prioritize resolution. 9 Reporting Results

The Usability Test Report will be provided at the conclusion of the usability test. It will consist of results based on the evaluation of usability metrics against goals, subjective evaluations and specific design problems and recommendations for resolution. The report is anticipated to be delivered on April 18, 2013. Template Reference: Usability.gov. Usability Test Plan Template. Retrieved from http://www.usability.gov/templates/index.html