Background These new HHS Web usability Guidelines carry forward one of the most enduring success ... more Background These new HHS Web usability Guidelines carry forward one of the most enduring success stories in user interface design. They continue the noble tradition of thoughtful practitioners who have hacked their way through the unruly design landscape and then distilled their experience into compact and generalizable aphorisms or patterns.
Since its introduction in 1986, the 10-item System Usability Scale (SUS) has been assumed to be u... more Since its introduction in 1986, the 10-item System Usability Scale (SUS) has been assumed to be unidimensional. Factor analysis of two independent SUS data sets reveals that the SUS actually has two factors-Usable (8 items) and Learnable (2 items-specifically, Items 4 and 10). These new scales have reasonable reliability (coefficient alpha of .91 and .70, respectively). They correlate highly with the overall SUS (r = .985 and .784, respectively) and correlate significantly with one another (r = .664), but at a low enough level to use as separate scales. A sensitivity analysis using data from 19 tests had a significant Test by Scale interaction, providing additional evidence of the differential utility of the new scales. Practitioners can continue to use the current SUS as is, but, at no extra cost, can also take advantage of these new scales to extract additional information from their SUS data. The data support the use of "awkward" rather than "cumbersome" in Item 8.
International Journal of Human–Computer Interaction, 2019
This article aims to investigate the Grand Challenges which arise in the current and emerging lan... more This article aims to investigate the Grand Challenges which arise in the current and emerging landscape of rapid technological evolution towards more intelligent interactive technologies, coupled with increased and widened societal needs, as well as individual and collective expectations that HCI, as a discipline, is called upon to address. A perspective oriented to humane and social values is adopted, formulating the challenges in terms of the impact of emerging intelligent interactive technologies on human life both at the individual and societal levels. Seven Grand Challenges are identified and presented in this article: Human-Technology Symbiosis; Human-Environment Interactions; Ethics, Privacy and Security; Well-being, Health and Eudaimonia; Accessibility and Universal Access; Learning and Creativity; and Social Organization and Democracy. Although not exhaustive, they summarize the views and research priorities of an international interdisciplinary group of experts, reflecting different scientific perspectives, methodological approaches and application domains. Each identified Grand Challenge is analyzed in terms of: concept and problem definition; main research issues involved and state of the art; and associated emerging requirements. BACKGROUND This article presents the results of the collective effort of a group of 32 experts involved in the community of the Human Computer Interaction International (HCII) Conference series. The group's collaboration started in early 2018 with the collection of opinions from all group members, each asked to independently list and describe five HCI grand challenges. During a one-day meeting held on the 20th July 2018 in the context of the HCI International 2018 Conference in Las Vegas, USA, the identified topics were debated and challenges were formulated in terms of the impact of emerging intelligent interactive technologies on human life both at the individual and societal levels. Further analysis and consolidation led to a set of seven Grand Challenges presented herein. This activity was organized and supported by the HCII Conference series.
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 2006
I examined samples from a number of companys' user publications using several automated reading m... more I examined samples from a number of companys' user publications using several automated reading measures and a graphics/text ratio. The goal was to answer two questions: Were there reliable differences in writing style among the competitors? If so, were these differences related to their rank position in published surveys of user satisfaction with documentation? Of the measures included in the study, only the Cloudiness Count had any significant relationship to rank position in the surveys. A second evaluation, focused on the components of the Cloudiness Count, indicated that both of its components (passive voice and 'empty' words-a type of infrequent word) contributed equally to its effectiveness. This is consistent with psycholinguistic research that indicates that it is harder for people to extract the meaning from a passive sentence relative to its active counterpart, and that word frequency is the variable with the most influence on the speed of lexical access.
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 1999
Predictive keyboards are software keyboards that, to conserve screen real estate, display a subse... more Predictive keyboards are software keyboards that, to conserve screen real estate, display a subset of the full set of alphabetic keys at any one time, predicting the letters to display on the basis of tables of letters' transitional probabilities. Using different types of test texts (names, words, random strings), we evaluated the influence of various manipulations on the efficiency of letter selection with a predictive keyboard. The results of this study indicated that, across tested text types (names, words, random strings), (1) there was little benefit gained from adding a fourth transitional table modeling the likelihood of a letter's use as a function of a space and two preceding letters, (2) there was a potential benefit from increasing the number of displayed letters from six to eight, and (3) for a personal communicator device, an adaptive strategy would probably be less effective than using multiple sets of letter probability tables.
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 2006
The primary purpose of this analysis was to investigate the effect of changing the level of descr... more The primary purpose of this analysis was to investigate the effect of changing the level of description of usability problems on the estimate of the problem discovery rate (p). A secondary purpose was to describe a method for using p to estimate the number of problems remaining available for discovery given the constraints associated with a particular participant population, application, and set of tasks. The level of problem description influenced estimates of p, and the direction of influence was predictable, with higher levels of description producing higher estimates of p. Practitioners need a level of description that flows easily into recommendations for redesigning products, but to keep usability studies as efficient as possible, practitioners also need to seek a level of description that takes advantage of common patterns in observed usability problems. Managing this tradeoff is only one of the challenges of usability evaluation, but it is an important one.
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2009
Correlations between prototypical usability metrics from 90 distinct usability tests were strong ... more Correlations between prototypical usability metrics from 90 distinct usability tests were strong when measured at the task-level (r between .44 and .60). Using test-level satisfaction ratings instead of task-level ratings attenuated the correlations (r between .16 and .24). The method of aggregating data from a usability test had a significant effect on the magnitude of the resulting correlations. The results of principal components and factor analyses on the prototypical usability metrics provided evidence for an underlying construct of general usability with objective and subjective factors.
This paper describes three selection accuracy experiments with the selection slot, a new menu sel... more This paper describes three selection accuracy experiments with the selection slot, a new menu selection device for pen-based computer systems. A selection slot is a slot into which a user places the tip of a pen or stylus for the purpose of selecting an item from a list of items. The results of the first experiment, using a prototype slot for notebook name selection with 3 mm per notebook name, indicated 100% selection accuracy. In a second experiment, the results indicated that with as little as 1.1 mm per notebook name, users could select notebook names with 100% accuracy. A third experiment evaluated page selection accuracy for 55-and 99-page slots using a 73 mm slot (0.74 mm per page for 99 pages). Participants in the 55-page condition used a single-page advance control to reach target pages greater than 55. Considering all targets, participants were slightly but consistently less accurate with the 99-page slots, but their within-slot selection accuracies were equal. All participants preferred the 99-page slot due to the inconvenience of acquiring pages greater than 55 with the single-page advance control.
The Mean Opinion Scale (MOS) is a questionnaire used to obtain listeners' subjective assessme... more The Mean Opinion Scale (MOS) is a questionnaire used to obtain listeners' subjective assessments of synthetic speech. This paper documents the motivation, method, and results of six experiments conducted from 1999 to 2002 that investigated the psychometric properties of the MOS and expanded the range of speech characteristics it evaluates. Our initial experiments documented the reliability, validity, sensitivity, and factor
Overestimation of the likelihood of problem discovery (p) in usability studies is serious because... more Overestimation of the likelihood of problem discovery (p) in usability studies is serious because it leads to underestimation of required sample sizes. The current experiments demonstrated that both normalization and multiple regression can significantly reduce this overestimation. Normalization proved to be as accurate as discounting and multiple regression methods, but tended to produce underestimates rather than overestimates of p. Normalization is the preferred method because it does not require empirical estimation of regression weights. A combination of normalization and Good-Turing discounting appears to provide very accurate sample size projection, even with p estimated from as few as two participants.
As the cost of the technology falls, touch screens are becoming more common. The purpose of this ... more As the cost of the technology falls, touch screens are becoming more common. The purpose of this report is to document the human factors research in touch screens from 1980 to 1992. It covers the topics of touch-selection strategies, button size, touch-screen keyboards, touch-feedback strategies, mouse-emulation strategies, touch biases and screen angles. The literature review provides the basis for making recommendations to touch screen designers.
This paper provides frequency distributions (constrained and unconstrained) for words and names a... more This paper provides frequency distributions (constrained and unconstrained) for words and names associated with the letters of the English alphabet. These distributions can be useful when developing voice spelling user interfaces (Lewis & Commarford, 2003). The data suggest that the unconstrained word distribution is preferable to the names distribution because, contrary to expectation, the overall consistency of unconstrained responses was not poorer than the responses in the names distribution and the occurrence of cases in which participants did not provide a response for the names distribution was significantly greater than for the unconstrained distribution.
International Journal of Human-Computer Interaction, 1995
Group 1 1. Overall, I am satisfied with how easy it is to use this system. Group 2 19. Overall, I... more Group 1 1. Overall, I am satisfied with how easy it is to use this system. Group 2 19. Overall, I am satisfied with this system. Group 3 2. It was simple to use this system. 3. I can effectively complete my work using this system. 4. I am able to complete my work quickly using this system. 5. I am able to efficiently complete my work using this system. 6. I feel comfortable using this system. 7. It was easy to learn to use this system. 8. I believe I became productive quickly using this system. 9. The system gives error messages that clearly tell me how to fix problems. 10. Whenever I make a mistake using the system, I recover easily and quickly. 11. The information provided with this system is clear. 12. It is easy to find the information I needed. 13. The information provided for the system is easy to understand. 14. The information is effective in helping me complete the tasks and scenarios. 15. The organization of the information on the systems screens is clear. 16. The interface of the system is pleasant. 17. I like using the interface of this system. 18. This system has all the functions and capabilities I expect it to have. Appended items related specifically to colorectal cancer screening: Group 4 20. It is easy to find information about the patient's colorectal cancer screening history in this system. Group 5 21. It is easy to find the patient's current status with regard to colorectal cancer screening in this system. Group 6 22. The system provides helpful patient education materials for CRC screening.
An emerging area of research in engineering psychology is the evaluation of text entry for mobile... more An emerging area of research in engineering psychology is the evaluation of text entry for mobile devices using a small number of keys for the control of cursor direction and character selection from a matrix of characters (i.e., selection-based data entry). The present article describes a software tool designed to reduce time and effort in the development of prototypes of alternative selection-based text-entry schemes and their empirical evaluation. The tool, available for distribution to researchers, educators, and students, uses Action Script code compiled into an executable file that has an embedded Adobe Flash Player and is compatible with most operating systems (including Microsoft Windows, Apple OSX, and Linux).
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 1999
The eight participants in this experiment used two different commercially available speech recogn... more The eight participants in this experiment used two different commercially available speech recognition dictation systems to complete a variety of reading transcription tasks. Participants enrolled fully in both systems. They received training in two correction strategies for both systems: multimodal correction (voice plus mouse plus keyboard) and hands-free correction (voice-only), and used both strategies during the experiment. The key findings were: • Both dictation systems were equally accurate. • Throughput (corrected words per minute) was significantly (63%) faster using multimodal correction. • Speaking rates were the same for both systems and correction strategies, averaging around 105-110 utterances (words and commands) per minute. • Correction speeds for the multimodal correction strategy (13.2 seconds per correction) were significantly faster than (a little more than twice as fast as) those for hands-free correction (29.1 seconds per correction). • At the end of the experi...
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 1999
The purpose of this study was to compare three input methods (standard keyboard, predictive keybo... more The purpose of this study was to compare three input methods (standard keyboard, predictive keyboard, and perfect handwriting recognition) considering user input rates and preferences. Six participants used all three input methods to enter both normal text and addresses. Participants indicated that they preferred to use the standard keyboard. The average input rate for handwriting was fastest, but also much more variable than the other methods. Despite its speed, participants generally found it difficult to write comfortably and legibly on the small (35times115 mm) display. The input rate for the standard keyboard was more than twice the input rate for the predictive keyboard. These results suggest that, for small devices, neither handwriting recognition nor predictive keyboards would effectively replace the standard keyboard layout. Even with perfect handwriting recognition, users seem to prefer tapping on a small standard keyboard unless the device's hardware design allows com...
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2011
When designing questionnaires there is a tradition of including items with both positive and nega... more When designing questionnaires there is a tradition of including items with both positive and negative wording to minimize acquiescence and extreme response biases. Two disadvantages of this approach are respondents accidentally agreeing with negative items (mistakes) and researchers forgetting to reverse the scales (miscoding).
The original System Usability Scale (SUS) and an all positively worded version were administered in two experiments (n=161 and n=213) across eleven websites. There was no evidence for differences in the response biases between the different versions. A review of 27 SUS datasets found 3 (11%) were miscoded by researchers and 21 out of 158 questionnaires (13%) contained mistakes from users.
We found no evidence that the purported advantages of including negative and positive items in usability questionnaires outweigh the disadvantages of mistakes and miscoding. It is recommended that researchers using the standard SUS verify the proper coding of scores and include procedural steps to ensure error-free completion of the SUS by users.
Researchers can use the all positive version with confidence because respondents are less likely to make mistakes when responding, researchers are less likely to make errors in coding, and the scores will be similar to the standard SUS.
This report describes a method that provides users with an improved method for entering data into... more This report describes a method that provides users with an improved method for entering data into mobile handheld devices such as cell phones and personal digital assistants (PDAs) by increasing the likelihood of displaying a desired word in a word completion interface. This solution is unique in that it is sensitive to implicit rejection of the word completion candidates. ITIRC Keywords Handheld Implicit rejection Input Mobile device PDA Word completion ,iv ,v Contents
It is important to help users navigate speech-based interactive voice response systems (IVRs) by ... more It is important to help users navigate speech-based interactive voice response systems (IVRs) by presenting the global navigation commands at any potential task end point. This paper presents an analysis we conducted to determine the optimal pause between presentation of a menu at a task terminal point and the presentation of the global navigation commands. Based on the distribution of response times to menu selection lists as a function of whether a desired option is in the list and whether the menu has been previously presented, we recommend a 1500 -2000 ms pause before presenting the global navigation commands.
Background These new HHS Web usability Guidelines carry forward one of the most enduring success ... more Background These new HHS Web usability Guidelines carry forward one of the most enduring success stories in user interface design. They continue the noble tradition of thoughtful practitioners who have hacked their way through the unruly design landscape and then distilled their experience into compact and generalizable aphorisms or patterns.
Since its introduction in 1986, the 10-item System Usability Scale (SUS) has been assumed to be u... more Since its introduction in 1986, the 10-item System Usability Scale (SUS) has been assumed to be unidimensional. Factor analysis of two independent SUS data sets reveals that the SUS actually has two factors-Usable (8 items) and Learnable (2 items-specifically, Items 4 and 10). These new scales have reasonable reliability (coefficient alpha of .91 and .70, respectively). They correlate highly with the overall SUS (r = .985 and .784, respectively) and correlate significantly with one another (r = .664), but at a low enough level to use as separate scales. A sensitivity analysis using data from 19 tests had a significant Test by Scale interaction, providing additional evidence of the differential utility of the new scales. Practitioners can continue to use the current SUS as is, but, at no extra cost, can also take advantage of these new scales to extract additional information from their SUS data. The data support the use of "awkward" rather than "cumbersome" in Item 8.
International Journal of Human–Computer Interaction, 2019
This article aims to investigate the Grand Challenges which arise in the current and emerging lan... more This article aims to investigate the Grand Challenges which arise in the current and emerging landscape of rapid technological evolution towards more intelligent interactive technologies, coupled with increased and widened societal needs, as well as individual and collective expectations that HCI, as a discipline, is called upon to address. A perspective oriented to humane and social values is adopted, formulating the challenges in terms of the impact of emerging intelligent interactive technologies on human life both at the individual and societal levels. Seven Grand Challenges are identified and presented in this article: Human-Technology Symbiosis; Human-Environment Interactions; Ethics, Privacy and Security; Well-being, Health and Eudaimonia; Accessibility and Universal Access; Learning and Creativity; and Social Organization and Democracy. Although not exhaustive, they summarize the views and research priorities of an international interdisciplinary group of experts, reflecting different scientific perspectives, methodological approaches and application domains. Each identified Grand Challenge is analyzed in terms of: concept and problem definition; main research issues involved and state of the art; and associated emerging requirements. BACKGROUND This article presents the results of the collective effort of a group of 32 experts involved in the community of the Human Computer Interaction International (HCII) Conference series. The group's collaboration started in early 2018 with the collection of opinions from all group members, each asked to independently list and describe five HCI grand challenges. During a one-day meeting held on the 20th July 2018 in the context of the HCI International 2018 Conference in Las Vegas, USA, the identified topics were debated and challenges were formulated in terms of the impact of emerging intelligent interactive technologies on human life both at the individual and societal levels. Further analysis and consolidation led to a set of seven Grand Challenges presented herein. This activity was organized and supported by the HCII Conference series.
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 2006
I examined samples from a number of companys' user publications using several automated reading m... more I examined samples from a number of companys' user publications using several automated reading measures and a graphics/text ratio. The goal was to answer two questions: Were there reliable differences in writing style among the competitors? If so, were these differences related to their rank position in published surveys of user satisfaction with documentation? Of the measures included in the study, only the Cloudiness Count had any significant relationship to rank position in the surveys. A second evaluation, focused on the components of the Cloudiness Count, indicated that both of its components (passive voice and 'empty' words-a type of infrequent word) contributed equally to its effectiveness. This is consistent with psycholinguistic research that indicates that it is harder for people to extract the meaning from a passive sentence relative to its active counterpart, and that word frequency is the variable with the most influence on the speed of lexical access.
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 1999
Predictive keyboards are software keyboards that, to conserve screen real estate, display a subse... more Predictive keyboards are software keyboards that, to conserve screen real estate, display a subset of the full set of alphabetic keys at any one time, predicting the letters to display on the basis of tables of letters' transitional probabilities. Using different types of test texts (names, words, random strings), we evaluated the influence of various manipulations on the efficiency of letter selection with a predictive keyboard. The results of this study indicated that, across tested text types (names, words, random strings), (1) there was little benefit gained from adding a fourth transitional table modeling the likelihood of a letter's use as a function of a space and two preceding letters, (2) there was a potential benefit from increasing the number of displayed letters from six to eight, and (3) for a personal communicator device, an adaptive strategy would probably be less effective than using multiple sets of letter probability tables.
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 2006
The primary purpose of this analysis was to investigate the effect of changing the level of descr... more The primary purpose of this analysis was to investigate the effect of changing the level of description of usability problems on the estimate of the problem discovery rate (p). A secondary purpose was to describe a method for using p to estimate the number of problems remaining available for discovery given the constraints associated with a particular participant population, application, and set of tasks. The level of problem description influenced estimates of p, and the direction of influence was predictable, with higher levels of description producing higher estimates of p. Practitioners need a level of description that flows easily into recommendations for redesigning products, but to keep usability studies as efficient as possible, practitioners also need to seek a level of description that takes advantage of common patterns in observed usability problems. Managing this tradeoff is only one of the challenges of usability evaluation, but it is an important one.
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2009
Correlations between prototypical usability metrics from 90 distinct usability tests were strong ... more Correlations between prototypical usability metrics from 90 distinct usability tests were strong when measured at the task-level (r between .44 and .60). Using test-level satisfaction ratings instead of task-level ratings attenuated the correlations (r between .16 and .24). The method of aggregating data from a usability test had a significant effect on the magnitude of the resulting correlations. The results of principal components and factor analyses on the prototypical usability metrics provided evidence for an underlying construct of general usability with objective and subjective factors.
This paper describes three selection accuracy experiments with the selection slot, a new menu sel... more This paper describes three selection accuracy experiments with the selection slot, a new menu selection device for pen-based computer systems. A selection slot is a slot into which a user places the tip of a pen or stylus for the purpose of selecting an item from a list of items. The results of the first experiment, using a prototype slot for notebook name selection with 3 mm per notebook name, indicated 100% selection accuracy. In a second experiment, the results indicated that with as little as 1.1 mm per notebook name, users could select notebook names with 100% accuracy. A third experiment evaluated page selection accuracy for 55-and 99-page slots using a 73 mm slot (0.74 mm per page for 99 pages). Participants in the 55-page condition used a single-page advance control to reach target pages greater than 55. Considering all targets, participants were slightly but consistently less accurate with the 99-page slots, but their within-slot selection accuracies were equal. All participants preferred the 99-page slot due to the inconvenience of acquiring pages greater than 55 with the single-page advance control.
The Mean Opinion Scale (MOS) is a questionnaire used to obtain listeners' subjective assessme... more The Mean Opinion Scale (MOS) is a questionnaire used to obtain listeners' subjective assessments of synthetic speech. This paper documents the motivation, method, and results of six experiments conducted from 1999 to 2002 that investigated the psychometric properties of the MOS and expanded the range of speech characteristics it evaluates. Our initial experiments documented the reliability, validity, sensitivity, and factor
Overestimation of the likelihood of problem discovery (p) in usability studies is serious because... more Overestimation of the likelihood of problem discovery (p) in usability studies is serious because it leads to underestimation of required sample sizes. The current experiments demonstrated that both normalization and multiple regression can significantly reduce this overestimation. Normalization proved to be as accurate as discounting and multiple regression methods, but tended to produce underestimates rather than overestimates of p. Normalization is the preferred method because it does not require empirical estimation of regression weights. A combination of normalization and Good-Turing discounting appears to provide very accurate sample size projection, even with p estimated from as few as two participants.
As the cost of the technology falls, touch screens are becoming more common. The purpose of this ... more As the cost of the technology falls, touch screens are becoming more common. The purpose of this report is to document the human factors research in touch screens from 1980 to 1992. It covers the topics of touch-selection strategies, button size, touch-screen keyboards, touch-feedback strategies, mouse-emulation strategies, touch biases and screen angles. The literature review provides the basis for making recommendations to touch screen designers.
This paper provides frequency distributions (constrained and unconstrained) for words and names a... more This paper provides frequency distributions (constrained and unconstrained) for words and names associated with the letters of the English alphabet. These distributions can be useful when developing voice spelling user interfaces (Lewis & Commarford, 2003). The data suggest that the unconstrained word distribution is preferable to the names distribution because, contrary to expectation, the overall consistency of unconstrained responses was not poorer than the responses in the names distribution and the occurrence of cases in which participants did not provide a response for the names distribution was significantly greater than for the unconstrained distribution.
International Journal of Human-Computer Interaction, 1995
Group 1 1. Overall, I am satisfied with how easy it is to use this system. Group 2 19. Overall, I... more Group 1 1. Overall, I am satisfied with how easy it is to use this system. Group 2 19. Overall, I am satisfied with this system. Group 3 2. It was simple to use this system. 3. I can effectively complete my work using this system. 4. I am able to complete my work quickly using this system. 5. I am able to efficiently complete my work using this system. 6. I feel comfortable using this system. 7. It was easy to learn to use this system. 8. I believe I became productive quickly using this system. 9. The system gives error messages that clearly tell me how to fix problems. 10. Whenever I make a mistake using the system, I recover easily and quickly. 11. The information provided with this system is clear. 12. It is easy to find the information I needed. 13. The information provided for the system is easy to understand. 14. The information is effective in helping me complete the tasks and scenarios. 15. The organization of the information on the systems screens is clear. 16. The interface of the system is pleasant. 17. I like using the interface of this system. 18. This system has all the functions and capabilities I expect it to have. Appended items related specifically to colorectal cancer screening: Group 4 20. It is easy to find information about the patient's colorectal cancer screening history in this system. Group 5 21. It is easy to find the patient's current status with regard to colorectal cancer screening in this system. Group 6 22. The system provides helpful patient education materials for CRC screening.
An emerging area of research in engineering psychology is the evaluation of text entry for mobile... more An emerging area of research in engineering psychology is the evaluation of text entry for mobile devices using a small number of keys for the control of cursor direction and character selection from a matrix of characters (i.e., selection-based data entry). The present article describes a software tool designed to reduce time and effort in the development of prototypes of alternative selection-based text-entry schemes and their empirical evaluation. The tool, available for distribution to researchers, educators, and students, uses Action Script code compiled into an executable file that has an embedded Adobe Flash Player and is compatible with most operating systems (including Microsoft Windows, Apple OSX, and Linux).
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 1999
The eight participants in this experiment used two different commercially available speech recogn... more The eight participants in this experiment used two different commercially available speech recognition dictation systems to complete a variety of reading transcription tasks. Participants enrolled fully in both systems. They received training in two correction strategies for both systems: multimodal correction (voice plus mouse plus keyboard) and hands-free correction (voice-only), and used both strategies during the experiment. The key findings were: • Both dictation systems were equally accurate. • Throughput (corrected words per minute) was significantly (63%) faster using multimodal correction. • Speaking rates were the same for both systems and correction strategies, averaging around 105-110 utterances (words and commands) per minute. • Correction speeds for the multimodal correction strategy (13.2 seconds per correction) were significantly faster than (a little more than twice as fast as) those for hands-free correction (29.1 seconds per correction). • At the end of the experi...
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 1999
The purpose of this study was to compare three input methods (standard keyboard, predictive keybo... more The purpose of this study was to compare three input methods (standard keyboard, predictive keyboard, and perfect handwriting recognition) considering user input rates and preferences. Six participants used all three input methods to enter both normal text and addresses. Participants indicated that they preferred to use the standard keyboard. The average input rate for handwriting was fastest, but also much more variable than the other methods. Despite its speed, participants generally found it difficult to write comfortably and legibly on the small (35times115 mm) display. The input rate for the standard keyboard was more than twice the input rate for the predictive keyboard. These results suggest that, for small devices, neither handwriting recognition nor predictive keyboards would effectively replace the standard keyboard layout. Even with perfect handwriting recognition, users seem to prefer tapping on a small standard keyboard unless the device's hardware design allows com...
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2011
When designing questionnaires there is a tradition of including items with both positive and nega... more When designing questionnaires there is a tradition of including items with both positive and negative wording to minimize acquiescence and extreme response biases. Two disadvantages of this approach are respondents accidentally agreeing with negative items (mistakes) and researchers forgetting to reverse the scales (miscoding).
The original System Usability Scale (SUS) and an all positively worded version were administered in two experiments (n=161 and n=213) across eleven websites. There was no evidence for differences in the response biases between the different versions. A review of 27 SUS datasets found 3 (11%) were miscoded by researchers and 21 out of 158 questionnaires (13%) contained mistakes from users.
We found no evidence that the purported advantages of including negative and positive items in usability questionnaires outweigh the disadvantages of mistakes and miscoding. It is recommended that researchers using the standard SUS verify the proper coding of scores and include procedural steps to ensure error-free completion of the SUS by users.
Researchers can use the all positive version with confidence because respondents are less likely to make mistakes when responding, researchers are less likely to make errors in coding, and the scores will be similar to the standard SUS.
This report describes a method that provides users with an improved method for entering data into... more This report describes a method that provides users with an improved method for entering data into mobile handheld devices such as cell phones and personal digital assistants (PDAs) by increasing the likelihood of displaying a desired word in a word completion interface. This solution is unique in that it is sensitive to implicit rejection of the word completion candidates. ITIRC Keywords Handheld Implicit rejection Input Mobile device PDA Word completion ,iv ,v Contents
It is important to help users navigate speech-based interactive voice response systems (IVRs) by ... more It is important to help users navigate speech-based interactive voice response systems (IVRs) by presenting the global navigation commands at any potential task end point. This paper presents an analysis we conducted to determine the optimal pause between presentation of a menu at a task terminal point and the presentation of the global navigation commands. Based on the distribution of response times to menu selection lists as a function of whether a desired option is in the list and whether the menu has been previously presented, we recommend a 1500 -2000 ms pause before presenting the global navigation commands.
Uploads
Papers by Jim Lewis
The original System Usability Scale (SUS) and an all positively worded version were administered in two experiments (n=161 and n=213) across eleven websites. There was no evidence for differences in the response biases between the different versions. A review of 27 SUS datasets found 3 (11%) were miscoded by researchers and 21 out of 158 questionnaires (13%) contained mistakes from users.
We found no evidence that the purported advantages of including negative and positive items in usability questionnaires outweigh the disadvantages of mistakes and miscoding. It is recommended that researchers using the standard SUS verify the proper coding of scores and include procedural steps to ensure error-free completion of the SUS by users.
Researchers can use the all positive version with confidence because respondents are less likely to make mistakes when responding, researchers are less likely to make errors in coding, and the scores will be similar to the standard SUS.
The original System Usability Scale (SUS) and an all positively worded version were administered in two experiments (n=161 and n=213) across eleven websites. There was no evidence for differences in the response biases between the different versions. A review of 27 SUS datasets found 3 (11%) were miscoded by researchers and 21 out of 158 questionnaires (13%) contained mistakes from users.
We found no evidence that the purported advantages of including negative and positive items in usability questionnaires outweigh the disadvantages of mistakes and miscoding. It is recommended that researchers using the standard SUS verify the proper coding of scores and include procedural steps to ensure error-free completion of the SUS by users.
Researchers can use the all positive version with confidence because respondents are less likely to make mistakes when responding, researchers are less likely to make errors in coding, and the scores will be similar to the standard SUS.