In recent years, we have witnessed a growing adoption of serious games in telerehabilitation by t... more In recent years, we have witnessed a growing adoption of serious games in telerehabilitation by taking advantage of advanced multimedia technologies such as motion capture and virtual reality devices. Current serious game solutions for telerehabilitation suffer form lack of personalization and adaptiveness to patients’ needs and performance. This paper introduces “RehaBot”, a framework for adaptive generation of personalized serious games in the context of remote rehabilitation, using 3D motion tracking and virtual reality environments. A personalized and versatile gaming platform with embedded virtual assistants, called “Rehab bots”, is created. Utilizing these rehab bots, all workout session scenes will include a guide with various sets of motions to direct patients towards performing the prescribed exercises correctly. Furthermore, the rehab bots employ a robust technique to adjust the workout difficulty level in real-time to match the patients’ performance. This technique correl...
This research uses multiple research methodologies guided by Information Systems Design Theory (I... more This research uses multiple research methodologies guided by Information Systems Design Theory (ISDT) to design and evaluate a mobile multimedia information system for Emergency Medical Services (EMS). We examined the impact of multimedia information for EMS information exchange and decision-making. A field study was designed and conducted in the Boise, Idaho region for three months to evaluate the system and validate ISDT design propositions. Findings from
This research uses multiple methods to investigate the use of an enterprise mobile multimedia inf... more This research uses multiple methods to investigate the use of an enterprise mobile multimedia information system aimed at improving handover of patient and emergency incident information from pre-hospital Emergency Medical Services (EMS) to hospital emergency department providers. A field study was conducted across EMS and hospital organizations in the Boise, Idaho region of the United States for three months to examine use of the system and to assess practitioner perspectives. Findings include perceived benefits and challenges to using digital audio recordings and digital pictures, captured using a smartphone application, for improving the timeliness, completeness, accuracy, convenience, and security of patient information for handover in EMS; limitations on how much data can be collected in the field due to a wide variety of contextual constraints; and a need to better understand the value of video within the EMS handover context.
Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization, 2019
This paper introduces 'RehaBot', a framework for building adaptive serious games in the c... more This paper introduces 'RehaBot', a framework for building adaptive serious games in the context of telerehabilitation. RehaBot takes advantage of 3D motion tracking and virtual reality devices, to develop an immersive and gamified telerehabilitation environment. A personalized and adaptive gaming system is developed, which allows patients to perform exercises with the help of embedded virtual assistants, hereafter called 'rehab bots', that are dynamically displayed within scenes to guide the patient through the different sets of gestures required to complete the session. These rehab bots have the ability to learn and adapt to the best level of difficulty in real-time based on the user performance. An intelligent alerting and automatic correction technique is incorporated within our engine, so that pre-calculated gesture patterns are correlated and matched with patients' gestures. Consequently, the system estimates the perceived difficulty of gestures by the patient, and automatically adjusts the game behavior to ensure a highly engaging and adaptive gaming experience. Furthermore, multimodal instructions are conveyed to users with details on joints that are not performing as expected, and to guide them towards improving the current gesture. A pilot study has been conducted to prove the usability and effectiveness of our adaptive physiotherapy solution.
Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments, 2019
This paper introduces a framework for building adaptive serious games in the context of telerehab... more This paper introduces a framework for building adaptive serious games in the context of telerehabilitation. This framework takes advantage of 3D motion tracking and virtual reality devices, to present a personalized and adaptive gaming system, which allows patients to perform exercises as suggested by their physiotherapy experts, while being fully engaged in a gamified 3D scene. Virtual characters are dynamically displayed within scenes to guide users through the different set of required gestures. Virtual assistants have the ability to learn and adapt to the best level of difficulty in real-time based on the user performance. An intelligent alerting and automatic correction components are embedded in our engine, so that pre-calculated gesture patterns are correlated and matched with patients' gestures. Consequently, the system estimates the perceived difficulty of gestures by the patient, so that automatic adjustments are executed to ensure a highly engaging and adaptive serious game experience. A pilot study has been conducted to prove the usability and effectiveness of our gamified physiotherapy solution.
Rehabilitative therapy is usually very expensive and confined to specialized rehabilitation cente... more Rehabilitative therapy is usually very expensive and confined to specialized rehabilitation centers or hospitals, leading to slower recovery times for corresponding patients. Therefore, there is a high demand for the development of technology-based personalized solutions to guide and encourage patients towards performing online rehabilitation program that can help them live independently at home. This paper introduces an innovative e-health framework that develops adaptive serious games for people with hand disabilities. The aim of this work is to provide a patient-adaptive environment for the gamification of hand therapies in order to facilitate and encourage rehabilitation issues. Theoretical foundations (i.e., therapy and patient models) and algorithms to match therapy-based hand gestures to navigational movements in 3D space within the serious game environment have been developed. A novel game generation module is introduced, which translates those movements into a 3D therapy-driven route on a real-world map and with different levels of difficulty based on the patient profile and capabilities. In order to enrich the user navigation experience, a 3D spatio-temporal validation region is also generated, which tracks and adjusts the patient movements throughout the session. The gaming environment also creates and adds semantics to different types of attractive and repellent objects in space depending on the difficulty level of the game. Relevant benchmarks to assess the patient interaction with the environment along with a usability and performance testing of our framework are introduced to ensure quantitative as well as qualitative improvements. Trial tests in one disability center were conducted with a total number of five subjects, having hand motor controls problems, who used our gamified physiotherapy solution to help us in measuring the usability and users’ satisfaction levels. The obtained results and feedback from therapists and patients are very encouraging.
Proceedings of the 22nd International Conference on Intelligent User Interfaces Companion, 2017
Cerebral Palsy, trauma, and strokes are common causes for the loss of hand movements and the decr... more Cerebral Palsy, trauma, and strokes are common causes for the loss of hand movements and the decrease in muscle strength for both children and adults. Improving fine motor skills usually involves the synchronization of wrists and fingers by performing appropriate tasks and activities. This demo introduces a novel patient-centered framework for the gamification of hand therapies in order to facilitate and encourage the rehabilitation process. This framework consists of an adaptive therapy-driven 3D environment augmented with our motion-based natural user interface. An intelligent game generator is developed, which translates the patient's gestures into navigational movements with therapy-driven goals, while adapting the level of difficulty based on the patient profile and real-time performance. A comprehensive evaluation and clinical-based assessments were conducted in a local children disability center, and highlights of the results are presented.
2015 IEEE/ACS 12th International Conference of Computer Systems and Applications (AICCSA), 2015
Recommending an optimized path for a large crowd poses a unique challenge to existing routing alg... more Recommending an optimized path for a large crowd poses a unique challenge to existing routing algorithms due to the interactions between users and the dynamic changes over road networks. Industries, researchers and end users show an enormous interest in crowdsourced data comprising social networks and user-generated content to remain updated with their concerns. In this paper, we present a data collection framework that helps users to find optimized routes in a dynamic environment. We have developed a data collection framework to collect dynamic road conditions via a set of location-based services to support a very large Hajj crowd by capturing their locations using smartphones. We also collect geotagged social network data that provides more details about road conditions. The system leverages geotagged crowdsourced information to identify constraints such as accidents, congestions, and roadblocks. Moreover, by continuously collecting real-time geotagged data of moving users, the system can also find the flow of traffic and road conditions. We propose a spatial grid index to compute the optimized path, and to identify the affected users within impact zones. The plan is to test the whole application and back-end server during Hajj 2016, where over three million pilgrims from all over the world gather to perform their rituals.
This research uses multiple research methodologies guided by Information Systems Design Theory (I... more This research uses multiple research methodologies guided by Information Systems Design Theory (ISDT) to design and evaluate a mobile multimedia information system for Emergency Medical Services (EMS). We examined the impact of multimedia information for EMS information exchange and decision-making. A field study was designed and conducted in the Boise, Idaho region for three months to evaluate the system and validate ISDT design propositions. Findings from qualitative analysis illustrated the value of digital images and audio recordings for improving information exchange and augmenting medical decision-making. This paper describes the problem and justification, presents the system design, the pilot test methodology and findings and overall implications and future research directions.
Proceedings of the 23rd ACM international conference on Multimedia, 2015
Aggregating interesting spatio-temporal events of daily life in a diary used to be favorite pasti... more Aggregating interesting spatio-temporal events of daily life in a diary used to be favorite pastime for many individuals. However, crescive use of social media through portable digital devices is replacing traditional diary writing. Thanks to ubiquity of smartphones, people can now add multiple media types in order to record joyful moments and events with a time line. In this paper, we propose a spatio-temporal multimedia authoring tool by leveraging the ubiquity of geo-tagged multimedia that represents activities of interest to the user and allowing users to aggregate the multimedia for generating a digital diary. The authoring tool allows users to aggregate one's own spatial media with that of publicly available geo-tagged multimedia artifacts. The complex digital diary can then be saved into a big data repository for personal use as well as for sharing with a community of interest (COI). During the authoring process, the system allows a novel visualization technique of browsing clustered multiple points of interests (CMPOI) at a single GPS location, thereby exploring POIs in a multi-story building, shopping mall, etc. To facilitate efficient and user friendly discovery of CMPOIs, the authoring environment also supports spatio-temporal multimedia queries such as range and KNN (K nearest neighbor) queries from the big data environment. We plan to incorporate this multimedia authoring tool with our existing large scale multimedia supported crowdsourcing environment that is envisioned to support millions of pilgrims during the Hajj 2015 event.
Proceedings of the 23rd ACM international conference on Multimedia, 2015
Logging on to a system using a conventional keyboard may not be feasible in certain environments,... more Logging on to a system using a conventional keyboard may not be feasible in certain environments, such as, in a surgical operation theatre or in an industrial manufacturing facility. We have developed a multi-sensory gesture based login system that allows a user to access secure information using body gestures. The system can be configured to use different types of gestures according to the type of sensors available to the user. We have proposed a simple scheme to represent all alphanumeric characters required for password entry as gestures within the multi-sensory environment. Our scheme is scalable enough to support sensors that detect a large number of gestures to those that can only accept a few. This allows the system to be used in a variety of situations such as usage by disabled persons with limited ability to perform gestures. We are in the midst of deploying our developed system in a clinical environment.
Proceedings of the 23rd ACM international conference on Multimedia, 2015
Traditional routing algorithms for calculating the fastest or shortest path become ineffective or... more Traditional routing algorithms for calculating the fastest or shortest path become ineffective or difficult to use when both source and destination are dynamic or unknown. To solve the problem, we propose a novel semantic routing system that leverages geo-tagged rich crowdsourced multimedia information such as images, audio, video and text to add semantics to the conventional routing. Our proposed system includes a Semantic Multimedia Routing Algorithm (SMRA) that uses an indexed spatial big data environment to answer multimedia spatiotemporal queries in real-time. The results are customized to the users' smartphone bandwidth and resolution requirements. The system has been designed to be able to handle a very large number of multimedia spatio-temporal requests at any given moment. A proof of concept of the system will be demonstrated through two scenarios. These are 1) multimedia enhanced routing and 2) finding lost individuals in a large crowd using multimedia. We plan to test the system's performance and usability during Hajj 2015, where over four million pilgrims from all over the world gather to perform their rituals.
2015 IEEE International Conference on Big Data (Big Data), 2015
Crowdsourced multimedia data poses several challenges when it is collected, stored, indexed, retr... more Crowdsourced multimedia data poses several challenges when it is collected, stored, indexed, retrieved, and visualized. Examples of crowd source multimedia data are social sensors, vehicle sensors, physical sensors, human sensors, etc. Analyzing such multimodal and diversified crowdsourced data provides very rich understanding about the need of individuals within a crowd. Such understanding makes it possible to tailor services to individuals' needs, also called context-aware services. In this paper, we propose a spatial multimedia big data framework that can collect multimedia data from 1) a very large crowd equipped with multi-sensory smartphones, 2) vehicles, and 3) social networks. A set of multimedia services are offered to users to support their spatio-temporal activities. These include but not limited to 1) simple user interfaces to utilize multimedia services for instant guidance, 2) navigation to points of interests (POI), and 3) efficient and cost effective intra-city rides to users. The big data framework is designed to handle a very large number of multimedia spatio-temporal queries in real-time. The system is a pilot project and will be deployed during the event of Hajj 2015 when over three million pilgrims from all over the world will visit Makkah, Saudi Arabia to perform their Hajj rituals.
Proceedings of the 8th ACM SIGSPATIAL International Workshop on Location-Based Social Networks, 2015
The intensive use of social media through mobile devices has leveraged the development of digital... more The intensive use of social media through mobile devices has leveraged the development of digital diary applications that keep track of social events as well as geotagged multimedia content. In a large crowd where users with cultural diversity perform spatiotemporal activities, such geotagged multimedia content facilitates users' navigation through points of interest (POI) based on their preferences. This work presents a crowdsourced geo-spatial multimedia data aggregation tool that allows users to develop diary chapters relevant to forthcoming users' spatio-temporal activities. Our proposed solution provides users with the ability to add POIs through an authoring environment with multiple dimensions, such as spatio-temporal filters, multimedia categories, and event types. Specific application domains such as emergency situations, leisure trips, journalism, and tourism can take benefit of this technique. This authoring environment also visualizes geo-spatial multimedia content for collocated points of interest (CPOI) with moving users' timelines. We plan to integrate our proposed authoring environment as a proof of concept into our existing large-scale crowdsourcing environment that is envisioned to support millions of users during the Hajj 2015 event.
Mobile crowdsourcing offers great opportunities when considering users' contexts with the aim... more Mobile crowdsourcing offers great opportunities when considering users' contexts with the aim of recognizing their interests, locations, history, and activities. This paper introduces a mobile data collection scheme that takes into account users' spatio-temporal activities, as well as other contextual dimensions towards the design of quality and context-aware crowdsourcing framework. We propose a framework that offers a set of cloud-based services as an incentive to support a very large crowd in a real scenario by capturing their contexts using smartphones. The proposed framework captures data from an individual, validates its quality by different measures to define his context and provides a set of adapted services to end users. The framework allows being in touch with a subset of one's community of interest. We leverage an approach requiring restricted spatio-temporal zoning model that applies to the scenario where millions of users need to perform some activities in a...
Proceedings of the 1st ACM SIGSPATIAL International Workshop on Privacy in Geographic Information Collection and Analysis - GeoPrivacy '14, 2014
ABSTRACT Protecting patient geo-privacy while allowing for valid geographic analyses of the data ... more ABSTRACT Protecting patient geo-privacy while allowing for valid geographic analyses of the data is a major challenge [1]. As a consequence, a variety of methods have been introduced to mask patients' locational information, also called geo-masking methods [2]. This study assessed the five main geo-masking methods as cited by [3] in terms of re-identification risk and performance. These five methods are Random Direction and Fixed Radius, Random Perturbation within a Circle, Gaussian Displacement, Donut Masking, and Bimodal Gaussian Displacement. Based on the assessment, the study highlighted two major gaps in the design of these geo-masking methods. First, all five geo-masking methods used only population density in calculating the displacement distances between the original locations of points and their new locations. However, other criteria that might be as important as population density were not considered in designing these five methods. These include data sensitivity, research types, quasi-indicator availability, previously generated maps availability, end-users' types, and the possibility of temporal synergy of data. Second, the Donut Masking and the Bimodal Gaussian Displacement methods were found to be superior in terms of minimizing re-identifying risks, but they were also found to be consuming much more processing power compared to the other three geo-masking methods. To address these gaps, this study proposed a new geo-masking method, called the "Triangular Displacement". The primary design, development, and evaluation of the Triangular Displacement method were based on the Design Science Research (DSR) Process Model [4], also known as DSRM. The expected next step is to implement the resultant geo-masking method as a tool to help healthcare data guardians de-identify large sets of PHR automatically. A pilot study with a large Southern Californian healthcare provider has been outlined to examine the efficacy of the developed solution.
Background: Trauma is the leading cause of death for patients between 1-44 years old and the fift... more Background: Trauma is the leading cause of death for patients between 1-44 years old and the fifth leading source of mortality in the United States [1]. The most common mechanism contributing to trauma injury is motor vehicle crashes (MVC). MVC’s account for approximately 42,000 fatalities per year [2], of which 60% (70% for Minnesota alone) are on rural roads. Timely and effective emergency medical services (EMS), is an important strategy for reducing the likelihood of death, disability and economic consequences of MVC’s. However, critical information gaps exist between the time when an incident is reported and when the patient receives definitive care at an emergency department (ED) or trauma center [3]. For example, prior studies have shown that only 49% of EMS agencies report collecting electronic data at the patient’s side [3], and verbal handoffs occur for only 44% of EMS patients [4]. Objectives: Existing communication methods between EMS and ED were assessed and a new alternative mobile computing centric strategy was designed, developed, and implemented. A design science research (DSR) approach was employed to design a solution according to the needs of two different core user groups – EMS and ED staff. The design goals were to improve trauma communications, emergency medical decision-making, and patient medical outcomes. The system, CrashHelp, consists of a mobile smartphone equipped with an application where EMS staff (paramedics, EMT’s) can securely capture pictures, digital audio recordings, video, patient and incident data and send it securely to the ED. Data collected includes patient name, gender, date of birth, origin of incident, mechanism of injury, trauma level, interventions, and protocol driven trauma specific data including the Glasgow coma scale (GCS), the highest heart rate, and the lowest blood pressure. Methods: An exploratory field test was conducted in rural Minnesota to assess usage and practitioner perceptions of usefulness. Staging scenarios were conducted at Tri-County Hospital (T-CH) in Wadena, MN, to model crash, response, information obtained and transmitted to the ED. Group discussions were conducted with EMS and ED staff to validate the trauma scenarios, determine trauma use cases, and implement the system. A three-month live pilot test was then conducted. Eleven real MVC trauma runs were selected and used for evaluating the system for its perceived impact on trauma communications, emergency medical decision-making, and medical outcomes. A total of 11 participants at T-CH were interviewed including four paramedics, six charge nurses, and one physician. Results: Key findings include improved trauma information collection by on-scene EMS personnel; improved trauma communication between pre-hospital transport and hospital organizations (ED/Trauma Centers); and improved hospital resource utilization by hospital personnel. Conclusion: The pilot provided an initial exploratory assessment and proof of concept that such a system could aid in improving the timeliness, completeness, accuracy, and richness of a trauma report from EMS to the ED/Trauma Centers. Additional research is needed to further determine the clinical value of using asynchronous data communications for point-of-care decisions; understanding the secondary use of data and value for clinical quality improvement and trauma audits; and integrating the data into hospital systems. [1] Peitzman, A. B., Fabian, T. C., Rhodes, M., Yealy, D. M. and Schwab, C. W. The trauma manual: trauma and acute care surgery. Lippincott Williams & Wilkins, 2012. [2] Peterson, J. A. Medical-Related Causes of Death in the United States. ACSM's Health & Fitness Journal, 17, 3 2013), 44 10.1249/FIT.1240b1013e3182905d3182912. [3] Schooley, B., Murad, A., Abed, Y. and Horan, T. A mHealth System for Patient Handover in Emergency Medical Services. [4] Benner, J. P., Hilton, J., Carr, G., Robbins, K., Schutt, R. C., Borloz, M. P., Alibertis, K., Sojka, B., Hudson, K., Haugh, D. and Brady, W. Information transfer from prehospital to ED health care providers. The American Journal of Emergency Medicine, 26, 2 2008), 233-235. []
In recent years, we have witnessed a growing adoption of serious games in telerehabilitation by t... more In recent years, we have witnessed a growing adoption of serious games in telerehabilitation by taking advantage of advanced multimedia technologies such as motion capture and virtual reality devices. Current serious game solutions for telerehabilitation suffer form lack of personalization and adaptiveness to patients’ needs and performance. This paper introduces “RehaBot”, a framework for adaptive generation of personalized serious games in the context of remote rehabilitation, using 3D motion tracking and virtual reality environments. A personalized and versatile gaming platform with embedded virtual assistants, called “Rehab bots”, is created. Utilizing these rehab bots, all workout session scenes will include a guide with various sets of motions to direct patients towards performing the prescribed exercises correctly. Furthermore, the rehab bots employ a robust technique to adjust the workout difficulty level in real-time to match the patients’ performance. This technique correl...
This research uses multiple research methodologies guided by Information Systems Design Theory (I... more This research uses multiple research methodologies guided by Information Systems Design Theory (ISDT) to design and evaluate a mobile multimedia information system for Emergency Medical Services (EMS). We examined the impact of multimedia information for EMS information exchange and decision-making. A field study was designed and conducted in the Boise, Idaho region for three months to evaluate the system and validate ISDT design propositions. Findings from
This research uses multiple methods to investigate the use of an enterprise mobile multimedia inf... more This research uses multiple methods to investigate the use of an enterprise mobile multimedia information system aimed at improving handover of patient and emergency incident information from pre-hospital Emergency Medical Services (EMS) to hospital emergency department providers. A field study was conducted across EMS and hospital organizations in the Boise, Idaho region of the United States for three months to examine use of the system and to assess practitioner perspectives. Findings include perceived benefits and challenges to using digital audio recordings and digital pictures, captured using a smartphone application, for improving the timeliness, completeness, accuracy, convenience, and security of patient information for handover in EMS; limitations on how much data can be collected in the field due to a wide variety of contextual constraints; and a need to better understand the value of video within the EMS handover context.
Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization, 2019
This paper introduces 'RehaBot', a framework for building adaptive serious games in the c... more This paper introduces 'RehaBot', a framework for building adaptive serious games in the context of telerehabilitation. RehaBot takes advantage of 3D motion tracking and virtual reality devices, to develop an immersive and gamified telerehabilitation environment. A personalized and adaptive gaming system is developed, which allows patients to perform exercises with the help of embedded virtual assistants, hereafter called 'rehab bots', that are dynamically displayed within scenes to guide the patient through the different sets of gestures required to complete the session. These rehab bots have the ability to learn and adapt to the best level of difficulty in real-time based on the user performance. An intelligent alerting and automatic correction technique is incorporated within our engine, so that pre-calculated gesture patterns are correlated and matched with patients' gestures. Consequently, the system estimates the perceived difficulty of gestures by the patient, and automatically adjusts the game behavior to ensure a highly engaging and adaptive gaming experience. Furthermore, multimodal instructions are conveyed to users with details on joints that are not performing as expected, and to guide them towards improving the current gesture. A pilot study has been conducted to prove the usability and effectiveness of our adaptive physiotherapy solution.
Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments, 2019
This paper introduces a framework for building adaptive serious games in the context of telerehab... more This paper introduces a framework for building adaptive serious games in the context of telerehabilitation. This framework takes advantage of 3D motion tracking and virtual reality devices, to present a personalized and adaptive gaming system, which allows patients to perform exercises as suggested by their physiotherapy experts, while being fully engaged in a gamified 3D scene. Virtual characters are dynamically displayed within scenes to guide users through the different set of required gestures. Virtual assistants have the ability to learn and adapt to the best level of difficulty in real-time based on the user performance. An intelligent alerting and automatic correction components are embedded in our engine, so that pre-calculated gesture patterns are correlated and matched with patients' gestures. Consequently, the system estimates the perceived difficulty of gestures by the patient, so that automatic adjustments are executed to ensure a highly engaging and adaptive serious game experience. A pilot study has been conducted to prove the usability and effectiveness of our gamified physiotherapy solution.
Rehabilitative therapy is usually very expensive and confined to specialized rehabilitation cente... more Rehabilitative therapy is usually very expensive and confined to specialized rehabilitation centers or hospitals, leading to slower recovery times for corresponding patients. Therefore, there is a high demand for the development of technology-based personalized solutions to guide and encourage patients towards performing online rehabilitation program that can help them live independently at home. This paper introduces an innovative e-health framework that develops adaptive serious games for people with hand disabilities. The aim of this work is to provide a patient-adaptive environment for the gamification of hand therapies in order to facilitate and encourage rehabilitation issues. Theoretical foundations (i.e., therapy and patient models) and algorithms to match therapy-based hand gestures to navigational movements in 3D space within the serious game environment have been developed. A novel game generation module is introduced, which translates those movements into a 3D therapy-driven route on a real-world map and with different levels of difficulty based on the patient profile and capabilities. In order to enrich the user navigation experience, a 3D spatio-temporal validation region is also generated, which tracks and adjusts the patient movements throughout the session. The gaming environment also creates and adds semantics to different types of attractive and repellent objects in space depending on the difficulty level of the game. Relevant benchmarks to assess the patient interaction with the environment along with a usability and performance testing of our framework are introduced to ensure quantitative as well as qualitative improvements. Trial tests in one disability center were conducted with a total number of five subjects, having hand motor controls problems, who used our gamified physiotherapy solution to help us in measuring the usability and users’ satisfaction levels. The obtained results and feedback from therapists and patients are very encouraging.
Proceedings of the 22nd International Conference on Intelligent User Interfaces Companion, 2017
Cerebral Palsy, trauma, and strokes are common causes for the loss of hand movements and the decr... more Cerebral Palsy, trauma, and strokes are common causes for the loss of hand movements and the decrease in muscle strength for both children and adults. Improving fine motor skills usually involves the synchronization of wrists and fingers by performing appropriate tasks and activities. This demo introduces a novel patient-centered framework for the gamification of hand therapies in order to facilitate and encourage the rehabilitation process. This framework consists of an adaptive therapy-driven 3D environment augmented with our motion-based natural user interface. An intelligent game generator is developed, which translates the patient's gestures into navigational movements with therapy-driven goals, while adapting the level of difficulty based on the patient profile and real-time performance. A comprehensive evaluation and clinical-based assessments were conducted in a local children disability center, and highlights of the results are presented.
2015 IEEE/ACS 12th International Conference of Computer Systems and Applications (AICCSA), 2015
Recommending an optimized path for a large crowd poses a unique challenge to existing routing alg... more Recommending an optimized path for a large crowd poses a unique challenge to existing routing algorithms due to the interactions between users and the dynamic changes over road networks. Industries, researchers and end users show an enormous interest in crowdsourced data comprising social networks and user-generated content to remain updated with their concerns. In this paper, we present a data collection framework that helps users to find optimized routes in a dynamic environment. We have developed a data collection framework to collect dynamic road conditions via a set of location-based services to support a very large Hajj crowd by capturing their locations using smartphones. We also collect geotagged social network data that provides more details about road conditions. The system leverages geotagged crowdsourced information to identify constraints such as accidents, congestions, and roadblocks. Moreover, by continuously collecting real-time geotagged data of moving users, the system can also find the flow of traffic and road conditions. We propose a spatial grid index to compute the optimized path, and to identify the affected users within impact zones. The plan is to test the whole application and back-end server during Hajj 2016, where over three million pilgrims from all over the world gather to perform their rituals.
This research uses multiple research methodologies guided by Information Systems Design Theory (I... more This research uses multiple research methodologies guided by Information Systems Design Theory (ISDT) to design and evaluate a mobile multimedia information system for Emergency Medical Services (EMS). We examined the impact of multimedia information for EMS information exchange and decision-making. A field study was designed and conducted in the Boise, Idaho region for three months to evaluate the system and validate ISDT design propositions. Findings from qualitative analysis illustrated the value of digital images and audio recordings for improving information exchange and augmenting medical decision-making. This paper describes the problem and justification, presents the system design, the pilot test methodology and findings and overall implications and future research directions.
Proceedings of the 23rd ACM international conference on Multimedia, 2015
Aggregating interesting spatio-temporal events of daily life in a diary used to be favorite pasti... more Aggregating interesting spatio-temporal events of daily life in a diary used to be favorite pastime for many individuals. However, crescive use of social media through portable digital devices is replacing traditional diary writing. Thanks to ubiquity of smartphones, people can now add multiple media types in order to record joyful moments and events with a time line. In this paper, we propose a spatio-temporal multimedia authoring tool by leveraging the ubiquity of geo-tagged multimedia that represents activities of interest to the user and allowing users to aggregate the multimedia for generating a digital diary. The authoring tool allows users to aggregate one's own spatial media with that of publicly available geo-tagged multimedia artifacts. The complex digital diary can then be saved into a big data repository for personal use as well as for sharing with a community of interest (COI). During the authoring process, the system allows a novel visualization technique of browsing clustered multiple points of interests (CMPOI) at a single GPS location, thereby exploring POIs in a multi-story building, shopping mall, etc. To facilitate efficient and user friendly discovery of CMPOIs, the authoring environment also supports spatio-temporal multimedia queries such as range and KNN (K nearest neighbor) queries from the big data environment. We plan to incorporate this multimedia authoring tool with our existing large scale multimedia supported crowdsourcing environment that is envisioned to support millions of pilgrims during the Hajj 2015 event.
Proceedings of the 23rd ACM international conference on Multimedia, 2015
Logging on to a system using a conventional keyboard may not be feasible in certain environments,... more Logging on to a system using a conventional keyboard may not be feasible in certain environments, such as, in a surgical operation theatre or in an industrial manufacturing facility. We have developed a multi-sensory gesture based login system that allows a user to access secure information using body gestures. The system can be configured to use different types of gestures according to the type of sensors available to the user. We have proposed a simple scheme to represent all alphanumeric characters required for password entry as gestures within the multi-sensory environment. Our scheme is scalable enough to support sensors that detect a large number of gestures to those that can only accept a few. This allows the system to be used in a variety of situations such as usage by disabled persons with limited ability to perform gestures. We are in the midst of deploying our developed system in a clinical environment.
Proceedings of the 23rd ACM international conference on Multimedia, 2015
Traditional routing algorithms for calculating the fastest or shortest path become ineffective or... more Traditional routing algorithms for calculating the fastest or shortest path become ineffective or difficult to use when both source and destination are dynamic or unknown. To solve the problem, we propose a novel semantic routing system that leverages geo-tagged rich crowdsourced multimedia information such as images, audio, video and text to add semantics to the conventional routing. Our proposed system includes a Semantic Multimedia Routing Algorithm (SMRA) that uses an indexed spatial big data environment to answer multimedia spatiotemporal queries in real-time. The results are customized to the users' smartphone bandwidth and resolution requirements. The system has been designed to be able to handle a very large number of multimedia spatio-temporal requests at any given moment. A proof of concept of the system will be demonstrated through two scenarios. These are 1) multimedia enhanced routing and 2) finding lost individuals in a large crowd using multimedia. We plan to test the system's performance and usability during Hajj 2015, where over four million pilgrims from all over the world gather to perform their rituals.
2015 IEEE International Conference on Big Data (Big Data), 2015
Crowdsourced multimedia data poses several challenges when it is collected, stored, indexed, retr... more Crowdsourced multimedia data poses several challenges when it is collected, stored, indexed, retrieved, and visualized. Examples of crowd source multimedia data are social sensors, vehicle sensors, physical sensors, human sensors, etc. Analyzing such multimodal and diversified crowdsourced data provides very rich understanding about the need of individuals within a crowd. Such understanding makes it possible to tailor services to individuals' needs, also called context-aware services. In this paper, we propose a spatial multimedia big data framework that can collect multimedia data from 1) a very large crowd equipped with multi-sensory smartphones, 2) vehicles, and 3) social networks. A set of multimedia services are offered to users to support their spatio-temporal activities. These include but not limited to 1) simple user interfaces to utilize multimedia services for instant guidance, 2) navigation to points of interests (POI), and 3) efficient and cost effective intra-city rides to users. The big data framework is designed to handle a very large number of multimedia spatio-temporal queries in real-time. The system is a pilot project and will be deployed during the event of Hajj 2015 when over three million pilgrims from all over the world will visit Makkah, Saudi Arabia to perform their Hajj rituals.
Proceedings of the 8th ACM SIGSPATIAL International Workshop on Location-Based Social Networks, 2015
The intensive use of social media through mobile devices has leveraged the development of digital... more The intensive use of social media through mobile devices has leveraged the development of digital diary applications that keep track of social events as well as geotagged multimedia content. In a large crowd where users with cultural diversity perform spatiotemporal activities, such geotagged multimedia content facilitates users' navigation through points of interest (POI) based on their preferences. This work presents a crowdsourced geo-spatial multimedia data aggregation tool that allows users to develop diary chapters relevant to forthcoming users' spatio-temporal activities. Our proposed solution provides users with the ability to add POIs through an authoring environment with multiple dimensions, such as spatio-temporal filters, multimedia categories, and event types. Specific application domains such as emergency situations, leisure trips, journalism, and tourism can take benefit of this technique. This authoring environment also visualizes geo-spatial multimedia content for collocated points of interest (CPOI) with moving users' timelines. We plan to integrate our proposed authoring environment as a proof of concept into our existing large-scale crowdsourcing environment that is envisioned to support millions of users during the Hajj 2015 event.
Mobile crowdsourcing offers great opportunities when considering users' contexts with the aim... more Mobile crowdsourcing offers great opportunities when considering users' contexts with the aim of recognizing their interests, locations, history, and activities. This paper introduces a mobile data collection scheme that takes into account users' spatio-temporal activities, as well as other contextual dimensions towards the design of quality and context-aware crowdsourcing framework. We propose a framework that offers a set of cloud-based services as an incentive to support a very large crowd in a real scenario by capturing their contexts using smartphones. The proposed framework captures data from an individual, validates its quality by different measures to define his context and provides a set of adapted services to end users. The framework allows being in touch with a subset of one's community of interest. We leverage an approach requiring restricted spatio-temporal zoning model that applies to the scenario where millions of users need to perform some activities in a...
Proceedings of the 1st ACM SIGSPATIAL International Workshop on Privacy in Geographic Information Collection and Analysis - GeoPrivacy '14, 2014
ABSTRACT Protecting patient geo-privacy while allowing for valid geographic analyses of the data ... more ABSTRACT Protecting patient geo-privacy while allowing for valid geographic analyses of the data is a major challenge [1]. As a consequence, a variety of methods have been introduced to mask patients' locational information, also called geo-masking methods [2]. This study assessed the five main geo-masking methods as cited by [3] in terms of re-identification risk and performance. These five methods are Random Direction and Fixed Radius, Random Perturbation within a Circle, Gaussian Displacement, Donut Masking, and Bimodal Gaussian Displacement. Based on the assessment, the study highlighted two major gaps in the design of these geo-masking methods. First, all five geo-masking methods used only population density in calculating the displacement distances between the original locations of points and their new locations. However, other criteria that might be as important as population density were not considered in designing these five methods. These include data sensitivity, research types, quasi-indicator availability, previously generated maps availability, end-users' types, and the possibility of temporal synergy of data. Second, the Donut Masking and the Bimodal Gaussian Displacement methods were found to be superior in terms of minimizing re-identifying risks, but they were also found to be consuming much more processing power compared to the other three geo-masking methods. To address these gaps, this study proposed a new geo-masking method, called the "Triangular Displacement". The primary design, development, and evaluation of the Triangular Displacement method were based on the Design Science Research (DSR) Process Model [4], also known as DSRM. The expected next step is to implement the resultant geo-masking method as a tool to help healthcare data guardians de-identify large sets of PHR automatically. A pilot study with a large Southern Californian healthcare provider has been outlined to examine the efficacy of the developed solution.
Background: Trauma is the leading cause of death for patients between 1-44 years old and the fift... more Background: Trauma is the leading cause of death for patients between 1-44 years old and the fifth leading source of mortality in the United States [1]. The most common mechanism contributing to trauma injury is motor vehicle crashes (MVC). MVC’s account for approximately 42,000 fatalities per year [2], of which 60% (70% for Minnesota alone) are on rural roads. Timely and effective emergency medical services (EMS), is an important strategy for reducing the likelihood of death, disability and economic consequences of MVC’s. However, critical information gaps exist between the time when an incident is reported and when the patient receives definitive care at an emergency department (ED) or trauma center [3]. For example, prior studies have shown that only 49% of EMS agencies report collecting electronic data at the patient’s side [3], and verbal handoffs occur for only 44% of EMS patients [4]. Objectives: Existing communication methods between EMS and ED were assessed and a new alternative mobile computing centric strategy was designed, developed, and implemented. A design science research (DSR) approach was employed to design a solution according to the needs of two different core user groups – EMS and ED staff. The design goals were to improve trauma communications, emergency medical decision-making, and patient medical outcomes. The system, CrashHelp, consists of a mobile smartphone equipped with an application where EMS staff (paramedics, EMT’s) can securely capture pictures, digital audio recordings, video, patient and incident data and send it securely to the ED. Data collected includes patient name, gender, date of birth, origin of incident, mechanism of injury, trauma level, interventions, and protocol driven trauma specific data including the Glasgow coma scale (GCS), the highest heart rate, and the lowest blood pressure. Methods: An exploratory field test was conducted in rural Minnesota to assess usage and practitioner perceptions of usefulness. Staging scenarios were conducted at Tri-County Hospital (T-CH) in Wadena, MN, to model crash, response, information obtained and transmitted to the ED. Group discussions were conducted with EMS and ED staff to validate the trauma scenarios, determine trauma use cases, and implement the system. A three-month live pilot test was then conducted. Eleven real MVC trauma runs were selected and used for evaluating the system for its perceived impact on trauma communications, emergency medical decision-making, and medical outcomes. A total of 11 participants at T-CH were interviewed including four paramedics, six charge nurses, and one physician. Results: Key findings include improved trauma information collection by on-scene EMS personnel; improved trauma communication between pre-hospital transport and hospital organizations (ED/Trauma Centers); and improved hospital resource utilization by hospital personnel. Conclusion: The pilot provided an initial exploratory assessment and proof of concept that such a system could aid in improving the timeliness, completeness, accuracy, and richness of a trauma report from EMS to the ED/Trauma Centers. Additional research is needed to further determine the clinical value of using asynchronous data communications for point-of-care decisions; understanding the secondary use of data and value for clinical quality improvement and trauma audits; and integrating the data into hospital systems. [1] Peitzman, A. B., Fabian, T. C., Rhodes, M., Yealy, D. M. and Schwab, C. W. The trauma manual: trauma and acute care surgery. Lippincott Williams & Wilkins, 2012. [2] Peterson, J. A. Medical-Related Causes of Death in the United States. ACSM's Health & Fitness Journal, 17, 3 2013), 44 10.1249/FIT.1240b1013e3182905d3182912. [3] Schooley, B., Murad, A., Abed, Y. and Horan, T. A mHealth System for Patient Handover in Emergency Medical Services. [4] Benner, J. P., Hilton, J., Carr, G., Robbins, K., Schutt, R. C., Borloz, M. P., Alibertis, K., Sojka, B., Hudson, K., Haugh, D. and Brady, W. Information transfer from prehospital to ED health care providers. The American Journal of Emergency Medicine, 26, 2 2008), 233-235. []
Uploads
Papers by Abdullah Murad