In this paper we present a Finger Walking in Place (FWIP) interaction technique that allows a use... more In this paper we present a Finger Walking in Place (FWIP) interaction technique that allows a user to travel in a virtual world as her/his bare fingers slide on a multi-touch sensitive surface. Traveling is basically realized by translating and rotating the user's viewpoint in the virtual world. The user can translate and rotate a viewpoint by moving her/his fingers in place. Currently, our FWIP technique can be used to navigate in a plane but it can be extended to navigate in the third axis, so that the user can move to any direction in a 3D virtual world. Since our FWIP technique only uses bare fingers and a multi-touch device, finger motions are not precisely detected, especially compared with the use of data gloves or similar sensing devices. However, our experiments show that FWIP can be used as a novel traveling technique even without accurate motion detection. Our experiment tasks include finding and reaching the target(s) with FWIP, and the participants successfully completed the tasks. The experiments illustrate our efforts to make the FWIP technique robust as a scaled-down walking-in-place locomotion technique, so that it can be used as a reliable traveling technique.
This paper describes a sensor-fusion-based wireless walking-inplace (WIP) interaction technique. ... more This paper describes a sensor-fusion-based wireless walking-inplace (WIP) interaction technique. We devised a new humanwalking detection algorithm that is based on a sensor-fusion using both acceleration and magnetic sensors integrated within a smart phone. Our sensor-fusion approach can be useful for the cases when the detection capability of a single sensor is limited to a certain range of walking speeds, when a system power source is limited, and/or when computation power is limited. The proposed algorithm is versatile enough to handle possible data-loss and random delay in the wireless communication environment, resulting in reduced wireless communication load and computation overhead. The initial study demonstrated that the algorithm can detect dynamic speeds of human walking. The algorithm can be implemented on any mobile device equipped with magnetic and acceleration sensors.
This paper describes a 3D, tangible user interface system and related interaction techniques for ... more This paper describes a 3D, tangible user interface system and related interaction techniques for a CAVE environment. The developed system is based on off-the shelf software and hardware component to provide 3D input data for applications. A CAVE installation with four sides (three walls and a floor) is used as a display and interaction space. Interaction tasks (navigation, selection, manipulation) are performed using interaction techniques based on manipulation of physical objects (props). All virtual objects are directly manipulated using the corresponding props to which AR physical markers are attached. Each physical marker corresponds to a specific virtual object. The floor projection (a white rectangle overlaid on top of the application generated video stream) or a directional light is used to create illumination necessary for computer vision based marker detection using ARToolKit. Initial evaluation results are positive and provide directions for future research.
This thesis presents a new 3-dimensional (3D) user interface system for a Cave Automated Virtual ... more This thesis presents a new 3-dimensional (3D) user interface system for a Cave Automated Virtual Environment (CAVE TM ) application, based on Virtual Reality (VR), Augmented Reality (AR), and Tangible User Interface (TUI). We explore fundamental 3D interaction tasks with our user interface for the CAVE TM system. User interface (UI) is comprised of a specific set of components, including input/output devices and interaction techniques. Our approach is based on TUIs using ARToolKit, which is currently the most popular toolkit for use in AR projects. Physical objects (props) are used as input devices instead of any tethered electromagnetic trackers. An off-the-shelf webcam is used to get tracking input data. A unique pattern marker is attached to the prop, which is easily and simply tracked by ARToolKit. Our interface system is developed on CAVE TM infrastructure, which is a semi-immersive environment. All virtual objects are directly manipulated with props, each of which corresponds to a certain virtual object. To navigate, the user can move the background itself, while virtual objects remain in place. The user can actually feel the prop's movement through the virtual space. Thus, fundamental 3D interaction tasks such as object selection, object manipulation, and navigation are performed with our interface. To feel immersion, the user is allowed to wear stereoscopic glasses with a head tracker. This is the only tethered device for our work. Since our interface is based on tangible input tools, seamless transition between one and two-handed operation is provided. We went through three design phases to achieve better task performance. In the first phase, we conducted the pilot study, focusing on the question whether or not this approach is applicable to 3D immersive environments. After the pilot study, we redesigned props and developed ARBox. ARBox is used for as interaction space while the CAVE TM system is only used for display space. In this phase, we also developed interaction techniques for fundamental 3D interaction tasks. Our summative user evaluation was conducted with ARDesk, which is redesigned after our formative user evaluation. Two user studies aim to get user feedback and to improve interaction techniques as well as interface tools' design. The results from our user studies show that our interface can be intuitively and naturally applied to 3D immersive environments even though there are still some issues with our system design. This thesis shows that effective interactions in a CAVE TM system can be generated using AR technique and tangible objects. Most of all, I thank my family,far from here, for always supporting me without question.
munger1~ is a new and enhanced version of a powerful Max/MSP real-time granular synthesis externa... more munger1~ is a new and enhanced version of a powerful Max/MSP real-time granular synthesis external found in the PeRColate library. Apart from added features and optimizations, including ability to generate theoretically unlimited number of grains per second using real-time input, munger1~ also offers GPL-licensed platformtransparent code using the flext library. Thus, munger1~ is compatible with both Max/MSP and Pure-Data without any alterations to the source. Due to object's complexity, as well as a significantly less common porting path from the Max/MSP/C to a platformagnostic C++/flext, this project has resulted in a number of improvements in flext build scripts and supporting documentation. It has also generated an invaluable list of caveats which are presented in this paper in hope to foster more platform-agnostic object design and porting efforts. Since its release, munger1~ has been featured in two interactive multimedia creations whose technical and artistic impact is also addressed in this paper.
Spatial knowledge, necessary for efficient navigation, comprises route knowledge (memory of landm... more Spatial knowledge, necessary for efficient navigation, comprises route knowledge (memory of landmarks along a route) and survey knowledge (overall representation like a map). Virtual environments (VEs) have been suggested as a power tool for understanding some issues associated with human navigation, such as spatial knowledge acquisition. The Finger-Walking-in-Place (FWIP) interaction technique is a locomotion technique for navigation tasks in immersive virtual environments (IVEs). The FWIP was designed to map a human's embodied ability overlearned by natural walking for navigation, to finger-based interaction technique. Its implementation on Lemur and iPhone/iPod Touch devices was evaluated in our previous studies. In this paper, we present a comparative study of the joystick's flying technique versus the FWIP. Our experiment results show that the FWIP results in better performance than the joystick's flying for route knowledge acquisition in our maze navigation tasks.
iPhone and iPod Touch are multi-touch handheld devices that provide new possibilities for interac... more iPhone and iPod Touch are multi-touch handheld devices that provide new possibilities for interaction techniques. We describe iPhone/iPod Touch implementation of a navigation interaction technique originally developed for a larger multi-touch device (i.e. Lemur). The interaction technique implemented on an iPhone/iPod Touch was used for navigation tasks in a CAVE virtual environment. We performed a pilot study to measure the control accuracy and to observe how human subjects respond to the interaction technique on the iPhone and iPod Touch devices. We used the preliminary results to improve the design of the interaction technique.
In this paper we present a Finger Walking in Place (FWIP) interaction technique that allows a use... more In this paper we present a Finger Walking in Place (FWIP) interaction technique that allows a user to travel in a virtual world as her/his bare fingers slide on a multi-touch sensitive surface. Traveling is basically realized by translating and rotating the user's viewpoint in the virtual world. The user can translate and rotate a viewpoint by moving her/his fingers in place. Currently, our FWIP technique can be used to navigate in a plane but it can be extended to navigate in the third axis, so that the user can move to any direction in a 3D virtual world. Since our FWIP technique only uses bare fingers and a multi-touch device, finger motions are not precisely detected, especially compared with the use of data gloves or similar sensing devices. However, our experiments show that FWIP can be used as a novel traveling technique even without accurate motion detection. Our experiment tasks include finding and reaching the target(s) with FWIP, and the participants successfully completed the tasks. The experiments illustrate our efforts to make the FWIP technique robust as a scaled-down walking-in-place locomotion technique, so that it can be used as a reliable traveling technique.
This paper describes a sensor-fusion-based wireless walking-inplace (WIP) interaction technique. ... more This paper describes a sensor-fusion-based wireless walking-inplace (WIP) interaction technique. We devised a new humanwalking detection algorithm that is based on a sensor-fusion using both acceleration and magnetic sensors integrated within a smart phone. Our sensor-fusion approach can be useful for the cases when the detection capability of a single sensor is limited to a certain range of walking speeds, when a system power source is limited, and/or when computation power is limited. The proposed algorithm is versatile enough to handle possible data-loss and random delay in the wireless communication environment, resulting in reduced wireless communication load and computation overhead. The initial study demonstrated that the algorithm can detect dynamic speeds of human walking. The algorithm can be implemented on any mobile device equipped with magnetic and acceleration sensors.
This paper describes a 3D, tangible user interface system and related interaction techniques for ... more This paper describes a 3D, tangible user interface system and related interaction techniques for a CAVE environment. The developed system is based on off-the shelf software and hardware component to provide 3D input data for applications. A CAVE installation with four sides (three walls and a floor) is used as a display and interaction space. Interaction tasks (navigation, selection, manipulation) are performed using interaction techniques based on manipulation of physical objects (props). All virtual objects are directly manipulated using the corresponding props to which AR physical markers are attached. Each physical marker corresponds to a specific virtual object. The floor projection (a white rectangle overlaid on top of the application generated video stream) or a directional light is used to create illumination necessary for computer vision based marker detection using ARToolKit. Initial evaluation results are positive and provide directions for future research.
This thesis presents a new 3-dimensional (3D) user interface system for a Cave Automated Virtual ... more This thesis presents a new 3-dimensional (3D) user interface system for a Cave Automated Virtual Environment (CAVE TM ) application, based on Virtual Reality (VR), Augmented Reality (AR), and Tangible User Interface (TUI). We explore fundamental 3D interaction tasks with our user interface for the CAVE TM system. User interface (UI) is comprised of a specific set of components, including input/output devices and interaction techniques. Our approach is based on TUIs using ARToolKit, which is currently the most popular toolkit for use in AR projects. Physical objects (props) are used as input devices instead of any tethered electromagnetic trackers. An off-the-shelf webcam is used to get tracking input data. A unique pattern marker is attached to the prop, which is easily and simply tracked by ARToolKit. Our interface system is developed on CAVE TM infrastructure, which is a semi-immersive environment. All virtual objects are directly manipulated with props, each of which corresponds to a certain virtual object. To navigate, the user can move the background itself, while virtual objects remain in place. The user can actually feel the prop's movement through the virtual space. Thus, fundamental 3D interaction tasks such as object selection, object manipulation, and navigation are performed with our interface. To feel immersion, the user is allowed to wear stereoscopic glasses with a head tracker. This is the only tethered device for our work. Since our interface is based on tangible input tools, seamless transition between one and two-handed operation is provided. We went through three design phases to achieve better task performance. In the first phase, we conducted the pilot study, focusing on the question whether or not this approach is applicable to 3D immersive environments. After the pilot study, we redesigned props and developed ARBox. ARBox is used for as interaction space while the CAVE TM system is only used for display space. In this phase, we also developed interaction techniques for fundamental 3D interaction tasks. Our summative user evaluation was conducted with ARDesk, which is redesigned after our formative user evaluation. Two user studies aim to get user feedback and to improve interaction techniques as well as interface tools' design. The results from our user studies show that our interface can be intuitively and naturally applied to 3D immersive environments even though there are still some issues with our system design. This thesis shows that effective interactions in a CAVE TM system can be generated using AR technique and tangible objects. Most of all, I thank my family,far from here, for always supporting me without question.
munger1~ is a new and enhanced version of a powerful Max/MSP real-time granular synthesis externa... more munger1~ is a new and enhanced version of a powerful Max/MSP real-time granular synthesis external found in the PeRColate library. Apart from added features and optimizations, including ability to generate theoretically unlimited number of grains per second using real-time input, munger1~ also offers GPL-licensed platformtransparent code using the flext library. Thus, munger1~ is compatible with both Max/MSP and Pure-Data without any alterations to the source. Due to object's complexity, as well as a significantly less common porting path from the Max/MSP/C to a platformagnostic C++/flext, this project has resulted in a number of improvements in flext build scripts and supporting documentation. It has also generated an invaluable list of caveats which are presented in this paper in hope to foster more platform-agnostic object design and porting efforts. Since its release, munger1~ has been featured in two interactive multimedia creations whose technical and artistic impact is also addressed in this paper.
Spatial knowledge, necessary for efficient navigation, comprises route knowledge (memory of landm... more Spatial knowledge, necessary for efficient navigation, comprises route knowledge (memory of landmarks along a route) and survey knowledge (overall representation like a map). Virtual environments (VEs) have been suggested as a power tool for understanding some issues associated with human navigation, such as spatial knowledge acquisition. The Finger-Walking-in-Place (FWIP) interaction technique is a locomotion technique for navigation tasks in immersive virtual environments (IVEs). The FWIP was designed to map a human's embodied ability overlearned by natural walking for navigation, to finger-based interaction technique. Its implementation on Lemur and iPhone/iPod Touch devices was evaluated in our previous studies. In this paper, we present a comparative study of the joystick's flying technique versus the FWIP. Our experiment results show that the FWIP results in better performance than the joystick's flying for route knowledge acquisition in our maze navigation tasks.
iPhone and iPod Touch are multi-touch handheld devices that provide new possibilities for interac... more iPhone and iPod Touch are multi-touch handheld devices that provide new possibilities for interaction techniques. We describe iPhone/iPod Touch implementation of a navigation interaction technique originally developed for a larger multi-touch device (i.e. Lemur). The interaction technique implemented on an iPhone/iPod Touch was used for navigation tasks in a CAVE virtual environment. We performed a pilot study to measure the control accuracy and to observe how human subjects respond to the interaction technique on the iPhone and iPod Touch devices. We used the preliminary results to improve the design of the interaction technique.
Uploads
Papers by Ji-Sun Kim