2018 International Conference on Sustainable Energy, Electronics and coMputing Systems (SEEMS)

Review of Skinput Technology: Input Through Skin

1
Supriya Singh & 2Awadhesh Kumar
1
P.G Student, Department of Computer Science, KNIT, Sultanpur
2
Dept. Computer Science and Engineering, KNIT, Sultanpur
[email protected], [email protected]

Abstract— Skinput technology takes input through the body not to shrink the surface area of the device on which users can
surface and acts as an interface. So input can be given by tapping a perform their task easily. In Skinput technology, acoustics
finger on the skin surface. This new input technique is based on sounds play vital rolls, which take profit of the human body's
bio-acoustic sensing which detects the acoustic signals on the in which acoustic sounds travel. This allows the Skinput
surface of the body as an impact of a finger and this signal technology to perform the task while taking input from the
propagates through the body. This technology provides input skin.
through tapping finger which is naturally portable and contains
minimally interfering. The technology is coupled with the bio- II. THEORITICAL REVIEW
acoustic detector, sensor, and pico-projectors. An interactive
Skinput technology takes input by tapping finger on palm
interface can be visible directly on the body surface as a graphical
image. or hand surface: Advances in electronics have allowed novel
Keywords— Skinput, Bio-acoustic, Finger,and Pico-Projector. devices to become very small so, much that they can benefit to
extra space area available for interaction, however, we don’t
I. INTRODUCTION want to carry around extra surface especially input, it’s a
clever way to use appropriate surface which is already around
As computer scientist work long and hard to make
us like our arm, palm or hand surface. As we know that touch
computer smaller, faster, cheaper and did their work by
technology is everywhere but still texting choosing graphing
placing internet in your pocket with the Smartphone.
entities, performing an operation and so on are still difficult
Wherever we go it’s by our side and makes us smarter. These
because one has to really struggle in dealing with the small
technologies are crafting our relations with devices by
interface in the device. Skinput technology combines with a
changing the culture and shaping the way we live. We got a
bio-acoustic detector, pico-projector, and sensors with
problem here as we shrink our device we also shrink ability to
sophisticated machine learning which enable the user to use
hold the device as well as its interface. As people use their
their finger or forearms as their touch screen or interface. Bio-
devices while traveling, all their concentration is on the device
acoustic detectors are used to detect the sound when we tap
with their hand and mind on it rather than people around
the finger on hand or forearms it has been found that while
them.“We’ve cool technologies that profoundly shape how we
tapping the finger on different parts of the arm produce
live. The bottleneck is the interface; therefore, the challenge is
distinct acoustic sound. Skinput technology provides various
to create a rich way for humans and device to communicate”.
input through the skin.
Imagine you are free from holding the device with the large
interface and riching natural way of interacting without device
and computing environment with people around you. The
approach we are taking is that looking at novel sensor and
sensing techniques which you may carry or wear it on your
body in order to create the ability to interact without the
device. These things are done with the help of sensor, bio-
acoustic detector and pico-projector combined in a device; we
are able to track the posture, gesture by electrical muscles Figure. 1: Bio-acoustic sensor in Skinput technology with sophisticated
machine learning which turns human palm or arm into “touch typing”.[1]
sensing which are able to sense the fine finger motions
without using your hands all the time. We are going to talk
Skinput technology is more than finger tapping selection
about the technology which is fairly at its early age that is
when we tap a finger on the surface of skin then multiple
Skinput technology. Skinput uses a bio-acoustic detector to
detect the sounds of the body in order to turn your arm or any distinct sounds are produced at the distinct location on the
part of your body into touchscreen while tapping a finger on surface of the skin. The skin provides varies inputs like
the surface of your body. Skinput technology is an input type squeezing, twisting, pressing, pulling, scratching and tapping.
of technology in which bio-acoustic sensing sensors are used This increases the number of inputs on the skin with varieties
to a particular place where the finger is taped to the skin for of interactions and gestures.
achieving input. When added with a Pico-projector, the
Skinput device can provide an interface on the user's skin or
on the body of the user's. This Skinput technology was
developed by Chris Harrison, Desney Tan, and Dan Morris, at
Microsoft Research's Computational User Experiences Group. Figure. 2:Variety of inputs: (a)Touch, (b)Grab, (c)Pull, (d)Press,(e)Scratch,
Skinput technology aim was to shrink the size of the device (f)Shear (g)Squeeze, (h)Twist.[2]

978-1-5386-5866-6/18/$31.00 ©2018 IEEE

 2018 International Conference on Sustainable Energy, Electronics and coMputing Systems (SEEMS)

addition when the impact of finger create transverse waves on

the surface of the arm, at that time some energy is transmitted
The set of input result exhibit in fig. 2 was acquired from inward, toward the skeleton, producing vibration among soft
formal touch interface as a result of studies on bio-machines tissues which are sensed by the sensor which is called
of skin. This input varies from on-surface interaction to longitudinal (compressive) waves.
intense skin deformation note that these varieties of inputs are
defined from the perspective of the user and not from a
technology-centered one. [2]
III. PRINCIPLE
Skinput technology works on bio-acoustic detection.
Whenever finger taps on the skin, the impact creates a sound
which is acoustic signals and these signals are captured by
bio-acoustic detectors. When tapping of a finger is slow down Figure. 3: Propagation of transverse waves on the body surface which is
sensed by the sensor when it passes underneath it.[1]
fourteen times we can see transverse wave on the surface of
the skin, however complex longitudinal (compressive) waves
are also formed and propagate through the body using soft
tissues of the skin surface. Bio-acoustic detectors are used to
capture both longitudinal waves and transverse waves which
are passed through the body surface and detect variation in
bone density, mass, and size which are passed from soft Figure. 4: Propagation of longitudinal waves in which some of the acoustic
signals travel towards the skeleton structure making soft tissue to vibrate due
tissues and joints with different locations and with distinct to which waves are emitted outwards from bone towards the skin.[1]
acoustic signals. An adult average body surface area is 1.73m2
which is greater than touch screen phone by 400 times that is These two separate transverse waves and longitudinal
0.004m2. Skinput technology adds Pico-projector to project an waves carry energy over different locations and at different
image of graphic-display on your hand with the sensor in its frequencies.[1]
technology resulting in arm vibration control.[4]
B. Bio-sensing
IV. WORKING Cantilevered sensor capture specific sets of frequencies
Skinput technology uses information of the acoustic signal which response to tapping impact of a finger. Whereas
to interpret information. This software developed its senses irrelevant frequencies cause the high signal to noise ratio
from the impact of a finger and classifies them into distinct which shows flat curve response. In this input technology
interactive capabilities which can be bounded to the different especially, lower frequency signals (e.g. 25 Hz) are low
location. In this technology, we choose to focus on arm or sensitive to the sensor. The solution to above problem is that
hand for inputs and developed the armband for sensing. The by adding an array with high sensing sensor in single sensing
armband contains the pico-projector which visualize graphic - sensor. In cantilever, small weights are added at the end to
display on users arm or hand which operates more similarly alter the given unique, narrow low-frequency response from
like the cell phone. Although the Skinput technology could be resonant frequencies, to sense element handed-down in
applied anywhere on the body surface but here the focus of acoustic spectrum. The additional weights are added to align
this technology is on arm or hand because this is an attractive with particular frequencies to study the bio-acoustic input
plus continuous and flat area for projecting 200 times bigger characteristics.
graphic- display. Furthermore, arm skeleton structure contains The perpendicular motion of the skin plane is rich
an assembly of bones which is complex in nature so, it responsive to cantilever as it perfectly captures the
provides an increase in distinct acoustic sounds at different longitudinal waves and transverse wave from the skin
locations. [5] surface.[6]
A. Bio-acoustic
16
When a finger is tapped by the user on their body surface, 14
Sensitivity(dB re

at different locations distinct acoustic energy is produced. This 12

10
nominial)

acoustic energy is radiated into the different medium like air 8

and some energy is absorbed by body surface. In air medium, 6
4
acoustic energy radiated is lost and the absorbed acoustic 2
0
energy is converted into two waves that are transverse waves
100
50
52
59
61
68
70
74
78
82
88
95

and longitudinal waves. The transverse waves are propagated

through the surface of the skin because of the finger impact. Hertz
Tapping finger on point of body surface, outward transverse
waves is generated to appear on the body which propagates
from that point when spotted by the high-speed camera. In Figure. 5: Shows resonant frequencies with respect to sensitivity[6]
 2018 International Conference on Sustainable Energy, Electronics and coMputing Systems (SEEMS)

C. Aramband Input is segmented and waveforms are analyzed. The nature of

In this prototype, armband contains two array features with individual taps is highly discrete in nature means that acoustic
five sensing elements, which corporate with each other into an signals are not particularly expressed overtimes. The entire
armband. The resonant frequencies are collected by armband computed window are featured and temporal dynamics are not
which contains pilot. There is two sensor package, the first captured. The brute force machine learning approach
one is upper sensor package, uses to sense the lower- computes total 186 features. Gross information includes the
frequency signals transmitted through fleshier body surface standard deviation, average amplitude and total waveforms
which dies down while propagating towards the sensor. The energy in each channel with all 30 features. From these, all
second one is lower sensor package, use to sense the high- average amplitude ratios between channel pairs including 45
frequency signals captured through transmitting from denser features are calculated. This Support Vector Machine (SVM)
bone on the body surface.[3] classifier is passed with 186 features.[6]

Figure. 6: Armband Prototype.[2]

Figure..7:Acoustic data generated by three finger taps on ten different
D. Processing channels on the forearm. Red color shows average exponential of the channel
and Green color show input Segmented windows. [6]
In the prototype system, Mackie Onyx 1200F audio
interface is taken to capture digital data from the 10 sensors.
Firewire was used to convert this to a conventional computer
desktop, Where Audio Stream Input/ Output (ASIO) protocol
in C interface was written. The sample from each channel was
taken at 5.5 kHz; this sample rate was too low for
environmental audio and speech but represents some
frequencies that were transmitted while taping on the skin.
Therefore sample rate was reduced when observed which
made technique quit portable to be embedded in processors.
For example, the ATmega168 processor which was employed
by the Arduino platform which shows an analog sampling
readings at 77kHz with no quality loss which was required by Figure. 8. Three position of whole-arm-centric condition accuracy.[6]
Skinput technology to provide full sample power that is about
55kHz in total. Data written in Java was sent from thin client
to primary application over the local socket. This contains 3 V. EXAMINING OF EXPERIMENTAL CONDITION
key functions. Firstly, Acoustic features are identified from 10 A. Fingertip ( 5location)
sensors which contain data from live visualization. Secondly, The users have to tap on their fingertip in each of its five
Individual taps contain segments of inputs which were fingers. Including fingertips, fourteen knuckles together make
retrieved from data streams. Thirdly, Individual taps are total nineteen distinct locations isolated input on the finger.
classified and audio stream segment was done into individual Potentially uses of the interface are provided by linear order
taps using the average value of absolute exponential from all fingers such as menu selection, number entry, and magnitude
10 channels. From (fig. 8, blue line) [6] intensity of threshold control. As we know that fingers are more uniform sub-
was recorded from different time periods from starting of the sections of the body due to which acoustic variations and
taps. If the intensity is not falling between 100ms and 700ms differences among them are more difficult to detect. The
the event was discarded. If the acoustic data was between signal needs to be passed from finger and wrist joint, to reach
60ms plus on the buffer, input event was considered (fig.8, forearm due to which signals are slightly wet. Placing an array
green vertical regions) [6]. Due to extreme noise suspended in sensor just below the elbow on the forearm provide the better
sensing sensor, this trial and error proved to be highly experimental condition. Even with these entire difficulties
vigorous. experiment shows measurable distinct acoustic signal through
Resonant frequencies in the two sensor packages.[6] the finger. All these theories are primarily based on size and
Upper Array 25 Hz 27 Hz 30 Hz 38 Hz 78 Hz thickness of the finger as well as the complex structure of
Lower Array 25 Hz 27 Hz 40 Hz 44 Hz 64 Hz wrist bone through which acoustic transmission takes place on
the forearm.[1]
 2018 International Conference on Sustainable Energy, Electronics and coMputing Systems (SEEMS)

and nutrition. Its uses "MotionX" sensor which is inbuilt to

B. Whole-arm (5 locations): track activity and transfers the data to iPhone, iPad or iPod
An investigation is done on the forearm or hand by taking touch via the headphone jack.
input at five different locations that are the middle finger,
palm, wrist, arm, and thumb. The reason for selecting these
locations are firstly in this location user doesn’t have to train
or mark them. Secondly, these locations provide distinct
acoustic signal offering separate input point with more
variations. In this experiment different conditions are used in
these locations firstly, the condition of the sensor should be
placed below and above the elbow to measure the loss of
accuracy around the elbow. In additional condition, the user
was told to close their eye and face forward for testing and
training purpose. Figure. 9. Jawbone UP device [7]

The other device known as Cicret Bracelet whose co-

C. Forearm (10 locations) founder is Guillaume Pommier and the device look like small
Forearm location provides very high input location wristband which is similar to the Jawbone Up. The Cicret
density but it also depends on the high degree of physical Bracelet contains a pico-projector and eight sensors in the row
uniformity of the forearm, these are following factor which that refects towards the user's forearm. It operates alone
could make difficulties in sensing acoustic signal. Forearm without connecting with any other device. when Cicret
location provides a large and flat area which can be Bracelet is activated by twisting of the wrist, projects an
immediately accessed, that is why this location provides an Android interface onto the users arm, much like Chris
ideal projection surface for the dynamic interface. In this, the Harrison's Skinput research. The proximity sensors detects the
user's finger or fingers tap on users arm and allow them to
sensor was placed below and above on the elbow. Small
interact with other Android device. If the technology of Cicret
stickers of different color are marked on the input target to Bracelet can improved more then, it will be something we'd
reduce confusion and to increase consistency of input. actually want to use.
This device include features like a vibration module, an
accelerometer and along with an LED which provide
notifications and allow users to send and receive emails,
browse the web and play games. It is possible for users to pair
their Cicret Bracelet device with smartphone, to answer their
incoming phone calls and to activate the speakerphone
functionality.

Figure..7: Input location sets evaluated as per study conducted at Carnegie

mellon university by Chris Harrison, Desney tan, Dan morris.[1]

D. Technology feasibility
The overall accuracy of the experiment at five different
locations was high, across 87.6% average accuracy. In finger
(5 locations) accuracy remain high, averaging to 87.7% where
there is approximately 40 cm of segregation present between
input target with multiple knuckles passing and sensor. In
whole-arm (5 locations) provide the best performance below
the elbow placement of the sensor, average accuracy about Figure. 10. Cicret Bracelet device [7]
95.5%. Whereas, when the sensor is moved above the elbow
its accuracy decereases to 88.3% and the difference is about
VII. CONCLUSION
7.2% compared with the sensor placed below the elbow. In
eye free condition the input average accuracy was around Skinput technology takes input through human body
85.0% this represents the loss of accuracy around 10% as which acts as an interface. Pico-projector shows the graphical
compared to visual assessment. representation of the surface area of hand or arm to show the
tapping location of a finger, as well as a novel and wearable
VI. LATEST INNOVATIONS bio-acoustic detector with sensing array, are built into the
Jawbone UP health and activity monitor is a device armband. Experimental results show that when the human
which was developed by Hosain Rahman, This device is body is taken as an interface for providing input, still we
stylish along with wrist-worn monitoring tracks activity, sleep acquire the admirable system performance for a series of
 2018 International Conference on Sustainable Energy, Electronics and coMputing Systems (SEEMS)

gestures. Even if the body is in motion this system performs

very well. In addition, future work can be explored as the
human hand or arm could be users i-phone and handset could
be watch sized on users wrist. At last, we terminate with the
illustration of various prototype applications in which the rich
design space is signified and skinput enables.

REFERENCES

[1] Skinput: Appropriating the Skin as an Interactive Canvas By Chris

Harrison, Desney Tan, and Dan Morris. august
2011,vol.54no.8,communications of the ACM.
[2] Skinput: Advance Input Technology Shaikh Abdur Rehman
Mohammed Sadique1, Pragnesh N Shah2, Vol-
05,February2015,International Journal of Computational Engineering
Research (IJCER).
[3] Chris Harrison, Desney Tan, Dan Morris Skinput: Appropriating the
Body as an Input Surface Communications of the ACM, Vol. 54 No. 8.
[4] Scott Klemmer Skintroducing the Future Communication of the ACM
2011 VOL.54 NO.8.
[5] Paul Wallis How to wear a working computer display on your skin.
2010.
[6] Binu Pv." Seminar report on Skinput" Cochin University Of Science
And Technology, Kochi-682022,August 2010.
[7] https://newatlas.com/jawbone-up-activity-monitor/20401.
[8] https://newatlas.com/cicret-bracelet/35109.