%\subsection{The haptic pitfall path}
-The design of haptic systems requires joint efforts between specialists in several scientific domains as depicted on Figure~\ref{fig:hapticpath}.
+The design of haptic systems requires joint efforts between specialists in several scientific domains as depicted in Figure~\ref{fig:hapticpath}.
This pipeline has two steps on the system side and two steps on the human side.
-Both the system and human have a step in the physical world and one outside the physical world.
+Both the system and humans have a step in the physical world and one outside the physical world.
The objective of this pipeline is to transmit information to users through their sense of touch.
This is indeed a simplified pipeline, which nevertheless shows that several scientific disciplines are interested in this topic. %, and each step is essentially studied by several distinct research communities.
While each step of the pipeline is essentially studied by one or two scientific fields, the role of Human-Computer Interaction is to connect them in a meaningful and useful way for people.
\paragraph{Haptic systems}
The objective of the pipeline is to render virtual objects or pieces of information into haptic sensations.
-The pipeline starts with the software part of the system, which uses the data associated with this information to compute a command, or a signal.
+The pipeline starts with the software part of the system, which uses the data associated with this information to compute a command or a signal.
This is the system side, out of the physical world.
Here we consider, relatively speaking, that the software is the computer's mind.
These commands activate various kinds of actuators to produce a mechanical effect.
The mechanical effect results from the centrifugal force of the rotating mass.
There is a delay before the motor spins fast enough, and inertia when the signal stops.
Therefore there is no easy way to control both the frequency and amplitude of the produced mechanical effect with these actuators.
-\defword{Voice coil} actuators actuators work similarly to speakers, and they can actually be controlled with sound systems.
+\defword{Voice coil} actuators work similarly to speakers, and they can actually be controlled with sound systems.
The only difference is the frequency range, which is 1-1000Hz, compared to 200-20000Hz for sound.
They produce vibrations with a coil and a permanent magnet~\cite{mortimer07,yao10}.
The precision of these actuators enables fine control of both frequency and amplitude.
These sensors are different type of nerves and similarly to electronic sensors they transform physical effects into electric signals.
This document will not cover this in detail.
The interested reader will find explanations in many Ph.D. manuscripts about haptics~\cite{brown07,crossan03,hoggan10,oakley03}, including mine~\cite{pietrzak08}.
-We will just mention cornerstone studies about the perception of touch that established the limits of tactile perception~\cite{goff67}, which depend on body part, sex and laterality~\cite{weinstein68}.
-In particular authors established the range of frequencies humans can perceive (roughly up to 1000Hz), with a peak perception around 250Hz.
+We will just mention cornerstone studies about the perception of touch that established the limits of tactile perception~\cite{goff67}, which depend on the body part, sex, and laterality~\cite{weinstein68}.
+In particular, authors established the range of frequencies humans can perceive (roughly up to 1000Hz), with a peak perception around 250Hz.
This study is the motivation why most of vibrotactile systems use a 250 Hz signal, sometimes modulated with another frequency~\cite{brown06}.
-My opinion on this choice is that other frequencies provide different, and interesting sensations.
+My opinion on this choice is that other frequencies provide different and interesting sensations.
They just require a higher amplitude to be perceivable.
-However the amplitude produced with usual vibrotactile systems is sufficient for a large range of frequencies.
+However, the amplitude produced with usual vibrotactile systems is sufficient for a large range of frequencies.
%This is not necessarily an issue on a design point of view because high amplitude vibrations are uncomfortable anyway.
This is why we typically used different frequencies in our studies~\cite{gupta16,gupta17}.
-The perception of touch is however not only a matter of received signal.
+The perception of touch is nevertheless not only a matter of received signal.
Gibson showed that people are more efficient at tactile exploration with active touch~\cite{gibson62}.
This means the brain combines sensations with other information, including exploratory movements that resulted in these sensations.
-Given the diversity of touch sensations: weight, shape, temperature, etc., Lederman and Klatzky describe specific exploratory procedure that enable people to perceive these types of sensations~\cite{lederman96}.
+Given the diversity of touch sensations: weight, shape, temperature, etc., Lederman and Klatzky describe specific exploratory procedures that enable people to perceive these types of sensations~\cite{lederman96}.
This chapter will not get into details regarding the relation between action and perception since we will cover this specific topic in chapter~\ref{chap:loop}.
-Independently of active touch, our perception of haptic signals is an interpretation of the brain.
-For example people may perceive only one vibration while there are two distinct stimulation if they are too close in space~\cite{weinstein68} or time~\cite{goodfellow34}.
-We can use such interpretations to create tactile illusions.
-For example the funneling illusion, also called phantom sensations enables to create a virtual vibration in between two points\cite{alles70,vonbekesy58}.
-To do so, one must place two actuators, one at each end.
-The amplitude of the signal on both actuators is a ratio corresponding to the desired position of the virtual vibration.
-For example let $A$ be the maximum amplitude and two actuators $a$ and $b$ placed $4cm$ apart.
-If the amplitude of actuator $a$ is $A(a) = \frac{A}{4}$ and the amplitude of actuator $b$ is $A(b) = \frac{3A}{4}$, then the users feel a vibration between $a$ and $b$, $1cm$ away from $a$ and $3cm$ away from $b$.
-Tactile saltation, also called the cutaneous rabbit illusion, gives the illusion of a sequence of equally spaced vibration~\cite{geldard72}.
-The tactile stimulation however is a repeated vibration on a smaller subset of locations.
-For example with three actuators $a$, $b$ and $c$ equally spaced on a straight line, the stimulation is three vibrations on $a$, three on $b$, and three on $c$.
-The person feels nine equally spaced stimulations between $a$ and $c$.
-Israr \etal combined both funneling and saltation to produce 2D tactile motions, not only in straight line but also on curves~\cite{israr11}.
-
-\begin{figure}[htb]
+\begin{figure}[!b]
\definecolor{cellred}{rgb} {0.98,0.17,0.15}
-\definecolor{virt}{rgb} {0.98,0.77,0.75}
-\definecolor{cellblue}{rgb} {0.17,0.60,0.99}
+\definecolor{virt}{rgb} {0.17,0.60,0.99}
+\definecolor{cellblue}{rgb} {0.17,0.60,0.99}
\newcommand{\actuator}[3]{\node[x=1mm,y=1mm, fill=#3, text=white, circle, minimum size=1cm] at (#1) {#2};}
\newcommand{\vibrdot}[2]{\node[x=1mm,y=1mm, fill=#2, circle, minimum size=1mm] at (#1) {};}
\begin{tikzpicture}
- \draw[x=1mm,y=1mm, loosely dotted] (0,20) grid (80,30);
+ \draw[x=1mm,y=1mm, dotted] (0,20) grid (80,30);
\actuator{0,25}{a}{cellred}
\actuator{80,25}{b}{cellred}
\actuator{20,25}{v}{virt}
-% \node[x=1mm,y=1mm, fill=blue, draw, circle, minimum size=1cm] at
- %(8,0) {b};
\draw[x=1mm,y=1mm, <->]
(0,15) -- (0,0) -- (85,0) node [below] {$pos.$};
\draw[x=1mm,y=1mm]
\draw[x=1mm,y=1mm]
(0,0) -- (80,10);
\node[x=1mm,y=1mm, anchor=south, rotate=90] () at (0,13){$amp.$};
-% \draw[x=1mm,y=1mm]
-% (0,10) -- (40,5) node [midway, above] {$A_a$} -- (80,0);
-% \draw[x=1mm,y=1mm]
-% (0,0) -- (40,5) -- (80,10) node [midway, above] {$A_b$};
\draw[x=1mm,y=1mm, thick, draw=cellred]
(20,0) -- (20,7.5) -- (0,7.5) node [left] {$A_a$};
\draw[x=1mm,y=1mm, thick, draw=cellred]
\node[x=1mm,y=1mm, anchor=center] () at (40,-5){Funneling or Phantom illusion};
-% \draw[x=1mm,y=0.5mm, xshift=85mm, ultra thick]
-
- \draw[x=1mm,y=1mm, step=8, loosely dotted, yshift=18mm] (96,0) grid (160,8);
+ \draw[x=1mm,y=1mm, step=8, dotted, yshift=18mm] (96,0) grid (160,8);
\node[x=1mm,y=1mm, anchor=south, rotate=90] () at (96,28){$pos.$};
\draw[x=1mm,y=1mm, <->]
- (96,30) -- (96,18) -- (160,18) node [below] {$time$};
- \node[x=1mm,y=1mm, anchor=west] () at (98,30){Sensation};
+ (96,30) -- (96,18) -- (160,18) node [below, xshift=-5] {$time$};
+ \node[x=1mm,y=1mm, anchor=west] () at (98,30){Stimulation};
+
+ \foreach \x in {0,...,8} {
+ \pgfmathtruncatemacro{\y}{18 + floor(\x / 3) * 4}
+ \vibrdot{96 + \x * 8, \y}{cellred}
+ }
- \draw[x=1mm,y=1mm, step=8, loosely dotted] (96,0) grid (160,8);
+ \draw[x=1mm,y=1mm, step=8, dotted] (96,0) grid (160,8);
\node[x=1mm,y=1mm, anchor=south, rotate=90] () at (96,10){$pos.$};
\draw[x=1mm,y=1mm, <->]
- (96,12) -- (96,0) -- (160,0) node [below] {$time$};
- \node[x=1mm,y=1mm, anchor=west] () at (98,12){Stimulation};
-
-
- \vibrdot{96,18}{virt}
- \vibrdot{104,19}{virt}
- \vibrdot{112,20}{virt}
- \vibrdot{120,21}{virt}
- \vibrdot{128,22}{virt}
- \vibrdot{136,23}{virt}
- \vibrdot{144,24}{virt}
- \vibrdot{152,25}{virt}
- \vibrdot{160,26}{virt}
-
- \vibrdot{96,0}{cellred}
- \vibrdot{104,0}{cellred}
- \vibrdot{112,0}{cellred}
- \vibrdot{120,4}{cellred}
- \vibrdot{128,4}{cellred}
- \vibrdot{136,4}{cellred}
- \vibrdot{144,8}{cellred}
- \vibrdot{152,8}{cellred}
- \vibrdot{160,8}{cellred}
+ (96,12) -- (96,0) -- (160,0) node [below, xshift=-5] {$time$};
+ \node[x=1mm,y=1mm, anchor=west] () at (98,12){Sensation};
- \node[x=1mm,y=1mm, anchor=center] () at (127.5,-5){Saltation or Cutaneous rabbit illusion};
+ \foreach \x in {0,...,8} {
+ \vibrdot{96 + \x * 8, \x}{virt}
+ }
-% \actuator{96,25}{a}{cellred}
-% \actuator{128,25}{b}{cellred}
-% \actuator{160,25}{c}{cellred}
+ \node[x=1mm,y=1mm, anchor=center] () at (127.5,-5){Saltation or Cutaneous rabbit illusion};
\end{tikzpicture}
-\caption[Tactile illusions: funneling and saltation]{Examples of tactile illusions. Left: funneling, right: saltation.}
+\caption[Tactile illusions: funneling and saltation]{Examples of tactile illusions. Left: funneling or phantom illusion, right: saltation or cutaneous rabbit illusion.}
\label{fig:illusions}
\end{figure}
+Independently of active touch, our perception of haptic signals is an interpretation of the brain.
+For example, people may perceive only one vibration while there are two distinct stimulation if they are too close in space~\cite{weinstein68} or time~\cite{goodfellow34}.
+We can use such interpretations to create tactile illusions.
+For example, the funneling illusion, also called phantom sensations, produces a virtual vibration in between two points\cite{alles70,vonbekesy58}.
+To do so, one must place two actuators, one at each end.
+The amplitude of the signal on both actuators is a ratio corresponding to the desired position of the virtual vibration.
+For example, let $A$ be the maximum amplitude and two actuators $a$ and $b$ placed $4cm$ apart, as depicted on the left part of Figure~\ref{fig:illusions}.
+If the amplitude of actuator $a$ is $A(a) = \frac{A}{4}$ and the amplitude of actuator $b$ is $A(b) = \frac{3A}{4}$, then the users feel a vibration between $a$ and $b$, $1cm$ away from $a$ and $3cm$ away from $b$.
+Tactile saltation, also called the cutaneous rabbit illusion, gives the illusion of a sequence of equally spaced vibration~\cite{geldard72}.
+The tactile stimulation however is a repeated vibration on a smaller subset of locations.
+For example, with three actuators $a$, $b$, and $c$ equally spaced on a straight line, the stimulation is three vibrations on $a$, three on $b$, and three on $c$, as shown on the right part of Figure~\ref{fig:illusions}.
+The person feels nine equally spaced stimulations between $a$ and $c$.
+Israr \etal combined both funneling and saltation to produce 2D tactile motions, not only in straight line but also on curves~\cite{israr11}.
+
\section{Research questions}
-The haptic rendering pipeline illustrates the multidisciplinary aspect of such research.
+The haptic rendering pipeline described above illustrates the multidisciplinary aspect of such research.
It also reveals the many pitfalls at all levels that can make users perceive something different than what was intended.
-Users interact with \fixme{digital} systems with \defword{interaction techniques}, which are combinations of a a device (part of the electro-machanical system) and an interactive language (part of the software controller)~\cite{nigay95}.
-Devices are usually computer peripherals, we presented some of them previously.
-The interactive language is a systematic description of the interactive features of this device.
-Similarly to programming languages, they have three levels: lexical, syntactic and semantic.
+Any step in this pipeline potentially introduces a drift with respect to the original message.
+For example the resolution of commands could be too low, the mechanical response to the command can be non-linear, the quality of the contact between the haptic system and the user can be sub-optimal, the mechanical effects can be out of the perceptual range, and haptic illusions can alter the interpretation of sensations.
+While the work of researchers in specialized domains mentioned in Figure~\ref{fig:hapticpath} is to avoid, or at least minimize such drifts in their part of the pipeline, my work as an HCI researcher consists in selecting, adjusting, combining, designing, implementing and evaluating such parts to build \defword{interactive systems}.
+My research is therefore fundamentally interdisciplinary, and I combine, complement or replace expertise of different domains, depending of the needs of each research project, and the expertise of my collaborators.
+
+%I prefer the term “interactive system” over “interface” because it better conveys my systemic approach.
+%In my opinion, sytems are not a computing core with an interface to feed inputs and display outputs.
+%I rather
+
+Interactive systems are made of \defword{interaction techniques}, which are combinations of a device (part of the electro-machanical system) and an interactive language (part of the software controller)~\cite{nigay95}.
+%Users interact with \fixme{digital} systems with \defword{interaction techniques}, which are combinations of a device (part of the electro-machanical system) and an interactive language (part of the software controller)~\cite{nigay95}.
+%Devices are usually computer peripherals, we presented some of them previously.
+We presented technologies for haptic devices above.
+Interactive languages are systematic descriptions of the inputs and outputs of an interactive device.
+Similar to programming languages, they have three levels: lexical, syntactic, and semantic.
The \emph{lexical level} defines the basic vocabulary of the device.
-For example, we can control vibrations in frequency, amplitude, shape and duration (Figure~\ref{fig:lexical}).
+For example, we can control vibrations in frequency, amplitude, shape, and duration (Figure~\ref{fig:lexical}).
Hence the command for a vibrotactile actuator is an electrical signal made of these elements.
%For example the output vocabulary of a 3 degrees-of-freedom (DOF) force-feedback device is a force vector.
%Hence, a software controller sends three values to the electro-mechanical system, which will convert it to forces.
(110,0) sin (111,10) cos (112,0) sin (113,-10) cos (114,0) sin (115,10) cos (116,0) sin (117,-10) cos (118,0) sin (119,10) cos (120,0) sin (121,-10) cos (122,0) sin (123,10) cos (124,0) sin (125,-10) cos (126,0);
\node[x=1mm,y=1mm, anchor=center] () at (112,-9){Duration};
- \draw[x=1mm,y=0.5mm, xshift=143mm, ultra thick]
+ \draw[x=1mm,y=0.5mm, xshift=140mm, ultra thick]
(0,0) sin (1,10) cos (2,0) sin (3,-10) cos
(4,0) sin (5,10) cos (6,0) sin (7,-10) cos
(8,0) sin (9,10) cos (10,0) sin (11,-10) cos
(12,0);
- \draw[x=1mm,y=0.5mm, xshift=158mm, ultra thick]
+ \draw[x=1mm,y=0.5mm, xshift=155mm, ultra thick]
(0,0) -- (0,10) -- (2,10) -- (2,-10) --
(4,-10) -- (4,10) -- (6,10) -- (6,-10) --
(8,-10) -- (8,10) -- (10,10) -- (10,-10) --
(12,-10) -- (12,0);
- \node[x=1mm,y=1mm, anchor=center] () at (156,-9){Shape};
+ \node[x=1mm,y=1mm, anchor=center] () at (153,-9){Shape};
\end{tikzpicture}
\caption{Four parameters of the vibrotactile output vocabulary: frequency, amplitude, duration and shape.}
\label{fig:lexical}
(52,0) sin (57,10) cos (62,0);
\node[x=1mm,y=1mm, anchor=center] () at (18,-13){Frequency modulation};
- \draw[x=1mm,y=0.5mm, xshift=72mm, ultra thick]
+ \draw[x=1mm,y=0.5mm, xshift=71mm, ultra thick]
(0,0) sin (1,5) cos (2,0) sin (3,-5) cos
(4,0) sin (5,7) cos (6,0) sin (7,-7) cos
(8,0) sin (9,9) cos (10,0) sin (11,-9) cos
(20,0) sin (21,7) cos (22,0) sin (23,-7) cos
(24,0) sin (25,5) cos (26,0) sin (27,-5) cos
(28,0);
- \node[x=1mm,y=1mm, anchor=center] () at (85,-13){Amplitude modulation};
+ \node[x=1mm,y=1mm, anchor=center] () at (84,-13){Amplitude modulation};
- \draw[x=0.5mm,y=0.5mm, xshift=142mm, ultra thick]
+ \draw[x=0.5mm,y=0.5mm, xshift=140mm, ultra thick]
(0,0) sin (1,10) cos (2,0) sin (3,-10) cos
(4,0) sin (5,10) cos (6,0) sin (7,-10) cos
(8,0) sin (9,10) cos (10,0) sin (11,-10) cos
(48,0) sin (49,10) cos (50,0) sin (51,-10) cos
(52,0) sin (53,10) cos (54,0) sin (55,-10) cos
(56,0);
- \node[x=1mm,y=1mm, anchor=center] () at (156,-13){Rhythm};
+ \node[x=1mm,y=1mm, anchor=center] () at (154,-13){Rhythm};
\end{tikzpicture}
\caption{Three examples of haptic phrases: frequency modulation, amplitude modulation and rhythm.}
\label{fig:syntactic}
\end{figure}
The \emph{semantic level} represents the mapping between the haptic effect and its associated meaning.
-For example if we create Tactons for messaging system alerts, we can encode the caller ID with a rhythm, and the urgency with roughness.
+For example, if we create Tactons for messaging system alerts, we can encode the caller ID with a rhythm, and the urgency with roughness.
The combination of both parameters enables encoding every level of urgency for every caller ID.
-One may think that because we are computer scientists, we only have to design and implement interaction techniques.
-However this design not only depends on technological factors, but also human factors.
-Moreover, the diversity of haptic sensations and technologies to produce them multiplies the research necessary to create such interaction techniques.
-Typically when a new haptic technology emerges, their designers need information about their specifications: size of actuators, voltage, required amplitude, resonant frequency, maximum force (continuous and peak), etc.
-Therefore we perform human factors studies with prototypes
+%One may think that because we are computer scientists, we only have to design and implement interaction techniques at the software controller level.
+%However, this design not only depends on technological factors (the electro-mechanical system), but also human factors (both sensory and cognitive systems).
+Not only the design of interaction techniques depends on technological factors (the electro-mechanical system), but it also depends on human factors (both sensory and cognitive systems).
+The expansion of the haptic interactive vocabulary goes in both direction.
+On one hand, the diversity of haptic sensations and body parts that sense them motivate the design of a large diversity of haptic devices.
+On the other hand, the frequent emergence of new haptic technologies expands the diversity of haptic sensations we leverage for the design of interaction techniques.
+This diversity of sensations and technologies is a richness, but also a limitation for the development and integration of haptic sytems.
+Indeed, the sense of touch combines of all these sensations, but no single haptic technology can produce all of them.
+Therefore interactive systems that leverage the sense of touch focus on a subset of haptic sensations relevant to their context of use.
+
+On a practical point of view, when a new haptic technology emerges, their designers need information about their specifications: the size of actuators, voltage, required amplitude, resonant frequency, maximum force (continuous and peak), etc.
+They often simulate the behavior of their actuators, and evaluate the relation between the command and the physical effect~\cite{poncet17,poncet16}.
+However, they need to know the desired parameters and values of the physical effects they can produce in order to propose a suitable output vocabulary that we can use to design interaction techniques.
+Therefore we perform iterations of prototyping and evaluation of human factors in order to understand these requirements.
%Technical and user issues, evaluation…
%The software controller provides users with \defword{modalities}, or \defword{interaction techniques}, which are combinations of a a device (the electro-machanical system) and an interactive language~\cite{nigay95}.
-
%Leverage the sense of touch to improve interaction.
-
Potential issues at all levels: difficulties to reproduce effects in a consistent way across devices and people.
Coding \texttt{=>} Command \texttt{=>} physical effect \texttt{=>} sensation \texttt{=>} information…
projects / hdr.git / commitdiff