% \draw [->, -stealth', thick]
% (software.north) edge (mechanics.south);
\end{tikzpicture}
-\caption[Haptic rendering pipeline]{Haptic rendering pipeline.}
+\caption[Haptic rendering pipeline.]{Haptic rendering pipeline.}
\label{fig:hapticpath}
\end{figure}
}
\node[x=1mm,y=1mm, anchor=center] () at (127.5,-5){Saltation or Cutaneous rabbit illusion};
-
\end{tikzpicture}
-\caption[Tactile illusions: funneling and saltation]{Examples of tactile illusions. Left: funneling or phantom illusion, right: saltation or cutaneous rabbit illusion.}
+\caption[Tactile illusions: funneling and saltation.]{Examples of tactile illusions. Left: funneling, or phantom illusion; right: saltation or cutaneous rabbit illusion.}
\label{fig:illusions}
\end{figure}
(12,-10) -- (12,0);
\node[x=1mm,y=1mm, anchor=center] () at (153,-9){Shape};
\end{tikzpicture}
- \caption{Four parameters of the vibrotactile output vocabulary: frequency, amplitude, duration and shape.}
+ \caption[Haptic vocabulary.]{Four parameters of the vibrotactile output vocabulary: frequency, amplitude, duration and shape.}
\label{fig:lexical}
\end{figure}
(56,0);
\node[x=1mm,y=1mm, anchor=center] () at (154,-13){Rhythm};
\end{tikzpicture}
-\caption{Three examples of haptic phrases: frequency modulation, amplitude modulation and rhythm.}
+\caption[Haptic phrases.]{Three examples of haptic phrases: frequency modulation, amplitude modulation and rhythm.}
\label{fig:syntactic}
\end{figure}
\draw[<->, stealth-stealth, thick] (l3) -- (d3);
\end{tikzpicture}
- \caption{Illustration of a semantic mapping between 3-parameters Tactons and a 3-level information, adapted from~\cite{brown06}. Three values of rhythm are mapped to three types of messages, three values of roughness are mapped to three degrees of importance, and three spatial locations are mapped to three values of delay.}
+ \caption[Mapping information with Tactons.]{Illustration of a semantic mapping between 3-parameters Tactons and a 3-level information, adapted from~\cite{brown06}. Three values of rhythm are mapped to three types of messages, three values of roughness are mapped to three degrees of importance, and three spatial locations are mapped to three values of delay.}
\label{fig:semantic}
\end{figure}
It feels like a physical button.
But when it is powered off, this haptic effect disappears.
Further, studies show that vibrotactile feedback increases typing performance~\cite{hoggan08} on touch keyboards.
-We will discuss in \ref{sec:printgets} the design of vibrotactile widgets for a touch dashboard.
+I will discuss in \ref{sec:printgets} the design of vibrotactile widgets for a touch dashboard.
\subsection{Haptics and tangible interaction}
Changing the shape of objects can change the way we use them, therefore their function.
For example, changing the shape of a knob changes the way we grip it~\cite{michelitsch04}.
A knob can even become a slider~\cite{kim16}.
-In section~\ref{sec:metamorphe-livingdesktop}, we will discuss how the actuation of computer peripherals can bring them new interactive properties.
+In section~\ref{sec:metamorphe-livingdesktop}, I will discuss how the actuation of computer peripherals can bring them new interactive properties.
% Physical properties of objects also give haptic feedback\\
\section{Contributions}
+The previous section presented challenges, but also opportunities for the design and implementation of haptic interactive systems.
+These are questions I regularly address in my research.
+The way I describe the research projects below does not necessarily follow the narrative of the papers that resulted from these works.
+I rather discuss the contribution of these studies to the research questions above.
+
% Haptic variables, vocabulary \texttt{=>} Tactons. Diversity of sensations \cite{lederman87} \texttt{=>} diversity of devices~\cite{seifi19}.
% The features are: linguistic/nonlinguistic, analogue/non-analogue, arbitrary/non-arbitrary, static/dynamic\cite{bernsen93a}
- % 8. Touch language Touch letters, numerals, words, other touch language related signs, text, list and table orderings.
+ % 8. Touch language Touch letters, numerals, words, other touch language related signs, text, list, and table orderings.
% Example: Braille
% 18. Real-world touch Single touch representations, touch sequences.
% 20. Touch graphs 1D, 2D or 3D graph space with geometrical forms.
% Pure charts (dot charts, bar charts, pie charts, etc.).
- % 24. Arbitrary touch Touch signals of differents sorts.
+ % 24. Arbitrary touch Touch signals of different sorts.
% 28. Touch structures Form fields, frames, grids, line separations, trees.
- \subsection{Tactons}
- \label{sec:activibe}
- Haptic feedback for activity monitoring (Activibe)\cite{cauchard16}
+\subsection{Tactons for activity monitoring}
+\label{sec:activibe}
- Off-the-shelf smartwatch \texttt{=>} simple ERM actuator \texttt{=>} simple feedback, limited vocabulary
+Using haptics as an output vocabulary to transmit information to users directly stems from my Master and Ph.D. work.
+I had the privilege and pleasure to collaborate with Stephen Brewster, who is also the inveentor of the concept of Tactons~\cite{brewster04}.
+I worked on several Tacton sets at the time: active~\cite{pietrzak05} and passive~\cite{pietrzak05a} force feedback, as well as pin-array Tactons~\cite{pietrzak06,pietrzak09}.
+Here, I will discuss another project on Tactons called Activibe~\cite{cauchard16}.
- Can people notice and interpret correctly information when they do not expect the tactile cues?
+The idea was to design haptic feedback for activity monitoring.
+Fitness trackers became mainstream in the last decade.
+People use wearable devices such as smartwatches or bracelets to measure their activities, such as their daily steps~\cite{consolvo08}.
+They typically define daily objectives, and need regular reminders to help them meeting their goal.
+These activity trackers provide visual information with small screens or LEDs.
+Therefore users have to actively look at their device to know their progress.
+This is a limitation of the incentive aspect of activity moitoring towards behavior change.
+Still today, the solution in consumer electronics products is a simple vibration that encourages users to look at their device to check for information visually.
+In this project we decided to provide progress information with vibrotactile feedback, thus not requiring users to look at the device while they perform their daily activities.
+We describe below the design and evaluation of these Tactons.
- \subsection{Tactile textures}
- \label{sec:stimtac}
+\subsubsection{Tacton design}
- command/effect relation
- singularity of the effects produced by research prototypes
+The context of activity monitoring implies several limitation on the design of the Tactons.
+We believed it would be difficult for us to motivate activity manufacturers to include a better tactile actuator than the existing ones.
+Therefore we decided to use an off-the-shelf smartwatch: the Pebble\footnote{\href{https://en.wikipedia.org/wiki/Pebble_(watch)}{https://en.wikipedia.org/wiki/Pebble\_(watch)}}.
+It includes an eccentric rotating mass (ERM) actuator.
+This low-resolution DC motor constrains the choice of tactile parameters to amplitude, duration and rhythm.
+Given that the perception of amplitude varies from one user to another or the way the smartwatch is fastened, we use the duration and rhythm parameters only.
- Definition
+\paragraph{Information}
- Tactile Textures~\cite{potier12,potier16}
+%%From Activibe
- \subsection{Vibrotactile widgets}
- \label{sec:printgets}
+ActiVibe was designed as a set of vibrotactile icons to represent progress.
+As activity performace is generally evaluated as a percentage or as a value on a scale, we created vibrations corresponding to the values 1 to 10, with the objective of representing 10\% increments.
+Because of our choice of using a single basic vibration actuator, we encode the vibrations using the duration and rhythm parameters only.
+Since there was no prior encoding of discrete numbers found in the literature using duration and rhythm only, we first had to determine the best encoding pattern for ActiVibe.
+%We designed a total of six patterns (Figure~\ref{fig:activibesets}) and in our subsequent evaluations used the pattern with the highest accuracy rate for ActiVibe.
- Replacing physical controls with touchscreens have advantages: updates, reconfigurable, visual feedback, but most of haptic properties are lost: click sensations of buttons, detents on slides. Impact on interaction. Technologies to restore haptic feedback.
-
- Vibrotactile widgets~\cite{frisson17,frisson20}
-
- Leverages vibrotactile feedback for touch surfaces.
-
- \subsection{Actuated computer peripherals}
- \label{sec:metamorphe-livingdesktop}
- % Leverage the physical properties of computer peripherals. Use them as tangibles~\cite{pietrzak17}.
- % Actuated peripherals.
+Figure~\ref{fig:activibesets} shows a visual representation of the pattern sets that we evaluated.
+Each individual squiggly line represents a single short pulse, while a long line represents a longer vibration.
+We first designed the series of vibration sets (A-E) that were evaluated in a first laboratory setting.
+The results of the first study helped us to design pattern F, which was then compared in a second laboratory study to the best sets from the first study (A, C, and E).
+
+\begin{figure}[htb]
+ \newcommand{\sine}[3]{\foreach \x in {1,...,{#3}}
+ {
+ \draw[thick]
+ ({4*\x*#1},0) sin
+ ({4*\x*#1+#1},#2) cos
+ ({4*\x*#1+2*#1},0) sin
+ ({4*\x*#1+3*#1},{-#2}) cos
+ ({4*\x*#1+4*#1},0);
+ }
+ }
+ \begin{tikzpicture}
+ \def\s{0.03}
+ \begin{scope}[] %Set A
+ \node[x=1mm,y=1mm, anchor=center] () at (14,-36){Set A};
+ \foreach \x in {1,...,10}
+ {
+ \begin{scope}[yshift=(-(\x-1)*10)]
+ \foreach \y in {1,...,\x}
+ {
+ \begin{scope}[xshift=(\y*6)]
+ \sine{\s}{.10}{1}
+ \end{scope}
+ }
+ \end{scope}
+ }
+ \end{scope}
+ \begin{scope}[xshift=3cm] %Set B
+ \node[x=1mm,y=1mm, anchor=center] () at (14,-36){Set B};
+ \foreach \x in {1,...,10}
+ {
+ \begin{scope}[yshift=(-(\x-1)*10)]
+ \sine{\s}{.10}{\x}
+ \end{scope}
+ }
+ \end{scope}
+ \begin{scope}[xshift=6cm] %Set C
+ \node[x=1mm,y=1mm, anchor=center] () at (14,-36){Set C};
+ \foreach \x in {1,...,10}
+ {
+ \begin{scope}[yshift=(-(\x-1)*10)]
+ \pgfmathtruncatemacro{\l}{20-2*\x}
+ \ifnum \x < 10
+ \sine{\s}{.10}{\l}
+ \fi
+ \pgfmathsetmacro{\ll}{\l*\s*40-1}
+ \begin{scope}[xshift=\ll mm]
+ \foreach \y in {1,...,\x} {
+ \pgfmathsetmacro{\m}{\y*\s*80}
+ \begin{scope}[xshift=\m mm]
+ \sine{\s}{.10}{1}
+ \end{scope}
+ }
+ \end{scope}
+ \end{scope}
+ }
+ \end{scope}
+ \begin{scope}[xshift=9cm] %Set D
+ \node[x=1mm,y=1mm, anchor=center] () at (14,-36){Set D};
+ \foreach \x in {10,...,1}
+ {
+ \begin{scope}[yshift=(-(10-\x)*10)]
+ \foreach \y in {1,...,\x} {
+ \pgfmathsetmacro{\m}{\y*\s*80}
+ \begin{scope}[xshift=\m mm]
+ \sine{\s}{.10}{1}
+ \end{scope}
+ }
+ \pgfmathtruncatemacro{\l}{20-2*\x}
+ \pgfmathsetmacro{\ll}{\x*\s*80+2}
+ \ifnum \x < 10
+ \begin{scope}[xshift=\ll mm]
+ \sine{\s}{.10}{\l}
+ \end{scope}
+ \fi
+ \end{scope}
+ }
+ \end{scope}
+ \begin{scope}[xshift=12cm] %Set E
+ \node[x=1mm,y=1mm, anchor=center] () at (14,-36){Set E};
+ \foreach \x in {1,...,10}
+ {
+ \begin{scope}[yshift=(-(\x-1)*10)]
+ \foreach \y in {1,...,\x} {
+ \pgfmathsetmacro{\m}{\y*\s*80}
+ \begin{scope}[xshift=\m mm]
+ \sine{\s}{.10}{1}
+ \end{scope}
+ }
+ \pgfmathtruncatemacro{\l}{20-2*\x}
+ \pgfmathsetmacro{\ll}{\x*\s*80+2}
+ \ifnum \x < 10
+ \begin{scope}[xshift=\ll mm]
+ \sine{\s}{.10}{\l}
+ \end{scope}
+ \fi
+ \end{scope}
+ }
+ \end{scope}
+ \begin{scope}[xshift=15cm] %Set F
+ \node[x=1mm,y=1mm, anchor=center] () at (14,-36){Set F};
+ \foreach \x in {1,...,4}
+ {
+ \pgfmathtruncatemacro{\xx}{\x-1}
+ \begin{scope}[yshift=(-\xx*10)]
+ \foreach \y in {0,...,\xx}
+ {
+ \begin{scope}[xshift=(\y*6)]
+ \sine{\s}{.10}{1}
+ \end{scope}
+ }
+ \end{scope}
+ }
+ \begin{scope}[yshift=-40]
+ \sine{\s}{.10}{8}
+ \end{scope}
+ \foreach \x in {1,...,4}
+ {
+ \begin{scope}[yshift=(-(\x+4)*10)]
+ \sine{\s}{.10}{8}
+ \begin{scope}[xshift=26]
+ \foreach \y in {1,...,\x}
+ {
+ \begin{scope}[xshift=(\y*6)]
+ \sine{\s}{.10}{1}
+ \end{scope}
+ }
+ \end{scope}
+ \end{scope}
+ }
+ \begin{scope}[yshift=-90]
+ \sine{\s}{.10}{8}
+ \begin{scope}[xshift=32]
+ \sine{\s}{.10}{8}
+ \end{scope}
+ \end{scope}
+ \end{scope}
+% \node[x=1mm,y=1mm, anchor=center] () at (70,-36){Set C};
+% \node[x=1mm,y=1mm, anchor=center] () at (103,-36){Set D};
+% \node[x=1mm,y=1mm, anchor=center] () at (136,-36){Set E};
+% \node[x=1mm,y=1mm, anchor=center] () at (160,-36){Set F};
+ \end{tikzpicture}
+ \caption[6 pattern sets evaluated in Activibe.]{Visual representation of the 6 pattern sets we evaluated in two laboratory studies, and a longitudinal study.}
+ \label{fig:activibesets}
+\end{figure}
+
+\paragraph{Design Rationale}
+Our design is driven by the semantics of the values we want to convey.
+Our intent is to represent discrete values of a progression.
+We explored two possibilities: 1) represent the actual value only; 2) represent the value as well as the scale.
+
+%\paragraph{Duration Only}
+When representing the actual value only, the duration of the vibrotactile pattern depends on the value it represents (Figure~\ref{fig:activibesets}, sets A \& B).
+The pattern has a short average duration, and thus it may be hard to understand the distance to the end of the event represented.
+We distinguish two variations.
+In set A, each value is represented by a series of short pulses, separated by short pauses.
+In set B, a continuous vibration represents each value with the duration corresponding to the value.
+
+The disadvantage of representing only the value is that even if the user has an idea of the current value, there is no clue about the distance between this value and the maximum value.
+Introducing a scale enables the positioning of a value relative to the beginning and the end of a progression.
+Sets C-E represent both the value and the scale of a progression in several ways.
+Either the current value is represented by a series of short vibrations and the scale by filling the sequence with a long vibration, either before or after the value (Figure~\ref{fig:activibesets}, sets C \& E), or the current value is represented by a long vibration and the scale is represented by filling the sequence with a series of short vibrations (Figure~\ref{fig:activibesets}, set D).
+Set F was defined using the results from the first laboratory study as a combination of short pulses and long vibrations.
+
+\begin{table}[htb]
+ \centering
+ \renewcommand{\arraystretch}{0.7}
+ \begin{tabular}{ c c c c }
+ \toprule
+ \textbf{Set} & \textbf{Information} & \textbf{Representation} & \textbf{Padding}\\
+ \midrule
+ A & Value only & Shorts & \\
+ B & Value only & Long & \\
+ C & Value + scale & Shorts (padding: Long) & Before \\
+ D & Value + scale & Long (padding: Shorts) & Before \\
+ E & Value + scale & Shorts (padding: Long) & After \\
+ F & Value + scale & Long + Shorts & \\
+ \bottomrule
+ \end{tabular}
+ \caption[Design space of the Activibe pattern sets.]{Design space of the Activibe pattern sets.}
+ \label{tab:activibedesignspace}
+\end{table}
+
+Table~\ref{tab:activibedesignspace} summarizes the design space of the vibration pattern sets we developed for our laboratory studies.
+We were interested in knowing which sets of icons were most suitable for representing progression values, and what is the best precision we can obtain using these representations.
+We can clearly see that we did not explore all the possible combinations.
+This is mostly due to the limited number of sets we could reasonably evaluate in a laboratory study.
+Other dimensions are also be relevant, like adding a warning vibration before the actual code, to help users paying attention to the signal while doing their daily activities.
+However in some cases, the padding vibration more or less play this role.
+In any case we analyzed this aspect in the longitudinal study.
+
+\paragraph{Parameter values}
+
+We ran short pilot studies to estimate the shortest vibration pulse that users are able to perceive with the apparatus (Pebble watch), as well as the shortest pause between two vibrations so that users can distinguish multiple short pulses.
+We obtained 100ms for the pulse and 150ms for the break between two pulses.
+Therefore, the longest patterns last less than 3 seconds.
+
+In the longitudinal study, we increased the vibrations durations given that participants will not be paying as much attention to the vibration as they had in the laboratory studies.
+Short vibration lasted $150ms$, long vibration $600ms$, and pauses were $200ms$ long.
+Finally, the pre-vibration were long vibrations of $700ms$.
+
+\subsubsection{Evaluation}
+
+I will not give the details about the evaluations in this study.
+I will rather discuss the main findings with hindsight.
+Readers interested in the details about the experiment protocols and the results should refer to \cite{cauchard16}.
+
+\paragraph{Counting VS duration}
+
+The first findings are related to the participants' strategy for interpreting the patterns.
+Either they counted the short vibrations, or estimated the duration of the long ones.
+The results showed that it is easier to count vibrations than estimating the duration.
+This is at least true within the tested range: 1-10, and $3s$ maximum.
+Indeed, we believe it would be much more complicated to keep the count with higher values.
+Durations are much complex to interpret, in particular because of the perception of time.
+Other modalities can influence the perception of the duration of haptic signals~\cite{grondin09}.
+
+Now, sets C-E contained both short and long vibrations that potentially both encoded the target number.
+Participants clearly stated that they counted the short vibrations to identify the answer.
+Therefore, with set D in which the sequence of small vibrations was a padding before the long vibration supposedly representing the answer, participants reported counting backwards.
+
+The set F was designed after the first user study.
+The rationale was to reduce the counting task by using a long vibration for marking 5, and additional short vibrations to represent increments.
+Therefore, users have to count up to 5 short vibrations, and they do not have to estimate the duration of long vibrations.
+We compared this set with sets A, C and E in a second laboratory study.
+Participants identified these patterns with the same or better performance than the patterns of the other sets.
+
+\paragraph{Distance between input and answer}
+
+In the context of activity monitoring, the distance between the correct value and the user answer matters.
+Indeed, interpreting a $60\%$ progression instead of $70\%$ is a lesser concern than interpreting $20\%$ instead of $70\%$.
+Therefore we analyzed this prrecision as a measure of distance between the input and the actual answer.
+We observed differences in terms of precision across pattern sets.
+For example with sets A and B, the precision gets worse as the number is high.
+However, the proportions are larger with set B, which confirms the idea that counting short vibrations is easier than estimating the duration of long vibrations.
+
+Interestingly, with set D the participants' precision was higher on the middle-range values.
+We explain this with the confusion due to the numbers being represented by the duration of the long vibrations at the end of the patterns.
+Since participants reported they counted backwards, this result shows that they sometimes forgot to do so.
+It had a strong influence on the result, which is another reason to reject this patterns set.
+
+\paragraph{Pre-attentive signals}
+
+Many participants mentioned that with pattern set C, they liked the long vibrations, which helped them focusing on the upcoming short vibrations and cound them.
+We believed this aspect would be important in a real context when users receive information while they perform their daily activities.
+Therefore we conducted a 28-days longitudinal study.
+Participants were presented random numbers of set F.
+Half-of them received a pre-attentive vibration of $700ms$ between the actual pattern.
+In order to make sure participants do not expect the notifications, they were sent twelve vibrations per day at semi-random times within one-hour window between 7am and 8pm.
+This schedule helped to cover different activities from the users such as their commute, when they bring their kids to school, their day at work, and some evening activities.
+
+The results did not show a significant difference in terms of answer accurracy rate.
+However, when discussing with participants after the experiment, most of the group who did not have the vibration answered that a pre-vibration would have been useful.
+Almost all of the participants in the group who had it answered that the pre-vibration was useful to them.
+So despite a lack of quantitative improvements, we recommend to provide a pre-attentive vibration because users prefer to have it.
+
+%Can people notice and interpret correctly information when they do not expect the tactile cues?
+
+\subsubsection{Discussion and Conclusion}
+
+Participants identified the patterns with a high accurracy both in the laboratory (96\%) and in-situ (89\%).
+We note that in the longitudinal study, the patterns were presented in a random order.
+However in an activity monitoring scenario, they would be presented in an increasing order.
+Hence we conjecture that participants could identify them better.
+
+The main limitation of this work is certainly its scalability.
+The most efficient technique for interpreting the patterns is counting the vibrations, and distinguishing short and long vibrations.
+We only used 10 patterns in each set, but if we would like to expand it to larger sets, we will need another way of coding numbers.
+
+Finally, our starting constraint of this work was using an off-the-shelf wearable device.
+We used a Pebble watch, which includes a low-quality ERM actuator.
+Indeed, it strongly limited the output vocabulary we could use.
+However haptic technologies are evolving, and we can expect better actuators even in consumer electronics products in the future.
+Therefore it would be interesting to push this work forward with better vibrotactile actuators \cite{mortimer07,yao10}, and use other tactile parameters such as frequency.
+
+\subsection{Tactile textures}
+\label{sec:stimtac}
+
+command/effect relation
+singularity of the effects produced by research prototypes
+
+Definition
+
+Tactile Textures~\cite{potier12,potier16}
+
+\subsection{Vibrotactile widgets}
+\label{sec:printgets}
+
+Replacing physical controls with touchscreens have advantages: updates, reconfigurable, visual feedback, but most of haptic properties are lost: click sensations of buttons, detents on slides. Impact on interaction. Technologies to restore haptic feedback.
+
+Vibrotactile widgets~\cite{frisson17,frisson20}
+
+Leverages vibrotactile feedback for touch surfaces.
+
+\subsection{Actuated computer peripherals}
+\label{sec:metamorphe-livingdesktop}
+% Leverage the physical properties of computer peripherals. Use them as tangibles~\cite{pietrzak17}.
+% Actuated peripherals.
In the early days of tangible interaction, Ullmer and Ishii described Tangible interaction this way: “TUIs will augment the real physical world by coupling digital information to everyday physical objects and environments.”\cite{ishii97}.
The idea is to break the barrier between the physical and the digital world.
projects / hdr.git / commitdiff