\scrollbar{0}{-10}{black!50}{black!50}
\scrollbar{4}{-10}{myblue}{black!50}
\scrollbar{8}{-10}{black!50}{black!50}
- \node[] () at (0,-12) {\includegraphics[height=2cm]{figures/handpinch2}};
+ \node[] () at (-5,-12) {\includegraphics[height=2cm]{figures/handpinch2}};
\node (lbl) at (0,-30) {(1)};
\node (lbl2) at (0,-34) {Summon};
\end{scope}
\scrollbar{0}{-10}{myblue}{black!50}
\scrollbar{4}{-10}{black!50}{black!50}
\scrollbar{8}{-10}{black!50}{black!50}
- \node[opacity=0.2] () at (0,-12) {\includegraphics[height=2cm]{figures/handpinch2}};
- \node[] () at (0,-17) {\includegraphics[height=2cm]{figures/handpinch2}};
- \draw[-Stealth, line width=2] (0,-10) -- (0,-20);
+ \node[opacity=0.2] () at (-5,-12) {\includegraphics[height=2cm]{figures/handpinch2}};
+ \node[] () at (-5,-17) {\includegraphics[height=2cm]{figures/handpinch2}};
+ \draw[-Stealth, line width=2] (-5,-10) -- (-5,-20);
\node (lbl) at (0,-30) {(2)};
\node (lbl2) at (0,-34) {Disambiguate};
\end{scope}
\scrollbar{0}{-10}{myblue}{myred}
\scrollbar{4}{-10}{black!50}{black!50}
\scrollbar{8}{-10}{black!50}{black!50}
- \node[] () at (-5,-15) {\includegraphics[height=2cm]{figures/handpinch1}};
+ \node[] () at (-5,-17) {\includegraphics[height=2cm]{figures/handpinch1}};
\node (lbl) at (0,-30) {(3.1)};
\end{scope}
\begin{scope}[xshift={\partw * 4}]
\scrollbar{0}{-1}{myblue}{myred}
\scrollbar{4}{-10}{black!50}{black!50}
\scrollbar{8}{-10}{black!50}{black!50}
- \node[opacity=0.2] () at (-5,-15) {\includegraphics[height=2cm]{figures/handpinch1}};
- \node[] () at (5,-15) {\includegraphics[height=2cm]{figures/handpinch1}};
- \draw[-Stealth, line width=2] (-5,-15) -- (5,-15);
+ \node[opacity=0.2] () at (-5,-17) {\includegraphics[height=2cm]{figures/handpinch1}};
+ \node[] () at (5,-17) {\includegraphics[height=2cm]{figures/handpinch1}};
+ \draw[-Stealth, line width=2] (-5,-17) -- (5,-17);
\node (lbl) at (0,-30) {(3.2)};
\node (lbl2) at (0,-34) {Manipulate};
\draw (lbl2.west) -- ++(-15,0) (lbl2.east) -- ++(15,0);
\scrollbar{0}{-1}{myblue}{myred}
\scrollbar{4}{-10}{black!50}{black!50}
\scrollbar{8}{-10}{black!50}{black!50}
- \node[] () at (5,-15) {\includegraphics[height=2cm]{figures/handpinch2}};
+ \node[] () at (5,-17) {\includegraphics[height=2cm]{figures/handpinch2}};
\node (lbl) at (0,-30) {(3.3)};
\end{scope}
\begin{scope}[xshift={\partw * 6}]
%\todo{Interest in human behavior for the design of interactive systems: 1) take inspiration of ot and reproduce a similar behavior. 2) make a better connection between humans and systems}
The second reason to study human behavior is to improve interaction between humans and machines.
Humans and interactive systems are distinct entities that need each other and must communicate to achieve their objective.
-Hence we will discuss below the architecture of interactive systems, the similarities and differences with humans, and how this is critical for improving interactions between them.
+Hence we will discuss below the architecture of interactive systems, the similarities and differences with humans, and how this is critical for improving interactions.
\paragraph{Computation models}
Initially, computers and programs were essentially based on theoretical models such as \defword{$\lambda$-calculus}~\cite{church32} or \defwords{Turing machines}{turing machine}~\cite{turing38}.
\section{Contributions}
In the previous chapters I presented contributions to improve output by leveraging the sense of touch, and input by leveraging the motor abilities.
+In this chapter we discussed in the \refsec{sec:limits} that this approach are not always sufficient to improve interaction.
The contributions below use the orthogonal approach as discussed above to improve interaction by leveraging the sensorimotor loop.
The first contribution is two interaction paradigms that leverage gestural interaction and vibrotactile feedback.
%The first one use semaphoric gestures to replace pointing in mid-air gestural interaction.
-The second contribution investigates the contributionn of haptics on the embodiment of an avatar in Virtual Reality.
+The second contribution investigates the contribution of haptics on the embodiment of an avatar in Virtual Reality.
\subsection{Haptic interaction paradigms}
\label{sec:hapticparadigms}
-Device described in \refsec{sec:limits}.
+In \refsec{sec:limits} we first wanted to measure quantitative benefits of haptic feedback for gestural interaction.
+The interaction paradigm we used was so inefficient that haptic feedback could not compensate its limitations.
+Here we propose a new interaction paradigm that gets around these limitations.
+Then, we will discuss a new interaction paradigm that brings direct manipulation to tactile displays.
+I implemented both paradigms with the device described in \refsec{sec:limits}.
\subsubsection{Summon interactions}
\label{sec:summon}
-~\cite{gupta17}
-
-3D gestural interaction in the air faces difficult challenges due to the contact-less nature of this interaction style.
-The absence of physical buttons or other unambiguous activation action makes it difficult to segment gestures.
-As a consequence, the system constantly interprets the users' actions.
-Therefore the user is always potentially interacting with the environment.
-This is known as the Midas touch problem, as a reference to the Midas king in the greek mythology, who turned everything he was touching to gold.
+%Finger count \cite{bailly10}
+%Shoesense \cite{bailly12}
+
+The main limitations we discussed in \refsec{sec:limits} are tracking difficulties and a lack of segmentation.
+The users are tracked without interruption.
+Therefore gesture segmentation is difficult.
+Moreover, every gesture the users perform are potentially interpreted.
+This is called \defword{Midas Touch}, as a reference of the curse of the king that turned everything he touched into gold in the Greek mythology.
+There are also issues when the user is outside the sensor field of view, either on the edges or when it is occluded.
+The users need additional feedback for this in order to avoid usability issues.
+
+These issues make it difficult to use standard GUI widgets.
+We discussed in \refsec{sec:limits} the simple case of buttons that require a different activation mechanism.
+3D gestural interfaces typically use dwell buttons that require users to hold their hand still over a button for a couple of second to select it.
+We proposed to simply add haptic feedback, but we were unable to measure a quantitative benefit.
+I believe we need deeper changes to improve interaction in this context.
+Therefore we proposes a different paradigm that does not rely on pointing \& selection.
+This new paradigm relies on summoning \& selection~\cite{gupta17}.
+
+This paradigm leverages a combination of semaphoric gestures, continuous gestures and tactile feedback (\reffig{fig:summonexample}).
+We first defined an segmentation hand posture (an open hand) to summon the GUI elements.
+Then we defined a different hand posture for different kinds of widgets: buttons, sliders, knobs, switches, spinboxes, and paired buttons.
+It is of course possible to add other kinds of widgets with other hand postures.
+When the users perform one of these postures, they can select one of the widgtes of this type.
+They receive a \qty{150}{\ms}/\qty{350}{\hertz} vibration pulse to confirm the selection, and the currently selected widget is highlighted.
+If the GUI has several widgets of this type, the users can disambiguate with a continuous movement.
+Then they perform a gesture for manipulating the widget, and receive immediate haptic feedback.
+For example they can pinch and drag to move a slider knob.
+They can release the knob by releasing the pinch, and release the slider by performing the segmentation gesture again.
+
+% Upon summoning, a 150ms pulse is played in both rings to indicate that the slider is summoned. When the user enters the drag state, a continuous pulse starts playing in both rings to mirror the grip of the slider bar. The pulse stops upon exit from the drag state. To reduce any per- ceived irritability from the vibration, the amplitude was set just above the perceivable level and the frequency was set at 350Hz. The 150ms pulse is played again upon release.
+
+We conduct user studies to measure the benefits of this new paradigm.
+In the first one we showed that this paradigm avoids the Midas touch issues, and we compared two disambiguation mechanisms.
+In the second study we showed that this paradigm has quantitative and qualitative benefits compared to pointing.
+Despite these benefits, this new paradigm has challenges that are still to be addressed.
+In particular it relies on semaphoric gestures that users have to know.
+It contradicts Nielsen's \emph{recognition rather than recall} heuristic~\cite{nielsen90,nielsen94}, which is one of the essential benefits of the point \& select paradigm.
+Therefore, we still have to evaluate the discoverability and learnability of the gestures, and improve them if necessary.
+We can for example encourage learnability and discoverability with feedforward visual cues in the vicinity of the widgets~\cite{malacria13}.
\input{figures/summonexample.tex}
projects / hdr.git / commitdiff