The contributions below use the orthogonal approach as discussed above to improve interaction by leveraging the sensorimotor loop.
The first contribution is two interaction paradigms that leverage gestural interaction and vibrotactile feedback.
%The first one use semaphoric gestures to replace pointing in mid-air gestural interaction.
-The second contribution investigates the contribution of haptics on the embodiment of an avatar in Virtual Reality.
+The second contribution investigates the contribution of haptics on the sense of embodiment of an avatar in Virtual Reality.
\subsection{Haptic interaction paradigms}
\label{sec:hapticparadigms}
%Finger count \cite{bailly10}
%Shoesense \cite{bailly12}
-The main limitations we discussed in \refsec{sec:limits} are tracking difficulties and a lack of segmentation.
+The main limitations of 3D gestural interaction we discussed in \refsec{sec:limits} are tracking difficulties and the lack of segmentation.
The users are tracked without interruption.
Therefore gesture segmentation is difficult.
Moreover, every gesture the users perform are potentially interpreted.
This is called \defword{Midas Touch}, as a reference of the curse of the king that turned everything he touched into gold in the Greek mythology.
There are also issues when the user is outside the sensor field of view, either on the edges or when it is occluded.
The users need additional feedback for this in order to avoid usability issues.
+And even though, it makes interaction more complicated.
These issues make it difficult to use standard GUI widgets.
We discussed in \refsec{sec:limits} the simple case of buttons that require a different activation mechanism.
3D gestural interfaces typically use dwell buttons that require users to hold their hand still over a button for a couple of second to select it.
We proposed to simply add haptic feedback, but we were unable to measure a quantitative benefit.
I believe we need deeper changes to improve interaction in this context.
-Therefore we proposes a different paradigm that does not rely on pointing \& selection.
+Therefore we proposed a different paradigm that does not rely on pointing \& selection.
This new paradigm relies on summoning \& selection~\cite{gupta17}.
This paradigm leverages a combination of semaphoric gestures, continuous gestures and tactile feedback (\reffig{fig:summonexample}).
Then we defined a different hand posture for different kinds of widgets: buttons, sliders, knobs, switches, spinboxes, and paired buttons.
It is of course possible to add other kinds of widgets with other hand postures.
When the users perform one of these postures, they can select one of the widgtes of this type.
-They receive a \qty{150}{\ms}/\qty{350}{\hertz} vibration pulse to confirm the selection, and the currently selected widget is highlighted.
+They receive a \qty{150}{\ms}/\qty{350}{\hertz} vibration pulse to confirm the selection, and the currently selected widget is visually highlighted.
If the GUI has several widgets of this type, the users can disambiguate with a continuous movement.
Then they perform a gesture for manipulating the widget, and receive immediate haptic feedback with continuous vibrations on the thumb and index finger.
For example they can pinch and drag to move a slider knob.
We conduct user studies to measure the benefits of this new paradigm.
In the first one we showed that this paradigm avoids the Midas touch issues, and we compared two disambiguation mechanisms.
-In the second study we showed that this paradigm has quantitative and qualitative benefits compared to pointing.
+In the second study we showed that this paradigm has quantitative and qualitative benefits compared to midair pointing.
Despite these benefits, this new paradigm has challenges that are still to be addressed.
In particular it relies on semaphoric gestures that users have to know.
It contradicts Nielsen's \emph{recognition rather than recall} heuristic~\cite{nielsen90,nielsen94}, which is one of the essential benefits of the point \& select paradigm.
-Therefore, we still have to evaluate the discoverability and learnability of the gestures, and improve them if necessary.
+Therefore, we still have to evaluate the discoverability and learnability of the gestures, and improve them if necessary~\cite{cockburn14}.
We can for example encourage learnability and discoverability with feedforward visual cues in the vicinity of the widgets~\cite{malacria13}.
\input{figures/summonexample.tex}
It is one of the most important concepts of GUIs.
Its properties provide valuable usability benefits that highly contributed to the success of GUIs over command line interfaces.
Yet, this paradigm was tailored for visual interfaces.
-The question whether this concept could be used in tactile display was open.
+The question whether this concept could be used or adapted to tactile display was open.
Therefore we studied the adaptation of direct manipulation to tactile displays~\cite{pietrzak15,gupta16,gupta16a}.
The most challenging direct manipulation property for tactile display is certainly the one stating that objects of interest have to be visible.
The display is divided into four quarters in between the actuators.
The cursor is a phantom sensation created by interpolating the signal amplitude of the two edge actuators of the corresponding quarter.
Targets are represented with a \qty{250}{\hertz} frequency and the background with \qty{100}{\hertz}.
-Not only it is an easily distinguishable vibration, but the \qty{100}{\hertz} is subtle and avoids numbness.
+Not only it is an easily distinguishable vibration, but the \qty{100}{\hertz} is subtle and avoids or at least reduce numbness.
The inputs use a multitouch smartwatch.
Up and down swypes move the cursor in either directions.
The tracking states trigger with one contact point and the dragging state triggers with two contact points.
The users are generally represented in virtual environments with an \defword{avatar}.
This avatar usually has a visual representation, which is not necessarily realistic or even human~\cite{olivier20}.
The users explore the virtual environment through this avatar.
-They can also perform operations that the users cannot in the physical world like telekinesis or teleportation.
+They can also perform operations thatare impossible in the physical world, like telekinesis or teleportation.
In fact, the appearance or behavior of the avatar has an influence on the way the users behave in the virtual environment.
For example, the \defword{Proteus effect} describes the way the visual representation of an avatar influences the behavior of the users that control it~\cite{yee07}.
At the opposite, visuo-tactile stimulation can lead people to consider a rubber hand as part of their body~\cite{botvinick98}, or that they have a sixth finger on their hand~\cite{hoyet16}.
\subsubsection{Methodologies for measuring the sense of embodiment}
-There is a number of embodiment questionaires in the literature to measure the sense of embodiment.
+There is a number of questionaires in the literature to measure the sense of embodiment.
We discuss some of them in one of our studies~\cite{richard22}.
%We can measure the embodiment of an avatar in a virtual environment with questionnaires~\cite{roth20,gonzalezfranco18,peck21}.
There are recent attemps to standardize these questionnaires.
-For example Roth \etal propose a questionnaire with \defword{ownership}, \defword{agency}, and perceived change in the \defword{body schema}~\cite{roth20}.
+For example Roth \etal propose a questionnaire with subcomponents: \defword{ownership}, \defword{agency}, and perceived change in the \defword{body schema}~\cite{roth20}.
The latter notion is larger than \emph{self location} as it refers to any difference the users may perceive between their own body and the avatar.
Gonzalez Franco and Peck proposed another questionnaire in which they added to Kilteni \etal's subcomponents : \emph{tactile sensations}, \emph{external appearance}, and \emph{response to external stimuli}~\cite{gonzalezfranco18}.
They later improved and simplified their questionnaire, and evaluated it with many different tasks~\cite{peck21}.
In a between-subjects study, users are assigned to one of the conditions.
There is therefore potentially a bias if the groups are not well balanced.
We investigated this effect on embodiment studies~\cite{richard22}.
-We experimented a visuo-motor task with a synchronous condition and an asynchronous condition with a latency of \qty{300}{\ms} between the input and output response.
+We experimented a visuo-motor task with a synchronous condition and an asynchronous condition with a latency of \qty{300}{\ms} between the inputs and output response.
This value is known to have a medium effect on embodiment in the literature~\cite{botvinick98,kilteni12,kokkinara14}.
We chose a simple experimental task that requires no special equipment to facilitate replication.
Participants were seated on a chair, with the legs on a table, and had to perform gestures with their feet (\reffig{fig:expewithin}), similarly to~\cite{kokkinara14}.
92 participants performed this task in a balanced within-subjects design.
To study the effect of the sample size and its effect on the statistical analysis we analyzed random data subsets of 10 to 92 participants.
-To study the effect of the experiment design we simulated between-subjects design by selecting the first condition of participants.
+To study the effect of the experiment design we simulated between-subjects design by selecting the first condition ever participant made.
We considered the analysis of all participants with the within-subjects design as the ground truth, which gave the same result as the literature~\cite{botvinick98,kilteni12,kokkinara14}.
\begin{figure}[htb]
Our results show that all the random subsets with at least \num{40} participants with the within-subjects design gave the same result as the ground truth.
However, regardless of the number of participants the between-subject analyses do not reveal the ground truth effect.
-Based on the debrieffing with participants, our main explanation of this phenomenon is that participants needed a reference to provide a value for each question.
+Based on the debrieffing with participants, our main explanation of this phenomenon is that participants needed a reference to provide a meaningful answer for each question.
Therefore they calibrated their answers to the second condition relatively to the first one.
Hence, we could not measure the effect with the first condition only.
We discuss recommendation and possible mitigation strategies in the paper~\cite{richard22}.
However, we did not observe these differences between the tactile and control conditions.
Besides the detailed discussion in the paper, it is important to note that in some ways this task favored the force feedback condition over the tactile condition.
Participants certainly expected to feel the stiffness of hard surfaces.
-However, the vibrotactile feedback was symbolic because participants only received tactile guidance.
+Similarly to realistic visual feedback~\cite{argelaguet16}, this realistic force feedback aspect reinforced the sense of ownership
+At the contrary the vibrotactile feedback was symbolic because participants only received tactile guidance.
+And we did not observe any improvement of embodiment.
It does not necessarily mean that the sense of embodiment requires realistic haptic feedback.
-It is not the case for visual feedback~\cite{argelaguet16}.
-But in our task force feedback constrains the stylus tip movement to prevent it from getting through the surface.
-I believe it favored sensorimotor integration, therefore participants focused on the painting task rather than controlling the stylus to paint the canvas.
+For example, non-realistic visual feedback improved the sense of agency~\cite{argelaguet16}.
+But in our task force feedback \emph{constrained} the stylus tip movement to prevent it from getting through the surface, while vibrotactile feedback only \emph{guided} it.
+Therefore I believe the force feedback condition had a stronger sensorimotor integration, which helped participants focusing on the painting task rather than controlling the stylus to paint the canvas.
The workload analysis discussed in the paper gives supports this explanation.
%It gave users immediate feedback that could guide them to stay close to the spatial location of the surface.
Further studies should investigate other tasks or a variation of this one in which tactile feedback favors sensorimotor integration.
projects / hdr.git / commitdiff