\begin{figure}[htb]
\centering
\includegraphics[width=\textwidth]{figures/wristband3protos}
- \caption[Haptic wristband prototypes]{Three iterations of haptic wristband prototypes. On the first prototypes, components were connected with conductive thread. On the second prototype, the components were soldered on protoboards that were connected with conductive thread. The third prototype used a homemade PCB connected with wires.}
+ \caption[Haptic wristband prototypes.]{Three iterations of haptic wristband prototypes. On the first prototypes, components were connected with conductive thread. On the second prototype, the components were soldered on protoboards that were connected with conductive thread. The third prototype used a homemade PCB connected with wires.}
\label{fig:wristbandprototypes}
\end{figure}
\section{Conclusion}
+In this chapter, I discussed the sensorimotor loop, how it is connected to many research domains, and aspects of the design, implementation, and evaluation of interactive systems.
+I discussed two examples of rejected submissions in which the work had the common fundamental flaw that we overlooked the sensorimotor loop.
+The reason why it happened is probably that my previous research that leveraged the sense of touch barely required inputs because they were essentially notification mechanisms.
+Later, I worked on the motor ability and visual feedback was sufficient.
+The interesting fact with these studies is that people who tried my 3D gestural interaction prototypes with haptic feedback appreciated this feedback, even if we did not measure quantitative benefits, and barely observed qualitative benefits.
+%People are heavily disturbed when they receive no visual feedback, and most of the time they cannot interact anymore.
+%Without the sense of touch they still can, they can adapt and compensate with their other senses.
+% To understand this
+% My question is: why are people disturbed when they have no visual feedback to the the point that they cannot interact anymore but at the same time they can adapt to the absence of
+%I later described contributions that provided quantitative and others that provided qualitative benefits.
+%These contributions consisted in a
+
+The issue with the initial quantitative study was basically that the gestural interaction we used for target selection was too inefficient.
+The dwell time to activate the button was \qty{3}{\second} while the mean selection time was \qty{3.5}{\second}.
+This means that \qty{85}{\percent} of the selection time is inevitable.
+The \qty{3}{\second} correspond to replications of selection in Kinect applications.
+There is certainly a slight progression margin, but nothing close to fast selection like with a mouse or a touchpad.
+Therefore, we proposed a new interaction paradigm for 3D gestural interaction that replaces point \& select by summon \& select.
+This new paradigm avoids the typical issues of gestural interaction: lack of segmentation, Midas touch, and tracking in general.
+However, these advantages are balanced with new potential issues.
+This paradigm relies on semaphoric postures and gestures that users have to know beforehand.
+Therefore there are potentially guessability and learnability issues.
+We did not study this aspect, but it has to be addressed or at least considered before implementation in an actual interactive system.
+
+The initial qualitative study showed an increase of presence thanks to tactile feedback, in particular for the \emph{sensory} and \emph{realism} factors.
+Some of the controls remained difficult though.
+The task consisted in driving a racing car with the hands in the air as if participants were holding a steering wheel.
+However, a physical steering wheel has many advantages in addition to precise and reliable steering.
+In particular, users can rest their arms on the steering wheel.
+But the difficulty with the input mapping is that both the steering and acceleration/braking degrees of freedom were integrated.
+It made it difficult to brake and steer at the same time, which is a common task when driving.
+Our next studies on the qualitative aspect of haptic feedback focused on the sense of embodiment in immersive virtual reality.
+On one hand, we showed that participant answered embodiment questionnaires relatively, which mean they either need calibration or we must use a within-subjects design.
+On the other hand, we showed that force feedback has an effect on embodiment for a painting task.
+We did not observe any effect of tactile feedback on embodiment.
+We explained this difference by the fact that force feedback prevented interpenetration, therefore participants could focus on the task.
+We must investigate other tasks in which tactile feedback contributes to sensorimotor integration.
+
+% My interpretation is that when we interact with physical objects our sense of touch is constantly stimulated.
+% Whether when we press buttons, when we grab a pen, or turn a steering wheel.
+% Even without physical objects we feel the wind and heat.
+% When we interact in virtual environments without haptic feedback, many aspects could visually look realistic and similar to physical interaction.
+% %This is why these are often called \emph{natural} user interfaces.
+% However, the sense of touch is not stimulated.
+
+% My explanation is that we essentially rely on our visual sense and most of our visual exploratory motions are unconscious.
+% Therefore the sensorimotor loop is the reason why vision is so efficient, and we never experience vision without it.
+% The sense of touch is a bit different because our exploratory motions involve our limbs and hands.
+% These manipulations are often conscious, and the situation is inverted.
+% We get sensations from our sense of touch, and we cannot touch without getting these sensations.
+% But the focus here is often on our manipulations rather than our sensations.
+% %This is probably why we can perform visuomotor tasks with little to no haptic feedback in virtual environments.
+
+% So, adding haptic feedback to gestural interaction does not bring something new, but restore something that should have been there right at the beginning.
+% My focus on the sensorimotor loop is also motivated by my impression that this is the most underrated mechanism of interactive systems.
+
+The interaction models I describe mostly describe current good practices.
+Therefore I tend to claim they have descriptive properties.
+But at this stage, I am still uncertain of their generative properties.
+These are the results of years of thinking about what started as an attempt to make a bridge between HCI and theoretical computing.
+Theoretical computing, and in particular logic was a large part of my educational background.
+It significantly shaped my vision of science at the beginning of my career.
+When I started thinking about this document years ago I struggled with avoiding crossing the boundaries of my expertise in these topics.
+There is therefore much more to say on the connections between interaction and computing, but with less confidence about relevance.
+%But it is more opinions that scientific
+This is also the reason why I needed to make the connection between interaction and design models, and shape my own vision of HCI research that I put into practice in my research.
+
+
+% For example, we can think about the question whether a machine that fullfill all our needs can exist.
+% This would not only require to know the current state of our environment, but also its future state.
+% Part of it is predictable, but the environment is essentially unpredictable.
+% We can only design systems with our current knowledge of the environment, and later evolve them later when this knowledge improves.
+% Therefore even autonomous systems rely on the flexibility and adaptability of the human mind.
+% I see here a similar pattern than Gödel's' first incompleteness theorem~\cite{godel31} that basically says that in any sufficiently expressive formal systems there are theorems that we cannot prove with this system.
+% To prove these theorems we must extend this formal system with additional axioms or inference rules.
+% %This reasoning worths what it is worth.
+% %However, it shows the intertwined relationship between people and machines at a meta level.
+% %
+% Following this idea, we can also compare an application to a living entity.
+% Its code is the equivalent of DNA and its execution is analogous to life.
+% New versions of this application are its next generations.
+% Humans have the role of the nature: we reproduce them, and we make them evolve.
+
+% Interaction is another way to leverage skills and capacilities of other entities, in particular humans.
+% %Wegner describe several kinds of what he calls \defwords{interaction machines}{interaction machine}.
+% Wegner describe several kinds of interaction machines.
+% For example, he describes a machine that simply echoes an input stream to an output stream~\cite{wegner97} to demonstrate the power of the interaction phenomenon.
+% Such a basic machine connected to two humans can pass the Turing test without being particularly intelligent.
+% It can win half of chess games between these two people.
+% % When I discussed this with \fixme{Gérard Berry}, he said this was cheating.
+% %\todo{Should I remove the name?}
+% When I discussed this with theoretical computer scientists, they said this was cheating.
+% But it is no different from AlphaZero which processes human knowledge to beat chess world champions~\cite{silver18}.
+% When the machine wins the game the reward goes to the machine designer not to the machine.
+% Therefore, even the skills of the most powerful machines are the result of an interaction between humans and the machine.
+% %In fact in this latter case we often cite it as a superiority of machines.
+% Therefore I consider machines as crystallized human knowledge and behavior at a given time.
+% They process data and perform operations with this knowledge and behavior state until humans update them to a newer knowledge and behavior state.
+
+% The important question is therefore not whether we should replace all human labor by machines.
+
+
+
+% \begin{idee}
+% Something about this incompletude phenomenon: ideas on how to tackle the problem
+
+% Gödel's' first incompleteness theorem: in any consistent formal system F within which a certain amount of arithmetic can be carried out, there are statements of the language of F which can neither be proved nor disproved in F.
+
+% The interactive incompleteness theorem: in any non trivial interactive program, there are desirable features which are not available.
+
+% The interactive incompleteness theorem v2: in any non trivial interactive program, there are tasks users cannot perform with available features.
+
+% Software evolution\cite{mens05}
+
+% Interactive programs have limited number of features and capabilities.
+% They are bounded by their inputs.
+% More features and capabilities are possible, but new input is necessary.
+% It can be new inputs, other kind of inputs or higher precision inputs.
+% In any case the program must evolve to take these changes into account.
+% These changes are made by humans.
+
+% Conway's Law~\cite{conway68}: Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.
+
+% Formal systems deal with Gödel's incompleteness theorem by adding axioms. Interactive programs add features.
+% \end{idee}
+
-Interdisciplinarity: better describe the connections between domainns involved in the design, implementation and evaluation of interactive systems.
-One of the main difficulties of interdisciplinarity is to understand each other's research questions and roles.
-A common model gives space to everybody.
-
-Sensorimotor loop and its numerous connections to many domains.
-
-Descriptive and predictive aspect: ok
-Generative aspect?
-
-%Arduino made electronics mainstream / available to hobbyists
-
+% Descriptive and predictive aspect: ok
+% Generative aspect?
+% Arduino made electronics mainstream / available to hobbyists
-\todo{Maybe move stuff below to the discussion…}
-Wegner describe several kinds of what he calls \defwords{interaction machines}{interaction machine}.
-He gives the a machine that simply echoes an input stream to an output stream~\cite{wegner97} to demonstrate the power of the interaction phenomenon.
-Such a basic machine connected to two humans can pass the Turing test without being particularly intelligent.
-It can win half of chess games between these two people.
-When I discussed this with \fixme{Gérard Berry}, he said this was cheating.
-\todo{Should I remove the name?}
-But it is no different from AlphaZero which processes human knowledge to beat chess world champions~\cite{silver18}.
-When the machine wins the game the reward is for the machine designer not for the machine.
-%In fact in this latter case we often cite it as a superiority of machines.
-Further, machines are crystallized human knowledge and behavior at a given time.
-They process data and perform operations with this knowledge and behavior state until humans update them to a newer knowledge and behavior state.
+% \todo{Maybe move stuff below to the discussion…}
-Implementation depends on ethnographic background of programmers \cite{rode04}
+% Implementation depends on ethnographic background of programmers \cite{rode04}
-Software architecture reproduce the organization structure\cite{conway68}
+% Software architecture reproduce the organization structure\cite{conway68}
-\begin{idee}
- We observe that any changes in either the human or system behavior will affect the other entity.
- This is known as \defword{co-adaptation}~\cite{mackay90}.
+% \begin{idee}
+% We observe that any changes in either the human or system behavior will affect the other entity.
+% This is known as \defword{co-adaptation}~\cite{mackay90}.
- => Towards computing affordance, a generalized notion of computability
+% => Towards computing affordance, a generalized notion of computability
-Pb: it reacts always the same way to the same entries. Humans tend to evolve. => ML?
-Flexibility? Adaptability?
+% Pb: it reacts always the same way to the same entries. Humans tend to evolve. => ML?
+% Flexibility? Adaptability?
-Curiosity~\cite{laversannefinot18}
+% Curiosity~\cite{laversannefinot18}
-Computers are human inventions, and they are build to follow human-made specifications.
-Their behavior is
+% Computers are human inventions, and they are build to follow human-made specifications.
+% Their behavior is
-Control and automation: Moravec's paradox
-Moravec paradox: sensorimotor and perception skills are harder for machines and processing intelligence is harder for humans\cite{moravec88}
-=> intelligence reduced to information processing
+% Control and automation: Moravec's paradox
+% Moravec paradox: sensorimotor and perception skills are harder for machines and processing intelligence is harder for humans\cite{moravec88}
+% => intelligence reduced to information processing
-\end{idee}
+% \end{idee}
\addcontentsline{toc}{chapter}{\nameref{chap:ccl}}
\addcontentsline{lof}{chapter}{\nameref{chap:ccl}}
-Something about this incompletude phenomenon: ideas on how to tackle the problem
+In the previous chapters I either focused on output and the sense of touch, or input and the motor ability.
+The models I described in these chapters corresponded to my initial vision of interactive systems with separate input and output pipelines.
+I showed interesting contributions made with this vision.
+It is relevant in notification systems in which input is limited anyway.
+But even in these chapters the sensorimotor loop was already there.
+How could it be different?
+In this chapter I wanted to describe how thinkg with the sensorimotor loop in mind changed the way I approach HCI research.
+I described the foundations of the field, and their connections to many other research domains.
+Of course, each of these notions were individually studied for decades since their discovery, and they are still studied and extended today.
+But instead of focusing on a precise notion, I rather wanted to take this opportunity to look at the whole picture.
+It felt like a giant mind puzzle because all these notions are connected.
+It reflects the systemic approach in my research, and life in general.
+I like to understand all the sides of a problem rather than focusing on a specific aspect.
-\section{Future work}
-\begin{idee}
-Interactive programs have limited number of features and capabilities.
-They are bounded by their inputs.
-More features and capabilities are possible, but new input is necessary.
-It can be new inputs, other kind of inputs or higher precision inputs.
-In any case the program must evolve to take these changes into account.
-These changes are made by humans.
-\end{idee}
-\begin{idee}
-Gödel's' first incompleteness theorem: in any consistent formal system F within which a certain amount of arithmetic can be carried out, there are statements of the language of F which can neither be proved nor disproved in F.
-The interactive incompleteness theorem: in any non trivial interactive program, there are desirable features which are not available.
+\begin{idee}
-The interactive incompleteness theorem v2: in any non trivial interactive program, there are tasks users cannot perform with available features.
-Software evolution\cite{mens05}
+ Interdisciplinarity: better describe the connections between domainns involved in the design, implementation and evaluation of interactive systems.
+ One of the main difficulties of interdisciplinarity is to understand each other's research questions and roles.
+ A common model gives space to everybody.
+
+ Sensorimotor loop and its numerous connections to many domains.
+\end{idee}
-Conway's Law~\cite{conway68}: Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.
-Formal systems deal with Gödel's incompleteness theorem by adding axioms. Interactive programs add features.
-\end{idee}
+\section{Future work}
\begin{idee}
change blindness for haptic
projects / hdr.git / commitdiff