% 6) Wrist ulnar/radial bend %(19/16)
% 7) Wrist flexion extension %(62/40)
Knowledge about the range of movements and maximum forces is necessary for the design and layout of workstations.
-For example, NASA documents these values with and with gravity, or with pressurization because they need to provice precise and documented specifications for spacecrafts~\cite{nasa14}.
-They also use these specifications to design clothings that astronauts can wear comfortably to perform routine tasks.
-In our case, we design input systems.
-Therefore we need to know the range and precision of movements we have to sense.
+For example, NASA documents these values with and with gravity, or with pressurization because they need to provide precise and documented specifications for spacecrafts~\cite{nasa14}.
+They also use these specifications to design clothes that astronauts can wear comfortably to perform routine tasks.
%motion range has application to
%- Workstation Design and Layout
%- design clothing for tasks: lift helmet visor, open door,
+In our case, we design input systems.
+Therefore we need to know the range and precision of movements we have to sense.
+However these are not the only human factors that have an impact on imput systems.
+Users may ignore the way to perform a specific action.
+The discoverability~\cite{norman13,pong19} and learnability~\cite{cockburn14,grossman09} of interaction are typical research problems in the HCI field related to this point.
+Then the physical actions users have to perform can require high motor skills or training~\cite{schmidt05}.
+For example, performing some multiple-key keyboard shortcuts with one hand can be challenging~\cite{pietrzak14}.
+\fixme{Extend this a little?}
\paragraph{Input systems}
%I will describe below the most frequent technologies in input systems.
The first family is electro-mechanical components.
In this family, various types of switches and push buttons sense contacts.
-Keyboard and controller buttons use variations of this technology.
+Keyboard and buttons use variations of this technology.
Linear and rotary potentiometers sense continuous movements.
Joysticks typically contain two of them to sense rotations in two directions.
Encoders sense discrete movements.
-Ball mice used two of them to sense positions, and they are still used today in mice wheels.
+Ball mice used two of them to sense positions, and mice wheels still use one.
The second family of input sensing technologies is electrical sensors, such as resistive or capacitive sensors.
They sense the position of one or more contact points.
They are used in current touchscreens, including mobile phones and tablets.
The signal coming from these sensors requires several transformations.
Contact inputs such as buttons require a software or hardware \defword{debouncing} mechanism to avoid unwanted multiple activations.
-Threshold-based input such as capacitive sensing not only require adjusting a sensitivity and threshold value, but they often require an \defword{hysteresis} mechanism to avoir multiple activations.
+Threshold-based input such as capacitive sensing not only requires adjusting a sensitivity and threshold value, but they often require an \defword{hysteresis} mechanism to avoid multiple activations as well.
Analog signals must be transformed to digital values with an Analog-to-digital converter (\defword{ADC}).
Input values often have noise that must be \defwords{filtered}{filter}.
-There many possible filters that remove noise, at the cost of latency~\cite{casiez12}.
+There are many possible filters that remove noise, at the cost of latency~\cite{casiez12}.
Some kinds of input require further transformation.
-In particular, pointing input require a \defword{transfer functions} that computes the movement of the cursor on the screen depending on the physical movements on the input device.
+In particular, pointing input require a \defword{transfer functions} that computes the movement of the cursor on the screen depending on the physical movements of the input device.
These transfer functions usually take into account the ballistic-then-corrective nature of our movements~\cite{meyer88}.
Vision-based technologies are sensible to occlusions.
Therefore the software part of the pipeline extrapolates data to fill gaps in the input streams.
The combination of several sources of inputs is challenging as well, but provides more precision in some cases.
-For example data from accelerometers require mathemathical integrations for position sensing.
+For example, data from accelerometers require mathematical integrations for position sensing.
Not only it requires calibration, but it is also sensible to drifts due to the data precision.
The fusion of accelerometers, gyroscopes, and magnetometers provides better tracking, at the cost of increased processing complexity.
The software processes described in the previous paragraph are either computed on the device or on the host computer.
For example, the device systematically debounces inputs with analog low-pass filters.
They also implement hysteresis effects with a Schmitt trigger\footnote{\href{https://en.wikipedia.org/wiki/Schmitt_trigger}{https://en.wikipedia.org/wiki/Schmitt\_trigger}}.
-The ADCs are microcontrollers peripherals on the device.
-Filters are commonly implemented both on the device and host side.
-For example, mouse out touchpad movements are filtered on the device.
-At the opposite, inputs of depth-cameras such as a Kinect\footnote{\href{https://en.wikipedia.org/wiki/Kinect}{https://en.wikipedia.org/wiki/Kinect}} are filtered on the host side, because the host retreives raw data and computes a skeleton for example~\cite{shotton11}.
-The transfer function is typically computed on the host, because it requires information about the screen.
+The ADCs are either microcontrollers peripherals or dedicated components.
+Filters are commonly implemented either on the device firmware or the host drivers.
+For example, mouse and touchpad movements are filtered on the device.
+%At the opposite, inputs of depth-cameras such as a Kinect\footnote{\href{https://en.wikipedia.org/wiki/Kinect}{https://en.wikipedia.org/wiki/Kinect}} are filtered on the host side, because the host retreives raw data and computes a skeleton for example~\cite{shotton11}.
+At the opposite, inputs of depth-cameras are filtered on the host side, because the host retreives raw data and computes a skeleton for example~\cite{shotton11}.
+The transfer function is typically computed on the host because it requires information about display.
When devices are integrated in the interactive system, they are connected to the host with a simple bus like SPI or I2C\footnote{\href{https://en.wikipedia.org/wiki/Serial_Peripheral_Interface}{https://en.wikipedia.org/wiki/Serial\_Peripheral\_Interface} \href{https://en.wikipedia.org/wiki/I\%C2\%B2C}{https://en.wikipedia.org/wiki/I\textsuperscript{2}C}}.
In this case, the device implements a communication protocol that the host has to follow.
There is no standard protocol, but the overall idea is usually similar.
-The previous buses use no correction codes.
-Therefore they are fast, but sensible to intereferences.
-Thus, devices that users can plug use more robust buses.
-Today's most frequent bus is certainly USB with the Human Interface Devices (\defword{HID}) class\footnote{\href{https://www.usb.org/hid}{https://www.usb.org/hid}}.
-This class defines a standard communication protocol.
-When the device is plugged, it sends descriptors that lists its features.
-In particular, the HID descriptor details the format and semantic of the data packets it will send at a fixed frequency.
+These buses use no correction codes, therefore they are fast but sensible to interferences.
+Thus, devices that users can plug use more robust buses such as USB with the Human Interface Devices (\defword{HID}) class\footnote{\href{https://www.usb.org/hid}{https://www.usb.org/hid}}.
+This class defines a standard communication protocol for interactive devices.
+When the device is plugged, it sends descriptors that list its features.
+In particular, the HID descriptor details the format and the semantic of the data packets the device will send at a fixed frequency.
Thanks to this protocol, the host can interpret virtually any HID device with a generic driver.
-Regardless of the communication method between the host and the device, the operating system creates \defwords{input events}{input event} that applications will interpret for their own use.
+Regardless of the communication method between the host and the device, the drivers of the operating system create \defwords{input events}{input event} that applications will interpret for their own use.
%Buxton collection of interactive devices\footnote{\href{https://www.microsoft.com/buxtoncollection}{https://www.microsoft.com/buxtoncollection}}
\section{Research questions}
+Similar to the haptic pipeline, the motor pipeline reveals pitfalls that could lead systems to behave differently than what users had in mind.
+%The first limitations are due to the limited capacities of humans.
+Users may ignore the way to perform they intend to do.
+They can know the the action they have to do but it is challenging to perform.
+There can be obstacles in the physical world that prevent systems to sense these actions correctly.
+The range of physical effect can be out of the sensing range of the system.
+The system may interpret what it sensed incorrectly.
+This pipeline is therefore a profuse source of HCI research questions.
+I focus here on three categories of research questions that I adressed in my research in the last decade.
+
\subsection{Sensing and interpretation}
-=> Lagmeter
+Interactive systems have a limited set of sensors, which sense a limited subset of the users' actions features.
+This is sometimes a desired property: a button is pressed regardless of the actual motion that produced its activation.
+When users lift a mouse, its position is not tracked anymore.
+This behaviour enables \defword{clutching}, which extends the motion range of the mouse cursor with the same required physical area.
+Sensing limitations are sometimes constrains for interaction though.
+For example occlusion phenomenons or the limited field of view of vision-based sensors can impede gesture recognition.
+Many factors can have a negative influence on the processing of input signals.
+For example gesture recognition failures provoke interaction errors.
+Beyond errors, the \defword{segmentation} of gestures is a key challenge of gestural interaction because typical gesture sensors observe users all the time, even when users do not want to interact with the system.
+The main consequence is the \defword{Midas touch}\footnote{\href{https://en.wikipedia.org/wiki/Midas\#Golden_Touch}{https://en.wikipedia.org/wiki/Midas\#{}Golden\_Touch}} effect that forces users to interact at all times.
+In the next chapter, section~\ref{sec:summon}, I will discuss this issue and a mitigation strategy for this specific problem.
+In the section~\ref{sec:lagmeter} of this chapter I will discuss a method and measurement system we designed for characterizing the latency of input systems.
\subsection{Input vocabulary}
Raycursor, facial expressions
+
\section{Contributions}
\subsection{Input latency measurement}
-
+\label{sec:lagmeter}
Latency \cite{casiez17}
\subsection{Flexible pens}
+\label{sec:flexiblepens}
FlexStylus \cite{fellion17}
Hyperbrush \cite{guerrero21}
\subsection{Finger identification}
+\label{sec:fingeridentification}
FingerCuts \cite{goguey14,goguey14a,goguey17}
Leverages finger identification for command selection and direct manipulation of parameters.
\subsection{Interaction in VR}
+\label{sec:interactionvr}
RayCursor \cite{baloup19,baloup18,baloup19a}
projects / hdr.git / commitdiff