Knowledge about the range of movements and maximum forces is necessary for the design and layout of workstations.
For example, NASA documents these values with and with gravity, or with pressurization because they need to provide precise and documented specifications for spacecrafts~\cite{nasa14}.
They also use these specifications to design clothes that astronauts can wear comfortably to perform routine tasks.
-
+sss
%motion range has application to
%- Workstation Design and Layout
%- design clothing for tasks: lift helmet visor, open door,
Further studies leverage the raw data of multi-touch surfaces, either from cameras or capacitive sensors.
With this data we can detect contact blobs instead of single coordinates, which increases the input vocabulary \cite{roudaut09}.
Other research investigated technologies that sense other gestures properties such as the part of the finger touching the surface \cite{harrison11}, make a distinction between contacts from different users \cite{dietz01}, or analyze the hand posture \cite{murugappan12}.
+This has been an active research area in the last decades, therefore this is just a tiny overview of what has been done on this topic.
In this work we were interested \defword{finger identification}.
It is an additional hand gesture property, which says which finger of which hand produced a given contact point.
-There is currently no technology that sense this property directly in consumer electronic products.
-A workaround is to ask users to press all fingers before releasing some of them \cite{lepinski10}.
-Some projects use a different processing of sensed data.
+There is still no technology that sense this property directly in consumer electronic products.
+A workaround with existing technologies is to ask users to press all fingers before releasing some of them \cite{lepinski10}.
+Other projects use a different processing of sensed data.
For example, the old generations tabletops used an infrared projector under a translucent surface, pointed towards the user.
An infrared camera sensed the reflection of the infrared light that created blobs at contact points.
The multi-touch coordinates were computed as the centroid of these blobs.
-However, the whole hand was visible on this image and it is possible to identify fingers with image processing of this data \cite{ewerling12}.
-Several studies in the literature use heuristics to detect detect finger chords.
-They detect the palm of the hand and deduce the position of fingers \cite{bailly08} or just assumptions based on the relative position of contact points \cite{ghomi13,wagner14}.
-Other research investigated finger identification with prototyping technologies:
-fibers under a touchpad that enable the capture of fingerprints \cite{holz13}, a glove with piezoelectric vibration sensors \cite{masson17}, or a glove with fiducial markers \cite{marquardt11}.
+However, the whole hand is visible on this image and it is possible to identify fingers with image processing of this data \cite{ewerling12}.
+Several other studies in the literature use heuristics to detect detect finger chords.
+They detect the palm of the hand and deduce the position of fingers \cite{bailly08} or make assumptions based on the relative position of contact points \cite{ghomi13,wagner14}.
+Other research investigated finger identification with prototyping technologies: optical fibers under a touchpad that enable the capture of fingerprints \cite{holz13}, a glove with piezoelectric vibration sensors \cite{masson17}, or a glove with fiducial markers \cite{marquardt11}.
%Fiberio for example uses optical fibers under a touchpad that enable the capture of fingerprints, hence identifying fingers \cite{holz13}.
%Whichfingers is another solutiuon that does works with any multi-touch interfaces, it uses a glove with piezoelectric vibration sensors on each finger \cite{masson17}.
-In the next section we will discuss the finger identification techniques we used in our studies.
+In the next section we will discuss the finger identification sensing technologies we used in our studies.
One of the simplest applications of finger identification is probably mapping a different command to every finger.
Research showed that this is already sufficient to increase the input thoughput of touch interaction \cite{roy15}.
This is particularly useful because multitouch applications typically cannot provide as many commands as desktop applications.
-For example, in 2014 Wagner \etal report that there were 648 commands in Adobe Photoshop’s menus compared to 35 commands on the tablet version.
+For example, in 2014 Wagner \etal report that there were 648 commands in the menus of the desktop version of Adobe Photoshop compared to 35 commands on the tablet version.
Interaction techniques that use finger identification is a solution.
-However the input space is huge: $2^{10}-1$ fingers chords, to which we can add gestures.
-We will discuss how we leverage this new input space in a systematic way.
+However, the input space is huge: $2^{10}-1$ fingers chords, to which we can add gestures.
+We will discuss how we leverage this new input space in a systematic way, in particular how we reduce this input space.
\subsubsection{Prototyping apparatus}
-This project is a typical HCI project in which we investigate the benefits and limitation of an interaction paradigm before the technology is ready to implement it in consumer electronics products.
-The advantage is that research for such a technology will happen only if and when the benefits will balance the cost.
-Even if such technology is not ready yet, the HCI community regularly uses alternative technologies that necessarrily make compromises.
+This project is a typical HCI project in which we investigate the benefits and limitations of an interaction paradigm before the technology is ready to implement it in consumer electronics products.
+The advantage is that research for such a technology will happen only if and when the benefits will balance the costs.
+Even if such technology is not ready yet, the HCI community regularly uses alternative technologies that necessarily make compromises.
In some cases we just imagine the technology is there and works perfectly.
The wizard of Oz technique consists in having an operator executing the actions on the users behalf, with any other technology.
-This is not always necessary.
+This is not always even necessary.
In the case of finger identifications, the Glass+Skin study just indicated participants the finger they wanted them to use.
Other methods make assumptions on the inputs \cite{ghomi13,wagner14}, but it limits the resulting interaction techniques.
-Finally we can use alternative technologies.
-Either they only work for specific devices, like projection-based tabletops, or they use invasive methods with markers.
+We can also use alternative technologies.
+Either they only work for specific devices, like projection-based tabletops, or they use invasive methods like markers.
We prefer the last method since it allowed us to build prototypes for tabletops, tablets and smartphones, without limitations on the input vocabulary.
\begin{figure}[htb]
\caption[Finger identification prototypes.]{Three implementations of finger identification prototypes. The first one uses gametracks to locate fingers on a tabletop. The two other ones use color markers with a camera to track fingers on a tablet and a smartphone.}
\end{figure}
-The \reffig{fig:hotfingers-prototypes} shows our three prototypes \cite{goguey14,goguey14a,goguey17}.
-The tabletop prototype combines two sources of information.
+The \reffig{fig:hotfingers-prototypes} shows our three prototypes.
+% \cite{goguey14,goguey14a,goguey17}.
+The tabletop prototype combines two sources of information to identify fingers.
The first one is the 2D coordinates of the contact points.
The second one is the 3D position of every finger that we sense with GameTraks\footnote{\href{https://en.wikipedia.org/wiki/Gametrak}{https://en.wikipedia.org/wiki/Gametrak}}.
GameTracks sense a 3D position by measuring the length and angles of a string attached to a joystick.
-This method requires a calibration and strings attached to every finger, but it is not sensible to occlusion forr example.
+This method requires a calibration and strings attached to every finger, but it is not sensible to occlusion for example.
The tablet and smartphone prototypes used color markers on every finger, as well as an external camera.
It requires calibration as well and it is sensible to occlusion.
-However having no strings attached to the fingers is more convenient for interacting on smaller devices.
+However, having no strings attached to the fingers is more convenient for interacting on smaller devices.
+These prototyping technologies are clearly not usable in consumer electronics products.
+But they are robust enough and add little constrains on multi-touch interaction, so they are convenient for exploring the input space of multi-touch interaction with finger identification.
\subsubsection{Input vocabulary}
Without finger identification, the input space is just the number of contacts, between 1 and 10 for one user, and coordinates for each contact point.
With finger identification, the theoretical input space much larger: each finger can either be pressed or not, which leads to $2^{10}-1$ possibilities.
However, chords with too many fingers are less likely used.
-For a given number of fingers $k$ there are still $C_k^{10}$ possible combinations.
+For a given number of fingers $k$ there are still $C_k^{10}$ possible combinations of two-hands chords.
This leads to 45 combinations of 2 fingers and 120 combinations of 3 fingers.
People cannot perform all chords efficiently because of biomechanical constraints~\cite{ghomi13}.
However, this input space remains still high therefore the design of a command selection technique requires a systematic approach.
The inspiration for our design rationale stems from command selection with keyboard shortcuts.
-They use modifier keys and an alphanumeric key.
+Keyboard shortcuts use modifier keys and an alphanumeric key.
Different combinations of modifier keys change the command activated with a given alphanumeric key.
Then, when the associated action is being executed the users can apply constraints with modifier keys.
The \reffig{fig:hotfingers} shows how we applied this principle to multi-touch interaction with finger identification.
\centering
\includegraphics[width=\columnwidth]{figures/hotfingers}
\label{fig:hotfingers}
- \caption[Command selection with finger identification.]{Command selection with finger identification. A chord with the left hand whows a crib sheet of commands users can select with the right hand. The user selects a command and adjusts the parameters with direct manipulation. The user can lift the left hand while adjusting the command parameters. The user can invoke additional constraints on the command parameters with the left hand.}
+ \caption[Command selection with finger identification.]{Command selection with finger identification. A chord with the left hand shows a crib sheet of commands users can select with the right hand. The user selects a command and adjusts the parameters with direct manipulation. The user can lift the left hand while adjusting the command parameters. The user can invoke additional constraints on the command parameters with the left hand.}
\end{figure}
\subsubsection{Discussion}
The \reffig{fig:hotfingers-helps} shows the visual feedforward items we use to promote the discoverability of commands.
On the left, when all fingers touch the surface, bubbles show all the possible chords associated with modifiers or commands.
On the left, crib sheets represent the visual icons of available commands or or constraints for the current chord of the left hand.
+All this visual feedforward information is displayed around the corresponding contact points, and follow them continuously when they move.
+The actual crib sheets are updated continuously as fingers touch or leave the surface.
-Linear menus and toolbars in desktop application show, in principle, all the available commands of an application.
+Linear menus and toolbars in desktop applications show, in principle, all the available commands of an application.
In our case, the visual representation of commands is only available on demand.
The command mapping and constraints change in real time as the users press or releases fingers.
It encourages users to explore the system and discover the available commands.
This exploration would be an issue if the explorable input space was too large.
However, we essentially use one-hand chords.
There are therefore $C_k^5$ possible chords with $k$ fingers, which gives 5 combinations of one or four fingers, and 10 combinations of 2 or 3 fingers.
-Therefore there are $5+10+10+5=30$ modifier chords for a maximum of $150$ single finger commands with the right hand.
+Therefore there are $5+10+10+5=30$ modifier chords for a maximum of $150$ single-finger commands with the right hand.
We assume this is a reasonnable size for an explorable input space.
-However, we did not conduct a user study to evaluate this aspect.
+However, we did not conduct a user study to evaluate this aspect and validate this claim.
\begin{figure}[htb]
\centering
\hfill
\includegraphics[height=3.3cm]{figures/hotfingers-lefthand2}
\label{fig:hotfingers-helps}
- \caption[Discoverability features for FingerCuts.]{Discoverability features for FingerCuts. Users can see all the available commands when touching the surface with all their fingers. Crib sheets show the avaiable commands and constraints when fingers of the left hand are touching the surface.}
+ \caption[Discoverability features for multi-touch interaction with finger identification.]{Discoverability features for multi-touch interaction with finger identification. Users can see all the available commands when touching the surface with all their fingers. Crib sheets show the available commands and constraints when fingers of the left hand are touching the surface.}
\end{figure}
-
+Keyboard shortcuts with multiple modifier keys are sometimes difficult to execute because the keys position is fixed.
+Therefore sometimes it requires users to stretch their fingers to reach all the keys, especially if they want to keep one hand on the mouse.
+In this context we studied the possibility to select modifiers on the mouse so that the left hand only has to select the alphanumeric key~\cite{pietrzak14}.
+The input space remained limited, especially because we only had two modifier buttons on the mouse.
+Il the case of our command selection technique for multi-touch interaction with finger identification, the advantage is that there is no actual button to press.
+Therefore, the location of the contact point does not matter.
+Hence, users can touch the surface with a comfortable posture.
\subsection{Interaction in VR}
\label{sec:interactionvr}
\section{Conclusion}
+
+All these input techniques use hands dexterity and our capacity to touch and manipulate.
+But the sense of touch is barely used.
+
+Problem with input space too small or too large.
\ No newline at end of file
projects / hdr.git / commitdiff