-\documentclass[11pt,a4paper,twoside,titlepage,openbib]{phdthesis}\r
+\documentclass[10pt,a4paper,twoside,titlepage,openbib]{phdthesis}\r
\r
\usepackage[utf8]{inputenc}\r
-\usepackage[T1]{fontenc}\r
\r
\usepackage{getcommit}\r
\usepackage{xstring}\r
\r
\frontmatter\r
\r
- \begin{titlepage}\r
- \thispagestyle{empty}\r
-% \vspace{1mm}\r
- \hrule\r
- \vspace{1mm}\r
- %\raggedleft\r
- Département Informatique\hfill École Doctorale SPI\r
-\r
- %\raggedleft\r
- Faculté de Sciences et Technologies\r
-\r
- \begin{centering}\r
-\r
- \vspace*{\stretch{2}}\r
- {\LARGE\textsc{Habilitation à diriger des recherches}}\r
-\r
- \vspace*{\stretch{1}}\r
- présentée par\r
-\r
- \vspace*{\stretch{1}}\r
- {\LARGE \scshape Thomas Pietrzak}\r
-\r
- \vspace*{\stretch{2}}\r
- Discipline : Informatique\r
-\r
- \vspace*{\stretch{1}}\r
- Spécialité : Interaction Homme-Machine\r
-\r
- \vspace*{\stretch{2}}\r
- {\LARGE\bfseries Forging digital hammers: the design and engineering of empowering interaction techniques and devices}\r
-\r
- \vspace*{\stretch{2}}\r
- XXX 2020\r
-\r
- \vspace*{\stretch{2}}\r
- \end{centering}\r
-\r
- \noindent\begin{center}\r
- \noindent\begin{tabular}{@{}l@{\hspace{2.5mm}}l@{\hspace{2.5mm}}l@{}}\r
- \emph{Rapporteurs :} & XXX & Professeur à XXX\\\r
- & XXX & Professeur à XXX\\\r
- & XXX & Professeur à XXX\\\r
- \\\r
- \emph{Examinateurs :} & XXX & Professeur à XXX\\\r
- & XXX & Professeur à XXX\\\r
- & XXX & Professeur à XXX\\\r
- & XXX & Professeur à XXX\\\r
- \end{tabular}\r
- \end{center}\r
- \vspace*{\stretch{1}}\r
- \hrule\r
- \vspace{1mm}\r
- \centering\r
- Habilitation à diriger des recherches préparée au sein de l'équipe projet LOKI commune à\\l'Université de Lille, CRIStAL --- UMR CNRS 9189 et Inria Lille - Nord Europe\\\r
- \vspace{1cm}\r
- \includegraphics[width=3.5cm]{logo_univ_lille}\r
- \hfill\r
- \includegraphics[width=3.5cm]{logo_cristal}\r
- \hfill\r
- \includegraphics[width=3.5cm]{logo_inria}\\\r
- \vspace{1cm}\r
- Revision: \texttt{\StrLeft{\commit}{7}} on branch \texttt{\branch}\\\r
- Compiled on \today\ at \currenttime\\\r
- \end{titlepage}\r
-\r
- \newgeometry{inner=1.5cm, outer=2.5cm, vmargin=3cm}\r
-\r
- \cleardoublepage\r
+\input{tex/0-titre}\r
\r
\tableofcontents\r
\cleardoublepage\r
\listofdefinitions\r
\cleardoublepage\r
\r
+\input{tex/00-remerciements}\r
+ \cleardoublepage\r
+\r
\mainmatter\r
\r
-\input{tex/introduction}\r
+\input{tex/1-introduction}\r
+ \cleardoublepage\r
+\r
+\input{tex/2-output}\r
\cleardoublepage\r
\r
-\input{tex/evolutions}\r
+\input{tex/3-input}\r
\cleardoublepage\r
\r
-\input{tex/vocabulary}\r
+\input{tex/4-loop}\r
\cleardoublepage\r
\r
\r
%\input{tex/something}\r
% \cleardoublepage\r
\r
-\input{tex/conclusion}\r
+\input{tex/5-conclusion}\r
\r
\backmatter\r
\r
\usepackage{tikz}
\usetikzlibrary{arrows.meta,arrows,positioning}
\usepackage{booktabs}
-\usepackage{lmodern}
+
+%fonts
+% font-latin-modern
+%\usepackage{tgadventor}
+%\usepackage{lmodern}
\usepackage{palatino}
+
+\usepackage[Latin,Greek]{ucharclasses}
+\usepackage{fontspec}
+\newfontfamily\mynormal{Palatino}% Linotype}
+\setDefaultTransitions{\mynormal}{}
+% font-computer-modern
+\newfontfamily\mygreek{CMU Serif}
+\setTransitionsForGreek{\mygreek}{}
+
\usepackage{graphicx}
\usepackage{epigraph}
\setlength{\epigraphrule}{0pt}
\setlength{\epigraphwidth}{.5\textwidth}
\usepackage{xspace}
\usepackage{parskip}
-\usepackage{caption}
-\usepackage{subcaption}
+%\usepackage{caption}
+%\usepackage{subcaption}
+%\captionsetup{justification=raggedright,singlelinecheck=false}
\usepackage{eurosym}
\usepackage[ruled,vlined]{algorithm2e}
+\usepackage[absolute,overlay]{textpos}
%\usepackage[french]{babel}
%\usepackage{icomma}
\definecolor{gray90}{rgb}{0.9,0.9,0.9}
\definecolor{linkcolor}{HTML}{992222}
\definecolor{ideecolor}{HTML}{f9eeee}
+\definecolor{mainheader}{rgb}{0.9,0.9,0.9}
+\definecolor{chapterback}{rgb}{0.7,0.7,0.7}
+
+
+\usepackage{tikz}
+\usetikzlibrary{calc}
+
+\newcommand\brectangles{%
+\begin{tikzpicture}[overlay, remember picture]
+\fill[chapterback]
+ (current page.north west) --
+ (current page.north east) --
+ ( $ (current page.north east) + (0,-6cm) $) --
+ ( $ (current page.north west) + (0,-7.5cm) $) --
+ cycle;
+%\fill[chapterback!50]
+% (current page.south west) --
+% (current page.south east) --
+% ( $ (current page.south east) - (0,-6cm) $) --
+% ( $ (current page.south west) - (0,-2cm) $) --
+% cycle;
+\end{tikzpicture}%
+}
+
+\newcommand\arectangles{%
+\begin{tikzpicture}[overlay, remember picture]
+\fill[mainheader]
+ (current page.north west) --
+ (current page.north east) --
+ ( $ (current page.north east) + (0,-4cm) $) --
+ ( $ (current page.north west) + (0,-5.5cm) $) --
+ cycle;
+%\fill[chapterback!50]
+% (current page.south west) --
+% (current page.south east) --
+% ( $ (current page.south east) - (0,-6cm) $) --
+% ( $ (current page.south west) - (0,-2cm) $) --
+% cycle;
+\end{tikzpicture}%
+}
\usepackage{titlesec}
+% % \titleformat{⟨command⟩}[⟨shape⟩]{⟨format⟩}{⟨label⟩}{⟨sep⟩}{⟨before⟩}[⟨after⟩]
+% \titleformat{\chapter}
+% {\fontsize{23}{25}\fontfamily{qag}\selectfont}%\sffamily\scshape
+% {\color{chaptercolor}\fontsize{50pt}{50pt}\selectfont\textbf{\thechapter}}
+% {0em}
+% {\hfill}
+% [\color{gray95}{\titlerule[2mm]}]
+% \titleformat{name=\chapter,numberless}
+% {\fontsize{23}{25}\sffamily\scshape}
+% {}
+% {0em}
+% {\hfill}
+% [\color{gray95}{\titlerule[2mm]}]
+
% \titleformat{⟨command⟩}[⟨shape⟩]{⟨format⟩}{⟨label⟩}{⟨sep⟩}{⟨before⟩}[⟨after⟩]
+
\titleformat{\chapter}
- {\fontsize{23}{25}\sffamily\scshape}
- {\color{chaptercolor}\fontsize{50pt}{50pt}\selectfont\textbf{\thechapter}}
- {0em}
- {\hfill}
- [\color{gray95}{\titlerule[2mm]}]
+ {\vspace{-3cm}\fontsize{30pt}{30pt}\bfseries\sffamily\color{white}\hyphenpenalty=10000}
+ {\thispagestyle{empty}\brectangles}
+ {-1cm}
+ {
+ \hspace{0.5cm}{\fontsize{50pt}{50pt}\color{bulletcolor}\thechapter}\hspace{1cm}%
+ }
+ [\vspace{2cm}]
+
\titleformat{name=\chapter,numberless}
- {\fontsize{23}{25}\sffamily\scshape}
- {}
- {0em}
- {\hfill}
- [\color{gray95}{\titlerule[2mm]}]
+ {\vspace{-3cm}\fontsize{30pt}{30pt}\bfseries\sffamily\color{white}\hyphenpenalty=10000}
+ {\thispagestyle{empty}\brectangles}
+ {0cm}
+ { }
+ [\vspace{2cm}]
\titleformat{\section}
{\Large\sffamily}
\def\fps@figure{tbp}
\def\ftype@figure{1}
\def\ext@figure{lod}
+
\newtheorem{@definition}{Definition}[chapter]
\newenvironment{definition}[1]{
\addcontentsline{lod}{section}{#1}
%\newcommand{\tab}{{\tt\small Tab}\xspace}
\newcommand{\tab}{\modk{Tab}\xspace}
+%\DeclareFontShape{TU}{lmr}{m}{n} { <-> ssub * ppl/m/n }{}
+%\DeclareFontShape{LGR}{ppl}{m}{n} { <-> ssub * cmr/m/n }{}
+%LGR/ppl/m/n' undefined(Font) using `LGR/cmr/m/n
\fi \@xsect{#5}}\r
\r
% FIGURES AND TABLES\r
-\long\def\@makefigurecaption#1#2{\centering \vskip 3mm #1 -- #2\par}\r
+\long\def\@makefigurecaption#1#2{\vskip 3mm #1 -- #2\par}\r
\r
-\long\def\@maketablecaption#1#2{\centering \vskip 3mm #1 -- #2\par}\r
+\long\def\@maketablecaption#1#2{\vskip 3mm #1 -- #2\par}\r
% \long\def\@maketablecaption#1#2{\hbox to \hsize{\parbox[t]{\hsize}\r
% {\mbox{}\hfill #1 \\ \centering #2}}\vskip 1.3ex}\r
\r
--- /dev/null
+%!TEX root = ../hdrmain.tex
+
+\begin{titlepage}
+
+\begin{tikzpicture}[overlay, remember picture]
+\fill[chapterback!50]
+ (current page.north west) --
+ (current page.north east) --
+ ( $ (current page.north east) + (0,-2cm) $) --
+ ( $ (current page.north west) + (0,-5cm) $) --
+ cycle;
+\fill[chapterback!50]
+ (current page.south west) --
+ (current page.south east) --
+ ( $ (current page.south east) - (0,-5cm) $) --
+ ( $ (current page.south west) - (0,-2cm) $) --
+ cycle;
+\end{tikzpicture}%
+
+ \begin{textblock*}{\columnwidth}(20mm,0mm)%
+ \begin{center}
+ \vspace{10mm}
+ \raggedright
+ Département Informatique\hfill École Doctorale SPI\\
+ Faculté de Sciences et Technologies%\hfill
+ \end{center}
+ \end{textblock*}
+
+ \thispagestyle{empty}
+
+% \arectangles
+% \brectangles
+% \vspace{1mm}
+ %\hrule
+ %\vspace{1mm}
+ %\raggedleft
+
+ \begin{centering}
+
+ \vspace{20mm}
+% \vspace*{\stretch{2}}
+ {\LARGE\textsc{Habilitation à diriger des recherches}}
+
+ \vspace*{\stretch{1}}
+ présentée par
+
+ \vspace*{\stretch{1}}
+ {\LARGE \scshape Thomas Pietrzak}
+
+ \vspace*{\stretch{2}}
+ Discipline : Informatique
+
+ \vspace*{\stretch{1}}
+ Spécialité : Interaction Homme-Machine
+
+ \vspace*{\stretch{2}}
+ {\LARGE\bfseries Forging digital hammers: the design and engineering of empowering interaction techniques and devices}
+
+ \vspace*{\stretch{2}}
+ XXX 2021
+
+ \vspace*{\stretch{2}}
+ \end{centering}
+
+ \noindent\begin{center}
+ \noindent\begin{tabular}{@{}l@{\hspace{2.5mm}}l@{\hspace{2.5mm}}l@{}}
+ \emph{Rapporteurs :} & XXX & Professeur à XXX\\
+ & XXX & Professeur à XXX\\
+ & XXX & Professeur à XXX\\
+ \\
+ \emph{Examinateurs :} & XXX & Professeur à XXX\\
+ & XXX & Professeur à XXX\\
+ & XXX & Professeur à XXX\\
+ & XXX & Professeur à XXX\\
+ \end{tabular}
+ \end{center}
+ %\vspace*{\stretch{1}}
+ %\hrule
+
+ \vspace{4cm}
+
+
+ \begin{textblock*}{\columnwidth}(20mm,250mm)%
+ \begin{center}
+ \raggedright
+ \includegraphics[width=3.5cm]{logo_univ_lille}
+ \end{center}
+ \end{textblock*}
+
+ \begin{textblock*}{\columnwidth}(20mm,242mm)%
+ \begin{center}
+ \centering
+ \includegraphics[width=3.5cm]{logo_cristal}
+ \end{center}
+ \end{textblock*}
+
+
+ \begin{textblock*}{\columnwidth}(20mm,230mm)%
+ \begin{center}
+ \raggedleft
+ \includegraphics[width=3.5cm]{logo_inria}\\
+ \end{center}
+ \end{textblock*}
+
+ \begin{textblock*}{\paperwidth}(0mm,275mm)%
+ \begin{center}
+ \centering
+ \centering
+ Habilitation à diriger des recherches préparée au sein de l'équipe projet LOKI commune à\\l'Université de Lille, CRIStAL --- UMR CNRS 9189 et Inria Lille - Nord Europe\\
+ \end{center}
+ \end{textblock*}
+
+ \begin{textblock*}{\columnwidth}(20mm,260mm)%
+ \begin{center}
+ \raggedleft
+ \color{gray75}
+ Revision: \texttt{\StrLeft{\commit}{7}} on branch \texttt{\branch}\\
+ Compiled on \today\ at \currenttime\\
+ \end{center}
+ \end{textblock*}
+
+
+ \end{titlepage}
+
+ \newgeometry{inner=1.5cm, outer=2.5cm, vmargin=3cm}
+
+ \cleardoublepage
--- /dev/null
+%!TEX root = ../hdrmain.tex
+
+\chapter*{Remerciements}
\ No newline at end of file
--- /dev/null
+%!TEX root = ../hdrmain.tex
+
+\chapter*{Introduction}
+\label{chap:intro}
+\addcontentsline{toc}{chapter}{\nameref{chap:intro}}
+\addcontentsline{lof}{chapter}{\nameref{chap:intro}}
+ \epigraph{\lorem \lorem \lorem}{Auteur}
+
+ \vspace{1cm}
+
+\newcommand\phdyear{2008}
+\newcommand\yearssincephd{\the\numexpr \the\year - \phdyear \relax} % F=m*a
+
+
+This manuscript starts where my Ph.D. manuscript ended, \yearssincephd~years ago.
+The objective of my research was to use haptics to to help visually impaired children at school.
+My main contributions consisted in encoding information with touch, using Brewster's concept of Tactons~\cite{brewster04}.
+It was a pleasure and honor to me to have the opportunity to collaborate with his team on some of these projects.
+At this time, the way I viewed haptic at this point was a way to send information to users through their sense of touch.
+I designed different kinds of Tacton sets: force feedback with active exploration~\cite{pietrzak05}, passive exploration~\cite{pietrzak05a}, and pin arrays~\cite{pietrzak09,pietrzak06}.
+I used them in two educational applications: an electric circuits exploration software~\cite{pietrzak07a,pietrzak09a}, and a geometric shapes exploration software~\cite{pietrzak09a,pietrzak09b}.
+Let $n$ be the number of Tactons in the set, the theoretical number of bits each Tacton in this set can transmit is $log_2(n)$.
+For example, each Tacton in a set of 8 Tactons can transmit up to $3$ bits.
+I performed user studies to analyze confusions, and deduced the number of bits effectively transmitted to users.
+This was an estimation of the throughput of this communication channel.
+Finally I designed a multimodal API~\cite{pietrzak07b} to use these output modalities along with sound and 3D graphics.
+
+During the committee questions after my defense presentation, Yves Guiard talked during 15 minutes to tell me how he “liked my work but does not agree”.
+This was the most exciting moment of my defense.
+To this day we still talk about that each time we have the chance to meet.
+He point referred to Gibson's work on perception, especially his paper on active touch~\cite{gibson62}.
+In this paper, Gibson describes an experiment in which participants had to recognize the shape of cookie cutters with their sense of touch only.
+There was two conditions: either participants could only feel the cookie cutter pressed on their hand, or they could explore the contours with their fingers.
+It turned out that participants of the first condition recognized $29\%$ of the shapes, and participants of the other condition $95\%$ of the shapes.
+This experiments clearly shows that our exploration actions are as important as our sensations in our understanding of our environment.
+In my work I made a consensual distinction between tactile and force feedback.
+I called tactile feedback sensations coming from the mechanoreceptors in the skin, and force feedback sensations coming from muscles, tendons and joints~\cite{oakley00}.
+Pr. Guiard argued for a different taxonomy in which tactile feedback refers sensations resulting from tangential movements, and force feedback refers to sensations resulting from normal movements.
+His taxonomy is certainly in line with Gibson's idea that our understanding of our environment depends on our exploratory movements.
+However, my understanding is that the actual sensations got lost in the way.
+We can however fill the gap with Lederman and Klatzky's work on exploratory movements~\cite{lederman87}.
+
+Indeed, I discussed the relation between movement and perception to some extend in my Ph.D. manuscript at several occasions, even if I was not aware of Gibson's work yet.
+The first time was when I designed force feedback icons.
+The first Tacton set I published used active exploration with force feedback~\cite{pietrzak05}.
+Users manipulated a force feedback stylus (Phantom), which tip was constraint on a line.
+These Tactons consisted in bumps that varied in number, direction and amplitude.
+Later, I created Tactons with passive exploration~\cite{pietrzak05a}.
+They also used a force feedback stylus, but this time the tip was stuck and dragged around in different directions, amplitudes, multiple times.
+The main difference between these Tactons is the active or passive exploration from users.
+Participants in my user studies perceived both kind of Tactons.
+The difference was essentially the situations in which they could be used.
+Typically, in the electric circuits schematics exploration software I designed~\cite{pietrzak07a,pietrzak09a}, active exploration encoded electrical components.
+It enabled users to scan components and deduce their type.
+Passive exploration Tactons were used at joints, in which users could query the adjacent directions, and help them to explore the topology of circuits.
+
+
+
+From Ancient Greek + {ἁπτικός} + (haptikós, “able to come in contact with”), from ἅπτω (háptō, “to touch”) + -ικός (-ikós, “suffix forming an adjective from a noun”)\footnote{\href{https://en.wiktionary.org/wiki/haptic}{https://en.wiktionary.org/wiki/haptic}}.
+
+
+
+
+
--- /dev/null
+%!TEX root = ../hdrmain.tex
+
+\chapter{Haptics as an output modality}
+ \epigraph{\lorem}{Auteur}
+
+\begin{Abstract}
+\loremipsum
+\end{Abstract}
+
+
+
+\section{Haptic output vocabulary}
+
+
+
+\section{Haptic feedback for activity monitoring (Activibe)}
+\cite{cauchard16}
+
+\section{Tactile textures with programmable friction}
+
+Tactile Textures~\cite{potier12,potier16}
+
+\section{Contribution of haptic to sense of embodiment}
+
+\cite{richard20}
\ No newline at end of file
--- /dev/null
+%!TEX root = ../hdrmain.tex
+
+\chapter{Haptic as an input modality}
+ \epigraph{\lorem}{Auteur}
+
+\begin{Abstract}
+\loremipsum
+\end{Abstract}
+
+
+Passive haptics:
+
+Métamorphe \cite{bailly13}
+
+Living Desktop \cite{bailly16}
+
+CtrlMouse \cite{pietrzak14}
+
+Pseudo haptics \cite{lecuyer01}
+
+\section{Lagmeter}
+
+Latency \cite{casiez17}
+
+\section{Extention of interaction vocabularies}
+
+\subsection{FlexStylus: a flexible digital pen}
+
+FlexStylus \cite{fellion17}
+Hyperbrush
+
+\section{New interaction techniques with additional vocabulary}
+
+\subsection{RayCursor: a 3D pointing technique in VR}
+
+RayCursor \cite{baloup19,baloup18,baloup19a}
+
+Leverages 3D input and combines with 3D motion for more efficient pointing
+
+\subsection{FingerCuts: leveraging finger identification for multi-touch interaction}
+
+FingerCuts \cite{goguey14,goguey14a,goguey17}
+
+Leverages finger identification for command selection and direct manipulation of parameters.
\ No newline at end of file
--- /dev/null
+%!TEX root = ../hdrmain.tex
+
+\chapter{Haptic interaction}
+ \epigraph{\lorem}{Auteur}
+
+\begin{Abstract}
+\loremipsum
+\end{Abstract}
+
+Animals such as humans explore their environment by probing them, and perceiving properties. Perception is a combination of actions, resulting sensations, and cognitions that mixes this with memory, experience, etc.
+
+Perception/action cycle~\cite{gibson79}
+
+Sensorimotor loop~\cite{oregan01a}
+
+Theory of action~\cite{norman88}
+
+Direct manipulation~\cite{schneiderman83}
+
+\cite{mbl00}
+
+Arch~\cite{arch92}, MVC~\cite{reenskaug79,reenskaug79a}, PAC~\cite{coutaz87}
+
+Implementation depends on ethnographic background of programmers \cite{rode04}
+
+Software architecture reproduce the organization structure\cite{conway68}
+
+\section{Something}
+
+\subsection{Vibrotactile widgets}
+
+Replacing physical controls with touchscreens have advantages: updates, reconfigurable, visual feedback, but most of haptic properties are lost: click sensations of buttons, detents on slides. Impact on interaction. Technologies to restore haptic feedback.
+
+Vibrotactile widgets~\cite{frisson17,frisson20}
+
+Leverages vibrotactile feedback for touch surfaces.
+,
+
+\subsection{Direct Manipulation in tactile displays}
+
+Direct Manipulation~\cite{gupta16,gupta16a}
+
+Direct manipulation is one of the most fundamental foundation of GUIs~\cite{schneiderman83}.
+It provid,es valuable usability benefits that highly contributed to the success of GUIs over command line interfaces.
+Yet, the description of this paradigm relied essentially on visual cues.
+In this project we explained how to adapt the concept of direct manipulation to tactile displays.
+
+\subsection{Summon interactions}
+
+Summon interactions~\cite{gupta17}
+
+3D gestural interaction in the air faces difficult challenges due to the contact-less nature of this interaction style.
+The absence of physical buttons, or other unambiguous activation action makes it difficult to segment gestures.
+As a consequence, the system constantly interprets the users' actions.
+Therefore the user is always potentially interacting with the environment.
+This is known as the Midas touch problem, as a reference to the Midas king in the greek mythology, who turned everything he was touching to gold.
--- /dev/null
+%!TEX root = ../hdrmain.tex
+
+\chapter*{Conclusion}
+\label{chap:ccl}
+\addcontentsline{toc}{chapter}{\nameref{chap:ccl}}
+\addcontentsline{lof}{chapter}{\nameref{chap:ccl}}
+
+Something about this incompletude phenomenon: ideas on how to tackle the problem
+
+
+\section{Future work}
+
+\begin{idee}
+Interactive programs have limited number of features and capabilities.
+They are bounded by their inputs.
+More features and capabilities are possible, but new input is necessary.
+It can be new inputs, other kind of inputs or higher precision inputs.
+In any case the program must evolve to take these changes into account.
+These changes are made by humans.
+\end{idee}
+
+\begin{idee}
+Gödel's' first incompleteness theorem: in any consistent formal system F within which a certain amount of arithmetic can be carried out, there are statements of the language of F which can neither be proved nor disproved in F.
+
+The interactive incompleteness theorem: in any non trivial interactive program, there are desirable features which are not available.
+
+The interactive incompleteness theorem v2: in any non trivial interactive program, there are tasks users cannot perform with available features.
+
+Software evolution\cite{mens05}
+
+
+Conway's Law~\cite{conway68}: Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.
+
+Formal systems deal with Gödel's incompleteness theorem by adding axioms. Interactive programs add features.
+\end{idee}
+
+\begin{idee}
+change blindness for haptic
+\end{idee}
--- /dev/null
+%!TEX root = ../hdrmain.tex
+
+\chapter{Reshaping foundations of interaction}
+ \epigraph{\lorem}{Auteur}
+
+\begin{Abstract}
+\loremipsum
+\end{Abstract}
+
+\section{Introduction}
+
+\section{Revisiting the desktop}
+
+Desktop interaction initiated personal computing.
+Used for work, education, entertainment.
+Most of the concepts established before it became mainstream.
+\begin{itemize}
+ \item screen
+ \item keyboard
+ \item mouse
+ \item windows
+ \item menus
+ \item undo/redo
+ \item copy/paste
+\end{itemize}
+
+
+\subsection{CtrlMouse and TouchCtrl: duplicating mode switchers on the mouse}
+
+Command selection and mode switching are fundamental interactions that existed even before GUIs.
+The first keyboard used as a direct input on a computer was a \defword{Friden Flexowriter}\footnote{\url{https://en.wikipedia.org/wiki/Friden_Flexowriter}} teleprinter, connected to MIT's \defword{Whirlwind I}\footnote{\url{https://en.wikipedia.org/wiki/Whirlwind_I}} computer in 1956~\cite{bardini00}.
+This is eight years after the computer construction started.
+The \ctrl key was later introduced to reset the first two bits of ASCII characters, giving access to commands encoded in the first 32 ASCII characters.
+For example, one can type \ctrl~+~\modk{h} for a backspace, or \ctrl~+~\modk{j} for a line feed.
+The Apple I used these features, and abbreviations were printed on some physical keys, like \verb+TAB+ on the \modk{I} key, and \verb+TAPE+ on the \modk{R} and \modk{T} keys (device control commands).
+These commands are still used in terminal communications, represented \verb+^H+ or \verb+^J+ for example.
+However the use of the \ctrl key in GUIs evolved to other commands beyond the ASCII table.
+The cut/copy/paste commands is a typical set of commands, usually mapped to \ctrl~+~\modk{x}/\modk{c}/\modk{v}.
+We remark tough that some keyboards, such as those of Sun workstations had dedicated keys for typical commands such as cut/copy/paste.
+However it is obviously not possible to have dedicated keys for all possible commands.
+Therefore in text-based applications, in which the keyboard is primarily used for typing text, the \ctrl key behaves as a \defword{mode switcher}.
+It enables switching between the text entry mode, and the command selection mode.
+We assume that the fact that such mode switchers are located on the keyboard is essentially due to historical reasons described above.
+
+\paragraph{The Mouse}
+
+The computer mouse was first publicly demonstrated by Engelbart in 1968~\cite{engelbart68}, commercialized with the \defword{Xerox Alto} in 1973, and first became a commercial success with the \defword{Apple Macintosh} in 1984.
+At this time, the mouse behavior essentially consisted in selecting pointed objects or invoking \defword{contextual menus} (called pop-up menus back then\cite{canfield82}).
+Scrolling with the mouse only came up about ten years later~\cite{ohno85}.
+In this article, Ohno~\etal describe a mouse with 5 buttons and a wheel.
+Interestingly, the wheel was not used for scrolling, but for selecting a particular object among several ones under the cursor.
+It was meant to be used with the thumb.
+Alternatively, the user could scroll documents with a pressure-sensitive button on the thumb with 4~bits resolution.
+There was another pressure sensitive button under the index finger, for picking objects.
+Pressure input provided more information about the object, with a zoom for example.
+The three other fingers had simple buttons, for opening a menu, undoing actions or quitting the current command.
+The scrolling wheel as we know it appeared ten years later \cite{venolia93,venolia94}, and was commercialized with the Microsoft IntelliMouse\footnote{\url{https://en.wikipedia.org/wiki/IntelliMouse}} in 1996.
+
+\paragraph{Command selection with a mouse}
+
+Besides the 5-button mouse by Ohno~\etal, which commands were fixed by design, command selection with the mouse used to rely on actions associated to selected objects such as icons or menu items.
+Even advanced interaction techniques such as Toolglasses~\cite{bier93} consisted in selecting menu items.
+The UNIA\footnote{\url{https://www.microsoft.com/buxtoncollection/detail.aspx?id=150}} mouse was likely the first one to enable direct command selection, without mediator.
+It has a numeric keypad, as well as modifier keys under a removable cover.
+This disposition makes it difficult to both use the device as a mouse, and use the additional keys at the same time.
+PadMouse was another modified mouse, which enabled command selection with marking menus~\cite{balakrishnan98}.
+The user can draw marks on a touchpad mounted on the mouse.
+This device later inspired the design of commercial products, such as the
+Apple Magic Mouse~\footnote{\url{https://www.apple.com/shop/product/MRME2/}} or the
+Microsoft Touch Mouse~\footnote{\url{http://www.microsoft.com/hardware/en-us/p/touch-mouse}}.
+In 1999, Microsoft released the IntelliMouse Explorer, which not only used an optical sensor, but also featured 5 buttons, one on each finger.
+These additional buttons are typically used for navigating to the previous/next page in a browser, or trigger actions in video games.
+The scrolling wheel button was typically used for pasting on X11\footnote{\url{https://www.x.org/wiki/}} and its following versions.
+While these devices map commands to additional mouse buttons, none of them considered using these buttons as mode switchers.
+This is the topic of the \emph{CtrlMouse} and \emph{TouchCtrl} projects~\cite{pietrzak14}.
+
+\defword{CtrlMouse} and \defword{TouchCtrl} are two implementations that enable us to reconsider the position of mode switchers (\reffig{figure:ctrlmousetouchctrl}).
+These two systems duplicate the \ctrl and \shift keys on a mouse, and differ by their type of \emph{activation}.
+\emph{CtrlMouse} mirrors classic mode delimiters on the mouse.
+Users \emph{explicitly} change the mode by pressing the buttons located on the vicinity of the thumb, similarly to pressing the \ctrl and \shift keys on the keyboard.
+\emph{TouchCtrl} triggers the command mode \emph{implicitly} when the user's hand is in contact with the pointing device.
+The user can then press the thumb buttons for activating additional modifiers.
+
+\begin{figure}[htb]
+ \centering
+ \setlength{\fboxsep}{0pt}%
+ \setlength{\fboxrule}{1pt}%
+ \fbox{\includegraphics[height=4.1cm]{touchctrl}}
+ \hspace{0.3cm}
+ \fbox{\includegraphics[height=4.1cm]{ctrlmouseOn}
+ \includegraphics[height=4.1cm]{ctrlmouseOff}}
+ \caption[CtrlMouse and TouchCtrl]{We explore the impact of duplicating modifier keys on the mouse. \emph{CtrlMouse} (left) mirrors the \protect\ctrl and \protect\shift keys on the mouse to delimit text entry and command selection and reduce the needs of coordination of the left hand. \emph{TouchCtrl} (right) implicitly triggers the command mode when the hand hovers the mouse.}
+ \label{figure:ctrlmousetouchctrl}
+\end{figure}
+
+\paragraph{\emph{CtrlMouse}}
+\emph{\emph{CtrlMouse}} is an interaction technique that facilitates the use of keyboard shortcuts when the user's hand is near the pointing device. \emph{CtrlMouse} uses the mouse buttons located in the vicinity of the thumb as keyboard modifiers (figure \ref{fig:ctrlmouse}). The keyboard switches from \emph{text mode} to \emph{command mode} when users press one of these buttons and return to the text-mode when users release these buttons (quasimode).
+
+\begin{figure}[htb]
+\centering
+\includegraphics[height=3.2cm]{_nopress.pdf}
+\includegraphics[height=3.2cm]{_singlepress.pdf}
+\includegraphics[height=3.2cm]{_dualpress.pdf}
+ \caption[CtrlMouse]{\emph{CtrlMouse} mirrors modifiers on the mouse (left). Users can either press \protect\ctrl (middle) or \protect\ctrl+\protect\shift (right) thanks to a simple small amplitude thumb movement. Users don't have to home to keyboard anymore to perform complicated keyboard shortcuts.}
+ \label{fig:ctrlmouse}
+\end{figure}
+
+As modifiers are dedicated to the right hand, users are neither constrained to perform chord gestures, nor bringing back the right hand to the keyboard when executing complex keyboard shortcuts. More precisely, mirroring the modifiers on the mouse has several advantages:
+
+\emph{Expressivity}. \emph{CtrlMouse} is not meant to replace modifiers on the keyboard, it provides another alternative for users. Users are free to use modifiers on the keyboard or on the mouse according to their preferences.
+
+\emph{Finger coordination.} Traditional keyboard shortcuts often require users to perform chording gestures. Depending on the hotkey location and the number of modifiers, typing these keyboard shortcuts can require subtle finger coordination. It can also result in uncomfortable static muscular loads \cite{putz88}. By splitting hotkeys and modifier keys between both hands, \emph{CtrlMouse} reduces the need for precise finger coordination.
+
+\emph{Hotkey mapping}. Commands are generally assigned to mnemonic hotkeys to favor memorization. However, we also observe that frequent hotkeys are often associated to keys localized on the left side of the keyboard to keep it close enough to the Left \ctrl key. As a result, it is sometimes necessary to increase the number of modifiers to the hotkeys localized on the left side of the keyboard. As \emph{CtrlMouse} alleviates the chording constraints, hotkeys can be uniformly distributed on the keyboard. It provides more flexibility to the designers to define efficient command-hotkey mappings and can reduce the number of hotkeys requiring several modifiers~\cite{bailly13}.
+
+\emph{Multiple modifiers}. When multiple modifiers are still necessary, \emph{CtrlMouse} provides a significant advantage. It supports the activation of two modifiers at once using a single mechanical action of a thumb press.
+
+\emph{Mouse/keyboard transitions.} Some keyboard shortcuts are difficult to perform because of the location of the hotkey, especially if located on the right side of the keyboard \cite{bailly13}. While users can use the \ctrl key on the right side of the keyboard, we observed that with pointing-based applications, users often still use the left \ctrl key with the left hand and press the hotkey with the right hand. \emph{CtrlMouse} buttons avoid these Mouse/Keyboard transitions with the right hand.
+
+\emph{Compatibility}. Usual actions mapped to mouse buttons are simple clicks, wheareas modifiers are maintained until a hotkey is pressed. This means \emph{CtrlMouse} is compatible with usual actions associated with these buttons, such as \modk{Previous} or \modk{Next}.
+
+\emph{Implementation}. \emph{CtrlMouse} does not require additional hardware since several mouses controllers (e.g, Logitech M705, Logitech M905) already have two thumb buttons.
+Technically, presses shorter than $200ms$ trigger the usual action, and a longer presses are interpreted as modifiers.
+
+To sum up, \emph{CtrlMouse} differs from traditional hotkeys by the \emph{location} of the modifier: users can press the dedicated buttons on the mouse instead of modifier keys of the keyboard. However, using the mouse for activating a keyboard modifier offers additional possibilities illustrated with \emph{TouchCtrl}.
+
+
+\paragraph{\emph{TouchCtrl}}
+
+\emph{\emph{TouchCtrl}} has also been designed to investigate the impact of duplicating modifiers on the mouse controller. It \emph{implicitly} switches the keyboard to command mode as soon as the right hand grabs the mouse. \emph{TouchCtrl} is based on the hypothesis that users seldom use the keyboard to type text while the right hand holds the mouse. As a result \emph{TouchCtrl} is, unlike \emph{CtrlMouse}, limited to a single modifier (typically \ctrl). Yet, \emph{TouchCtrl} has similar advantages than \emph{CtrlMouse} (better finger coordination, less movements, easier shortcut mapping, and less keyboard/mouse transitions), but also requires less efforts from the right hand.
+
+\emph{Compatibility}. \emph{TouchCtrl} is based on the hypothesis that users do not type long texts when the right hand hold the mouse as we observed that these situations are rare. However, if users still want to enter text with the left hand, they can briefly lift their right hand up from the mouse.
+
+\emph{Several modifiers}. \emph{TouchCtrl} is more sensible to multiple modifiers than \emph{CtrlMouse} because the binary information (contact between the hand and the mouse) can only replace one modifier. An explicit modifier is still required to trigger keyboard shortcuts that use several modifiers.
+
+\emph{Hardware implementation}. \emph{TouchCtrl} is easily supported by state-of-the-art multitouch mouse such as Apple MagicMouse or Microsoft TouchMouse.
+Moreover, \emph{TouchCtrl} can also be used with a touchpad to activate commands when at least a finger touches the surface.
+
+%Duplicating modifiers on the mouse can provide both qualitative and quantitative advantages compared to the keyboard. In the next sections, we detail two laboratory studies and a field study to compare the use of keyboard and mouse modifiers.
+
+\paragraph{User studies}
+
+We envisioned two factors that could affect the adoption of mouse mouse switchers.
+The first one is the position of the letter key of keyboard shortcuts.
+Some of them make are difficult to type while holding the \ctrl key.
+The second factor is the \emph{Text Entry Demand}.
+
+In the first user study we evaluated the effect of these two factors on the adoption of mouse mode switchers..
+
+
+In the second study we evaluated the user performance.
+
+\paragraph{Discussion}
+
+Uni-manual interaction for typical bi-manual actions: scroll/zoom, multiple item selections, boolean operations on free-hand selections, manipulation constraints.
+
+Trigger for existing techniques~\cite{bier93,grossman05}
+
+\subsection{Métamorphe: a keyboard with actuated keys}
+
+Métamorphe \cite{bailly13}
+
+Keyboard keys have a limited vocabulary.
+They are either pressed or released.
+We actuated keys so that they can be up or down.
+In addition to the additional output modality, in enables new interactions with the sides of the keys: pushes and pinches.
+
+\subsection{Living Desktop: actuated peripherals}
+
+Living Desktop \cite{bailly16}
+
+Beyond the digital desktop, we extend the physical desktop.
+Besides being input and output devices, computer peripherals are tangible objects.
+We not only manipulate them the way they were designed to be used, but also in other ways.
+We explored other uses, and how actuating computer peripherals could ease such tasks.
+
+\section{Revisiting direct manipulation}
+
+\subsection{Direct Manipulation in tactile displays}
+
+Direct Manipulation~\cite{gupta16}
+
+Direct manipulation is one of the most fundamental foundation of GUIs~\cite{schneiderman83}.
+It provides valuable usability benefits that highly contributed to the success of GUIs over command line interfaces.
+Yet, the description of this paradigm relied essentially on visual cues.
+In this project we explained how to adapt the concept of direct manipulation to tactile displays.
+
+\subsection{Summon interactions}
+
+Summon interactions~\cite{gupta17}
+
+3D gestural interaction in the air faces difficult challenges due to the contact-less nature of this interaction style.
+The absence of physical buttons, or other unambiguous activation action makes it difficult to segment gestures.
+As a consequence, the system constantly interprets the users' actions.
+Therefore the user is always potentially interacting with the environment.
+This is known as the Midas touch problem, as a reference to the Midas king in the greek mythology, who turned everything he was touching to gold.
+
+
+\section{Conclusion}
--- /dev/null
+%!TEX root = ../hdrmain.tex
+
+\chapter{Extention of interaction modalities}
+ \epigraph{\lorem}{Auteur}
+
+\begin{Abstract}
+\loremipsum
+\end{Abstract}
+
+\section{Introduction}
+
+\section{Extention of interaction vocabularies}
+
+\subsection{FlexStylus: a flexible digital pen}
+
+FlexStylus \cite{fellion17}
+
+\subsection{Activibe: vibrotactile feedback for activity monitoring}
+
+Activibe~\cite{cauchard16}
+
+\subsection{Tactile textures with programmable friction}
+
+Tactile Textures~\cite{potier12,potier16}
+
+
+\section{New interaction techniques with additional vocabulary}
+
+\subsection{RayCursor: a 3D pointing technique in VR}
+
+RayCursor \cite{baloup19}
+
+Leverages 3D input and combines with 3D motion for more efficient pointing
+
+\subsection{FingerCuts: leveraging finger identification for multi-touch interaction}
+
+FingerCuts \cite{goguey14,goguey14a,goguey17}
+
+Leverages finger identification for command selection and direct manipulation of parameters.
+
+\subsection{Vibrotactile widgets}
+
+Vibrotactile widgets~\cite{frisson17}
+
+Leverages vibrotactile feedback for touch surfaces.
+
+\section{Conclusion}
projects / hdr.git / commitdiff