I am a PhD student in the Advanced Interactive Technologies lab at ETH Zurich. I have double-degree in Human-Computer Interaction & Design from Aalto University, Finland and the University of Twente, the Netherlands. During my Master's I did an internship in the User Interfaces group led by Antti Oulasvirta at Aalto University. Before that I completed a B.Sc. in Industrial Design at the University of Technology Eindhoven, the Netherlands.

### Research Interests

My current research is at the intersection of Human-Computer Interaction, Optimization and Reinforcement Learing. I focus on modelling user interactions with their environment via heirarchical control strategies. Prior to this I focused on haptic feedback systems. Specifically I built custom actuators and developed novel control strategies for haptic interactions in Virtual and Augmented Reality.

# Publications

## Papers

#### Hedgehog: Handheld Spherical Pin Array based on a Central Electromagnetic Actuator

Authors: A. Abler, J. Zarate, T. Langerak, V. Vechev, O. Hilliges
In Proceedings: Proceedings of the IEEE World Haptics Conference, Virtual, July 2021
Honorable Mention - Best Paper Award

Abstract

We present Hedgehog, a single-actuator spherical pin-array device that produces cutaneous haptic sensations to the user’s palms. Hedgehog can enrich digital experiences by providing dynamic haptic patterns over a spherical surface using a simple, hand-held device. The key to our design is that it uses a single central actuator, a spherical omnidirectional electromagnet, to control the extension of all the 86 movable pins. This keeps our design simple to fabricate and scalable. A core challenge with this type of design is that the pins in the array, made out of permanent magnets, need to have a stable position when retracted. We present a method to compute such an arrays’ spatial stability, evaluate our hardware implementation in terms of its output force and pin’s extension and compare it against our method’s predictions. We also report our findings from three user studies investigating the perceived force and speed of traveling patterns. Finally, we present insights on the possible applications of Hedgehog.

bibtex
@inproceedings{abler2021a,
author = {Abler Aline and Zarate, Juan and Langerak, Thomas and Vechev, Velko and Hilliges, Otmar},
title     = {Hedgehog: Handheld Spherical Pin Array based on a CentralElectromagnetic Actuator},
booktitle = {World Haptics Conference},
year={2021}
}}

#### Omni: Volumetric Sensing and Actuation of Passive Magnetic Tools for Dynamic Haptic Feedback

Authors: T. Langerak*, J. Zarate*, D. Lindlbauer, C. Holz, O. Hilliges
In Proceedings: Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology, Virtual Event, USA, 2020
* These two authors contributed equally to this work.

Abstract

We present Omni, a self-contained 3D haptic feedback system that is capable of sensing and actuating an untethered, passive tool containing only a small embedded permanent magnet. Omni enriches AR, VR and desktop applications by providing an active haptic experience using a simple apparatus centered around an electromagnetic base. The spatial haptic capabilities of Omni are enabled by a novel gradient-based method to reconstruct the 3D position of the permanent magnet in midair using the measurements from eight off-the-shelf hall sensors that are integrated into the base. Omni's 3 DoF spherical electromagnet simultaneously exerts dynamic and precise radial and tangential forces in a volumetric space around the device. Since our system is fully integrated, contains no moving parts and requires no external tracking, it is easy and affordable to fabricate. We describe Omni’s hardware implementation, the 3D reconstruction algorithm, and evaluate the tracking and actuation performance in depth. Finally, we demonstrate its capabilities via a set of interactive usage scenarios.

bibtex
@inbook{10.1145/3379337.3415589,
author = {Langerak, Thomas and Z\'{a}rate, Juan Jos\'{e} and Lindlbauer, David and Holz, Christian and Hilliges, Otmar},
title = {Omni: Volumetric Sensing and Actuation of Passive Magnetic Tools for Dynamic Haptic Feedback},
year = {2020},
isbn = {9781450375146},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3379337.3415589},
abstract = {We present Omni, a self-contained 3D haptic feedback system that is capable of sensing and actuating an untethered, passive tool containing only a small embedded permanent magnet. Omni enriches AR, VR and desktop applications by providing an active haptic experience using a simple apparatus centered around an electromagnetic base. The spatial haptic capabilities of Omni are enabled by a novel gradient-based method to reconstruct the 3D position of the permanent magnet in midair using the measurements from eight off-the-shelf hall sensors that are integrated into the base. Omni's 3 DoF spherical electromagnet simultaneously exerts dynamic and precise radial and tangential forces in a volumetric space around the device. Since our system is fully integrated, contains no moving parts and requires no external tracking, it is easy and affordable to fabricate. We describe Omni's hardware implementation, our 3D reconstruction algorithm, and evaluate the tracking and actuation performance in depth. Finally, we demonstrate its capabilities via a set of interactive usage scenarios.},
booktitle = {Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology},
pages = {594–606},
numpages = {13}
}

#### Optimal Control for Electromagnetic Haptic Guidance Systems

Authors: T. Langerak, J. Zarate, V. Vechev, D. Lindlbauer, D. Panozzo, O. Hilliges
In Proceedings: Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology, Virtual Event, USA, 2020

Abstract

We introduce an optimal control method for electromagnetic haptic guidance systems. Our real-time approach assists users in pen-based tasks such as drawing, sketching or designing. The key to our control method is that it guides users, yet does not take away agency. Existing approaches force the stylus to a continuously advancing setpoint on a target trajectory, leading to undesirable behavior such as loss of haptic guidance or unintended snapping. Our control approach, in contrast, gently pulls users towards the target trajectory, allowing them to always easily override the system to adapt their input spontaneously and draw at their own speed. To achieve this flexible guidance, our optimization iteratively predicts the motion of an input device such as a pen, and adjusts the position and strength of an underlying dynamic electromagnetic actuator accordingly. To enable real-time computation, we additionally introduce a novel and fast approximate model of an electromagnet. We demonstrate the applicability of our approach by implementing it on a prototypical hardware platform based on an electromagnet moving on a bi-axial linear stage, as well as a set of applications. Experimental results show that our approach is more accurate and preferred by users compared to open-loop and time-dependent closed-loop approaches

bibtex
@inbook{10.1145/3379337.3415593,
author = {Langerak, Thomas and Z\'{a}rate, Juan Jos\'{e} and Vechev, Velko and Lindlbauer, David and Panozzo, Daniele and Hilliges, Otmar},
title = {Optimal Control for Electromagnetic Haptic Guidance Systems},
year = {2020},
isbn = {9781450375146},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3379337.3415593},
abstract = {We introduce an optimal control method for electromagnetic haptic guidance systems. Our real-time approach assists users in pen-based tasks such as drawing, sketching or designing. The key to our control method is that it guides users, yet does not take away agency. Existing approaches force the stylus to a continuously advancing setpoint on a target trajectory, leading to undesirable behavior such as loss of haptic guidance or unintended snapping. Our control approach, in contrast, gently pulls users towards the target trajectory, allowing them to always easily override the system to adapt their input spontaneously and draw at their own speed. To achieve this flexible guidance, our optimization iteratively predicts the motion of an input device such as a pen, and adjusts the position and strength of an underlying dynamic electromagnetic actuator accordingly. To enable real-time computation, we additionally introduce a novel and fast approximate model of an electromagnet. We demonstrate the applicability of our approach by implementing it on a prototypical hardware platform based on an electromagnet moving on a bi-axial linear stage, as well as a set of applications. Experimental results show that our approach is more accurate and preferred by users compared to open-loop and time-dependent closed-loop approaches.},
booktitle = {Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology},
pages = {951–965},
numpages = {15}
}

#### Contact-free Nonplanar Haptics with a Spherical Electromagnet

Authors: J. Zarate*, T. Langerak*, B. Thomaszewski, O. Hilliges
In Proceedings: 2020 IEEE Haptics Symposium (HAPTICS), 2020
* These two authors contributed equally to this work.

Abstract

In this paper we introduce a novel contact-free volumetric haptic feedback device. A symmetric electromagnet is used in combination with a dipole magnet model and a simple control law to deliver dynamically adjustable forces onto a hand-held tool. The tool only requires an embedded permanent magnet and thus can be entirely untethered. The force, however, while contact-free, remains grounded via the spherical electromagnet and relatively large forces (1N at contact) can be felt by the user. The device is capable of rendering both attracting and repulsive forces in a thin shell around the electromagnet. We report findings from a user experiment with 6 participants, characterizing force delivery aspects and perceived precision of our system. We found that users can discern at least 25 locations for repulsive forces.

bibtex
@inproceedings{zarate2020contact,
title={Contact-free Nonplanar Haptics with a Spherical Electromagnet},
author={Zarate, Juan Jose and Langerak, Thomas and Thomaszewski, Bernhard and Hilliges, Otmar},
booktitle={2020 IEEE Haptics Symposium (HAPTICS)},
pages={698--704},
year={2020},
organization={IEEE}
}

## Posters, Demos & Workshops

#### Generalizing User Models through Hybrid Hierarchical Control

Authors: T. Langerak, S. Christen, A. Feit, O. Hilliges
In Proceedings: Workshop on Reinforcement Learning for Humans, Computer, and Interaction, 2021

Abstract

Reinforcement Learning has two main challenges in the field of Human-Computer Interaction. The first challenge is generalization across tasks and environments. The second challenge is to achieve human-likeness. We propose a Hybrid Hierarchical Control framework for pointing tasks to address both challenges simultaneously. In our framework, we separate high-level decision-making from low-level motor and gaze control. This hierarchical structure promotes generalizability. By constraining the low-level control to human-like capabilities we aim to achieve human-like results. Finally, we present some applications that our framework could be used for.

bibtex
@article{langerak2021generalizing,
title={Generalizing User Models through Hybrid Hierarchical Control},
author={Langerak, Thomas and Christen, Sammy and Feit, Anna Maria and Hilliges, Otmar},
year={2021}}

#### A demonstration on dynamic drawing guidance via electromagnetic haptic feedback

Authors: T. Langerak, J. Zarate, V. Vechev, D. Panozzo, O. Hilliges
In Proceedings: The Adjunct Publication of the 32nd Annual ACM Symposium on User Interface Software and Technology, 2019

Abstract

We demonstrate a system to deliver dynamic guidance in drawing, sketching and handwriting tasks via an electromagnet moving underneath a high refresh rate pressure sensitive tablet. The system allows the user to move the pen at their own pace and style and does not take away control. Using a closed-loop time-free approach allows for error-correcting behavior. The user will experience to be smoothly and natural pulled back to the desired trajectory rather than pushing or pulling the pen to a continuously advancing setpoint. The optimization of the setpoint with regard to the user is unique in our approach.

bibtex
@inproceedings{langerak2019demonstration,
title={A demonstration on dynamic drawing guidance via electromagnetic haptic feedback},
author={Langerak, Thomas and Zarate, Juan and Vechev, Velko and Panozzo, Daniele and Hilliges, Otmar},
booktitle={The Adjunct Publication of the 32nd Annual ACM Symposium on User Interface Software and Technology},
pages={110--112},
year={2019}
}

#### Aalto Interface Metrics (AIM) A Service and Codebase for Computational GUI Evaluation

Authors: A. Oulasvirta, S. De Pascale, J. Koch, T. Langerak, J. Jokinen, K. Todi, M. Laine, M. Kristhombuge, Y. Zhu, A. Miniukovich, G. Palmas, T. Weinkauf
In Proceedings: The Adjunct Publication of the 31st Annual ACM Symposium on User Interface Software and Technology, 2018

Abstract

Aalto Interface Metrics (AIM) pools several empirically validated models and metrics of user perception and attention into an easy-to-use online service for the evaluation of graphical user interface (GUI) designs. Users input a GUI design via URL, and select from a list of 17 different metrics covering aspects ranging from visual clutter to visual learnability. AIM presents detailed breakdowns, visualizations, and statistical comparisons, enabling designers and practitioners to detect shortcomings and possible improvements. The web service and code repository are available at interfacemetrics.aalto.fi

bibtex
@inproceedings{10.1145/3266037.3266087,
author = {Oulasvirta, Antti and De Pascale, Samuli and Koch, Janin and Langerak, Thomas and Jokinen, Jussi and Todi, Kashyap and Laine, Markku and Kristhombuge, Manoj and Zhu, Yuxi and Miniukovich, Aliaksei and Palmas, Gregorio and Weinkauf, Tino},
title = {Aalto Interface Metrics (AIM): A Service and Codebase for Computational GUI Evaluation},
year = {2018},
isbn = {9781450359498},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3266037.3266087},
doi = {10.1145/3266037.3266087},
abstract = {Aalto Interface Metrics (AIM) pools several empirically validated models and metrics of user perception and attention into an easy-to-use online service for the evaluation of graphical user interface (GUI) designs. Users input a GUI design via URL, and select from a list of 17 different metrics covering aspects ranging from visual clutter to visual learnability. AIM presents detailed breakdowns, visualizations, and statistical comparisons, enabling designers and practitioners to detect shortcomings and possible improvements. The web service and code repository are available at interfacemetrics.aalto.fi.},
booktitle = {The 31st Annual ACM Symposium on User Interface Software and Technology Adjunct Proceedings},
pages = {16–19},
numpages = {4},
keywords = {user interfaces, computational evaluation, metrics, ui layouts},
location = {Berlin, Germany},
}

# Other

## Reviewing

• 2021: CHI, IEEE Sensors, IEEE WHC
• 2020: CHI, UIST
• 2019: CHI, UIST

2021

2020 2019