I am a Principal Researcher at Microsoft Research, in the Extended Perception, Interaction & Cognition (EPIC) and Future Experiences (FX) group. Before joining Microsoft, I was Professor of Interaction Design and Innovation at University College London, UCL Interaction Centre, in the Faculty of Engineering and the Faculty of Brain Sciences. I am an Honorary Professor at UCL, Department of Computer Science, and UCL Interaction Centre.
My current research invents expressive, real‑time direct‑manipulation techniques with generative AI — turning models into instruments that people can create with, ideate with, and steer continuously, as expressive input modalities complementing natural language AI interactions. Recent projects explore text as a living, AI‑responsive material; generative models as parameterised reusable instruments and tools; and ways to capture intent as first‑class inputs to AI systems. More broadly, my work builds collaborative technology that empowers people: AI‑supported expressive techniques for crafting and manipulating content; sensor‑driven interaction tuned to spatial and social context; and novel hardware and form‑factors that situate these tools in everyday environments. My research spans across generative AI human–computer interaction, collaborative systems, multi‑device computing, and prototyping platforms and toolkits. I am co‑author of the Sketching User Experiences Workbook (Morgan Kaufmann, 2011) and Proxemic Interactions (Morgan & Claypool, 2015), and have received multiple ACM best‑paper and honourable‑mention awards, as well as two 10‑year Impact Awards at ACM ISS for our work on Proxemic Interactions (2019) and Spatially‑Aware Displays for Collaboration (2023).
Research
Current research projects exploring direct-manipulation interaction with generative AI.

Texterial
Material-inspired interactions for rich direct manipulation and editing of text on the canvas.

AI-Instruments
Tangible instruments that turn prompts into reusable, GUI-style tools for working with generative AI.

ImaginationVellum
A canvas where spatial prompts, generative strokes and AI ideation flow together.
List of 126 publications below (86 full papers and journal articles). Google Scholar, DBLP, Microsoft Research profile. Download the complete BibTeX bibliography (.bib) or expand any entry to copy the citation.
2026
-
Texterial: A Text-as-Material Interaction Paradigm for LLM-Mediated Writing
CHI 2026 · Barcelona, Spain Honourable Mention
-
Proscenium: Exploring Design Spaces of Layered Information Experience on a Large Dual-Layer Transparent Display
CHI EA 2026 · Barcelona, Spain
2025
-
AI-Instruments: Embodying Prompts as Instruments to Abstract & Reflect Graphical Interface Commands as General-Purpose Tools
CHI 2025 · Yokohama, Japan Honourable Mention
-
ImaginationVellum: Generative-AI Ideation Canvas with Spatial Prompts, Generative Strokes, and Ideation History
UIST 2025 · Busan, Republic of Korea
-
2024
-
BlendScape: Enabling End-User Customization of Video-Conferencing Environments through Generative AI
UIST 2024 · Pittsburgh, PA, USA Honourable Mention
-
SpaceBlender: Creating Context-Rich Collaborative Spaces Through Generative 3D Scene Blending
UIST 2024 · Pittsburgh, PA, USA
-
Exploring the Design Space of Input Modalities for Working in Mixed Reality on Long-haul Flights
DIS 2024 · Copenhagen, Denmark
-
HoloDevice: Holographic Cross-Device Interactions for Remote Collaboration
CoRR 2024 · Honolulu, HI, USA
-
-
2023
-
-
-
-
-
Analysis of Product Architectures of Pin Array Technologies for Tactile Displays
Proc. ACM Hum. Comput. Interact. 2023
2022
-
-
-
-
Eliciting User-Defined Touch and Mid-air Gestures for Co-located Mobile Gaming
Proc. ACM Hum. Comput. Interact. 2022
-
Tangible Interaction with In-Car Smart Intelligence
AutomotiveUI (Adjunct Proceedings) 2022
2021
-
Sketchnote Components, Design Space Dimensions, and Strategies for Effective Visual Note Taking
CHI 2021 Honourable Mention
-
-
-
-
-
-
-
-
-
-
2020
-
-
-
-
Human Observer and Automatic Assessment of Movement Related Self-Efficacy in Chronic Pain: From Exercise to Functional Activity
IEEE Trans. Affect. Comput. 2020
-
-
-
PACM HCI V4 ISS, November 2020 - Editorial
Proc. ACM Hum. Comput. Interact. 2020
-
-
2019
-
How Can Affect Be Detected and Represented in Technological Support for Physical Rehabilitation?
ACM Trans. Comput. Hum. Interact. 2019
-
-
-
-
-
-
2018
-
SketchBlog #1: the rise and rise of the sketchnote
Interactions 2018
-
Feeling SketCHI?: the lasting appeal of the drawn image in HCI
Interactions 2018
-
-
-
-
-
-
-
EagleView: A Video Analysis Tool for Visualising and Querying Spatial Interactions of People and Devices
ISS 2018 Honourable Mention
-
-
-
2017
-
-
-
From the BBC micro to micro: bit and beyond: a British innovation
Interactions 2017
-
-
-
Opportunities and challenges for cross-device interactions in the wild
Interactions 2017
-
-
LOOP: A Physical Artifact to Facilitate Seamless Interaction with Personal Data in Everyday Life
DIS (Companion Volume) 2017
-
-
-
2016
-
-
Physikit: Data Engagement Through Physical Ambient Visualizations in the Home
CHI 2016 Honourable Mention
-
-
-
-
-
Proxemic Mobile Collocated Interactions
CHI EA 2016
-
Advances in DIY Health and Wellbeing
CHI EA 2016
-
Democratizing children's engagement with the internet of things through connectUs
UbiComp Adjunct 2016
-
-
2015
-
-
As Light as your Footsteps: Altering Walking Sounds to Change Perceived Body Weight, Emotional State and Gait
CHI 2015 Honourable Mention
-
-
Proxemic Interactions: From Theory to Practice
Proxemic Interactions 2015
-
EXCITE: EXploring Collaborative Interaction in Tracked Environments
INTERACT (2) 2015
-
-
-
-
Sketching User Experiences: The Hands-on Course
CHI EA 2015
-
SenCity 2: visualizing the hidden pulse of a city
UbiComp/ISWC Adjunct 2015
2014
-
The design of slow-motion feedback
DIS 2014
-
HuddleLamp: Spatially-Aware Mobile Displays for Ad-hoc Around-the-Table Collaboration
ITS 2014 10-Year Impact
-
-
-
-
The Dark Patterns of Proxemic Sensing
Computer 2014
2013
-
-
-
Proxemic Interactions in Ubiquitous Computing Ecologies
University of Calgary, Alberta, Canada 2013
2012
-
Informing the Design of Proxemic Interactions
IEEE Pervasive Comput. 2012
-
-
-
-
-
Sketching User Experiences - The Workbook
Academic Press 2012
-
-
2011
2010
-
-
-
-
Creating, Editing, and Sharing Complex Ubiquitous Computing Environment Configurations with CollaborationBus
Scalable Comput. Pract. Exp. 2010
-
Proxemic interactions: the new ubicomp?
Interactions 2010
-
Proxemic interaction: designing for a proximity and orientation-aware environment
ITS 2010 Best Paper 10-Year Impact
-
Visible and controllable RFID tags
CHI EA 2010
2009
-
The Haptic Tabletop Puck: the video
ITS 2009
-
2007
2006
-
Sens-ation: a service-oriented platform for developing sensor-based infrastructures
Int. J. Internet Protoc. Technol. 2006
