Robot Learning

Learning a Thousand Tasks in a Day

We discovered that decomposing manipulation trajectories into object alignment followed by object interaction enables learning new tasks from single demonstrations, compared to tens or hundreds typically required. Through 3,450 real-world experiments, we developed Multi-Task Trajectory Transfer (MT3), which combines trajectory decomposition with retrieval in learned latent spaces. Thanks to MT3's data efficiency, we were able to teach a robot 1000 distinct manipulation tasks in less than 24 hours of human demonstrator time, while also generalizing effectively to novel objects. This work was featured on the cover of the November 2025 issue of Science Robotics.

Adapting Skills to Novel Grasps: A Self-Supervised Approach

We found that robots typically fail to transfer skills when grasping objects differently than during training. To address this, we developed a self-supervised data collection method that enables grasp-invariant skill transfer. Our approach allows robots to successfully adapt manipulation skills to novel grasps without requiring additional human demonstrations, demonstrating robust generalization across varying grasp configurations through systematic real-world validation.

One-Shot Imitation Learning: A Pose Estimation Perspective

We investigated how combining trajectory transfer with unseen object pose estimation enables robots to learn new tasks from single demonstrations, eliminating the need for additional data collection or training. Through systematic experimentation, we characterized the impact of pose estimation errors and camera calibration errors on task success rates, revealing critical tolerance thresholds for reliable one-shot learning. This analysis provides practical insights for deploying vision-based manipulation systems in real-world environments.

Computer Vision for Robotics

Learning Eye-in-Hand Camera Calibration from a Single Image

We explored learning-based approaches for extrinsic calibration of wrist-mounted RGB cameras using only single images. Through comparative analysis of multiple methods, we discovered that direct regression of calibration parameters outperformed both alternative learning-based techniques and classical marker-based calibration approaches. This finding demonstrates that data-driven methods can achieve superior calibration accuracy while eliminating the need for physical markers, streamlining robot deployment in new environments.

Hybrid ICP

We found that traditional ICP algorithms rely on fixed data association methods and error metrics, limiting their adaptability to varying scene conditions. To address this, we developed a dynamic ICP variant that automatically optimizes both data association and error metrics in real-time based on live object observations and current pose estimates. Through experimental validation, this adaptive approach achieved superior registration accuracy compared to fixed-parameter methods, particularly in challenging scenarios with partial occlusions and varying object geometries.

Combinatorial Bayesian Optimisation

Framework and Benchmarks for Combinatorial and Mixed-variable Bayesian Optimization

We discovered that the field of Mixed-variable and Combinatorial Bayesian Optimization lacked systematic benchmarking standards, hindering meaningful comparison of approaches. To address this, we developed a modular framework that enables seamless integration of Bayesian Optimization components across diverse tasks. Through over 4,000 experiments comparing 47 novel algorithms against 12 existing solvers across ten benchmarking tasks, we identified a superior combination of MCBO primitives that outperforms previous methods. Our analysis revealed that model fit and trust region design are critical factors for achieving robust optimization performance across mixed-variable spaces.

Toward real-world automated antibody design with combinatorial Bayesian optimization

We found that therapeutic antibody design requires simultaneously optimizing antigen-binding affinity and developability properties across vast sequence spaces that prohibit exhaustive search. To tackle this challenge, we formulated the problem as combinatorial black-box optimization, where oracle functions simulate binding affinity from antibody-antigen sequence pairs. Through our computational framework, we developed efficient search mechanisms that identify optimal antibody sequences maximizing binding affinity while preserving favorable biophysical properties, dramatically reducing the experimental candidates requiring laboratory validation.