The second Workshop on
Reconstruction of Human-Object Interactions (RHOBIN)

Bridging the Gap: Harnessing Human Knowledge for Robotic Manipulation

The internet is a treasure trove of data showcasing human-object interactions, offering rich knowledge for robotic object manipulation. However, a significant challenge remains: how can we translate this wealth of knowledge into practical applications for robotic systems, given the differences in physical capabilities and embodiments between humans and robots? In this talk, I will unveil innovative strategies to bridge this embodiment gap, enabling robots to perform complex object manipulations by learning from human behaviors. Discover how these cutting-edge techniques are revolutionizing robotic manipulation in real-world scenarios.

Human-centric 3D Scenes

Understanding human interactions within 3D scenes is crucial for many applications, including robotics, virtual reality, and animation. However, limited ground truth 3D/4D data of human interactions makes such interaction understanding and modeling very challenging. We first propose to model dynamic human-object interactions through a contact-guided diffusion formulation, to enable text-guided human-object interaction synthesis. In order to achieve wide generality, we then propose to distill knowledge from powerful vision-language models to inform 3D human-scene interaction synthesis in a new zero-shot formulation. We hope this will pave the way for more realistic, generalized paradigms for interaction analysis and synthesis.