The second Workshop on
Reconstruction of Human-Object Interactions (RHOBIN)

June 17, 13²⁵ - 18⁰⁰ @ CVPR 2024,Seattle WA, USA
Summit 427 @ Seattle Convention Center

Challenge Winners

Human Reconstruction track	1st place	Fabien Baradel, Thomas Lucas, Matthieu Armando, Romain Brégier, Salma Galaaoui, Philippe Weinzaepfel, Gregory Rogez from NAVER LABS
Human Reconstruction track	2nd place	Yangxu Yan¹, Anlong Ming¹, Huadong Ma¹, Zhaowen Lin¹, Weihong Yao² from ¹Beijing University of Posts and Telecommunications and ²Shanghai Vision Era Co. Ltd.
Joint Human-Object Reconstruction track	1st place	Yuyang Jing¹, Anlong Ming¹, Yongchang Zhang¹, Xiaohai Yu¹, Huadong Ma¹, Weihong Yao² from ¹Beijing University of Posts and Telecommunications and ²Shanghai Vision Era Co. Ltd.
Joint Human-Object Reconstruction track	2nd place	Feng Xue¹, Ke Han¹, Tianrui Ye² from ¹University of Trento and ²University of North Carolina at Chapel Hill
3D Contact Estimation track	1st place	Qian Wu¹, Jun Liu^1,2 from ¹Singapore University of Technology and Design and ²Lancaster University
	2nd place	Chengfeng Wang, Yuhang Yang, Wei Zhai, Yang Cao, Zhengjun Zha from University of Science and Technology of China (USTC)
	3rd place	Seungeun Lee¹, Donguk Kim², Kyungmoon Lee³ from ¹KLLEON, ²SHIFTUP, and ³NALBI

Paper Presentations

1	Mara Levy (University of Maryland), Abhinav Shrivastava (University of Maryland)	V-VIPE: Variational View Invariant Pose Embedding Abstract: Learning to represent three dimensional (3D) human pose given a two dimensional (2D) image of a person, is a challenging problem. In order to make the problem less ambiguous it has become common practice to estimate 3D pose in the camera coordinate space. However, this makes the task of comparing two 3D poses difficult. In this paper, we address this challenge by separating the problem of estimating 3D pose from 2D images into two steps. We use a variational autoencoder (VAE) to find an embedding that represents 3D poses in world coordinate space using a canonical camera viewpoint. We refer to this embedding as variational view-invariant pose embedding V-VIPE. Using V-VIPE we can encode 2D and 3D poses and use the embedding for downstream tasks, like retrieval and classification. We can estimate 3D poses from these embeddings using the decoder as well as generate unseen 3D poses. The variability of our encoding allows it to generalize well to unseen camera views when mapping from 2D space. To the best of our knowledge, V-VIPE is the only representation to offer this diversity of applications.
2	Bedirhan Uğuz (Middle East Technical University [METU]), Özhan Suat (METU), Batuhan Karagoz (METU), Emre Akbas (METU)	MoCap-to-Visual Domain Adaptation for Efficient Human Mesh Estimation from 2D Keypoints Abstract: This paper presents Key2Mesh, a model that takes a set of 2D human pose keypoints as input and estimates the corresponding body mesh. Since this process does not involve any visual (i.e. RGB image) data, the model can be trained on large-scale motion capture (MoCap) datasets, thereby overcoming the scarcity of image datasets with 3D labels. To enable the model's application on RGB images, we first run an off-the-shelf 2D pose estimator to obtain the 2D keypoints, and then feed these 2D keypoints to Key2Mesh. To improve the performance of our model on RGB images, we apply an adversarial domain adaptation (DA) method to bridge the gap between the MoCap and visual domains. Crucially, our DA method does not require 3D labels for visual data, which enables adaptation to target sets without the need for costly labels. We evaluate Key2Mesh for the task of estimating 3D human meshes from 2D keypoints, in the absence of RGB and mesh label pairs. Our results on widely used H3.6M and 3DPW datasets show that Key2Mesh sets the new state-of-the-art by outperforming other models in PA-MPJPE for both datasets, and in MPJPE and PVE for the 3DPW dataset. Thanks to our model's simple architecture, it operates at least 12x faster than the prior state-of-the-art model, LGD. We provide sample video outputs and additional qualitative samples in the supplementary material. Code will be released.
3	Md Mushfiqur Azam (The University of Texas at San Antonio), Kevin Desai (University of Texas at San Antonio)	A Survey on 3D Egocentric Human Pose Estimation Abstract: Egocentric human pose estimation aims to estimate human body poses and develop body representations from a first-person camera perspective. It has gained vast popularity in recent years because of its wide range of applications in sectors like XR-technologies, human-computer interaction, and fitness tracking. However, to the best of our knowledge, there is no systematic literature review based on the proposed solutions regarding egocentric 3D human pose estimation. To that end, the aim of this survey paper is to provide an extensive overview of the current state of egocentric pose estimation research. In this paper, we categorize and discuss the popular datasets and the different pose estimation models, highlighting the strengths and weaknesses of different methods by comparative analysis. This survey can be a valuable resource for both researchers and practitioners in the field, offering insights into key concepts and cutting-edge solutions in egocentric pose estimation, its wide-ranging applications, as well as the open problems with future scope.

Contact Info

E-mail: rhobinchallenge@gmail.com