Project summary

The objective of the FIGHS project - Fine Grained Three-dimensional Human Sensing -- is to advance the fundamental methodology for the detailed three-dimensional analysis of multiple people from image and video data. While significant progress has been made in 3d human sensing, no existing system can analyze scene containing multiple people, potentially occluding each other and involved in interactions, in detail. To this end, our proposal aims to (a) create large-scale state of the art in-the-wild datasets of multiple interacting humans with synchronized ground truth images and three dimensional shape and pose for training and evaluation purposes, as well as (b) self-supervised meta-learning, modeling and optimization techniques, that would support the reconstruction of complex scenes containing interacting humans, in detail. A successful fine-grained 3d human system would be widely useful for health, communication, entertainment, analysis of sports performances, robotics and self-driving systems, or manufacturing. Such a system would also support extensive and accurate fine-grained analysis of human social distancing during a pandemic.