Decoupling Human and Camera Motion from Videos in the Wild

Vickie Ye、Georgios Pavlakos、Jitendra Malik、Angjoo Kanazawa

University of California, Berkeley

从野外视频中分离人和摄像机运动

要点：

1.提出了一种从野外视频重建全球人类轨迹的方法。该优化方法将相机和人的运动解耦，这允许我们将人放在同一个世界坐标系中。

2.大多数现有方法都没有对相机运动建模；依赖于背景像素来推断3D人体运动的方法通常需要全场景重建，这在野生视频中通常是不可能的。然而，即使当现有SLAM系统不能恢复精确的场景重建时，背景像素运动仍然提供足够的信号来约束相机运动。

3.论文表明，相对相机估计以及数据驱动的人类运动先验可以解决场景尺度的模糊性并恢复全局人类轨迹。我们的方法在野外挑战视频（如PoseTrack）中稳健地恢复了人们的全球3D轨迹。

一句话总结：

在三维人类数据集Egobody上量化了我们对现有方法的改进，进一步证明，论文种恢复的相机比例允许在共享坐标系中推理多个人的运动，这提高了PoseTrack中下游跟踪的性能。代码和视频结果可在https://vye16.github.io/slahmr[机器翻译+人工校对]

Vickie Ye, Georgios Pavlakos, Jitendra Malik, Angjoo Kanazawa
We propose a method to reconstruct global human trajectories from videos in the wild. Our optimization method decouples the camera and human motion, which allows us to place people in the same world coordinate frame. Most existing methods do not model the camera motion; methods that rely on the background pixels to infer 3D human motion usually require a full scene reconstruction, which is often not possible for in-the-wild videos. However, even when existing SLAM systems cannot recover accurate scene reconstructions, the background pixel motion still provides enough signal to constrain the camera motion. We show that relative camera estimates along with data-driven human motion priors can resolve the scene scale ambiguity and recover global human trajectories. Our method robustly recovers the global 3D trajectories of people in challenging in-the-wild videos, such as PoseTrack. We quantify our improvement over existing methods on 3D human dataset Egobody. We further demonstrate that our recovered camera scale allows us to reason about motion of multiple people in a shared coordinate frame, which improves performance of downstream tracking in PoseTrack. Code and video results can be found at https://vye16.github.io/slahmr.

https://arxiv.org/pdf/2302.12827.pdf

内容中包含的图片若涉及版权问题，请及时与我们联系删除