Introduction

The Pose-Ball Estimation (PBE) system is a synthetic system for ball trajectory tracking and poses capturing.

I present the PBE to offer a 3D, accurate, and multi-viewpoint replay of ball trajectory tracking and pose estimation. It was always hard for coaches to obtain athletes’ sports performances through videos due to restrictions including single viewpoint and non-stereoscopic observation. In the PBE, however, we offer great multi-view realistic and animated replays with skeletons and balls. With multiple angles of videos/images in overlapped time, it gives the trajectory of the ball and the skeleton representation of humans.

DEMO: Example listed below.

Result: The results offer free viewpoint replay with 3d skeletons and reprojections of skeletons and ball trajectories back to the original view.

However, due to the limited camera resolution and calibration deviation, there would naturally be some errors in reprojecting positions.

Raw: Videos were shot from 11 viewpoints, here I present one of eleven views as an example.

Restricted by camera shooting range and light conditions, only few out of eleven views are available for reconstruction. Through experiments, we need at least three available viewpoints to meet the minimum requirements (more is better).

Features:

  1. High adaptability: Given the input video, the system automatically match timestamps, upload to the designated GPU, process detection and estimation, do post-processing including filtering and interpolation, and save both image output and video output.
  2. Various arguments to be set: You could use your own backbone of pose estimation and your own model for object detection; You could easily apply different camera extrinsic; You could modify detecting and estimating confidence, etc.
  3. 3D skeletons from all angles could be obtained through our system. Moreover, you could specify if only part of the input sequence is needed to be processed.