Abstract

This paper presents a novel approach for autonomous live sports broadcasting using visual 3D model-based tracking and a vertical take-off and landing (VTOL) unmanned aerial vehicle (UAV) such as a quadcopter or hexacopter in GPS-impaired environments. To achieve this level of autonomy, position estimation is essential and is a highly challenging problem using a monocular camera due to the scale ambiguity. In this paper, we track a tennis court, that is standard in dimension, using a moving edge-based tracker, and recover the scale with the prior knowledge of the fixed playing field. Experimental results are demonstrated in 3 different environments including static scenes, real broadcast video, and indoor flying. We also evaluate the proposed approach with the ground truth provided by a motion capture system and achieve a position estimation with less than 0:02m standard deviation in the error.

References

• C. Chen, O. Wang, S. Heinzle, P. Carr, A. Smolic, and M. Gross. Computational sports broadcasting: Automated director assistance for live sports. In Multimedia and Expo (ICME), 2013 IEEE International Conference on, pages 1–6, July 2013.
• Lawrence L Cone. Skycam-An aerial robotic camera system. Byte, 10(10):122, 1985.
• Christopher Hanson. Television Sport Broadcasting and Technology. Producing Television, Spectator, 2008.
• Sergio Ilarri, Eduardo Mena, Arantza Illarramendi, Roberto Yus, Maider Laka, and Gorka Marcos. A friendly location-aware system to facilitate the work of technical directors when broadcasting sport events. Mobile Information Systems, 8(1):17–43, 2012.
• Jinjun Wang, Changsheng Xu, Engsiong Chng, Hanqing Lu, and Qi Tian. Automatic composition of broadcast sports video. Multimedia Systems, 14(4):179–193, 2008.
• Guangyu Zhu, Changsheng Xu, Qingming Huang, and Wen Gao. Automatic multi-player detection and tracking in broadcast sports video using support vector machine and particle filter. In Multimedia and Expo, 2006 IEEE International Conference on, pages 1629–1632. IEEE, 2006.
• Chung-Yuan Chao, Huang-Chia Shih, and Chung-Lin Huang. Semantics-based highlight extraction of soccer program using DBN. In Acoustics, Speech, and Signal Processing, 2005. Proceedings. (ICASSP'05). IEEE International Conference on, volume 2, pages ii–1057. IEEE, 2005.
• Yoshiaki Itoh, Shigenobu Sakaki, Kazunori Kojima, and Masaaki Ishigame. Highlight scene extraction of sports broadcasts using sports news programs. In Multimedia Signal Processing, 2008 IEEE 10th Workshop on, pages 646–649. IEEE, 2008.
• Jinjun Wang, Changsheng Xu, Engsiong Chng, and Qi Tian. Sports highlight detection from keyword sequences using HMM. In Multimedia and Expo, 2004. ICME'04. 2004 IEEE International Conference on, volume 1, pages 599–602. IEEE, 2004.
• PB Sujit, Joao Sousa, and F Lobo Pereira. UAV and AUVs coordination for ocean exploration. In Oceans 2009-Europe, pages 1–7. IEEE, 2009.
• David W Casbeer, RW Beard, TW McLain, Sai-Ming Li, and Raman K Mehra. Forest fire monitoring with multiple small UAVs. In American Control Conference, 2005. Proceedings of the 2005, pages 3530–3535. IEEE, 2005.
• St D Ilcev and Ivan Skoryk. Automatic landing of animal tracking Lighter-Than-Air Systems (LTAS) on the recharging platform. In Microwave & Telecommunication Technology (CriMiCo), 2014 24th International Crimean Conference, pages 275–277. IEEE, 2014.
• Gurkan Tuna, Tarik Veli Mumcu, and Kayhan Gulez. Design strategies of unmanned aerial vehicle-aided communication for disaster recovery. In High Capacity Optical Networks and Emerging/Enabling Technologies. 2012.
• Inkyu Sa, Stefan Hrabar, and Peter Corke. Inspection of Pole-Like Structures Using a Vision-Controlled VTOL UAV and Shared Autonomy. In IEEE International Conference on Intelligent Robots and Systems, pages 4819–4826, Sep. 2014.
• J Berni, Pablo J Zarco-Tejada, Lola Su´arez, and Elias Fereres. Thermal and narrowband multispectral remote sensing for vegetation monitoring from an unmanned aerial vehicle. Geoscience and Remote Sensing, IEEE Transactions on, 47(3):722–738, 2009.
• S Nebiker, A Annen, M Scherrer, and D Oesch. A light-weight multispectral sensor for micro UAV—Opportunities for very high resolution airborne remote sensing. The international archives of the photogrammetry, remote sensing and spatial information sciences, 37:1193–1200, 2008.
• Aaron Mcfadyen, Peter Corke, and Luis Mejias. Rotorcraft Collision Avoidance using Spherical Image-based Visual Servoing and Single Point Features. In Proc. of the International Conference on Intelligent Robot Systems (IROS), Oct. 2012.
• Inkyu Sa and Peter Corke. System Identification, Estimation and Control for a Cost Effective Open-Source Quadcopter. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2012.
• Stephan M. Weiss. Vision based navigation for micro helicopters. PhD thesis, ETHZ, 2012.
• Vincent Lepetit and Pascal Fua. Monocular model-based 3D tracking of rigid objects: A survey. Foundations and trends in computer graphics and vision, 1(CVLAB-ARTICLE-2005-002):1–89, 2005.
• C´eline Teuliere, Eric Marchand, and Laurent Eck. 3D Model-Based Tracking for UAV Indoor Localization. IEEE Trans. on Cybernetics, 2014.
• C. Teuli`ere, L. Eck, E. Marchand, and N. Gu´enard. 3D model-based tracking for UAV position control. In Intelligent Robots and Systems (IROS), 2010 IEEE/RSJ International Conference on, pages 1084–1089, Oct 2010.
• ´ Eric Marchand, Fabien Spindler, and Franc¸ois Chaumette. ViSP for visual servoing: a generic software platform with a wide class of robot control skills. Robotics & Automation Magazine, IEEE, 12(4):40–52, 2005.
• Patrick Bouthemy. A maximum likelihood framework for determining moving edges. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 11(5):499–511, 1989.
• Zhengyou Zhang. A flexible new technique for camera calibration. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 22(11):1330–1334, 2000.
• S. Hutchinson, G. D. Hager, and P. I. Corke. A tutorial on visual servo control. Robotics and Automation, IEEE Transactions on, 12(5):651 –670, oct 1996.
• Lagadic project. Visual Servoing Platform, Computer vision algorithms, 2012.