Department of Electrical Engineering (ISY)

Model-Based Video Coding using Colour and Depth Cameras

David Sandberg, Per-Erik Forssén, Jens Ogniewski
DICTA11, Noosa Heads, Australia
Digital Image Computing: Techniques and Applications (DICTA11)
December 2011

Abstract

In this paper, we present a model-based video coding method that uses input from colour and depth cameras, such as the Microsoft Kinect. The model-based approach uses a 3D representation of the scene, enabling several other applications besides video playback. Some of these applications are stereoscopic viewing, object insertion for augmented reality and free viewpoint viewing. The video encoding step uses computer vision to estimate the camera motion. The scene geometry is represented by keyframes, which are encoded as 3D quads using a quadtree, allowing good compression rates. Camera motion in-between keyframes is approximated to be linear. The relative camera positions at keyframes and the scene geometry are then compressed and transmitted to the decoder. Our experiments demonstrate that the model-based approach delivers a high level of detail at competitively low bitrates.

Full Paper

Portable document format file PDF ()
This paper was awarded the Canon best colour paper prize

---

Bibtex entry

@InProceedings{sandberg11,
  author = 	 {David Sandberg and Per-Erik Forss\'en and Jens Ogniewski},
  title = 	 {Model-Based Video Coding using Colour and Depth Cameras},
  booktitle = {Digital Image Computing: Techniques and Applications ({DICTA11})},
  OPTpages = 	 {},
  year = 	 {2011},
  OPTeditor = 	 {},
  address = 	 {Noosa Heads, Australia},
  month = 	 {December},
  organization = {{IEEE}},
  isbn =         {978-1-4577-2006-2},
  OPTpublisher = {}
}