Tele-Immersive Display with Live-Streamed Video Wai-Kwan Tang

Tele-immersion is a rapidly growing technology which allows geographically separated users to collaborate in a shared virtual environment. Video systems in those tele-immersion applications, however, usually suffer from a limited field-of-view. To capture wide field-of-view video, specially designed video systems are required.

In this thesis, we present the techniques for the real-time construction of live panoramic video from multiple live video streams obtained by normal CCD video cameras. A system framework for tele-immersive applications is designed. In our approach, we first obtain sample video frames from the video cameras. Using techniques of generating image mosaic, the video frames are combined together to form a large field of view image. We obtain a list of triangles with texture coordinates. In the running time, the live video streams are used as texture map which are rendered into a live panoramic video stream. The generated video is then projected onto an immersive display.

Our system framework can allow many potential tele-immersive applications. For example, almost all current video conferencing applications are limited to a small field of view video. Using our system framework, instead of seeing only a talking head, the users can get the feeling of immersing into a virtual conference room. Virtual control room, immersive remote navigation, virtual laboratory, tele-medicine are some possible directions which can gain benefit from our system framework.

Related publication:

• " A System for Real-time Panorama Generation and Display in Tele-immersive Applications",
  W. K. Tang, T. T. Wong and P. A. Heng,
  IEEE Transactions on Multimedia, Vol. 7, No. 2, April 2005, pp. 280-292.

• "The Immersive Cockpit System for Capturing Natural Heritage",
  W. K. Tang, T. T. Wong and P. A. Heng,
  Journal of System Simulation, Vol. 15, No. 3, March 2003, to appear.

• "The Immersive Cockpit",
  W. K. Tang, T. T. Wong and P. A. Heng,
  in Proceedings of ACM Multimedia 2002, Juan Les Pins, France, December 2002, pp. 658-659.

• "The Immersive Cockpit",
  W. K. Tang, T. T. Wong and P. A. Heng,
  in Proceedings of International Workshop on Immersive Telepresence (ITP 2002), Juan Les Pins, France, December 2002, pp. 36-39.