Interactive Digital Cinema Project

Thumbnail
selfbuilt 3D DLP LED projector

Interactive Digital Cinema Project

Immersive Interactive Multisensory Cinema Capture and Playback system studies novel aesthetic, expressive and narrative possibilities enabled by creative misuses of hardware and software technologies and computer vision methods. The project utilizes appropriated, hacked and modified hardware, microcontrollers, multiple 3d and video softwares using a custom workflow, and a minimalist viewing furniture based on a customized version of finnish industrial designer Esa Vesmanen's lounger design.

Core system components are: A) A 17 1920x1080 progressive HD cam 3D capture rig, B) Software based capture workflow consisting of 3D surface reconstruction/image based rendering, batch processing, 3D file formatting and importing to Unity3D Pro realtime game engine, C) A custom selfbuilt 1920x1080 progressive LED backlit stereoscopic 3D DLP front projector built from hijacked Samsung/Texas Instruments components, D) A custom lounger chair modified together with designer Esa Vesmanen based on his lounger chair designs (www.puredesign.fi). The chair consists of a selfmade USB scent stick engine microcontroller board, scent designs by artist Hilda Kozari, Clark Synthesis Platinum transducer, novel phase accurate speakers for speaker based crosstalk cancelled binaural 3D audio. User Interaction is tracked in a subtle manner based on tracking of the viewer's  head in 6 degrees of freedom (X, Y, Z, pitch, yaw, roll) by an OpenCV computer vision routine, connected to Unity3D game engine.

Projector front panel:

Marko projector front image

17 cam hd capture rig:

Marko Capture rig

USB scent engine microcontroller board:

Microcontroller USB scent engine

Lounger prototype by Esa Vesmanen of Puredesign:

Lounger prototype

Collaboration Partners: Marko Tandefelt: overall concept, 3D scanning rig, scent engine microcontroller, interaction development; Donnie Bugden: game interaction programming; designer Esa Vesmanen of Puredesign: lounger industrial design; artist Hilda Kozari: scent design; Antti Louhivaara, designer of Aurelia speakers: speaker design

Special thank yous to: Daniel LĂ©lis Baggio of visualblaster.com for his generosity of spirit and brilliant OpenCV/Unity3d 6 dof head tracking code examples

USB scent sticks kindly provided by Aroma USB/Debreu Ltd.

Unity3D Pro game engine kindly provided by Unity3D

DXG 595 video cameras, related custom firmware kindly provided by DXG USA

Initial stages of the project have been supported by Greta and William Lehtinen Foundation and AVEK The Promotion Centre for Audiovisual Culture Vieteri grant (Spring grant)

 

 

AVEK logo Greta and William Lehtinen Foundation logo

 

Unity3D logo Aroma USA logo DXG The Digital Camera Company logo

Project Created: 
January 2009