Interactive Digital Cinema Project

Immersive Interactive Multisensory Cinema Capture and Playback system studies novel aesthetic, expressive and narrative possibilities enabled by creative misuses of hardware and software technologies and computer vision methods. The project utilizes appropriated, hacked and modified hardware, microcontrollers, multiple 3d and video softwares using a custom workflow, and a minimalist viewing furniture based on a customized version of finnish industrial designer Esa Vesmanen's lounger design.

Core system components are: A) A 17 1920x1080 progressive HD cam 3D capture rig, B) Software based capture workflow consisting of 3D surface reconstruction/image based rendering, batch processing, 3D file formatting and importing to Unity3D Pro realtime game engine, C) A custom selfbuilt 1920x1080 progressive LED backlit stereoscopic 3D DLP front projector built from hijacked Samsung/Texas Instruments components, D) A custom lounger chair modified together with designer Esa Vesmanen based on his lounger chair designs (www.puredesign.fi). The chair consists of a selfmade USB scent stick engine microcontroller board, scent designs by artist Hilda Kozari, Clark Synthesis Platinum transducer, novel phase accurate speakers for speaker based crosstalk cancelled binaural 3D audio. User Interaction is tracked in a subtle manner based on tracking of the viewer's  head in 6 degrees of freedom (X, Y, Z, pitch, yaw, roll) by an OpenCV computer vision routine, connected to Unity3D game engine.

Projector front panel:

17 cam hd capture rig:

USB scent engine microcontroller board:

Lounger prototype by Esa Vesmanen of Puredesign:

Collaboration Partners: Marko Tandefelt: overall concept, 3D scanning rig, scent engine microcontroller, interaction development; Donnie Bugden: game interaction programming; designer Esa Vesmanen of Puredesign: lounger industrial design; artist Hilda Kozari: scent design; Antti Louhivaara, designer of Aurelia speakers: speaker design

Special thank yous to: Daniel Lélis Baggio of visualblaster.com for his generosity of spirit and brilliant OpenCV/Unity3d 6 dof head tracking code examples

USB scent sticks kindly provided by Aroma USB/Debreu Ltd.

Unity3D Pro game engine kindly provided by Unity3D

DXG 595 video cameras, related custom firmware kindly provided by DXG USA

Initial stages of the project have been supported by Greta and William Lehtinen Foundation and AVEK The Promotion Centre for Audiovisual Culture Vieteri grant (Spring grant)