For the past year I’ve been developing an application called NextStage that turns the Kinect into a realtime virtual production camera.
By tracking retroreflective markers NextStage can instantly find the Kinect’s position and rotation. NextStage can also separate actors and objects from the background without the need for greenscreen, record uncompressed RGBA video, export data to sync an external camera, or stream the tracking data out to other applications.
There are two versions. NextStage Lite is available for free, and NextStage Pro can export tracking data as a collada .dae file. NextStage Pro also comes with a built in preset so that the tracking data can be easily imported into Blender.
For the beta release NextStage only supports .obj files as sets, and lighting effects need to be pre-baked. All of the demo Virtual Sets that ship with NextStage were pre-baked using Cycles (Suzanne the monkey makes a cameo in the intro video).
I’ve been developing this application pretty much in a vacuum, but I’m very excited to finally get it out into the world. Please let me know if you have any questions, comments or concerns.
This is really awesome! Very useful for young filmmakers, enthusiasts and even professionals! I`ll be checking this out as soon as I get a Kinect 2. Thanks!
That is correct. You might be able to get away with it in the shade or on a cloudy day, but direct sunlight won’t work. It’s primarily meant for indoor/studio use.
What’s the maximum distance that you can use this on an indoor scenario? On my tests Kinect 2 could only reach around 5 meters as maximum for the depth information, is this the same for you?
That sounds about right for the Kinect. As long as there is depth data and the Kinect can see at least 3 markers, it will be able to track. For longer distances the markers may have to be a bit larger (1.5"-2" instead of 1").
For now it will only work with the second generation Kinect (Kinect V2 and Kinect for Xbox One). There are qualities of the infrared camera (global shutter, short exposure, flood fill infrared) that make this technique possible and no other sensor currently has.
I’m actively investigating other technologies and include those as they become available.
