spatial/temporal (3D) blender stabilizer - Proof of Concept

Hey,
There have been recently a thread about Microsoft Research’s attempt to stabilize GoPro timelapses, and that triggered me ideas on how to get a somewhat similar result in Blender.

The workflow would be the following:

  • Motion track the footage, and get a decent camera path
  • Create a plane that takes exactly the camera view cone
  • Parent the plane to the camera
  • Apply motion, use Frame Duplication to get all the different frames positions
  • Assign them the correct frame as the texture
  • Low-pass (or rework) the camera transform
  • Bring the “correct” plane for each frame so that the movie still resembles to the original
  • Render
    Have a stabilized video !

Now this is ultra-theoric, but technically you can do it right now on Blender, it might be better to make it a script so that you can process 2000+ frames a little more quickly :slight_smile:

I don’t know how good it would be, but I guess this is a hacky version of a 3D stabilizer !

Hi,
So I’ve been trying to not drown in the API, and came up with a rusty proof of concept.
With the following script I can use the camera path to put planes on its field of view, with each plane having its corresponding frame.

Now of course more work is needed, and I’m still struggling, but as far as placing frames in a 3D world, it is working :wink:

All you need is a camera with a path (otherwise all the frames will be at one place), and an image sequence (yes, because I’m working with Voodoo so image sequence is what I needed first). It can be any length long.
Start and End of animation/processing is controlled via the Start/End frames of the Timeline.
Name of image sequence and offset is controllable from the script. Run it to get an eventual result.

Link: https://dl.dropboxusercontent.com/u/98670603/FramesTo3Dscene.blend

Oh, I wondered this a while ago too. that is, could you paint a scene from the camera’s POV? That is a sort of UV projection onto a large field plane, in your case onto many planes with frozen images.

Also I wondered if you could projection paint an object in this way?

Unwrap using 3D view while on Camera view, it’s a hacky way of painting an image onto an object using Camera coordinates. I have no idea how to animate the UV map though.