VSE Compositor Node proxy

Hi everyone.
I was reading some discussion about the compositor, and how it would be interesting to integrate the VSE to become a more in-place feature, rather than compositing all pre assembled footage.

This script is designed to read the VSE strips, and create handled nodes which will expose useful information to the UI when compositing. The idea is that the VSE is not actually used for rendering, and is only used as a GUI.

This script is very bare-bones. Currently, I’ve implemented Image and Movie nodes, and only the cross effect. I did have a different form of effect node which simply exposed a normalised factor for the effect, but I decided that the purpose of this script is to reduce the need to create your own node graphs manually. You can still read the factor value of the effect node output.

The script creates a new node group type for each input sequence strip. The reason for this is it is not possible to instantiate node groups, so any data you modify on their node tree affects all the instances. I could solve this by exposing all the data to input sockets, but that looks unclean, and this is merely a P.O.C.



(Movie node)


(User-arranged node tree)

If an effect is applied to two strips, it can connect them to the effect, but if two input strips overlap, you will need to fulfil that connection yourself.This would be somewhat straightforward to implement, but I won’t do it for now.


(Cross Effect node)

Node sockets:
Each node will have a number of common pins:

  • Image - output image of node (effect, movie, image …)
  • Current frame - current VSE frame. All nodes expose it in case it is needed for custom effects
  • Start Frame - start frame (global frame in VSE, accounting for trim) of clip
  • End Frame - end frame (global frame in VSE, accounting for trim) of clip
  • Layer In - Image to use when before the start frame of this node
  • Layer out - Image to use when after the end frame of this node

Effect nodes also expose:

  • Fac - linear interpolation between start and end frame according to current frame. Used for fading, other linear effects.
  • Image Upper - Higher channel Image
  • Image Lower - Lower channel Image

Be aware - there is a limitation, that there can be only one user of any input data blocks. This limitation comes from the Movie input only, but I have extended that to the image to keep things consistent.

Don’t manually delete the nodes, they’re created from the VSE, and the system doesn’t check if they still exist, deleting them will cause problems. Instead, delete the VSE sequence.

When a VSE clip is removed, the associated data block will be deleted (user_clear() and then deleted)
To update the nodes, simply change the current frame.

Script:
http://www.pasteall.org/51941/python

One step closer to Flame in Blender. :).

This’s a good direction. Thanks!

Hey fella,

Looks really quite cool, I gave it a quick blast and it seems to work, though running it more than once seems to confuse it a bit (im using 2.70a release). It also doesn’t seem to update when the sequencer edit changes but I’m not sure if this is outside the scope of the current script or not?

As you say it’s only a Proof of Concept right now but really shows that this could work very well indeed - are you going to put a link to this on the compositor mailing list or shall I?

cheers,

Christian

Once it has been run once, it doesn’t need running again until you close blender or open another file. I should make it so that it attempts to recover existing node setups. I’m not sure about linking to it yet, I feel it should be more functional first, but feel free :slight_smile:

Currently there is no way of detecting when the sequencer updates, unless I overloaded the operators some How. Instead just changing the time-line frame will perform an update, so if you move, add or delete a clip it will adjust the nodes.

Does the button in the VSE timeline called “refresh sequencer” force an update?

That’s a really nice addon, and it works very smoothly as I can see from a quick test.
Once completed it would certainly speed up tremendously the workflow while compositing
Just a couple of comment about stuff that agoose77 may have already thought about and would be very nice to have:

  1. offset isn’t taken into account: if a sequence starts from a value different from 1 you can’t see the picture where expected according to the position of strip in the timeline
  2. extrapolation of the sequence: you could take into account the behavior of the strip before and after the first and last frame with something like old first/last frame, loop or ping pong taking into account the extra length of the strip if you move the end over the last frame
  3. trim point of the sequence: at the moment if a sequence is shortened at the start or at the end this isn’t reflected on the node

Again, your add on is tremendously useful hope you would move this proof of concept to a complete tool!
We definitely have to rise the attention to your addon in the bf-compositor ML too!

Currently, it does not (for my script, it does for the actual VSE), but changing the VSE frame does. I don’t think I can access anything in that aspect of the BPY UI

Some more information on how layering is performed:


In this image, the nodes are arranged roughly matching the VSE timeline. This helps visualise what is going on. The Layer In socket is used for strips that play before this strip, and the Layer Out for those that play after wards. Even if these inputs are still playing whilst the strip in question is playing, they will be ignored as it takes precedence.

The top node has playback priority, hence we composite with the output of that node. The topmost priority node always renders the output. Priority is the same as the channel, higher channel number means greater priority.

If two strips have the same highest priority, then simply use the Layer In/Out system to stack them



(Before stacking)


(After stacking)
In this image we also include the Purple Image Sequence input into the layer out, so that when the second cross effect is completed, it continues to play the image sequence

Here is a simple cross effect



In this first image, we simple ask the node to cross between the left node and the right node


Here we also instruct the node to play the initial clip before the fade occurs, and the final clip when the fade is completed.

Can someone help me? The scrpit almost works, but there is a problem when I want to render animation. When I’m setting up nodes I can see in viewer node, that it’s changes the movie clips from VSE, but when I click button “RENDER” or “ANIMATION” it renders pure frames from VSE, without composing ; ( Blender 2.72b

Go to Render settings, Post Processing painel and uncheck “Sequencer” (if it’s checked).