How to Display Time as 3rd Dimension in an Image Sequence?

Hi all,
I have an idea that as been baking in my head for some time, but I don’t quite know how to accomplish it.
Let’s say we have a movie or image sequence…
I would like to access different frames of the movie clip at different areas of the display.
For example, what would it look like if the movie is played at normal speed at the left and more slowly as you look further right?
Or, what if I want to create a static image that shows the progression of the whole movie over time as one looks further down the image? So the horizontal axis would be as it is in the original movie clip, but the vertical axis would be the original vertical axis as it progresses in time. Let me know if that makes any sense.

One way I considered accomplishing this was to make an object, say a plane, that is UV-mapped (or Global XY-mapped) to the image sequence texture, then the geometry’s Z is mapped to time (i.e. the frame number). So when the plane is animated to move straight up, it essentially plays the movie as it normally would, to an orthographic camera pointing down. But! If the plane is rotated or its individual vertices change in Z, the camera sees parts of the movie later in time and parts earlier in time.
However, as far as I know, only one frame can be accessed at a given time in Blender. I would have expected to be able to connect a value to a “frame” node in the Image Texture block, in the Materials node editor, but there is no frame node, only a frame number selector. I would connect the Geometry’s position in Z to the Frame node.
Maybe this would be possible via Python scripts? Or does Blender itself need to be modified? Maybe there’s some existing tool that I am unaware of?

If you got confused by this post, I’m right there with you. Please let me know what I can help clarify. But, if you were able to follow it, I would love to hear your comments on how this could be accomplished!

Thanks!

Spacetime! That’s the word I was looking for!
vpaint looks like a very useful tool for 2D animation! I downloaded it.
Yeah the visualization looks a lot like what I was thinking about. Imagine if, in that 3D spacetime visualization, you rotated the “current frame” plane or re-shaped it so parts of it would be more in the “future” in spacetime than others.

It would be cool to see a transparent curved mesh of an animation/movie like it shows in vpaint, too!
Ooh, there are ways of vectorizing images, right?

Thanks for your input!