Some day, we'll see those old videos in hi-def

It's been interesting to see how TV shows from the 60s and 70s are being made available in HDTV formats. I've watched a few of Classic Star Trek, where they not only rescanned the old film at better resolution, but also created new computer graphics to replace the old 60s-era opticals. (Oddly, because the relative budget for these graphics is small, some of the graphics look a bit cheesy in a different way, even though much higher in technical quality.)

The earliest TV was shot live. My mother was a TV star in the 50s and 60s, but this was before videotape was cheap. Her shows all were done live, and the only recording was a Kinescope -- a film shot off the TV monitor. These kinneys are low quality and often blown out. The higher budget shows were all shot and edited on film, and can all be turned into HD. Then broadcast quality videotape got cheap enough that cheaper shows, and then even expensive shows began being shot on it. This period will be known in the future as a strange resolution "dark ages" when the quality of the recordings dropped. No doubt they will find today's HD recordings low-res as well, and many productions are now being shot on "4K" cameras which have about 8 megapixels.

But I predict the future holds a surprise for us. We can't do it yet, but I imagine software will arise that will be able to take old, low quality videos and turn them into some thing better. They will do this by actually modeling the scenes that were shot to create higher-resolution images and models of all the things which appear in the scene. In order to do this, it will be necessary that everything move. Either it has to move (as people do) or the camera must pan over it. In some cases having multiple camera views may help.

When an object moves against a video camera, it is possible to capture a static image of it in sub-pixel resolution. That's because the multiple frames can be combined to generate more information than is visible in any one frame. A video taken with a low-res camera that slowly pans over an object (in both dimensions) can produce a hi-res still. In addition, for most TV shows, a variety of production stills are also taken at high resolution, and from a variety of angles. They are taken for publicity, and also for continuity. If these exist, it makes the situation even easier. This will be easiest at first for the backgrounds in these low-resolution videos, particularly if the cameras moved freely over them. Once a hi-res image of the background is generated, it will not be hard to then re-map that image into a new video, adjusting for lighting changes and shadows in each given frame. The moving things, particularly people, will create a larger problem because they are 3-dimensional. While sub-pixel detail should be extractable, it may not be for every frame. For actors, a wide variety of still shots will also be available.

All of this can feed into the "synthespian" technology which is hoped to generate a complete computer model of a person which bridges the uncanny valley and looks realistic. If this can be done with an actor, then all that is required is to do motion capture on the low-res video and map it into the animated representation to produce a hi-res version. But it may be possible even before that, just from mapping higher resolution textures with the right shading onto a low-res original.

At first, this will be expensive, but in time systems might arise that could be used even on all of our old VHS camcorder tapes. At first, the results will be varied -- they might produce a movie that occasionally is blurry in certain areas. That might be jarring, but still might be preferred to the poor quality original.

Other possibilities might include not increasing resolution but at least increasing dynamic range. Most videotapes suffer from poor range, with highlights blown out or shadows black due to AGC on the brightness. Smarter software might notice if it has a suitably exposed image of a region that is blown out and fix this problem.

You might view this as akin to the aborted effort in the 1990s to re-colourize old B&W movies. This failed for a number of reasons. First, the colourizations didn't really look very good or realistic, something I believe we could change if it were tried today. Secondly most of those shows were designed to be shot in B&W and the colourized versions did not have the same artistic cinematography of the originals. Some felt it was even a violation of the original filmmaker's intent. That's less likely to be true here -- few filmmakers would deliberately shoot with low-resolution video, though it is certainly true that props, sets and makeup and other things were often designed to look good only at low-res. That's obvious already in some of the old filmed TV shows being re-released in HD. For example, some of the papier-mâché rocks in Star Trek, are really obviously papier-mâché in HD. Those sort of things can be fixed with CGI if there is a budget.

None of this will be trivial. And in some cases, the information will have been truly lost due to various circumstances and could only be re-invented rather than restored. But in many cases it's clear that videos from the past hold within them far more information than you can see just watching them, and software of the future will be able to re-extract it.

Already some tools are available (though not generally at the consumer level) to recover stuff from old tapes. Some things, like removing dropouts and jitter, are well within today's technology. Having better, more accurate heads to read the video could help a lot with that too, for those who care about old VHS. Decaying oxide is a big problem, you may not get more than one chance to read a tape.

Comments

http://en.wikipedia.org/wiki/Looker

Also:

The unfortunate story of Babylon 5, where the live action footage was shot at 16x9 on super 35mm film, but to save money the CGI was done at 4x3 (on video, I think). The idea was that when they were ready to do hi def releases they'd go back to the original CGI models and re-render them at a suitably high quality. Unfortunately some or most of the original CGI data has been lost as both Foundation Imaging and its successor, Netter Digital, have gone out of business. So while the live action footage is hi def-ready, the CGI would have to be done over from scratch. Of course there's always the chance that some past employee kept some of the CGI data and it's sitting on a forgotten hard drive somewhere (or more likely backup tape, since hard drives were much more expensive when Babylon 5 was first produced).

Looker was not a great movie, neither was SimOne, another film about a synthetic actress. We've seen more advanced tools today which are not quite ready (like the young Jeff Bridges in TRON:Legacy) but it's coming along, and all I point out is that before long this will be doable and cheap and allow these enhancements.

I am not worried about B5. As noted, they re-did Star Trek and just re-did the graphics from scratch. Graphics today are vastly cheaper than they were, though human time still costs the same which would have made it nice to recover that old code. Today we have single individuals making CGI films with graphics as good as the best films of 25 years ago, though there is still artistry and painstaking human labour which costs money.

Not denying it can, and will, someday, be done.

Just pointing out a "prior art" reference, as well as how B5 had HOPED to "future proof" (for values of "future" <= ~25 years) their production, but didn't spend enough effort on future proofing the data.

Star Trek does have two advantages over B5 in regards to getting their effects redone: 1) A lot fewer, and simpler effects shots, and 2) A much larger audience likely to buy the remastered product.

I intended to mention how in high school in the 80's I was talking to other friends who were interested in film production about how actors at some point were likely to be replaced by digital characters. I expected we'd have convincing digital "live actor" replacements by now, but I didn't count on (and wasn't aware of) the uncanny valley that was a big bump in the road along the way.

And I think of things like The Lawnmower Man, which I only finally saw just last summer. Woof.

So, cryonics for movies. I like it!

"We can rebuild it. We have the technology."

A fine reference to a show that deserves an update, gwern. With sufficiently advanced software, not only could one improve the appearance of the video, but perhaps one could improve the quality of the acting.

Add new comment