Using Photographs To Enhance Videos

seussman71 writes with a link to some very interesting research out of the University of Washington that employs "a method of using high quality photographs to enhance a video taken of the same subject. The project page gives a good overview of what they are doing and the video on the page gives some really nice examples of how their technology works. Hopefully someone can take the technology and run with it, but one thing's for sure: this could make amateur video-making look even better than it does now." And if adding mustaches would improve your opinion of the people in amateur videos, check out the unwrap-mosaics technique from Microsoft Research.

3D models from videos

[4D6963]

The other cool part of it is that it derives a cloud of points from the video, meaning it can turn a video into a 3D model, apparently. However it seems like their program only uses it internally.

Of the two, I find the Microsoft one to be better

[spoco2]

Really, the ability for their software to 'unwrap' a 3D object and allow you to fiddle with it as you wish is very cool.

And not limited to a 'static' scene.

And, really, if you're going to go to the effort of videoing a scene, then photographing the scene, then passing the video and the photos through their software. All to get better exposure and resolution.

Um.

Wouldn't it be a far better cost/effort equation to just buy a better HD camera in the damn first place?

Re:Of the two, I find the Microsoft one to be bett

[cnettel]

The more interesting aspect is that you can tweak those still photos, and then transfer them back. Photoshop some key frames, and you have suddenly created a video with the same manipulation. The video is just a cheap source for spatial data, which you can then texture with your photos.

A better use?

[neokushan]

All of these techniques are pretty awesome and will certainly be a boon to home video enthusiasts the world over (plus plenty of commercial places that are on a tight budget), but I've got another idea.

You see it on TV all the time, CCTV footage of robberies and the like, couldn't this technology be used to effectively map out a 3D image of the purpetrator?
I know it wont be perfect and most CCTV is probably too low quality to be used, but it would certainly be pretty cool (and useful) to have a vaugely accurate 3D model of the guy, giving you height, build, etc. and with the help of supplementary images, a really easy way to adjust it's appearance.

CREEPY!

[Ohrion]

Did anyone else notice a very creepy effect in the "enhanced" video with the bust? It made it look like the head was turning to look at you as you moved around it. *shudder*

Re:A better use?

[Pingmaster]

I would say that mounting a high res still camera in parallel with the CCTV camera and taking, say 1 picture every 10 seconds after the CCTV montion sensors are tripped, which would have quality comparable to a high-end consumer camera (i.e. 7-8 Mpixels), then use that data to enhance the video taken to aid in identifying suspects

That said, I don't think these 'enhanced' videos should be admissible as evidence, since the videos have been effectively tampered with and given the possibility of altering identifying features of a suspect using superimposition of a different picture on the video could either cause the wrong person to be jailed, or the actual criminal set free

Dealing with copyright problems

[DouglasR]

This might be helpful to deal with copyright-protected material that gets into frame, for instance, billboards, logos on T-shirts, posters and art-work on walls. Take a single frame into a photo-editor, replace the unauthorised image with an authorised one, and this technique could potentially replace it throughout the sequence. It could equally be used to replace moving images, for instance on a TV screen, with a "blue screen" (or green), that normal video compositing software can then replace with a desired image.