Using Photographs To Enhance Videos

seussman71 writes with a link to some very interesting research out of the University of Washington that employs "a method of using high quality photographs to enhance a video taken of the same subject. The project page gives a good overview of what they are doing and the video on the page gives some really nice examples of how their technology works. Hopefully someone can take the technology and run with it, but one thing's for sure: this could make amateur video-making look even better than it does now." And if adding mustaches would improve your opinion of the people in amateur videos, check out the unwrap-mosaics technique from Microsoft Research.

Is anyone else sick of demos?

[QuantumG]

Why is UW not releasing their source code? If they intend to spin off commercial products, why are they releasing demos? Hell, even *Microsoft* is releasing demos of this stuff.. is Apple and Google the only companies that can ship product these days (even if it is "beta" you can at least freakin' use it).

No more demos. We know you're smart, now make something useful please.

Patent Encumbered?

[reality-bytes]

I always get this feeling when I see a university-styled promotional release that the *software* patents are already pending.

I haven't the time to search just now but I'll bet there's at least one application pertaining to this method which encompasses a hell of a lot more.

Re:Is anyone else sick of demos?

[imsabbel]

Simple reason:
They _say_ that it does this automatically.

Translation: We put some phd student on it who spend some months optimizing the settings for the 2 selected scenes so we can make a nice publication and maybe get more money.

If you just look at their steps of the workflow, the way they discribe it just isnt possible (like the way they "stereoscopically" create a depth-map from a _single_ still photograph..).
Not to mention that the first scene looks like a bad video game level after their "improvement".

Find Video Frame - Apply

[KalvinB]

The easy way would be to use the already calculated depth field from the frame in the video that best matches the photo.

It's going to be big...

[bill_kress]

When the ability to deconstruct a video into a 3-d model & skin (the opposite of what a video card does now) is placed into an open-source API, the possibilities are going to be HUGE (and a little frightening).

Anyone want to post a few ideas? I'll give you a few topics to kick things off:

Change detection (Finding lost objects in a room, seeing boxes left in a government office, where's my remote)
Change observation (plant growth, things that change too gradually for us to notice)
Creating 3-d models from humans (extracted from old films, walking down the street)
Weapon systems (Undetectable lasers blinding targets, Unmanned guns with perfect accuracy)
Home interaction (Make a sign with your hand, computer changes the channel, lighting, heat, ...)
Office monitoring (Exactly where each person is any time just by typing "Where's bill" into your PC)

All things that could be done by any hobbiest/hacker with the right API.

(I assume that to get real-time you could use the massively parallel abilities of a video card, making this stuff run on any hardware...)

Also, just storing models and skins is extremely efficient--You could film a room for years in extremely high resolution and use virtually no storage (almost none except when something or someone new enters the room, then just one new high-def skin)

Other ideas?

Well there goes using video as evidence in court

[Brynath]

With most if not all video cameras storing the video digitally, and now with all these new techniques for editing video, why would any court allow for video evidence?

evidence fabrication?

[Atilla]

This software, if it actually works as described, could also be used to easily fabricate video "evidence". An average viewer would not be able to tell the difference.

Kinda scary...

Re:Of the two, I find the Microsoft one to be bett

[jebrew]

I'd just connect a camera to the bottom of my camcorder (they both have a spot for mounting).

Then just have the still camera do continuous shooting @ ~1fps while you video. Match them up in this software when you're done and you're good to go...now if I could just get a hold of their software.

Re:Of the two, I find the Microsoft one to be bett

[imess]

Wouldn't it be a far better cost/effort equation to just buy a better HD camera in the damn first place?

Hint: years old amateur/family/etc video meets modern high-res camera.

Re:Of the two, I find the Microsoft one to be bett

[dword]

if you're going to go to the effort of videoing a scene

"you..videoing..." isn't the only application. This could be used to enhance other videos. Let's say someone else made a great video (captured some really great scenes, focused on some details) and you want to publish it but even if they focused on cool details, they're not enough. You take a few pictures and enhance their video.
Also, this is just the start. They are currently enhancing static videos but I'm sure in the near future, if this is worked on enough, it could be used to enhance any kind of video scene. So you have something interesting happening - by some people's standards, two squirrels fighting over a nut is interesting enough - but the overall quality of your video is just awful. You won't be able to re-take the shot because the squirrels canceled their contract so you'll take some pictures, match them against the video and voila, high-quality video or a forest and two squirrels kicking each other in the nuts over a nut while filmed by a nut.
Do you by any chance remember those huge radios? I mean those REALLY huge radios weighting about 50kg? They weren't very practical and to the final consumer they were cool but they were heavy and incredibly expensive. Now I carry an MP3 player in my pocket that also has an FM radio integrated, just for the hell of it. === POOF === 20 years later === Do you by any chance remember those projects that they started, to enhance videos of static scenes using photographs? There was an article on a site named "Slashdot" which was taken down after it started WW3... I doubt that you'd remember that, but look where we are today: with a couple of high-quality pictures (100 gpixel ;) you can enhance any video.
This is what this whole project is about. Studying something cool and then enhancing it until it gains practical applications. Why the hell won't /. users stop bitching about "this isn't very useful" and "i don't see the point"? It's not useful now, but it will be, otherwise nobody would invest in it and I think people who pay tons of money for this kind of research know a bit more about what research is good for than you but unfortunately they're too busy making money and changing the world to spend their time on /. The fact that you don't see it's point means only that you don't see it, it doesn't mean there is no point. In stead of saying "this isn't useful" why don't you ask "what could this be used for?" Maybe that change in some people would help us progress faster because they will question the applications of certain research which causes debates which lead the faster progress (not at the time of the debates but a couple of years later people draw conclusions and they start to get along and pretend they never asked dumb questions). It would also encourage researches by showing them that if they give you applications for their work you might embrace it. But NOOOOOOOOOOOOOOOO you just say "i can't do anything with this" which basically insults and shuts the trap of anyone who might give you a couple of uses and it just starts a flame war which is basically a debate focused on insults and swear words in stead of what it should be - focused on pros and cons.