New AI Model Turns Photos Into Explorable 3D Worlds, With Caveats 18
An anonymous reader quotes a report from Ars Technica: On Tuesday, Tencent released HunyuanWorld-Voyager, a new open-weights AI model that generates 3D-consistent video sequences from a single image, allowing users to pilot a camera path to "explore" virtual scenes. The model simultaneously generates RGB video and depth information to enable direct 3D reconstruction without the need for traditional modeling techniques. However, it won't be replacing video games anytime soon.
The results aren't true 3D models, but they achieve a similar effect: The AI tool generates 2D video frames that maintain spatial consistency as if a camera were moving through a real 3D space. Each generation produces just 49 frames -- roughly two seconds of video -- though multiple clips can be chained together for sequences lasting "several minutes," according to Tencent. Objects stay in the same relative positions when the camera moves around them, and the perspective changes correctly as you would expect in a real 3D environment. While the output is video with depth maps rather than true 3D models, this information can be converted into 3D point clouds for reconstruction purposes. There are some caveats with the tool. It doesn't generate true 3D models (only 2D frames with depth maps) and each run produces just two seconds of footage, with errors compounding during longer or complex camera motions like full 360-degree rotations. Furthermore, because it relies heavily on training data patterns, its ability to generalize is limited and it demands enormous GPU power (60-80GB of memory) to run effectively. On top of that, licensing restricts use in the EU, UK, and South Korea, with large-scale deployments requiring special agreements.
Tencent published the model weights on Hugging Face.
The results aren't true 3D models, but they achieve a similar effect: The AI tool generates 2D video frames that maintain spatial consistency as if a camera were moving through a real 3D space. Each generation produces just 49 frames -- roughly two seconds of video -- though multiple clips can be chained together for sequences lasting "several minutes," according to Tencent. Objects stay in the same relative positions when the camera moves around them, and the perspective changes correctly as you would expect in a real 3D environment. While the output is video with depth maps rather than true 3D models, this information can be converted into 3D point clouds for reconstruction purposes. There are some caveats with the tool. It doesn't generate true 3D models (only 2D frames with depth maps) and each run produces just two seconds of footage, with errors compounding during longer or complex camera motions like full 360-degree rotations. Furthermore, because it relies heavily on training data patterns, its ability to generalize is limited and it demands enormous GPU power (60-80GB of memory) to run effectively. On top of that, licensing restricts use in the EU, UK, and South Korea, with large-scale deployments requiring special agreements.
Tencent published the model weights on Hugging Face.
mmm .. (Score:2)
... can it duplicate them too?
Only the generous survive (Score:2)
Philosophically, this
Re: (Score:2)
-Tesla's open patents accelerated EV infrastructure and cemented its leadership.
Who used Tesla's "open" (in exchange for agreeing not to sue them ever, which is really just a licensing scheme) patents?
Re: (Score:2)
Their connector has been widely adopted and standardized, if not any patents.
OK, so the answer is "nobody".
Do you agree with the broader point besides this nitpick?
I agree with openness promoting uptake and being good in general. However, the comment was also potentially misleading. Besides Tesla's "open" patents (which were a scam) there's also that IBM did not intend the PC to be as open as it was. It was the fashion of the day to publish schematics and documentation for personal computers. IBM did not mean for the BIOS to be cloned, they fought tooth and nail against it. If Compaq didn't fought their way through, the PC would have rema
Re: (Score:2)
Have a look at Nvidia's models. Many of them have a license that prohibits you from using them on non-Nvidia hardware. I wonder if it would really be enforceable.
Hardware and time (Score:2, Interesting)
The only real innovation is somehow getting 100,000 video trained on hardware powerful and large enough to hold all this both on disk and in memory. Its a 'me too' or a 'Chinese innovation' to compete with Google Genie 3, which is far better from what I can see.
Whatever your feelings are about AI today, in the year 2025, where AI is both useful and stupid as shit, you should understand that the real 'battle' is being fought over resources to run it, both in terms of silicon and electricity. AI (prob
Re: (Score:2)
You're forgetting quantization. Starting with just 8 GB of VRAM you can run very useful "AI assistant" LLMs. Similar video models like this were also quantized to run on less RAM than the full weights. If I had to guess, I'd say people get it down to 32 GB VRAM to run it on a 5090 and possibly someone gets a smaller version to the 24 GB of the 30/4090.
The "Voyager" part on the hugging face repo (so without the VAE and text encoder) has a 30 GB filesize, most of the other required memory is probably required
Just what I always wanted! (Score:2)
Some days already seem like "only 2D frames with depth maps", and this is shaping up to be one of those days. Sigh...
Vore (Score:4, Funny)
Man the things we'll be able to do with porn. Wanking will never be the same.
Re: (Score:2)
The point is not to generate models. They point is to generate views.
I should move my midget porn collection elsewhere (Score:2)
One does not want an explorable 3D world based on such imagery.
Finally AI makes Red Dwarf possible (Score:2)
I can't wait to enhance my photos in this way. https://youtu.be/EMBkWtDAPBY?s... [youtu.be]