OpenAI Releases Point-E, an AI For 3D Modeling (engadget.com) 12
OpenAI, the Elon Musk-founded artificial intelligence startup behind popular DALL-E text-to-image generator, announced (PDF) on Tuesday the release of its newest picture-making machine POINT-E, which can produce 3D point clouds directly from text prompts. Engadget reports: Whereas existing systems like Google's DreamFusion typically require multiple hours -- and GPUs to generate their images, Point-E only needs one GPU and a minute or two. Point-E, unlike similar systems, "leverages a large corpus of (text, image) pairs, allowing it to follow diverse and complex prompts, while our image-to-3D model is trained on a smaller dataset of (image, 3D) pairs," the OpenAI research team led by Alex Nichol wrote in Point-E: A System for Generating 3D Point Clouds from Complex Prompts, published last week. "To produce a 3D object from a text prompt, we first sample an image using the text-to-image model, and then sample a 3D object conditioned on the sampled image. Both of these steps can be performed in a number of seconds, and do not require expensive optimization procedures."
If you were to input a text prompt, say, "A cat eating a burrito," Point-E will first generate a synthetic view 3D rendering of said burrito-eating cat. It will then run that generated image through a series of diffusion models to create the 3D, RGB point cloud of the initial image -- first producing a coarse 1,024-point cloud model, then a finer 4,096-point. "In practice, we assume that the image contains the relevant information from the text, and do not explicitly condition the point clouds on the text," the research team points out. These diffusion models were each trained on "millions" of 3d models, all converted into a standardized format. "While our method performs worse on this evaluation than state-of-the-art techniques," the team concedes, "it produces samples in a small fraction of the time." OpenAI has posted the projects open-source code on Github.
If you were to input a text prompt, say, "A cat eating a burrito," Point-E will first generate a synthetic view 3D rendering of said burrito-eating cat. It will then run that generated image through a series of diffusion models to create the 3D, RGB point cloud of the initial image -- first producing a coarse 1,024-point cloud model, then a finer 4,096-point. "In practice, we assume that the image contains the relevant information from the text, and do not explicitly condition the point clouds on the text," the research team points out. These diffusion models were each trained on "millions" of 3d models, all converted into a standardized format. "While our method performs worse on this evaluation than state-of-the-art techniques," the team concedes, "it produces samples in a small fraction of the time." OpenAI has posted the projects open-source code on Github.
Well. Now we can have 3d body horror! (Score:2)
Cause the things coming out of stable diffusion are often one step from nightmares.
Re:Well. Now we can have 3d body horror! (Score:4, Interesting)
It would be very handy if you could give it a photo of a broken part, and it gave you an optimised 3D printable model back.
Re:Well. Now we can have 3d body horror! (Score:4, Interesting)
Provided someone already created the model of the broken part, and made the model available for this program to sample, it's entirely possible that this program could find the exact model you are looking for.
I suppose that if you spent many hours refining your criteria, it could generate something very close to what you're looking for. The description would have to be very, very detailed and specific, so specialized software would have to be written to allow for that kind of minute detail to be specified.
Eventually someone will create such software and give it a cool name; something like (just off the top of my head) "Blender". The name will have special meaning to its creator, and sound rather random to everyone else, but it will probably happen.
Re: (Score:2)
I just want to point my phone camera at it, and get a printable model out.
You can pay someone on Fivr or similar service to do it today. Now I remembered that I'm suddenly a lot less interested in the AI solution.
More dubious Elon Musk fawning (Score:3)
Re: (Score:1)
and resigned from the company after a few years.
So, if he's not part of the company any longer, how is the recent Open AI hype equating to "Elon Musk fawning" in your mind?
Re:More dubious Elon Musk fawning (Score:5, Insightful)
I dunno about fawning, but the inclusion of his name is odd every time OpenAI gets brought up, considering he resigned from the board 4 years ago and doesn't have anything to do with the company's decisions or operations anymore. Microsoft is by far the largest single funder, contributing a billion dollars on their own.
Re: (Score:2)
Re: (Score:3)
Was he an actual co-founder, or is this another case of "I'll invest a bunch of money but you first have to sign this contract agreeing to refer to me as a founder"?