Point-E, an AI that creates 3D models, is made available via OpenAI
3D model generators may be the next innovation to rock the field of AI. Point-E, a machine learning system that generates a 3D object from a text prompt, was made available to the public this week by OpenAI. A article that was published along with the code base claims that Point-E can create 3D models on a single Nvidia V100 GPU in one to two minutes.
In the conventional sense, Point-E does not produce 3D objects. Instead, it creates point clouds, which are discrete collections of data points in space that reflect 3D shapes; hence, the playful abbreviation. (The "E" in Point-E stands for "efficiency," as it purports to be quicker than earlier 3D object production techniques.) From a computational perspective, point clouds are simpler to create, but they are currently a major drawback of Point-E because they cannot capture an object's fine-grained shape or texture.
The Point-E team trained an additional AI system to transform Point-point E's clouds to meshes in order to get around this restriction. In 3D modeling and design, meshes—collections of vertices, edges, and faces—are frequently used to define objects. However, they make a point in the report that the model occasionally misses specific item details, resulting in blocky or deformed shapes.
Point-E is made up of two models: a text-to-image model and an image-to-3D model, in addition to the mesh-generating model, which is a standalone model. The text-to-image model was trained on tagged images to comprehend the relationships between words and visual concepts, much like generative art systems like OpenAI's own DALL-E 2 and Stable Diffusion. The image-to-3D model, on the other hand, was taught to effectively translate between the two by being fed a set of photographs coupled with 3D objects.
Point-text-to-image E's model creates a synthetic rendered item from a text prompt, such as "a 3D printed gear, a single gear, 3 inches in diameter and half inch thick," and feeds it to the image-to-3D model, which creates a point cloud.
Point-E could generate colored point clouds that commonly matched word prompts after training the models on a dataset of "several million" 3D objects and related metadata, according to the OpenAI researchers. It's not flawless; occasionally Point-image-to-3D E's model cannot interpret the image from the text-to-image model, leading to a shape that does not correspond to the text prompt. Even so, the OpenAI team claims that it is orders of magnitude faster than the prior state-of-the-art.
They said in the report, "While our method produces samples in a small fraction of the time, it performs worse on this evaluation than state-of-the-art techniques. This might make it more useful for particular applications or might open the door to the discovery of 3D objects of greater quality.
What exactly are the applications? The point clouds created by Point-E, however, might be utilized to create actual objects, for instance through 3D printing, according to the OpenAI researchers. Once it's a little more refined, the system might also find use in processes for game and animation production thanks to the addition of the mesh-converting model.
Although it may be the most recent business to enter the 3D object generation market, OpenAI is by no means the first, as was previously said. A more developed version of Dream Fields, a generative 3D technology that Google revealed back in 2021, was released earlier this year under the name DreamFusion. DreamFusion, in contrast to Dream Fields, doesn't need any prior training, therefore it can create 3D models of objects without 3D data.
While 2D art generators are currently the focus of attention, model-synthesizing AI has the potential to be the next major industrial disruptor. 3D models are frequently utilized in the domains of science, interior design, architecture, film, and television. Engineers utilize models as designs for new equipment, vehicles, and structures, while architectural firms use them to demonstrate proposed buildings and landscapes.
However, creating 3D models typically takes a while - anywhere from a few hours to a few days. If the bugs are ever ironed out, AI like Point-E might alter that and bring OpenAI a respectable profit.
What kinds of intellectual property issues might eventually occur is the question. There is a sizable market for 3D models, and artists can sell their original work on a number of online marketplaces, such as CGStudio and CreativeMarket. Model artists may object if Point-E is successful and its models are released on the market, citing evidence that contemporary generative AI heavily draws from its training data, in this example, existing 3D models. Point-E, like DALL-E 2, does not mention or give credit to any of the artists who might have had an impact on its generations.
But OpenAI will save that topic for another day. The GitHub page and the Point-E document both make no mention of copyright.
To their credit, the researchers do acknowledge that they anticipate Point-E to have other issues, such as biases inherited from the training data and a lack of protections for models that might be exploited to build "hazardous objects." It's possible that this is the reason they are careful to describe Point-E as a "beginning point" that they hope would motivate "additional study" in the area of text-to-3D synthesis.
My3D.Cloud - a platform for Architects and Contractors who uses 3D laser scanning and 3D modelling in their work
- Storage
- Viewers 3D files (Point Cloud | CAD | MESH)
- Converting 3D files(Point Cloud | CAD | MESH)
- Sharing
- Teamwork
- Mobile
We have created this platform to improve project delivery and collaboration experience. Thanks to it, users get the opportunity to increase the loyalty of their customers, and in turn, customers get time savings and maximum convenience.