How To Use AI to Convert a Photo to a 3D Model
Is there anything that AI can’t do? Well, fixing me a drink or doing the dishes is still a way off, but in the meantime, a new wave of AI tools can take a single photo and convert it into a 3D model.
Converting photos into 3D models is not new: photogrammetry, the technique of taking lots of photos of an object, loading them into an app like Meshmixer, and converting them to a 3D model, has been around for a while.
But sites like Meshy and Alpha 3D are taking this to the next level: both have recently released tools that can take a single image of an object and create a quite convincing 3D model from it. It is still a work in progress, but it’s a great way to create a 3D model of something you only have a single photo of.
For both of these tools, the sites warn that they are in testing, so you might get unexpected results. They also advise that they work best with certain types of objects and with single objects on a plain background: it can’t pick an object out of a complex photo yet.
Alpha 3D says its current 2D to 3D model is currently trained on a few categories of objects, including furniture and shoes. The tool also warns that by furniture, it really means couches. So, would it work on a gaming chair, which is a bit like a couch? I uploaded a photo of the Inland MACH 2 gaming chair, a top seller, and got the following result.
Image: Richard BaguleyLet’s try the same thing on Meshy.ai. They similarly warn that their 2D to 3D tool is still in alpha. It works on a credit system, where each model costs five credits to generate, and they give you 200 credits when you sign up. This time, it took about five minutes to create the 3D model.
Image: Richard BaguleyThe results here were far more impressive. The AI got the chair's shape right, the legs are correct, and it has wheels. However, it did miss a few things: the back of the real chair is pink, not the gray that the model assumed, and it missed the shape of the spring section at the rear of the base. Still, a very impressive final render.
Let’s try this with some more complex objects. Next, I tried Meshy.ai on a motherboard, an MSI C670 MAG Tomahawk. That’s a highly complex object that has numerous small features. Will it be able to pick out parts like the heatsinks and sockets? Rather than use a top-down view, I chose a photo from a slight angle to give it a few clues about the shape of the thing.
Image: Richard BaguleyFive minutes (and five credits) later, I had a result. I suspect that the AI model behind the Meshy.ai service hasn’t been trained on objects as unusual as mother boards. It figured out the raised hood over the back panel, but also seems to have assumed that it was also present on the underside of the motherboard. It also didn't zero in on the sockets and heatsinks on the surface of the motherboard. Still, it understood a lot of the image.
Finally, let’s try one of my favorite objects in the world, the Gayer-Anderson Cat, a unique late-period ancient Egyptian statue dedicated to the cat goddess Bastet. The original is in the British Museum. Can Meshy generate a model from a single photo of this object? To find out, I used a public domain photo from Peter Tillman because it isolates the cat from the background, which most pictures don’t, because the original is on exhibit, and the background is not the clean white that Meshy.ai prefers.
This took a bit longer to process: I gave it the full resolution original from Flickr, so it had more pixels to chew on. The model is available to view here.
Image: Richard BaguleyIt’s not bad at all, but it is missing a few things. The nose ring and earrings aren’t there, and the front feet look mushy. And, perhaps most importantly, the tail is missing, except for a stub: our Egyptian Sphinx cat has become a Manx.
If you want s clearer view, there is an excellent 3D model that restores the original finish and decoration that you can buy for $5.
My overall impressions of these tools? Interesting, if still limited in these early days. The reason you might want to use a tool like this is that you want to get a 3D model of an unusual object, something that you can’t easily model or scan yourself. To be fair, both Alpha 3D and Meshy.io point out that their tools are still experimental. No doubt, like most AI tools, they will get better as they are trained on a wider data set.
Read more: AI Tools and Tips
- What is TOPS? The AI Performance Metric Explained
- Copilot Plus PCs vs AI PCs: What's The Difference?
- Microsoft Launches a New Era of Copilot Plus PCs
- Hands-on with the Faster, Smarter ChatGPT-4o AI
- Why Coders are Learning to Love Copilot
- Roll Your Own GPT: Setting Up Your Computer for Local AI
- How to Get NVIDIA Chat with RTX: Local AI for Everyone
- How to Make Sure Your Next Computer Is AI Ready
