{"id":3019,"date":"2023-06-01T14:42:49","date_gmt":"2023-06-01T14:42:49","guid":{"rendered":"https:\/\/salarydistribution.com\/machine-learning\/2023\/06\/01\/digital-renaissance-nvidia-neuralangelo-research-reconstructs-3d-scenes\/"},"modified":"2023-06-01T14:42:49","modified_gmt":"2023-06-01T14:42:49","slug":"digital-renaissance-nvidia-neuralangelo-research-reconstructs-3d-scenes","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2023\/06\/01\/digital-renaissance-nvidia-neuralangelo-research-reconstructs-3d-scenes\/","title":{"rendered":"Digital Renaissance: NVIDIA Neuralangelo Research Reconstructs 3D Scenes"},"content":{"rendered":"
\n

Neuralangelo, a new AI model by NVIDIA Research for 3D reconstruction using neural networks, turns 2D video clips into detailed 3D structures \u2014 generating lifelike virtual replicas of buildings, sculptures and other real-world objects.<\/p>\n

Like Michelangelo sculpting stunning, life-like visions from blocks of marble, Neuralangelo<\/a> generates 3D structures with intricate details and textures. Creative professionals can then import these 3D objects into design applications, editing them further for use in art, video game development, robotics and industrial digital twins<\/a>.<\/p>\n

Neuralangelo\u2019s ability to translate the textures of complex materials \u2014 including roof shingles, panes of glass and smooth marble \u2014 from 2D videos to 3D assets significantly surpasses prior methods. The high fidelity makes its 3D reconstructions easier for developers and creative professionals to rapidly create usable virtual objects for their projects using footage captured by smartphones.<\/p>\n

\u201cThe 3D reconstruction capabilities Neuralangelo offers will be a huge benefit to creators, helping them recreate the real world in the digital world,\u201d said Ming-Yu Liu, senior director of research and co-author on the paper. \u201cThis tool will eventually enable developers to import detailed objects \u2014 whether small statues or massive buildings \u2014 into virtual environments for video games or industrial digital twins.\u201d<\/p>\n

In a demo, NVIDIA researchers showcased how the model could recreate objects as iconic as Michelangelo\u2019s David and as commonplace as a flatbed truck. Neuralangelo can also reconstruct building interiors and exteriors \u2014 demonstrated with a detailed 3D model of the park at NVIDIA\u2019s Bay Area campus.<\/p>\n<\/p>\n

Neural Rendering Model Sees in 3D<\/b><\/h2>\n

Prior AI models to reconstruct 3D scenes have struggled to accurately capture repetitive texture patterns, homogenous colors and strong color variations. Neuralangelo adopts instant neural graphics primitives, the technology behind NVIDIA Instant NeRF<\/a>, to help capture these finer details.<\/p>\n

Using a 2D video of an object or scene filmed from various angles, the model selects several frames that capture different viewpoints \u2014 like an artist considering a subject from multiple sides to get a sense of depth, size and shape.<\/p>\n

Once it\u2019s determined the camera position of each frame, Neuralangelo\u2019s AI creates a rough 3D representation of the scene, like a sculptor starting to chisel the subject\u2019s shape.<\/p>\n

The model then optimizes the render to sharpen the details, just as a sculptor painstakingly hews stone to mimic the texture of fabric or a human figure.<\/p>\n

The final result is a 3D object or large-scale scene that can be used in virtual reality applications, digital twins or robotics development.<\/p>\n

Find NVIDIA Research at CVPR, June 18-22<\/b><\/h2>\n

Neuralangelo<\/a> is one of nearly 30 projects by NVIDIA Research<\/a> to be presented at the Conference on Computer Vision and Pattern Recognition (CVPR), taking place June 18-22 in Vancouver. The papers span topics including pose estimation, 3D reconstruction and video generation.<\/p>\n

One of these projects, DiffCollage<\/a>, is a diffusion method that creates large-scale content \u2014 including long landscape orientation, 360-degree panorama and looped-motion images. When fed a training dataset of images with a standard aspect ratio, DiffCollage treats these smaller images as sections of a larger visual \u2014 like pieces of a collage. This enables diffusion models to generate cohesive-looking large content without being trained on images of the same scale.<\/p>\n

\"sunset<\/p>\n

The technique can also transform text prompts into video sequences, demonstrated using a pretrained diffusion model that captures human motion:<\/p>\n

Learn more about NVIDIA Research at CVPR<\/a>.<\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

https:\/\/blogs.nvidia.com\/blog\/2023\/06\/01\/neuralangelo-ai-research-3d-reconstruction\/<\/p>\n","protected":false},"author":0,"featured_media":3020,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[],"_links":{"self":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/3019"}],"collection":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/comments?post=3019"}],"version-history":[{"count":0,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/3019\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media\/3020"}],"wp:attachment":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media?parent=3019"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/categories?post=3019"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/tags?post=3019"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}