{"id":2513,"date":"2022-08-19T16:51:35","date_gmt":"2022-08-19T16:51:35","guid":{"rendered":"https:\/\/salarydistribution.com\/machine-learning\/2022\/08\/19\/meet-the-omnivore-startup-in3d-turns-selfies-into-talking-dancing-avatars-with-nvidia-omniverse\/"},"modified":"2022-08-19T16:51:35","modified_gmt":"2022-08-19T16:51:35","slug":"meet-the-omnivore-startup-in3d-turns-selfies-into-talking-dancing-avatars-with-nvidia-omniverse","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2022\/08\/19\/meet-the-omnivore-startup-in3d-turns-selfies-into-talking-dancing-avatars-with-nvidia-omniverse\/","title":{"rendered":"Meet the Omnivore: Startup in3D Turns Selfies Into Talking, Dancing Avatars With NVIDIA Omniverse"},"content":{"rendered":"
\n

Editor\u2019s note: This post is a part of our <\/i>Meet the Omnivore<\/i><\/a> series, which features individual creators and developers who use <\/i>NVIDIA Omniverse<\/i><\/a> to accelerate their 3D workflows and create virtual worlds.<\/i><\/p>\n

Imagine taking a selfie and using it to get a moving, talking, customizable 3D avatar of yourself in just seconds.<\/p>\n

A new extension for NVIDIA Omniverse<\/a>, a design collaboration and world simulation platform, enables just that.<\/p>\n

Created by developers at software startup in3D<\/a>, the extension<\/a> lets people instantly import 3D avatars of themselves into virtual environments using their smartphones. Omniverse Extensions<\/a> are the core building blocks that let anyone create and extend functions of Omniverse Apps<\/a>.<\/p>\n

The in3D app can now bring people, in their digital forms, into Omniverse. It helps creators build engaging virtual worlds<\/a> and use these avatars as heroes, actors or spectators in their stories. The app works on any phone with a camera, recreating a user\u2019s full geometry and texture based on a video selfie.<\/p>\n

The avatars can even be added into 3D worlds with animations and a customizable wardrobe.<\/p>\n<\/p>\n

In3D is a member of NVIDIA Inception<\/a>, a free, global program that nurtures cutting-edge startups.<\/p>\n

Simple and Scalable Avatar Creation<\/b><\/h2>\n

Creating a photorealistic 3D avatar<\/a> has traditionally taken up to several months, with costs reaching up to tens of thousands of dollars<\/a>. Photogrammetry, a standard approach to creating 3D references of humans from images, is extremely costly, requires a digital studio and lacks scalability.<\/p>\n

With in3D, the process of creating 3D avatars is simple and scalable. The app understands the geometry, texture, depth and various vectors of a person via a mobile scan \u2014 and uses this information to replicate lifelike detail and create predictive animations for avatars.<\/p>\n

Dmitry Ulyanov, CEO of in3D, which is based in Tel Aviv, Israel, said the app captures even small details with centimeter-grade accuracy and automatically fixes lighting. This allows for precise head geometry from a single selfie, as well as estimation of a user\u2019s exact body shape.<\/p>\n

\"\"<\/p>\n

For creators building 3D worlds, in3D software can save countless hours, increase productivity and result in substantial cost savings, Ulyanov said.<\/p>\n

\u201cManually creating one avatar can take up to months,\u201d he added. \u201cWith in3D\u2019s scanning app and software development kit, a user can scan and upload 21,000 people with a single GPU and mobile phone in the same amount of time.\u201d<\/p>\n

Connecting to Omniverse<\/b><\/h2>\n

Ulyanov said that using in3D\u2019s extension with NVIDIA Omniverse Avatar Cloud Engine (ACE)<\/a> opens up many possibilities for avatar building, as users can easily customize imported avatars from in3D to engage and interact with their virtual worlds \u2014 in real time and at scale.<\/p>\n

In3D uses Universal Scene Description (USD)<\/a>, an open-source, extensible file format, to seamlessly integrate its high-fidelity avatars into Omniverse. All avatar data is contained in a USD file, removing the need for complex shaders or embeddings. And bringing the avatars into Omniverse only requires a simple drag and drop.<\/p>\n

Once imported into Omniverse via USD, the avatars can be used in apps like Omniverse Create<\/a> and Audio2Face<\/a>. Users have a complete toolset within Omniverse to support holistic content creation, whether animating avatars\u2019 bodies with the retargeting tool or crafting their facial expressions with Audio2Face.<\/p>\n<\/p>\n

To build the Omniverse Extension, in3D used Omniverse Kit<\/a> and followed the development flow using the VSCode computer program. Being able to put a breakpoint anywhere in the code made VSCode an easy-to-use, convenient, out-of-the-box solution for connecting in3D to Omniverse, Ulyanov said.<\/p>\n

\u201cThe ability to centralize our SDK alongside other software for 3D developers is game changing,\u201d he said. \u201cWith our Omniverse Extension now available, we\u2019re looking to expand the base of developers who use our avatars.\u201d<\/p>\n

\u201cHaving the ability to upload our SDK and connect it with all the tools that 3D developers use has made in3D a tangible solution to deploy across all 3D development environments,\u201d said Sergei Sherman, chief marketing officer at in3D. \u201cThis was something we wouldn\u2019t have been able to achieve on our own in such a short amount of time.\u201d<\/p>\n

Join In on the Creation<\/b><\/h2>\n

Creators and developers across the world can download NVIDIA Omniverse for free<\/a>, and enterprise teams<\/a> can use the platform for their 3D projects.<\/p>\n

Learn how to connect and create virtual worlds with Omniverse at NVIDIA GTC, the design and simulation conference for the era of AI and the metaverse, running online Sept. 19-22. Registration is free<\/a> and offers access to dozens of sessions and special events.<\/p>\n

Developers can use Omniverse Code<\/a> to create their own Omniverse Extension for the inaugural #ExtendOmniverse contest<\/a> by Friday, Sept. 9, at 5 p.m. PT, for a chance to win an NVIDIA RTX GPU. The winners will be announced in the NVIDIA Omniverse User Group<\/a> at GTC.<\/p>\n

Find additional documentation and tutorials in the Omniverse Resource Center<\/a>, which details how developers like Ulyanov can build custom USD-based applications and extensions for the platform.<\/p>\n

Follow NVIDIA Omniverse on <\/i>Instagram<\/i><\/a>, <\/i>Medium<\/i><\/a>, <\/i>Twitter<\/i><\/a> and <\/i>YouTube<\/i><\/a> for additional resources and inspiration. Check out the Omniverse <\/i>forums<\/i><\/a>, and join our <\/i>Discord server<\/i><\/a> and <\/i>Twitch <\/i><\/a>channel<\/i> to chat with the community.<\/i><\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

https:\/\/blogs.nvidia.com\/blog\/2022\/08\/19\/omniverse-developer-in3d\/<\/p>\n","protected":false},"author":0,"featured_media":2514,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[],"_links":{"self":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/2513"}],"collection":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/comments?post=2513"}],"version-history":[{"count":0,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/2513\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media\/2514"}],"wp:attachment":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media?parent=2513"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/categories?post=2513"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/tags?post=2513"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}