{"id":3233,"date":"2023-10-25T14:50:35","date_gmt":"2023-10-25T14:50:35","guid":{"rendered":"https:\/\/salarydistribution.com\/machine-learning\/2023\/10\/25\/next-gen-neural-networks-nvidia-research-announces-array-of-ai-advancements-at-neurips\/"},"modified":"2023-10-25T14:50:35","modified_gmt":"2023-10-25T14:50:35","slug":"next-gen-neural-networks-nvidia-research-announces-array-of-ai-advancements-at-neurips","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2023\/10\/25\/next-gen-neural-networks-nvidia-research-announces-array-of-ai-advancements-at-neurips\/","title":{"rendered":"Next-Gen Neural Networks: NVIDIA Research Announces Array of AI Advancements at NeurIPS"},"content":{"rendered":"<div id=\"bsf_rt_marker\">\n<p>NVIDIA researchers are collaborating with academic centers worldwide to advance <a href=\"https:\/\/www.nvidia.com\/en-us\/glossary\/data-science\/generative-ai\/\">generative AI<\/a>, robotics and the natural sciences \u2014 and more than a dozen of these projects will be shared at <a href=\"https:\/\/neurips.cc\/\" target=\"_blank\" rel=\"noopener\">NeurIPS<\/a>, one of the world\u2019s top AI conferences.<\/p>\n<p>Set for Dec. 10-16 in New Orleans, NeurIPS brings together experts in generative AI, machine learning, computer vision and more. Among the innovations <a href=\"https:\/\/www.nvidia.com\/en-us\/research\/\">NVIDIA Research<\/a> will present are new techniques for transforming text to images, photos to 3D avatars, and specialized robots into multi-talented machines.<\/p>\n<p>\u201cNVIDIA Research continues to drive progress across the field \u2014 including generative AI models that transform text to images or speech, autonomous AI agents that learn new tasks faster, and neural networks that calculate complex physics,\u201d said Jan Kautz, vice president of learning and perception research at NVIDIA. \u201cThese projects, often done in collaboration with leading minds in academia, will help accelerate developers of virtual worlds, simulations and autonomous machines.\u201d<\/p>\n<h2><b>Picture This: Improving Text-to-Image Diffusion Models<\/b><\/h2>\n<p>Diffusion models have become the most popular type of generative AI models to turn text into realistic imagery. NVIDIA researchers have collaborated with universities on multiple projects advancing diffusion models that will be presented at NeurIPS.<\/p>\n<ul>\n<li>A paper accepted as an oral presentation focuses on improving generative AI models\u2019 ability to <a href=\"https:\/\/neurips.cc\/virtual\/2023\/oral\/73870\" target=\"_blank\" rel=\"noopener\">understand the link between modifier words and main entities<\/a> in text prompts. While existing text-to-image models asked to depict a yellow tomato and a red lemon may incorrectly generate images of yellow lemons and red tomatoes, the new model analyzes the syntax of a user\u2019s prompt, encouraging a bond between an entity and its modifiers to deliver a more faithful visual depiction of the prompt.<\/li>\n<li>SceneScape, a new framework using <a href=\"https:\/\/neurips.cc\/virtual\/2023\/poster\/71859\" target=\"_blank\" rel=\"noopener\">diffusion models to create long videos of 3D scenes from text prompts<\/a>, will be presented as a poster. The project combines a text-to-image model with a depth prediction model that helps the videos maintain plausible-looking scenes with consistency between the frames \u2014 generating videos of art museums, haunted houses and ice castles (pictured above).<\/li>\n<li>Another poster describes work that improves how text-to-image models <a href=\"https:\/\/neurips.cc\/virtual\/2023\/poster\/70922\" target=\"_blank\" rel=\"noopener\">generate concepts rarely seen in training data<\/a>. Attempts to generate such images usually result in low-quality visuals that aren\u2019t an exact match to the user\u2019s prompt. The new method uses a small set of example images that help the model identify good seeds \u2014 random number sequences that guide the AI to generate images from the specified rare classes.<\/li>\n<li>A third poster shows how a text-to-image diffusion model can <a href=\"https:\/\/neurips.cc\/virtual\/2023\/poster\/70648\" target=\"_blank\" rel=\"noopener\">use the text description of an incomplete point cloud<\/a> to generate missing parts and create a complete 3D model of the object. This could help complete point cloud data collected by lidar scanners and other depth sensors for robotics and autonomous vehicle AI applications. Collected imagery is often incomplete because objects are scanned from a specific angle \u2014 for example, a lidar sensor mounted to a vehicle would only scan one side of each building as the car drives down a street.<\/li>\n<\/ul>\n<p><a href=\"https:\/\/blogs.nvidia.com\/wp-content\/uploads\/2023\/10\/Point-Cloud-Completion.jpg\"><\/p>\n<p><img decoding=\"async\" loading=\"lazy\" src=\"https:\/\/blogs.nvidia.com\/wp-content\/uploads\/2023\/10\/Point-Cloud-Completion-672x222.jpg\" alt=\"\" width=\"672\" height=\"222\"><\/p>\n<p><\/a><\/p>\n<h2><b>Character Development: Advancements in AI Avatars<\/b><\/h2>\n<p>AI avatars combine multiple generative AI models to create and animate virtual characters, produce text and convert it to speech. Two NVIDIA posters at NeurIPS present new ways to make these tasks more efficient.<\/p>\n<ul>\n<li>A poster describes a new method to <a href=\"https:\/\/neurips.cc\/virtual\/2023\/poster\/72615\" target=\"_blank\" rel=\"noopener\">turn a single portrait image into a 3D head avatar<\/a> while capturing details including hairstyles and accessories. Unlike current methods that require multiple images and a time-consuming optimization process, this model achieves high-fidelity 3D reconstruction without additional optimization during inference. The avatars can be animated either with blendshapes, which are 3D mesh representations used to represent different facial expressions, or with a reference video clip where a person\u2019s facial expressions and motion are applied to the avatar.<\/li>\n<li>Another poster by NVIDIA researchers and university collaborators advances zero-shot text-to-speech synthesis with P-Flow, a generative AI model that can <a href=\"https:\/\/neurips.cc\/virtual\/2023\/poster\/69899#:~:text=Our%20work%20proposes%20P%2DFlow,quality%20and%20fast%20speech%20synthesis.\" target=\"_blank\" rel=\"noopener\">rapidly synthesize high-quality personalized speech<\/a> given a three-second reference prompt. P-Flow features better pronunciation, human likeness and speaker similarity compared to recent state-of-the-art counterparts. The model can near-instantly convert text to speech on a single <a href=\"https:\/\/www.nvidia.com\/en-us\/data-center\/a100\/\">NVIDIA A100 Tensor Core GPU<\/a>.<\/li>\n<\/ul>\n<h2><b>Research Breakthroughs in Reinforcement Learning, Robotics<\/b><\/h2>\n<p>In the fields of reinforcement learning and robotics, NVIDIA researchers will present two posters highlighting innovations that improve the generalizability of AI across different tasks and environments.<\/p>\n<ul>\n<li>The first proposes a <a href=\"https:\/\/neurips.cc\/virtual\/2023\/poster\/72040\" target=\"_blank\" rel=\"noopener\">framework for developing reinforcement learning algorithms<\/a> that can adapt to new tasks while avoiding the common pitfalls of gradient bias and data inefficiency. The researchers showed that their method \u2014 which features a novel meta-algorithm that can create a robust version of <i>any<\/i> meta-reinforcement learning model \u2014 performed well on multiple benchmark tasks.<\/li>\n<li>Another by an NVIDIA researcher and university collaborators tackles the challenge of <a href=\"https:\/\/neurips.cc\/virtual\/2023\/poster\/71709\" target=\"_blank\" rel=\"noopener\">object manipulation in robotics<\/a>. Prior AI models that help robotic hands pick up and interact with objects can handle specific shapes but struggle with objects unseen in the training data. The researchers introduce a new framework that estimates how objects across different categories are geometrically alike \u2014 such as drawers and pot lids that have similar handles \u2014 enabling the model to more quickly generalize to new shapes.<\/li>\n<\/ul>\n<h2><b>Supercharging Science: AI-Accelerated Physics, Climate, Healthcare<\/b><\/h2>\n<p>NVIDIA researchers at NeurIPS will also present papers across the natural sciences \u2014 covering physics simulations, climate models and AI for healthcare.<\/p>\n<ul>\n<li>To accelerate <a href=\"https:\/\/neurips.cc\/virtual\/2023\/poster\/72670\" target=\"_blank\" rel=\"noopener\">computational fluid dynamics for large-scale 3D simulations<\/a>, a team of NVIDIA researchers proposed a neural operator architecture that combines accuracy and computational efficiency to estimate the pressure field around vehicles \u2014 the first deep learning-based computational fluid dynamics method on an industry-standard, large-scale automotive benchmark. The method achieved 100,000x acceleration on a single <a href=\"https:\/\/www.nvidia.com\/en-us\/data-center\/tensor-cores\/\">NVIDIA Tensor Core GPU<\/a> compared to another GPU-based solver, while reducing the error rate. Researchers can incorporate the model into their own applications using the open-source <a href=\"https:\/\/github.com\/neuraloperator\/neuraloperator\" target=\"_blank\" rel=\"noopener\">neuraloperator library<\/a>.<\/li>\n<\/ul>\n<p><a href=\"https:\/\/blogs.nvidia.com\/wp-content\/uploads\/2023\/10\/Computational-Fluid-Dynamics-1.jpg\"><\/p>\n<p><img decoding=\"async\" loading=\"lazy\" src=\"https:\/\/blogs.nvidia.com\/wp-content\/uploads\/2023\/10\/Computational-Fluid-Dynamics-1-672x390.jpg\" alt=\"\" width=\"672\" height=\"390\"><\/p>\n<p><\/a><\/p>\n<p>\u00a0<\/p>\n<ul>\n<li>A consortium of climate scientists and machine learning researchers from universities, national labs, research institutes, Allen AI and NVIDIA collaborated on <a href=\"https:\/\/neurips.cc\/virtual\/2023\/poster\/73569\" target=\"_blank\" rel=\"noopener\">ClimSim<\/a>, a massive dataset for physics and machine learning-based climate research that will be shared in an oral presentation at NeurIPS. The dataset covers the globe over multiple years at high resolution \u2014 and machine learning emulators built using that data can be plugged into existing operational climate simulators to improve their fidelity, accuracy and precision. This can help scientists produce better predictions of storms and other extreme events.<\/li>\n<li>NVIDIA Research interns are presenting a poster introducing an AI algorithm that provides personalized <a href=\"https:\/\/neurips.cc\/virtual\/2023\/poster\/71940\" target=\"_blank\" rel=\"noopener\">predictions of the effects of medicine dosage<\/a> on patients. Using real-world data, the researchers tested the model\u2019s predictions of blood coagulation for patients given different dosages of a treatment. They also analyzed the new algorithm\u2019s predictions of the antibiotic vancomycin levels in patients who received the medication \u2014 and found that prediction accuracy significantly improved compared to prior methods.<\/li>\n<\/ul>\n<p><a href=\"https:\/\/www.nvidia.com\/en-us\/research\/\"><i>NVIDIA Research<\/i><\/a><i> comprises hundreds of scientists and engineers worldwide, with teams focused on topics including AI, computer graphics, computer vision, self-driving cars and robotics. <\/i><\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>https:\/\/blogs.nvidia.com\/blog\/2023\/10\/25\/neurips-ai-research\/<\/p>\n","protected":false},"author":0,"featured_media":3234,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[],"_links":{"self":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/3233"}],"collection":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/comments?post=3233"}],"version-history":[{"count":0,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/3233\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media\/3234"}],"wp:attachment":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media?parent=3233"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/categories?post=3233"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/tags?post=3233"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}