{"id":3671,"date":"2024-07-22T17:47:49","date_gmt":"2024-07-22T17:47:49","guid":{"rendered":"https:\/\/salarydistribution.com\/machine-learning\/2024\/07\/22\/sustainable-strides-how-ai-and-accelerated-computing-are-driving-energy-efficiency\/"},"modified":"2024-07-22T17:47:49","modified_gmt":"2024-07-22T17:47:49","slug":"sustainable-strides-how-ai-and-accelerated-computing-are-driving-energy-efficiency","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2024\/07\/22\/sustainable-strides-how-ai-and-accelerated-computing-are-driving-energy-efficiency\/","title":{"rendered":"Sustainable Strides: How AI and Accelerated Computing Are Driving Energy Efficiency"},"content":{"rendered":"
\n\t\t<\/span> <\/span> <\/span><\/span><\/p>\n

AI and accelerated computing<\/a> \u2014 twin engines NVIDIA continuously improves \u2014 are delivering energy efficiency<\/a> for many industries.<\/p>\n

It\u2019s progress the wider community is starting to acknowledge.<\/p>\n

\u201cEven if the predictions that data centers will soon account for 4% of global energy consumption become a reality, AI is having a major impact on reducing the remaining 96% of energy consumption,\u201d said a report<\/a> from Lisbon Council Research, a nonprofit formed in 2003 that studies economic and social issues.<\/p>\n

The article from the Brussels-based research group is among a handful of big-picture AI policy studies starting to emerge. It uses Italy\u2019s Leonardo supercomputer<\/a>, accelerated with nearly 14,000 NVIDIA GPUs, as an example of a system advancing work in fields from automobile design and drug discovery to weather forecasting.<\/p>\n

\"\"<\/a>
Energy-efficiency gains over time for the most efficient supercomputer on the TOP500 list. Source: TOP500.org<\/figcaption><\/figure>\n

Why Accelerated Computing Is Sustainable Computing<\/b><\/h2>\n

Accelerated computing uses the parallel processing of NVIDIA GPUs to do more work in less time. As a result, it consumes less energy than general-purpose servers that employ CPUs built to handle one task at a time.<\/p>\n

That\u2019s why accelerated computing is sustainable computing<\/a>.<\/p>\n

\"\"<\/a>
Accelerated systems use parallel processing on GPUs to do more work in less time, consuming less energy than CPUs.<\/figcaption><\/figure>\n

The gains are even greater when accelerated systems apply AI<\/a>, an inherently parallel form of computing that\u2019s the most transformative technology of our time.<\/p>\n

\u201cWhen it comes to frontier applications like machine learning or deep learning, the performance of GPUs is an order of magnitude better than that of CPUs,\u201d the report said.<\/p>\n

\"\"<\/a>
NVIDIA offers a combination of GPUs, CPUs, and DPUs tailored to maximize energy efficiency with accelerated computing.<\/figcaption><\/figure>\n

User Experiences With Accelerated AI<\/b><\/h2>\n

Users worldwide are documenting energy-efficiency gains with AI and accelerated computing.<\/p>\n

In financial services, Murex<\/a> \u2014 a Paris-based company with a trading and risk-management platform used daily by more than 60,000 people \u2014 tested the NVIDIA Grace Hopper Superchip<\/a>. On its workloads, the CPU-GPU combo delivered a 4x reduction in energy consumption and a 7x reduction in time to completion compared with CPU-only systems (see chart below).<\/p>\n

\u201cOn risk calculations, Grace is not only the fastest processor, but also far more power-efficient, making green IT a reality in the trading world,\u201d said Pierre Spatz, head of quantitative research at Murex.<\/p>\n

\"\"<\/a><\/p>\n

In manufacturing, Taiwan-based Wistron<\/a> built a digital copy of a room where NVIDIA DGX systems<\/a> undergo thermal stress tests to improve operations at the site. It used NVIDIA Omniverse<\/a>, a platform for industrial digitization, with a surrogate model, a version of AI that emulates simulations.<\/p>\n

The digital twin, linked to thousands of networked sensors, enabled Wistron to increase the facility\u2019s overall energy efficiency by up to 10%. That amounts to reducing electricity consumption by 120,000 kWh per year and carbon emissions by a whopping 60,000 kilograms.<\/p>\n

Up to 80% Fewer Carbon Emissions<\/b><\/h2>\n

The RAPIDS Accelerator for Apache Spark<\/a> can reduce the carbon footprint for data analytics, a widely used form of machine learning, by as much as 80% while delivering 5x average speedups and 4x reductions in computing costs, according to a recent benchmark<\/a>.<\/p>\n

Thousands of companies \u2014 about 80% of the Fortune 500 \u2014 use Apache Spark<\/a> to analyze their growing mountains of data. Companies using NVIDIA\u2019s Spark accelerator include Adobe, AT&T<\/a> and the U.S. Internal Revenue Service<\/a>.<\/p>\n<\/p>\n

In healthcare, Insilico Medicine<\/a> discovered and put into phase 2 clinical trials a drug candidate for a relatively rare respiratory disease, thanks to its NVIDIA-powered AI platform.<\/p>\n

Using traditional methods, the work would have cost more than $400 million<\/a> and taken up to six years<\/a>. But with generative AI<\/a>, Insilico hit the milestone for one-tenth of the cost in one-third of the time.<\/p>\n

\u201cThis is a significant milestone not only for us, but for everyone in the field of AI-accelerated drug discovery,\u201d said Alex Zhavoronkov, CEO of Insilico Medicine.<\/p>\n

This is just a sampler of results that users of accelerated computing and AI are pursuing at companies such as Amgen<\/a>, BMW<\/a>, Foxconn<\/a>, PayPal<\/a> and many more.<\/p>\n

Speeding Science With Accelerated AI\u00a0<\/b><\/h2>\n

In basic research, the National Energy Research Scientific Computing Center (NERSC<\/a>), the U.S. Department of Energy\u2019s lead facility for open science, measured results<\/a> on a server with four NVIDIA A100 Tensor Core GPUs<\/a> compared with dual-socket x86 CPU servers across four of its key high-performance computing and AI applications.<\/p>\n

Researchers found that the apps, when accelerated with the NVIDIA A100 GPUs, saw energy efficiency<\/a> rise 5x on average (see below). One application, for weather forecasting, logged gains of nearly 10x.<\/p>\n

\"\"<\/a><\/p>\n

Scientists and researchers worldwide depend on AI and accelerated computing to achieve high performance and efficiency.<\/p>\n

In a recent ranking<\/a> of the world\u2019s most energy-efficient supercomputers, known as the Green500<\/a>, NVIDIA-powered systems swept the top six spots, and 40 of the top 50.<\/p>\n

Underestimated Energy Savings<\/b><\/h2>\n

The many gains across industries and science are sometimes overlooked in forecasts that extrapolate only the energy consumption of training the largest AI models. That misses the benefits from most of an AI model\u2019s life when it\u2019s consuming relatively little energy, delivering the kinds of efficiencies users described above.<\/p>\n

In an analysis citing dozens of sources, a recent study<\/a> debunked as misleading and inflated projections based on training models.<\/p>\n

\u201cJust as the early predictions about the energy footprints of e-commerce and video streaming ultimately proved to be exaggerated, so too will those estimates about AI likely be wrong,\u201d said the report from the Information Technology and Innovation Foundation (ITIF), a Washington-based think tank.<\/p>\n

The report notes as much as 90% of the cost \u2014 and all the efficiency gains \u2014 of running an AI model are in deploying it in applications after it\u2019s trained.<\/p>\n

\u201cGiven the enormous opportunities to use AI to benefit the economy and society \u2014 including transitioning to a low-carbon future \u2014 it is imperative that policymakers and the media do a better job of vetting the claims they entertain about AI\u2019s environmental impact,\u201d said the report\u2019s author, who described his findings in a recent podcast<\/a>.<\/p>\n

Others Cite AI\u2019s Energy Benefits<\/b><\/h2>\n

Policy analysts from the R Street Institute, also in Washington, D.C., agreed.<\/p>\n

\u201cRather than a pause, policymakers need to help realize the potential for gains from AI,\u201d the group wrote in a 1,200-word article<\/a>.<\/p>\n

\u201cAccelerated computing and the rise of AI hold great promise for the future, with significant societal benefits in terms of economic growth and social welfare,\u201d it said, citing demonstrated benefits of AI in drug discovery<\/a>, banking<\/a>, stock trading<\/a> and insurance<\/a>.<\/p>\n

AI can make the electric grid, manufacturing and transportation sectors more efficient, it added.<\/p>\n

AI Supports Sustainability Efforts<\/b><\/h2>\n

The reports also cited the potential of accelerated AI to fight climate change and promote sustainability.<\/p>\n

\u201cAI can enhance the accuracy of weather modeling<\/a> to improve public safety as well as generate more accurate predictions of crop yields<\/a>. The power of AI can also contribute to \u2026 developing more precise climate models<\/a>,\u201d R Street said.<\/p>\n

The Lisbon report added that AI plays \u201ca crucial role in the innovation needed to address climate change\u201d for work such as discovering more efficient battery materials.<\/p>\n

How AI Can Help the Environment<\/b><\/h2>\n

ITIF called on governments to adopt AI as a tool in efforts to decarbonize their operations.<\/p>\n

Public and private organizations are already applying NVIDIA AI to protect coral reefs<\/a>, improve tracking of wildfires<\/a> and extreme weather<\/a>, and enhance sustainable agriculture<\/a>.<\/p>\n

For its part, NVIDIA is working with hundreds of startups<\/a> employing AI to address climate issues. NVIDIA also announced plans for Earth-2<\/a>, expected to be the world\u2019s most powerful AI supercomputer dedicated to climate science.<\/p>\n

Enhancing Energy Efficiency Across the Stack<\/b><\/h2>\n

Since its founding in 1993, NVIDIA has worked on energy efficiency across all its products \u2014 GPUs, CPUs, DPUs<\/a>, networks, systems and software, as well as platforms such as Omniverse.<\/p>\n

In AI, the brunt of an AI model\u2019s life is in inference, delivering insights that help users achieve new efficiencies. The NVIDIA GB200 Grace Blackwell Superchip<\/a> has demonstrated 25x energy efficiency over the prior NVIDIA Hopper GPU generation in AI inference.<\/p>\n

Over the last eight years, NVIDIA GPUs have advanced a whopping 45,000x in their energy efficiency running large language models (see chart below).<\/p>\n

\"\"<\/a><\/p>\n

Recent innovations in software include TensorRT-LLM<\/a>. It can help GPUs reduce 3x the energy consumption of LLM inference.<\/p>\n

Here\u2019s an eye-popping stat: If the efficiency of cars improved as much as NVIDIA has advanced the efficiency of AI on its accelerated computing platform, cars would get 280,000 miles per gallon. That means you could drive to the moon on less than a gallon of gas.<\/p>\n

The analysis applies to the fuel efficiency of cars NVIDIA\u2019s whopping 10,000x efficiency gain in AI training and inference from 2016 to 2025 (see chart below).<\/p>\n

\"\"<\/a>
How the big AI efficiency leap from the NVIDIA P100 GPU to the NVIDIA Grace Blackwell compares to car fuel-efficiency gains.<\/figcaption><\/figure>\n

Driving Data Center Efficiency<\/b><\/h2>\n

NVIDIA delivers many optimizations through system-level innovations. For example, NVIDIA BlueField-3 DPUs<\/a> can reduce power consumption<\/a> up to 30% by offloading essential data center networking and infrastructure functions from less efficient CPUs.<\/p>\n

\"\"<\/a><\/p>\n

Last year, NVIDIA received a $5 million grant<\/a> from the U.S. Department of Energy \u2014 the largest of 15 grants from a pool of more than 100 applications \u2014 to design a new liquid-cooling technology for data centers. It will run 20% more efficiently than today\u2019s air-cooled approaches and has a smaller carbon footprint.<\/p>\n

These are just some of the ways NVIDIA contributes to the energy efficiency of data centers.<\/p>\n

\"\"<\/a><\/p>\n

Data centers are among the most efficient users of energy and one of the largest consumers of renewable energy.<\/p>\n

The ITIF report notes that between 2010 and 2018, global data centers experienced a 550% increase in compute instances and a 2,400% increase in storage capacity, but only a 6% increase in energy use, thanks to improvements across hardware and software.<\/p>\n

NVIDIA continues to drive energy efficiency for accelerated AI, helping users in science, government and industry accelerate their journeys toward sustainable computing.<\/p>\n

Try NVIDIA\u2019s <\/i>energy-efficiency calculator<\/i><\/a> to find ways to improve energy efficiency. And check out NVIDIA\u2019s <\/i>sustainable computing site<\/i><\/a> and <\/i>corporate sustainability report<\/i><\/a> for more information.\u00a0<\/i><\/p>\n<\/p>\n

\t\t<\/p><\/div>\n","protected":false},"excerpt":{"rendered":"

https:\/\/blogs.nvidia.com\/blog\/accelerated-ai-energy-efficiency\/<\/p>\n","protected":false},"author":0,"featured_media":3672,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[],"_links":{"self":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/3671"}],"collection":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/comments?post=3671"}],"version-history":[{"count":0,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/3671\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media\/3672"}],"wp:attachment":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media?parent=3671"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/categories?post=3671"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/tags?post=3671"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}