{"id":919,"date":"2021-09-22T19:17:55","date_gmt":"2021-09-22T19:17:55","guid":{"rendered":"https:\/\/salarydistribution.com\/machine-learning\/2021\/09\/22\/nvidia-extends-ai-inference-performance-leadership-with-debut-results-on-arm-based-servers\/"},"modified":"2021-09-22T19:17:55","modified_gmt":"2021-09-22T19:17:55","slug":"nvidia-extends-ai-inference-performance-leadership-with-debut-results-on-arm-based-servers","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2021\/09\/22\/nvidia-extends-ai-inference-performance-leadership-with-debut-results-on-arm-based-servers\/","title":{"rendered":"NVIDIA Extends AI Inference Performance Leadership, with Debut Results on Arm-based Servers"},"content":{"rendered":"
\n

NVIDIA delivers the best results in AI inference using either x86 or Arm-based CPUs, according to benchmarks released today<\/a>.<\/p>\n

It\u2019s the third consecutive time NVIDIA has set records in performance and energy efficiency on inference tests from MLCommons<\/a>, an industry benchmarking group formed in May 2018.<\/p>\n

And it\u2019s the first time the data-center category tests have run on an Arm-based system, giving users more choice in how they deploy AI, the most transformative technology of our time.<\/p>\n

Tale of the Tape<\/b><\/h2>\n

NVIDIA AI platform-powered computers topped all seven performance tests of inference in the latest round with systems from NVIDIA and nine of our ecosystem partners including Alibaba, Dell Technologies, Fujitsu, GIGABYTE, Hewlett Packard Enterprise, Inspur, , Nettrix and Supermicro.<\/p>\n

And NVIDIA is the only company to report results on all MLPerf tests in this and every round to date.<\/p>\n

<\/p>\n

\"MLPerf<\/p>\n

<\/a><\/p>\n

Inference is what happens when a computer runs AI software to recognize an object or make a prediction. It\u2019s a process that uses a deep learning model to filter data, finding results no human could capture.<\/p>\n

MLPerf\u2019s inference benchmarks are based on today\u2019s most popular AI workloads and scenarios, covering computer vision, medical imaging, natural language processing, recommendation systems, reinforcement learning and more.<\/p>\n

So, whatever AI applications they deploy, users can set their own records with NVIDIA.<\/p>\n

Why Performance Matters<\/b><\/h2>\n

AI models and datasets continue to grow as AI use cases expand from the data center to the edge and beyond. That\u2019s why users need performance that\u2019s both dependable and flexible to deploy.<\/p>\n

MLPerf gives users the confidence to make informed buying decisions. It\u2019s backed by dozens of industry leaders, including Alibaba, Arm, Baidu, Google, Intel and NVIDIA, so the tests are transparent and objective.<\/p>\n

Flexing Arm for Enterprise AI<\/b><\/h2>\n

The Arm architecture is making headway into data centers around the world, in part thanks to its energy efficiency, performance increases and expanding software ecosystem.<\/p>\n

The latest benchmarks show that as a GPU-accelerated platform, Arm-based servers using Ampere Altra CPUs deliver near-equal performance to similarly configured x86-based servers for AI inference jobs. In fact, in one of the tests, the Arm-based server out-performed a similar x86 system.<\/p>\n

NVIDIA has a long tradition of supporting every CPU architecture, so we\u2019re proud to see Arm prove its AI prowess in a peer-reviewed industry benchmark.<\/p>\n

\u201cArm, as a founding member of MLCommons, is committed to the process of creating standards and benchmarks to better address challenges and inspire innovation in the accelerated computing industry,\u201d said David Lecomber, a senior director of HPC and tools at Arm.<\/p>\n

\u201cThe latest inference results demonstrate the readiness of Arm-based systems powered by Arm-based CPUs and NVIDIA GPUs for tackling a broad array of AI workloads in the data center,\u201d he added.<\/p>\n

\"MLPerf<\/p>\n

<\/a><\/p>\n

Partners Show Their AI Powers<\/b><\/h2>\n

NVIDIA\u2019s AI technology is backed by a large and growing ecosystem.<\/p>\n

Seven OEMs submitted a total of 22 GPU-accelerated platforms in the latest benchmarks.<\/p>\n

Most of these server models are NVIDIA-Certified, validated for running a diverse range of accelerated workloads. And many of them support NVIDIA AI Enterprise<\/a>, software officially released last month.<\/p>\n

Our partners participating in this round included Dell Technologies, Fujitsu, Hewlett Packard Enterprise, Inspur, Lenovo, Nettrix and Supermicro as well as cloud-service provider Alibaba.<\/p>\n

The Power of Software<\/b><\/h2>\n

A key ingredient of NVIDIA\u2019s AI success across all use cases is our full software stack.<\/p>\n

For inference, that includes pre-trained AI models for a wide variety of use cases. The NVIDIA TAO Toolkit<\/a> customizes those models for specific applications using transfer learning.<\/p>\n

Our NVIDIA TensorRT<\/a> software optimizes AI models so they make best use of memory and run faster. We routinely use it for MLPerf tests, and it\u2019s available for both x86 and Arm-based systems.<\/p>\n

We also employed our NVIDIA Triton Inference Server<\/a> software and Multi-Instance GPU (MIG<\/a>) capability in these benchmarks. They deliver for all developers the kind of performance that usually requires expert coders.<\/p>\n

Thanks to continuous improvements in this software stack, NVIDIA achieved gains up to 20 percent in performance and 15 percent in energy efficiency from previous MLPerf inference benchmarks<\/a> just four months ago.<\/p>\n

All the software we used in the latest tests is available from the MLPerf repository, so anyone can reproduce our benchmark results. We continually add this code into our deep learning frameworks and containers available on NGC<\/a>, our software hub for GPU applications.<\/p>\n

It\u2019s part of a full-stack AI offering, supporting every major processor architecture, proven in the latest industry benchmarks and available to tackle real AI jobs today.<\/p>\n

To learn more about the NVIDIA inference platform, check out our NVIDIA Inference Technology Overview<\/a>.<\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

http:\/\/feedproxy.google.com\/~r\/nvidiablog\/~3\/Vnwv2fz5Q3w\/<\/p>\n","protected":false},"author":0,"featured_media":920,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[],"_links":{"self":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/919"}],"collection":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/comments?post=919"}],"version-history":[{"count":0,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/919\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media\/920"}],"wp:attachment":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media?parent=919"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/categories?post=919"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/tags?post=919"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}