{"id":3673,"date":"2024-07-23T00:08:30","date_gmt":"2024-07-23T00:08:30","guid":{"rendered":"https:\/\/salarydistribution.com\/machine-learning\/2024\/07\/23\/nvidias-ai-masters-sweep-kdd-cup-2024-data-science-competition\/"},"modified":"2024-07-23T00:08:30","modified_gmt":"2024-07-23T00:08:30","slug":"nvidias-ai-masters-sweep-kdd-cup-2024-data-science-competition","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2024\/07\/23\/nvidias-ai-masters-sweep-kdd-cup-2024-data-science-competition\/","title":{"rendered":"NVIDIA\u2019s AI Masters Sweep KDD Cup 2024 Data Science Competition"},"content":{"rendered":"
\n\t\t<\/span> <\/span> <\/span><\/span><\/p>\n

Team NVIDIA has triumphed at the Amazon KDD Cup 2024<\/a>, securing first place Friday across all five competition tracks.<\/p>\n

The team \u2014 consisting of NVIDIANs Ahmet Erdem<\/a>, Benedikt Schifferer<\/a>, Chris Deotte<\/a>, Gilberto Titericz<\/a>, Ivan Sorokin<\/a> and Simon Jegou<\/a> \u2014 demonstrated its prowess in generative AI, winning in categories that included text generation, multiple-choice questions, name entity recognition, ranking, and retrieval.<\/p>\n

\"\"<\/a><\/p>\n

The competition, themed \u201cMulti-Task Online Shopping Challenge for LLMs<\/a>,\u201d asked participants to solve various challenges using limited datasets.<\/p>\n

\u201cThe new trend in LLM competitions is that they don\u2019t give you training data,\u201d said Deotte, a senior data scientist at NVIDIA. \u201cThey give you 96 example questions \u2014 not enough to train a model \u2014 so we came up with 500,000 questions on our own.\u201d<\/p>\n

Deotte explained that the NVIDIA team generated a variety of questions by writing some themselves, using a large language model<\/a> to create others, and transforming existing e-commerce datasets.<\/p>\n

\u201cOnce we had our questions, it was straightforward to use existing frameworks to fine-tune a language model,\u201d he said.<\/p>\n

The competition organizers hid the test questions to ensure participants couldn\u2019t exploit previously known answers. This approach encourages models that generalize well to any question about e-commerce, proving the model\u2019s capability to handle real-world scenarios effectively.<\/p>\n

Despite these constraints, Team NVIDIA\u2019s innovative approach outperformed all competitors by using Qwen2-72B, a just-released LLM with 72 billion parameters, fine-tuned on eight NVIDIA A100 Tensor Core GPUs, and employing QLoRA, a technique for fine-tuning models with datasets.<\/p>\n

About the KDD Cup 2024<\/strong><\/h2>\n

The KDD Cup, organized by the Association for Computing Machinery\u2019s Special Interest Group on Knowledge Discovery and Data Mining, or ACM SIGKDD, is a prestigious annual competition that promotes research and development in the field.<\/p>\n

This year\u2019s challenge, hosted by Amazon, focused on mimicking the complexities of online shopping with the goal of making it a more intuitive and satisfying experience using large language models. Organizers utilized the test dataset ShopBench \u2014 a benchmark that replicates the massive challenge for online shopping with 57 tasks and about 20,000 questions derived from real-world Amazon shopping data \u2014 to evaluate participants\u2019 models.<\/p>\n

The ShopBench benchmark focused on four key shopping skills, along with a fifth \u201call-in-one\u201d challenge:<\/p>\n

    \n
  1. Shopping Concept Understanding: Decoding complex shopping concepts and terminologies.<\/li>\n
  2. Shopping Knowledge Reasoning: Making informed decisions with shopping knowledge.<\/li>\n
  3. User Behavior Alignment: Understanding dynamic customer behavior.<\/li>\n
  4. Multilingual Abilities: Shopping across languages.<\/li>\n
  5. All-Around: Solving all tasks from the previous tracks in a unified solution.<\/li>\n<\/ol>\n

    NVIDIA\u2019s Winning Solution<\/strong><\/h2>\n

    NVIDIA\u2019s winning solution involved creating a single model for each track.<\/p>\n

    The team fine-tuned the just-released Qwen2-72B model using eight NVIDIA A100 Tensor Core GPUs for approximately 24 hours. The GPUs provided fast and efficient processing, significantly reducing the time required for fine-tuning.<\/p>\n

    \"\"<\/a><\/p>\n

    First, the team generated training datasets based on the provided examples and synthesized additional data using Llama 3 70B hosted on build.nvidia.com<\/a>.<\/p>\n

    Next, they employed QLoRA (Quantized Low-Rank Adaptation), a training process using the data created in step one. QLoRA modifies a smaller subset of the model\u2019s weights, allowing efficient training and fine-tuning.<\/p>\n

    The model was then quantized \u2014 making it smaller and able to run on a system with a smaller hard drive and less memory \u2014 with AWQ 4-bit and used the vLLM inference library to predict the test datasets on four NVIDIA T4 Tensor Core GPUs within the time constraints.<\/p>\n

    This approach secured the top spot in each individual track and the overall first place in the competition\u2014a clean sweep for NVIDIA for the second year in a row.<\/p>\n

    The team plans to submit a detailed paper on its solution next month and plans to present its findings at KDD 2024 in Barcelona.<\/p>\n

    \t\t<\/p><\/div>\n","protected":false},"excerpt":{"rendered":"

    https:\/\/blogs.nvidia.com\/blog\/nvidia-ai-masters-kdd-cup-2024\/<\/p>\n","protected":false},"author":0,"featured_media":3674,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[],"_links":{"self":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/3673"}],"collection":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/comments?post=3673"}],"version-history":[{"count":0,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/3673\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media\/3674"}],"wp:attachment":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media?parent=3673"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/categories?post=3673"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/tags?post=3673"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}