{"id":3633,"date":"2024-06-15T01:12:54","date_gmt":"2024-06-15T01:12:54","guid":{"rendered":"https:\/\/salarydistribution.com\/machine-learning\/2024\/06\/15\/nvidia-releases-open-synthetic-data-generation-pipeline-for-training-large-language-models\/"},"modified":"2024-06-15T01:12:54","modified_gmt":"2024-06-15T01:12:54","slug":"nvidia-releases-open-synthetic-data-generation-pipeline-for-training-large-language-models","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2024\/06\/15\/nvidia-releases-open-synthetic-data-generation-pipeline-for-training-large-language-models\/","title":{"rendered":"NVIDIA Releases Open Synthetic Data Generation Pipeline for Training Large Language Models"},"content":{"rendered":"
\n\t\t<\/span> <\/span> <\/span><\/span><\/p>\n

NVIDIA today announced Nemotron-4 340B, a family of open models that developers can use to generate synthetic data for training large language models (LLMs) for commercial applications across healthcare, finance, manufacturing, retail and every other industry.<\/p>\n

High-quality training data plays a critical role in the performance, accuracy and quality of responses from a custom LLM \u2014 but robust datasets can be prohibitively expensive and difficult to access.<\/p>\n

Through a uniquely permissive open model license<\/a>, Nemotron-4 340B gives developers a free, scalable way to generate synthetic data that can help build powerful LLMs.<\/p>\n

The Nemotron-4 340B family includes base, instruct and reward models that form a pipeline to generate synthetic data used for training and refining LLMs. The models are optimized to work with NVIDIA NeMo<\/a>, an open-source framework for end-to-end model training, including data curation, customization and evaluation. They\u2019re also optimized for inference with the open-source NVIDIA TensorRT-LLM<\/a> library.<\/p>\n

Nemotron-4 340B can be downloaded now from Hugging Face<\/a>. Developers will soon be able to access the models at ai.nvidia.com<\/a>, where they\u2019ll be packaged as an NVIDIA NIM<\/a> microservice with a standard application programming interface that can be deployed anywhere.<\/p>\n

Navigating Nemotron to Generate Synthetic Data<\/h2>\n

LLMs can help developers generate synthetic training data in scenarios where access to large, diverse labeled datasets is limited.<\/p>\n

The Nemotron-4 340B Instruct<\/a> model creates diverse synthetic data that mimics the characteristics of real-world data, helping improve data quality to increase the performance and robustness of custom LLMs across various domains.<\/p>\n

Then, to boost the quality of the AI-generated data, developers can use the Nemotron-4 340B Reward<\/a> model to filter for high-quality responses. Nemotron-4 340B Reward grades responses on five attributes: helpfulness, correctness, coherence, complexity and verbosity. It\u2019s currently first place on the Hugging Face RewardBench leaderboard<\/a>, created by AI2<\/a>, for evaluating the capabilities, safety and pitfalls of reward models.<\/p>\n

\"nemotron<\/a>
In this synthetic data generation pipeline, (1) the Nemotron-4 340B Instruct model is first used to produce synthetic text-based output. An evaluator model, (2) Nemotron-4 340B Reward, then assesses this generated text \u2014 providing feedback that guides iterative improvements and ensures the synthetic data is accurate, relevant and aligned with specific requirements.<\/figcaption><\/figure>\n

Researchers can also create their own instruct or reward models by customizing the Nemotron-4 340B Base<\/a> model using their proprietary data, combined with the included HelpSteer2 dataset<\/a>.<\/p>\n

Fine-Tuning With NeMo, Optimizing for Inference With TensorRT-LLM<\/h2>\n

Using open-source NVIDIA NeMo and NVIDIA TensorRT-LLM, developers can optimize the efficiency of their instruct and reward models to generate synthetic data and to score responses.<\/p>\n

All Nemotron-4 340B models are optimized with TensorRT-LLM to take advantage of tensor parallelism, a type of model parallelism in which individual weight matrices are split across multiple GPUs and servers, enabling efficient inference at scale.<\/p>\n

Nemotron-4 340B Base, trained on 9 trillion tokens, can be customized using the NeMo framework to adapt to specific use cases or domains. This fine-tuning process benefits from extensive pretraining data and yields more accurate outputs for specific downstream tasks.<\/p>\n

A variety of customization methods are available through the NeMo framework, including supervised fine-tuning and parameter-efficient fine-tuning methods such as low-rank adaptation, or LoRA.<\/p>\n

To boost model quality, developers can align their models with NeMo Aligner<\/a> and datasets annotated by Nemotron-4 340B Reward. Alignment is a key step in training LLMs, where a model\u2019s behavior is fine-tuned using algorithms like reinforcement learning from human feedback (RLHF) to ensure its outputs are safe, accurate, contextually appropriate and consistent with its intended goals.<\/p>\n

Businesses seeking enterprise-grade support and security for production environments can also access NeMo and TensorRT-LLM through the cloud-native NVIDIA AI Enterprise<\/a> software platform, which provides accelerated and efficient runtimes for generative AI foundation models.<\/p>\n

Evaluating Model Security and Getting Started<\/h2>\n

The Nemotron-4 340B Instruct model underwent extensive safety evaluation, including adversarial tests, and performed well across a wide range of risk indicators. Users should still perform careful evaluation of the model\u2019s outputs to ensure the synthetically generated data is suitable, safe and accurate for their use case.<\/p>\n

For more information on model security and safety evaluation, read the model card.<\/p>\n

Download Nemotron-4 340B models via\u00a0Hugging Face<\/a>. For more details, read the research papers on the model<\/a> and dataset<\/a>.<\/p>\n

See <\/i>notice<\/i><\/a> regarding software product information.<\/i><\/p>\n

\t\t<\/p><\/div>\n","protected":false},"excerpt":{"rendered":"

https:\/\/blogs.nvidia.com\/blog\/nemotron-4-synthetic-data-generation-llm-training\/<\/p>\n","protected":false},"author":0,"featured_media":3634,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[],"_links":{"self":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/3633"}],"collection":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/comments?post=3633"}],"version-history":[{"count":0,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/3633\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media\/3634"}],"wp:attachment":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media?parent=3633"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/categories?post=3633"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/tags?post=3633"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}