{"id":1255,"date":"2021-11-24T08:26:48","date_gmt":"2021-11-24T08:26:48","guid":{"rendered":"https:\/\/salarydistribution.com\/machine-learning\/2021\/11\/24\/build-mlops-workflows-with-amazon-sagemaker-projects-gitlab-and-gitlab-pipelines\/"},"modified":"2021-11-24T08:26:48","modified_gmt":"2021-11-24T08:26:48","slug":"build-mlops-workflows-with-amazon-sagemaker-projects-gitlab-and-gitlab-pipelines","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2021\/11\/24\/build-mlops-workflows-with-amazon-sagemaker-projects-gitlab-and-gitlab-pipelines\/","title":{"rendered":"Build MLOps workflows with Amazon SageMaker projects, GitLab, and GitLab pipelines"},"content":{"rendered":"
\n

Machine learning operations (MLOps) are key to effectively transition from an experimentation phase to production. The practice provides you the ability to create a repeatable mechanism to build, train, deploy, and manage machine learning models. To quickly adopt MLOps, you often require capabilities that use your existing toolsets and expertise. Projects<\/a> in Amazon SageMaker<\/a> give organizations the ability to easily set up and standardize developer environments for data scientists and CI\/CD (continuous integration, continuous delivery) systems for MLOps engineers. With SageMaker projects, MLOps engineers or organization administrators can define templates that bootstrap the ML workflow with source version control, automated ML pipelines, and a set of code to quickly start iterating over ML use cases. With projects, dependency management, code repository management, build reproducibility, and artifact sharing and management become easy for organizations to set up. SageMaker projects are provisioned using AWS Service Catalog<\/a> products. Your organization can use project templates to provision projects for each of your users.<\/p>\n

In this post, you use a custom SageMaker project template to incorporate CI\/CD practices with GitLab and GitLab pipelines. You automate building a model using Amazon SageMaker Pipelines<\/a> for data preparation, model training, and model evaluation. SageMaker projects builds on Pipelines by implementing the model deployment steps and using SageMaker Model Registry<\/a>, along with your existing CI\/CD tooling, to automatically provision a CI\/CD pipeline. In our use case, after the trained model is approved in the model registry, the model deployment pipeline is triggered via a GitLab pipeline.<\/p>\n

Prerequisites<\/h2>\n

For this walkthrough, you should have the following prerequisites:<\/p>\n

This post provides a detailed explanation of the SageMaker projects, GitLab, and GitLab pipelines integration. We review the code and discuss the components of the solution. To deploy the solution, reference the GitHub repo<\/a>, which provides step-by-step instructions for implementing a MLOps workflow using a SageMaker project template with GitLab and GitLab pipelines.<\/p>\n

Solution overview<\/h2>\n

The following diagram shows the architecture we build using a custom SageMaker project template.<\/p>\n

\"\"<\/a><\/p>\n

Let\u2019s review the components of this architecture to understand the end-to-end setup:<\/p>\n