{"id":1219,"date":"2021-11-18T08:32:51","date_gmt":"2021-11-18T08:32:51","guid":{"rendered":"https:\/\/salarydistribution.com\/machine-learning\/2021\/11\/18\/use-amazon-sagemaker-ack-operators-to-train-and-deploy-machine-learning-models\/"},"modified":"2021-11-18T08:32:51","modified_gmt":"2021-11-18T08:32:51","slug":"use-amazon-sagemaker-ack-operators-to-train-and-deploy-machine-learning-models","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2021\/11\/18\/use-amazon-sagemaker-ack-operators-to-train-and-deploy-machine-learning-models\/","title":{"rendered":"Use Amazon SageMaker ACK Operators to train and deploy machine learning models"},"content":{"rendered":"
\n

AWS recently released the new Amazon SageMaker Operators for Kubernetes using the AWS Controllers for Kubernetes (ACK)<\/a>. ACK is a framework for building Kubernetes custom controllers, where each controller communicates with an AWS service API. These controllers allow Kubernetes users to provision AWS resources like databases or message queues simply by using the Kubernetes API. The new SageMaker ACK Operators make it easier for machine learning (ML) developers and data scientists who use Kubernetes as their control plane to train, tune, and deploy ML models in Amazon SageMaker<\/a> without signing in to the SageMaker console.<\/p>\n

Kubernetes and SageMaker<\/h2>\n

Building scalable ML workflows involves many iterative steps, including sourcing and preparing data, building ML models, training and evaluating these models, deploying them to production, and monitoring workloads after deployment.<\/p>\n

SageMaker is a fully managed service designed and optimized specifically for managing these ML workflows. It removes the undifferentiated heavy lifting of infrastructure management and eliminates the need to invest in IT and DevOps to manage clusters for ML model building, training, and inference. Compute resources are only provisioned when requested, scaled as needed, and shut down automatically when jobs complete, thereby providing near 100% utilization. SageMaker provides many performance and cost optimizations for distributed training, spot training, automatic model tuning, inference latency, and multi-model endpoints.<\/p>\n

Many AWS customers who have portability requirements implement a hybrid cloud approach, or implement on-premises and use Kubernetes, an open-source, general-purpose container orchestration system, to set up repeatable ML pipelines running training and inference workloads. However, to support ML workloads, these developers still need to write custom code to optimize the underlying ML infrastructure, provide high availability and reliability, provide data science productivity tools, and comply with appropriate security and regulatory requirements. Kubernetes customers therefore want to use fully managed ML services such as SageMaker for cost-optimized and managed infrastructure, but want platform and infrastructure teams to continue using Kubernetes for orchestration and managing pipelines to retain standardization and portability.<\/p>\n

To address this need, AWS allows you to train, tune, and deploy models in SageMaker by using the new SageMaker ACK Operators, which includes a set of custom resource definitions for SageMaker resources that extends the Kubernetes API. With the SageMaker ACK Operators, you can take advantage of fully managed SageMaker infrastructure, tools, and optimizations natively from Kubernetes.<\/p>\n

How did we get here?<\/h2>\n

In late 2019, AWS introduced the SageMaker Operators for Kubernetes<\/a> to enable developers and data scientists to manage the end-to-end SageMaker training and production lifecycle using Kubernetes as the control plane. SageMaker operators were installed from the GitHub repo<\/a> by downloading a YAML configuration file that configured your Kubernetes cluster with the custom resource definitions and operator controller service.<\/p>\n

In 2020, AWS introduced ACK to facilitate a Kubernetes-native way of managing AWS Cloud resources. ACK includes a common controller runtime, a code generator, and a set of AWS service-specific controllers, one of which is the SageMaker controller<\/a>.<\/p>\n

Going forward, new functionality will be added to the SageMaker Operators for Kubernetes through the ACK project.<\/p>\n

How does ACK work?<\/h2>\n

The following diagram illustrates how ACK works.<\/p>\n

\"\"<\/p>\n

In this example, Alice is a Kubernetes user. She wants to run model training on SageMaker from within the Kubernetes cluster using the Kubernetes API. Alice issues a call to kubectl apply<\/code>, passing in a file that describes a Kubernetes custom resource<\/a> describing her SageMaker training job. kubectl apply<\/code> passes this file, called a manifest<\/a>, to the Kubernetes API server running in the Kubernetes controller node (Step 1 in the workflow diagram).<\/p>\n

The Kubernetes API server receives the manifest with the SageMaker training job specification and determines whether Alice has permissions<\/a> to create a custom resource of kind<\/a> sageMaker.services.k8s.aws\/TrainingJob<\/code>, and whether the custom resource is properly formatted (Step 2).<\/p>\n

If Alice is authorized and the custom resource is valid, the Kubernetes API server writes (Step 3) the custom resource to its etcd<\/code> data store and then responds back (Step 4) to Alice that the custom resource has been created.<\/p>\n

The SageMaker controller, which is running on a Kubernetes worker node within the context of a normal Kubernetes Pod<\/a>, is notified (Step 5) that a new custom resource of kind SageMaker.services.k8s.aws\/TrainingJob<\/code> has been created.<\/p>\n

The SageMaker controller then communicates (Step 6) with the SageMaker API, calling the SageMaker CreateTrainingJob API to create the training job in AWS. After communicating with the SageMaker API, the SageMaker controller calls the Kubernetes API server to update (Step 7) the custom resource\u2019s status with information it received from SageMaker. The SageMaker controller therefore provides the same information to the developers that they would have received using the AWS SDK. This results in a better and consistent developer experience.<\/p>\n

Machine learning use case<\/h2>\n

For this post, we follow the SageMaker example provided in the following notebook<\/a>. However, you can reuse the components in this example with your preference of SageMaker built-in or custom algorithms and your own datasets.<\/p>\n

We use the Abalone dataset<\/a> originally from the UCI data repository [1]. In the libsvm converted version, the nominal feature (male\/female\/infant) has been converted into a real valued feature. The age of abalone is to be predicted from eight physical measurements. This dataset is already processed and stored in Amazon Simple Storage Service<\/a> (Amazon S3). We train an XGBoost model on the UCI Abalone dataset to replicate the flow in the example Jupyter notebook.<\/p>\n

Prerequisites<\/h2>\n

For this walkthrough, you should have the following prerequisites:<\/p>\n

An existing Amazon Elastic Kubernetes Service<\/a> (Amazon EKS) cluster. It should be Kubernetes version 1.16+. For automated cluster creation using <\/span>eksctl<\/span><\/code>, see <\/span>Getting started with Amazon EKS \u2013 eksctl<\/code><\/span><\/a> and create your cluster with Amazon EC2 Linux managed nodes.<\/span><\/p>\n

Install the following tools on the client machine used to access your Kubernetes cluster (you can use AWS Cloud9<\/a>, a cloud-based integrated development environment (IDE) for the Kubernetes cluster setup):<\/p>\n