{"id":1373,"date":"2021-12-15T00:19:34","date_gmt":"2021-12-15T00:19:34","guid":{"rendered":"https:\/\/salarydistribution.com\/machine-learning\/2021\/12\/15\/add-automl-functionality-with-amazon-sagemaker-autopilot-across-accounts\/"},"modified":"2021-12-15T00:19:34","modified_gmt":"2021-12-15T00:19:34","slug":"add-automl-functionality-with-amazon-sagemaker-autopilot-across-accounts","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2021\/12\/15\/add-automl-functionality-with-amazon-sagemaker-autopilot-across-accounts\/","title":{"rendered":"Add AutoML functionality with Amazon SageMaker Autopilot across accounts"},"content":{"rendered":"<div id=\"\">\n<p>AutoML is a powerful capability, provided by <a href=\"https:\/\/aws.amazon.com\/sagemaker\/autopilot\/\" target=\"_blank\" rel=\"noopener noreferrer\">Amazon SageMaker Autopilot<\/a>, that allows non-experts to create machine learning (ML) models to invoke in their applications.<\/p>\n<p>The problem that we want to solve arises when, due to governance constraints, <a href=\"https:\/\/aws.amazon.com\/sagemaker\/\" target=\"_blank\" rel=\"noopener noreferrer\">Amazon SageMaker<\/a> resources can\u2019t be deployed in the same AWS account where they are used.<\/p>\n<p>Examples of such a situation are:<\/p>\n<ul>\n<li>A multi-account enterprise setup of AWS where the Autopilot resources must be deployed in a specific AWS account (the trusting account), and should be accessed from trusted accounts<\/li>\n<li>A software as a service (SaaS) provider that offers AutoML to their users and adopts the resources in the customer AWS account so that the billing is associated to the end customer<\/li>\n<\/ul>\n<p>This post walks through an implementation using the SageMaker Python SDK. It\u2019s divided into two sections:<\/p>\n<ul>\n<li>Create the <a href=\"http:\/\/aws.amazon.com\/iam\" target=\"_blank\" rel=\"noopener noreferrer\">AWS Identity and Access Management<\/a> (IAM) resources needed for cross-account access<\/li>\n<li>Perform the Autopilot job, deploy the top model, and make predictions from the trusted account accessing the trusting account<\/li>\n<\/ul>\n<p>The solution described in this post is provided in the Jupyter notebook available in this <a href=\"https:\/\/github.com\/aws-samples\/amazon-sagemaker-autopilot-cross-account\" target=\"_blank\" rel=\"noopener noreferrer\">GitHub repository<\/a>.<\/p>\n<p>For a full explanation of Autopilot, you can refer to the examples available in GitHub, particularly <a href=\"https:\/\/github.com\/aws\/amazon-sagemaker-examples\/blob\/master\/autopilot\/autopilot_customer_churn_high_level_with_evaluation.ipynb\" target=\"_blank\" rel=\"noopener noreferrer\">Top Candidates Customer Churn Prediction with Amazon SageMaker Autopilot and Batch Transform (Python SDK)<\/a>.<\/p>\n<h2>Prerequisites<\/h2>\n<p>We have two AWS accounts:<\/p>\n<ul>\n<li><strong>Customer (trusting) account<\/strong> \u2013 Where the SageMaker resources are deployed<\/li>\n<li><strong>SaaS (trusted) account<\/strong> \u2013 Drives the training and prediction activities<\/li>\n<\/ul>\n<p>You have to <a href=\"https:\/\/docs.aws.amazon.com\/IAM\/latest\/UserGuide\/id_users_create.html#id_users_create_console\" target=\"_blank\" rel=\"noopener noreferrer\">create a user<\/a> for each account, with programmatic access enabled and the <code>IAMFullAccess<\/code> managed policy associated.<\/p>\n<p>You have to <a href=\"https:\/\/docs.aws.amazon.com\/cli\/latest\/userguide\/cli-configure-profiles.html\">configure the user profiles<\/a> in the <code>.aws\/credentials<\/code> file:<\/p>\n<ul>\n<li><code>customer_config<\/code> for the user configured in the customer account<\/li>\n<li><code>saas_config<\/code> for the user configured in the SaaS account<\/li>\n<\/ul>\n<p>To update the SageMaker SDK, run the following command in your Python environment:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-bash\">!pip install --update sagemaker<\/code><\/pre>\n<\/p><\/div>\n<p>The procedure has been tested in the SageMaker environment <code>conda_python3<\/code>.<\/p>\n<h2>Common modules and initial definitions<\/h2>\n<p>Import common Python modules used in the script:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">import boto3\nimport json\nimport sagemaker\nfrom botocore.exceptions import ClientError<\/code><\/pre>\n<\/p><\/div>\n<p>Let\u2019s define the AWS Region that will host the resources:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">REGION = boto3.Session().region_name<\/code><\/pre>\n<\/p><\/div>\n<p>and the reference to the dataset for the training of the model:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">DATASET_URI = \"s3:\/\/sagemaker-sample-files\/datasets\/tabular\/synthetic\/churn.txt\"<\/code><\/pre>\n<\/p><\/div>\n<h2>Set up the IAM resources<\/h2>\n<p>The following diagram illustrates the IAM entities that we create, which allow the cross-account implementation of the Autopilot job.<br \/><a href=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2021\/10\/25\/ML-4196-img1.png\"><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-29816\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2021\/10\/25\/ML-4196-img1.png\" alt=\"\" width=\"641\" height=\"241\"><\/a><\/p>\n<p>On the customer account, we define the single role <code>customer_trusting_saas<\/code>, which consolidates the permissions for <a href=\"http:\/\/aws.amazon.com\/s3\" target=\"_blank\" rel=\"noopener noreferrer\">Amazon Simple Storage Service<\/a> (Amazon S3) and SageMaker access needed for the following:<\/p>\n<ul>\n<li>The local SageMaker service that performs the Autopilot actions<\/li>\n<li>The principal in the SaaS account that initiates the actions in the customer account<\/li>\n<\/ul>\n<p>On the SaaS account, we define the following:<\/p>\n<ul>\n<li>The <code>AutopilotUsers<\/code> group with the policy required to assume the <code>customer_trusting_saas<\/code> role via <a href=\"https:\/\/docs.aws.amazon.com\/STS\/latest\/APIReference\/welcome.html\" target=\"_blank\" rel=\"noopener noreferrer\">AWS Security Token Service<\/a> (AWS STS)<\/li>\n<li>The <code>saas_user<\/code>, which is a member of the <code>AutopilotUsers<\/code> group and is the actual principal triggering the Autopilot actions<\/li>\n<\/ul>\n<p>For additional security, in the cross-account trust relationship, we use the external ID to mitigate the <a href=\"https:\/\/docs.aws.amazon.com\/IAM\/latest\/UserGuide\/id_roles_create_for-user_externalid.html\" target=\"_blank\" rel=\"noopener noreferrer\">confused deputy problem<\/a>.<\/p>\n<p>Let\u2019s proceed with the setup.<\/p>\n<p>For each of the two accounts, we complete the following tasks:<\/p>\n<ol>\n<li>Create the Boto3 session with the profile of the respective configuration user.<\/li>\n<li>Retrieve the AWS account ID by means of AWS STS.<\/li>\n<li>Create the IAM client that performs the configuration steps in the account.<\/li>\n<\/ol>\n<p>For the customer account, use the following code:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">customer_config_session = boto3.session.Session(profile_name=\"customer_config\")\nCUSTOMER_ACCOUNT_ID = customer_config_session.client(\"sts\").get_caller_identity()[\"Account\"]\ncustomer_iam_client = customer_config_session.client(\"iam\")<\/code><\/pre>\n<\/p><\/div>\n<p>Use the following code in the SaaS account:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">saas_config_session = boto3.session.Session(profile_name=\"saas_config\")\nSAAS_ACCOUNT_ID = saas_config_session.client(\"sts\").get_caller_identity()[\"Account\"]\nsaas_iam_client = saas_config_session.client(\"iam\")<\/code><\/pre>\n<\/p><\/div>\n<h3>Set up the IAM entities in the customer account<\/h3>\n<p>Let\u2019s first define the role needed to perform cross-account tasks from the SaaS account in the customer account.<\/p>\n<p>For simplicity, the same role is adopted for trusting SageMaker in the customer account. Ideally, consider splitting this role into two roles with fine-grained permissions in line with the principle of granting <a href=\"https:\/\/docs.aws.amazon.com\/IAM\/latest\/UserGuide\/best-practices.html#grant-least-privilege\" target=\"_blank\" rel=\"noopener noreferrer\">least privilege<\/a>.<\/p>\n<p>The role name and the references to the ARN of the SageMaker AWS managed policies are as follows:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">CUSTOMER_TRUST_SAAS_ROLE_NAME = \"customer_trusting_saas\"\nCUSTOMER_TRUST_SAAS_ROLE_ARN = \"arn:aws:iam::{}:role\/{}\".format(CUSTOMER_ACCOUNT_ID, CUSTOMER_TRUST_SAAS_ROLE_NAME)\nSAGEMAKERFULLACCESS_POLICY_ARN = \"arn:aws:iam::aws:policy\/AmazonSageMakerFullAccess\"<\/code><\/pre>\n<\/p><\/div>\n<p>The following customer managed policy gives the role the permissions to access the Amazon S3 resources that are needed for the SageMaker tasks and for the cross-account copy of the dataset.<\/p>\n<p>We restrict the access to the S3 buckets dedicated to SageMaker in the AWS Region for the customer account. See the following code:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">CUSTOMER_S3_POLICY_NAME = \"customer_s3\"\nCUSTOMER_S3_POLICY = \n{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n    {\n      \"Effect\": \"Allow\",\n      \"Action\": [\n         \"s3:GetObject\",\n         \"s3:PutObject\",\n         \"s3:DeleteObject\",\n         \"s3:ListBucket\"\n      ],\n      \"Resource\": [\n         \"arn:aws:s3:::sagemaker-{}-{}\".format(REGION, CUSTOMER_ACCOUNT_ID),\n         \"arn:aws:s3:::sagemaker-{}-{}\/*\".format(REGION, CUSTOMER_ACCOUNT_ID)\n      ]\n    }\n  ]\n}<\/code><\/pre>\n<\/p><\/div>\n<p>Then we define the external ID to mitigate the <a href=\"https:\/\/docs.aws.amazon.com\/IAM\/latest\/UserGuide\/id_roles_create_for-user_externalid.html\" target=\"_blank\" rel=\"noopener noreferrer\">confused deputy problem<\/a>:<\/p>\n<p>The trust relationships policy allows the principals from the trusted account and SageMaker to assume the role:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">CUSTOMER_TRUST_SAAS_POLICY = \n{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n    {\n      \"Effect\": \"Allow\",\n      \"Principal\": {\n        \"AWS\": \"arn:aws:iam::{}:root\".format(SAAS_ACCOUNT_ID)\n      },\n      \"Action\": \"sts:AssumeRole\",\n      \"Condition\": {\n        \"StringEquals\": {\n          \"sts:ExternalId\": EXTERNAL_ID\n        }\n      }\n    },\n    {\n      \"Effect\": \"Allow\",\n      \"Principal\": {\n        \"Service\": \"sagemaker.amazonaws.com\"\n      },\n      \"Action\": \"sts:AssumeRole\"\n    }\n  ]\n}<\/code><\/pre>\n<\/p><\/div>\n<p>For simplicity, we don\u2019t include the management of the exceptions in the following snippets. See the Jupyter notebook for the full code.<\/p>\n<p>We create the customer managed policy in the customer account, create the new role, and attach the two policies. We use the <a href=\"https:\/\/docs.aws.amazon.com\/IAM\/latest\/UserGuide\/id_roles_use.html#id_roles_use_view-role-max-session\" target=\"_blank\" rel=\"noopener noreferrer\">maximum session duration<\/a> parameter to manage long-running jobs. See the following code:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">MAX_SESSION_DURATION = 10800\ncreate_policy_response = customer_iam_client.create_policy(PolicyName=CUSTOMER_S3_POLICY_NAME,\n                                                           PolicyDocument=json.dumps(CUSTOMER_S3_POLICY))\ncustomer_s3_policy_arn = create_policy_response[\"Policy\"][\"Arn\"]\n\ncreate_role_response = customer_iam_client.create_role(RoleName=CUSTOMER_TRUST_SAAS_ROLE_NAME,\n                                                       AssumeRolePolicyDocument=json.dumps(CUSTOMER_TRUST_SAAS_POLICY),\n                                                       MaxSessionDuration=MAX_SESSION_DURATION)\n\ncustomer_iam_client.attach_role_policy(RoleName=CUSTOMER_TRUST_SAAS_ROLE_NAME,\n                                       PolicyArn=customer_s3_policy_arn)\ncustomer_iam_client.attach_role_policy(RoleName=CUSTOMER_TRUST_SAAS_ROLE_NAME,\n                                       PolicyArn=SAGEMAKERFULLACCESS_POLICY_ARN)<\/code><\/pre>\n<\/p><\/div>\n<h3>Set up IAM entities in the SaaS account<\/h3>\n<p>We define the following in the SaaS account:<\/p>\n<ul>\n<li>A group of users allowed to perform the Autopilot job in the customer account<\/li>\n<li>A policy associated with the group for assuming the role defined in the customer account<\/li>\n<li>A policy associated with the group for uploading data to Amazon S3 and managing bucket policies<\/li>\n<li>A user that is responsible for the implementation of the Autopilot jobs \u2013 the user has programmatic access<\/li>\n<li>A user profile to store the user access key and secret in the file for the credentials<\/li>\n<\/ul>\n<p>Let\u2019s start with defining the name of the group (<code>AutopilotUsers<\/code>):<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-bash\">SAAS_USER_GROUP_NAME = \"AutopilotUsers\"<\/code><\/pre>\n<\/p><\/div>\n<p>The first policy refers to the customer account ID and the role:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">SAAS_ASSUME_ROLE_POLICY_NAME = \"saas_assume_customer_role\"\nSAAS_ASSUME_ROLE_POLICY = \n{\n    \"Version\": \"2012-10-17\",\n    \"Statement\": [\n        {\n            \"Effect\": \"Allow\",\n            \"Action\": \"sts:AssumeRole\",\n            \"Resource\": \"arn:aws:iam::{}:role\/{}\".format(CUSTOMER_ACCOUNT_ID, CUSTOMER_TRUST_SAAS_ROLE_NAME)\n        }\n    ]\n}<\/code><\/pre>\n<\/p><\/div>\n<p>The second policy is needed to download the dataset, and to manage the Amazon S3 bucket used by SageMaker:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">SAAS_S3_POLICY_NAME = \"saas_s3\"\nSAAS_S3_POLICY = \n{\n    \"Version\": \"2012-10-17\",\n    \"Statement\": [\n        {\n            \"Effect\": \"Allow\",\n            \"Action\": [\n                \"s3:GetObject\"\n            ],\n            \"Resource\": [\n                \"arn:aws:s3:::{}\".format(DATASET_URI.split(':\/\/')[1])\n            ]\n        },\n        {\n            \"Effect\": \"Allow\",\n            \"Action\": [\n                \"s3:CreateBucket\",\n                \"s3:GetObject\",\n                \"s3:PutObject\",\n                \"s3:DeleteObject\",\n                \"s3:PutBucketPolicy\",\n                \"s3:DeleteBucketPolicy\"\n            ],\n            \"Resource\": [\n                \"arn:aws:s3:::sagemaker-{}-{}\".format(REGION, SAAS_ACCOUNT_ID),\n                \"arn:aws:s3:::sagemaker-{}-{}\/*\".format(REGION, SAAS_ACCOUNT_ID)\n            ]\n        }\n    ]\n}<\/code><\/pre>\n<\/p><\/div>\n<p>For simplicity, we give the same value to the user name and user profile:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-bash\">SAAS_USER_PROFILE = SAAS_USER_NAME = \"saas_user\"<\/code><\/pre>\n<\/p><\/div>\n<p>Now we create the two new managed policies. Next, we create the group, attach the policies to the group, create the user with programmatic access, and insert the user into the group. See the following code:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">create_policy_response = saas_iam_client.create_policy(PolicyName=SAAS_ASSUME_ROLE_POLICY_NAME,\n                                                       PolicyDocument=json.dumps(SAAS_ASSUME_ROLE_POLICY))      \nsaas_assume_role_policy_arn = create_policy_response[\"Policy\"][\"Arn\"]\n\ncreate_policy_response = saas_iam_client.create_policy(PolicyName=SAAS_S3_POLICY_NAME,\n                                                       PolicyDocument=json.dumps(SAAS_S3_POLICY))\nsaas_s3_policy_arn = create_policy_response[\"Policy\"][\"Arn\"]\n\nsaas_iam_client.create_group(GroupName=SAAS_USER_GROUP_NAME)\n\nsaas_iam_client.attach_group_policy(GroupName=SAAS_USER_GROUP_NAME,PolicyArn=saas_assume_role_policy_arn)\nsaas_iam_client.attach_group_policy(GroupName=SAAS_USER_GROUP_NAME,PolicyArn=saas_s3_policy_arn)\n\nsaas_iam_client.create_user(UserName=SAAS_USER_NAME)\nsaas_iam_client.create_access_key(UserName=SAAS_USER_NAME)\n\nadd_user_to_group(GroupName=SAAS_USER_GROUP_NAME,UserName=SAAS_USER_NAME)<\/code><\/pre>\n<\/p><\/div>\n<h3>Update the credentials file<\/h3>\n<p>Create the user profile for <code>saas_user<\/code> in the <code>.aws\/credentials<\/code> file:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">from pathlib import Path\nimport configparser\n\ncredentials_config = configparser.ConfigParser()\ncredentials_config.read(str(Path.home()) + \"\/.aws\/credentials\")\n\nif not credentials_config.has_section(SAAS_USER_PROFILE):\n    credentials_config.add_section(SAAS_USER_PROFILE)\n    \ncredentials_config[SAAS_USER_PROFILE][\"aws_access_key_id\"] = create_akey_response[\"AccessKey\"][\"AccessKeyId\"]\ncredentials_config[SAAS_USER_PROFILE][\"aws_secret_access_key\"] = create_akey_response[\"AccessKey\"][\"SecretAccessKey\"]\n\nwith open(str(Path.home()) + \"\/.aws\/credentials\", \"w\") as configfile:\n    credentials_config.write(configfile, space_around_delimiters=False)<\/code><\/pre>\n<\/p><\/div>\n<p>This completes the configuration of IAM entities that are needed for the cross-account implementation of the Autopilot job.<\/p>\n<h2>Autopilot cross-account access<\/h2>\n<p>This is the core objective of the post, where we demonstrate the main differences with respect to the single-account scenario.<\/p>\n<p>First, we prepare the dataset the Autopilot job uses for training the models.<\/p>\n<h3>Data<\/h3>\n<p>We reuse the same dataset adopted in the SageMaker example: <a href=\"https:\/\/github.com\/aws\/amazon-sagemaker-examples\/blob\/master\/autopilot\/autopilot_customer_churn_high_level_with_evaluation.ipynb\" target=\"_blank\" rel=\"noopener noreferrer\">Top Candidates Customer Churn Prediction with Amazon SageMaker Autopilot and Batch Transform (Python SDK)<\/a>.<\/p>\n<p>For a full explanation of the data, refer to the original example.<\/p>\n<p>We skip the data inspection and proceed directly to the focus of this post, which is the cross-account Autopilot job invocation.<\/p>\n<p>Download the churn dataset with the following <a href=\"http:\/\/aws.amazon.com\/cli\" target=\"_blank\" rel=\"noopener noreferrer\">AWS Command Line Interface<\/a> (AWS CLI) command:<\/p>\n<div class=\"hide-language\">\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">!aws s3 cp $DATASET_URI .\/ --profile saas_user<\/code><\/pre>\n<\/p><\/div>\n<\/p><\/div>\n<h3>Split the dataset for the Autopilot job and the inference phase<\/h3>\n<p>After you load the dataset, split it into two parts:<\/p>\n<ul>\n<li>80% for the Autopilot job to train the top model<\/li>\n<li>20% for testing the model that we deploy<\/li>\n<\/ul>\n<p>Autopilot applies a <a href=\"https:\/\/aws.amazon.com\/it\/about-aws\/whats-new\/2021\/05\/amazon-sagemaker-autopilot-adds-automatic-cross-validation-to-im\/\" target=\"_blank\" rel=\"noopener noreferrer\">cross-validation<\/a> resampling procedure, on the dataset passed as input, to all candidate algorithms to test their ability to predict data they have not been trained on.<\/p>\n<p>Split the dataset with the following code:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">import pandas as pd\nimport numpy as np\n\nchurn = pd.read_csv(\".\/churn.txt\")\ntrain_data = churn.sample(frac=0.8,random_state=200)\ntest_data = churn.drop(train_data.index)\ntest_data_no_target = test_data.drop(columns=[\"Churn?\"])<\/code><\/pre>\n<\/p><\/div>\n<p>Let\u2019s save the training data into a file locally that we pass to the fit method of the AutoML estimator:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">train_file = \"train_data.csv\"\ntrain_data.to_csv(train_file, index=False, header=True)<\/code><\/pre>\n<\/p><\/div>\n<h3>Autopilot training job, deployment, and prediction overview<\/h3>\n<p>The training, deployment, and prediction process is illustrated in the following diagram.<br \/><a href=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2021\/10\/28\/image-12.png\"><img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-29962 size-full\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2021\/10\/28\/image-12.png\" alt=\"\" width=\"541\" height=\"302\"><\/a><\/p>\n<p>The following are the steps for the cross-account invocation:<\/p>\n<ol>\n<li>Initiate a session as <code>saas_user<\/code> in the SaaS account and load the profile from the credentials.<\/li>\n<li>Assume the role in the customer account via the AWS STS.<\/li>\n<li>Set up and train the AutoML estimator in the customer account.<\/li>\n<li>Deploy the top candidate model proposed by AutoML in the customer account.<\/li>\n<li>Invoke the deployed model endpoint for the prediction on test data.<\/li>\n<\/ol>\n<h3>Initiate the user session in the SaaS account<\/h3>\n<p>The setup procedure of IAM entities, explained at the beginning of the post, created the <code>saas_user<\/code>, identified by the <code>saas_user<\/code> profile in the <code>.aws\/credentials<\/code> file. We initiate a Boto3 session with this profile:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">saas_user_session = boto3.session.Session(profile_name=SAAS_USER_PROFILE, \n                                          region_name=REGION)<\/code><\/pre>\n<\/p><\/div>\n<p>The <code>saas_user<\/code> inherits from the <code>AutopilotUsers<\/code> group the permission to assume the <code>customer_trusting_saas<\/code> role in the customer account.<\/p>\n<h3>Assume the role in the customer account via AWS STS<\/h3>\n<p>AWS STS provides the credentials for a temporary session that is initiated in the customer account:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">saas_sts_client = saas_user_session.client(\"sts\", region_name=REGION)<\/code><\/pre>\n<\/p><\/div>\n<p>The default session duration (the <a href=\"https:\/\/docs.aws.amazon.com\/STS\/latest\/APIReference\/API_AssumeRole.html\" target=\"_blank\" rel=\"noopener noreferrer\">DurationSeconds<\/a> parameter) is 1 hour. We set it to the maximum duration session value set for the role. If the session expires, you can recreate it by performing the following steps again. See the following code:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">assumed_role_object = saas_sts_client.assume_role(RoleArn=CUSTOMER_TRUST_SAAS_ROLE_ARN,\n                                                  RoleSessionName=\"sagemaker_autopilot\",\n                                                  ExternalId=EXTERNAL_ID,\n                                                  DurationSeconds=MAX_SESSION_DURATION)\n\nassumed_role_credentials = assumed_role_object[\"Credentials\"]\n\t\t\t \nassumed_role_session = boto3.Session(aws_access_key_id=assumed_role_credentials[\"AccessKeyId\"],\n                                     aws_secret_access_key=assumed_role_credentials[\"SecretAccessKey\"],\n                                     aws_session_token=assumed_role_credentials[\"SessionToken\"],\n                                     region_name=REGION)\n\t\t\t\t\t\t\t\t\t \nsagemaker_session = sagemaker.Session(boto_session=assumed_role_session)<\/code><\/pre>\n<\/p><\/div>\n<p>The <code>sagemaker_session<\/code> parameter is needed for using the high-level AutoML estimator.<\/p>\n<h3>Set up and train the AutoML estimator in the customer account<\/h3>\n<p>We use the AutoML estimator from the SageMaker Python SDK to invoke the Autopilot job to train a set of candidate models for the training data.<\/p>\n<p>The setup of the AutoML object is similar to the single-account scenario, but with the following differences for the cross-account invocation:<\/p>\n<ul>\n<li>The role for SageMaker access in the customer account is <code>CUSTOMER_TRUST_SAAS_ROLE_ARN<\/code><\/li>\n<li>The <code>sagemaker_session<\/code> is the temporary session created by AWS STS<\/li>\n<\/ul>\n<p>See the following code:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">target_attribute_name = \"Churn?\"\n\nfrom sagemaker import AutoML\nfrom time import gmtime, strftime, sleep\n\ntimestamp_suffix = strftime(\"%d-%H-%M-%S\", gmtime())\nbase_job_name = \"automl-churn-sdk-\" + timestamp_suffix\n\ntarget_attribute_name = \"Churn?\"\ntarget_attribute_values = np.unique(train_data[target_attribute_name])\ntarget_attribute_true_value = target_attribute_values[1] # 'True.'\n\nautoml = AutoML(role=CUSTOMER_TRUST_SAAS_ROLE_ARN,\n                target_attribute_name=target_attribute_name,\n                base_job_name=base_job_name,\n                sagemaker_session=sagemaker_session,\n                max_candidates=10)<\/code><\/pre>\n<\/p><\/div>\n<p>We now launch the Autopilot job by calling the fit method of the AutoML estimator in the same way as in the single-account example. We consider the following alternative options for providing the training dataset to the estimator.<\/p>\n<h4>First option: upload a local file and train by fit method<\/h4>\n<p>We simply pass the training dataset by referring to the local file that the fit method uploads into the default Amazon S3 bucket used by SageMaker in the customer account:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">automl.fit(train_file, job_name=base_job_name, wait=False, logs=False)<\/code><\/pre>\n<\/p><\/div>\n<h4>Second option: cross-account copy<\/h4>\n<p>Most likely, the training dataset is located in an Amazon S3 bucket owned by the SaaS account. We copy the dataset from the SaaS account into the customer account and refer to the URI of the copy in the fit method.<\/p>\n<ol>\n<li>Upload the dataset into a local bucket of the SaaS account. For convenience, we use the SageMaker default bucket in the Region.\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">DATA_PREFIX = \"auto-ml-input-data\"\nlocal_session = sagemaker.Session(boto_session=saas_user_session)\nlocal_session_bucket = local_session.default_bucket()\ntrain_data_s3_path = local_session.upload_data(path=train_file,key_prefix=DATA_PREFIX)<\/code><\/pre>\n<\/p><\/div>\n<\/li>\n<li>To allow the cross-account copy, we set the following policy in the local bucket, only for the time needed for the copy operation:\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">train_data_s3_arn = \"arn:aws:s3:::{}\/{}\/{}\".format(local_session_bucket,DATA_PREFIX,train_file)\nbucket_policy = {\n    \"Version\": \"2012-10-17\",\n    \"Statement\": [\n        {\n            \"Effect\": \"Allow\",\n            \"Principal\": {\n                \"AWS\": CUSTOMER_TRUST_SAAS_ROLE_ARN\n            },\n            \"Action\": \"s3:GetObject\",\n            \"Resource\": train_data_s3_arn\n        }\n    ]\n}\nbucket_policy = json.dumps(bucket_policy)\n\nsaas_s3_client = saas_user_session.client(\"s3\")\nsaas_s3_client.put_bucket_policy(Bucket=local_session_bucket,Policy=bucket_policy)<\/code><\/pre>\n<\/p><\/div>\n<\/li>\n<li>Then the copy is performed by the assumed role in the customer account:\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">assumed_role_s3_client = boto3.client(\"s3\",\n                                       aws_access_key_id=assumed_role_credentials[\"AccessKeyId\"],\n                                       aws_secret_access_key=assumed_role_credentials[\"SecretAccessKey\"],\n                                       aws_session_token=assumed_role_credentials[\"SessionToken\"])\ntarget_train_key = \"{}\/{}\".format(DATA_PREFIX, train_file)\nassumed_role_s3_client.copy_object(Bucket=sagemaker_session.default_bucket(), \n                                   CopySource=train_data_s3_path.split(\":\/\/\")[1], \n                                   Key=target_train_key)<\/code><\/pre>\n<\/p><\/div>\n<\/li>\n<li>Delete the bucket policy so that the access has been granted only for the time of the copy:\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">saas_s3_client.delete_bucket_policy(Bucket=local_session_bucket)<\/code><\/pre>\n<\/p><\/div>\n<\/li>\n<li>Finally, we launch the Autopilot job, passing the URI of the object copy:<\/li>\n<\/ol>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">target_train_uri = \"s3:\/\/{}\/{}\".format(sagemaker_session.default_bucket(), \n                                       target_train_key)\nautoml.fit(target_train_uri, job_name=base_job_name, wait=False, logs=False)<\/code><\/pre>\n<\/p><\/div>\n<p>Another option is to refer to the URI of the source dataset in the bucket in SaaS account. In this case, the bucket policy should include the <code>s3:ListBucket<\/code> action for the source bucket.<\/p>\n<p>The bucket policy should be assigned for the duration of all the training and allow the <code>s3:ListBucket<\/code> action for the source bucket, including a statement like the following:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">{\n  \"Effect\": \"Allow\",\n  \"Principal\": {\n     \"AWS\": \"arn:aws:iam::CUSTOMER_ACCOUNT_ID:role\/customer_trusting_saas\"\n  },\n  \"Action\": \"s3:ListBucket\",\n  \"Resource\": \"arn:aws:s3:::sagemaker-REGION-SAAS_ACCOUNT_ID\"\n}<\/code><\/pre>\n<\/p><\/div>\n<p>We can use the <code>describe_auto_ml_job<\/code> method to track the status of our SageMaker Autopilot job:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">describe_response = automl.describe_auto_ml_job()\nprint (describe_response[\"AutoMLJobStatus\"] + \" - \" + describe_response[\"AutoMLJobSecondaryStatus\"])\njob_run_status = describe_response[\"AutoMLJobStatus\"]\n\nwhile job_run_status not in (\"Failed\", \"Completed\", \"Stopped\"):\n    describe_response = automl.describe_auto_ml_job()\n    job_run_status = describe_response[\"AutoMLJobStatus\"]\n    \n    print(describe_response[\"AutoMLJobStatus\"] + \" - \" + describe_response[\"AutoMLJobSecondaryStatus\"])\n    sleep(30)<\/code><\/pre>\n<\/p><\/div>\n<p>Because an Autopilot job can take a long time, if the session token expires during the fit, you can create a new session following the steps described earlier and retrieve the current Autopilot job reference by implementing the following code:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">automl = AutoML.attach(auto_ml_job_name=base_job_name,sagemaker_session=sagemaker_session)<\/code><\/pre>\n<\/p><\/div>\n<h3>Deploy the top candidate model proposed by AutoML<\/h3>\n<p>The Autopilot job trains and returns a set of trained candidate models, identifying among them the top candidate that optimizes the evaluation metric related to the <a href=\"https:\/\/docs.aws.amazon.com\/sagemaker\/latest\/dg\/autopilot-problem-types.html\" target=\"_blank\" rel=\"noopener noreferrer\">ML problem<\/a>.<\/p>\n<p>In this post, we only demonstrate the deployment of the top candidate proposed by AutoML, but you can choose a different candidate that better fits your business criteria.<\/p>\n<p>First, we review the performance achieved by the top candidate in the cross-validation:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">best_candidate = automl.describe_auto_ml_job()[\"BestCandidate\"]\nbest_candidate_name = best_candidate[\"CandidateName\"]\nprint(\"n\")\nprint(\"CandidateName: \" + best_candidate_name)\nprint(\"FinalAutoMLJobObjectiveMetricName: \" + best_candidate[\"FinalAutoMLJobObjectiveMetric\"][\"MetricName\"])\nprint(\"FinalAutoMLJobObjectiveMetricValue: \" + str(best_candidate[\"FinalAutoMLJobObjectiveMetric\"][\"Value\"]))<\/code><\/pre>\n<\/p><\/div>\n<p>If the performance is good enough for our business criteria, we deploy the top candidate in the customer account:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">from sagemaker.predictor import Predictor\nfrom sagemaker.serializers import CSVSerializer\nfrom sagemaker.deserializers import CSVDeserializer\n\ninference_response_keys = [\"predicted_label\", \"probability\"]\n\npredictor = automl.deploy(initial_instance_count=1,\n                          instance_type=\"ml.m5.large\",\n                          inference_response_keys=inference_response_keys,\n                          predictor_cls=Predictor,\n                          serializer=CSVSerializer(),\n                          deserializer=CSVDeserializer())\n\nprint(\"Created endpoint: {}\".format(predictor.endpoint_name))<\/code><\/pre>\n<\/p><\/div>\n<p>The instance is deployed and billed to the customer account.<\/p>\n<h3>Prediction on test data<\/h3>\n<p>Finally, we access the model endpoint for the prediction of the label output for the test data:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">predictor.predict(test_data_no_target.to_csv(sep=\",\", \n                                             header=False, \n                                             index=False))<\/code><\/pre>\n<\/p><\/div>\n<p>If the session token expires after the deployment of the endpoint, you can recreate a new session following the steps described earlier and connect to the already deployed endpoint by implementing the following code:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">predictor = Predictor(predictor.endpoint_name, \n                      sagemaker_session = sagemaker_session,\n                      serializer=CSVSerializer(), \n                      deserializer=CSVDeserializer())<\/code><\/pre>\n<\/p><\/div>\n<h2>Clean up<\/h2>\n<p>To avoid incurring unnecessary charges, delete the endpoints and resources that were created when deploying the model after they are no longer needed.<\/p>\n<h3>Delete the model endpoint<\/h3>\n<p>The model endpoint is deployed in a container that is always active. We delete it first to avoid consumption of credits:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">predictor.delete_endpoint()<\/code><\/pre>\n<\/p><\/div>\n<h3>Delete the artifacts generated by the Autopilot job<\/h3>\n<p>Delete all the artifacts created by the Autopilot job, such as the generated candidate models, scripts, and notebook.<\/p>\n<p>We use the <a href=\"https:\/\/boto3.amazonaws.com\/v1\/documentation\/api\/latest\/guide\/resources.html\" target=\"_blank\" rel=\"noopener noreferrer\">high-level resource for Amazon S3<\/a> to simplify the operation:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">assumed_role_s3_resource = boto3.resource(\"s3\",\n                                          aws_access_key_id=assumed_role_credentials[\"AccessKeyId\"],\n                                          aws_secret_access_key=assumed_role_credentials[\"SecretAccessKey\"],\n                                          aws_session_token=assumed_role_credentials[\"SessionToken\"])\n\ns3_bucket = assumed_role_s3_resource.Bucket(automl.sagemaker_session.default_bucket())\ns3_bucket.objects.filter(Prefix=base_job_name).delete()<\/code><\/pre>\n<\/p><\/div>\n<h3>Delete the training dataset copied into the customer account<\/h3>\n<p>Delete the training dataset in the customer account with the following code:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">from urllib.parse import urlparse\n\ntrain_data_uri = automl.describe_auto_ml_job()[\"InputDataConfig\"][0][ \"DataSource\"][\"S3DataSource\"][\"S3Uri\"]\n\no = urlparse(train_data_uri, allow_fragments=False)\nassumed_role_s3_resource.Object(o.netloc, o.path.lstrip(\"\/\")).delete()<\/code><\/pre>\n<\/p><\/div>\n<h3>Clean up IAM resources<\/h3>\n<p>We delete the IAM resources in reverse order to the creation phase.<\/p>\n<ol>\n<li>Remove the user from the group, and the profile from the credentials, and delete the user:\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">saas_iam_client.remove_user_from_group(GroupName = SAAS_USER_GROUP_NAME,\n                                       UserName = SAAS_USER_NAME)\n                                      \ncredentials_config.remove_section(SAAS_USER_PROFILE)\nwith open(str(Path.home()) + \"\/.aws\/credentials\", \"w\") as configfile:\n    credentials_config.write(configfile, space_around_delimiters=False)\n    \nuser_access_keys = saas_iam_client.list_access_keys(UserName=SAAS_USER_NAME)\nfor AccessKeyId in [element[\"AccessKeyId\"] for element in user_access_keys[\"AccessKeyMetadata\"]]:\n    saas_iam_client.delete_access_key(UserName=SAAS_USER_NAME, AccessKeyId=AccessKeyId)\n\t\nsaas_iam_client.delete_user(UserName=SAAS_USER_NAME)<\/code><\/pre>\n<\/p><\/div>\n<\/li>\n<li>Detach the policies from the group in the SaaS account, and delete the group and policies:\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">attached_group_policies = saas_iam_client.list_attached_group_policies(GroupName=SAAS_USER_GROUP_NAME)\nfor PolicyArn in [element[\"PolicyArn\"] for element in attached_group_policies[\"AttachedPolicies\"]]:\n    saas_iam_client.detach_group_policy(GroupName=SAAS_USER_GROUP_NAME, PolicyArn=PolicyArn)\n    \nsaas_iam_client.delete_group(GroupName=SAAS_USER_GROUP_NAME)\nsaas_iam_client.delete_policy(PolicyArn=saas_assume_role_policy_arn)\nsaas_iam_client.delete_policy(PolicyArn=saas_s3_policy_arn)<\/code><\/pre>\n<\/p><\/div>\n<\/li>\n<li>Detach the AWS policies from the role in the customer account, then delete the role and the policy:\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">attached_role_policies = customer_iam_client.list_attached_role_policies(RoleName=CUSTOMER_TRUST_SAAS_ROLE_NAME)\nfor PolicyArn in [element[\"PolicyArn\"] for element in attached_role_policies[\"AttachedPolicies\"]]:\n    customer_iam_client.detach_role_policy(RoleName=CUSTOMER_TRUST_SAAS_ROLE_NAME, PolicyArn=PolicyArn)\n\ncustomer_iam_client.delete_role(RoleName=CUSTOMER_TRUST_SAAS_ROLE_NAME)\ncustomer_iam_client.delete_policy(PolicyArn=customer_s3_policy_arn)<\/code><\/pre>\n<\/p><\/div>\n<\/li>\n<\/ol>\n<h2>Conclusion<\/h2>\n<p>This post described a possible implementation, using the SageMaker Python SDK, of an Autopilot training job, model deployment, and prediction in a cross-account configuration. The originating account owns the data for the training and it delegates the activities to the account hosting the SageMaker resources.<\/p>\n<p>You can use the API calls shown in this post to incorporate AutoML capabilities into a SaaS application, by delegating the management and billing of SageMaker resources to the customer account.<\/p>\n<p>SageMaker decouples the environment where the data scientist drives the analysis from the containers that perform each phase of the ML process.<\/p>\n<p>This capability simplifies other cross-account scenarios. For example: a SaaS provider who owns sensitive data, instead of sharing its data with the customer, could expose certified training algorithms and generate models on behalf of the customer. The customer will receive the trained model at the end of the Autopilot job.<\/p>\n<p>For more examples of how to integrate Autopilot into SaaS products, see the following posts:<\/p>\n<hr>\n<h3>About the Authors<\/h3>\n<p><strong><a href=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2021\/11\/17\/Francesco-Polimeni2.png\"><img decoding=\"async\" loading=\"lazy\" class=\"size-full wp-image-30952 alignleft\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2021\/11\/17\/Francesco-Polimeni2.png\" alt=\"\" width=\"100\" height=\"125\"><\/a> Francesco Polimeni<\/strong> is a Sr Solutions Architect at AWS with focus on Machine Learning. He has over 20 years of experience in professional services and pre-sales organizations for IT management software solutions.<\/p>\n<p><strong><a href=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2021\/11\/17\/Mehmet-Bakkaloglu.jpg\"><img decoding=\"async\" loading=\"lazy\" class=\"size-full wp-image-30953 alignleft\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2021\/11\/17\/Mehmet-Bakkaloglu.jpg\" alt=\"\" width=\"100\" height=\"144\"><\/a>Mehmet Bakkaloglu<\/strong> is a Sr Solutions Architect at AWS. He has vast experience in data analytics and cloud architecture, having provided technical leadership for transformation programs and pre-sales activities in a variety of sectors.<\/p>\n<p>       <!-- '\"` -->\n      <\/div>\n","protected":false},"excerpt":{"rendered":"<p>https:\/\/aws.amazon.com\/blogs\/machine-learning\/add-automl-functionality-with-amazon-sagemaker-autopilot-across-accounts\/<\/p>\n","protected":false},"author":0,"featured_media":1374,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[],"_links":{"self":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/1373"}],"collection":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/comments?post=1373"}],"version-history":[{"count":0,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/1373\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media\/1374"}],"wp:attachment":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media?parent=1373"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/categories?post=1373"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/tags?post=1373"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}