{"id":1017,"date":"2021-10-12T08:39:31","date_gmt":"2021-10-12T08:39:31","guid":{"rendered":"https:\/\/salarydistribution.com\/machine-learning\/2021\/10\/12\/build-tune-and-deploy-an-end-to-end-churn-prediction-model-using-amazon-sagemaker-pipelines\/"},"modified":"2021-10-12T08:39:31","modified_gmt":"2021-10-12T08:39:31","slug":"build-tune-and-deploy-an-end-to-end-churn-prediction-model-using-amazon-sagemaker-pipelines","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2021\/10\/12\/build-tune-and-deploy-an-end-to-end-churn-prediction-model-using-amazon-sagemaker-pipelines\/","title":{"rendered":"Build, tune, and deploy an end-to-end churn prediction model using Amazon SageMaker Pipelines"},"content":{"rendered":"

The ability to predict that a particular customer is at a high risk of churning, while there is still time to do something about it, represents a huge potential revenue source for every online business. Depending on the industry and business objective, the problem statement can be multi-layered. The following are some business objectives based on this strategy:<\/p>\n

This post discusses how you can orchestrate an end-to-end churn prediction model across each step: data preparation, experimenting with a baseline model and hyperparameter optimization (HPO), training and tuning, and registering the best model. You can manage your Amazon SageMaker<\/a> training and inference workflows using Amazon SageMaker Studio<\/a> and the SageMaker Python SDK. SageMaker offers all the tools you need to create high-quality data science solutions.<\/p>\n

SageMaker helps data scientists and developers prepare, build, train, and deploy high-quality machine learning (ML) models quickly by bringing together a broad set of capabilities purpose-built for ML.<\/p>\n

Studio provides a single, web-based visual interface where you can perform all ML development steps, improving data science team productivity by up to 10 times.<\/p>\n

Amazon SageMaker Pipelines<\/a> is a tool for building ML pipelines that takes advantage of direct SageMaker integration. With Pipelines, you can easily automate the steps of building a ML model, catalog models in the model registry, and use one of several templates provided in SageMaker Projects to set up continuous integration and continuous delivery (CI\/CD) for the end-to-end ML lifecycle at scale.<\/p>\n

After the model is trained, you can use Amazon SageMaker Clarify<\/a> to identify and limit bias and explain predictions to business stakeholders. You can share these automated reports with business and technical teams for downstream target campaigns or to determine features that are key differentiators for customer lifetime value.<\/p>\n

By the end of this post, you should have enough information to successfully use this end-to-end template using Pipelines to train, tune, and deploy your own predictive analytics use case. The full instructions are available on the GitHub repo<\/a>.<\/p>\n

In this solution, your entry point is the Studio integrated development environment (IDE) for rapid experimentation. Studio offers an environment to manage the end-to-end Pipelines experience. With Studio, you can bypass the AWS Management Console<\/a> for your entire workflow management. For more information on managing Pipelines from Studio, see View, Track, and Execute SageMaker Pipelines in SageMaker Studio<\/a>.<\/p>\n

The following diagram illustrates the high-level architecture of the data science workflow.<\/p>\n

After you create the Studio domain, select your user name and choose Open Studio<\/strong>. A web-based IDE opens that allows you to store and collect all the things that you need\u2014whether it\u2019s code, notebooks, datasets, settings, or project folders.<\/p>\n

Pipelines is integrated directly with SageMaker, so you don\u2019t need to interact with any other AWS services. You also don\u2019t need to manage any resources because Pipelines is a fully managed service, which means that it creates and manages resources for you. For more information the various SageMaker components that are both standalone Python APIs along with integrated components of Studio, see the SageMaker service page<\/a>.<\/p>\n

For this use case, you use the following components for the fully automated model development process:<\/p>\n

A SageMaker pipeline is a series of interconnected steps that is defined by a JSON pipeline definition. This pipeline definition encodes a pipeline using a directed acyclic graph (DAG). This DAG gives information on the requirements for and relationships between each step of your pipeline. The structure of a pipeline\u2019s DAG is determined by the data dependencies between steps. These data dependencies are created when the properties of a step\u2019s output are passed as the input to another step.<\/p>\n

For this post, our use case is a classic ML problem that aims to understand what various marketing strategies based on consumer behavior we can adopt to increase customer retention for a given retail store. The following diagram illustrates the complete ML workflow for the churn prediction use case.<\/p>\n

Let\u2019s go through the accelerated ML workflow development process in detail.<\/p>\n

To follow along with this post, you need to download and save the sample dataset<\/a> in the default Amazon Simple Storage Service<\/a> (Amazon S3) bucket associated with your SageMaker session, and in the S3 bucket of your choice. For rapid experimentation or baseline model building, you can save a copy of the dataset under your home directory in Amazon Elastic File System<\/a> (Amazon EFS) and follow the Jupyter notebook Customer_Churn_Modeling.ipynb<\/code>.<\/p>\n

The following screenshot shows the sample set with the target variable as retained 1, if customer is assumed to be active, or 0 otherwise.<\/p>\n

Run the following code in a Studio notebook to preprocess the dataset and upload it to your own S3 bucket:<\/p>\n

\n
import boto3\nimport pandas as pd\nimport numpy as np\n\n## Preprocess the dataset\ndef preprocess_data(file_path):\u00a0 \n  df = pd.read_csv(file_path)\n  ## Convert to datetime columns\n  df[\"firstorder\"]=pd.to_datetime(df[\"firstorder\"],errors='coerce')\n  df[\"lastorder\"] = pd.to_datetime(df[\"lastorder\"],errors='coerce')\n  ## Drop Rows with null values\n  df = df.dropna()\u00a0\u00a0\u00a0 \n  ## Create Column which gives the days between the last order and the first order\n  df[\"first_last_days_diff\"] = (df['lastorder']-df['firstorder']).dt.days\n  ## Create Column which gives the days between when the customer record was created and the first order\n  df['created'] = pd.to_datetime(df['created'])\n  df['created_first_days_diff']=(df['created']-df['firstorder']).dt.days\n  ## Drop Columns\n  df.drop(['custid','created','firstorder','lastorder'],axis=1,inplace=True)\n  ## Apply one hot encoding on favday and city columns\n  df = pd.get_dummies(df,prefix=['favday','city'],columns=['favday','city'])\n  return df\n\n## Set the required configurations\nmodel_name = \"churn_model\"\nenv = \"dev\"\n## S3 Bucket\ndefault_bucket = \"customer-churn-sm-pipeline\"\n## Preprocess the dataset\nstoredata = preprocess_data(f\"s3:\/\/{default_bucket}\/data\/storedata_total.csv\")<\/code><\/pre>\n<\/p><\/div>\n
With Studio notebooks with elastic compute, you can now easily run multiple training and tuning jobs. For this use case, you use the SageMaker built-in XGBoost algorithm and SageMaker HPO with objective function as \"binary:logistic\"<\/code> and \"eval_metric\":\"auc\"<\/code>.<\/p>\n
\n
def split_datasets(df):\n    y=df.pop(\"retained\")\n    X_pre = df\n    y_pre = y.to_numpy().reshape(len(y),1)\n    feature_names = list(X_pre.columns)\n    X= np.concatenate((y_pre,X_pre),axis=1)\n    np.random.shuffle(X)\n    train,validation,test=np.split(X,[int(.7*len(X)),int(.85*len(X))])\n    return feature_names,train,validation,test\n\n# Split dataset\nfeature_names,train,validation,test = split_datasets(storedata)\n\n# Save datasets in Amazon S3\npd.DataFrame(train).to_csv(f\"s3:\/\/{default_bucket}\/data\/train\/train.csv\",header=False,index=False)\npd.DataFrame(validation).to_csv(f\"s3:\/\/{default_bucket}\/data\/validation\/validation.csv\",header=False,index=False)\npd.DataFrame(test).to_csv(f\"s3:\/\/{default_bucket}\/data\/test\/test.csv\",header=False,index=False)\n<\/code><\/pre>\n
Train, tune, and find the best candidate model with the following code:<\/p>\n
\n
# Training and Validation Input for SageMaker Training job\ns3_input_train = TrainingInput(\n    s3_data=f\"s3:\/\/{default_bucket}\/data\/train\/\",content_type=\"csv\")\ns3_input_validation = TrainingInput(\n    s3_data=f\"s3:\/\/{default_bucket}\/data\/validation\/\",content_type=\"csv\")\n\n# Hyperparameter used\nfixed_hyperparameters = {\n    \"eval_metric\":\"auc\",\n    \"objective\":\"binary:logistic\",\n    \"num_round\":\"100\",\n    \"rate_drop\":\"0.3\",\n    \"tweedie_variance_power\":\"1.4\"\n}\n\n# Use the built-in SageMaker algorithm\n\nsess = sagemaker.Session()\ncontainer = sagemaker.image_uris.retrieve(\"xgboost\",region,\"0.90-2\")\n\nestimator = sagemaker.estimator.Estimator(\n    container,\n    role,\n    instance_count=1,\n    hyperparameters=fixed_hyperparameters,\n    instance_type=\"ml.m4.xlarge\",\n    output_path=\"s3:\/\/{}\/output\".format(default_bucket),\n    sagemaker_session=sagemaker_session\n)\n\nhyperparameter_ranges = {\n    \"eta\": ContinuousParameter(0, 1),\n    \"min_child_weight\": ContinuousParameter(1, 10),\n    \"alpha\": ContinuousParameter(0, 2),\n    \"max_depth\": IntegerParameter(1, 10),\n}\nobjective_metric_name = \"validation:auc\"\ntuner = HyperparameterTuner(\nestimator, objective_metric_name,\nhyperparameter_ranges,max_jobs=10,max_parallel_jobs=2)\n\n# Tune\ntuner.fit({\n    \"train\":s3_input_train,\n    \"validation\":s3_input_validation\n    },include_cls_metadata=False)\n\n## Explore the best model generated\ntuning_job_result = boto3.client(\"sagemaker\").describe_hyper_parameter_tuning_job(\n    HyperParameterTuningJobName=tuner.latest_tuning_job.job_name\n)\n\njob_count = tuning_job_result[\"TrainingJobStatusCounters\"][\"Completed\"]\nprint(\"%d training jobs have completed\" %job_count)\n## 10 training jobs have completed\n\n## Get the best training job\n\nfrom pprint import pprint\nif tuning_job_result.get(\"BestTrainingJob\",None):\n    print(\"Best Model found so far:\")\n    pprint(tuning_job_result[\"BestTrainingJob\"])\nelse:\n    print(\"No training jobs have reported results yet.\")\n<\/code><\/pre>\n<\/p><\/div>\n
\"ML-4931-HyperParameterTuning\"<\/p>\n
After you establish a baseline, you can use Amazon SageMaker Debugger<\/a> for offline model analysis. Debugger is a capability within SageMaker that automatically provides visibility into the model training process for real-time and offline analysis. Debugger saves the internal model state at periodic intervals, which you can analyze in real time during training and offline after the training is complete. For this use case, you use the explainability tool SHAP (SHapley Additive exPlanation) and the native integration of SHAP with Debugger. Refer to the following notebook<\/a> for detailed analysis.<\/p>\n
The following summary plot explains the positive and negative relationships of the predictors with the target variable. For example, the top variable here, esent<\/code>, is defined as number of emails sent. This plot is made of all data points in the training set. Blue indicates dragging the final output to class 0, and pink represents class 1. Key influencing features are ranked in descending order.<\/p>\n
\"ML-4931-Shapley-Plots\"<\/p>\n
Now you can proceed with the deploy and manage step of the ML workflow.<\/p>\n
Develop and automate the workflow<\/h2>\n
Let\u2019s start with the project structure:<\/p>\n