{"id":578,"date":"2020-11-21T02:55:37","date_gmt":"2020-11-21T02:55:37","guid":{"rendered":"https:\/\/machine-learning.webcloning.com\/2020\/11\/21\/getting-started-with-amazon-kendra-servicenow-online-connector\/"},"modified":"2020-11-21T02:55:37","modified_gmt":"2020-11-21T02:55:37","slug":"getting-started-with-amazon-kendra-servicenow-online-connector","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2020\/11\/21\/getting-started-with-amazon-kendra-servicenow-online-connector\/","title":{"rendered":"Getting started with Amazon Kendra ServiceNow Online connector"},"content":{"rendered":"<div id=\"\">\n<p><a href=\"https:\/\/aws.amazon.com\/kendra\/\" target=\"_blank\" rel=\"noopener noreferrer\">Amazon Kendra<\/a> is a highly accurate and easy-to-use intelligent search service powered by machine learning (ML). To make it simple to search data across multiple content repositories, Amazon Kendra offers a number of native data source connectors to help get your documents easily ingested and indexed.<\/p>\n<p>This post describes how you can use the Amazon Kendra ServiceNow connector. To allow the connector to access your ServiceNow site, you need to know your ServiceNow version, the Amazon Kendra index, the ServiceNow host URL, and the credentials of a user with the ServiceNow admin role attached to it. The ServiceNow credentials needed for the Amazon Kendra ServiceNow connector to work are securely stored in <a href=\"https:\/\/aws.amazon.com\/secrets-manager\/\" target=\"_blank\" rel=\"noopener noreferrer\">AWS Secrets Manager<\/a>, and can be entered during the connector setup.<\/p>\n<p>Currently, Amazon Kendra has two provisioning editions: the Amazon Kendra Developer Edition for building proof of concepts (POCs), and the Amazon Kendra Enterprise Edition. Amazon Kendra connectors work with both these editions.<\/p>\n<p>The Amazon Kendra ServiceNow Online connector indexes Service Catalog items and public articles that have a published state, so a knowledge base article must have the public role under <strong>Can Read<\/strong>, and <strong>Cannot Read<\/strong> must be null or not set.<\/p>\n<h2>Prerequisites<\/h2>\n<p>To get started, you need the following:<\/p>\n<ul>\n<li>The ServiceNow host URL<\/li>\n<li>Username and Password of a user with the admin role<\/li>\n<li>Know your ServiceNow version<\/li>\n<\/ul>\n<p>The user that you use for the connector needs to have the admin role in ServiceNow. This is defined on ServiceNow\u2019s <strong>User Administration<\/strong> page (see the section <strong>Insufficient Permissions <\/strong>for more information).<\/p>\n<p>When setting up the ServiceNow connector, we need to define if our build is London or a different ServiceNow version. To obtain our build name, we can go on the <strong>System Diagnostics<\/strong> menu and choose <strong>Stats<\/strong>.<\/p>\n<p>In the following screenshot, my build name is Orlando, so I indicate on the connector that my version is <strong>Others<\/strong>.<\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-18719 size-full\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/11\/20\/Screenshot.png\" alt=\"\" width=\"800\" height=\"228\"><\/p>\n<h2>Creating a ServiceNow connector in the Amazon Kendra console<\/h2>\n<p>The following section describes the process of deploying an Amazon Kendra index and configuring a ServiceNow connector.<\/p>\n<ol>\n<li>Create your index. For instructions, see <a href=\"https:\/\/aws.amazon.com\/blogs\/machine-learning\/getting-started-with-the-amazon-kendra-sharepoint-online-connector\/\" target=\"_blank\" rel=\"noopener noreferrer\">Getting started with the Amazon Kendra SharePoint Online connector<\/a>.<\/li>\n<\/ol>\n<p>If you already have an index, you can skip this step.<\/p>\n<p>The next step is to set up the data sources. One of the advantages of implementing Amazon Kendra is that you can use a set of pre-built connectors for data sources, such as <a href=\"http:\/\/aws.amazon.com\/s3\" target=\"_blank\" rel=\"noopener noreferrer\">Amazon Simple Storage Service<\/a> (Amazon S3), <a href=\"http:\/\/aws.amazon.com\/rds\" target=\"_blank\" rel=\"noopener noreferrer\">Amazon Relational Database Service<\/a> (Amazon RDS), Salesforce, ServiceNow, and SharePoint Online, among others.<\/p>\n<p>For this post, we use the ServiceNow connector.<\/p>\n<ol start=\"2\">\n<li>On the Amazon Kendra console, choose <strong>Indexes<\/strong>.<\/li>\n<li>Choose <strong>MyServiceNowIndex<\/strong>.<\/li>\n<li>Choose <strong>Add data sources<\/strong>.<\/li>\n<\/ol>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-18489\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/11\/17\/Amazon-Kendra-2.jpg\" alt=\"\" width=\"798\" height=\"508\"><\/p>\n<ol start=\"5\">\n<li>Choose <strong>ServiceNow Online<\/strong>.<\/li>\n<\/ol>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-18633 size-full\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/11\/19\/Getting-Started-with-Amazon-Kendra.png\" alt=\"\" width=\"800\" height=\"358\"><\/p>\n<ol start=\"6\">\n<li>For <strong>Name<\/strong>, enter a connector name.<\/li>\n<li>For <strong>Description<\/strong>, enter an optional description.<\/li>\n<li>For <strong>Tags<\/strong>\u00b8 you can optionally <a href=\"https:\/\/docs.aws.amazon.com\/general\/latest\/gr\/aws_tagging.html\" target=\"_blank\" rel=\"noopener noreferrer\">assign tags<\/a> to your data source.<\/li>\n<li>Choose <strong>Next<\/strong>.<\/li>\n<\/ol>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-18491\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/11\/17\/Amazon-Kendra-4.jpg\" alt=\"\" width=\"800\" height=\"605\"><\/p>\n<p>In this next step, we define targets.<\/p>\n<ol start=\"10\">\n<li>For <strong>ServiceNow host<\/strong>, enter the host name.<\/li>\n<li>For <strong>ServiceNow version<\/strong>, enter your version (for this post, we choose <strong>Others<\/strong>).<\/li>\n<li>For <strong>IAM role<\/strong>, we can create a new <a href=\"http:\/\/aws.amazon.com\/iam\" target=\"_blank\" rel=\"noopener noreferrer\">AWS Identity and Access Management<\/a> (IAM) role or use an existing one.<\/li>\n<\/ol>\n<p>For more information, see <a href=\"https:\/\/docs.aws.amazon.com\/kendra\/latest\/dg\/iam-roles.html#iam-roles-ds-sn\" target=\"_blank\" rel=\"noopener noreferrer\">IAM role for ServiceNow data sources<\/a>.<\/p>\n<p>This role has four functions:<\/p>\n<p>If you use an existing IAM role, you have to grant permissions to this secret in Secrets Manager. If you create a new IAM and a new secret, no further action is required.<\/p>\n<ol start=\"13\">\n<li>Choose <strong>Next<\/strong>.<\/li>\n<\/ol>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-18492\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/11\/17\/Amazon-Kendra-5.jpg\" alt=\"\" width=\"800\" height=\"618\"><\/p>\n<p>You then need to define ServiceNow authentication details, the content to index, and the synchronization schedule.<\/p>\n<p>The ServiceNow user you provide for the connector needs to have the admin role.<\/p>\n<ol start=\"14\">\n<li>In the <strong>Authentication section<\/strong>, for <strong>Type of authentication<\/strong>, choose an existing secret or create a new one. For this post, we choose <strong>New<\/strong>.<\/li>\n<li>Enter your secret\u2019s name, username, and password.<\/li>\n<\/ol>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-18493\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/11\/17\/Amazon-Kendra-6.jpg\" alt=\"\" width=\"800\" height=\"563\"><\/p>\n<ol start=\"16\">\n<li>In the <strong>ServiceNow configuration<\/strong> section, we define the content types we need to index: <strong>Knowledge articles<\/strong>, <strong>Service catalog items<\/strong>, or both.<\/li>\n<li>You also define if it include the item attachments.<\/li>\n<\/ol>\n<p>Amazon Kendra only indexes public articles that have a published state, so a knowledge base article must have the public role under <strong>Can Read<\/strong>, and <strong>Cannot Read<\/strong> must be null or not set.<\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-18494\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/11\/17\/Amazon-Kendra-7.jpg\" alt=\"\" width=\"333\" height=\"120\"><\/p>\n<ol start=\"18\">\n<li>You can include or exclude some file extensions (for example, for Microsoft Word, we have six different types of extensions).<\/li>\n<\/ol>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-18495\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/11\/17\/Amazon-Kendra-8.jpg\" alt=\"\" width=\"800\" height=\"579\"><\/p>\n<ol start=\"19\">\n<li>For <strong>Frequency<\/strong>, choose <strong>Run on demand<\/strong>.<\/li>\n<\/ol>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-18496\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/11\/17\/Amazon-Kendra-9.jpg\" alt=\"\" width=\"800\" height=\"226\"><\/p>\n<ol start=\"20\">\n<li>Add field mappings.<\/li>\n<\/ol>\n<p>Even though this is an optional step, it\u2019s a good idea to add this extra layer of metadata to our documents from ServiceNow. This metadata enables you to improve accuracy through manual tuning, filtering, and faceting. There is no way to add metadata to already ingested documents, so if you want to add metadata later, you need to delete this data source and recreate a data source with metadata and ingest your documents again.<\/p>\n<p>If you map fields through the console when setting up the ServiceNow connector for the first time, these fields are created automatically. If you configure the connector via the API, you need <a href=\"https:\/\/docs.aws.amazon.com\/kendra\/latest\/dg\/API_UpdateIndex.html\" target=\"_blank\" rel=\"noopener noreferrer\">update your index<\/a> first and define those new fields.<\/p>\n<p>You can <a href=\"https:\/\/docs.aws.amazon.com\/kendra\/latest\/dg\/data-source-servicenow.html\" target=\"_blank\" rel=\"noopener noreferrer\">map ServiceNow properties to Amazon Kendra index fields<\/a>. The following table is the list of fields that we can map.<\/p>\n<table border=\"1px\" cellpadding=\"5px\">\n<tbody>\n<tr>\n<td width=\"150\"><strong>ServiceNow Field Name<\/strong><\/td>\n<td width=\"236\"><strong>Suggested Amazon Kendra Field Name<\/strong><\/td>\n<\/tr>\n<tr>\n<td width=\"150\">content<\/td>\n<td width=\"236\">_document_body<\/td>\n<\/tr>\n<tr>\n<td width=\"150\">displayUrl<\/td>\n<td width=\"236\">sn_display_url<\/td>\n<\/tr>\n<tr>\n<td width=\"150\">first_name<\/td>\n<td width=\"236\">sn_ka_first_name<\/td>\n<\/tr>\n<tr>\n<td width=\"150\">kb_category<\/td>\n<td width=\"236\">sn_ka_category<\/td>\n<\/tr>\n<tr>\n<td width=\"150\">kb_catagory_name<\/td>\n<td width=\"236\">_category<\/td>\n<\/tr>\n<tr>\n<td width=\"150\">kb_knowledge_base<\/td>\n<td width=\"236\">sn_ka_knowledge_base<\/td>\n<\/tr>\n<tr>\n<td width=\"150\">last_name<\/td>\n<td width=\"236\">sn_ka_last_name<\/td>\n<\/tr>\n<tr>\n<td width=\"150\">number<\/td>\n<td width=\"236\">sn_kb_number<\/td>\n<\/tr>\n<tr>\n<td width=\"150\">published<\/td>\n<td width=\"236\">sn_ka_publish_date<\/td>\n<\/tr>\n<tr>\n<td width=\"150\">repItemType<\/td>\n<td width=\"236\">sn_repItemType<\/td>\n<\/tr>\n<tr>\n<td width=\"150\">short_description<\/td>\n<td width=\"236\">_document_title<\/td>\n<\/tr>\n<tr>\n<td width=\"150\">sys_created_by<\/td>\n<td width=\"236\">sn_createdBy<\/td>\n<\/tr>\n<tr>\n<td width=\"150\">sys_created_on<\/td>\n<td width=\"236\">_created_at<\/td>\n<\/tr>\n<tr>\n<td width=\"150\">sys_id<\/td>\n<td width=\"236\">sn_sys_id<\/td>\n<\/tr>\n<tr>\n<td width=\"150\">sys_updated_by<\/td>\n<td width=\"236\">sn_updatedBy<\/td>\n<\/tr>\n<tr>\n<td width=\"150\">sys_updated_on<\/td>\n<td width=\"236\">_last_updated_at<\/td>\n<\/tr>\n<tr>\n<td width=\"150\">url<\/td>\n<td width=\"236\">sn_url<\/td>\n<\/tr>\n<tr>\n<td width=\"150\">user_name<\/td>\n<td width=\"236\">sn_ka_user_name<\/td>\n<\/tr>\n<tr>\n<td width=\"150\">valid_to<\/td>\n<td width=\"236\">sn_ka_valid_to<\/td>\n<\/tr>\n<tr>\n<td width=\"150\">workflow_state<\/td>\n<td width=\"236\">sn_ka_workflow_state<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>Even though there are suggested Kendra field names you can define, you can map a field into a different name.<\/p>\n<p>The following table summarizes the available service catalog fields.<\/p>\n<table border=\"1px\" cellpadding=\"5px\">\n<tbody>\n<tr>\n<td width=\"146\"><strong>ServiceNow Field Name<\/strong><\/td>\n<td width=\"236\"><strong>Suggested Amazon Kendra Field Name<\/strong><\/td>\n<\/tr>\n<tr>\n<td width=\"146\">category<\/td>\n<td width=\"236\">sn_sc_category<\/td>\n<\/tr>\n<tr>\n<td width=\"146\">category_full_name<\/td>\n<td width=\"236\">sn_sc_category_full_name<\/td>\n<\/tr>\n<tr>\n<td width=\"146\">category_name<\/td>\n<td width=\"236\">_category<\/td>\n<\/tr>\n<tr>\n<td width=\"146\">description<\/td>\n<td width=\"236\">_document_body<\/td>\n<\/tr>\n<tr>\n<td width=\"146\">displayUrl<\/td>\n<td width=\"236\">sn_display_url<\/td>\n<\/tr>\n<tr>\n<td width=\"146\">repItemType<\/td>\n<td width=\"236\">sn_repItemType<\/td>\n<\/tr>\n<tr>\n<td width=\"146\">sc_catalogs<\/td>\n<td width=\"236\">sn_sc_catalogs<\/td>\n<\/tr>\n<tr>\n<td width=\"146\">sc_catalogs_name<\/td>\n<td width=\"236\">sn_sc_catalogs_name<\/td>\n<\/tr>\n<tr>\n<td width=\"146\">short_description<\/td>\n<td width=\"236\">_document_body<\/td>\n<\/tr>\n<tr>\n<td width=\"146\">sys_created_by<\/td>\n<td width=\"236\">sn_createdBy<\/td>\n<\/tr>\n<tr>\n<td width=\"146\">sys_created_on<\/td>\n<td width=\"236\">_created_at<\/td>\n<\/tr>\n<tr>\n<td width=\"146\">sys_id<\/td>\n<td width=\"236\">sn_sys_id<\/td>\n<\/tr>\n<tr>\n<td width=\"146\">sys_updated_by<\/td>\n<td width=\"236\">sn_updatedBy<\/td>\n<\/tr>\n<tr>\n<td width=\"146\">sys_updated_on<\/td>\n<td width=\"236\">_last_updated_at<\/td>\n<\/tr>\n<tr>\n<td width=\"146\">title<\/td>\n<td width=\"236\">_document_title<\/td>\n<\/tr>\n<tr>\n<td width=\"146\">url<\/td>\n<td width=\"236\">sn_url<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>For this post, our Amazon Kendra index has a custom index field called MyCustomUsername, which you can use to map the Username field from different data sources. This custom field was created under the index\u2019s facet definition. The following screenshot shows a custom mapping.<\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-18497\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/11\/17\/Amazon-Kendra-10.jpg\" alt=\"\" width=\"800\" height=\"589\"><\/p>\n<ol start=\"21\">\n<li>Review the settings and choose <strong>Create data source<\/strong>.<\/li>\n<\/ol>\n<p>After your ServiceNow data source is created, you see a banner similar to the following screenshot.<\/p>\n<ol start=\"22\">\n<li>Choose <strong>Sync now<\/strong> to start the syncing and document ingestion process.<\/li>\n<\/ol>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-18498\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/11\/17\/Amazon-Kendra-11.jpg\" alt=\"\" width=\"800\" height=\"35\"><\/p>\n<p>If everything goes as expected, you can see the status as Succeeded.<\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-18499\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/11\/17\/Amazon-Kendra-12.jpg\" alt=\"\" width=\"800\" height=\"69\"><\/p>\n<h3>Testing<\/h3>\n<p>Now that you have synced your ServiceNow site you can test it on the Amazon Kendra\u2019s search console.<\/p>\n<p>In my case, my ServiceNow site has the demo examples, so I asked what is the storage on the ipad 3, which returned information from a service catalog item:<\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-18500\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/11\/17\/Amazon-Kendra-13.jpg\" alt=\"\" width=\"800\" height=\"333\"><\/p>\n<h2>Creating a ServiceNow connector with Python<\/h2>\n<p>We saw how to create an index on the Amazon Kendra console; now we create a new Amazon Kendra index and a ServiceNow connector and sync it by using the <a href=\"https:\/\/aws.amazon.com\/sdk-for-python\/\" target=\"_blank\" rel=\"noopener noreferrer\">AWS SDK for Python (Boto3)<\/a>. Boto3 makes it easy to integrate your Python application, library, or script with AWS services, including Amazon Kendra.<\/p>\n<p>My personal preference to test my Python scripts is to spin up an <a href=\"https:\/\/aws.amazon.com\/sagemaker\/\">Amazon SageMaker<\/a> notebook instance, a fully managed ML <a href=\"http:\/\/aws.amazon.com\/ec2\">Amazon Elastic Compute Cloud<\/a> (Amazon EC2) instance that runs the Jupyter Notebook app. For instructions, see <a href=\"https:\/\/docs.aws.amazon.com\/sagemaker\/latest\/dg\/gs-setup-working-env.html\" target=\"_blank\" rel=\"noopener noreferrer\">Create an Amazon SageMaker Notebook Instance<\/a>.<\/p>\n<p>To create an index using the AWS SDK, we need to have the policy <a href=\"https:\/\/docs.aws.amazon.com\/kendra\/latest\/dg\/security_iam_id-based-policy-examples.html#security_iam_id-predefined-policies\" target=\"_blank\" rel=\"noopener noreferrer\">AmazonKendraFullAccess<\/a> attached to the role we use.<\/p>\n<p>Also, Amazon Kendra requires different roles to operate:<\/p>\n<ul>\n<li>IAM roles for indexes, which are needed by Amazon Kendra to write to Amazon CloudWatch Logs.<\/li>\n<li>IAM roles for data sources, which are needed when we use the <code>CreateDataSource<\/code> These roles require a specific set of permissions depending on the connector we use. Because we use ServiceNow data sources, it must provide permissions to:\n<ul>\n<li>Secrets Manager, where the ServiceNow online credentials are stored.<\/li>\n<li>Permission to use the <a href=\"http:\/\/aws.amazon.com\/kms\" target=\"_blank\" rel=\"noopener noreferrer\">AWS Key Management Service<\/a> (AWS KMS) customer master Key (CMK) to decrypt the credentials by Secrets Manager.<\/li>\n<li>Permission to use the <code>BatchPutDocument<\/code> and <code>BatchDeleteDocument<\/code> operations to update the index.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>For more information, see <a href=\"https:\/\/docs.aws.amazon.com\/kendra\/latest\/dg\/iam-roles.html\" target=\"_blank\" rel=\"noopener noreferrer\">IAM access roles for Amazon Kendra<\/a>.<\/p>\n<p>Our current requirements are:<\/p>\n<ul>\n<li>Amazon SageMaker Notebooks execution role with permission to create an Amazon Kendra index using an Amazon SageMaker notebook<\/li>\n<li>Amazon Kendra IAM role for CloudWatch<\/li>\n<li>Amazon Kendra IAM role for ServiceNow connector<\/li>\n<li>ServiceNow credentials stored on Secrets Manager<\/li>\n<\/ul>\n<p>To create an index, we use the following code:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">import boto3\r\n from botocore.exceptions import ClientError\r\n import pprint\r\n import time\r\n  \r\n kendra = boto3.client(\"kendra\")\r\n  \r\n print(\"Creating an index\")\r\n  \r\n description = &lt;YOUR_INDEX_DESCRIPTION&gt;\r\n index_name = &lt;YOUR_NEW_INDEX_NAME&gt;\r\n role_arn = &lt;KENDRA_ROLE_WITH_CLOUDWATCH_PERMISSIONS ROLE&gt;\r\n  \r\n try:\r\n     index_response = kendra.create_index(\r\n         Description = &lt;DESCRIPTION&gt;,\r\n         Name = index_name,\r\n         RoleArn = role_arn,\r\n         Edition = \"DEVELOPER_EDITION\",\r\n         Tags=[\r\n         {\r\n             'Key': 'Project',\r\n             'Value': 'SharePoint Test'\r\n         } \r\n         ]\r\n     )\r\n  \r\n     pprint.pprint(index_response)\r\n  \r\n     index_id = index_response['Id']\r\n  \r\n     print(\"Wait for Kendra to create the index.\")\r\n  \r\n     while True:\r\n         # Get index description\r\n         index_description = kendra.describe_index(\r\n             Id = index_id\r\n         )\r\n         # If status is not CREATING quit\r\n         status = index_description[\"Status\"]\r\n         print(\"    Creating index. Status: \"+status)\r\n         if status != \"CREATING\":\r\n             break\r\n         time.sleep(60)\r\n  \r\n except  ClientError as e:\r\n         print(\"%s\" % e)\r\n  \r\n print(\"Done creating index.\")<\/code><\/pre>\n<\/div>\n<p>While our index is being created, we obtain regular updates (every 60 seconds to be exact, check line 38) until the process is finished. See the following code:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">Creating an index\r\n {'Id': '3311b507-bfef-4e2b-bde9-7c297b1fd13b',\r\n  'ResponseMetadata': {'HTTPHeaders': {'content-length': '45',\r\n                                       'content-type': 'application\/x-amz-json-1.1',\r\n                                       'date': 'Wed, 12 Aug 2020 12:58:19 GMT',\r\n                                       'x-amzn-requestid': 'a148a4fc-7549-467e-b6ec-6f49512c1602'},\r\n                       'HTTPStatusCode': 200,\r\n                       'RequestId': 'a148a4fc-7549-467e-b6ec-6f49512c1602',\r\n                       'RetryAttempts': 2}}\r\n Wait for Kendra to create the index.\r\n     Creating index. Status: CREATING\r\n     Creating index. Status: CREATING\r\n     Creating index. Status: CREATING\r\n     Creating index. Status: CREATING\r\n     Creating index. Status: CREATING\r\n     Creating index. Status: CREATING\r\n     Creating index. Status: CREATING\r\n     Creating index. Status: CREATING\r\n     Creating index. Status: CREATING\r\n     Creating index. Status: CREATING\r\n     Creating index. Status: CREATING\r\n     Creating index. Status: CREATING\r\n     Creating index. Status: CREATING\r\n     Creating index. Status: CREATING\r\n     Creating index. Status: CREATING\r\n     Creating index. Status: CREATING\r\n     Creating index. Status: CREATING\r\n     Creating index. Status: ACTIVE\r\n Done creating index<\/code><\/pre>\n<\/div>\n<p>The preceding code indicates that our index has been created and our new index ID is <code>3311b507-bfef-4e2b-bde9-7c297b1fd13b<\/code> (your ID is different from our example code). This information is included as ID in the response.<\/p>\n<p>Our Amazon Kendra index is up and running now.<\/p>\n<p>If you have metadata attributes associated with your ServiceNow articles, you want to do three things:<\/p>\n<ol>\n<li>Determine the Amazon Kendra attribute name you want for each of your ServiceNow metadata attributes. By default, Amazon Kendra has six <a href=\"https:\/\/docs.aws.amazon.com\/kendra\/latest\/dg\/field-mapping.html\" target=\"_blank\" rel=\"noopener noreferrer\">reserved fields<\/a> (<code>_category, created_at, _file_type, _last_updated_at, _source_uri, and _view_count<\/code>).<\/li>\n<li>Update the index with the <a href=\"https:\/\/docs.aws.amazon.com\/kendra\/latest\/dg\/API_UpdateIndex.html\" target=\"_blank\" rel=\"noopener noreferrer\">UpdateIndex<\/a> API call with the Amazon Kendra attribute names.<\/li>\n<li>Map each ServiceNow metadata attribute to each Amazon Kendra metadata attribute.<\/li>\n<\/ol>\n<p>You can find a table with metadata attributes and the suggested Amazon Kendra fields under step 20 on the previous section.<\/p>\n<p>For this post, I have the metadata attribute <code>UserName<\/code> associated with my ServiceNow article and I want to map it to the field <code>MyCustomUsername<\/code> on my index. The following code shows how to add the attribute <code>MyCustomUsername<\/code> to my Amazon Kendra index. After we create this custom field in our index, we map our field Username from ServiceNow to it. See the following code:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-json\">try:\r\n     update_response = kendra.update_index(\r\n         Id='3311b507-bfef-4e2b-bde9-7c297b1fd13b',\r\n         RoleArn='arn:aws:iam::&lt;MY-ACCOUNT-NUMBER&gt;:role\/service-role\/AmazonKendra-us-east-1-KendraRole',\r\n         DocumentMetadataConfigurationUpdates=[\r\n         {\r\n             'Name': &lt;MY_CUSTOM_FIELD_NAME&gt;,\r\n             'Type': 'STRING_VALUE',\r\n             'Search': {\r\n                 'Facetable': True,\r\n                 'Searchable': True,\r\n                 'Displayable': True\r\n             }\r\n         }   \r\n     ]\r\n     )\r\n except  ClientError as e:\r\n         print('%s' % e)   \r\n pprint.pprint(update_response)<\/code><\/pre>\n<\/div>\n<p>If everything goes well, we receive a 200 response:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-http\">{'ResponseMetadata': {'HTTPHeaders': {'content-length': '0',\r\n                                       'content-type': 'application\/x-amz-json-1.1',\r\n                                       'date': 'Wed, 12 Aug 2020 12:17:07 GMT',\r\n                                       'x-amzn-requestid': '3eba66c9-972b-4757-8d92-37be17c8f8a2},\r\n                       'HTTPStatusCode': 200,\r\n                       'RequestId': '3eba66c9-972b-4757-8d92-37be17c8f8a2',\r\n                       'RetryAttempts': 0}} \r\n  \r\n\r\n }<\/code><\/pre>\n<\/div>\n<p>We also need to have <code>GetSecretValue<\/code> for our secret stored in Secrets Manager.<\/p>\n<p>If you need to create a new secret in Secrets Manager to store your ServiceNow credentials, make sure the role you use has permissions to <code>CreateSecret<\/code> and tagging for Secrets Manager. The policy should look like the following code:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-json\">{\r\n    \"Version\": \"2012-10-17\",\r\n    \"Statement\": [\r\n        {\r\n            \"Sid\": \"SecretsManagerWritePolicy\",\r\n            \"Effect\": \"Allow\",\r\n            \"Action\": [\r\n                \"secretsmanager:UntagResource\",\r\n                \"secretsmanager:CreateSecret\",\r\n                \"secretsmanager:TagResource\"\r\n            ],\r\n            \"Resource\": \"*\"\r\n        }\r\n    ]\r\n}<\/code><\/pre>\n<\/div>\n<p>The following code creates a secret in Secrets Manager:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">secretsmanager = boto3.client('secretsmanager')\r\n\r\nSecretName = <span><em>&lt;YOURSECRETNAME&gt;<\/em><\/span>\r\nSharePointCredentials = \"{'username': <span><em>&lt;YOUR_SERVICENOW_SITE_USERNAME&gt;<\/em><\/span>, 'password': <span><em>&lt;YOUR_SERVICENOW_SITE_PASSWORD&gt;<\/em><\/span>}\"\r\n\r\ntry:\r\n  create_secret_response = secretsmanager.create_secret(\r\n  Name=SecretName,\r\n  Description='Secret for a servicenow data source connector',\r\n  SecretString=SharePointCredentials,\r\n  Tags=[\r\n   {\r\n    'Key': 'Project',\r\n    'Value': 'ServiceNow Test'\r\n   }\r\n ]\r\n )\r\nexcept ClientError as e:\r\n  print('%s' % e)\r\n  pprint.pprint(create_secret_response)<\/code><\/pre>\n<\/div>\n<p>If everything goes well, you get a response with your secret\u2019s ARN:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-json\">{'ARN':<span><em>&lt;YOUR_SECRETS_ARN&gt;<\/em><\/span>,\r\n 'Name': <span><em>&lt;YOUR_SECRET_NAME&gt;<\/em><\/span>,\r\n 'ResponseMetadata': {'HTTPHeaders': {'connection': 'keep-alive',\r\n                                      'content-length': '161',\r\n                                      'content-type': 'application\/x-amz-json-1.1',\r\n                                      'date': 'Sat, 22 Aug 2020 14:44:13 GMT',\r\n                                      'x-amzn-requestid': '68c9a153-c08e-42df-9e6d-8b82550bc412'},\r\n                      'HTTPStatusCode': 200,\r\n                      'RequestId': '68c9a153-c08e-42df-9e6d-8b82550bc412',\r\n                      'RetryAttempts': 0},\r\n 'VersionId': 'bee74dab-6beb-4723-a18b-4829d527aad8'}<\/code><\/pre>\n<\/div>\n<p>Now that we have our Amazon Kendra index, our custom field, and our ServiceNow credentials, we can proceed with creating our data source.<\/p>\n<p>To ingest documents from this data source, we need an IAM role with <code>Kendra:BatchPutDocument<\/code> and <code>kendra:BatchDeleteDocument<\/code> permissions. For more information, see <a href=\"https:\/\/docs.aws.amazon.com\/kendra\/latest\/dg\/iam-roles.html#iam-roles-ds-spo\" target=\"_blank\" rel=\"noopener noreferrer\">IAM roles for Microsoft SharePoint Online data sources<\/a>. We use the ARN for this IAM role when invoking the <a href=\"https:\/\/boto3.amazonaws.com\/v1\/documentation\/api\/latest\/reference\/services\/kendra.html#kendra.Client.create_data_source\" target=\"_blank\" rel=\"noopener noreferrer\">CreateDataSource<\/a> API.<\/p>\n<p>Make sure the role you use for your data source connector has a trust relationship with Amazon Kendra. It should look like the following code:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-json\">{\r\n  \"Version\": \"2012-10-17\",\r\n  \"Statement\": [\r\n    {\r\n      \"Effect\": \"Allow\",\r\n      \"Principal\": {\r\n        \"Service\": \"kendra.amazonaws.com\"\r\n      },\r\n      \"Action\": \"sts:AssumeRole\"\r\n    }\r\n  ]<\/code><\/pre>\n<\/div>\n<p>The following code is the <a href=\"https:\/\/docs.aws.amazon.com\/kendra\/latest\/dg\/iam-roles.html#iam-roles-ds-spo\" target=\"_blank\" rel=\"noopener noreferrer\">policy<\/a> structure we need:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-json\">{\r\n    \"Version\": \"2012-10-17\",\r\n    \"Statement\": [\r\n        {\r\n            \"Effect\": \"Allow\",\r\n            \"Action\": [\r\n                \"secretsmanager:GetSecretValue\"\r\n            ],\r\n            \"Resource\": [\r\n                \"arn:aws:secretsmanager:region:account ID:secret:secret ID\"\r\n            ]\r\n        },\r\n        {\r\n            \"Effect\": \"Allow\",\r\n            \"Action\": [\r\n                \"kms:Decrypt\"\r\n            ],\r\n            \"Resource\": [\r\n                \"arn:aws:kms:region:account ID:key\/key ID\"\r\n            ]\r\n        },\r\n        {\r\n            \"Effect\": \"Allow\",\r\n            \"Action\": [\r\n                \"kendra:BatchPutDocument\",\r\n                \"kendra:BatchDeleteDocument\"\r\n            ],\r\n            \"Resource\": [\r\n                \"arn:aws:kendra:&lt;REGION&gt;:&lt;account_ID&gt;:index\/&lt;index_ID&gt;\"\r\n            ],\r\n            \"Condition\": {\r\n                \"StringLike\": {\r\n                    \"kms:ViaService\": [\r\n                        \"kendra.amazonaws.com\"\r\n                    ]\r\n                }\r\n            }\r\n        },\r\n        {\r\n            \"Effect\": \"Allow\",\r\n            \"Action\": [\r\n                \"s3:GetObject\"\r\n            ],\r\n            \"Resource\": [\r\n                \"arn:aws:s3:::<span><em><strong>&lt;BUCKET_NAME&gt;<\/strong><\/em><\/span>\/*\"\r\n            ]\r\n        }\r\n    ]\r\n}<\/code><\/pre>\n<\/div>\n<p>Finally, the following code is my role\u2019s ARN:<\/p>\n<p><code>arn:aws:iam::<span><em>&lt;MY_ACCOUNT_NUMBER&gt;<\/em><\/span>:role\/Kendra-Datasource<\/code><\/p>\n<p>Following the <a href=\"https:\/\/docs.aws.amazon.com\/IAM\/latest\/UserGuide\/best-practices.html#grant-least-privilege\" target=\"_blank\" rel=\"noopener noreferrer\">least privilege principle<\/a>, we only allow our role to put and delete documents in our index, and read the secrets to connect to our ServiceNow site.<\/p>\n<p>One detail we can specify when creating a data source is the sync schedule, which indicates how often our index syncs with the data source we create. This schedule is defined on the <code>Schedule<\/code> key of our request. You can use <a href=\"https:\/\/docs.aws.amazon.com\/AmazonCloudWatch\/latest\/events\/ScheduledEvents.html\" target=\"_blank\" rel=\"noopener noreferrer\">schedule expressions for rules<\/a> to define how often you want to sync your data source. For this post, I use the <code>ScheduleExpression 'cron(0 11 * * ? *)'<\/code>, which means that our data source is synced every day at 11:00 AM.<\/p>\n<p>I use the following code. Make sure you match your <code>SiteURL<\/code> and <code>SecretARN,<\/code> as well as your <code>IndexID<\/code>. Additionally, <code>FieldMappings<\/code> is where you map between the ServiceNow attribute names with the Amazon Kendra index attribute names. I chose the same attribute name in both, but you can call the Amazon Kendra attribute whatever you\u2019d like.<\/p>\n<p>For more details, see <a href=\"https:\/\/boto3.amazonaws.com\/v1\/documentation\/api\/latest\/reference\/services\/kendra.html#kendra.Client.create_data_source\" target=\"_blank\" rel=\"noopener noreferrer\">\u00a0create_data_source(**kwargs)<\/a>.<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">print('Create a data source')\r\n \r\nSecretArn= \"<span><em>&lt;YOUR_SERVICENOW_ONLINE_USER_AND_PASSWORD_SECRETS_ARN&gt;<\/em><\/span>\"\r\nSiteUrl = \"<span><em>&lt;YOUR_SERVICENOW_SITE_URL&gt;<\/em><\/span>\"\r\nDSName= \"<em><span>&lt;YOUR_NEW_DATASOURCE_NAME&gt;<\/span><\/em>\"\r\nIndexId= \"<span><em>&lt;YOUR_INDEX_ID&gt;<\/em><\/span>\"\r\nDSRoleArn= \"<span><em>&lt;YOUR_DATA_SOURCE_ROLE&gt;<\/em><\/span>\"\r\nScheduleExpression='cron(0 11 * * ? *)'\r\ntry:try:\r\n    datasource_response = kendra.create_data_source(\r\n    Name=DSName,\r\n    IndexId=IndexId,        \r\n    Type='SERVICENOW',\r\n    Configuration={\r\n        'ServiceNowConfiguration': {\r\n            'HostUrl': SiteUrl,\r\n            'SecretArn': SecretArn,\r\n            'ServiceNowBuildVersion': 'OTHERS',\r\n            'KnowledgeArticleConfiguration': \r\n            {\r\n                'CrawlAttachments': True,\r\n                'DocumentDataFieldName': 'content',\r\n                'DocumentTitleFieldName': 'short_description',\r\n                'FieldMappings': \r\n                [\r\n                    {\r\n                        'DataSourceFieldName': 'sys_created_on',\r\n                        'DateFieldFormat': 'yyyy-MM-dd hh:mm:ss',\r\n                        'IndexFieldName': '_created_at'\r\n                    },\r\n                    {\r\n                        'DataSourceFieldName': 'sys_updated_on',\r\n                        'DateFieldFormat': 'yyyy-MM-dd hh:mm:ss',\r\n                        'IndexFieldName': '_last_updated_at'\r\n                    },\r\n                    {\r\n                        'DataSourceFieldName': 'kb_category_name',\r\n                        'IndexFieldName': '_category'\r\n                    },\r\n                    {\r\n                        'DataSourceFieldName': 'sys_created_by',\r\n                        'IndexFieldName': 'MyCustomUsername'\r\n                    }\r\n                ],\r\n                'IncludeAttachmentFilePatterns': \r\n                [\r\n                    '.*\\.(dotm|ppt|pot|pps|ppa)$',\r\n                    '.*\\.(doc|dot|docx|dotx|docm)$',\r\n                    '.*\\.(pptx|ppsx|pptm|ppsm|html)$',\r\n                    '.*\\.(txt)$',\r\n                    '.*\\.(hml|xhtml|xhtml2|xht|pdf)$'\r\n                ]\r\n            },\r\n            'ServiceCatalogConfiguration': {\r\n                'CrawlAttachments': True,\r\n                'DocumentDataFieldName': 'description',\r\n                'DocumentTitleFieldName': 'title',\r\n                'FieldMappings': \r\n                [\r\n                    {\r\n                        'DataSourceFieldName': 'sys_created_on',\r\n                        'DateFieldFormat': 'yyyy-MM-dd hh:mm:ss',\r\n                        'IndexFieldName': '_created_at'\r\n                    },\r\n                    {\r\n                        'DataSourceFieldName': 'sys_updated_on',\r\n                        'DateFieldFormat': 'yyyy-MM-dd hh:mm:ss',\r\n                        'IndexFieldName': '_last_updated_at'\r\n                    },\r\n                    {\r\n                        'DataSourceFieldName': 'category_name',\r\n                        'IndexFieldName': '_category'\r\n                    }\r\n                ],\r\n                'IncludeAttachmentFilePatterns': \r\n                [\r\n                    '.*\\.(dotm|ppt|pot|pps|ppa)$',\r\n                    '.*\\.(doc|dot|docx|dotx|docm)$',\r\n                    '.*\\.(pptx|ppsx|pptm|ppsm|html)$',\r\n                    '.*\\.(txt)$',\r\n                    '.*\\.(hml|xhtml|xhtml2|xht|pdf)$'\r\n                ]\r\n            },\r\n        },\r\n    },\r\n    Description='My ServiceNow Datasource',\r\n    RoleArn=DSRoleArn,\r\n    Schedule=ScheduleExpression,\r\n    Tags=[\r\n        {\r\n            'Key': 'Project',\r\n            'Value': 'ServiceNow Test'\r\n        }\r\n    ])\r\n    pprint.pprint(datasource_response)\r\n    print('Waiting for Kendra to create the DataSource.')\r\n    datasource_id = datasource_response['Id']\r\n    while True:\r\n        # Get index description\r\n        datasource_description = kendra.describe_data_source(\r\n            Id=datasource_id,\r\n            IndexId=IndexId\r\n        )\r\n        # If status is not CREATING quit\r\n        status = datasource_description[\"Status\"]\r\n        print(\"    Creating index. Status: \"+status)\r\n        if status != \"CREATING\":\r\n            break\r\n        time.sleep(60)    \r\n\r\nexcept  ClientError as e:\r\n        pprint.pprint('%s' % e)\r\npprint.pprint(datasource_response)\r\n<\/code><\/pre>\n<\/div>\n<p>If everything goes well, we should receive a 200 status response:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-http\">Create a DataSource\r\n{'Id': '507686a5-ff4f-4e82-a356-32f352fb477f',\r\n 'ResponseMetadata': {'HTTPHeaders': {'content-length': '45',\r\n                                      'content-type': 'application\/x-amz-json-1.1',\r\n                                      'date': 'Sat, 22 Aug 2020 15:50:08 GMT',\r\n                                      'x-amzn-requestid': '9deaea21-1d38-47b0-a505-9bb2efb0b74f'},\r\n                      'HTTPStatusCode': 200,\r\n                      'RequestId': '9deaea21-1d38-47b0-a505-9bb2efb0b74f',\r\n                      'RetryAttempts': 0}}\r\nWaiting for Kendra to create the DataSource.\r\n    Creating index. Status: CREATING\r\n    Creating index. Status: ACTIVE\r\n{'Id': '507686a5-ff4f-4e82-a356-32f352fb477f',\r\n 'ResponseMetadata': {'HTTPHeaders': {'content-length': '45',\r\n                                      'content-type': 'application\/x-amz-json-1.1',\r\n                                      'date': 'Sat, 22 Aug 2020 15:50:08 GMT',\r\n                                      'x-amzn-requestid': '9deaea21-1d38-47b0-a505-9bb2efb0b74f'},\r\n                      'HTTPStatusCode': 200,\r\n                      'RequestId': '9deaea21-1d38-47b0-a505-9bb2efb0b74f',\r\n                      'RetryAttempts': 0}}<\/code><\/pre>\n<\/div>\n<p>Even though we have defined a schedule for syncing my data source, we can sync on demand by using the method <a href=\"https:\/\/boto3.amazonaws.com\/v1\/documentation\/api\/latest\/reference\/services\/kendra.html#kendra.Client.start_data_source_sync_job\" target=\"_blank\" rel=\"noopener noreferrer\">start_data_source_sync_job<\/a>:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">DSId=<span><em>&lt;YOUR DATA SOURCE ID&gt;<\/em><\/span>\r\nIndexId=<span><em>&lt;YOUR INDEX ID&gt;<\/em><\/span>\r\n \r\ntry:\r\n    ds_sync_response = kendra.start_data_source_sync_job(\r\n    Id=DSId,\r\n    IndexId=IndexId\r\n)\r\nexcept  ClientError as e:\r\n        print('%s' % e)  \r\n        \r\npprint.pprint(ds_sync_response)<\/code><\/pre>\n<\/div>\n<p>The response should look like the following code:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-http\">{'ExecutionId': '3d11e6ef-3b9e-4283-bf55-f29d0b10e610',\r\n 'ResponseMetadata': {'HTTPHeaders': {'content-length': '54',\r\n                                      'content-type': 'application\/x-amz-json-1.1',\r\n                                      'date': 'Sat, 22 Aug 2020 15:52:36 GMT',\r\n                                      'x-amzn-requestid': '55d94329-50af-4ad5-b41d-b173f20d8f27'},\r\n                      'HTTPStatusCode': 200,\r\n                      'RequestId': '55d94329-50af-4ad5-b41d-b173f20d8f27',\r\n                      'RetryAttempts': 0}}<\/code><\/pre>\n<\/div>\n<h3>Testing<\/h3>\n<p>Finally, we can query our index. See the following code:<\/p>\n<div class=\"hide-language\">\n<pre><code class=\"lang-python\">response = kendra.query(\r\nIndexId=&lt;YOUR_INDEX_ID&gt;,\r\nQueryText='Is there a service that has 11 9s of durability?')\r\nif response['TotalNumberOfResults'] &gt; 0:\r\n    print(response['ResultItems'][0]['DocumentExcerpt']['Text'])\r\n    print(\"More information: \"+response['ResultItems'][0]['DocumentURI'])\r\nelse:\r\n    print('No results found, please try a different search term.')<\/code><\/pre>\n<\/div>\n<h2>Common errors<\/h2>\n<p>In this section, we discuss errors that may occur, whether using the Amazon Kendra console or the Amazon Kendra API.<\/p>\n<p>You should look at CloudWatch logs and error messages returned in the Amazon Kendra console or via the Amazon Kendra API. The CloudWatch logs help you determine the reason for a particular error, whether you experience it using the console or programmatically.<\/p>\n<p>Common errors when trying to access ServiceNow as a data source are:<\/p>\n<ul>\n<li>Insufficient permissions<\/li>\n<li>Invalid credentials<\/li>\n<li>Secrets Manager error<\/li>\n<\/ul>\n<h3>Insufficient permissions<\/h3>\n<p>A common scenario you may come across is when you have the right credentials but your user doesn\u2019t have enough permissions for the Amazon Kendra ServiceNow connector to crawl your knowledge base and service catalog items.<\/p>\n<p>You receive the following error message:<\/p>\n<p><code>We couldn't sync the following data source: 'MyServiceNowOnline', at start time Sep 12, 2020, 1:08 PM CDT. Amazon Kendra can't connect to the ServiceNow server with the specified credentials. Check your credentials and try your request again.<\/code><\/p>\n<p>If you can log in to your ServiceNow instance, make sure that the user you designed for the connector has the admin role.<\/p>\n<ol>\n<li>On your ServiceNow instance, under <strong>User Administration<\/strong>, choose <strong>Users<\/strong>.<\/li>\n<\/ol>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-18501\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/11\/17\/Amazon-Kendra-14.jpg\" alt=\"\" width=\"558\" height=\"240\"><\/p>\n<ol start=\"2\">\n<li>On the users list, choose the user ID of the user you want to use for the connector.<\/li>\n<\/ol>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-18502\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/11\/17\/Amazon-Kendra-15.jpg\" alt=\"\" width=\"800\" height=\"141\"><\/p>\n<ol start=\"3\">\n<li>On the <strong>Roles <\/strong>tab, verify that your user has the admin<\/li>\n<\/ol>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-18503\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/11\/17\/Amazon-Kendra-16.jpg\" alt=\"\" width=\"800\" height=\"250\"><\/p>\n<ol start=\"4\">\n<li>If you don\u2019t have that role attached to your user, choose <strong>Edit <\/strong>to add it.<\/li>\n<\/ol>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-18504\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/11\/17\/Amazon-Kendra-17.jpg\" alt=\"\" width=\"800\" height=\"613\"><\/p>\n<ol start=\"5\">\n<li>On the Amazon Kendra console, on your connector configuration page, choose <strong>Sync now<\/strong>.<\/li>\n<\/ol>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-18505\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/11\/17\/Amazon-Kendra-18.jpg\" alt=\"\" width=\"402\" height=\"74\"><\/p>\n<h3>Invalid credentials<\/h3>\n<p>You may encounter an error with the following message:<\/p>\n<p><code>We couldn't sync the following data source: 'MyServiceNowOnline', at start time Jul 28, 2020, 3:59 PM CDT. Amazon Kendra can't connect to the ServiceNow server with the specified credentials. Check your credentials and try your request again.<\/code><\/p>\n<p>To investigate, complete the following steps:<\/p>\n<ol>\n<li>Choose the error message to review the CloudWatch logs.<\/li>\n<\/ol>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-18506\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/11\/17\/Amazon-Kendra-19.jpg\" alt=\"\" width=\"800\" height=\"65\"><\/p>\n<p>You\u2019re redirected <a href=\"https:\/\/docs.aws.amazon.com\/AmazonCloudWatch\/latest\/logs\/AnalyzingLogData.html\" target=\"_blank\" rel=\"noopener noreferrer\">CloudWatch Logs Insights<\/a>.<\/p>\n<ol start=\"2\">\n<li>Choose <strong>Run Query <\/strong>to start analyzing the logs.<\/li>\n<\/ol>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-18507\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/11\/17\/Amazon-Kendra-20.jpg\" alt=\"\" width=\"800\" height=\"126\"><\/p>\n<p>We can verify our credentials by going to Secrets Manager and reviewing our credentials stored in the secret.<\/p>\n<ol start=\"3\">\n<li>Choose your secret name.<\/li>\n<\/ol>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-18508\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/11\/17\/Amazon-Kendra-21.jpg\" alt=\"\" width=\"800\" height=\"24\"><\/p>\n<ol start=\"4\">\n<li>Choose <strong>Retrieve secret value<\/strong>.<\/li>\n<\/ol>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-18519 size-full\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/11\/17\/Amazon-Kendra-22.jpg\" alt=\"\" width=\"800\" height=\"54\"><\/p>\n<ol start=\"5\">\n<li>If your password doesn\u2019t match, choose <strong>Edit<\/strong>,<\/li>\n<\/ol>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-18510\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/11\/17\/Amazon-Kendra-23.jpg\" alt=\"\" width=\"800\" height=\"165\"><\/p>\n<ol start=\"6\">\n<li>And the username or password and choose <strong>Save<\/strong>.<\/li>\n<\/ol>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-18511\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/11\/17\/Amazon-Kendra-24.jpg\" alt=\"\" width=\"608\" height=\"336\"><\/p>\n<ol start=\"7\">\n<li>Go back to your data source in Amazon Kendra, and choose <strong>Sync now<\/strong>.<\/li>\n<\/ol>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-18512\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/11\/17\/Amazon-Kendra-25.jpg\" alt=\"\" width=\"800\" height=\"27\"><\/p>\n<h3>Secrets Manager error<\/h3>\n<p>You may encounter an error stating that the customer\u2019s secret can\u2019t be fetched. This may happen if you use an existing secret and the IAM role used for syncing your ServiceNow data source doesn\u2019t have permissions to access the secret.<\/p>\n<p>To address this issue, first we need our secret\u2019s ARN.<\/p>\n<ol>\n<li>On the Secrets Manager console, choose your secret\u2019s name (for this post, <code>AmazonKendra-ServiceNow-demosite<\/code>).<\/li>\n<li>Copy the secret\u2019s ARN.<\/li>\n<li>On the IAM console, search for the role we use to sync our ServiceNow data source (for this post, <code>AmazonKendra-servicenow-role<\/code>).<\/li>\n<li>For <strong>Permissions<\/strong>, choose <strong>Add inline policy<\/strong>.<\/li>\n<li>Following the <a href=\"https:\/\/docs.aws.amazon.com\/IAM\/latest\/UserGuide\/best-practices.html\" target=\"_blank\" rel=\"noopener noreferrer\">least privilege principle<\/a>, for <strong>Service<\/strong>, choose <strong>Secrets Manager<\/strong>.<\/li>\n<li>For <strong>Access Level<\/strong>, choose <strong>Read<\/strong> and <strong>GetSecretValue<\/strong>.<\/li>\n<li>For <strong>Resources<\/strong>, enter our secret\u2019s ARN.<\/li>\n<\/ol>\n<p>Your settings should look similar to the following screenshot.<\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-18513\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/11\/17\/Amazon-Kendra-26.jpg\" alt=\"\" width=\"800\" height=\"288\"><\/p>\n<ol start=\"8\">\n<li>Enter a name for your policy.<\/li>\n<li>Choose <strong>Create Policy<\/strong>.<\/li>\n<\/ol>\n<p>After your policy has been created and attached to your data source role, try to sync again.<\/p>\n<h2>Conclusion<\/h2>\n<p>You have now learned how to ingest the documents from your ServiceNow site into your Amazon Kendra index. We hope this post helps you take advantage of the intelligent search capabilities in Amazon Kendra to find accurate answers from your enterprise content.<\/p>\n<p>For more information about Amazon Kendra, see\u00a0<a href=\"https:\/\/www.youtube.com\/watch?v=7-31KgImGgU&amp;t=7904s\" target=\"_blank\" rel=\"noopener noreferrer\">AWS re:Invent 2019 \u2013 Keynote with Andy Jassy<\/a>\u00a0on YouTube,\u00a0<a href=\"https:\/\/aws.amazon.com\/kendra\/faqs\/\" target=\"_blank\" rel=\"noopener noreferrer\">Amazon Kendra FAQs<\/a>, and\u00a0<a href=\"https:\/\/docs.aws.amazon.com\/kendra\/latest\/dg\/what-is-kendra.html\" target=\"_blank\" rel=\"noopener noreferrer\">What is Amazon Kendra?<\/a><\/p>\n<p>\u00a0<\/p>\n<hr>\n<h3>About the Authors<\/h3>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"size-full wp-image-15317 alignleft\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/08\/27\/David-Shute.jpg\" alt=\"\" width=\"101\" height=\"136\"><strong>David Shute<\/strong> is a Senior ML GTM Specialist at Amazon Web Services focused on Amazon Kendra. When not working, he enjoys hiking and walking on a beach.<\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"size-full wp-image-17854 alignleft\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/11\/01\/Juan-Bustos.jpg\" alt=\"\" width=\"100\" height=\"150\"><strong>Juan Pablo Bustos<\/strong> is an AI Services Specialist Solutions Architect at Amazon Web Services, based in Dallas, TX. Outside of work, he loves spending time writing and playing music as well as trying random restaurants with his family.<\/p>\n<\/p><\/div>\n","protected":false},"excerpt":{"rendered":"<p>https:\/\/aws.amazon.com\/blogs\/machine-learning\/getting-started-with-amazon-kendra-servicenow-online-connector\/<\/p>\n","protected":false},"author":0,"featured_media":579,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[],"_links":{"self":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/578"}],"collection":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/comments?post=578"}],"version-history":[{"count":0,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/578\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media\/579"}],"wp:attachment":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media?parent=578"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/categories?post=578"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/tags?post=578"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}