{"id":210,"date":"2020-09-12T16:01:36","date_gmt":"2020-09-12T16:01:36","guid":{"rendered":"https:\/\/machine-learning.webcloning.com\/2020\/09\/12\/aggregate-and-query-information-from-web-portals\/"},"modified":"2020-09-12T16:01:36","modified_gmt":"2020-09-12T16:01:36","slug":"aggregate-and-query-information-from-web-portals","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2020\/09\/12\/aggregate-and-query-information-from-web-portals\/","title":{"rendered":"Aggregate and query information from web portals"},"content":{"rendered":"<div id=\"\">\n<!-- begin main body content --><\/p>\n<h2 id=\"summary\">Summary<\/h2>\n<p>In this code pattern, build an application that uses IBM\u00ae Watson\u2122 Discovery service APIs to create, query, get a status, and delete a document collection of web portals. The application renders the query results on a custom-built web user interface. This provides flexibility for an app developer to design and build the web user interface to suit specific information and visualization requirements.<\/p>\n<h2 id=\"description\">Description<\/h2>\n<p>There is a lot of information on the internet, and people are always interested in getting information relevant to them. Two common scenarios drive the need for it:<\/p>\n<ul>\n<li>There is usually a specific item of interest, and you want to get all of the relevant information about the item<\/li>\n<li>You have a question and are searching to find the answers for it on the internet<\/li>\n<\/ul>\n<p>This code pattern addresses the previous scenarios where you want to query for relevant information from a group of internet pages. Watson Discovery provides the ability to crawl through web pages and build a queryable collection. You use this feature to build an application where you can:<\/p>\n<ul>\n<li>Specify a list of URLs that Watson Discovery will crawl to build the collection<\/li>\n<li>Specify a query in natural language and get relevant results with insights from Watson Discovery<\/li>\n<li>Visualize the top five matching documents, passages, and entities for the query<\/li>\n<\/ul>\n<p>When you have completed this code pattern, you understand how to:<\/p>\n<ul>\n<li>Create a collection using a web crawl<\/li>\n<li>Get the status of a collection<\/li>\n<li>Query the collection using natural language<\/li>\n<li>Delete a collection<\/li>\n<li>Parse, read, and visualize the results from Watson Discovery<\/li>\n<\/ul>\n<h2 id=\"flow\">Flow<\/h2>\n<p><img class=\"lazycontent\" data-src=\"https:\/\/developer.ibm.com\/developer\/default\/patterns\/retrieve-and-visualize-relevant-information-from-collection-of-web-portals\/images\/discovery-web-crawl-insights-flow.png\" alt=\"Retrieve and visualize relevant information flow\"><\/p>\n<ol>\n<li>The user requests a creation, status, or deletion of a collection, or queries a collection through a custom-built web UI.<\/li>\n<li>The request is sent to a server application on the cloud.<\/li>\n<li>The application invokes an API on the Watson Discovery service using the Watson SDK.<\/li>\n<li>The Watson Discovery service processes the results and sends it back to the application. The results are then visualized by the user.<\/li>\n<\/ol>\n<h2 id=\"instructions\">Instructions<\/h2>\n<p>Find the detailed steps for this pattern in the <a href=\"https:\/\/github.com\/IBM\/discovery-webcrawl-insights\/blob\/master\/README.md\">readme<\/a> file. The steps will show you how to:<\/p>\n<ol>\n<li>Clone the GitHub repository.<\/li>\n<li>Deploy the application.<\/li>\n<li>Analyze the results.<\/li>\n<\/ol>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>https:\/\/developer.ibm.com\/patterns\/retrieve-and-visualize-relevant-information-from-collection-of-web-portals\/<\/p>\n","protected":false},"author":0,"featured_media":211,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[],"_links":{"self":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/210"}],"collection":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/comments?post=210"}],"version-history":[{"count":0,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/210\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media\/211"}],"wp:attachment":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media?parent=210"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/categories?post=210"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/tags?post=210"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}