{"id":851,"date":"2021-09-05T15:53:38","date_gmt":"2021-09-05T15:53:38","guid":{"rendered":"https:\/\/salarydistribution.com\/machine-learning\/2021\/09\/05\/a-gentle-introduction-to-approximation\/"},"modified":"2021-09-05T15:53:38","modified_gmt":"2021-09-05T15:53:38","slug":"a-gentle-introduction-to-approximation","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2021\/09\/05\/a-gentle-introduction-to-approximation\/","title":{"rendered":"A Gentle Introduction To Approximation"},"content":{"rendered":"<div id=\"\">\n<p>When it comes to machine learning tasks such as classification or regression, approximation techniques play a key role in learning from the data. Many machine learning methods approximate a function or a mapping between the inputs and outputs via a learning algorithm.<\/p>\n<p>In this tutorial, you will discover what is approximation and its importance in machine learning and pattern recognition.<\/p>\n<p>After completing this tutorial, you will know:<\/p>\n<ul>\n<li>\u00a0What is approximation<\/li>\n<li>Importance of approximation in machine learning<\/li>\n<\/ul>\n<p>Let\u2019s get started.<\/p>\n<div id=\"attachment_12764\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/MMani.png\"><img aria-describedby=\"caption-attachment-12764\" loading=\"lazy\" class=\"wp-image-12764 \" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/MMani-300x224.png\" alt=\"A Gentle Introduction To Approximation. Photo by M Mani, some rights reserved.\" width=\"530\" height=\"395\"><img decoding=\"async\" aria-describedby=\"caption-attachment-12764\" loading=\"lazy\" class=\"wp-image-12764 \" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/MMani-300x224.png\" alt=\"A Gentle Introduction To Approximation. Photo by M Mani, some rights reserved.\" width=\"530\" height=\"395\"><\/a><\/p>\n<p id=\"caption-attachment-12764\" class=\"wp-caption-text\">A Gentle Introduction To Approximation. Photo by M Mani, some rights reserved.<\/p>\n<\/div>\n<h2><b>Tutorial Overview<\/b><\/h2>\n<p>This tutorial is divided into 3 parts; they are:<\/p>\n<ol>\n<li>What is approximation?<\/li>\n<li>Approximation when the form of function is not known<\/li>\n<li>Approximation when the form of function is known<\/li>\n<\/ol>\n<h2><b>What Is Approximation?<\/b><\/h2>\n<p>We come across approximation very often. For example, the irrational number \u03c0 can be approximated by the number 3.14. A more accurate value is 3.141593, which remains an approximation. You can similarly approximate the values of all irrational numbers like sqrt(3), sqrt(7), etc.<\/p>\n<p>Approximation is used whenever a numerical value, a model, a structure or a function is either unknown or difficult to compute. In this article we\u2019ll focus on function approximation and describe its application to machine learning problems. There are two different cases:<\/p>\n<ol>\n<li>The function is known but it is difficult or numerically expensive to compute its exact value. In this case approximation methods are used to find values, which are close to the function\u2019s actual values.<\/li>\n<li>The function itself is unknown and hence a model or learning algorithm is used to closely find a function that can produce outputs close to the unknown function\u2019s outputs.<\/li>\n<\/ol>\n<h2><b>Approximation When Form of Function is Known<\/b><\/h2>\n<p>If the form of a function is known, then a well known method in calculus and mathematics is approximation via Taylor series. The Taylor series of a function is the sum of infinite terms, which are computed using function\u2019s derivatives. The Taylor series expansion of a function is discussed in this <a href=\"https:\/\/machinelearningmastery.com\/a-gentle-introduction-to-taylor-series\">tutorial<\/a>.<\/p>\n<p>Another well known method for approximation in calculus and mathematics is <a href=\"https:\/\/en.wikipedia.org\/wiki\/Newton%27s_method\">Newton\u2019s method<\/a>. It can be used to approximate the roots of polynomials, hence making it a useful technique for approximating quantities such as the square root of different values or the reciprocal of different numbers, etc.<\/p>\n<h2><b>Approximation When Form of Function is Unknown<\/b><\/h2>\n<p>In data science and machine learning, it is assumed that there is an underlying function that holds the key to the relationship between the inputs and outputs. The form of this function is unknown. Here, we discuss several machine learning problems that employ approximation.<\/p>\n<h3><b>Approximation in Regression<\/b><\/h3>\n<p>Regression involves the prediction of an output variable when given a set of inputs. In regression, the function that truly maps the input variables to outputs is not known. It is assumed that some linear or non-linear regression model can approximate the mapping of inputs to outputs.<span class=\"Apple-converted-space\">\u00a0<\/span><\/p>\n<p>For example, we may have data related to consumed calories per day and the corresponding blood sugar. To describe the relationship between the calorie input and blood sugar output, we can assume a straight line relationship\/mapping function. The straight line is therefore the approximation of the mapping of inputs to outputs. A learning method such as the method of least squares is used to find this line.<\/p>\n<div id=\"attachment_12717\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/approx1.png\"><img aria-describedby=\"caption-attachment-12717\" loading=\"lazy\" class=\"wp-image-12717 \" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/approx1-300x240.png\" alt=\"A straight line approximation to relationship between caloric count and blood sugar\" width=\"485\" height=\"388\"><img decoding=\"async\" aria-describedby=\"caption-attachment-12717\" loading=\"lazy\" class=\"wp-image-12717 \" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/approx1-300x240.png\" alt=\"A straight line approximation to relationship between caloric count and blood sugar\" width=\"485\" height=\"388\"><\/a><\/p>\n<p id=\"caption-attachment-12717\" class=\"wp-caption-text\">A straight line approximation to relationship between caloric count and blood sugar<\/p>\n<\/div>\n<h3><b>Approximation in Classification<\/b><\/h3>\n<p>A classic example of models that approximate functions in classification problems is that of neural networks. It is assumed that the neural network as a whole can approximate a true function that maps the inputs to the class labels. Gradient descent or some other learning algorithm is then used to learn that function approximation by adjusting the weights of the neural network.<\/p>\n<div id=\"attachment_12719\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/approx3.png\"><img aria-describedby=\"caption-attachment-12719\" loading=\"lazy\" class=\"wp-image-12719 \" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/approx3-300x74.png\" alt=\"A neural network approximates an underlying function that maps inputs to outputs\" width=\"482\" height=\"119\"><img decoding=\"async\" aria-describedby=\"caption-attachment-12719\" loading=\"lazy\" class=\"wp-image-12719 \" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/approx3-300x74.png\" alt=\"A neural network approximates an underlying function that maps inputs to outputs\" width=\"482\" height=\"119\"><\/a><\/p>\n<p id=\"caption-attachment-12719\" class=\"wp-caption-text\">A neural network approximates an underlying function that maps inputs to outputs<\/p>\n<\/div>\n<h3><b>Approximation in Unsupervised Learning<\/b><\/h3>\n<p>Below is a typical example of unsupervised learning. Here we have points in 2D space and the label of none of these points is given. A clustering algorithm generally assumes a model according to which a point can be assigned to a class or label. For example, k-means learns the labels of data by assuming that data clusters are circular, and hence, assigns the same label or class to points lying in the same circle or an n-sphere in case of multi-dimensional data. In the figure below we are approximating the relationship between points and their labels via circular functions.<\/p>\n<div id=\"attachment_12718\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/approx2.png\"><img aria-describedby=\"caption-attachment-12718\" loading=\"lazy\" class=\"wp-image-12718 \" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/approx2-300x230.png\" alt=\"A clustering algorithm approximates a model that determines clusters or unknown labels of input points\" width=\"467\" height=\"358\"><img decoding=\"async\" aria-describedby=\"caption-attachment-12718\" loading=\"lazy\" class=\"wp-image-12718 \" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/approx2-300x230.png\" alt=\"A clustering algorithm approximates a model that determines clusters or unknown labels of input points\" width=\"467\" height=\"358\"><\/a><\/p>\n<p id=\"caption-attachment-12718\" class=\"wp-caption-text\">A clustering algorithm approximates a model that determines clusters or unknown labels of input points<\/p>\n<\/div>\n<h2><b>Extensions<\/b><\/h2>\n<p>This section lists some ideas for extending the tutorial that you may wish to explore.<\/p>\n<ul>\n<li>Maclaurin series<\/li>\n<li>Taylor\u2019s series<\/li>\n<\/ul>\n<p>If you explore any of these extensions, I\u2019d love to know. Post your findings in the comments below.<\/p>\n<h2><b>Further Reading<\/b><\/h2>\n<p>This section provides more resources on the topic if you are looking to go deeper.<\/p>\n<h3><b>Tutorials<\/b><\/h3>\n<h3><b>Resources<\/b><\/h3>\n<h3><b>Books<\/b><\/h3>\n<h2><b>Summary<\/b><\/h2>\n<p>In this tutorial, you discovered what is approximation. Specifically, you learned:<\/p>\n<ul>\n<li>Approximation<\/li>\n<li>Approximation when the form of a function is known<\/li>\n<li>Approximation when the form of a function is unknown<\/li>\n<\/ul>\n<h2><b>Do you have any questions?<\/b><\/h2>\n<p>Ask your questions in the comments below and I will do my best to answer<\/p>\n<\/p><\/div>\n","protected":false},"excerpt":{"rendered":"<p>https:\/\/machinelearningmastery.com\/a-gentle-introduction-to-approximation\/<\/p>\n","protected":false},"author":0,"featured_media":852,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[],"_links":{"self":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/851"}],"collection":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/comments?post=851"}],"version-history":[{"count":0,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/851\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media\/852"}],"wp:attachment":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media?parent=851"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/categories?post=851"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/tags?post=851"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}