{"id":849,"date":"2021-09-05T15:53:37","date_gmt":"2021-09-05T15:53:37","guid":{"rendered":"https:\/\/salarydistribution.com\/machine-learning\/2021\/09\/05\/a-gentle-introduction-to-taylor-series\/"},"modified":"2021-09-05T15:53:37","modified_gmt":"2021-09-05T15:53:37","slug":"a-gentle-introduction-to-taylor-series","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2021\/09\/05\/a-gentle-introduction-to-taylor-series\/","title":{"rendered":"A Gentle Introduction to Taylor Series"},"content":{"rendered":"<div id=\"\">\n<p><b>A Gentle Introduction to Taylor Series<\/b><\/p>\n<p>Taylor series expansion is an awesome concept, not only the world of mathematics, but also in optimization theory, function approximation and machine learning. It is widely applied in numerical computations when estimates of a function\u2019s values at different points are required.<\/p>\n<p>In this tutorial, you will discover Taylor series and how to approximate the values of a function around different points using its Taylor series expansion.<\/p>\n<p>After completing this tutorial, you will know:<\/p>\n<ul>\n<li>Taylor series expansion of a function<\/li>\n<li>How to approximate functions using Taylor series expansion<\/li>\n<\/ul>\n<p>Let\u2019s get started.<\/p>\n<div id=\"attachment_12763\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/Muhammad-Khubaib-Sarfraz.png\"><img aria-describedby=\"caption-attachment-12763\" loading=\"lazy\" class=\"wp-image-12763 \" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/Muhammad-Khubaib-Sarfraz-300x224.png\" alt=\"A Gentle Introduction To Taylor Series. Photo by Muhammad Khubaib Sarfraz, some rights reserved.\" width=\"506\" height=\"378\"><img decoding=\"async\" aria-describedby=\"caption-attachment-12763\" loading=\"lazy\" class=\"wp-image-12763 \" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/Muhammad-Khubaib-Sarfraz-300x224.png\" alt=\"A Gentle Introduction To Taylor Series. Photo by Muhammad Khubaib Sarfraz, some rights reserved.\" width=\"506\" height=\"378\"><\/a><\/p>\n<p id=\"caption-attachment-12763\" class=\"wp-caption-text\">A Gentle Introduction To Taylor Series. Photo by Muhammad Khubaib Sarfraz, some rights reserved.<\/p>\n<\/div>\n<h2><b>Tutorial Overview<\/b><\/h2>\n<p>This tutorial is divided into 3 parts; they are:<\/p>\n<ol>\n<li>Power series and Taylor series<\/li>\n<li>Taylor polynomials<\/li>\n<li>Function approximation using Taylor polynomials<\/li>\n<\/ol>\n<h2><b>What Is A Power Series?<\/b><\/h2>\n<p>The following is a power series about the center x=a and constant coefficients c_0, c_1, etc.<\/p>\n<p><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/tayloreq1.png\"><img loading=\"lazy\" class=\"wp-image-12759 aligncenter\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/tayloreq1-300x39.png\" alt=\"\" width=\"492\" height=\"64\"><img decoding=\"async\" loading=\"lazy\" class=\"wp-image-12759 aligncenter\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/tayloreq1-300x39.png\" alt=\"\" width=\"492\" height=\"64\"><\/a><\/p>\n<h2><b>What Is A Taylor Series?<\/b><\/h2>\n<p>It is an amazing fact that functions which are infinitely differentiable can generate a power series called the Taylor series. Suppose we have a function f(x) and f(x) has derivatives of all orders on a given interval, then the Taylor series generated by f(x) at x=a is given by:<\/p>\n<p><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/tayloreq2.png\"><img loading=\"lazy\" class=\"wp-image-12758 aligncenter\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/tayloreq2-300x63.png\" alt=\"\" width=\"538\" height=\"113\"><img decoding=\"async\" loading=\"lazy\" class=\"wp-image-12758 aligncenter\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/tayloreq2-300x63.png\" alt=\"\" width=\"538\" height=\"113\"><\/a><\/p>\n<p>The second line of the above expression gives the value of the kth coefficient.<\/p>\n<p>If we set a=0, then we have an expansion called the Maclaurin series expansion of f(x).<\/p>\n<h2><b>Examples Of Taylor Series Expansion<\/b><\/h2>\n<p>Taylor series generated by f(x) = 1\/x can be found by first differentiating the function and finding a general expression for the kth derivative.<span class=\"Apple-converted-space\">\u00a0<\/span><\/p>\n<p><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/tayloreq3.png\"><img loading=\"lazy\" class=\"wp-image-12757 aligncenter\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/tayloreq3-300x32.png\" alt=\"\" width=\"646\" height=\"69\"><img decoding=\"async\" loading=\"lazy\" class=\"wp-image-12757 aligncenter\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/tayloreq3-300x32.png\" alt=\"\" width=\"646\" height=\"69\"><\/a><\/p>\n<p>The Taylor series about various points can now be found. For example:<\/p>\n<p><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/tayloreq4.png\"><img loading=\"lazy\" class=\"wp-image-12756 aligncenter\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/tayloreq4-300x125.png\" alt=\"\" width=\"607\" height=\"253\"><img decoding=\"async\" loading=\"lazy\" class=\"wp-image-12756 aligncenter\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/tayloreq4-300x125.png\" alt=\"\" width=\"607\" height=\"253\"><\/a><\/p>\n<p>\u00a0<\/p>\n<h2><b>Taylor Polynomial<\/b><\/h2>\n<p>A Taylor polynomial of order k, generated by f(x) at x=a is given by:<\/p>\n<p><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/tayloreq5.png\"><img loading=\"lazy\" class=\"wp-image-12755 aligncenter\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/tayloreq5-300x47.png\" alt=\"\" width=\"587\" height=\"92\"><img decoding=\"async\" loading=\"lazy\" class=\"wp-image-12755 aligncenter\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/tayloreq5-300x47.png\" alt=\"\" width=\"587\" height=\"92\"><\/a><\/p>\n<p>For the example of f(x)=1\/x, the Taylor polynomial of order 2 is given by:<\/p>\n<p><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/tayloreq6.png\"><img loading=\"lazy\" class=\"wp-image-12754 aligncenter\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/tayloreq6-300x273.png\" alt=\"\" width=\"256\" height=\"233\"><img decoding=\"async\" loading=\"lazy\" class=\"wp-image-12754 aligncenter\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/tayloreq6-300x273.png\" alt=\"\" width=\"256\" height=\"233\"><\/a><\/p>\n<h2><b>Approximation via Taylor Polynomials<\/b><\/h2>\n<p>We can approximate the value of a function at a point x=a using Taylor polynomials. The higher the order of the polynomial, the more the terms in the polynomial and the closer the approximation is to the actual value of the function at that point.<\/p>\n<p>In the graph below, the function 1\/x is plotted around the point x=1 (left) and x=3 (right). The line in green is the actual function f(x)= 1\/x. The pink line represents the approximation via an order 2 polynomial.<\/p>\n<div id=\"attachment_12761\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/taylor1-1.png\"><img aria-describedby=\"caption-attachment-12761\" loading=\"lazy\" class=\"wp-image-12761\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/taylor1-1-300x124.png\" alt=\"The actual function (green) and its approximation (pink)\" width=\"842\" height=\"348\"><img decoding=\"async\" aria-describedby=\"caption-attachment-12761\" loading=\"lazy\" class=\"wp-image-12761\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/taylor1-1-300x124.png\" alt=\"The actual function (green) and its approximation (pink)\" width=\"842\" height=\"348\"><\/a><\/p>\n<p id=\"caption-attachment-12761\" class=\"wp-caption-text\">The actual function (green) and its approximation (pink)<\/p>\n<\/div>\n<h2>More Examples of Taylor Series<\/h2>\n<p>Let\u2019s look at the function g(x) = e^x.<span class=\"Apple-converted-space\">\u00a0 <\/span>Noting the fact that the kth order derivative of g(x) is also g(x), the expansion of g(x) about x=a, is given by:<\/p>\n<p><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/tayloreq7.png\"><img loading=\"lazy\" class=\"wp-image-12753 aligncenter\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/tayloreq7-300x48.png\" alt=\"\" width=\"488\" height=\"78\"><img decoding=\"async\" loading=\"lazy\" class=\"wp-image-12753 aligncenter\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/tayloreq7-300x48.png\" alt=\"\" width=\"488\" height=\"78\"><\/a><\/p>\n<p>Hence, around x=0, the series expansion of g(x) is given by (obtained by setting a=0):<\/p>\n<p><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/tayloreq8.png\"><img loading=\"lazy\" class=\"size-medium wp-image-12752 aligncenter\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/tayloreq8-300x87.png\" alt=\"\" width=\"300\" height=\"87\"><img decoding=\"async\" loading=\"lazy\" class=\"size-medium wp-image-12752 aligncenter\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/tayloreq8-300x87.png\" alt=\"\" width=\"300\" height=\"87\"><\/a><\/p>\n<p>The polynomial of order k generated for the function e^x around the point x=0 is given by:<\/p>\n<p><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/tayloreq9.png\"><img loading=\"lazy\" class=\"size-medium wp-image-12751 aligncenter\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/tayloreq9-300x79.png\" alt=\"\" width=\"300\" height=\"79\"><img decoding=\"async\" loading=\"lazy\" class=\"size-medium wp-image-12751 aligncenter\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/tayloreq9-300x79.png\" alt=\"\" width=\"300\" height=\"79\"><\/a><\/p>\n<p>The plots below show polynomials of different orders that estimate the value of e^x around x=0. We can see that as we move away from zero, we need more terms to approximate e^x more accurately. The green line representing the actual function is hiding behind the blue line of the approximating polynomial of order 7.<\/p>\n<div id=\"attachment_12762\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/taylor2.png\"><img aria-describedby=\"caption-attachment-12762\" loading=\"lazy\" class=\"wp-image-12762\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/taylor2-300x238.png\" alt=\"\" width=\"442\" height=\"351\"><img decoding=\"async\" aria-describedby=\"caption-attachment-12762\" loading=\"lazy\" class=\"wp-image-12762\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/07\/taylor2-300x238.png\" alt=\"\" width=\"442\" height=\"351\"><\/a><\/p>\n<p id=\"caption-attachment-12762\" class=\"wp-caption-text\">Polynomials of varying degrees that approximate e^x<\/p>\n<\/div>\n<h2><b>Taylor Series In Machine Learning<\/b><\/h2>\n<p>A popular method in machine learning for finding the optimal points of a function is the Newton\u2019s method. Newton\u2019s method uses the second order polynomials to approximate a function\u2019s value at a point. Such methods that use second order derivatives are called second order optimization algorithms.<span class=\"Apple-converted-space\">\u00a0<\/span><\/p>\n<h2><b>Extensions<\/b><\/h2>\n<p>This section lists some ideas for extending the tutorial that you may wish to explore.<\/p>\n<ul>\n<li>Newton\u2019s method<\/li>\n<li>Second order optimization algorithms<\/li>\n<\/ul>\n<p>If you explore any of these extensions, I\u2019d love to know. Post your findings in the comments below.<\/p>\n<h2><b>Further Reading<\/b><\/h2>\n<p>This section provides more resources on the topic if you are looking to go deeper.<\/p>\n<h3><b>Tutorials<\/b><\/h3>\n<h3><b>Resources<\/b><\/h3>\n<h3><b>Books<\/b><\/h3>\n<ul>\n<li><a href=\"https:\/\/www.amazon.com\/Pattern-Recognition-Learning-Information-Statistics\/dp\/0387310738\">Pattern recognition and machine learning<\/a> by Christopher M. Bishop.<\/li>\n<li><a href=\"https:\/\/www.amazon.com\/Deep-Learning-Adaptive-Computation-Machine\/dp\/0262035618\/ref=as_li_ss_tl?dchild=1&amp;keywords=deep+learning&amp;qid=1606171954&amp;s=books&amp;sr=1-1&amp;linkCode=sl1&amp;tag=inspiredalgor-20&amp;linkId=0a0c58945768a65548b639df6d1a98ed&amp;language=en_US\">Deep learning<\/a> by Ian Goodfellow, Joshua Begio, Aaron Courville.<\/li>\n<li><a href=\"https:\/\/amzn.to\/35Yeolv\">Thomas Calculus<\/a>, 14th edition, 2017. (based on the original works of George B. Thomas, revised by Joel Hass, Christopher Heil, Maurice Weir)<\/li>\n<li><a href=\"https:\/\/www.amazon.com\/Calculus-3rd-Gilbert-Strang\/dp\/0980232759\/ref=as_li_ss_tl?dchild=1&amp;keywords=Gilbert+Strang+calculus&amp;qid=1606171602&amp;s=books&amp;sr=1-1&amp;linkCode=sl1&amp;tag=inspiredalgor-20&amp;linkId=423b93db012f7cc6bb92cb7494a3095f&amp;language=en_US\">Calculus<\/a>, 3rd Edition, 2017. (Gilbert Strang)<\/li>\n<li><a href=\"https:\/\/amzn.to\/3kS9I52\">Calculus<\/a>, 8th edition, 2015. (James Stewart)<\/li>\n<\/ul>\n<h2><b>Summary<\/b><\/h2>\n<p>In this tutorial, you discovered what is Taylor series expansion of a function about a point. Specifically, you learned:<\/p>\n<ul>\n<li>Power series and Taylor series<\/li>\n<li>Taylor polynomials<\/li>\n<li>How to approximate functions around a value using Taylor polynomials<\/li>\n<\/ul>\n<h2><b>Do you have any questions?<\/b><\/h2>\n<p>Ask your questions in the comments below and I will do my best to answer<\/p>\n<\/p><\/div>\n","protected":false},"excerpt":{"rendered":"<p>https:\/\/machinelearningmastery.com\/a-gentle-introduction-to-taylor-series\/<\/p>\n","protected":false},"author":0,"featured_media":850,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[],"_links":{"self":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/849"}],"collection":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/comments?post=849"}],"version-history":[{"count":0,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/849\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media\/850"}],"wp:attachment":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media?parent=849"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/categories?post=849"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/tags?post=849"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}