{"id":847,"date":"2021-09-05T15:53:36","date_gmt":"2021-09-05T15:53:36","guid":{"rendered":"https:\/\/salarydistribution.com\/machine-learning\/2021\/09\/05\/calculus-in-action-neural-networks\/"},"modified":"2021-09-05T15:53:36","modified_gmt":"2021-09-05T15:53:36","slug":"calculus-in-action-neural-networks","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2021\/09\/05\/calculus-in-action-neural-networks\/","title":{"rendered":"Calculus in Action: Neural Networks"},"content":{"rendered":"<div id=\"\">\n<p>An artificial neural network is a computational model that approximates a mapping between inputs and outputs.<span class=\"Apple-converted-space\">\u00a0<\/span><\/p>\n<p>It is inspired by the structure of the human brain, in that it is similarly composed of a network of interconnected neurons that propagate information upon receiving sets of stimuli from neighbouring neurons.<\/p>\n<p>Training a neural network involves a process that employs the backpropagation and gradient descent algorithms in tandem. As we will be seeing, both of these algorithms make extensive use of calculus.<\/p>\n<p>In this tutorial, you will discover how aspects of calculus are applied in neural networks.<span class=\"Apple-converted-space\">\u00a0<\/span><\/p>\n<p>After completing this tutorial, you will know:<\/p>\n<ul>\n<li>An artificial neural network is organized into layers of neurons and connections, where the latter are attributed a weight value each.<\/li>\n<li>Each neuron implements a nonlinear function that maps a set of inputs to an output activation.<\/li>\n<li>In training a neural network, calculus is used extensively by the backpropagation and gradient descent algorithms.<span class=\"Apple-converted-space\">\u00a0<\/span><\/li>\n<\/ul>\n<p>Let\u2019s get started.<span class=\"Apple-converted-space\">\u00a0<\/span><\/p>\n<p>\u00a0<\/p>\n<div id=\"attachment_12787\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_cover-scaled.jpg\"><img aria-describedby=\"caption-attachment-12787\" loading=\"lazy\" class=\"wp-image-12787 size-large\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_cover-1024x576.jpg\" alt=\"\" width=\"1024\" height=\"576\"><img decoding=\"async\" aria-describedby=\"caption-attachment-12787\" loading=\"lazy\" class=\"wp-image-12787 size-large\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_cover-1024x576.jpg\" alt=\"\" width=\"1024\" height=\"576\"><\/a><\/p>\n<p id=\"caption-attachment-12787\" class=\"wp-caption-text\">Calculus in Action: Neural Networks<br \/>Photo by <a href=\"https:\/\/unsplash.com\/photos\/T1Wru10gKhg\">Tomoe Steineck<\/a>, some rights reserved.<\/p>\n<\/div>\n<p>\u00a0<\/p>\n<h2><b>Tutorial Overview<\/b><\/h2>\n<p>This tutorial is divided into three parts; they are:<\/p>\n<ul>\n<li>An Introduction to the Neural Network<\/li>\n<li>The Mathematics of a Neuron<\/li>\n<li>Training the Network<\/li>\n<\/ul>\n<h2><b>Prerequisites<\/b><\/h2>\n<p>For this tutorial, we assume that you already know what are:<\/p>\n<p>You can review these concepts by clicking on the links given above.<\/p>\n<h2><b>An Introduction to the Neural Network<\/b><\/h2>\n<p>Artificial neural networks can be considered as function approximation algorithms.<span class=\"Apple-converted-space\">\u00a0<\/span><\/p>\n<p>In a supervised learning setting, when presented with many input observations representing the problem of interest, together with their corresponding target outputs, the artificial neural network will seek to approximate the mapping that exists between the two.<span class=\"Apple-converted-space\">\u00a0<\/span><\/p>\n<blockquote>\n<p><i>A neural network is a computational model that is inspired by the structure of the human brain.<\/i><span class=\"Apple-converted-space\">\u00a0<\/span><\/p>\n<p>\u2013 Page 65, <a href=\"https:\/\/www.amazon.com\/Deep-Learning-Press-Essential-Knowledge\/dp\/0262537559\/ref=sr_1_11?dchild=1&amp;keywords=deep+learning&amp;qid=1627991691&amp;sr=8-11\">Deep Learning<\/a>, 2019.<span class=\"Apple-converted-space\">\u00a0<\/span><\/p>\n<\/blockquote>\n<p>The human brain consists of a massive network of interconnected neurons (around one hundred billion of them), with each comprising a cell body, a set of fibres called dendrites, and an axon:<\/p>\n<div id=\"attachment_12774\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_1.png\"><img aria-describedby=\"caption-attachment-12774\" loading=\"lazy\" class=\"wp-image-12774\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_1-1024x455.png\" alt=\"\" width=\"450\" height=\"200\"><img decoding=\"async\" aria-describedby=\"caption-attachment-12774\" loading=\"lazy\" class=\"wp-image-12774\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_1-1024x455.png\" alt=\"\" width=\"450\" height=\"200\"><\/a><\/p>\n<p id=\"caption-attachment-12774\" class=\"wp-caption-text\">A Neuron in the Human Brain<\/p>\n<\/div>\n<p>The dendrites act as the input channels to a neuron, whereas the axon acts as the output channel. Therefore, a neuron would receive input signals through its dendrites, which in turn would be connected to the (output) axons of other neighbouring neurons. In this manner, a sufficiently strong electrical pulse (also called an action potential) can be transmitted along the axon of one neuron, to all the other neurons that are connected to it. This permits signals to be propagated along the structure of the human brain.<span class=\"Apple-converted-space\">\u00a0<\/span><\/p>\n<blockquote>\n<p><i>So, a neuron acts as an all-or-none switch, that takes in a set of inputs and either outputs an action potential or no output.<span class=\"Apple-converted-space\">\u00a0<\/span><\/i><\/p>\n<p>\u2013 Page 66, <a href=\"https:\/\/www.amazon.com\/Deep-Learning-Press-Essential-Knowledge\/dp\/0262537559\/ref=sr_1_11?dchild=1&amp;keywords=deep+learning&amp;qid=1627991691&amp;sr=8-11\">Deep Learning<\/a>, 2019.<\/p>\n<\/blockquote>\n<p>An artificial neural network is analogous to the structure of the human brain, because (1) it is similarly composed of a large number of interconnected neurons that, (2) seek to propagate information across the network by, (3) receiving sets of stimuli from neighbouring neurons and mapping these to outputs, to be fed to the next layer of neurons.<span class=\"Apple-converted-space\">\u00a0<\/span><\/p>\n<p>The structure of an artificial neural network is typically organised into layers of neurons (<a href=\"https:\/\/machinelearningmastery.com\/?p=12720&amp;preview=true\">recall<\/a> the depiction of a tree diagram). For example, the following diagram illustrates a fully-connected<span class=\"Apple-converted-space\">\u00a0\u00a0<\/span>neural network, where all the neurons in one layer are connected to all the neurons in the next layer:<\/p>\n<div id=\"attachment_12790\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_21.png\"><img aria-describedby=\"caption-attachment-12790\" loading=\"lazy\" class=\"wp-image-12790\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_21-1024x874.png\" alt=\"\" width=\"473\" height=\"404\"><img decoding=\"async\" aria-describedby=\"caption-attachment-12790\" loading=\"lazy\" class=\"wp-image-12790\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_21-1024x874.png\" alt=\"\" width=\"473\" height=\"404\"><\/a><\/p>\n<p id=\"caption-attachment-12790\" class=\"wp-caption-text\">A Fully-Connected, Feedforward Neural Network<\/p>\n<\/div>\n<p>The inputs are presented on the left hand side of the network, and the information propagates<span class=\"Apple-converted-space\">\u00a0 <\/span>(or flows) rightward towards the outputs at the opposite end. Since the information is, hereby, propagating in the <i>forward<\/i> direction through the network, then we would also refer to such a network as a <i>feedforward neural network<\/i>.<span class=\"Apple-converted-space\">\u00a0<\/span><\/p>\n<p>The layers of neurons in between the input and output layers are called <i>hidden<\/i> layers, because they are not directly accessible.<span class=\"Apple-converted-space\">\u00a0<\/span><\/p>\n<p>Each connection (represented by an arrow in the diagram) between two neurons is attributed a weight, which acts on the data flowing through the network, as we will see shortly.<span class=\"Apple-converted-space\">\u00a0<\/span><\/p>\n<h2><b>The Mathematics of a Neuron<\/b><\/h2>\n<p>More specifically, let\u2019s say that a particular artificial neuron (or a <em>perceptron<\/em>, as Frank Rosenblatt had initially named it) receives <i>n<\/i> inputs, [<i>x<\/i><sub>1<\/sub>, \u2026, <i>x<\/i><sub>n<\/sub>], where each connection is attributed a corresponding weight, [<i>w<\/i><sub>1<\/sub>, \u2026, <i>w<\/i><sub>n<\/sub>].<span class=\"Apple-converted-space\">\u00a0<\/span><\/p>\n<p>The first operation that is carried out multiplies the input values by their corresponding weight, and adds a bias term, <i>b<\/i>, to their sum, producing an output, <i>z<\/i>:<\/p>\n<p><i>z<\/i> = ((<i>x<\/i><sub>1<\/sub> \u00d7 <i>w<\/i><sub>1<\/sub>) + (<i>x<\/i><sub>2<\/sub> \u00d7 <i>w<\/i><sub>2<\/sub>) + \u2026 + (<i>x<\/i><sub>n<\/sub> \u00d7 <i>w<\/i><sub>n<\/sub>)) + <i>b<\/i><\/p>\n<p>We can, alternatively, represent this operation in a more compact form as follows:<\/p>\n<p><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_5.png\"><img loading=\"lazy\" class=\"aligncenter wp-image-12778\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_5.png\" alt=\"\" width=\"179\" height=\"66\"><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter wp-image-12778\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_5.png\" alt=\"\" width=\"179\" height=\"66\"><\/a><\/p>\n<p>This weighted sum calculation that we have performed so far is a linear operation. If every neuron had to implement this particular calculation alone, then the neural network would be restricted to learning only linear input-output mappings.<span class=\"Apple-converted-space\">\u00a0<\/span><\/p>\n<blockquote>\n<p><i>However, many of the relationships in the world that we might want to model are nonlinear, and if we attempt to model these relationships using a linear model, then the model will be very inaccurate.<span class=\"Apple-converted-space\">\u00a0<\/span><\/i><\/p>\n<p>\u2013 Page 77, <a href=\"https:\/\/www.amazon.com\/Deep-Learning-Press-Essential-Knowledge\/dp\/0262537559\/ref=sr_1_11?dchild=1&amp;keywords=deep+learning&amp;qid=1627991691&amp;sr=8-11\">Deep Learning<\/a>, 2019.<\/p>\n<\/blockquote>\n<p>Hence, a second operation is performed by each neuron that transforms the weighted sum by the application of a nonlinear activation function, <i>a<\/i>(.):<span class=\"Apple-converted-space\">\u00a0<\/span><\/p>\n<p><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_6.png\"><img loading=\"lazy\" class=\"aligncenter wp-image-12779\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_6-1024x259.png\" alt=\"\" width=\"296\" height=\"75\"><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter wp-image-12779\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_6-1024x259.png\" alt=\"\" width=\"296\" height=\"75\"><\/a><\/p>\n<p>We can represent the operations performed by each neuron even more compactly, if we had to integrate the bias term into the sum as another weight, <i>w<\/i><sub>0<\/sub> (notice that the sum now starts from 0):<\/p>\n<p><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_7.png\"><img loading=\"lazy\" class=\"aligncenter wp-image-12780\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_7.png\" alt=\"\" width=\"221\" height=\"70\"><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter wp-image-12780\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_7.png\" alt=\"\" width=\"221\" height=\"70\"><\/a><\/p>\n<p>The operations performed by each neuron can be illustrated as follows:<\/p>\n<div id=\"attachment_12776\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_3.png\"><img aria-describedby=\"caption-attachment-12776\" loading=\"lazy\" class=\"wp-image-12776\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_3-1024x898.png\" alt=\"\" width=\"321\" height=\"282\"><img decoding=\"async\" aria-describedby=\"caption-attachment-12776\" loading=\"lazy\" class=\"wp-image-12776\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_3-1024x898.png\" alt=\"\" width=\"321\" height=\"282\"><\/a><\/p>\n<p id=\"caption-attachment-12776\" class=\"wp-caption-text\">Nonlinear Function Implemented by a Neuron<\/p>\n<\/div>\n<p>Therefore, each neuron can be considered to implement a nonlinear function that maps a set of inputs to an output activation.<span class=\"Apple-converted-space\">\u00a0<\/span><\/p>\n<h2><b>Training the Network<\/b><\/h2>\n<p>Training an artificial neural network involves the process of searching for the set of weights that model best the patterns in the data. It is a process that employs the backpropagation and gradient descent algorithms in tandem. Both of these algorithms make extensive use of calculus.<span class=\"Apple-converted-space\">\u00a0<\/span><\/p>\n<p>Each time that the network is traversed in the forward (or rightward) direction, the error of the network can be calculated as the difference between the output produced by the network and the expected ground truth, by means of a loss function (such as the sum of squared errors (SSE)). The backpropagation algorithm, then, calculates the gradient (or the rate of change) of this error to changes in the weights. In order to do so, it requires the use of the chain rule and partial derivatives.<span class=\"Apple-converted-space\">\u00a0<\/span><\/p>\n<p>For simplicity, consider a network made up of two neurons connected by a single path of activation. If we had to break them open, we would find that the neurons perform the following operations in cascade:<\/p>\n<div id=\"attachment_12777\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_4.png\"><img aria-describedby=\"caption-attachment-12777\" loading=\"lazy\" class=\"wp-image-12777\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_4-1024x413.png\" alt=\"\" width=\"589\" height=\"238\"><img decoding=\"async\" aria-describedby=\"caption-attachment-12777\" loading=\"lazy\" class=\"wp-image-12777\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_4-1024x413.png\" alt=\"\" width=\"589\" height=\"238\"><\/a><\/p>\n<p id=\"caption-attachment-12777\" class=\"wp-caption-text\">Operations Performed by Two Neurons in Cascade<\/p>\n<\/div>\n<p>The first application of the chain rule connects the overall error of the network to the input, <i>z<\/i><sub>2<\/sub>, of the activation function <i>a<\/i><sub>2<\/sub> of the second neuron, and subsequently to the weight, <i>w<\/i><sub>2<\/sub>, as follows:<\/p>\n<p><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_8.png\"><img loading=\"lazy\" class=\"aligncenter wp-image-12781\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_8-1024x177.png\" alt=\"\" width=\"324\" height=\"56\"><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter wp-image-12781\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_8-1024x177.png\" alt=\"\" width=\"324\" height=\"56\"><\/a><\/p>\n<p>You may notice that the application of the chain rule involves, among other terms, a multiplication by the partial derivative of the neuron\u2019s activation function with respect to its input, <i>z<\/i><sub>2<\/sub>. There are different activation functions to choose from, such as the sigmoid or the logistic functions. If we had to take the logistic function as an example, then its partial derivative would be computed as follows:<\/p>\n<p><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_9.png\"><img loading=\"lazy\" class=\"aligncenter wp-image-12782\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_9-1024x154.png\" alt=\"\" width=\"372\" height=\"56\"><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter wp-image-12782\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_9-1024x154.png\" alt=\"\" width=\"372\" height=\"56\"><\/a><\/p>\n<p>Hence, we can compute ?<sub>2<\/sub> as follows:<\/p>\n<p><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_10.png\"><img loading=\"lazy\" class=\"aligncenter wp-image-12783\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_10-1024x96.png\" alt=\"\" width=\"341\" height=\"32\"><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter wp-image-12783\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_10-1024x96.png\" alt=\"\" width=\"341\" height=\"32\"><\/a><\/p>\n<p>Here, <i>t<\/i><sub>2<\/sub> is the expected activation, and in finding the difference between <i>t<\/i><sub>2<\/sub> and <i>a<\/i><sub>2<\/sub> we are, therefore, computing the error between the activation generated by the network and the expected ground truth.<span class=\"Apple-converted-space\">\u00a0<\/span><\/p>\n<p>Since we are computing the derivative of the activation function, it should, therefore, be continuous and differentiable over the entire space of real numbers. In the case of deep neural networks, the error gradient is propagated backwards over a large number of hidden layers. This can cause the error signal to rapidly diminish to zero, especially if the maximum value of the derivative function is already small to begin with (for instance, the inverse of the logistic function has a maximum value of 0.25). This is known as the <i>vanishing gradient problem<\/i>. The ReLU function has been so popularly used in deep learning to alleviate this problem, because its derivative in the positive portion of its domain is equal to 1.<span class=\"Apple-converted-space\">\u00a0<\/span><\/p>\n<p>The next weight backwards is deeper into the network and, hence, the application of the chain rule can similarly be extended to connect the overall error to the weight, <i>w<\/i><sub>1<\/sub>, as follows:<\/p>\n<p><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_11.png\"><img loading=\"lazy\" class=\"aligncenter wp-image-12784\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_11-1024x144.png\" alt=\"\" width=\"398\" height=\"56\"><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter wp-image-12784\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_11-1024x144.png\" alt=\"\" width=\"398\" height=\"56\"><\/a><\/p>\n<p>If we take the logistic function again as the activation function of choice, then we would compute ?<sub>1<\/sub> as follows:<\/p>\n<p><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_12.png\"><img loading=\"lazy\" class=\"wp-image-12785 aligncenter\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_12-1024x427.png\" alt=\"\" width=\"339\" height=\"141\"><img decoding=\"async\" loading=\"lazy\" class=\"wp-image-12785 aligncenter\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_12-1024x427.png\" alt=\"\" width=\"339\" height=\"141\"><\/a><\/p>\n<p>Once we have computed the gradient of the network error with respect to each weight, then the gradient descent algorithm can be applied to update each weight for the next <i>forward propagation<\/i> at time, <i>t<\/i>+1. For the weight, <i>w<\/i><sub>1<\/sub>, the weight update rule using gradient descent would be specified as follows:<\/p>\n<p><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_13.png\"><img loading=\"lazy\" class=\"aligncenter wp-image-12786\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_13-1024x242.png\" alt=\"\" width=\"236\" height=\"56\"><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter wp-image-12786\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/08\/neural_networks_13-1024x242.png\" alt=\"\" width=\"236\" height=\"56\"><\/a><\/p>\n<p>Even though we have hereby considered a simple network, the process that we have gone through can be extended to evaluate more complex and deeper ones, such convolutional neural networks (CNNs).<span class=\"Apple-converted-space\">\u00a0<\/span><\/p>\n<p>If the network under consideration is characterised by multiple branches coming from multiple inputs (and possibly flowing towards multiple outputs), then its evaluation would involve the summation of different derivative chains for each path, similarly to how we have previously derived the generalized chain rule.<span class=\"Apple-converted-space\">\u00a0<\/span><\/p>\n<h2><b>Further Reading<\/b><\/h2>\n<p>This section provides more resources on the topic if you are looking to go deeper.<\/p>\n<h3><b>Books<\/b><\/h3>\n<h2><b>Summary<\/b><\/h2>\n<p>In this tutorial, you discovered\u00a0how aspects of calculus are applied in neural networks.<span class=\"Apple-converted-space\">\u00a0<\/span><\/p>\n<p>Specifically, you learned:<\/p>\n<ul>\n<li>An artificial neural network is organized into layers of neurons and connections, where the latter are each attributed a weight value.<\/li>\n<li>Each neuron implements a nonlinear function that maps a set of inputs to an output activation.<\/li>\n<li>In training a neural network, calculus is used extensively by the backpropagation and gradient descent algorithms.<\/li>\n<\/ul>\n<p>Do you have any questions?<br \/>Ask your questions in the comments below and I will do my best to answer.<\/p>\n<\/p><\/div>\n","protected":false},"excerpt":{"rendered":"<p>https:\/\/machinelearningmastery.com\/calculus-in-action-neural-networks\/<\/p>\n","protected":false},"author":0,"featured_media":848,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[],"_links":{"self":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/847"}],"collection":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/comments?post=847"}],"version-history":[{"count":0,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/847\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media\/848"}],"wp:attachment":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media?parent=847"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/categories?post=847"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/tags?post=847"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}