{"id":993,"date":"2021-10-05T08:39:42","date_gmt":"2021-10-05T08:39:42","guid":{"rendered":"https:\/\/salarydistribution.com\/machine-learning\/2021\/10\/05\/understanding-simple-recurrent-neural-networks-in-keras\/"},"modified":"2021-10-05T08:39:42","modified_gmt":"2021-10-05T08:39:42","slug":"understanding-simple-recurrent-neural-networks-in-keras","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2021\/10\/05\/understanding-simple-recurrent-neural-networks-in-keras\/","title":{"rendered":"Understanding Simple Recurrent Neural Networks In Keras"},"content":{"rendered":"<div id=\"\">\n<p>This tutorial is designed for anyone looking for an understanding of how recurrent neural networks (RNN) work and how to use them via the Keras deep learning library. While all the methods required for solving problems and building applications are provided by the Keras library, it is also important to gain an insight on how everything works. In this article, the computations taking place in the RNN model are shown step by step. Next, a complete end to end system for time series prediction is developed.<\/p>\n<p>After completing this tutorial, you will know:<\/p>\n<ul>\n<li>The structure of RNN<\/li>\n<li>How RNN computes the output when given an input<\/li>\n<li>How to prepare data for a SimpleRNN in Keras<\/li>\n<li>How to train a SimpleRNN model<\/li>\n<\/ul>\n<p>Let\u2019s get started.<\/p>\n<div id=\"attachment_12889\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/09\/IMG_9433-scaled.jpg\"><img aria-describedby=\"caption-attachment-12889\" loading=\"lazy\" class=\"wp-image-12889\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/09\/IMG_9433-300x225.jpg\" alt=\"Umstead state park\" width=\"745\" height=\"559\"><img decoding=\"async\" aria-describedby=\"caption-attachment-12889\" loading=\"lazy\" class=\"wp-image-12889\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/09\/IMG_9433-300x225.jpg\" alt=\"Umstead state park\" width=\"745\" height=\"559\"><\/a><\/p>\n<p id=\"caption-attachment-12889\" class=\"wp-caption-text\">Understanding Simple Recurrent Neural Networks In Keras Photo by Mehreen Saeed, some rights reserved.<\/p>\n<\/div>\n<h2 id=\"Tutorial-Overview\">Tutorial Overview<\/h2>\n<p>This tutorial is divided into two parts; they are:<\/p>\n<ol>\n<li>The structure of the RNN\n<ol>\n<li>Different weights and biases associated with different layers of the RNN.<\/li>\n<li>How computations are performed to compute the output when given an input.<\/li>\n<\/ol>\n<\/li>\n<li>A complete application for time series prediction.<\/li>\n<\/ol>\n<h2 id=\"Prerequisites\">Prerequisites<\/h2>\n<p>It is assumed that you have a basic understanding of RNNs before you start implementing them. <a href=\"https:\/\/machinelearningmastery.com\/an-introduction-to-recurrent-neural-networks-and-the-math-that-powers-them\" target=\"_blank\" rel=\"noopener\">An Introduction To Recurrent Neural Networks And The Math That Powers Them<\/a> gives you a quick overview of RNNs.<\/p>\n<p>Let\u2019s now get right down to the implementation part.<\/p>\n<h2 id=\"Import-section\">Import section<\/h2>\n<p>To start the implementation of RNNs, let\u2019s add the import section.<\/p>\n<div id=\"urvanov-syntax-highlighter-615b506a0925e123833014\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-mac print-yes notranslate\" data-settings=\" minimize scroll-mouseover disable-anim\">\n<p><textarea class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" readonly><br \/>\nfrom pandas import read_csv<br \/>\nimport numpy as np<br \/>\nfrom keras.models import Sequential<br \/>\nfrom keras.layers import Dense, SimpleRNN<br \/>\nfrom sklearn.preprocessing import MinMaxScaler<br \/>\nfrom sklearn.metrics import mean_squared_error<br \/>\nimport math<br \/>\nimport matplotlib.pyplot as plt<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\">\n<p><span class=\"crayon-st\">from<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">pandas <\/span><span class=\"crayon-r\">import<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">read_csv<\/span><\/p>\n<p><span class=\"crayon-r\">import<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">numpy <\/span><span class=\"crayon-st\">as<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">np<\/span><\/p>\n<p><span class=\"crayon-st\">from<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">keras<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">models <\/span><span class=\"crayon-r\">import<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">Sequential<\/span><\/p>\n<p><span class=\"crayon-st\">from<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">keras<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">layers <\/span><span class=\"crayon-r\">import<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">Dense<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">SimpleRNN<\/span><\/p>\n<p><span class=\"crayon-st\">from<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">sklearn<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">preprocessing <\/span><span class=\"crayon-r\">import<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">MinMaxScaler<\/span><\/p>\n<p><span class=\"crayon-st\">from<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">sklearn<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">metrics <\/span><span class=\"crayon-r\">import<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">mean_squared_error<\/span><\/p>\n<p><span class=\"crayon-r\">import<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-k \">math<\/span><\/p>\n<p><span class=\"crayon-r\">import<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">matplotlib<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">pyplot <\/span><span class=\"crayon-st\">as<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">plt<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<h2 id=\"Kera's-SimpleRNN\">Keras SimpleRNN<\/h2>\n<p>The function below returns a model that includes a <code>SimpleRNN<\/code> layer and a <code>Dense<\/code> layer for learning sequential data. The <code>input_shape<\/code> specifies the parameter <code>(time_steps x features)<\/code>. We\u2019ll simplify everything and use univariate data, i.e., one feature only; the time_steps are discussed below.<\/p>\n<div id=\"urvanov-syntax-highlighter-615b506a09263375353476\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-mac print-yes notranslate\" data-settings=\" minimize scroll-mouseover disable-anim\">\n<p><textarea class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" readonly><br \/>\ndef create_RNN(hidden_units, dense_units, input_shape, activation):<br \/>\n    model = Sequential()<br \/>\n    model.add(SimpleRNN(hidden_units, input_shape=input_shape,<br \/>\n                        activation=activation[0]))<br \/>\n    model.add(Dense(units=dense_units, activation=activation[1]))<br \/>\n    model.compile(loss=&#8217;mean_squared_error&#8217;, optimizer=&#8217;adam&#8217;)<br \/>\n    return model<\/p>\n<p>demo_model = create_RNN(2, 1, (3,1), activation=[&#8216;linear&#8217;, &#8216;linear&#8217;])<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\">\n<p><span class=\"crayon-r\">def<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">create_RNN<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">hidden_units<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">dense_units<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">input_shape<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">activation<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">model<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">Sequential<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">model<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">add<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e\">SimpleRNN<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">hidden_units<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">input_shape<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-v\">input_shape<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">activation<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-v\">activation<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">model<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">add<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e\">Dense<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">units<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-v\">dense_units<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">activation<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-v\">activation<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">model<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-k \">compile<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">loss<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8216;mean_squared_error&#8217;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">optimizer<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8216;adam&#8217;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">model<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-v\">demo_model<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">create_RNN<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">2<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">3<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">activation<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-s\">&#8216;linear&#8217;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8216;linear&#8217;<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<p>The object <code>demo_model<\/code> is returned with 2 hidden units created via a the <code>SimpleRNN<\/code> layer and 1 dense unit created via the <code>Dense<\/code> layer. The <code>input_shape<\/code> is set at 3\u00d71 and a <code>linear<\/code> activation function is used in both layers for simplicity. Just to recall the linear activation function $f(x) = x$ makes no change in the input. The network looks as follows:<\/p>\n<p>If we have $m$ hidden units ($m=2$ in the above case), then:<\/p>\n<ul>\n<li>Input: $x in R$<\/li>\n<li>Hidden unit: $h in R^m$<\/li>\n<li>Weights for input units: $w_x in R^m$<\/li>\n<li>Weights for hidden units: $w_h in R^{mxm}$<\/li>\n<li>Bias for hidden units: $b_h in R^m$<\/li>\n<li>Weight for the dense layer: $w_y in R^m$<\/li>\n<li>Bias for the dense layer: $b_y in R$<\/li>\n<\/ul>\n<p>Let\u2019s look at the above weights. Note: As the weights are initialized randomly, the results pasted here will be different from yours. The important thing is to learn what the structure of each object being used looks like and how it interacts with others to produce the final output.<\/p>\n<div id=\"urvanov-syntax-highlighter-615b506a09264943401642\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-mac print-yes notranslate\" data-settings=\" minimize scroll-mouseover disable-anim\">\n<p><textarea class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" readonly><br \/>\nwx = demo_model.get_weights()[0]<br \/>\nwh = demo_model.get_weights()[1]<br \/>\nbh = demo_model.get_weights()[2]<br \/>\nwy = demo_model.get_weights()[3]<br \/>\nby = demo_model.get_weights()[4]<\/p>\n<p>print(&#8216;wx = &#8216;, wx, &#8216; wh = &#8216;, wh, &#8216; bh = &#8216;, bh, &#8216; wy =&#8217;, wy, &#8216;by = &#8216;, by)<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\">\n<p><span class=\"crayon-v\">wx<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">demo_model<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">get_weights<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-v\">wh<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">demo_model<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">get_weights<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-v\">bh<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">demo_model<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">get_weights<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">2<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-v\">wy<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">demo_model<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">get_weights<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">3<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-v\">by<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">demo_model<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">get_weights<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">4<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-k \">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8216;wx = &#8216;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">wx<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8216; wh = &#8216;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">wh<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8216; bh = &#8216;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">bh<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8216; wy =&#8217;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">wy<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8216;by = &#8216;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">by<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<div id=\"urvanov-syntax-highlighter-615b506a09265320830188\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-mac print-yes notranslate\" data-settings=\" minimize scroll-mouseover disable-anim\">\n<p><textarea class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" readonly><br \/>\nwx =  [[ 0.18662322 -1.2369459 ]]  wh =  [[ 0.86981213 -0.49338293]<br \/>\n [ 0.49338293  0.8698122 ]]  bh =  [0. 0.]  wy = [[-0.4635998]<br \/>\n [ 0.6538409]] by =  [0.]<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\">\n<p>wx =\u00a0\u00a0[[ 0.18662322 -1.2369459 ]]\u00a0\u00a0wh =\u00a0\u00a0[[ 0.86981213 -0.49338293]<\/p>\n<p> [ 0.49338293\u00a0\u00a00.8698122 ]]\u00a0\u00a0bh =\u00a0\u00a0[0. 0.]\u00a0\u00a0wy = [[-0.4635998]<\/p>\n<p> [ 0.6538409]] by =\u00a0\u00a0[0.]<\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<p>Now let\u2019s do a simple experiment to see how the layers from a SimpleRNN and Dense layer produce an output. Keep this figure in view.<\/p>\n<div id=\"attachment_12886\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/09\/rnnCode1.png\"><img aria-describedby=\"caption-attachment-12886\" loading=\"lazy\" class=\"wp-image-12886\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/09\/rnnCode1-300x100.png\" alt=\"Layers Of A Recurrent Neural Network\" width=\"666\" height=\"222\"><img decoding=\"async\" aria-describedby=\"caption-attachment-12886\" loading=\"lazy\" class=\"wp-image-12886\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/09\/rnnCode1-300x100.png\" alt=\"Layers Of A Recurrent Neural Network\" width=\"666\" height=\"222\"><\/a><\/p>\n<p id=\"caption-attachment-12886\" class=\"wp-caption-text\">Layers Of A Recurrent Neural Network<\/p>\n<\/div>\n<p>We\u2019ll input <code>x<\/code> for three time steps and let the network generate an output. The values of the hidden units at time steps 1, 2 and 3 will be computed. $h_0$ is initialized to the zero vector. The output $o_3$ is computed from $h_3$ and $w_y$. An activation function is not required as we are using linear units.<\/p>\n<div id=\"urvanov-syntax-highlighter-615b506a09266830779867\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-mac print-yes notranslate\" data-settings=\" minimize scroll-mouseover disable-anim\">\n<p><textarea class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" readonly><br \/>\nx = np.array([1, 2, 3])<br \/>\n# Reshape the input to the required sample_size x time_steps x features<br \/>\nx_input = np.reshape(x,(1, 3, 1))<br \/>\ny_pred_model = demo_model.predict(x_input)<\/p>\n<p>m = 2<br \/>\nh0 = np.zeros(m)<br \/>\nh1 = np.dot(x[0], wx) + h0 + bh<br \/>\nh2 = np.dot(x[1], wx) + np.dot(h1,wh) + bh<br \/>\nh3 = np.dot(x[2], wx) + np.dot(h2,wh) + bh<br \/>\no3 = np.dot(h3, wy) + by<\/p>\n<p>print(&#8216;h1 = &#8216;, h1,&#8217;h2 = &#8216;, h2,&#8217;h3 = &#8216;, h3)<\/p>\n<p>print(&#8220;Prediction from network &#8220;, y_pred_model)<br \/>\nprint(&#8220;Prediction from our computation &#8220;, o3)<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<div class=\"urvanov-syntax-highlighter-nums-content\">\n<p>1<\/p>\n<p>2<\/p>\n<p>3<\/p>\n<p>4<\/p>\n<p>5<\/p>\n<p>6<\/p>\n<p>7<\/p>\n<p>8<\/p>\n<p>9<\/p>\n<p>10<\/p>\n<p>11<\/p>\n<p>12<\/p>\n<p>13<\/p>\n<p>14<\/p>\n<p>15<\/p>\n<p>16<\/p>\n<p>17<\/p>\n<\/div>\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\">\n<p><span class=\"crayon-v\">x<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-k \">array<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">2<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">3<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-c\"># Reshape the input to the required sample_size x time_steps x features <\/span><\/p>\n<p><span class=\"crayon-v\">x_input<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">reshape<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">x<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">3<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-v\">y_pred_model<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">demo_model<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">predict<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">x_input<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-v\">m<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">2<\/span><\/p>\n<p><span class=\"crayon-v\">h0<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">zeros<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">m<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-v\">h1<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">dot<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">x<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">wx<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">h0<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">bh<\/span><\/p>\n<p><span class=\"crayon-v\">h2<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">dot<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">x<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">wx<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">dot<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">h1<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-v\">wh<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">bh<\/span><\/p>\n<p><span class=\"crayon-v\">h3<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">dot<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">x<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">2<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">wx<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">dot<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">h2<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-v\">wh<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">bh<\/span><\/p>\n<p><span class=\"crayon-v\">o3<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">dot<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">h3<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">wy<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">by<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-k \">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8216;h1 = &#8216;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">h1<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-s\">&#8216;h2 = &#8216;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">h2<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-s\">&#8216;h3 = &#8216;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">h3<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-k \">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;Prediction from network &#8220;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_pred_model<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-k \">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;Prediction from our computation &#8220;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">o3<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<div id=\"urvanov-syntax-highlighter-615b506a09267718298126\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-mac print-yes notranslate\" data-settings=\" minimize scroll-mouseover disable-anim\">\n<p><textarea class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" readonly><br \/>\nh1 =  [[ 0.18662322 -1.23694587]] h2 =  [[-0.07471441 -3.64187904]] h3 =  [[-1.30195881 -6.84172557]]<br \/>\nPrediction from network  [[-3.8698118]]<br \/>\nPrediction from our computation  [[-3.86981216]]<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\">\n<p>h1 =\u00a0\u00a0[[ 0.18662322 -1.23694587]] h2 =\u00a0\u00a0[[-0.07471441 -3.64187904]] h3 =\u00a0\u00a0[[-1.30195881 -6.84172557]]<\/p>\n<p>Prediction from network\u00a0\u00a0[[-3.8698118]]<\/p>\n<p>Prediction from our computation\u00a0\u00a0[[-3.86981216]]<\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<h2 id=\"Running-The-RNN-On-Sunspots-Dataset\">Running The RNN On Sunspots Dataset<\/h2>\n<p>Now that we understand how the SimpleRNN and Dense layers are put together. Let\u2019s run a complete RNN on a simple time series dataset. We\u2019ll need to follow these steps<\/p>\n<ol>\n<li>Read the dataset from a given URL<\/li>\n<li>Split the data into training and test set<\/li>\n<li>Prepare the input to the required Keras format<\/li>\n<li>Create an RNN model and train it<\/li>\n<li>Make the predictions on training and test sets and print the root mean square error on both sets<\/li>\n<li>View the result<\/li>\n<\/ol>\n<h3 id=\"Step-1,-2:-Reading-Data-and-Splitting-Into-Train-And-Test\">Step 1, 2: Reading Data and Splitting Into Train And Test<\/h3>\n<p>The following function reads the train and test data from a given URL and splits it into a given percentage of train and test data. It returns single dimensional arrays for train and test data after scaling the data between 0 and 1 using <code>MinMaxScaler<\/code> from scikit-learn.<\/p>\n<div id=\"urvanov-syntax-highlighter-615b506a09268129916794\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-mac print-yes notranslate\" data-settings=\" minimize scroll-mouseover disable-anim\">\n<p><textarea class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" readonly><br \/>\n# Parameter split_percent defines the ratio of training examples<br \/>\ndef get_train_test(url, split_percent=0.8):<br \/>\n    df = read_csv(url, usecols=[1], engine=&#8217;python&#8217;)<br \/>\n    data = np.array(df.values.astype(&#8216;float32&#8217;))<br \/>\n    scaler = MinMaxScaler(feature_range=(0, 1))<br \/>\n    data = scaler.fit_transform(data).flatten()<br \/>\n    n = len(data)<br \/>\n    # Point for splitting data into train and test<br \/>\n    split = int(n*split_percent)<br \/>\n    train_data = data[range(split)]<br \/>\n    test_data = data[split:]<br \/>\n    return train_data, test_data, data<\/p>\n<p>sunspots_url = &#8216;https:\/\/raw.githubusercontent.com\/jbrownlee\/Datasets\/master\/monthly-sunspots.csv&#8217;<br \/>\ntrain_data, test_data, data = get_train_test(sunspots_url)<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\">\n<p><span class=\"crayon-c\"># Parameter split_percent defines the ratio of training examples<\/span><\/p>\n<p><span class=\"crayon-r\">def<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">get_train_test<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">url<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">split_percent<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">0.8<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">read_csv<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">url<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">usecols<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">engine<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8216;python&#8217;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-k \">array<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">values<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">astype<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8216;float32&#8217;<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">scaler<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">MinMaxScaler<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">feature_range<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">scaler<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">fit_transform<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">flatten<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">n<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-k \">len<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-c\"># Point for splitting data into train and test<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">split<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-k \">int<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">n<\/span><span class=\"crayon-o\">*<\/span><span class=\"crayon-v\">split_percent<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">train_data<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-k \">range<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">split<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">test_data<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">split<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">train_data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">test_data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">data<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-v\">sunspots_url<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8216;https:\/\/raw.githubusercontent.com\/jbrownlee\/Datasets\/master\/monthly-sunspots.csv&#8217;<\/span><\/p>\n<p><span class=\"crayon-v\">train_data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">test_data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">get_train_test<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">sunspots_url<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<h3 id=\"Step-3:-Reshaping-Data-For-Keras\">Step 3: Reshaping Data For Keras<\/h3>\n<p>The next step is to prepare the data for Keras model training. The input array should be shaped as: <code>total_samples x time_steps x features<\/code>.<\/p>\n<p>There are many ways of preparing time series data for training. We\u2019ll create input rows with non-overlapping time steps. An example for time_steps = 2 is shown in the figure below. Here time_steps denotes the number of previous time steps to use for predicting the next value of the time series data.<\/p>\n<div id=\"attachment_12887\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/09\/rnnCode2.png\"><img aria-describedby=\"caption-attachment-12887\" loading=\"lazy\" class=\"wp-image-12887\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/09\/rnnCode2-300x159.png\" alt=\"How Data Is Prepared For Sunspots Example\" width=\"719\" height=\"381\"><img decoding=\"async\" aria-describedby=\"caption-attachment-12887\" loading=\"lazy\" class=\"wp-image-12887\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/09\/rnnCode2-300x159.png\" alt=\"How Data Is Prepared For Sunspots Example\" width=\"719\" height=\"381\"><\/a><\/p>\n<p id=\"caption-attachment-12887\" class=\"wp-caption-text\">How Data Is Prepared For Sunspots Example<\/p>\n<\/div>\n<p>The following function <code>get_XY()<\/code> takes a one dimensional array as input and converts it to the required input <code>X<\/code>\u00a0and target <code>Y<\/code>\u00a0arrays. We\u2019ll use 12 <code>time_steps<\/code> for the sunspots dataset as the sunspots generally have a cycle of 12 months. You can experiment with other values of <code>time_steps<\/code>.<\/p>\n<div id=\"urvanov-syntax-highlighter-615b506a09269516801201\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-mac print-yes notranslate\" data-settings=\" minimize scroll-mouseover disable-anim\">\n<p><textarea class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" readonly><br \/>\n# Prepare the input X and target Y<br \/>\ndef get_XY(dat, time_steps):<br \/>\n    # Indices of target array<br \/>\n    Y_ind = np.arange(time_steps, len(dat), time_steps)<br \/>\n    Y = dat[Y_ind]<br \/>\n    # Prepare X<br \/>\n    rows_x = len(Y)<br \/>\n    X = dat[range(time_steps*rows_x)]<br \/>\n    X = np.reshape(X, (rows_x, time_steps, 1))<br \/>\n    return X, Y<\/p>\n<p>time_steps = 12<br \/>\ntrainX, trainY = get_XY(train_data, time_steps)<br \/>\ntestX, testY = get_XY(test_data, time_steps)<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\">\n<p><span class=\"crayon-c\"># Prepare the input X and target Y<\/span><\/p>\n<p><span class=\"crayon-r\">def<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">get_XY<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">dat<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">time_steps<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-c\"># Indices of target array<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">Y_ind<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">arange<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">time_steps<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-k \">len<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">dat<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">time_steps<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">Y<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">dat<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">Y_ind<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-c\"># Prepare X<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">rows_x<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-k \">len<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">Y<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">dat<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-k \">range<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">time_steps<\/span><span class=\"crayon-o\">*<\/span><span class=\"crayon-v\">rows_x<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">reshape<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">rows_x<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">time_steps<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">Y<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-v\">time_steps<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">12<\/span><\/p>\n<p><span class=\"crayon-v\">trainX<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">trainY<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">get_XY<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">train_data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">time_steps<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-v\">testX<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">testY<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">get_XY<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">test_data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">time_steps<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<h3 id=\"Step-4:-Create-RNN-Model-And-Train\">Step 4: Create RNN Model And Train<\/h3>\n<p>For this step, we can reuse our <code>create_RNN()<\/code> function that was defined above.<\/p>\n<div id=\"urvanov-syntax-highlighter-615b506a0926a058540779\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-mac print-yes notranslate\" data-settings=\" minimize scroll-mouseover disable-anim\">\n<p><textarea class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" readonly><br \/>\nmodel = create_RNN(hidden_units=3, dense_units=1, input_shape=(time_steps,1),<br \/>\n                   activation=[&#8216;tanh&#8217;, &#8216;tanh&#8217;])<br \/>\nmodel.fit(trainX, trainY, epochs=20, batch_size=1, verbose=2)<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\">\n<p><span class=\"crayon-v\">model<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">create_RNN<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">hidden_units<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">3<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">dense_units<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">input_shape<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">time_steps<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 <\/span><span class=\"crayon-v\">activation<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-s\">&#8216;tanh&#8217;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8216;tanh&#8217;<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-v\">model<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">fit<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">trainX<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">trainY<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">epochs<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">20<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">batch_size<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">verbose<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">2<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<h3 id=\"Step-5:-Compute-And-Print-The-Root-Mean-Square-Error\">Step 5: Compute And Print The Root Mean Square Error<\/h3>\n<p>The function <code>print_error()<\/code> computes the mean square error between the actual values and the predicted values.<\/p>\n<div id=\"urvanov-syntax-highlighter-615b506a0926b913488223\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-mac print-yes notranslate\" data-settings=\" minimize scroll-mouseover disable-anim\">\n<p><textarea class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" readonly><br \/>\ndef print_error(trainY, testY, train_predict, test_predict):<br \/>\n    # Error of predictions<br \/>\n    train_rmse = math.sqrt(mean_squared_error(trainY, train_predict))<br \/>\n    test_rmse = math.sqrt(mean_squared_error(testY, test_predict))<br \/>\n    # Print RMSE<br \/>\n    print(&#8216;Train RMSE: %.3f RMSE&#8217; % (train_rmse))<br \/>\n    print(&#8216;Test RMSE: %.3f RMSE&#8217; % (test_rmse))    <\/p>\n<p># make predictions<br \/>\ntrain_predict = model.predict(trainX)<br \/>\ntest_predict = model.predict(testX)<br \/>\n# Mean square error<br \/>\nprint_error(trainY, testY, train_predict, test_predict)<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\">\n<p><span class=\"crayon-r\">def<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">print_error<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">trainY<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">testY<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">train_predict<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">test_predict<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-c\"># Error of predictions<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">train_rmse<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-k \">math<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">sqrt<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e\">mean_squared_error<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">trainY<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">train_predict<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">test_rmse<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-k \">math<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">sqrt<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e\">mean_squared_error<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">testY<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">test_predict<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-c\"># Print RMSE<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-k \">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8216;Train RMSE: %.3f RMSE&#8217;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">%<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">train_rmse<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-k \">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8216;Test RMSE: %.3f RMSE&#8217;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">%<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">test_rmse<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-c\"># make predictions<\/span><\/p>\n<p><span class=\"crayon-v\">train_predict<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">model<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">predict<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">trainX<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-v\">test_predict<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">model<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">predict<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">testX<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-c\"># Mean square error<\/span><\/p>\n<p><span class=\"crayon-e\">print_error<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">trainY<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">testY<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">train_predict<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">test_predict<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<div id=\"urvanov-syntax-highlighter-615b506a0926c220587635\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-mac print-yes notranslate\" data-settings=\" minimize scroll-mouseover disable-anim\">\n<p><textarea class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" readonly><br \/>\nTrain RMSE: 0.058 RMSE<br \/>\nTest RMSE: 0.077 RMSE<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\">\n<p>Train RMSE: 0.058 RMSE<\/p>\n<p>Test RMSE: 0.077 RMSE<\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<h3 id=\"Step-6:-View-The-result\">Step 6: View The result<\/h3>\n<p>The following function plots the actual target values and the predicted value. The red line separates the training and test data points.<\/p>\n<div id=\"urvanov-syntax-highlighter-615b506a0926d228794727\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-mac print-yes notranslate\" data-settings=\" minimize scroll-mouseover disable-anim\">\n<p><textarea class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" readonly><br \/>\n# Plot the result<br \/>\ndef plot_result(trainY, testY, train_predict, test_predict):<br \/>\n    actual = np.append(trainY, testY)<br \/>\n    predictions = np.append(train_predict, test_predict)<br \/>\n    rows = len(actual)<br \/>\n    plt.figure(figsize=(15, 6), dpi=80)<br \/>\n    plt.plot(range(rows), actual)<br \/>\n    plt.plot(range(rows), predictions)<br \/>\n    plt.axvline(x=len(trainY), color=&#8217;r&#8217;)<br \/>\n    plt.legend([&#8216;Actual&#8217;, &#8216;Predictions&#8217;])<br \/>\n    plt.xlabel(&#8216;Observation number after given time steps&#8217;)<br \/>\n    plt.ylabel(&#8216;Sunspots scaled&#8217;)<br \/>\n    plt.title(&#8216;Actual and Predicted Values. The Red Line Separates The Training And Test Examples&#8217;)<br \/>\nplot_result(trainY, testY, train_predict, test_predict)<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\">\n<p><span class=\"crayon-c\"># Plot the result<\/span><\/p>\n<p><span class=\"crayon-r\">def<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">plot_result<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">trainY<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">testY<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">train_predict<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">test_predict<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">actual<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">append<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">trainY<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">testY<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">predictions<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">append<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">train_predict<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">test_predict<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">rows<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-k \">len<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">actual<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">plt<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">figure<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">figsize<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">15<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">6<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">dpi<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">80<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">plt<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">plot<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-k \">range<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">rows<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">actual<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">plt<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">plot<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-k \">range<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">rows<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">predictions<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">plt<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">axvline<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">x<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-k \">len<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">trainY<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">color<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8216;r&#8217;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">plt<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">legend<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-s\">&#8216;Actual&#8217;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8216;Predictions&#8217;<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">plt<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">xlabel<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8216;Observation number after given time steps&#8217;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">plt<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">ylabel<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8216;Sunspots scaled&#8217;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">plt<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">title<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8216;Actual and Predicted Values. The Red Line Separates The Training And Test Examples&#8217;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-e\">plot_result<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">trainY<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">testY<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">train_predict<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">test_predict<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<p>The following plot is generated:<\/p>\n<p><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/09\/rnnCode3.png\"><img loading=\"lazy\" class=\"alignnone wp-image-12885\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/09\/rnnCode3-300x130.png\" alt=\"\" width=\"766\" height=\"332\"><img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-12885\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/09\/rnnCode3-300x130.png\" alt=\"\" width=\"766\" height=\"332\"><\/a><\/p>\n<h2 id=\"Consolidated-Code\">Consolidated Code<\/h2>\n<p>Given below is the entire code for this tutorial. Do try this out at your end and experiment with different hidden units and time steps. You can add a second <code>SimpleRNN<\/code> to the network and see how it behaves. You can also use the <code>scaler<\/code> object to rescale the data back to its normal range.<\/p>\n<div id=\"urvanov-syntax-highlighter-615b506a0926e239320425\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-mac print-yes notranslate\" data-settings=\" minimize scroll-mouseover disable-anim\">\n<p><textarea class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" readonly><br \/>\n# Parameter split_percent defines the ratio of training examples<br \/>\ndef get_train_test(url, split_percent=0.8):<br \/>\n    df = read_csv(url, usecols=[1], engine=&#8217;python&#8217;)<br \/>\n    data = np.array(df.values.astype(&#8216;float32&#8242;))<br \/>\n    scaler = MinMaxScaler(feature_range=(0, 1))<br \/>\n    data = scaler.fit_transform(data).flatten()<br \/>\n    n = len(data)<br \/>\n    # Point for splitting data into train and test<br \/>\n    split = int(n*split_percent)<br \/>\n    train_data = data[range(split)]<br \/>\n    test_data = data[split:]<br \/>\n    return train_data, test_data, data<\/p>\n<p># Prepare the input X and target Y<br \/>\ndef get_XY(dat, time_steps):<br \/>\n    Y_ind = np.arange(time_steps, len(dat), time_steps)<br \/>\n    Y = dat[Y_ind]<br \/>\n    rows_x = len(Y)<br \/>\n    X = dat[range(time_steps*rows_x)]<br \/>\n    X = np.reshape(X, (rows_x, time_steps, 1))<br \/>\n    return X, Y<\/p>\n<p>def create_RNN(hidden_units, dense_units, input_shape, activation):<br \/>\n    model = Sequential()<br \/>\n    model.add(SimpleRNN(hidden_units, input_shape=input_shape, activation=activation[0]))<br \/>\n    model.add(Dense(units=dense_units, activation=activation[1]))<br \/>\n    model.compile(loss=&#8217;mean_squared_error&#8217;, optimizer=&#8217;adam&#8217;)<br \/>\n    return model<\/p>\n<p>def print_error(trainY, testY, train_predict, test_predict):<br \/>\n    # Error of predictions<br \/>\n    train_rmse = math.sqrt(mean_squared_error(trainY, train_predict))<br \/>\n    test_rmse = math.sqrt(mean_squared_error(testY, test_predict))<br \/>\n    # Print RMSE<br \/>\n    print(&#8216;Train RMSE: %.3f RMSE&#8217; % (train_rmse))<br \/>\n    print(&#8216;Test RMSE: %.3f RMSE&#8217; % (test_rmse))    <\/p>\n<p># Plot the result<br \/>\ndef plot_result(trainY, testY, train_predict, test_predict):<br \/>\n    actual = np.append(trainY, testY)<br \/>\n    predictions = np.append(train_predict, test_predict)<br \/>\n    rows = len(actual)<br \/>\n    plt.figure(figsize=(15, 6), dpi=80)<br \/>\n    plt.plot(range(rows), actual)<br \/>\n    plt.plot(range(rows), predictions)<br \/>\n    plt.axvline(x=len(trainY), color=&#8217;r&#8217;)<br \/>\n    plt.legend([&#8216;Actual&#8217;, &#8216;Predictions&#8217;])<br \/>\n    plt.xlabel(&#8216;Observation number after given time steps&#8217;)<br \/>\n    plt.ylabel(&#8216;Sunspots scaled&#8217;)<br \/>\n    plt.title(&#8216;Actual and Predicted Values. The Red Line Separates The Training And Test Examples&#8217;)<\/p>\n<p>sunspots_url = &#8216;https:\/\/raw.githubusercontent.com\/jbrownlee\/Datasets\/master\/monthly-sunspots.csv&#8217;<br \/>\ntime_steps = 12<br \/>\ntrain_data, test_data, data = get_train_test(sunspots_url)<br \/>\ntrainX, trainY = get_XY(train_data, time_steps)<br \/>\ntestX, testY = get_XY(test_data, time_steps)<\/p>\n<p># Create model and train<br \/>\nmodel = create_RNN(hidden_units=3, dense_units=1, input_shape=(time_steps,1),<br \/>\n                   activation=[&#8216;tanh&#8217;, &#8216;tanh&#8217;])<br \/>\nmodel.fit(trainX, trainY, epochs=20, batch_size=1, verbose=2)<\/p>\n<p># make predictions<br \/>\ntrain_predict = model.predict(trainX)<br \/>\ntest_predict = model.predict(testX)<\/p>\n<p># Print error<br \/>\nprint_error(trainY, testY, train_predict, test_predict)<\/p>\n<p>#Plot result<br \/>\nplot_result(trainY, testY, train_predict, test_predict)<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<div class=\"urvanov-syntax-highlighter-nums-content\">\n<p>1<\/p>\n<p>2<\/p>\n<p>3<\/p>\n<p>4<\/p>\n<p>5<\/p>\n<p>6<\/p>\n<p>7<\/p>\n<p>8<\/p>\n<p>9<\/p>\n<p>10<\/p>\n<p>11<\/p>\n<p>12<\/p>\n<p>13<\/p>\n<p>14<\/p>\n<p>15<\/p>\n<p>16<\/p>\n<p>17<\/p>\n<p>18<\/p>\n<p>19<\/p>\n<p>20<\/p>\n<p>21<\/p>\n<p>22<\/p>\n<p>23<\/p>\n<p>24<\/p>\n<p>25<\/p>\n<p>26<\/p>\n<p>27<\/p>\n<p>28<\/p>\n<p>29<\/p>\n<p>30<\/p>\n<p>31<\/p>\n<p>32<\/p>\n<p>33<\/p>\n<p>34<\/p>\n<p>35<\/p>\n<p>36<\/p>\n<p>37<\/p>\n<p>38<\/p>\n<p>39<\/p>\n<p>40<\/p>\n<p>41<\/p>\n<p>42<\/p>\n<p>43<\/p>\n<p>44<\/p>\n<p>45<\/p>\n<p>46<\/p>\n<p>47<\/p>\n<p>48<\/p>\n<p>49<\/p>\n<p>50<\/p>\n<p>51<\/p>\n<p>52<\/p>\n<p>53<\/p>\n<p>54<\/p>\n<p>55<\/p>\n<p>56<\/p>\n<p>57<\/p>\n<p>58<\/p>\n<p>59<\/p>\n<p>60<\/p>\n<p>61<\/p>\n<p>62<\/p>\n<p>63<\/p>\n<p>64<\/p>\n<p>65<\/p>\n<p>66<\/p>\n<p>67<\/p>\n<p>68<\/p>\n<p>69<\/p>\n<p>70<\/p>\n<p>71<\/p>\n<\/div>\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\">\n<p><span class=\"crayon-c\"># Parameter split_percent defines the ratio of training examples<\/span><\/p>\n<p><span class=\"crayon-r\">def<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">get_train_test<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">url<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">split_percent<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">0.8<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">read_csv<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">url<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">usecols<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">engine<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8216;python&#8217;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-k \">array<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">values<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">astype<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8216;float32&#8217;<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">scaler<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">MinMaxScaler<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">feature_range<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">scaler<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">fit_transform<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">flatten<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">n<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-k \">len<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-c\"># Point for splitting data into train and test<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">split<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-k \">int<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">n<\/span><span class=\"crayon-o\">*<\/span><span class=\"crayon-v\">split_percent<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">train_data<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-k \">range<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">split<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">test_data<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">split<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">train_data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">test_data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">data<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-c\"># Prepare the input X and target Y<\/span><\/p>\n<p><span class=\"crayon-r\">def<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">get_XY<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">dat<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">time_steps<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">Y_ind<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">arange<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">time_steps<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-k \">len<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">dat<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">time_steps<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">Y<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">dat<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">Y_ind<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">rows_x<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-k \">len<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">Y<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">dat<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-k \">range<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">time_steps<\/span><span class=\"crayon-o\">*<\/span><span class=\"crayon-v\">rows_x<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">reshape<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">rows_x<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">time_steps<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">Y<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-r\">def<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">create_RNN<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">hidden_units<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">dense_units<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">input_shape<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">activation<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">model<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">Sequential<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">model<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">add<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e\">SimpleRNN<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">hidden_units<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">input_shape<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-v\">input_shape<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">activation<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-v\">activation<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">model<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">add<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e\">Dense<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">units<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-v\">dense_units<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">activation<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-v\">activation<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">model<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-k \">compile<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">loss<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8216;mean_squared_error&#8217;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">optimizer<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8216;adam&#8217;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">model<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-r\">def<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">print_error<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">trainY<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">testY<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">train_predict<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">test_predict<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-c\"># Error of predictions<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">train_rmse<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-k \">math<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">sqrt<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e\">mean_squared_error<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">trainY<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">train_predict<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">test_rmse<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-k \">math<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">sqrt<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e\">mean_squared_error<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">testY<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">test_predict<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-c\"># Print RMSE<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-k \">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8216;Train RMSE: %.3f RMSE&#8217;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">%<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">train_rmse<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-k \">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8216;Test RMSE: %.3f RMSE&#8217;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">%<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">test_rmse<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-c\"># Plot the result<\/span><\/p>\n<p><span class=\"crayon-r\">def<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">plot_result<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">trainY<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">testY<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">train_predict<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">test_predict<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">actual<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">append<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">trainY<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">testY<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">predictions<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">append<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">train_predict<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">test_predict<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">rows<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-k \">len<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">actual<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">plt<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">figure<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">figsize<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">15<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">6<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">dpi<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">80<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">plt<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">plot<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-k \">range<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">rows<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">actual<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">plt<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">plot<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-k \">range<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">rows<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">predictions<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">plt<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">axvline<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">x<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-k \">len<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">trainY<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">color<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8216;r&#8217;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">plt<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">legend<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-s\">&#8216;Actual&#8217;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8216;Predictions&#8217;<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">plt<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">xlabel<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8216;Observation number after given time steps&#8217;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">plt<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">ylabel<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8216;Sunspots scaled&#8217;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">plt<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">title<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8216;Actual and Predicted Values. The Red Line Separates The Training And Test Examples&#8217;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-v\">sunspots_url<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8216;https:\/\/raw.githubusercontent.com\/jbrownlee\/Datasets\/master\/monthly-sunspots.csv&#8217;<\/span><\/p>\n<p><span class=\"crayon-v\">time_steps<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">12<\/span><\/p>\n<p><span class=\"crayon-v\">train_data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">test_data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">get_train_test<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">sunspots_url<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-v\">trainX<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">trainY<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">get_XY<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">train_data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">time_steps<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-v\">testX<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">testY<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">get_XY<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">test_data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">time_steps<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-c\"># Create model and train<\/span><\/p>\n<p><span class=\"crayon-v\">model<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">create_RNN<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">hidden_units<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">3<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">dense_units<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">input_shape<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">time_steps<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 <\/span><span class=\"crayon-v\">activation<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-s\">&#8216;tanh&#8217;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8216;tanh&#8217;<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-v\">model<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">fit<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">trainX<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">trainY<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">epochs<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">20<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">batch_size<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">verbose<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">2<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-c\"># make predictions<\/span><\/p>\n<p><span class=\"crayon-v\">train_predict<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">model<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">predict<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">trainX<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-v\">test_predict<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">model<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">predict<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">testX<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-c\"># Print error<\/span><\/p>\n<p><span class=\"crayon-e\">print_error<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">trainY<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">testY<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">train_predict<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">test_predict<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-c\">#Plot result<\/span><\/p>\n<p><span class=\"crayon-e\">plot_result<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">trainY<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">testY<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">train_predict<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">test_predict<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<h2 id=\"Further-Reading\">Further Reading<\/h2>\n<p>This section provides more resources on the topic if you are looking to go deeper.<\/p>\n<h3 id=\"Books\">Books<\/h3>\n<h3 id=\"Articles\">Articles<\/h3>\n<h2 id=\"Summary\">Summary<\/h2>\n<p>In this tutorial, you discovered recurrent neural networks and their various architectures.<\/p>\n<p>Specifically, you learned:<\/p>\n<ul>\n<li>The structure of RNNs<\/li>\n<li>How the RNN computes an output from previous inputs<\/li>\n<li>How to implement an end to end system for time series forecasting using an RNN<\/li>\n<\/ul>\n<p>Do you have any questions about RNNs discussed in this post? Ask your questions in the comments below and I will do my best to answer.<\/p>\n<\/p><\/div>\n","protected":false},"excerpt":{"rendered":"<p>https:\/\/machinelearningmastery.com\/understanding-simple-recurrent-neural-networks-in-keras\/<\/p>\n","protected":false},"author":0,"featured_media":994,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[],"_links":{"self":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/993"}],"collection":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/comments?post=993"}],"version-history":[{"count":0,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/993\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media\/994"}],"wp:attachment":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media?parent=993"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/categories?post=993"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/tags?post=993"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}