{"id":1221,"date":"2021-11-18T08:33:01","date_gmt":"2021-11-18T08:33:01","guid":{"rendered":"https:\/\/salarydistribution.com\/machine-learning\/2021\/11\/18\/using-cnn-for-financial-time-series-prediction\/"},"modified":"2021-11-18T08:33:01","modified_gmt":"2021-11-18T08:33:01","slug":"using-cnn-for-financial-time-series-prediction","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2021\/11\/18\/using-cnn-for-financial-time-series-prediction\/","title":{"rendered":"Using CNN for financial time series prediction"},"content":{"rendered":"<div id=\"\">\n<p id=\"last-modified-info\">Last Updated on November 15, 2021<\/p>\n<p>Convolutional neural networks have their roots in image processing. It was first published in LeNet to recognize the MNIST handwritten digits. However, convolutional neural networks are not limited to handling images.<\/p>\n<p>In this tutorial, we are going to look at an example of using CNN for time series prediction with an application from financial markets. By way of this example, we are going to explore some techniques in using Keras for model training as well.<\/p>\n<p>After completing this tutorial, you will know<\/p>\n<ul>\n<li>What a typical multidimensional financial data series looks like?<\/li>\n<li>How can CNN applied to time series in a classification problem<\/li>\n<li>How to use generators to feed data to train a Keras model<\/li>\n<li>How to provide a custom metric for evaluating a Keras model<\/li>\n<\/ul>\n<p>Let\u2019s get started<\/p>\n<div id=\"attachment_4963\" class=\"wp-caption aligncenter\"><img aria-describedby=\"caption-attachment-4963\" loading=\"lazy\" class=\"size-full wp-image-4963\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/11\/aron-visuals-BXOXnQ26B7o-unsplash.jpg\" alt=\"Using CNN for financial time series prediction\" width=\"640\" height=\"480\"><img decoding=\"async\" aria-describedby=\"caption-attachment-4963\" loading=\"lazy\" class=\"size-full wp-image-4963\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/11\/aron-visuals-BXOXnQ26B7o-unsplash.jpg\" alt=\"Using CNN for financial time series prediction\" width=\"640\" height=\"480\"><\/p>\n<p id=\"caption-attachment-4963\" class=\"wp-caption-text\">Using CNN for financial time series prediction<br \/>Photo by <a href=\"https:\/\/unsplash.com\/photos\/BXOXnQ26B7o\">Aron Visuals<\/a>, some rights reserved.<\/p>\n<\/div>\n<h2>Tutorial overview<\/h2>\n<p>This tutorial is divided into 7 parts; they are:<\/p>\n<ol>\n<li>Background of the idea<\/li>\n<li>Preprocessing of data<\/li>\n<li>Data generator<\/li>\n<li>The model<\/li>\n<li>Training, validation, and test<\/li>\n<li>Extensions<\/li>\n<li>Does it work?<\/li>\n<\/ol>\n<h2>Background of the idea<\/h2>\n<p>In this tutorial we are following the paper titled \u201cCNNpred: CNN-based stock market prediction using a iverse set of variables\u201d by Ehsan Hoseinzade and Saman Haratizadeh. The data file and sample code from the author are available in github:<\/p>\n<p>The goal of the paper is simple: To predict the next day\u2019s direction of the stock market (i.e., up or down compared to today), hence it is a binary classification problem. However, it is interesting to see how this problem are formulated and solved.<\/p>\n<p>We have seen the examples on using CNN for sequence prediction. If we consider Dow Jones Industrial Average (DJIA) as an example, we may build a CNN with 1D convolution for prediction. This makes sense because a 1D convolution on a time series is roughly computing its moving average or using digital signal processing terms, applying a filter to the time series. It should provide some clues about the trend.<\/p>\n<p>However, when we look at financial time series, it is quite a common sense that some derived signals are useful for predictions too. For example, price and volume together can provide a better clue. Also some other technical indicators such as the moving average of different window size are useful too. If we put all these align together, we will have a table of data, which each time instance has multiple <strong>features<\/strong>, and the goal is still to predict the direction of <strong>one<\/strong> time series.<\/p>\n<p>In the CNNpred paper, 82 such features are prepared for the DJIA time series:<\/p>\n<div id=\"attachment_4963\" class=\"wp-caption aligncenter\"><img aria-describedby=\"caption-attachment-4963\" loading=\"lazy\" class=\"aligncenter wp-image-13058\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/11\/features_table.png\" alt=\"\" width=\"1024\" height=\"874\"><img decoding=\"async\" aria-describedby=\"caption-attachment-4963\" loading=\"lazy\" class=\"aligncenter wp-image-13058\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/11\/features_table.png\" alt=\"\" width=\"1024\" height=\"874\"><\/p>\n<p id=\"caption-attachment-4963\" class=\"wp-caption-text\">Excerpt from the CNNpred paper showing the list of features used.<\/p>\n<\/div>\n<p>Unlike LSTM, which there is an explicit concept of time steps applied, we present data as a matrix in CNN models. As shown in the table below, the features across multiple time steps are presented as a 2D array.<\/p>\n<p><img loading=\"lazy\" class=\"aligncenter wp-image-13059\" data-cfsrc=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/11\/data-excel.png\" alt=\"\" width=\"1024\" height=\"660\"><\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter wp-image-13059\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2021\/11\/data-excel.png\" alt=\"\" width=\"1024\" height=\"660\"><\/p>\n<h2>Preprocessing of data<\/h2>\n<p>In the following, we try to implement the idea of the CNNpred from scratch using Tensorflow\u2019s keras API. While there is a reference implementation from the author in the github link above, we reimplement it differently to illustrate some Keras techniques.<\/p>\n<p>Firstly the data are five CSV files, each for a different market index, under the <code>Dataset<\/code> directory from github repository above, or we can also get a copy here:<\/p>\n<p>The input data has a date column and a name column to identify the ticker symbol for the market index. We can leave the date column as time index and remove the name column. The rest are all numerical.<\/p>\n<p>As we are going to predict the market direction, we first try to create the classification label. The market direction is defined as the closing index of tomorrow compared to today. If we have read the data into a pandas DataFrame, we can use <code>X[\"Close\"].pct_change()<\/code> to find the percentage change, which a positive change for the market goes up. So we can shift this to one time step back as our label:<\/p>\n<div id=\"urvanov-syntax-highlighter-61954f5e772f5803858389\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" minimize scroll-mouseover disable-anim\">\n<p><textarea class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" readonly><br \/>\n&#8230;<br \/>\nX[&#8220;Target&#8221;] = (X[&#8220;Close&#8221;].pct_change().shift(-1) &gt; 0).astype(int)<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\">\n<p><span class=\"crayon-sy\">.<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-sy\">.<\/span><\/p>\n<p><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-s\">&#8220;Target&#8221;<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-s\">&#8220;Close&#8221;<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">pct_change<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">shift<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">&gt;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">astype<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-t\">int<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<p>The line of code above is to compute the percentage change of the closing index and align the data with the previous day. Then convert the data into either 1 or 0 for whether the percentage change is positive.<\/p>\n<p>For five data file in the directory, we read each of them as a separate pandas DataFrame and keep them in a Python dictionary:<\/p>\n<div id=\"urvanov-syntax-highlighter-61954f5e772fc245080338\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" minimize scroll-mouseover disable-anim\">\n<p><textarea class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" readonly><br \/>\n&#8230;<br \/>\ndata = {}<br \/>\nfor filename in os.listdir(DATADIR):<br \/>\n    if not filename.lower().endswith(&#8220;.csv&#8221;):<br \/>\n        continue # read only the CSV files<br \/>\n    filepath = os.path.join(DATADIR, filename)<br \/>\n    X = pd.read_csv(filepath, index_col=&#8221;Date&#8221;, parse_dates=True)<br \/>\n    # basic preprocessing: get the name, the classification<br \/>\n    # Save the target variable as a column in dataframe for easier dropna()<br \/>\n    name = X[&#8220;Name&#8221;][0]<br \/>\n    del X[&#8220;Name&#8221;]<br \/>\n    cols = X.columns<br \/>\n    X[&#8220;Target&#8221;] = (X[&#8220;Close&#8221;].pct_change().shift(-1) &gt; 0).astype(int)<br \/>\n    X.dropna(inplace=True)<br \/>\n    # Fit the standard scaler using the training dataset<br \/>\n    index = X.index[X.index &amp;lt; TRAIN_TEST_CUTOFF]<br \/>\n    index = index[:int(len(index) * TRAIN_VALID_RATIO)]<br \/>\n    scaler = StandardScaler().fit(X.loc[index, cols])<br \/>\n    # Save scale transformed dataframe<br \/>\n    X[cols] = scaler.transform(X[cols])<br \/>\n    data[name] = X<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<div class=\"urvanov-syntax-highlighter-nums-content\">\n<p>1<\/p>\n<p>2<\/p>\n<p>3<\/p>\n<p>4<\/p>\n<p>5<\/p>\n<p>6<\/p>\n<p>7<\/p>\n<p>8<\/p>\n<p>9<\/p>\n<p>10<\/p>\n<p>11<\/p>\n<p>12<\/p>\n<p>13<\/p>\n<p>14<\/p>\n<p>15<\/p>\n<p>16<\/p>\n<p>17<\/p>\n<p>18<\/p>\n<p>19<\/p>\n<p>20<\/p>\n<p>21<\/p>\n<\/div>\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\">\n<p><span class=\"crayon-sy\">.<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-sy\">.<\/span><\/p>\n<p><span class=\"crayon-v\">data<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">{<\/span><span class=\"crayon-sy\">}<\/span><\/p>\n<p><span class=\"crayon-st\">for<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">filename <\/span><span class=\"crayon-st\">in<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">os<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">listdir<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">DATADIR<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">not<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">filename<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">lower<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">endswith<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;.csv&#8221;<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">continue<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-p\"># read only the CSV files<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">filepath<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">os<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">path<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">join<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">DATADIR<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">filename<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">pd<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">read_csv<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">filepath<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">index_col<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;Date&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">parse_dates<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-t\">True<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># basic preprocessing: get the name, the classification<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Save the target variable as a column in dataframe for easier dropna()<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">name<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-s\">&#8220;Name&#8221;<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-i\">del<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-s\">&#8220;Name&#8221;<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">cols<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-i\">columns<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-s\">&#8220;Target&#8221;<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-s\">&#8220;Close&#8221;<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">pct_change<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">shift<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">&gt;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">astype<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-t\">int<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">dropna<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">inplace<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-t\">True<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Fit the standard scaler using the training dataset<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">&amp;<\/span><span class=\"crayon-v\">lt<\/span><span class=\"crayon-sy\">;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">TRAIN_TEST_CUTOFF<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-t\">int<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e\">len<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">*<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">TRAIN_VALID_RATIO<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">scaler<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">StandardScaler<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">fit<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">loc<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">cols<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Save scale transformed dataframe<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">cols<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">scaler<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">transform<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">cols<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">name<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">X<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<p>The result of the above code is a DataFrame for each index, which the classification label is the column \u201cTarget\u201d while all other columns are input features. We also normalize the data with a standard scaler.<\/p>\n<p>In time series problems, it is generally reasonable not to split the data into training and test sets randomly, but to set up a cutoff point in which the data before the cutoff is training set while that afterwards is the test set. The scaling above are based on the training set but applied to the entire dataset.<\/p>\n<h2>Data generator<\/h2>\n<p>We are not going to use all time steps at once, but instead, we use a fixed length of N time steps to predict the market direction at step N+1. In this design, the window of N time steps can start from anywhere. We can just create a large number of DataFrames with large amount of overlaps with one another. To save memory, we are going to build a data generator for training and validation, as follows:<\/p>\n<div id=\"urvanov-syntax-highlighter-61954f5e772fe187347576\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" minimize scroll-mouseover disable-anim\">\n<p><textarea class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" readonly><br \/>\n&#8230;<br \/>\nTRAIN_TEST_CUTOFF = &#8216;2016-04-21&#8217;<br \/>\nTRAIN_VALID_RATIO = 0.75<\/p>\n<p>def datagen(data, seq_len, batch_size, targetcol, kind):<br \/>\n    &#8220;As a generator to produce samples for Keras model&#8221;<br \/>\n    batch = []<br \/>\n    while True:<br \/>\n        # Pick one dataframe from the pool<br \/>\n        key = random.choice(list(data.keys()))<br \/>\n        df = data[key]<br \/>\n        input_cols = [c for c in df.columns if c != targetcol]<br \/>\n        index = df.index[df.index &amp;lt; TRAIN_TEST_CUTOFF]<br \/>\n        split = int(len(index) * TRAIN_VALID_RATIO)<br \/>\n        if kind == &#8216;train&#8217;:<br \/>\n            index = index[:split]   # range for the training set<br \/>\n        elif kind == &#8216;valid&#8217;:<br \/>\n            index = index[split:]   # range for the validation set<br \/>\n        # Pick one position, then clip a sequence length<br \/>\n        while True:<br \/>\n            t = random.choice(index)      # pick one time step<br \/>\n            n = (df.index == t).argmax()  # find its position in the dataframe<br \/>\n            if n-seq_len+1 &amp;lt; 0:<br \/>\n                continue # can&#8217;t get enough data for one sequence length<br \/>\n            frame = df.iloc[n-seq_len+1:n+1]<br \/>\n            batch.append([frame[input_cols].values, df.loc[t, targetcol]])<br \/>\n            break<br \/>\n        # if we get enough for a batch, dispatch<br \/>\n        if len(batch) == batch_size:<br \/>\n            X, y = zip(*batch)<br \/>\n            X, y = np.expand_dims(np.array(X), 3), np.array(y)<br \/>\n            yield X, y<br \/>\n            batch = []<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<div class=\"urvanov-syntax-highlighter-nums-content\">\n<p>1<\/p>\n<p>2<\/p>\n<p>3<\/p>\n<p>4<\/p>\n<p>5<\/p>\n<p>6<\/p>\n<p>7<\/p>\n<p>8<\/p>\n<p>9<\/p>\n<p>10<\/p>\n<p>11<\/p>\n<p>12<\/p>\n<p>13<\/p>\n<p>14<\/p>\n<p>15<\/p>\n<p>16<\/p>\n<p>17<\/p>\n<p>18<\/p>\n<p>19<\/p>\n<p>20<\/p>\n<p>21<\/p>\n<p>22<\/p>\n<p>23<\/p>\n<p>24<\/p>\n<p>25<\/p>\n<p>26<\/p>\n<p>27<\/p>\n<p>28<\/p>\n<p>29<\/p>\n<p>30<\/p>\n<p>31<\/p>\n<p>32<\/p>\n<p>33<\/p>\n<\/div>\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\">\n<p><span class=\"crayon-sy\">.<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-sy\">.<\/span><\/p>\n<p><span class=\"crayon-v\">TRAIN_TEST_CUTOFF<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8216;2016-04-21&#8217;<\/span><\/p>\n<p><span class=\"crayon-v\">TRAIN_VALID_RATIO<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">0.75<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">datagen<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">batch_size<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">targetcol<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">kind<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;As a generator to produce samples for Keras model&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">batch<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">while<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-t\">True<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Pick one dataframe from the pool<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">key<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">random<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">choice<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e\">list<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">keys<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">key<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">input_cols<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-i\">c<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">for<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">c<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">in<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">columns <\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">c<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">!=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">targetcol<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">&amp;<\/span><span class=\"crayon-v\">lt<\/span><span class=\"crayon-sy\">;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">TRAIN_TEST_CUTOFF<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">split<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-t\">int<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e\">len<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">*<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">TRAIN_VALID_RATIO<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">kind<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">==<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8216;train&#8217;<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-v\">split<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-h\">\u00a0\u00a0 <\/span><span class=\"crayon-p\"># range for the training set<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">elif <\/span><span class=\"crayon-v\">kind<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">==<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8216;valid&#8217;<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">split<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-h\">\u00a0\u00a0 <\/span><span class=\"crayon-p\"># range for the validation set<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Pick one position, then clip a sequence length<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">while<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-t\">True<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">t<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">random<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">choice<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># pick one time step<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">n<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">==<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">t<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">argmax<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\">\u00a0\u00a0<\/span><span class=\"crayon-p\"># find its position in the dataframe<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">n<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">&amp;<\/span><span class=\"crayon-v\">lt<\/span><span class=\"crayon-sy\">;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">continue<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-p\"># can&#8217;t get enough data for one sequence length<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">frame<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">iloc<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">n<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-v\">n<\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">batch<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">append<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">frame<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">input_cols<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">values<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">loc<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">t<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">targetcol<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">break<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># if we get enough for a batch, dispatch<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">len<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">batch<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">==<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">batch_size<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">zip<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-o\">*<\/span><span class=\"crayon-v\">batch<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">expand_dims<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-t\">array<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">3<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-t\">array<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-i\">yield<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">y<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">batch<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<p>Generator is a special function in Python that does not <code>return<\/code> a value but to <code>yield<\/code> in iterations, such that a sequence of data are produced from it. For a generator to be used in Keras training, it is expected to <code>yield<\/code> a batch of input data and target. This generator supposed to run indefinitely. Hence the generator function above is created with an infinite loop starts with <code>while True<\/code>.<\/p>\n<p>In each iteration, it randomly pick one DataFrame from the Python dictionary, then within the range of time steps of the training set (i.e., the beginning portion), we start from a random point and take N time steps using the pandas <code>iloc[start:end]<\/code> syntax to create a input under the variable <code>frame<\/code>. This DataFrame will be a 2D array. The target label is that of the last time step. The input data and the label are then appended to the list <code>batch<\/code>. Until we accumulated for one batch\u2019s size, we dispatch it from the generator.<\/p>\n<p>The last four lines at the code snippet above is to dispatch a batch for training or validation. We collect the list of input data (each a 2D array) as well as a list of target label into variables <code>X<\/code> and <code>y<\/code>, then convert them into numpy array so it can work with our Keras model. We need to add one more dimension to the numpy array <code>X<\/code> using <code>np.expand_dims()<\/code> because of the design of the network model, as explained below.<\/p>\n<h2>The Model<\/h2>\n<p>The 2D CNN model presented in the original paper accepts an input tensor of shape $Ntimes m times 1$ for N the number of time steps and m the number of features in each time step. The paper assumes $N=60$ and $m=82$.<\/p>\n<p>The model comprises of three convolutional layers, as described as follows:<\/p>\n<div id=\"urvanov-syntax-highlighter-61954f5e772ff313108600\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" minimize scroll-mouseover disable-anim\">\n<p><textarea class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" readonly><br \/>\n&#8230;<br \/>\ndef cnnpred_2d(seq_len=60, n_features=82, n_filters=(8,8,8), droprate=0.1):<br \/>\n    &#8220;2D-CNNpred model according to the paper&#8221;<br \/>\n    model = Sequential([<br \/>\n        Input(shape=(seq_len, n_features, 1)),<br \/>\n        Conv2D(n_filters[0], kernel_size=(1, n_features), activation=&#8221;relu&#8221;),<br \/>\n        Conv2D(n_filters[1], kernel_size=(3,1), activation=&#8221;relu&#8221;),<br \/>\n        MaxPool2D(pool_size=(2,1)),<br \/>\n        Conv2D(n_filters[2], kernel_size=(3,1), activation=&#8221;relu&#8221;),<br \/>\n        MaxPool2D(pool_size=(2,1)),<br \/>\n        Flatten(),<br \/>\n        Dropout(droprate),<br \/>\n        Dense(1, activation=&#8221;sigmoid&#8221;)<br \/>\n    ])<br \/>\n    return model<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\">\n<p><span class=\"crayon-sy\">.<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-sy\">.<\/span><\/p>\n<p><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">cnnpred_2d<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">60<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">n_features<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">82<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">n_filters<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">8<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">8<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">8<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">droprate<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">0.1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;2D-CNNpred model according to the paper&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">model<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">Sequential<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">[<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">Input<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">shape<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">n_features<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">Conv2D<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">n_filters<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">kernel_size<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">n_features<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">activation<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;relu&#8221;<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">Conv2D<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">n_filters<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">kernel_size<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">3<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">activation<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;relu&#8221;<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">MaxPool2D<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">pool_size<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">2<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">Conv2D<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">n_filters<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">2<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">kernel_size<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">3<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">activation<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;relu&#8221;<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">MaxPool2D<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">pool_size<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">2<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">Flatten<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">Dropout<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">droprate<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">Dense<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">activation<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;sigmoid&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">model<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<p>and the model is presented by the following:<\/p>\n<div id=\"urvanov-syntax-highlighter-61954f5e77300158498702\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" minimize scroll-mouseover disable-anim\">\n<p><textarea class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" readonly><br \/>\nModel: &#8220;sequential&#8221;<br \/>\n_________________________________________________________________<br \/>\nLayer (type)                 Output Shape              Param #<br \/>\n=================================================================<br \/>\nconv2d (Conv2D)              (None, 60, 1, 8)          664<br \/>\n_________________________________________________________________<br \/>\nconv2d_1 (Conv2D)            (None, 58, 1, 8)          200<br \/>\n_________________________________________________________________<br \/>\nmax_pooling2d (MaxPooling2D) (None, 29, 1, 8)          0<br \/>\n_________________________________________________________________<br \/>\nconv2d_2 (Conv2D)            (None, 27, 1, 8)          200<br \/>\n_________________________________________________________________<br \/>\nmax_pooling2d_1 (MaxPooling2 (None, 13, 1, 8)          0<br \/>\n_________________________________________________________________<br \/>\nflatten (Flatten)            (None, 104)               0<br \/>\n_________________________________________________________________<br \/>\ndropout (Dropout)            (None, 104)               0<br \/>\n_________________________________________________________________<br \/>\ndense (Dense)                (None, 1)                 105<br \/>\n=================================================================<br \/>\nTotal params: 1,169<br \/>\nTrainable params: 1,169<br \/>\nNon-trainable params: 0<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<div class=\"urvanov-syntax-highlighter-nums-content\">\n<p>1<\/p>\n<p>2<\/p>\n<p>3<\/p>\n<p>4<\/p>\n<p>5<\/p>\n<p>6<\/p>\n<p>7<\/p>\n<p>8<\/p>\n<p>9<\/p>\n<p>10<\/p>\n<p>11<\/p>\n<p>12<\/p>\n<p>13<\/p>\n<p>14<\/p>\n<p>15<\/p>\n<p>16<\/p>\n<p>17<\/p>\n<p>18<\/p>\n<p>19<\/p>\n<p>20<\/p>\n<p>21<\/p>\n<p>22<\/p>\n<p>23<\/p>\n<\/div>\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\">\n<p>Model: &#8220;sequential&#8221;<\/p>\n<p>_________________________________________________________________<\/p>\n<p>Layer (type)\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Output Shape\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Param #<\/p>\n<p>=================================================================<\/p>\n<p>conv2d (Conv2D)\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0(None, 60, 1, 8)\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0664<\/p>\n<p>_________________________________________________________________<\/p>\n<p>conv2d_1 (Conv2D)\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0(None, 58, 1, 8)\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0200<\/p>\n<p>_________________________________________________________________<\/p>\n<p>max_pooling2d (MaxPooling2D) (None, 29, 1, 8)\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a00<\/p>\n<p>_________________________________________________________________<\/p>\n<p>conv2d_2 (Conv2D)\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0(None, 27, 1, 8)\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0200<\/p>\n<p>_________________________________________________________________<\/p>\n<p>max_pooling2d_1 (MaxPooling2 (None, 13, 1, 8)\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a00<\/p>\n<p>_________________________________________________________________<\/p>\n<p>flatten (Flatten)\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0(None, 104)\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0<\/p>\n<p>_________________________________________________________________<\/p>\n<p>dropout (Dropout)\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0(None, 104)\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0<\/p>\n<p>_________________________________________________________________<\/p>\n<p>dense (Dense)\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0(None, 1)\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 105<\/p>\n<p>=================================================================<\/p>\n<p>Total params: 1,169<\/p>\n<p>Trainable params: 1,169<\/p>\n<p>Non-trainable params: 0<\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<p>The first convolutional layer has 8 units, and is applied across all features in each time step. It is followed by a second convolutional layer to consider three consecutive days at once, for it is a common belief that three days can make a trend in the stock market. It is then applied to a max pooling layer and another convolutional layer before it is flattened into a one-dimensional array and applied to a fully-connected layer with sigmoid activation for binary classification.<\/p>\n<h2>Training, validation, and test<\/h2>\n<p>That\u2019s it for the model. The paper used MAE as the loss metric and also monitor for accuracy and F1 score to determine the quality of the model. We should point out that F1 score depends on precision and recall ratios, which are both considering the positive classification. The paper, however, consider the average of the F1 from positive and negative classification. Explicitly, it is the F1-macro metric:<br \/>$$<br \/>F_1 = frac{1}{2}left(<br \/>frac{2cdot frac{TP}{TP+FP} cdot frac{TP}{TP+FN}}{frac{TP}{TP+FP} + frac{TP}{TP+FN}}<br \/>+<br \/>frac{2cdot frac{TN}{TN+FN} cdot frac{TN}{TN+FP}}{frac{TN}{TN+FN} + frac{TN}{TN+FP}}<br \/>right)<br \/>$$<br \/>The fraction $frac{TP}{TP+FP}$ is the precision with TP and FP the number of true positive and false positive. Similarly $frac{TP}{TP+FN}$ is the recall. The first term in the big parenthesis above is the normal F1 metric that considered positive classifications. And the second term is the reverse, which considered the negative classifications.<\/p>\n<p>While this metric is available in scikit-learn as <code>sklearn.metrics.f1_score()<\/code> there is no equivalent in Keras. Hence we would create our own by borrowing code from <a href=\"https:\/\/datascience.stackexchange.com\/questions\/45165\/\">this stackexchange question<\/a>:<\/p>\n<div id=\"urvanov-syntax-highlighter-61954f5e77301451807836\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" minimize scroll-mouseover disable-anim\">\n<p><textarea class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" readonly><br \/>\nfrom tensorflow.keras import backend as K<\/p>\n<p>def recall_m(y_true, y_pred):<br \/>\n    true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))<br \/>\n    possible_positives = K.sum(K.round(K.clip(y_true, 0, 1)))<br \/>\n    recall = true_positives \/ (possible_positives + K.epsilon())<br \/>\n    return recall<\/p>\n<p>def precision_m(y_true, y_pred):<br \/>\n    true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))<br \/>\n    predicted_positives = K.sum(K.round(K.clip(y_pred, 0, 1)))<br \/>\n    precision = true_positives \/ (predicted_positives + K.epsilon())<br \/>\n    return precision<\/p>\n<p>def f1_m(y_true, y_pred):<br \/>\n    precision = precision_m(y_true, y_pred)<br \/>\n    recall = recall_m(y_true, y_pred)<br \/>\n    return 2*((precision*recall)\/(precision+recall+K.epsilon()))<\/p>\n<p>def f1macro(y_true, y_pred):<br \/>\n    f_pos = f1_m(y_true, y_pred)<br \/>\n    # negative version of the data and prediction<br \/>\n    f_neg = f1_m(1-y_true, 1-K.clip(y_pred,0,1))<br \/>\n    return (f_pos + f_neg)\/2<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<div class=\"urvanov-syntax-highlighter-nums-content\">\n<p>1<\/p>\n<p>2<\/p>\n<p>3<\/p>\n<p>4<\/p>\n<p>5<\/p>\n<p>6<\/p>\n<p>7<\/p>\n<p>8<\/p>\n<p>9<\/p>\n<p>10<\/p>\n<p>11<\/p>\n<p>12<\/p>\n<p>13<\/p>\n<p>14<\/p>\n<p>15<\/p>\n<p>16<\/p>\n<p>17<\/p>\n<p>18<\/p>\n<p>19<\/p>\n<p>20<\/p>\n<p>21<\/p>\n<p>22<\/p>\n<p>23<\/p>\n<p>24<\/p>\n<\/div>\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\">\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-v\">tensorflow<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">keras <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">backend <\/span><span class=\"crayon-st\">as<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">K<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">recall_m<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_true<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">true_positives<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">sum<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">round<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">clip<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e \">y_true *<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">possible_positives<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">sum<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">round<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">clip<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_true<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">recall<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">true_positives<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">\/<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">possible_positives<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">epsilon<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">recall<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">precision_m<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_true<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">true_positives<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">sum<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">round<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">clip<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e \">y_true *<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">predicted_positives<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">sum<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">round<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">clip<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">precision<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">true_positives<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">\/<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">predicted_positives<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">epsilon<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">precision<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">f1_m<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_true<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">precision<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">precision_m<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_true<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">recall<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">recall_m<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_true<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">2<\/span><span class=\"crayon-o\">*<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e \">precision*<\/span><span class=\"crayon-v\">recall<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">\/<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">precision<\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-v\">recall<\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">epsilon<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">f1macro<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_true<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">f_pos<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">f1_m<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_true<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># negative version of the data and prediction<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">f_neg<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">f1_m<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-v\">y_true<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">clip<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">f_pos<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">f_neg<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">\/<\/span><span class=\"crayon-cn\">2<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<p>The training process can take hours to complete. Hence we want to save the model in the middle of the training so that we may interrupt and resume it. We can make use of checkpoint features in Keras:<\/p>\n<div id=\"urvanov-syntax-highlighter-61954f5e77302532522933\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" minimize scroll-mouseover disable-anim\">\n<p><textarea class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" readonly><br \/>\ncheckpoint_path = &#8220;.\/cp2d-{epoch}-{val_f1macro:.2f}.h5&#8243;<br \/>\ncallbacks = [<br \/>\n    ModelCheckpoint(checkpoint_path,<br \/>\n                    monitor=&#8217;val_f1macro&#8217;, mode=&#8221;max&#8221;, verbose=0,<br \/>\n                    save_best_only=True, save_weights_only=False, save_freq=&#8221;epoch&#8221;)<br \/>\n]<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\">\n<p><span class=\"crayon-v\">checkpoint_path<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;.\/cp2d-{epoch}-{val_f1macro:.2f}.h5&#8221;<\/span><\/p>\n<p><span class=\"crayon-v\">callbacks<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">[<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">ModelCheckpoint<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">checkpoint_path<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">monitor<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8216;val_f1macro&#8217;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">mode<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;max&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">verbose<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">save_best_only<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-t\">True<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">save_weights_only<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-t\">False<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">save_freq<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;epoch&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-sy\">]<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<p>We set up a filename template <code>checkpoint_path<\/code> and ask Keras to fill in the epoch number as well as validation F1 score into the filename. We save it by monitoring the validation\u2019s F1 metric, and this metric is supposed to increase when the model gets better. Hence we pass in the <code>mode=\"max\"<\/code> to it.<\/p>\n<p>It should now be trivial to train our model, as follows:<\/p>\n<div id=\"urvanov-syntax-highlighter-61954f5e77303634314082\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" minimize scroll-mouseover disable-anim\">\n<p><textarea class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" readonly><br \/>\nseq_len    = 60<br \/>\nbatch_size = 128<br \/>\nn_epochs   = 20<br \/>\nn_features = 82<\/p>\n<p>model = cnnpred_2d(seq_len, n_features)<br \/>\nmodel.compile(optimizer=&#8221;adam&#8221;, loss=&#8221;mae&#8221;, metrics=[&#8220;acc&#8221;, f1macro])<br \/>\nmodel.fit(datagen(data, seq_len, batch_size, &#8220;Target&#8221;, &#8220;train&#8221;),<br \/>\n          validation_data=datagen(data, seq_len, batch_size, &#8220;Target&#8221;, &#8220;valid&#8221;),<br \/>\n          epochs=n_epochs, steps_per_epoch=400, validation_steps=10, verbose=1,<br \/>\n          callbacks=callbacks)<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\">\n<p><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">60<\/span><\/p>\n<p><span class=\"crayon-v\">batch_size<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">128<\/span><\/p>\n<p><span class=\"crayon-v\">n_epochs<\/span><span class=\"crayon-h\">\u00a0\u00a0 <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">20<\/span><\/p>\n<p><span class=\"crayon-v\">n_features<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">82<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-v\">model<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">cnnpred_2d<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">n_features<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-v\">model<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">compile<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">optimizer<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;adam&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">loss<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;mae&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">metrics<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-s\">&#8220;acc&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">f1macro<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-v\">model<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">fit<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e\">datagen<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">batch_size<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;Target&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;train&#8221;<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">validation_data<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-e\">datagen<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">batch_size<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;Target&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;valid&#8221;<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">epochs<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-v\">n_epochs<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">steps_per_epoch<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">400<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">validation_steps<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">10<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">verbose<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">callbacks<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-v\">callbacks<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<p>Two points to note in the above snippets. We supplied <code>\"acc\"<\/code> as the accuracy as well as the function <code>f1macro<\/code> defined above as the <code>metrics<\/code> parameter to the <code>compile()<\/code> function. Hence these two metrics will be monitored during training. Because the function is named <code>f1macro<\/code>, we refer to this metric in the checkpoint\u2019s <code>monitor<\/code> parameter as <code>val_f1macro<\/code>.<\/p>\n<p>Separately, in the <code>fit()<\/code> function, we provided the input data through the <code>datagen()<\/code> generator as defined above. Calling this function will produce a generator, which during the training loop, batches are fetched from it one after another. Similarly, validation data are also provided by the generator.<\/p>\n<p>Because the nature of a generator is to dispatch data indefinitely. We need to tell the training process on how to define a epoch. Recall that in Keras terms, a batch is one iteration of doing gradient descent update. An epoch is supposed to be one cycle through all data in the dataset. At the end of an epoch is the time to run validation. It is also the opportunity for running the checkpoint we defined above. As Keras has no way to infer the size of the dataset from a generator, we need to tell how many batch it should process in one epoch using the <code>steps_per_epoch<\/code> parameter. Similarly, it is the <code>validation_steps<\/code> parameter to tell how many batch are used in each validation step. The validation does not affect the training, but it will report to us the metrics we are interested. Below is a screenshot of what we will see in the middle of training, which we will see that the metric for training set are updated on each batch but that for validation set is provided only at the end of epoch:<\/p>\n<div id=\"urvanov-syntax-highlighter-61954f5e77304533246631\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" minimize scroll-mouseover disable-anim\">\n<p><textarea class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" readonly><br \/>\nEpoch 1\/20<br \/>\n400\/400 [==============================] &#8211; 43s 106ms\/step &#8211; loss: 0.4062 &#8211; acc: 0.6184 &#8211; f1macro: 0.5237 &#8211; val_loss: 0.4958 &#8211; val_acc: 0.4969 &#8211; val_f1macro: 0.4297<br \/>\nEpoch 2\/20<br \/>\n400\/400 [==============================] &#8211; 44s 111ms\/step &#8211; loss: 0.2760 &#8211; acc: 0.7489 &#8211; f1macro: 0.7304 &#8211; val_loss: 0.5007 &#8211; val_acc: 0.4984 &#8211; val_f1macro: 0.4833<br \/>\nEpoch 3\/20<br \/>\n 60\/400 [===&gt;&#8230;&#8230;&#8230;&#8230;&#8230;&#8230;&#8230;&#8230;..] &#8211; ETA: 39s &#8211; loss: 0.2399 &#8211; acc: 0.7783 &#8211; f1macro: 0.7643<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\">\n<p>Epoch 1\/20<\/p>\n<p>400\/400 [==============================] &#8211; 43s 106ms\/step &#8211; loss: 0.4062 &#8211; acc: 0.6184 &#8211; f1macro: 0.5237 &#8211; val_loss: 0.4958 &#8211; val_acc: 0.4969 &#8211; val_f1macro: 0.4297<\/p>\n<p>Epoch 2\/20<\/p>\n<p>400\/400 [==============================] &#8211; 44s 111ms\/step &#8211; loss: 0.2760 &#8211; acc: 0.7489 &#8211; f1macro: 0.7304 &#8211; val_loss: 0.5007 &#8211; val_acc: 0.4984 &#8211; val_f1macro: 0.4833<\/p>\n<p>Epoch 3\/20<\/p>\n<p> 60\/400 [===&gt;&#8230;&#8230;&#8230;&#8230;&#8230;&#8230;&#8230;&#8230;..] &#8211; ETA: 39s &#8211; loss: 0.2399 &#8211; acc: 0.7783 &#8211; f1macro: 0.7643<\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<p>After the model finished training, we can test it with unseen data, i.e., the test set. Instead of generating the test set randomly, we create it from the dataset in a deterministic way:<\/p>\n<div id=\"urvanov-syntax-highlighter-61954f5e77305978139527\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" minimize scroll-mouseover disable-anim\">\n<p><textarea class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" readonly><br \/>\ndef testgen(data, seq_len, targetcol):<br \/>\n    &#8220;Return array of all test samples&#8221;<br \/>\n    batch = []<br \/>\n    for key, df in data.items():<br \/>\n        input_cols = [c for c in df.columns if c != targetcol]<br \/>\n        # find the start of test sample<br \/>\n        t = df.index[df.index &gt;= TRAIN_TEST_CUTOFF][0]<br \/>\n        n = (df.index == t).argmax()<br \/>\n        for i in range(n+1, len(df)+1):<br \/>\n            frame = df.iloc[i-seq_len:i]<br \/>\n            batch.append([frame[input_cols].values, frame[targetcol][-1]])<br \/>\n    X, y = zip(*batch)<br \/>\n    return np.expand_dims(np.array(X),3), np.array(y)<\/p>\n<p># Prepare test data<br \/>\ntest_data, test_target = testgen(data, seq_len, &#8220;Target&#8221;)<\/p>\n<p># Test the model<br \/>\ntest_out = model.predict(test_data)<br \/>\ntest_pred = (test_out &gt; 0.5).astype(int)<br \/>\nprint(&#8220;accuracy:&#8221;, accuracy_score(test_pred, test_target))<br \/>\nprint(&#8220;MAE:&#8221;, mean_absolute_error(test_pred, test_target))<br \/>\nprint(&#8220;F1:&#8221;, f1_score(test_pred, test_target))<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<div class=\"urvanov-syntax-highlighter-nums-content\">\n<p>1<\/p>\n<p>2<\/p>\n<p>3<\/p>\n<p>4<\/p>\n<p>5<\/p>\n<p>6<\/p>\n<p>7<\/p>\n<p>8<\/p>\n<p>9<\/p>\n<p>10<\/p>\n<p>11<\/p>\n<p>12<\/p>\n<p>13<\/p>\n<p>14<\/p>\n<p>15<\/p>\n<p>16<\/p>\n<p>17<\/p>\n<p>18<\/p>\n<p>19<\/p>\n<p>20<\/p>\n<p>21<\/p>\n<p>22<\/p>\n<p>23<\/p>\n<\/div>\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\">\n<p><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">testgen<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">targetcol<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;Return array of all test samples&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">batch<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">for<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">key<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">df <\/span><span class=\"crayon-st\">in<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">items<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">input_cols<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-i\">c<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">for<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">c<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">in<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">columns <\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">c<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">!=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">targetcol<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># find the start of test sample<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">t<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">&gt;=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">TRAIN_TEST_CUTOFF<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">n<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">==<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">t<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">argmax<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">for<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">i<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">in<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">range<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">n<\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">len<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">frame<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">iloc<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">i<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-v\">i<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">batch<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">append<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">frame<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">input_cols<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">values<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">frame<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">targetcol<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">zip<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-o\">*<\/span><span class=\"crayon-v\">batch<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">expand_dims<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-t\">array<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">3<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-t\">array<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># Prepare test data<\/span><\/p>\n<p><span class=\"crayon-v\">test_data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">test_target<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">testgen<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;Target&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># Test the model<\/span><\/p>\n<p><span class=\"crayon-v\">test_out<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">model<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">predict<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">test_data<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-v\">test_pred<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">test_out<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">&gt;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">0.5<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">astype<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-t\">int<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;accuracy:&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">accuracy_score<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">test_pred<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">test_target<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;MAE:&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">mean_absolute_error<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">test_pred<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">test_target<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;F1:&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">f1_score<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">test_pred<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">test_target<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<p>The structure of the function <code>testgen()<\/code> is resembling that of <code>datagen()<\/code> we defined above. Except in <code>datagen()<\/code> the output data\u2019s first dimension is the number of samples in a batch but in <code>testgen()<\/code> is the the entire test samples.<\/p>\n<p>Using the model for prediction will produce a floating point between 0 and 1 as we are using the sigmoid activation function. We will convert this into 0 or 1 by using the threshold at 0.5. Then we use the functions from scikit-learn to compute the accuracy, mean absolute error and F1 score (which accuracy is just one minus the MAE).<\/p>\n<p>Tying all these together, the complete code is as follows:<\/p>\n<div id=\"urvanov-syntax-highlighter-61954f5e77306068980583\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" minimize scroll-mouseover disable-anim\">\n<p><textarea class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" readonly><br \/>\nimport os<br \/>\nimport random<\/p>\n<p>import numpy as np<br \/>\nimport pandas as pd<br \/>\nimport tensorflow as tf<br \/>\nfrom tensorflow.keras import backend as K<br \/>\nfrom tensorflow.keras.layers import Dense, Dropout, Flatten, Conv2D, MaxPool2D, Input<br \/>\nfrom tensorflow.keras.models import Sequential, load_model<br \/>\nfrom tensorflow.keras.callbacks import ModelCheckpoint<br \/>\nfrom sklearn.preprocessing import StandardScaler<br \/>\nfrom sklearn.metrics import accuracy_score, f1_score, mean_absolute_error<\/p>\n<p>from f1metrics import f1macro<\/p>\n<p>DATADIR = &#8220;.\/Dataset&#8221;<br \/>\nTRAIN_TEST_CUTOFF = &#8216;2016-04-21&#8217;<br \/>\nTRAIN_VALID_RATIO = 0.75<\/p>\n<p># https:\/\/datascience.stackexchange.com\/questions\/45165\/how-to-get-accuracy-f1-precision-and-recall-for-a-keras-model<br \/>\n# to implement F1 score for validation in a batch<br \/>\ndef recall_m(y_true, y_pred):<br \/>\n    true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))<br \/>\n    possible_positives = K.sum(K.round(K.clip(y_true, 0, 1)))<br \/>\n    recall = true_positives \/ (possible_positives + K.epsilon())<br \/>\n    return recall<\/p>\n<p>def precision_m(y_true, y_pred):<br \/>\n    true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))<br \/>\n    predicted_positives = K.sum(K.round(K.clip(y_pred, 0, 1)))<br \/>\n    precision = true_positives \/ (predicted_positives + K.epsilon())<br \/>\n    return precision<\/p>\n<p>def f1_m(y_true, y_pred):<br \/>\n    precision = precision_m(y_true, y_pred)<br \/>\n    recall = recall_m(y_true, y_pred)<br \/>\n    return 2*((precision*recall)\/(precision+recall+K.epsilon()))<\/p>\n<p>def f1macro(y_true, y_pred):<br \/>\n    f_pos = f1_m(y_true, y_pred)<br \/>\n    # negative version of the data and prediction<br \/>\n    f_neg = f1_m(1-y_true, 1-K.clip(y_pred,0,1))<br \/>\n    return (f_pos + f_neg)\/2<\/p>\n<p>def cnnpred_2d(seq_len=60, n_features=82, n_filters=(8,8,8), droprate=0.1):<br \/>\n    &#8220;2D-CNNpred model according to the paper&#8221;<br \/>\n    model = Sequential([<br \/>\n        Input(shape=(seq_len, n_features, 1)),<br \/>\n        Conv2D(n_filters[0], kernel_size=(1, n_features), activation=&#8221;relu&#8221;),<br \/>\n        Conv2D(n_filters[1], kernel_size=(3,1), activation=&#8221;relu&#8221;),<br \/>\n        MaxPool2D(pool_size=(2,1)),<br \/>\n        Conv2D(n_filters[2], kernel_size=(3,1), activation=&#8221;relu&#8221;),<br \/>\n        MaxPool2D(pool_size=(2,1)),<br \/>\n        Flatten(),<br \/>\n        Dropout(droprate),<br \/>\n        Dense(1, activation=&#8221;sigmoid&#8221;)<br \/>\n    ])<br \/>\n    return model<\/p>\n<p>def datagen(data, seq_len, batch_size, targetcol, kind):<br \/>\n    &#8220;As a generator to produce samples for Keras model&#8221;<br \/>\n    batch = []<br \/>\n    while True:<br \/>\n        # Pick one dataframe from the pool<br \/>\n        key = random.choice(list(data.keys()))<br \/>\n        df = data[key]<br \/>\n        input_cols = [c for c in df.columns if c != targetcol]<br \/>\n        index = df.index[df.index &amp;lt; TRAIN_TEST_CUTOFF]<br \/>\n        split = int(len(index) * TRAIN_VALID_RATIO)<br \/>\n        assert split &gt; seq_len, &#8220;Training data too small for sequence length {}&#8221;.format(seq_len)<br \/>\n        if kind == &#8216;train&#8217;:<br \/>\n            index = index[:split]   # range for the training set<br \/>\n        elif kind == &#8216;valid&#8217;:<br \/>\n            index = index[split:]   # range for the validation set<br \/>\n        else:<br \/>\n            raise NotImplementedError<br \/>\n        # Pick one position, then clip a sequence length<br \/>\n        while True:<br \/>\n            t = random.choice(index)     # pick one time step<br \/>\n            n = (df.index == t).argmax() # find its position in the dataframe<br \/>\n            if n-seq_len+1 &amp;lt; 0:<br \/>\n                continue # this sample is not enough for one sequence length<br \/>\n            frame = df.iloc[n-seq_len+1:n+1]<br \/>\n            batch.append([frame[input_cols].values, df.loc[t, targetcol]])<br \/>\n            break<br \/>\n        # if we get enough for a batch, dispatch<br \/>\n        if len(batch) == batch_size:<br \/>\n            X, y = zip(*batch)<br \/>\n            X, y = np.expand_dims(np.array(X), 3), np.array(y)<br \/>\n            yield X, y<br \/>\n            batch = []<\/p>\n<p>def testgen(data, seq_len, targetcol):<br \/>\n    &#8220;Return array of all test samples&#8221;<br \/>\n    batch = []<br \/>\n    for key, df in data.items():<br \/>\n        input_cols = [c for c in df.columns if c != targetcol]<br \/>\n        # find the start of test sample<br \/>\n        t = df.index[df.index &gt;= TRAIN_TEST_CUTOFF][0]<br \/>\n        n = (df.index == t).argmax()<br \/>\n        # extract sample using a sliding window<br \/>\n        for i in range(n+1, len(df)+1):<br \/>\n            frame = df.iloc[i-seq_len:i]<br \/>\n            batch.append([frame[input_cols].values, frame[targetcol][-1]])<br \/>\n    X, y = zip(*batch)<br \/>\n    return np.expand_dims(np.array(X),3), np.array(y)<\/p>\n<p># Read data into pandas DataFrames<br \/>\ndata = {}<br \/>\nfor filename in os.listdir(DATADIR):<br \/>\n    if not filename.lower().endswith(&#8220;.csv&#8221;):<br \/>\n        continue # read only the CSV files<br \/>\n    filepath = os.path.join(DATADIR, filename)<br \/>\n    X = pd.read_csv(filepath, index_col=&#8221;Date&#8221;, parse_dates=True)<br \/>\n    # basic preprocessing: get the name, the classification<br \/>\n    # Save the target variable as a column in dataframe for easier dropna()<br \/>\n    name = X[&#8220;Name&#8221;][0]<br \/>\n    del X[&#8220;Name&#8221;]<br \/>\n    cols = X.columns<br \/>\n    X[&#8220;Target&#8221;] = (X[&#8220;Close&#8221;].pct_change().shift(-1) &gt; 0).astype(int)<br \/>\n    X.dropna(inplace=True)<br \/>\n    # Fit the standard scaler using the training dataset<br \/>\n    index = X.index[X.index &amp;lt; TRAIN_TEST_CUTOFF]<br \/>\n    index = index[:int(len(index) * TRAIN_VALID_RATIO)]<br \/>\n    scaler = StandardScaler().fit(X.loc[index, cols])<br \/>\n    # Save scale transformed dataframe<br \/>\n    X[cols] = scaler.transform(X[cols])<br \/>\n    data[name] = X<\/p>\n<p>seq_len    = 60<br \/>\nbatch_size = 128<br \/>\nn_epochs   = 20<br \/>\nn_features = 82<\/p>\n<p># Produce CNNpred as a binary classification problem<br \/>\nmodel = cnnpred_2d(seq_len, n_features)<br \/>\nmodel.compile(optimizer=&#8221;adam&#8221;, loss=&#8221;mae&#8221;, metrics=[&#8220;acc&#8221;, f1macro])<br \/>\nmodel.summary() # print model structure to console<\/p>\n<p># Set up callbacks and fit the model<br \/>\n# We use custom validation score f1macro() and hence monitor for &#8220;val_f1macro&#8221;<br \/>\ncheckpoint_path = &#8220;.\/cp2d-{epoch}-{val_f1macro:.2f}.h5&#8243;<br \/>\ncallbacks = [<br \/>\n    ModelCheckpoint(checkpoint_path,<br \/>\n                    monitor=&#8217;val_f1macro&#8217;, mode=&#8221;max&#8221;, verbose=0,<br \/>\n                    save_best_only=True, save_weights_only=False, save_freq=&#8221;epoch&#8221;)<br \/>\n]<\/p>\n<p>model.fit(datagen(data, seq_len, batch_size, &#8220;Target&#8221;, &#8220;train&#8221;),<br \/>\n          validation_data=datagen(data, seq_len, batch_size, &#8220;Target&#8221;, &#8220;valid&#8221;),<br \/>\n          epochs=n_epochs, steps_per_epoch=400, validation_steps=10, verbose=1, callbacks=callbacks)<\/p>\n<p># Prepare test data<br \/>\ntest_data, test_target = testgen(data, seq_len, &#8220;Target&#8221;)<\/p>\n<p># Test the model<br \/>\ntest_out = model.predict(test_data)<br \/>\ntest_pred = (test_out &gt; 0.5).astype(int)<br \/>\nprint(&#8220;accuracy:&#8221;, accuracy_score(test_pred, test_target))<br \/>\nprint(&#8220;MAE:&#8221;, mean_absolute_error(test_pred, test_target))<br \/>\nprint(&#8220;F1:&#8221;, f1_score(test_pred, test_target))<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<div class=\"urvanov-syntax-highlighter-nums-content\">\n<p>1<\/p>\n<p>2<\/p>\n<p>3<\/p>\n<p>4<\/p>\n<p>5<\/p>\n<p>6<\/p>\n<p>7<\/p>\n<p>8<\/p>\n<p>9<\/p>\n<p>10<\/p>\n<p>11<\/p>\n<p>12<\/p>\n<p>13<\/p>\n<p>14<\/p>\n<p>15<\/p>\n<p>16<\/p>\n<p>17<\/p>\n<p>18<\/p>\n<p>19<\/p>\n<p>20<\/p>\n<p>21<\/p>\n<p>22<\/p>\n<p>23<\/p>\n<p>24<\/p>\n<p>25<\/p>\n<p>26<\/p>\n<p>27<\/p>\n<p>28<\/p>\n<p>29<\/p>\n<p>30<\/p>\n<p>31<\/p>\n<p>32<\/p>\n<p>33<\/p>\n<p>34<\/p>\n<p>35<\/p>\n<p>36<\/p>\n<p>37<\/p>\n<p>38<\/p>\n<p>39<\/p>\n<p>40<\/p>\n<p>41<\/p>\n<p>42<\/p>\n<p>43<\/p>\n<p>44<\/p>\n<p>45<\/p>\n<p>46<\/p>\n<p>47<\/p>\n<p>48<\/p>\n<p>49<\/p>\n<p>50<\/p>\n<p>51<\/p>\n<p>52<\/p>\n<p>53<\/p>\n<p>54<\/p>\n<p>55<\/p>\n<p>56<\/p>\n<p>57<\/p>\n<p>58<\/p>\n<p>59<\/p>\n<p>60<\/p>\n<p>61<\/p>\n<p>62<\/p>\n<p>63<\/p>\n<p>64<\/p>\n<p>65<\/p>\n<p>66<\/p>\n<p>67<\/p>\n<p>68<\/p>\n<p>69<\/p>\n<p>70<\/p>\n<p>71<\/p>\n<p>72<\/p>\n<p>73<\/p>\n<p>74<\/p>\n<p>75<\/p>\n<p>76<\/p>\n<p>77<\/p>\n<p>78<\/p>\n<p>79<\/p>\n<p>80<\/p>\n<p>81<\/p>\n<p>82<\/p>\n<p>83<\/p>\n<p>84<\/p>\n<p>85<\/p>\n<p>86<\/p>\n<p>87<\/p>\n<p>88<\/p>\n<p>89<\/p>\n<p>90<\/p>\n<p>91<\/p>\n<p>92<\/p>\n<p>93<\/p>\n<p>94<\/p>\n<p>95<\/p>\n<p>96<\/p>\n<p>97<\/p>\n<p>98<\/p>\n<p>99<\/p>\n<p>100<\/p>\n<p>101<\/p>\n<p>102<\/p>\n<p>103<\/p>\n<p>104<\/p>\n<p>105<\/p>\n<p>106<\/p>\n<p>107<\/p>\n<p>108<\/p>\n<p>109<\/p>\n<p>110<\/p>\n<p>111<\/p>\n<p>112<\/p>\n<p>113<\/p>\n<p>114<\/p>\n<p>115<\/p>\n<p>116<\/p>\n<p>117<\/p>\n<p>118<\/p>\n<p>119<\/p>\n<p>120<\/p>\n<p>121<\/p>\n<p>122<\/p>\n<p>123<\/p>\n<p>124<\/p>\n<p>125<\/p>\n<p>126<\/p>\n<p>127<\/p>\n<p>128<\/p>\n<p>129<\/p>\n<p>130<\/p>\n<p>131<\/p>\n<p>132<\/p>\n<p>133<\/p>\n<p>134<\/p>\n<p>135<\/p>\n<p>136<\/p>\n<p>137<\/p>\n<p>138<\/p>\n<p>139<\/p>\n<p>140<\/p>\n<p>141<\/p>\n<p>142<\/p>\n<p>143<\/p>\n<p>144<\/p>\n<p>145<\/p>\n<p>146<\/p>\n<p>147<\/p>\n<p>148<\/p>\n<p>149<\/p>\n<p>150<\/p>\n<p>151<\/p>\n<p>152<\/p>\n<p>153<\/p>\n<p>154<\/p>\n<p>155<\/p>\n<p>156<\/p>\n<p>157<\/p>\n<p>158<\/p>\n<p>159<\/p>\n<p>160<\/p>\n<p>161<\/p>\n<\/div>\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\">\n<p><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">os<\/span><\/p>\n<p><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">random<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">numpy <\/span><span class=\"crayon-st\">as<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">np<\/span><\/p>\n<p><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">pandas <\/span><span class=\"crayon-st\">as<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">pd<\/span><\/p>\n<p><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">tensorflow <\/span><span class=\"crayon-st\">as<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">tf<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-v\">tensorflow<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">keras <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">backend <\/span><span class=\"crayon-st\">as<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">K<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-v\">tensorflow<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">keras<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">layers <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-v\">Dense<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">Dropout<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">Flatten<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">Conv2D<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">MaxPool2D<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">Input<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-v\">tensorflow<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">keras<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">models <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-v\">Sequential<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">load_model<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-v\">tensorflow<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">keras<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">callbacks <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">ModelCheckpoint<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-v\">sklearn<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">preprocessing <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">StandardScaler<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-v\">sklearn<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">metrics <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-v\">accuracy_score<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">f1_score<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">mean_absolute_error<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-e\">f1metrics <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">f1macro<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-v\">DATADIR<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;.\/Dataset&#8221;<\/span><\/p>\n<p><span class=\"crayon-v\">TRAIN_TEST_CUTOFF<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8216;2016-04-21&#8217;<\/span><\/p>\n<p><span class=\"crayon-v\">TRAIN_VALID_RATIO<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">0.75<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># https:\/\/datascience.stackexchange.com\/questions\/45165\/how-to-get-accuracy-f1-precision-and-recall-for-a-keras-model<\/span><\/p>\n<p><span class=\"crayon-p\"># to implement F1 score for validation in a batch<\/span><\/p>\n<p><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">recall_m<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_true<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">true_positives<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">sum<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">round<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">clip<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e \">y_true *<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">possible_positives<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">sum<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">round<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">clip<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_true<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">recall<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">true_positives<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">\/<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">possible_positives<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">epsilon<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">recall<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">precision_m<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_true<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">true_positives<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">sum<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">round<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">clip<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e \">y_true *<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">predicted_positives<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">sum<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">round<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">clip<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">precision<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">true_positives<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">\/<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">predicted_positives<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">epsilon<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">precision<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">f1_m<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_true<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">precision<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">precision_m<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_true<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">recall<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">recall_m<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_true<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">2<\/span><span class=\"crayon-o\">*<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e \">precision*<\/span><span class=\"crayon-v\">recall<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">\/<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">precision<\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-v\">recall<\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">epsilon<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">f1macro<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_true<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">f_pos<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">f1_m<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_true<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># negative version of the data and prediction<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">f_neg<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">f1_m<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-v\">y_true<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">clip<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">f_pos<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">f_neg<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">\/<\/span><span class=\"crayon-cn\">2<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">cnnpred_2d<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">60<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">n_features<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">82<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">n_filters<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">8<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">8<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">8<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">droprate<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">0.1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;2D-CNNpred model according to the paper&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">model<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">Sequential<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">[<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">Input<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">shape<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">n_features<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">Conv2D<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">n_filters<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">kernel_size<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">n_features<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">activation<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;relu&#8221;<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">Conv2D<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">n_filters<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">kernel_size<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">3<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">activation<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;relu&#8221;<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">MaxPool2D<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">pool_size<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">2<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">Conv2D<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">n_filters<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">2<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">kernel_size<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">3<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">activation<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;relu&#8221;<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">MaxPool2D<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">pool_size<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">2<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">Flatten<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">Dropout<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">droprate<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">Dense<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">activation<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;sigmoid&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">model<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">datagen<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">batch_size<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">targetcol<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">kind<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;As a generator to produce samples for Keras model&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">batch<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">while<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-t\">True<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Pick one dataframe from the pool<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">key<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">random<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">choice<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e\">list<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">keys<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">key<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">input_cols<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-i\">c<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">for<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">c<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">in<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">columns <\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">c<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">!=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">targetcol<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">&amp;<\/span><span class=\"crayon-v\">lt<\/span><span class=\"crayon-sy\">;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">TRAIN_TEST_CUTOFF<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">split<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-t\">int<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e\">len<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">*<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">TRAIN_VALID_RATIO<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">assert<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">split<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">&gt;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;Training data too small for sequence length {}&#8221;<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">format<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">kind<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">==<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8216;train&#8217;<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-v\">split<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-h\">\u00a0\u00a0 <\/span><span class=\"crayon-p\"># range for the training set<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">elif <\/span><span class=\"crayon-v\">kind<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">==<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8216;valid&#8217;<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">split<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-h\">\u00a0\u00a0 <\/span><span class=\"crayon-p\"># range for the validation set<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">else<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">raise <\/span><span class=\"crayon-i\">NotImplementedError<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Pick one position, then clip a sequence length<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">while<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-t\">True<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">t<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">random<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">choice<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0 <\/span><span class=\"crayon-p\"># pick one time step<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">n<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">==<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">t<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">argmax<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-p\"># find its position in the dataframe<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">n<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">&amp;<\/span><span class=\"crayon-v\">lt<\/span><span class=\"crayon-sy\">;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">continue<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-p\"># this sample is not enough for one sequence length<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">frame<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">iloc<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">n<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-v\">n<\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">batch<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">append<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">frame<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">input_cols<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">values<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">loc<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">t<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">targetcol<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">break<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># if we get enough for a batch, dispatch<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">len<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">batch<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">==<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">batch_size<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">zip<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-o\">*<\/span><span class=\"crayon-v\">batch<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">expand_dims<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-t\">array<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">3<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-t\">array<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-i\">yield<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">y<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">batch<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">testgen<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">targetcol<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;Return array of all test samples&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">batch<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">for<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">key<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">df <\/span><span class=\"crayon-st\">in<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">items<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">input_cols<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-i\">c<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">for<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">c<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">in<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">columns <\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">c<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">!=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">targetcol<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># find the start of test sample<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">t<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">&gt;=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">TRAIN_TEST_CUTOFF<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">n<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">==<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">t<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">argmax<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># extract sample using a sliding window<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">for<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">i<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">in<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">range<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">n<\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">len<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">frame<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">iloc<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">i<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-v\">i<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">batch<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">append<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">frame<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">input_cols<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">values<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">frame<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">targetcol<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">zip<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-o\">*<\/span><span class=\"crayon-v\">batch<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">expand_dims<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-t\">array<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">3<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-t\">array<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># Read data into pandas DataFrames<\/span><\/p>\n<p><span class=\"crayon-v\">data<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">{<\/span><span class=\"crayon-sy\">}<\/span><\/p>\n<p><span class=\"crayon-st\">for<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">filename <\/span><span class=\"crayon-st\">in<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">os<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">listdir<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">DATADIR<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">not<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">filename<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">lower<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">endswith<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;.csv&#8221;<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">continue<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-p\"># read only the CSV files<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">filepath<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">os<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">path<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">join<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">DATADIR<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">filename<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">pd<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">read_csv<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">filepath<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">index_col<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;Date&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">parse_dates<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-t\">True<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># basic preprocessing: get the name, the classification<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Save the target variable as a column in dataframe for easier dropna()<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">name<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-s\">&#8220;Name&#8221;<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-i\">del<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-s\">&#8220;Name&#8221;<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">cols<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-i\">columns<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-s\">&#8220;Target&#8221;<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-s\">&#8220;Close&#8221;<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">pct_change<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">shift<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">&gt;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">astype<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-t\">int<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">dropna<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">inplace<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-t\">True<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Fit the standard scaler using the training dataset<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">&amp;<\/span><span class=\"crayon-v\">lt<\/span><span class=\"crayon-sy\">;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">TRAIN_TEST_CUTOFF<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-t\">int<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e\">len<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">*<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">TRAIN_VALID_RATIO<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">scaler<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">StandardScaler<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">fit<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">loc<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">cols<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Save scale transformed dataframe<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">cols<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">scaler<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">transform<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">cols<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">name<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">X<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">60<\/span><\/p>\n<p><span class=\"crayon-v\">batch_size<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">128<\/span><\/p>\n<p><span class=\"crayon-v\">n_epochs<\/span><span class=\"crayon-h\">\u00a0\u00a0 <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">20<\/span><\/p>\n<p><span class=\"crayon-v\">n_features<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">82<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># Produce CNNpred as a binary classification problem<\/span><\/p>\n<p><span class=\"crayon-v\">model<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">cnnpred_2d<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">n_features<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-v\">model<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">compile<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">optimizer<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;adam&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">loss<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;mae&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">metrics<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-s\">&#8220;acc&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">f1macro<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-v\">model<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">summary<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-p\"># print model structure to console<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># Set up callbacks and fit the model<\/span><\/p>\n<p><span class=\"crayon-p\"># We use custom validation score f1macro() and hence monitor for &#8220;val_f1macro&#8221;<\/span><\/p>\n<p><span class=\"crayon-v\">checkpoint_path<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;.\/cp2d-{epoch}-{val_f1macro:.2f}.h5&#8221;<\/span><\/p>\n<p><span class=\"crayon-v\">callbacks<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">[<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">ModelCheckpoint<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">checkpoint_path<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">monitor<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8216;val_f1macro&#8217;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">mode<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;max&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">verbose<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">save_best_only<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-t\">True<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">save_weights_only<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-t\">False<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">save_freq<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;epoch&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-sy\">]<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-v\">model<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">fit<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e\">datagen<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">batch_size<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;Target&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;train&#8221;<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">validation_data<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-e\">datagen<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">batch_size<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;Target&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;valid&#8221;<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">epochs<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-v\">n_epochs<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">steps_per_epoch<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">400<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">validation_steps<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">10<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">verbose<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">callbacks<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-v\">callbacks<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># Prepare test data<\/span><\/p>\n<p><span class=\"crayon-v\">test_data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">test_target<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">testgen<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;Target&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># Test the model<\/span><\/p>\n<p><span class=\"crayon-v\">test_out<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">model<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">predict<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">test_data<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-v\">test_pred<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">test_out<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">&gt;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">0.5<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">astype<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-t\">int<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;accuracy:&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">accuracy_score<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">test_pred<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">test_target<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;MAE:&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">mean_absolute_error<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">test_pred<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">test_target<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;F1:&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">f1_score<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">test_pred<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">test_target<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<h2>Extensions<\/h2>\n<p>The original paper called the above model \u201c2D-CNNpred\u201d and there is a version called \u201c3D-CNNpred\u201d. The idea is not only consider the many features of one stock market index but cross compare with many market indices to help prediction on <strong>one index<\/strong>. Refer to the table of features and time steps above, the data for one market index is presented as 2D array. If we stack up multiple such data from different indices, we constructed a 3D array. While the target label is the same, but allowing us to look at a different market may provide some additional information to help prediction.<\/p>\n<p>Because the shape of the data changed, the convolutional network also defined slightly different, and the data generators need some modification accordingly as well. Below is the complete code of the 3D version, which the change from the previous 2d version should be self-explanatory:<\/p>\n<div id=\"urvanov-syntax-highlighter-61954f5e77308100471906\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" minimize scroll-mouseover disable-anim\">\n<p><textarea class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" readonly><br \/>\nimport os<br \/>\nimport random<\/p>\n<p>import numpy as np<br \/>\nimport pandas as pd<br \/>\nimport tensorflow as tf<br \/>\nfrom tensorflow.keras import backend as K<br \/>\nfrom tensorflow.keras.layers import Dense, Dropout, Flatten, Conv2D, MaxPool2D, Input<br \/>\nfrom tensorflow.keras.models import Sequential, load_model<br \/>\nfrom tensorflow.keras.callbacks import ModelCheckpoint<br \/>\nfrom sklearn.preprocessing import StandardScaler<br \/>\nfrom sklearn.metrics import accuracy_score, f1_score, mean_absolute_error<\/p>\n<p>from f1metrics import f1macro<\/p>\n<p>DATADIR = &#8220;.\/Dataset&#8221;<br \/>\nTRAIN_TEST_CUTOFF = &#8216;2016-04-21&#8217;<br \/>\nTRAIN_VALID_RATIO = 0.75<\/p>\n<p># https:\/\/datascience.stackexchange.com\/questions\/45165\/how-to-get-accuracy-f1-precision-and-recall-for-a-keras-model<br \/>\n# to implement F1 score for validation in a batch<br \/>\ndef recall_m(y_true, y_pred):<br \/>\n    true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))<br \/>\n    possible_positives = K.sum(K.round(K.clip(y_true, 0, 1)))<br \/>\n    recall = true_positives \/ (possible_positives + K.epsilon())<br \/>\n    return recall<\/p>\n<p>def precision_m(y_true, y_pred):<br \/>\n    true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))<br \/>\n    predicted_positives = K.sum(K.round(K.clip(y_pred, 0, 1)))<br \/>\n    precision = true_positives \/ (predicted_positives + K.epsilon())<br \/>\n    return precision<\/p>\n<p>def f1_m(y_true, y_pred):<br \/>\n    precision = precision_m(y_true, y_pred)<br \/>\n    recall = recall_m(y_true, y_pred)<br \/>\n    return 2*((precision*recall)\/(precision+recall+K.epsilon()))<\/p>\n<p>def f1macro(y_true, y_pred):<br \/>\n    f_pos = f1_m(y_true, y_pred)<br \/>\n    # negative version of the data and prediction<br \/>\n    f_neg = f1_m(1-y_true, 1-K.clip(y_pred,0,1))<br \/>\n    return (f_pos + f_neg)\/2<\/p>\n<p>def cnnpred_3d(seq_len=60, n_stocks=5, n_features=82, n_filters=(8,8,8), droprate=0.1):<br \/>\n    &#8220;3D-CNNpred model according to the paper&#8221;<br \/>\n    model = Sequential([<br \/>\n        Input(shape=(n_stocks, seq_len, n_features)),<br \/>\n        Conv2D(n_filters[0], kernel_size=(1,1), activation=&#8221;relu&#8221;, data_format=&#8221;channels_last&#8221;),<br \/>\n        Conv2D(n_filters[1], kernel_size=(n_stocks,3), activation=&#8221;relu&#8221;),<br \/>\n        MaxPool2D(pool_size=(1,2)),<br \/>\n        Conv2D(n_filters[2], kernel_size=(1,3), activation=&#8221;relu&#8221;),<br \/>\n        MaxPool2D(pool_size=(1,2)),<br \/>\n        Flatten(),<br \/>\n        Dropout(droprate),<br \/>\n        Dense(1, activation=&#8221;sigmoid&#8221;)<br \/>\n    ])<br \/>\n    return model<\/p>\n<p>def datagen(data, seq_len, batch_size, target_index, targetcol, kind):<br \/>\n    &#8220;As a generator to produce samples for Keras model&#8221;<br \/>\n    # Learn about the data&#8217;s features and time axis<br \/>\n    input_cols = [c for c in data.columns if c[0] != targetcol]<br \/>\n    tickers = sorted(set(c for _,c in input_cols))<br \/>\n    n_features = len(input_cols) \/\/ len(tickers)<br \/>\n    index = data.index[data.index &amp;lt; TRAIN_TEST_CUTOFF]<br \/>\n    split = int(len(index) * TRAIN_VALID_RATIO)<br \/>\n    assert split &gt; seq_len, &#8220;Training data too small for sequence length {}&#8221;.format(seq_len)<br \/>\n    if kind == &#8220;train&#8221;:<br \/>\n        index = index[:split]   # range for the training set<br \/>\n    elif kind == &#8216;valid&#8217;:<br \/>\n        index = index[split:]   # range for the validation set<br \/>\n    else:<br \/>\n        raise NotImplementedError<br \/>\n    # Infinite loop to generate a batch<br \/>\n    batch = []<br \/>\n    while True:<br \/>\n        # Pick one position, then clip a sequence length<br \/>\n        while True:<br \/>\n            t = random.choice(index)<br \/>\n            n = (data.index == t).argmax()<br \/>\n            if n-seq_len+1 &amp;lt; 0:<br \/>\n                continue # this sample is not enough for one sequence length<br \/>\n            frame = data.iloc[n-seq_len+1:n+1][input_cols]<br \/>\n            # convert frame with two level of indices into 3D array<br \/>\n            shape = (len(tickers), len(frame), n_features)<br \/>\n            X = np.full(shape, np.nan)<br \/>\n            for i,ticker in enumerate(tickers):<br \/>\n                X[i] = frame.xs(ticker, axis=1, level=1).values<br \/>\n            batch.append([X, data[targetcol][target_index][t]])<br \/>\n            break<br \/>\n        # if we get enough for a batch, dispatch<br \/>\n        if len(batch) == batch_size:<br \/>\n            X, y = zip(*batch)<br \/>\n            yield np.array(X), np.array(y)<br \/>\n            batch = []<\/p>\n<p>def testgen(data, seq_len, target_index, targetcol):<br \/>\n    &#8220;Return array of all test samples&#8221;<br \/>\n    input_cols = [c for c in data.columns if c[0] != targetcol]<br \/>\n    tickers = sorted(set(c for _,c in input_cols))<br \/>\n    n_features = len(input_cols) \/\/ len(tickers)<br \/>\n    t = data.index[data.index &gt;= TRAIN_TEST_CUTOFF][0]<br \/>\n    n = (data.index == t).argmax()<br \/>\n    batch = []<br \/>\n    for i in range(n+1, len(data)+1):<br \/>\n        # Clip a window of seq_len ends at row position i-1<br \/>\n        frame = data.iloc[i-seq_len:i]<br \/>\n        target = frame[targetcol][target_index][-1]<br \/>\n        frame = frame[input_cols]<br \/>\n        # convert frame with two level of indices into 3D array<br \/>\n        shape = (len(tickers), len(frame), n_features)<br \/>\n        X = np.full(shape, np.nan)<br \/>\n        for i,ticker in enumerate(tickers):<br \/>\n            X[i] = frame.xs(ticker, axis=1, level=1).values<br \/>\n        batch.append([X, target])<br \/>\n    X, y = zip(*batch)<br \/>\n    return np.array(X), np.array(y)<\/p>\n<p># Read data into pandas DataFrames<br \/>\ndata = {}<br \/>\nfor filename in os.listdir(DATADIR):<br \/>\n    if not filename.lower().endswith(&#8220;.csv&#8221;):<br \/>\n        continue # read only the CSV files<br \/>\n    filepath = os.path.join(DATADIR, filename)<br \/>\n    X = pd.read_csv(filepath, index_col=&#8221;Date&#8221;, parse_dates=True)<br \/>\n    # basic preprocessing: get the name, the classification<br \/>\n    # Save the target variable as a column in dataframe for easier dropna()<br \/>\n    name = X[&#8220;Name&#8221;][0]<br \/>\n    del X[&#8220;Name&#8221;]<br \/>\n    cols = X.columns<br \/>\n    X[&#8220;Target&#8221;] = (X[&#8220;Close&#8221;].pct_change().shift(-1) &gt; 0).astype(int)<br \/>\n    X.dropna(inplace=True)<br \/>\n    # Fit the standard scaler using the training dataset<br \/>\n    index = X.index[X.index &amp;lt; TRAIN_TEST_CUTOFF]<br \/>\n    index = index[:int(len(index) * TRAIN_VALID_RATIO)]<br \/>\n    scaler = StandardScaler().fit(X.loc[index, cols])<br \/>\n    # Save scale transformed dataframe<br \/>\n    X[cols] = scaler.transform(X[cols])<br \/>\n    data[name] = X<\/p>\n<p># Transform data into 3D dataframe (multilevel columns)<br \/>\nfor key, df in data.items():<br \/>\n    df.columns = pd.MultiIndex.from_product([df.columns, [key]])<br \/>\ndata = pd.concat(data.values(), axis=1)<\/p>\n<p>seq_len    = 60<br \/>\nbatch_size = 128<br \/>\nn_epochs   = 20<br \/>\nn_features = 82<br \/>\nn_stocks   = 5<\/p>\n<p># Produce CNNpred as a binary classification problem<br \/>\nmodel = cnnpred_3d(seq_len, n_stocks, n_features)<br \/>\nmodel.compile(optimizer=&#8221;adam&#8221;, loss=&#8221;mae&#8221;, metrics=[&#8220;acc&#8221;, f1macro])<br \/>\nmodel.summary() # print model structure to console<\/p>\n<p># Set up callbacks and fit the model<br \/>\n# We use custom validation score f1macro() and hence monitor for &#8220;val_f1macro&#8221;<br \/>\ncheckpoint_path = &#8220;.\/cp3d-{epoch}-{val_f1macro:.2f}.h5&#8243;<br \/>\ncallbacks = [<br \/>\n    ModelCheckpoint(checkpoint_path,<br \/>\n                    monitor=&#8217;val_f1macro&#8217;, mode=&#8221;max&#8221;, verbose=0,<br \/>\n                    save_best_only=True, save_weights_only=False, save_freq=&#8221;epoch&#8221;)<br \/>\n]<\/p>\n<p>model.fit(datagen(data, seq_len, batch_size, &#8220;DJI&#8221;, &#8220;Target&#8221;, &#8220;train&#8221;),<br \/>\n          validation_data=datagen(data, seq_len, batch_size, &#8220;DJI&#8221;, &#8220;Target&#8221;, &#8220;valid&#8221;),<br \/>\n          epochs=n_epochs, steps_per_epoch=400, validation_steps=10, verbose=1, callbacks=callbacks)<\/p>\n<p># Prepare test data<br \/>\ntest_data, test_target = testgen(data, seq_len, &#8220;DJI&#8221;, &#8220;Target&#8221;)<\/p>\n<p># Test the model<br \/>\ntest_out = model.predict(test_data)<br \/>\ntest_pred = (test_out &gt; 0.5).astype(int)<br \/>\nprint(&#8220;accuracy:&#8221;, accuracy_score(test_pred, test_target))<br \/>\nprint(&#8220;MAE:&#8221;, mean_absolute_error(test_pred, test_target))<br \/>\nprint(&#8220;F1:&#8221;, f1_score(test_pred, test_target))<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<div class=\"urvanov-syntax-highlighter-nums-content\">\n<p>1<\/p>\n<p>2<\/p>\n<p>3<\/p>\n<p>4<\/p>\n<p>5<\/p>\n<p>6<\/p>\n<p>7<\/p>\n<p>8<\/p>\n<p>9<\/p>\n<p>10<\/p>\n<p>11<\/p>\n<p>12<\/p>\n<p>13<\/p>\n<p>14<\/p>\n<p>15<\/p>\n<p>16<\/p>\n<p>17<\/p>\n<p>18<\/p>\n<p>19<\/p>\n<p>20<\/p>\n<p>21<\/p>\n<p>22<\/p>\n<p>23<\/p>\n<p>24<\/p>\n<p>25<\/p>\n<p>26<\/p>\n<p>27<\/p>\n<p>28<\/p>\n<p>29<\/p>\n<p>30<\/p>\n<p>31<\/p>\n<p>32<\/p>\n<p>33<\/p>\n<p>34<\/p>\n<p>35<\/p>\n<p>36<\/p>\n<p>37<\/p>\n<p>38<\/p>\n<p>39<\/p>\n<p>40<\/p>\n<p>41<\/p>\n<p>42<\/p>\n<p>43<\/p>\n<p>44<\/p>\n<p>45<\/p>\n<p>46<\/p>\n<p>47<\/p>\n<p>48<\/p>\n<p>49<\/p>\n<p>50<\/p>\n<p>51<\/p>\n<p>52<\/p>\n<p>53<\/p>\n<p>54<\/p>\n<p>55<\/p>\n<p>56<\/p>\n<p>57<\/p>\n<p>58<\/p>\n<p>59<\/p>\n<p>60<\/p>\n<p>61<\/p>\n<p>62<\/p>\n<p>63<\/p>\n<p>64<\/p>\n<p>65<\/p>\n<p>66<\/p>\n<p>67<\/p>\n<p>68<\/p>\n<p>69<\/p>\n<p>70<\/p>\n<p>71<\/p>\n<p>72<\/p>\n<p>73<\/p>\n<p>74<\/p>\n<p>75<\/p>\n<p>76<\/p>\n<p>77<\/p>\n<p>78<\/p>\n<p>79<\/p>\n<p>80<\/p>\n<p>81<\/p>\n<p>82<\/p>\n<p>83<\/p>\n<p>84<\/p>\n<p>85<\/p>\n<p>86<\/p>\n<p>87<\/p>\n<p>88<\/p>\n<p>89<\/p>\n<p>90<\/p>\n<p>91<\/p>\n<p>92<\/p>\n<p>93<\/p>\n<p>94<\/p>\n<p>95<\/p>\n<p>96<\/p>\n<p>97<\/p>\n<p>98<\/p>\n<p>99<\/p>\n<p>100<\/p>\n<p>101<\/p>\n<p>102<\/p>\n<p>103<\/p>\n<p>104<\/p>\n<p>105<\/p>\n<p>106<\/p>\n<p>107<\/p>\n<p>108<\/p>\n<p>109<\/p>\n<p>110<\/p>\n<p>111<\/p>\n<p>112<\/p>\n<p>113<\/p>\n<p>114<\/p>\n<p>115<\/p>\n<p>116<\/p>\n<p>117<\/p>\n<p>118<\/p>\n<p>119<\/p>\n<p>120<\/p>\n<p>121<\/p>\n<p>122<\/p>\n<p>123<\/p>\n<p>124<\/p>\n<p>125<\/p>\n<p>126<\/p>\n<p>127<\/p>\n<p>128<\/p>\n<p>129<\/p>\n<p>130<\/p>\n<p>131<\/p>\n<p>132<\/p>\n<p>133<\/p>\n<p>134<\/p>\n<p>135<\/p>\n<p>136<\/p>\n<p>137<\/p>\n<p>138<\/p>\n<p>139<\/p>\n<p>140<\/p>\n<p>141<\/p>\n<p>142<\/p>\n<p>143<\/p>\n<p>144<\/p>\n<p>145<\/p>\n<p>146<\/p>\n<p>147<\/p>\n<p>148<\/p>\n<p>149<\/p>\n<p>150<\/p>\n<p>151<\/p>\n<p>152<\/p>\n<p>153<\/p>\n<p>154<\/p>\n<p>155<\/p>\n<p>156<\/p>\n<p>157<\/p>\n<p>158<\/p>\n<p>159<\/p>\n<p>160<\/p>\n<p>161<\/p>\n<p>162<\/p>\n<p>163<\/p>\n<p>164<\/p>\n<p>165<\/p>\n<p>166<\/p>\n<p>167<\/p>\n<p>168<\/p>\n<p>169<\/p>\n<p>170<\/p>\n<p>171<\/p>\n<p>172<\/p>\n<p>173<\/p>\n<p>174<\/p>\n<p>175<\/p>\n<p>176<\/p>\n<p>177<\/p>\n<p>178<\/p>\n<p>179<\/p>\n<\/div>\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\">\n<p><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">os<\/span><\/p>\n<p><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">random<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">numpy <\/span><span class=\"crayon-st\">as<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">np<\/span><\/p>\n<p><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">pandas <\/span><span class=\"crayon-st\">as<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">pd<\/span><\/p>\n<p><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">tensorflow <\/span><span class=\"crayon-st\">as<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">tf<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-v\">tensorflow<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">keras <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">backend <\/span><span class=\"crayon-st\">as<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">K<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-v\">tensorflow<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">keras<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">layers <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-v\">Dense<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">Dropout<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">Flatten<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">Conv2D<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">MaxPool2D<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">Input<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-v\">tensorflow<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">keras<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">models <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-v\">Sequential<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">load_model<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-v\">tensorflow<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">keras<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">callbacks <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">ModelCheckpoint<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-v\">sklearn<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">preprocessing <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">StandardScaler<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-v\">sklearn<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">metrics <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-v\">accuracy_score<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">f1_score<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">mean_absolute_error<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-e\">f1metrics <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">f1macro<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-v\">DATADIR<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;.\/Dataset&#8221;<\/span><\/p>\n<p><span class=\"crayon-v\">TRAIN_TEST_CUTOFF<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8216;2016-04-21&#8217;<\/span><\/p>\n<p><span class=\"crayon-v\">TRAIN_VALID_RATIO<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">0.75<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># https:\/\/datascience.stackexchange.com\/questions\/45165\/how-to-get-accuracy-f1-precision-and-recall-for-a-keras-model<\/span><\/p>\n<p><span class=\"crayon-p\"># to implement F1 score for validation in a batch<\/span><\/p>\n<p><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">recall_m<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_true<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">true_positives<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">sum<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">round<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">clip<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e \">y_true *<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">possible_positives<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">sum<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">round<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">clip<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_true<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">recall<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">true_positives<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">\/<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">possible_positives<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">epsilon<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">recall<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">precision_m<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_true<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">true_positives<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">sum<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">round<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">clip<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e \">y_true *<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">predicted_positives<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">sum<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">round<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">clip<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">precision<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">true_positives<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">\/<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">predicted_positives<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">epsilon<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">precision<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">f1_m<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_true<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">precision<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">precision_m<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_true<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">recall<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">recall_m<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_true<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">2<\/span><span class=\"crayon-o\">*<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e \">precision*<\/span><span class=\"crayon-v\">recall<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">\/<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">precision<\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-v\">recall<\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">epsilon<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">f1macro<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_true<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">f_pos<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">f1_m<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_true<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># negative version of the data and prediction<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">f_neg<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">f1_m<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-v\">y_true<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-v\">K<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">clip<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y_pred<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">f_pos<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">f_neg<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">\/<\/span><span class=\"crayon-cn\">2<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">cnnpred_3d<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">60<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">n_stocks<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">5<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">n_features<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">82<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">n_filters<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">8<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">8<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">8<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">droprate<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">0.1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;3D-CNNpred model according to the paper&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">model<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">Sequential<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">[<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">Input<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">shape<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">n_stocks<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">n_features<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">Conv2D<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">n_filters<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">kernel_size<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">activation<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;relu&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">data_format<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;channels_last&#8221;<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">Conv2D<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">n_filters<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">kernel_size<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">n_stocks<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">3<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">activation<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;relu&#8221;<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">MaxPool2D<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">pool_size<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">2<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">Conv2D<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">n_filters<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">2<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">kernel_size<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">3<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">activation<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;relu&#8221;<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">MaxPool2D<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">pool_size<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-cn\">2<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">Flatten<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">Dropout<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">droprate<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">Dense<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">activation<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;sigmoid&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">model<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">datagen<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">batch_size<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">target_index<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">targetcol<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">kind<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;As a generator to produce samples for Keras model&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Learn about the data&#8217;s features and time axis<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">input_cols<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-i\">c<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">for<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">c<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">in<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">columns <\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">c<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">!=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">targetcol<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">tickers<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">sorted<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e\">set<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-i\">c<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">for<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">_<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-i\">c<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">in<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">input_cols<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">n_features<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">len<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">input_cols<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-c\">\/\/ len(tickers)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">&amp;<\/span><span class=\"crayon-v\">lt<\/span><span class=\"crayon-sy\">;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">TRAIN_TEST_CUTOFF<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">split<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-t\">int<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e\">len<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">*<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">TRAIN_VALID_RATIO<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">assert<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">split<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">&gt;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;Training data too small for sequence length {}&#8221;<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">format<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">kind<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">==<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;train&#8221;<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-v\">split<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-h\">\u00a0\u00a0 <\/span><span class=\"crayon-p\"># range for the training set<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">elif <\/span><span class=\"crayon-v\">kind<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">==<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8216;valid&#8217;<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">split<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-h\">\u00a0\u00a0 <\/span><span class=\"crayon-p\"># range for the validation set<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">else<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">raise <\/span><span class=\"crayon-i\">NotImplementedError<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Infinite loop to generate a batch<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">batch<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">while<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-t\">True<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Pick one position, then clip a sequence length<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">while<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-t\">True<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">t<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">random<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">choice<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">n<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">==<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">t<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">argmax<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">n<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">&amp;<\/span><span class=\"crayon-v\">lt<\/span><span class=\"crayon-sy\">;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">continue<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-p\"># this sample is not enough for one sequence length<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">frame<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">iloc<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">n<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-v\">n<\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">input_cols<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># convert frame with two level of indices into 3D array<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">shape<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e\">len<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">tickers<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">len<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">frame<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">n_features<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">full<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">shape<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">nan<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">for<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">i<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-e\">ticker <\/span><span class=\"crayon-st\">in<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">enumerate<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">tickers<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">i<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">frame<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">xs<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">ticker<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">axis<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">level<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">values<\/span><\/p>\n<p><span class=\"crayon-e\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">batch<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">append<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">targetcol<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">target_index<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">t<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">break<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># if we get enough for a batch, dispatch<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">len<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">batch<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">==<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">batch_size<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">zip<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-o\">*<\/span><span class=\"crayon-v\">batch<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">yield <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-t\">array<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-t\">array<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">batch<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">testgen<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">target_index<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">targetcol<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;Return array of all test samples&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">input_cols<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-i\">c<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">for<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">c<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">in<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">columns <\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">c<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">!=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">targetcol<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">tickers<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">sorted<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e\">set<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-i\">c<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">for<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">_<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-i\">c<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">in<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">input_cols<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">n_features<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">len<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">input_cols<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-c\">\/\/ len(tickers)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">t<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">&gt;=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">TRAIN_TEST_CUTOFF<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">n<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">==<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">t<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">argmax<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">batch<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">for<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">i<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">in<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">range<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">n<\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">len<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Clip a window of seq_len ends at row position i-1<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">frame<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">iloc<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">i<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-v\">i<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">target<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">frame<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">targetcol<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">target_index<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">frame<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">frame<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">input_cols<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># convert frame with two level of indices into 3D array<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">shape<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e\">len<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">tickers<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">len<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">frame<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">n_features<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">full<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">shape<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">nan<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">for<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">i<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-e\">ticker <\/span><span class=\"crayon-st\">in<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">enumerate<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">tickers<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">i<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">frame<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">xs<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">ticker<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">axis<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">level<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">values<\/span><\/p>\n<p><span class=\"crayon-e\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">batch<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">append<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">target<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">zip<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-o\">*<\/span><span class=\"crayon-v\">batch<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-t\">array<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">np<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-t\">array<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">y<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># Read data into pandas DataFrames<\/span><\/p>\n<p><span class=\"crayon-v\">data<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">{<\/span><span class=\"crayon-sy\">}<\/span><\/p>\n<p><span class=\"crayon-st\">for<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">filename <\/span><span class=\"crayon-st\">in<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">os<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">listdir<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">DATADIR<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">not<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">filename<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">lower<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">endswith<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;.csv&#8221;<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">continue<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-p\"># read only the CSV files<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">filepath<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">os<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">path<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">join<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">DATADIR<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">filename<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">pd<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">read_csv<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">filepath<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">index_col<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;Date&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">parse_dates<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-t\">True<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># basic preprocessing: get the name, the classification<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Save the target variable as a column in dataframe for easier dropna()<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">name<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-s\">&#8220;Name&#8221;<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-i\">del<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-s\">&#8220;Name&#8221;<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">cols<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-i\">columns<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-s\">&#8220;Target&#8221;<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-s\">&#8220;Close&#8221;<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">pct_change<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">shift<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">&gt;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">astype<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-t\">int<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">dropna<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">inplace<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-t\">True<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Fit the standard scaler using the training dataset<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">&amp;<\/span><span class=\"crayon-v\">lt<\/span><span class=\"crayon-sy\">;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">TRAIN_TEST_CUTOFF<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-t\">int<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e\">len<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">*<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">TRAIN_VALID_RATIO<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">scaler<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">StandardScaler<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">fit<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">loc<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">index<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">cols<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Save scale transformed dataframe<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">cols<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">scaler<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">transform<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">cols<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">name<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">X<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># Transform data into 3D dataframe (multilevel columns)<\/span><\/p>\n<p><span class=\"crayon-st\">for<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">key<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">df <\/span><span class=\"crayon-st\">in<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">items<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">columns<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">pd<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">MultiIndex<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">from_product<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">df<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">columns<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">key<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-v\">data<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">pd<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">concat<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">values<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">axis<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">60<\/span><\/p>\n<p><span class=\"crayon-v\">batch_size<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">128<\/span><\/p>\n<p><span class=\"crayon-v\">n_epochs<\/span><span class=\"crayon-h\">\u00a0\u00a0 <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">20<\/span><\/p>\n<p><span class=\"crayon-v\">n_features<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">82<\/span><\/p>\n<p><span class=\"crayon-v\">n_stocks<\/span><span class=\"crayon-h\">\u00a0\u00a0 <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">5<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># Produce CNNpred as a binary classification problem<\/span><\/p>\n<p><span class=\"crayon-v\">model<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">cnnpred_3d<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">n_stocks<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">n_features<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-v\">model<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">compile<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">optimizer<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;adam&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">loss<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;mae&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">metrics<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-s\">&#8220;acc&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">f1macro<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-v\">model<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">summary<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-p\"># print model structure to console<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># Set up callbacks and fit the model<\/span><\/p>\n<p><span class=\"crayon-p\"># We use custom validation score f1macro() and hence monitor for &#8220;val_f1macro&#8221;<\/span><\/p>\n<p><span class=\"crayon-v\">checkpoint_path<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;.\/cp3d-{epoch}-{val_f1macro:.2f}.h5&#8221;<\/span><\/p>\n<p><span class=\"crayon-v\">callbacks<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">[<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">ModelCheckpoint<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">checkpoint_path<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">monitor<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8216;val_f1macro&#8217;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">mode<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;max&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">verbose<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">save_best_only<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-t\">True<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">save_weights_only<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-t\">False<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">save_freq<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;epoch&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-sy\">]<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-v\">model<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">fit<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e\">datagen<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">batch_size<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;DJI&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;Target&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;train&#8221;<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">validation_data<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-e\">datagen<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">batch_size<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;DJI&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;Target&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;valid&#8221;<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">epochs<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-v\">n_epochs<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">steps_per_epoch<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">400<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">validation_steps<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">10<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">verbose<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">callbacks<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-v\">callbacks<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># Prepare test data<\/span><\/p>\n<p><span class=\"crayon-v\">test_data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">test_target<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">testgen<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">seq_len<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;DJI&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;Target&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># Test the model<\/span><\/p>\n<p><span class=\"crayon-v\">test_out<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">model<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">predict<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">test_data<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-v\">test_pred<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">test_out<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">&gt;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">0.5<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">astype<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-t\">int<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;accuracy:&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">accuracy_score<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">test_pred<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">test_target<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;MAE:&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">mean_absolute_error<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">test_pred<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">test_target<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;F1:&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">f1_score<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">test_pred<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">test_target<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<p>While the model above is for next-step prediction, it does not stop you from making prediction for k steps ahead if you replace the target label to a different calculation. This may be an exercise for you.<\/p>\n<h2>Does it work?<\/h2>\n<p>As in all prediction projects in the financial market, it is always unrealistic to expect a high accuracy. The training parameter in the code above can produce slightly more than 50% accuracy in the testing set. While the number of epochs and batch size are deliberately set smaller to save time, there should not be much room for improvement.<\/p>\n<p>In the original paper, it is reported that the 3D-CNNpred performed better than 2D-CNNpred but only attaining the F1 score of less than 0.6. This is already doing better than three baseline models mentioned in the paper. It may be of some use, but not a magic that can help you make money quick.<\/p>\n<p>From machine learning technique perspective, here we classify a panel of data into whether the market direction is up or down the next day. Hence while the data is not an image, it resembles one since both are presented in the form of a 2D array. The technique of convolutional layers can therefore applied, but we may use a different filter size to match the intuition we usually have for financial time series.<\/p>\n<h2>Further readings<\/h2>\n<p>The original paper is available at:<\/p>\n<p>If you are new to finance application and want to build the connection between machine learning techniques and finance, you may find this book useful:<\/p>\n<p>On the similar topic, we have a previous post on using CNN for time series, but using 1D convolutional layers;<\/p>\n<p>You may also find the following documentation helpful to explain some syntax we used above:<\/p>\n<h2>Summary<\/h2>\n<p>In this tutorial, you discovered how a CNN model can be built for prediction in financial time series.<\/p>\n<p>Specifically, you learned:<\/p>\n<ul>\n<li>How to create 2D convolutional layers to process the time series<\/li>\n<li>How to present the time series data in a multidimensional array so that the convolutional layers can be applied<\/li>\n<li>What is a data generator for Keras model training and how to use it<\/li>\n<li>How to monitor the performance of model training with a custom metric<\/li>\n<li>What to expect in predicting financial market<\/li>\n<\/ul><\/div>\n","protected":false},"excerpt":{"rendered":"<p>https:\/\/machinelearningmastery.com\/using-cnn-for-financial-time-series-prediction\/<\/p>\n","protected":false},"author":0,"featured_media":1222,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[],"_links":{"self":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/1221"}],"collection":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/comments?post=1221"}],"version-history":[{"count":0,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/1221\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media\/1222"}],"wp:attachment":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media?parent=1221"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/categories?post=1221"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/tags?post=1221"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}