{"id":112,"date":"2020-08-20T17:20:17","date_gmt":"2020-08-20T17:20:17","guid":{"rendered":"https:\/\/machine-learning.webcloning.com\/2020\/08\/20\/how-to-calculate-the-bias-variance-trade-off-with-python\/"},"modified":"2020-08-20T17:20:17","modified_gmt":"2020-08-20T17:20:17","slug":"how-to-calculate-the-bias-variance-trade-off-with-python","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2020\/08\/20\/how-to-calculate-the-bias-variance-trade-off-with-python\/","title":{"rendered":"How to Calculate the Bias-Variance Trade-off with Python"},"content":{"rendered":"<div id=\"\">\n<p id=\"last-modified-info\">Last Updated on August 20, 2020<\/p>\n<p>The performance of a machine learning model can be characterized in terms of the <strong>bias<\/strong> and the <strong>variance<\/strong> of the model.<\/p>\n<p>A model with high bias makes strong assumptions about the form of the unknown underlying function that maps inputs to outputs in the dataset, such as linear regression. A model with high variance is highly dependent upon the specifics of the training dataset, such as unpruned decision trees. We desire models with low bias and low variance, although there is often a trade-off between these two concerns.<\/p>\n<p>The bias-variance trade-off is a useful conceptualization for selecting and configuring models, although generally cannot be computed directly as it requires full knowledge of the problem domain, which we do not have. Nevertheless, in some cases, we can estimate the error of a model and divide the error down into bias and variance components, which may provide insight into a given model\u2019s behavior.<\/p>\n<p>In this tutorial, you will discover how to calculate the bias and variance for a machine learning model.<\/p>\n<p>After completing this tutorial, you will know:<\/p>\n<ul>\n<li>Model error consists of model variance, model bias, and irreducible error.<\/li>\n<li>We seek models with low bias and variance, although typically reducing one results in a rise in the other.<\/li>\n<li>How to decompose mean squared error into model bias and variance terms.<\/li>\n<\/ul>\n<p><strong>Kick-start your project<\/strong> with my new book <a href=\"https:\/\/machinelearningmastery.com\/machine-learning-with-python\/\">Machine Learning Mastery With Python<\/a>, including <em>step-by-step tutorials<\/em> and the <em>Python source code<\/em> files for all examples.<\/p>\n<p>Let\u2019s get started.<\/p>\n<div id=\"attachment_10713\" class=\"wp-caption aligncenter\">\n<img decoding=\"async\" aria-describedby=\"caption-attachment-10713\" loading=\"lazy\" class=\"size-full wp-image-10713\" src=\"https:\/\/3qeqpr26caki16dnhd19sv6by6v-wpengine.netdna-ssl.com\/wp-content\/uploads\/2020\/08\/How-to-Calculate-the-Bias-Variance-Trade-off-in-Python.jpg\" alt=\"How to Calculate the Bias-Variance Trade-off in Python\" width=\"799\" height=\"534\"><\/p>\n<p id=\"caption-attachment-10713\" class=\"wp-caption-text\">How to Calculate the Bias-Variance Trade-off in Python<br \/>Photo by <a href=\"https:\/\/flickr.com\/photos\/nathalie-photos\/38618514704\/\">Nathalie<\/a>, some rights reserved.<\/p>\n<\/div>\n<h2>Tutorial Overview<\/h2>\n<p>This tutorial is divided into three parts; they are:<\/p>\n<ol>\n<li>Bias, Variance, and Irreducible Error<\/li>\n<li>Bias-Variance Trade-off<\/li>\n<li>Calculate the Bias and Variance<\/li>\n<\/ol>\n<h2>Bias, Variance, and Irreducible Error<\/h2>\n<p>Consider a machine learning model that makes predictions for a predictive modeling task, such as regression or classification.<\/p>\n<p>The performance of the model on the task can be described in terms of the prediction error on all examples not used to train the model. We will refer to this as the model error.<\/p>\n<p>The model error can be decomposed into three sources of error: the <strong>variance<\/strong> of the model, the <strong>bias<\/strong> of the model, and the variance of the <strong>irreducible error<\/strong> in the data.<\/p>\n<ul>\n<li>Error(Model) = Variance(Model) + Bias(Model) + Variance(Irreducible Error)<\/li>\n<\/ul>\n<p>Let\u2019s take a closer look at each of these three terms.<\/p>\n<h3>Model Bias<\/h3>\n<p>The bias is a measure of how close the model can capture the mapping function between inputs and outputs.<\/p>\n<p>It captures the rigidity of the model: the strength of the assumption the model has about the functional form of the mapping between inputs and outputs.<\/p>\n<blockquote>\n<p>This reflects how close the functional form of the model can get to the true relationship between the predictors and the outcome.<\/p>\n<\/blockquote>\n<p>\u2014 Page 97, <a href=\"https:\/\/amzn.to\/3a7Yzrc\">Applied Predictive Modeling<\/a>, 2013.<\/p>\n<p>A model with high bias is helpful when the bias matches the true but unknown underlying mapping function for the predictive modeling problem. Yet, a model with a large bias will be completely useless when the functional form for the problem is mismatched with the assumptions of the model, e.g. assuming a linear relationship for data with a high non-linear relationship.<\/p>\n<ul>\n<li>\n<strong>Low Bias<\/strong>: Weak assumptions regarding the functional form of the mapping of inputs to outputs.<\/li>\n<li>\n<strong>High Bias<\/strong>: Strong assumptions regarding the functional form of the mapping of inputs to outputs.<\/li>\n<\/ul>\n<p>The bias is always positive.<\/p>\n<h3>Model Variance<\/h3>\n<p>The variance of the model is the amount the performance of the model changes when it is fit on different training data.<\/p>\n<p>It captures the impact of the specifics the data has on the model.<\/p>\n<blockquote>\n<p>Variance refers to the amount by which [the model] would change if we estimated it using a different training data set.<\/p>\n<\/blockquote>\n<p>\u2014 Page 34, <a href=\"https:\/\/amzn.to\/2RC7ElX\">An Introduction to Statistical Learning with Applications in R<\/a>, 2014.<\/p>\n<p>A model with high variance will change a lot with small changes to the training dataset. Conversely, a model with low variance will change little with small or even large changes to the training dataset.<\/p>\n<ul>\n<li>\n<strong>Low Variance<\/strong>: Small changes to the model with changes to the training dataset.<\/li>\n<li>\n<strong>High Variance<\/strong>: Large changes to the model with changes to the training dataset.<\/li>\n<\/ul>\n<p>The variance is always positive.<\/p>\n<h3>Irreducible Error<\/h3>\n<p>On the whole, the error of a model consists of reducible error and irreducible error.<\/p>\n<ul>\n<li>Model Error = Reducible Error + Irreducible Error<\/li>\n<\/ul>\n<p>The reducible error is the element that we can improve. It is the quantity that we reduce when the model is learning on a training dataset and we try to get this number as close to zero as possible.<\/p>\n<p>The irreducible error is the error that we can not remove with our model, or with any model.<\/p>\n<p>The error is caused by elements outside our control, such as statistical noise in the observations.<\/p>\n<blockquote>\n<p>\u2026 usually called \u201cirreducible noise\u201d and cannot be eliminated by modeling.<\/p>\n<\/blockquote>\n<p>\u2014 Page 97, <a href=\"https:\/\/amzn.to\/3a7Yzrc\">Applied Predictive Modeling<\/a>, 2013.<\/p>\n<p>As such, although we may be able to squash the reducible error to a very small value close to zero, or even zero in some cases, we will also have some irreducible error. It defines a lower bound in performance on a problem.<\/p>\n<blockquote>\n<p>It is important to keep in mind that the irreducible error will always provide an upper bound on the accuracy of our prediction for Y. This bound is almost always unknown in practice.<\/p>\n<\/blockquote>\n<p>\u2014 Page 19, <a href=\"https:\/\/amzn.to\/2RC7ElX\">An Introduction to Statistical Learning with Applications in R<\/a>, 2014.<\/p>\n<p>It is a reminder that no model is perfect.<\/p>\n<h2>Bias-Variance Trade-off<\/h2>\n<p>The bias and the variance of a model\u2019s performance are connected.<\/p>\n<p>Ideally, we would prefer a model with low bias and low variance, although in practice, this is very challenging. In fact, this could be described as the goal of applied machine learning for a given predictive modeling problem,<\/p>\n<p>Reducing the bias can easily be achieved by increasing the variance. Conversely, reducing the variance can easily be achieved by increasing the bias.<\/p>\n<blockquote>\n<p>This is referred to as a trade-off because it is easy to obtain a method with extremely low bias but high variance [\u2026] or a method with very low variance but high bias \u2026<\/p>\n<\/blockquote>\n<p>\u2014 Page 36, <a href=\"https:\/\/amzn.to\/2RC7ElX\">An Introduction to Statistical Learning with Applications in R<\/a>, 2014.<\/p>\n<p>This relationship is generally referred to as the <strong>bias-variance trade-off<\/strong>. It is a conceptual framework for thinking about how to choose models and model configuration.<\/p>\n<p>We can choose a model based on its bias or variance. Simple models, such as linear regression and logistic regression, generally have a high bias and a low variance. Complex models, such as random forest, generally have a low bias but a high variance.<\/p>\n<p>We may also choose model configurations based on their effect on the bias and variance of the model. The k hyperparameter in k-nearest neighbors controls the bias-variance trade-off. Small values, such as k=1, result in a low bias and a high variance, whereas large k values, such as k=21, result in a high bias and a low variance.<\/p>\n<p>High bias is not always bad, nor is high variance, but they can lead to poor results.<\/p>\n<p>We often must test a suite of different models and model configurations in order to discover what works best for a given dataset. A model with a large bias may be too rigid and underfit the problem. Conversely, a large variance may overfit the problem.<\/p>\n<p>We may decide to increase the bias or the variance as long as it decreases the overall estimate of model error.<\/p>\n<h2>Calculate the Bias and Variance<\/h2>\n<p>I get this question all the time:<\/p>\n<blockquote>\n<p>How can I calculate the bias-variance trade-off for my algorithm on my dataset?<\/p>\n<\/blockquote>\n<p>Technically, we cannot perform this calculation.<\/p>\n<p>We cannot calculate the actual bias and variance for a predictive modeling problem.<\/p>\n<p>This is because we do not know the true mapping function for a predictive modeling problem.<\/p>\n<p>Instead, we use the bias, variance, irreducible error, and the bias-variance trade-off as tools to help select models, configure models, and interpret results.<\/p>\n<blockquote>\n<p>In a real-life situation in which f is unobserved, it is generally not possible to explicitly compute the test MSE, bias, or variance for a statistical learning method. Nevertheless, one should always keep the bias-variance trade-off in mind.<\/p>\n<\/blockquote>\n<p>\u2014 Page 36, <a href=\"https:\/\/amzn.to\/2RC7ElX\">An Introduction to Statistical Learning with Applications in R<\/a>, 2014.<\/p>\n<p>Even though the bias-variance trade-off is a conceptual tool, we can estimate it in some cases.<\/p>\n<p>The <a href=\"http:\/\/rasbt.github.io\/mlxtend\/\">mlxtend library<\/a> by <a href=\"https:\/\/sebastianraschka.com\/\">Sebastian Raschka<\/a> provides the <a href=\"http:\/\/rasbt.github.io\/mlxtend\/user_guide\/evaluate\/bias_variance_decomp\/\">bias_variance_decomp() function<\/a> that can estimate the bias and variance for a model over multiple bootstrap samples.<\/p>\n<p>First, you must install the mlxtend library; for example:<\/p>\n<p><!-- Urvanov Syntax Highlighter v2.8.12 --><\/p>\n<p><!-- [Format Time: 0.0001 seconds] --><\/p>\n<p>The example below loads the <a href=\"https:\/\/github.com\/jbrownlee\/Datasets\/blob\/master\/housing.names\">Boston housing dataset<\/a> directly via URL, splits it into train and test sets, then estimates the mean squared error (MSE) for a linear regression as well as the bias and variance for the model error over 200 bootstrap samples.<\/p>\n<p><!-- Urvanov Syntax Highlighter v2.8.12 --><\/p>\n<div id=\"urvanov-syntax-highlighter-5f3eafcb629c2466933617\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" minimize scroll-mouseover\">\n<p><textarea class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" readonly><br \/>\n# estimate the bias and variance for a regression model<br \/>\nfrom pandas import read_csv<br \/>\nfrom sklearn.model_selection import train_test_split<br \/>\nfrom sklearn.linear_model import LinearRegression<br \/>\nfrom mlxtend.evaluate import bias_variance_decomp<br \/>\n# load dataset<br \/>\nurl = &#8216;https:\/\/raw.githubusercontent.com\/jbrownlee\/Datasets\/master\/housing.csv&#8217;<br \/>\ndataframe = read_csv(url, header=None)<br \/>\n# separate into inputs and outputs<br \/>\ndata = dataframe.values<br \/>\nX, y = data[:, :-1], data[:, -1]<br \/>\n# split the data<br \/>\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=1)<br \/>\n# define the model<br \/>\nmodel = LinearRegression()<br \/>\n# estimate bias and variance<br \/>\nmse, bias, var = bias_variance_decomp(model, X_train, y_train, X_test, y_test, loss=&#8217;mse&#8217;, num_rounds=200, random_seed=1)<br \/>\n# summarize results<br \/>\nprint(&#8216;MSE: %.3f&#8217; % mse)<br \/>\nprint(&#8216;Bias: %.3f&#8217; % bias)<br \/>\nprint(&#8216;Variance: %.3f&#8217; % var)<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<div class=\"urvanov-syntax-highlighter-nums-content\">\n<p>1<\/p>\n<p>2<\/p>\n<p>3<\/p>\n<p>4<\/p>\n<p>5<\/p>\n<p>6<\/p>\n<p>7<\/p>\n<p>8<\/p>\n<p>9<\/p>\n<p>10<\/p>\n<p>11<\/p>\n<p>12<\/p>\n<p>13<\/p>\n<p>14<\/p>\n<p>15<\/p>\n<p>16<\/p>\n<p>17<\/p>\n<p>18<\/p>\n<p>19<\/p>\n<p>20<\/p>\n<p>21<\/p>\n<\/div>\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\">\n<p><span class=\"crayon-p\"># estimate the bias and variance for a regression model<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-e\">pandas <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">read_csv<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-v\">sklearn<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">model_selection <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">train_test_split<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-v\">sklearn<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">linear_model <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">LinearRegression<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-v\">mlxtend<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">evaluate <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-v\">bias_variance<\/span><span class=\"crayon-sy\">_<\/span>decomp<\/p>\n<p><span class=\"crayon-p\"># load dataset<\/span><\/p>\n<p><span class=\"crayon-v\">url<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8216;https:\/\/raw.githubusercontent.com\/jbrownlee\/Datasets\/master\/housing.csv&#8217;<\/span><\/p>\n<p><span class=\"crayon-v\">dataframe<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">read_csv<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">url<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">header<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-v\">None<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-p\"># separate into inputs and outputs<\/span><\/p>\n<p><span class=\"crayon-v\">data<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">dataframe<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-i\">values<\/span><\/p>\n<p><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">data<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-p\"># split the data<\/span><\/p>\n<p><span class=\"crayon-v\">X_train<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">X_test<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_train<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_test<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">train_test_split<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">X<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">test_size<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">0.33<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">random_state<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-p\"># define the model<\/span><\/p>\n<p><span class=\"crayon-v\">model<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">LinearRegression<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-p\"># estimate bias and variance<\/span><\/p>\n<p><span class=\"crayon-v\">mse<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">bias<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-t\">var<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">bias_variance_decomp<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">model<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">X_train<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_train<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">X_test<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">y_test<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">loss<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8216;mse&#8217;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">num_rounds<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">200<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">random_seed<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-p\"># summarize results<\/span><\/p>\n<p><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8216;MSE: %.3f&#8217;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">%<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">mse<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8216;Bias: %.3f&#8217;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">%<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">bias<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8216;Variance: %.3f&#8217;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">%<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-t\">var<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table>\n<\/div>\n<\/div>\n<p><!-- [Format Time: 0.0008 seconds] --><\/p>\n<p>Running the example reports the estimated error as well as the estimated bias and variance for the model error.<\/p>\n<p>Your specific results may vary given the stochastic nature of the evaluation routine. Try running the example a few times.<\/p>\n<p>In this case, we can see that the model has a high bias and a low variance. This is to be expected given that we are using a linear regression model. We can also see that the sum of the estimated mean and variance equals the estimated error of the model, e.g. 20.726 + 1.761 = 22.487.<\/p>\n<p><!-- Urvanov Syntax Highlighter v2.8.12 --><\/p>\n<div id=\"urvanov-syntax-highlighter-5f3eafcb629c3425041026\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" minimize scroll-mouseover\">\n<p><textarea class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" readonly><br \/>\nMSE: 22.487<br \/>\nBias: 20.726<br \/>\nVariance: 1.761<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\">\n<p>MSE: 22.487<\/p>\n<p>Bias: 20.726<\/p>\n<p>Variance: 1.761<\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table>\n<\/div>\n<\/div>\n<p><!-- [Format Time: 0.0000 seconds] --><\/p>\n<h2>Further Reading<\/h2>\n<p>This section provides more resources on the topic if you are looking to go deeper.<\/p>\n<h3>Tutorials<\/h3>\n<h3>Books<\/h3>\n<h3>Articles<\/h3>\n<h2>Summary<\/h2>\n<p>In this tutorial, you discovered how to calculate the bias and variance for a machine learning model.<\/p>\n<p>Specifically, you learned:<\/p>\n<ul>\n<li>Model error consists of model variance, model bias, and irreducible error.<\/li>\n<li>We seek models with low bias and variance, although typically reducing one results in a rise in the other.<\/li>\n<li>How to decompose mean squared error into model bias and variance terms.<\/li>\n<\/ul>\n<p><strong>Do you have any questions?<\/strong><br \/>Ask your questions in the comments below and I will do my best to answer.<\/p>\n<div class=\"widget_text awac-wrapper\">\n<div class=\"widget_text awac widget custom_html-78\">\n<div class=\"textwidget custom-html-widget\">\n<div>\n<h2>Discover Fast Machine Learning in Python!<\/h2>\n<p><a href=\"\/machine-learning-with-python\/\" rel=\"nofollow\"><img decoding=\"async\" src=\"https:\/\/3qeqpr26caki16dnhd19sv6by6v-wpengine.netdna-ssl.com\/wp-content\/uploads\/2014\/07\/MachineLearningMasteryWithPython-220px.png\" alt=\"Master Machine Learning With Python\" align=\"left\"><\/a><\/p>\n<h4>Develop Your Own Models in Minutes<\/h4>\n<p>&#8230;with just a few lines of scikit-learn code<\/p>\n<p>Learn how in my new Ebook:<br \/><a href=\"\/machine-learning-with-python\/\" rel=\"nofollow\">Machine Learning Mastery With Python<\/a><\/p>\n<p>Covers <strong>self-study tutorials<\/strong> and <strong>end-to-end projects<\/strong> like:<br \/><em>Loading data<\/em>, <em>visualization<\/em>, <em>modeling<\/em>, <em>tuning<\/em>, and much more&#8230;<\/p>\n<h4>Finally Bring Machine Learning To<br \/>Your Own Projects<\/h4>\n<p>Skip the Academics. Just Results.<\/p>\n<p><a href=\"\/machine-learning-with-python\/\" class=\"woo-sc-button  custom\"><span class=\"woo-\">See What&#8217;s Inside<\/span><\/a><\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div><\/div>\n","protected":false},"excerpt":{"rendered":"<p>https:\/\/machinelearningmastery.com\/calculate-the-bias-variance-trade-off\/<\/p>\n","protected":false},"author":0,"featured_media":113,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[],"_links":{"self":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/112"}],"collection":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/comments?post=112"}],"version-history":[{"count":0,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/112\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media\/113"}],"wp:attachment":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media?parent=112"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/categories?post=112"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/tags?post=112"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}