{"id":1327,"date":"2021-12-08T05:02:15","date_gmt":"2021-12-08T05:02:15","guid":{"rendered":"https:\/\/salarydistribution.com\/machine-learning\/2021\/12\/08\/programming-image-processing\/"},"modified":"2021-12-08T05:02:15","modified_gmt":"2021-12-08T05:02:15","slug":"programming-image-processing","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2021\/12\/08\/programming-image-processing\/","title":{"rendered":"Programming Image Processing"},"content":{"rendered":"
\n

So far, to showcase BigML\u2019s upcoming Image Processing release<\/a><\/strong>, we have demonstrated how composite sources<\/a><\/strong> work, how you can label your images<\/a><\/strong> on the platform, we have covered an example\u00a0use case<\/strong> in Manufacturing<\/strong><\/a>, and how to execute the newly available features using the\u00a0BigML Dashboard<\/a><\/strong>. In contrast, this installment demonstrates how to perform image classification by calling the\u00a0BigML REST API. As mentioned before<\/strong><\/a>, image classification is a supervised learning technique for images to identify various classes of images and has a tremendous amount of applications. Let\u2019s jump in and see how we can put it to use programmatically.<\/p>\n

\"\"<\/figure>\n

Authentication<\/h2>\n

Before using the API, you must set up your environment variables. You should set\u00a0BIGML_USERNAME, BIGML_API_KEY, <\/strong>and BIGML_AUTH<\/strong>\u00a0in your .bash_profile. BIGML_USERNAME is just your username. Your BIGML_API_KEY can be found on the Dashboard by clicking on your username to pull up the account page, and then clicking on \u2018API Key\u2019. Finally, BIGML_AUTH is simply the combination of these elements.<\/p>\n

export BIGML_USERNAME=my_name\nexport BIGML_API_KEY=123456789\nexport BIGML_AUTH=\u201cusername=$BIGML_USERNAME;api_key=$BIGML_API_KEY;\u201c<\/pre>\n
\"\"<\/figure>\n
Upload Your Data<\/h2>\n
For this tutorial, we\u2019re using the same MNIST dataset used in our previous tutorial demonstrating the new BigML Dashboard capabilities. Once again, a zip file containing a large number of images is organized into multiple folders, each one representing a handwritten digit.<\/p>\n
Data sources can be uploaded to BigML in many different ways, so this step should be appropriately adapted to your data with the help of the API documentation<\/strong><\/a>. Here, we will create our data source using a local zip file.<\/p>\n
curl \"https:\/\/bigml.io\/source?$BIGML_AUTH\" -F file=@mnist-traing.zip<\/pre>\n
You could also upload images included in the MNIST dataset to BigML independently and later build a new composite source to group them like this.<\/p>\n
curl \"https:\/\/bigml.io\/source?$BIGML_AUTH\" \n  -X POST \n  -H 'content-type: application\/json'\n  -d '{\"sources\": [\"603de3a8edc1580a58029253\", \"603de3a8edc1580a58029254\", ...]}'<\/pre>\n
Once you have a composite source, you can add, remove or replace the images inside the source by updating the source with arguments: add_sources, remove_sources, or sources. For example, you can use the following command to remove an image from the composite source.<\/p>\n
curl \"https:\/\/bigml.io\/source\/602e12b74e72556f5100000a?$BIGML_AUTH\" \n  -X PUT \n  -H 'content-type: application\/json'\n  -d '{\"remove_sources\": [\"603de3a8edc1580a58029253\"]}'<\/pre>\n
Preparing Your Data<\/h2>\n
In our previous tutorial of the BigML Dashboard, we explained image classification needs labels to build models and showed you how to create them in the Dashboard. Now, we explain how to do the same using the BigML API.<\/p>\n
First off, BigML creates a label field automatically if it\u2019s able to detect a folder structure in the zip file. To create a label field via the API, you can use the following command, which specifies the required info for the new field, that is, a name<\/strong> and the optype<\/strong>.<\/p>\n
curl \"https:\/\/bigml.io\/source\/602e12b74e72556f5100000a?$BIGML_AUTH\" \n  -X PUT \n  -H 'content-type: application\/json'\n  -d '{\n        \"new_fields\": [\n          {\"name\": \"Label\", \"optype\": \"numeric\"}\n        ]\n      }'<\/pre>\n
You can define as many label fields as you need.<\/p>\n
Now, you have to give a value to the label field for the images contained in the composite source with the class that better describes the image.  You can do this by using this command, which specifies the list of components (images in composite) to update, the field id of label field, and the value<\/strong> to assign.<\/p>\n
curl \"https:\/\/bigml.io\/source\/602e12b74e72556f5100000a?$BIGML_AUTH\" \n  -X PUT \n  -H 'content-type: application\/json'\n  -d '{\n        \"row_values\": [\n          {\n            \"components\": [\n              \"603de3a8edc1580a58029253\",\n              \"603de3a8edc1580a58029254\"\n            ],\n            \"field\": \"100002\",\n            \"value\": 6\n          }\n        ]\n      }'<\/pre>\n
Kindly remember that the full list of arguments can be found in our\u00a0API documentation<\/strong><\/a>.<\/p>\n
Creating a Dataset<\/h2>\n
A BigML dataset is a separate resource and is a serialized form of your data. In the Dashboard, it\u2019s displayed with some simple summary statistics and is the resource consumed by Machine Learning algorithms. To create a dataset from your uploaded data via the API, you can use the following command that specifies the source used to generate the dataset.<\/p>\n
curl \"https:\/\/bigml.io\/dataset?$BIGML_AUTH\" \n  -X POST \n  -H 'content-type: application\/json' \n  -d '{\"source\": \"source\/603de3344e72554f7d0000e0\"}'<\/pre>\n
Creating a Deepnet<\/h2>\n
In order to create the Deepnet, you just need your dataset ID.  In this case, BigML will train a particular class of deep neural networks \u2014 Convolutional Neural Networks (CNN).<\/p>\n
curl \"https:\/\/bigml.io\/deepnet?$BIGML_AUTH\" \n  -X POST \n  -H 'content-type: application\/json' \n  -d '{\"dataset\": \"dataset\/602e12b74e72556f5100000a\"}'<\/pre>\n
You guessed it! The full list of Deepnet arguments can also be found in our\u00a0API documentation<\/strong><\/a>.<\/p>\n
Evaluating the Deepnet<\/h2>\n
Once ready, you can evaluate the deepnets predictive performance. You just need to use the deepnet<\/strong> ID and the dataset<\/strong> containing the instances that you want to evaluate against. By default, BigML will provide multiple performance metrics some of which may be more relevant than others for your use case.<\/p>\n
curl \"https:\/\/bigml.io\/evaluation?$BIGML_AUTH\" \n  -X POST \n  -H 'content-type: application\/json' \n  -d '{\"dataset\": \"dataset\/602ca0994e72552267000021\",\n       \"deepnet\": \"deepnet\/603534b44e72550e1e000000\"}'<\/pre>\n
Once you have your Evaluation, you may decide that you want to change some of your Deepnets parameters to improve its performance. No problem. If so, just repeat the previous step with different parameters.<\/p>\n
Classifying Images: Making Predictions<\/h2>\n
When you are satisfied with the outstanding performance of your Deepnet, you can begin to use it to classify new images. <\/p>\n
To classify an image, which is also referred to as making a single prediction, you must have previously uploaded the image to the BigML platform, which implies that you already have a source created for it. You can select an individual image in existing sources, either single images or images included in composite sources. You just need to pass the value of the source<\/strong> id as the value for the label field.<\/p>\n
curl \"https:\/\/bigml.io\/prediction?$BIGML_AUTH\" \n-X POST \n-H 'content-type: application\/json' \n-d '{\n      \"deepnet\": \"deepnet\/603534b44e72550e1e000000\", \n      \"input_data\": {\n        \"000002\": \"source\/602fd3c48411750dac7bbd3a\"\n      }\n    }'<\/pre>\n
You can also classify many images at the same time by using batch predictions. In this case, you need a dataset containing the images you want to classify.<\/p>\n
curl \"https:\/\/bigml.io\/batchprediction?$BIGML_AUTH\" \n-X POST \n-H 'content-type: application\/json' \n-d '{\"deepnet\": \"deepnet\/603534b44e72550e1e000000\", \n     \"dataset\" : \"dataset\/603de3344e72554f7d0000a5\"}'<\/pre>\n
You can also configure the output of the batch prediction, whose full list of arguments can be found in our\u00a0API documentation<\/strong><\/a>. How about that? Once the bath prediction is finished, you can easily download the result as a CSV file, or generate an output dataset whichever suits your needs.<\/p>\n
Want to know more about Image Processing?<\/h2>\n
If you have any questions or you would like to learn more about how Image classification works, please visit the\u00a0release page<\/strong><\/a>. It includes a series of blog posts to gently introduce Image Processing<\/a><\/strong> from scratch. And remember to register for our free live webinar<\/strong><\/a> that will take place on December 15<\/strong> at 8:30 AM PST \/ 10:30 AM CST \/ 5:30 PM CET.<\/p>\n
\n
\n
Like this:<\/h3>\n
Like<\/span><\/span> Loading…<\/span><\/p>\n
<\/span><\/a><\/div>\n
Relacionado<\/em><\/h3>\n<\/div><\/div>\n","protected":false},"excerpt":{"rendered":"
https:\/\/blog.bigml.com\/2021\/12\/07\/programming-image-processing\/<\/p>\n","protected":false},"author":0,"featured_media":1328,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[],"_links":{"self":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/1327"}],"collection":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/comments?post=1327"}],"version-history":[{"count":0,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/1327\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media\/1328"}],"wp:attachment":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media?parent=1327"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/categories?post=1327"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/tags?post=1327"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}