{"id":1339,"date":"2021-12-09T15:39:49","date_gmt":"2021-12-09T15:39:49","guid":{"rendered":"https:\/\/salarydistribution.com\/machine-learning\/2021\/12\/09\/automating-machine-learning-with-images\/"},"modified":"2021-12-09T15:39:49","modified_gmt":"2021-12-09T15:39:49","slug":"automating-machine-learning-with-images","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2021\/12\/09\/automating-machine-learning-with-images\/","title":{"rendered":"Automating Machine Learning with Images"},"content":{"rendered":"
\n

Following our previous posts on Image processing in BigML<\/a><\/strong>, the turn has arrived to discuss automation for datasets with images. As BigMLers will already know, BigML offers automation that can be used on the server-side thanks to the WhizzML<\/strong><\/a> language, which has been designed especially for Machine Learning tasks, but it also offers client-side bindings<\/a><\/strong> for many programming languages. In this post, we\u2019ll review the Python bindings<\/a><\/strong> approach.<\/p>\n

\"\"<\/figure>\n

The File-Field Duality of Images<\/h2>\n

From the Machine Learning point of view, each image is a source for many fields, like the light levels in some regions, their color information, or the shapes it contains. In that sense, we need to think of an image as a composed field, just as we could think about other composed fields like text or date-time fields. At the same time, all this information in packed in an image file, and, as you probably know, uploading any file to BigML generates a Source<\/strong> resource. This will also happen in this case for any uploaded image. But then, how do we manage to work with images as any other field? Well, simply by allowing Sources<\/strong> to contain other sources as components. That\u2019s why BigML now offers Composite Sources<\/strong>.<\/p>\n

Creating Composite Sources<\/h2>\n

The simplest way to create a Composite Source<\/strong> from a list of images is to pack them in a compressed file and upload them as you would upload any file to BigML. Of course, if the file is located in your computer, you will need to use some kind of client-side binding to help you automate that. After installing<\/a> the Python bindings and setting your credentials<\/a> as environment variables, the code to upload data to BigML will be:<\/p>\n

from<\/span> bigml.api<\/span> import<\/span> BigML
<\/span>api<\/span> =<\/span> BigML<\/span>()
<\/span>composite_source<\/span> =<\/span> api<\/span>.<\/span>create_source<\/span>(<\/span>\"my_images.zip\"<\/span>)<\/span>\n<\/pre>\n
BigML opens the compressed file and creates one Source<\/strong> per image file plus a Composite Source<\/strong> that contains one image per row. In addition to that, if the images in the compressed file are organized in folders, a new label<\/strong> field is added to each row to store the name of the folder<\/strong> the image was in. Thus, if you upload a compressed file containing two folders: cats<\/em> and dogs<\/em> with the corresponding images,\u00a0 the Composite Source<\/strong> will automatically handle labeling for you.<\/p>\n
In order to apply some Machine Learning to the image information, the next step is deciding what kind of features will we extract from the image that might be relevant to our problem. The Composite Source<\/strong> is the right place to configure the parsing<\/em> that will be used to extract the image information. BigML offers different options for image analysis<\/a>, and they can be applied at the source level to determine the interpretation of your image. Of course, you can choose to let BigML decide for you and go with defaults, but in case you want to change the type of analysis to use, just update your Composite Source<\/strong>.<\/p>\n
from<\/span> bigml.api<\/span> import<\/span> BigML
<\/span>api<\/span> =<\/span> BigML<\/span>()
<\/span>composite_source<\/span> =<\/span> api<\/span>.<\/span>update_source<\/span>(
<\/span>    composite_source<\/span>,<\/span>\n    {<\/span>\"image_analysis\"<\/span>:<\/span> {<\/span>\n     \"enabled\"<\/span>:<\/span> True<\/span>,<\/span>\n     \"extracted_features\"<\/span>:<\/span> [<\/span>\"average_pixels\"<\/span>]}})<\/span>\n<\/pre>\n
Annotating your Images<\/h2>\n
In order to be able to use images as one more field in your Datasets<\/strong>, you need to be able to assign other additional information to each image you upload to BigML. Let\u2019s start with the simple case of image classification.<\/p>\n
In order to train an image classifier, you\u2019ll need to provide the images and the label that you want to associate to each of them. The simplest way to do that is to upload a compressed file that contains, not only the images, but also a CSV<\/strong> or JSON<\/strong> file with the annotations. The annotations file should have at least a field that contains the path to the image file, as found in the compressed file, and another field where the label is stored.\u00a0<\/p>\n
from<\/span> bigml.api<\/span> import<\/span> BigML<\/span>\n# my_annotated_images.zip contains a list of images plus an annotations file<\/span>\n# e.g.<\/span>\n# image1.jpg<\/span>\n# image2.jpg<\/span>\n# annotations.csv<\/span>\n\napi<\/span> =<\/span> BigML<\/span>()<\/span>\ncomposite_source<\/span> =<\/span> api<\/span>.<\/span>create_source<\/span>(<\/span>\"my_annotated_images.zip\"<\/span>)<\/span>\n \n<\/pre>\n
The result will be a table+image<\/em> Composite Source<\/strong>, where each row will contain both the associated label and the Id of the Source<\/strong> created when uploading the corresponding image. This automated link between the contents of your CSV and the image Sources<\/strong> will happen when BigML is able to find a field that contains the names of the image files. Keep in mind, the CSV can contain any number of additional fields too, regardless of their type.<\/p>\n
Editing Annotations<\/h2>\n
Probably you\u2019ve experienced the need to change or add new labels to an existing set of images. There are plenty of reasons for that: you did not finish your labeling yet, or you redefined your labels in order to improve your results. In that case, you should be able to change the contents of the labels associated to the images in your Composite Source<\/strong>. The table+image<\/em> format will not allow you to alter its contents out of the box, but there\u2019s a different way of creating a Composite Source<\/strong> for annotated images that will:<\/p>\n
from<\/span> bigml.api<\/span> import<\/span> BigML
<\/span>api<\/span> =<\/span> BigML<\/span>()
<\/span>composite_source<\/span> =<\/span> api<\/span>.<\/span>create_annotated_source<\/span>(<\/span>\"annotated_images.json\"<\/span>)<\/span><\/pre>\n
The annotated_images.json<\/em> file should point to two files: the zip file, containing exclusively the images, and the annotations file that contains the labels. The structure for this annotations file would be:<\/p>\n
{\"description<\/strong>\": \"Fruit images to test colour distributions\",
\"images_file<\/strong>\": \".\/fruits_hist.zip\",
\"new_fields<\/strong>\": [{\"name\": \"new_label\", \"optype\": \"categorical\"}],
\"source_id<\/strong>\": null,
\"annotations<\/strong>\": \".\/annotations_detail.json\"}<\/pre>\n
You can see that the images_file<\/em> attribute points to the images zip file. The source_id<\/em> attribute can be set to the Composite Source<\/strong> Id once it has been created. If used, images will not be uploaded again. A new_fields<\/em> attribute declares the new_label<\/em> field that will be added to the Source. The annotations\u00a0<\/em>attribute points to the annotations file, which would be similar to this one:<\/p>\n
[{\"file<\/strong>\": \"apples\/fruits1f.png\", \"new_label<\/strong>\": \"Green\"},
{\"file<\/strong>\": \"apples\/fruits1.png\", \"new_label<\/strong>\": \"Red\"},
{\"file<\/strong>\": \"berries\/fruits2e.png\", \"new_label<\/strong>\": \"Blue\"}]<\/pre>\n
By using this method, an editable image<\/em> Composite Source is created, and its labels can be modified until a Dataset is built<\/strong> from it (or you decide to close<\/strong> the Source for editing).<\/p>\n
But why should we ever close<\/em> the Source<\/strong> and avoid any further editing?<\/p>\n
From Immutability to Interpretability<\/h2>\n
On top of a good perfomance, today\u2019s Machine Learning solutions need to provide accountability, interpretability and reproducibility. The fact that resources in BigML are immutable has always ensured those qualities. However, as shown in the previous section, Composite Sources<\/strong> allow some significant editing, like labeling our images. In order to maintain traceability of the Machine Learning models, we need to block that editing the minute we use the contents of the Source<\/strong> to produce other Machine Learning resources from them, that is, when building a Dataset<\/strong>. Once a Dataset is built from it, the Source is automatically closed (its closed<\/code> attribute is set to True<\/code>) for editing, and accepts no more modifications. If you need to modify a closed Source<\/strong>, you\u2019ll need to clone it.<\/p>\n
from<\/span> bigml.api<\/span> import<\/span> BigML<\/span>\napi<\/span> =<\/span> BigML<\/span>()<\/span>\nclosed_source<\/span> =<\/span> \"source\/61786d54520f9017b5000005\"<\/span> # this source is closed<\/span>\nopen_cloned_source<\/span> =<\/span> api<\/span>.<\/span>clone_source<\/span>(<\/span>closed_source<\/span>) <\/span># this one is open<\/span><\/span> <\/pre>\n
The resulting Source\u00a0<\/strong>is an open clone of the original one, ready to be changed again.<\/p>\n
As you\u2019ve seen, adding images to BigML\u2019s models is an easy task that starts by uploading them as Composite Sources<\/strong>. From then on, images can be analyzed not only by Deepnets<\/strong> and their Convolutional Neural Networks but also by all kind of Supervised<\/strong> and Unsupervised<\/strong> models, like Clusters<\/strong> or Anomaly Detectors<\/strong>, thanks to BigML\u2019s automated feature extraction.\u00a0\u00a0<\/p>\n
Want to know more about Image Processing?<\/h2>\n
We still have things to show about the BigML release and how building Image-based Machine Learning models will be a piece of cake from now on. Please\u00a0visit the dedicated release page<\/strong><\/a>\u00a0and\u00a0join the FREE live webinar on Wednesday, December 15\u00a0<\/strong>at 8:30 AM PST \/ 10:30 AM CST \/ 5:30 PM CET to learn more.\u00a0Register today,<\/a><\/strong>\u00a0space is limited!<\/strong><\/p>\n
\n
\n
Like this:<\/h3>\n
Like<\/span><\/span> Loading…<\/span><\/p>\n
<\/span><\/a><\/div>\n
Relacionado<\/em><\/h3>\n<\/div><\/div>\n","protected":false},"excerpt":{"rendered":"
https:\/\/blog.bigml.com\/2021\/12\/09\/automating-machine-learning-with-images\/<\/p>\n","protected":false},"author":0,"featured_media":1340,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[],"_links":{"self":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/1339"}],"collection":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/comments?post=1339"}],"version-history":[{"count":0,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/1339\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media\/1340"}],"wp:attachment":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media?parent=1339"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/categories?post=1339"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/tags?post=1339"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
That configuration will cause every image in your Dataset<\/strong> to be represented by the the average intensity of pixels in different regions (see the API documents<\/a> to learn about the available options and their derived features). Of course, you can apply more than one type of extraction to your images. The extracted_features<\/em> attribute will accept a list of the options that you want to use, and features will be generated when a Dataset<\/strong> is built from your Composite Source<\/strong>.<\/p>\n