{"id":2067,"date":"2022-04-15T13:40:17","date_gmt":"2022-04-15T13:40:17","guid":{"rendered":"https:\/\/salarydistribution.com\/machine-learning\/2022\/04\/15\/a-night-to-behold-researchers-use-deep-learning-to-bring-color-to-night-vision\/"},"modified":"2022-04-15T13:40:17","modified_gmt":"2022-04-15T13:40:17","slug":"a-night-to-behold-researchers-use-deep-learning-to-bring-color-to-night-vision","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2022\/04\/15\/a-night-to-behold-researchers-use-deep-learning-to-bring-color-to-night-vision\/","title":{"rendered":"A Night to Behold: Researchers Use Deep Learning to Bring Color to Night Vision"},"content":{"rendered":"
\n

Talk about a bright idea. A team of scientists has used GPU-accelerated deep learning to show how color can be brought to night-vision systems.\u00a0<\/span><\/p>\n

In a paper\u00a0<\/span>published this week in the journal PLOS One<\/span><\/a>, a team of researchers at the University of California, Irvine led by Professor Pierre Baldi<\/a> and Dr. Andrew Browne<\/a>, describes how they reconstructed color images of photos of faces using an infrared camera.\u00a0<\/span><\/p>\n

The study is a step toward predicting and reconstructing what humans would see using cameras that collect light using imperceptible near-infrared illumination.\u00a0<\/span><\/p>\n

The study\u2019s authors explain that humans see light in the so-called \u201cvisible spectrum,\u201d or light with wavelengths of between 400 and 700 nanometers.<\/span><\/p>\n

Typical night vision systems rely on cameras that collect infrared light outside this spectrum that we can\u2019t see.\u00a0<\/span><\/p>\n

Information gathered by these cameras is then transposed to a display that shows a monochromatic representation of what the infrared camera detects, the researchers explain.<\/span><\/p>\n

The team at UC Irvine developed an imaging algorithm that relies on deep learning to predict what humans would see using light captured by an infrared camera.<\/span><\/p>\n

\u00a0<\/p>\n

\n

\"\"

Researchers at the University of California, Irvine, aimed to use deep learning to predict visible spectrum images using infrared illumination alone. Source: Browne, et al.\u00a0<\/a><\/figcaption><\/figure>\n

\u00a0<\/span><\/p>\n

In other words, they\u2019re able to digitally render a scene for humans using cameras operating in what, to humans, would be complete \u201cdarkness.\u201d\u00a0<\/span><\/p>\n

To do this, the researchers used a monochromatic camera sensitive to visible and near-infrared light to acquire an image dataset of printed images of faces.\u00a0<\/span><\/p>\n

These images were gathered under multispectral illumination spanning standard visible red, green, blue and infrared wavelengths.\u00a0<\/span><\/p>\n

The researchers then optimized a\u00a0<\/span>convolutional neural network<\/span><\/a>\u00a0with a U-Net-like architecture \u2014 a specialized convolutional neural network first developed for biomedical\u00a0<\/span>image segmentation<\/span><\/a>\u00a0at the Computer Science Department of the\u00a0<\/span>University of Freiburg<\/span><\/a>\u00a0\u2014 to predict visible spectrum images from near-infrared images.<\/span><\/p>\n

\n

\"\"

On the left, visible spectrum ground truth image composed of red, green and blue input images. On the right, predicted reconstructions for UNet-GAN, UNet and linear regression using three infrared input images. Source: Browne, et al.\u00a0<\/a><\/figcaption><\/figure>\n

The system was trained using NVIDIA GPUs and 140 images of human faces for training, 40 for validation and 20 for testing.\u00a0\u00a0<\/span><\/p>\n

The result: the team successfully recreated color portraits of people taken by an infrared camera in darkened rooms. In other words, they created systems that could \u201csee\u201d color images in the dark.\u00a0\u00a0<\/span><\/p>\n

To be sure, these systems aren\u2019t yet ready for general purpose use. These systems would need to be trained to predict the color of different kinds of objects \u2014 such as flowers or faces.<\/span><\/p>\n

Nevertheless, the study could one day lead to night vision systems able to see color, just as we do in daylight, or allow scientists to study biological samples sensitive to visible light.<\/span><\/p>\n

Featured image source: Browne, et al.\u00a0<\/a><\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

https:\/\/blogs.nvidia.com\/blog\/2022\/04\/15\/color-night-vision\/<\/p>\n","protected":false},"author":0,"featured_media":2068,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[],"_links":{"self":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/2067"}],"collection":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/comments?post=2067"}],"version-history":[{"count":0,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/2067\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media\/2068"}],"wp:attachment":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media?parent=2067"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/categories?post=2067"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/tags?post=2067"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}