The Year in Image Recognition Image Recognition

March 2017

Snap Inc IPO suggests the future is bright for computer vision-enabled AR

Snap Inc debuts on the New York Stock Exchange, with its shares soaring 10 percent on the first day. Besides its massive and young user base, Snap’s appeal for investors is buoyed by its accessible forays into augmented reality (AR), which is expected to grow into a $220 billion market by 2020. Future versions of Snap’s popular AR Lenses and app will use computer vision to identify landscapes, faces, objects, and more and overlay relevant animations over them in real time.

Feburary 2017

Pinterest Lens Takes Visual Search and Shopping to the Next Level

Pinterest launches the image-recognition-enabled Lens beta, whereby aiming at an object with the in-app camera immediately brings up related pins. Focusing on, say, an Eames chair immediately brings up a link to that chair, along with other mid-century furniture that would go well with it, while putting the lens on a tomato might bring up different recipes from pasta sauce to bisque. Welcome to visual search meets virtual personal assistant meets online shopping.

January 2017

Self-driving AI Cars Take Over CES

Largely credited with developing the GPUs that are used to power neural networks that learn how to recognize things in images, NVIDIA led a keynote at CES that announced a fully self-driving car that it is developing with Audi, which is set to hit the road by 2020. Using computer vision and in-car neural networks, Audi’s Q7 CES prototype was able to teach itself how to drive a course in just four days.

December

GumGum Sports uses computer vision to calculate the true value of sports sponsorships across platforms

In December, GumGum partnered with Catalyst Sports and Media on the launch of GumGum Sports, a computer vision-powered system for automatically detecting brand presence in signage across TV, streaming video, and social media. The technology allows brands to, for the first time, get a comprehensive accounting of exactly how visible their branding presence is (or isn't) across the sporting events they sponsor.

November

Amazon’s computer vision tech can tell the difference between dog breeds

In November, Amazon announced that its computer vision system, called Rekognition, was capable of making instant determinations such as "looks like a face," "appears to be female," "appears to be happy," and could even identify the breed of a dog. The company launched the service through its cloud division, Amazon Web Services.

October

IBM puts computer vision to work fighting cancer

In October, IBM scientists released research showing how computer vision can help to detect and identify melanoma, the deadliest form of skin cancer. Working with the Memorial Sloan-Kettering Cancer Center in New York, the scientists put their computer vision system up against a group of expert dermatologists and found their system was more accurate at figuring out what was actually melanoma and what wasn’t.

September

Intel snatches up a computer-vision chip maker

Intel announced it was buying Movidius, the computer vision chip maker (see the April item, above). In a statement about the acquisition, an Intel exec said “we will look to deploy the technology across our efforts in augmented, virtual and merged reality, drones, robotics, digital security cameras and beyond.”

August

Facebook shares computer-vision code with the world

The Facebook AI Research (FAIR) group of artificial intelligence researchers released open-source code for its computer vision software, including tools called SharpMask and MultiPathNet, which help with object detection in images at the pixel level. The so-called “segmentation” approach to computer vision behind these tools is all about understanding all the various parts of a complex image rather than looking for an overall pattern. By open-sourcing its code and research (much as Google did with its TensorFlor system last year), FAIR hopes to spur computer vision technologies to grow more powerful more quickly.

July

Google buys some computer-vision smarts

In July, Google snatched up Paris-based startup Moodstocks, the creator of computer vision systems with APIs and ready-to-use software-development kits that allow developers to integrate image-scanning/recognition into mobile devices without having to do their own coding. According to Google, the acquisition will help it improve its image search, while in an admirably lofty statement, Moodstocks itself declared that “Our dream has been to give eyes to machines by turning cameras into smart sensors able to make sense of their surroundings.”

June

Your iPhone or iPad can now recognize your friends.

In June, Apple introduced a new Photo app in iOS 10 with powerful computer vision built right in. Capable of performing 11 billion computations on a single photo to “see” into its content, the algorithm behind the new and improved app can recognize faces, so that, say, a friend who appears in multiple random photos over the years can automatically be grouped into one album. Unlike cloud-based image-rec systems (like those used by Facebook), which rely on powerful server-based processing, Apple’s new software does all its computations, sorting and ID-ing within your phone or tablet.

May

Amazon moves to give its delivery drones intelligent sight

In early May, The Verge reported that Amazon had assembled a crack team of computer vision experts focused on one huge task: giving Amazon Prime Air drones smarter “sight.” Drones are equipped with cameras, but making sense of what those cameras are recording in real time requires some serious processing power. The Amazon computer vision team, based in Graz, Austria, is tasked with helping Prime Air drones figure out the difference between trees, buildings, the sky, and all manner of obstacles, from patio furniture to windows.

April

Fathom Neural Compute Stick brings computer vision to any device via USB

Movidius, which specializes in computer vision technology that’s baked right into the chips it makes, introduced its Fathom Neural Compute Stick, a tiny USB-drive-sized computer. The device is designed to add plug-and-play computer vision smarts by using a flavor of the Google-built TensorFlow software for vision processing, and can, for instance, help a drone make better sense of its surroundings. A bit like adding a powerful graphics card to a PC to enable better gaming, the stick turbocharges a device’s existing visual-processing systems.

March

Computer vision helps researchers make sense of the evolution of plants

Researchers debuted a new computer vision-based means of categorizing plant leaves into precise categories, helping to revolutionize the way scientists make sense of fossil identification and the evolution of plant life. "It normally takes a trained person a few hours to describe one leaf according to the standard protocol, which uses about 50 terms," said Peter Wilf, professor of geosciences at Penn State in a statement. “The computer program is thousands of times faster.”

February

New Microsoft app can “see” signs and menus and translate them instantly

Microsoft launched Translator, an iOS app that makes translating a sign or menu (in one of 21 supported languages) as easy as pointing your camera at it. (An Android version of the app was released in April.) The translation occurs in real time — it also works with existing images from your phone’s photo album — by deploying Microsoft’s custom deep learning engine, which relies more heavily on computer vision than competing visual translators (which depend, in part, on crowdsourced translations). The app also translates group conversations in real time, even video chats, relying on computer vision to help with precision translation, too.

January

Google and Lenovo team up on a computer vision smartphone project

At CES in Las Vegas, Lenovo and Google announced that they were working together to develop smartphones with computer vision capabilities, leveraging Google’s hardware/software technology known as Project Tango. The plan: to give phones the ability to build 3D models of your immediate environment complete with precise measurements, so that you could, for instance, figure out if a couch at a furniture store would fit in your living room without ever having to pull out a measuring tape. The first result of the collaboration, the Phab2, would end up being released in June.