Teaching machines to see: Facebook releases three image identification algorithms to the public

 

 

Facebook is constantly researching new technology and sometimes it release the results of its efforts to the public for coders to work with. Facebook released three of its image identification algorithms recently in a move that they hope will "help rapidly advance the field of machine vision."

Users of the social media network will know that Facebook has already been using image analysis for quite some time as the site can automatically identify and tag people in images using facial recognition algorithms. The newly-released technology is designed in part to help identify what is in an image and when there aren't people, Facebook's networks can skip the facial recognition and save resources.

The released code includes DeepMask+SharpMask, which works to identify where and how many objects occupy an image. DeepMask roughly outlines objects and then SharpMask refines the output created by DeepMask. Basically DeepMask is high-level and then SharpMask is low-level. At the pixel level, as a matter of fact.

After the two masks do their thing, an image's objects have been masked, but neither masking code has the ability to identify objects; that's where MultiPathNet comes in. MultiPathNet, while not perfect, is able to identify objects based on their shapes. You can read more about the process in a very interesting research article written by Facebook Research.

 

Masks created by DeepMask and refined by SharpMask
 

 

While not perfect, MultiPathNet does a much better job identifying objects (masked by DeepMask and SharpMask) than we saw only a few years ago.

The technology isn't foolproof. Images can be immensely complex and there's an essentially infinite amount of variation in a real-world scene. Nonetheless, Facebook's research continues to push machine vision forward and who knows how far the boundaries can be pushed in the coming years. Hopefully with the release of the code (which you can find here and here) the public will be able to speed up the process. Who knows, maybe someday our cameras will come with built-in algorithms that will correctly identify every object in our photos and help us capture better images easier.

(Seen via DIY Photography)