Merlin: Instant, intelligent bird photo recognition in the palm of your hand

The Cornell Lab of Ornithology and Visipedia have collaborated to develop Merlin, a machine-learning program that helps to identify various species of birds in a photograph, both at home and in the field.

Merlin, available both online and as a native apps for both iOS and Android, is designed for amateur and intermediate bird photographers, who will inevitably come across the question, ‘What bird is this?’ at some point in their career.

Since Merlin differs in the browser from its mobile app, I’ll break down the two different implementations of the same technology.

Web App

To discover what bird it is you photographed, Merlin’s web app relies on user-inputted data in addition to image-recognition algorithms.

After uploading the image of a bird, you’re asked to input the location and date it was taken. This information is the first step of the process, which helps to narrow down the search using known living areas of certain bird species at certain times of the year.

The next step is to give Merlin ‘hints’ in the form of mapping out the body structure of the bird in the photograph. It asks you first to drag a box around the bird. Once inside the box, you’re asked to mark the location of the bird’s bill, eye and tail.

Once completed, Merlin gets to work. When the recognition is complete, you’re presented with the name of the bird and a number of other photographs from the 70 million sightings shared on eBird.

Mobile App

The mobile app is meant to be used in the field more than in the office. Instead of using image recognition algorithms to determine what bird you’re capturing, you’re asked five simple questions in a step-by-step method:

• Where did you see the bird? (It will automatically detect this for you, or you can manually enter it)
• When did you see the bird? (It will automatically have today’s date chosen)
• What size was the bird?
• What color was the bird?
• What was the bird doing?

When you’ve answered the five questions, Merlin almost instantaneously presents its best guess as to what bird you’re looking at. Also presented are photographs and audio recordings of bird species from the Macaulay Library.

Overview

I’m not a bird photographer myself, but Merlin is one of the most incredible resources I’ve come across. It uses multiple libraries, technologies and archives into a single, cohesive experience that provides impressive results.

To take Merlin for a spin in the browser, head over to its Photo ID page. If you want the field guide app, you can download it for free for both iOS and Android.