Putting Merlin’s New Bird Photo ID Tool to the Test

Last week the team from the Cornell Lab of Ornithology rolled out a new tool on its popular Merlin app that identifies the birds in your photos for you.

I know what you’re thinking. It’s a Christmas miracle! This is the greatest gift a lazy birder could ever get!

Well, I'm happy to report that the app is indeed very easy to use, requiring negative 30 calories from beginning to end. Just snap a smartphone picture of a species, or upload a photo you've already taken, and note where and when you saw the bird. Then, using “statistical pattern recognition” (basically facial recognition, but not just for faces) and eBird data, the software calculates some probabilities and tries to figure out what bird you’ve got. It covers 650 North American species—the same as the field-guide part of the app. And it works offline, which means you don't have to waste any data on dumb sparrow IDs.

Sounds fancy, but how well does it work? And is it a practical tool for people who are photographing birds on their phones, or is it really for people who are taking pro pics and want to suss out the identifications later?

With so many questions to answer, I had to test it for myself. Here’s what I found.

I wanted to start simple, so I found some photos I had on my SLR camera of clearly visible, easily identifiable birds. If the app can’t get these right, then we’re in trouble.

First up was a White-eyed Vireo that I photographed in Maryland a few years ago, with its big ol’ white eye looking straight into the lens.

I took a picture of the camera screen with my smartphone, selected the image from my gallery, made sure the bird was centered, and . . . boom! White-eyed Vireo! Great start, Merlin!

Next I found a photo of a Band-tailed Pigeon I grabbed a few months ago in California. This is a less-common species, so this was a tougher test. Still, it was no sweat for Merlin. Impressive stuff so far.

Time to turn up the heat. I pulled up a photo I had of a juvenile Pomarine Jaeger from a couple years ago around North Carolina. It’s an unusual bird, in unusual plumage. Let’s see how it goes.

Okay, so that might not have been totally fair, but at least we found a limit to what the machine has learned so far. The beauty of the software is that it trains itself by making mistakes; the more junk we feed it, the better it becomes at underplaying bad answers (the way the system is built, it never completely eliminates an option). Adding examples of terrible photos and birds that aren't even covered by the app builds out its memory, allowing it to sidestep red herrings and fine-tune its “definition” of each species. By trolling the system, you're really doing it a service.  

Now, back to the birds. What about difficult ID challenges among common species? This seems like the most practical use for the app. I gave it a shot with a Downy Woodpecker and a juvenile Yellow-crowned Night-Heron.

For these species, the computer narrowed it down to a couple choices, but it didn’t know exactly what the answer was. That’s a good start, but I imagine the majority of birders can get to this point on their own.

The next test takes place where a lot of the birds are: outdoors. I wondered how well the app would do with grainy and distant shots taken with my garbage iPhone camera. When it comes to birds, my neighborhood isn’t really hopping, but I did manage a pretty good shot of a female House Sparrow and a set of Mourning Doves on a wire.

Mixed results. The House Sparrow info is useful, and could guide people to the final conclusion. The Mourning Doves were understandable, but disappointing: This species is readily identifiable by most birders. The system has some work to do when it comes to recognizing silhouettes.

Time for more abstract thinking. The app works on photographs, but does it work on art? (Does it have a soul?) I took pictures of the Northern Shrike page in my Sibley Guide to Birds and of an American Goldfinch holiday card I had lying around, and gave it a shot. The system recognized both, which makes sense, because this kind of bird art is supposed to look as realistic as possible. Still, impressive.

But what about really bad art? For example, this Scarlet Tanager I doodled?

Merlin Photo ID was more confident in identifying this image than any of my previous ones. I don’t know if this result says more about Merlin or my art skills. I’m not going to question it.

My takeaway: Merlin Photo ID is helpful in identifying birds—or at least narrowing down possibilities—as long as you have a clear, well-lit, not-too-busy, decently close-up photo to give it. It isn’t a prodigy just yet, but it might just grow up to be a genius. Download the Merlin app (for free!) and enjoy.