Capturing Hummingbirds with Deep Learning

Hummingbirds are neat to look at, so it’s not surprising that hummingbird feeders are popular garden additions. If you have the time to sit around and wait, you can watch them hover pleasingly around the feeder. Fun! But, if the idea of waiting for long periods of time just to catch a glimpse of one doesn’t appeal to you, there is another solution. Chris Lam of Hackernoon demonstrates how to use deep learning to capture video of hummingbirds feeding.

The first step is to attract the hummingbirds, which is easy enough to do with the feeders you can find at your local home and garden store. Unlike most other birds, hummingbirds are happy to feed on sugar water, which has the energy density to keep them going. Next, a GoPro — or any other action camera — is attached to the feeder with a 3D-printed mount. That’ll happily capture hours of video, but almost all of that video is going to be devoid of hummingbird activity and boring to watch.

That’s why Lam turned to deep learning to extract just the good stuff. He started with a pre-trained RESNET that analyzed the entirety of each frame. That didn’t work well, because there was too much going on and the classifier was overwhelmed. To get around that, Lam set up a Python script to first crop each frame to just the areas around the “flowers” of the feeder where the hummingbirds would likely be. With that system, hummingbirds are classified with an accuracy of about 97%. Now, Lam can enjoy videos of the hummingbirds without having to sit around waiting.