Birds And Algorithms

Birds and Algorithms

Birds are important indicators of ecosystem health. They play vital roles in pollination, keeping the population of insects under control, and in plant dispersal. The lower Himalayan region, where IIT Mandi is located, is home to several species of birds, including the endangered Himalayan Griffon vulture. This makes IIT Mandi an ideal location to develop and set up automatic systems for the long-term monitoring of bird populations.

The Birds and Algorithms Group, MAS Lab, IIT Mandi
But why study birds?

Birds are vulnerable to human-induced change in their surroundings and several species all over the world, are under the threat of population decline. For ecologists, the use of technology supplements many traditional field studies, which are usually human as well as cost-intensive and are limited in their scalability. Technology can be used to create systems which cut down human effort and can scale up to collect and analyse vast amount of data.

Three faculty members (Dr. Padmanabhan Rajan, Dr. Dileep A. D. and Dr. Arnav Bhavsar), from the Multimedia Analytics and Systems Lab, at IIT Mandi initiated creating algorithms to analyse data about birds.

Birds can be detected by both sight and sound. In many cases, the bird is more heard than seen. This makes acoustic detection a good way to detect their presence. Similarly, visual detection can also help in determining the species, in cases when a bird is not singing, or when there are different types and numbers of birds in close vicinity. Hence, sounds, images and videos are useful in detecting and identifying bird species. The information provided by these modalities can be combined to provide more reliable mechanisms to detect and identify birds. At a coarser level, this is inherently similar to how human birdwatchers look and listen for birds.

In the recent, Bird Activity Detection (BAD) challenge (conducted by the Machine Listening Lab of Queen Mary University, London), the submission from the Multimedia Analytics and Systems (MAS) lab at IIT Mandi regarding this interesting study won the Judge's award.

So, what actually happens in such a study?

Sounds are captured using microphones, and images and videos using cameras. Then by using machine learning algorithms, specific tasks such as the automatic identification of birds etc. are performed. Machine learning algorithms try to mimic how humans learn, and are an active research field. For example, for the task of identifying bird species, a small number of sounds, images or videos from several species are provided to the algorithm. Using techniques from signal processing, computer vision, statistics and optimization, the algorithm “learns” how to distinguish one species from another. Then, given a new sound, image or video, the algorithm is able to predict which species is present.

Of course, the problem is much more complex than what is described above. The microphone records not just bird sounds, but all sounds near it. This includes humans talking, passing vehicles, the wind, other animals etc. Similarly, the camera captures birds only when they come into the field of view. Sometimes when captured through the camera, birds could be occluded by leaves, may be out of focus or be only partly visible. Thus, even detecting whether a bird is present, partly present, or not present is a challenge in itself. Varying degrees of noise in the signals, including background noise for sounds, varying degrees of pose, illumination for images and videos increase the challenges.

The team involved in this exciting research includes a Ph.D. student (Anshul Thakur), three B.Tech. students, and two interns. They were guided by three assistant professors in the School of Computing and Electrical Engineering. Other collaborators include researchers in engineering as well as ecologists from other institutions.

IIT Mandi’s framework for sounds

Bird detection: One exciting contribution from IIT Mandi is the development of a bird activity detector (BAD) system. The BAD system is used to determine if a bird sound is present or absent in an audio recording. IIT Mandi’s BAD framework uses signal processing and powerful discriminative classifiers to develop a computationally efficient system.

Species identification: The IIT Mandi team has also developed a machine learning framework to identify bird species from their sounds. The system was able to achieve more than 95% accuracy on 26 species of birds from the lower Himalayan regions.

Detection bird sound in audio recordings. In the top figure, the blue signal indicates a bird call waveform. The red lines detect the call regions and ignore the empty regions. The bottom figure represents the spectral signature of the top figure.

IIT Mandi’s species identification framework for images

For the visual identification, the IIT Mandi team is developing algorithms for:
Bird detection in images: This essentially involves the process of localizing regions which contains the bird, and mask out the background region, which may not be useful for the process of bird identification.
Bird identification from images: The bird regions detected above is provided to an identification algorithm, which focuses on discrimination based on local traits of the birds (e.g. beak, wings, tail etc.). For both the above tasks, image processing and machine learning algorithms such as support vector machines, deep learning are employed.

Bird detection from images. The red bounding boxes show the bird detected against varied backgrounds.


The project is funded by a grant from IIT Mandi for Rs 30 lakh. Further funding, (totally worth about 60 lakhs) was provided by DST-SERB.

Future Work

One of the future directions of this research is to develop low-cost recording devices, and a website for sharing and analysing bird sounds. This is in collaboration with C-DAC Bangalore, and IISER Tirupati.