How bird’s species made new Professor Serge Belongie world-famous within Computer Vision

“Wouldn’t it be cool if we could photograph any bird flying by and have a computer tell us the species?“

What started as a stray thought during a casual conversation with other computer scientists at the University of California, San Diego, would turn out to become a defining moment in the career of Serge Belongie. Now, 11 years later, several start-ups have emerged, and his name is associated with a range of innovative tools in the fields of computer vision and machine learning.

Given his merits in computer vision, Serge Belongie was an obvious choice for the position as director of Denmark’s new pioneer center for AI research, which will be opening in Copenhagen later this year.

In Copenhagen, Serge Belongie will have a dual title as Director of the Pioneer Center for AI and as Professor at the Department of Computer Science, University of Copenhagen (DIKU). While the center will be located at the campus of University of Copenhagen, activities will involve departments at Aalborg University, Danish Technical University, the IT University, Aarhus University, and, of course, University of Copenhagen. 

A fantastic environment for computer vision

For some years, Serge Belongie has been an Honorary Professor at DTU.

- The idea popped up of doing something across the five Danish universities that all have strong positions in computer vision. I immediately found the idea interesting but wasn’t aware that it would include a role for me, he admits.

- For sure, coming to Europe has been a big move for me. Part of my motivation comes from my private life since my wife is Danish, but actually, I have been to the Scandinavian countries several times also before we met. Already as an undergraduate, I visited labs in Copenhagen, Stockholm and Oslo. I have always found the environment around computer vision here to be fantastic.

Another attractive feature for a computer science researcher is the data management regime in Denmark:

- While strict measures safeguarding personal data are in place, it is still relatively simple to work with anonymized data. If you look at the efforts by Professor Mads Nielsen in computer-assisted analysis of breast cancer imagery, something similar would be very difficult to do in the US. Primarily because the data is not centralized as it is in Denmark.

Domain experts take things to the next level

Returning to the original idea of teaching a computer to identify a bird’s species from a photograph, Serge Belongie and his colleagues were successful. He later co-created an image recognition platform which combines computer and human capabilities.

- As we began developing our first tool for bird species recognition, we soon realized that we needed to talk to experts outside computing. It was an enlightening experience and made us humble. This is a philosophy that I have taken to heart and used ever since. Only by engaging with domain expertise will you be able to take things to the next level.

The same approach will be recommended for the PhD students and other young scientists involved in the Pioneer Center. 

- I want to emphasize that the scope of the center is scientific. We are not a start-up incubator. Still, it is important that we maximize the societal impact of the AI research. To that end, we have identified a number of core areas that we look to address.

Serge continues:

- Examples are health tech, public transportation, and climate change mitigation. Sometimes the best way to ensure societal benefit is through spin-outs, but the primary goal of the projects should always be scientific. We are walking a fine line here. However, that is nothing new for me, Serge says.

Three successful startups

Should a project lead to some kind of entrepreneurship, Serge Belongie may be able to offer advice to that end also.

- Being able to determine a bird’s species is highly useful in biology research, but we soon found the software behind the application to be interesting in many other contexts also. Just to give one example, one of the start-ups that we initiated develops software that assists users in organizing their personal photos, explains Serge Belongie.

This start-up, Anchovi Labs, was later acquired by DropBox, Inc. Serge Belongie has also co-founded Digital Persona – the world’s first supplier of mass-market fingerprint identification devices – later acquired by Crossmatch, Inc; and computer vision company Orpix, Inc.

Following the initial breakthrough in species recognition, Serge Belongie has sustained an interest in that field. Here, he found yet another Danish connection. It just so happens, that GBIF (the Global Biodiversity Information Facility) is headquartered at the Zoological Museum in Copenhagen.

- GBIF is an impressive initiative with participation from multiple governmental institutions. As a computer vision scientist, the many datasets in the facility represent fertile grounds. We should always remember that a large collection of data is not science – but it is a potential for doing science!, says Serge.