Talk: Supervised Learning for video summarization

Speaker: Fei Sha (University of Southern California)

Today there is far more video being captured — by consumers, scientists, defense analysts, and others — than can ever be watched. With this explosion of video data comes a pressing need to develop automatic video summarization algorithms. Video summarization takes a long video as input and produces a short video as output, while preserving its information content as much as possible. As such, summarization techniques have great potential to make large video collections substantially more efficient to browse, search, disseminate, and facilitate communication. Such increased efficiency will play a vital role in many important application areas. For example, with reliable summarization systems, a biologist gathering long videos of her animal subjects could quickly browse a week’s worth of their activity before deciding where to inspect the data most closely. A young student searching YouTube to learn about Yellowstone National Park could see at a glance what content exists, much better than today’s simple thumbnail images can depict. An intelligence agent could rapidly sift through reams of aerial video, reducing the resources required to analyze surveillance data to identify suspicious activities.

In this talk, I will describe our recent efforts in developing supervised  learning techniques for video summarization. Concretely, I will discuss how to formulate video summarization as a subset selection problem and to describe the subset selection with a probabilistic model known as determinantal point processes (DPPs). I will demonstrate the empirical success of this approach and its superiority over many other competing methods.


Dr. Fei Sha is the Jack Munushian Early Career Chair and an associate professor at the University of Southern California, Dept. of Computer Science. His primary research interests are machine learning and its application to speech and language processing, computer vision, and robotics. He had won outstanding student paper awards at NIPS 2006 and ICML 2004. He was selected as a Sloan Research Fellow in 2013, won an Army Research Office Young Investigator Award in 2012, and was a member of DARPA 2010 Computer Science Study Panel. He has a Ph.D (2007) from Computer and Information Science from U. of Pennsylvania and B.Sc and M.Sc from Southeast University (Nanjing, China)