Pioneer Centre for AI talk: Semantic Manipulation of Visual Content

This is the first talk in the Pioneer Centre for AI’s Colloquium Talk series. The speaker is Sagie Benaim who is a new postdoc working with Centre director and Professor, Serge Belongie.

Abstract

Methods for visual content manipulation, such as texture and style transfer, are a subject of long-term interest in computer vision, but often lack the ability to manipulate content semantically. In this talk, Sagie will discuss approaches for the semantic manipulation of images, videos, and 3D objects.

In the image domain, Sagie will discuss a method for generating a 'structural analogy' between two images A and B: that is, an image that keeps the appearance and style of B, but has a structural arrangement that corresponds to A. 

Moving the videos, Sagie will discuss a method that learns to automatically predict the 'speediness' of moving objects in videos in a self-supervised manner. This can be used for semantic adaptive video speedup as well as for self-supervised action recognition and video retrieval. 

Lastly, Sagie will discuss a method for the semantic stylization of 3D meshes conforming to a target text prompt. The method can handle low-quality meshes (non-manifold, boundaries, etc.) with arbitrary genus, and does not require parameterization. It can be used to synthesize a myriad of semantic styles over a wide variety of 3D meshes in the wild.