As a step toward mapping out the visual aspects of the fashion world, we introduce the Fashionpedia ontology and fashion segmentation dataset. The Fashionpedia consists of two parts: (1) an ontology built by fashion experts containing 27 main apparel objects, 19 apparel parts, and 92 finegrained attributes and their relationships and (2) a dataset consisting of everyday and celebrity event fashion images annotated with segmentation masks and their associated fine-grained attributes, built upon the backbone of the Fashionpedia ontology structure. The aim of our work is to cultivate research connections between the computer vision and fashion communities through the creation of a high quality dataset and associated open competitions, thereby advancing the state-of-the-art in fine-grained visual recognition for fashion and apparel.