Convolutional Networks with Adaptive Inference Graphs
Research output: Contribution to journal › Journal article › Research › peer-review
Do convolutional networks really need a fixed feed-forward structure? What if, after identifying the high-level concept of an image, a network could move directly to a layer that can distinguish fine-grained differences? Currently, a network would first need to execute sometimes hundreds of intermediate layers that specialize in unrelated aspects. Ideally, the more a network already knows about an image, the better it should be at deciding which layer to compute next. In this work, we propose convolutional networks with adaptive inference graphs (ConvNet-AIG) that adaptively define their network topology conditioned on the input image. Following a high-level structure similar to residual networks (ResNets), ConvNet-AIG decides for each input image on the fly which layers are needed. In experiments on ImageNet we show that ConvNet-AIG learns distinct inference graphs for different categories. Both ConvNet-AIG with 50 and 101 layers outperform their ResNet counterpart, while using 20 % and 38 % less computations respectively. By grouping parameters into layers for related classes and only executing relevant layers, ConvNet-AIG improves both efficiency and overall classification quality. Lastly, we also study the effect of adaptive inference graphs on the susceptibility towards adversarial examples. We observe that ConvNet-AIG shows a higher robustness than ResNets, complementing other known defense mechanisms.
Original language | English |
---|---|
Journal | International Journal of Computer Vision |
Volume | 128 |
Issue number | 3 |
Pages (from-to) | 730-741 |
Number of pages | 12 |
ISSN | 0920-5691 |
DOIs | |
Publication status | Published - 1 Mar 2020 |
Externally published | Yes |
Bibliographical note
Publisher Copyright:
© 2019, Springer Science+Business Media, LLC, part of Springer Nature.
- Convolutional neural networks, Gumbel-Softmax, Residual networks
Research areas
ID: 301823501