Accéder directement au contenu Accéder directement à la navigation
Communication dans un congrès

Paying more attention to attention: improving the performance of convolutional neural networks via attention transfer

Abstract : Attention plays a critical role in human visual experience. Furthermore, it has recently been demonstrated that attention can also play an important role in the context of applying artificial neural networks to a variety of tasks from fields such as computer vision and NLP. In this work we show that, by properly defining attention for convolutional neural networks, we can actually use this type of information in order to significantly improve the performance of a student CNN network by forcing it to mimic the attention maps of a powerful teacher network. To that end, we propose several novel methods of transferring attention, showing consistent improvement across a variety of datasets and convolutional neural network architectures. Code and models for our experiments are available at this https URL : https://github.com/szagoruyko/attention-transfer
Type de document :
Communication dans un congrès
Liste complète des métadonnées

https://hal-enpc.archives-ouvertes.fr/hal-01832769
Contributeur : Nikos Komodakis <>
Soumis le : lundi 9 juillet 2018 - 08:34:17
Dernière modification le : mercredi 26 février 2020 - 19:06:07

Lien texte intégral

Identifiants

  • HAL Id : hal-01832769, version 1
  • ARXIV : 1612.03928

Collections

Citation

Nikos Komodakis, Sergey Zagoruyko. Paying more attention to attention: improving the performance of convolutional neural networks via attention transfer. ICLR, Jun 2017, Paris, France. ⟨hal-01832769⟩

Partager

Métriques

Consultations de la notice

301