TY - JOUR
TI - Separable Layers Enable Structured Efficient Linear Substitutions
AU - Gray, Gavin
AU - Crowley, Elliot J.
AU - Storkey, Amos
T2 - arXiv:1906.00859 [cs, stat]
AB - In response to the development of recent efficient dense layers, this paper shows that something as simple as replacing linear components in pointwise convolutions with structured linear decompositions also produces substantial gains in the efficiency/accuracy tradeoff. Pointwise convolutions are fully connected layers and are thus prepared for replacement by structured transforms. Networks using such layers are able to learn the same tasks as those using standard convolutions, and provide Pareto-optimal benefits in efficiency/accuracy, both in terms of computation (mult-adds) and parameter count (and hence memory). Code is available at https://github.com/BayesWatch/deficient-efficient.
DA - 2019/06/03/
PY - 2019
DP - arXiv.org
UR - http://arxiv.org/abs/1906.00859
AN - http://arxiv.org/abs/1906.00859
DB - arXiv.org
Y2 - 2019/07/01/09:17:21
KW - Computer Science - Machine Learning
KW - Statistics - Machine Learning
ER -