Key points are not available for this paper at this time.
The fully-connected layers of deep convolutional neural networks typically contain over 90% of the network parameters. Reducing the number of parameters while preserving predictive performance is critically important for training big models in distributed systems and for deployment in embedded devices. In this paper, we introduce a novel Adaptive Fastfood transform to reparameterize the matrix-vector multiplication of fully connected layers. Reparameterizing a fully connected layer with d inputs and n outputs with the Adaptive Fastfood transform reduces the storage and computational costs costs from O(nd) to O(n) and O(n log d) respectively. Using the Adaptive Fastfood transform in convolutional networks results in what we call a deep fried convnet. These convnets are end-to-end trainable, and enable us to attain substantial reductions in the number of parameters without affecting prediction accuracy on the MNIST and ImageNet datasets.
Building similarity graph...
Analyzing shared references across papers
Loading...
Zichao Yang
Ocean University of China
Marcin Moczulski
University of Michigan
Misha Denil
Google (United States)
University of Oxford
Carnegie Mellon University
Georgia Institute of Technology
Building similarity graph...
Analyzing shared references across papers
Loading...
Yang et al. (Tue,) studied this question.
synapsesocial.com/papers/6a127e90a4bed3c7b16733a7 — DOI: https://doi.org/10.1109/iccv.2015.173