Part of Proceedings of Machine Learning and Systems 1 (MLSys 2019)
Paul Whatmough, Chuteng Zhou, Patrick Hansen, Shreyas Venkataramanaiah, Jae-sun Seo, Matthew Mattina
The computational demands of computer vision tasks based on state-of-the-art Convolutional Neural Network (CNN) image classification far exceed the energy budgets of mobile devices. This paper proposes FixyNN, which consists of a fixed-weight feature extractor that generates ubiquitous CNN features, and a conventional programmable CNN accelerator which processes a dataset-specific CNN. Image classification models for FixyNN are trained end-to-end via transfer learning, with the common feature extractor representing the transfered part, and the programmable part being learnt on the target dataset. Experimental results demonstrate FixyNN hardware can achieve very high energy efficiencies up to 26.6 TOPS/W (4.81x better than iso-area programmable accelerator). Over a suite of six datasets we trained models via transfer learning with an accuracy loss of <1% resulting in up to 11.2 TOPS/W -- nearly 2x more efficient than a conventional programmable CNN accelerator of the same area.