QuadraLib: A Performant Quadratic Neural Network Library for Architecture Optimization and Design Exploration

Part of Proceedings of Machine Learning and Systems 4 pre-proceedings (MLSys 2022)

Paper

Bibtek download is not available in the pre-proceeding


Authors

Zirui Xu, Fuxun Yu, Jinjun Xiong, Xiang Chen

Abstract

The significant success of Deep Neural Networks (DNNs) is highly promoted by the multiple sophisticated DNN libraries. On the contrary, although some work have proved that Quadratic Deep Neuron Networks (QDNNs) show better non-linearity and learning capability than the traditional first-order DNNs, their neuron design suffers certain drawbacks from theoretical performance to practical deployment. In this paper, we first proposed a new QDNN neuron architecture design, and further developed QuadraLib, a QDNN library to provide architecture optimization and design exploration for QDNNs. Extensive experiments show that our design has better performance regarding prediction accuracy and computation consumption on multiple learning tasks.