Compacting Deep Neural Networks for Internet of Things: Methods and Applications

【Author】 Zhang, Ke; Ying, Hanbo; Dai, Hong-Ning; Li, Lin; Peng, Yuanyuan; Guo, Keyi; Yu, Hongfang

【Source】IEEE INTERNET OF THINGS JOURNAL

【影响因子】10.238

【Abstract】Deep neural networks (DNNs) have shown great success in completing complex tasks. However, DNNs inevitably bring high computational cost and storage consumption due to the complexity of hierarchical structures, thereby hindering their wide deployment in Internet-of-Things (IoT) devices, which have limited computational capability and storage capacity. Therefore, it is a necessity to investigate the technologies to compact DNNs. Despite tremendous advances in compacting DNNs, few surveys summarize compacting-DNNs technologies, especially for IoT applications. Hence, this article presents a comprehensive study on compacting-DNNs technologies. We categorize compacting-DNNs technologies into three major types: 1) network model compression; 2) knowledge distillation (KD); and 3) modification of network structures. We also elaborate on the diversity of these approaches and make side-by-side comparisons. Moreover, we discuss the applications of compacted DNNs in various IoT applications and outline future directions.

【Keywords】Internet of Things; Biological system modeling; Knowledge engineering; Data models; Computational modeling; Neurons; Convolution; Deep learning (DL); deep neural networks (DNNs); Internet of Things (IoT); model compression

【发表时间】2021 44774

【收录时间】2022-01-02

【文献类型】

【主题类别】

--