TY - GEN
T1 - Local Normalization Based BN Layer Pruning
AU - Liu, Yuan
AU - Jia, Xi
AU - Shen, Linlin
AU - Ming, Zhong
AU - Duan, Jinming
N1 - Publisher Copyright:
© 2019, Springer Nature Switzerland AG.
PY - 2019
Y1 - 2019
N2 - Compression and acceleration of convolutional neural network (CNN) have raised extensive research interest in the past few years. In this paper, we proposed a novel channel-level pruning method based on gamma (scaling parameters) of Batch Normalization layer to compress and accelerate CNN models. Local gamma normalization and selection was proposed to address the over-pruning issue and introduce local information into channel selection. After that, an ablation based beta (shifting parameters) transfer, and knowledge distillation based fine-tuning were further applied to improve the performance of the pruned model. The experimental results on CIFAR-10, CIFAR-100 and LFW datasets suggest that our approach can achieve much more efficient pruning in terms of reduction of parameters and FLOPs, e.g., 8.64 × compression and 3.79 × acceleration of VGG were achieved on CIFAR, with slight accuracy loss.
AB - Compression and acceleration of convolutional neural network (CNN) have raised extensive research interest in the past few years. In this paper, we proposed a novel channel-level pruning method based on gamma (scaling parameters) of Batch Normalization layer to compress and accelerate CNN models. Local gamma normalization and selection was proposed to address the over-pruning issue and introduce local information into channel selection. After that, an ablation based beta (shifting parameters) transfer, and knowledge distillation based fine-tuning were further applied to improve the performance of the pruned model. The experimental results on CIFAR-10, CIFAR-100 and LFW datasets suggest that our approach can achieve much more efficient pruning in terms of reduction of parameters and FLOPs, e.g., 8.64 × compression and 3.79 × acceleration of VGG were achieved on CIFAR, with slight accuracy loss.
KW - Convolutional neural network (CNN)
KW - Knowledge distillation
KW - Model compression and acceleration
KW - Pruning
UR - http://www.scopus.com/inward/record.url?scp=85072858272&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-30484-3_28
DO - 10.1007/978-3-030-30484-3_28
M3 - Conference contribution
AN - SCOPUS:85072858272
SN - 9783030304836
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 334
EP - 346
BT - Artificial Neural Networks and Machine Learning – ICANN 2019
A2 - Tetko, Igor V.
A2 - Karpov, Pavel
A2 - Theis, Fabian
A2 - Kurková, Vera
PB - Springer Verlag
T2 - 28th International Conference on Artificial Neural Networks, ICANN 2019
Y2 - 17 September 2019 through 19 September 2019
ER -