TY - GEN
T1 - Clustering-Based Adaptive Dropout for CNN-Based Classification
AU - Wen, Zhiwei
AU - Ke, Zhiwei
AU - Xie, Weicheng
AU - Shen, Linlin
N1 - Publisher Copyright:
© Springer Nature Switzerland AG 2020.
PY - 2020
Y1 - 2020
N2 - Dropout has been widely used to improve the generalization ability of a deep network, while current dropout variants rarely adapt the dropout probabilities of the network hidden units or weights dynamically to their contributions on the network optimization. In this work, a clustering-based dropout based on the network characteristics of features, weights or their derivatives is proposed, where the dropout probabilities for these characteristics are updated self-adaptively according to the corresponding clustering group to differentiate their contributions. Experimental results on the databases of Fashion-MNIST and CIFAR10 and expression databases of FER2013 and CK+ show that the proposed clustering-based dropout achieves better accuracy than the original dropout and various dropout variants, and the most competitive performances compared with state-of-the-art algorithms.
AB - Dropout has been widely used to improve the generalization ability of a deep network, while current dropout variants rarely adapt the dropout probabilities of the network hidden units or weights dynamically to their contributions on the network optimization. In this work, a clustering-based dropout based on the network characteristics of features, weights or their derivatives is proposed, where the dropout probabilities for these characteristics are updated self-adaptively according to the corresponding clustering group to differentiate their contributions. Experimental results on the databases of Fashion-MNIST and CIFAR10 and expression databases of FER2013 and CK+ show that the proposed clustering-based dropout achieves better accuracy than the original dropout and various dropout variants, and the most competitive performances compared with state-of-the-art algorithms.
KW - Facial expression recognition
KW - Feature and weight clustering
KW - Feature derivative dropout
KW - Self-adaptive dropout probability
UR - http://www.scopus.com/inward/record.url?scp=85081591198&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-41404-7_4
DO - 10.1007/978-3-030-41404-7_4
M3 - Conference contribution
AN - SCOPUS:85081591198
SN - 9783030414030
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 46
EP - 58
BT - Pattern Recognition - 5th Asian Conference, ACPR 2019, Revised Selected Papers
A2 - Palaiahnakote, Shivakumara
A2 - Sanniti di Baja, Gabriella
A2 - Wang, Liang
A2 - Yan, Wei Qi
PB - Springer
T2 - 5th Asian Conference on Pattern Recognition, ACPR 2019
Y2 - 26 November 2019 through 29 November 2019
ER -