Learning parameters rotation method

Khusaenov Artem Azatovich

Learning parameters rotation method

Khusaenov A. A.

Khusaenov Artem Azatovich

Postgraduate, Lomonosov Moscow State University, Faculty of Mechanics and Mathematics, Chair of Mathematical Theory of Intellectual Systems

Published: 2023, vol. 27, issue 2, pp. 19–47

Download full text (In Russian)

Abstract

A method is proposed for improving the quality of training of convolutional artiﬁcial neural networks (ANN) by dividing the parameters according to their ability to expand the receptive ﬁeld. For example, ResNet50 accuracy increase is achieved by only 4 layers freeze. It is shown that the model generalizing ability increase is achieved by eliminating the excessive contribution of individual signiﬁcant (occlusive) image elements in the feature maps. In favor of these assumptions the results of experiments in the transfer learning task and reasoning about the existence of this problem are presented. The proposed approaches can be useful in training ANNs on small data or distillation of the training set, where the problems of overﬁtting on individual occlusive features are of high importance.

Keywords: convolutional artiﬁcial neural network, neuron receptive ﬁeld, model overﬁtting problems, feature occlusion in convolutional artiﬁcial neural networks.

BibTeX

@article{IS-Khusaenov2023,
  author  = {Khusaenov, Artem Azatovich},
  title   = {{Learning parameters rotation method}},
  journal = {Intelligent Systems. Theory and Applications},
  year    = {2023},
  volume  = {27},
  number  = {2},
  pages   = {19--47},
}

AMSBIB

\Bibitem{IS-Khusaenov2023}
\by A.\,A.~Khusaenov
\paper Learning parameters rotation method
\jour Intelligent Systems. Theory and Applications
\yr 2023
\vol 27
\issue 2
\pages 19--47
\lang In Russian

Published under Creative Commons Attribution 4.0 International (CC BY 4.0)

← Back to issue