TY - JOUR
TI - Gradient Regularization Improves Accuracy of Discriminative Models
AU - Varga, Dániel
AU - Csiszárik, Adrián
AU - Zombori, Zsolt
TI - Gradient Regularization Improves Accuracy of Discriminative Models
AB - 
	Regularizing the gradient norm of the output of a neural network is a powerful technique, rediscovered several times. This paper presents evidence that gradient regularization can consistently improve classification accuracy on vision tasks, using modern deep neural networks, especially when the amount of training data is small. We introduce our regularizers as members of a broader class of Jacobian-based regularizers. We demonstrate empirically on real and synthetic data that the learning process leads to gradients controlled beyond the training points, and results in solutions that generalize well.
VL - 2018
IS - Volume 27
PY - 2018
SN - 1732-3916
C1 - 2083-8476
SP - 31
EP - 45
DO - 10.4467/20838476SI.18.003.10408
UR - https://ejournals.eu/en/journal/schedae-informaticae/article/gradient-regularization-improves-accuracy-of-discriminative-models
KW - neural network
KW - generalization
KW - gradient regularization
KW - spectral norm
KW - Frobenius norm