Volume 27

2018

Publication date: 2018

Licence: CC BY-NC-ND 4.0

Editorial team

Editor-in-Chief Stanisław Migórski

Deputy Editor-in-Chief Andrzej Bielecki

Secretary Krzysztof Misztal

Issue content

Deep learning-based initialization for object packing

Maciej Wołczyk

Schedae Informaticae, Volume 27, 2018, pp. 9-17

https://doi.org/10.4467/20838476SI.18.001.10406

One of the most important optimization tasks in the industry at the current time is the object packing problem. Although several methods have been developed for the purpose of solving it, they are usually only able to optimize placement locally and as such are heavily dependent on the choice of the initial setting -- hence the need for trying out multiple possible starting points, which impacts algorithm running time. In this paper we present a neural network-based model which provides sensible starting points in a linear time.

PDF

Traffic Signal Settings Optimization Using Gradient Descent

Marcin Możejko, Maciej Brzeski, Łukasz Mądry, Łukasz Skowronek, Paweł Gora

Schedae Informaticae, Volume 27, 2018, pp. 19-30

https://doi.org/10.4467/20838476SI.18.002.10407

We investigate performance of a gradient descent optimization (GR) applied to the traffic signal setting problem and compare it to genetic algorithms. We used neural networks as metamodels evaluating quality of signal settings and discovered that both optimization methods produce similar results, e.g., in both cases the accuracy of neural networks close to local optima depends on an activation function (e.g., TANH activation makes optimization process converge to different minima than ReLU activation).

PDF

Gradient Regularization Improves Accuracy of Discriminative Models

Dániel Varga, Adrián Csiszárik, Zsolt Zombori

Schedae Informaticae, Volume 27, 2018, pp. 31-45

https://doi.org/10.4467/20838476SI.18.003.10408

Regularizing the gradient norm of the output of a neural network is a powerful technique, rediscovered several times. This paper presents evidence that gradient regularization can consistently improve classification accuracy on vision tasks, using modern deep neural networks, especially when the amount of training data is small. We introduce our regularizers as members of a broader class of Jacobian-based regularizers. We demonstrate empirically on real and synthetic data that the learning process leads to gradients controlled beyond the training points, and results in solutions that generalize well.

PDF

LOSSGRAD: Automatic Learning Rate in Gradient Descent

Bartosz Wójcik, Łukasz Maziarka, Jacek Tabor

Schedae Informaticae, Volume 27, 2018, pp. 47-57

https://doi.org/10.4467/20838476SI.18.004.10409

In this paper, we propose a simple, fast and easy to implement algorithm LOSSGRAD (locally optimal step-size in gradient descent), which automatically modifies the step-size in gradient descent during neural networks training. Given a function f, a point x, and the gradient ▽_xf of f, we aim to find the step-size h which is (locally) optimal, i.e. satisfies:

h = arg min f(x - t▽_xf).
t≥0

Making use of quadratic approximation, we show that the algorithm satisfies the above assumption. We experimentally show that our method is insensitive to the choice of initial learning rate while achieving results comparable to other methods.

PDF

On Some Goodness of Fit Tests for Normality Based on the Optimal Transport Distance

Marcin Mazur, Piotr Kościelniak

Schedae Informaticae, Volume 27, 2018, pp. 59-68

https://doi.org/10.4467/20838476SI.18.005.10410

We apply the optimal transport distance to construct two goodness of fit tests for (univariate) normality. The derived statistics are then compared with those used by the Shapiro-Wilk, the Anderson-Darling and the Cramer-von Mises tests. In particular, we preform Monte Carlo experiments, involving computations of the test power against some selected alternatives and wide range of sample sizes, which show efficiency of the obtained test procedures.

PDF

Sliced Generative Models

Szymon Knop, Marcin Mazur, Jacek Tabor, Igor T. Podolak, Przemysław Spurek

Schedae Informaticae, Volume 27, 2018, pp. 69-79

https://doi.org/10.4467/20838476SI.18.006.10411

In this paper we discuss a class of AutoEncoder based generative models based on one dimensional sliced approach. The idea is based on the reduction of the discrimination between samples to one-dimensional case.

Our experiments show that methods can be divided into two groups. First consists of methods which are a modification of standard normality tests, while the second is based on classical distances between samples.

It turns out that both groups are correct generative models, but the second one gives a slightly faster decrease rate of Frechet Inception Distance (FID).

PDF

Image Inpainting with Gradient Attention

Michał Sadowski, Aleksandra Grzegorczyk

Schedae Informaticae, Volume 27, 2018, pp. 81-91

https://doi.org/10.4467/20838476SI.18.007.10412

We present a novel modification of context encoder loss function, which results in more accurate and plausible inpainting. For this purpose, we introduce gradient attention loss component of loss function, to suppress the common problem of inconsistency in shapes and edges between the inpainted region and its context. To this end, the mean absolute error is computed not only for the input and output images, but also for their derivatives. Therefore, model concentrates on areas with larger gradient, which are crucial for accurate reconstruction. The positive effects on inpainting results are observed both for fully-connected and fully-convolutional models tested on MNIST and CelebA datasets.

PDF

Słowa kluczowe: cutting & packing, optimization, object packing problem, phi-functions, deep learning, traffic optimization, metamodels, activation functions, genetic algorithm, gradient descent, neural network, generalization, gradient regularization, spectral norm, Frobenius norm, gradient descent, optimization methods, adaptive step size, dynamic learning rate, neural networks, Goodness of fit test (for normality), optimal transport distance, Wasserstein distance, autoencoder-based generative model., Generative model, AutoEncoder, Wasserstein distances

Volume 27

2018 Next

Issue content

2018