On Certain Limitations of Recursive Representation Model
cytuj
pobierz pliki
RIS BIB ENDNOTEChoose format
RIS BIB ENDNOTEPublication date: 24.03.2017
Schedae Informaticae, 2016, Volume 25, pp. 37 - 47
https://doi.org/10.4467/20838476SI.16.003.6184Authors
There is a strong research eort towards developing models that can achieve state-of-the-art results without sacrificing interpretability and simplicity. One of such is recently proposed Recursive Random Support Vector Machine (R2SVM) model, which is composed of stacked linear models. R2SVM was reported to learn deep representations outperforming many strong classiffiers like Deep Convolutional Neural Network. In this paper we try to analyze it both from theoretical and empirical perspective and show its important limitations.Analysis of similar model Deep Representation Extreme Learning Machine (DrELM) is also included. It is concluded that models in its current form achieves lower accuracy scores than Support Vector Machine with Radial Basis Function kernel.
[1] Bengio Y., Learning deep architectures for AI. Foundations and Trends in Machine Learning, 2009, 2.
[2] Podolak I.T., Roman A., Theoretical foundations and experimental results for a hierarchical classiffer with overlapping clusters. Computational Intelligence, 2013, 29 (2), pp. 357-388.
[3] Czarnecki W.M., Tabor J., Extreme entropy machines: Robust information theoretic classiffcation. arXiv preprint arXiv:1501.05279, 2015.
[4] Huang G.B., Zhu Q.Y., Siew C.K., Extreme learning machine: Theory and applications. Neurocomputing, 2006, 70 (1-3), pp. 489-501.
[5] Tissera M., McDonnell M., Deep extreme learning machines for classiffcation. In: Proceedings of ELM-2014 Volume 1. vol. 3 of Proceedings in Adaptation, Learning and Optimization. Springer International Publishing 2015, pp. 345-354.
[6] Vinyals O., Jia Y., Deng L., Darrell T., Learning with recursive perceptual representations. In: Advances in Neural Information Processing Systems 25. Curran Associates, Inc. 2012, pp. 2825-2833.
[7] Yu W., Zhuang F., He Q., Shi Z., Learning deep representations via extreme learning machines. Neurocomputing, 2015, 149, pp. 308-315.
[8] Wolpert D.H., Stacked generalization. Neural Networks, 1992, 5, pp. 241-259.
[9] Cortes C., Vapnik V., Support-vector networks. Machine Learning, 1995, 20 (3), pp. 273297.
[10] Tang Y., Deep learning using linear support vector machines. In: In ICML, 2013.
[11] Lichman M., UCI machine learning repository, 2013.
[12] Chang C.C., Lin C.J., LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2011, 2, pp. 27:1-27:27.
[13] Pedregosa F., Varoquaux G., Gramfort A., Michel V., Thirion B., Grisel O., Blondel M., Prettenhofer P., Weiss R., Dubourg V., Vanderplas J., Passos A., Cournapeau D., Brucher M., Perrot M., Duchesnay E., Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 2011, 12, pp. 2825-2830.
Information: Schedae Informaticae, 2016, Volume 25, pp. 37 - 47
Article type: Original article
Titles:
Faculty of Mathematics and Computer Science, Jagiellonian University, Krakow, Poland
Jagiellonian University in Kraków
Published at: 24.03.2017
Article status: Open
Licence: None
Percentage share of authors:
Article corrections:
-Publication languages:
EnglishView count: 2089
Number of downloads: 1409