Zaawansowane procedury NLP jako przesłanka rekonstrukcji idei wiedzy

Rafał  Maciąg

Zaawansowane procedury NLP jako przesłanka rekonstrukcji idei wiedzy

Publication date: 30.05.2022

Culture Management, 2022, Volume 23, Issue 1, pp. 37 - 53

https://doi.org/10.4467/20843976ZK.22.003.15869

Authors

Pobierz pełny tekst

PDF

Titles

Zaawansowane procedury NLP jako przesłanka rekonstrukcji idei wiedzy

Abstract

Advanced NLP Procedures as Premises for the Reconstruction of the Idea of Knowledge

The article presents the current state of development of the Natural Language Processing (NLP) technology, in particular the GPT-3 language model, and presents its consequences for understanding the phenomenon of knowledge. The NLP technology has been experiencing remarkable development recently. The GPT-3 language model presents a level of advancement that allows it to generate texts as answers to general questions, as summaries of the presented text, etc., which reach the level surpassing the analogous level of human texts. These algorithmic operations lead to the determination of the probability distribution of its components. Texts generated by such a model should be considered as autonomous texts, using immanent, implicit knowledge embedded in language. This conclusion raises questions about the status of such knowledge. Help in the analysis is provided also by the theory of discourse, as well as the theory of discursive space based on it, that proposes the interpretation of knowledge as a trajectory of discourses in a dynamical space. Recognizing that knowledge may also be autonomous, and in particular not be at the exclusive disposal of humans, leads to the question of the status of artificial cognitive agents, such as the GPT-3 language model.

Źródła finansowania publikacji: Artykuł powstał w wyniku realizacji projektu badawczego o numerze

2018/29/B/HS1/01882, finansowanego ze środków Narodowego Centrum Nauki.

Źródła finansowania publikacji: Artykuł powstał w wyniku realizacji projektu badawczego o numerze

2018/29/B/HS1/01882, finansowanego ze środków Narodowego Centrum Nauki.

Keywords

knowledge, Natural Language Processing, GPT-3, discourse, hermeneutics

References

Download references

Aggarwal Charu C. (2018). Machine Learning for Text. Cham, Switzerland: Springer International Publishing.

Angermuller Johannes, Maingueneau Dominique, Wodak Ruth (2014). An Introduction. W: Johannes Angermuller, Dominique Maingueneau, Ruth Wodak (red.), The Discourse Studies Reader: Main Currents in Theory and Analysis. Amsterdam–Philadelphia: John

Benjamins Publishing Company, 1–14. Bender Emily M., Koller Alexander (2020). Climbing Towards NLU: On Meaning, Form, and Understanding in the Age of Data. W: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 5185–5198, https://doi.org/10.18653/v1/2020.acl-main.463 [odczyt: 21.06.2022]. Bleicher Joseph (1980). Contemporary Hermeneutics: Hermeneutics as Method, Philosophy and Critique (reprint, 1982 edition). London–Boston: Routledge & Kegan Paul.

Brown Tom B., Mann Benjamin, Ryder Nick, Subbiah Melanie, Kaplan Jared, Dhariwal Prafulla, Neelakantan Arvind, Shyam Pranav, Sastry Girish, Askell Amanda, Agarwal Sandhini, Herbert-Voss Ariel, Krueger Gretchen, Henighan Tom, Child Rewon, Ramesh Aditya, Ziegler Daniel M., Wu Jeffrey, Winter Clemens, Hesse Christopher, Chen Mark, Sigler Eric, Litwin Mateusz, Gray Scott, Chess Benjamin, Clark Jack, Berner Christopher, McCandlish Sam, Radford Alec, Sutskever Ilya, Amodei Dario (2020). Language Models are Few-Shot Learners, ArXiv:2005.14165 [Cs], https://arxiv.org/abs/2005.14165 [odczyt: 21.06.2022].

Charniak Eugene (2019). Introduction to Deep Learning. Cambridge, Massachusetts: The MIT Press.

Common Crawl Foundation (2021). Common Crawl, https://commoncrawl.org/ [odczyt: 21.06.2022].

Crane Lesley (2016). Knowledge and Discourse Matters: Relocating Knowledge Management’s Sphere of Interest onto Language. Hoboken, New Jersey: John Wiley & Sons, Inc.

Deng Li, Liu Yang (red.) (2018). Deep Learning in Natural Language Processing. Singapore: Springer Singapore, https://doi.org/10.1007/978-981-10-5209-5 [odczyt: 21.06.2022].

Descartes Rene (1637). Discours de la methode pour bien conduire sa raison, & chercher la verité dans les sciences. Plus La dioptrique. Les meteores. Et La geometrie. Qui sont des essais de cete methode. Leyde: De l’imprimerie de Ian Maire.

Descartes Rene (2013). Rozprawa o metodzie. Właściwego kierowania rozumem i poszukiwania prawdy w naukach, tłum. Tadeusz Boy-Żeleński. Kęty: Wydawnictwo ANTYK.

Dijk Teun Adrianus van (2013). Discourse and Knowledge. W: James Paul Gee, Michael Handford (red.), The Routledge Handbook of Discourse Analysis. London: Routledge, 587–603.

Fairclough Norman (2003). Analysing Discourse: Textual Analysis for Social Research. London: Routledge.

Flasiński Mariusz (2016). Introduction to Artificial Intelligence. Cham, Switzerland: Springer.

Fortune Business Insights (2021). Natural Language Processing (NLP) Market Size & Growth, 2028, https://www.fortunebusinessinsights.com/industry-reports/natural-language-processing-nlp-market-101933 [odczyt: 21.06.2022].

Foucault Michel (1966). Les mots et les choses: Une archéologie des sciences humaines. Paris: Gallimard.

Foucault Michel (1969). L’archéologie du savoir. Paris: Gallimard.

Foucault Michel (1971). L’ordre du discours: Leçon inaugurale au College de France prononcée le 2 décembre 1970. Paris: Gallimard.

Foucault Michel (1981). The Order of Discourse. W: Robert E. Young (red.), Untying the Text: A Post-Structuralist Reader. Boston: Routledge & Kegan Paul, 51–78.

Goodfellow Ian, Bengio Yoshua, Courville Aaron (2016). Deep Learning. Cambridge, Massachusetts: MIT Press, https://www.deeplearningbook.org [odczyt: 21.06.2022].

Helwe Chadi, Clavel Chloé, Suchanek Fabian M. (2021). Reasoning with Transformer-based Models: Deep Learning, but Shallow Reasoning, 3rd Conference on Automated Knowledge Base Construction, June 22, https://openreview.net/forum?id=Ozp1WrgtF5_ [odczyt: 21.06.2022].

Hochreiter Sepp, Schmidhuber Jürgen (1997). Long Short-Term Memory. Neural Computation, 9(8), 1735–1780, https://doi.org/10.1162/neco.1997.9.8.1735 [odczyt: 21.06.2022].

Hyland Ken, Paltridge Brian (red.) (2011). Bloomsbury Companion to Discourse Analysis. London–New York: Bloomsbury.

Jorgensen Marianne, Phillips Louise (2002). Discourse Analysis as Theory and Method. London–Thousand Oaks, California: Sage Publications.

Jurafsky Dan, Martin James H. (2020). Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition, 3rd ed. Upper Saddle River, New Jersey: Prentice Hall.

Kozlowski Austin C., Taddy Matt, Evans James A. (2019). The Geometry of Culture: Analyzing Meaning through Word Embeddings. American Sociological Review, 84(5), 905–949, https://doi.org/10.1177/0003122419877135 [odczyt: 21.06.2022].

Lévy Pierre (1999). Collective Intelligence, Mankind’s Emerging World in Cyberspace. Cambridge, Massachusetts: Perseus Books.

Lyotard Jean-François (1979). La condition postmoderne: Rapport sur le savoir. Paris: Éd. de Minuit. Maciąg Rafał (2018). Discursive Space and Its Consequences for Understanding Knowledge and Information. Philosophies, 3(4), https://doi.org/10.3390/philosophies3040034 [odczyt: 21.06.2022].

Maciąg Rafał (2020). Ontological Basis of Knowledge in the Theory of Discursive Space and Its Consequences. Proceedings, 47(1), 11, https://doi.org/10.3390/proceedings47010011 [odczyt: 21.06.2022].

Maciąg Rafał (2022). Wiedza jako opowieść. Przestrzeń dyskursywna. Kraków: TAiWPN Universitas.

Malpas Jeff, Gander Hans-Helmut (red.) (2015). The Routledge Companion to Hermeneutics. Abingdon, Oxon: Routledge.

Merrill William, Goldberg Yoav, Schwartz Roy, Smith Noah A. (2021). Provable Limitations of Acquiring Meaning from Ungrounded Form: What Will Future Language Models Understand? Transactions of the Association for Computational Linguistics, 9, 1047–1060, https://doi.org/10.1162/tacl_a_00412 [odczyt: 21.06.2022].

Mikolov Tomáš, Sutskever Ilya, Chen Kai, Corrado Greg, Dean Jeffrey (2013). Distributed Representations of Words and Phrases and their Compositionality, ArXiv:1310.4546 [Cs, Stat], https://arxiv.org/abs/1310.4546 [odczyt: 21.06.2022].

Neelakantan Arvind, Xu Tao, Puri Raul, Radford Alec, Han Jesse Michael, Tworek Jerry, Yuan Qiming, Tezak Nikolas, Kim Jong Wook, Hallacy Chris, Heidecke Johannes, Shyam Pranav, Power Boris, Nekoul Tyna Eloundou, Sastry Girish, Krueger Gretchen, Schnurr David, Such Felipe Petroski, Hsu Kenny, Thompson Madeleine, Khan Tabarak, Sherbakov Toki, Jang Joanne, Welinder Peter, Weng Lilian (2022). Text and Code Embeddings by Contrastive Pre-Training, ArXiv:2201.10005 [Cs], https://arxiv.org/abs/2201.10005 [odczyt: 21.06.2022].

Ouyang Long, Wu Jeff, Jiang Xu, Almeida Diogo, Wainwright Carroll L., Mishkin Pamela, Zhang Chong, Agarwal Sandhini, Slama Katarina, Ray Alex, Schulman John, Hilton Jacob, Kelton Fraser, Miller Luke, Simens Maddie, Askell Amanda, Welinder Peter, Christiano Paul, Leike Jan, Lowe Ryan (2022). Training Language Models to Follow Instructions with Human Feedback, https://arxiv.org/abs/2203.02155 [odczyt: 21.06.2022].

Paltridge Brian (2006). Discourse Analysis: An Introduction. London–New York: Continuum. Petroni Fabio, Rocktäschel Tim, Lewis Patrick, Bakhtin Anton, Wu Yuxiang, Miller Alexander H., Riedel Sebastian (2019). Language Models as Knowledge Bases? W: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2463–2473, https://doi.org/10.18653/v1/D19-1250 [odczyt: 21.06.2022].

Porada Ian, Sordoni Alessandro, Cheung Jackie C.K. (2021). Does Pre-training Induce Systematic Inference? How Masked Language Models Acquire Commonsense Knowledge, ArXiv:2112.08583 [Cs], https://arxiv.org/abs/2112.08583 [odczyt: 21.06.2022].

Porter Stanley E., Robinson Jason C. (2011). Hermeneutics: An Introduction to Interpretative Theory. Grand Rapids, Michigan–Cambridge, U.K.: William B. Eerdmans Publishing Company.

Qiu Wenjun, Xu Yang (2022). HistBERT: A Pre-trained Language Model for Diachronic Lexical Semantic Analysis, ArXiv:2202.03612 [Cs], https://doi.org/10.13140/RG.2.2.14905.44649 [odczyt: 21.06.2022].

Radford Alec, Wu Jeffrey, Child Rewon, Luan David, Amodei Dario, Sutskever Ilya (2019). Language Models are Unsupervised Multitask Learners, https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf [odczyt: 21.06.2022].

Rajpurkar Pranav, Jia Robin, Liang Percy (2018). Know What You Don’t Know: Unanswerable Questions for SQuAD, ArXiv:1806.03822 [Cs], https://arxiv.org/abs/1806.03822 [odczyt: 21.06.2022].

Rajpurkar Pranav, Zhang Jian, Lopyrev Konstantin, Liang Percy (2016). SQuAD: 100, 000+ Questions for Machine Comprehension of Text, CoRR, ArXiv:1606.05250, https://arxiv.org/abs/1606.05250 [odczyt: 21.06.2022].

Safavi Tara, Koutra Danai (2021). Relational World Knowledge Representation in Contextual Language Models: A Review, ArXiv:2104.05837 [Cs], https://arxiv.org/abs/2104.05837 [odczyt: 21.06.2022].

Sowa John F. (2010). The Role of Logic and Ontology in Language and Reasoning. W: Roberto Poli, Johanna Seibt (red.), Theory and Applications of Ontology: Philosophical Perspectives.Dordrecht: Springer Netherlands, 231–263, https://doi.org/10.1007/978-90-481-8845-1_11 [odczyt: 21.06.2022].

Tamkin Alex, Brundage Miles, Clark Jack, Ganguli Deep (2021). Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models, ArXiv:2102.02503, https://arxiv.org/abs/2102.02503v1 [odczyt: 21.06.2022].

Traylor Aaron, Feiman Roman, Pavlick Ellie (2021). AND does not Mean OR: Using Formal Languages to Study Language Models’ Representations. W: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. Vol. 2: Short papers, 158–167, https://doi.org/10.18653/v1/2021.acl-short.21 [odczyt: 21.06.2022].

Turing Alan M. (1950). Computing Machinery and Intelligence. Mind, 236(14), 433–460.

Vaswani Ashish, Shazeer Noam, Parmar Niki, Uszkoreit Jakob, Jones Llion, Gomez Aidan N., Kaiser Lukasz, Polosukhin Illia (2017). Attention Is All You Need, ArXiv:1706.03762 [Cs], https://arxiv.org/abs/1706.03762 [odczyt: 21.06.2022].

Wang Alex, Singh Amanpreet, Michael Julian, Hill Felix, Levy Omer, Bowman Samuel R. (2018). GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding, ArXiv:1804.07461, https://arxiv.org/abs/1804.07461 [odczyt: 21.06.2022].

Wittgenstein Ludwig (1958). Philosophical Investigations (second edition). Oxford: Basil Blackwell. Yoshida Ryo, Oseki Yohei (2022). Learning Argument Structures with Recurrent Neural Network Grammars. W: Proceedings of the Society for Computation in Linguistics, 5(1), 101–111, https://doi.org/10.7275/kne0-hc86 [odczyt: 21.06.2022].

Zhang Daniel, Maslej Nestor, Brynjolfsson Erik, Etchemendy John, Lyons Terah, Manyika James, Ngo Helen, Niebles Juan C., Sellitto Michael, Sakhaee Ellie, Shoham Yoav, Clark Jack, Perrault Raymond (2021). The AI Index 2021 Annual Report, Stanford, California: AI Index Steering Committee, Human-Centered AI Institute, Stanford University.

Zhou Xuhui, Zhang Yue, Cui Leyang, Huang Dandan (2020). Evaluating Commonsense in Pre-Trained Language Models. W: Proceedings of the AAAI Conference on Artificial Intelligence, 34(05), 9733–9740, https://doi.org/10.1609/aaai.v34i05.6523 [odczyt: 21.06.2022].

Information

Information: Culture Management, 2022, Volume 23, Issue 1, pp. 37 - 53

DOI: https://doi.org/10.4467/20843976ZK.22.003.15869

Article type: Original article

Titles:

Polish:

Zaawansowane procedury NLP jako przesłanka rekonstrukcji idei wiedzy

English:

Advanced NLP Procedures as Premises for the Reconstruction of the Idea of Knowledge

Authors

Rafał Maciąg

https://orcid.org/0000-0003-1812-3538

Jagiellonian University in Kraków, Gołębia 24, 31-007 Kraków, Poland

Published at: 30.05.2022

Received at: 30.12.2021

Accepted at: 05.04.2022

Article status: Open

Licence: CC BY

Percentage share of authors:

Rafał Maciąg (Author) - 100%

Article corrections:

Publication languages:

Polish

View count: 676

Number of downloads: 345

<p> Advanced NLP Procedures as Premises for the Reconstruction of the Idea of Knowledge</p>

Download xml