AI-POWERED LANGUAGE MODELS ENHANCING NATURAL LANGUAGE UNDERSTANDING AND GENERATION
Keywords:
Large Language Models (LLMs), Natural Language Understanding (NLU), Artificial IntelligenceAbstract
A new era of revolutionary developments in Generative AI has begun with the introduction of Large Language Models (LLMs). When it comes to NLG and Natural Language Understanding (NLU) problems, these models—which have billions of parameters—have shown to be unmatched. This study explores the history of generative AI, with a focus on how LLMs were crucial. We investigate how these models' ability to handle massive volumes of textual data and produce coherent, contextually appropriate text has transformed NLU and NLG. In addition, we explore the methods and approaches used to tap into the potential of LLMs for a variety of applications, such as chatbots, content generation, machine translation, and sentiment analysis. We also take a look at the problems that come with LLM-based generative AI, including things like model bias, the amount of computing power needed for training and fine-tuning, and ethical considerations. We conclude by suggesting avenues for further study in this area, with an eye towards improving LLMs for more general use, reducing their shortcomings, and guaranteeing their ethical implementation in practical settings. This paper provides a thorough review of generative AI as it is right now, explaining how it could change our interactions with and creation of natural language material.
References
A. Chernyavskiy, D. Ilvovsky, P. Nakov, Transformers:“the end of history” for natural language processing?, in: Machine Learning and Knowledge Discovery in Databases. Research Track: European Conference, ECML PKDD 2021, Bilbao, Spain, September 13–17, 2021, Proceedings, Part III 21, Springer, 2021, pp. 677–693. 1
A. Wang, Y. Pruksachatkun, N. Nangia, A. Singh, J. Michael, F. Hill, O. Levy, S. Bowman, Superglue: A stickier benchmark for generalpurpose language understanding systems, Advances in neural information processing systems 32 (2019). 1, 24, 29
D. Adiwardana, M.-T. Luong, D. R. So, J. Hall, N. Fiedel, R. Thoppilan, Z. Yang, A. Kulshreshtha, G. Nemade, Y. Lu, et al., Towards a humanlike open-domain chatbot, arXiv preprint arXiv:2001.09977 (2020). 1
B. A. y Arcas, Do large language models understand us?, Daedalus 151 (2) (2022) 183–197. 2
A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, et al., Language models are unsupervised multitask learners, OpenAI blog 1 (8) (2019) 9. 2, 7
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al., Language models are few-shot learners, Advances in neural information processing systems 33 (2020) 1877–1901. 2, 6, 7, 8, 9, 16, 17, 22, 23, 24, 25, 33
J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, arXiv preprint arXiv:1810.04805 (2018). 2, 18, 24
M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, L. Zettlemoyer, Deep contextualized word representations, in: NAACLHLT, Association for Computational Linguistics, 2018, pp. 2227–2237. 2
M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, L. Zettlemoyer, Bart: Denoising sequence-to-sequence pretraining for natural language generation, translation, and comprehension, arXiv preprint arXiv:1910.13461 (2019). 2
C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, P. J. Liu, Exploring the limits of transfer learning with a unified text-to-text transformer, The Journal of Machine Learning Research 21 (1) (2020) 5485–5551. 2, 7, 8, 17, 19, 24, 25, 26, 28, 30, 31
L. Xue, N. Constant, A. Roberts, M. Kale, R. Al-Rfou, A. Siddhant, A. Barua, C. Raffel, mt5: A massively multilingual pre-trained text-totext transformer, arXiv preprint arXiv:2010.11934 (2020). 2, 7, 8, 24, 25, 28, 30
Z. Zhang, Y. Gu, X. Han, S. Chen, C. Xiao, Z. Sun, Y. Yao, F. Qi, J. Guan, P. Ke, et al., Cpm-2: Large-scale cost-effective pre-trained language models, AI Open 2 (2021) 216–224. 2, 8, 25
T. L. Scao, A. Fan, C. Akiki, E. Pavlick, S. Ilic, D. Hesslow, ´ R. Castagné, A. S. Luccioni, F. Yvon, M. Gallé, et al., Bloom: A 176bparameter open-access multilingual language model, arXiv preprint arXiv:2211.05100 (2022). 2, 4, 9, 11, 22, 23, 24, 25, 30
S. Zhang, S. Roller, N. Goyal, M. Artetxe, M. Chen, S. Chen, C. Dewan, M. Diab, X. Li, X. V. Lin, et al., Opt: Open pre-trained transformer language models, arXiv preprint arXiv:2205.01068 (2022). 2, 9, 11, 23, 24, 25
A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann, et al., Palm: Scaling language modeling with pathways, arXiv preprint arXiv:2204.02311 (2022).
Downloads
Published
Issue
Section
License
Copyright (c) 2024 Venkata Sai Swaroop Reddy, Nallapa Reddy (Author)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
