Виявлення намірів користувача при спілкуванні з ботом
DOI:
https://doi.org/10.35681/1560-9189.2023.25.2.300436Ключові слова:
zero-shot-класифікації, виявлення намірів користувача, чат-бот, моделі NLIАнотація
Розглянуто проблему класифікації намірів користувачів у чат-боті, зосереджуючись на використанні моделей машинного та глибинного навчання. Проведено аналіз існуючих методів і наборів даних, і визначено задачу виявлення намірів через zero-shot-класифікацію за допомогою моделей Natural Language Inference (NLI). Запропоновано різнотипні підходи до класифікації і використання моделей NLI для виявлення намірів у повідомленнях користувача. Експериментально перевірено їхню ефективність на різних корпусах різних доменів, порівняно з існуючими методами, включаючи попередньо навчені моделі та великі мовні моделі, такі як GPT-3.5. Отримані результати свідчать про перспективність підвищення точності zero-shot-класифікації намірів за допомогою різнотипних моделей NLI та шаблонів гіпотез, хоча важливо враховувати, що найвищі показники точності не досягають значень попередньо навчених моделей і GPT-3.5.
Посилання
Serhiy Telenyk, Hzhehozh Novakovs'kyy, Yevheniy Vovk, Ihor Anosov. Rozvytok i realizatsiya tekhnolohiyi stvorennya shyrokoho klasu zastosuvan' typu chat-botiv na osnovi formal'nykh modeley. Naukovi zapysky NaUKMA. Komp"yuterni nauky. 2022. Tom 5. S. 97–107. ISSN 2617-3808. DOI: 10.18523/2617-3808.2022.5.97-107.
A suite of Text Analytics and Identity Analytics products to analyze Big Data using AI and MachineLearning-based technologies. URL: http://www.sra.com/netowl
Leading provider of text analytics solutions: Attensity. URL: http://www.attensity.com
Stumme G., Hotho A., Berendt B. Semantic Web Mining. State of the art and future directions. Web Semantics: Science, Services and Agents on the World Wide Web. 2006. No. 4. P. 124–143.
OWL 2 Web Ontology Language:Structural Specification and Functional-Style Syntax Boris Motik, Peter F. Patel-Schneider, Bijan Parsia, eds. W3C Working Draft, 2008. URL: http://www.w3.org/ TR/2008/WD-owl2-syntax-20081202/
Alexander Koller, Alex Lascarides A Logic of Semantic Representations for Shallow Parsing. Proceedings of the 12th Conference of the European Chapter of the ACL. 200. P. 30–39.
Pavlov A.A., Telenik S.F. Informacionnye tehnologii i algoritmizacija v upravlenii. Kiev: Tehnika, 2002. 344 c.
Ann Copestake, Dan Flickinger, Ivan Sag and Carl Pollard. Minimal Recursion Semantics: An introduction. Journal of Research on Language and Computation. 2005. No. 3. P. 281–332
Tore Amble. The Understanding Computer. Natural Language Understanding in Practice. Norweg, 2008. 201 p.
Larson S., & Leach K. (2022). A Survey of Intent Classification and Slot-Filling Datasets for Task-Oriented Dialog. arXiv preprint arXiv:2207.13211.
Hemphill C.T., Godfrey J.J., & Doddington G.R. The ATIS spoken language systems pilot corpus. In Speech and Natural Language: Proceedings of a Workshop Held at Hidden Valley. Pennsylvania. June 24-27, 1990.
Hirschman L. Multi-site data collection for a spoken language corpus. In Speech and Natural Language: Proceedings of a Workshop Held at Harriman. New York. February 23–26, 1992.
Hirschman L., Bates M., Dahl D.A., Fisher W.M., Garofolo J.S., Pallett D.S., ... & Tzouker-mann E. Multi-site data collection and evaluation in spoken language understanding. In Human Language Technology: Proceedings of a Workshop Held at Plainsboro. New Jersey, March 21–24, 1993.
Upadhyay S., Faruqui M., Tur G., Dilek H.T., & Heck L. (Almost) zero-shot cross-lingual spoken language understanding. In 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP). 2018, April. Р. 6034–6038.
Xu W., Haider B., & Mansour S. (2020). End-to-end slot alignment and recognition for cross-lingual NLU arXiv preprint arXiv:2004.14353.
Dao M.H., Truong T.H., & Nguyen D.Q. (2021). Intent detection and slot filling for Vietnamese. arXiv preprint arXiv:2104.02021.
Wang Y., Shen Y., & Jin H. (2018). A bi-model based rnn semantic frame parsing model for intent detection and slot filling. arXiv preprint arXiv:1812.10235.
Chen X., Ghoshal A., Mehdad,Y., Zettlemoyer L., & Gupta S. (2020). Low-resource domain adaptation for compositional task-oriented semantic parsing. arXiv preprint arXiv:2010.03546.
Larson S., Mahendran A., Peper J.J., Clarke C., Lee A., Hill P., ... & Mars J. (2019). An evaluation dataset for intent classification and out-of-scope prediction. arXiv preprint arXiv:1909.02027.
Papers with Code: Text Classification on clinc_oos. URL: https://paperswithcode.com/ sota/text-classification-on-clinc-oos
Casanueva I., Temcinas T., Gerz D., Henderson M., & Vulic I. (2020). Efficient intent detection with dual sentence encoders. arXiv preprint arXiv:2003.04807.
FitzGerald J., Hench C., Peris C., Mackie S., Rottmann K., Sanchez A., ... & Natarajan P. (2022). Massive: A 1m-example multilingual natural language understanding dataset with 51 typologically-diverse languages. arXiv preprint arXiv:2204.08582.
Loureiro D., Barbieri F., Neves L., Anke L.E., & Camacho-Collados J. (2022). Timelms: Diachronic language models from twitter. arXiv preprint arXiv:2202.03829.
The platform where the machine learning community collaborates on models, datasets, and applications: Toxicity Classification Model. URL: https://huggingface.co/s-nlp/roberta_toxicity_classifier
Conneau A., Khandelwal K., Goyal N., Chaudhary V., Wenzek G., Guzman F., ... & Stoyanov V. (2019). Unsupervised cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02116.
The platform where the machine learning community collaborates on models, datasets, and applications Sowanski Marcin, 2023: xml-r-base-amazon-massive-intent. URL: https://huggingface.co/ cartesinus/xlm-r-base-amazon-massive-intent
Lin Y.T., Papangelis A., Kim S., Lee S., Hazarika D., Namazifar M., ... & Hakkani-Tur D. (2023). Selective in-context data augmentation for intent detection using pointwise v-information. arXiv preprint arXiv:2302.05096.
Clarke C., Heng Y., Kang Y., Flautner K., Tang L., & Mars J. (2023). Label Agnostic Pre-training for Zero-shot Text Classification. arXiv preprint arXiv:2305.16521.
Bastianelli E., Vanzo A., Swietojanski P., & Rieser V. (2020). SLURP: A spoken language understanding resource package. arXiv preprint arXiv:2011.13205.
Adina Williams, Nikita Nangia, and Samuel Bowman. A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2018. Volume 1 (Long Papers), Р. 1112–1122, New Orleans, Louisiana. Association for Computational Linguistics.
Ma T., Yao J. G., Lin C. Y., & Zhao T. (2021, August). Issues with entailment-based zero-shot text classification. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers) (Р. 786–796).
Coucke A., Saade A., Ball A., Bluche T., Caulier A., Leroy D., ... & Dureau J. (2018). Snips voice platform: an embedded spoken language understanding system for private-by-design voice interfaces. arXiv preprint arXiv:1805.10190.
Gururangan S., Swayamdipta S., Levy O., Schwartz R., Bowman S.R., & Smith N.A. (2018). Annotation artifacts in natural language inference data. arXiv preprint arXiv:1803.02324.
The platform where the machine learning community collaborates on models, datasets, and applications: bard-large-mnli. URL: https://huggingface.co/facebook/bart-large-mnli
Schuster T., Fisch A., & Barzilay R. Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021, June. Р. 624–643.
Laban P., Schnabel T., Bennett P. N., & Hearst M.A. SummaC: Re-visiting NLI-based models for inconsistency detection in summarization. Transactions of the Association for Computational Linguistics. 2022. 10. Р. 163–177.
Nie Y., Chen H., & Bansal M. Combining fact extraction and verification with neural semantic matching networks. In Proceedings of the AAAI conference on artificial intelligence. 2019, July. Vol. 33, No. 01. Р. 6859–6866.
Nie Y., Williams A., Dinan E., Bansal M., Weston J., & Kiela D. Adversarial NLI: A New Benchmark for Natural Language Understanding. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020, July. Р. 4885–4901.
Parrish A., Huang W., Agha O., Lee S.H., Nangia N., Warstadt A., ... & Bowman S. (2021, November). Does Putting a Linguist in the Loop Improve NLU Data Collection?. In Findings of the Association for Computational Linguistics: EMNLP 2021. Р. 4886–4901.
