InterrogateLLM: Zero-Resource Hallucination Detection in LLM-Generated Answers

Yakir Yehuda, Itzik Malkiel, Oren Barkan, Jonathan Weill, Royi Ronen, Noam Koenigstein

نتاج البحث: فصل من :كتاب / تقرير / مؤتمرمنشور من مؤتمرمراجعة النظراء

ملخص

Despite the many advances of Large Language Models (LLMs) and their unprecedented rapid evolution, their impact and integration into every facet of our daily lives is limited due to various reasons. One critical factor hindering their widespread adoption is the occurrence of hallucinations, where LLMs invent answers that sound realistic, yet drift away from factual truth. In this paper, we present a novel method for detecting hallucinations in large language models, which tackles a critical issue in the adoption of these models in various real-world scenarios. Through extensive evaluations across multiple datasets and LLMs, including Llama-2, we study the hallucination levels of various recent LLMs and demonstrate the effectiveness of our method to automatically detect them. Notably, we observe up to 87% hallucinations for Llama-2 in a specific experiment, where our method achieves a Balanced Accuracy of 81%, all without relying on external knowledge.

اللغة الأصليةالإنجليزيّة
عنوان منشور المضيفLong Papers
المحررونLun-Wei Ku, Andre F. T. Martins, Vivek Srikumar
ناشرAssociation for Computational Linguistics (ACL)
الصفحات9333-9347
عدد الصفحات15
رقم المعيار الدولي للكتب (الإلكتروني)9798891760943
حالة النشرنُشِر - 2024
منشور خارجيًانعم
الحدث62nd Annual Meeting of the Association for Computational Linguistics, ACL 2024 - Bangkok, تايلند
المدة: ١١ أغسطس ٢٠٢٤١٦ أغسطس ٢٠٢٤

سلسلة المنشورات

الاسمProceedings of the Annual Meeting of the Association for Computational Linguistics
مستوى الصوت1
رقم المعيار الدولي للدوريات (المطبوع)0736-587X

!!Conference

!!Conference62nd Annual Meeting of the Association for Computational Linguistics, ACL 2024
الدولة/الإقليمتايلند
المدينةBangkok
المدة١١/٠٨/٢٤١٦/٠٨/٢٤

ملاحظة ببليوغرافية

Publisher Copyright:
© 2024 Association for Computational Linguistics.

بصمة

أدرس بدقة موضوعات البحث “InterrogateLLM: Zero-Resource Hallucination Detection in LLM-Generated Answers'. فهما يشكلان معًا بصمة فريدة.

قم بذكر هذا