Authors: Bhor Vaishnavi S., Professor Bangar Abhishek P., Professor Karad Vandana A., Professor Bhosale Sachin B.
Abstract: Chatbots based on Large Language Models (LLMs) can provide quick, fluent, and informative answers, but they some-times generate incorrect or unsupported information known as hallucinations. These hallucinations can reduce user trust and create problems in areas such as education, healthcare, research, and customer support. Therefore, detecting and preventing hallucinated outputs is important for developing reliable and trustworthy AI systems. Retrieval-Augmented Generation (RAG) is a useful approach that retrieves relevant information from trusted sources before generating an answer. This retrieved information can be used to check whether the chatbot's response is accurate and supported by available evidence. However, RAG alone cannot completely prevent hallucinations because the generated answer may still contain incorrect facts, irrelevant information, or unsupported statements. This paper proposes a conceptual frame-work for detecting hallucinations in chatbot responses by combining information retrieval, evidence-based verification, and rule-based validation. The system compares the generated answer with retrieved information and identifies state-ments that are incorrect or lack sufficient supporting evidence. Based on the verification results, the system classifies the response as reliable or flagged. If the response contains unsupported information, the system can suggest corrections or prevent the answer from reaching the user without proper verification. The proposed approach aims to improve an-swer accuracy, reduce misleading information, and increase transparency in AI-based conversational systems. Although this framework offers a promising direction, its effectiveness depends on the quality of retrieved information, verifica-tion methods, and decision thresholds. Future evaluation should measure detection accuracy, response time, and com-putational cost. This approach can support the development of safer, more reliable, and trustworthy chatbots across different application areas.
