Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Models

Retrieval augmented generation (RAG) can be hindered by imperfect retrieval that contains irrelevant, misleading, or malicious information. Knowledge conflicts between a large language model's internal knowledge and this external information are a significant bottleneck for RAG systems. A novel approach, Astute RAG, is proposed to be more resilient to imperfect retrieval augmentation. It adaptively elicits information from the LLM's internal knowledge, iteratively consolidates internal and external knowledge with source-awareness, and finalizes the answer based on information reliability. Experiments demonstrate that Astute RAG improves performance and trustworthiness, even surpassing conventional LLM use in worst-case scenarios.