Grounding & Retrieval (RAG) Techniques

Grounding & Retrieval (RAG) Techniques

Traditional Retrieval-Augmented Generation (RAG) architectures typically follow a straightforward sequence across the preparation and query phases, as described below.

Preparation Phase

  • Segment source documents into fixed-size chunks suitable for the chosen embedding model → Chunks
  • Generate embeddings for each chunk → Chunk Embeddings
  • Store the resulting embeddings in a vector database

Query Phase

  • Generate embeddings for the user query → Query Embeddings
  • Execute a similarity search in the vector database using the query embeddings → Top‑K candidate documents
  • Aggregate the retrieved Top‑K documents → Context
  • Provide the context and query to the LLM to produce the final Answer

In RAG systems, chunking refers to the process of dividing large documents into smaller, manageable segments or “chunks”.

The quality and structure of these chunks significantly influence the performance of the system.

While this baseline approach performs adequately in many scenarios (often achieving success rates above 50%), achieving accuracy levels greater than 90% requires more advanced techniques. This basic approach is commonly referred to as “Naive RAG”.

Reference:

https://www.ibm.com/think/topics/rag-techniques

However, Naive RAG introduces several inherent limitations:

  • Fixed-size chunking may split sentences or ideas, resulting in loss of semantic continuity
  • Context fragmentation occurs as small, isolated chunks may not capture the broader meaning of the source material
  • Single‑vector querying can miss relevant documents expressed using different phrasing or terminology
  • Similarity-based retrieval returns the closest matches, which are not necessarily the most relevant ones for the user intent

To address these challenges, more sophisticated techniques – discussed in the following sections – are required to significantly improve retrieval quality and overall system performance. These techniques have been organized in four groups based on which part of the RAG flow is enhanced the most:

  • Chunking Techniques
  • Query Optimization Techniques
  • Relevance Improvements
  • Advanced Methods

Chunking Techniques

Context-Aware Chunking

Context-aware chunking is a crucial technique in RAG systems that enhances the relevance and accuracy of generated outputs by ensuring that information is segmented into meaningful, context-rich pieces. Here are some key reasons why context-aware chunking is essential:

  • Maintaining Semantic Integrity: Context-aware chunking ensures that each chunk captures a complete idea or concept, preventing the dilution of important information. This is crucial for generating coherent and relevant responses.
  • Optimizing Retrieval Efficiency: By creating well-defined chunks, RAG systems can retrieve relevant information more quickly and accurately. This leads to improved performance and reduced computational overhead.
  • Addressing Context Window Constraints: LLMs have strict limits on the amount of context they can process. Properly sized chunks help avoid exceeding these limits, ensuring that the most relevant information is fed into the model.

The disadvantages of using context-aware chunking are:

  • Slower than fixed-size chunking
  • Requires proper document parsing

Note: It is advised to use context-aware chunking in all cases instead of fixed-size chunking.

Contextual Retrieval

Contextual retrieval is an enhancement to traditional RAG systems, designed to improve the accuracy of information retrieval by preserving the context of document chunks. The key principles of contextual retrieval are the following:

  • Context Preservation: Each chunk is enriched with a short, context-specific explanation that situates it within the overall document. For example, a chunk about Berlin might be contextualized as: “Berlin is the capital and largest city of Germany, known for being the EU’s most populous city within its limits”.
  • Improved Retrieval Accuracy: By adding contextual information, retrieval errors can be reduced significantly. Anthropic’s research shows a 49% reduction in errors when using contextual embeddings.
  • Further Improved Retrieval Accuracy: Contextual retrieval can be combined with hybrid search methods (semantic and keyword-based) and re-ranking techniques to further enhance retrieval quality.

Reference:

https://www.anthropic.com/engineering/contextual-retrieval

The disadvantages of using contextual retrieval are:

  • Can be expensive, since it requires one LLM query per chunk during preparation
  • Documents ingestion is slower
  • Creates larger index sizes

Note: Use this method for important documents where accuracy matters more than costs.

Hierarchical RAG

Hierarchical Retrieval-Augmented Generation is a technique that enhances the performance of RAG systems by organizing information into structured layers.

The key characteristics of hierarchical RAG are the following:

  • Hierarchical RAG builds on traditional RAG by using a structured method to find information at different levels, improving accuracy and relevance.
  • It operates through a two-step retrieval process:
    • Generic-to-Specific approach:
      • first, it queries a broader index to identify relevant categories, and
      • then it performs a more focused search for specific information
    • Specific-to-Generic approach:
      • first, it queries an index of focused, small-sized pieces of text to identify relevant information, and
      • then it aggregates larger pieces of texts/documents to feed the LLM
  • This RAG approach reduces retrieval noise and supports multi-hop reasoning, making it more efficient for handling complex queries.
  • Hierarchical RAG is particularly useful in scenarios where information is organized into hierarchical structures, such as documents or knowledge graphs, allowing for better navigation and context-aware responses.

Reference:

https://arxiv.org/abs/2405.14446

The following diagram depicts the two phases of hierarchical RAG (chunking & embeddings creation, and searching):

--- Image: RAG Technique Hierarchical RAG ---

The benefits of using hierarchical RAG are:

  • Balances level of precision with context length
  • Reduces noise in search, since it fetches most relevant matching results
  • Natural for structured documents where there is already a hierarchy defined

The disadvantages of using hierarchical RAG are:

  • Requires to build / create a parent-child schema and persist in a DB
    • Needs careful hierarchy design
  • More complexing indexing strategy due to multi-level hierarchy

Note: Use it when documents have clear hierarchical structure (technical manuals, legal documents, research papers).

Late Chunking

Late chunking is an advanced technique in RAG that enhances context retention and retrieval accuracy by embedding the entire document before splitting it into smaller chunks. Traditional chunking methods often split documents into smaller segments before embedding them, which can lead to a loss of critical context and semantic relationships. Late chunking, on the other hand, involves embedding the entire document first and then splitting it into chunks, allowing for better context preservation and more accurate retrieval results.

The advantages of using late chunking are the following:

  • Context Preservation: By embedding the full document, late chunking retains the broader context, which is crucial for understanding references and relationships within the text. This is particularly important for documents with anaphoric references, where terms like “the company” or “her” may refer to concepts introduced earlier in the text.
  • Improved Retrieval Accuracy: Research indicates that late chunking can enhance retrieval accuracy by 10-12% compared to traditional methods. This improvement is especially noticeable in documents where key information is spread across multiple sections.
  • Semantic Coherence: Late chunking allows for embeddings that capture the semantic meaning of the entire document, leading to more meaningful and relevant results during the retrieval phase.

References:

https://www.bluetickconsultants.com/unlocking-better-text-retrieval-with-late-chunking-a-revolutionary-approach-for-rag-applications/

https://medium.com/@visrow/what-is-late-chunking-in-rag-how-can-you-improve-your-rag-with-late-chunking-f981a0cb39bb

--- Image: RAG Technique Hierarchical RAG ---

The benefits of using late chunking are the following:

  • Maintains full document context
  • Leverages long-context models effectively
  • Better semantic understanding

The drawbacks of late chunking are:

  • Requires long-context embedding models
    • if the text/document can’t fit in the embedding model input, this method can’t be used
  • The implementation of this method is very complex
  • Limited by model’s max tokens input

Note: Use it when document context is crucial for understanding chunks (dense technical documents, legal contracts).

Query Optimization Techniques

Query Expansion

Query expansion is a technique used in RAG systems to improve the quality of retrieved documents before passing them to the language model. The idea is to transform or enrich the user’s query into a more detailed version so that the retriever can find more relevant and comprehensive context.

The key benefits of query expansion are:

  • Covers synonyms and paraphrases: Users may phrase queries differently than the documents in the knowledge base.
  • Improves recall: Retrieves more potentially relevant documents.
  • Reduces retrieval misses: Handles vocabulary mismatch between query and documents.
  • Boosts answer accuracy: The LLM gets richer, more relevant context.

The disadvantages of using query expansion are:

  • Slower than pure vector (similarity) search
  • May impose slightly higher costs due to extra LLM query

Note: Use it when it is expected that users ask ambiguous questions (i.e. chatbots or search).

Multi-Query RAG

Multi-query RAG extends the query expansion method, in the sense it transforms the user’s query into multiple semantically related queries so that the retriever can find more relevant and comprehensive context. The similarity searches can execute in parallel to decrease latency.

# Generate multiple related queries
from langchain_openai import ChatOpenAI
from langchain.prompts import PromptTemplate
from langchain_community.vectorstores import FAISS
from langchain_community.embeddings import OpenAIEmbeddings
# Initialize LLM and embeddings
llm = ChatOpenAI(model="gpt-4", temperature=0)
embeddings = OpenAIEmbeddings()
# Example FAISS vector store (replace with your own)
vectorstore = FAISS.load_local("my_faiss_index", embeddings)
# Prompt to generate multiple query variations
prompt = PromptTemplate(
    input_variables=["query"],
    template="""# Search Query Paraphrase Generator
Generate **3 distinct alternative phrasings** of the search query below.

Each variation should:

* Preserve the **original intent and meaning**.
* Use **meaningfully different wording or sentence structure**.
* Reflect a **natural way a real user might search for the same information**.
* Avoid introducing new assumptions, constraints, or details.
* Avoid near-duplicate variations.

**Query:**
`{query}`

**Output format:**
Return exactly 3 variations, one per line, with no additional commentary.

**Variations:**
1.
2.
3.
    """
)

def expand_query(query):
    """Generate multiple query variations using an LLM."""
    expansion_prompt = prompt.format(query=query)
    response = llm.predict(expansion_prompt)
    variations = [q.strip("- ").strip() for q in response.split("\n") if q.strip()]
    return [query] + variations  # Include original query
def rag_with_multiple_queries(user_query, top_k=3):
    """Perform RAG retrieval with query expansion."""
    queries = expand_query(user_query)
    all_docs = []
    for q in queries:
        docs = vectorstore.similarity_search(q, k=top_k)
        all_docs.extend(docs)
    # Deduplicate by document content
    unique_docs = list({doc.page_content: doc for doc in all_docs}.values())
    return unique_docs
# Example usage
if __name__ == "__main__":
    query = "symptoms of heart attack"
    docs = rag_with_multiple_queries(query)
    for i, doc in enumerate(docs, 1):
        print(f"Doc {i}: {doc.page_content[:200]}...")

Multi-query RAG suffers from the same disadvantages as query expansion. Furthermore, it queries the vector DB at least 3 times more (even though execution can be parallelized).

Relevance Improvements

Re-ranking

Re-ranking is a technique used in RAG systems to refine the initial set of retrieved documents. After an initial search, which may use semantic or keyword matching, a re-ranker evaluates and reorders these documents based on their relevance to the specific query. This two-step process ensures that the most relevant information is presented to the user, improving the overall quality of the output.

Reference:

https://cookbook.openai.com/examples/search_reranking_with_cross-encoders

The key advantages of re-ranking are the following:

  • Improved Relevance: Re-ranking helps filter out irrelevant documents that may have been retrieved initially, ensuring that only the most pertinent information is considered for generating responses.
  • Cost Efficiency: By focusing on the most relevant documents, re-ranking reduces the computational load on downstream systems, saving resources and costs associated with processing unnecessary data.
  • Enhanced User Experience: By delivering more accurate and contextually relevant responses, re-ranking significantly improves user satisfaction and the effectiveness of AI-driven applications.

The disadvantages of using re-ranking are:

  • Slower than pure vector (similarity) search
  • May impose slightly higher costs

Note: Use it when precision is more important than speed.

Advanced Methods

Basic Agentic RAG

Agentic RAG extends traditional Retrieval‑Augmented Generation by incorporating autonomous, goal‑driven behaviors into the retrieval and reasoning process. Instead of following a fixed sequence of retrieve‑then‑generate, an agentic system actively evaluates what it knows, identifies gaps in its understanding, and decides when additional retrieval steps are necessary. This approach utlizes the tools that are available to the RAG agent, to enhance the retrieval process. Many problems can be solved / answered by combining various data sources, structured or not. Agent tools can help in this front to provide the necassary information for a successful RAG application.

Reference:

https://www.ibm.com/think/topics/agentic-rag

from langchain.tools import tool
from langchain_openai import ChatOpenAI
from langchain.agents import AgentExecutor, create_tool_calling_agent
from langchain_core.prompts import ChatPromptTemplate
# -------------------------------------
# Language Model
# -------------------------------------
llm = ChatOpenAI(
    model="gpt-4o",
    temperature=0
)
# -------------------------------------
# Tools
# -------------------------------------
@tool
async def search_knowledge_base(query: str, limit: int = 5) -> str:
    """Semantic search over document chunks."""
    query_embedding = await embedder.embed_query(query)
    results = await db.match_chunks(query_embedding, limit)
    return format_results(results)

@tool
async def retrieve_full_document(document_title: str) -> str:
    """Retrieve full document when chunk context is insufficient."""
    result = await db.query(
        "SELECT title, content FROM documents WHERE title ILIKE %s",
        f"%{document_title}%"
    )
    return f"**{result['title']}**\n\n{result['content']}"

@tool
async def sql_query(question: str) -> str:
    """Query structured database for specific data."""

    return execute_safe_sql(question)

tools = [search_knowledge_base, retrieve_full_document, sql_query]
# -------------------------------------
# System Prompt
# -------------------------------------
prompt = ChatPromptTemplate.from_messages([
    ("system",
     """You are a RAG assistant with multiple retrieval tools. Your task is to select the most appropriate tool(s) to answer each user query accurately and efficiently.

For each query:

1. **Understand the need**: Identify key entities, intent, constraints, and required information type.
2. **Select tools**: Choose the best-matching tool(s) based on data source and capability. Use multiple tools only if necessary for completeness or validation. Avoid irrelevant calls.
3. **Prefer quality sources**: Prioritize authoritative, specialized, or first-party sources. Use structured retrieval for precise facts and semantic retrieval for conceptual queries.
4. **Decompose if needed**: Split complex queries into sub-questions and retrieve per subtask, then combine results.
5. **Resolve uncertainty pragmatically**: If unclear, pick the most likely tool or run minimal exploratory retrieval. Do not guess or fabricate.
6. **Minimize calls**: Use the fewest tool invocations needed; avoid redundant or already-available information.
7. **Ground responses**: Base answers on retrieved evidence, distinguish inference from facts, and state limitations when evidence is insufficient.

**Goal:** maximize relevance, factual accuracy, and source quality while minimizing retrieval cost."""),
    ("human", "{input}")
])
# -------------------------------------
# Create an LLM agent using tool-calling
# -------------------------------------
agent = create_tool_calling_agent(
    llm=llm,
    tools=tools,
    prompt=prompt
)
executor = AgentExecutor(
    agent=agent,
    tools=tools,
    verbose=True
)
# -------------------------------------
# Usage
# -------------------------------------
async def run_agent(query: str):
    return await executor.ainvoke({"input": query})

The key benefits of agentic RAG are the following:

  • Flexibility: Agentic RAG applications pull data from multiple external knowledge bases and allow for external tool use.
  • Adaptability: The system has the ability to adapt to changing contexts and (if applicable) access the right data sources specific to the query.
  • Versatility: Can be combined with other RAG strategies for optimum results

The disadvantages of using agentic RAG are:

  • Higher latency due to the number of LLM queries
  • May exhibit less predictable behavior
  • High costs due to extensive usage of LLMs

Note: Use it when data can be retrieved from heterogeneous data sources (documents, databases, APIs) and queries vary widely in complexity.

Self-Reflective RAG (full agentic)

Self-reflective RAG is based on the Generate, Reflect and Refine agentic AI pattern (see Self-Reflection & ), and to some extend is an evolution of Agentic RAG approach. Unlike traditional RAG systems that simply retrieve information and generate responses, self-reflective RAG incorporates a self-reflection mechanism, that allows the model to evaluate the relevance and accuracy of the information it retrieves and generates. If the response (Generate phase) is at acceptable level (Reflect phase), the system returns it to the user, otherwise it refines the query (Refine phase) and runs the process again.

Warning: To avoid endless loops in self-reflection, it is essential to define stopping criteria that prevent the system from continuing the process indefinitely. These criteria can include a fixed number of iterations, a threshold for quality or correctness, or a maximum time limit for the reflection process. By implementing these stopping criteria, the model can ensure that the self-reflection loop is not prolonged unnecessarily, allowing for a more efficient and effective improvement process.

from openai import OpenAI
client = OpenAI()
def self_reflective_rag(query: str, limit: int = 5, max_iterations: int = 3) -> dict:
    """Self-reflective search loop."""
    for iteration in range(max_iterations):
        # Perform search (your existing synchronous function)
        results = vector_search(query, limit)
        # Prepare grading prompt
        grade_prompt = f"""You are a relevance grader.
Query:
{query}

Retrieved documents:
{format_docs_for_grading(results)}

Rate how relevant the retrieved documents are to answering the query using this scale:

* 1 = Completely irrelevant
* 2 = Mostly irrelevant; only weak or tangential connection
* 3 = Partially relevant; contains some useful information but has notable gaps
* 4 = Highly relevant; directly addresses most of the query
* 5 = Fully relevant; directly and comprehensively addresses the query

Evaluate relevance based on whether the documents contain information that would help answer the query. Do not judge writing quality, factual correctness, or completeness beyond what affects relevance.

Output exactly one integer from 1 to 5. Do not include explanations, punctuation, or any other text.
"""
        # Call OpenAI chat completion
        grade_response = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[
                {"role": "user", "content": grade_prompt}
            ],
            temperature=0
        )
        grade_text = grade_response.choices[0].message.content.strip()
        grade = int(grade_text.split()[0])
        # Good enough → return
        if grade >= 3:
            return {
                "results": results,
                "iterations": iteration + 1,
                "final_query": query
            }
        # If poor results and not finished, ask for query refinement
        if iteration < max_iterations - 1:
            refine_prompt = f"""Given the search query:
"{query}"

The query produced low-relevance document search results.

Rewrite it to maximize retrieval relevance by preserving the user's core intent while:

* replacing vague or ambiguous terms with precise, domain-relevant language,
* adding important concepts, entities, synonyms, or likely document terminology,
* removing unnecessary words,
* avoiding unsupported assumptions or invented details.

Return exactly one improved search query and nothing else.
"""
            refined_response = client.chat.completions.create(
                model="gpt-4o-mini",
                messages=[
                    {"role": "user", "content": refine_prompt}
                ],
                temperature=0.5
            )
            query = refined_response.choices[0].message.content.strip()
    # After max iterations, return best attempt
    return {
        "results": results,
        "iterations": max_iterations,
        "final_query": query
    }

The key benefits of self-reflective RAG are the following:

  • Enhanced Accuracy: By allowing models to self-correct, this method reduces the likelihood of generating misleading or incorrect information, addressing a common issue in AI responses.
  • Greater Control: The use of reflection tokens provides more control over the generation process, enabling the model to tailor its behavior to specific tasks and requirements.
  • Versatility: Self-reflective RAG maintains the versatility of language models while improving their reliability, making them more effective for knowledge-intensive tasks.

The disadvantages of using self-reflective RAG are:

  • Higher latency due to the number of LLM queries
  • Most expensive method (due to extensive usage of LLMs)

Note: Use it when response accuracy is critical and latency is acceptable. Great for cases where queries are more complex than usual.

Knowledge Graph-Based RAG

Knowledge Graphs are structured representations of information, consisting of entities (nodes) and relationships (edges) between them. They play a pivotal role in RAG applications by enabling structured knowledge retrieval and contextual understanding for generating accurate and coherent responses.

Knowledge graphs enhance RAG systems by providing structured, interconnected data that supports inferential reasoning and contextual understanding. For example, a knowledge graph can represent relationships like “Sarah works for Prismatic AI” enabling the system to infer that Sarah and Michael work for the same company. This structured approach improves the quality and relevance of generated responses.

Reference:

https://www.datacamp.com/tutorial/knowledge-graph-rag

Refer to Knowledge Graph Databases vs Vector Databases for a comparison of the two database paradigms most commonly used in RAG-based applications.

from langchain_community.document_loaders import DirectoryLoader, TextLoader
from langchain_community.graphs.graph_document import GraphDocument
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_core.documents import Document
from langchain_openai import ChatOpenAI
from langchain_experimental.graph_transformers import LLMGraphTransformer
def extract_graph(text: str) -> list[GraphDocument]:
 
    documents = [ Document(page_content=text, metadata={"source": "file.txt"}) ]
    # It first tries to keep paragraphs intact, then sentences, then words if needed.
    text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=20)
    texts = text_splitter.split_documents(documents)
    # Initialize LLM
    llm = ChatOpenAI(
        model='gpt-4.1-mini', 
        temperature=0, 
        max_tokens=4096, 
        api_key=os.environ['OPENAI_KEY'])
    # Extract Knowledge Graph
    llm_transformer = LLMGraphTransformer(llm=llm)
    graph_documents = llm_transformer.convert_to_graph_documents(texts)
    return graph_documents
# Example usage
if __name__ == '__main__':
    text = """Sarah is an employee at prismaticAI, a leading technology company based in Westside Valley. She has been working there for the past three years as a software engineer.
Michael is also an employee at prismaticAI, where he works as a data scientist. He joined the company two years ago after completing his graduate studies.
prismaticAI is a well-known technology company that specializes in developing cutting-edge software solutions and artificial intelligence applications. The company has a diverse workforce of talented individuals from various backgrounds.
Both Sarah and Michael are highly skilled professionals who contribute significantly to prismaticAI's success. They work closely with their respective teams to develop innovative products and services that meet the evolving needs of the company's clients."""
    docs = extract_graph(text)
    for i, doc in enumerate(docs, 1):
        print(f'nodes: {len(doc.nodes)}')
        for n in doc.nodes:
            print(f'  {n.type}: {n.id}')
            print(f'    props: {n.properties}')
        print(f'rels: {len(doc.relationships)}')
        for r in doc.relationships:
            print(f'  {r.source.id} --> {r.type} --> {r.target.id}')
            print(f'    props: {r.properties}')

from langchain_neo4j import Neo4jGraph, GraphCypherQAChain
from langchain_openai import ChatOpenAI
# Query graph example
if __name__ == '__main__':
    # Store Knowledge Graph in Neo4j
    graph_store = Neo4jGraph(
        url='neo4j://localhost:7687',
        username='neo4j',
        password='neo4j-pwd')
    # Initialize LLM
    llm = ChatOpenAI(
        model='gpt-4.1-mini', 
        temperature=0, 
        max_tokens=4096, 
        api_key=os.environ['OPENAI_KEY'])
    chain = GraphCypherQAChain.from_llm(llm, 
        graph=graph_store, verbose=True)
    #question = 'Where does Sarah work?'
    #chain.run(question)
    #question = 'Who works for prismaticAI?'
    #chain.run(question)
    question = 'Does Michael work for the same company as Sarah?'
    result = chain.invoke(question)
    print(f'Q: {question}\n')
    print(result)

Reference:

https://www.datacamp.com/tutorial/knowledge-graph-rag

Note: The Graphiti Python library (https://github.com/getzep/graphiti) provides an integrated search framework that combines semantic, keyword-based, and graph‑driven retrieval methods, along with built‑in re‑ranking capabilities – offering a comprehensive, all‑in‑one search solution.

Fine-tuned Embeddings Model

Fine-tuning embedding models in a Retrieval-Augmented Generation (RAG) pipeline enhances the model’s ability to retrieve domain-specific and contextually relevant information. This process adapts embeddings to provided training dataset, improving retrieval accuracy and response generation.

The key benefits of fine-tuned embeddings model are the following:

  • Improved Retrieval: Tailors embeddings to domain-specific language, enhancing relevance.
  • Cost-Effective: Focuses on query optimization without re-embedding the entire knowledge base.

The considerations to take into account when using fine-tuned embeddings models are the following:

  • Training dataset size: Ensure the training dataset contains sufficient labeled or synthetic data for effective training.
  • Training dataset quality: Ensure the training dataset covers all the possible technical terms / glossary.

RAG Strategies

Selecting a single RAG strategy can improve search accuracy, but only to a limited extent. To fully unlock the potential of RAG and achieve the best results, it’s essential to integrate multiple techniques into a combined approach.

In the following paragraphs, we explore several combined strategies that demonstrate how integrating multiple RAG techniques can deliver significantly better performance.

Strategy 1 – Baseline for production

The first strategy is the proposed baseline for general purpose production RAG applications. The set of techniques to use are:

  • Context-Aware Chunking – to make sure (text) chunks retain coherent context
  • Query Expansion – to cope with vague questions
  • Re-ranking – to add context relevance ordering in VectorDB similarity search results
  • Basic Agentic RAG – to be able to adapt to complex queries (i.e. multiple available data sources to choose from)

Strategy 2 – Best for critical use cases

This strategy is best for user cases where the system must ensure the validity of the responses it generates, by utilizing self-correction actions. The set of techniques to use are:

  • Contextual Retrieval – to make sure (text) chunks are self-contained
  • Multi-Query RAG – to capture all aspects of the underline query
  • Re-ranking – to add context relevance ordering in VectorDB similarity search results
  • Self-Reflective RAG – to be able to evaluate the relevance and accuracy of the information it retrieves from the LLMs and re-iterate if required (self-correction).

Strategy 3 – Domain specific

The third strategy is suitable for special cases where deep domain knowledge is required at every part of the solution.

  • (Optional) Fine-tuned Embeddings Model – to let the model understand the terminology used per domain
    • If the terminology is not big in size, providing it as system prompt instruction may be sufficient
  • Contextual Retrieval – to make sure (text) chunks are self-contained
  • Knowledge Graph-based – to capture the underline relationships in the selected domain
  • Re-ranking – to add domain context relevance ordering in VectorDB similarity search results