graphiti/TODOs.md

7. Suggestions for Improving Graphiti's RAG Techniques

Graphiti already employs a sophisticated RAG pipeline that surpasses standard vector search by using a dynamic knowledge graph. The following suggestions aim to build upon this strong foundation by incorporating more advanced RAG strategies to enhance retrieval accuracy, contextual understanding, and overall performance.

---

1. Advanced Query Pre-processing

The current approach of combining recent messages into a single search query is effective for context. However, it can be enhanced by adding a more structured query analysis step before retrieval.

Current State: A recent conversation history is concatenated into a single string for searching.

• Source Snippet (agent.ipynb):

  graphiti_query = f'{\"SalesBot\" if isinstance(last_message, AIMessage) else state[\"user_name\"]}: {last_message.content}'
  

Suggested Improvements:

• Query Decomposition: For multi-faceted questions, an LLM call could break the query down into multiple sub-queries that are executed against the graph. The results can then be synthesized.

  • Example: A query like "What non-wool shoes would John like that are available in his size (10)?" could be decomposed into:
    1. search("John's preferences") -> Retrieves facts like John LIKES "Basin Blue".
    2. search("John's shoe size") -> Retrieves John HAS_SHOE_SIZE "10".
    3. search("shoes NOT made of wool") -> Retrieves products made of cotton, tree fiber, etc.
  • Benefit: This provides more targeted retrieval than a single, complex query, reducing noise and improving the relevance of retrieved facts.

• Hypothetical Document Embeddings (HyDE): Before performing a semantic search, the LLM can generate a hypothetical "perfect" answer to the user's query. The embedding of this hypothetical answer is then used for the vector search, which often yields more relevant results than embedding the query itself.

  • Example: For the query "What makes the Men's Couriers special?", the LLM might generate a hypothetical fact: "The Men's Couriers are special because they feature a retro silhouette and are made from natural materials like cotton." The embedding of this sentence is then used to find real facts in the graph.
  • Benefit: This bridges the gap between the "question space" and the "answer space" in vector embeddings, leading to better semantic matches.

---

2. Enhanced Graph-Native Retrieval and Summarization

Graphiti's core strength is the graph itself. Retrieval can be made even more powerful by leveraging graph topology more deeply.

Current State: The system uses graph traversal for re-ranking via center_node_uuid and detects communities (build_communities in the Neptune example).

Suggested Improvements:

• Recursive Retrieval & Graph-Based Summarization: Retrieval can be a multi-step process.

  1. Initial Retrieval: Retrieve a set of initial facts/nodes as is currently done.
  2. Neighbor Exploration: For the top N initial nodes, automatically retrieve their direct neighbors. This can uncover critical context that wasn't captured by the initial query.
  3. LLM-Powered Summarization: Pass the retrieved sub-graph (initial nodes + neighbors) to an LLM with a prompt like: "Summarize the key information and relationships in the following set of facts: [facts list]".
  • Benefit: This moves beyond retrieving a simple list of facts to retrieving a synthesized insight from a relevant portion of the graph, which is a much more compressed and potent form of context for the final generation step.

• Leverage Pre-computed Communities: The build_communities method is a powerful feature. These communities can be used to generate summary nodes ahead of time.

  • Implementation: After running community detection, for each community, create a new Summary node. Use an LLM to generate a description for this node that summarizes the entities within that community.
  • Example: A community of nodes related to "SuperLight Wool Runners" could have a summary node: "This product family is characterized by its lightweight SuperLight Foam technology and wool-based materials."
  • Benefit: During retrieval, if the query matches this summary node, the system can retrieve a single, dense summary instead of dozens of individual product facts, leading to massive prompt compression and faster, more coherent context.

---

3. Optimized Ranking and Post-processing

The final step of filtering and ranking retrieved facts is crucial for prompt quality.

Current State: Graphiti uses a hybrid search and likely some form of score fusion (like RRF, as hinted in search_config_recipes) to rank results.

Suggested Improvements:

• LLM-based Re-ranking: After retrieving an initial set of candidate facts (e.g., the top 20-30), use a smaller, faster LLM to perform a final re-ranking pass.

  • Implementation: The LLM would be prompted with the original query and each fact, and asked to output a relevance score (e.g., 1-10) or a simple "relevant/not relevant" judgment.
  • Benefit: This can catch nuanced relevance that semantic or keyword scores might miss, providing a final layer of polish to the retrieved context. It is more expensive but can significantly improve the quality of the top-k results.

• Structured Fact Output: Instead of returning a flat list of fact strings, the retrieval endpoint could return a structured JSON object that preserves the Subject-Predicate-Object nature of the facts.

  • Example:

    [
      { "subject": "John", "predicate": "IS_ALLERGIC_TO", "object": "wool" },
      { "subject": "John", "predicate": "HAS_SHOE_SIZE", "object": "10" }
    ]
    

  • Benefit: This structured format is more easily parsed by the final generation LLM, which can be explicitly prompted to "pay attention to the relationships between subjects and objects." This is a more direct way of "prompting a graph" and can lead to more logical and accurate responses. This also helps the LLM differentiate between entities and their attributes more clearly.