Skip to content

rag.md

Latest commit

 

History

History
320 lines (237 loc) · 11.3 KB

rag.md

File metadata and controls

320 lines (237 loc) · 11.3 KB
Raw
title RAG System Documentation

The MultiMind SDK's RAG (Retrieval Augmented Generation) system combines document processing, embeddings, and vector storage so you can retrieve relevant context and feed it to any MultiMind model. This document covers the system's components, configuration, and usage.

Table of Contents

  1. Overview
  2. Components
  3. Installation
  4. Basic Usage
  5. Advanced Features
  6. Agent Integration
  7. Best Practices
  8. API Reference

Overview

The RAG system lets you:

  • Process and chunk documents with configurable sizes and overlap
  • Embed documents and store the vectors in a vector store (FAISS and others)
  • Perform semantic search over documents with optional metadata filtering
  • Generate answers by passing retrieved context to any MultiMind model
  • Integrate retrieval into the agent system as a custom tool

The core class is RAG (retrieval only — you compose it with a model for generation). It is configured with a single RAGConfig object that bundles the vector store, embedding, retrieval, and document-processing settings.

Components

1. Documents

Documents are plain data objects:

from multimind.document_processing.base import Document

doc = Document(
    id="doc_1",
    content="The MultiMind SDK is a powerful framework.",
    metadata={"category": "documentation"},
    source="example",
)

Chunking behavior is controlled by the document_config dict on RAGConfig (min_chunk_size, max_chunk_size, chunk_overlap). Note that RAG.add_documents(process=True) only re-chunks when the document processor has a model attached; with the default configuration documents are embedded as-is.

2. Embeddings

Embedding behavior is configured with EmbeddingConfig. Supported model_type values include "openai" (e.g. text-embedding-ada-002, 1536 dimensions) and "huggingface" (e.g. sentence-transformers/all-MiniLM-L6-v2, 384 dimensions, fully local). The OpenAI embedding client does not read OPENAI_API_KEY itself; pass the key through custom_params:

import os
from multimind.embeddings.embedding import EmbeddingConfig

embedding_config = EmbeddingConfig(
    model_name="text-embedding-ada-002",
    model_type="openai",
    batch_size=32,
    max_length=512,
    normalize=True,
    device="cpu",
    cache_dir=None,
    custom_params={"api_key": os.environ["OPENAI_API_KEY"]},
)

3. Vector Stores

Vector stores are configured with VectorStoreConfig. FAISS (in-memory, fast similarity search) has a convenience constructor; the dimension must match your embedding model:

from multimind.vector_store import VectorStoreConfig

vector_store_config = VectorStoreConfig.create_faiss_config(
    dimension=1536,       # 384 for all-MiniLM-L6-v2
    metric="cosine",
    index_type="flat",
)

Other backends (Chroma, Qdrant, Weaviate, Pinecone, Milvus, Elasticsearch, and more) are available via the vector-stores extra; see multimind.vector_store for the factory and per-backend configs. Not every backend in VectorStoreType is implemented: AwaDB, BagelDB, BaiduCloud, Clarifai, ClickHouse, DashVector, Databricks Vector Search, DeepLake, DingoDB, Elastic Vector Search (legacy) and Epsilla are stubs that raise NotImplementedError, and Cassandra, Azure AI Search, Azure Cosmos DB, LLMRails and TileDB are only partially implemented. The per-backend status is in feature-status.md.

Installation

Install the RAG system with all dependencies:

pip install "multimind-sdk[rag]"

Required environment variables (only for the providers you use):

export OPENAI_API_KEY="your-openai-key"        # For OpenAI models/embeddings
export ANTHROPIC_API_KEY="your-anthropic-key"  # For Claude models

HuggingFace embeddings run locally and need no API key.

Basic Usage

1. Initialize the RAG System

import asyncio
import os
from multimind import RAG, RAGConfig, OpenAIModel
from multimind.vector_store import VectorStoreConfig
from multimind.embeddings.embedding import EmbeddingConfig
from multimind.document_processing.base import Document

model = OpenAIModel(model_name="gpt-4o-mini")

config = RAGConfig(
    vector_store_config=VectorStoreConfig.create_faiss_config(
        dimension=1536, metric="cosine", index_type="flat"
    ),
    retrieval_config={"top_k": 3, "similarity_threshold": 0.5},
    embedding_config=EmbeddingConfig(
        model_name="text-embedding-ada-002",
        model_type="openai",
        batch_size=32,
        max_length=512,
        normalize=True,
        device="cpu",
        cache_dir=None,
        custom_params={"api_key": os.environ["OPENAI_API_KEY"]},
    ),
    document_config={"min_chunk_size": 100, "max_chunk_size": 1000, "chunk_overlap": 200},
)

rag = RAG(config)

2. Add Documents

async def add_docs():
    await rag.initialize()
    docs = [
        Document(id="doc_0", content="The MultiMind SDK is a powerful framework.",
                 metadata={"source": "direct"}, source="example"),
        Document(id="doc_1", content="RAG combines retrieval with generation.",
                 metadata={"source": "direct"}, source="example"),
    ]
    await rag.add_documents(docs, process=True)

asyncio.run(add_docs())

3. Retrieve and Generate

RAG.retrieve returns a list of multimind.retrieval.RetrievalResult objects (content, score, metadata, document_id, source, chunk_id) whose score is at or above similarity_threshold; you build the prompt and generate with any model:

async def answer(question: str) -> str:
    retrieved = await rag.retrieve(question, k=3)
    context = "\n\n".join(result.content for result in retrieved)
    prompt = f"Context:\n{context}\n\nQuestion: {question}\n\nAnswer:"
    return await model.generate(prompt, temperature=0.7)

print(asyncio.run(answer("What is the MultiMind SDK?")))

A complete runnable version of this flow (with similarity-threshold guard rails and a local no-API-key fallback) is in examples/rag/example_rag.py.

Advanced Features

1. Metadata Filtering

RAG.retrieve accepts a filter_criteria dict, but in 1.0.0 its entries are forwarded to the vector store as keyword arguments, so filter_criteria={"category": "documentation"} raises RetrievalError (VectorStore.search() got an unexpected keyword argument 'category'). Until that is fixed, over-fetch and filter on the result metadata:

results = await rag.retrieve("deployment steps", k=20)
results = [r for r in results if r.metadata.get("category") == "documentation"][:5]

2. Batch Queries

queries = ["query1", "query2", "query3"]
results = await asyncio.gather(*(rag.retrieve(q, k=3) for q in queries))

3. Model Switching

Retrieval is independent of the generating model, so you can pick a model per task:

from multimind import ClaudeModel, OpenAIModel

# Use Claude for complex reasoning
claude = ClaudeModel(model_name="claude-3-5-sonnet-20241022")
complex_answer = await claude.generate(prompt, temperature=0.7)

# Use a smaller model for simpler tasks
mini = OpenAIModel(model_name="gpt-4o-mini")
simple_answer = await mini.generate(prompt, temperature=0.3)

For switching models mid-conversation with context transfer, see ModelSession in the cookbook.

4. Clearing the Store

await rag.clear()   # remove all documents and vectors

Agent Integration

Retrieval plugs into the agent system as a custom tool (subclass BaseTool):

from multimind.agents import Agent, AgentMemory
from multimind.agents.tools import BaseTool, CalculatorTool

class RAGTool(BaseTool):
    def __init__(self, rag: RAG):
        super().__init__(
            name="rag_query",
            description="Retrieve relevant documents for a query",
        )
        self.rag = rag

    def get_parameters(self):
        return {
            "type": "object",
            "properties": {
                "query": {"type": "string"},
                "top_k": {"type": "integer", "default": 3},
            },
            "required": ["query"],
        }

    async def run(self, query: str, top_k: int = 3):
        results = await self.rag.retrieve(query, k=top_k)
        return "\n\n".join(r.content for r in results)

agent = Agent(
    model=model,
    memory=AgentMemory(max_history=100),
    tools=[RAGTool(rag), CalculatorTool()],
)

response = await agent.run("What are the main features of the MultiMind SDK?")  # returns a dict

CalculatorTool is the only tool shipped in multimind.agents.tools; anything else is a BaseTool subclass you write.

Best Practices

  1. Document Processing

    • Choose appropriate chunk sizes based on your use case
    • Use meaningful metadata for better filtering
    • Clean and normalize text before processing

  2. Embedding Models

    • Use OpenAI embeddings for production applications
    • Use HuggingFace models for local or cost-sensitive use cases
    • Match the vector store dimension to your embedding model

  3. Vector Stores

    • Use FAISS for high-performance, in-memory applications
    • Use a persistent backend (via the vector-stores extra) when data must survive restarts
    • Monitor memory usage with large document collections

  4. Retrieval Quality

    • Set a similarity_threshold and refuse to answer when nothing relevant is retrieved
    • Consider grounding-checking generated answers — see hallucination detection

  5. Agent Integration

    • Combine RAG with other tools for comprehensive solutions
    • Use memory to maintain context across interactions
    • Implement proper error handling and fallbacks

API Reference

RAG Class

class RAG:
    def __init__(self, config: RAGConfig)

    async def initialize(self) -> None

    async def add_documents(
        self,
        documents: List[Document],
        process: bool = True,
    ) -> None

    async def retrieve(
        self,
        query: str,
        k: int = 5,
        filter_criteria: Optional[Dict[str, Any]] = None,   # see "Metadata Filtering" above
    ) -> List[RetrievalResult]

    async def clear(self) -> None

RAGConfig

RAGConfig is a dataclass:

@dataclass
class RAGConfig:
    vector_store_config: VectorStoreConfig
    retrieval_config: Dict[str, Any]     # e.g. {"top_k": 3, "similarity_threshold": 0.5} (defaults: 5, 0.7)
    embedding_config: EmbeddingConfig
    document_config: Dict[str, Any]      # e.g. {"min_chunk_size": 100, "max_chunk_size": 1000, "chunk_overlap": 200}
    custom_params: Dict[str, Any] = None

multimind.rag also exports BaseRAG, RAGError, GraphRetriever, PostProcessor and PostProcessingConfig; a fluent RAGPipeline lives in multimind.rag.fluent (see examples/rag/fluent_rag_example.py).

For the HTTP RAG service, see rag_api.md and the generated openapi-rag.json.