If you find this project useful, please consider giving it a ⭐!
The Oracle AI Optimizer and Toolkit (the AI Optimizer) is an open-source platform for building, testing, and evaluating GenAI applications. Its agentic runtime grounds model responses at request time in authoritative private and sensitive data that remains outside the model's training knowledge and under your control in Oracle AI Database.
The Server exposes a REST API and Model Context Protocol (MCP) endpoint. An optional Streamlit web client provides configuration and experimentation. Portable AgentSpec definitions are loaded into a LangGraph runtime for an LLM-only agent, an NL2SQL agent backed by SQLcl MCP, and a Vector Search RAG flow. A combined router coordinates the NL2SQL and Vector Search routes.
- Application: Python, Streamlit client, and FastAPI/Uvicorn server.
- AI orchestration: LangChain, LangGraph, LiteLLM, AgentSpec/pyagentspec, and Model Context Protocol (MCP).
- Oracle integration: Oracle AI Database Vector Search, Oracle Deep Data Security, SQLcl MCP, OCI, OCI GenAI, and OCI Object Storage.
- RAG and evaluation: Docling, OracleDB/LangChain vector integrations, Giskard v3 testbed checks, and FAISS-backed local vector workflows.
- Model providers and runtimes: OCI GenAI, OpenAI-compatible APIs, OpenAI, Cohere, Perplexity, Ollama, Hugging Face, vLLM, and TGI-style hosted runtimes.
- Deployment and operations: Docker/Podman, Kubernetes, Helm, OpenTofu infrastructure automation, OpenTelemetry, and SigNoz.
- Run LLM-only and tool-enabled agents and flows
- Configure language and embedding models
- Split and embed documents for Vector Search
- Refresh vector stores from OCI Object Storage
- Manage system prompts
- Evaluate generated or existing Q&A datasets
- Apply Oracle Deep Data Security policies
- Inspect portable AgentSpec definitions
- Integrate external applications through the REST API or MCP endpoint
Run the AI Optimizer on a developer workstation, on premises, or in a cloud environment. Supported deployment options include bare metal, containers, Oracle Cloud Infrastructure (OCI), and Kubernetes.
For guided installation and configuration, see the documentation.
- Oracle AI Database, including Oracle AI Database Free.
- Access to a language model to run agents and flows
- Access to an embedding model for RAG and Vector Search
- SQLcl for NL2SQL
- uv for bare-metal installation
- A container runtime such as Docker or Podman for container installation
Language and embedding models can use third-party APIs or run on premises. Oracle recommends GPUs for models hosted on premises.
To run the application on bare metal, download the source:
-
Create the project environment and install the application:
uv sync --all-extras
-
Optionally create an environment file to preconfigure the application:
cp src/.env.example src/.env.dev
Edit
src/.env.devand set the requiredAIO_DB_*values for the Oracle AI Database connection, or complete the local database setup page when the client first starts. -
Start the AI Optimizer Client:
uv run python src/entrypoint.py client
-
Navigate to
http://localhost:8501. -
Configure the AI Optimizer.
To run the application in a container, download the source:
-
Build the all-in-one image.
Note: macOS Silicon users may need to specify
--arch amd64.podman build -f src/Dockerfile -t localhost/ai-optimizer-aio:dev . -
Start the container:
podman run -v ai-optimizer-data:/app/data -p 8501:8501 -it --rm localhost/ai-optimizer-aio:dev
-
Navigate to
http://localhost:8501. -
Complete the required Oracle AI Database setup page. A successful configuration is stored in the
ai-optimizer-datacontainer volume. -
Configure the AI Optimizer.
The AI Optimizer can be deployed in Oracle Cloud Infrastructure (OCI) using Infrastructure as Code (IaC).
Choose a virtual machine or Oracle Kubernetes Engine deployment, both with an Oracle Autonomous Database:
For more information, see the IaC documentation.
This project welcomes contributions from the community. Before submitting a pull request, please review our contribution guide.
Please consult the security guide for our responsible security vulnerability disclosure process.
Copyright (c) 2024 Oracle and/or its affiliates. Released under the Universal Permissive License v1.0 as shown at https://oss.oracle.com/licenses/upl/
See LICENSE for more details.