Skip to content

Oracle AI Optimizer and Toolkit

If you find this project useful, please consider giving it a ⭐!

Test Suite Validate IaC Validate Container Images Deploy Documentation

The Oracle AI Optimizer and Toolkit (the AI Optimizer) is an open-source platform for building, testing, and evaluating GenAI applications. Its agentic runtime grounds model responses at request time in authoritative private and sensitive data that remains outside the model's training knowledge and under your control in Oracle AI Database.

The Server exposes a REST API and Model Context Protocol (MCP) endpoint. An optional Streamlit web client provides configuration and experimentation. Portable AgentSpec definitions are loaded into a LangGraph runtime for an LLM-only agent, an NL2SQL agent backed by SQLcl MCP, and a Vector Search RAG flow. A combined router coordinates the NL2SQL and Vector Search routes.

Technology Stack

  • Application: Python, Streamlit client, and FastAPI/Uvicorn server.
  • AI orchestration: LangChain, LangGraph, LiteLLM, AgentSpec/pyagentspec, and Model Context Protocol (MCP).
  • Oracle integration: Oracle AI Database Vector Search, Oracle Deep Data Security, SQLcl MCP, OCI, OCI GenAI, and OCI Object Storage.
  • RAG and evaluation: Docling, OracleDB/LangChain vector integrations, Giskard v3 testbed checks, and FAISS-backed local vector workflows.
  • Model providers and runtimes: OCI GenAI, OpenAI-compatible APIs, OpenAI, Cohere, Perplexity, Ollama, Hugging Face, vLLM, and TGI-style hosted runtimes.
  • Deployment and operations: Docker/Podman, Kubernetes, Helm, OpenTofu infrastructure automation, OpenTelemetry, and SigNoz.

AI Optimizer Features

Getting Started

Run the AI Optimizer on a developer workstation, on premises, or in a cloud environment. Supported deployment options include bare metal, containers, Oracle Cloud Infrastructure (OCI), and Kubernetes.

For guided installation and configuration, see the documentation.

Prerequisites

  • Oracle AI Database, including Oracle AI Database Free.
  • Access to a language model to run agents and flows
  • Access to an embedding model for RAG and Vector Search
  • SQLcl for NL2SQL
  • uv for bare-metal installation
  • A container runtime such as Docker or Podman for container installation

Language and embedding models can use third-party APIs or run on premises. Oracle recommends GPUs for models hosted on premises.

Bare-Metal Installation

To run the application on bare metal, download the source:

  1. Create the project environment and install the application:

    uv sync --all-extras
  2. Optionally create an environment file to preconfigure the application:

    cp src/.env.example src/.env.dev

    Edit src/.env.dev and set the required AIO_DB_* values for the Oracle AI Database connection, or complete the local database setup page when the client first starts.

  3. Start the AI Optimizer Client:

    uv run python src/entrypoint.py client
  4. Navigate to http://localhost:8501.

  5. Configure the AI Optimizer.

Container Installation

To run the application in a container, download the source:

  1. Build the all-in-one image.

    Note: macOS Silicon users may need to specify --arch amd64.

    podman build -f src/Dockerfile -t localhost/ai-optimizer-aio:dev .
  2. Start the container:

    podman run -v ai-optimizer-data:/app/data -p 8501:8501 -it --rm localhost/ai-optimizer-aio:dev
  3. Navigate to http://localhost:8501.

  4. Complete the required Oracle AI Database setup page. A successful configuration is stored in the ai-optimizer-data container volume.

  5. Configure the AI Optimizer.

Got OCI?

The AI Optimizer can be deployed in Oracle Cloud Infrastructure (OCI) using Infrastructure as Code (IaC).

Choose a virtual machine or Oracle Kubernetes Engine deployment, both with an Oracle Autonomous Database:

Deploy to Oracle Cloud

For more information, see the IaC documentation.

Contributing

This project welcomes contributions from the community. Before submitting a pull request, please review our contribution guide.

Security

Please consult the security guide for our responsible security vulnerability disclosure process.

License

Copyright (c) 2024 Oracle and/or its affiliates. Released under the Universal Permissive License v1.0 as shown at https://oss.oracle.com/licenses/upl/

See LICENSE for more details.

Releases

Packages

Used by

Contributors

Languages

Generated from oracle/template-repo