Skip to content

[Suggestion] Agno (Phidata) voice-enabled agent with Deepgram STT and TTS (Python) #336

Description

@deepgram-robot

What to build

A working integration example that builds a voice-enabled AI agent using the Agno framework (formerly Phidata) with Deepgram STT for speech input and Deepgram TTS for speech output. The agent should accept voice input, process it through an LLM with tool calling, and respond with synthesized speech.

Why this matters

Agno (rebranded from Phidata) is one of the most popular Python agent frameworks with 20,000+ GitHub stars. Developers building voice-enabled AI agents in Python frequently reach for Agno for its tool calling, memory, and knowledge base features — but there is no example showing how to connect Deepgram's voice capabilities to an Agno agent. This integration enables developers to add voice I/O to their existing Agno-based agents without switching frameworks, and positions Deepgram as the voice layer for one of the largest agent ecosystems.

Suggested scope

  • Language: Python
  • Frameworks: Agno (latest), Deepgram Python SDK
  • Deepgram APIs: Streaming STT (Nova-3 or Flux), TTS (Aura)
  • Key components:
    • Agno agent with at least 2 tools (e.g., web search, calculator, or custom function)
    • Microphone capture → Deepgram streaming STT → agent input
    • Agent response → Deepgram TTS → audio playback
    • Conversation memory (Agno's built-in session memory)
    • Clean shutdown handling
  • Backend: No separate server needed — runs as a single Python script
  • Complexity: Medium — Agno setup, Deepgram streaming, audio I/O

Acceptance criteria

  • Runnable with minimal setup (clone, add API key, run)
  • README explains the pattern clearly
  • Uses current SDK version
  • Agno agent processes voice input and responds with voice output
  • At least 2 working tool calls demonstrated
  • Conversation memory persists across turns within a session
  • Works with both Nova-3 and Flux STT models

Raised by the DX intelligence system.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions