Add audio chat API support (gpt-4o-audio-preview, gpt-audio-1.5)

[BugorBN]

Summary

This PR implements audio-to-audio chat completion support, replacing the traditional STT→Chat→TTS pipeline with a single API call for 2-3x faster response times.

Supports both gpt-4o-audio-preview / gpt-4o-mini-audio-preview (preview models) and the new gpt-audio-1.5 (recommended by OpenAI for best performance with improved instruction following, tool calling, and multilingual accuracy).

Features

• Audio Chat API: Direct audio-to-audio conversations with audioChats() and audioChatsStream() methods
• Type-safe enums: AudioFormat (wav, mp3, flac, opus, pcm16) and Voice (alloy, echo, fable, onyx, nova, shimmer)
• Streaming support: Real-time audio chunk streaming with AsyncThrowingStream
• AudioConversationManager: Actor-based utility for managing multi-turn conversations with automatic history tracking and pruning
• Format requirements:
  • Input audio: wav and mp3 only
  • Output audio: wav, mp3, flac, opus, pcm16 (pcm16 recommended for streaming)
• Relaxed parsing support: Handles missing fields gracefully, consistent with ChatResult behavior

Implementation Details

• New models: AudioChatQuery, AudioChatResult, AudioChatStreamResult
• Model constants: .gpt_4o_audio_preview, .gpt_4o_mini_audio_preview, .gpt_audio_1_5
• Protocol updates: OpenAIProtocol, OpenAIAsync, with Combine support
• Comprehensive documentation in README.md with usage examples
• Full async/await and Combine publisher support

Testing

• 38 passing tests across 4 test suites with 100% coverage:
  • AudioChatQueryCodingTests (8 tests)
  • AudioChatResultTests (9 tests)
  • AudioChatStreamResultTests (10 tests)
  • AudioConversationManagerTests (11 tests)

Files Changed

• 14 files changed, 2,072 insertions, 4 deletions

Documentation

Added comprehensive Audio Chat section to README.md including:

• API overview and benefits
• Format requirements and recommendations
• Request/response structure documentation
• Non-streaming and streaming usage examples
• AudioConversationManager examples

Breaking Changes

None - this is a purely additive change.

Related

• OpenAI Audio Chat API: https://platform.openai.com/docs/guides/audio
• OpenAI model announcement: gpt-audio-1.5 recommended for Chat Completions

[Krivoblotsky]

Thank you for the work here, and apologies for the wait. It merges cleanly with main, builds, and all 217 tests pass. I want to be upfront about the direction, because it affects most of the diff.

ChatQuery already covers audio chat. On main today: ChatQuery.modalities, ChatQuery.audioOptions (11 voices including ash, ballad, coral, sage, verse; formats mp3, opus, flac, wav, pcm16), ContentPartAudioParam for input_audio, ChatResult.Choice.Message.audio (id, expiresAt, data, transcript) and the same on the streaming delta. AudioChatQuery, AudioChatResult and AudioChatStreamResult are a second way to call /chat/completions with a smaller surface (6 voices, fewer parameters), and audioChats / audioChatsStream add four protocol requirements, which the new API-breakage gate flags. The library should have one way to do a chat completion.

AudioConversationManager is conversation state management, which belongs in the app (or the Demo), not in the SDK.

What we would like to take from this PR:

• the gpt-audio-1.5 constant (plus gpt-audio and gpt-audio-mini), with gpt-4o-audio-preview marked deprecated since it was shut down on 7 May 2026;
• a README section "Audio input and output with Chat Completions" built on ChatQuery, which the README lacks entirely today; your examples adapt almost one to one;
• the coding tests, adapted to ChatQuery where they add coverage we do not have.

If you are up for reshaping the PR along those lines, that would be great and it would land quickly. If not, we can open a smaller PR that carries those pieces with you credited as co-author. Either way, thank you for pushing on audio support.

[BugorBN]

Thank you for the work here, and apologies for the wait. It merges cleanly with main, builds, and all 217 tests pass. I want to be upfront about the direction, because it affects most of the diff.

ChatQuery already covers audio chat. On main today: ChatQuery.modalities, ChatQuery.audioOptions (11 voices including ash, ballad, coral, sage, verse; formats mp3, opus, flac, wav, pcm16), ContentPartAudioParam for input_audio, ChatResult.Choice.Message.audio (id, expiresAt, data, transcript) and the same on the streaming delta. AudioChatQuery, AudioChatResult and AudioChatStreamResult are a second way to call /chat/completions with a smaller surface (6 voices, fewer parameters), and audioChats / audioChatsStream add four protocol requirements, which the new API-breakage gate flags. The library should have one way to do a chat completion.

AudioConversationManager is conversation state management, which belongs in the app (or the Demo), not in the SDK.

What we would like to take from this PR:

• the gpt-audio-1.5 constant (plus gpt-audio and gpt-audio-mini), with gpt-4o-audio-preview marked deprecated since it was shut down on 7 May 2026;
• a README section "Audio input and output with Chat Completions" built on ChatQuery, which the README lacks entirely today; your examples adapt almost one to one;
• the coding tests, adapted to ChatQuery where they add coverage we do not have.

If you are up for reshaping the PR along those lines, that would be great and it would land quickly. If not, we can open a smaller PR that carries those pieces with you credited as co-author. Either way, thank you for pushing on audio support.

Hey, thanks for the feedback, I'll reshape it in the next couple days