Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 60 additions & 0 deletions api/audio_buffer.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
.. _api_audio_buffer:

Audio Buffer & Stream Management API
####################################

The Audio Buffer and Stream Management API governs intermediate audio data storage,
circular stream pointer tracking, inter-component buffer negotiation, and lockless
cross-core ring buffer streaming.

Buffer Subsystem Architecture
*****************************

SOF audio buffers decouple processing components, absorbing execution jitter and
rate differences:

.. code-block:: text

+-------------------+ +-------------------+
| Upstream Copier | | Downstream Mixer |
| (Source Component)| | (Sink Component) |
+---------+---------+ +---------^---------+
| |
| audio_stream_produce() audio_stream_consume()|
v |
+---------+---------------------------------------------------+---------+
| struct comp_buffer |
| - id, size, free, avail, flags |
| - source -> upstream comp_dev sink -> downstream comp_dev |
| - stream (struct audio_stream) |
| * r_ptr, w_ptr (circular byte/frame pointers) |
| * addr, end_addr (underlying audio memory buffer) |
+-----------------------------------------------------------------------+

Buffer Types & Usage Profiles
*****************************

1. **Interconnect Buffers (`struct comp_buffer`)**:
Allocated in internal SRAM via `buffer_alloc()` or `rballoc()`. Connects adjacent
components within a pipeline. Supports transaction callbacks (`BUFF_CB_TYPE_*`)
to notify consumers when fresh audio data is written.

2. **Abstract Audio Buffers (`struct sof_audio_buffer`)**:
Unified buffer descriptor providing polymorphic operations (`struct audio_buffer_ops`)
across legacy component buffers (`BUFFER_TYPE_LEGACY_BUFFER`) and lockless
ring buffers (`BUFFER_TYPE_RING_BUFFER`).

3. **Lockless Ring Buffers (`struct ring_buffer`)**:
Designed for cross-core audio streaming (such as Inter-Domain Communication between
Core 0 and secondary DSP cores). Lockless atomic read/write head updates eliminate
spinlock contention between cores.

4. **Circular Audio Streams (`struct audio_stream`)**:
Format-aware circular buffer tracking sample rates, channels, and bit depths,
enabling sample-accurate and frame-accurate data production and consumption.

C API Reference
***************

.. doxygengroup:: audio_buffer_api
:project: SOF Project
37 changes: 37 additions & 0 deletions api/audio_format.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
.. _api_audio_format:

Audio Formats & Channel Routing API
###################################

The Audio Formats and Channel Routing API provides channel map parsing, multi-channel
spatial routing, and integer sample saturation arithmetic across SOF audio processing graphs.

Channel Mapping Infrastructure
******************************

Multi-channel audio streams require precise channel assignments across ALSA PCM devices,
DSP processing algorithms, and hardware Digital Audio Interfaces (DAI).

SOF utilizes channel mapping tokens (`struct sof_ipc_channel_map`) to configure:
- Channel slot numbering and spatial assignments (Front Left, Front Right, Center, LFE, Surround).
- Dynamic channel routing and demultiplexing.
- Helper functions: `chmap_get()` and `chmap_get_size()`.

PCM Sample Saturation Arithmetic
********************************

DSP filter and gain stages can generate audio sample values that exceed the representable
dynamic range of target fixed-point bit depths. Rather than allowing integer overflow
wrap-around distortion (which creates severe acoustic clicks), SOF provides inline
hardware-optimized saturation primitives:

- **`sat_int32(int64_t x)`**: Clamps 64-bit intermediate products to 32-bit signed range (`[INT32_MIN, INT32_MAX]`).
- **`sat_int24(int32_t x)`**: Clamps 32-bit values to 24-bit audio range (`[-8388608, 8388607]`).
- **`sat_int16(int32_t x)`**: Clamps 32-bit values to 16-bit audio range (`[-32768, 32767]`).
- **`sat_int8(int32_t x)`**: Clamps 32-bit values to 8-bit signed range (`[-128, 127]`).

C API Reference
***************

.. doxygengroup:: audio_format_api
:project: SOF Project
97 changes: 97 additions & 0 deletions api/component_infra.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
.. _api_component_infra:

Audio Component Infrastructure API
##################################

The Audio Component Infrastructure API provides the foundational driver model,
lifecycle state machine, copy execution interface, and Inter-Domain Communication (IDC)
binding protocol for all audio processing nodes within SOF.

Component Architecture & Driver Model
*************************************

Audio processing components implement the standardized operations vtable (`struct comp_ops`),
allowing the pipeline engine to manage disparate processing nodes (codecs, volume controls,
equalizers, mixers, and hardware DAIs) through a uniform interface:

.. code-block:: text

+-------------------------------------------------------------------------+
| struct comp_dev (Instance) |
| - id, type, state, core, is_shared, period, frames |
| - bsink_list, bsource_list (connected comp_buffer lists) |
| - drvdata (private component algorithm context) |
+------------------------------------+------------------------------------+
| Points to
v
+-------------------------------------------------------------------------+
| struct comp_driver (Driver) |
| - type, uid, tplg_type, ops |
+------------------------------------+------------------------------------+
| Contains
v
+-------------------------------------------------------------------------+
| struct comp_ops (VTable) |
| - new() / free() : Lifecycle allocation & cleanup |
| - params() / dai_hw_params() : Stream format negotiation |
| - prepare() / reset() : Buffer allocation & resource reset |
| - trigger() : START, STOP, PAUSE, RELEASE transitions |
| - copy() : Audio block processing algorithm loop |
| - bind() / unbind() : Multi-module IPC graph interconnection |
| - get_attribute() / set() : Runtime dynamic module parameters |
+-------------------------------------------------------------------------+

Component Lifecycle & State Machine
***********************************

Every component instance transitions through deterministic execution states in response
to pipeline trigger commands:

.. code-block:: text

+-----------------------+
| COMP_STATE_INIT | <--- comp_new()
+-----------+-----------+
| params()
v
+-----------------------+
+--------> | COMP_STATE_READY | <---------------------------+
| +-----------+-----------+ |
| | prepare() |
| v |
| +-----------------------+ |
| | COMP_STATE_PREPARE | |
| +-----------+-----------+ |
| | START / PRE_START |
| v |
| +-----------------------+ |
| | COMP_STATE_ACTIVE | <-------+ |
| +-----+-----------+-----+ | |
| | | | RELEASE |
| PAUSE | | STOP | |
| v v | |
| +------------------+ +-------------------+ |
| | COMP_STATE_PAUSED| | COMP_STATE_PREPARE| |
| +--------+---------+ +---------+---------+ |
| | | |
| +-----------------------+ |
| | RESET |
| v |
+------------------------+---------------------------------------+

Multi-Core IDC & IPC4 Module Binding
************************************

When components communicate across DSP core boundaries or connect dynamically in IPC4
firmware, binding information is propagated via `struct bind_info` and Inter-Domain
Communication (IDC) messages:

- **Local Core Binding**: Direct buffer attachment between source and sink components.
- **Remote Core Binding**: Uses `comp_ipc4_bind_remote()` to establish cross-core shared
buffers or lockless ring buffer streaming channels.

C API Reference
***************

.. doxygengroup:: component_infra_api
:project: SOF Project
157 changes: 151 additions & 6 deletions api/components.rst
Original file line number Diff line number Diff line change
@@ -1,11 +1,156 @@
.. _api_components:
.. _api_modules:

Audio Processing Components API
###############################
Audio Processing Components & Modules API
#########################################

SOF includes a rich library of standard audio processing components, covering volume control,
gain attenuation, channel mixing, IIR/FIR equalization, dynamic range compression (DRC),
sample rate conversion (SRC/ASRC), sound dose measurement, and acoustic filters.
In Sound Open Firmware (SOF), all audio processing components—including volume controllers,
mixers, equalizers (EQ FIR/IIR), dynamic range compressors (DRC), sample rate converters
(SRC/ASRC), crossover filters, and smart amplifiers—are architected as **processing modules**.

.. doxygengroup:: audio_components_api
Modules can be built statically into the base firmware binary or compiled and loaded
dynamically at runtime as relocatable ELF extensions using the Zephyr Loadable Extension
(LLEXT) framework.

Architecture & Module Adapter Layer
***********************************

The pipeline execution graph decouples pipeline orchestration from signal processing
algorithms through the **Module Adapter** layer:

.. code-block:: text

+-------------------------------------------------------------------------+
| Pipeline Graph |
| pipeline_create() / pipeline_connect() / walk_op |
+------------------------------------+------------------------------------+
| comp_ops (params, prepare, copy)
v
+-------------------------------------------------------------------------+
| Module Adapter Component |
| DECLARE_MODULE_ADAPTER(name) |
| - Adapts comp_ops into struct module_interface calls |
| - Binds comp_buffer streaming queues to sof_source and sof_sink APIs |
| - Handles parameter fragmentation (large configuration blobs) |
+------------------------------------+------------------------------------+
| module_interface
v
+-------------------------------------------------------------------------+
| Audio Processing Algorithm Module |
| (Volume, Mixer, EQ FIR/IIR, DRC, SRC, ASRC, Smart Amp, etc.) |
| - init() / prepare() / process() / apply_config() / free() |
| - Pulls audio from struct sof_source |
| - Pushes audio to struct sof_sink |
+------------------------------------+------------------------------------+
| System Services
v
+-------------------------------------------------------------------------+
| Host & RTOS System Services |
| (Dynamic Heap Memory, Timers, Tracing, UUID, Math) |
+-------------------------------------------------------------------------+

Processing Component Catalog
****************************

SOF provides an extensive suite of built-in processing modules optimized for low-latency
DSP execution:

.. list-table::
:widths: 20 22 28 30
:header-rows: 1

* - Component
- Processing Mode
- Hardware Acceleration
- Primary Function
* - **Volume**
- Source/Sink (LL / DP)
- SIMD HiFi3 / HiFi4 / HiFi5
- Smooth logarithmic/linear volume ramp, mute, gain attenuation, zero-crossing detection.
* - **Mixer / Mixout**
- Source/Sink (LL)
- Vectorized summation
- Multi-channel audio stream summation, channel splitting, and matrix routing.
* - **EQ (FIR & IIR)**
- Source/Sink (LL / DP)
- DF2T biquad SIMD
- Parametric equalization, shelving filters, high/low-pass filters, and speaker tuning.
* - **DRC & Multiband**
- Source/Sink (LL / DP)
- Vectorized RMS detector
- Dynamic range compression, peak limiting, soft/hard knee thresholds, and multiband crossover.
* - **SRC & ASRC**
- Source/Sink (LL / DP)
- Polyphase FIR filterbank
- Synchronous and asynchronous sample rate conversion with fractional drift compensation.
* - **Crossover**
- Source/Sink (LL / DP)
- 4th-order Linkwitz-Riley (LR4)
- Multi-way acoustic frequency splitting (woofer, midrange, tweeter).
* - **Smart Amp**
- Source/Sink (LL / DP)
- Thermal/Excursion observer
- Speaker transducer protection, coil temperature estimation, and IV-sense feedback.
* - **Sound Dose**
- Source/Sink (LL / DP)
- A-weighted filterbank
- Real-time acoustic energy integration complying with IEC 62368-1 and EN 50332 standards.
* - **KPB / MFCC**
- Source/Sink (LL / DP)
- FFT & Mel filterbank
- Keyword pre-roll circular buffering and Mel-frequency cepstral coefficient feature extraction.

Module Lifecycle & Interface Contracts
**************************************

All audio modules implement `struct module_interface`:

1. **Initialization (`init`)**:
Invoked when the module adapter component is instantiated (`.new()`). Allocates private
algorithm state memory and registers default parameters. Typically marked `__cold`.

2. **Stream Preparation (`prepare`)**:
Called when stream parameters are configured (`.prepare()`). Validates sample rates,
channel counts, and buffer formats across all attached sources and sinks.

3. **Audio Block Processing (`process`)**:
The primary execution callback running in the real-time processing loop. Consumes audio
samples from source endpoint buffers and emits processed audio samples into sink
endpoint buffers.

4. **Dynamic Configuration (`set_configuration` / `get_configuration`)**:
Transfers runtime parameter blobs (filter coefficients, DRC curves, tuning parameters)
from the host via IPC. Supports multi-fragment chunking for large configuration structures.

5. **Reset & Teardown (`reset`, `free`)**:
Restores algorithm states to initial default settings and frees runtime buffers.

Streaming Endpoints: Source & Sink APIs
***************************************

Modules consume audio data exclusively via `struct sof_source` and produce audio data
via `struct sof_sink`:

- **Source API (`source_get_data()`, `source_release_data()`)**:
Allows the module to request a contiguous block of audio samples from the input stream.
The underlying source handles circular buffer rollover, cache invalidation, and pointer advance.

- **Sink API (`sink_get_buffer()`, `sink_commit_buffer()`)**:
Provides a destination memory region for the module to store processed samples.
The sink commits the written bytes, flushes cache lines, and advances write pointers.

Dynamic LLEXT Loading & Identification
**************************************

Dynamic processing modules declare their ABI version, format compatibility tag, and unique
UUID using standard SOF registration macros:

- `SOF_DEFINE_UUID()`: Associates a 128-bit RFC 4122 UUID with the module driver.
- `struct sof_module_api_build_info`: Encodes module build timestamp, git hash, and API version.
- `SOF_LLEXT_EXPORT_SYMBOL()`: Exposes module entry points for dynamic symbol resolution.

C API Reference
***************

.. doxygengroup:: module_api
:project: SOF Project
20 changes: 15 additions & 5 deletions api/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -27,14 +27,23 @@ Overview of Documented Modules

The Doxygen documentation covers the entire public firmware and host-shared interface:

* **Audio Components & Pipelines**:
Core component driver lifecycle (``component.h``), component extensions and buffer helpers (``component_ext.h``), and PCM stream buffer utilities (``audio_stream.h``).
* **Audio Processing Components & Modules**:
Unified processing module interface (``module/interface.h``), lifecycle callbacks, streaming endpoints (``sink_api.h``, ``source_api.h``), dynamic LLEXT symbols, and built-in audio components (Volume, Mixer, EQ FIR/IIR, DRC, SRC, ASRC, Crossover, Smart Amp, Sound Dose).

* **Audio Pipelines & Buffers**:
Audio pipeline DAG graphs (``pipeline.h``), component driver lifecycle and ops (``component.h``, ``component_ext.h``), interconnect and lockless ring buffers (``buffer.h``, ``audio_buffer.h``, ``ring_buffer.h``, ``audio_stream.h``), and channel mapping routines (``channel_map.h``).

* **Hardware Drivers & Interfaces**:
Direct Memory Access (``dma.h``), Digital Audio Interfaces for I2S/SSP, SoundWire/ALH, DMIC/PDM, and HDA (``dai.h``), Power Management runtime (``pm_runtime.h``), and platform hardware timers and interrupt controllers (``platform.h``).

* **Core RTOS & System Services**:
Real-time task scheduling (EDF, LL-Timer, LL-DMA, Zephyr DataProcessing threads in ``schedule.h``), memory allocation heaps (``alloc.h``), and component/pipeline UUID declarations (``uuid.h``).
Spinlocks, mutexes, condition variables, clocks, and component/pipeline UUID declarations (``uuid.h``).

* **Scheduling Infrastructure**:
Real-time task schedulers (Low-Latency Timer and DMA in ``ll_schedule.h`` and ``ll_schedule_domain.h``, Data Processing preemptible threads in ``dp_schedule.h``, Thread With Budget in ``twb_schedule.h``, and task state machines in ``schedule.h`` and ``rtos/task.h``).

* **Memory Management Infrastructure**:
Multi-tier system heaps and flags (``rtos/alloc.h``), Virtual Memory Heap (``regions_mm.h``), virtual regions (``vregion.h``), cold DRAM staging (``fast-get.h``), and virtual page mapping (``vpage.h``).

* **IPC & Host Interfaces**:
User/Kernel IPC ABI protocols and messaging envelopes (``ipc/header.h``, ``ipc/control.h``), and SRAM Window 0 firmware status registers and telemetry offsets (``kernel/mailbox.h``).
Expand All @@ -49,9 +58,10 @@ Subsystem C API References
:maxdepth: 1

ipc4
modules
pipeline
components
pipeline
memory
scheduling
rtos
math

Expand Down
Loading
Loading