[RFC] ao_wasapi: add Windows Spatial Audio output

[Kajcsi]

This adds an opt-in Windows Spatial Audio output using
ISpatialAudioClient.

The regular WASAPI output uses the format and channel layout exposed by
the selected endpoint. In some setups, that means decoded surround audio
gets remixed before Windows Spatial Audio sees the original channel
layout.

wasapi-spatial instead sends decoded PCM channels to Windows as static
spatial audio objects. Windows then renders those objects for the selected
output device. This makes the backend useful with headphones, surround
speaker systems, home theater endpoints, and supported layouts with
height channels.

The new output stays out of the normal auto-probe order. It can be
selected explicitly while leaving regular WASAPI as a fallback:

ao=wasapi-spatial,wasapi

It reuses mpv's existing Windows audio device enumeration, selection,
and loading helpers, so --audio-device behaves the same way as it does
with the existing WASAPI output. The backend handles start, pause, reset,
and shutdown on its own render thread.

The backend accepts decoded PCM only. It rejects exclusive mode and
compressed passthrough. It also does not preserve native Dolby Atmos or
DTS:X object metadata. If those formats are decoded before reaching the
backend, wasapi-spatial receives the resulting channel bed and submits
that to Windows as static spatial audio objects.

I tested this through Plezy, a libmpv frontend, on Windows with
multichannel content. Windows received the surround channel bed without
exclusive access, and normal endpoint processing remained available. I
have not tested every output device or Windows spatial renderer, so I am
submitting this as an RFC.

This implements the static-object part of #11306.

I used AI assistance for parts of the Windows API research and while
preparing the patch. I reviewed the resulting code, tested the behavior
described above, and understand the changes I am submitting. I take
responsibility for the contribution and confirm that the code can be
submitted under the same license as the files it changes.

[kasper93]

Does is have to be separate AO? It complicates what normally would be auto negotiation of channel layout on output to user having to select specific AO. I would see this as an extension for WASAPI to support, non standard layouts. However, another question is, is WASAPI (non-spatial) still useful, if all channels can be sent as static objects? Is the rendering differs? Are those AO interchangeable or complementary to each other?

[Kajcsi]

Does is have to be separate AO? It complicates what normally would be auto negotiation of channel layout on output to user having to select specific AO. I would see this as an extension for WASAPI to support, non standard layouts. However, another question is, is WASAPI (non-spatial) still useful, if all channels can be sent as static objects? Is the rendering differs? Are those AO interchangeable or complementary to each other?

I don't think it has to be separate. I kept it separate mostly so I
could get the ISAC path working without changing the existing WASAPI
code.

They aren't completely interchangeable though. Static objects still go
through the spatial renderer, so the result can differ from normal
WASAPI. On headphones, for example, the whole bed can be spatialized,
including stereo. Normal WASAPI is also still useful for exclusive
output, passthrough, and cases where the spatial renderer can't
represent the requested layout.

I think folding this into wasapi makes more sense. The spatial path
could be another part of layout negotiation, used when it can preserve
a layout that normal WASAPI would otherwise remix or downmix. I'm not
sure it should replace normal WASAPI for layouts that are already
represented correctly, since that can change the rendering.

[kasper93]

I think folding this into wasapi makes more sense. The spatial path
could be another part of layout negotiation, used when it can preserve
a layout that normal WASAPI would otherwise remix or downmix. I'm not
sure it should replace normal WASAPI for layouts that are already
represented correctly, since that can change the rendering.

That's the main concern. I don't know either. It could be an option to use/prefer spatial even for standard layouts. But frankly I don't know at this point, I would have to dig into it. I also don't want ao_wasapi.c becoming a mess, if those two apis are really not sharing much of the code.

In short, we need to think about the nominal cases. --no-config and playing back various of files. One strategy could be ao=wasapi-spatial,wasapi where wasapi-spatial would reject things that "are not spatial" or based on some options. This way standard things would got into WASAPI. There are two issues here, it probes spatial thing always, which for vast majority of files is not really needed and like already said it needs to decide based on options if it should be used. Which makes it complicated for users.

Food for thought, I myself not sure yet, what is the best approach here.

[Kajcsi]

I also don't want ao_wasapi becoming a mess if those two apis are really not sharing much of the code.

OpenAL Soft has a useful example for this. It keeps one WASAPI backend,
but the plain and spatial paths are separate implementations internally.
Its spatial-api option is off by default; when enabled it tries the
spatial path first and falls back to normal WASAPI if that fails.

I think that could work here too. The spatial implementation could stay
separate internally, but be exposed through wasapi behind an opt-in
--wasapi-spatial option. --no-config would keep the current behavior
and wouldn't probe the spatial API at all. Exclusive mode and passthrough
would stay on the normal path.

I don't think channel layout alone is enough to select it automatically,
since the same standard layout can work through both paths while producing
different rendering. For now I'd leave spatial as an explicit preference
rather than trying to guess. Does that sound reasonable?

[kasper93]

Hmm, technically Windows Audio Session API is IAudioClient and Microsoft Spatial Sound ...API is ISpatialAudioClient. So they don't really call it WASAPI anymore. But I never seen it refered as WSSAPI or MSSAPI :D

I don't think channel layout alone is enough to select it automatically, since the same standard layout can work through both paths while producing different rendering. For now I'd leave spatial as an explicit preference rather than trying to guess.

If it's explicit preference, separate AO is fine too (maybe not even called wasapi). So whether it's separate AO or an option, doesn't matter for user. The end result would be the same. So, the only consideration is code duplication and overall organization. It sounds like it will be easier with separate AO, but if there are shared code and functions they should be shared in common code.

[Kajcsi]

Yeah that makes sense. Most of the code is specific to ISpatialAudioClient, so Ill keep it as a separate AO.
Device enumeration and selection are already shared with wasapi, the other obvious duplicate is opening the selected IMMDevice, so I'll share that helper too.

[garoto]

Peanut gallery here: cheers for the wall-of-text edit.