How should a proper audio system look like?

[Snarky]

As it seems like AGS 4.0 is moving closer to release, I would like to revive this thread, as I feel an improved audio API should be part of that revision.

I think the basic idea of getting rid of AudioChannel in favor of something tied to the playback instance, and guaranteed to be non-null, is probably the right approach. The API might not need to change all that much, but there are a few different things to consider:

- AudioClip
- Voice clips
- Frame-linked audio (esp. on looping animations)
- AudioType
- Playback with repeat

For example, with a looping or repeating sound, is it always the same playback instance, or a new one each time? And how would you configure frame-linked audio (volume, etc)? If there was a persistent playback instance, that might offer a good API.

I think @Crimson Wizard has also sometimes mentioned more advanced effects, like changing playback speed or adding echo. I would suggest that this shouldn't be a priority to implement at this time, but it might be worth keeping in mind how it might be added in future.

[Crimson Wizard]

I think the basic idea of getting rid of AudioChannel in favor of something tied to the playback instance, and guaranteed to be non-null, is probably the right approach.

Not ready to give a full reply right now, just couple of quick notes.
I am not certain about getting rid of AudioChannel is a good thing or not. It may be a good thing to not have a channel as a return value from Play(), but at the same time I found audio channels to be an interesting logical concept in how multiple simultaneous playbacks are organized.

In regards to a "null pointer" problem, there's something that hit me recently, and I wonder why did not I think about this earlier. What if Play() returned a dummy audio playback? There are 2 reasons for Play to fail: failure to decode and failure to play, the former happens when the file cannot be opened at all, and the latter when the sound device is not working, or game set to use "no driver".
We are mostly interested to prevent script errors in the second case, because the first case will be noticed immediately during the game development. In audio software that works with filter chains there's a concept of "null" output, where the processed sound is simply discarded. So what we could do is to process the sound internally (which will even update its Position property), but then discard it through such "null" output. I did not check yet, but it's possible that our sound library supports that already.

[Snarky]

Not ready to give a full reply right now, just couple of quick notes.
I am not certain about getting rid of AudioChannel is a good thing or not. It may be a good thing to not have a channel as a return value from Play(), but at the same time I found audio channels to be an interesting logical concept in how multiple simultaneous playbacks are organized.

I think if there is something like an AudioPlayback instance for everything that's currently playing, that will basically take the place of AudioChannels. You might still want a way to iterate through all the playing sounds (for example if you need to adjust the overall volume), but I'm not sure a fixed list of AudioChannels (that exist regardless of whether anything is playing on the channel or not) is the right way to do that if the limit on simultaneous playback is removed.

AudioTypes seem more useful to me as a way to easily set up game logic for sound. MaxChannels would still help control whether a track replaces what is currently playing or not, for example. It's also important to consider how speech will work. It would be good if the system could support voiced background speech; some way to control the speech "channel(s)"/get speech playback instances would be needed.

In regards to a "null pointer" problem, there's something that hit me recently, and I wonder why did not I think about this earlier. What if Play() returned a dummy audio playback? There are 2 reasons for Play to fail: failure to decode and failure to play, the former happens when the file cannot be opened at all, and the latter when the sound device is not working, or game set to use "no driver".
We are mostly interested to prevent script errors in the second case, because the first case will be noticed immediately during the game development. In audio software that works with filter chains there's a concept of "null" output, where the processed sound is simply discarded. So what we could do is to process the sound internally (which will even update its Position property), but then discard it through such "null" output. I did not check yet, but it's possible that our sound library supports that already.

I think it's clear that if it must never return null but playback may fail for whatever reason, it must then return a "dummy." If I understand you correctly, though, what you're saying is that the dummy may still "pretend" to play, specifically in terms of reporting playback position. Yes, I think that could be useful because other game logic may depend on it (use it for timing, for example, like if a cut-scene has been designed to sync to a music track), but at the same time I think it's important that it should be possible to tell somehow that playback has failed.

I think there may be cases where the file fails to open/play but this isn't known at design time (e.g. if using AudioClip.GetByName() or Game.PlayVoiceClip(), or using an external AUDIO.VOX file; perhaps a way to load audio files from the file system or even streaming will also be reintroduced in the future), but there is already an AudioClip.IsAvailable property that can deal with that, if the AudioClip isn't simply null.