I finally managed to solve my problem after some experimentation.
Setting a rolloff value that is below 1f seems to keep the sound from fading out after setting the max distance. You'll also want to make sure you're properly setting up your orientation to match your coordinate system; luckily mine was already designed to work with the default one, but be sure you do that. It's very important.
So, to make sure attenuation works correctly with reference and max angles, do the following:
- Set your distance model to
AL_LINEAR_DISTANCE_CLAMPED
- Set your listener data to the correct values (orientation is fully set up)
- Make sure your rolloff is 1f on the source
- Set the reference and max distances however you want on the source
- MAKE SURE the listener gain is not 0f or 1f, only in between. This does not count for sounds though, they can be 0f and 1f. 0f means "master sound is 0" and 1f means "no attenuation" for the listener.
That should be it.
Also, I've seen some talk during my studies of having to normalize your coordinates. It actually doesn't matter, as OpenAL doesn't expect you to do this (you can find this in the documentation around the bottom of page 32)
I hope I've helped someone else by clarifying all of these. Good luck on your own work, internet strangers.