This is not poor performance... it is deliberate so that clients don't experience re-buffering once playback has begun. Internet radio servers are not built for low-latency.
There are many sources of the delay:
- Capture buffer (<500ms)
- Encoding buffer (~250ms-1s depending on codec)
- Encoder-to-Server Network Transfer and Buffer (<10ms)
- Server buffer (the one you disabled, often set at 1MB which can be a long time depending on bitrate)
- Server-to-Client Network Transfer and Buffer (<10ms-1s typically, longer times for mobile)
- Client buffer (widely variable, but 2s is typical)
- Client playback buffer (<500ms)
The biggest of these as you can see is the server buffer. You have eliminated that, and that is all you can do. If you want something with lower latency, you need a different technology.