LibEV how to correctly buffer?

Question

How can I properly create a write buffer? Currently, I am using a linked list of allocated buffers but most of by writes are only a few bytes. What are your suggestions on creating an efficient write buffer?

Answer 1

Linked list is probably not the best choice here. Its traversal is not cache friendly and thus is expensive.

You can use a ring buffer [1], some consumer will put raw bytes into it and some producer will later grab the whole available content and send it in one syscall.

If writing to a continuous memory region is not desirable (e.g. you can't do it in zero-copy fashion), you can put struct iovec referencing your memory into the ring buffer. Consumer will then grab all iovec's and pass them to writev [2].

Speaking of producers and consumers, they might be OS level threads, and you will have to synchronize them around you buffer, or they might be fibers [3]. The latter is preferable if you are doing an event driven application. I have an implementation [4] of fibers specifically for libev that you can look at. Also it includes efficient virtual ring buffer implementation.

References:

1. http://en.wikipedia.org/wiki/Circular_buffer
2. http://man7.org/linux/man-pages/man2/readv.2.html
3. http://en.wikipedia.org/wiki/Fiber_(computer_science)
4. https://github.com/Lupus/libevfibers