Epoll in EPOLLET mode returning 2 EPOLLIN before reading from the socket

Question

The epoll manpage says that a fd registered with EPOLLET(edge triggered) shouldn't notify twice EPOLLIN if no read has been done.
So after an EPOLLIN you need to empty the buffer before epoll_wait being able to return a new EPOLLIN on new data.

However I'm experiencing problems with this approach as I'm seeing duplicated EPOLLIN events for untouched fds.
This is the strace output, 0x200 is EPOLLRDHUP that is not defined yet in my glibc headers but defined in the kernel.

30285 epoll_ctl(3, EPOLL_CTL_ADD, 9, {EPOLLIN|EPOLLPRI|EPOLLERR|EPOLLHUP|EPOLLET|0x2000, {u32=9, u64=9}}) = 0
30285 epoll_wait(3, {{EPOLLIN, {u32=9, u64=9}}}, 10, -1) = 1
30285 epoll_wait(3, {{EPOLLIN, {u32=9, u64=9}}}, 10, -1) = 1
30285 epoll_wait(3,  <unfinished ...>
30349 epoll_ctl(3, EPOLL_CTL_DEL, 9, NULL) = 0
30306 recv(9, "7u\0\0\10\345\241\312\t\20\f\32\r\10\27\20\2\30\200\10 \31(C0\17\32\r\10\27\20\2\30"..., 20000, 0) = 20000
30349 epoll_ctl(3, EPOLL_CTL_DEL, 9, NULL) = -1 ENOENT (No such file or directory)
30305 recv(9, " \31(C0\17\32\r\10\27\20\2\30\200\10 \31(C0\17\32\r\10\27\20\2\30\200\10 \31("..., 20000, 0) = 10011

So, after adding fd number 9 I do receive 2 consecutive EPOLLIN events before having recving the file descriptor, the syscall trace shows how I do delete the fd before reading but it should only happen once, one per event.
So either I am not reading the manpage properly or something is now working here.

Answer 1

I think you missed this part of the epoll man page:

Since even with the edge-triggered epoll multiple events can be generated upon receipt of multiple chunks of data, the caller has the option to specify the EPOLLONESHOT flag, to tell epoll to disable the associated file descriptor after the receipt of an event with epoll_wait(2). When the EPOLLONESHOT flag is specified, it is the caller's responsibility to rearm the file descriptor using epoll_ctl(2) with EPOLL_CTL_MOD.

That is: you got two chunks of data arriving in your receive queue before your first read() happened, which means you got two epoll events. It seems like EPOLLONESHOT is what you're after, which will atomically remove the file descriptor from the poll set when an event happens on it (so you won't need to do the EPOLL_CTL_DEL).

Answer 2

Edge trigger simply means (unless you've used EPOLLONESHOT) that you'll get 1 event when something enters the (kernel) buffer.

Thus, if you get 1 EPOLLIN event and do nothing about it, you'll get another EPOLLIN the next time some data arrives on that descriptor - if no new data arrives, you will not get an event though, even if you didn't read any data as indicated by the first event.

Answer 3

Well, to put it succinctly, EPOLLONESHOT just means that if you don't read the data you're supposed to read, they will be discarded.

Normally, you'd be notified with an event for the same data if you don't read them. With EPOLLONESHOT, however, not reading the data is perfectly legal and they will be just ignored. Hence, no further events will be generated.