how to assign a context to a socket or an epoll event

Question

I want to write an event based server using epoll.

each client has a distinct request, and the server should respond to them. the server will wait for connections, and when connections are available they're queued for read. data is read from the clients and they'll be queued for write. after processing the data, an appropriate response should be sent to each.

all the operations will be asynchronously.

the problem is, how can I determine, which response is for which socket when the sockets are ready for writing?? one way, I can store a (socket, data) tuple, but it's kind of bad programming.

I wonder if I could assign a context to each socket, or each epoll event, so I could determine which data belongs to which socket.

any idea?

is there any suggestions on using SIGIO instead of epoll? if I could assign a context to a file descriptor, or to a signal (I'm not familiar with linux programming) then I could sleep indefinitely and wait for signals...

now forget about networking, look at this example, I open a pre-created FIFO, and pause the thread till I get a SIGIO, in another case, consider I opened 10 FIFO s and assigned a random number to each, when I want to print that number to console, somehow I must be able to retrieve the number, maybe I can assign a context to the file descriptor?

#include <stdlib.h>
#include <stdio.h>
#include <sys/epoll.h>
#include <fcntl.h>
#include <signal.h>
#include <string.h>
static void sigioHandler(int sig)
{
}

int main()
{
    int fd, epfd, ret, i, nr_events, flags;
    struct sigaction sa;
    struct epoll_event event, *events;
    char buf[10];
    memset(buf, 0, 10);
    sa.sa_flags = SA_RESTART;
    sa.sa_handler = sigioHandler;
    if (sigaction(SIGIO, &sa, NULL) == -1)
    {
        perror("sigaction");
        exit(1);
    }
    events = malloc (sizeof (struct epoll_event) * 10);
    if (!events) {
          perror ("malloc");
          return 1;
  }

    fd = open("/tmp/foo", O_RDONLY);

    if(fcntl(fd, F_SETOWN, getpid())==-1){
        perror("own");
        exit(1);
    }
    flags = fcntl(fd, F_GETFL);
    if(fcntl(fd, F_SETFL, flags | O_ASYNC | O_NONBLOCK)==-1){
        perror("set");
        exit(1);
    }
    read(fd, buf, 10);
    epfd = epoll_create(10);
    if(epfd<0)
        perror("epoll_create");

    event.data.fd = fd;
    event.events = EPOLLIN | EPOLLET;

    ret = epoll_ctl(epfd, EPOLL_CTL_ADD, fd, &event);
    if(ret)
        perror("epol_ctl");
    while(1){
        pause();
        nr_events = epoll_wait (epfd, events, 10, -1);
        if (nr_events < 0) {
            perror ("epoll_wait");
            free (events);
            return 1;
        }

        for (i = 0; i < nr_events; i++) {
            if(events[i].events & EPOLLIN)
            {
                read(events[i].data.fd, buf, 10);
                if(buf[0] == '#')
                    goto end;
                printf("%s", buf);
            }
        }
    }
end:
    free (events);

    close(epfd);
    close(fd);
    return 0;
}

changed it a little bit:

static void sigioHandler(int status, siginfo_t *ioinfo, void * context)
{
    if(ioinfo == NULL)
        return;

    switch (ioinfo->si_code)
    {
        case POLL_IN:
            printf("signal received for input chars.sig:%d -%d\n",status, ioinfo->si_code);
            break;

        case POLL_OUT:
        default:
            printf("signal received for something else.sig:%d -%d\n",status, ioinfo->si_code);
            break;
    }
}

in main:
...
sa.sa_sigaction = sigioHandler;
...

I get a strange Segmentation Fault.

don't know if FreeBSD's "mac_set_fd(int fd, mac_t label);" is related to this problem.

Answer 1

The epoll_event structure that you pass to epoll_ctl(), and which is filled in by epoll_wait(), has a data_ptr field.

Answer 2

As Benito has pointed out the epoll_event structure passed into epoll_ctl has a data_ptr field. To be totally correct the field with which you create context is defined as

typedef union epoll_data 
    void        *ptr;
    int          fd;
    uint32_t     u32;
    uint64_t     u64;
} epoll_data_t;

i.e. a union. Often the fd field is used for the context in many examples. But I've found the ptr field to be of most use in C++ where it can be used to point to an object containing all the state for the socket or connection.

So to if you had an object like:

class Connection
{
  private:
    int m_fd;

  public:
    Connection(int fd) : m_fd(fd) {}

    // Other methods like Send/Receive
};


/////
// Somewhere in your server accept loop for a server
struct sockaddr sin;
socklen_t len = sizeof(sin);
int fd = accept(s_listen, &sin, &len);
if (fd == -1) {
    // handle error
}

// or if it's a client you'd have just created the socket with a call to socket()

// and then you add the new connection to epoll.
epoll_event ev;
ev.data.ptr = new Connection(fd);
ev.events = EPOLLIN | EPOLLOUT | EPOLLRDHUP;    
if (epoll_ctl(epfd, EPOLL_CTL_ADD, fd, &ev) < 0) {
    ; // handle error
}

Later when you get epoll events with epoll_wait, you can access the event.data.ptr and cast it back to a Connection type and call a default method to handle the event whatever it is. The class can then call appropriate methods to complete the work such as read/write.

Answer 3

About your first issue I've tried such a thing and it worked. For each of connections you have with other peers (assume you're using TCP) you have a new file descriptor. So each connection is unique by a descriptor.

In my project I've used select. You have to add descriptors which are result of connect() using FD_SET. For further information refer to Linux manual pages of select.