ioctl fails when trying to get more than 8 queue file descriptor from a tun interface

Question

After I heared that after kernel 3.8 linux added multi queue ability to tun tap device with flag IFF_MULTI_QUEUE, I upgraded my kernel to 3.10 and have its header in /usr/src then I altered my c code to have a thread to open a new queue file descriptor each time neaded. but the thread could just open 8 queue (with my earlier than 3.8 kernel, it could not open queue at all) and after that I get this "Arguments too long" error from ioctl in

ioctl(fd, TUNSETIFF, (void *)&ifr)

then I wrote another program to test how much queue fd can be opend in my ubuntu 12.10 linux box with this

uname -r
3.10.0-031000-generic 

kernel version in a much simpler program.

//essential
#include <net/if.h>
#include <linux/if_tun.h>
#include <sys/ioctl.h>
#include <fcntl.h>
#include <arpa/inet.h>
//
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/socket.h> 
#include <sys/select.h>
#include <sys/time.h>
#include <errno.h>
#include <stdarg.h>

#include <netdb.h>

#define IFF_MULTI_QUEUE 0x0100
int tun_alloc_mq(char *dev, int queues, int *fds)
{
    struct ifreq ifr;
    int fd, err=-1, i;
    char *clonedev = "/dev/net/tun";

    if (!dev){
        printf("dev");
        return -1;
    }

    memset(&ifr, 0, sizeof(ifr));
    ifr.ifr_flags = IFF_TAP | IFF_NO_PI | IFF_MULTI_QUEUE;
    strcpy(ifr.ifr_name, dev);
    int error=0;

    for (i = 0; i < queues; i++) {

        printf("loop %d\n",i);

        if( (fd = open(clonedev , O_RDWR)) < 0 ) {
            perror("Opening /dev/net/tun");
            error=1;
            break;
          }

          if(ioctl(fd, TUNSETIFF, (void *)&ifr) < 0 ) {
            printf("first error\n");
            error=1;
            close(fd);
            break;
          } 
    }
    if(error==1)
        return -1;
    return 0;
}
int main(int argc, char *argv[])
{

    int* fdsx;
    if(tun_alloc_mq("testmqtun0",20,fdsx)<0)
    {
        perror("tun");
        exit(1);
    }
    return 0;
}

and it turns out this is limited to 8 too. here is the output:

loop 0
loop 1
loop 2
loop 3
loop 4
loop 5
loop 6
loop 7
loop 8
first error
tun: Argument list too long

I tested it on another linux box and it had the same output. So is there a limit to opening more than 8 queue from a tun device in kernel? if so, how to fix that? please help me with that

Answer 1

From the tun driver we can see the following:

112 /* DEFAULT_MAX_NUM_RSS_QUEUES were choosed to let the rx/tx queues allocated for
113  * the netdevice to be fit in one page. So we can make sure the success of
114  * memory allocation. TODO: increase the limit. */
115 #define MAX_TAP_QUEUES DEFAULT_MAX_NUM_RSS_QUEUES
    [...]
500 static int tun_attach(struct tun_struct *tun, struct file *file)
501 {
    [...]
518         if (!tfile->detached &&
519             tun->numqueues + tun->numdisabled == MAX_TAP_QUEUES)
520                 goto out;
    [...]

DEFAULT_MAX_NUM_RSS_QUEUES is defined at linux/netdevice.h as:

2187 #define DEFAULT_MAX_NUM_RSS_QUEUES      (8)

So, 8 is the default value.

Answer 2

if you want to get more than 8 queue file descriptor from a tun interface, you can upgrade your linux kernel to 4.0+, it support 256 queues.