segfault on second calling of opendir

Question

Some background:

I am writing a module for nginx (web server) that allows for on-the-fly setting of root directories in a shared environment. I have a directory that contains other directories, one for each hosting user; given a directory path relating to a requested domain I need to determine which user it belongs to.

--public_dir
---user_1
---user_2
----com/example/www/_public/
---user_3

www.example.com is converted to com/example/www and the function should return "user_2"

The problem:

I successfully do this by opening public_dir, traversing it, and checking each 1st level directory (users) for the existence of com/example/www/_public/ and caching the result. All runs as expected the first time the function is called. The second time segfaults when attempting to open public_dir again (it has been successfully closed in the previous function call):

#define NGX_SHARED_ENV_SHARED_DIR "/var/www/public/"
#define NGX_SHARED_ENV_PUBLIC_DIR "_public"

char* ownerFromDir(char *dir){
    char *owner;
    char fullPath[strlen(NGX_SHARED_ENV_SHARED_DIR)+strlen(dir)+128];
    struct dirent *e;
    struct stat sb;

// ------------ERROR IN HERE----------------------------

    fprintf(stderr, "Before opendir\n");
    DIR *sharedDir = opendir(NGX_SHARED_ENV_SHARED_DIR);
    fprintf(stderr, "After opendir\n");

// -----------------------------------------------------

    if(sharedDir){
        while((e=readdir(sharedDir))!=NULL){
            //is the entry a directory?
            if(e->d_type!=DT_DIR){ 
                continue;
            }
            if(strcmp(e->d_name, ".")==0 || strcmp(e->d_name, "..")==0){
                continue;
            }
            // build a path to check for /var/www/public/userX/com/example/www/_public
            snprintf(fullPath, sizeof(fullPath), "%s%s/%s/%s", NGX_SHARED_ENV_SHARED_DIR, e->d_name, dir, NGX_SHARED_ENV_PUBLIC_DIR);
            stat(fullPath, &sb);
            //if a matching directory is found then copy the entry name into owner
            if(S_ISDIR(sb.st_mode)){ 
                int len = 1 + (int) strlen(e->d_name);
                owner = (char*) malloc(sizeof(char)*len);
                free(owner);
                memcpy(owner, &(e->d_name), len);
                break;
            }
        }
        closedir(sharedDir);
    }
    return owner;
}

gdb isn't very helpful even when compiling with gcc -g for debug flags

Breakpoint 1, ownerFromDir (dir=0x603450 "com/example") at ngx_module.h:43
43  char* ownerFromDir(char *dir){
(gdb) step
45      char fullPath[strlen(NGX_SHARED_ENV_SHARED_DIR)+strlen(dir)+128];
(gdb) step
48      fprintf(stderr, "Before opendir\n");
(gdb) step
Before opendir
49      DIR *sharedDir = opendir(NGX_SHARED_ENV_SHARED_DIR);
(gdb) step
50      fprintf(stderr, "After opendir\n");
(gdb) step
After opendir
51      if(sharedDir){
(gdb) continue
Continuing.

Breakpoint 1, ownerFromDir (dir=0x60b4a0 "com/example2") at ngx_module.h:43
43  char* ownerFromDir(char *dir){
(gdb) step
45      char fullPath[strlen(NGX_SHARED_ENV_SHARED_DIR)+strlen(dir)+128];
(gdb) step
48      fprintf(stderr, "Before opendir\n");
(gdb) step
Before opendir
49      DIR *sharedDir = opendir(NGX_SHARED_ENV_SHARED_DIR);
(gdb) step

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7a99df2 in ?? () from /lib/x86_64-linux-gnu/libc.so.6

EDIT: backtrace

(gdb) backtrace
#0  0x00007ffff7a99df2 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007ffff7a9b446 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x00007ffff7a9dfc5 in malloc () from /lib/x86_64-linux-gnu/libc.so.6
#3  0x00007ffff7ad651b in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#4  0x0000000000400aea in ownerFromDir (dir=0x60b4a0 "com/example2") at ./ngx_module.h:48
#5  0x0000000000400cb2 in ownerFromDom (domain=0x40116c "example2.com") at ./ngx_module.h:82
#6  0x0000000000400f3f in testOwner (domain=0x40116c "example2.com", expect=0x401150 "user2") at ./ngx_module.c:45
#7  0x0000000000400e27 in runTests () at ./ngx_module.c:29
#8  0x0000000000400d0b in main (argc=1, argv=0x7fffffffe708) at ./ngx_module.c:10

Can anyone please point me in the right direction?

Answer 1

Your code does:

        if(S_ISDIR(sb.st_mode)){
            int len = 1 + (int) strlen(e->d_name);
            owner = (char*) malloc(sizeof(char)*len);
            free(owner);
            memcpy(owner, &(e->d_name), len);
            break;
        }

and that leads to:

return owner;

You've freed the memory so the memcpy() is invalid (accessing memory that you have no right to access) and the return is at the very least dangerous and arguably invalid (you're passing back a pointer to freed memory).

Remove the free() in this code. Who knows what's going wrong in the calling code, but a moderately plausible guess is 'you free() the returned value', which is a double free, which is also a serious no-no.

As written, the memcpy() could easily corrupt your heap; a double free could corrupt your heap. So, with two sources of heap corruption, you're very likely to get a crash somewhere as a result, and opendir() does memory allocation, so it is a plausible victim. If you did a memory allocation before that, it would probably crash instead.