mmap file returns a pointer to an inaccessible place in memory

Question

I have this program that is supposed to mmap a file in read-write mode and be able to edit its contents. Also the file this is written for is about 40-50 GB, so I need mmap64. The problem is, while mmap64 does not return an error, the address it returns is not accessible.

#include <assert.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>
#include <unistd.h>

typedef unsigned long long u64;


void access_test(u64 p, u64 sz)
{
    u64 i;
    char tmp;
    for (i=0; i<sz; i++) {
        tmp = *(char*)(p+i);
    }
}

int main(int argc, char *argv[])
{
    int fd;
    long long int sz, p;
    struct stat buf;

    fd = open(argv[1], O_RDWR, 0x0666);
    if (fd == -1) {
        perror("open");
        return 1;
    }

    fstat64(fd, &buf);
    sz = buf.st_size;
    printf("File size: 0x%016llx\n", sz);

    p = mmap64 (0, buf.st_size, PROT_READ | PROT_WRITE , MAP_SHARED, fd, 0);
    if (p == -1) {
        perror ("mmap");
        return 1;
    }

    access_test(p,sz);

    if (close (fd) == -1) {
        perror ("close");
        return 1;
    }


    if (munmap ((void*)p, buf.st_size) == -1) {
        perror ("munmap");
        return 1;
    }

    return 0;
}

The result of this is on a small file:

$ ./testmmap minicom.log

File size: 0x0000000000000023
[1]    8282 segmentation fault (core dumped)  ./testmmap minicom.log

The same goes for the big one.

Answer 1

Always enable warnings when you compile

Here is the result with warnings enabled:

$ gcc mmp.c  -Wall -g
mmp.c: In function ‘access_test’:
mmp.c:18:10: warning: variable ‘tmp’ set but not used [-Wunused-but-set-variable]
     char tmp;
          ^
mmp.c: In function ‘main’:
mmp.c:36:5: warning: implicit declaration of function ‘fstat64’ [-Wimplicit-function-declaration]
     fstat64(fd, &buf);
     ^
mmp.c:40:5: warning: implicit declaration of function ‘mmap64’ [-Wimplicit-function-declaration]
     p = mmap64 (0, buf.st_size, PROT_READ | PROT_WRITE , MAP_SHARED, fd, 0);

The last two warnings here are extremely important. They say there is no prototype for mmap64. C therefore gives you a default prototype, and it is wrong, at least for the mmap64() call (since the prototype will return an int, which cannot represent a pointer on a 64-bit Linux host)

The argument to fstat64() is a struct stat64 too BTW, which is another issue.

Make the specific 64-bit functions available

If you want to make the fstat64()/mmap64() function available, you need to compile the code with the _LARGEFILE and LARGEFILE64_SOURCE #define, see information here, so you should compile this as e.g:

  gcc -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE mmp.c  -Wall -g

Or use #define _FILE_OFFSET_BITS=64

There is however no need to do this. Just call the normal fstat() and mmap() and #define _FILE_OFFSET_BITS=64 when compiling. e.g.:

   gcc -D_FILE_OFFSET_BITS=64 mmp.c  -Wall -g

This will enable support for large files, and e.g. translate the mmap() call to mmap64() if it is needed (e.g. if you're on a 32-bit host).

If you are trying to mmap() an 50 GB file, you anyway need to be on a 64-bit host, and on a 64-bit Linux host there's no need for any of this - mmap() and fstat() handles large files without any need to do anything.

Use pointers

The next issue is you're assigning the return value of mmap() to an integer. This might happen to work, but the code does look odd because of it. If you want to treat the thing as a char *, assign it to a char *. Don't play tricks with casting pointers around to a 64-bit integer type.

E.g. your access function should be:

void access_test(char *p, u64 sz)
{
    u64 i;
    char tmp;
    for (i=0; i<sz; i++) {
       tmp = p[i];
    }
}

And p should be declared as char *p; in main(), or use uint8_t *p; if you intend to treat the data as binary data.