How can I make system calls invoke my SIGSEGV handler when given protected memory?

Question

I'm working on a memory tracking library where we use mprotect to remove access to most of a program's memory and a SIGSEGV handler to restore access to individual pages as the program touches them. This works great most of the time.

My problem is that when the program invokes a system call (say read) with memory that my library has marked no access, the system call just returns -1 and sets errno to EFAULT. This changes behavior of the programs being tested in strange ways. I would like to be able to restore access to each page of memory given to a system call before it actually goes to the kernel.

My current approach is to create a wrapper for each system call that touches memory. Each wrapper would touch all the memory given to it before handing it off to the real system call. It seems like this will work for calls made directly from the program, but not for those made by libc (for instance, fread will call read directly without using my wrapper). Is there any better approach? How is it possible to get this behavior?

Answer 1

You can use ptrace(2) to achieve this. It allows you to monitor a process and get told whenever certain events occur. For your purposes, look at PTRACE_SYSCALL which allows you to stop the process upon syscall entry and exit.

You will have to change some of your memory tracking infrastructure, however, as ptrace operates such that a parent process monitors a child process, and as far as the child is concerned it doesn't have visibility of when a monitored event occurs. Having said that, you should be able to do something along the lines of:

• Setup ptrace parent and child, monitoring (at least) PTRACE_SYSCALL.
• Child process does a syscall; and parent is notified.
• Parent saves the requested syscall info; and uses PTRACE_GETREGS and PTRACE_SETREGS to change child state so instead of calling the syscall; the child process calls the 'memory unprotect' routine.
• Child unprotect's it's memory; then raises SIGUSR1 or similar to tell controlling parent that the memory work is complete.
• Parent catches SIGUSR, uses PTRACE_SETREGS to restore the previouly-saved syscall info and resumes the child.
• Child resumes and executes the orignal syscall.