Running code built for x86 unpaged protected mode

Question

I'm currently trying to reverse engineer a program (~30KB, mostly code) that appears to be built for x86 unpaged protected mode. It contains instructions that access physical memory (or memory mapped device registers to be more precise) directly, e.g.:

mov esi, 0FED400000h ; Some device base address
mov eax, [esi+18h]   ; Memory mapped status register of said device

I would like to run said program in QEMU and ideally be able to step through it in a debugger such as gdb.

Now the problem is that the absolute memory addresses encoded in the code result in segmenation faults in any operating system with paging enabled.

I've been thinking of ways to circumvent this:

• Map physical memory into user space by using mmap on /dev/mem and run the program in user space. While this allows access to physical memory, the hardcoded memory addresses in the analyzed program would need to be patched as the mapped address will not be the same as the physical address. This seems hardly feasible due to the large amount of encoded absolute addresses.
• Install a segmentation fault handler that tries to fix the illegal memory access by rewriting the address. Then run the program in user space.
• Write a bootloader that loads the program code from disk into memory, enters protected mode (including setup of the GDT) and jumps to it. As I've never written any comparable code, I'm not even sure this would work.

Are there any other options to run this program I've missed so far? Or do you have any suggestions, additions or experiences with the ideas mentioned above?

Answer 1

The few SINIT modules I've seen have a small real-mode loader part at the entry which parses and jumps into a plain PE file embedded at the end of the module. So, if you don't need to debug the loader, you can extract the PE and then use one of the DOS extenders which support PE files (e.g. HX-DOS, WDOSX) to load and debug it from DOS. Note, however, that the private TXT registers won't be accessible once the module has run and issued the CLOSE-PRIVATE command.

BTW, the 0xFED40000 range seems to belong to Intel's internal TPM (iTPM). See tpm_tis driver in Linux. So you could instead intercept these accesses (with a signal/exception handler) and redirect them to an actual TPM.