Stack overflow in IIRF (a C program, ISAPI)

Question

I am using IIRF - an ISAPI rewrite filter for pretty URL's. I haven't been able to get much help from the developer on these issues. I'm hoping by making some sense of this dump, so I can find the problematic area in the code and rebuild it myself. I am not super familiar with C, but can get around. Do I need to build this with debug symbols to get anything useful out of the dump?

These stack overflow exceptions are happening on our live production web server, so I'm not able to use debug mode to step into this code. What's happening is my application pool processes (w3wp.exe) are receiving this error event:

A process serving application pool 'dotNET Pool' suffered a fatal communication error with the World Wide Web Publishing Service. The process id was '6924'. The data field contains the error number.

Can someone help me make some sense of this data? It's a dump from the IIS Debug Diagnostic tool. How do I interpret it and find the source of the exception? It appears to be an exception inside the 3rd party PCRE regular expressions library.

WARNING - DebugDiag was not able to locate debug symbols for IsapiRewrite4.dll, so the information below may be incomplete.

In w3wp__PID__3760__Date__06_23_2009__ Time_01_29_55PM__916__Second_Chance_Exception_C00000FD.dmp the assembly instruction at IsapiRewrite4!pcre_exec+12f9 in C:\IIRF1.2.15R5\IsapiRewrite4.dll has caused a stack overflow exception (0xC00000FD) when trying to write to memory location 0x00fc2be8 on thread 5

Type of Analysis Performed   Crash Analysis 
Machine Name  WEB
Operating System   Windows Server 2003 Service Pack 2 
Number Of Processors   4 
Process ID   8056 
Process Image   c:\WINDOWS\system32\inetsrv\w3wp.exe 
System Up-Time   0 day(s) 09:26:25 
Process Up-Time   0 day(s) 02:17:00 

Thread 4 - System ID 6624
Entry point   w3tp!THREAD_MANAGER::ThreadManagerThread 
Create time   6/23/2009 11:12:56 AM 
Time spent in user mode   0 Days 0:0:40.906 
Time spent in kernel mode   0 Days 0:0:6.312 

Function                         Arg 1        Arg 2        Arg 3     Source 
IsapiRewrite4!pcre_exec+12f9     08166a27     01b6741f     081669b8    
IsapiRewrite4!pcre_exec+2779     08166a27     01b6746b     081669b8    
IsapiRewrite4!pcre_exec+1660     08166a26     01b6741f     081669b8    
IsapiRewrite4!pcre_exec+2779     08166a26     01b6746b     081669b8    
IsapiRewrite4!pcre_exec+1660     08166a25     01b6741f     081669b8    
IsapiRewrite4!pcre_exec+2779     08166a25     01b6746b     081669b8    
IsapiRewrite4!pcre_exec+1660     08166a24     01b6741f     081669b8    
IsapiRewrite4!pcre_exec+2779     08166a24     01b6746b     081669b8    
IsapiRewrite4!pcre_exec+1660     08166a23     01b6741f     081669b8    
IsapiRewrite4!pcre_exec+2779     08166a23     01b6746b     081669b8    
IsapiRewrite4!pcre_exec+1660     08166a22     01b6741f     081669b8    
[...snip...]

---

Update on this debug process

I believe that I have found the culprit, after tweaking the IIS Debug Diagnostic tools to provide more information, I have found URL's like the following. It appears to be a similar to a SQL injection attack, but I do not believe to be SQL Injection.

[original_url]/Forum+Result:+%E8%F1%EF%EE%EB%FC%E7%EE%E2%E0%ED+%ED%E8 %EA%ED%E5%E9%EC+%22Rifsadasy%22;%EF%E8%EA%F2%EE%EA%EE%E4+%E4%E5%F8%E8 %F4%F0%EE%E2%E0%ED;%E7%E0%F0%E5%E3%E8%F1%F2%F0%E8%F0%EE%E2%E0%EB%E8%F1 %FC+100%25+%28%E2%EA%EB%FE%F7%E5%ED+%F0%E5%E6%E8%EC+%F2%EE%EB%FC%EA%EE +%F0%E5%E3%E8%F1%F2%F0%E0%F6%E8%E8%29;

Has anyone seen this type of attack? Do they know what it is? I've tried to decode this using HEX, Base64 and some others, but I'm not coming up with anything except this ASCII text:

èñïîëüçîâàí+íèêíåéì+"Rifsadasy";ïèêòîêîä+äåøèôðîâàí;çàðåãèñòðèðîâàëèñü+100%+(âêëþ÷åí+ðåæèì+òîëüêî+ðåãèñòðàöèè);

Answer 1

I think the stack overflow is not due to a bug in the rewriter. It is due to a bug in the patterns used in the configuration of the filter configuration.

It is actually easy to create a single regex that loops endlessly, and neither PCRE nor IIRF have a way to prevent that. It is also possible to create logic loops in the rewrite rules, so that you redirect or rewrite endlessly. Again, there's no way for the filter to stop you from shooting yourself in the foot. You have to take care. These risks exist when using any rewriter that depends on PCRE, or any modular regex engine.

The stack overflow is happening in pcre_exec, which is where the regular expression is being executed. It's a degenerate case but one that should be handled in the configuration. A prior rule should have filtered out such invalid cases. Here's a post about using a rewriting filter as a security barrier.

Test early and often. Some people think that because the filter rules are "just configuration", it is not in need of rigorous testing. In general, that's not a safe assumption. Treat the IIRF rules like any code module. You're just using a different language.

Answer 2

PCRE is calling two functions recursively repeatedly, until it runs out of stack space. You can either try to upgrade the Rewrite DLL to one that isn't buggy, or try to change your rewrite rules so that it doesn't hit the PCRE bug.

Answer 3

looks like you've got pcre_exec recursing and using up all of your stack space. I would check and see what regular expressions you are using.

Answer 4

Based on that stack trace at the very bottom, it looks like the program is going into infinite recursion, with two functions that are calling each other without terminating. This will cause a stack overflow exception due to eventually running out of available stack.

Answer 5

Can you figure out the URL that's sending into the infinite recursion? It looks like you have two rewrite rules that are triggering each other. Unless you have the source to IsapiRewrite4.dll, it's not going to help much -- you can get to the assembly code -- but even if you had the source (you could decompile), it won't help, because I think you need to see the URL that got passed.

Did it also dump memory (or could you make it do that). Arg1, Arg2, or Arg3 might be a pointer to the URL?

Answer 6

So I believe I have found the cause error. Though I'm not sure if this is still really a bug in IIRF ISAPI filter? It at least should not iterate through the rewrite functions so many times that it causes a stack overflow.

I can reproduce the error I was seeing in the event log by requesting this URL on my server: [original_url]/Forum+Result:+%E8%F1%EF%EE%EB%FC%E7%EE%E2%E0%ED+%ED%E8 %EA%ED%E5%E9%EC+%22Rifsadasy%22;%EF%E8%EA%F2%EE%EA%EE%E4+%E4%E5%F8%E8 %F4%F0%EE%E2%E0%ED;%E7%E0%F0%E5%E3%E8%F1%F2%F0%E8%F0%EE%E2%E0%EB%E8%F1 %FC+100%25+%28%E2%EA%EB%FE%F7%E5%ED+%F0%E5%E6%E8%EC+%F2%EE%EB%FC%EA%EE +%F0%E5%E3%E8%F1%F2%F0%E0%F6%E8%E8%29;

So I've created a rewrite rule to return a 404 status for this URL.

But I needed to know if this was some type of attack, I cannot be fully sure yet, but here is what I think the encrypted string says. The URL was coming from IP addresses in Russia, so here is what I've done to translate it:

1. Hex -to- ASCII:

   èñïîëüçîâàí+íèêíåéì+"Rifsadasy";ïèêòîêîä+äåøèôðîâàí;çàðåãèñòðèðîâàëèñü+100%+(âêëþ÷åí+ðåæèì+òîëüêî+ðåãèñòðàöèè)

2. Then Cyrillic to Russian:

   использован+никнейм+"Rifsadasy";пиктокод+дешифрован;зарегистрировались+100%+(включен+режим+только+регистрации)

3. Then Russian to English:

   used nickname "Rifsadasy"; piktokod decrypt; registered 100 (mode only)
   or, similarly
   It is used никнейм "Rifsadasy"; пиктокод it is decoded; were registered 100 (the mode only registration is included)