How to make Apache copy certain requests to another webserver?

Question

I have a busy web server running apache. Now I am interested in certain request like:

http://myserver1/path1/somepage1.html?xxxxxx
http://myserver1/path2/somepage2.html?xxxxxx

What I want to do is to duplicate request like this and forward them to another webserver like:

http://myserver2/request_statistic/

But the original request must be served on myserver1 as they are now. myserver2 is only for research purpose, so I want the duplicated request headers and bodys are just as the original ones.

Can this be done? How?

Thank you.

Answer 1

Where would the response go?

You might try looking at mod_security, which has a number of useful features that would be of use... is your goal security/forensics, or performance analysis?

For performance analysis, I've found it more useful in the past to create a more comprehensive logging format that captures things like response-code, response Location header (for tracking redirects), selected request headers, timing information, etc.

If https is not in use, then you might be better served by something driven by packet-capture. I know that Oracle Real User Information (?) (RUI) works using that principle. For more casual diagnostic sessions, I've often gotten away with the following tcpdump:

tcpdump -s0 -A -p -nn tcp and port 80

That's enough to get the full requests (and responses), it is a little messy, but the data is all there. You can clean it up a bit with a script, such as the following (tcpdump-http-headers-only) -- its not perfect (particularly on a busy server where things get harder to track).

#!/bin/bash
#
# Pass in the output of 'tcpdump -s0 -A ...' to this and it will
# output only the HTTP request headers and response headers.
#
# Cameron Kerr <cameron.kerr.nz@gmail.com>
# 2013-02-14
#
grep --line-buffered -o \
        -e $'GET .*\r' \
        -e $'POST .*\r' \
        -e $'^[A-Z][A-Za-z0-9_-]*: .*\r' \
        -e $'HTTP/1.1 .*\r' \
        -e $'^\r$' \
  | sed --unbuffered -e 's,\r$,,'

Alternatively, you might like to capture them (perhaps in conjunction with the -W, -C or -G options) for later analysis. This can, depending on the cipher used, also work with https connections if the key is provided (useful for Wireshark)