Scripting an HTTP header request with netcat

Question

I'm trying to play around with netcat to learn more about how HTTP works. I'd like to script some of it in bash or Perl, but I've hit upon a stumbling block early on in my testing.

If I run netcat straight from the prompt and type in a HEAD request, it works and I receive the headers for the web server I'm probing.

This works:

    [romandas@localhost ~]$ nc 10.1.1.2 80
    HEAD / HTTP/1.0

    HTTP/1.1 200 OK
    MIME-Version: 1.0
    Server: Edited out
    Content-length: 0
    Cache-Control: public
    Expires: Sat, 01 Jan 2050 18:00:00 GMT

    [romandas@localhost ~]$

But when I put the same information into a text file and feed it to netcat through a pipe or via redirection, in preparation for scripting, it doesn't return the headers.
The text file consists of the HEAD request and two newlines:

HEAD / HTTP/1.0

Sending the same information via echo or printf doesn't work either.

$ printf "HEAD / HTTP/1.0\r\n"; |nc -n 10.1.1.2 80
$ /bin/echo -ne 'HEAD / HTTP/1.0\n\n' |nc 10.1.1.2 80

Any ideas what I'm doing wrong? Not sure if it's a bash problem, an echo problem, or a netcat problem.

I checked the traffic via Wireshark, and the successful request (manually typed) sends the trailing newline in a second packet, whereas the echo, printf, and text file methods keep the newline in the same packet, but I'm not sure what causes this behavior.

Answer 1

You need two lots of "\r\n", and also to tell netcat to wait for a response. printf "HEAD / HTTP/1.0\r\n\r\n" |nc -n -i 1 10.1.1.2 80 or similar should work.

Answer 2

Another way is to use what is called the 'heredoc' convention.

$ nc -n -i 1 10.1.1.2 80 <<EOF
> HEAD / HTTP/1.0
>
> EOF

Answer 3

Another way to get nc to wait for the response is to add a sleep to the input. e.g.

(printf 'GET / HTTP/1.0\r\n\r\n'; sleep 1) | nc HOST 80

Answer 4

You can use below netcat command to make your instance webserver:

MYIP=$(ifconfig eth0|grep 'inet addr'|awk -F: '{print $2}'| awk '{print $1}')
while true; do echo -e "HTTP/1.0 200 OK\r\n\r\nWelcome to $MYIP" | sudo nc -l -p 80 ; done&

Answer 5

This line will also work as equivalent:

echo -e "HEAD / HTTP/1.1\nHost: 10.1.1.2\nConnection: close\n\n\n\n" | netcat 10.1.1.2 80