Allow connections to only a specific URL via HTTPS with iptables, -m recent (potentially) and -m string (definitely)

Question

Let's say that, for example, I want to allow connections only to subdomain.mydomain.com; I have it partially working, but it sometimes gets in a freaky loop with the client key exchange once the Client Hello is allowed. Ah, to make it even more annoying, it's a self-signed certificate, and the page requires authentication, and HTTPS is listening on a non-standard port... So the TCP/SSL Handshake experience will differ greatly for many users.

Is -m recent the right route? Is there a more graceful method to allow the complete TCP stream once the string is seen?

Here's what I have so far:

#iptables -N SSL
#iptables -A INPUT -i eth0 -p tcp -j SSL
#iptables -A SSL -m recent --set -p tcp --syn --dport 400
#iptables -A SSL -m recent --update -p tcp --tcp-flags PSH,SYN,ACK SYN,ACK --sport 400
#iptables -A SSL -m recent --update -p tcp --tcp-flags PSH,SYN,ACK ACK --dport 400
#iptables -A SSL -m recent --remove -p tcp --tcp-flags PSH,ACK PSH,ACK --dport 400 -m string --algo kmp --string "subdomain.mydomain.com" -j ACCEPT

Yes, I have tried to get around this with nginx tweaks, but I can't get nginx to return a 444 before the client hello, if you can think of a way to achieve this instead, I'm all ears, err, eyes.

Answer 1

As I mentioned on serverfault, I have found a solution:

iptables -A INPUT -p tcp --tcp-flags FIN,PSH,ACK PSH,ACK --dport 400 -m connbytes --connbytes 0:512 --connbytes-dir original --connbytes-mode bytes -m string --algo bm ! --string "subdomain.mydomain.com" -j REJECT --reject-with tcp-reset
iptables -A INPUT -i eth0 -p tcp --dport 400 -j ACCEPT