NAT Implementation

Question

I'm developing an app that performs NAT between 2 network interfaces on a Windows machine, and I have a bit of a problem making sense of what's going on.

There are 2 network interfaces in the system:

• a physical network interface (00:0c:29:bc:4c:11 192.168.133.130) that uses as a gateway 00:50:56:eb:f5:15 - 192.168.133.2 (operated by VMware)
• a virtual TAP network interface (00:ff:15:08:ac:26 192.168.200.100) used by my app that uses as a gateway 03:03:03:03:03:03 - 192.168.200.1 (operated by my app)

What my app does:

• responds to ARP requests to 192.168.200.1 (the virtual gateway)
• performs one-to-one NAT for ICMP, UDP and TCP between the TAP interface and the physical one

To demonstrate what's going on I have attached a .cap file with the packets captured by Microsoft Network Monitor 3.4 (it can also be opened with Wireshark) during a test. The test consist in establishing a TCP connection with google.com:80. First the connection is established directly through the physical interface to prove that there is internet connectivity, and then the connection is established through the TAP interface to test the NAT.

Packet analysis from top to bottom:

• 1 - TCP SYN sent to google.com directly through the physical interface
• 2 - TCP SYN-ACK received from google.com through the physical if
• 3 - TCP ACK sent to google.com through physical if
• 4 - TCP RST-ACK sent to google.com through physical if, to close connection

All good, we have internet connectivity. Now I change the default gateway to be the virtual router operated by my app (192.168.200.1)

• 5 - Windows notices the default gateway changed to 192.168.200.1 and sends an ARP request to 192.168.200.1
• 6 - my app responds that 192.168.200.1 is at 03:03:03:03:03:03
• 7 - TCP SYN sent to google.com through the TAP interface (192.168.200.100) to the virtual router (192.168.200.1)
• 8 - my app performs NAT on the packet (source and destination MAC address are changed accordingly, and IP source address is changed to 192.168.133.130) and sends the packet on the physical interface to 00:50:56:eb:f5:15 (192.168.133.2)
• 9 - no response received, so a second TCP SYN is sent on TAP if
• 10 - my app performs NAT the same way and sends the packet on the physical if
• 11 - again no response, so a third attempt is made
• 12 - my app performs NAT in the same manner

After NAT, the packet that is sent on the physical interface, is nearly identical to the one sent in the first test, and nothing is changed in the network topology. Why is the second TCP connection unsuccessful? It doesn't make any sense.

The program is using WinPcap. Here is the code if interested.

Answer 1

I have checked the captured file and I don't see any difference at TCP level and MAC layer headers (between the successul SYN and the unsuccessful one). Only I see that in the successful SYN, the IP header checksum is 0, probably because the calculation is offloaded to the interface card. In the unsuccessful SYN, the checksum is calculated and correct according to Wireshark. That should not be a problem if the checksum is correct. That's the only difference I see at IP layer.

I would say either google or somebody in the middle is filtering out your next messages.

Now, something weird to me is why when your app closes the socket in stream index 0, it is sending a RST instead of FIN. Maybe some node is trying to prevent a RST attack from your IP address? Is there a way to close the socket and send a [FIN, ACK] message? Can you wait more time before running the second test?

Other weirdness is that in the second test both connections 192.168.200.100-your app (03:03:03:03:03:03) and "your app"-Google use the same source port 49181. That doesn't seem to be a problem because these are two different machines but I would investigate there as a very last resource. I don't think that's a problem though.

Edited:

After you explained the TCP segment is handed over unmodified I realized the problem is that you are not recalculating the TCP checksum!!! Both SYN messages (frame 7 and 8) have the same TCP checksum. If the IP addresses change, TCP checksum has to change.

BTW, this is a very interesting problem to look at.