RST, ACK after sending huge portion of data

Question

I have an IMAP server (Dovecot), on which I am trying to create 1,200 mailboxes (for performance testing). The server successfully creates the mailboxes.

After this operation I then want to list all created folders. The server sends some data, however, after some time (nearly 1 second) the CLIENT sends RST, ACK to the server in response to the server responding with IMAP protocol's command about the list of created folders.

Here is my Wireshark dump snippet:

IMAP: Src Port: imap (143), Dst Port: 56794 (56794), Seq: 29186, Ack: 20533, Len: 24
IMAP: Src Port: 56794 (56794), Dst Port: imap (143), Seq: 20533, Ack: 29210, Len: 15
IMAP: Src Port: imap (143), Dst Port: 56794 (56794), Seq: 29210, Ack: 20548, Len: 16384
TCP: 56794 > imap [ACK] Seq=20548 Ack=45594 Win=49408 Len=0 TSV=3940902 TSER=3940902
IMAP: Src Port: imap (143), Dst Port: 56794 (56794), Seq: 45594, Ack: 20548, Len: 16384
TCP: 56794 > imap [RST, ACK] Seq=20548 Ack=61978 Win=49408 Len=0 TSV=3940902 TSER=3940902

Edit: Well, I think I figured out why RST flag is sent by client. The reason is server exceed MTU value for my loopback interface. I have checked similar behavior for sample Mina server - and all is OK there, i.e. huge packets are spited by TCP/IP protocol. So Dovecot can't manage packets wisely. But I have my own IMAP server (based on MINA) and the problem still persist there!

So why TCP/IP protocol manages sent packets (split them according to MTU value) wisely only for some apps but not for all?

Answer 1

Your assumption about the causation of a TCP Reset being sent is incorrect. If you've exceeded the MTU, that's not managed by TCP. It's managed at the IP layer and an ICMP "fragmentation required" message will be sent to the client. Such a message should then cause the client to fragment the packets at the IP layer. This is not happening in your case based on the information that you've shared.

Regarding the loopback interface, this traffic doesn't go anywhere near the loopback interface, is it not two separate devices?

Sadly, your trace file still doesn't offer any insight into why this packet -

IMAP: Src Port: imap (143), Dst Port: 56794 (56794), Seq: 45594, Ack: 20548, Len: 16384

causes a TCP reset. There's nothing really further that I can deduce from this information.

TCP has an option called Maximum Segment size, which is similar but also different :) The TCP/IP stack is independent of applications and does not apply different settings to each application, it's system-wide.

Edit: Looking at you packet capture, there's nothing indicative of a MTU issue. There's no ICMP traffic anywhere so I suspect that it's not an issue. If there's a MTU issue, it should have happened in the previous response because both LIST responses from the IMAP server are of identical size and there's no issue with the window sizes.

The only thing that I can see is in relation to the first element of the final response (before the RST) where part of the reply looks to be malformed (see the attachment). There's something going wrong in the IMAP application and the data it's replying with is malformed. - see the difference with the bottom two responses that are consistent with all other LIST responses in the pcap.