How to deal with erroneous length headers for packets in TCP stream in Erlang?

Question

I'm receiving messages over TCP in Erlang using gen_tcp. The stream is divided into packets by using a 4 byte length header, as specified by the {packet, 4} option. This is how I call gen_tcp:listen/2:

gen_tcp:listen(Port, [binary, inet, {active, once}, {packet, 4}]).

As you can see, I use the {active, once} option so I can fetch my packets from the process mailbox without flooding it. This works fine as long as the length header is correct. If it's not, anything can happen. So I want to deal with the possibility of erroneous packets somehow.

This is a bit tricky as I'm actually dealing with a stream. Ignoring erroneous packets would be okay, but how do I get Erlang to skip these packets and recognize following packets? How is this problem usually dealt with?

Is it better to use a delimiter? I've looked at some other packet options for gen_tcp. In particular the following:

asn1 | cdr | sunrm | fcgi | tpkt | line

The only one I really understand the meaning of is line, but I don't think that would be a good option. I'm expecting to recieve packages sent and constructed in Objective C, which I'm not familiar with, containing many different kinds of data, not only strings.

Answer 1

I don't think you can skip "bad" packets when using {packet,N}

Main reason being, what's a bad packet?

any 4 bytes can indicate size, so if someone sent: Hello\n

you actually get: 0x48656c6c as the size header, indicating the frame is: 1214606444 bytes long followed by "o\n" and then you'd hang

You can set {packet_size,1024} (or whatever) to keep you from allocating 1.2gb on "hello"

Not sure if that returns an error or drops the socket.

There's really no way to recover from bad framing in TCP since messages are arbitrarily split up in transit.

if you don't use {packet,N} then you'll have to do your own segmentation control, say you get lines of text, you need to know when a message ends.

if you control the client/server protocols, you may want to use {packet,http} or a more full fledged we solution like cowboy

Then you have the client submit http requests (body can still by binary), and the framwork deals with malformed requests for you.

If you need a bi-directionaly channel, websockets might still be ok (assuming you have a websocket lib in objective c)