PHP Stream Write Buffer

Question

I am trying to write a large amount of data (~250K) to a stream in a non-blocking fashion.

Abstracting away the complexities and object structure, this is what I am running:

$fp = fsockopen('host', 80);
socket_set_blocking( $fp, false );
fwrite( $fp, $string, $length ); // Where $string is a 250K string

However the data doesn't all get written. Assuming this was PHP's write buffer coming into play, I set stream_set_write_buffer( $fp, 0 ) but that didn't solve the problem either.

I broke my fwrite into chunks of 4096B - and it looks like the client sends 3 complete batches (of 4096 bytes) and ~1500B of the fourth batch. Any and all successive calls to fwrite return 0 bytes written.

Does anyone have any idea how I can queue this data to all be sent out in a non-blocking fashion? If I remove socket_set_blocking( $fp, false ); - it all works fine. So clearly it's an issue with running it asynchronously.

What are your thoughts? Would the sockets extension help here at all? Does it handle buffers differently?

Note: I am intentionally writing this socket transport layer to avoid using curl for various reasons. Using curl_multi() is not an option.

Answer 1

Your problem is almost certainly due to the fact that fwrite operations on non-blocking socket streams can be interrupted by the arrival of new packets. As a result, you can't count on an fwrite to be atomic. In such cases you MUST rely on the return value of your fwrite invocation to tell you exactly how many bytes were written to the stream in that pass and continue writing until all your data is sent.

For example ...

$dataToWrite = 'my data';
$bytesToWrite = strlen($dataToWrite);
$totalBytesWritten = 0;

while ($totalBytesWritten < $bytesToWrite) {
    $bytes = fwrite($mySock, substr($dataToWrite, $totalBytesWritten));
    $totalBytesWritten += $bytes;
}

Obviously, a robust treatment of this problem must also account for situations where the socket connection goes away, etc.

Answer 2

Are you accounting for blocking and retries? I would assume you need to account for EAGAIN/EWOULDBLOCK/EINTR states otherwise you are going to end up blocking yourself.