Prevent output buffering with PHP and Apache

Question

I have a PHP script which sends a large number of records, and I want to flush each record as soon as it is available: the client is able to process each record as it arrives, it does not need to wait for the entire response. I realize it takes slightly longer for the entire transfer because it needs to be sent in multiple packets, but it still allows the client to start working sooner.

I've tried all the different flush() and ob_flush() functions but nothing seems to help get the data actually sent over the line before the page is finished. I've confirmed that it is not the web browser because I've tested it using telnet.

Answer 1

The only solution that worked for me was to set the output_buffering directive in php.ini to "Off". I didn't want to do this for the entire server, just this one specific resource. Normally you could use ini_set from the PHP script, but for whatever reason php doesn't allow output_buffering to be set in this way (see the php manual).

Well it turns out that if you're using Apache, you can set some php ini directives (including output_buffering) from your server config, including a .htaccess file. So I used the following in a .htaccess file to disable the output_buffering just for that one file:

<Files "q.php">
    php_value output_buffering Off
</Files>

And then in my static server configuration, I just needed AllowOverride Options=php_value (or a larger hammer, like AllowOverride All) in order for that to be allowed in a .htaccess file.

Answer 2

You don't mention what web server you are using, but I am going to go out on a limb here and guess Apache2. I hit almost the identical thing you describe. I was trying to get my cgi script to pass back information as it had it ready, instead of buffering the whole thing. Worked jiffy in curl, etc., but buffered in a browser (pretty much any browser), which was at least maddening. I went through the exact steps you describe. The resolution in my case was to modify sites-enabled/terrifico.com configuration file in Apache2 (the line in question starts with

SetEnvIfNoCase

(You can ignore the stuff above and below that line, I'm just showing it for reference of where I placed it.)

<VirtualHost *:80>
ServerAdmin webmaster@localhost
ServerName test.terrifico.com
ServerAlias test.terrifico.com

SetEnvIfNoCase Request_URI \.cgi$ no-gzip dont-vary

DocumentRoot /var/www/test.terrifico.com

From staring at the network traffic going back and forth, it finally dawned on me that the browser was advertising that it accepted deflation for anything (that was text). That was the difference between the browser and curl, for example. The salient bit was

Accept-Encoding:gzip,deflate,sdch

There was a bit about chunking, but that didn't impact this particular problem. So, the browser was requesting mod_deflate to kick in, which defeated my carefully spewing out bytes as I got them in my cgi script. You could change it in the browser, but it seemed more sensible to change it on the server once for the works.

Perhaps this helps.

Answer 3

To turn off output buffering at run time in PHP without changing php.ini or having a .htaccess file, just use ob_end_flush() or ob_end_clean() at the beginning of the script. For example:

This should output without buffering:

<?php
ob_end_clean();

for ($i = 0; $i < 5; $i++)
{
    echo "$i\n";
    flush();
    usleep(0.5e6);
}

This outputs with buffering (all at a time) if output_buffering is on, regardless of the flush() call:

<?php

for ($i = 0; $i < 5; $i++)
{
    echo "$i\n";
    flush();
    usleep(0.5e6);
}

Despite its name, ob_implicit_flush calls flush(), not ob_flush(), implicitly after every output. This can be handy in this instance after closing the output buffer at the beginning:

<?php
ob_end_clean(); // disable output buffer
ob_implicit_flush(); // call flush() automatically after every output

for ($i = 0; $i < 5; $i++)
{
    echo "$i\n";
    usleep(0.5e6);
}

This fixes the PHP side. There may be something else going on with mod_deflate or similar (see the answer by Ted Collins), and I've observed that Firefox needs at least 1024 bytes before it starts to output anything at all.