Does fopen have a size limitation?

Question

I'm having problems using fopen in php.

If I open the desired URL by hand I get:

OK,13:24:34.236 INFO [10] org.mortbay.http.SocketListener - Started SocketListener on 0.0.0.0:4443 13:24:34.236 INFO [10] org.mortbay.util.Container - Started org.mortbay.jetty.Server@52fe85 13:24:39.251 INFO [11] org.mortbay.util.Credential - Checking Resource aliases 13:24:39.251 INFO [11] org.openqa.selenium.server.SeleniumDriverResourceHandler - Command request: getNewBrowserSession[*firefox, http://dev.REPLACED.com] on session null 13:24:39.251 INFO [11] org.openqa.selenium.server.BrowserSessionFactory - creating new remote session 13:24:39.939 INFO [11] org.openqa.selenium.server.BrowserSessionFactory - Allocated session 179f69cb121941e3b45c7312664de890 for http://dev.REPLACED.com, launching... 13:24:40.048 INFO [11] org.openqa.selenium.server.browserlaunchers.FirefoxChromeLauncher - Preparing Firefox profile... 13:24:47.751 INFO [11] org.openqa.selenium.server.browserlaunchers.FirefoxChromeLauncher - Launching Firefox... 13:25:01.751 INFO [11] org.openqa.selenium.server.SeleniumDriverResourceHandler - Got result: OK,179f69cb121941e3b45c7312664de890 on session 179f69cb121941e3b45c7312664de890 13:25:01.861 INFO [18] org.openqa.selenium.server.SeleniumDriverResourceHandler - Command request: windowMaximize[, ] on session 179f69cb121941e3b45c7312664de890 13:25:03.657 INFO [18] org.openqa.selenium.server.SeleniumDriverResourceHandler - Got result: OK on session 179f69cb121941e3b45c7312664de890 13:25:04.267 INFO [11] org.openqa.selenium.server.SeleniumDriverResourceHandler - Command request: open[index.php?module=authsystem&func=logout, ] on session 179f69cb121941e3b45c7312664de890 13:25:11.064 INFO [11] org.openqa.selenium.server.SeleniumDriverResourceHandler - Got result: OK on session 179f69cb121941e3b45c7312664de890 13:25:11.673 INFO [11] org.openqa.selenium.server.SeleniumDriverResourceHandler - Command request: waitForPageToLoad[30000, ] on session 179f69cb121941e3b45c7312664de890 13:25:11.814 INFO [11] org.openqa.selenium.server.SeleniumDriverResourceHandler - Got result: OK on session 179f69cb121941e3b45c7312664de890 13:25:12.407 INFO [11] org.openqa.selenium.server.SeleniumDriverResourceHandler - Command request: type[uname4, Selenium] on session 179f69cb121941e3b45c7312664de890 13:25:12.423 INFO [11] org.openqa.selenium.server.SeleniumDriverResourceHandler - Got result: OK on session 179f69cb121941e3b45c7312664de890 13:25:13.079 INFO [11] org.openqa.selenium.server.SeleniumDriverResourceHandler - Command request: type[pass4, casetester] on session 179f69cb121941e3b45c7312664de890 13:25:13.111 INFO [11] org.openqa.selenium.server.SeleniumDriverResourceHandler - Got result: OK on session 179f69cb121941e3b45c7312664de890 13:25:13.704 INFO [11] org.openqa.selenium.server.SeleniumDriverResourceHandler - Command request: click[rememberme4, ] on session 179f69cb121941e3b45c7312664de890 13:25:13.782 INFO [11] org.openqa.selenium.server.SeleniumDriverResourceHandler - Got result: OK on session 179f69cb121941e3b45c7312664de890 13:25:14.361 INFO [11] org.openqa.selenium.server.SeleniumDriverResourceHandler - Command request: click[//input[@value='Log In'], ] on session 179f69cb121941e3b45c7312664de890 13:25:14.517 INFO [11] org.openqa.selenium.server.SeleniumDriverResourceHandler - Got result: OK on session 179f69cb121941e3b45c7312664de890 13:25:15.095 INFO [11] org.openqa.selenium.server.SeleniumDriverResourceHandler - Command request: waitForPageToLoad[30000, ] on session 179f69cb121941e3b45c7312664de890 13:25:16.001 INFO [11] org.openqa.selenium.server.SeleniumDriverResourceHandler - Got result: OK on session 179f69cb121941e3b45c7312664de890 13:25:16.642 INFO [11] org.openqa.selenium.server.SeleniumDriverResourceHandler - Command request: click[link=REPLACED, ] on session 179f69cb121941e3b45c7312664de890 13:25:16.751 INFO [11] org.openqa.selenium.server.SeleniumDriverResourceHandler - Got result: OK on session 179f69cb121941e3b45c7312664de890 13:25:17.345 INFO [11] org.openqa.selenium.server.SeleniumDriverResourceHandler - Command request: waitForPageToLoad[30000, ] on session 179f69cb121941e3b45c7312664de890 13:25:19.532 INFO [11] org.openqa.selenium.server.SeleniumDriverResourceHandler - Got result: OK on session 179f69cb121941e3b45c7312664de890 13:25:26.204 INFO [11] org.openqa.selenium.server.SeleniumDriverResourceHandler - Command request: retrieveLastRemoteControlLogs[, ] on session 179f69cb121941e3b45c7312664de890

Now, if I open that url using fopen:

echo fopen($myurl, "r");

The content displayed is cut in 1330 chars... Is there a limitation I should change for this whole content to be received as expected?

Answer 1

First of all:

echo fopen($myurl, "r");

Obviously isn't going to work. I assume you meant:

$handle = fopen($myurl, "r");
echo(fread($handle, $some_size));
fclose($handle);

In that case you're doing it wrong. fread() is not guaranteed to return all the data in one call; sometimes you need to call it multiple times to get the whole input. This is especially true if the remote HTTP server uses Transfer-Encoding: chunked.

See the fread() documentation for more information. Quoting the documentation:

When reading from anything that is not a regular local file, such as streams returned when reading remote files or from popen() and fsockopen(), reading will stop after a packet is available. This means that you should collect the data together in chunks as shown in the examples below.

$handle = fopen("http://www.example.com/", "rb");
$contents = '';
while (!feof($handle)) {
  $contents .= fread($handle, 8192);
}
fclose($handle);

A simpler solution would be to use file _ get _ contents(). However this function writes the entire file into a single string, which may or may not hurt performance depending on what you do with the input from the web page.

Answer 2

Try changing the output buffering from ON to a set size. For example

output_buffering = 4096