Why is raising plackup (or starman) memory usage?

Question

I have this simple PSGI application (app.psgi).

use strict;
use warnings;

my $app = sub {
    my $mem = `ps -o rss= -p $$`;
    $mem =~ s/^\s*|\s*$//gs;
    return [ 200, [ 'Content-Type' => 'text/text' ], [ $mem ]];
};

I was requested the above 1000 times and got increased memory usage. Depending on how was started the server, got:

• plackup - memory usage is raising at first 3 requests and remain constant for the next 997 requests

• plackup -r - memory usage is randomly raising (not at every request) by 4k.

• starman - like above, the memory usage is randomly raising by 4k, but with slower rate

The question is:

• WHY is raising the memory usage ? Where is the leak, and how achieve constant memory usage (especially on starman), because i don't want run out of memory at long run. (OK, it is possible define for example --max-requests 100), but it is not an answer for the memory usage.
• or - what is wrong in my example?

If anyone want test this too - here is my script for the fetching:

use strict;
use warnings;
use LWP::UserAgent;
my $ua = LWP::UserAgent->new;
my $req = HTTP::Request->new(GET => 'http://localhost:5000');

my $old_mem = 0;
print "req#\tmem\n";
foreach my $i (1..1000) {
    my $res = $ua->request($req);
    (my $mem = $res->content) =~ s/\D//g;
    next if( $mem == $old_mem );
    print "$i\t$mem\n";
    $old_mem = $mem;
}

My results:

plackup                 plackup -r              starman
req#    mem             req#    mem             req#    mem
1       7780            1       3924            1       3280
2       7800            2       4296            5       3728
3       7804            3       4304            8       3280
                        ...                     ...
                        ... deleted             ... deleted
                        ...                     ...
                        839     4596            994     3912
                        866     4600            998     3908
                        962     4604            1000    3912

So,

• why plackup raising in first 3 requests?
• plackup -r - 4k increase (see last lines) - at the beginning much more
• starman - raising too, but with default 5 workers in slower rate (3280->3912)

Versions:

# cpanm Plack Starman
Plack is up to date. (0.9979)
Starman is up to date. (0.2010)
# perl -v

This is perl 5, version 12, subversion 3 (v5.12.3) built for darwin-thread-multi-2level

Answer 1

Based on miyagava's comments the answer is:

"plackup - memory usage is raising at first 3 requests and remain constant for the next 997 requests" That means some modules are lazy-loaded in the first few requests. After that there's no leak. – miyagawa 14 hours ago

Starman by default enables keep-alive and HTTP pipelining, meaning if you send 1000 requests in the short period of time you'll have these connections connected, unless you explicitly disconnect them. I can confirm this using ApacheBench - the memory temporarily increases, but when they disconnect/timeout, the memory gets down to where it was. – miyagawa 14 hours ago

thanx.

Answer 2

Did you use the latest version? I cannot reproduce your output.

With "plackup":

sidburn@sid:~/perl/plack$ ./memory.pl 
req#    mem
1   5340
2   5380

With "plackup -r":

sidburn@sid:~/perl/plack$ ./memory.pl 
req#    mem
1   4860
2   5060

With "starman":

sidburn@sid:~/perl/plack$ ./memory.pl 
req#    mem
1   5176
5   5224
6   5176
7   5224

Versions:
Perl: 5.12.1 & 5.12.3
Plack: 0.9979
Starman: 0.2010