How should I handle session files that become too numerous?

Question

I am acting as a sysadmin for a couple of servers that hold Magento sites and occasionally they fill up with session files.

I have been told that managing these files isn't something that can be done from within Magento and I presume their temporary use means they can't just be turned off, but it seems weird that Magento has no way to handle the removal of these files?

My solution is a nightly crontab that performs something like this find /path/to/magento/sessions/ -name "sess*" -type f -delete but it feels inelegant to say the least.

What is the best way to handle these?

Answer 1

Apart from deleting session files with find using a custom modification time as mentioned by others, you also can:

• Save the sessions in your database. Of course this will put load on your database and it's not the fastest way, but you can handle way more sessions that way and you can share sessions between multiple frontend servers. You can change the setting in app/etc/local.xml by switching

  <session_save><![CDATA[files]]></session_save>
  

  to

  <session_save><![CDATA[db]]></session_save>
  

• Use memcached as your session storage. Magento also supports this by default. Have a look at app/etc/local.xml.additional for the configuration. I never used it in production but have heard that this may be a little bit tricky.

• Handle the sessions in Redis using Colin Mollenhours brilliant extension Cm_RedisSession. Setting up Redis shouldn't take up too long, can also be used for caching (see Cm_Cache_Backend_Redis) and combines a RAM cache with persistence on disk (in opposition to memcached, RAM disks or the like) which is always in case your server is crashing.

Answer 2

With file based sessions, they will be auto-pruned by the PHP session clean-up cron – so the files are likely to be deleted within ~7200 seconds of creation. So even on a busy site (30k uniques per day), there usually only around 4,000 session files in ./var/session – which is nothing even for a low-end Linux server.

However, the clean-up actually relies on the cron working - which doesn't normally look in the ./var/session directory of Magento. So you should set up a new system cron

/usr/bin/find /home/myuser/public_html/var/session -mindepth 1 -maxdepth 1 -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 -exec rm {} \; >/dev/null 2>&1

The default clean up period for sessions is 7200 seconds, which should be more than adequate, although you can change the above to suit.

With Memcache sessions, TCP/IP is the only overhead – which for a single-server deployment, would make it slower than file based. So then, you would use a unix socket instead, which removes that overhead and gives better security. But even still, your customer sessions will be truncated/limited as to the amount of RAM you can allocate. The average Magento session is 4Kb – so you’ll be able to support 256 active sessions, per MB you allocate. So be sure to set an appropriate limit to avoid customers randomly losing cart/session. And also bear in mind, a Memcache daemon restart will wipe out all existing sessions (BAD!).

With Redis (not native, but available via an extension), you get a similar level of support as Memcache, but with the added benefits of persistence (should you wish to use it). With the Cm_Redis extension, you'll also be able to take advantage of session compression. We've found this extension works very well on both CE and EE deployments.

The with DB, the default prune expiration setting is a mighty 1 week, so with the above store size as an example (30k uniques per day), you’ll be looking at a DB table size for core_cache_session of around 7GB – which will grind your store to a complete halt, for almost every session based operation.

From experience of hosting both large (230k unique visitors per day) and small (<1k unique visitors per day) stores, our recommendation is:

Single-server deployment - files

Multi-server deployment - Redis (using a separate database from the main Magento cache)

I wrote some really thorough replies here http://magebase.com/magento-tutorials/magento-session-storage-which-to-choose-and-why/comment-page-1/#comment-1980

Answer 3

I've asked a related question some time ago:

https://stackoverflow.com/questions/7828975/php-garbage-collection-clarification

What I never found out (I left that job for a new one, and the original problem became someone else's) is if Magento's sessions will honor these settings, or if they implement their session handling using Zend (and presumably some sort of zend.ini config file).

The php settings to look at:

session.gc_maxlifetime session.gc_probability session.gc_divisor

http://php.net/manual/en/session.configuration.php#ini.session.gc-probability

Answer 4

Usually a cron job is sufficient, but here is a few things to keep in mind:

1) Set the session to last no longer than session.gc_maxlifetime (php -i | grep session.gc_maxlifetime)seconds (this will set up expired sessions to be prepared for garbage collection by the php.ini or .htaccess)

2) You might want to store the sessions in the database see here for more info on how to do this (this option might be easier to manage via custom magento module)

3) Another option to consider is Memcached witch can also speed up the servers (though not completely connected to the question, i think it is useful to know about)

See this question for more info: https://stackoverflow.com/questions/4353875/how-long-do-the-magento-session-files-need-to-be-kept

Answer 5

On all of our setups we have a maintenance.php file which takes care of the cleaning of the logs and var directory once in a while. Since the sessions have to be either saved in the database or on the file system, this maintenance file will clean them up both. (See code below).

You can run the following command as a cron job to clean up the logs:

php maintenance.php clean=log

The above command will produce the following output:

catalogindex_aggregation has been truncated
catalogindex_aggregation_tag has been truncated
catalogindex_aggregation_to_tag has been truncated
catalog_compare_item has been truncated
dataflow_batch_export has been truncated
dataflow_batch_import has been truncated
log_customer has been truncated
log_quote has been truncated
log_summary has been truncated
log_summary_type has been truncated
log_url has been truncated
log_url_info has been truncated
log_visitor has been truncated
log_visitor_info has been truncated
log_visitor_online has been truncated
report_compared_product_index has been truncated
report_event has been truncated
report_viewed_product_index has been truncated

You can run the following command as a cron job to clean up the var folder:

php maintenance.php clean=var

The above command will produce the following output:

downloader/.cache/* has been emptied
downloader/pearlib/cache/* has been emptied
downloader/pearlib/download/* has been emptied
var/cache/ has been emptied
var/locks/ has been emptied
var/log/ has been emptied
var/report/ has been emptied
var/session/ has been emptied
var/tmp/ has been emptied

The actual code (Don't forget to adjust the path to your local.xml file):

<?php
$xml = simplexml_load_file('./app/etc/local.xml', NULL, LIBXML_NOCDATA);

$db['host'] = $xml->global->resources->default_setup->connection->host;
$db['name'] = $xml->global->resources->default_setup->connection->dbname;
$db['user'] = $xml->global->resources->default_setup->connection->username;
$db['pass'] = $xml->global->resources->default_setup->connection->password;
$db['pref'] = $xml->global->resources->db->table_prefix;

if (!isset($argv[1]) || !stristr($argv[1], 'clean=')) {
    echo 'Please use one of the commands below:' . PHP_EOL;
    echo 'php maintenance.php clean=log' . PHP_EOL;
    echo 'php maintenance.php clean=var' . PHP_EOL;
    die;
}

$method = str_replace('clean=', '', $argv[1]);

switch ($method) {
case 'log':
    clean_log_tables();
    break;
case 'var':
    clean_var_directory();
    break;
default:
    echo 'Please use one of the commands below:' . PHP_EOL;
    echo 'php maintenance.php clean=log' . PHP_EOL;
    echo 'php maintenance.php clean=var' . PHP_EOL;
    break;
}

function clean_log_tables() {
    global $db;

    $tables = array(
        'catalogindex_aggregation',
        'catalogindex_aggregation_tag',
        'catalogindex_aggregation_to_tag',
        'catalog_compare_item',
        'dataflow_batch_export',
        'dataflow_batch_import',
        'log_customer',
        'log_quote',
        'log_summary',
        'log_summary_type',
        'log_url',
        'log_url_info',
        'log_visitor',
        'log_visitor_info',
        'log_visitor_online',
        'report_compared_product_index',
        'report_event',
        'report_viewed_product_index'
    );

    mysql_connect($db['host'], $db['user'], $db['pass']) or die(mysql_error());
    mysql_select_db($db['name']) or die(mysql_error());

    foreach($tables as $v => $k) {
        @mysql_query('TRUNCATE `'.$db['pref'].$k.'`');
        echo $db['pref'] . $k . ' has been truncated' . PHP_EOL;
    }
}

function clean_var_directory() {
    $dirs = array(
        'downloader/.cache/*',
        'downloader/pearlib/cache/*',
        'downloader/pearlib/download/*',
        'var/cache/',
        'var/locks/',
        'var/log/',
        'var/report/',
        'var/session/',
        'var/tmp/'
    );

    foreach($dirs as $v => $k) {
        exec('rm -rf '.$k);
        echo $k . ' has been emptied' . PHP_EOL;
    }
}

Answer 6

For Magento CMS and the like (that aren't cleaning old sessions up), I just use cron jobs based on php.ini settings.

PHP5/Ubuntu 14.04/Debian

The system cron.d setup for php5 does not clean Magento ./var/session (or anything besides default session folder (/var/lib/php5 for Ubuntu and /var/lib/php5/sessions or /tmp/ for most others Linux dists).

But you can still use "sessionclean" and "maxlifetime" as per the default php5/Debian system cron:

Example you can try from Command line:

# sudo /usr/lib/php5/sessionclean /var/www/{yoursite}/var/session $(/usr/lib/php5/maxlifetime)

So just incorporate that into a system/root crontab or a crontab of user who has read/write permission for the session files:

$ sudo crontab -e

Add this is you want it to look similar to system php cron:

20,40 * * * * [ -x /usr/lib/php5/maxlifetime ] && [ -x /usr/lib/php5/sessionclean ] && [ -d /var/www/*/var/session ] && /usr/lib/php5/sessionclean /var/www/{yoursite}/var/session $(/usr/lib/php5/maxlifetime)

or - since we know those files/dirs exist:

20,40 * * * * /usr/lib/php5/sessionclean /var/www/*/var/session $(/usr/lib/php5/maxlifetime)

Now I have a manageable amount of sessions and it is kept clean via default garbage collection/lifetime via php.ini (cli) settings.

(You may leave the wildcard above or replace with sitename.)

EDIT (PHP7 / Ubuntu 16.xx / Debian):

The 'sessionclean' script has changed and maxlifetime script has been removed. For the system/php cron job it is now one script. You cannot really use this anymore as the file calls are now static to script.

The older php5 sessionclean script can still work for you if the system isn't cleaning up. What you can do is grab the older Debian php5 Package and extract sessionclean from it. Or you can simply copy this to your scripts area (giving proper /var/www/(site) permissions/ownership) :

#!/bin/sh

# first find all used files and touch them (hope it's not massive amount of files)
[ -x /usr/bin/lsof ] && /usr/bin/lsof -w -l +d "${1}" | awk -- '{ if (NR > 1) { print $9; } }' | xargs -i touch -c {}

# find all files older then maxlifetime
find "${1}" -depth -mindepth 1 -maxdepth 1 -ignore_readdir_race -type f -cmin "+${2}" -delete

I also recommend renaming it, so it's not confused with the new php 'sessionclean' cronjob. You can then plug your own "maxlifetime" number in like so:

     20,40 * * * * /home/-username-/scripts/MySessionClean /var/www/*/var/session 61

(61 being the example age (in minutes) and 'MySessionClean' being the renamed php5 script downloaded or copied from above).

In this manner we avoid php.ini/env calls entirely.

(EDIT 13DEC2016: Updated DEBIAN ARCHIVE REPO LINK)

Answer 7

I cleared the DB regualarly from all these old session files. It was irritating manual work untill I installed Magento Optimizer which makes all this routine work for me. Also, my cache is refreshed constantly and I don't it manually after changing products and static blocks. Oh, yes, error reports and abandoned carts are cleaned as well.

Answer 8

Of all the Comments above, I think this is the easy solution and hope its better than long scripts and installing 3rd party extension to manage old sessions files and keep new session files.

1. Create a file name "clean_session.sh" under your magento folder.
2. Paste these lines.

#!/bin/bash
# delete session files older than 14 days
find /home/DOMAIN/public_html/var/session -name 'sess_*' -type f -mtime +14 -exec rm {} \;

3. Then you can schedule cronjob weekly to perform clean up.

1 1 * * 6 /bin/sh /home/DOMAIN/public_html/clean_session.sh

4. Don't forget to make the file executable as

chmod u+x clean_session.sh

5. And you can also run it as

sh clean_session.sh

Answer 9

For my case, I run this script placed in magento/var/ directory for delete session files more than one week (-mtime +7) old:

#!/bin/sh
# Place this script in magento/var/ directory

for n in `seq 0 9`
  do
    for u in `seq 0 9`
    do
      for m in `seq 0 9`
        do
          name="sess_"$n$u$m*
          echo $name
          find session/ -name $name -type f -mtime +7 -delete
          echo $name ok
      done
      for m in {a..z}
        do
          name="sess_"$n$u$m*
          echo $name
          find session/ -name $name -type f -mtime +7 -delete
          echo $name ok
      done
    done
      for u in {a..z}
      do
        for m in `seq 0 9`
          do
            name="sess_"$n$u$m*
            echo $name
            find session/ -name $name -type f -mtime +7 -delete
            echo $name ok
        done
        for m in {a..z}
          do
            name="sess_"$n$u$m*
            echo $name
            find session/ -name $name -type f -mtime +7 -delete
            echo $name ok
        done
    done
done

for n in {a..z}
  do
    for u in `seq 0 9`
      do
        for m in `seq 0 9`
          do
            name="sess_"$n$u$m*
            echo $name
            find session/ -name $name -type f -mtime +7 -delete
            echo $name ok
        done
        for m in {a..z}
          do
            name="sess_"$n$u$m*
            echo $name
            find session/ -name $name -type f -mtime +7 -delete
            echo $name ok
        done
    done
    for u in {a..z}
      do
        for m in `seq 0 9`
          do
            name="sess_"$n$u$m*
            echo $name
            find session/ -name $name -type f -mtime +7 -delete
            echo $name ok
        done
        for m in {a..z}
          do
            name="sess_"$n$u$m*
            echo $name
            find session/ -name $name -type f -mtime +7 -delete
            echo $name ok
        done
    done
done

It's my first bash script (revision 2) and I think it can be optimized in several aspects. I'm open for any optimization suggestion.

This script can be retrieve at : https://gist.github.com/Nolwennig/a75dc2f8628be2864bb2

Answer 10

I created a script which empties the var/session directory. You can add it to a cron job to run once per day which should be enough and adjust as needed. You'll notice when you session directory gets filled, it's impossible to delete files via cpanel or ssh, this script will do the trick just place in magento root directory.

<?php
function adjustSessionFiles($dir, $pattern = "*")
{
    $files = glob($dir . "/$pattern");
    foreach ($files as $file) {
        if (is_dir($file) and !in_array($file, array('..', '.')))  {
            adjustSessionFiles($file, $pattern);
        }else if(is_file($file) and ($file != __FILE__)) {
            unlink($file);
        }
    }
}
$dir = __DIR__ . DIRECTORY_SEPARATOR . 'var' . DIRECTORY_SEPARATOR . 'session';
adjustSessionFiles($dir);

Answer 11

The session files should be deleted by php using the settings from your php.ini file.

You probably have session.gc_probability = 0 configured in your php.ini file. Which means that 0 in every $session.gc_divisor visitors will trigger the garbage collector.

You can:

a) Change session.gc_probability to 1 in order to make php do the cleaning for you

b) You can create a single php file and add it to your cron (recommended for production) - see PHP official example

Be aware: Solution A is simpler, but...

The A solution is fine if you know how many visitors your site have, so it's very easy to implement. However, if you clean the session too often, or in a very frequent way, this could be a problem. That's why solution B is recommended.

Answer 12

Working Solution...

You can remove by writing a simple Linux command on cron

find /var/lib/php/session -type f -cmin +$(echo "`/opt/php/bin/php -i|grep -i session.gc_maxlifetime|cut -d' ' -f3`"/60|bc) -exec rm -f {} \;

Where

/var/lib/php/session is your absolute session Directory Path
/opt/php/bin/php is PHP path on Server 60 [Minute] is File Life greater than (Means All files, which are not modiefied since last 60 minute)