Product Redirect Loops

Question

We have a Magento website (1.6.2) on a LEMP server which we have had running for just over 2 years. Recently we made a few changes;

• Changed the whole site to HTTPS.
• Created all new categories and got rid of the old ones but kept the same products

• Created server redirects in the form;

  location /this/old/category { rewrite ^(this/old/category)/?(.*)$ https://oursite.co.uk/new/category$2 permanent; }

This works for both category pages and any products within them and we thought we were all good!

However Google WMT started showing a lot of unfollowed URLs so we investigated and found that many products are being redirected strangely and resulting in redirect loops.

In Chrome's network tab I can see that when you go to the old URL of certain products the server redirect kicks in and you get the proper response. However, there then follows (seemingly infinite) redirects which take the format of;

random/old/category/random/old/category/random/old/category/random/old/category/random/old/category/random/old/category/random/old/category/random/old/category/random/old/category/product-name

Basically it adds "random/old/category" with each request, on and on. Our server hosts are adamant that is is a) Magento (and so not their problem) and b) should have a preceding slash.

I can not find any settings which would cause this.

• "use web server redirects" is set to yes.
• I have used "grep" on a text dump of our database to try and find any occurence of one of the affected URLs (with no joy, and I am sure this would not be the case due to point 1).
• I have also manually been through the admin area with a fine toothcomb and can not see any possible causes.

I hope this is enough information. Can anyone tell me where a redirect might be happening other than on the server?

UPDATE:

As Douglas pointed out, it would seem the new URL is being redirected also but I have checked the .conf file for the site (and others) and there is no rule which would cause this.

UPDATE2:

I have had a slight breakthrough. It seems that when Magento can not find the product in the category it is redirected to, it will go round and round in a loop. Why it does this rather than just a 404 is a mystery to me so help on that would be good too!

UPDATE3:

3 years, a server move, a whole lot of experience and an upgrade to CE 1.9.3.8 later I think I have made a breakthrough!

I now know for certain that this is a Magento routing problem, as when adding a die(); line in index.php the redirect loop does not occurr.

I added this logging code (which can be found in multiple places on the internet) to the Mage_Core_Controller_Varien_Front class:

Mage::log('----Matching routers------------------------------');
Mage::log('Total ' . count($this->_routers) . ': ' . implode(', ', 
array_keys($this->_routers)));
while (!$request->isDispatched() && $i++<100) {
    Mage::log('- Iteration ' . $i);
    $requestData = array(
        'path_info' => $request->getPathInfo(),
        'module' => $request->getModuleName(),
        'action' => $request->getActionName(),
        'controller' => $request->getControllerName(),
        'controller_module' => $request->getControllerModule(),
        'route' => $request->getRouteName()
    );

    $st = '';
    foreach ($requestData as $key => $val) {
        $st .= "[{$key}={$val}]";
    }
    Mage::log('Request: ' . $st);
    foreach ($this->_routers as $name => $router) {
        if ($router->match($this->getRequest())) {
            Mage::log('Matched by "' . $name . '" router, class ' . 
    get_class($router));
            break;
        }
    }
}

From a bit of research and understanding the code in this class; Magento looks for a router from the system (either core or modules) to take responsibility for the $request and once one matches it will send a dispatch signal.

With the logging code above I got a list of Routers but no information in the $requestData array suggesting that the router was never dispatching.

In each listed router I went to the Controller/Router.php file and found the match(Zend_Controller_Request_Http $request) method. In the lines before this method returned true (signifying a match) I added a simple echo line, E.G:

echo "matched in the xxx module"; die();

This resulted in me finding the router responsible and I suspect that is not dispatching the request after matching and hence it just keeps looping. The module responsible is MageWorx_SeoSuite, I suspect it has been modified by a third party in the past since I can find no references to this online. More investigation is needed which I'll do as soon as I can.

Answer 1

It turns out that the match(Zend_Controller_Request_Http $request) in the MageWorx_SeoSuite Router had a switch statement which resolved CMS, Rss, Category, Category_layer and other routes.

The problem was with the default (read: product) case of this switch statement. It was quite a unique situation (which explains why it only affected a handful of products).

It would;

• Check if the end of the url (i.e: the product urlKey) had a rewrite and if not;
• Check if the lowest level category of said product had a rewrite and if not;
• Load the product by ID and fetch the canonical Url from the MageWorx_SeoSuite helper class;
• Send a redirect to the canonical Url and update the response using an earlier defined HTTP response code depending on the configuration settings.
• exit();

If either of the first two conditional statement were false; the method would return false; (I.E: router did not match). However, if both of the first two conditionals evaluated to true and some other checks passed (canonical Url was not empty etc.); the statement would send the redirect and then simply exit();.

I'm new to the whole routing methology but I beleive there were a few things wrong with this;

1. The method did not return a boolean for all routes of the conditionals (E.G if a product and it's imediate category had no rewrite).
2. The method would "match" but was never dispatch'd.
3. This is based on my limited (undergraduate) knowledge but: this seems like really bad software design.

It's worth pointing out at this time, because I don't want to slate MageWorx without the facts: I am quite sure this code has been altered in the last 5 years (whereby which several third party companies could have been responsible).

In summary: I commented out the default: case statement in the switch so that the Magento core takes responsibility. The culprit Urls now go to a 404, no-route page, which is ideal as they are all old non-existent paths. I would not usually alter module code in this hap-hazard manner but I have documented this well and we are in a fortunate position that the new version of our website is under development, so the "fix" is temporary.