Init Resource Model Not Working - Returning Non Object

Question

I am attempting to extend the Crawler portion of the Enterprise PageCache module, however, I am getting the following error, I believe it has to do with the line $this->_init('vmr_crawler/crawler'); found in my model, I say this because if I switch out vmr_crawler, with enterprise_pagecache, it works... Here is the error

Fatal error: Call to a member function getUrlsPaths() on a non-object in /jlr/sites/mage/app/code/local/VMR/Crawler/Model/Crawler.php on line 37

Any ideas, snippets, comments are greatly appreciated! Thank you so much, I am posting all of my code below.

My Module Config

app/etc/modules/VMR_Crawler.xml

My Module Config Code

<?xml version="1.0"?>
<config>
<modules>
    <VMR_Crawler>
        <active>true</active>
        <codePool>local</codePool>
        <depends>
            <Mage_Core/>
        </depends>
    </VMR_Crawler>
</modules>

My Config

app/code/local/VMR/Crawler/etc/config.xml

My Config Code

<?xml version="1.0"?>
<config>
<modules>
    <VMR_Crawler>
        <version>0.0.1</version>
    </VMR_Crawler>
</modules>
<global>
    <models>
        <vmr_crawler>
            <class>VMR_Crawler_Model</class>
            <resourceModel>vmr_crawler_resource</resourceModel>
        </vmr_crawler>
        <vmr_crawler_resource>
            <class>VMR_Crawler_Model_Resource</class>
        </vmr_crawler_resource>
        <pagecache>
            <rewrite>
                <crawler>VMR_Crawler_Model_Crawler</crawler>
            </rewrite>
        </pagecache>
        <pagecache_resource_crawler>
            <rewrite>
                <attribute>VMR_Crawler_Model_Resource_Crawler</attribute>
            </rewrite>
        </pagecache_resource_crawler>
    </models>
</global>
</config> 

My Model

app/code/local/VMR/Crawler/Model/Crawler.php

My Model Code

<?php

class VMR_Crawler_Model_Crawler extends Enterprise_PageCache_Model_Crawler
{
protected function _construct()
{
    $this->_init('vmr_crawler/crawler');
}

public function crawl()
{
    if (!Mage::app()->useCache('full_page')) {
        return $this;
    }
    $storesInfo  = $this->getStoresInfo();
    $adapter     = new Varien_Http_Adapter_Curl();

    foreach ($storesInfo as $info) {
        $options = array(CURLOPT_USERAGENT => self::USER_AGENT);
        $storeId = $info['store_id'];
        $this->_visitedUrls = array();

        if (!Mage::app()->getStore($storeId)->getConfig(self::XML_PATH_CRAWLER_ENABLED)) {
            continue;
        }

        $threads = (int)Mage::app()->getStore($storeId)->getConfig(self::XML_PATH_CRAWLER_THREADS);
        if (!$threads) {
            $threads = 1;
        }
        if (!empty($info['cookie'])) {
            $options[CURLOPT_COOKIE] = $info['cookie'];
        }
        $urls       = array();
        $baseUrl    = $info['base_url'];
        $urlsCount  = $totalCount = 0;
        $urlsPaths  = $this->_getResource()->getUrlsPaths($storeId);
        foreach ($urlsPaths as $urlPath) {
            $url = $baseUrl . $urlPath;
            $urlHash = md5($url);
            if (isset($this->_visitedUrls[$urlHash])) {
                continue;
            }
            $urls[] = $url;
            $this->_visitedUrls[$urlHash] = true;
            $urlsCount++;
            $totalCount++;
            if ($urlsCount == $threads) {
                $adapter->multiRequest($urls, $options);
                $urlsCount = 0;
                $urls = array();
            }
        }
        if (!empty($urls)) {
            $adapter->multiRequest($urls, $options);
        }
    }
    return $this;
}
}

My Resource Model

app/code/local/VMR/Crawler/Model/Resource/Crawler.php

My Resource Model Code

<?php

class VMR_Crawler_Model_Resource_Crawler extends Enterprise_PageCache_Model_Resource_Crawler
{
    /**
     * Retrieve URLs paths that must be visited by crawler
     *
     * @param  $storeId
     * @return array
     */
    public function getUrlsPaths($storeId)
    {
        $bWorked = false;
        $collection = array();
        for($i = 0; $i < 3; $i++) 
        {
            echo "hi";
            try {
                $adapter = $this->_getReadAdapter();
                $select = $adapter->select()
                    ->from($this->getTable('core/url_rewrite'), array('request_path'))
                    ->where('store_id=?', $storeId)
                    ->where('is_system=1');
                $collection = $adapter->fetchCol($select);
                $bWorked = true;
            } 
            catch(Exception $ex)
            {

            }

            if($bWorked == true)
            {
                break;
            }
        }

        return $collection;
    }
}

My Resource Model Mysql4 Class

app/code/local/VMR/Crawler/Model/Mysql4/Crawler.php

My Resource Model Mysql4 Class Code

<?php

class VMR_Crawler_Model_Mysql4_Crawler extends Enterprise_PageCache_Model_Resource_Crawler
{

}

My Shell Class For Testing

shell/crawler.php

My Shell Class For Testing Code

<?php
/**
 * Full Page Cache Warming Shell Script
 *
 * @author Jeffrey L. Roberts
 **/

require_once 'abstract.php';

class Mage_Shell_Crawler extends Mage_Shell_Abstract
{
    /**
     * Run script
     *
     */
    public function run()
    {
        // echo "Starting Crawler";

        $crawler = new VMR_Crawler_Model_Crawler();
        $crawler->crawl();

        // echo "Crawler finished";
    }

    /**
     * Retrieve Usage Help Message
     *
     */
    public function usageHelp()
    {
        return <<<USAGE
Usage:  php -f crawler.php 

USAGE;
    }
}

$shell = new Mage_Shell_Crawler();
$shell->run(); 

To execute the shell

cd shell
php -f crawler.php

Answer 1

Shot in the dark here, as your post is a little dense and I'm not sure I've seen anyone extend both a model resource and override the _construct method in both, but the line 37 that PHP is complaining about looks to be

$urlsPaths  = $this->_getResource()->getUrlsPaths($storeId);

which is backed up by your error message

Fatal error: Call to a member function getUrlsPaths() on a non-object in /jlr/sites/mage/app/code/local/VMR/Crawler/Model/Crawler.php on line 37

So Magento is trying to instantiate your model's resource, but can't. So, if you're looking for a place to start, try tracing the code path of

$o = Mage::getModelResource('vmr_crawler/crawler');

and determine why its returning false.

A few guesses on that. First, you've included a

<deprecatedNode>vmr_crawler_mysql4</deprecatedNode>

but you don't have any vmr_crawler_mysql4 node in your XML file. Why is this here?

Secondly, your resource model for the crawler appears to be initializing with a core/url_rewrite?

protected function _construct()
{
    $this->_init('core/url_rewrite', 'url_rewrite_id');
}

I'm not sure what that's supposed to do, but it looks off (then again, I'm not exactly clear on what you're trying to do here, and I don't have any Enterprise versions available at the moment)