Why is my rails if / else getting a NoMethodError

Question

I am using Mechanize to scrape data off of KBB.com, my app takes a url and then uses Mechanize to go through the individual pages and find the price for the given url. There are a number of different pages that a url can lead to and my app needs to be able to figure out which page it has landed on. I am currently using an if / else statement to direct the logic but I am running into a problem that I seemingly cannot debug.

When my app arrives at the if / else statement and checks the condition:

if agent.page.link_with(:text => "Choose this style")

I get:

NoMethodError (undefined method `link_with' for nil:NilClass)

I tried to debug this by using:

 debugger if agent.page.link_with(:text => "Choose this style").nil?

but that gives me the same error as above, only this time it says the debugger code is the source of the problem.

How do I debug this? Or what is a better way to code this so that my app does not hit this exception?

Here's the code:

def kbb_value(url)
agent = Mechanize.new

# rescue bad urls
begin
  page = agent.get(url)
rescue Mechanize::ResponseCodeError
  begin
    agent.cookie_jar.clear!
    page = agent.get(url.sub('styles', 'categories'))
  rescue Mechanize::ResponseCodeError
    page = 'http://www.kbb.com/acura/rsx/2002-acura-rsx/styles/?intent=buy-used'
  end
end  


# kbb.com has cookie problems - solution is to clear cookies after every request

agent.cookie_jar.clear!

# program determines which starting page it is on via if / else
debugger if agent.page.link_with(:text => "Choose this style").nil?
if agent.page.link_with(:text => "Choose this style")
  agent.page.link_with(:text => "Choose this style").click

  # if the page allows additional options - skip that page 

  if agent.page.link_with(:text => "Choose price type")
    agent.cookie_jar.clear!
    agent.page.link_with(:text => "Choose price type").click
  end  

elsif agent.page.link_with(:text => "Choose price type")
  agent.page.link_with(:text => "Choose price type").click
else
  agent.page.link_with(:href => /bodystyle/).click
  agent.cookie_jar.clear!
  agent.page.link_with(:text => 'Choose this style').click
  agent.cookie_jar.clear!
  agent.page.link_with(:text => "Choose price type").click
end   

 # Get the 'Good' car price from kbb.com

agent.cookie_jar.clear!
agent.page.links_with(:text => "Get used car price")[2].click

# instead of getting the 'retail' value, substitute in 'private-party' in the url. Then get that page and grab the kbb value.

agent.cookie_jar.clear!
agent.get(agent.page.uri.to_s.sub('retail', 'private-party'))
@kbb_value = agent.page.at('.selected .value').text.delete('$')
end  

Answer 1

Try it with:

agent.page.link_with(:text => "Choose this style") rescue debugger

It seems that page is nil

Answer 2

Looks like it could be one of two things:

• I've never used mechanize, but you're assigning the result to a variable called page, not agent.page. Try changing it to compare against page.links_with, rather than agent.page.links_with, which may be null.

• Ruby has block variable scope, and any variable that is defined within a block will not exist outside of it. You're assigning your page inside the begin (which, I believe, is a block scope). Try changing it so you define page outside of the scope (as nil), and then just assign the value within the begin/rescue.

Code Fix Examples:

# Assign the page variable outside of the scope
page = nil
begin
  page = agent.get(url)
rescue Mechanize::ResponseCodeError
  begin
    agent.cookie_jar.clear!
    page = agent.get(url.sub('styles', 'categories'))
  rescue Mechanize::ResponseCodeError
    page = 'http://www.kbb.com/acura/rsx/2002-acura-rsx/styles/?intent=buy-used'
  end
end

# Check the "page" variable
debugger if page.link_with(:text => "Choose this style").nil?

Note: It's been a while since I've used Ruby, and forever since I've used Mechanize. Give these a shot, hope they help.

Answer 3

You get this error because page wasn't loaded by Mechanize.