Try it with:
agent.page.link_with(:text => "Choose this style") rescue debugger
It seems that page is nil
Pergunta
I am using Mechanize to scrape data off of KBB.com, my app takes a url and then uses Mechanize to go through the individual pages and find the price for the given url. There are a number of different pages that a url can lead to and my app needs to be able to figure out which page it has landed on. I am currently using an if / else statement to direct the logic but I am running into a problem that I seemingly cannot debug.
When my app arrives at the if / else statement and checks the condition:
if agent.page.link_with(:text => "Choose this style")
I get:
NoMethodError (undefined method `link_with' for nil:NilClass)
I tried to debug this by using:
debugger if agent.page.link_with(:text => "Choose this style").nil?
but that gives me the same error as above, only this time it says the debugger code is the source of the problem.
How do I debug this? Or what is a better way to code this so that my app does not hit this exception?
Here's the code:
def kbb_value(url)
agent = Mechanize.new
# rescue bad urls
begin
page = agent.get(url)
rescue Mechanize::ResponseCodeError
begin
agent.cookie_jar.clear!
page = agent.get(url.sub('styles', 'categories'))
rescue Mechanize::ResponseCodeError
page = 'http://www.kbb.com/acura/rsx/2002-acura-rsx/styles/?intent=buy-used'
end
end
# kbb.com has cookie problems - solution is to clear cookies after every request
agent.cookie_jar.clear!
# program determines which starting page it is on via if / else
debugger if agent.page.link_with(:text => "Choose this style").nil?
if agent.page.link_with(:text => "Choose this style")
agent.page.link_with(:text => "Choose this style").click
# if the page allows additional options - skip that page
if agent.page.link_with(:text => "Choose price type")
agent.cookie_jar.clear!
agent.page.link_with(:text => "Choose price type").click
end
elsif agent.page.link_with(:text => "Choose price type")
agent.page.link_with(:text => "Choose price type").click
else
agent.page.link_with(:href => /bodystyle/).click
agent.cookie_jar.clear!
agent.page.link_with(:text => 'Choose this style').click
agent.cookie_jar.clear!
agent.page.link_with(:text => "Choose price type").click
end
# Get the 'Good' car price from kbb.com
agent.cookie_jar.clear!
agent.page.links_with(:text => "Get used car price")[2].click
# instead of getting the 'retail' value, substitute in 'private-party' in the url. Then get that page and grab the kbb value.
agent.cookie_jar.clear!
agent.get(agent.page.uri.to_s.sub('retail', 'private-party'))
@kbb_value = agent.page.at('.selected .value').text.delete('$')
end
Solução
Try it with:
agent.page.link_with(:text => "Choose this style") rescue debugger
It seems that page is nil
Outras dicas
Looks like it could be one of two things:
I've never used mechanize, but you're assigning the result to a variable called page
, not agent.page
. Try changing it to compare against page.links_with
, rather than agent.page.links_with
, which may be null.
Ruby has block variable scope, and any variable that is defined within a block will not exist outside of it. You're assigning your page inside the begin (which, I believe, is a block scope). Try changing it so you define page outside of the scope (as nil), and then just assign the value within the begin/rescue
.
Code Fix Examples:
# Assign the page variable outside of the scope
page = nil
begin
page = agent.get(url)
rescue Mechanize::ResponseCodeError
begin
agent.cookie_jar.clear!
page = agent.get(url.sub('styles', 'categories'))
rescue Mechanize::ResponseCodeError
page = 'http://www.kbb.com/acura/rsx/2002-acura-rsx/styles/?intent=buy-used'
end
end
# Check the "page" variable
debugger if page.link_with(:text => "Choose this style").nil?
Note: It's been a while since I've used Ruby, and forever since I've used Mechanize. Give these a shot, hope they help.
You get this error because page wasn't loaded by Mechanize.