How to get changed DOM after Watir execute_script

Question

I am using Nokogiri to scrape the list of artist from http://www.englandgallery.com/artists.php. I need to use Watir, because the page loads artists using javascript. My code looks like this

browser = Watir::Browser.new(:phantomjs)
browser.goto "http://www.englandgallery.com/artists.php"
browser.execute_script("javascript:loadContent('ALL');")
indexPage = Nokogiri::HTML(browser.html)
artists = indexPage.css('a.artistlink')
artists.each do |artist|
    puts artist.text
end

The page by default displays only contemporary artists, so you need to execute js to load all artists. I'm sure loadContent('ALL') gets executed and actually loads all artists to the page, but only contemporary artists are selected to artists array. What am I missing? Do I need to somehow reload the browser to get the current DOM?

Answer 1

It seems that after executing javascript:loadContent('ALL'); it takes sometime until the list is actually updated. You could try sleep for a few seconds.

browser = Watir::Browser.new(:phantomjs)
browser.goto "http://www.englandgallery.com/artists.php"
browser.execute_script("javascript:loadContent('ALL');")
sleep 5
indexPage = Nokogiri::HTML(browser.html)
artists = indexPage.css('a.artistlink')
artists.each do |artist|
    puts artist.text
end