Ruby mecanizate la tabla de raspar no captura toda la fila

Question

Estoy tratando de raspar un sitio web de mesa con mecanismo. Quiero raspar la segunda fila.

cuando ejecuto:

agent.page.search('table.ea').search('tr')[-2].search('td').map{ |n| n.text }

esperaría que rascara la fila completa.Pero en su lugar, solo raspa: ["2011-02-17", "0,00"]

¿Por qué no está raspando todas las columnas en la fila, pero solo la primera y la última columna?

xpath: / HTML / CUERPO / CUERPO / CENTRO / TABLA / TODO / TR [2] / TD [2] / TABLA / TODO / TUDE / TR [3] / TD / TABLA / TODO / TR [2] / TD / TABLE / TODO / TR [2]

CSS PATH: HTML CUERPO CENTRO TABLE TUBLY TR TD TABLA TUBLIA TR TD TABLA TULY TR T TABLATE.EA TODO T TD.TOTAL

La página es similar a esta:

<table><table><table>
<table width="100%" border="0" cellpadding="0" cellspacing="1" class="ea">
<tr>
    <th><a href="#">Date</a></th>
    <th><a href="#">One</a></th>    
    <th><a href="#">Two</a></th>    
    <th><a href="#">Three</a></th>     
    <th><a href="#">Four</a></th>    
    <th><a href="#">Five</a></th>        
    <th><a href="#">Six</a></th>        
    <th><a href="#">Seven</a></th>      
    <th><a href="#">Eight</a></th>
</tr>
<tr>
    <td><a href="#">2011-02-17</a></td>
    <td align="right">0</td>    
    <td align="right">0</td>    
    <td align="right">0,00</td>     
    <td align="right">0</td>    
    <td align="right">0</td>        
    <td align="right">0</td>    
    <td align="right">0</td>        
    <td align="right">387</td>      
    <td align="right">0,00</td>     <!-- FOV -->
    <td align="right">0,00</td>
</tr>
<tr>
    <td class="total">Ialt</td>
    <td class="total" align="right">0</td>  
    <td class="total" align="right">40</td>     
    <td class="total" align="right">0,46</td>   
    <td class="total" align="right">2</td>      
    <td class="total" align="right">0</td>        
    <td class="total" align="right">0</td>      
    <td class="total" align="right">0</td>        
    <td class="total" align="right">3.060</td>      
    <td class="total" align="right">0,00</td>       
    <td class="total" align="right">18,58</td>
</tr>
</table>
</table></table></table>

Answer 1

Using the following Ruby code (https://gist.github.com/835603):

require 'mechanize'
require 'pp'

a = Mechanize.new { |agent|
  agent.user_agent_alias = 'Mac Safari'
}

a.get('http://binarymuse.net/table.html') do |page|
  pp page.search('table.ea').search('tr')[-2].search('td').map{ |n| n.text }
end

I get the following output:

["2011-02-17", "0", "0", "0,00", "0", "0", "0", "0", "387", "0,00", "0,00"]

Answer 2

I would recommend you to leave Mechanize to harder stuff than scraping a page. You can use Nokogiri much more simple than using Mechanize(but ofcourse you can do it with it) since you can just query the page.

Try it out!

here is a link to an answer regarding nokogiri

Personally I used Mechanize when I needed to send forms and stuff like that albeit there are tons of other uses to it!