Beautiful soup findAll didn't find all of them

Question

I'm using Calibre to make a recipe for a website.

The web source code is:

  <section>

  <h1 class="fly-title">Leaders</h1>

    <article>
  <h2><a href="/node/21537908" class="package-link">Democracy and its enemies</a></h2>

    <a href="/node/21537908"><img src="http://media.economist.com/sites/default/files/imagecache/news_package_primary_landscape/20120123_LDC001_0.gif" alt="" title=""  class="imagecache imagecache-news_package_primary_landscape" width="412" height="232" /></a>  
  <p>
    In the coming year the people who run the world will change—and so could the ideas, predicts John Micklethwait    <a href="/node/21537908/comments#comments" title="Comments" class="comment-icon"><span>(0)</span></a>  </p>

</article>  
    <ul class="package-item"><li class="first"><div class="">
  <a href="/node/21537909" class="package-link">The year of self-induced stagnation</a>  <a href="/node/21537909/comments#comments" title="Comments" class="comment-icon"><span>(7)</span></a></div>
</li>
<li class="even"><div class="">
  <a href="/node/21537914" class="package-link">How to run the euro?</a>  <a href="/node/21537914/comments#comments" title="Comments" class="comment-icon"><span>(2)</span></a></div>
</li>
<li class=""><div class="">
  <a href="/node/21537916" class="package-link">Wanted: a fantasy American president</a>  <a href="/node/21537916/comments#comments" title="Comments" class="comment-icon"><span>(0)</span></a></div>
</li>
<li class="even"><div class="">
  <a href="/node/21537917" class="package-link">Poking goes public</a>  <a href="/node/21537917/comments#comments" title="Comments" class="comment-icon"><span>(7)</span></a></div>
</li>
<li class=""><div class="">
  <a href="/node/21537918" class="package-link">Varied company</a>  <a href="/node/21537918/comments#comments" title="Comments" class="comment-icon"><span>(0)</span></a></div>
</li>
<li class="even"><div class="">
  <a href="/node/21537919" class="package-link">All eyes on London</a>  <a href="/node/21537919/comments#comments" title="Comments" class="comment-icon"><span>(0)</span></a></div>
</li>
<li class="last"><div class="">
  <a href="/node/21537921" class="package-link">And now for some non-events</a>  <a href="/node/21537921/comments#comments" title="Comments" class="comment-icon"><span>(2)</span></a></div>
</li>
</ul>  
  </section>

I want to find all <a href="/node/********" class="package-link">

So I used beautiful soup

for section in soup.findAll('section'): 
...
    for post in section.findAll('a', attrs={'class':['package-link']})

But only the first one was found (that is the one in <h2><a href="/node/21537908" class="package-link">Democracy and its enemies</a></h2>).

How can I find them all?

Answer 1

Works for me:

soup = BeautifulSoup.BeautifulSoup(xml)    
for section in soup.findAll("section"):
    for post in section.findAll('a', attrs={'class':['package-link']}):
        print post

results in:

<a href="/node/21537908" class="package-link">Democracy and its enemies</a>
<a href="/node/21537909" class="package-link">The year of self-induced stagnation</a>
<a href="/node/21537914" class="package-link">How to run the euro?</a>
<a href="/node/21537916" class="package-link">Wanted: a fantasy American president</a>
<a href="/node/21537917" class="package-link">Poking goes public</a>
<a href="/node/21537918" class="package-link">Varied company</a>
<a href="/node/21537919" class="package-link">All eyes on London</a>
<a href="/node/21537921" class="package-link">And now for some non-events</a>

Edit

Versions I use:

• Python 2.7.3
• BeautifulSoup 3.2.0