Link Fetching List

Question

so I've asked many questions regarding this one subject, and I'm sorry. But this is it.

So I have this code:

import urllib
import urllib.request
from bs4 import BeautifulSoup
import sys
from collections import defaultdict

m_num=int(input('Enter number of monsters to look up: '))
for x in range(m_num):
    name=input("Enter a monster's name: ")
    url_name=name.replace(' ','_')
    url=('http://yugioh.wikia.com/wiki/Card_Tips:{}'.format(url_name))
    page = urllib.request.urlopen(url)
    soup = BeautifulSoup(page.read())
    content = soup.find('div',id='mw-content-text')
    links = content.findAll('a')
    link_lists = defaultdict(list)
    for link in links:
        link_lists[x].append(link.get('title'))
all_lists = list(link_lists.values())
common_links = set(all_lists[0]).intersection(*all_lists[1:])
print('common links: ',common_links)

What I'm trying to do is for how many number of monsters the user specifies is how many lists are creatd. Each list is then filled with all the links from that specific site. And then in the ned all the lists are compared to see if they have similar strings inside of them. (Hopefully that makes sense).

So the problem I'm having is that when I run it and it gets to the print('common links:',common_links) part it only prints out the last list. It doesn't compare the lists nor does it even recognize that the other lists were created.

Can anyone lend a helping hand? I've been troubleshooting this and I'm just stuck.

Answer 1

link_lists refers to a new dictionary on each iteration. You could exclude it: put all_lists = [] before the for x in range(m_num) loop. And replace the last 3 line in the loop with: all_lists.append([link.get("title") for link in links]) Note: you don't need to know m_num in this case:

all_lists = []
for name in iter(lambda: input("monster name"), ""): # loop until empty name
    # ...
    titles = [link["title"] for link in content.findAll('a', title=True)]
    all_lists.append(titles)