split a comma separated list with links in with beautifulsoup
-
11-09-2019 - |
Question
I've got a comma separated list in a table cell in an HTML document, but some of items in the list are linked:
<table>
<tr>
<td>Names</td>
<td>Fred, John, Barry, <a href="http://www.example.com/">Roger</a>, James</td>
</tr>
</table>
I've been using beautiful soup to parse the html, and I can get to the table, but what is the best way to split it and return a data structure roughly like:
[
{'name':'Fred'},
{'name':'John'},
{'name':'Barry'},
{'name':'Roger', 'url':'http://www.example.com/'},
{'name':'James'},
]
Solution
This is one way you could do it:
import BeautifulSoup
soup = BeautifulSoup.BeautifulSoup('''<table>
<tr>
<td>Names</td>
<td>Fred, John, Barry, <a href="http://www.example.com/">Roger</a>, James</td>
</tr>
</table>''')
result = []
for tag in soup.table.findAll('td')[1]:
if isinstance(tag, BeautifulSoup.NavigableString):
for name in tag.string.split(','):
name = name.strip()
if name:
result.append({ 'name': name })
else:
result.append({ 'name': tag.string.strip(), 'url': tag["href"] })
print result
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow