How to run arbitrary string as command

Question

I see answers where getattr() is utilized for some simple one method/function call.

How about arbitrary string, e.g. doing web parsing here:

from bs4 import BeautifulSoup
import urllib

f = urllib.urlopen(link) # link comes from database, e.g. 'http://www.example.com'
soup = BeautifulSoup(f)

text = soup.find(True, 'text').get_text() # Now this is hardcoded

Works fine, but how about running parser string which comes from database? String can be like:

soup.find("div", "layout left").find(id=True).get_text()

or pretty match anything, depends on webpage.

Answer 1

You can use eval to evaluate an arbitrary Python expression stored in a string. However, this is dangerous. A hacker or an unscrupulous user could insert malicious code into the database (e.g., 1000000**1000000 to cause Python to go nuts).

Answer 2

Why can not you advance from the string line to construct a list and do something like this?

tags = soup.findAll(['div','span'])

or

soup.findAll(lambda tag: tag.name in ['div', 'span'] or tag['id'] == "eggs")

or maybe even better:

tags = soup.findAll(['div', 'span'])
tags.extend(soup.findAll(id="eggs"))

If you want to exclude some tags by condition you can add the condition to lambda expression.

Example:

From DB:

s = 'div;span;table' # or something like this with structure

Do like this:

tags_list = s.split(';')
tags = soup.findAll(tags_list)

I think you got the main idea.