If you feed the text/string version of the data into a StringIO.StringIO
(or io.StringIO
in Python 3.X), you can pass that object to the pandas parser. So your code becomes:
#!/usr/bin/env python2
import pandas as p
from BeautifulSoup import BeautifulSoup
import urllib2
import StringIO
def main():
url = "URL"
html = urllib2.urlopen(url)
soup = BeautifulSoup(html)
# pre is the tag that the data is within
tab_sepd_vals = soup.pre.string
# make the StringIO object
tsv = StringIO.StringIO(tab_sepd_vals)
# something like this
data = p.read_csv(tsv, sep='\t')
# then what you had
process(data)