Use stripped_strings
(or strings
if you need the whitespace):
In [16]: soup = BeautifulSoup('''<div>
<div>1</div>
<div>2</div>
3
<div>4</div>
</div>''')
In [19]: list(soup.stripped_strings)
Out[19]: [u'1', u'2', u'3', u'4']
In [20]: ''.join(soup.stripped_strings)
Out[20]: u'1234'
http://www.crummy.com/software/BeautifulSoup/bs4/doc/#strings-and-stripped-strings