Python ftplib UnicodeEncodeError when trying to upload an XML with unicode content

Question

I am trying to upload an XML with unicode content to a FTP server using ftplib, but getting the following exception when I try to upload the using storbinary method. The XML data is properly encoded to unicode (utf-8), I have made sure of that, I am not sure as to why storbinary is trying to encode it to 'ascii' while uploading. Can anyone please help?

--> 429         ftp.storbinary("STOR file.xml", xml)
    430 
    431     def run(self):

/usr/lib/python2.7/ftplib.pyc in storbinary(self, cmd, fp, blocksize, callback, rest)
    463             buf = fp.read(blocksize)
    464             if not buf: break
--> 465             conn.sendall(buf)
    466             if callback: callback(buf)
    467         conn.close()

/usr/lib/python2.7/socket.pyc in meth(name, self, *args)
    222 
    223 def meth(name,self,*args):
--> 224     return getattr(self._sock,name)(*args)
    225 
    226 for _m in _socketmethods:

UnicodeEncodeError: 'ascii' codec can't encode character u'\xae' in position 3368: ordinal not in range(128)

Answer 1

I was able to find the solution, the comment by @Cairnarvon was partially correct, I was encoding the string, but there were other bits of the string written to the StringIO instance that were not encoded. Finally I ended up creating the XML bit and encoding it as a whole. You can see my code in the pastebin link below;

http://pastebin.com/GugTLRQJ

Answer 2

You should pass a file opened in a binary mode to ftp.storbinary(). For example, if you want to upload a Unicode string as a filename file:

import io

assert isinstance(unicode_string, unicode)
file = io.BytesIO(unicode_string.encode("utf-8"))
ftp.storbinary("STOR filename", file)

If unicode_string contains xml; make sure that the character encoding used in the xml declaration is consistent with the encoding you use to store the file.