Encoding unicode path with filesystem encoding breaks it [duplicate]

Question

This question already has answers here:

Unicode filename to python subprocess.call() [duplicate] (7 answers)

Closed 6 years ago.

I do have the following path in the memory:

video_path = u'C:\\Documents and Settings\\user\\My Documents\\Downloads\\\xf5iv - Neon Phoenix [Free DL].mp3'

I'm trying to use it as a parameter in cmd, so I have to encode it.

video_path = video_path.encode(sys.getfilesystemencoding())
cmd = 'ffmpeg -y -i "%s" -vn -ac 2 -f mp3 audio.mp3' % video_path
subprocess.Popen(cmd)

However the string is not encoded in the right way - it converts the \xf5 to ? instead of õ. Therefore the file could not be found.

How can this happen? I'm using the default filesystem encoding (which is mbcs).

Answer 1

From an answer here:

In Py3K - at least from "Python" 3.2 - subprocess.Popen and sys.argv work consistently with (default unicode) str's on Windows. CreateProcessW and GetCommandLineW are used obviously.

In Python - up to v2.7.2 at least - subprocess.Popen is buggy with unicode arguments. It sticks to CreateProcessA (while os.* are consistent with unicode). And shlex.split creates additional nonsense. Pywin32's win32process.CreateProcess also doesn't auto-switch to the W version, nor is there a win32process.CreateProcessW. Same with GetCommandLine. Thus ctypes.windll.kernel32.CreateProcessW... needs to be used. The subprocess module perhaps should be fixed regarding this issue.

Therefore, subprocess.Popen can't handle unicode right at Python 2.x versions.

My solution was renaming the input file to something random (with os.rename, which supports unicode), convert with ffmpeg that i launch with subprocess.Popen, and then rename it back.

Answer 2

Try to encode using UTF-8:

video_path = video_path.encode("utf-8")

Answer 3

Unless I am totally mistaken, the double backslash in

video_path = u'C:...\\xf5iv...'

causes the problem. There should be only one:

video_path = u'C:...\xf5iv...'

Otherwise the backslash is preserved as a backslash and left for os.system(), rather than .encode(), to deal with.