PyAudio 'utf8' error when listing devices

Question

When using PyAudio (Portaudio binding) with ASIO+DirectSound support, this code :

import pyaudio

p = pyaudio.PyAudio()
for i in range(p.get_device_count()):
    print p.get_device_info_by_index(i)

... produces this error :

UnicodeDecodeError: 'utf8' codec can't decode byte 0xe9 in position 1: invalid continuation byte

How can we solve this problem ?

---

The problem may come from "pyaudio.py", line 990, because of an unsucessful utf8 decoding :

           return {'index' : index,
                    'structVersion' : device_info.structVersion,
                    'name' : device_info.name,

---

This answer here Special characters in audio devices name : Pyaudio ("don't use PyAudio") is not satisfactory.

---

Traceback

...
{'defaultSampleRate': 44100.0, 'defaultLowOutputLatency': 0.0, 'defaultLowInputLatency': 0.12, 'maxInputChannels': 2L, 'structVersion': 2L, 'hostApi': 1L, 'index': 8, 'defaultHighOutputLatency': 0.0, 'maxOutputChannels': 0L, 'name': u'Microphone interne (Conexant 20672 SmartAudio HD)', 'defaultHighInputLatency': 0.24}
Traceback (most recent call last):
  File "D:\test\test.py", line 5, in <module>
    print p.get_device_info_by_index(i)
  File "C:\ProgramData\Anaconda\lib\site-packages\pyaudio.py", line 977, in get_device_info_by_index
    pa.get_device_info(device_index)
  File "C:\ProgramData\Anaconda\lib\site-packages\pyaudio.py", line 990, in _make_device_info_dictionary
    'name' : device_info.name,
UnicodeDecodeError: 'utf8' codec can't decode byte 0xe9 in position 1: invalid continuation byte

Answer 1

The only successful solution found is :

• apply Tobias Erichsen'patch to PortAudio (as mentioned in @RossBencina's comment) that can be found here : https://www.assembla.com/spaces/portaudio/support/tickets/224-patch-for-windows-directsound-and-wmme-utf-8-device-names#/activity/ticket
• rebuild the whole thing

Many thanks to @cgohlke for having built new ready-to-use installers : http://www.lfd.uci.edu/~gohlke/pythonlibs/#pyaudio

Answer 2

The error 'invalid continuation byte' makes me think that the text is corrupt for that particular index.

If you're able to modify the pyaudio.py file (or get the pyaudio.py file to return just the name), you might be able to try handle the UTF-8 decoding yourself by using 'Unicode Dammit'. It pretty much takes a best guess at what the encoding can be. Here's a link to their tutorial (http://www.crummy.com/software/BeautifulSoup/bs4/doc/#unicode-dammit)

I think the code would look just like the tutorial:

from bs4 import UnicodeDammit

dammit = UnicodeDammit(audiodevicename)
print(dammit.unicode_markup) ## Wéird Device Name!

Answer 3

I've forked pyAudio and modified https://github.com/joelewis/PyAudio/blob/master/src/_portaudiomodule.c code to use

PyUnicode_DecodeFSDefault

instead of

 PyUnicode_FromString

which likely might solve the unicode issue. See if you could find it helpful.

fork: https://github.com/joelewis/PyAudio/

Answer 4

I think the clue here is

UnicodeDecodeError: 'utf8' codec can't decode byte 0xe9 in position 1: invalid continuation byte

For whatever reason something returned by get_device_info_by_index() (probably the name field) contains the byte 0xe9 which, if you are interpreting the string of bytes as UTF8, signifies a "continuation byte". This means that it expects some valid bytes to follow the 0xe9. valid bytes means some sequence of bytes that constitutes a legitimate UTF8 character. E.g.

http://hexutf8.com/?q=e981a8

uses 0xe9 with some valid continuation bytes.