how to check in python if a string contains characters only form specific code page?

Question

I know that python uses solely utf8 for string encoding , but what if I need to send data to older application that is non-unicode and supports only these characters that are included in windows code-page like windows-1251 (cyrylic) etc... So here I want to detect if utf-8 string contains any character that could not by represented by given code page.

Answer 1

There is lib for encoding detection: https://pypi.python.org/pypi/chardet (Character encoding auto-detection in Python. As smart as your browser. Open source)

Detects...

• ASCII, UTF-8, UTF-16 (2 variants), UTF-32 (4 variants)
• Big5, GB2312, EUC-TW, HZ-GB-2312, ISO-2022-CN (Traditional and Simplified Chinese)
• EUC-JP, SHIFT_JIS, ISO-2022-JP (Japanese)
• EUC-KR, ISO-2022-KR (Korean)
• KOI8-R, MacCyrillic, IBM855, IBM866, ISO-8859-5, windows-1251 (Cyrillic)
• ISO-8859-2, windows-1250 (Hungarian)
• ISO-8859-5, windows-1251 (Bulgarian)
• windows-1252 (English)
• ISO-8859-7, windows-1253 (Greek)
• ISO-8859-8, windows-1255 (Visual and Logical Hebrew)
• TIS-620 (Thai)

example from docs:

>>> import urllib
>>> rawdata = urllib.urlopen('http://yahoo.co.jp/').read()
>>> import chardet
>>> chardet.detect(rawdata)
{'encoding': 'EUC-JP', 'confidence': 0.99}

Answer 2

Where blah is your string and ascii is your encoding:

blah = u'\uffff'
try:
    blah.encode('ascii')
except UnicodeEncodeError:
    # nope - can't do it...