I have problem with displaying national characters from “ENGLISH_UNITED KINGDOM.US7ASCII” Oracle 11 database using Python 3.3 cx_Oracle 5.1.2 and "NLS_LANG" environment variable.
Db table column type is "VARCHAR2(2000 BYTE)"
How to display string "£aÀÁÂÃÄÅÆÇÈ" from Oracle US7ASCII in Python? This will be some sort of hack.
The hank works in every other scripting language Perl, PHP, PL/SQL and in Python 2.7, but it does not work in Python 3.3.
In Oracle 11 Database I created SECURITY_HINTS.ANSWER="£aÀÁÂÃÄÅÆÇÈ". ANSWER column type is "VARCHAR2(2000 BYTE)".
Now when using cx_Oracle and default NLS_LANG, I get "¿a¿¿¿¿¿¿¿¿¿"
and when using NLS_LANG="ENGLISH_UNITED KINGDOM.US7ASCII" I get
"UnicodeDecodeError: 'ascii' codec can't decode byte 0xa3 in position 0: ordinal not in range(128)"
Update1
I made some progress. When switching to Python 2.7 and cx_Oracle 5.1.2 for Python 2.7 the problem goes away (I get all >127 characters from db). In Python 2 strings are represented as bytes and in Python 3+ strings are represented as unicode. I still need best possible solution for Python 3.3.
Update2
One possible solution to the problem is to used rawtohex(utl_raw.cast_to_raw see code below.
cursor.execute("select rawtohex(utl_raw.cast_to_raw(ANSWER)) from security_hints where userid = '...'")
for rawValue in cursor:
print (''.join(['%c' % iterating_var for iterating_var in binascii.unhexlify(rawValue[0])]))
source code of my script is below or at GitHub and GitHub Sollution
def test_nls(nls_lang=None):
print (">>> run test_nls for %s" %(nls_lang))
if nls_lang:
os.environ["NLS_LANG"] = nls_lang
os.environ["ORA_NCHAR_LITERAL_REPLACE"] = "TRUE"
connection = get_connection()
cursor = connection.cursor()
print("version=%s\nencoding=%s\tnencoding=%s\tmaxBytesPerCharacter=%s" %(connection.version, connection.encoding,
connection.nencoding, connection.maxBytesPerCharacter))
cursor.execute("SELECT USERENV ('language') FROM DUAL")
for result in cursor:
print("%s" %(result))
cursor.execute("select ANSWER from SECURITY_HINTS where USERID = '...'")
for rawValue in cursor:
print("query returned [%s]" % (rawValue))
answer = rawValue[0]
str = ""
for iterating_var in answer:
str = ("%s [%d]" % (str, ord(iterating_var)))
print ("str %s" %(str))
cursor.close()
connection.close()
if __name__ == '__main__':
test_nls()
test_nls(".AL32UTF8")
test_nls("ENGLISH_UNITED KINGDOM.US7ASCII")
see log output below.
run test_nls for None
version=11.1.0.7.0
encoding=WINDOWS-1252 nencoding=WINDOWS-1252 maxBytesPerCharacter=1
ENGLISH_UNITED KINGDOM.US7ASCII
query returned [¿a¿¿¿¿¿¿¿¿¿]
str [191] [97] [191] [191] [191] [191] [191] [191] [191] [191] [191
run test_nls for .AL32UTF8
version=11.1.0.7.0
encoding=UTF-8 nencoding=UTF-8 maxBytesPerCharacter=4
AMERICAN_AMERICA.US7ASCII
query returned [�a���������]
str [65533] [97] [65533] [65533] [65533] [65533] [65533] [65533] [65533] [65533] [65533]
run test_nls for ENGLISH_UNITED KINGDOM.US7ASCII
version=11.1.0.7.0
encoding=US-ASCII nencoding=US-ASCII maxBytesPerCharacter=1
ENGLISH_UNITED KINGDOM.US7ASCII
Traceback (most recent call last):
File "C:/dev/tmp/Python_US7ASCII_cx_Oracle/showUS7ASCII.py", line 71, in <module>
test_nls("ENGLISH_UNITED KINGDOM.US7ASCII")
File "C:/dev/tmp/Python_US7ASCII_cx_Oracle/showUS7ASCII.py", line 55, in test_nls
for rawValue in cursor:
UnicodeDecodeError: 'ascii' codec can't decode byte 0xa3 in position 0: ordinal not in range(128)
I am trying to Display it in Django Web page. But each character comes as character with code 191 or 65533.
I looked at
choosing NLS_LANG for Oracle and
Importing from Oracle using the correct encoding with Python
Cannot Insert Unicode Using cx-Oracle