SQL Server database with Latin1 codepage shows Japanese Chars as “?”

Question

Three questions with the following scenario:

• SQL Server 2005 production db with a Latin1 codepage and showing "?" for invalid chars in Management Studio.
• SomeCompanyApp client as a service that populates the data from servers and workstations.
• SomeCompanyApp management console that shows "?" for Asian characters.

Since this is a prod db I will not write to it.

I don't know if the client app that is storing the data in the database is actually storing it correctly as Unicode and it simply doesn't show because they are using Latin1 for the console.

Q1: As I understand it, SQL Server stores nvarchar text as Unicode regardless of the codepage or am I completely wrong and if the codepage is Latin1 then everything that is not in that codepage gets converted to "?".

Q2: Is it the same with a text column?

Q3: Is there a way using SQL Server Management Studio or Visual Studio and some code (don't care which language :)) to query the db and show me if the chars really do show up as Japanese, Chinese, Korean, etc.?

My final goal is to extract data from the db and store it in another db using UTF-8 to show Japanese and other Asian chars as what they are in my own client webapp. I will settle for an answer to Q3. I can code in several languages and at the very least understand some others but I'm just not knowledgeable enough about Unicode. In case you want to know my webapp will be using pyodbc and cassandra but for these questions that doesn't matter.

Answer 1

When inserting into an NVARCHAR column in SSMS, you need to make absolutely sure you're prefixing your string with a N:

This will NOT work:

 INSERT INTO dbo.MyTable(NVarcharColumn) VALUES('Some Text with Special Char')

SQL Server will interpret your string in the VALUES(..) as VARCHAR and thus strip off any special characters.

You need this:

 INSERT INTO dbo.MyTable(NVarcharColumn) VALUES(N'Some Text with Special Char')

Prefixing your text literal with an N'..' tells SQL Server to treat this as NVARCHAR all the way.

Does this help you solve your Q3 ??