How to deal with single quotes and untranslatable characters

Question

My SAS table contains a text column that has sentences. Some of these sentences contain single quotes somthing like don't , space's . . .Whenever i try to upload this table to teradata it throws an error string contains untranslatable characters

When i remove the ' manually, it gets uploaded okay .

How can I upload the original sentences correctly in the original way ?

---

I suspect that this is not a regular single quote

I tried

varName = tranwrd(varName, "'",""); /* i also tried using ` instead of '  */

AND

varName = translate(varName, "","'"); /* i also tried using ` instead of '  */

But the string dod not change

Answer 1

One possibility: instead of removing it manually, 'cut' it, and paste that into SAS instead of typing the ' or ` characters.

More generally, you can identify what a character actually is by looking at the $HEX. version of the character.

One good approach is to first remove known 'good' characters.

data want;
set have;
step1 = compress(yourvar,,'ns');
put step1= $HEX.;
run;

That removes (English) letters and numbers, underscores, and space-type characters (space, tab, etc.). That should reduce your string to a relatively small number of characters; if you want to limit it further, add characters to the (currently missing) second argument.

Now, you can look at the hex output and identify what the character is, hopefully fairly easily. Every 2 hex characters map to one ASCII character, unless you're in a Unicode environment, where that may not be the case (it may be 2, it may be as many as 8). You should be able to see the 'bad' character. Once you do find it, you can use '##'x to construct a character identical to it, and use compress to remove it (compress does what you're trying to do with translate more efficiently). IE, if you have '0A'x (line feed), you could use

data want;
set have;
fixed=compress(yourvar,'0A'x);
run;

to remove the unwanted line feed.

Answer 2

Forgive me, I don't know teradata but if it's problematic in the same way as sql. Convert your text from something like

'don't'

to

'don''t'

This may help