How to force upload in SAS (or convert all to Unicode)?

Question

Is there any force function that would allow to finish the upload ?

data testlib.testTab
(
    dbtype=(
        var1 = 'VARCHAR(300)',
        var2 = 'VARCHAR(150)',
    )
)
;

set testt ; 
run ; 

This works fine, however rarely I could get an untranslatable character error . Last time I had a ' which was ONLY in one observation. I do not know why all other ' uploaded fine, except for this one. I presume it was neither in Unicode nor Latin.

Is there any way i could force upload, skip that observation . . . or whatever . . . anything but fail the whole sample .

Maybe convert all values to Unicode to be sure that upload succeeds

How can I prepare a SAS data set to make sure it gets uploaded fine ?

Answer 1

If your concern is that the encoding is different, you could consider using the KCVT function. That converts from one encoding to another. You could try, for example.

data want;
set have;
yourvar = kcvt(yourvar,'utf8','ansi');
run;

You would be best off identifying what encoding your source data is in and ensuring the 'from' matches that, as this won't work if the source isn't valid in that encoding.

To try to 'skip' the error, you might try enabling BULK_COPY and make sure the error limit is sufficiently high. See this doc page for more detail. An example from that page:

In the following example, SAS stops processing and pauses Fastload when it encounters the tenth error.

libname mydblib teradata user=terauser pw=XXXXXX ERRLIMIT=10;

data mydblib.trfload(bulkload=yes dbtype=(i='int check (i > 11)') );
     do
         i=1 to 50000;output;
     end;
run;

Set the ERRLIMIT sufficiently low to warn you of important issues and high enough to skip trivial issues like this; and use bulkload=yes in the data statement to ensure it uses FASTLOAD. Leave off the int check part of things unless you actually want to error on i>11, of course.

Answer 2

If you want to coerce the entire dataset to the Latin encoding, this should do:

data clean ( endoding= "latin1" );
    set dirty ( encoding= "asciiany" );
run;