Oracle starts to do full table scans when a column is changed from varchar to nclob

Question

I have a table with about 100.000 rows that used to look more or less like this:

id      varchar(20),
omg     varchar(10),
ponies  varchar(3000)

When adding support for international characters, we had to redefine the ponies column to an nclob, as 3000 (multibyte) characters is too big for an nvarchar

id      varchar(20),
omg     varchar(10),
ponies  nclob 

We read from the table using a prepared statement in java:

select omg, ponies from tbl where id = ?

After the 'ponies' column was changed to an NCLOB and some other tables where changed to use nchar columns, Oracle 11g decided to do a full table scan instead of using the index for the id column, which causes our application to grind to a halt.

When adding a hint to the query, the index is used and everything is "fine", or rather just a little bit more slow than it was when the column was a varchar.

We have defined the following connection properties:

 oracle.jdbc.convertNcharLiterals="true"
 defaultNChar=true

Btw, The database statistics are updated.

I have not had time to look at all queries, so I don't know if other indexes are ignored, but do I have to worry that the defaultNChar setting somehow is confusing the optimizer since the id is not a nchar? It would be rather awkward to either sprinkle hints on virtually all queries or redefine all keys.

Alternatively, is the full table scan regarded as insignificant as a "large" nclob is going to be loaded - that assumption seems to be off by 3 orders of magnitude, and I would like to believe that Oracle is smarter than that.

Or is it just bad luck? Or, something else? Is it possible to fix without hints?

Answer 1

The problem turns out to be the jdbc-flag defaultNChar=true.

Oracles optimizer will not use indexes created on char/varchar2 columns if the parameter is sent as a nchar/nvarchar. This is nearly making sense, as I suppose you could get phantom results.

We are mostly using stored procedures, with the parameters defined as char/varchar2 - forcing a conversion before the query is executed - so we didn't notice this effect except in a few places where dynamic sql is used.

The solution is to convert the database to AL32UTF8 and get rid of the nchar columns.

Answer 2

When you redid the statistics did you estimate or use dbms_stats.gather_table_stats with an estimate_percentage > 50%? If you didn't then use dbms_stats with a 100% estimate_percentage.

If your table is only 3 columns and these are the ones you're returning then the best index is all 3 columns no matter what you hint and even if the id index is unique. As it stands your explain plan should by a unique index scan followed by a table access by rowid. If you index all 3 columns this becomes a unique scan as all the information you're returning will be in the index already and there's no need to re-access the table to get it. The order would be id, omg, ponies to make use of it in the where clause. This would effectively make your table an index organized table, which would be easier than having a separate index. Obviously, gather stats after.

Saying all that I'm not actually certain you can index a nclob and no matter what you do the size of the column will have an impact as the longer it is the more disk reads you will have to do.

Answer 3

Sorry, but I don't understand why had you change your column ponies from varchar to clob. If your maximum lenght is 3000 char in this column, why don't you use a NVARCHAR2 column instead? As far as I know, nvarchar2 can hold up to 4000 characters.

But you're right, the maximum column size allowed is 2000 characters when the national character set is AL16UTF16 and 4000 when it is UTF8.