Try using temporary table to load all of your data into. Then transfer the data from your temporary table into your table using a generic insert into ... select * from temporary table where column is not null;
The Oracle loader is not really the best place to try do manipulations/clauses etc. on data files. Rather ensure your file is as correct as possible before you try load it (i.e. run it through a perl parser or something before), or load into a temp table do your manipulations from a temporary table into your final table.