Handling data truncation in Talend

Question

I am copying data from Excel sheet to the SQL server tables. In some of the sheets I have data bigger in size of the Table's schema in SQL.

i.e. Table's column has data type nvarchar(50) where as my Excel sheet has data of more than 50 characters in some of the shells.

Now while copying, the rows which has such data are not being inserted in to the database. Instead I would like to insert rows with such data by truncating extra characters. How do I do this?

Answer 1

You can use Java's substring method with a check to the length of the string with something like:

row1.foobar.length() > 50 ? row1.foobar.substring(0,50) : row1.foobar

This uses Java's String length method to test to see if it's longer than 50. If it is then it uses the substring method to get the characters between 0 and 50 (so the first 50 characters) and if it's not then it returns the whole string.

If you pop this in a tMap or a tJavaRow then you should be able to limit strings to 50 characters (or whatever you want with some tweaking):

If you'd prefer to remove any rows not compliant with your database schema then you should define your job's schema to match the database schema and then use a tSchemaComplianceCheck component to filter out the rows that don't match that schema.