Question

I have the following fields:

  • In the database I have the field property_industry_sector which is a list of comma separated ints, null or empty string.
  • In the Solr schema configuration I have the same field property_industry_sector of type int and multivalued.

My problem is that I have to handle to difference in the DataImportHandler configuration, and my attempt looks like this:

  <entity
    name="property_industry_sector_extractor"
    transformer="script:SplitIndustrySector"
    query="
      SELECT property_industry_sector
      FROM   job
      WHERE  job.id = ${job.id}
    ">
    <field column="property_industry_sector" name="property_industry_sector" />
  </entity>

Where the ScriptTransformer has the following definition:

function SplitIndustrySector(row) {
  //var logger = java.util.logging.Logger.getLogger("org.apache.solr");

  if(row.get('property_industry_sector') !== null) {
    if(false === row.get('property_industry_sector').isEmpty()) {

      var pieces = row.get('property_industry_sector').split(',');
      var arr    = new java.util.ArrayList();
      for(var i=0, len=pieces.length; i<len; i++) {
        arr.add(new java.lang.Integer(pieces[i]));
      }

      row.put('property_industry_sector', arr);
      return row;
    }
  }

  var arr = new java.util.ArrayList();
  arr.add(new java.lang.Integer(0));
  row.put('property_industry_sector', arr);
  return row;
}

The problem is with the general case, when the value is null or empty string, because no matter what the transformer does, I still get the following Exception

property_industry_sector=property_industry_sector(1.0)={[, 0]}}]
java.lang.NumberFormatException: For input string: ""
    at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
    at java.lang.Integer.parseInt(Integer.java:493)
    at java.lang.Integer.parseInt(Integer.java:514)
    at org.apache.solr.schema.TrieField.createField(TrieField.java:374)
    at org.apache.solr.schema.SchemaField.createField(SchemaField.java:97)
    at org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:203)
    at org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:276)
    at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:60)
    at org.apache.solr.handler.dataimport.SolrWriter.upload(SolrWriter.java:73)
    at org.apache.solr.handler.dataimport.DataImportHandler$1.upload(DataImportHandler.java:294)
    at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:631)
    at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:267)
    at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:186)
    at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:353)
    at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:411)
    at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:392)

I do not understand where the empty string comes from (which it tries to convert to Integer) while also being confused by the values it tries to insert above the exception:

property_industry_sector=property_industry_sector(1.0)={[, 0]}}]

I've tried clearing the row prior to the put() call. Return null, or just as with the current example return the row with a single value of 0.

Was it helpful?

Solution

Haven't found a way to work it out, but managed to solve the issue with an alternative solution. Instead of using the ScriptTransformer I was able to achieve the same goal with SQL transformations.

<entity name="industry_sector_hack" query='
    SELECT property_industry_sector AS property_industry_sector_ids
    FROM   job
    WHERE  id = ${job.id} AND
           property_industry_sector IS NOT NULL AND
           property_industry_sector &lt;&gt; ""
  '>
  <entity name="property_industry_sector" query='
      SELECT property.id AS property_industry_sector
      FROM   property
      WHERE  property.id IN (${industry_sector_hack.property_industry_sector_ids})
    '>
    <field column="property_industry_sector" name="property_industry_sector" />
  </entity>
</entity>
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top