How to generate an Id using DataImportHandler?

StackOverflow https://stackoverflow.com/questions/10381298

  •  04-06-2021
  •  | 
  •  

Question

I'm new in Solr and I'm struggling to import some XML Data which does not contain a ID field, although It's required as it says my schema.xml:

An XML example:

<results>
<estacions>
<estacio id="72400" nom="Aeroport"/>
<estacio id="79600" nom="Arenys de Mar"/>
...
</estacions>
</results>

Schema.xml:

<uniqueKey>id</uniqueKey>

At this point, I need to import this xml from http fetch, then I use DataimportHandler. This is my data-config.xml

<dataConfig>
    <dataSource type="URLDataSource" />
    <document>
            <entity name="renfe"                        
                    url="http://host_url/myexample.xml"
                    processor="XPathEntityProcessor"
                    forEach="/results/estacions/estacio"
                    transformer="script:generateCustomId">
                    <field column="idestacio"   xpath="/results/estacions/estacio/@id" commonField="true" />
                    <field column="nomestacio"  xpath="/results/estacions/estacio/@nom" commonField="true" />
            </entity>
    </document>

Then, it seems to work properly, but I got the following error: org.apache.solr.common.SolrException: [doc=null] missing required field: id

This makes me think that I should generate an automatic id while importing, and by using the data-config.xml, but I don't reach to see how to do it.

How should I do? Using a ScriptTransformer? Any idea is grateful

And another question: Can I force a value during the import ?

For ex: <field column="site" value="estacions"/> (obviously this does not work)

Was it helpful?

Solution

You can use code below to generate ID:

<dataConfig>
  <script><![CDATA[
        id = 1;
        function GenerateId(row) {
            row.put('id', (id ++).toFixed());
            return row;
        }
       ]]></script>
    <dataSource type="URLDataSource" />
    <document>
            <entity name="renfe"                        
                    url="http://host_url/myexample.xml"
                    processor="XPathEntityProcessor"
                    forEach="/results/estacions/estacio"
                    transformer="script:GenerateId">
                    <field column="idestacio"   xpath="/results/estacions/estacio/@id" commonField="true" />
                    <field column="nomestacio"  xpath="/results/estacions/estacio/@nom" commonField="true" />
            </entity>
    </document>
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top