Scheduled import from PostgreSQL to Solr4

Question

We have 'Article' table in PostgreSQL db with following structure (simplified):

CREATE TABLE article
(
  id integer,
  header text,
  body text,
  isSolrized boolean
)

I would like to import all non-solrized rows to solr4 on a regular basis.

Is there a tool or built-in solr's mechanism to do this?

I'm going to implement my own import tool with blackjack and hookers and run it using cron. Is there a better way?

Answer 1

Opted for DataImportHandler.

1. Download jdbc driver for postgresql and put it in dist/ dir

2. Add modules to config file:

   <lib dir="../../dist/" regex="apache-solr-dataimporthandler-\d.*\.jar" />
   <lib dir="../../dist/" regex="postgresql-\d.*\.jar" />
   

3. Follow instructions at Solr wiki

4. Remember to declare your scheme right in scheme.xml

Done in 30 minutes.

Answer 2

Implementing a Foreign data wrapper in PostgeSQL (details and examples here) may be better. It can contain any logic you desire.

After implementing, it can be used in a simple trigger, or called with cron (or PostgreSQL own planner).

Such thing could be useful to other people to, if you opensource it.