The best realtime search platform for realtime indexing of large db?

Question

I am building a site atm which requires realtime indexing of results (not 10,000 docs per second, I mean millisecond updates). I went anout researching the different techs and originally came up with dozens of different platforms. I have been able to narrow my choices down to about 3 through the use of deduction (doc complexity, different types of support etc):

• Lucence
• Xapian
• Sphinx

I originally tried to choose between these by the sites using them but then, to my surprise, lots and lots of high profile sites trust all three of these. I also found that all three of these also allow millisecond updates.

I thought about Sphinx originally because it is the only one of the three to say full realtime indexing instead of near realtime indexing only to find it is still in beta (not sure how reliable this tech would be in realtime indexing tbh).

I am leaning towards lucene since when solr gets realtime indexing moving my schema to solr will be insanely easy.

I am also leaning towards Xapian because a number of sites I know implement it very well.

I am having huge problems deciding between these techs and which one would be best suited.

I am looking at a site with millions maybe even tens of millions of records that needs an index that can be appended/deleted/updated in realtime.

Can anyone share their experiences on working with realtime search platforms to help me choose the right one for me? I am open to suggestions that are not here :).

P.S I use MongoDB so don't post SQL only search platforms please :).

Answer 1

I am answering this question with what I found, after a couple of weeks, was the best option.

I found Lucene the best actually since Zoies user base was, is.....**. I wanted to post a topic on the google group (the only form of support) and so far a couple of weeks later it still has not been moderated and approved for display.

That really put me off Zoie so in the end I decided to give Lucene a try.

Thanks anyway :).

Answer 2

I would recommend zoie based on lucene.