Is it advisable to use only Lucene index without Database?

Question

I am developing an iOS application which would consist of approx 600 book each about 90 pages (.pdf docs) in length. The approach I had taken was to place all the text in a database separated into paragraphs, which I then called and displayed. To improve searching speed, I indexed all the required data from the database using CLucene and realized, I now have two copies of my data, and I'm getting better access times from the index rather than the database as expected.

My question is: Is it advisable or recommended to use the application without the database and just draw all the data from the index, even in cases where searches are not being executed. This would improve both the speed and size of my application as the database would not be included but is it advisable?

I would greatly appreciate any assistance/advice on this as I'm inexperience in this area and its the first time I'm using any sort of search API.

Answer 1

Don't worry too much about the duplicate data. The thing you do have to watch though, is ensuring that only the necessary data is stored in your Lucene indexes. So, if you aren't actually going to display full chapters directly from the index, don't store them, just index them. This way, it's not duplicate data.

Also, you will find longer-term that if you want the data to be editable, maintainable and backup-able, it will be easier to keep it stored in a database. However, this does not mean that you have to use a relational database like MySQL or MS SQL. There are many NoSQL databases options too.

I won't go into too much detail around the pros and cons of NoSQL because you can google that. But you could for instance use MongoDB to store and serve all your data instead of using your database and Lucene and it would still be very quick and a scalable solution.