Question

we are still using (re)Tire for elasticsearch in our production system.
My question is how to run import task for index and providing it with sorted data from mongoid. Eg. I want to index 90 million records while the system is accessible by users, and I would like to achieve that the last inserted records will be indexed first, so that users can find the most recent ones (and most relevant ones - order data by created_at).
So is there any way how I can achieve this by using:
rake environment tire:import CLASS=Document FORCE=true

Was it helpful?

Solution

I would look at manually importing them. Write your own script, there is an example in the readme something like

User.order('created_at DESC').find_in_batches do |batch|
  Tire.index("users").import batch
end

Or use their import tool:

User.index.import User.order('created_at DESC')

If your site is widely used I would would be tempted to spin up a different core or instance of solr, import everything and then switch your app to use the new one.

Hope that helps.

Edit:

If you are using rails, then make sure you include your rails environment in the script (config/environment). If it is going to take a long time to import then you might want to run the script with 'nohup ruby my_script.rb > script_out.log 2>&1 &' so it isn't bound to your ssh session.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top