Indexing old data with Neo4j and jRuby on Rails

Question

We recently switched to JRuby on Rails so that we can use Neo4j, and even though we've only scratched the surface of what Neo4j can do, it's already pretty great. That said, I just ran into a rather vexing problem with indexing and finding.

I just added ':index => :exact' to my User model's created_at field and my Post model's created_at field (there already was an index on my Post model's custom_date field):

class User < Neo4j::Rails::Model
  property :created_at, :type => DateTime, :index => :exact

class Post < Neo4j::Rails::Model
  property :created_at, :type => DateTime, :index => :exact
  property :custom_date, :type => DateTime, :index => :exact

I then went into rails console to tried this code (which I adapted from the documentation):

User.find( :all, :created_at => 7.days.ago .. DateTime.now ).count

And it returned 0, which I know isn't the case as this code:

new_users = []
User.all.each{ |user| new_users << user if user.created_at > 7.days.ago }
new_users.count

Returns 104.

Similarly, when I run

Post.find( :all, :created_at => 7.days.ago .. DateTime.now ).count

I get 0, but when I run

Post.find( :all, :custom_date => 7.days.ago .. DateTime.now ).count

I get 305. (A post's custom_date is initially set to the posts's created_at.) The only difference is I had the index in place on the Post model's custom_date field when we ported over the old database, whereas I only just added the index to the created_at field.

Here's my question: how do I search my users and posts by their created_at times?

Answer 1

The problem is that the old data does not have a lucene index. You have only declared an index but not updated the old nodes. You need to use the add_index on all old nodes.

You should do something like this:

Neo4j::Transaction.run do
  User.all.each do |user|
     # Since there is a type converter on created_at, we need to find the data
     # as it is stored in neo4j.
     val = User._converter(:created_at).to_java(user[:created_at])
     user.add_index(:created_at, val) 
  end
end

See the RSpec for this https://github.com/andreasronge/neo4j/commit/cffe7a4a2fdb83b71e781e717a2a8011e2ef494d

I've also created a github issue because I think it should be much easier to do this - https://github.com/andreasronge/neo4j/issues/220

Pull requests are welcome :)

Btw, in the old 0.4.6 version of neo4j there was support for doing this with migrations.