Adding Secondary index on Cassandra indexes historical data?

Question

if on a particular column family i add a index on a column later on will it index the historical data too or data which comes now after adding the index.

Here in this When does Cassandra DB index data after updating a column as secondary index The accepted answer says it will index only data which is inserted after creating the index.

I tried creating a CF with index on a column.(i am using Cassandra 1.0.7)

create column family users with comparator=UTF8Type and column_metadata=[{column_name: full_name, validation_class: UTF8Type}, {column_name: birth_date, validation_class: LongType, index_type: KEYS}, {column_name: state, validation_class: UTF8Type, index_type: KEYS}];

Added some data , then did

removed index by drop index users.birth_date then added it back by updating CF

update column family users with comparator=UTF8Type and column_metadata=[{column_name: full_name, validation_class: UTF8Type}, {column_name: birth_date, validation_class: LongType, index_type: KEYS}, {column_name: state, validation_class: UTF8Type, index_type: KEYS}];

and then added some data again

But when i am querying on birth_data i get historical data too ?

Can someone clear my confusion on this ? Are there two ways to create index , one with historical data and one without ?

Answer 1

Maybe the previous version of Cassandra didn't build indexes for historical data, but according to the code post Cassandra 1.2, the index creation is an async process that does happen on historical data if you add a secondary index:

https://github.com/apache/cassandra/blob/cassandra-1.2.15/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java#L240

In your scenario, what has happened is that you removed the index and added the index. Because the old index files where already loaded and not removed from disk, Cassandra linked them for usage again. Otherwise, it would have attempted to create them.

In case you are not sure about your secondary indexes being in sync, you can use:

nodetool rebuild_index