Using ElasticSearch to store data without indexing or analysis (NEST client)

Question

We are using ES via the NEST client for search, and we'd like to try to leverage it to store some reports that the system generates as well.

The reports are strings containing CSV data and they can be quite large, 100mb+, and we've run into some problems. First we were exceeding the 100mb limit set in the http config, so I increased that and the error stopped.

Now we're getting System.OutOfMemoryExceptions.

With the reports, we don't need to analyse them or have them tokenized and indexed. We just need to be able to get them back out by their ID to send along to the browser. I haven't had a lot of luck finding details on how to use ES as a dumb key-value store though, or if that will help with the memory problem.

Additionally it crossed my mind to zip compress the data before sending it into ES, but again, not sure if that'd help or what would be involved.

Answer 1

Don't know how you have it currently configured, but you could try the string type with an index value of "not_analyzed" or "no". I'd also give the binary type a try. You should set store to "true" for either approach. That should prevent ES from attempting to analyze and index the field.

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-core-types.html