Excluding mongodb field from elasticsearch river import

Question

I have installed an elasticsearch river connecting mongodb with elasticsearch (using this plugin). I have a certain schema defined for mongodb and a certain mapping defined for elasticsearch.

What I cannot find is any info on how to exclude fields from mongodb from being processed by the river. The closest thing that I can see being feasible is modifying the elasticsearch mapping for those fields to store:no,index:not_analyzed,include_in_all:false.

I am still not clear whether using the aforementioned solution avoids any processing at all, since theoretically all json submissions are also stored in _source.

Anyone have any suggestions regarding this?

Answer 1

Yes, you can do in mongo-river.you need to specify exclude fields in river mapping.. i ve attached a sample code.. check it out..

curl -XPUT 'http://IP:9200/_river/mongodb/_meta' -d '{
"type": "mongodb", 
"mongodb": { 
    "db": "DATABASE_NAME", 
    "collection": "COLLECTION", 
    "gridfs": true
}, "options": { 
  "secondary_read_preference" : true, 
  "drop_collection": ${mongo.drop.collection}, 
  "exclude_fields": ${mongo.exclude.fields},
  "include_fields": ${mongo.include.fields},
  "include_collection": ${mongo.include.collection},
  "import_all_collections": ${mongo.import.all.collections},
  "initial_timestamp": {
    "script_type": ${mongo.initial.timestamp.script.type},
    "script": ${mongo.initial.timestamp.script}
  },
  "skip_initial_import" : ${mongo.skip.initial.import},
  "store_statistics" : ${mongo.store.statistics},
},

"index": { 
    "name": "ES_INDEX_NAME", 
    "type": "ES_TYPE_NAME" 
}
}'

for more information refer: https://github.com/richardwilly98/elasticsearch-river-mongodb/wiki

HOpe it helps..!