Data Re-Indexing

  • Every so often you may run into changes required in mapping for existing fields

  • As per our previous attempts updating mapping for existing fields is not possible

  • The only solution I am aware of: create new index with new mapping and move data over

  • To move data reliably in ETL (extract, transform, and load) like fashion - scan API is useful

  • Initiating a scroll:

    curl 'localhost:9200/logstash-2015.05.20/log/_search?scroll=1m'

  • To continue with scroll until no records left:

    curl -XGET  'localhost:9200/_search/scroll'  -d '
{
  "scroll" : "1m", 
  "scroll_id" : "c2Nhbjs2OzM0NDg1ODpzRlBLc0FXNlNyNm5JWUc1" 
}
'

  • What do you do with the data you receive?

  • How do you plan for re-indexing possibility without affecting ElasticSearch clients?

PreviousData SynchronizationNextHosted ElasticSearch

Last updated

Was this helpful?