Elasticsearch dump

It is often required to take a dump of all the data stored in elasticsearch for various reasons. Though elasticsearch provides Snapshot And Restore feature to persist and restore data from either a shared filesystem or remote repository you may still want to get a local dump.

elasticdump is a node module which can be used to get dump from elasticsearch index.

Step 1: Verify node is installed
To get started, first ensure node is installed on your local environment, if not, download and install node package from https://nodejs.org/en/download/

M25:dump nmn-notes$ node -v
v0.10.24

Step 2: Install elasticdump globally
M25:dump nmn-notes$ sudo npm install elasticdump -g

Step 3: Take dump
M25:dump nmn-notes$ elasticdump --input=http:///my_index --output=/tmp/es_dump/dump.json --type=data

Step 3 will fetch all data from my_index index to /tmp/es_dump/dump.json file. Each line in dump.json is going to be a json document.

You can also write a simple script and configure a cron job to run it periodically. Below is a sample script which will connect to localhost elasticsearch and take a dump of a requested index to an output directory.

es_dump.sh

M25:dump nmn-notes$ ./es_dump.sh my_index /tmp/es_data

The above command will take a dump from “my_index index and copy all its content to /tmp/es_data/my_index-2017-05-20:21-59-00.json file.

Advertisements