Elasticsearch Curate Features

• Alias Management – add, remove
• Shard routing allocation
• Indices Management – Close , Delete indices, Open closed indices, Optimize indices and modify number of replicas
• Snapshot(backups management) – Show, Backup, Restore.
• Change the number of replicas per shard for indices
• Pattern Matching for statements (e.g delete all indices matching .marvel*)

 

• Snapshots Index creation
  • All indices except Marvels regex support
• Restore from snapshots (using the Elasticsearch end point via curl ….. Not with curator currently)
  • All indices in the back up
  • Specific index from the backup
  • Keep cluster state as is
• Delete indices
  • Older than a date range
  • By a regex matching pattern

Background

• Started off as clearESindices.py for a simple goal to delete indices.
• It then became logstash_index_cleaner.py
• Then moved under logstash repository as “expire_logs”.
• After Jordan Sissel was hired by Elastic it then became Elasticsearch Curator and is now hosted at https://github.com/elastic/curator
• Today : Curator now performs many operations on your Elasticsearch indices, from delete to snapshot to shard allocation routing.

 

Curator all-in Command Line

• Curator 2.0 underlying feature was the first attempt to detach the popular Elasticsearch API from the CLI.
• These allows scripting to carry out any of the tasks featured above meaning you can for instance:
  • Automate your back up and restore procedures.
• Also with version 2.0, Curator ships in an API along side the wrapper scripts/ entry points. This API allows you to roll out your own scripts to perform similar or totally different tasks to Curator with the same underlying code that curator uses…
• Documentation can be found here

 

Curator installation for Mac

[code language=”bash”]

# you may ignore the python installations if you already have this
wget https://bootstrap.pypa.io/get-pip.py
python get-pip.py

wget https://bootstrap.pypa.io/get-pip.py
sudo python get-pip.py

wget https://pypi.python.org/packages/source/u/urllib3/urllib3-1.8.3.tar.gz
pip install urllib3-1.8.3.tar.gz

wget https://pypi.python.org/packages/source/c/click/click-3.3.tar.gz -O click-3.3.tar.gz
sudo pip install click-3.3.tar.gz

# install elasticsearch-py
wget https://github.com/elastic/elasticsearch-py/archive/1.6.0.tar.gz -O elasticsearch-py.tar.gz
sudo pip install elasticsearch-py.tar.gz

# install elasticsearch-curator
wget https://github.com/elastic/curator/archive/v3.3.0.tar.gz -O elasticsearch-curator.tar.gz
sudo pip install elasticsearch-curator.tar.gz

# To test : Verify version
curator –version
# should echo: "curator, version 3.3.0"

# To example Commands try the help menu
curator –help
# should echo : help menu options

[/code]

For more installation guides please see the official Elasticsearch Guides

 

Curator Command Line Flags

 

Index and Snapshot Selection

–newer-than

–older-than

–prefix

–suffix

–time-unit

–timestring

–regex

–exclude

Index selection only

–index

–all-indices

Snapshot selection only

–snapshot

–all-snapshots

–repository