Capturing and Restoring Snapshots in
Elasticsearch
Accelirate, Inc.
August 28th, 2019
Revision 1.0
1. Contents
3.1 Snapshot and Restoring of an Elasticsearch Data Cluster......................................................................2
3.2 Before Taking A Snapshot......................................................................................................................2
3.2.1 Editing the elasticsearch.yml File....................................................................................................2
3.2.1.1 Locate the Version of Elasticsearch and Kibana that are currently being used........................2
3.2.1.2 Access and Find Currently Running Elasticsearch and/or Kibana Services...............................4
3.2.1.3 Locate the Correct Elasticsearch.yml File.................................................................................5
3.2.1.4 Adding a directory to add the snapshot...................................................................................5
3.2.1.5 Restart.....................................................................................................................................5
3.3 Creating the snapshot............................................................................................................................5
3.3.1 Kibana Dev Tool Snapshot..............................................................................................................6
3.3.2 Curl Request Snapshot....................................................................................................................7
3.4 Deleting a Snapshot...............................................................................................................................8
3.4.1 Delete with Kibana.........................................................................................................................8
3.4.2 Delete with a Curl Request.............................................................................................................8
3.5 Restoring a Snapshot.............................................................................................................................8
3.5.1 Kibana Restore................................................................................................................................8
2. Document Revision
Date Version Number/Editor Document Changes
08/28/2019 1.0/ Justin Keefe Initial Draft
3.1 Snapshot and Restoring of an Elasticsearch Data
Cluster
The purpose of this document is to specify how to take snapshots of Elasticsearch Data Clusters on
Windows Servers using the Development Tools in Kibana or as Curl Requests to your Elasticsearch Data
Cluster without Kibana. This document assumes you have a basic understanding of Elasticsearch and
Kibana. This document will also mention some of the issues I ran into while performing a snapshot.
Disclaimer*: The information and screenshots were taken on an actual windows server environment. Test data for Elasticsearch
was acquired from https://www.elastic.co/guide/en/kibana/current/tutorial-sample-data.html, so no sensitive information was
used.
3.2 Before Taking A Snapshot
3.2.1 Editing the elasticsearch.yml File
First, you will need to locate the correct elasticsearch.yml file. This can sometimes be difficult because
there can be more than one instance of Configuration files found in the directory that Elasticsearch is
saved in or even multiple instances of Elasticsearch running at the same time.
3.2.1.1 Locate the Version of Elasticsearch and Kibana that are currently
being used.
Using Kibana
Navigate to the Kibana instance you are currently using to access Elasticsearch. Locate the Dev Tools
section on the side menu and click on it. Type the command:
Then Click on the Green Arrow and your results will be returned in a json format. Your values for the
categories may vary.
The Elasticsearch version will be found here:
The version of Kibana you are currently using can be found from the Kibana instance you are using. Click
on the Management tab and you will see displayed in the top of the page the version of Kibana you are
using. It is best practice to use the same version of Kibana and Elasticsearch.
Without Kibana
Without Kibana you will need to send a curl request. To send a curl request you will need to go to
command prompt. You will then need to run this command:
That will be the output and you can see the version number for Elasticsearch. There will be no version
for Kibana because you are not using it.
3.2.1.2 Access and Find Currently Running Elasticsearch and/or Kibana
Services
Elasticsearch and Kibana are ran as services through windows. This makes it easiest to find which Kibana
and Elasticsearch are running and where to look for the files you need.
To access the services panel in windows you will need to go to the search bar and type Services:
You will then click on the Services Panel and will see several services. Locate the ones that say
Elasticsearch and Kibana.
If your instance of Kibana and Elasticsearch are currently able to be accessed than they are running. This
will help to make your search for the correct version of Elasticsearch. You will then right click on the
running service for just Elasticsearch and then you will click the properties tab.
The path to the executable will show you where the Elasticsearch instance you are looking for is located.
3.2.1.3 Locate the Correct Elasticsearch.yml File
There may be multiple elasticsearch.yml files depending if you have had upgrades to Elasticsearch
before or depending on which version you are using. For the most part, when you install Elasticsearch it
will by default place the configurations file in the Program Data folder.
You will then navigate to the configuration file section and open the elasticsearch.yml. If the file does
not look like this then you are not in the right place.
You can try in the actual Elasticsearch directory that your instance is saved inside the config folder if you
do not have a configuration file here. Your configuration file should not have commented out sections in
it denoted by the # symbol. If most of the page is commented out, you are using the wrong
configuration file.
3.2.1.4 Adding a Directory to Add the Snapshot
Create a directory outside of your Elasticsearch directory to store the snapshot. You must specify the
location of your path in your elasticsearch.yml. It must be placed in your path.repo variable inside your
configuration file elasticsearch.yml.
3.2.1.5 Restart Elasticsearch and/or Kibana Services
Now you must restart your Elasticsearch service and your Kibana service (if you are using Kibana). From
the service panel right click on the service and then click restart. The service will take a few minutes to
restart.
Now you are ready to create a snapshot.
3.3 Creating the snapshot
To create a snapshot, you will have to run one command. You can run the command from Kibana or as a
curl request.
3.3.1 Kibana Dev Tool Snapshot
You will have to go to the Dev Tools tab on Kibana and type the command:
The PUT command tells Elasticsearch to create a copy of your directory. The /_snapshot/ calls the
command for the snapshot. The /es_backup_1/ is the directory that the snapshot should be placed in.
The /backup1 is the name of the snapshot.
If successful, the result will be:
If the response says acknowledged it is not saved correctly.
To check on the status of your snapshot you can type the command:
The result will be:
You will now have a saved snapshot of all the data in your Elasticsearch data cluster including the global
state cluster. The global state cluster contains all metadata (schema, location, size) for all shards in the
entire cluster and is kept current on every node in the cluster.
3.3.2 Curl Request Snapshot
The curl request will need to be ran from the command line.
The command will be:
curl -X PUT “(your-host-name-here):9200/_snapshot/(your-directory-here)/(your-snapshot-name-here)?
pretty” – H ‘Content-Type: application/json’ -d ‘ {“type” : “fs”, “settings”: { “location”: “C:/(your-
directory-here)”, “compress”: true}} ‘
You will get the same responses as you would from Kibana if it was successful.
3.4 Deleting a Snapshot
There are two ways to delete a snapshot with Kibana or a curl request
3.4.1 Delete with Kibana
Run the delete command from Kibana in the Dev Tools Section. The command is to delete one snapshot:
To Delete a whole repository from Kibana Dev Tools. Run the command:
3.4.2 Delete with a Curl Request
Run the delete command from the cmd. The command is to delete one snapshot:
curl -X DELETE "(your-host-name):9200/_snapshot/(your-directory-here)/(your-snapshot-name-here)?
pretty"
To Delete a whole repository from the cmd. The command is:
curl -X DELETE "(your-host-name):9200/_snapshot/(your-directory-here)?pretty"
3.5 Restoring a Snapshot
To restore a snapshot, you must have a snapshot and run a specific command. This can be done through
a curl request or the Kibana Dev Tools.
3.5.1 Kibana Restore
To restore the basic snapshot with no added features, use the command:
To restore the snapshot with added features, use the command:
*I would advise restoring of the global cluster state to as accurately as possibly replace the cluster. Using
the include_global_state as true.
You will now need to wait until the cluster has fully been restored. This could take some time. To get the
status of the currently running screenshot type the command:
3.5.2 Curl Request Restore
To do the basic restore you run the curl command from the windows cmd:
curl -X POST "(your-local-host-name):9200/_snapshot/(your-directory-name)/(your-snapshot-name)/
_restore?pretty"
To restore the snapshot with added features:
curl -X POST "localhost:9200/_snapshot/my_backup/snapshot_1/_restore?pretty" -H 'Content-Type:
application/json' -d'
{
"indices": "index_1,index_2",
"ignore_unavailable": true,
"include_global_state": true,
"rename_pattern": "index_(.+)",
"rename_replacement": "restored_index_$1"
}'
*I would advise restoring of the global cluster state to as accurately as possibly replace the cluster. Using
the include_global_state as true.
You will now need to wait until the cluster has fully been restored. This could take some time. To get the
status of the currently running screenshot type the command:
curl -X GET "(your-host-name-here):9200/_snapshot/(your-directory-here)/(your-snapshot-here)/
_status?pretty"
