ElasticSearch 映射到 hdfs的快照

一、下载：

https://artifacts.elastic.co/downloads/elasticsearch-plugins/repository-hdfs/repository-hdfs-5.4.0.zip

二、安装

./bin/elasticsearch-plugin install file:///opt/elasticsearch-5.4.0/repository-hdfs-5.4.0.zip

三、插件的bug，遇到修改

vim /opt/elasticsearch-5.4.0/plugins/repository-hdfs/plugin-security.policy

1grant {
2  // Hadoop UserGroupInformation, HdfsConstants, PipelineAck clinit
3  permission java.lang.RuntimePermission "getClassLoader";
4
5  // UserGroupInformation (UGI) Metrics clinit
6  permission java.lang.RuntimePermission "accessDeclaredMembers";
7  permission java.lang.reflect.ReflectPermission "suppressAccessChecks";
8
9  // org.apache.hadoop.util.StringUtils clinit
10  permission java.util.PropertyPermission "*", "read,write";
11
12  // org.apache.hadoop.util.ShutdownHookManager clinit
13  permission java.lang.RuntimePermission "shutdownHooks";
14
15  // JAAS is used always, we use a fake subject, hurts nobody
16  permission javax.security.auth.AuthPermission "getSubject";
17  permission javax.security.auth.AuthPermission "doAs";
18  permission javax.security.auth.AuthPermission "modifyPrivateCredentials";
19## 注意，以下为我添加的信息
20  permission java.lang.RuntimePermission "accessDeclaredMembers";
21  permission java.lang.RuntimePermission "getClassLoader";
22  permission java.lang.RuntimePermission "shutdownHooks";
23  permission java.lang.reflect.ReflectPermission "suppressAccessChecks";
24  permission javax.security.auth.AuthPermission "doAs";
25  permission javax.security.auth.AuthPermission "getSubject";
26  permission javax.security.auth.AuthPermission "modifyPrivateCredentials";
27  permission java.security.AllPermission;
28  permission java.util.PropertyPermission "*", "read,write";
29  permission javax.security.auth.PrivateCredentialPermission "org.apache.hadoop.security.Credentials * \"*\"", "read";
30};

vim /opt/elasticsearch-5.4.0/config/jvm.options

1添加以下
2
3-Djava.security.policy=file:////opt/elasticsearch-5.4.0/plugins/repository-hdfs/plugin-security.policy

四、复制 hdfs-site.xml 到/opt/elasticsearch-5.4.0/config/

五、添加创建快照仓库

1PUT _snapshot/my_hdfs_repository/snapshot_url
2{
3  "type": "hdfs",
4  "settings": {
5    "uri": "hdfs://master200:8020",
6    "path": "/logs/elasticsearch/my_hdfs_repository",
7    "conf_location": "hdfs-site.xml"
8  }
9}

六、创建快照

1PUT _snapshot/my_hdfs_repository/snapshot_url
2{"indices":"url,url2"}

七、恢复数据

1备份恢复
2
3POST _snapshot/my_hdfs_repository/snapshot_url/_restore
4
5备份删除
6
7curl -XDELETE "localhost:9200/_snapshot/backup/snapshot_1"
8查看仓库信息
9
10curl -XGET 'http://localhost:9200/_snapshot/backup?pretty'