Symptom
- Elasticsearch is not available after some Elasticsearch indices have been deleted accidentally from folder /usr/share/elasticsearch/data
- To troubleshoot the problem, switching Elasticsearch to Maintenance Mode did not help: command kubectl -n $NAMESPACE exec diagnostics-elasticsearch-0 -it -- /usr/local/bin/docker-entrypoint.sh results in error:
[ERROR][o.e.g.GatewayMetaState ] [diagnostics-elasticsearch-master] failed to read or upgrade local state, exiting...
java.io.IOException: failed to find metadata for existing index logstash-2020.04.24 [location: 5Kg8xlC4Q3Wm7ser4FsT8g, generation: 48]
at org.elasticsearch.gateway.MetaStateService.loadFullState(MetaStateService.java:99) ~[elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.gateway.GatewayMetaState.upgradeMetaData(GatewayMetaState.java:141) [elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.gateway.GatewayMetaState.<init>(GatewayMetaState.java:95) [elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.node.Node.<init>(Node.java:492) [elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.node.Node.<init>(Node.java:258) [elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:221) [elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:221) [elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349) [elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) [elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) [elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) [elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) [elasticsearch-cli-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-cli-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) [elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) [elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
[2020-08-06T07:12:10,275][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [diagnostics-elasticsearch-master] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: ElasticsearchException[failed to bind service]; nested: IOException[failed to find metadata for existing index logstash-2020.04.24 [location: 5Kg8xlC4Q3Wm7ser4FsT8g, generation: 48]];
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
Caused by: org.elasticsearch.ElasticsearchException: failed to bind service
at org.elasticsearch.node.Node.<init>(Node.java:617) ~[elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.node.Node.<init>(Node.java:258) ~[elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:221) ~[elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:221) ~[elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349) ~[elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
... 6 more
Caused by: java.io.IOException: failed to find metadata for existing index logstash-2020.04.24 [location: 5Kg8xlC4Q3Wm7ser4FsT8g, generation: 48]
at org.elasticsearch.gateway.MetaStateService.loadFullState(MetaStateService.java:99) ~[elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.gateway.GatewayMetaState.upgradeMetaData(GatewayMetaState.java:141) ~[elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.gateway.GatewayMetaState.<init>(GatewayMetaState.java:95) ~[elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.node.Node.<init>(Node.java:492) ~[elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.node.Node.<init>(Node.java:258) ~[elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:221) ~[elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:221) ~[elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349) ~[elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.3.0-SNAPSHOT.jar:7.3.0-SNAPSHOT]
... 6 more
command terminated with exit code 1
Read more...
Environment
SAP Data Intelligence (DI) 3.0 Patch 2 (3.0.24)
Product
Keywords
KBA , CA-DI-ONP , SAP DI, on-premise: Please use CA-DI instead , CA-DI , Data Intelligence , Problem
About this page
This is a preview of a SAP Knowledge Base Article. Click more to access the full version on SAP for Me (Login required).Search for additional results
Visit SAP Support Portal's SAP Notes and KBA Search.
SAP Knowledge Base Article - Preview