OSS logstash with AWS Opendistro for Elasticsearch¶
To get logstash talking to the Open Distro Elasticsearch the first thing that should be understood is that open distro only works with the OSS (Apache 2.0 Licensed) edition of the Elastic tools and not the Elastic licensed edition (Xpack).
Open Distro = AWS sponsored Apache 2.0 licensed distro (there is no logstash distributed by Open Distro)
OSS = Apache 2.0 licensed version from Elastic
X-Pack = Original Elastic licensed version with the enterprise Xpack features (Not compatible with the Apache licensed stuff)
If you point logstash output to a non OSS version of Elastic you will see this error on the OSS Elasticsearch logs (and a 500 error in the logstash logs). Ensure you do not use the Elastic licensed edition of Logstash and use the pure Apache 2.0 licensed distribution (OSS).
ddfe-node1] Unexpected exception [_license] InvalidIndexNameException[Invalid index name [_license], must not start with '_'.
If you get the following error then see below on how to fix the trust store
PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target"}
To get TLS trust working, open the elastic search endpoint in a browser and inspect the https certificate to export it to a binary encoded .DER format. Import it into the java trust store. Note that the OSS edition of logstash ships with its own JRE (there is also a version shipping without it) so ensure you install the certificate into the correct keystore (not the system Java) (it does not matter what the alias is it just needs to be unique to that keystore)
sudo keytool -importcert -alias local-CAes2 -keystore /home/brent/esoss/logstash-7.10.0/jdk/lib/security/cacerts -file ~/es2.der.cer
The next issue is the Elastic search output in the logstash config will error with the below. To fix it, disable the Index lifecycle management (ILM) on the elastic output as follows:
output {
elasticsearch {
hosts => ["https://localhost:9200"]
index => "brent"
user => "admin"
password => "admin"
# Disable ilm for compatibility with open distro elastic
ilm_enabled => false
}
}
[2020-11-22T20:24:42,514][ERROR][logstash.outputs.elasticsearch][main] Failed to install template. {:message=>"Got response code '500' contacting Elasticsearch at URL 'https://localhost:9200/_xpack'", :class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError", :backtrace=>["/home/brent/esoss/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/http_client/manticore_adapter.rb:80:in `perform_request'", "/home/brent/esoss/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:332:in `perform_request_to_url'", "/home/brent/esoss/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:319:in `block in perform_request'", "/home/brent/esoss/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:414:in `with_connection'", "/home/brent/esoss/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:318:in `perform_request'", "/home/brent/esoss/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:326:in `block in Pool'", "/home/brent/esoss/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/http_client.rb:162:in `get'", "/home/brent/esoss/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/http_client.rb:378:in `get_xpack_info'", "/home/brent/esoss/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/ilm.rb:57:in `ilm_ready?'", "/home/brent/esoss/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/ilm.rb:28:in `ilm_in_use?'", "/home/brent/esoss/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/template_manager.rb:15:in `install_template'", "/home/brent/esoss/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/common.rb:218:in `install_template'", "/home/brent/esoss/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/common.rb:49:in block in setup_after_successful_connection'"]}
warning: thread "Ruby-0-Thread-18: :1" terminated with exception (report_on_exception is true):
LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError: Got response code '500' contacting Elasticsearch at URL 'https://localhost:9200/_xpack'
perform_request at /home/brent/esoss/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/http_client/manticore_adapter.rb:80
perform_request_to_url at /home/brent/esoss/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:332
perform_request at /home/brent/esoss/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:319
with_connection at /home/brent/esoss/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:414
perform_request at /home/brent/esoss/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:318
Pool at /home/brent/esoss/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:326
get at /home/brent/esoss/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/http_client.rb:162
get_xpack_info at /home/brent/esoss/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/http_client.rb:378
ilm_ready? at /home/brent/esoss/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/ilm.rb:57
ilm_in_use? at /home/brent/esoss/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/ilm.rb:28
setup_after_successful_connection at /home/brent/esoss/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/common.rb:50
Comments
comments powered by Disqus