ELK企业级日志分析平台——ES集群监控

官网：https://www.elastic.co/guide/en/elasticsearch/reference/7.6/configuring-tls.html#node-certificates

在elk1上生成证书

[root@elk1 ~]# cd /usr/share/elasticsearch/[root@elk1 elasticsearch]# bin/elasticsearch-certutil ca[root@elk1 elasticsearch]# bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12[root@elk1 elasticsearch]# cp elastic-certificates.p12 /etc/elasticsearch[root@elk1 elasticsearch]# cd /etc/elasticsearch/[root@elk1 elasticsearch]# chown elasticsearch elastic-certificates.p12

[root@elk1 elasticsearch]# vim elasticsearch.yml...
http.cors.allow-headers: Authorization,X-Requested-With,Content-Length,Content-Typexpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /etc/elasticsearch/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /etc/elasticsearch/elastic-certificates.p12

[root@elk1 elasticsearch]# systemctl  restart elasticsearch.service

拷贝证书到其它集群节点

[root@elk1 elasticsearch]# scp elastic-certificates.p12 server2:/etc/elasticsearch/[root@elk1 elasticsearch]# scp elastic-certificates.p12 server3:/etc/elasticsearch/

elk2配置

[root@elk2 ~]# chown elasticsearch /etc/elasticsearch/elastic-certificates.p12[root@elk2 ~]# vim /etc/elasticsearch/elasticsearch.yml...
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /etc/elasticsearch/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /etc/elasticsearch/elastic-certificates.p12[root@elk2 ~]# systemctl  restart elasticsearch.service

elk3配置

[root@elk3 ~]# chown elasticsearch /etc/elasticsearch/elastic-certificates.p12[root@elk3 ~]# vim /etc/elasticsearch/elasticsearch.yml...
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /etc/elasticsearch/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /etc/elasticsearch/elastic-certificates.p12[root@elk3 ~]# systemctl  restart elasticsearch.service

等待所有集群节点重启成功后，设置认证账户密码

[root@elk1 elasticsearch]# cd /usr/share/elasticsearch/[root@elk1 elasticsearch]# bin/elasticsearch-setup-passwords interactive

交互式输入密码

ES启用xpack之后，其它服务都需要配置认证

elasticsearch-head插件

head插件在访问时，需要在url中输入用户和密码

[root@k8s1 ~]# cd elasticsearch-head-master/[root@k8s1 elasticsearch-head-master]# npm run start &

访问:192.168.92.11:9100/?auth_user=elastic&auth_password=westos

cerebro插件

cerebro访问：

[root@k8s1 ~]# docker start cerebro

访问：192.168.92.11:9000

​

kibana访问

[root@elk5 kibana]# vim kibana.yml
...
elasticsearch.username: "kibana"
elasticsearch.password: "westos"[root@elk5 kibana]# systemctl  restart kibana.service

​

metricbeat监控

官网：https://www.elastic.co/guide/en/elasticsearch/reference/7.6/configuring-metricbeat.html

首先使用内部检测，然后进入设置模式

​

选择启动metricbeat的节点，然后根据提示步骤操作

安装软件

[root@elk1 ~]# rpm -ivh metricbeat-7.6.1-x86_64.rpm

启动elasticsearch-xpack模块

[root@elk1 ~]# cd /etc/metricbeat/modules.d[root@elk1 modules.d]# metricbeat modules enable elasticsearch-xpack

编辑elasticsearch-xpack模块配置文件

[root@elk1 modules.d]# vim elasticsearch-xpack.yml- module: elasticsearchmetricsets:- ccr- cluster_stats- enrich- index- index_recovery- index_summary- ml_job- node_stats- shardperiod: 10shosts: ["http://localhost:9200"]username: "remote_monitoring_user"password: "westos"xpack.enabled: true

编辑metricbeat配置文件

[root@elk1 modules.d]# cd ..[root@elk1 metricbeat]# vim metricbeat.yml...
output.elasticsearch:hosts: ["http://192.168.56.11:9200"]username: "elastic"password: "westos"

启动metricbeat服务

[root@elk1 metricbeat]# systemctl enable --now metricbeat.service

​

其它节点依次类推

​

最后退出设置模式

kibana 监控

安装软件

[root@elk5 ~]# rpm -ivh metricbeat-7.6.1-x86_64.rpm

启动kibana-xpack模块

[root@elk5 ~]# cd /etc/metricbeat/modules.d/[root@elk5 ~]# metricbeat modules enable kibana-xpack

编辑kibana-xpack模块配置文件

[root@elk5 modules.d]# vim kibana-xpack.ymlusername: "elastic"password: "westos"

编辑metricbeat配置文件

[root@elk5 modules.d]# cd ..[root@elk5 metricbeat]# vim metricbeat.ymlhosts: ["http://192.168.92.31:9200"]username: "elastic"
password: "westos"

启动metricbeat服务

[root@elk5 ~]# systemctl enable --now metricbeat.service

禁用内部检测

[root@elk5 ~]# vim /etc/kibana/kibana.ymlxpack.monitoring.kibana.collection.enabled: false

重启kibana服务

[root@elk5 ~]# systemctl restart kibana.service

filebeat日志采集

文档：https://www.elastic.co/guide/en/beats/filebeat/7.6/filebeat-module-elasticsearch.html

安装软件

[root@elk1 ~]# rpm -ivh filebeat-7.6.1-x86_64.rpm

启动elasticsearch模块

[root@elk1 ~]# cd /etc/filebeat/modules.d[root@elk1 modules.d]# filebeat modules enable elasticsearch

编辑elasticsearch模块配置文件

[root@elk1 modules.d]# vim elasticsearch.yml- module: elasticsearch# Server logserver:enabled: true# Set custom paths for the log files. If left empty,# Filebeat will choose the paths depending on your OS.var.paths:- /var/log/elasticsearch/*.log          # Plain text logs- /var/log/elasticsearch/*_server.json  # JSON logsgc:enabled: true# Set custom paths for the log files. If left empty,# Filebeat will choose the paths depending on your OS.var.paths:- /var/log/elasticsearch/gc.log.[0-9]*- /var/log/elasticsearch/gc.logaudit:enabled: true# Set custom paths for the log files. If left empty,# Filebeat will choose the paths depending on your OS.var.paths:- /var/log/elasticsearch/*_access.log  # Plain text logs- /var/log/elasticsearch/*_audit.json  # JSON logsslowlog:enabled: true# Set custom paths for the log files. If left empty,# Filebeat will choose the paths depending on your OS.var.paths:- /var/log/elasticsearch/*_index_search_slowlog.log     # Plain text logs- /var/log/elasticsearch/*_index_indexing_slowlog.log   # Plain text logs- /var/log/elasticsearch/*_index_search_slowlog.json    # JSON logs- /var/log/elasticsearch/*_index_indexing_slowlog.json  # JSON logsdeprecation:enabled: true# Set custom paths for the log files. If left empty,# Filebeat will choose the paths depending on your OS.var.paths:- /var/log/elasticsearch/*_deprecation.log   # Plain text logs- /var/log/elasticsearch/*_deprecation.json  # JSON logs

编辑filebeat配置文件

[root@elk1 modules.d]# cd ..[root@elk1 filebeat]# vim filebeat.yml...
output.elasticsearch:hosts: ["http://192.168.92.31:9200"]username: "elastic"password: "westos"

启动filebeat服务

[root@elk1 filebeat]# systemctl  enable --now filebeat.service

​

其它节点依次类推

​