【ELK】容器化部署Elasticsearch1.14.3集群【亲测可用】

提示：文章写完后，目录可以自动生成，如何生成可参考右边的帮助文档

1. 部署

按照官网流程进行部署

使用 Docker 安装 Elasticsearch |Elasticsearch 指南 [8.14] |弹性的

需要提前准备好 elasticsearch:8.14.3 镜像

1.1 单节点

1. docker-compose.yml

   # es
   docker run --name es01 --net elastic -p 9200:9200 -it -m 1GB docker.elastic.co/elasticsearch/elasticsearch:8.14.3
   # kibana
   docker run --name kibana --net elastic -p 5601:5601 docker.elastic.co/kibana/kibana:8.14.3
   

   官网是docker run命令，我们直接改成docker-compose格式进行部署：

   version: "3.8"
   services:es01:image: elasticsearch:8.14.3container_name: es01networks:- elasticports:- "9200:9200"mem_limit: 1gkibana:image: kibana:8.14.3container_name: kibananetworks:- elasticports:- "5601:5601"networks:elastic:driver: bridge
   

2. Cosign 验证

   Cosign 是一个用于签名和验证容器镜像的工具，它允许用户对镜像进行数字签名，并验证签名以确保镜像的完整性和来源。通过使用 Cosign，你可以创建和管理镜像的签名，并在部署时验证这些签名，以确保镜像的安全性。

   总结起来就是对镜像的上传和下载做签名验证，本文不做验证

3. 启动查看

   docker ps

   

   容器启动正常

   查看kibana登录地址docker logs kibana

   

   尽管我们已经配置了kibana的端口映射，但是由于我们目前没有任何kibana相关配置，所以启动需要用kibana生成出来的地址，不然还得找，就像下面这样

   

   从生成的地址登录到kibana

   

   生成令牌，贴到框中

   # 生成令牌命令
   docker exec -it es01 /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana# 输出如下，一个base64编码的字符串
   eyJ2ZXIiOiI4LjE0LjAiLCJhZHIiOlsiMTcyLjE4LjAuMzo5MjAwIl0sImZnciI6IjM4YjQxZTVmYjI1ODFiZjg0OGVjMTZkMDA1ZGRlNDliODlhZWI2OGQyYWU5NjEwOWNhNTJlNDRkM2VkYTYzNTkiLCJrZXkiOiJpYmdzdnBNQmhFZWlxSUlWZ21qUzpTUFFINFVqU1NsLWpGN0tWTVRTTk1RIn0=
   

   

   configure Elastic

   

   跑完后会进入到登录页面：

   

   Username默认是 elastic

   密码我们不知道，直接生成一个新的

   docker exec -it es01 bin/elasticsearch-reset-password -u elastic
   

   

   用新密码登录，登录成功

4. 查看es节点状态

   GET /_cat/health?v&format=json

   green就是健康

   

1.2 新节点加入集群

1. 使用现有节点生成令牌

   令牌注册时间为30分钟

   docker exec -it es01 /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node
   

   

2. 启动新的 Elasticsearch 容器。将注册令牌作为环境变量包含在内

   docker run -d -e ENROLLMENT_TOKEN="eyJ2ZXIiOiI4LjE0LjAiLCJhZHIiOlsiMTcyLjE4LjAuMzo5MjAwIl0sImZnciI6IjM4YjQxZTVmYjI1ODFiZjg0OGVjMTZkMDA1ZGRlNDliODlhZWI2OGQyYWU5NjEwOWNhNTJlNDRkM2VkYTYzNTkiLCJrZXkiOiJ0TGpVdnBNQmhFZWlxSUlWZ0doWDp1Nk1MOWV3dlNBT09XX2hSX090b0ZRIn0=" --name es02 --net bigdata_elastic -it -m 1GB elasticsearch:8.14.3
   

4. 查看节点是否加入集群

   GET /_cat/nodes?v&format=json

   

   GET /_cat/health?v&format=json

   

5. 加入成功

1.3 docker-compose部署集群

1. 创建.env

   # Password for the 'elastic' user (at least 6 characters)
   ELASTIC_PASSWORD=Qianyue@2024# Password for the 'kibana_system' user (at least 6 characters)
   KIBANA_PASSWORD=Qianyue@2024# Version of Elastic products
   STACK_VERSION=8.14.3# Set the cluster name
   CLUSTER_NAME=elastic-cluster# Set to 'basic' or 'trial' to automatically start the 30-day trial
   LICENSE=basic
   #LICENSE=trial# Port to expose Elasticsearch HTTP API to the host
   ES_PORT=9200
   #ES_PORT=127.0.0.1:9200# Port to expose Kibana to the host
   KIBANA_PORT=5601
   #KIBANA_PORT=80# Increase or decrease based on the available host memory (in bytes)
   MEM_LIMIT=1073741824# Project namespace (defaults to the current folder name if not set)
   #COMPOSE_PROJECT_NAME=myproject
   

2. docker-compose.yml

   version: "2.2"services:setup:image: elasticsearch:${STACK_VERSION}volumes:- certs:/usr/share/elasticsearch/config/certsuser: "0"command: >bash -c 'if [ x${ELASTIC_PASSWORD} == x ]; thenecho "Set the ELASTIC_PASSWORD environment variable in the .env file";exit 1;elif [ x${KIBANA_PASSWORD} == x ]; thenecho "Set the KIBANA_PASSWORD environment variable in the .env file";exit 1;fi;if [ ! -f config/certs/ca.zip ]; thenecho "Creating CA";bin/elasticsearch-certutil ca --silent --pem -out config/certs/ca.zip;unzip config/certs/ca.zip -d config/certs;fi;if [ ! -f config/certs/certs.zip ]; thenecho "Creating certs";echo -ne \"instances:\n"\"  - name: es01\n"\"    dns:\n"\"      - es01\n"\"      - localhost\n"\"    ip:\n"\"      - 127.0.0.1\n"\"  - name: es02\n"\"    dns:\n"\"      - es02\n"\"      - localhost\n"\"    ip:\n"\"      - 127.0.0.1\n"\"  - name: es03\n"\"    dns:\n"\"      - es03\n"\"      - localhost\n"\"    ip:\n"\"      - 127.0.0.1\n"\> config/certs/instances.yml;bin/elasticsearch-certutil cert --silent --pem -out config/certs/certs.zip --in config/certs/instances.yml --ca-cert config/certs/ca/ca.crt --ca-key config/certs/ca/ca.key;unzip config/certs/certs.zip -d config/certs;fi;echo "Setting file permissions"chown -R root:root config/certs;find . -type d -exec chmod 750 \{\} \;;find . -type f -exec chmod 640 \{\} \;;echo "Waiting for Elasticsearch availability";until curl -s --cacert config/certs/ca/ca.crt https://es01:9200 | grep -q "missing authentication credentials"; do sleep 30; done;echo "Setting kibana_system password";until curl -s -X POST --cacert config/certs/ca/ca.crt -u "elastic:${ELASTIC_PASSWORD}" -H "Content-Type: application/json" https://es01:9200/_security/user/kibana_system/_password -d "{\"password\":\"${KIBANA_PASSWORD}\"}" | grep -q "^{}"; do sleep 10; done;echo "All done!";'healthcheck:test: ["CMD-SHELL", "[ -f config/certs/es01/es01.crt ]"]interval: 1stimeout: 5sretries: 120es01:depends_on:setup:condition: service_healthyimage: elasticsearch:${STACK_VERSION}volumes:- certs:/usr/share/elasticsearch/config/certs- esdata01:/usr/share/elasticsearch/dataports:- ${ES_PORT}:9200environment:- node.name=es01- cluster.name=${CLUSTER_NAME}- cluster.initial_master_nodes=es01,es02,es03- discovery.seed_hosts=es02,es03- ELASTIC_PASSWORD=${ELASTIC_PASSWORD}- bootstrap.memory_lock=true- xpack.security.enabled=true- xpack.security.http.ssl.enabled=true- xpack.security.http.ssl.key=certs/es01/es01.key- xpack.security.http.ssl.certificate=certs/es01/es01.crt- xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt- xpack.security.transport.ssl.enabled=true- xpack.security.transport.ssl.key=certs/es01/es01.key- xpack.security.transport.ssl.certificate=certs/es01/es01.crt- xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt- xpack.security.transport.ssl.verification_mode=certificate- xpack.license.self_generated.type=${LICENSE}mem_limit: ${MEM_LIMIT}ulimits:memlock:soft: -1hard: -1healthcheck:test:["CMD-SHELL","curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",]interval: 10stimeout: 10sretries: 120es02:depends_on:- es01image: elasticsearch:${STACK_VERSION}volumes:- certs:/usr/share/elasticsearch/config/certs- esdata02:/usr/share/elasticsearch/dataenvironment:- node.name=es02- cluster.name=${CLUSTER_NAME}- cluster.initial_master_nodes=es01,es02,es03- discovery.seed_hosts=es01,es03- bootstrap.memory_lock=true- xpack.security.enabled=true- xpack.security.http.ssl.enabled=true- xpack.security.http.ssl.key=certs/es02/es02.key- xpack.security.http.ssl.certificate=certs/es02/es02.crt- xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt- xpack.security.transport.ssl.enabled=true- xpack.security.transport.ssl.key=certs/es02/es02.key- xpack.security.transport.ssl.certificate=certs/es02/es02.crt- xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt- xpack.security.transport.ssl.verification_mode=certificate- xpack.license.self_generated.type=${LICENSE}mem_limit: ${MEM_LIMIT}ulimits:memlock:soft: -1hard: -1healthcheck:test:["CMD-SHELL","curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",]interval: 10stimeout: 10sretries: 120es03:depends_on:- es02image: elasticsearch:${STACK_VERSION}volumes:- certs:/usr/share/elasticsearch/config/certs- esdata03:/usr/share/elasticsearch/dataenvironment:- node.name=es03- cluster.name=${CLUSTER_NAME}- cluster.initial_master_nodes=es01,es02,es03- discovery.seed_hosts=es01,es02- bootstrap.memory_lock=true- xpack.security.enabled=true- xpack.security.http.ssl.enabled=true- xpack.security.http.ssl.key=certs/es03/es03.key- xpack.security.http.ssl.certificate=certs/es03/es03.crt- xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt- xpack.security.transport.ssl.enabled=true- xpack.security.transport.ssl.key=certs/es03/es03.key- xpack.security.transport.ssl.certificate=certs/es03/es03.crt- xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt- xpack.security.transport.ssl.verification_mode=certificate- xpack.license.self_generated.type=${LICENSE}mem_limit: ${MEM_LIMIT}ulimits:memlock:soft: -1hard: -1healthcheck:test:["CMD-SHELL","curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",]interval: 10stimeout: 10sretries: 120kibana:depends_on:es01:condition: service_healthyes02:condition: service_healthyes03:condition: service_healthyimage: kibana:${STACK_VERSION}volumes:- certs:/usr/share/kibana/config/certs- kibanadata:/usr/share/kibana/dataports:- ${KIBANA_PORT}:5601environment:- SERVERNAME=kibana- ELASTICSEARCH_HOSTS=https://es01:9200- ELASTICSEARCH_USERNAME=kibana_system- ELASTICSEARCH_PASSWORD=${KIBANA_PASSWORD}- ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES=config/certs/ca/ca.crtmem_limit: ${MEM_LIMIT}healthcheck:test:["CMD-SHELL","curl -s -I http://localhost:5601 | grep -q 'HTTP/1.1 302 Found'",]interval: 10stimeout: 10sretries: 120volumes:certs:driver: localesdata01:driver: localesdata02:driver: localesdata03:driver: localkibanadata:driver: local
   

3. 启动

   docker-compose up -d

   

   kibana查看节点
   

4. 成功部署