当前位置： 首页 > news >正文

Kubernetes (K8s) 部署Doris

news 2025/8/1 10:28:15

官网提供yaml地址下载部署

https://doris.apache.org/zh-CN/docs/2.0/install/cluster-deployment/k8s-deploy/install-env/

禁用和关闭 swap

在部署 Doris 时，建议关闭 swap 分区。

通过以下命令可以永久关闭 swap 分区。

echo "vm.swappiness = 0">> /etc/sysctl.conf
swapoff -a && swapon -a
sysctl -p

设置系统最大打开文件句柄数

vi /etc/security/limits.conf 
* soft nofile 65536
* hard nofile 65536

修改虚拟内存区域数量

修改虚拟内存区域至少 2000000

sysctl -w vm.max_map_count=2000000

关闭透明大页

在部署 Doris 时，建议关闭透明大页。

echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag

部署 Doris Operator
添加 Doris Cluster 资源定义
Doris Operator 使用自定义资源定义（Custom Resource Definition, CRD）扩展 Kubernetes。Doris Cluster 的 CRD 中封装了对 Doris 对象的描述，例如对 FE 或 BE 的描述，详细内容可以参考 doris-operator-api。在部署 Doris 前，需要先创建 Doris Cluster 的 CRD。
通过以下命令可以在 Kubernetes 环境中部署 Doris Cluster CRD：

kubectl create -f https://raw.githubusercontent.com/apache/doris-operator/master/config/crd/bases/doris.selectdb.com_dorisclusters.yaml

如果没有外网，先将 CRD 文件下载到本地：

wget https://raw.githubusercontent.com/apache/doris-operator/master/config/crd/bases/doris.selectdb.com_dorisclusters.yaml
kubectl create -f ./doris.selectdb.com_dorisclusters.yaml

以下是期望输出结果：

customresourcedefinition.apiextensions.k8s.io/dorisclusters.doris.selectdb.com created

在创建了 Doris Cluster CRD 后，可以通过以下命令查看创建的 CRD。

kubectl get crd | grep doris

以下为期望输出结果：

dorisclusters.doris.selectdb.com                      2024-02-22T16:23:13Z

添加 Doris Operator
方案一：快速部署 Doris Operator
可以直接拉去仓库中的 Doris Operator 模板进行快速部署。
使用以下命令可以在 Kubernetes 集群中部署 Doris Operator：

kubectl apply -f https://raw.githubusercontent.com/apache/doris-operator/master/config/operator/operator.yaml

以下为期望输出结果：

namespace/doris created
role.rbac.authorization.k8s.io/leader-election-role created
rolebinding.rbac.authorization.k8s.io/leader-election-rolebinding created
clusterrole.rbac.authorization.k8s.io/doris-operator created
clusterrolebinding.rbac.authorization.k8s.io/doris-operator-rolebinding created
serviceaccount/doris-operator created
deployment.apps/doris-operator created

方案二：自定义部署 Doris Operator
在创建完 CRD 后，在 Kubernetes 集群上部署 Doris Operator 有两种方式：在线与离线部署。
在 operator.yaml 文件中规范了部署 operator 的服务的最低要求。为了适配复杂的生产环境，可以下载 operator.yaml 文件后，按照期望更新其中配置。
在线安装 Doris Operator
在修改 operator.yaml 文件后，可以使用以下命令部署 Doris Operator 服务：

kubectl apply -f /data/k8s-yaml/doris/operator.yaml

以下为期望输出结果：

namespace/doris created
role.rbac.authorization.k8s.io/leader-election-role created
rolebinding.rbac.authorization.k8s.io/leader-election-rolebinding created
clusterrole.rbac.authorization.k8s.io/doris-operator created
clusterrolebinding.rbac.authorization.k8s.io/doris-operator-rolebinding created
serviceaccount/doris-operator created
deployment.apps/doris-operator created

离线安装 Doris Operator
下载 operator 运行所需镜像文件
如果服务器没有连通外网，需要先下载对应的 operator 镜像文件。Doris Operator 用到以下的镜像：

selectdb/doris.k8s-operator:latest

在可以连通外网的服务器中运行以下的命令，可以将镜像下载下来：

download doris operator image

docker pull selectdb/doris.k8s-operator:latest

save the doris operator image as a tar package

docker save -o doris.k8s-operator-latest.tar selectdb/doris.k8s-operator:latest

将已打包的 tar 文件放置到所有的 Kubernetes node 节点中，运行以下命令上传镜像：

docker load -i doris.k8s-operator-latest.tar

配置 Doris Operator
下载 operator.yaml 文件后，可以根据生产环境期望修改模板。

Doris Operator 在 Kubernetes 集群中是一个无状态的 Deployment，可以根据需求修改如 limits、replica、label、namespace 等项目。如需要指定某一版本的 doirs operator 镜像，可以在上传镜像后对 operator.yaml 文件做如下修改：

...
containers:- command:- /dorisoperatorargs:- --leader-electimage: selectdb/doris.k8s-operator:v1.0.0name: dorisoperatorsecurityContext:allowPrivilegeEscalation: falsecapabilities:drop:- "ALL"...

安装 Doris Operator
在修改 Doris Operator 模板后，可以使用 apply 命令部署 Operator：

kubectl apply -f ./data/k8s-yaml/doris/operator.yaml

第三种方式部署
根据服务器环境，修改doris_be.yml文件。

apiVersion: v1
kind: Service
metadata:name: doris-be-cluster1labels:app: doris-be-cluster1
spec:ports:- port: 9060name: be-port- port: 8040name: webserver-port- port: 9050name: heartbeat-port #This name should be fixed. Doris will get the port information through this name- port: 8060name: brpc-portclusterIP: Noneselector:app: doris-be-cluster1
---
apiVersion: v1
kind: Service
metadata:name: doris-be-cluster1labels:app: doris-be-cluster1
spec:ports:- port: 9060name: be-port- port: 8040name: webserver-port- port: 9050name: heartbeat-port #This name should be fixed. Doris will get the port information through this name- port: 8060name: brpc-portclusterIP: Noneselector:app: doris-be-cluster1
---
apiVersion: apps/v1
kind: StatefulSet
metadata:name: doris-be-cluster1labels:app: doris-be-cluster1
spec:selector:matchLabels:app: doris-be-cluster1serviceName: doris-be-cluster1replicas: 3template:metadata:name: doris-be-cluster1labels:app: doris-be-cluster1spec:containers:- name: doris-be-cluster1#Need to change to real mirror information#image: apache-doris-be:test# 修改点1： 镜像地址修改为真实doris镜像，可在https://hub.docker.com/r/apache/doris/tags找到需要的镜像版本image: apache/doris:2.0.0_alpha-be-x86_64imagePullPolicy: IfNotPresent#节点选择nodeSelector:node: middlewareenv:#Specify the startup type as k8s to bypass some restrictions of the official image initialization script- name: BUILD_TYPEvalue: "k8s"# 修改点2： 增加环境变量，写明FE的IP与端口- name: FE_MASTER_IPvalue: "doris-follower-cluster1-0.doris-follower-cluster1.doris.svc.cluster.local"- name: FE_MASTER_PORTvalue: "9030"ports:- containerPort: 9060name: be-port- containerPort: 8040name: webserver-port- containerPort: 9050name: heartbeat-port- containerPort: 8060name: brpc-portvolumeMounts:#Mount the configuration file in the way of configmap- name: confmountPath: /data/apache-doris/be/conf#Ifnot mounted, when enable_profile, error will be reported when querying the data from jdbc catalog#Error message: error setting certificate verify locations: CAfile:/etc/pki/tls/certs/ca-bundle.crt CApath: none- name: sysmountPath: /etc/pki# 修改点3： 挂载存储subPath: pkireadOnly: true# 同修改点3- name: sysmountPath: /data/apache-doris/be/storagesubPath: storagevolumes:- name: confconfigMap:name: be-conf- name: sys# 修改点4：不使用hostpath，注释掉#hostPath:#path: /etc/pki# 修改点5： 增加存储配置，此处使用nfs-dynamic-class或者longhornvolumeClaimTemplates:- metadata:name: sysspec:storageClassName: nfs-dynamic-classaccessModes: [ "ReadWriteOnce" ]resources:requests:storage: 50Gi 
---
apiVersion: v1
kind: ConfigMap
metadata:name: be-conf
data:be.conf: |PPROF_TMPDIR="$DORIS_HOME/log/"sys_log_level = INFObe_port = 9060webserver_port = 8040heartbeat_service_port = 9050brpc_port = 8060# 修改点6： 修改网段为k8s使用网段，配置数据存储路径priority_networks = 10.44.0.0/16storage_root_path = /data/apache-doris/be/storage

根据服务器环境，修改doris_follower.yml文件

apiVersion: v1
kind: Service
metadata:name: doris-follower-cluster1labels:app: doris-follower-cluster1
spec:ports:- port: 8030name: http-port- port: 9020name: rpc-port- port: 9030name: query-port- port: 9010name: edit-log-port #This name should be fixed. Doris will get the port information through this nameclusterIP: Noneselector:app: doris-follower-cluster1
---
apiVersion: v1
kind: Service
metadata:name: doris-follower-cluster1labels:app: doris-follower-cluster1
spec:ports:- port: 8030name: http-port- port: 9020name: rpc-port- port: 9030name: query-port- port: 9010name: edit-log-port #This name should be fixed. Doris will get the port information through this nameclusterIP: Noneselector:app: doris-follower-cluster1
---
apiVersion: apps/v1
kind: StatefulSet
metadata:name: doris-follower-cluster1labels:app: doris-follower-cluster1
spec:selector:matchLabels:app: doris-follower-cluster1serviceName: doris-follower-cluster1# 修改点1： 修改fe副本为1replicas: 1template:metadata:name: doris-follower-cluster1labels:app: doris-follower-cluster1spec:containers:- name: doris-follower-cluster1#Need to change to real mirror information# 修改点2： 镜像地址修改为真实doris镜像，可在https://hub.docker.com/r/apache/doris/tags找到需要的镜像版本image: apache/doris:2.0.0_alpha-fe-x86_64imagePullPolicy: IfNotPresent#节点选择nodeSelector:node: middlewareenv:# 修改点3： 增加了APP_NAMESPACE与FE_IPADDRESS环境变量- name: APP_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespace- name: FE_IPADDRESSvalueFrom:fieldRef:fieldPath: status.podIP#Specify the startup type as k8s to bypass some restrictions of the official image initialization script- name: BUILD_TYPEvalue: "k8s"#Initialize the fe of three nodes- name: FE_INIT_NUMBER# 修改点4： 将数量改为1value: "1"#ServiceName of bakend_cn node,（if do not have bakend_cn node,do not configure this environment variable）# 修改点5： 不使用cn节点，注释变量CN_SERVICE  CN_STATEFULSET#- name: CN_SERVICE#  value: "doris-cn-cluster1"#StatefulSetName of bakend_cn node,（if do not have bakend_cn node,do not configure this environment variable)#- name: CN_STATEFULSET#  value: "doris-cn-cluster1"#ServiceName of bakend node,（if do not have bakend node,do not configure this environment variable）- name: BE_SERVICEvalue: "doris-be-cluster1"#StatefulSetName of bakend node,（if do not have bakend node,do not configure this environment variable)- name: BE_STATEFULSETvalue: "doris-be-cluster1"#ServiceName of follower node,（if do not have follower node,do not configure this environment variable）- name: FE_SERVICEvalue: "doris-follower-cluster1"##StatefulSetName of follower node,（if do not have follower node,do not configure this environment variable)- name: FE_STATEFULSETvalue: "doris-follower-cluster1"ports:- containerPort: 8030name: http-port- containerPort: 9020name: rpc-port- containerPort: 9030name: query-port- containerPort: 9010name: edit-log-portvolumeMounts:#Mount the configuration file in the way of configmap- name: confmountPath: /data/apache-doris/fe/conf#In order to call the api of k8s- name: kube# 使用本地配置则为/root/.kube/configmountPath: /root/.kubereadOnly: true# 修改点6： 配置存储，用于元数据持久化- name: metadatamountPath: /data/apache-doris/fe/doris-metavolumes:- name: confconfigMap:name: follower-conf- name: kube# 修改点7： 修改为使用configMap(此处可以不修改，使用本地配置)#hostPath:#path: /root/.kube/configconfigMap:name: kube-conf# 修改点8： 增加存储配置，此处使用nfs-dynamic-class 或者 longhornvolumeClaimTemplates:- metadata:name: metadataspec:storageClassName: nfs-dynamic-class accessModes: [ "ReadWriteOnce" ]resources:requests:storage: 10Gi
---
apiVersion: v1
kind: ConfigMap
metadata:name: follower-conf
data:fe.conf: |# 修改点9： 修改网段为k8s使用网段priority_networks = 10.44.0.0/16#It can automatically maintain node information by getting the number of replicas of StatefulSet, similar to alter system add/drop backenable_deploy_manager = k8s#Automatically adjust the IP of the node according to the domain name (for example, after the pod is restarted, the domain name is still doris-be-cluster1-0-doris-be-cluster1.default.svc.cluster.local, but the IP may change from 172.16.0.9 to 172.16.0.10)enable_fqdn_mode = trueLOG_DIR = ${DORIS_HOME}/logsys_log_level = INFOhttp_port = 8030rpc_port = 9020query_port = 9030edit_log_port = 9010#Doris needs to generate the log4j configuration file according to the fe.yml configuration information, which is written in the same directory as fe.yml by default, but the config we mount is readonly, so specify this configuration to write the log4j file to another locationcustom_config_dir = /data/apache-doris/#when set to false, the backend will not be dropped and remaining in DECOMMISSION statedrop_backend_after_decommission = false# 修改点10： 增加元数据、java等配置mysql_service_nio_enabled = trueJAVA_OPTS = "-Xmx8192m -XX:+UseMembar -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xloggc:$DORIS_HOME/log/fe.gc.log.$DATE"JAVA_OPTS_FOR_JDK_9 = "-Xmx8192m -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+CMSClassUnloadingEnabled -XX:-CMSParalle=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xlog:gc*:$DORIS_HOME/log/fe.gc.log.$DATE:time"meta_dir = /data/apache-doris/fe/doris-meta#metadata_failure_recovery = true

外部访问sevice

kind: Service
apiVersion: v1
metadata:name: doris-follower-loacllabels:app: doris-follower-loacl
spec:ports:- name: httpprotocol: TCPport: 8030targetPort: 8030nodePort: 28030- name: tcpprotocol: TCPport: 9030targetPort: 9030nodePort: 29030selector:app: doris-follower-cluster1type: NodePort

部署

kubectl create ns doris
kubectl apply -f doris_be.yml -n doris
kubectl apply -f doris_follower.yml -n doris
kubectl apply -f doris-svc.yaml -n doris
  1. 访问及使用, 默认账号为root，无密码
# web端访问地址
http://[节点IP]:20803/login
# 使用mysql client连接地址
host: [节点IP]
port: 29030
user: root
pass: 
# 使用mysql client连接后可修改root密码
SET PASSWORD FOR 'root' = PASSWORD('your_password');
http://www.lryc.cn/news/606139.html

相关文章：

  • Redis过期策略
  • 【嵌入式电机控制#23】BLDC：开环运动控制框架
  • 设计模式：命令模式 Command
  • 法国声学智慧 ，音响品牌SK (SINGKING AUDIO) 重构专业音频边界
  • Web开发-PHP应用原生语法全局变量数据接受身份验证变量覆盖任意上传（代码审计案例）
  • HighgoDB查询慢SQL和阻塞SQL
  • 电商项目_性能优化_高并发缓存一致性
  • 当过滤条件不符合最左前缀时，如何有效利用索引？ | OceanBase SQL 优化实践
  • 0731 IO进程基础
  • FATFS文件系统
  • 从“救火”到“先知”：润建曲尺运维大模型如何重构网络运维价值链
  • 科研快报 |无人机+AI：广东防控基孔热背后的技术革命
  • 大疆无人机开发：MQTT 赋能机场系统集成的Java实战之旅
  • 九识智能与星逻智能达成战略合作，共推“无人车 + 无人机”空地一体巡检升级
  • 5G 单兵终端 + 无人机：消防应急场景的 “空 - 地” 救援协同体系
  • 无人机光伏巡检缺陷检出率↑32%：陌讯多模态融合算法实战解析
  • Lombok 字段魔法：用 @FieldDefaults 解锁“隐身+锁死”双重特效
  • php session 和 jwt 区别和使用场景
  • Java试题-选择题（2）
  • sqli-labs：Less-13关卡详细解析
  • 数据大集网：引领精准获客新时代的优质平台
  • 智慧医院导航系统：基于GPS+蓝牙ibeacon多源融合定位与deepseek•AI导诊问答的设计与实现
  • Linux 时钟同步配置：基础管理与 chrony 工具应用
  • 多架构镜像整合全攻略：在Docker中实现单一镜像支持同时支持amd64和arm64架构
  • hive新增列之后插入新数据时，新列为NULL的解决办法
  • CentOS 7 编译 Redis 6.x 完整教程（解决 GCC 版本不支持 C11）
  • 告别物业思维：科技正重构产业园区的价值坐标系
  • AR智能巡检：工业4.0时代的降本增效利器
  • [人工智能-综述-17]：AI革命：重塑职业版图，开启文明新篇
  • 数据集归一化