当前位置： 首页 > news >正文

JuiceFS-K8s部署

news 2025/8/4 11:49:18

目录

    • 1、部署JuiceFS-CSI驱动
    • 2、创建OBS认证信息Secret
    • 3、创建存储类
    • 4、创建PVC--PVC创建时会自动创建PV
    • 5、创建测试Pod--测试Pod创建容器内是否挂载成功

官网文档地址：https://juicefs.com/docs/zh/csi/introduction/

1、部署JuiceFS-CSI驱动

部署yaml如下，镜像已进行更改为镜像仓库内地址

# DO NOT EDIT: generated by 'kustomize build'
apiVersion: v1
kind: ServiceAccount
metadata:labels:app.kubernetes.io/instance: juicefs-csi-driverapp.kubernetes.io/name: juicefs-csi-driverapp.kubernetes.io/version: mastername: juicefs-csi-controller-sanamespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:labels:app.kubernetes.io/instance: juicefs-csi-driverapp.kubernetes.io/name: juicefs-csi-driverapp.kubernetes.io/version: mastername: juicefs-csi-node-sanamespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:labels:app.kubernetes.io/instance: juicefs-csi-driverapp.kubernetes.io/name: juicefs-csi-driverapp.kubernetes.io/version: mastername: juicefs-csi-external-node-service-role
rules:
- apiGroups:- ""resources:- pods- pods/log- secretsverbs:- get- list- create- update- delete- patch- watch
- apiGroups:- batchresources:- jobsverbs:- get- list- create- update- delete- patch- watch
- apiGroups:- ""resources:- nodes/proxyverbs:- '*'
- apiGroups:- ""resources:- persistentvolumesverbs:- get- list
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:labels:app.kubernetes.io/instance: juicefs-csi-driverapp.kubernetes.io/name: juicefs-csi-driverapp.kubernetes.io/version: mastername: juicefs-external-provisioner-role
rules:
- apiGroups:- ""resources:- persistentvolumesverbs:- get- list- watch- create- delete
- apiGroups:- ""resources:- persistentvolumeclaimsverbs:- get- list- watch- update
- apiGroups:- storage.k8s.ioresources:- storageclassesverbs:- get- list- watch
- apiGroups:- ""resources:- eventsverbs:- get- list- watch- create- update- patch
- apiGroups:- storage.k8s.ioresources:- csinodesverbs:- get- list- watch
- apiGroups:- ""resources:- nodesverbs:- get- list- watch
- apiGroups:- ""resources:- secretsverbs:- get- list- create- update- patch- delete
- apiGroups:- ""resources:- pods- pods/logverbs:- get- list- watch- create- update- patch- delete
- apiGroups:- batchresources:- jobsverbs:- get- list- watch- create- update- patch- delete
- apiGroups:- ""resources:- endpointsverbs:- get- list- watch- create- update- patch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:labels:app.kubernetes.io/instance: juicefs-csi-driverapp.kubernetes.io/name: juicefs-csi-driverapp.kubernetes.io/version: mastername: juicefs-csi-node-service-binding
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: juicefs-csi-external-node-service-role
subjects:
- kind: ServiceAccountname: juicefs-csi-node-sanamespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:labels:app.kubernetes.io/instance: juicefs-csi-driverapp.kubernetes.io/name: juicefs-csi-driverapp.kubernetes.io/version: mastername: juicefs-csi-provisioner-binding
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: juicefs-external-provisioner-role
subjects:
- kind: ServiceAccountname: juicefs-csi-controller-sanamespace: kube-system
---
apiVersion: apps/v1
kind: StatefulSet
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: juicefs-csi-driverapp.kubernetes.io/name: juicefs-csi-driverapp.kubernetes.io/version: mastername: juicefs-csi-controllernamespace: kube-system
spec:replicas: 1selector:matchLabels:app: juicefs-csi-controllerapp.kubernetes.io/instance: juicefs-csi-driverapp.kubernetes.io/name: juicefs-csi-driverapp.kubernetes.io/version: masterserviceName: juicefs-csi-controllertemplate:metadata:labels:app: juicefs-csi-controllerapp.kubernetes.io/instance: juicefs-csi-driverapp.kubernetes.io/name: juicefs-csi-driverapp.kubernetes.io/version: masterspec:containers:- args:- --endpoint=$(CSI_ENDPOINT)- --logtostderr- --nodeid=$(NODE_NAME)- --v=5env:- name: CSI_ENDPOINTvalue: unix:///var/lib/csi/sockets/pluginproxy/csi.sock- name: NODE_NAMEvalueFrom:fieldRef:fieldPath: spec.nodeName- name: JUICEFS_MOUNT_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespace- name: POD_NAMEvalueFrom:fieldRef:fieldPath: metadata.name- name: HOST_IPvalueFrom:fieldRef:fieldPath: status.hostIP- name: KUBELET_PORTvalue: "10250"- name: JUICEFS_MOUNT_PATHvalue: /var/lib/juicefs/volume- name: JUICEFS_CONFIG_PATHvalue: /var/lib/juicefs/configimage: harbor.t3caic.com/juicefs/juicefs-csi-driver:v0.17.5livenessProbe:failureThreshold: 5httpGet:path: /healthzport: healthzinitialDelaySeconds: 10periodSeconds: 10timeoutSeconds: 3name: juicefs-pluginports:- containerPort: 9909name: healthzprotocol: TCPresources:limits:cpu: 1000mmemory: 1Girequests:cpu: 100mmemory: 512MisecurityContext:capabilities:add:- SYS_ADMINprivileged: truevolumeMounts:- mountPath: /var/lib/csi/sockets/pluginproxy/name: socket-dir- mountPath: /jfsmountPropagation: Bidirectionalname: jfs-dir- mountPath: /root/.juicefsmountPropagation: Bidirectionalname: jfs-root-dir- args:- --csi-address=$(ADDRESS)- --timeout=60s- --v=5env:- name: ADDRESSvalue: /var/lib/csi/sockets/pluginproxy/csi.sockimage: harbor.t3caic.com/juicefs/csi-provisioner:v1.6.0name: csi-provisionervolumeMounts:- mountPath: /var/lib/csi/sockets/pluginproxy/name: socket-dir- args:- --csi-address=$(ADDRESS)- --health-port=$(HEALTH_PORT)env:- name: ADDRESSvalue: /csi/csi.sock- name: HEALTH_PORTvalue: "9909"image: harbor.t3caic.com/juicefs/livenessprobe:v1.1.0name: liveness-probevolumeMounts:- mountPath: /csiname: socket-dirpriorityClassName: system-cluster-criticalserviceAccount: juicefs-csi-controller-satolerations:- key: CriticalAddonsOnlyoperator: Existsvolumes:- emptyDir: {}name: socket-dir- hostPath:path: /var/lib/juicefs/volumetype: DirectoryOrCreatename: jfs-dir- hostPath:path: /var/lib/juicefs/configtype: DirectoryOrCreatename: jfs-root-dirvolumeClaimTemplates: []
---
apiVersion: apps/v1
kind: DaemonSet
metadata:labels:app.kubernetes.io/component: nodeapp.kubernetes.io/instance: juicefs-csi-driverapp.kubernetes.io/name: juicefs-csi-driverapp.kubernetes.io/version: mastername: juicefs-csi-nodenamespace: kube-system
spec:selector:matchLabels:app: juicefs-csi-nodeapp.kubernetes.io/instance: juicefs-csi-driverapp.kubernetes.io/name: juicefs-csi-driverapp.kubernetes.io/version: mastertemplate:metadata:labels:app: juicefs-csi-nodeapp.kubernetes.io/instance: juicefs-csi-driverapp.kubernetes.io/name: juicefs-csi-driverapp.kubernetes.io/version: masterspec:containers:- args:- --endpoint=$(CSI_ENDPOINT)- --logtostderr- --nodeid=$(NODE_NAME)- --v=5- --enable-manager=trueenv:- name: CSI_ENDPOINTvalue: unix:/csi/csi.sock- name: NODE_NAMEvalueFrom:fieldRef:fieldPath: spec.nodeName- name: JUICEFS_MOUNT_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespace- name: POD_NAMEvalueFrom:fieldRef:fieldPath: metadata.name- name: HOST_IPvalueFrom:fieldRef:fieldPath: status.hostIP- name: KUBELET_PORTvalue: "10250"- name: JUICEFS_MOUNT_PATHvalue: /var/lib/juicefs/volume- name: JUICEFS_CONFIG_PATHvalue: /var/lib/juicefs/configimage: harbor.t3caic.com/juicefs/juicefs-csi-driver:v0.17.5lifecycle:preStop:exec:command:- /bin/sh- -c- rm /csi/csi.socklivenessProbe:failureThreshold: 5httpGet:path: /healthzport: healthzinitialDelaySeconds: 10periodSeconds: 10timeoutSeconds: 3name: juicefs-pluginports:- containerPort: 9909name: healthzprotocol: TCPresources:limits:cpu: 1000mmemory: 1Girequests:cpu: 100mmemory: 512MisecurityContext:privileged: truevolumeMounts:- mountPath: /var/lib/kubeletmountPropagation: Bidirectionalname: kubelet-dir- mountPath: /csiname: plugin-dir- mountPath: /devname: device-dir- mountPath: /jfsmountPropagation: Bidirectionalname: jfs-dir- mountPath: /root/.juicefsmountPropagation: Bidirectionalname: jfs-root-dir- args:- --csi-address=$(ADDRESS)- --kubelet-registration-path=$(DRIVER_REG_SOCK_PATH)- --v=5env:- name: ADDRESSvalue: /csi/csi.sock- name: DRIVER_REG_SOCK_PATHvalue: /var/lib/kubelet/csi-plugins/csi.juicefs.com/csi.sockimage: harbor.t3caic.com/juicefs/csi-node-driver-registrar:v2.1.0name: node-driver-registrarvolumeMounts:- mountPath: /csiname: plugin-dir- mountPath: /registrationname: registration-dir- args:- --csi-address=$(ADDRESS)- --health-port=$(HEALTH_PORT)env:- name: ADDRESSvalue: /csi/csi.sock- name: HEALTH_PORTvalue: "9909"image: harbor.t3caic.com/juicefs/livenessprobe:v1.1.0name: liveness-probevolumeMounts:- mountPath: /csiname: plugin-dirdnsPolicy: ClusterFirstWithHostNetpriorityClassName: system-node-criticalserviceAccount: juicefs-csi-node-satolerations:- key: CriticalAddonsOnlyoperator: Existsvolumes:- hostPath:path: /var/lib/kubelettype: Directoryname: kubelet-dir- hostPath:path: /var/lib/kubelet/csi-plugins/csi.juicefs.com/type: DirectoryOrCreatename: plugin-dir- hostPath:path: /var/lib/kubelet/plugins_registry/type: Directoryname: registration-dir- hostPath:path: /devtype: Directoryname: device-dir- hostPath:path: /var/lib/juicefs/volumetype: DirectoryOrCreatename: jfs-dir- hostPath:path: /var/lib/juicefs/configtype: DirectoryOrCreatename: jfs-root-dir
---
apiVersion: storage.k8s.io/v1
kind: CSIDriver
metadata:labels:app.kubernetes.io/instance: juicefs-csi-driverapp.kubernetes.io/name: juicefs-csi-driverapp.kubernetes.io/version: mastername: csi.juicefs.com
spec:attachRequired: falsepodInfoOnMount: false

2、创建OBS认证信息Secret

需要修改的信息
metaurl: “用户名:密码@IP:端口/库名”
postgres://用户名:密码@IP:端口/库名 数据库需要提前进入进行创建库

storage: “obs” 存储类型标识
bucket: “” 桶的域名
access-key: “”
secret-key: “”

apiVersion: v1
kind: Secret
metadata:name: juicefs-sc-secretnamespace: kube-system
type: Opaque
stringData:name: "test"metaurl: ""storage: ""bucket: ""access-key: ""secret-key: ""

3、创建存储类

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:name: juicefs-sc
provisioner: csi.juicefs.com
reclaimPolicy: Retain
volumeBindingMode: Immediate
parameters:csi.storage.k8s.io/node-publish-secret-name: juicefs-sc-secretcsi.storage.k8s.io/node-publish-secret-namespace: kube-systemcsi.storage.k8s.io/provisioner-secret-name: juicefs-sc-secretcsi.storage.k8s.io/provisioner-secret-namespace: kube-system

4、创建PVC–PVC创建时会自动创建PV

目前而言，JuiceFS CSI 驱动不支持设置存储容量。在 PersistentVolume 和 PersistentVolumeClaim 中指定的容量会被忽略，填写任意有效值即可，例如 10Gi

apiVersion: v1
kind: PersistentVolumeClaim
metadata:name: "juicefs-test-pvc"namespace: test-juicefs
spec:storageClassName: sc-topolvmaccessModes:- ReadWriteOncevolumeMode: Filesystemresources:requests:storage: 10GistorageClassName: juicefs-sc

5、创建测试Pod–测试Pod创建容器内是否挂载成功

apiVersion: apps/v1
kind: Deployment
metadata:name: nginx-run
spec:selector:matchLabels:app: nginxtemplate:metadata:labels:app: nginxspec:containers:- name: nginximage: harbor.t3caic.com/nginxdemos/hello:plain-textports:- containerPort: 80volumeMounts:- mountPath: /configname: web-datavolumes:- name: web-datapersistentVolumeClaim:claimName: juicefs-test-pvc

ERR注意：如果测试Pod创建卡死，查看kube-system命名空间的juicefs-XXX-pvc容器起来了吗
如因为拉取不下镜像，请在此容器所在节点手动解决Pod拉取 docker pull
如果镜像拉取下来Pod一直在重启可以考虑一下是否是OBS连接不上，如果OBS需要做域名解析可以在CoreDNS里配置

kubectl edit configmap coredns -n kube-system
添加hosts {100.125.32.189 juicefs.obs.cn-east-263.t3caic.comfallthrough}
http://www.lryc.cn/news/66469.html

相关文章：

  • 2023最新版本Camtasia电脑录屏软件好不好用?
  • 第三章 Linux 初步
  • linux环境安装使用mysql详解
  • SUNTANS模型学习（9）——学习Tidal forcing算例
  • ​力扣解法汇总1010. 总持续时间可被 60 整除的歌曲
  • 利用老毛桃pe启动U盘启动ubuntu.iso，完成ubuntu系统的安装
  • 分享2个教学视频录制的方法！
  • 「SQL面试题库」 No_63 报告的记录 II
  • 【事务】怎么去理解事务？
  • camunda流程变量如何使用
  • CMIP6：WRF模式动力降尺度、单点降尺度、统计方法区域降尺度
  • 2023建筑设计师们有哪些好用的AI设计工具？
  • mysql主从复制与读写分离
  • 技术控，看这里，一款支持断点调试的数据科学工具
  • 论文导读 | 大语言模型上的精调策略
  • 进阶自动化测试，这3点你一定要知道的...
  • 网络编程套接字API
  • 数字藏品的价值和意义
  • Unity物理系统脚本编程（上）
  • Java基础（十七）File类与IO流
  • 跑步课程导入能力，助力科学训练
  • MySQL---8、创建和管理表
  • 图像分类简单介绍
  • 很多博主用Markdown格式文章？直呼真不错！
  • 【2023/05/07】汇编语言
  • AI 生成第3篇测试文章：怎么编写测试计划？
  • 怎么洗稿容易过稿-在线洗稿软件
  • 图书馆客流人数统计分析系统方案
  • linux命令之crontab详解
  • 浅谈一下接口工具（jmeter、postman、swagger等）