k8s环境使用Operator部署Seaweedfs集群（下）
@作者：闫乾苓
文章目录
- 4.4.3 部署seaweedfs集群
- 4.4.4 验证集群运行状态
- 4.4.5 测试集群功能
4.4.3 部署seaweedfs集群
集群Yaml示例
apiVersion: seaweed.seaweedfs.com/v1
kind: Seaweed
metadata:name: seaweed1namespace: default
spec:image: chrislusf/seaweedfs:latestvolumeServerDiskCount: 1master:replicas: 3volumeSizeLimitMB: 1024volume:replicas: 3requests:storage: 5GistorageClassName: "local-storage"filer:replicas: 2s3: true # 启用S3 API persistence:enabled: truestorageClassName: "local-storage"resources:requests:storage: 2Giconfig: |[leveldb2]enabled = truedir = "/data/filerldb2"# S3相关配置可以在这里添加 [s3]enabled = true
报错提示，crd中filer.persistence没有这个字段：
kubectl apply -f cluster_seaweedfs.yaml
error: error validating "cluster_seaweedfs.yaml": error validating data: ValidationError(Seaweed.spec.filer): unknown field "persistence" in com.seaweedfs.seaweed.v1.Seaweed.spec.filer; if you choose to ignore these errors, turn validation off with --validate=false
需要部署最新的crd yaml
Kubectl apply -f seaweedfs-operator/config/crd/bases/seaweed.seaweedfs.com_seaweeds.yaml
重新部署集群：
[root@master cluster]# kubectl apply -f cluster_seaweedfs.yaml
seaweed.seaweed.seaweedfs.com/seaweed1 created
4.4.4 验证集群运行状态
查看pod 运行状态：
[root@master cluster]# kubectl get pod
NAME READY STATUS RESTARTS AGE
seaweed1-filer-0 1/1 Running 0 2m39s
seaweed1-filer-1 1/1 Running 0 2m39s
seaweed1-master-0 1/1 Running 1 (3m8s ago) 3m12s
seaweed1-master-1 1/1 Running 1 (3m8s ago) 3m12s
seaweed1-master-2 1/1 Running 2 (2m51s ago) 3m12s
seaweed1-volume-0 1/1 Running 1 (2m30s ago) 2m39s
seaweed1-volume-1 1/1 Running 1 (2m30s ago) 2m39s
seaweed1-volume-2 1/1 Running 3 (99s ago) 2m39s
seaweedfs-operator-5c44d464cc-7bl8g 1/1 Running 0 132m
查看pvc状态，STATUS显示正常应该都为Bound 状态
[root@master cluster]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
mount0-seaweed1-volume-0 Bound seaweedfs-volume-pv-2 5Gi RWO local-storage 4m57s
mount0-seaweed1-volume-1 Bound seaweedfs-volume-pv-1 5Gi RWO local-storage 4m57s
mount0-seaweed1-volume-2 Bound seaweedfs-volume-pv-3 5Gi RWO local-storage 4m57s
seaweed1-filer-seaweed1-filer-0 Bound seaweedfs-filer-pv-2 2Gi RWO local-storage 4m57s
seaweed1-filer-seaweed1-filer-1 Bound seaweedfs-filer-pv-1 2Gi RWO local-storage 4m57s
查看pv, STATUS 显示正常应该都为Bound 状态
[root@master cluster]# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
seaweedfs-filer-pv-1 2Gi RWO Retain Bound default/seaweed1-filer-seaweed1-filer-1 local-storage 8m18s
seaweedfs-filer-pv-2 2Gi RWO Retain Bound default/seaweed1-filer-seaweed1-filer-0 local-storage 8m18s
seaweedfs-volume-pv-1 5Gi RWO Retain Bound default/mount0-seaweed1-volume-1 local-storage 8m18s
seaweedfs-volume-pv-2 5Gi RWO Retain Bound default/mount0-seaweed1-volume-0 local-storage 8m18s
seaweedfs-volume-pv-3 5Gi RWO Retain Bound default/mount0-seaweed1-volume-2 local-storage 8m18s
svc默认都为ClusterIP类型，为方便使用，本例将master和filer改为NodePort类型（修改过程略），修改后查看svc的状态如下：
[root@master cluster]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 11d
seaweed1-filer NodePort 10.101.219.113 <none> 8888:32082/TCP,18888:30125/TCP,8333:32697/TCP 12m
seaweed1-filer-peer ClusterIP None <none> 8888/TCP,18888/TCP,8333/TCP 12m
seaweed1-master NodePort 10.100.21.71 <none> 9333:30145/TCP,19333:32207/TCP 12m
seaweed1-master-peer ClusterIP None <none> 9333/TCP,19333/TCP 12m
seaweed1-volume-0 ClusterIP 10.103.109.209 <none> 8444/TCP,18444/TCP 12m
seaweed1-volume-1 ClusterIP 10.103.155.224 <none> 8444/TCP,18444/TCP 12m
seaweed1-volume-2 ClusterIP 10.104.119.13 <none> 8444/TCP,18444/TCP 12m
seaweed1-volume-peer ClusterIP None <none> 8444/TCP,18444/TCP 12m
seaweedfs-operator ClusterIP 10.101.55.140 <none> 8080/TCP 142m
seaweedfs-operator-webhook ClusterIP 10.106.70.105 <none> 443/TCP 142m 137m
4.4.5 测试集群功能
浏览器访问master webUI查看集群运行信息：
浏览器访问filer webUI 测试filer文件管理（上传，下载，创建目录，删除）测试
查看k8s node 节点pv映射的数据目录，已经有数据写入。
[root@node1 data]# ls seaweedfs-filer-pv-data/ seaweedfs-volume-pv-data/ -l
seaweedfs-filer-pv-data/:
总用量 4
drwxr-xr-x 10 root root 4096 7月 3 15:58 filerldb2seaweedfs-volume-pv-data/:
总用量 41012
-rw-r--r-- 1 root root 25166344 7月 3 16:14 1.dat
-rw-r--r-- 1 root root 96 7月 3 16:14 1.idx
-rw-r--r-- 1 root root 146 7月 3 16:13 1.vif
-rw-r--r-- 1 root root 16802960 7月 3 16:14 3.dat
-rw-r--r-- 1 root root 112 7月 3 16:14 3.idx
-rw-r--r-- 1 root root 146 7月 3 16:13 3.vif
-rw-r--r-- 1 root root 36 7月 3 15:57 vol_dir.uuid
使用aws cli测试S3 服务。
Operator部署的集群默认没有配置Accessid和Accesskey,所以配置为空即可。
aws configure
AWS Access Key ID [****************ykey]:
AWS Secret Access Key [****************cret]:
Default region name [us-east-1]:
Default output format [None]:
创建bucket和上传文件测试：
[root@node3 ~]# aws --endpoint-url=http://192.168.61.21:32697 s3 ls
[root@node3 ~]# aws --endpoint-url=http://192.168.61.21:32697 s3 mb s3://test-bucket
make_bucket: test-bucketecho aaa > file01.txt
[root@node3 ~]# aws --endpoint-url=http://192.168.61.21:32697 s3 cp file01.txt s3://test-bucket/
upload: ./file01.txt to s3://test-bucket/file01.txt
filer webUI 同步显示bucket和文件已经上传。
