安装向量数据库Milvus
createtime: 2025-03-01
updatetime: 2025-08-06
author:薄刀刀
硬件:至少使用2个CPU和8GB的初始内存
联网: 需要
主机:192.168.10.1
硬盘要求:SSD固态硬盘 (生产环境需要,测试环境不需要)
耗时:1.5h
版本: 2.3.2 ,然后升级到2.5.5
docker 搭建 milvus
使用Docker Compose独立安装Milvus,安装前请检查硬件和软件的要求:至少使用2个vcpu和8gb的初始内存。否则可能导致安装失败。
下载docker-compose配置文件
mkdir -p /database/app/milvus && cd /database/app/milvus (SSD)
或 mkdir -p /opt/app/milvus && cd /opt/app/milvus (非SSD)
注:/database为SSD固态盘的挂载目录,/opt为普通硬盘的挂载目录
下载配置文件,随后通过docker-compose进行安装milvus
wget https://github.com/milvus-io/milvus/releases/download/v2.3.2/milvus-standalone-docker-compose.yml
如果无法访问,这里提供yml
vim milvus-standalone-docker-compose.yml
version: '3.5'services:etcd:container_name: milvus-etcdimage: quay.io/coreos/etcd:v3.5.5environment:- ETCD_AUTO_COMPACTION_MODE=revision- ETCD_AUTO_COMPACTION_RETENTION=1000- ETCD_QUOTA_BACKEND_BYTES=4294967296- ETCD_SNAPSHOT_COUNT=50000volumes:- ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/etcd:/etcdcommand: etcd -advertise-client-urls=http://127.0.0.1:2379 -listen-client-urls http://0.0.0.0:2379 --data-dir /etcdhealthcheck:test: ["CMD", "etcdctl", "endpoint", "health"]interval: 30stimeout: 20sretries: 3minio:container_name: milvus-minioimage: minio/minio:RELEASE.2023-03-20T20-16-18Zenvironment:MINIO_ACCESS_KEY: minioadminMINIO_SECRET_KEY: minioadminports:- "9001:9001"- "9000:9000"volumes:- ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/minio:/minio_datacommand: minio server /minio_data --console-address ":9001"healthcheck:test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]interval: 30stimeout: 20sretries: 3standalone:container_name: milvus-standaloneimage: milvusdb/milvus:v2.3.2command: ["milvus", "run", "standalone"]security_opt:- seccomp:unconfinedenvironment:ETCD_ENDPOINTS: etcd:2379MINIO_ADDRESS: minio:9000volumes:- ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/milvus:/var/lib/milvushealthcheck:test: ["CMD", "curl", "-f", "http://localhost:9091/healthz"]interval: 30sstart_period: 90stimeout: 20sretries: 3ports:- "19530:19530"- "9091:9091"depends_on:- "etcd"- "minio"networks:default:name: milvus
安装 docker
# docker --version
版本如果过低(例如1.13.1),需要重装,否则后面使用Attu可视化工具会报错
重装docker
卸载原有docker
[root@*****]# rpm -qa|grep docker
docker-buildx-plugin-0.14.1-1.el7.x86_64
docker-ce-cli-26.1.4-1.el7.x86_64
docker-ce-19.03.15-3.el7.x86_64
docker-compose-plugin-2.27.1-1.el7.x86_64
[root@*****]# rpm -e docker-buildx-plugin-0.14.1-1.el7.x86_64 --nodepsrpm -e docker-ce-cli-26.1.4-1.el7.x86_64 --nodepsrpm -e docker-ce-19.03.15-3.el7.x86_64 --nodepsrpm -e docker-compose-plugin-2.27.1-1.el7.x86_64 --nodeps
rm -rf /var/lib/docker/*
或yum remove docker \docker-client \docker-client-latest \docker-common \docker-latest \docker-latest-logrotate \docker-logrotate \docker-engine
安装docker
cd /etc/yum.repos.dmkdir bakmv *.repo bak/curl -o /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repoyum install -y yum-utils device-mapper-persistent-data lvm2yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.reposed -i -e '/mirrors.cloud.aliyuncs.com/d' -e '/mirrors.aliyuncs.com/d' /etc/yum.repos.d/CentOS-Base.repoyum install docker-ce-19.03.* -y 如果报错或是因为网络原因再次执行即可systemctl enable docker --now 开机启动并立即启动
确认启动
# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
看到标头,表示正常# systemctl status docker|grep activeActive: active (running)
看到 active (running),表示启动正常
查看docker版本
# docker version 大于24
Client: Docker Engine - CommunityVersion: 26.1.4API version: 1.40 (downgraded from 1.45)Go version: go1.21.11Git commit: 5650f9bBuilt: Wed Jun 5 11:32:04 2024OS/Arch: linux/amd64Context: defaultServer: Docker Engine - CommunityEngine:Version: 19.03.15API version: 1.40 (minimum version 1.12)Go version: go1.13.15Git commit: 99e3ed8919Built: Sat Jan 30 03:16:33 2021OS/Arch: linux/amd64Experimental: falsecontainerd:Version: 1.6.33GitCommit: d2d58213f83a351ca8f528a95fbd145f5654e957runc:Version: 1.1.12GitCommit: v1.1.12-0-g51d5e94docker-init:Version: 0.18.0GitCommit: fec3683
。
启动
docker compose -f milvus-standalone-docker-compose.yml up -d
该步骤会拉取镜像 Pulling-->解压-->启动
如果报错 Error response from daemon: Range of CPUs is from 0.01 to 2.00, as there are only 2 CPUs available
是因为机器配置较低,调整配置文件milvus-standalone-docker-compose.yml ,将cpu部分改低即可
# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
75030ccc3afa milvusdb/milvus:v2.3.2 "/tini -- milvus run…" 13 seconds ago Up 11 seconds (health: starting) 0.0.0.0:9091->9091/tcp, 0.0.0.0:19530->19530/tcp milvus-standalone
ea7388460edc minio/minio:RELEASE.2023-03-20T20-16-18Z "/usr/bin/docker-ent…" 13 seconds ago Up 12 seconds (health: starting) 0.0.0.0:9000-9001->9000-9001/tcp milvus-minio
d174608777a9 quay.io/coreos/etcd:v3.5.5 "etcd -advertise-cli…" 13 seconds ago Up 12 seconds (health: starting) 2379-2380/tcp milvus-etcd
看到milvusdb,minio,etcd均已启动
搭建python环境
因anaconda存在商业授权争议,强烈建议使用miniforge替换
Linux下安装miniforge
Linux下下载、安装miniforge,并去掉mamba的banner。
cd /database/app
mkdir -p miniforge
安装Miniforge
cd /database/app/miniforge 或 cd /opt/app/miniforge
wget https://mirrors.tuna.tsinghua.edu.cn/github-release/conda-forge/miniforge/LatestRelease/Miniforge3-Linux-x86_64.shchmod +x Miniforge3-Linux-x86_64.sh ./Miniforge3-Linux-x86_64.sh -b -f -p ~/.miniforge~/.miniforge/bin/mamba init
bash
conda config --set auto_activate_base false
Remove Mamba Banner
mamba activate base
mamba env config vars set MAMBA_NO_BANNER=1
mamba deactivate
mamba activate base
conda换清华源
echo "channels:- defaults
show_channel_urls: true
default_channels:- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2
custom_channels:conda-forge: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloudmsys2: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloudbioconda: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloudmenpo: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloudpytorch: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloudpytorch-lts: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloudsimpleitk: https://mirrors.tuna.tsinghua.edu.cn/anaconda/clouddeepmodeling: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/" > ~/.condarc
pip换清华源
# pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple
Writing to /root/.config/pip/pip.conf
退出重新登入会话
milvus需要的是python3.7+ 的运行环境。环境搭建好后,需要安装依赖库 pymilvus
详见官网介绍:https://milvus.io/docs/install-pymilvus.md
注意:pymilvus 默认conda 是安装不来的,但是pip3 可以直接安装
(base) # conda install numpy==2.2.2
(base)# pip3 install pymilvus==2.5.5
报错1
pip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out.
原因:网速太慢,超时
解决:设置代理报错2
pip3 install pymilvus==2.3.2
报错 from __future__ import annotationsSyntaxError: future feature annotations is not defined
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-4udr2cvc/ujson/
原因:使用了老版本pip
解决:退出重登
(base) [root@vm-prd-dba-dba-185-45 ~]# pip3 --version
pip 23.2.1 from /database/anaconda3/lib/python3.11/site-packages/pip (python 3.11)
(base) [root@vm-prd-dba-dba-185-45 ~]# pip3 install pymilvus==2.3.2报错3
Failed to build grpcio
ERROR: Failed to build installable wheels for some pyproject.toml based projects (grpcio)
原因:python版本较高
解决:pip3 install pymilvus
可以用如下命令验证环境是否搭建成功(没报错就说明成功了)
python3 -c "from pymilvus import Collection"报错 from backports import tarfileImportError: cannot import name 'tarfile' from 'backports' (/database/anaconda3/lib/python3.11/site-packages/backports/__init__.py)
原因:backports 包可能未正确安装或部分文件丢失。
解决 pip3 install --force-reinstall backports.tarfile
连接 Milvus
milvus使用的端口是19530
查看容器使用的端口:
# docker port milvus-standalone
19530/tcp -> 0.0.0.0:19530
9091/tcp -> 0.0.0.0:9091查看容器是否在某个端口上进行监听:
# docker port milvus-standalone 19530/tcp
0.0.0.0:19530
检查服务
telnet 192.168.10.1 19530
安装 Attu 可视化工具
在github上有安装命令 :https://github.com/zilliztech/attu/blob/main/doc/zh-CN/attu_install-docker.md
将Milvus的IP从 127.0.0.1改为本机IP(enp0s3)或docker(docker0)的IP,都是可以的
使用修改IP后的命令重启Attu容器
docker run -d -p 8008:3000 -e HOST_URL=http://192.168.10.1:8008 -e MILVUS_URL=127.0.0.1:19530 zilliz/attu:latest
docker run -d -p 8008:3000 -e HOST_URL=http://192.168.10.1:8008 -e MILVUS_URL=127.0.0.1:19530 zilliz/attu:2.4.10
若拉取镜像失败,其他机器拉镜像,再传到这台机器上
检查服务
telnet 192.168.10.1 8008
然后使用web浏览器输入 http://192.168.10.1:8008,进行连接,并登录
如果打开页面 能看到Attu的logo,但是页面是空白,请尝试更换浏览器,使用腾讯QQ浏览器解决
另外,你也可以将Attu加入到docker-compose.xml文件中,让它们一起启动,命令如下
attu:container_name: attuimage: zilliz/attu:v2.2.6environment:MILVUS_URL: milvus-standalone:19530ports:- "8000:3000"depends_on:- "standalone"
参考文章:
参考文章:
1,一文带你入门向量数据库milvus:含docker安装、milvus安装使用、attu 可视化,完整指南启动 Milvus 进行了向量相似度搜索-CSDN博客
2,华为云论坛_问答专区_云计算论坛_开发者论坛_技术论坛-华为云
3,https://zhuanlan.zhihu.com/p/379567315
4,https://www.cnblogs.com/fang-d/p/17832995.html