Sparse4D运行笔记

Sparse4D有三个版本，其中V1和V2版本的官方文档中环境依赖写得比较模糊且依赖库有版本冲突。

1. Sparse4D V1

创建环境

conda create sparse4dv1 python=3.8

激活环境

conda activate sparse4dv1

安装torch, torchvision, torchaudio

pip install torch==1.13.0+cu116 torchvision==0.14.0+cu116 torchaudio==0.13.0 --extra-index-url https://download.pytorch.org/whl/cu116

安装mmcv

pip install mmcv_full==1.7.1

安装mmdet3d

pip install spconv-cu116==2.2.4pip install mmdet3d==1.0.0rc1

安装其它依赖

pip install mmdet==2.28.2 urllib3==1.26.16 pyquaternion==0.9.9 nuscenes-devkit==1.1.10 yapf==0.33.0 tensorboard==2.14.0 motmetrics==1.1.3 pandas==1.1.5

2. Sparse4D V2

创建环境

conda create sparse4dv2 python=3.8

激活环境

conda activate sparse4dv2

安装torch, torchvision, torchaudio

pip install torch==1.13.0+cu116 torchvision==0.14.0+cu116 torchaudio==0.13.0 --extra-index-url https://download.pytorch.org/whl/cu116

安装mmcv

pip install mmcv_full==1.7.1

安装其它依赖

pip install mmdet==2.28.2 urllib3==1.26.16 pyquaternion==0.9.9 nuscenes-devkit==1.1.10 yapf==0.33.0 tensorboard==2.14.0 motmetrics==1.1.3 pandas==1.1.5

3. Sparse4D V3

3.1. 环境配置

创建环境

conda create sparse4dv3 python=3.8

激活环境

conda activate sparse4dv3

安装torch, torchvision, torchaudio

pip install torch==1.13.0+cu116 torchvision==0.14.0+cu116 torchaudio==0.13.0 --extra-index-url https://download.pytorch.org/whl/cu116

安装mmcv

pip install mmcv_full==1.7.1

安装mmdet3d_plugin

cd projects/mmdet3d_plugin/ops
python3 setup.py develop
cd ../../../

安装其它依赖

pip install mmdet==2.28.2 urllib3==1.26.16 pyquaternion==0.9.9 nuscenes-devkit==1.1.10 yapf==0.33.0 tensorboard==2.14.0 motmetrics==1.1.3 pandas==1.1.5

3.2. 数据准备

首先下载并解压nuscenes数据，然后把它链接到本地

cd ${sparse4d_path}
mkdir data
ln -s path/to/nuscenes ./data/nuscenes

 数据转换

pkl_path="data/nuscenes_anno_pkls"
mkdir -p ${pkl_path}
python3 tools/nuscenes_converter.py --version v1.0-mini --info_prefix ${pkl_path}/nuscenes-mini
python3 tools/nuscenes_converter.py --version v1.0-trainval,v1.0-test --info_prefix ${pkl_path}/nuscenes

使用K-means生成锚点

python3 tools/anchor_generator.py --ann_file ${pkl_path}/nuscenes_infos_train.pkl

3.3. 训练与测试

下载预训练的backbone

mkdir ckpt
wget https://download.pytorch.org/models/resnet50-19c8e357.pth -O ckpt/resnet50-19c8e357.pth

训练与测试

# train
bash local_train.sh sparse4dv3_temporal_r50_1x8_bs6_256x704# test
bash local_test.sh sparse4dv3_temporal_r50_1x8_bs6_256x704  path/to/checkpoint

参考文献

GitHub - HorizonRobotics/Sparse4D