大模型推理环境安装过程中踩坑记录

1.找不到libsndfile

  File "/.conda/envs/gemma/lib/python3.11/site-packages/soundfile.py", line 181, in <module>import _soundfile_data  # ImportError if this doesn't exist^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named '_soundfile_data'During handling of the above exception, another exception occurred:Traceback (most recent call last):File "/.conda/envs/gemma/lib/python3.11/site-packages/soundfile.py", line 190, in <module>raise OSError('sndfile library not found using ctypes.util.find_library')
OSError: sndfile library not found using ctypes.util.find_libraryDuring handling of the above exception, another exception occurred:Traceback (most recent call last):File "gemma-app1.py", line 83, in predictinputs = processor.apply_chat_template(^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^File "/.conda/envs/gemma/lib/python3.11/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_funcreturn func(*args, **kwargs)^^^^^^^^^^^^^^^^^^^^^File "/.conda/envs/gemma/lib/python3.11/site-packages/transformers/processing_utils.py", line 1598, in apply_chat_templatebatch_audios.append(load_audio(fname, sampling_rate=mm_load_kwargs["sampling_rate"]))^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^File "/.conda/envs/gemma/lib/python3.11/site-packages/transformers/audio_utils.py", line 62, in load_audioaudio = librosa.load(audio, sr=sampling_rate)[0]^^^^^^^^^^^^File "/.conda/envs/gemma/lib/python3.11/site-packages/lazy_loader/__init__.py", line 83, in __getattr__attr = getattr(submod, name)^^^^^^^^^^^^^^^^^^^^^File "/.conda/envs/gemma/lib/python3.11/site-packages/lazy_loader/__init__.py", line 82, in __getattr__submod = importlib.import_module(submod_path)^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^File "/.conda/envs/gemma/lib/python3.11/importlib/__init__.py", line 126, in import_modulereturn _bootstrap._gcd_import(name[level:], package, level)^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^File "<frozen importlib._bootstrap>", line 1204, in _gcd_importFile "<frozen importlib._bootstrap>", line 1176, in _find_and_loadFile "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlockedFile "<frozen importlib._bootstrap>", line 690, in _load_unlockedFile "<frozen importlib._bootstrap_external>", line 940, in exec_moduleFile "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removedFile "/.conda/envs/gemma/lib/python3.11/site-packages/librosa/core/audio.py", line 10, in <module>import soundfile as sfFile "/.conda/envs/gemma/lib/python3.11/site-packages/soundfile.py", line 212, in <module>_snd = _ffi.dlopen(_explicit_libname)^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: cannot load library 'libsndfile.so': libsndfile.so: cannot open shared object file: No such file or directory
Error generating response: cannot load library 'libsndfile.so': libsndfile.so: cannot open shared object file: No such file or directory

解决方法：

conda install -c conda-forge libsndfile

2.Input type (torch.cuda.FloatTensor) and weight type (CUDABFloat16Type) should be the same

原始代码如下：

inputs = processor.apply_chat_template(messages,add_generation_prompt=True,tokenize=True,return_dict=True,return_tensors="pt",).to(device)

修改后：

inputs = processor.apply_chat_template(messages,add_generation_prompt=True,tokenize=True,return_dict=True,return_tensors="pt",).to(device, dtype=torch.bfloat16)