transformers==4.42.0会有一个BUG

transformers==4.42.0版本下，自动安装模型时出现一个BUG（自动从Hugging Faces上下载）。

2025-07-02 14:07:08,641 - __main__ - ERROR - 模型加载失败: Failed to import transformers.models.llama.tokenization_llama_fast because of the following error (look up to see its traceback):
(unicode error) 'utf-8' codec can't decode bytes in position 3344-3345: invalid continuation byte (tokenization_llama_fast.py, line 120)
ERROR:    Traceback (most recent call last):
  File "/root/miniconda3/envs/我的conda名称/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1560, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
  File "/root/miniconda3/envs/我的conda名称/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 879, in exec_module
  File "<frozen importlib._bootstrap_external>", line 1017, in get_code
  File "<frozen importlib._bootstrap_external>", line 947, in source_to_code
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/root/miniconda3/envs/我的conda名称/lib/python3.10/site-packages/transformers/models/llama/tokenization_llama_fast.py", line 120
    """
       ^
SyntaxError: (unicode error) 'utf-8' codec can't decode bytes in position 3344-3345: invalid continuation byte

其实表面上看就知道这个是一个字符编码的问题，但是网上没有什么简单的针对方案。其实你找到这个文件，打开一开似乎还看不出 tokenization_llama_fast.py, line 120 有什么问题，而且你还会发现这一行其实就是一个注释，最简单的方式就是清除注释，之后运行就正常了，非常简单。