Skip to content

加载模型报错ValueError: prefetch_factor option could only be specified in multiprocessing.let num_workers > 0 to enable multiprocessing, otherwise set prefetch_factor to None. #453

Description

@licamla

Describe the bug

from fastHan import FastHan

model = FastHan()
model.set_cws_style('cityu')

model('加载模型报错')

加载模型报错

  File "/lib/python3.10/site-packages/fastNLP/core/dataloaders/torch_dataloader/fdl.py", line 152, in __init__
    super().__init__(**dl_kwargs)
  File "/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 245, in __init__
    raise ValueError('prefetch_factor option could only be specified in multiprocessing.'
ValueError: prefetch_factor option could only be specified in multiprocessing.let num_workers > 0 to enable multiprocessing, otherwise set prefetch_factor to None.

Desktop (please complete the following information):

torch 2.0.0+cpu
FastNLP 1.0.1

Additional context
torch的DataLoader参数num_workers=0时,prefetch_factor需要设置成None。
而fastNLP里的子类TorchDataLoader的默认num_workers=0,prefetch_factor=2。按道理来说TorchDataLoader应该像DataLoader默认prefetch_factor=None

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions