Skip to content

Inconsistent inference acceleration parameters across functions + additional optimization opportunities #236

Description

@Mr-Milk

Several functions in LazySlide accept parameters for accelerating model inference (amp, autocast_dtype, compile, compile_kws), but these are not consistently available across the full inference pipeline. Additionally, there are other well-established PyTorch acceleration techniques that could be integrated.

  • tl.feature_extraction: Missing compile, compile_kws
  • tl.feature_aggregation: Missing compile, compile_kws
  • tl.tile_prediction: Missing compile, compile_kws
  • tl.text_embedding: Missing compile, compile_kws
  • tl.zero_shot_score: Missing amp, autocast_dtype, compile, compile_kws
  • tl.slide_caption: Missing amp, autocast_dtype, compile, compile_kws
  • tl.virtual_stain: Missing compile, compile_kws
  • tl.image_generation: Missing compile, compile_kws
  • seg.cells: Missing compile, compile_kws
  • seg.cell_types: Missing compile, compile_kws
  • seg.tissue: Missing compile, compile_kws
  • seg.artifact: Missing compile, compile_kws

Dataloader optimizations:

  1. Set pin_meory=True?
  2. Expose prefetch_factor to the user?
  3. Set persistent_workers=True when num_workers > 0

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions