Skip to content

perf(grpo): use PEFT named adapters for reference policy #41

Description

@abrichr

_compute_ref_log_probs clones all LoRA weights, swaps to reference, runs forward, restores — per step per rollout (120+ swaps per training step).

Use PEFT's set_adapter() or named adapters ("current" / "reference") instead.

Also: LoRA state capture uses fragile "lora" string matching on state_dict(); use get_adapter_state_dict() if available.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestperformancePerformance improvement

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions