_compute_ref_log_probs clones all LoRA weights, swaps to reference, runs forward, restores — per step per rollout (120+ swaps per training step).
Use PEFT's set_adapter() or named adapters ("current" / "reference") instead.
Also: LoRA state capture uses fragile "lora" string matching on state_dict(); use get_adapter_state_dict() if available.
_compute_ref_log_probsclones all LoRA weights, swaps to reference, runs forward, restores — per step per rollout (120+ swaps per training step).Use PEFT's
set_adapter()or named adapters ("current" / "reference") instead.Also: LoRA state capture uses fragile "lora" string matching on
state_dict(); useget_adapter_state_dict()if available.