How is variance caclulated over runs

[SaitejaUtpala]

Hi,

Great work, I have question about how is variance calculated, where is random seed being changed.

• Is it at train/val split level ?

python data/split_tasks.py \
    --json_path datasets/sciknoweval/chemistry.json \
    --output_dir datasets/sciknoweval/chemistry \
    --test_ratio 0.1 \
    --seed 42 (change here)

• Or is it seed that controls sampling randomness during generation ?

vllm_async_server.py: "seed": self.config.get("seed", 0)

• Or is it seed that controls controls mini-batch ordering during PPO updates

actor.data_loader_seed: 42 (in actor.yaml, also mirrored to critic.data_loader_seed)

[Doow501]

Hi,did u solve this question？I also meet this question.