Failed to run plotter.py

[levilovearch]

Hi @yihaosun1124
Thanks for sharing your great work!
I failed to run python run_example/plotter.py --algos "rambo" --task "hopper-medium-replay-v2", after following your setup instruction.
The outputs are:

Traceback (most recent call last):
  File "run_example/plotter.py", line 173, in <module>
    csv_file = merge_csv(path, args.query_file, args.query_x, args.query_y)
  File "run_example/plotter.py", line 42, in merge_csv
    assert len(results) > 0
AssertionError

After reading plotter.py, I'm not sure what this assert is for. Would you check from your side?

Best,
Levi

[levilovearch]

Hi,
I would also mention that RAMBO failed to train from my side. Currently, I don't have enough time to check which part is wrong, so I just attached the training log for your reference.

Best,
Levi
policy_training_progress.csv

dynamics_training_progress.csv

[yihaosun1124]

This error in plotter.py occurred because the path you specified does not exist. You may carefully inspect whether the specified path "log/hopper-medium-replay-v2/rambo" exists.

For the second issue, could you please additionally provide the "hyper_params.json" file? It will help us to better locate the bug.

[levilovearch]

You may carefully inspect whether the specified path "log/hopper-medium-replay-v2/rambo" exists.

No, I didn't create this path. I will create this path and try again. Thank you.

For the second issue, could you please additionally provide the "hyper_params.json" file? It will help us to better locate the bug.

There you go, sir.
{"algo_name": "rambo", "task": "hopper-medium-v2", "seed": 0, "actor_lr": 0.0001, "critic_lr": 0.0003, "dynamics_lr": 0.0003, "dynamics_adv_lr": 0.0003, "hidden_dims": [256, 256, 256], "gamma": 0.99, "tau": 0.005, "alpha": 0.2, "auto_alpha": true, "target_entropy": -3.0, "alpha_lr": 0.0001, "dynamics_hidden_dims": [200, 200, 200, 200], "dynamics_weight_decay": [2.5e-05, 5e-05, 7.5e-05, 7.5e-05, 0.0001], "n_ensemble": 7, "n_elites": 5, "rollout_freq": 250, "dynamics_update_freq": 1000, "adv_batch_size": 256, "rollout_batch_size": 50000, "rollout_length": 5, "adv_weight": 0.0003, "model_retain_epochs": 5, "real_ratio": 0.5, "load_dynamics_path": null, "epoch": 2000, "step_per_epoch": 1000, "eval_episodes": 10, "batch_size": 256, "device": "cuda", "include_ent_in_adv": false, "load_bc_path": null, "bc_lr": 0.0001, "bc_epoch": 50, "bc_batch_size": 256, "obs_shape": [11], "action_dim": 3.0, "max_action": 1.0}

[yihaosun1124]

Hi, I would also mention that RAMBO failed to train from my side. Currently, I don't have enough time to check which part is wrong, so I just attached the training log for your reference.

Best, Levi policy_training_progress.csv

dynamics_training_progress.csv

Sorry for the late reply! We've fixed the bug in rambo and the new version of rambo works well on several tested tasks (hopper-medium, hopper-medium-replay, walker-medium). Please pull the newest code! Evaluation on more tasks is going on and we'll list the full results soon.

[levilovearch]

Hi, I have seen your recent PR

           -   observations = next_observations
           +   observations = next_observations.copy()

That makes sense to me. Well done, sir!
I'm testing the new code from my side and will update whether it works fine or not

[levilovearch]

Hi,
Unfortunately, it does work for the same hyperparameter listed above. I will try with a different random seed and see if there is any difference.