Hi,
I have been working on reproducing the cup in the wild experiment for a couple of weeks now. I have a UR5e robot and an OnRobot RG6 gripper. I created the driver for the gripper and it seems to work ok.
The problem I'm facing is that I do not know from where to start to tune the parameters of the program. I followed the guide in the README, so I can run the eval_real.py script, but when I start the inference, the robot either jams and raises a kinematic error or it crashes the gripper into the table. I thought it was going for the cup for a while, until I switched the cup for the saucer and now it goes for the saucer (it's basically just moving forward and down).
From reading the paper, I know that some of the parameters to set in order to have a hope of getting a functionning inference are the following :
- mirror_swap should be ON (See page 8, Effect of side mirrors [HD3]).
- observation and action pose representation should be relative (See page 7, Alternative action spaces [PD2]).
Question 1 :
I was wondering if someone could explain which steps to take and in which order to do them to tune the parameters of the program.
The parameters I'm the most curious about are "steps_per_inference", "frequency", "command_latency", "gripper_speed", "robot_obs_latency" and "robot_action_latency". I have no clue on how to set them up (do I have to?). I was expecting to run the demo and have a pseudo-functional setup out of the box, as most of the hardware I use is the same as what you guys used.
Question 2 :
What speed percentage is the robot set at? It's obviously not 100% (ask me how I know). When we look at the videos on the project page, the footage seems to be accelerated, but we don't know by how much, making it impossible to reverse engineer the speed setting of the robot.
Question 3 :
When setting the "tcp_offset" parameter in eval_robots_config.yaml, from where is the measurement taken?
From the wrist flange to the TCP?
From the camera mount to the TCP?
My setup :
Arm : UR5e
Gripper : OnRobot RG6
Capture : Elgato HD260X
Camera : GoPro Hero9 Black with 155° Fisheye lens (the same used for the research)
Cups and saucers : From Amazon, the same as the ones used for cup in the wild and some 3D printed
Environment : Industrial lab, no direct sunlight, no disturbance in background
OS : Ubuntu 22.04.5 LTS
CPU : Ryzen 9 7950X
GPU : RTX 4090
RAM : 64GB
Network : Gigabit Ethernet
Thank you in advance.
Alfred
Hi,
I have been working on reproducing the cup in the wild experiment for a couple of weeks now. I have a UR5e robot and an OnRobot RG6 gripper. I created the driver for the gripper and it seems to work ok.
The problem I'm facing is that I do not know from where to start to tune the parameters of the program. I followed the guide in the README, so I can run the eval_real.py script, but when I start the inference, the robot either jams and raises a kinematic error or it crashes the gripper into the table. I thought it was going for the cup for a while, until I switched the cup for the saucer and now it goes for the saucer (it's basically just moving forward and down).
From reading the paper, I know that some of the parameters to set in order to have a hope of getting a functionning inference are the following :
Question 1 :
I was wondering if someone could explain which steps to take and in which order to do them to tune the parameters of the program.
The parameters I'm the most curious about are "steps_per_inference", "frequency", "command_latency", "gripper_speed", "robot_obs_latency" and "robot_action_latency". I have no clue on how to set them up (do I have to?). I was expecting to run the demo and have a pseudo-functional setup out of the box, as most of the hardware I use is the same as what you guys used.
Question 2 :
What speed percentage is the robot set at? It's obviously not 100% (ask me how I know). When we look at the videos on the project page, the footage seems to be accelerated, but we don't know by how much, making it impossible to reverse engineer the speed setting of the robot.
Question 3 :
When setting the "tcp_offset" parameter in eval_robots_config.yaml, from where is the measurement taken?
From the wrist flange to the TCP?
From the camera mount to the TCP?
My setup :
Arm : UR5e
Gripper : OnRobot RG6
Capture : Elgato HD260X
Camera : GoPro Hero9 Black with 155° Fisheye lens (the same used for the research)
Cups and saucers : From Amazon, the same as the ones used for cup in the wild and some 3D printed
Environment : Industrial lab, no direct sunlight, no disturbance in background
OS : Ubuntu 22.04.5 LTS
CPU : Ryzen 9 7950X
GPU : RTX 4090
RAM : 64GB
Network : Gigabit Ethernet
Thank you in advance.
Alfred