Hey everyone,
I’m training a Diffusion Policy for a task that requires active force feedback and correction, and I’m running into an issue.
When I include force–torque (FT) data in the low-dim observations (concatenated with end-effector position and velocity) alongside two RGB images, the policy seems to ignore the FT signals during inference. I’ve tested multiple datasets, training runs and inference runs :
Inference : The policy performs well and executes task properly.
Behavioral checks:
Zero-Force test: zeroing FT inputs leaves performance unchanged (policy still completes the task) → suggests the policy largely ignores FT.
High-Force test: injecting large FT values changes behavior drastically and the robot stops trying the task → FT is “seen” but not used in a helpful way.
Setup details:
Controller: Position Controller via UFactory XArm API
policy input : two RGB cameras + low-dimensional state (end-effector pose/velocity + FT).
Policy output: absolute Cartesian actions [X, Y, Z, Roll, Pitch, Yaw]
models used : raw diffusion policy with and without imagenet weights.
Question:
What’s the most effective way to increase the influence of FT during training so the policy actually uses it at inference?
Would be cool to have :
- Pointers, papers, or resources where similar problem can be found.
- Practical tips (architectural or training-time) that made FT signals matter in your setups.
If anyone wants to discuss this whole issue further, please feel free to reach out to me.
Hey everyone,
I’m training a Diffusion Policy for a task that requires active force feedback and correction, and I’m running into an issue.
When I include force–torque (FT) data in the low-dim observations (concatenated with end-effector position and velocity) alongside two RGB images, the policy seems to ignore the FT signals during inference. I’ve tested multiple datasets, training runs and inference runs :
Inference : The policy performs well and executes task properly.
Behavioral checks:
Zero-Force test: zeroing FT inputs leaves performance unchanged (policy still completes the task) → suggests the policy largely ignores FT.
High-Force test: injecting large FT values changes behavior drastically and the robot stops trying the task → FT is “seen” but not used in a helpful way.
Setup details:
Controller: Position Controller via UFactory XArm API
policy input : two RGB cameras + low-dimensional state (end-effector pose/velocity + FT).
Policy output: absolute Cartesian actions [X, Y, Z, Roll, Pitch, Yaw]
models used : raw diffusion policy with and without imagenet weights.
Question:
What’s the most effective way to increase the influence of FT during training so the policy actually uses it at inference?
Would be cool to have :
If anyone wants to discuss this whole issue further, please feel free to reach out to me.