This repository documents an exploratory research project on converting and recreating simplified OpenSim exoskeleton / dynamic walking models in a ROS 2 and Ignition Gazebo simulation environment. The longer-term motivation is to create a pipeline where biomechanical models from OpenSim can be represented in an online robotics simulation stack, controlled through ROS 2, visualized through RViz, and eventually integrated with higher-level policies such as OpenVLA or other Vision-Language-Action (VLA) models.
This project is not a finished locomotion controller or validated OpenSim-to-Gazebo converter. It is a working exploratory prototype and handoff package for future students to build on.
This project began from a discussion with Professor Yasser Shoukry about possible research directions involving control systems, exoskeleton simulation, and Vision-Language-Action models. The initial proposed direction was:
- Get familiar with Vision-Language-Action models, especially OpenVLA.
- Get familiar with OpenSim and exoskeleton / dynamic walking examples.
- Investigate whether OpenSim can be connected to ROS or whether an OpenSim model can be represented in a ROS-compatible simulation framework.
- Explore how OpenVLA could eventually be integrated with an exoskeleton robot simulation.
The original suggested technical path was:
- Step 1: Get familiar with OpenSim examples and tutorials.
- Step 2: Since OpenSim is not built around ROS, consider whether a ROS wrapper could extract sensor data, visual feedback, and accept control commands.
- Step 3: Integrate OpenVLA to control an example exoskeleton or dynamic walking setup.
During the project, I found that OpenSim is excellent for biomechanical modeling and offline analysis, but it is not naturally structured as an online ROS 2 closed-loop simulation environment. Because of that, I shifted toward extracting or recreating the relevant kinematic and inertial structure from an OpenSim model into URDF/Xacro/SDF and simulating it in Ignition Gazebo.
- General VLA overview: https://ianloe.medium.com/the-complete-guide-to-vision-language-action-models-how-robots-are-learning-to-think-f1a788d003ed
- OpenVLA paper: https://arxiv.org/abs/2406.09246
- OpenVLA codebase: https://github.com/openvla/openvla
- OpenVLA tutorial: https://www.youtube.com/watch?v=-0s0v3q7mBk
- OpenSim home page: https://simtk.org/projects/opensim
- OpenSim documentation and tutorials: https://opensimconfluence.atlassian.net/wiki/spaces/OpenSim/overview?homepageId=53084162
- Dynamic Walking Challenge example: https://opensimconfluence.atlassian.net/wiki/spaces/OpenSim/pages/53084247/Dynamic+Walking+Challenge+Go+the+Distance
- Example paper on OpenSim-ROS integration: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=11442793
- Symbolic muscle moment arm library: https://simtk.org/projects/sym-moment-arm
The high-level objective is to explore a software pipeline like this:
OpenSim .osim model
↓
Extract kinematic, inertial, contact, and actuator information
↓
URDF / Xacro / SDF robot model
↓
ROS 2 + Ignition Gazebo simulation
↓
Low-level joint / locomotion controller
↓
High-level VLA or OpenVLA-style command interface
The long-term research question is whether high-fidelity biomechanical / exoskeleton models from OpenSim can be represented in an online robotics simulation stack while preserving enough model fidelity to support control, visual feedback, and eventually learning-based policies.
OpenSim is strong for:
- Biomechanical modeling.
- Musculoskeletal simulation.
- Offline force and motion analysis.
- Evaluating changes in model parameters.
- Replaying and analyzing simulation or sensor-derived motion data.
- Studying muscle paths, wrapping surfaces, and moment arms.
OpenSim is not naturally designed around:
- ROS 2 topic-based communication.
- Real-time online feedback control.
- Standard robotics sensors such as cameras, lidar, odometry, and TF trees.
- Gazebo / RViz style interactive simulation and visualization.
- Direct integration with common ROS 2 control pipelines.
A ROS wrapper around OpenSim may be possible, but a more practical first step was to recreate a simplified model in URDF/Xacro/SDF and control it through ROS 2 and Gazebo.
A useful OpenSim-to-ROS/Gazebo pipeline would need to extract and validate several layers of model information.
The pipeline should extract:
- Links / bodies.
- Joint topology.
- Joint types.
- Joint axes.
- Joint limits.
- Relative transforms between bodies.
Validation idea:
- Compare joint positions and link poses for a set of prescribed movements in OpenSim and Gazebo.
- Verify that both models produce equivalent kinematic motion.
The pipeline should extract:
- Link masses.
- Centers of mass.
- Moments of inertia.
- Joint force / torque relationships.
- Contact geometry and contact parameters.
Validation idea:
- Compare joint forces, joint torques, and contact forces under matched motion cases.
- Compare falling, stepping, or dynamic walking behaviors under equivalent initial conditions.
The OpenSim dynamic walking example uses contact force modeling, including Hertzian-style compliant contact. A Gazebo representation would need either:
- An approximate physics-engine contact model, or
- Additional force sensors / contact plugins to compare against OpenSim contact force outputs.
Validation idea:
- Add force/contact sensors in Gazebo.
- Compare foot-ground contact forces between OpenSim and Gazebo over similar motion cases.
A future, higher-fidelity pipeline may need to represent muscles as actuator constraints rather than simple joint position commands.
Potential future path:
- Use a symbolic moment-arm library to extract muscle moment arm information.
- Convert muscle actuation into equivalent torque or constraint models.
- Begin with direct joint control, then later replace ideal joint actuation with muscle-inspired actuator models.
OpenVLA is a Vision-Language-Action model trained primarily for robot manipulation policies. The released OpenVLA model takes an image observation and language instruction and predicts robot action tokens, such as relative end-effector motion and gripper commands.
For a biped / exoskeleton robot, the action space and control problem are different. A walking robot needs stable low-level control, proprioception, contact handling, joint coordination, and relatively fast feedback. Therefore, my current interpretation is that the most practical architecture would be:
Camera observation + language command
↓
OpenVLA or VLA-style model
↓
High-level task or motion command
↓
Stable low-level locomotion controller
↓
Joint commands / torques
↓
Gazebo / physical robot
Example high-level commands could include:
- Walk forward.
- Stop.
- Step backward.
- Turn left / right.
- Move toward an object.
The lower-level controller could be model-based, learning-based, or hybrid. Candidate approaches include:
- Model Predictive Control (MPC).
- Learning-based MPC.
- Reinforcement learning in a MuJoCo / Gym-style environment.
- A gait generator plus stabilizing feedback controller.
The current repository does not implement OpenVLA integration. It provides an initial ROS 2 / Gazebo simulation and joint command interface that could support future integration.
This repository contains a working ROS 2 package named exoskeleton.
Implemented components include:
- A simplified dynamic walker / exoskeleton robot model.
- OpenSim
.osimreference files. - Xacro / URDF / SDF robot descriptions.
- Ignition Gazebo world file.
- ROS 2 launch file for Gazebo, RViz, bridges, camera, lidar, odometry, and TF.
- ROS-Gazebo topic bridges.
- RViz configuration.
- Camera feeds.
- Robot camera model for external visual observation.
- Lidar scan visualization.
- Odometry publishing.
- Static transforms for camera and lidar frames.
- A keyboard-based joint controller for commanding each leg joint.
exoskeleton-main/
├── README.md
├── package.xml
├── setup.py
├── setup.cfg
├── exoskeleton/
│ ├── __init__.py
│ └── joint_keyboard_controller.py
├── launch/
│ └── robot_control_launch.py
├── rviz/
│ └── dynamic_walker.rviz
├── urdf/
│ ├── DynamicWalkerModel.osim
│ ├── DynamicWalkerModel.sdf
│ ├── DynamicWalkerModel.urdf
│ ├── DynamicWalkerModel.xacro
│ └── DynamicWalkerModel_FullDOF.xacro
└── world/
└── robot_world.sdf
This project was developed with:
- Ubuntu 22.04
- ROS 2 Humble
- Ignition Gazebo / Gazebo Sim bridge packages
From the ROS 2 workspace root:
cd ~/ros2_ws
colcon build --packages-select exoskeleton
source install/setup.bashIf needed, source ROS 2 first:
source /opt/ros/humble/setup.bashros2 launch exoskeleton robot_control_launch.pyThis launches:
- Ignition Gazebo with
robot_world.sdf. - ROS-Gazebo topic bridges.
- Image bridges for camera feeds.
- RViz with
dynamic_walker.rviz. robot_state_publisher.- Static TF publishers for camera / lidar frames.
In a second terminal:
cd ~/ros2_ws
source install/setup.bash
ros2 run exoskeleton joint_keyboard_controllerThe controller supports direct joint angle commands:
set lt -30
set ls 10
set rt 30
set rs 15
Joint abbreviations:
lt = left_thigh
ls = left_shin
rt = right_thigh
rs = right_shin
Joystick mode:
joy
Current keyboard mapping:
q/a : left_thigh
w/s : left_shin
t/f : right_shin
y/g : right_thigh
Other commands:
zero : command all joints to 0 deg
status : print current commanded angles
step N : set joystick step size to N degrees
help : show help menu
quit : exit
ros2 topic list | grep camera
ros2 topic echo /camera/image_raw --once
ros2 topic echo /robotcamera/image_raw --once
ros2 topic hz /camera/image_raw
ros2 topic hz /robotcamera/image_rawros2 topic list | grep scan
ros2 topic echo /scan --onceros2 topic list | grep odom
ros2 topic echo /odom --once
ros2 run tf2_ros tf2_echo odom base_linkros2 run tf2_tools view_framesign topic -l
ign topic -e -t /odom
ign topic -e -t /world/robot_world/pose/inforos2 run xacro xacro DynamicWalkerModel.xacro > DynamicWalkerModel.urdfcheck_urdf DynamicWalkerModel.urdfgz sdf -p DynamicWalkerModel.urdf > DynamicWalkerModel.sdfgz sdf -k DynamicWalkerModel.sdfign gazeboign gazebo world/robot_world.sdfWorking:
- Gazebo simulation launches - the robot_world.sdf file is from a saved Gazebo world with manual edits inside to support sensor plugins and joint control
- Robot model appears in Gazebo - these robots models are manually created from .xacro files
- RViz displays the robot model.
- Camera feeds are bridged into ROS 2.
- Robot camera feed works.
- Lidar scan is bridged and visualized.
- Odometry publishes.
- TF frames are available for RViz visualization.
- Individual joint commands can be sent from ROS 2 to Gazebo.
- Keyboard controller can command absolute joint angles or incremental joystick-style motion.
Not yet implemented:
- Stable walking gait.
- Closed-loop balancing controller.
- MPC or RL controller.
- MuJoCo simulation.
- OpenVLA integration.
- Automated OpenSim-to-URDF/SDF conversion.
- Quantitative validation against OpenSim simulation outputs.
- Muscle actuator model.
- Force/contact validation against Hertzian contact behavior.
-
The model is simplified.
The current robot is a simplified biped model, not a validated full exoskeleton model. -
The controller is not a walking controller.
The keyboard controller only sends joint position commands. It is useful for testing joints and ROS/Gazebo integration, but it does not stabilize walking. -
The OpenSim conversion is manual.
The current model was manually recreated from OpenSim-inspired structure. There is no automated.osimto.urdfor.sdfconverter in this repository. -
The dynamics are not validated.
Masses, inertias, and contact behavior should be compared against OpenSim before claiming model fidelity. -
Contact modeling needs more work.
Gazebo contact forces and OpenSim Hertzian-style contact models may not match directly without careful tuning and validation. -
OpenVLA is not integrated.
The repository has camera feeds that could support future visual policy integration, but there is no OpenVLA inference node yet.
- Clean up the launch file and use package share paths instead of workspace-relative
src/...paths. - Install
urdf,world, andrvizdirectories properly throughsetup.py. - Add documentation for all ROS topics and frames.
- Add a simple scripted gait sequence to test coordinated leg movement.
- Add joint state logging for commanded vs. actual joint positions.
- Add force/contact sensors to the feet.
- Create a minimal validation script comparing Gazebo output to expected OpenSim-derived quantities.
Possible next controller stages:
-
Open-loop gait playback
Command a simple periodic gait and observe stability. -
Feedback gait controller
Use joint states, odometry, and possibly IMU/contact sensors for feedback. -
Model Predictive Control
Formulate a simplified biped model and solve for stabilizing joint trajectories. -
Reinforcement Learning
Train a walking policy in MuJoCo/Gym, then attempt transfer back to Gazebo/ROS. -
Hierarchical control
Use a high-level VLA-style model for task intent and a low-level controller for stability.
A future OpenSim-to-ROS conversion tool could attempt to extract:
- Body names and hierarchy.
- Joint definitions.
- Joint limits and axes.
- Body inertias.
- Contact geometry.
- Muscle paths and wrapping surfaces.
- Moment arm information.
- Actuator constraints.
Validation should be done in stages:
- Kinematic validation.
- Dynamic validation.
- Contact validation.
- Muscle/actuator validation.
- Closed-loop controller validation.
Possible integration directions:
- Use the robot camera feed as the visual input to a high-level VLA model.
- Define a discrete set of high-level commands such as
walk_forward,stop,turn_left, andturn_right. - Let a stable low-level controller handle joint-level execution.
- Alternatively, collect demonstrations and fine-tune a VLA-style model on a newly defined exoskeleton action space.
The OpenSim dynamic walking example uses compliant contact modeling to represent ground interaction. Hertzian contact models are often used to approximate normal contact force as a nonlinear function of penetration depth. A simplified conceptual form is:
F_n ≈ k * δ^n + damping terms
where:
F_nis normal contact force.δis penetration depth.kis an effective contact stiffness.nis commonly related to the contact geometry.
For this project, the main concern is not deriving the full contact law, but validating whether Gazebo contact behavior produces comparable forces and motion trends to the OpenSim model. Future work could add contact sensors to the robot feet and compare contact force profiles across similar walking or falling motions.
The most useful starting point for a future student is likely:
- Launch the current Gazebo/RViz simulation.
- Run the joint keyboard controller.
- Verify that the four leg joints respond to command topics.
- Review the Xacro/URDF/SDF files to understand the simplified robot model.
- Decide whether the next goal is:
- improving model fidelity,
- building a walking controller,
- validating against OpenSim,
- or integrating a high-level VLA interface.
Recommended immediate next step:
Build a simple stable closed-loop controller for the simplified biped before attempting OpenVLA integration.
The reason is that OpenVLA or another VLA-style model is unlikely to solve low-level biped stabilization directly without a carefully defined action space and training dataset. A stable controller should exist first, then higher-level visual/language commands can be layered on top.
Han Nguyen
UC Berkeley B.S. Mechanical Engineering, 2023
Johns Hopkins M.S. Robotics and Autonomous Systems, 2026