ORC-Assignment-3

Assignment 3 of the Course Optimization-Based Robot Control held by Andrea Del Prete during Academic Year 2022-2023 (Department of Industrial Engineering (DII)).

Introduction

The goal of this assignment is to implement the Deep Q-Learning algorithm with a simple continuous time (state-space) 1-pendulumn environment, using discrete control input space. The environment used for implementing this final project made use of Keras which is a framework built on top of TensorFlow.

Description of files and folders contained in the repository

Folders

1. saved_models : Folder containing the model's weights got after the training of the NN (to save best performance of the algorithm)
2. Papers : Folder containing the text, questions of the assignment and the scientific papers used for both the DQN algorithm implementation and for writing down the final report.
3. Tests_plots : Folder containing the different tests/simulations both for the single and double pendulum in order to try to get the optimal hyper-parameters for stabilizing the pendulum around the top position.
4. Videos : Folder containing the videos of both Single and Double Pendulum resulting in the best performance we could have been achieved.
5. Prova : Folder containing some plots (only for plotting purposes during perfomed tests )
6. src : Folder containing the source code developed during the development of the DQN algorithm

Files (source code)

1. display.py : Connects to gepetto-viewer or webbrowser
2. pendulum.py : Creates a continuous state simulation environment for a N-pendulum
3. dpendulum.py : Describes a continuous state pendulum environment with discrete control input
4. dqn.py : Creates a Deep-Q-Network learning agent
5. main_dqn.py: main file for the dqn algorithm applied to the environment pendulum
6. network_utils.py: Contains functions for creating the NN
7. plot_utils.py: Contains functions for plotting, mainly V table, policy table and the trajectories of state and control