The project implements an Automatic Differentiation library in OCaml.
Automatic differentiation is an algorithm used to evaluate partial derivatives of functions. Automatic differentiation is more efficient than symbolic differentiation, and more accurate than numeric differentiation. Our library can be used to define functions using our Variable and Tensor modules. One can then find gradients and various partial derivatives of these functions.
Examples are listed below. We also use the Bigarray library to build our tensors, so that needs to be installed.
Clone the repository and build the project.
git clone https://github.com/mxberner/Autodiff-Ocaml
cd Autodiff-Ocaml
dune buildNext there are a bunch of examples that make it easy to see the library work in practice.
dune exec <example_name>- basic - Runs some basic operations using tensor module.
- simple - Run autodifferentiation on a simple function.
- nn - Train a single layer neural network on some random data.
- visualize - Visualizes the dynamically built computation graph.
- printed - Demonstrates the underlying Hashtable.
When building a function representation, the function is split into primitives defined by Variable module.
This results in a directed acyclic graph (DAG) where each node contains an id, the actual Tensor.t value and its calculated gradient based on function primitive type. Additionally there is an operation tag that specifies if the graph entry is a function of two variables, or one variable. This is mostly useful for demo purposes and looking into the computational graph.
This graph is implicitly formed through the use of a hashtable of type v in the Variable module. Both the Tensor and Variable modules have accompanying tests. The oplot and graph libraries that we downloaded weren't ultimately of use, so they just remain with their demos.
- First order derivative
- Vectorization
- N-th order derivative
- Example
- Basic
- Simple Neural Network
- Visualizing autodiff computation graph
- Demonstration of Hashtable