Improve tutorial + documentation

docs/src/tutorial.md

@@ -10,6 +10,7 @@ as a sum of element functions $f_i$.
 PartiallySeparableNLPModels.jl relies on [ExpressionTreeForge.jl](https://github.com/JuliaSmoothOptimizers/ExpressionTreeForge.jl) to detect the partially-separable structure. Then, it defines suitable partitioned structures using [PartitionedStructures.jl](https://github.com/JuliaSmoothOptimizers/PartitionedStructures.jl) and [PartitionedVectors.jl](https://github.com/paraynaud/PartitionedVectors.jl).
 Any `NLPModels` from PartiallySeparableNLPModels.jl rely on `PartitionedVector <: AbstractVector` instead of `Vector`.
 Any model from PartiallySeparableNLPModels.jl may be defined either from a `ADNLPModel` or a `MathOptNLPModel`.
+
 Let starts with an example using an `ADNLPModel` (`MathOptNLPModel` will follow):
 ```@example PSNLP
 using PartiallySeparableNLPModels, ADNLPModels
@@ -44,20 +45,18 @@ fx = NLPModels.obj(psnlp, x) # compute the objective function
 
 ```@example PSNLP
 gx = NLPModels.grad(psnlp, x) # compute the gradient
+Vector(gx) # compute and return the value of the vector that gx::PartitionedVector represents
 ```
 
 ```@example PSNLP
 v = similar(x)
-v .= 1
+v .= 2
 hv = NLPModels.hprod(psnlp, x, v)
-```
-`fx`, `gx` and `hv` accumulate contributions from element functions, either its evaluation $f_i(U_ix)$, its gradient $\nabla f_i(U_ix)$ or its element Hessian-vector $\nabla^2 f_i(U_i x) U_i v$.
-You can get the `Vector` value of `gx` and `hv` with 
-```@example PSNLP
 Vector(hv)
-Vector(gx)
 ```
-and you can find more detail about `PartitionedVector`s in [PartitionedVectors.jl tutorial](https://paraynaud.github.io/PartitionedVectors.jl/stable/).
+`fx`, `gx` and `hv` accumulate contributions from element functions, either its evaluation $f_i(U_ix)$, its gradient $\nabla f_i(U_ix)$ or its element Hessian-vector $\nabla^2 f_i(U_i x) U_i v$.
+You can get the `Vector` value of `gx` and `hv` with `Vector(hv), Vector(gx)`.
+You can find more detail about `PartitionedVector`s in [PartitionedVectors.jl tutorial](https://paraynaud.github.io/PartitionedVectors.jl/stable/).
 
 The same procedure can be applied to `MathOptNLPModel`s:
 ```@example PSNLP
@@ -81,6 +80,7 @@ psnlp = PSNLPModel(jumpnlp)
 
 fx = NLPModels.obj(psnlp, x) # compute the objective function
 gx = NLPModels.grad(psnlp, x) # compute the gradient
+Vector(gx)
 ```
 
 ## Partitioned quasi-Newton `NLPModel`s
@@ -158,10 +158,10 @@ using JSOSolvers
 
 trunk_solver = TrunkSolver(pbfgsnlp)
 ```
-which define properly the PartitionedVectors mandatory for running `trunk`
-`turnk_solver` can be `solve` afterward with:
+which define properly the PartitionedVectors mandatory for running `trunk`.
+`trunk_solver` can be `solve` afterward with:
 ```@example PSNLP
 solve!(trunk_solver, pbfgsnlp)
 ```
 
-For now, `TrunkSolver` is the sole `Solver` defined for `PartiallySeparableNLPModel`s, if you want to add another `Solver`, you should define it similarly to `TrunkSolver` in `src/trunk.jl`.
+For now, `TrunkSolver` is the sole `Solver` from `Krylov.jl` that support the `PartiallySeparableNLPModel`s, if you want to add another Krylov `Solver`, you should proceed similarly to `TrunkSolver` defined in `src/trunk.jl`.

src/AbstractPNLPModels.jl

@@ -13,20 +13,24 @@ export update_nlp, hess_approx, update_nlp!
 abstract type AbstractPartiallySeparableNLPModel{T, S} <: AbstractNLPModel{T, S} end
 abstract type AbstractPQNNLPModel{T, S} <: QuasiNewtonModel{T, S} end
 
-""" Accumulate the supported NLPModels. """
+""" 
+    SupportedNLPModel = Union{ADNLPModel, MathOptNLPModel}
+
+Union Type accumulating the NLPModels supported.
+"""
 SupportedNLPModel = Union{ADNLPModel, MathOptNLPModel}
 
 """
     ElementFunction
 
-A type that gathers the information indentifying an element function in a `PartiallySeparableNLPModel`, and its properties.
+Type gathering the information to indentify an element function in a `PartiallySeparableNLPModel`; also inform some of its properties.
 `ElementFunction` has fields:
 
 * `i`: the index of the element function;
-* `index_element_tree`: the index occupied in the element-function vector after the deletion of redundant element functions;
+* `index_element_tree`: the effective index in the vector gathering the element-function expression trees after the deletion of redundant element functions;
 * `variable_indices`: list of elemental variables of `ElementFunction`;
-* `type`: `constant`, `linear`, `quadratic`, `cubic` or `general`;
-* `convexity_status`: `constant`, `linear`, `convex`, `concave` or `unknown`.
+* `type`: either `constant`, `linear`, `quadratic`, `cubic` or `general`;
+* `convexity_status`: either `constant`, `linear`, `convex`, `concave` or `unknown`.
 """
 mutable struct ElementFunction
   i::Int # the index of the function 1 ≤ i ≤ N

src/PQNNLPmethods.jl

@@ -131,7 +131,7 @@ function NLPModels.reset_data!(pqnnlp::AbstractPQNNLPModel; name = pqnnlp.name,
   (name == :psr1) && (op = epm_from_epv(epv))
   (name == :pse) && (op = epm_from_epv(epv))
   (name == :plbfgs) && (op = eplo_lbfgs_from_epv(epv; kwargs...))
-  (name == :plsr1) && (op = eplo_lsr1_from_epv(epv))
+  (name == :plsr1) && (op = eplo_lsr1_from_epv(epv; kwargs...))
   (name == :plse) && (op = eplo_lose_from_epv(epv; kwargs...))
   if name == :pcs
     convex_vector = map(eem -> PartitionedStructures.get_convex(eem), pqnnlp.op.eem_set)

src/backends/GeneralBackends/ElementNonlinearModel.jl

@@ -19,7 +19,7 @@ end
 Return an `ElementMOIModelBackend` from a `Vector` of expression trees
 (supported by [ExpressionTreeForge.jl](https://github.com/JuliaSmoothOptimizers/ExpressionTreeForge.jl))
 of size `length(vec_elt_expr_tree)=M` and `index_element_tree` which redirects each element function `i`
- to its corresponding expression tree (1 ≤ `index_element_tree[i]` ≤ M, 1 ≤ i ≤ N).
+ to its corresponding expression tree (1 ≤ `index_element_tree[i]` ≤ M, ∀ 1 ≤ i ≤ N).
 """
 function ElementMOIModelBackend(
   vec_elt_expr_tree::Vector,

src/backends/GeneralBackends/ModifiedNonlinearBackends.jl

@@ -5,9 +5,10 @@ export set_vector_from_pv!, set_pv_from_vector!
     ModifiedObjectiveMOIModelBackend{T}
 
 Composed of:
-- `vec_element_evaluators::Vector{MOI.Nonlinear.Evaluator{MOI.Nonlinear.ReverseAD.NLPEvaluator}}`, M distinct element function `MOI.Nonlinear.Model`;
-- `index_element_tree::Vector{Int}`, from which any of the N element function may associate a gradient tape from `vec_element_gradient_tapes`.
-Each `MOI.Nonlinear.Evaluator{MOI.Nonlinear.ReverseAD.NLPEvaluator}` accumulates the element-function's contribution in a element-vector of a `PartitionedVector`.
+- `evaluator`: the `Evaluator` of a modified partially-separable objective function : f(x) = ∑ᵢ fᵢ(x), where x ∈ ℜⁿ where n = ∑ᵢ nᵢ;
+- `x_modified`: a vector x ∈ ℜⁿ where n = ∑ᵢ nᵢ;
+- `v_modified`: a vector v ∈ ℜⁿ where n = ∑ᵢ nᵢ;
+Each partial derivative of f corresponds to a partial derivative of an element function fᵢ.
 """
 mutable struct ModifiedObjectiveMOIModelBackend{T} <: PartitionedBackend{Float64}
   evaluator::MOI.Nonlinear.Evaluator{MOI.Nonlinear.ReverseAD.NLPEvaluator}

src/backends/GeneralBackends/SparseJacobianBackend.jl

@@ -4,10 +4,11 @@ export set_pv_from_sparse_jacobian!
 """
     SparseJacobianMoiModelBackend{T}
 
+Compute the partitioned derivatives from the sparse Jacobian of the constraints.
 Composed of:
-- `evaluator::MOI.Nonlinear.Evaluator{MOI.Nonlinear.ReverseAD.NLPEvaluator}`, an evaluator of a `MOI.Nonlinear.Model` constrained by every element functions;
-- `sparse_jacobian::Vector{T}`, mandatory to store in place the sparse Jacobianwith MathOptInterface;
-- `translated_x::PartitionedVector{T}`, to handle variables translations, especially when the model doesn't depend on every variable.
+- `evaluator::MOI.Nonlinear.Evaluator{MOI.Nonlinear.ReverseAD.NLPEvaluator}`: an evaluator of a `MOI.Nonlinear.Model` with a partially-separable objective and `N` constraits (=0) each parametrized by an element functions;
+- `sparse_jacobian::Vector{T}`: mandatory to store in place the sparse Jacobianwith MathOptInterface;
+- `translated_x::PartitionedVector{T}`: handle the partial derivative translations, usefull when the model doesn't depend on every variable.
 """
 mutable struct SparseJacobianMoiModelBackend{T} <: PartitionedBackend{Float64}
   evaluator::MOI.Nonlinear.Evaluator{MOI.Nonlinear.ReverseAD.NLPEvaluator}

src/backends/GradientBackends/ElementReverseDiffGradient.jl

@@ -5,7 +5,7 @@ _ni(element_variables::Vector{Int}) = isempty(element_variables) ? 0 : maximum(e
 """
     element_gradient_tape = compiled_grad_element_function(element_function::T; ni::Int = length(ExpressionTreeForge.get_elemental_variables(element_function)), type = Float64) where {T}
 
-Return the `elment_gradient_tape::GradientTape` which speed up the gradient computation of `element_function` with `ReverseDiff`.
+Return the `element_gradient_tape::GradientTape` to compute the gradient computation of an `element_function` with `ReverseDiff`.
 """
 function compiled_grad_element_function(
   element_function::G;
@@ -23,8 +23,8 @@ end
     ElementReverseDiffGradient{T}
 
 Composed of:
-- `vec_element_gradient_tapes::Vector{ReverseDiff.CompiledTape}`, M distinct element function tapes;
-- `index_element_tree::Vector{Int}`, from which any of the N element function may associate a gradient tape from `vec_element_gradient_tapes`.
+- `vec_element_gradient_tapes::Vector{ReverseDiff.CompiledTape}`: M distinct element function tapes;
+- `index_element_tree::Vector{Int}`: from which any of the N element function may associate a gradient tape from `vec_element_gradient_tapes`.
 Each `ReverseDiff.CompiledTape` accumulates the element-function's contribution in a element-vector of a `PartitionedVector`.
 """
 mutable struct ElementReverseDiffGradient{T} <: AbstractGradientBackend{T}
@@ -38,7 +38,7 @@ end
 Return an `ElementReverseDiffGradient` from a `Vector` of expression trees
 (supported by [ExpressionTreeForge.jl](https://github.com/JuliaSmoothOptimizers/ExpressionTreeForge.jl))
 of size `length(vec_elt_expr_tree)=M` and `index_element_tree` which redirects each element function `i`
- to its corresponding expression tree (1 ≤ `index_element_tree[i]` ≤ M, 1 ≤ i ≤ N).
+ to its corresponding expression tree (1 ≤ `index_element_tree[i]` ≤ M, ∀ 1 ≤ i ≤ N).
 """
 function ElementReverseDiffGradient(
   vec_elt_expr_tree::Vector,

src/backends/HprodBackends/ElementReverseForwardHprod.jl

@@ -5,8 +5,8 @@ export ElementReverseForwardHprod
 
 Compute the partitioned Hessian-product with `partitioned_hessian_prod!` by applying successively ReverseDiff and ForwardDiff.
 It is composed of:
-- `vec_elt_complete_expr_tree::Vector{G}`, the expression trees of the distinct element functions (of size M);
-- `index_element_tree::Vector{Int}`, reffering to index in `vec_elt_complete_expr_tree` of any of the N element function;
+- `vec_elt_complete_expr_tree::Vector{G}`: the expression trees of the distinct element functions (of size M);
+- `index_element_tree::Vector{Int}`: store for each one of the N element functions its corresponding index in `vec_elt_complete_expr_tree`;
 """
 mutable struct ElementReverseForwardHprod{T, G} <: AbstractHprodBackend{T}
   vec_elt_complete_expr_tree::Vector{G} # of size M
@@ -17,9 +17,9 @@ end
     hprod_backend = ElementReverseForwardHprod(complete_trees::Vector, index_element_tree::Vector{Int}; type=Float64)
 
 Return an `ElementReverseForwardHprod` from a `Vector` of expression trees
-(supported by [ExpressionTreeForge.jl](https://github.com/JuliaSmoothOptimizers/ExpressionTreeForge.jl))
-`vec_elt_expr_tree` of size `length(vec_elt_expr_tree)=M` and `index_element_tree` which redirects each element function `i`
- to its corresponding expression tree (1 ≤ `index_element_tree[i]` ≤ M, 1 ≤ i ≤ N).
+(supported by [ExpressionTreeForge.jl](https://github.com/JuliaSmoothOptimizers/ExpressionTreeForge.jl)).
+`vec_elt_expr_tree` is of size `M` and `index_element_tree` redirects any element function `i`
+ to its corresponding expression tree (1 ≤ `index_element_tree[i]` ≤ M, ∀ 1 ≤ i ≤ N).
 """
 function ElementReverseForwardHprod(
   vec_elt_expr_tree::Vector{G},

src/backends/ObjectiveBackends/MOIObjectiveBackend.jl

@@ -3,8 +3,8 @@ export MOIObjectiveBackend
 """
     MOIObjectiveBackend{T, Model}
 
-Composed of `nlp::Model`, it evaluates the objective function from `NLPModels.obj(nlp, x::AbstractVector{T})`.
-The user has to make sure `nlp` is can evaluate `x::AbstractVector{T}` with a suitable type `T`.
+Evaluate the objective function from `NLPModels.obj(nlp, x::AbstractVector{T})`.
+The user has to make sure `nlp` can evaluate `x::AbstractVector{T}` with a suitable type `T`.
 """
 mutable struct MOIObjectiveBackend{T} <: AbstractObjectiveBackend{T}
   evaluator::MathOptInterface.Nonlinear.Evaluator{MathOptInterface.Nonlinear.ReverseAD.NLPEvaluator}

src/backends/ObjectiveBackends/NLPObjectiveBackend.jl

@@ -4,7 +4,7 @@ export NLPObjectiveBackend
     NLPObjectiveBackend{T, Model}
 
 Composed of `nlp::Model`, it evaluates the objective function from `NLPModels.obj(nlp, x::AbstractVector{T})`.
-The user has to make sure `nlp` is can evaluate `x::AbstractVector{T}` with a suitable type `T`.
+The user has to make sure `nlp` can evaluate `x::AbstractVector{T}` with a suitable type `T`.
 """
 mutable struct NLPObjectiveBackend{T, Model} <: AbstractObjectiveBackend{T}
   nlp::Model # ADNLPModel or MathOptNLPModel

src/backends/PartitionedBackends.jl

@@ -29,7 +29,7 @@ objective(backend::PartitionedBackend{T}, x::AbstractVector{T}) where {T} =
     partitioned_gradient!(backend::AbstractObjectiveBackend{T}, x::AbstractVector{T}, g::AbstractVector{T})
 
 Compute the partitioned gradient from `backend` at the point `x` in place of `g`.
-This method is designed for `PartitionedVector{T}<:AbstractVector{T}` (for now, both `x` and `g`).
+This method support only `x` and `g` as `PartitionedVector{T}<:AbstractVector{T}`.
 """
 partitioned_gradient!(
   backend::PartitionedBackend{T},
@@ -41,7 +41,7 @@ partitioned_gradient!(
     partitioned_hessian_prod!(backend::AbstractHprodBackend{T}, x::AbstractVector{T}, v::AbstractVector{T}, Hv::AbstractVector{T})
 
 Compute the partitioned Hessian-vector product ∇² f(x) v from `backend` in place of `Hv`.
-This method is designed for `PartitionedVector{T}<:AbstractVector{T}` (`x`, `v` and `Hv`).
+This method support only `x, v` and `Hv` as `PartitionedVector{T}<:AbstractVector{T}`.
 """
 partitioned_hessian_prod!(
   backend::PartitionedBackend{T},

src/partitionedNLPModels/PBFGSNLPModels.jl

@@ -20,14 +20,14 @@ Deduct and allocate the partitioned structures of a NLPModel using partitioned B
 * `counters`: count how many standards methods of `NLPModels` are called;
 * `n`: the size of the problem;
 * `N`: the number of element functions;
-* `vec_elt_fun`: a `ElementFunction` vector, of size `N`;
+* `vec_elt_fun`: a vector of `ElementFunction`, of size `N`;
 * `M`: the number of distinct element-function expression trees;
-* `vec_elt_complete_expr_tree`: a `Complete_expr_tree` vector, of size `M`;
+* `vec_elt_complete_expr_tree`: a vector of `Complete_expr_tree`, of size `M`;
 * `element_expr_tree_table`: a vector of size `M`, the i-th element `element_expr_tree_table[i]::Vector{Int}` informs which element functions use the `vec_elt_complete_expr_tree[i]` expression tree;
 * `index_element_tree`: a vector of size `N` where each component indicates which `Complete_expr_tree` from `vec_elt_complete_expr_tree` is used for the corresponding element;
-* `vec_compiled_element_gradients`: the vector gathering the compiled tapes for every element gradient evaluations;
+* `vec_compiled_element_gradients`: a vector of compiled tapes needed for every element gradient evaluation;
 * `op`: the partitioned matrix (main memory cost);
-* `name`: the name of partitioned quasi-Newton update performed
+* `name`: the name of the partitioned quasi-Newton update performed
 """
 mutable struct PBFGSNLPModel{
   G,

src/partitionedNLPModels/PCSNLPModels.jl

@@ -20,14 +20,14 @@ Deduct and allocate the partitioned structures of a NLPModel using a PCS Hessian
 * `counters`: count how many standards methods of `NLPModels` are called;
 * `n`: the size of the problem;
 * `N`: the number of element functions;
-* `vec_elt_fun`: a `ElementFunction` vector, of size `N`;
+* `vec_elt_fun`: a vector of `ElementFunction`, of size `N`;
 * `M`: the number of distinct element-function expression trees;
-* `vec_elt_complete_expr_tree`: a `Complete_expr_tree` vector, of size `M`;
+* `vec_elt_complete_expr_tree`: a vector of `Complete_expr_tree`, of size `M`;
 * `element_expr_tree_table`: a vector of size `M`, the i-th element `element_expr_tree_table[i]::Vector{Int}` informs which element functions use the `vec_elt_complete_expr_tree[i]` expression tree;
 * `index_element_tree`: a vector of size `N` where each component indicates which `Complete_expr_tree` from `vec_elt_complete_expr_tree` is used for the corresponding element;
-* `vec_compiled_element_gradients`: the vector gathering the compiled tapes for every element gradient evaluations;
+* `vec_compiled_element_gradients`: a vector of compiled tapes needed for every element gradient evaluation;
 * `op`: the partitioned matrix (main memory cost);
-* `name`: the name of partitioned quasi-Newton update performed
+* `name`: the name of the partitioned quasi-Newton update performed
 """
 mutable struct PCSNLPModel{
   G,

src/partitionedNLPModels/PLBFGSNLPModels.jl

@@ -20,14 +20,14 @@ Deduct and allocate the partitioned structures of a NLPModel using a PLBFGS Hess
 * `counters`: count how many standards methods of `NLPModels` are called;
 * `n`: the size of the problem;
 * `N`: the number of element functions;
-* `vec_elt_fun`: a `ElementFunction` vector, of size `N`;
+* `vec_elt_fun`: a vector of `ElementFunction`, of size `N`;
 * `M`: the number of distinct element-function expression trees;
-* `vec_elt_complete_expr_tree`: a `Complete_expr_tree` vector, of size `M`;
+* `vec_elt_complete_expr_tree`: a vector of `Complete_expr_tree`, of size `M`;
 * `element_expr_tree_table`: a vector of size `M`, the i-th element `element_expr_tree_table[i]::Vector{Int}` informs which element functions use the `vec_elt_complete_expr_tree[i]` expression tree;
 * `index_element_tree`: a vector of size `N` where each component indicates which `Complete_expr_tree` from `vec_elt_complete_expr_tree` is used for the corresponding element;
-* `vec_compiled_element_gradients`: the vector gathering the compiled tapes for every element gradient evaluations;
+* `vec_compiled_element_gradients`: a vector of compiled tapes needed for every element gradient evaluation;
 * `op`: the partitioned matrix (main memory cost);
-* `name`: the name of partitioned quasi-Newton update performed
+* `name`: the name of the partitioned quasi-Newton update performed
 """
 mutable struct PLBFGSNLPModel{
   G,

src/partitionedNLPModels/PLSENLPModels.jl

@@ -20,14 +20,14 @@ Deduct and allocate the partitioned structures of a NLPModel using a PLSE Hessia
 * `counters`: count how many standards methods of `NLPModels` are called;
 * `n`: the size of the problem;
 * `N`: the number of element functions;
-* `vec_elt_fun`: a `ElementFunction` vector, of size `N`;
+* `vec_elt_fun`: a vector of `ElementFunction`, of size `N`;
 * `M`: the number of distinct element-function expression trees;
-* `vec_elt_complete_expr_tree`: a `Complete_expr_tree` vector, of size `M`;
+* `vec_elt_complete_expr_tree`: a vector of `Complete_expr_tree`, of size `M`;
 * `element_expr_tree_table`: a vector of size `M`, the i-th element `element_expr_tree_table[i]::Vector{Int}` informs which element functions use the `vec_elt_complete_expr_tree[i]` expression tree;
 * `index_element_tree`: a vector of size `N` where each component indicates which `Complete_expr_tree` from `vec_elt_complete_expr_tree` is used for the corresponding element;
-* `vec_compiled_element_gradients`: the vector gathering the compiled tapes for every element gradient evaluations;
+* `vec_compiled_element_gradients`: a vector of compiled tapes needed for every element gradient evaluation;
 * `op`: the partitioned matrix (main memory cost);
-* `name`: the name of partitioned quasi-Newton update performed
+* `name`: the name of the partitioned quasi-Newton update performed
 """
 mutable struct PLSENLPModel{
   G,