Problems with predict for MLJ implementations of `PegasosClassifier` and `PerceptronClassifier`

[ablaom]

These models are not working in 0.12.4.

using MLJ
using MLJTestIntegration

A = @load PegasosClassifier pkg = BetaML

X, y = MLJTestIntegration.datasets(A)[2];

mach = machine(A(), X, y)
MLJ.fit!(mach, rows=1:4);
MLJ.predict(mach, X)

# ERROR: BoundsError: attempt to access 1-element Vector{Vector{Float64}} at index [2]
# Stacktrace:
#  [1] throw_boundserror(A::Vector{Vector{Float64}}, I::Tuple{Int64})
#    @ Base ./essentials.jl:14
#  [2] getindex
#    @ ./essentials.jl:916 [inlined]
#  [3] predict(x::Matrix{Float64}, θ::Vector{Vector{Float64}}, θ₀::Vector{Float64}, classes::Vector{String})
#    @ BetaML.Perceptron ~/.julia/packages/BetaML/esA5J/src/Perceptron/Perceptron_classic.jl:266
#  [4] predict(model::BetaML.Bmlj.PegasosClassifier, fitresult::Tuple{…}, Xnew::@NamedTuple{…}
# )
#    @ BetaML.Bmlj ~/.julia/packages/BetaML/esA5J/src/Bmlj/Perceptron_mlj.jl:262
#  [5] predict(mach::Machine{…}, Xraw::@NamedTuple{…})
#    @ MLJBase ~/.julia/packages/MLJBase/F8Zzu/src/operations.jl:135
#  [6] top-level scope
#    @ REPL[48]:1
# Some type information was truncated. Use `show(err)` to see complete types.


B = @load PerceptronClassifier pkg = BetaML

X, y = MLJTestIntegration.datasets(B)[2];

mach = machine(A(), X, y)
MLJ.fit!(mach, rows=1:4);
MLJ.predict(mach, X)

# ERROR: BoundsError: attempt to access 1-element Vector{Vector{Float64}} at index [2]
# Stacktrace:
#  [1] throw_boundserror(A::Vector{Vector{Float64}}, I::Tuple{Int64})
#    @ Base ./essentials.jl:14
#  [2] getindex
#    @ ./essentials.jl:916 [inlined]
#  [3] predict(x::Matrix{Float64}, θ::Vector{Vector{Float64}}, θ₀::Vector{Float64}, classes::Vector{String})
#    @ BetaML.Perceptron ~/.julia/packages/BetaML/esA5J/src/Perceptron/Perceptron_classic.jl:266
#  [4] predict(model::BetaML.Bmlj.PegasosClassifier, fitresult::Tuple{…}, Xnew::@NamedTuple{…}
# )
#    @ BetaML.Bmlj ~/.julia/packages/BetaML/esA5J/src/Bmlj/Perceptron_mlj.jl:262
#  [5] predict(mach::Machine{…}, Xraw::@NamedTuple{…})
#    @ MLJBase ~/.julia/packages/MLJBase/F8Zzu/src/operations.jl:135
#  [6] top-level scope
#    @ REPL[53]:1
# Some type information was truncated. Use `show(err)` to see complete types.

[sylvaticus]

Sorry, I'll work on this in the next few days. I also have to check why I don't catch these errors in my tests, perhaps I need to review how I test the MLJ interface.

[ablaom]

I'm guessing you are using MLJTestInterface.test. You could instead use the full integration tests we are doing in MLJ. Something like:

using MLJTestIntegration # instead of MLJTestInterface
import BetaML

M = BetaML.Bmlj.PerceptronClassifier

MLJTestIntegration.test(M; mod=@__MODULE__, level=4, throw=true, verbosity=2)

[sylvaticus]

I use MLJBase, and my tests are typically:

import MLJBase

data                       = MLJBase.@load_xxx
model                      = BetaML.Bmlj.MyModelX(...)
regressor_dtr              = MLJBase.machine(model, [data])
(fitresult, cache, report) = MLJBase.fit(model, 0, [data])
predictions               = MLJBase.predict(model, fitresult, X)
@test [appropriate metric]

I tought that using MLJTestIntegration/Interface I would have been forced to load the published verion of BetaML model instead of the one under local development, but it seems not.
I'll try them and check if the test itself is not taking too long...

[sylvaticus]

Solved and added an optional test for the complete suite of MLJ tests for BetaML models using MLJTestIntegration.test (Pkg.test("BetaML", test_args=["MLJ"]))

[ablaom]

I appreciate the addition of the integrations tests in this case, given the activity here around MLJ interfaces.