The group_sparsity regularizer of the Stacked2dCore the convolution layers are referred to as conv, but that does not generalize to other conv layer types like ds_conv for instance.
|
def group_sparsity(self): |
|
""" |
|
Sparsity regularization on the filters of all the conv2d layers except the first one. |
|
""" |
|
ret = 0 |
|
for feature in self.features[1:]: |
|
ret = ret + feature.conv.weight.pow(2).sum(3, keepdim=True).sum(2, keepdim=True).sqrt().mean() |
|
return ret / ((self.num_layers - 1) if self.num_layers > 1 else 1) |
|
|
|
def regularizer(self): |
|
return self.group_sparsity() * self.gamma_hidden + self.gamma_input * self.laplace() |
The
group_sparsityregularizer of theStacked2dCorethe convolution layers are referred to asconv, but that does not generalize to other conv layer types likeds_convfor instance.neuralpredictors/neuralpredictors/layers/cores/conv2d.py
Lines 247 to 257 in 7f1b303