Skip to content

Commit 324e030

Browse files
committed
feat(models): add plain channel-independent MLP forecaster
MLP applies the same input_len->output_len network to every channel with no normalization, no residual block, and no linear shortcut. Configurable via --hidden_dim (128), --n_layers (2), --dropout (0). Added to docs/models.md (table row, cited Rumelhart et al. 1986 - the OG MLP); README count 77 -> 78.
1 parent 5386187 commit 324e030

3 files changed

Lines changed: 52 additions & 1 deletion

File tree

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ Get started with FedProC in just a few steps:
5050
- **[Usage](docs/usage.md)** - Basic usage and examples
5151
- **[Strategies](docs/strategies.md)** - Available federated learning strategies (55 strategies)
5252
- **[Datasets](docs/datasets.md)** - Supported datasets and data preparation (31 datasets)
53-
- **[Models](docs/models.md)** - Available models and architectures (77 models)
53+
- **[Models](docs/models.md)** - Available models and architectures (78 models)
5454
- **[Augmentations](docs/augs.md)** - Custom GPU-native PyTorch augmentations (8 augmentations)
5555
- **[Losses](docs/losses.md)** - Loss functions and metrics (19 losses)
5656
- **[Optimizers](docs/optimizers.md)** - Optimization methods (9 optimizers)

docs/models.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ Auxiliary (eg: static time-varying features, future time-varying features, etc)
99
| Sonnet | Wavelet + Koopman + Attention | Multivariate | AAAI (Oral) | 2026 | Sonnet: Spectral Operator Neural Network for Multivariable Time Series Forecasting | [Arxiv](https://arxiv.org/abs/2505.15312) - [GitHub](https://github.com/ClaudiaShu/Sonnet) |
1010
| SimTS | Causal CNN (contrastive pre-training) | Multivariate | ICASSP | 2024 | SimTS: Rethinking Contrastive Representation Learning for Time Series Forecasting | [Arxiv](https://arxiv.org/abs/2303.18205) - [IEEE](https://ieeexplore.ieee.org/document/10446875) - [GitHub](https://github.com/xingyu617/SimTS_Representation_Learning) |
1111
| Amplifier | MLP | Multivariate | AAAI | 2025 | Amplifier: Bringing Attention to Neglected Low-Energy Components in Time Series Forecasting | [Arxiv](https://arxiv.org/abs/2501.17216) - [REF](https://github.com/aikunyi/Amplifier/blob/main/models/Amplifier.py) |
12+
| MLP | MLP | Univariate | Nature | 1986 | Learning representations by back-propagating errors | [Nature](https://www.nature.com/articles/323533a0) |
1213
| Linear | MLP | Univariate | AAAI | 2023 | Are Transformers Effective for Time Series Forecasting? | [Arxiv](https://arxiv.org/abs/2205.13504) - [REF](https://github.com/cure-lab/LTSF-Linear/blob/main/models/Linear.py) |
1314
| LinearIC | MLP | Univariate | AAAI | 2023 | Are Transformers Effective for Time Series Forecasting? | [Arxiv](https://arxiv.org/abs/2205.13504) - [REF](https://github.com/cure-lab/LTSF-Linear/blob/main/models/Linear.py) |
1415
| NLinear | MLP | Univariate | AAAI | 2023 | Are Transformers Effective for Time Series Forecasting? | [Arxiv](https://arxiv.org/abs/2205.13504) - [REF](https://github.com/cure-lab/LTSF-Linear/blob/main/models/NLinear.py) |

models/MLP.py

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
import torch.nn as nn
2+
3+
4+
class MLP(nn.Module):
5+
"""Plain channel-independent MLP forecaster.
6+
7+
Minimal over-parameterizable nonlinear baseline. Unlike TiDE there is no
8+
normalization, no residual block, and no linear shortcut, so the
9+
memorize -> plateau -> generalize (grokking) dynamics are not damped by an
10+
auxiliary linear path. The same MLP is applied to every channel
11+
(input_len -> output_len), keeping the parameter count small and the
12+
capacity/data ratio easy to control.
13+
"""
14+
15+
optional = {
16+
"hidden_dim": 128,
17+
"n_layers": 2,
18+
"dropout": 0.0,
19+
}
20+
21+
@classmethod
22+
def args_update(cls, parser):
23+
parser.add_argument("--hidden_dim", type=int, default=None)
24+
parser.add_argument("--n_layers", type=int, default=None)
25+
parser.add_argument("--dropout", type=float, default=None)
26+
27+
def __init__(self, configs):
28+
super().__init__()
29+
in_dim = configs.input_len
30+
out_dim = configs.output_len
31+
h = configs.hidden_dim
32+
n = configs.n_layers
33+
dropout = configs.dropout
34+
35+
dims = [in_dim] + [h] * n
36+
layers = []
37+
for a, b in zip(dims[:-1], dims[1:]):
38+
layers.append(nn.Linear(a, b))
39+
layers.append(nn.ReLU())
40+
if dropout and dropout > 0:
41+
layers.append(nn.Dropout(dropout))
42+
layers.append(nn.Linear(dims[-1], out_dim))
43+
self.net = nn.Sequential(*layers)
44+
45+
def forward(self, x, **kwargs):
46+
# x: [batch, input_len, channels]
47+
x = x.permute(0, 2, 1) # [batch, channels, input_len]
48+
x = self.net(x) # [batch, channels, output_len]
49+
x = x.permute(0, 2, 1) # [batch, output_len, channels]
50+
return x

0 commit comments

Comments
 (0)