MonarchGRU is a compact recurrent neural network for always-on keyword spotting on IoT devices. Its largest model (h = 64) has 29K parameters and reaches 97.9% test accuracy on the 12-class Google Speech Commands v2. It replaces most GRU weight matrices with Monarch matrices, which with 4 blocks need half the weights and multiply-accumulates (MACs) of a dense matrix.
AICAS'26 | DOI: coming soon | Citation
MonarchGRU: A Low-Footprint Recurrent Neural Network for Audio Keyword Spotting System
2026 IEEE 8th International Conference on Artificial Intelligence Circuits and Systems (AICAS 2026), pp. 310–314
Soujanya Bhowmick1, Sangyeob Kim2, Qinyu Chen3, Chang Gao4, Kwantae Kim1,*
1Aalto University, Finland 2Yonsei University, South Korea
3Leiden University, The Netherlands 4TU Delft, The Netherlands
*Corresponding author
This work was recognized with a 2026 IEEE CASS Student Travel Grant (Flagship Conferences) for AICAS 2026.
Keyword spotting with MonarchGRU. Fig. 1 of the paper, © 2026 IEEE.
This repo contains the PyTorch code to train and test MonarchGRU on the 12-class Google Speech Commands v2 task (10 keywords, plus unknown and silence), and a trained checkpoint of the model with hidden size h = 64 and b = 4 Monarch blocks. Each one-second clip becomes 16 log-Mel coefficients per 10 ms frame, plus their first and second time derivatives: 48 features × 101 frames. Please see our paper (AICAS 2026 proceedings, pp. 310–314) for the full results and the comparison with other keyword-spotting models.
Instructions: Setup | Data | Training | Pretrained model
- Sep 2026: MonarchGRU appeared at AICAS 2026 in an oral session (Ha Long Bay, Vietnam, 16–18 September 2026). The paper is in the conference proceedings, pp. 310–314.
An always-on keyword spotter keeps its weights in on-chip SRAM, and in earlier keyword-spotting chips the weight SRAM takes 41–56% of the accelerator power. A GRU layer with input size d and hidden size h has 3h(d + h) weights, so its memory and MACs grow with h².
MonarchGRU replaces each h × h matrix with a Monarch matrix M = P L Pᵀ R. L and R are block-diagonal with b blocks, and P is a fixed stride permutation, which is only wiring and has no weights. This needs 2h²/b weights and MACs instead of h², so half at b = 4. In layer 1 the input matrices (h × 48) stay dense and the hidden matrices use Monarch. In layer 2 both are Monarch.
Each gate also gets dual-path normalization: batch normalization on its input projection and layer normalization on its recurrent projection. In the paper, this raises the test accuracy of the b = 4 model at h = 64 from 96.95% to 97.94% (Table II).
Vanilla GRU cell (left) and MonarchGRU cell with dual-path normalization (right). Fig. 3 of the paper, © 2026 IEEE.
Test accuracy on the 12-class Google Speech Commands v2 test set, as reported in the paper (Tables I and III; two layers, b = 4, dual-path normalization):
| Hidden size h | Parameters | MACs per clip | Accuracy (%) |
|---|---|---|---|
| 16 | 3.9 K | 349 K | 94.36 |
| 32 | 10.0 K | 0.93 M | 96.87 |
| 48† | 15.0 K | 1.4 M | 97.32 |
| 64 | 29.2 K | 2.8 M | 97.94 |
† At h = 48 the layer-1 input matrices are square (48 × 48), so they use Monarch as well
(in this code: --hidden_size 48 --no-dense_input_layer1, 14,988 parameters).
The models in the table were trained for 500 epochs. The training command below is for h = 64, b = 4;
for the other rows, change --hidden_size (for h = 48, also replace --dense_input_layer1 with
--no-dense_input_layer1; see †). Accuracy changes a little from seed to seed.
-
Clone the repo and create the conda environment:
git clone https://github.com/TSircGroup/monarchgru.git cd monarchgru conda env create -f env.yml conda activate monarch-gru-envThe environment is for Linux with an NVIDIA GPU whose driver supports CUDA 12.8 or newer. On a machine without such a driver (for example a cluster login node), create it with
CONDA_OVERRIDE_CUDA=12.9 conda env create -f env.yml.env.ymllists the main packages at the versions we use: Python 3.12, PyTorch 2.13, torchaudio 2.11 and CUDA 12.9, on V100 GPUs. -
Check the install. This builds the model and prints its size (29,196 parameters); no data is needed:
python main.py --check_model
It reads no audio, so it does not check torchaudio and torchcodec (training and
inference.pyuse them). Run this and all later commands from the repository folder:main.pyreads./dataand writes./Resultsin the current folder.
Add --download to your first training command. It downloads Speech Commands v2 and its official test
set into ./data and builds the 12-class split (about 12 GB of disk space in total):
data/
speech_commands_v0.02/
train_12class/
valid_12class/
_background_noise_/
speech_commands_v0.02_split/
speech_commands_test_set_v0.02/
The download runs at the start of the first training run, so the machine it runs on (for a Slurm job, the
compute node) needs internet access. Start other training runs only after that run has printed
Download and 12-class split finished.; until then the 12-class split is incomplete.
If the compute nodes have no internet access, start the training command once with --download on a
machine that has it (for example a login node). It downloads the data and then starts training, so stop
it with Ctrl+C after it has printed Download and 12-class split finished., and then train without
--download.
Use --download only once: each time, it deletes data/speech_commands_v0.02 and
data/speech_commands_test_set_v0.02 and downloads them again. If you already have the data in this
layout, link it instead: ln -s /path/to/data data (--download refuses to run when ./data is a link).
To only measure the pretrained model's test accuracy, the test set alone is enough; see Pretrained model.
-
Train the h = 64, b = 4 model with the settings in
submit_mn4.sh:export CUBLAS_WORKSPACE_CONFIG=:4096:8 python -u main.py \ --hidden_size 64 --nblocks 4 --num_layers 2 --layer_dropout 0.0 \ --epochs 1000 --batch_size 128 --lr 0.005 --weight_decay 0.03 --grad_clip 4.0 \ --warmup_epochs 20 --label_smoothing 0.05 --lr_schedule cos --optimizer adamw \ --n_mels 16 --use_deltas --specaug --freq_mask 7 --time_mask 20 \ --use_swa --swa_start_frac 0.98 --valid_silence_noise --resample_unknown \ --norm dual --dense_input_layer1 --seed 456 --gpu 0 --deterministic \ --run_name my_runReplace
my_runwith a new name for every run. One epoch takes about 2 to 4 minutes on a V100, so the 1000 epochs take about 1.5 to 3 days; the run needs up to about 35 GB of host memory. On a Slurm cluster,sbatch submit_mn4.shruns the same command with the Slurm job ID as the run name; if the compute nodes have internet access, add--downloadto it for the first job only (see Data before you submit more jobs), and adjust its partition and module lines for your cluster. -
The results are saved in
Results/my_run/:gru.pt: weights of the epoch with the best validation accuracy.gru_last.pt: weights after the last epoch.gru_selected.pt: the checkpoint used for the test; with--use_swait is the average of the weights of the last 20 epochs.selection.json: how it was chosen, and all run settings.test_evaluation.json: test accuracy, confusion matrix and per-clip predictions.final_metrics.json: summary of the validation and test accuracy.validation_predictions.json: the chosen checkpoint's predictions on the validation set.*.pkl: per-epoch curves (training loss, validation accuracy, learning rate, gradient norm, fraction of clipped steps) and the final test accuracy.
The test set is scored once, after training, with
gru_selected.pt. To compare settings, train a few seeds (--seed 456,--seed 457, ...). All options:python main.py --help.
checkpoints/gru.pt is a trained h = 64, b = 4 model (29,196 parameters). Its accuracy is 97.955% on the
official test set (4790 of 4890 clips); we picked it from our runs with the test results in view. The model
runs on the GPU, or on the CPU if there is no GPU.
-
Predict the keyword in your own clips (mono 16 kHz WAV files of at most one second):
python inference.py clip.wav more.wav
After a short model summary, it prints one line per clip with the keyword and its probability, for example
clip.wav: yes (94.2%). The classes areyes no up down left right on off stop go,_unknown_and_silence_. -
Measure the accuracy on the official test set. This needs only the test set (113 MB), not the full dataset; skip the download steps if you already have the test set in
./data(see Data):mkdir -p data/speech_commands_test_set_v0.02 cd data/speech_commands_test_set_v0.02 curl -LO https://storage.googleapis.com/download.tensorflow.org/data/speech_commands_test_set_v0.02.tar.gz tar -xzf speech_commands_test_set_v0.02.tar.gz cd ../.. python inference.py --eval_test
It prints
Test accuracy: 97.955% (4790/4890).submit_inference.shruns the same evaluation as a Slurm job, on the CPU (its comments say how to use a GPU); as withsubmit_mn4.sh, adjust its partition and module lines for your cluster.
| File | Contents |
|---|---|
main.py |
Training, validation and the final test |
inference.py |
Keyword prediction and test accuracy with checkpoints/gru.pt |
monarch_gru.py |
The MonarchGRU cell, layers and classifier |
monarch_linear.py, structured_linear.py, blockdiag_butterfly_multiply.py |
The Monarch layer, adapted from HazyResearch/fly |
utils.py |
Dataset download and split, data loading, augmentation, log-Mel features |
checkpoints/gru.pt |
Trained h = 64, b = 4 model |
assets/ |
Figures from the paper |
submit_mn4.sh, submit_inference.sh |
Slurm scripts for training and test evaluation |
env.yml |
Conda environment (the main packages) |
LICENSE, LICENSE-apache, LICENSE-bcresnet, NOTICE.md |
Licenses (see License) |
If you find our work useful, please consider citing our paper.
Conference publication in AICAS'26:
@inproceedings{2026-AICAS-Bhowmick-monarchgru,
author={Bhowmick, Soujanya and Kim, Sangyeob and Chen, Qinyu and Gao, Chang and Kim, Kwantae},
booktitle={2026 IEEE 8th International Conference on Artificial Intelligence Circuits and Systems (AICAS)},
title={{MonarchGRU}: A Low-Footprint Recurrent Neural Network for Audio Keyword Spotting System},
year={2026},
volume={},
number={},
pages={310-314}
}The DOI will be added when the paper is on IEEE Xplore.
This work was supported by the MIcroELectronics (MIELi) doctoral pilot programme, funded by the Finnish Ministry of Education and Culture.
The Monarch layer is adapted from HazyResearch/fly, the code of the Monarch paper
(Dao et al., ICML 2022).
The data pipeline in utils.py is adapted from Qualcomm-AI-research/bcresnet
(Kim et al., Interspeech 2021).
Our code and the checkpoint are released under the MIT License. Some files keep the license of the project they come from:
monarch_linear.py,structured_linear.pyandblockdiag_butterfly_multiply.pyare modified from HazyResearch/fly and stay under the Apache License 2.0 (LICENSE-apache).- Parts of
utils.py(see Acknowledgements) stay under the BSD-3-Clause-Clear license (LICENSE-bcresnet), Copyright (c) 2023 Qualcomm Technologies, Inc. - The figures in
assets/are from the paper, © 2026 IEEE, and are not covered by the MIT License.
See NOTICE.md for details. Questions and bug reports: please open a GitHub issue.