3DMattX: Gamified, real-time 3D cluster visualization and load monitoring.
MattX is a modern, out-of-tree Linux kernel module that brings the magic of Single System Image (SSI) clustering to the Linux 7.x era.
Inspired by the legendary openMosix project, MattX allows multiple physical or virtual machines to act as one massive, unified computer. It autonomously load-balances CPU-bound tasks by seamlessly teleporting running processes across the network—preserving their memory, CPU registers, file descriptors, and network sockets—without requiring any modifications to the Linux kernel source code.
September 2026
A huge shoutout to #openMosix legend Kris Buytaert for his featured article on the "European Environment for Scientific Software Installations (EESSI)" blog about "Testing MattX with EESSI"
Many Thanks Kris! Awesome 🔥
August 2026 The impossible has been achieved. Historically, multi-threaded applications were the ultimate boogeyman of Single System Image (SSI) clustering. Splitting threads across a network meant split-brain deadlocks, shattered memory spaces, and kernel panics. Not anymore.
With MattX 1.8, we are incredibly proud to introduce Native Gang Migration! 🐙
We have completely re-engineered the MattX core to support complex, multi-threaded workloads, bringing us one massive step closer to full MPI/MPICH cluster compatibility.
In MattX, threads sharing a TGID are considered a "Gang" and are never split up. When the Mother task migrates, the entire Gang teleports through the TCP Wormhole with her.
- Native Futex Synchronization: Because the entire Gang shares the same physical RAM on the remote node, local synchronizers like
futexandrseqexecute 100% natively. No network tunneling required! - The Gang Grower 🌱 & Ghost Exorcist 👻: MattX dynamically adapts to thread lifecycles. If threads are born on the remote node, the Home Node natively spawns missing dummy threads during Recall. If threads die remotely, the Home Node silently assassinates the leftovers.
- The True Freeze 🧊: Ironclad kernel-level synchronization guarantees threads are perfectly frozen at the user-space boundary before a single byte of memory is carved.
Why tunnel static, read-only system libraries over the network? MattX 1.8 introduces a configurable bypass that allows the remote node to natively load system libraries (e.g., /lib/libc.so.6, /etc/ld.so.cache) from its own local disk. This saves massive amounts of Wormhole bandwidth, drops latency to zero, and allows complex C++ stack unwinders to execute flawlessly!
Our memory injection pipeline has been upgraded with a surgical "Hole Carver". It dynamically detects growing thread stacks and heap expansions, carving out exact memory holes using MAP_FIXED without fragmenting the VMA tree or destroying file-backed executable memory.
To achieve our goal of full MPI/MPICH compatibility, MattX 1.8 introduces a massive array of newly intercepted and routed syscalls, ensuring complex HPC apps feel right at home:
- Advanced Networking:
getsockname,getpeername,setsockopt,getsockopt,sendmsg,recvmsg - Event Loops (epoll):
epoll_create,epoll_create1,epoll_ctl,epoll_wait - Metadata & VFS:
statfs,fstatfs,newfstatat,faccessat2,readlink,readlinkat,getdents64,pipe2,pread64 - Process Control:
uname,prlimit64,prctl,fcntl,ioctl - Local Memory & Sched (Native VM2 Execution):
mprotect,madvise,mbind,sched_getaffinity,sched_setaffinity,sched_yield
June 2026 MattX v1.7 is our most stable and architecturally advanced release yet! We have completely re-architected the system to cleanly separate the Control Plane (process migration, lifecycle) from the Data Plane (File and Network I/O).
- Kworker + Native Injector: We have moved away from the legacy "Deputy Hijack" (waking the sleeping home process via signals) for most syscalls. Instead, MattX now spawns background kernel threads (
kworkers) that execute syscalls natively and forcefully inject the resultingstruct file *directly into the sleeping Deputy's FD table. All core File and Network I/O syscalls have been refactored to this blazing-fast architecture! - Alma Linux & SELinux Victory: MattX is now fully supported on RPM/RHEL-based distributions like Alma Linux. Even better? Because our Native Injector perfectly mimics standard local process behavior, MattX runs flawlessly with SELinux set to ENFORCING!
- Version Interface: You can now check your running cluster version via the new read-only
/proc/mattx/versioninterface.
June 2026
A massive shoutout to openMosix legend Kris Buytaert for contributing the official MattX-TestSuite! This powerful Makefile/Bash toolkit uses libvirt and official cloud images to fully automate the deployment of a MattX cluster. With a single make debcluster command, you can spin up a fully configured, multi-node MattX development environment in under 5 minutes. It currently supports Debian, Ubuntu, and Alma Linux!
Please find it at:
https://github.com/KrisBuytaert/mattx-testsuite
👉 Read older updates in our NEWS archive
MattX migrates running processes safely using a custom Syscall Drainer. Instead of violently freezing processes with SIGSTOP while they hold kernel locks, MattX uses modern task_work_add(TWA_SIGNAL) to gently guide the process to the user-space boundary. Once in a 100% stable state, its memory map (VMAs), CPU registers (pt_regs), and credentials (struct cred) are extracted and streamed over a custom TCP data pump to the target node.
Historically, multi-threaded applications have been the "boogeyman" of Single System Image (SSI) clustering. Because threads share memory, file descriptors, and signal handlers, splitting them across a network introduces catastrophic latency and split-brain deadlocks. MattX solves this natively using a strict Gang Migration architecture.
To ensure migrated processes can still access their original files, MattX introduces MattXFS, a custom Virtual File System (VFS).
When a process arrives on a remote node, MattX spawns a surrogate (mattx-stub) inside a private Linux Mount Namespace (unshare(CLONE_NEWNS)). The stub bind-mounts the remote MattXFS cluster tree (e.g., /mattxfs/709/) directly over its own root directory (/).
- The Result: The Linux VFS natively routes all file operations (
open,read,write,seek,fsync) across the network back to the Home Node. This completely eliminates the need for hacky Kprobes for file I/O!
For true High-Performance Computing (HPC), sending file I/O over a TCP wormhole is a bottleneck. MattX supports DFSA. If your cluster shares a SAN or NFS drive (e.g., /mnt/shared), you can configure MattX to bypass the network. The Namespace Illusion will bind-mount the local SAN directly into the surrogate's glass box, allowing migrated processes to read and write shared data at bare-metal disk speeds.
Migrated processes don't lose their network connections! MattX uses Kretprobes to intercept network system calls (socket, bind, listen, connect, sendto, recvfrom). It tunnels these requests back to the Home Node, allowing a process running on Node B to maintain an active TCP connection established on Node A. It even features a Distributed Wait Queue to seamlessly route asynchronous poll() and select() calls across the cluster!
MattX maintains strict tracking of exported "Deputies" and remote "Surrogates." If a migrated process finishes its work and exits naturally, the remote node sends a Death Certificate back home to cleanly terminate the Deputy. If a Sysadmin types kill -9 <pid> on the Home Node, an Assassination Order is instantly dispatched to terminate the remote Surrogate. No ghost processes, no memory leaks.
MattX is designed to be "Zero-Config" via UDP Multicast auto-discovery, but it offers powerful toggles for cluster administrators:
# Network interface and UDP Multicast group for Zero-Config Auto-Discovery
INTERFACE=eth0
MULTICAST_GROUP=239.0.0.1
PORT=7225
# Feature Toggles (A/B Testing & Fallbacks)
MIGRATE_NETWORK_IO=true # Set to false to disable socket routing
MIGRATE_FILE_IO=true # Set to false to fallback to Kprobe File I/O
MATTXFS_ENABLED=true # Set to true to enable the MattXFS Namespace Illusion
# Direct File System Access (DFSA)
DFSA_DIR=/mnt/shared # Directory to bypass network routing (e.g., shared NFS/SAN)MattX embraces the UNIX philosophy. The entire cluster can be monitored and controlled using simple text files in the /proc/mattx/ virtual directory.
cat /proc/mattx/nodes- Displays a live table of all connected nodes, their IP addresses, CPU load, and Free Memory.cat /proc/mattx/remote- Lists all processes that have been migrated away from this node (<orig_pid>:<target_node>).cat /proc/mattx/guests- Lists all foreign processes currently running on this node (<local_surrogate_pid>:<home_node>).cat /proc/mattx/version- Displays the current MattX version and license information.
You can control the cluster dynamically by echoing commands into /proc/mattx/admin:
# Disable the automatic load balancer (useful for manual testing)
echo "balancer 0" > /proc/mattx/admin
# Manually migrate a specific PID to a specific Node ID
echo "migrate 1234 814" > /proc/mattx/admin
# Recall a migrated process back to its Home Node!
echo "migrate 1234 home" > /proc/mattx/adminmattx.ko&mattxfs.ko: The core kernel modules.mattx-discd: The user-space UDP discovery daemon. It reads/etc/mattx.confand configures the kernel via Generic Netlink.mattx-stub: The user-space surrogate binary. It carves memory, builds the Namespace Illusion, and sacrifices itself to receive the brain transplant of a migrated process.3DMattX: A gamified, WebGL-based 3D cluster visualization tool. Fly through your cluster in real-time and watch the CPU load bars rise and fall as processes migrate!
MattX is developed and tested on Debian Trixie for Kernel 6.x and Ubuntu 26.04 for Kernel 7.x. It is also fully supported on Alma Linux (Kernel 6.x) with SELinux enforcing!
MattX is proudly open-source and released under the GNU General Public License v2.0 (GPLv2) to ensure maximum compatibility with the Linux kernel ecosystem.
Copyright (c) 2026 by Matthias Rechenburg.