[Feature] Add MESH expert residency with io_uring direct I/O for MoE inference#2003

Open

RaQiu wants to merge 61 commits into

kvcache-ai:mainfrom

Commits on Feb 12, 2026

Commits on Mar 11, 2026

feat: integrate triton-windows for Windows platform support

RaQiu
and
claude
committed
fix: downgrade git hooks install failure from FATAL_ERROR to WARNING

RaQiu
and
claude
committed
fix: replace shell-based git hooks install with pure CMake

RaQiu
and
claude
committed
Revert "fix: replace shell-based git hooks install with pure CMake"
RaQiu
committed
fix: add Windows bat script for git hooks installation

RaQiu
and
claude
committed
fix: auto-download hwloc on Windows when pkg-config unavailable

RaQiu
and
claude
committed
fix: skip libnuma on Windows, use built-in NUMA API

RaQiu
and
claude
committed
fix: add Windows compat for pthread/numa/posix_memalign in cpu_backend

RaQiu
and
claude
committed
[fix] Windows build: add NOMINMAX and wrap numa.h in moe.hpp

RaQiu
and
claude
committed
[fix] Windows: support .pyd extension in CPU variant loader

RaQiu
and
claude
committed
[fix] Windows: fix extension output directory and .pyd detection in setup.py

RaQiu
and
claude
committed
fix(windows): support .pyd extension in doctor.py diagnostics

RaQiu
and
claude
committed
feat: add kt-frontend desktop app with dashboard, chat, and server management

RaQiu
and
claude
committed
feat: implement three-tier MoE weight management with mmap and NUMA promotion

RaQiu
and
claude
committed
feat: add weight-strategy CLI option and frontend mmap toggle

RaQiu
and
claude
committed
fix: bind mmap pages to NUMA nodes and prefetch before promote

RaQiu
and
claude
committed
feat: add tier0 parameter support via environment variables

RaQiu
and
claude
committed
fix: remove unsupported SGLang parameters, use env vars only

RaQiu
and
claude
committed
feat: auto-detect available memory for tier0 allocation

RaQiu
and
claude
committed
[fix]: C++ compilation and sglang backward compatibility
RaQiu
committed
feat: add tier0 memory management parameter support
RaQiu
committed

Commits on Mar 14, 2026

kt-kernel: enable bf16 tiered mmap lazy packing
RaQiu
committed

Commits on Mar 27, 2026

[fix]: improve native conversion and weight runtime
KT Fix
committed

Commits on Mar 29, 2026

Commits on Mar 30, 2026

[merge]: upstream/qwen3.5 - Qwen3.5 MoE packed format support
RaQiu
committed

Commits on May 7, 2026

Commits on May 8, 2026

[fix]: harden iouring promotion concurrency
RaQiu
committed

Commits on May 12, 2026

[fix](mesh): guard resident io by buffer layout
RaQiu
committed

Commits on May 13, 2026

[feat](mesh): add prefill layer-window residency mode
RaQiu
committed

Commits on May 14, 2026

[feat](mesh): restore prefill expert-window path
RaQiu
committed

Commits on May 19, 2026

Commits on May 30, 2026

feat(mesh): add rolling prefetch + runtime helpers for paper benchmarks
RaQiu
committed