An imperative command-line-interface for AI workload orchestration

A policy layer above transport for KV movement, workload-aware admissibility, and explainable routing in disaggregated inference.

Python control plane for inference and self-improving systems: launch, route, scale, and observe LLM workloads on your cloud with a clean SDK, from disaggregated serving to RL rollouts.

Research fork of LLMServingSim 2.0 investigating Head-of-Line (HoL) blocking mitigation in disaggregated LLM serving via novel scheduling algorithms.

A fault-tolerant LLM routing system that decouples inference from AWS Bedrock by routing prefill and decode tasks through SQS and ensuring zero-downtime scaling with a graceful drain sidecar.

DiSpec — a from-scratch LLM inference engine: paged attention, continuous batching, CUDA-graph decode, speculative decoding, and prefill/decode disaggregation