streaming-llm

⚡ Zero-Stall MoE Inference via Lookahead Prediction & Async DMA Prefetching. Optimized for SSD I/O with Hybrid MLA+Sliding Window Attention.

A real-time, browser-native digital twin: your webcam drives a 3D humanoid that mirrors your pose, reads your emotion, and chats back through a streaming Llama 3.1 voice pipeline.

GenPark AI Agent Skill - Streaming attention sink preserver and FIFO sliding-window KV cache eviction manager for infinite-context agent loops.

GenPark AI Agent Skill - Streaming attention sink preserver and FIFO sliding-window KV cache eviction manager for infinite-context agent loops.

Multiplatform 3D chess app powered by Stockfish, featuring on-device AI move coaching and cloud-based RAG for position chat.

Proof of concept for a streaming input, concurrent output, no-compaction inferencing implementation based on the StreamingLLM paper

Agentic AI assistant powered by Google Gemini 2.5, with streaming LLM output, multi-tool data routing, and cross-platform Flutter deployment.

Real-time LLM streaming with local CPU-based Piper TTS, sentence buffering, text cleaning, and smooth FFmpeg audio processing.

Real-time SSE event-stream chat component for Laravel Livewire and React, integrated with streaming LLM providers.