-
Notifications
You must be signed in to change notification settings - Fork 26
Issues
is:issue state:open
is:issue state:open
Issue creation is restricted in this repository
Search results
perf(server): tick-cooperative speculative decoding to remove the burst head-of-line block
area:inferenceGeneration, sampling, decoding (incl. speculative, DRY)Generation, sampling, decoding (incl. speculative, DRY)priority:mediumMedium priorityMedium prioritystatus:readyReady to be worked onReady to be worked ontype:performancePerformance improvementsPerformance improvementsStatus: Open.#734 In lablup/mlxcel;perf(server): decode runs ~11% slower after a prompt-cache hit than after a cold prefill
area:inferenceGeneration, sampling, decoding (incl. speculative, DRY)Generation, sampling, decoding (incl. speculative, DRY)priority:lowLow priorityLow prioritystatus:readyReady to be worked onReady to be worked ontype:performancePerformance improvementsPerformance improvementsStatus: Open.#730 In lablup/mlxcel;test(surgery): empty_surgery_pipeline_matches_baseline_output is flaky on CUDA (post-EOS tail nondeterminism)
area:cliCommand-line interface / CLI flagsCommand-line interface / CLI flagsarea:surgeryModel surgery operations (merge, interpolate, replace, etc.)Model surgery operations (merge, interpolate, replace, etc.)priority:lowLow priorityLow prioritystatus:readyReady to be worked onReady to be worked ontype:bugBug fixes, error corrections, or issue resolutionsBug fixes, error corrections, or issue resolutionsStatus: Open.#728 In lablup/mlxcel;perf(cuda/quant): amortizing small-M quantized GEMM for batched decode on Blackwell
area:coremlxcel-core: MLX FFI, primitives, KV cache, layersmlxcel-core: MLX FFI, primitives, KV cache, layerspriority:mediumMedium priorityMedium prioritystatus:readyReady to be worked onReady to be worked ontype:performancePerformance improvementsPerformance improvementsStatus: Open.#725 In lablup/mlxcel;ci: generate and publish a CycloneDX SBOM on release
priority:mediumMedium priorityMedium prioritystatus:readyReady to be worked onReady to be worked ontype:choreMaintenance tasks (build, CI, etc.)Maintenance tasks (build, CI, etc.)Status: Open.#719 In lablup/mlxcel;fix(server): dense batched-decode + --max-kv-size mis-decodes prompts longer than the cap
area:coremlxcel-core: MLX FFI, primitives, KV cache, layersmlxcel-core: MLX FFI, primitives, KV cache, layerspriority:mediumMedium priorityMedium prioritystatus:readyReady to be worked onReady to be worked ontype:bugBug fixes, error corrections, or issue resolutionsBug fixes, error corrections, or issue resolutionsStatus: Open.#718 In lablup/mlxcel;epic: CUDA (MLX backend) prefill and decode acceleration program
area:coremlxcel-core: MLX FFI, primitives, KV cache, layersmlxcel-core: MLX FFI, primitives, KV cache, layersarea:inferenceGeneration, sampling, decoding (incl. speculative, DRY)Generation, sampling, decoding (incl. speculative, DRY)priority:mediumMedium priorityMedium prioritystatus:readyReady to be worked onReady to be worked ontype:performancePerformance improvementsPerformance improvementsStatus: Open.#623 In lablup/mlxcel;feat: extend the OpenXLA packed-quant path to bf16 scales/biases
area:inferenceGeneration, sampling, decoding (incl. speculative, DRY)Generation, sampling, decoding (incl. speculative, DRY)priority:lowLow priorityLow prioritystatus:readyReady to be worked onReady to be worked ontype:enhancementNew features, capabilities, or significant additionsNew features, capabilities, or significant additionsStatus: Open.#603 In lablup/mlxcel;track(upstream-iree): fused dequant->matmul for the packed int8 bandwidth win
area:inferenceGeneration, sampling, decoding (incl. speculative, DRY)Generation, sampling, decoding (incl. speculative, DRY)priority:lowLow priorityLow prioritystatus:blockedBlocked by dependencies or other issuesBlocked by dependencies or other issuestype:performancePerformance improvementsPerformance improvementsStatus: Open.#574 In lablup/mlxcel;epic: OpenXLA multimodal / VLM architecture support (Qwen2-VL, Gemma3n, Phi4MM, Molmo, and more)
area:architectureArchitecture and code structure changesArchitecture and code structure changesarea:inferenceGeneration, sampling, decoding (incl. speculative, DRY)Generation, sampling, decoding (incl. speculative, DRY)priority:lowLow priorityLow prioritystatus:readyReady to be worked onReady to be worked ontype:enhancementNew features, capabilities, or significant additionsNew features, capabilities, or significant additionsStatus: Open.#566 In lablup/mlxcel;epic: OpenXLA SSM / hybrid / recurrent architecture support (Mamba2, Nemotron-H, Jamba, and more)
area:architectureArchitecture and code structure changesArchitecture and code structure changesarea:inferenceGeneration, sampling, decoding (incl. speculative, DRY)Generation, sampling, decoding (incl. speculative, DRY)priority:lowLow priorityLow prioritystatus:readyReady to be worked onReady to be worked ontype:enhancementNew features, capabilities, or significant additionsNew features, capabilities, or significant additionsStatus: Open.#564 In lablup/mlxcel;feat(kimi-vl): 3D MoonViT video support (image + video patch embedding)
area:modelsModel architectures, weights, loading, metadataModel architectures, weights, loading, metadatapriority:lowLow priorityLow prioritystatus:readyReady to be worked onReady to be worked ontype:enhancementNew features, capabilities, or significant additionsNew features, capabilities, or significant additionsStatus: Open.#551 In lablup/mlxcel;