Feature/vlm+toploc by fedor-konovalenko · Pull Request #2 · machine-intelligence-laboratory/gonka-vllm

fedor-konovalenko · 2026-03-06T11:10:21Z

Essential Elements of an Effective PR Description Checklist

The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
The test plan, such as providing test command.
The test results, such as pasting the results comparison before and after, or e2e results
(Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS ABOVE HAVE BEEN CONSIDERED.

Purpose

Test Plan

Test Result

(Optional) Documentation Update

BEFORE SUBMITTING, PLEASE READ https://docs.vllm.ai/en/latest/contributing (anything written below this line will be removed by GitHub Actions)

Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>

…econds metric (#32661) This PR completes the removal of the deprecated vllm:time_per_output_token_seconds metric that was deprecated in v0.11, hidden in v0.12, scheduled for removal in v0.13, but delayed until v0.15. Signed-off-by: carlory <baofa.fan@daocloud.io> Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Signed-off-by: yisheng <yi.sheng@intel.com>

…n (#32603) Signed-off-by: linhaifeng <1371675203@qq.com>

Signed-off-by: whx-sjtu <2952154980@qq.com>

Signed-off-by: yewentao256 <zhyanwentao@126.com>

Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com> Co-authored-by: Hongxia Yang <hongxia.yang@amd.com> Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

Signed-off-by: JJJYmmm <92386084+JJJYmmm@users.noreply.github.com>

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Signed-off-by: rahul-tuli <rtuli@redhat.com>

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

…piledFunction (#25205) Signed-off-by: dolpm <34420038+dolpm@users.noreply.github.com>

Signed-off-by: Shinichi Hemmi <50256998+Alnusjaponica@users.noreply.github.com> Co-authored-by: Kenichi Maehashi <maehashi@preferred.jp>

Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>

Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>

Signed-off-by: Micah Williamson <micah.williamson@amd.com>

Signed-off-by: Or Ozeri <oro@il.ibm.com>

…dels by Default (#32299) Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>

Signed-off-by: Nick Hill <nickhill123@gmail.com>

…neration) (#32542) Signed-off-by: gopalsarda <gopal.sarda@servicenow.com>

…048) Signed-off-by: shanjiaz <zsjwpianpian@gmail.com> Signed-off-by: shanjiaz <43143795+shanjiaz@users.noreply.github.com>

Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>

Signed-off-by: Paco Xu <paco.xu@daocloud.io>

Signed-off-by: ricky-chaoju <ricky.chen@infinirc.com> Signed-off-by: vllm-dev <ricky.chen@infinirc.com>

… (#32741) Signed-off-by: Lucas Kabela <lucaskabela@meta.com>

…put-ids" This reverts commit 49aabe0, reversing changes made to 44cfa7c.

Signed-off-by: fedor-konovalenko <fdkonovalenko@gmail.com>

…hed tokens default parameters baked into vllm

Add logprobs mode as request param + on-validation classifier

Tg/cornercases vllm15

Made-with: Cursor

feat: stronger seed

…ligence-laboratory/gonka-vllm into feature/vlm+toploc Signed-off-by: fedor-konovalenko <fdkonovalenko@gmail.com>

chaunceyjiang and others added 30 commits January 20, 2026 18:48

[Bugfix] Fix the fp8_mqa_logits dim mismatch (#32652)

c4e5bdf

Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>

[4/N] Initialize MM components in context managers (M-P) (#32663)

fda3f03

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

[XPU]Support AgRsAll2AllManager on XPU device (#32654)

13f6630

Signed-off-by: yisheng <yi.sheng@intel.com>

[Bugfix] Fix Off-by-one error in _num_tokens_to_min_blocks calculatio…

7901109

…n (#32603) Signed-off-by: linhaifeng <1371675203@qq.com>

[PluggableLayer][1/N] Define PluggableLayer (#32331)

4ca62a0

Signed-off-by: whx-sjtu <2952154980@qq.com>

[Perf] Only clone when needed for moe_permute (#32273)

6c97b9b

Signed-off-by: yewentao256 <zhyanwentao@126.com>

[Doc] [ROCm] Update ROCm getting started doc (#32580)

c025263

Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com> Co-authored-by: Hongxia Yang <hongxia.yang@amd.com> Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

[Bugfix] fix the ima issue of qwen-vit (#32687)

04a9e06

Signed-off-by: JJJYmmm <92386084+JJJYmmm@users.noreply.github.com>

[Model Runner V2] Support FLASHINFER_MLA backend (#32709)

9ab4388

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

[Doc] Update docs for MM model development with context usage (#32691)

09194b9

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Test: added acceptance length tests (#32030)

f0feb1c

Signed-off-by: rahul-tuli <rtuli@redhat.com>

[5/N] Initialize MM components in context managers (Q-Z) (#32695)

193069d

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

[AOT compilation] support torch.compile inductor artifacts in VllmCom…

7c5dedc

…piledFunction (#25205) Signed-off-by: dolpm <34420038+dolpm@users.noreply.github.com>

[Bugfix] Fix byte fallback handling when using outlines (#31391)

86c69dc

Signed-off-by: Shinichi Hemmi <50256998+Alnusjaponica@users.noreply.github.com> Co-authored-by: Kenichi Maehashi <maehashi@preferred.jp>

[Misc] Remove pad_for_cudagraphs from config (#30143)

2261340

Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>

[Bugfix] Suppress log on non-ROCm platform (#32703)

9b67338

Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>

[ROCm][CI] Remove DS async eplb accuracy test from AMD CI (#32717)

22375f8

Signed-off-by: Micah Williamson <micah.williamson@amd.com>

fp8 online quant: split out Fp8OnlineLinearMethod (#32189)

d2389c1

Revert "[PluggableLayer][1/N] Define PluggableLayer" (#32725)

c78ee24

OffloadingConnector: Prevent redundant loads (#29087)

7013e9a

Signed-off-by: Or Ozeri <oro@il.ibm.com>

[Bugfix] Fix Granite Vision / Don't use Siglip Pooling Head Nested Mo…

27b81e0

…dels by Default (#32299) Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>

[Cleanup] Remove unused KVConnectorModelRunnerMixin methods (#32077)

6f067b1

Signed-off-by: Nick Hill <nickhill123@gmail.com>

Enable Eagle3 speculative decoding for Pixtral (LlavaForConditionalGe…

0900ced

…neration) (#32542) Signed-off-by: gopalsarda <gopal.sarda@servicenow.com>

Added qwen3 vision language moe support for speculative decoding (#32…

7ab80a8

…048) Signed-off-by: shanjiaz <zsjwpianpian@gmail.com> Signed-off-by: shanjiaz <43143795+shanjiaz@users.noreply.github.com>

Update FlashMLA (#32491)

b4f64e5

Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

[Bugfix] Fix Nemotron-Nano-v2-vlm static resolution (#32682)

27ca95b

Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>

[Docs] Fix GitHub handle in governance process (#32582)

360aa93

Signed-off-by: Paco Xu <paco.xu@daocloud.io>

[Bugfix] Support HF sharded weights for Mistral3/Pixtral models (#32673)

f23fb5a

Signed-off-by: ricky-chaoju <ricky.chen@infinirc.com> Signed-off-by: vllm-dev <ricky.chen@infinirc.com>

[Documentation] Fix typo in docs/design/torch_compile_multimodal.md…

c80f92c

… (#32741) Signed-off-by: Lucas Kabela <lucaskabela@meta.com>

baychak and others added 29 commits March 15, 2026 03:57

Revert "Merge pull request gonka-ai#5 from kaitakuai/fix/poc-dummy-in…

2156692

…put-ids" This reverts commit 49aabe0, reversing changes made to 44cfa7c.

skip compiled fix

a1a057c

response reformatting

b2ebbf8

Signed-off-by: fedor-konovalenko <fdkonovalenko@gmail.com>

fix compilation skip

2abe4f9

add scratchpad

de73476

add scratchpad

cf2417a

add logprobs mode as request param + on-validation classifier

31f6c31

remove unused env variables

fb6fc7b

Attention backend, logprobs mode, compilation config and max num batc…

2182f84

…hed tokens default parameters baked into vllm

Fix attention backend

fadde79

fix default args

0021c0c

Merge pull request gonka-ai#25 from gonka-ai/tg/logprobes_switch

591ff97

Add logprobs mode as request param + on-validation classifier

Fix cornercases together with attention metadata

300267e

default to all

f71fb45

Merge pull request gonka-ai#27 from gonka-ai/tg/cornercases_vllm15

6f0a305

Tg/cornercases vllm15

Add test suite

e11fe3f

update test

509ce27

Update priority gating drain -> abort

28ff0ba

Made-with: Cursor

add cli args

539ce25

hardcode dtype auto

80dadb5

feat: stronger seed

bf45627

Merge pull request gonka-ai#30 from gonka-ai/fi/seed-015

2163850

feat: stronger seed

GPU util 0.925 by default

517d056

fix

dd1ddce

results of tests

021a1c4

Merge branch 'feature/vlm+toploc' of https://github.com/machine-intel…

561514c

…ligence-laboratory/gonka-vllm into feature/vlm+toploc Signed-off-by: fedor-konovalenko <fdkonovalenko@gmail.com>

results of tests

de6e3cf

results of tests

69325a9

ruff and precommit

cc98419

fedor-konovalenko closed this Apr 15, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Feature/vlm+toploc#2

Feature/vlm+toploc#2
fedor-konovalenko wants to merge 2211 commits into
tg/token_format_patchfrom
feature/vlm+toploc

fedor-konovalenko commented Mar 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Uh oh!

Conversation

fedor-konovalenko commented Mar 6, 2026

Essential Elements of an Effective PR Description Checklist

Purpose

Test Plan

Test Result

(Optional) Documentation Update

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants