trt-llm

Here are 3 public repositories matching this topic...

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

reinforcement-learning inference rdma disaggregation llm vllm sglang kvcache trt-llm tokenspeed

Benchmark and optimize LLM inference backends on Amazon SageMaker for latency, throughput, and cost.

aws benchmarking inference sagemaker llm llama-cpp vllm trt-llm

Failure-mode diagnosis of TensorRT-LLM quantization on Qwen2-VL-2B. FP8 KV cache causes silent VLM regression; SmoothQuant is Pareto-optimal.

Add a description, image, and links to the trt-llm topic page so that developers can more easily learn about it.

To associate your repository with the trt-llm topic, visit your repo's landing page and select "manage topics."