Search results

[Bug]: XQA multi_block_mode crashes with CUDA_ERROR_INVALID_VALUE under concurrent inference (v1.0.0)

bugSomething isn't workingSomething isn't working

Customized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.

Inference runtime<NV>General operational aspects of TRTLLM execution not in other categories.<NV>General operational aspects of TRTLLM execution not in other categories.

Status: Open.

#15537 In NVIDIA/TensorRT-LLM;

· xuxiongjie272 opened on Jun 23, 2026

[Bug]: GLM-5.1-NVFP4 serve fails at warmup: PRE_MLP_FUSION accesses missing gate_up_proj.input_scale on excluded dense layers

Customized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.

Model optimization<NV>Model-specific performance optimizations and tuning<NV>Model-specific performance optimizations and tuning

Status: Open.

#15516 In NVIDIA/TensorRT-LLM;

· muma378 opened on Jun 22, 2026

[Doc]: dtype="auto" resolution chain undocumented for LLM and KvCacheConfig

bugSomething isn't workingSomething isn't working

Doc<NV>TRTLLM's textual/illustrative materials: API refs, guides, tutorials. Improvement & clarity.<NV>TRTLLM's textual/illustrative materials: API refs, guides, tutorials. Improvement & clarity.

Status: Open.

#15512 In NVIDIA/TensorRT-LLM;

· ojas4414 opened on Jun 21, 2026

PyTorch backend (1.3.0rc15): KV-cache block reuse changes first-token logits between first and subsequent identical requests

Inference runtime<NV>General operational aspects of TRTLLM execution not in other categories.<NV>General operational aspects of TRTLLM execution not in other categories.

Pytorch<NV>Pytorch backend related issues<NV>Pytorch backend related issues

Status: Open.

#15503 In NVIDIA/TensorRT-LLM;

· ShuaiShao93 opened on Jun 20, 2026

[Bug]: chunked-prefill fallback does not synchronize llm_args.enable_chunked_prefill

bugSomething isn't workingSomething isn't working

Pytorch<NV>Pytorch backend related issues<NV>Pytorch backend related issues

Status: Open.

#15463 In NVIDIA/TensorRT-LLM;

· DhineshPonnarasan opened on Jun 17, 2026

KVCacheManagerV2 host cache pool allocation is very slow for large host_cache_size (single-threaded faulting + THP compaction stalls)

KV-Cache Managementkv-cache management for efficient LLM inferencekv-cache management for efficient LLM inference

Status: Open.

#15430 In NVIDIA/TensorRT-LLM;

· nafis271 opened on Jun 16, 2026

[Bug]: KVCacheManagerV2: windowed pool min_slots underflow causes scheduler deadlock at high concurrency

bugSomething isn't workingSomething isn't working

KV-Cache Managementkv-cache management for efficient LLM inferencekv-cache management for efficient LLM inference

Status: Open.

#15401 In NVIDIA/TensorRT-LLM;

· lwlsaysnuaa opened on Jun 16, 2026

[Feature Request] Context-parallel (Helix) sharding for MLA in AutoDeploy

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#15367 In NVIDIA/TensorRT-LLM;

· greg-kwasniewski1 opened on Jun 15, 2026

[Bug]: [pyexecutor] SM121 appears to be unintentionally excluded from MLA block-reuse and chunked-prefill allowlists

bugSomething isn't workingSomething isn't working

Inference runtime<NV>General operational aspects of TRTLLM execution not in other categories.<NV>General operational aspects of TRTLLM execution not in other categories.

Status: Open.

#15344 In NVIDIA/TensorRT-LLM;

· DhineshPonnarasan opened on Jun 13, 2026

[Feature]: Add TRTLLM-gen FMHA Dense paged GQA generation cubins for P16

Customized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#15339 In NVIDIA/TensorRT-LLM;

· elwhyjay opened on Jun 13, 2026

[Parity with vLLM, SGLang, ATOM]: Public Nightly NGC docker images too

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#15328 In NVIDIA/TensorRT-LLM;

· functionstackx opened on Jun 12, 2026

[Feature]: Expose per-request scheduling priority in the trtllm-serve OpenAI frontend

LLM API<NV>High-level LLM Python API & tools (e.g., trtllm-llmapi-launch) for TRTLLM inference/workflows.<NV>High-level LLM Python API & tools (e.g., trtllm-llmapi-launch) for TRTLLM inference/workflows.

Status: Open.

#15327 In NVIDIA/TensorRT-LLM;

· sopwg612 opened on Jun 12, 2026