Topics tagged inference-server-triton

Topic	Replies	Views	Activity
Deterministic Inference at Scale: Moving Beyond Agents and MoE in Regulated Workloads TensorRT jetson-inference , inference-server-triton , nim , llama	0	2	December 14, 2025
CUDA Buffer Sharing Failure Between Triton and DeepStream Containers on WSL2 DeepStream SDK cuda , wsl , inference-server-triton , deepstream	2	30	December 8, 2025
TensorRT built-in NMS output lost when using Triton dynamic batching TensorRT tensorrt , cudnn , inference-server-triton	2	56	December 2, 2025
Bug Report Summary \| Product : NVIDIA NIM for Image OCR (NeMo Retriever OCR v1) \| Version: 1.1.0 \| Severity: High (Production Blocker) Technical Support (PhysicsNeMo Only) nemo , inference-server-triton , gpu , nim	0	42	November 18, 2025
Segmentation Fault Loading YOLO v4 TensorRT Model with Triton TensorRT tensorrt , inference-server-triton	1	40	November 18, 2025
NIM to Triton Server Pipeline Models inference-server-triton , nim	1	134	November 14, 2025
Creating a container for seminar Fundamentals of Deep Learning Courses and Workshops cuda , kernel , docker , pytorch , inference-server-triton	0	17	November 9, 2025
Nvinfer yields constant OCR text with NHWC engine (fast_plate_ocr – cct_s_v1_global_model) while nvinferserver returns correct results DeepStream SDK inference-server-triton , deepstream	3	64	November 7, 2025
Gray image in Triton DeepStream SDK gstreamer , inference-server-triton , deepstream	3	38	October 31, 2025
Running Llama-3.1-8B-FP4 get triton error. Value 'sm_121a' is not defined for option 'gpu-name' DGX Spark / GB10 inference-server-triton , llama	2	315	October 24, 2025
Tensor-RT rejects engine cache pre-built on same device type Jetson AGX Orin tensorrt , cudnn , inference-server-triton	5	90	October 21, 2025
tritonclient.utils.InferenceServerException: Fail to connect to remote host ipv4:127.0.0.1:8001 in TRELLIS NIM Models inference-server-triton , nim	0	83	September 23, 2025
Connection problem due to lack of CORS support in Triton Server, which blocks requests from frontend web applications TensorRT python , cudnn , onnx , inference-server-triton	3	104	September 12, 2025
Triton + TensorRT-LLM (Llama 3.1 8B) – Feasibility of Stateful Serving + KV Cache Reuse + Priority Caching JAX inference-server-triton , llama	1	76	September 5, 2025
How to access labelfile_path in custom classifier parser for nvinferserver? DeepStream SDK gstreamer , inference-server-triton , jetson , deepstream	2	79	August 19, 2025
Feature Proposal: Enable Deterministic Algorithms in Triton server PyTorch Backend CUDA Programming and Performance inference-server-triton	0	78	August 5, 2025
Error reading checkpoint.tl cuDNN inference-server-triton	1	98	July 31, 2025
Triton server GPU memory leak for grpc cuda shared memory request GPU - Hardware cuda , inference-server-triton , gpu	3	195	August 8, 2025
Nvcr.io/nvidia/l4t-triton:r35.2.1 access denied Jetson Orin Nano inference-server-triton , jetson	3	90	August 13, 2025
NSight with AGX Orin and Deepstream + Triton DeepStream SDK inference-server-triton , deepstream , jetson-orin	11	215	July 15, 2025
Intermittent Artifacts in DeepStream RTSP Output with Dynamic Multi-Stream Video Analytics with triton inference server with python backend DeepStream SDK gstreamer , inference-server-triton , deepstream	88	819	July 8, 2025
Deploying Triton Server with TensorRT-LLM on Jetson AGX Orin (JetPack 6.2) — Any Working Example? Jetson AGX Orin tensorrt , jetson-inference , inference-server-triton , generative_ai , llm	10	739	June 17, 2025
How to get model configuration from HTTP API without first loading the model in EXPLICIT mode? AI & Data Science inference-server-triton	1	68	April 30, 2025
Windows systems perfomance issue TensorRT tensorrt , inference-server-triton	1	86	April 30, 2025
How to load specific version of a model using EXPLICIT mode? AI & Data Science inference-server-triton	0	52	April 29, 2025
tritonclient.utils.InferenceServerException: No field is set NGC GPU Cloud inference-server-triton	1	1705	April 17, 2025
CUDA shared memory doesn't work (failed to open CUDA IPC handle: invalid device context) DeepStream SDK inference-server-triton , deepstream	9	515	April 14, 2025
Deepstream + triton infer server DeepStream SDK inference-server-triton , deepstream	4	155	March 25, 2025
Invalid argument: model input NHWC/NCHW require 3 dims for visual_changenet_segmentation_tao TAO Toolkit tensorrt , inference-server-triton	5	111	March 13, 2025
NvInferServer implementation of LSTM model DeepStream SDK inference-server-triton , deepstream	9	171	March 10, 2025