Dhruv Mullick

PostsComments

RRunPod

•Created by Dhruv Mullick on 4/5/2024 in #⛅｜pods

TensorRT-LLM setup

Has anyone been able to successfully install tensorrt_llm? I'm trying with pip, but I'm running into mpi related errors: Cannot open configuration file /build-result/hpcx-v2.16-gcc-inbox-ubuntu22.04-cuda12-gdrcopy2-nccl2.18-x86_64/ompi/share/openmpi/mpicc-wrapper-data.txt Error parsing data file mpicc: Not found I've tried a few templates (runpod/pytorch:2.2.0-py3.10-cuda12.1.1-devel-ubuntu22.04; nvcr.io/nvidia/tritonserver:24.03-trtllm-python-py3) on A100 and on a 4090. Cuda 12.2

53 replies

RRunPod

•Created by Dhruv Mullick on 3/4/2024 in #⛅｜pods

Frequent GPU problem with H100

23 replies