Описание
vLLM is an inference and serving engine for large language models (LLMs). From 0.5.5 until 0.23.1rc0, integer truncation of tensor dimensions in vLLM's GGUF dequantize kernels (csrc/quantization/gguf/gguf_kernel.cu) causes partial tensor processing. The output tensor is allocated at full size via torch::empty (uninitialized memory), but the dequantize CUDA kernel processes only a truncated number of elements. The unfilled portion of the output tensor retains whatever was previously in GPU memory. In multi-tenant inference deployments, this residual GPU memory may contain tensor data from other users' inference requests, constituting information disclosure. This vulnerability is fixed in 0.23.1rc0.
A flaw was found in vLLM. Integer truncation of tensor dimensions in vLLM's GGUF dequantize kernels leads to partial tensor processing. This results in the output tensor retaining previously used GPU memory, which, in multi-tenant inference deployments, can expose sensitive tensor data from other users' requests. This constitutes an information disclosure vulnerability.
Отчет
Red Hat rates this issue as having Low impact for Red Hat AI products. The upstream issue is limited information disclosure via integer truncation in vLLM sampling parameters. Red Hat OpenShift AI, Red Hat AI Inference Server, and Red Hat Enterprise Linux AI images are not considered affected because untrusted clients cannot control the vulnerable parameters in supported deployment models.
Меры по смягчению последствий
No mitigation is required for unaffected deployments. Restrict untrusted access to inference APIs as a general hardening measure.
Затронутые пакеты
| Платформа | Пакет | Состояние | Рекомендация | Релиз |
|---|---|---|---|---|
| Red Hat AI Inference Server | rhaiis/vllm-cpu-rhel9 | Not affected | ||
| Red Hat AI Inference Server | rhaiis/vllm-cuda-rhel9 | Not affected | ||
| Red Hat AI Inference Server | rhaiis/vllm-neuron-rhel9 | Not affected | ||
| Red Hat AI Inference Server | rhaiis/vllm-rocm-rhel9 | Not affected | ||
| Red Hat AI Inference Server | rhaiis/vllm-spyre-rhel9 | Not affected | ||
| Red Hat AI Inference Server | rhaiis/vllm-tpu-rhel9 | Not affected | ||
| Red Hat AI Inference Server | rhaii/vllm-cpu-rhel9 | Not affected | ||
| Red Hat AI Inference Server | rhaii/vllm-cuda-rhel9 | Not affected | ||
| Red Hat AI Inference Server | rhaii/vllm-gaudi-rhel9 | Not affected | ||
| Red Hat AI Inference Server | rhaii/vllm-neuron-rhel9 | Not affected |
Показывать по
Ссылки на источники
Дополнительная информация
Статус:
4.3 Medium
CVSS3
Связанные уязвимости
vLLM is an inference and serving engine for large language models (LLMs). From 0.5.5 until 0.23.1rc0, integer truncation of tensor dimensions in vLLM's GGUF dequantize kernels (csrc/quantization/gguf/gguf_kernel.cu) causes partial tensor processing. The output tensor is allocated at full size via torch::empty (uninitialized memory), but the dequantize CUDA kernel processes only a truncated number of elements. The unfilled portion of the output tensor retains whatever was previously in GPU memory. In multi-tenant inference deployments, this residual GPU memory may contain tensor data from other users' inference requests, constituting information disclosure. This vulnerability is fixed in 0.23.1rc0.
vLLM is an inference and serving engine for large language models (LLM ...
vLLM: GGUF dequantize kernel int truncation exposes uninitialized GPU memory in multi-tenant serving
4.3 Medium
CVSS3