Описание
vLLM is an inference and serving engine for large language models (LLMs). From 0.18.0 to before 0.20.0, the extract_hidden_states speculative decoding proposer in vLLM returns a tensor with an incorrect shape after the first decode step, causing a RuntimeError that crashes the EngineCore process. The crash is triggered when any request in the batch uses sampling penalty parameters (repetition_penalty, frequency_penalty, or presence_penalty). A single request with a penalty parameter (e.g., "repetition_penalty": 1.1) is sufficient to crash the server. This vulnerability is fixed in 0.20.0.
A flaw was found in vLLM, an inference and serving engine for large language models (LLMs). The extract_hidden_states speculative decoding proposer returns a tensor with an incorrect shape after the first decode step. This can be triggered by a remote attacker sending a request that uses sampling penalty parameters, such as repetition_penalty, frequency_penalty, or presence_penalty. Successful exploitation leads to a RuntimeError that crashes the EngineCore process, resulting in a denial of service (DoS).
Отчет
This Important denial of service flaw in vLLM, as used in Red Hat AI Inference Server and Red Hat OpenShift AI, allows a remote attacker to crash the EngineCore process. By sending a request with specific sampling penalty parameters, an attacker can trigger an incorrect tensor shape, leading to a service disruption for affected AI inference workloads.
Меры по смягчению последствий
Mitigation for this issue is either not available or the currently available options do not meet the Red Hat Product Security criteria comprising ease of use and deployment, applicability to widespread installation base, or stability.
Затронутые пакеты
| Платформа | Пакет | Состояние | Рекомендация | Релиз |
|---|---|---|---|---|
| Red Hat AI Inference Server | rhaiis/vllm-cpu-rhel9 | Will not fix | ||
| Red Hat AI Inference Server | rhaiis/vllm-cuda-rhel9 | Affected | ||
| Red Hat AI Inference Server | rhaiis/vllm-neuron-rhel9 | Will not fix | ||
| Red Hat AI Inference Server | rhaiis/vllm-rocm-rhel9 | Affected | ||
| Red Hat AI Inference Server | rhaiis/vllm-spyre-rhel9 | Affected | ||
| Red Hat AI Inference Server | rhaiis/vllm-tpu-rhel9 | Will not fix | ||
| Red Hat AI Inference Server | rhaii/vllm-cpu-rhel9 | Affected | ||
| Red Hat AI Inference Server | rhaii/vllm-cuda-rhel9 | Affected | ||
| Red Hat AI Inference Server | rhaii/vllm-gaudi-rhel9 | Affected | ||
| Red Hat AI Inference Server | rhaii/vllm-neuron-rhel9 | Affected |
Показывать по
Дополнительная информация
Статус:
EPSS
7.5 High
CVSS3
Связанные уязвимости
vLLM is an inference and serving engine for large language models (LLMs). From 0.18.0 to before 0.20.0, the extract_hidden_states speculative decoding proposer in vLLM returns a tensor with an incorrect shape after the first decode step, causing a RuntimeError that crashes the EngineCore process. The crash is triggered when any request in the batch uses sampling penalty parameters (repetition_penalty, frequency_penalty, or presence_penalty). A single request with a penalty parameter (e.g., "repetition_penalty": 1.1) is sufficient to crash the server. This vulnerability is fixed in 0.20.0.
vLLM is an inference and serving engine for large language models (LLM ...
vLLM: extract_hidden_states speculative decoding crashes server on any request with penalty parameters
EPSS
7.5 High
CVSS3