Описание
Envoy's global rate limit may crash when the response phase limit is enabled and the response phase request is failed directly
Summary
At the rate limit filter, if we enabled the response phase limit with apply_on_stream_done in the rate limit configuration and the response phase limit request fails directly, it may crash Envoy.
Details
When both the request phase limit and response phase limit are enabled, the safe gRPC client instance will be re-used for both the request phase request and response phase request.
But after the request phase request is done, the inner state of the request phase limit request in gRPC client is not cleaned up. When we send the second limit request at response phase, and the second limit request fails directly, we may access the previous request's inner state and result in crash.
PoC
This need to mock the network failure. But we have reproduced by unit test locally.
Impact
This only happens when both the request phase limit and response phase limit are enabled in the rate limit filter, and requires the request to rate limit service fails directly (For example, if from Envoy's perspective, no healthy endpoint for rate limit service may result the request fails directly). That's say, not easy to trigger this.
To workaround
This could be worked around by splitting the rate limit filter. That is, if there is a rate limit filter that contains normal rate limit configuration (request phase limit, without apply_on_stream_done) and also rate limit configuration with apply_on_stream_done (response phase limit). Splitting them into two rate limit filters and ensure one filter only contains normal rate limit configuration (without apply_on_stream_done), and one only contains rate limit configuration with apply_on_stream_done could avoid this problem.
Credit
Mandar Jog (mandarjog@gmail.com)
Пакеты
github.com/envoyproxy/envoy
= 1.37.0
Отсутствует
github.com/envoyproxy/envoy
>= 1.36.0, <= 1.36.4
Отсутствует
Связанные уязвимости
Envoy is a high-performance edge/middle/service proxy. Prior to 1.37.1, 1.36.5, 1.35.8, and 1.34.13, At the rate limit filter, if the response phase limit with apply_on_stream_done in the rate limit configuration is enabled and the response phase limit request fails directly, it may crash Envoy. When both the request phase limit and response phase limit are enabled, the safe gRPC client instance will be re-used for both the request phase request and response phase request. But after the request phase request is done, the inner state of the request phase limit request in gRPC client is not cleaned up. When a second limit request is sent at response phase, and the second limit request fails directly, the previous request's inner state may be accessed and result in crash. This vulnerability is fixed in 1.37.1, 1.36.5, 1.35.8, and 1.34.13.
Envoy is a high-performance edge/middle/service proxy. Prior to 1.37.1 ...