[Bug] After using H200*8 to deploy DeepSeekR1, the large stress test model crashes

[wangguo1230]

Checklist

• 1. I have searched related issues but cannot get the expected help.
• 2. The bug has not been fixed in the latest version.
• 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
• 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
• 5. Please use English, otherwise it will be closed.

Describe the bug

If you use a stress test tool to stress test an API, an error is reported in the 500 request and the following symptoms occur:

1. The utilization rate of 7 graphics cards is 100%, and 1 card is idle
2. The model will get stuck and will no longer answer
3. All requests return 200 status codes in an instant

Reproduction

H200*8

docker run -d --name sglang --gpus all --shm-size 512g -p 20011:20011 -v /data1/models:/workspace --ipc=host --network=host --privileged lmsysorg/sglang:latest python3 -m sglang.launch_server --model-path /workspace/DeepSeek-R1 --host 0.0.0.0 --served-model-name DeepSeek-R1 --context-length 32000 --tp 8 --trust-remote-code --enable-dp-attention --mem-fraction-static 0.9 --enable-flashinfer-mla --port 20011

Environment

docker image lmsysorg/sglang:latest

[Fridge003]

Hi @wangguo1230, please try removing --enable-dp-attention or tuning the value of --mem-fraction-static and see whether this will help.

[wangguo1230]

It's okay to remove -enable-dp-attention, but then there is no high throughput

[Fridge003]

You can refer to #3956 for other optimization options. --enable-dp-attention is not a stable one.