手上有一台服务器,配置如下

显卡是4090*2
通过llama.cpp 部署的 Qwen3.8-27B-Uncensored-Q5_K_M.gguf
启动参数为:
ExecStart=/usr/local/bin/llama-server \
-m /home/xxxx/ollama/qwen3.8/Qwen3.8-27B-Uncensored-Q5_K_M.gguf \
--mmproj /home/xxxx/ollama/qwen3.8/mmproj-Qwen3.8-27B-Uncensored-f16.gguf \
-a Qwen3.8-27B \
--host 0.0.0.0 \
--port 8888 \
-ngl all \
-np 1 \
-c 262144 \
-kvu \
--temp 1.00 \
--top-k 20 \
--top-p 0.95 \
--min-p 0.00 \
--repeat-penalty 1.00 \
--presence-penalty 0.00 \
--frequency-penalty 0.00 \
--spec-type draft-mtp \
--spec-draft-n-max 1 \
--load-mode mlock \
--reasoning-preserve \
--reasoning-effort medium
GPU内存占用:
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 595.84 Driver Version: 595.84 CUDA Version: 13.2 |
+-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 4090 Off | 00000000:02:00.0 Off | Off |
| 30% 48C P0 82W / 450W | 20761MiB / 24564MiB | 7% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 1 NVIDIA GeForce RTX 4090 Off | 00000000:03:00.0 Off | Off |
| 30% 35C P8 21W / 450W | 21566MiB / 24564MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| 0 N/A N/A 2374 G /usr/lib/xorg/Xorg 4MiB |
| 0 N/A N/A 621097 G /usr/bin/gnome-shell 406MiB |
| 0 N/A N/A 621387 C+G ...c/gnome-remote-desktop-daemon 477MiB |
| 0 N/A N/A 621481 G /usr/bin/Xwayland 11MiB |
| 0 N/A N/A 779483 C /usr/local/bin/llama-server 19732MiB |
| 0 N/A N/A 793036 G ...tigravity-ide/antigravity-ide 30MiB |
| 0 N/A N/A 892515 G ...y/Antigravity-x64/antigravity 11MiB |
| 1 N/A N/A 2374 G /usr/lib/xorg/Xorg 4MiB |
| 1 N/A N/A 779483 C /usr/local/bin/llama-server 21536MiB |
+-----------------------------------------------------------------------------------------+
部署完的模型接入到sub2api,在opencode中使用,
opencode中的这个模型的配置如下:
"Qwen3.8": {
"name": "Qwen3.8",
"thinking": true,
"attachment": true,
"limit": {
"context": 262144,
"output": 65535
}
}
目前发现一个问题,每次会话即将要压缩,或者压缩的时候, opencode中就会出现超时, 服务器的风扇狂转, 请问一下各位佬是什么原因?哪里可以优化一下 ^-^
sub2api前面的nginx的错误如下:
upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.110.234, server: xx.xxxxxx.xx, request: "POST /sub2api/v1/chat/completions HTTP/1.1", upstream: "http://192.168.110.80:8080/v1/chat/completions", host: "xx.xxxxxx.xx:11443"