9654 x 2 + 630内存 + 4060ti 跑GLM-5.3-Flash-Q4_K_M

3.35.792.741 D srv operator (): all results received, terminating stream

3.35.792.757 D srv operator (): http: streamed chunk: data: [DONE]

3.35.792.797 D srv operator (): http: stream ended

[ Prompt: 7.1 t/s | Generation: 2.5 t/s ]

Exiting...

3.35.797.233 D que start_loop: processing new tasks

3.35.797.235 D que process_new_: terminate

3.35.797.552 I srv operator (): operator (): cleaning up before exit...

3.35.814.072 I common_memory_breakdown_print: | memory breakdown [MiB] | total free self model context compute unaccounted |

3.35.814.081 I common_memory_breakdown_print: | - CUDA0 (RTX 4060 Ti) | 8187 = 0 + ( 7436 = 5316 + 0 + 2120) + 751 |

3.35.814.082 I common_memory_breakdown_print: | - Host | 175712 = 174922 + 673 + 116 |

3.35.814.239 D ~llama_context: CUDA0 compute buffer size is 2120.0234 MiB, matches expectation of 2120.0234 MiB

3.35.814.241 D ~llama_context: CUDA_Host compute buffer size is 116.4122 MiB, matches expectation of 116.4122 MiB

^-^

个人还是别玩了。

这是Windows 跑的，Linux可以会快一点点。