本地跑 gemma4-12B-qat 推理 60tok/s 还能怎么提高？

RT

+-----------------------------------------------------------------------------------------+

| NVIDIA-SMI 595.79                 Driver Version: 595.79         CUDA Version: 13.2     |

+-----------------------------------------+------------------------+----------------------+

| GPU  Name                  Driver-Model | Bus-Id          Disp.A | Volatile Uncorr. ECC |

| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |

|                                         |                        |               MIG M. |

|=========================================+========================+======================|

|   0  NVIDIA GeForce RTX 5070 ...  WDDM  |   00000000:01:00.0  On |                  N/A |

| N/A   59C    P2            112W /  130W |   10069MiB /  12227MiB |     63%      Default |

|                                         |                        |                  N/A |

+-----------------------------------------+------------------------+----------------------+

Processing Prompt [BATCH] (2149 / 2149 tokens)

Generating (1 / 2000 tokens)sched_reserve: reserving ...

sched_reserve: max_nodes = 1024

sched_reserve: reserving full memory module

sched_reserve: worst-case: n_tokens = 512, n_seqs = 1, n_outputs = 1

sched_reserve:      CUDA0 compute buffer size =  1034.50 MiB

sched_reserve:  CUDA_Host compute buffer size =    24.88 MiB

sched_reserve: graph nodes  = 132

sched_reserve: graph splits = 2

sched_reserve: reserve took 26.98 ms, sched copies = 1

Generating (1054 / 2000 tokens)

(EOS token triggered! ID:106)

[01:32:47] CtxLimit:3203/8192, Init:0.01s, Processed:2149 in 1.37s (1565.19T/s), Generated:1054/2000 in 17.25s (61.09T/s), Total:18.63s

还有请问怎么开启批量处理？对沉静式翻译用途来说。