llama-cpp: set batch size to 4096 · 6d47f02a0f - server-config - Simon Gardling's Gitea instance

titaniumtown/server-config

Archived

llama-cpp: set batch size to 4096

All checks were successful

Build and Deploy / deploy (push) Successful in 1m22s

This commit is contained in:

Simon Gardling

2026-04-06 02:29:37 -04:00

parent 9addb1569a

commit 6d47f02a0f

1 changed files with 4 additions and 0 deletions

									
										4

services/llama-cpp/llama-cpp.nix
									
												View File
												
					@@ -40,6 +40,10 @@ in

					      "--metrics"

					      "--metrics"

					      "--alias"

					      "--alias"

					      modelAlias

					      modelAlias

					      "-b"

					      "4096"

					      "-ub"

					      "4096"

					    ];

					    ];

					  };

					  };