feat: Variet Engine v1.0 + 5-model tuning complete

Phase 01 (LLM Tuning):
- Gemma4 26B: 74.65 t/s (fast)
- Qwen 35B: 61.62 t/s (balanced)
- Gemma4 31B: 16.0 t/s (deep-coder)
- Qwen 27B: 16.7 t/s (deep-logic)
- Qwen 122B: 8.95 t/s (ultra, GPU 1 only)

Phase 02 (API Engine):
- FastAPI reverse proxy on port 8000
- /engine/switch hot-swap with 503 protection
- config/engine_models.json as single source of truth
- Replaced 4 individual .bat files with unified engine

File cleanup:
- scripts/ 85 files -> 9 + _archive/
- Root .bat files -> _archive/
This commit is contained in:
Variet-Worker
2026-04-07 18:08:58 +09:00
parent 7c7a899fd5
commit c111b3a9b0
414 changed files with 3402 additions and 68598 deletions

View File

@@ -0,0 +1,31 @@
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 24575 MiB):
Device 0: NVIDIA GeForce RTX 3060, compute capability 8.6, VMM: yes, VRAM: 12287 MiB
Device 1: NVIDIA GeForce RTX 3060, compute capability 8.6, VMM: yes, VRAM: 12287 MiB
-dev, --device <dev1,dev2,..> comma-separated list of devices to use for offloading (none = don't
use --list-devices to see a list of available devices
(env: LLAMA_ARG_DEVICE)
--list-devices print list of available devices and exit
-ot, --override-tensor <tensor name pattern>=<buffer type>,...
override tensor buffer type
(env: LLAMA_ARG_OVERRIDE_TENSOR)
-cmoe, --cpu-moe keep all Mixture of Experts (MoE) weights in the CPU
-ncmoe, --n-cpu-moe N keep the Mixture of Experts (MoE) weights of the first N layers in the
-sm, --split-mode {none,layer,row} how to split the model across multiple GPUs, one of:
- layer (default): split layers and KV across GPUs
- row: split rows across GPUs
(env: LLAMA_ARG_SPLIT_MODE)
-ts, --tensor-split N0,N1,N2,... fraction of the model to offload to each GPU, comma-separated list of
(env: LLAMA_ARG_TENSOR_SPLIT)
-mg, --main-gpu INDEX the GPU to use for the model (with split-mode = none), or for
intermediate results and KV (with split-mode = row) (default: 0)
-fit, --fit [on|off] whether to adjust unset arguments to fit in device memory ('on' or
target margin per device for --fit, comma-separated list of values,
single value is broadcast across all devices, default: 1024
--check-tensors check model tensor data for invalid values (default: false)
--op-offload, --no-op-offload whether to offload host tensor operations to device (default: true)
-otd, --override-tensor-draft <tensor name pattern>=<buffer type>,...
override tensor buffer type for draft model
-cmoed, --cpu-moe-draft keep all Mixture of Experts (MoE) weights in the CPU for the draft
-ncmoed, --n-cpu-moe-draft N keep the Mixture of Experts (MoE) weights of the first N layers in the
-devd, --device-draft <dev1,dev2,..> comma-separated list of devices to use for offloading the draft model
use --list-devices to see a list of available devices