feat: Variet Engine v1.0 + 5-model tuning complete
Phase 01 (LLM Tuning): - Gemma4 26B: 74.65 t/s (fast) - Qwen 35B: 61.62 t/s (balanced) - Gemma4 31B: 16.0 t/s (deep-coder) - Qwen 27B: 16.7 t/s (deep-logic) - Qwen 122B: 8.95 t/s (ultra, GPU 1 only) Phase 02 (API Engine): - FastAPI reverse proxy on port 8000 - /engine/switch hot-swap with 503 protection - config/engine_models.json as single source of truth - Replaced 4 individual .bat files with unified engine File cleanup: - scripts/ 85 files -> 9 + _archive/ - Root .bat files -> _archive/
This commit is contained in:
30
_archive/start_gemma4_26b_api.bat
Normal file
30
_archive/start_gemma4_26b_api.bat
Normal file
@@ -0,0 +1,30 @@
|
||||
@echo off
|
||||
chcp 65001 >nul
|
||||
echo =========================================================
|
||||
echo Gemma4 26B-A4B API Server (256K Context - Final Optimal)
|
||||
echo [CORE] --n-cpu-moe 10: VRAM 12GB 최적화용 Expert 오프로드
|
||||
echo [TUNED] -t 4 -ub 512: CPU 병목 방지 및 SWA 캐시 최적화
|
||||
echo [PERF] Speed: ~30.9 t/s (1x RTX 3060)
|
||||
echo =========================================================
|
||||
echo.
|
||||
|
||||
llama_bin_run\llama-server.exe ^
|
||||
--model models\gemma-4-26B-A4B-it-Q4_K_M.gguf ^
|
||||
-ngl 999 ^
|
||||
--n-cpu-moe 10 ^
|
||||
-c 262144 ^
|
||||
-np 1 ^
|
||||
-fa on ^
|
||||
--cache-type-k q4_0 ^
|
||||
--cache-type-v q4_0 ^
|
||||
-ub 512 ^
|
||||
-b 2048 ^
|
||||
-t 4 ^
|
||||
-tb 4 ^
|
||||
--mlock ^
|
||||
--prio 3 ^
|
||||
--poll 50 ^
|
||||
--port 8000 ^
|
||||
--host 0.0.0.0
|
||||
|
||||
pause
|
||||
Reference in New Issue
Block a user