c111b3a9b0
Phase 01 (LLM Tuning): - Gemma4 26B: 74.65 t/s (fast) - Qwen 35B: 61.62 t/s (balanced) - Gemma4 31B: 16.0 t/s (deep-coder) - Qwen 27B: 16.7 t/s (deep-logic) - Qwen 122B: 8.95 t/s (ultra, GPU 1 only) Phase 02 (API Engine): - FastAPI reverse proxy on port 8000 - /engine/switch hot-swap with 503 protection - config/engine_models.json as single source of truth - Replaced 4 individual .bat files with unified engine File cleanup: - scripts/ 85 files -> 9 + _archive/ - Root .bat files -> _archive/
47 lines
943 B
JSON
47 lines
943 B
JSON
[
|
|
{
|
|
"name": "Baseline: all expert CPU",
|
|
"avg_tps": 8.72,
|
|
"best_tps": 8.74,
|
|
"vram_gpu0": 620,
|
|
"vram_gpu1": 6493,
|
|
"vram_total": 7113,
|
|
"status": "OK"
|
|
},
|
|
{
|
|
"name": "n-cpu-moe=60 (4 layers expert GPU)",
|
|
"avg_tps": 8.72,
|
|
"best_tps": 8.77,
|
|
"vram_gpu0": 638,
|
|
"vram_gpu1": 6493,
|
|
"vram_total": 7131,
|
|
"status": "OK"
|
|
},
|
|
{
|
|
"name": "n-cpu-moe=56 (8 layers expert GPU)",
|
|
"avg_tps": 8.72,
|
|
"best_tps": 8.8,
|
|
"vram_gpu0": 624,
|
|
"vram_gpu1": 6493,
|
|
"vram_total": 7117,
|
|
"status": "OK"
|
|
},
|
|
{
|
|
"name": "n-cpu-moe=52 (12 layers expert GPU)",
|
|
"avg_tps": 8.76,
|
|
"best_tps": 8.79,
|
|
"vram_gpu0": 634,
|
|
"vram_gpu1": 6493,
|
|
"vram_total": 7127,
|
|
"status": "OK"
|
|
},
|
|
{
|
|
"name": "n-cpu-moe=48 (16 layers expert GPU)",
|
|
"avg_tps": 8.81,
|
|
"best_tps": 8.95,
|
|
"vram_gpu0": 632,
|
|
"vram_gpu1": 6493,
|
|
"vram_total": 7125,
|
|
"status": "OK"
|
|
}
|
|
] |