diff --git a/Phase-01-LLM-Tuning.-.md b/Phase-01-LLM-Tuning.-.md new file mode 100644 index 0000000..a81d134 --- /dev/null +++ b/Phase-01-LLM-Tuning.-.md @@ -0,0 +1,57 @@ +# Phase 01 — LLM Tuning + +## 5 모델 확정 성능 (v3 재튜닝 포함) + +| # | 모델 | 역할 | 측정 t/s | 컨텍스트 | 비고 | +|:-:|------|------|:--------:|:-------:|------| +| 1 | Gemma 4 26B-A4B Q4_K_M | fast | 71.89 | 256K | Vision GPU (~1s) | +| 2 | Qwen 3.5 35B-A3B Q4_K_M | **balanced (primary)** | 64.16 | 256K | Vision CPU (6.4s), thinking mode | +| 3 | Gemma 4 31B Dense Q4_K_M | deep-coder | 16.0 | 192K | 코딩 특화 | +| 4 | Qwen 3.5 27B Dense Q4_K_M | deep-logic | 16.7 | 256K | 깊은 추론 | +| 5 | Qwen 3.5 122B-A10B Q4_K_M | ultra | 8.95 | 256K | GPU 1 단독 | + +## Hardware Asymmetry + +``` +GPU 0 PCIe 3.0 x4 (3.94 GB/s) ← bottleneck +GPU 1 PCIe 4.0 x16 (31.5 GB/s) +Total VRAM: 24 GB +``` + +GPU 0의 PCIe x4 제약이 이중 GPU 추론 속도 상한을 결정합니다. + +## v3 재튜닝 변경사항 (2026-04-11) + +### balanced (Qwen 3.5 35B-A3B) +- `-ub 128 → 256` (prefill +78%: 649 → 1,157 t/s) +- `-ts 0.5,0.5 → 0.48,0.52` (PP 활성화) +- `--mmproj models/mmproj-F16.gguf` + `--no-mmproj-offload` (비전 CPU) +- `--mlock/--poll/--prio/-t/-tb` 제거 + +### fast (Gemma 4 26B-A4B) +- `cache-type: f16 → q8_0` (VRAM 2.5 GB 절약) +- `-ts 0.43,0.57` (13/17 layer split) +- `--mmproj models/gemma-4-26B-mmproj-F16.gguf` (Vision GPU) +- `--mlock/--poll/--prio/-t/-tb` 제거 + +## 아키텍처 발견 + +### Qwen 3.5 35B-A3B (Gated Delta Net) +- 40 레이어 중 **10개만 full attention** (매 4번째) +- 나머지 30개는 SSM (Mamba-like) 레이어 +- KV 캐시 실제 크기: 1.4 GB (q4_0, 256K) — 초기 추정(5GB) 대비 작음 + +### Gemma 4 26B-A4B +- 30 레이어 중 **5개만 full attention** (매 6번째) +- 나머지 25개는 SWA (Sliding Window, 1,536 cells) +- 128 experts / 8 active (MoE) + +## Reference + +- [docs/v3_balanced_retuning_log.md](../../docs/v3_balanced_retuning_log.md) +- [docs/v3_fast_retuning_log.md](../../docs/v3_fast_retuning_log.md) +- [.planning/phases/01-llm-tuning/VERIFICATION.md](../../.planning/phases/01-llm-tuning/VERIFICATION.md) + +## Status + +**Phase 01**: ✅ COMPLETE (2026-04-07 초기, 2026-04-11 재검증)