diff --git a/Phase-01-LLM-Tuning.md b/Phase-01-LLM-Tuning.md new file mode 100644 index 0000000..a81d134 --- /dev/null +++ b/Phase-01-LLM-Tuning.md @@ -0,0 +1,57 @@ +# Phase 01 — LLM Tuning + +## 5 모델 확정 성능 (v3 재튜닝 포함) + +| # | 모델 | 역할 | 측정 t/s | 컨텍스트 | 비고 | +|:-:|------|------|:--------:|:-------:|------| +| 1 | Gemma 4 26B-A4B Q4_K_M | fast | 71.89 | 256K | Vision GPU (~1s) | +| 2 | Qwen 3.5 35B-A3B Q4_K_M | **balanced (primary)** | 64.16 | 256K | Vision CPU (6.4s), thinking mode | +| 3 | Gemma 4 31B Dense Q4_K_M | deep-coder | 16.0 | 192K | 코딩 특화 | +| 4 | Qwen 3.5 27B Dense Q4_K_M | deep-logic | 16.7 | 256K | 깊은 추론 | +| 5 | Qwen 3.5 122B-A10B Q4_K_M | ultra | 8.95 | 256K | GPU 1 단독 | + +## Hardware Asymmetry + +``` +GPU 0 PCIe 3.0 x4 (3.94 GB/s) ← bottleneck +GPU 1 PCIe 4.0 x16 (31.5 GB/s) +Total VRAM: 24 GB +``` + +GPU 0의 PCIe x4 제약이 이중 GPU 추론 속도 상한을 결정합니다. + +## v3 재튜닝 변경사항 (2026-04-11) + +### balanced (Qwen 3.5 35B-A3B) +- `-ub 128 → 256` (prefill +78%: 649 → 1,157 t/s) +- `-ts 0.5,0.5 → 0.48,0.52` (PP 활성화) +- `--mmproj models/mmproj-F16.gguf` + `--no-mmproj-offload` (비전 CPU) +- `--mlock/--poll/--prio/-t/-tb` 제거 + +### fast (Gemma 4 26B-A4B) +- `cache-type: f16 → q8_0` (VRAM 2.5 GB 절약) +- `-ts 0.43,0.57` (13/17 layer split) +- `--mmproj models/gemma-4-26B-mmproj-F16.gguf` (Vision GPU) +- `--mlock/--poll/--prio/-t/-tb` 제거 + +## 아키텍처 발견 + +### Qwen 3.5 35B-A3B (Gated Delta Net) +- 40 레이어 중 **10개만 full attention** (매 4번째) +- 나머지 30개는 SSM (Mamba-like) 레이어 +- KV 캐시 실제 크기: 1.4 GB (q4_0, 256K) — 초기 추정(5GB) 대비 작음 + +### Gemma 4 26B-A4B +- 30 레이어 중 **5개만 full attention** (매 6번째) +- 나머지 25개는 SWA (Sliding Window, 1,536 cells) +- 128 experts / 8 active (MoE) + +## Reference + +- [docs/v3_balanced_retuning_log.md](../../docs/v3_balanced_retuning_log.md) +- [docs/v3_fast_retuning_log.md](../../docs/v3_fast_retuning_log.md) +- [.planning/phases/01-llm-tuning/VERIFICATION.md](../../.planning/phases/01-llm-tuning/VERIFICATION.md) + +## Status + +**Phase 01**: ✅ COMPLETE (2026-04-07 초기, 2026-04-11 재검증) diff --git a/Session-2026-04-11.md b/Session-2026-04-11.md new file mode 100644 index 0000000..78ce524 --- /dev/null +++ b/Session-2026-04-11.md @@ -0,0 +1,67 @@ +# Session 2026-04-11 — Phase 01 Retuning & Hermes Agent v0.8.0 + +**Type**: Maintenance +**Focus**: LLM tuning re-verification, Hermes Agent update, primary model switch + +--- + +## Summary + +Phase 01 (LLM Tuning) 재검증 세션과 Hermes Agent 주요 업데이트를 병행하여 진행. 두 LLM 역할 최적화 파라미터 갱신 및 Qwen 3.5 35B를 메인 모델로 승격, Hermes Agent v0.7.x → v0.8.0 업데이트 완료. + +## Key Outcomes + +### LLM Engine +- **balanced** (Qwen 3.5 35B-A3B): 61.62 → **64.16 t/s** (+4.1%) +- **fast** (Gemma 4 26B-A4B): 74.65 → **71.89 t/s** (Vision GPU 추가 trade-off) +- Default role: `fast` → `balanced` +- 두 역할 모두 `--mlock/--poll/--prio/-t/-tb` 제거 + +### Qwen Primary Promotion +- 속도 차이 1.25 t/s (negligible) +- 35B > 26B 품질 우위 +- Thinking mode + 한국어/코딩 강점 +- Vision CPU 오프로드 수용 (6.4s/image) + +### Hermes Agent +- 버전: `fff237e1` → `e902e55b` (340 commits, v0.8.0) +- 로컬 8개 패치 자동 병합 (0 conflict) +- 설정: `custom/qwen3.5-35b-a3b` + `DISCORD_HOME_CHANNEL` 수동 설정 + +## Experiments (Not Adopted) + +- **Speculative Decoding** (E2B draft): +14% gen vs -31% cold start → 채택 안 함 +- **llama.cpp b8757**: Gemma 4 9% 회귀 → b8660 유지 + +## Files Changed + +- `config/engine_models.json` +- `docs/v3_balanced_retuning_log.md` (new) +- `docs/v3_fast_retuning_log.md` (new) +- `.planning/reports/20260411-session-report.md` (new) +- `.planning/phases/01-llm-tuning/VERIFICATION.md` (updated) +- `.planning/STATE.md` (updated) +- `.planning/HANDOFF.json` (updated) +- `agents/hermes-agent` (submodule bump) +- `scripts/bench_short.py`, `bench_long.py`, `test_ts_ratios.py` (new utilities) + +## Git History + +``` +e02626f chore(session): pause work — Qwen promoted to primary + Hermes v0.8.0 +0dee779 refactor(phase-01): v3 retune fast & balanced roles +``` + +## Hardware Constraints Documented + +- GPU 0: PCIe 3.0 x4 (3.94 GB/s) — bottleneck +- GPU 1: PCIe 4.0 x16 (31.5 GB/s) +- Total VRAM: 24 GB (2x RTX 3060 12GB) + +## Next Session + +- Run `/gsd-resume-work` to reload state from `HANDOFF.json` +- Options: + - Start Hermes Agent (`run_hermes_agent.bat`) + - Resume Phase 05 (VS Code Extension Packaging) + - New feature development