Variet/variet_llm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
26 Commits 1 Branch 0 Tags
e02626fda8c41610017bf2220f9700a94025a15a
Commit Graph

23 Commits

Author SHA1 Message Date
Variet-Worker
e02626fda8 chore(session): pause work — Qwen promoted to primary + Hermes v0.8.0 
Engine:
- default_role: fast → balanced (Qwen 3.5 35B primary)
- balanced: remove -t/-tb (no impact with -ngl 999)

Hermes Agent submodule:
- Update fff237e1 → e902e55b (v0.8.0, 340 commits merged)
- Local 8 file patches auto-merged (stash/pop, 0 conflicts)
- mmproj CPU offload, DISCORD_HOME_CHANNEL fix via external ~/.hermes/config.yaml

Decisions:
- Speculative decoding experiment rejected (+14% gen vs -31% cold start)
- llama.cpp kept at b8660 (b8757 has 9% Gemma 4 regression)
- Qwen superior for thinking/Korean/coding; speed diff negligible

Session handoff: .planning/HANDOFF.json

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-11 18:14:41 +09:00
Variet-Worker
0dee779a73 refactor(phase-01): v3 retune fast & balanced roles 
fast (Gemma 4 26B-A4B):
- Enable mmproj GPU loading (vision ~1s, 12x faster than CPU)
- KV f16 → q8_0 (save ~2.5 GB VRAM for mmproj)
- Tensor split 0.5,0.5 → 0.43,0.57 (13/17 layers)
- Remove --mlock/--poll/--prio/-t/-tb (no measurable impact)
- measured_tps 74.65 → 71.89 (trade 3.7% speed for vision)

balanced (Qwen 3.5 35B-A3B):
- Tensor split 0.5,0.5 → 0.48,0.52 (enables pipeline parallelism)
- Ubatch 128 → 256 (prefill +78%: 649 → 1,157 t/s)
- mmproj + --no-mmproj-offload (CPU vision, VRAM headroom)
- Remove useless flags same as fast
- measured_tps 61.62 → 64.16 (+4.1%)

Other:
- Document full retuning in docs/v3_{fast,balanced}_retuning_log.md
- Session report at .planning/reports/20260411-session-report.md
- Add bench utilities: bench_short/bench_long/test_ts_ratios
- Speculative decoding (E2B draft) experimented but rejected
  (+14% gen vs -31% cold start + tokenizer mismatch + mmproj conflict)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-11 14:55:27 +09:00
Variet-Worker
219985b9ce wip: [06-install-and-evaluate-hermes-agent] paused at task 5/5 2026-04-09 21:55:23 +09:00
Variet-Worker
bef6c410cc wip: 06-install-and-evaluate-hermes-agent paused at task 4/4 2026-04-08 23:15:21 +09:00
Variet-Worker
96c91cb57a feat(phase-06): complete Hermes Agent windows fixes & deployment 2026-04-08 23:04:20 +09:00
Variet-Worker
e1fef677d7 wip: 06-install-and-evaluate-hermes-agent paused at task 4/4 2026-04-08 17:56:34 +09:00
Variet-Worker
ff06664441 docs(05): create VS Code Extension Packaging plan 2026-04-08 10:58:22 +09:00
Variet-Worker
6014dec75b wip: milestone v1.1 paused at phase 05 (2/3 complete) 2026-04-07 22:42:34 +09:00
Variet-Worker
7f0c688f11 docs(phase-04): complete phase execution 2026-04-07 22:40:01 +09:00
Variet-Worker
8d2ce634e8 feat(phase-04): agentRouting config + hot-swap + agent loop verified 2026-04-07 22:38:06 +09:00
Variet-Worker
5f02557313 docs(phase-03): complete phase execution 2026-04-07 21:25:25 +09:00
Variet-Worker
8c4bb0e614 feat(phase-03): complete CLI build and provider connection 2026-04-07 21:24:55 +09:00
Variet-Worker
543e7f6735 docs(phase-03): create CLI build & provider connection plan 2026-04-07 21:01:13 +09:00
Variet-Worker
2ac47dbe2e docs: create milestone v1.1 roadmap (3 phases) 2026-04-07 20:58:12 +09:00
Variet-Worker
36521d7853 docs: define milestone v1.1 requirements 2026-04-07 20:57:07 +09:00
Variet-Worker
7e39c002b6 docs: start milestone v1.1 OpenClaude CLI Integration 2026-04-07 20:39:36 +09:00
Variet-Worker
c111b3a9b0 feat: Variet Engine v1.0 + 5-model tuning complete 
Phase 01 (LLM Tuning):
- Gemma4 26B: 74.65 t/s (fast)
- Qwen 35B: 61.62 t/s (balanced)
- Gemma4 31B: 16.0 t/s (deep-coder)
- Qwen 27B: 16.7 t/s (deep-logic)
- Qwen 122B: 8.95 t/s (ultra, GPU 1 only)

Phase 02 (API Engine):
- FastAPI reverse proxy on port 8000
- /engine/switch hot-swap with 503 protection
- config/engine_models.json as single source of truth
- Replaced 4 individual .bat files with unified engine

File cleanup:
- scripts/ 85 files -> 9 + _archive/
- Root .bat files -> _archive/
 2026-04-07 18:08:58 +09:00
Variet-Worker
626a089b6b wip: [01-llm-tuning] paused at task 3/5 2026-04-06 21:17:39 +09:00
Variet-Worker
a09736e930 wip: [01-llm-tuning] paused at task 1/3 2026-04-05 22:59:01 +09:00
Variet-Worker
66778b750d wip: initialization paused 2026-04-05 00:52:46 +09:00
Variet-Worker
e37f65af80 docs: initialize Variet LLM project 2026-04-05 00:49:30 +09:00
Variet-Worker
93f4182384 chore: add Variet LLM project config 2026-04-05 00:45:54 +09:00
Variet-Worker
7890ff6644 chore: initial project setup with agent guide 2026-04-05 00:43:39 +09:00