diff --git a/.agents/references/known-issues.md b/.agents/references/known-issues.md index a918863..a1239b6 100644 --- a/.agents/references/known-issues.md +++ b/.agents/references/known-issues.md @@ -172,7 +172,17 @@ - `ResolveOutstandingSteps` → `run state not found` (500 에러, 실제로는 CANCEL 동작) - `sendChatActionMessage`, `executeCascadeAction` → 119개 명령 중 미등록 - 존재하는 approval-like 명령: `agentAcceptAllInFile` (코드 diff), `agentAcceptFocusedHunk` (hunk), `acceptCompletion` (자동완성) — 터미널 승인과 무관 -- **해결**: Renderer DOM Click 구현됨 (미검증). Extension→Renderer HTTP `/trigger-click` 엔드포인트. **AG 완전 재시작 필요** (Reload Window 불가). 실패 시 → `TerminalExecutionPolicy.EAGER` 탐색 -- **주의**: `agentPanel.focus`도 미등록, `agentSidePanel.focus`만 존재. Renderer가 webview iframe 내부 버튼에 접근 가능한지 미확인 +- **해결**: ~~Renderer DOM Click 구현됨 (미검증)~~ → **v1 검증 실패: webview iframe 격리 확인**. v3 `deepFindButtons()`로 업그레이드 (iframe contentDocument + webview.executeJavaScript + shadow DOM). AG 완전 재시작 후 DOM-DUMP로 접근 가능 여부 확인 필요 +- **주의**: `agentPanel.focus`도 미등록, `agentSidePanel.focus`만 존재 +### [2026-03-09] Renderer DOM — webview iframe 격리 확인 + v3 deep traversal +- **증상**: Renderer trigger-click이 `document.querySelectorAll('button')`으로 버튼 검색 → Run 버튼 미발견. 감지된 것은 외부 DOM의 trust-level 버튼(`RunAlt+?`)뿐 +- **원인**: Run/Accept 버튼은 AG 채팅 webview iframe (`vscode-webview://` origin) 안에 렌더링. 외부 workbench DOM (`vscode-file://` origin)에서 cross-origin으로 접근 불가 +- **해결**: Renderer v3 `deepFindButtons()` 구현: + 1. Main document 검색 (기존) + 2. `iframe.contentDocument` 접근 시도 (same-origin이면 성공) + 3. `.executeJavaScript()` 접근 시도 (Electron API) + 4. Shadow DOM 재귀 탐색 + → **미검증** (AG 재시작 후 DOM-DUMP 결과 필요) +- **주의**: 커뮤니티 auto-accept 확장들은 CDP(Chrome DevTools Protocol)를 사용하지만, 이는 `--remote-debugging-port` 플래그가 필요한 비표준 접근. 먼저 표준 DOM API로 관통 가능한지 확인 diff --git a/docs/devlog/2026-03-09.md b/docs/devlog/2026-03-09.md index 5fb9a9f..009db55 100644 --- a/docs/devlog/2026-03-09.md +++ b/docs/devlog/2026-03-09.md @@ -4,3 +4,4 @@ |---|------|----------|------|------| | 001 | 08:00~09:17 | 승인 실행 메커니즘 연구 + step-type별 VS Code 명령 분기 구현 | included in 002 | 🔧 | | 002 | 09:21~15:07 | SDK 승인 명령 미등록 확정 + Renderer DOM Click 구현 | `4497e96` | 🔧 | +| 003 | 15:32~17:59 | Renderer v3 deep DOM traversal (iframe/webview/shadow 관통) | pending | 🔧 | diff --git a/docs/devlog/entries/20260309-003.md b/docs/devlog/entries/20260309-003.md new file mode 100644 index 0000000..d7362d7 --- /dev/null +++ b/docs/devlog/entries/20260309-003.md @@ -0,0 +1,46 @@ +# Renderer v3 — webview iframe 격리 확인 + deep DOM traversal + +- **시작**: 15:32 KST +- **종료**: 17:59 KST +- **상태**: 🔧 미완료 (AG 재시작 후 DOM-DUMP 결과 확인 필요) + +## 핵심 발견 + +### webview iframe 격리 확인 +- Bridge HTTP 서버(port 34332) 정상 동작: `/ping→pong`, `/trigger-click→{action:null}` +- Extension log 분석: Renderer DOM Observer가 감지한 버튼들은 `RunAlt+?` 등 외부 DOM trust-level 버튼 +- **Run/Accept 버튼은 webview iframe (`vscode-webview://` origin) 안에 존재** +- 외부 workbench DOM (`vscode-file://` origin)에서는 cross-origin으로 접근 불가 + +### v3 deep DOM traversal 구현 +`generateApprovalObserverScript()` 전면 업그레이드: + +1. **`deepFindButtons()`** — 패턴 매칭 버튼을 재귀적으로 검색: + - Main document (기존) + - iframe `contentDocument` (same-origin이면 접근 가능) + - `.executeJavaScript()` (Electron API — webview 내부 코드 실행) + - Shadow DOM 재귀 탐색 + +2. **`dumpDOMStructure()`** — 시작 3초 후 1회 실행: + - 모든 iframe 목록 + 접근성 (ACCESSIBLE/BLOCKED) + - 모든 `` 목록 + 접근성 + - 메인 DOM 버튼 목록 (처음 10개) + +3. **3-Phase trigger-click**: + - Phase 1: deep DOM search (main + iframes + shadow) + - Phase 2: `.executeJavaScript()` (DOM 미접근 시 fallback) + - Phase 3: iframe direct access retry + +## 다음 단계 + +1. **AG 완전 재시작** → Renderer v3 스크립트 로딩 확인 +2. DevTools에서 `[GB Observer] DOM-DUMP:` 로그 확인 +3. iframe/webview 접근성 확인 → ACCESSIBLE이면 즉시 E2E 테스트 +4. BLOCKED이면 다음 대안 탐색 + +## 파일 변경 + +| 파일 | 변경 | +|------|------| +| `extension/src/extension.ts` | v3 renderer: deepFindButtons, dumpDOMStructure, 3-phase trigger-click | +| `.agents/references/known-issues.md` | webview iframe 격리 확인 + v3 기록 | diff --git a/extension/out/extension.js b/extension/out/extension.js index 418deb9..ae08693 100644 --- a/extension/out/extension.js +++ b/extension/out/extension.js @@ -574,16 +574,122 @@ function startObserverHttpBridge() { function generateApprovalObserverScript(_port) { // Port is hardcoded as fallback, but renderer also reads ag-bridge-ports.json for multi-bridge return ` -// ── Gravity Bridge v2: Approval Observer (MutationObserver-first, throttled) ── +// ── Gravity Bridge v3: Approval Observer (deep DOM traversal — iframes, webviews, shadow DOMs) ── (function(){ 'use strict'; var BASE='',_obs=false,_sent={},_ready=false; var _scanScheduled=false,_lastScanTs=0; var THROTTLE_MS=100; var CLEANUP_MS=300000; + var _domDumped=false; function log(m){console.log('[GB Observer] '+m);} - log('v2 Script loaded — discovering bridge port...'); + log('v3 Script loaded — deep DOM traversal enabled'); + + // ── Deep DOM Traversal: find buttons across ALL boundaries ── + // Searches: main document → iframes (contentDocument) → webview elements → shadow DOMs + function deepFindButtons(patterns){ + var results=[]; + // 1. Main document buttons + collectButtons(document,results,patterns,'main'); + // 2. Iframe traversal (try contentDocument — works if same-origin or webSecurity off) + var iframes=document.querySelectorAll('iframe'); + for(var i=0;i tag — has executeJavaScript) + var webviews=document.querySelectorAll('webview'); + for(var w=0;wshadow'); + } + }catch(e){} + } + + // ── DOM Structure Dump (one-time on startup) ── + function dumpDOMStructure(){ + if(_domDumped)return; + _domDumped=true; + try{ + // Count iframes + var iframes=document.querySelectorAll('iframe'); + log('DOM-DUMP: '+iframes.length+' iframes found'); + for(var i=0;i elements'); + for(var w=0;w0){ + log(emoji+' TRIGGER-CLICK: clicking "'+found[0].text+'" from '+found[0].source); + found[0].btn.click(); + return; + } + + // Phase 2: Try .executeJavaScript for inaccessible webviews + var webviews=document.querySelectorAll('webview'); + if(webviews.length>0){ + log('TRIGGER-CLICK: trying '+webviews.length+' webview(s) via executeJavaScript...'); + var patternsStr=patterns.map(function(re){return re.source;}).join('|'); + var clickScript='(function(){'+ + 'var re=new RegExp("'+patternsStr+'","i");'+ + 'var btns=document.querySelectorAll("button");'+ + 'for(var i=0;i0){ + log('TRIGGER-CLICK: trying '+iframes.length+' iframe(s) — checking accessibility...'); + var clickedAny=false; + for(var fi=0;fi { function generateApprovalObserverScript(_port: number): string { // Port is hardcoded as fallback, but renderer also reads ag-bridge-ports.json for multi-bridge return ` -// ── Gravity Bridge v2: Approval Observer (MutationObserver-first, throttled) ── +// ── Gravity Bridge v3: Approval Observer (deep DOM traversal — iframes, webviews, shadow DOMs) ── (function(){ 'use strict'; var BASE='',_obs=false,_sent={},_ready=false; var _scanScheduled=false,_lastScanTs=0; var THROTTLE_MS=100; var CLEANUP_MS=300000; + var _domDumped=false; function log(m){console.log('[GB Observer] '+m);} - log('v2 Script loaded — discovering bridge port...'); + log('v3 Script loaded — deep DOM traversal enabled'); + + // ── Deep DOM Traversal: find buttons across ALL boundaries ── + // Searches: main document → iframes (contentDocument) → webview elements → shadow DOMs + function deepFindButtons(patterns){ + var results=[]; + // 1. Main document buttons + collectButtons(document,results,patterns,'main'); + // 2. Iframe traversal (try contentDocument — works if same-origin or webSecurity off) + var iframes=document.querySelectorAll('iframe'); + for(var i=0;i tag — has executeJavaScript) + var webviews=document.querySelectorAll('webview'); + for(var w=0;wshadow'); + } + }catch(e){} + } + + // ── DOM Structure Dump (one-time on startup) ── + function dumpDOMStructure(){ + if(_domDumped)return; + _domDumped=true; + try{ + // Count iframes + var iframes=document.querySelectorAll('iframe'); + log('DOM-DUMP: '+iframes.length+' iframes found'); + for(var i=0;i elements'); + for(var w=0;w0){ + log(emoji+' TRIGGER-CLICK: clicking "'+found[0].text+'" from '+found[0].source); + found[0].btn.click(); + return; + } + + // Phase 2: Try .executeJavaScript for inaccessible webviews + var webviews=document.querySelectorAll('webview'); + if(webviews.length>0){ + log('TRIGGER-CLICK: trying '+webviews.length+' webview(s) via executeJavaScript...'); + var patternsStr=patterns.map(function(re){return re.source;}).join('|'); + var clickScript='(function(){'+ + 'var re=new RegExp("'+patternsStr+'","i");'+ + 'var btns=document.querySelectorAll("button");'+ + 'for(var i=0;i0){ + log('TRIGGER-CLICK: trying '+iframes.length+' iframe(s) — checking accessibility...'); + var clickedAny=false; + for(var fi=0;fi