feat(bridge): renderer v3 deep DOM traversal (iframe/webview/shadow) #task-255
- deepFindButtons(): traverse iframe contentDocument, webview.executeJavaScript, shadow DOMs - dumpDOMStructure(): startup diagnostic dump of all iframes/webviews/buttons - 3-phase trigger-click: deep DOM → webview execJS → iframe direct - known-issues: webview iframe isolation confirmed, v3 solution documented
This commit is contained in:
@@ -172,7 +172,17 @@
|
||||
- `ResolveOutstandingSteps` → `run state not found` (500 에러, 실제로는 CANCEL 동작)
|
||||
- `sendChatActionMessage`, `executeCascadeAction` → 119개 명령 중 미등록
|
||||
- 존재하는 approval-like 명령: `agentAcceptAllInFile` (코드 diff), `agentAcceptFocusedHunk` (hunk), `acceptCompletion` (자동완성) — 터미널 승인과 무관
|
||||
- **해결**: Renderer DOM Click 구현됨 (미검증). Extension→Renderer HTTP `/trigger-click` 엔드포인트. **AG 완전 재시작 필요** (Reload Window 불가). 실패 시 → `TerminalExecutionPolicy.EAGER` 탐색
|
||||
- **주의**: `agentPanel.focus`도 미등록, `agentSidePanel.focus`만 존재. Renderer가 webview iframe 내부 버튼에 접근 가능한지 미확인
|
||||
- **해결**: ~~Renderer DOM Click 구현됨 (미검증)~~ → **v1 검증 실패: webview iframe 격리 확인**. v3 `deepFindButtons()`로 업그레이드 (iframe contentDocument + webview.executeJavaScript + shadow DOM). AG 완전 재시작 후 DOM-DUMP로 접근 가능 여부 확인 필요
|
||||
- **주의**: `agentPanel.focus`도 미등록, `agentSidePanel.focus`만 존재
|
||||
|
||||
### [2026-03-09] Renderer DOM — webview iframe 격리 확인 + v3 deep traversal
|
||||
- **증상**: Renderer trigger-click이 `document.querySelectorAll('button')`으로 버튼 검색 → Run 버튼 미발견. 감지된 것은 외부 DOM의 trust-level 버튼(`RunAlt+?`)뿐
|
||||
- **원인**: Run/Accept 버튼은 AG 채팅 webview iframe (`vscode-webview://` origin) 안에 렌더링. 외부 workbench DOM (`vscode-file://` origin)에서 cross-origin으로 접근 불가
|
||||
- **해결**: Renderer v3 `deepFindButtons()` 구현:
|
||||
1. Main document 검색 (기존)
|
||||
2. `iframe.contentDocument` 접근 시도 (same-origin이면 성공)
|
||||
3. `<webview>.executeJavaScript()` 접근 시도 (Electron API)
|
||||
4. Shadow DOM 재귀 탐색
|
||||
→ **미검증** (AG 재시작 후 DOM-DUMP 결과 필요)
|
||||
- **주의**: 커뮤니티 auto-accept 확장들은 CDP(Chrome DevTools Protocol)를 사용하지만, 이는 `--remote-debugging-port` 플래그가 필요한 비표준 접근. 먼저 표준 DOM API로 관통 가능한지 확인
|
||||
|
||||
|
||||
@@ -4,3 +4,4 @@
|
||||
|---|------|----------|------|------|
|
||||
| 001 | 08:00~09:17 | 승인 실행 메커니즘 연구 + step-type별 VS Code 명령 분기 구현 | included in 002 | 🔧 |
|
||||
| 002 | 09:21~15:07 | SDK 승인 명령 미등록 확정 + Renderer DOM Click 구현 | `4497e96` | 🔧 |
|
||||
| 003 | 15:32~17:59 | Renderer v3 deep DOM traversal (iframe/webview/shadow 관통) | pending | 🔧 |
|
||||
|
||||
46
docs/devlog/entries/20260309-003.md
Normal file
46
docs/devlog/entries/20260309-003.md
Normal file
@@ -0,0 +1,46 @@
|
||||
# Renderer v3 — webview iframe 격리 확인 + deep DOM traversal
|
||||
|
||||
- **시작**: 15:32 KST
|
||||
- **종료**: 17:59 KST
|
||||
- **상태**: 🔧 미완료 (AG 재시작 후 DOM-DUMP 결과 확인 필요)
|
||||
|
||||
## 핵심 발견
|
||||
|
||||
### webview iframe 격리 확인
|
||||
- Bridge HTTP 서버(port 34332) 정상 동작: `/ping→pong`, `/trigger-click→{action:null}`
|
||||
- Extension log 분석: Renderer DOM Observer가 감지한 버튼들은 `RunAlt+?` 등 외부 DOM trust-level 버튼
|
||||
- **Run/Accept 버튼은 webview iframe (`vscode-webview://` origin) 안에 존재**
|
||||
- 외부 workbench DOM (`vscode-file://` origin)에서는 cross-origin으로 접근 불가
|
||||
|
||||
### v3 deep DOM traversal 구현
|
||||
`generateApprovalObserverScript()` 전면 업그레이드:
|
||||
|
||||
1. **`deepFindButtons()`** — 패턴 매칭 버튼을 재귀적으로 검색:
|
||||
- Main document (기존)
|
||||
- iframe `contentDocument` (same-origin이면 접근 가능)
|
||||
- `<webview>.executeJavaScript()` (Electron API — webview 내부 코드 실행)
|
||||
- Shadow DOM 재귀 탐색
|
||||
|
||||
2. **`dumpDOMStructure()`** — 시작 3초 후 1회 실행:
|
||||
- 모든 iframe 목록 + 접근성 (ACCESSIBLE/BLOCKED)
|
||||
- 모든 `<webview>` 목록 + 접근성
|
||||
- 메인 DOM 버튼 목록 (처음 10개)
|
||||
|
||||
3. **3-Phase trigger-click**:
|
||||
- Phase 1: deep DOM search (main + iframes + shadow)
|
||||
- Phase 2: `<webview>.executeJavaScript()` (DOM 미접근 시 fallback)
|
||||
- Phase 3: iframe direct access retry
|
||||
|
||||
## 다음 단계
|
||||
|
||||
1. **AG 완전 재시작** → Renderer v3 스크립트 로딩 확인
|
||||
2. DevTools에서 `[GB Observer] DOM-DUMP:` 로그 확인
|
||||
3. iframe/webview 접근성 확인 → ACCESSIBLE이면 즉시 E2E 테스트
|
||||
4. BLOCKED이면 다음 대안 탐색
|
||||
|
||||
## 파일 변경
|
||||
|
||||
| 파일 | 변경 |
|
||||
|------|------|
|
||||
| `extension/src/extension.ts` | v3 renderer: deepFindButtons, dumpDOMStructure, 3-phase trigger-click |
|
||||
| `.agents/references/known-issues.md` | webview iframe 격리 확인 + v3 기록 |
|
||||
@@ -574,16 +574,122 @@ function startObserverHttpBridge() {
|
||||
function generateApprovalObserverScript(_port) {
|
||||
// Port is hardcoded as fallback, but renderer also reads ag-bridge-ports.json for multi-bridge
|
||||
return `
|
||||
// ── Gravity Bridge v2: Approval Observer (MutationObserver-first, throttled) ──
|
||||
// ── Gravity Bridge v3: Approval Observer (deep DOM traversal — iframes, webviews, shadow DOMs) ──
|
||||
(function(){
|
||||
'use strict';
|
||||
var BASE='',_obs=false,_sent={},_ready=false;
|
||||
var _scanScheduled=false,_lastScanTs=0;
|
||||
var THROTTLE_MS=100;
|
||||
var CLEANUP_MS=300000;
|
||||
var _domDumped=false;
|
||||
|
||||
function log(m){console.log('[GB Observer] '+m);}
|
||||
log('v2 Script loaded — discovering bridge port...');
|
||||
log('v3 Script loaded — deep DOM traversal enabled');
|
||||
|
||||
// ── Deep DOM Traversal: find buttons across ALL boundaries ──
|
||||
// Searches: main document → iframes (contentDocument) → webview elements → shadow DOMs
|
||||
function deepFindButtons(patterns){
|
||||
var results=[];
|
||||
// 1. Main document buttons
|
||||
collectButtons(document,results,patterns,'main');
|
||||
// 2. Iframe traversal (try contentDocument — works if same-origin or webSecurity off)
|
||||
var iframes=document.querySelectorAll('iframe');
|
||||
for(var i=0;i<iframes.length;i++){
|
||||
try{
|
||||
var idoc=iframes[i].contentDocument||iframes[i].contentWindow.document;
|
||||
if(idoc){collectButtons(idoc,results,patterns,'iframe#'+i+'('+iframes[i].className.substring(0,30)+')');}
|
||||
}catch(e){
|
||||
// Cross-origin — can't access. Log only on first dom dump
|
||||
if(!_domDumped)log('iframe#'+i+' cross-origin: '+e.message.substring(0,60));
|
||||
}
|
||||
}
|
||||
// 3. Webview elements (Electron <webview> tag — has executeJavaScript)
|
||||
var webviews=document.querySelectorAll('webview');
|
||||
for(var w=0;w<webviews.length;w++){
|
||||
try{
|
||||
var wvDoc=webviews[w].contentDocument;
|
||||
if(wvDoc){collectButtons(wvDoc,results,patterns,'webview#'+w);}
|
||||
}catch(e){
|
||||
if(!_domDumped)log('webview#'+w+' access error: '+e.message.substring(0,60));
|
||||
}
|
||||
}
|
||||
return results;
|
||||
}
|
||||
|
||||
function collectButtons(doc,results,patterns,source){
|
||||
if(!doc||!doc.querySelectorAll)return;
|
||||
var btns=doc.querySelectorAll('button');
|
||||
for(var i=0;i<btns.length;i++){
|
||||
var b=btns[i];
|
||||
if(b.disabled||b.hidden)continue;
|
||||
try{if(!b.offsetParent&&b.style.display!=='fixed')continue;}catch(e){}
|
||||
var txt=(b.textContent||'').trim();
|
||||
if(!txt)continue;
|
||||
for(var p=0;p<patterns.length;p++){
|
||||
if(patterns[p].test(txt)){
|
||||
results.push({btn:b,text:txt,source:source});
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
// 4. Recurse into shadow DOMs
|
||||
try{
|
||||
var allEls=doc.querySelectorAll('*');
|
||||
for(var j=0;j<allEls.length;j++){
|
||||
var sr=allEls[j].shadowRoot;
|
||||
if(sr)collectButtons(sr,results,patterns,source+'>shadow');
|
||||
}
|
||||
}catch(e){}
|
||||
}
|
||||
|
||||
// ── DOM Structure Dump (one-time on startup) ──
|
||||
function dumpDOMStructure(){
|
||||
if(_domDumped)return;
|
||||
_domDumped=true;
|
||||
try{
|
||||
// Count iframes
|
||||
var iframes=document.querySelectorAll('iframe');
|
||||
log('DOM-DUMP: '+iframes.length+' iframes found');
|
||||
for(var i=0;i<iframes.length;i++){
|
||||
var f=iframes[i];
|
||||
var fInfo=' iframe#'+i+' class="'+f.className.substring(0,50)+'" src="'+(f.src||'').substring(0,80)+'"';
|
||||
try{
|
||||
var idoc=f.contentDocument;
|
||||
if(idoc){
|
||||
var btns=idoc.querySelectorAll('button');
|
||||
fInfo+=' → ACCESSIBLE ('+btns.length+' buttons)';
|
||||
// Dump first 5 button texts
|
||||
for(var b=0;b<Math.min(5,btns.length);b++){
|
||||
log(' btn['+b+']: "'+((btns[b].textContent||'').trim()).substring(0,40)+'"');
|
||||
}
|
||||
}
|
||||
}catch(e){
|
||||
fInfo+=' → BLOCKED ('+e.message.substring(0,40)+')';
|
||||
}
|
||||
log(fInfo);
|
||||
}
|
||||
// Count webview elements
|
||||
var webviews=document.querySelectorAll('webview');
|
||||
log('DOM-DUMP: '+webviews.length+' <webview> elements');
|
||||
for(var w=0;w<webviews.length;w++){
|
||||
var wInfo=' webview#'+w+' src="'+(webviews[w].src||'').substring(0,80)+'"';
|
||||
try{
|
||||
var wdoc=webviews[w].contentDocument;
|
||||
if(wdoc)wInfo+=' → ACCESSIBLE';
|
||||
else wInfo+=' → null contentDocument';
|
||||
}catch(e){wInfo+=' → BLOCKED ('+e.message.substring(0,40)+')';}
|
||||
log(wInfo);
|
||||
}
|
||||
// Main doc buttons
|
||||
var mainBtns=document.querySelectorAll('button');
|
||||
log('DOM-DUMP: '+mainBtns.length+' buttons in main document');
|
||||
for(var m=0;m<Math.min(10,mainBtns.length);m++){
|
||||
log(' main-btn['+m+']: "'+((mainBtns[m].textContent||'').trim()).substring(0,50)+'"');
|
||||
}
|
||||
// Report to bridge
|
||||
fetch(BASE+'/ping?dom_dump='+iframes.length+'f_'+webviews.length+'wv_'+mainBtns.length+'btn').catch(function(){});
|
||||
}catch(e){log('DOM-DUMP error: '+e.message);}
|
||||
}
|
||||
|
||||
// ── Port Discovery: async fetch()-based (sync XHR blocked in Electron renderer) ──
|
||||
var HARDCODED_PORT=${_port};
|
||||
@@ -615,7 +721,7 @@ function generateApprovalObserverScript(_port) {
|
||||
discoverPort(function(port){
|
||||
BASE='http://127.0.0.1:'+port;
|
||||
fetch(BASE+'/ping').then(function(r){return r.text();}).then(function(t){
|
||||
if(t==='pong'){log('Bridge connected on port '+port);_ready=true;startObserver();}
|
||||
if(t==='pong'){log('Bridge connected on port '+port);_ready=true;startObserver();setTimeout(dumpDOMStructure,3000);}
|
||||
else log('Bridge ping failed: '+t);
|
||||
}).catch(function(e){log('Bridge unreachable: '+e.message);});
|
||||
});
|
||||
@@ -871,49 +977,100 @@ function generateApprovalObserverScript(_port) {
|
||||
|
||||
// ── TRIGGER-CLICK: Extension→Renderer bridge for programmatic button clicks ──
|
||||
// Extension sets clickTrigger via tryApprovalStrategies → renderer polls and clicks
|
||||
// v3: uses deepFindButtons() to traverse iframes, webviews, shadow DOMs
|
||||
setInterval(function(){
|
||||
if(!_ready||!BASE)return;
|
||||
fetch(BASE+'/trigger-click?t='+Date.now()).then(function(r){return r.json();}).then(function(d){
|
||||
if(!d.action)return;
|
||||
log('🔔 TRIGGER-CLICK received: action='+d.action);
|
||||
// Find first visible approve or reject button
|
||||
|
||||
var approveRe=[/^Run$/i,/^Run /i,/^Accept/i,/^Allow/i,/^Approve/i,/^Continue$/i,/^Proceed$/i];
|
||||
var rejectRe=[/^Reject/i,/^Cancel$/i,/^Deny$/i,/^Stop$/i,/^Decline$/i];
|
||||
var patterns=(d.action==='approve')?approveRe:rejectRe;
|
||||
var emoji=(d.action==='approve')?'✅':'❌';
|
||||
|
||||
// Phase 1: deepFindButtons in main doc + accessible iframes + shadow DOMs
|
||||
var found=deepFindButtons(patterns);
|
||||
if(found.length>0){
|
||||
log(emoji+' TRIGGER-CLICK: clicking "'+found[0].text+'" from '+found[0].source);
|
||||
found[0].btn.click();
|
||||
return;
|
||||
}
|
||||
|
||||
// Phase 2: Try <webview>.executeJavaScript for inaccessible webviews
|
||||
var webviews=document.querySelectorAll('webview');
|
||||
if(webviews.length>0){
|
||||
log('TRIGGER-CLICK: trying '+webviews.length+' webview(s) via executeJavaScript...');
|
||||
var patternsStr=patterns.map(function(re){return re.source;}).join('|');
|
||||
var clickScript='(function(){'+
|
||||
'var re=new RegExp("'+patternsStr+'","i");'+
|
||||
'var btns=document.querySelectorAll("button");'+
|
||||
'for(var i=0;i<btns.length;i++){'+
|
||||
'var b=btns[i];if(b.disabled||b.hidden)continue;'+
|
||||
'var t=(b.textContent||"").trim();'+
|
||||
'if(re.test(t)){b.click();return "CLICKED:"+t;}'+
|
||||
'}'+
|
||||
'return "NOT_FOUND:"+btns.length+"_buttons";'+
|
||||
'})()';
|
||||
for(var w=0;w<webviews.length;w++){
|
||||
(function(wv,idx){
|
||||
try{
|
||||
if(typeof wv.executeJavaScript==='function'){
|
||||
wv.executeJavaScript(clickScript).then(function(result){
|
||||
log(emoji+' TRIGGER-CLICK webview#'+idx+': '+result);
|
||||
}).catch(function(e){
|
||||
log('TRIGGER-CLICK webview#'+idx+' execJS error: '+e.message);
|
||||
});
|
||||
}
|
||||
}catch(e){
|
||||
log('TRIGGER-CLICK webview#'+idx+' error: '+e.message);
|
||||
}
|
||||
})(webviews[w],w);
|
||||
}
|
||||
}
|
||||
|
||||
// Phase 3: Try iframes via postMessage (cross-origin fallback)
|
||||
var iframes=document.querySelectorAll('iframe');
|
||||
if(iframes.length>0){
|
||||
log('TRIGGER-CLICK: trying '+iframes.length+' iframe(s) — checking accessibility...');
|
||||
var clickedAny=false;
|
||||
for(var fi=0;fi<iframes.length;fi++){
|
||||
try{
|
||||
var idoc=iframes[fi].contentDocument||iframes[fi].contentWindow.document;
|
||||
if(!idoc)continue;
|
||||
var ibtns=idoc.querySelectorAll('button');
|
||||
for(var bi=0;bi<ibtns.length;bi++){
|
||||
var ib=ibtns[bi];
|
||||
if(ib.disabled||ib.hidden)continue;
|
||||
var itxt=(ib.textContent||'').trim();
|
||||
for(var pi=0;pi<patterns.length;pi++){
|
||||
if(patterns[pi].test(itxt)){
|
||||
log(emoji+' TRIGGER-CLICK iframe#'+fi+': clicking "'+itxt+'"');
|
||||
ib.click();
|
||||
clickedAny=true;
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
}catch(e){}
|
||||
}
|
||||
}
|
||||
|
||||
if(!found.length){
|
||||
// Log what we DID find for debugging
|
||||
var allBtns=document.querySelectorAll('button');
|
||||
if(d.action==='approve'){
|
||||
// Click first Run/Accept/Allow/Continue button
|
||||
var approveRe=[/^Run/i,/^Accept/i,/^Allow/i,/^Approve/i,/^Continue$/i,/^Proceed$/i];
|
||||
for(var i=0;i<allBtns.length;i++){
|
||||
var b=allBtns[i];
|
||||
if(b.disabled||b.hidden||!b.offsetParent)continue;
|
||||
var txt=(b.textContent||'').trim();
|
||||
for(var p=0;p<approveRe.length;p++){
|
||||
if(approveRe[p].test(txt)){
|
||||
log('✅ TRIGGER-CLICK: clicking "'+txt+'"');
|
||||
b.click();
|
||||
return;
|
||||
var btnTexts=[];
|
||||
for(var di=0;di<Math.min(10,allBtns.length);di++){
|
||||
btnTexts.push('"'+((allBtns[di].textContent||'').trim()).substring(0,30)+'"');
|
||||
}
|
||||
}
|
||||
}
|
||||
log('⚠️ TRIGGER-CLICK: no approve button found in DOM');
|
||||
} else if(d.action==='reject'){
|
||||
for(var j=0;j<allBtns.length;j++){
|
||||
var b2=allBtns[j];
|
||||
if(b2.disabled||b2.hidden||!b2.offsetParent)continue;
|
||||
var txt2=(b2.textContent||'').trim();
|
||||
for(var r=0;r<REJECT_RE.length;r++){
|
||||
if(REJECT_RE[r].test(txt2)){
|
||||
log('❌ TRIGGER-CLICK: clicking "'+txt2+'"');
|
||||
b2.click();
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
log('⚠️ TRIGGER-CLICK: no reject button found in DOM');
|
||||
log('⚠️ TRIGGER-CLICK: no '+d.action+' button found. Main DOM has '+allBtns.length+' btns: ['+btnTexts.join(',')+']');
|
||||
log('⚠️ iframes='+document.querySelectorAll('iframe').length+' webviews='+document.querySelectorAll('webview').length);
|
||||
}
|
||||
}).catch(function(){});
|
||||
},1000);
|
||||
|
||||
_obs=true;
|
||||
log('v2 Observer active — MutationObserver + 3s fallback + trigger-click polling');
|
||||
log('v3 Observer active — deep DOM traversal + MutationObserver + trigger-click polling');
|
||||
}
|
||||
})();
|
||||
`;
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -546,16 +546,122 @@ function startObserverHttpBridge(): Promise<number> {
|
||||
function generateApprovalObserverScript(_port: number): string {
|
||||
// Port is hardcoded as fallback, but renderer also reads ag-bridge-ports.json for multi-bridge
|
||||
return `
|
||||
// ── Gravity Bridge v2: Approval Observer (MutationObserver-first, throttled) ──
|
||||
// ── Gravity Bridge v3: Approval Observer (deep DOM traversal — iframes, webviews, shadow DOMs) ──
|
||||
(function(){
|
||||
'use strict';
|
||||
var BASE='',_obs=false,_sent={},_ready=false;
|
||||
var _scanScheduled=false,_lastScanTs=0;
|
||||
var THROTTLE_MS=100;
|
||||
var CLEANUP_MS=300000;
|
||||
var _domDumped=false;
|
||||
|
||||
function log(m){console.log('[GB Observer] '+m);}
|
||||
log('v2 Script loaded — discovering bridge port...');
|
||||
log('v3 Script loaded — deep DOM traversal enabled');
|
||||
|
||||
// ── Deep DOM Traversal: find buttons across ALL boundaries ──
|
||||
// Searches: main document → iframes (contentDocument) → webview elements → shadow DOMs
|
||||
function deepFindButtons(patterns){
|
||||
var results=[];
|
||||
// 1. Main document buttons
|
||||
collectButtons(document,results,patterns,'main');
|
||||
// 2. Iframe traversal (try contentDocument — works if same-origin or webSecurity off)
|
||||
var iframes=document.querySelectorAll('iframe');
|
||||
for(var i=0;i<iframes.length;i++){
|
||||
try{
|
||||
var idoc=iframes[i].contentDocument||iframes[i].contentWindow.document;
|
||||
if(idoc){collectButtons(idoc,results,patterns,'iframe#'+i+'('+iframes[i].className.substring(0,30)+')');}
|
||||
}catch(e){
|
||||
// Cross-origin — can't access. Log only on first dom dump
|
||||
if(!_domDumped)log('iframe#'+i+' cross-origin: '+e.message.substring(0,60));
|
||||
}
|
||||
}
|
||||
// 3. Webview elements (Electron <webview> tag — has executeJavaScript)
|
||||
var webviews=document.querySelectorAll('webview');
|
||||
for(var w=0;w<webviews.length;w++){
|
||||
try{
|
||||
var wvDoc=webviews[w].contentDocument;
|
||||
if(wvDoc){collectButtons(wvDoc,results,patterns,'webview#'+w);}
|
||||
}catch(e){
|
||||
if(!_domDumped)log('webview#'+w+' access error: '+e.message.substring(0,60));
|
||||
}
|
||||
}
|
||||
return results;
|
||||
}
|
||||
|
||||
function collectButtons(doc,results,patterns,source){
|
||||
if(!doc||!doc.querySelectorAll)return;
|
||||
var btns=doc.querySelectorAll('button');
|
||||
for(var i=0;i<btns.length;i++){
|
||||
var b=btns[i];
|
||||
if(b.disabled||b.hidden)continue;
|
||||
try{if(!b.offsetParent&&b.style.display!=='fixed')continue;}catch(e){}
|
||||
var txt=(b.textContent||'').trim();
|
||||
if(!txt)continue;
|
||||
for(var p=0;p<patterns.length;p++){
|
||||
if(patterns[p].test(txt)){
|
||||
results.push({btn:b,text:txt,source:source});
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
// 4. Recurse into shadow DOMs
|
||||
try{
|
||||
var allEls=doc.querySelectorAll('*');
|
||||
for(var j=0;j<allEls.length;j++){
|
||||
var sr=allEls[j].shadowRoot;
|
||||
if(sr)collectButtons(sr,results,patterns,source+'>shadow');
|
||||
}
|
||||
}catch(e){}
|
||||
}
|
||||
|
||||
// ── DOM Structure Dump (one-time on startup) ──
|
||||
function dumpDOMStructure(){
|
||||
if(_domDumped)return;
|
||||
_domDumped=true;
|
||||
try{
|
||||
// Count iframes
|
||||
var iframes=document.querySelectorAll('iframe');
|
||||
log('DOM-DUMP: '+iframes.length+' iframes found');
|
||||
for(var i=0;i<iframes.length;i++){
|
||||
var f=iframes[i];
|
||||
var fInfo=' iframe#'+i+' class="'+f.className.substring(0,50)+'" src="'+(f.src||'').substring(0,80)+'"';
|
||||
try{
|
||||
var idoc=f.contentDocument;
|
||||
if(idoc){
|
||||
var btns=idoc.querySelectorAll('button');
|
||||
fInfo+=' → ACCESSIBLE ('+btns.length+' buttons)';
|
||||
// Dump first 5 button texts
|
||||
for(var b=0;b<Math.min(5,btns.length);b++){
|
||||
log(' btn['+b+']: "'+((btns[b].textContent||'').trim()).substring(0,40)+'"');
|
||||
}
|
||||
}
|
||||
}catch(e){
|
||||
fInfo+=' → BLOCKED ('+e.message.substring(0,40)+')';
|
||||
}
|
||||
log(fInfo);
|
||||
}
|
||||
// Count webview elements
|
||||
var webviews=document.querySelectorAll('webview');
|
||||
log('DOM-DUMP: '+webviews.length+' <webview> elements');
|
||||
for(var w=0;w<webviews.length;w++){
|
||||
var wInfo=' webview#'+w+' src="'+(webviews[w].src||'').substring(0,80)+'"';
|
||||
try{
|
||||
var wdoc=webviews[w].contentDocument;
|
||||
if(wdoc)wInfo+=' → ACCESSIBLE';
|
||||
else wInfo+=' → null contentDocument';
|
||||
}catch(e){wInfo+=' → BLOCKED ('+e.message.substring(0,40)+')';}
|
||||
log(wInfo);
|
||||
}
|
||||
// Main doc buttons
|
||||
var mainBtns=document.querySelectorAll('button');
|
||||
log('DOM-DUMP: '+mainBtns.length+' buttons in main document');
|
||||
for(var m=0;m<Math.min(10,mainBtns.length);m++){
|
||||
log(' main-btn['+m+']: "'+((mainBtns[m].textContent||'').trim()).substring(0,50)+'"');
|
||||
}
|
||||
// Report to bridge
|
||||
fetch(BASE+'/ping?dom_dump='+iframes.length+'f_'+webviews.length+'wv_'+mainBtns.length+'btn').catch(function(){});
|
||||
}catch(e){log('DOM-DUMP error: '+e.message);}
|
||||
}
|
||||
|
||||
// ── Port Discovery: async fetch()-based (sync XHR blocked in Electron renderer) ──
|
||||
var HARDCODED_PORT=${_port};
|
||||
@@ -587,7 +693,7 @@ function generateApprovalObserverScript(_port: number): string {
|
||||
discoverPort(function(port){
|
||||
BASE='http://127.0.0.1:'+port;
|
||||
fetch(BASE+'/ping').then(function(r){return r.text();}).then(function(t){
|
||||
if(t==='pong'){log('Bridge connected on port '+port);_ready=true;startObserver();}
|
||||
if(t==='pong'){log('Bridge connected on port '+port);_ready=true;startObserver();setTimeout(dumpDOMStructure,3000);}
|
||||
else log('Bridge ping failed: '+t);
|
||||
}).catch(function(e){log('Bridge unreachable: '+e.message);});
|
||||
});
|
||||
@@ -843,49 +949,100 @@ function generateApprovalObserverScript(_port: number): string {
|
||||
|
||||
// ── TRIGGER-CLICK: Extension→Renderer bridge for programmatic button clicks ──
|
||||
// Extension sets clickTrigger via tryApprovalStrategies → renderer polls and clicks
|
||||
// v3: uses deepFindButtons() to traverse iframes, webviews, shadow DOMs
|
||||
setInterval(function(){
|
||||
if(!_ready||!BASE)return;
|
||||
fetch(BASE+'/trigger-click?t='+Date.now()).then(function(r){return r.json();}).then(function(d){
|
||||
if(!d.action)return;
|
||||
log('🔔 TRIGGER-CLICK received: action='+d.action);
|
||||
// Find first visible approve or reject button
|
||||
|
||||
var approveRe=[/^Run$/i,/^Run /i,/^Accept/i,/^Allow/i,/^Approve/i,/^Continue$/i,/^Proceed$/i];
|
||||
var rejectRe=[/^Reject/i,/^Cancel$/i,/^Deny$/i,/^Stop$/i,/^Decline$/i];
|
||||
var patterns=(d.action==='approve')?approveRe:rejectRe;
|
||||
var emoji=(d.action==='approve')?'✅':'❌';
|
||||
|
||||
// Phase 1: deepFindButtons in main doc + accessible iframes + shadow DOMs
|
||||
var found=deepFindButtons(patterns);
|
||||
if(found.length>0){
|
||||
log(emoji+' TRIGGER-CLICK: clicking "'+found[0].text+'" from '+found[0].source);
|
||||
found[0].btn.click();
|
||||
return;
|
||||
}
|
||||
|
||||
// Phase 2: Try <webview>.executeJavaScript for inaccessible webviews
|
||||
var webviews=document.querySelectorAll('webview');
|
||||
if(webviews.length>0){
|
||||
log('TRIGGER-CLICK: trying '+webviews.length+' webview(s) via executeJavaScript...');
|
||||
var patternsStr=patterns.map(function(re){return re.source;}).join('|');
|
||||
var clickScript='(function(){'+
|
||||
'var re=new RegExp("'+patternsStr+'","i");'+
|
||||
'var btns=document.querySelectorAll("button");'+
|
||||
'for(var i=0;i<btns.length;i++){'+
|
||||
'var b=btns[i];if(b.disabled||b.hidden)continue;'+
|
||||
'var t=(b.textContent||"").trim();'+
|
||||
'if(re.test(t)){b.click();return "CLICKED:"+t;}'+
|
||||
'}'+
|
||||
'return "NOT_FOUND:"+btns.length+"_buttons";'+
|
||||
'})()';
|
||||
for(var w=0;w<webviews.length;w++){
|
||||
(function(wv,idx){
|
||||
try{
|
||||
if(typeof wv.executeJavaScript==='function'){
|
||||
wv.executeJavaScript(clickScript).then(function(result){
|
||||
log(emoji+' TRIGGER-CLICK webview#'+idx+': '+result);
|
||||
}).catch(function(e){
|
||||
log('TRIGGER-CLICK webview#'+idx+' execJS error: '+e.message);
|
||||
});
|
||||
}
|
||||
}catch(e){
|
||||
log('TRIGGER-CLICK webview#'+idx+' error: '+e.message);
|
||||
}
|
||||
})(webviews[w],w);
|
||||
}
|
||||
}
|
||||
|
||||
// Phase 3: Try iframes via postMessage (cross-origin fallback)
|
||||
var iframes=document.querySelectorAll('iframe');
|
||||
if(iframes.length>0){
|
||||
log('TRIGGER-CLICK: trying '+iframes.length+' iframe(s) — checking accessibility...');
|
||||
var clickedAny=false;
|
||||
for(var fi=0;fi<iframes.length;fi++){
|
||||
try{
|
||||
var idoc=iframes[fi].contentDocument||iframes[fi].contentWindow.document;
|
||||
if(!idoc)continue;
|
||||
var ibtns=idoc.querySelectorAll('button');
|
||||
for(var bi=0;bi<ibtns.length;bi++){
|
||||
var ib=ibtns[bi];
|
||||
if(ib.disabled||ib.hidden)continue;
|
||||
var itxt=(ib.textContent||'').trim();
|
||||
for(var pi=0;pi<patterns.length;pi++){
|
||||
if(patterns[pi].test(itxt)){
|
||||
log(emoji+' TRIGGER-CLICK iframe#'+fi+': clicking "'+itxt+'"');
|
||||
ib.click();
|
||||
clickedAny=true;
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
}catch(e){}
|
||||
}
|
||||
}
|
||||
|
||||
if(!found.length){
|
||||
// Log what we DID find for debugging
|
||||
var allBtns=document.querySelectorAll('button');
|
||||
if(d.action==='approve'){
|
||||
// Click first Run/Accept/Allow/Continue button
|
||||
var approveRe=[/^Run/i,/^Accept/i,/^Allow/i,/^Approve/i,/^Continue$/i,/^Proceed$/i];
|
||||
for(var i=0;i<allBtns.length;i++){
|
||||
var b=allBtns[i];
|
||||
if(b.disabled||b.hidden||!b.offsetParent)continue;
|
||||
var txt=(b.textContent||'').trim();
|
||||
for(var p=0;p<approveRe.length;p++){
|
||||
if(approveRe[p].test(txt)){
|
||||
log('✅ TRIGGER-CLICK: clicking "'+txt+'"');
|
||||
b.click();
|
||||
return;
|
||||
var btnTexts=[];
|
||||
for(var di=0;di<Math.min(10,allBtns.length);di++){
|
||||
btnTexts.push('"'+((allBtns[di].textContent||'').trim()).substring(0,30)+'"');
|
||||
}
|
||||
}
|
||||
}
|
||||
log('⚠️ TRIGGER-CLICK: no approve button found in DOM');
|
||||
} else if(d.action==='reject'){
|
||||
for(var j=0;j<allBtns.length;j++){
|
||||
var b2=allBtns[j];
|
||||
if(b2.disabled||b2.hidden||!b2.offsetParent)continue;
|
||||
var txt2=(b2.textContent||'').trim();
|
||||
for(var r=0;r<REJECT_RE.length;r++){
|
||||
if(REJECT_RE[r].test(txt2)){
|
||||
log('❌ TRIGGER-CLICK: clicking "'+txt2+'"');
|
||||
b2.click();
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
log('⚠️ TRIGGER-CLICK: no reject button found in DOM');
|
||||
log('⚠️ TRIGGER-CLICK: no '+d.action+' button found. Main DOM has '+allBtns.length+' btns: ['+btnTexts.join(',')+']');
|
||||
log('⚠️ iframes='+document.querySelectorAll('iframe').length+' webviews='+document.querySelectorAll('webview').length);
|
||||
}
|
||||
}).catch(function(){});
|
||||
},1000);
|
||||
|
||||
_obs=true;
|
||||
log('v2 Observer active — MutationObserver + 3s fallback + trigger-click polling');
|
||||
log('v3 Observer active — deep DOM traversal + MutationObserver + trigger-click polling');
|
||||
}
|
||||
})();
|
||||
`;
|
||||
|
||||
Reference in New Issue
Block a user