This commit is contained in:
oss-sync
@@ -0,0 +1,114 @@
|
||||
# Add segmented screenshots to BrowseWeb
|
||||
|
||||
> **Status: Implemented (2026-07-03).** 設計との主な差分は下記「Implementation notes」を参照。
|
||||
|
||||
## Implementation notes (設計との差分)
|
||||
|
||||
- **既定を分割に変更**: 当初案は `screenshotSegments: true` のオプトインだったが、実際のユーザー意図は「スクショは既定で 1 画面ぶんずつ区切る」だったため、**BrowseWeb では分割を既定挙動**にした。フルページ 1 枚が欲しい場合は `screenshotSegments: false`（基本モード）/ アクションの `segments: false` でオプトアウトする。
|
||||
- **短ページは連番なし**: 1 画面に収まるページは連番を付けず `output/<name>.png` の 1 枚のまま（後方互換）。2 画面ぶん以上のときだけ `-001` / `-002` の連番になる。
|
||||
- **分割単位・撮影方式**: ビューポート高さ（1 画面ぶん）を単位に、`fullPage: true` + `clip` でページを縦に切り出す。スクロールしないため、(1) `position:fixed`/`sticky` なヘッダーが各セグメント先頭に重複して本文を隠さない、(2) 撮影後にページのスクロール位置を変えないので後続アクション（ref/selector クリック等）に副作用が出ない。最後のセグメントは残り高さぶんだけ切り出し、下端に余白を作らない。
|
||||
- **枚数の絶対上限**: `maxSegments` はユーザー（LLM）指定でも `ABSOLUTE_MAX_SEGMENTS`（50）を超えない。巨大な `scrollHeight` を返すページや極端な指定値による撮影ループの暴走（worker 時間・ディスク枯渇）を防ぐ最終防波堤。非有限値（NaN/Infinity）は既定値にフォールバック。
|
||||
- **TestWorkspaceApp は現状維持**: 共通ハンドラ（`runPageActions`）の既定を分割にしたため、`TestWorkspaceApp` の screenshot アクションには `segments: false` を明示して単一フルページ撮影を維持（本 issue の scope 指示どおり）。
|
||||
- **打ち切り通知**: `maxSegments`（既定 10）で打ち切った場合は戻り値にその旨を出力（無音打ち切りにしない）。
|
||||
- 実装: `src/engine/tools/browser.ts`（`planScreenshotSegments` / `segmentFilename` / `captureScreenshots`）、テスト: `browser.screenshot-segments.test.ts`（純関数）・`browser.screenshot-capture.test.ts`（実ブラウザ、`CONTAINER=1` ゲート）。
|
||||
|
||||
## Summary
|
||||
|
||||
Long HTML pages are hard for agents to verify from a single screenshot. A full-page
|
||||
capture can become extremely tall, causing image understanding to miss details or
|
||||
compress important UI states. Viewport-only screenshots avoid giant images, but they
|
||||
only show the first visible area.
|
||||
|
||||
Add a segmented screenshot mode to `BrowseWeb` so agents can capture a page as a
|
||||
sequence of viewport-sized images.
|
||||
|
||||
## Problem
|
||||
|
||||
For generated HTML verification and UI review, agents often need to inspect the
|
||||
whole rendered page. Current screenshot modes are not ideal:
|
||||
|
||||
- viewport screenshot: readable, but only captures one screen
|
||||
- full-page screenshot: complete, but can be too tall and information-dense
|
||||
|
||||
This makes visual verification unreliable for long reports, dashboards, and other
|
||||
HTML outputs.
|
||||
|
||||
## Proposed Design
|
||||
|
||||
Keep existing screenshot behavior and add an explicit segmented mode.
|
||||
|
||||
### Basic BrowseWeb mode
|
||||
|
||||
```js
|
||||
BrowseWeb({
|
||||
url: "output/report.html",
|
||||
screenshot: "report.png",
|
||||
screenshotSegments: true
|
||||
})
|
||||
```
|
||||
|
||||
Expected output files:
|
||||
|
||||
```text
|
||||
output/report-001.png
|
||||
output/report-002.png
|
||||
output/report-003.png
|
||||
```
|
||||
|
||||
### Action mode
|
||||
|
||||
```js
|
||||
BrowseWeb({
|
||||
actions: [
|
||||
{ type: "goto", url: "output/report.html" },
|
||||
{ type: "screenshot", value: "report.png", segments: true, maxSegments: 8 }
|
||||
]
|
||||
})
|
||||
```
|
||||
|
||||
### Behavior
|
||||
|
||||
- Scroll the page from top to bottom and capture one viewport-sized screenshot
|
||||
per segment.
|
||||
- Use stable generated names based on the requested screenshot filename:
|
||||
`name-001.ext`, `name-002.ext`, etc.
|
||||
- Return the saved file list in the tool output.
|
||||
- Default to a bounded number of segments, for example `maxSegments: 10`, to
|
||||
avoid runaway captures on infinite-scroll pages.
|
||||
- Allow callers to override the cap with `maxSegments`.
|
||||
- Preserve existing modes:
|
||||
- default screenshot remains a single viewport image
|
||||
- full-page screenshot remains available via the explicit full-page option
|
||||
- Do not change `TestWorkspaceApp` iframe screenshots unless a separate need is
|
||||
identified.
|
||||
|
||||
## Files Likely Involved
|
||||
|
||||
- `src/engine/tools/browser.ts`
|
||||
- extend `BrowseWebAction` with `segments?: boolean` and `maxSegments?: number`
|
||||
- add basic-mode inputs `screenshotSegments` and `screenshotMaxSegments`
|
||||
- implement shared segmented capture helper
|
||||
|
||||
- `src/engine/tools/browser.runpageactions.test.ts`
|
||||
- add regression coverage for segmented screenshots on a tall local HTML page
|
||||
- assert filenames and image heights/counts
|
||||
|
||||
- `docs/tools/browseweb.md`
|
||||
- document segmented screenshot usage
|
||||
|
||||
- `ui/src/content/help/00-changelog.md`
|
||||
- add a user-facing changelog entry when implemented
|
||||
|
||||
- `ui/src/content/help/16-tools.md`
|
||||
- mention that browser screenshots can be captured as viewport, full-page, or
|
||||
segmented images
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- `BrowseWeb({ url, screenshot, screenshotSegments: true })` saves multiple
|
||||
viewport-sized screenshots for a tall page.
|
||||
- Action-mode `screenshot` supports `segments: true`.
|
||||
- The tool output lists all saved segment paths.
|
||||
- Segment count is capped by default and configurable.
|
||||
- Existing viewport and full-page screenshot behavior continues to work.
|
||||
- Tests cover viewport, full-page, and segmented capture behavior.
|
||||
Reference in New Issue
Block a user