325 lines
10 KiB
Markdown
325 lines
10 KiB
Markdown
# Improve task memory and conversation recall
|
|
|
|
## Summary
|
|
|
|
Long-running local tasks can lose early user instructions, constraints, and
|
|
decisions after several exchanges. The codebase already has partial mechanisms:
|
|
|
|
- `MissionUpdate` / Mission Brief pins `goal`, `done`, `open`, and
|
|
`clarifications`.
|
|
- `Conversation` persists `logs/transcript.jsonl` and can replay prior turns on
|
|
continuation.
|
|
- `buildLocalConversationContext()` injects recent task comments.
|
|
- `handoffContext` carries the previous piece result.
|
|
|
|
However, these do not give the agent a reliable way to preserve task state or
|
|
actively search older conversation history.
|
|
|
|
This issue proposes three related improvements:
|
|
|
|
1. Make Mission Brief a stronger task-state ledger.
|
|
2. Add agent-facing conversation search/read tools.
|
|
3. Update prompts so the agent proactively revisits prior conversation when
|
|
task context is ambiguous or long-running.
|
|
|
|
## Problem
|
|
|
|
In multi-turn tasks, the agent often appears to forget early conversation
|
|
details, especially:
|
|
|
|
- original user constraints
|
|
- decisions made after clarification
|
|
- "do not change X" instructions
|
|
- previous failed attempts
|
|
- which files were already inspected or changed
|
|
- why a plan was chosen
|
|
|
|
Current behavior is understandable from the implementation:
|
|
|
|
- `MissionUpdate` is optional and model-driven; it is not enforced at movement
|
|
boundaries.
|
|
- Mission Brief fields are too coarse for long implementation tasks.
|
|
- local task context only injects the recent conversation window.
|
|
- transcript replay is automatic context carry-over, not searchable retrieval.
|
|
- prompt compaction/summarization can remove or compress important details.
|
|
|
|
## Existing Mechanisms To Reuse
|
|
|
|
### Mission Brief
|
|
|
|
Source: `src/engine/tools/mission.ts`
|
|
|
|
Mission Brief is the best existing place for pinned task state because it is:
|
|
|
|
- per local task
|
|
- visible at the top of every movement prompt
|
|
- editable by the agent through `MissionUpdate`
|
|
- editable by the user in the UI
|
|
- intended to prevent long-conversation drift
|
|
|
|
This should be extended rather than replaced.
|
|
|
|
### Conversation transcript
|
|
|
|
Source: `src/engine/context/conversation.ts`
|
|
|
|
`logs/transcript.jsonl` is already written during execution and replayed on
|
|
continuation. It should remain the raw audit source. New search tools should
|
|
read from it instead of stuffing the whole transcript into prompt context.
|
|
|
|
### Local task comments
|
|
|
|
Source: `src/engine/local-context.ts`
|
|
|
|
Task comments are the user-facing conversation layer. They should be searchable
|
|
alongside transcript entries because user instructions often live in comments,
|
|
requests, and interjections.
|
|
|
|
## Proposed Design
|
|
|
|
### 1. Extend Mission Brief schema
|
|
|
|
Add fields that separate task facts from progress:
|
|
|
|
```ts
|
|
interface MissionBrief {
|
|
goal?: string;
|
|
user_constraints?: string;
|
|
decisions?: string;
|
|
done?: string;
|
|
open?: string;
|
|
current_focus?: string;
|
|
touched_files?: string;
|
|
risks?: string;
|
|
clarifications?: string;
|
|
last_updated_by_movement?: string;
|
|
}
|
|
```
|
|
|
|
Keep backwards compatibility with existing `goal`, `done`, `open`, and
|
|
`clarifications`.
|
|
|
|
### 2. Make movement-boundary updates explicit
|
|
|
|
At the end of each movement, encourage or enforce a Mission Brief update when
|
|
the movement made meaningful progress.
|
|
|
|
Suggested behavior:
|
|
|
|
- On movement start, render the current Mission Brief as today.
|
|
- During movement, the model can still call `MissionUpdate`.
|
|
- Before `transition` or `complete`, prompt guidance should require the agent to
|
|
update `done`, `open`, `current_focus`, and relevant `decisions` if stale.
|
|
- Optionally add a lightweight stale-check:
|
|
- if a movement used tools or edited files
|
|
- and no `MissionUpdate` occurred
|
|
- inject a reminder before accepting terminal control calls
|
|
|
|
Do not make this a hard blocker initially; use a reminder first to avoid
|
|
creating loops.
|
|
|
|
### 3. Add `SearchTaskConversation`
|
|
|
|
Add a META tool available to all local-task pieces.
|
|
|
|
Suggested signature:
|
|
|
|
```ts
|
|
SearchTaskConversation({
|
|
query: string;
|
|
source?: "comments" | "transcript" | "both";
|
|
author?: "user" | "agent" | "system";
|
|
kind?: "request" | "comment" | "interjection" | "result" | "ask" | "progress" | "handoff";
|
|
limit?: number;
|
|
})
|
|
```
|
|
|
|
Search sources:
|
|
|
|
- DB local task comments
|
|
- `runtimeDir/transcript.jsonl` when available
|
|
|
|
Return compact excerpts only:
|
|
|
|
```md
|
|
## Conversation Search Results
|
|
|
|
- comment:123 user/comment 2026-06-30T...
|
|
excerpt: ...
|
|
|
|
- transcript:42 user 2026-06-30T...
|
|
excerpt: ...
|
|
```
|
|
|
|
Constraints:
|
|
|
|
- cap result count
|
|
- cap excerpt length
|
|
- never return full transcript by default
|
|
- search only the current task's conversation
|
|
- respect existing task/space authorization boundaries
|
|
|
|
### 4. Add `ReadTaskConversation`
|
|
|
|
Add a companion tool to read context around a known hit.
|
|
|
|
Suggested signature:
|
|
|
|
```ts
|
|
ReadTaskConversation({
|
|
ref: "comment:123" | "transcript:42";
|
|
before?: number;
|
|
after?: number;
|
|
})
|
|
```
|
|
|
|
Return nearby entries with strict caps. This keeps search cheap and lets the
|
|
agent inspect the exact conversation around an old decision.
|
|
|
|
### 5. Wire conversation retrieval into Mission Brief refresh
|
|
|
|
Update prompt guidance for long tasks:
|
|
|
|
- If user constraints or decisions are unclear, use `SearchTaskConversation`
|
|
before asking the user.
|
|
- When updating `user_constraints` or `decisions`, prefer citing old comments or
|
|
transcript refs in the brief.
|
|
- Before asking the user a clarification, search the task conversation when the
|
|
missing information may already have been stated earlier.
|
|
- Before changing previously touched files or revising a prior decision, search
|
|
for older constraints/decisions related to the file or topic.
|
|
|
|
Example Mission Brief snippet:
|
|
|
|
```md
|
|
### User constraints
|
|
- Keep existing auth flow unchanged. Source: comment:17
|
|
- Do not rewrite unrelated UI. Source: transcript:42
|
|
|
|
### Decisions
|
|
- Extend Mission Brief instead of creating a second TaskState store. Source: comment:23
|
|
```
|
|
|
|
### 6. Add proactive recall guidance to system/movement prompts
|
|
|
|
The new tools should not be passive. Update prompt guidance so the agent
|
|
actively uses them when forgetting early conversation is likely.
|
|
|
|
Suggested rules:
|
|
|
|
- At the start of a follow-up task, review Mission Brief first.
|
|
- If Mission Brief is missing, stale, or too vague, call
|
|
`SearchTaskConversation` before proceeding with assumptions.
|
|
- If the task has multiple user turns or interjections, search conversation
|
|
history for constraints before making broad edits.
|
|
- If the agent is about to ask the user something, first search prior comments
|
|
and transcript for the answer unless the question is truly new.
|
|
- If a movement resumes after context compaction, search for relevant older
|
|
decisions before modifying files.
|
|
- When search results reveal durable constraints or decisions, update
|
|
Mission Brief immediately.
|
|
|
|
Candidate prompt copy:
|
|
|
|
```md
|
|
## Conversation recall
|
|
For long-running or follow-up tasks, do not rely only on the visible recent
|
|
context. If an earlier user constraint, decision, or clarification may affect
|
|
your next action, use SearchTaskConversation / ReadTaskConversation before
|
|
proceeding. Prefer retrieving prior context over asking the user to repeat it.
|
|
When you find durable task facts, update MissionUpdate so they remain pinned.
|
|
```
|
|
|
|
Keep this guidance concise so it does not crowd out movement-specific
|
|
instructions.
|
|
|
|
## Files Likely Involved
|
|
|
|
- `src/db/repository.ts`
|
|
- extend `MissionBrief`
|
|
- update parse/update tests
|
|
- add query helpers for task comments if needed
|
|
|
|
- `src/engine/tools/mission.ts`
|
|
- extend `MissionUpdate` schema
|
|
- clamp/render new fields
|
|
- tests for partial updates
|
|
|
|
- `src/engine/agent-loop/prompt.ts`
|
|
- render new Mission Brief fields
|
|
- update movement guidance around state refresh
|
|
- add concise proactive recall guidance for long/follow-up tasks
|
|
|
|
- `src/engine/agent-loop/watchdogs.ts`
|
|
- optionally add stale Mission Brief reminder logic
|
|
- optionally remind the agent to search conversation history when Mission Brief
|
|
is empty/stale during later iterations
|
|
|
|
- `src/engine/tools/index.ts`
|
|
- add new conversation tools to META_TOOLS
|
|
|
|
- `src/engine/tools/conversation.ts` or `src/engine/tools/task-conversation.ts`
|
|
- implement `SearchTaskConversation`
|
|
- implement `ReadTaskConversation`
|
|
|
|
- `src/engine/tools/core.ts`
|
|
- expose safe context fields needed by conversation tools, e.g. taskId,
|
|
runtimeDir, workspacePath, repo access if needed
|
|
|
|
- `src/worker.ts`
|
|
- ensure ToolContext has enough current-task context for the new tools
|
|
|
|
- `ui/src/components/detail/tabs/OverviewTab.tsx`
|
|
- expose new Mission Brief fields for user editing
|
|
|
|
- `ui/src/content/help/*.md`
|
|
- update help docs if behavior/UI changes
|
|
|
|
- `ui/src/content/help/00-changelog.md`
|
|
- add a user-facing changelog entry
|
|
|
|
## Acceptance Criteria
|
|
|
|
- Mission Brief supports new fields while preserving existing data.
|
|
- Existing tasks with old Mission Brief JSON still load correctly.
|
|
- `MissionUpdate` can update new fields independently.
|
|
- Mission Brief rendering keeps a bounded prompt size.
|
|
- Movement guidance encourages updating Mission Brief before transitions.
|
|
- Prompt guidance tells the agent to search prior conversation before asking the
|
|
user to repeat information or making assumptions in long/follow-up tasks.
|
|
- Prompt guidance tells the agent to search prior conversation before asking the
|
|
user to repeat information or making assumptions in long/follow-up tasks.
|
|
- `SearchTaskConversation` can find old user comments by keyword.
|
|
- `SearchTaskConversation` can find transcript entries when `transcript.jsonl`
|
|
exists.
|
|
- `ReadTaskConversation` can return bounded context around a search result.
|
|
- Tools are scoped to the current task and cannot read other task logs.
|
|
- Tests cover:
|
|
- old Mission Brief compatibility
|
|
- new Mission Brief fields
|
|
- conversation search over comments
|
|
- conversation search over transcript
|
|
- prompt text includes proactive recall guidance
|
|
- bounded output and excerpt truncation
|
|
- authorization/task scoping
|
|
|
|
## Non-goals
|
|
|
|
- Do not put task-specific state into `UpdateUserMemory`; that is user-wide
|
|
memory and would pollute future unrelated tasks.
|
|
- Do not inject the full transcript into every prompt.
|
|
- Do not replace `Conversation` replay; retrieval should complement it.
|
|
- Do not make Mission Brief update a hard terminal blocker until reminder-only
|
|
behavior has been observed.
|
|
|
|
## Suggested Implementation Order
|
|
|
|
1. Extend Mission Brief schema and UI rendering.
|
|
2. Extend `MissionUpdate` and prompt rendering.
|
|
3. Add reminder-only movement-boundary guidance.
|
|
4. Implement `SearchTaskConversation` for DB comments.
|
|
5. Add transcript search.
|
|
6. Add `ReadTaskConversation`.
|
|
7. Add proactive recall prompt guidance and tests.
|
|
8. Update help docs and changelog.
|