maestro/docs/issues/2026-06-30-task-memory-and-conversation-search.md

Improve task memory and conversation recall

Summary

Long-running local tasks can lose early user instructions, constraints, and decisions after several exchanges. The codebase already has partial mechanisms:

• MissionUpdate / Mission Brief pins goal, done, open, and clarifications.
• Conversation persists logs/transcript.jsonl and can replay prior turns on continuation.
• buildLocalConversationContext() injects recent task comments.
• handoffContext carries the previous piece result.

However, these do not give the agent a reliable way to preserve task state or actively search older conversation history.

This issue proposes three related improvements:

1. Make Mission Brief a stronger task-state ledger.
2. Add agent-facing conversation search/read tools.
3. Update prompts so the agent proactively revisits prior conversation when task context is ambiguous or long-running.

Problem

In multi-turn tasks, the agent often appears to forget early conversation details, especially:

• original user constraints
• decisions made after clarification
• "do not change X" instructions
• previous failed attempts
• which files were already inspected or changed
• why a plan was chosen

Current behavior is understandable from the implementation:

• MissionUpdate is optional and model-driven; it is not enforced at movement boundaries.
• Mission Brief fields are too coarse for long implementation tasks.
• local task context only injects the recent conversation window.
• transcript replay is automatic context carry-over, not searchable retrieval.
• prompt compaction/summarization can remove or compress important details.

Existing Mechanisms To Reuse

Mission Brief

Source: src/engine/tools/mission.ts

Mission Brief is the best existing place for pinned task state because it is:

• per local task
• visible at the top of every movement prompt
• editable by the agent through MissionUpdate
• editable by the user in the UI
• intended to prevent long-conversation drift

This should be extended rather than replaced.

Conversation transcript

Source: src/engine/context/conversation.ts

logs/transcript.jsonl is already written during execution and replayed on continuation. It should remain the raw audit source. New search tools should read from it instead of stuffing the whole transcript into prompt context.

Local task comments

Source: src/engine/local-context.ts

Task comments are the user-facing conversation layer. They should be searchable alongside transcript entries because user instructions often live in comments, requests, and interjections.

Proposed Design

1. Extend Mission Brief schema

Add fields that separate task facts from progress:

interface MissionBrief {
  goal?: string;
  user_constraints?: string;
  decisions?: string;
  done?: string;
  open?: string;
  current_focus?: string;
  touched_files?: string;
  risks?: string;
  clarifications?: string;
  last_updated_by_movement?: string;
}

Keep backwards compatibility with existing goal, done, open, and clarifications.

2. Make movement-boundary updates explicit

At the end of each movement, encourage or enforce a Mission Brief update when the movement made meaningful progress.

Suggested behavior:

• On movement start, render the current Mission Brief as today.
• During movement, the model can still call MissionUpdate.
• Before transition or complete, prompt guidance should require the agent to update done, open, current_focus, and relevant decisions if stale.
• Optionally add a lightweight stale-check:
  • if a movement used tools or edited files
  • and no MissionUpdate occurred
  • inject a reminder before accepting terminal control calls

Do not make this a hard blocker initially; use a reminder first to avoid creating loops.

3. Add SearchTaskConversation

Add a META tool available to all local-task pieces.

Suggested signature:

SearchTaskConversation({
  query: string;
  source?: "comments" | "transcript" | "both";
  author?: "user" | "agent" | "system";
  kind?: "request" | "comment" | "interjection" | "result" | "ask" | "progress" | "handoff";
  limit?: number;
})

Search sources:

• DB local task comments
• runtimeDir/transcript.jsonl when available

Return compact excerpts only:

## Conversation Search Results

- comment:123 user/comment 2026-06-30T...
  excerpt: ...

- transcript:42 user 2026-06-30T...
  excerpt: ...

Constraints:

• cap result count
• cap excerpt length
• never return full transcript by default
• search only the current task's conversation
• respect existing task/space authorization boundaries

4. Add ReadTaskConversation

Add a companion tool to read context around a known hit.

Suggested signature:

ReadTaskConversation({
  ref: "comment:123" | "transcript:42";
  before?: number;
  after?: number;
})

Return nearby entries with strict caps. This keeps search cheap and lets the agent inspect the exact conversation around an old decision.

5. Wire conversation retrieval into Mission Brief refresh

Update prompt guidance for long tasks:

• If user constraints or decisions are unclear, use SearchTaskConversation before asking the user.
• When updating user_constraints or decisions, prefer citing old comments or transcript refs in the brief.
• Before asking the user a clarification, search the task conversation when the missing information may already have been stated earlier.
• Before changing previously touched files or revising a prior decision, search for older constraints/decisions related to the file or topic.

Example Mission Brief snippet:

### User constraints
- Keep existing auth flow unchanged. Source: comment:17
- Do not rewrite unrelated UI. Source: transcript:42

### Decisions
- Extend Mission Brief instead of creating a second TaskState store. Source: comment:23

6. Add proactive recall guidance to system/movement prompts

The new tools should not be passive. Update prompt guidance so the agent actively uses them when forgetting early conversation is likely.

Suggested rules:

• At the start of a follow-up task, review Mission Brief first.
• If Mission Brief is missing, stale, or too vague, call SearchTaskConversation before proceeding with assumptions.
• If the task has multiple user turns or interjections, search conversation history for constraints before making broad edits.
• If the agent is about to ask the user something, first search prior comments and transcript for the answer unless the question is truly new.
• If a movement resumes after context compaction, search for relevant older decisions before modifying files.
• When search results reveal durable constraints or decisions, update Mission Brief immediately.

Candidate prompt copy:

## Conversation recall
For long-running or follow-up tasks, do not rely only on the visible recent
context. If an earlier user constraint, decision, or clarification may affect
your next action, use SearchTaskConversation / ReadTaskConversation before
proceeding. Prefer retrieving prior context over asking the user to repeat it.
When you find durable task facts, update MissionUpdate so they remain pinned.

Keep this guidance concise so it does not crowd out movement-specific instructions.

Files Likely Involved

• src/db/repository.ts

  • extend MissionBrief
  • update parse/update tests
  • add query helpers for task comments if needed

• src/engine/tools/mission.ts

  • extend MissionUpdate schema
  • clamp/render new fields
  • tests for partial updates

• src/engine/agent-loop/prompt.ts

  • render new Mission Brief fields
  • update movement guidance around state refresh
  • add concise proactive recall guidance for long/follow-up tasks

• src/engine/agent-loop/watchdogs.ts

  • optionally add stale Mission Brief reminder logic
  • optionally remind the agent to search conversation history when Mission Brief is empty/stale during later iterations

• src/engine/tools/index.ts

  • add new conversation tools to META_TOOLS

• src/engine/tools/conversation.ts or src/engine/tools/task-conversation.ts

  • implement SearchTaskConversation
  • implement ReadTaskConversation

• src/engine/tools/core.ts

  • expose safe context fields needed by conversation tools, e.g. taskId, runtimeDir, workspacePath, repo access if needed

• src/worker.ts

  • ensure ToolContext has enough current-task context for the new tools

• ui/src/components/detail/tabs/OverviewTab.tsx

  • expose new Mission Brief fields for user editing

• ui/src/content/help/*.md

  • update help docs if behavior/UI changes

• ui/src/content/help/00-changelog.md

  • add a user-facing changelog entry

Acceptance Criteria

• Mission Brief supports new fields while preserving existing data.
• Existing tasks with old Mission Brief JSON still load correctly.
• MissionUpdate can update new fields independently.
• Mission Brief rendering keeps a bounded prompt size.
• Movement guidance encourages updating Mission Brief before transitions.
• Prompt guidance tells the agent to search prior conversation before asking the user to repeat information or making assumptions in long/follow-up tasks.
• Prompt guidance tells the agent to search prior conversation before asking the user to repeat information or making assumptions in long/follow-up tasks.
• SearchTaskConversation can find old user comments by keyword.
• SearchTaskConversation can find transcript entries when transcript.jsonl exists.
• ReadTaskConversation can return bounded context around a search result.
• Tools are scoped to the current task and cannot read other task logs.
• Tests cover:
  • old Mission Brief compatibility
  • new Mission Brief fields
  • conversation search over comments
  • conversation search over transcript
  • prompt text includes proactive recall guidance
  • bounded output and excerpt truncation
  • authorization/task scoping

Non-goals

• Do not put task-specific state into UpdateUserMemory; that is user-wide memory and would pollute future unrelated tasks.
• Do not inject the full transcript into every prompt.
• Do not replace Conversation replay; retrieval should complement it.
• Do not make Mission Brief update a hard terminal blocker until reminder-only behavior has been observed.

Suggested Implementation Order

1. Extend Mission Brief schema and UI rendering.
2. Extend MissionUpdate and prompt rendering.
3. Add reminder-only movement-boundary guidance.
4. Implement SearchTaskConversation for DB comments.
5. Add transcript search.
6. Add ReadTaskConversation.
7. Add proactive recall prompt guidance and tests.
8. Update help docs and changelog.