Context Management Plugin

The Context Management Plugin proactively maintains multi-resolution state descriptions of conversations and dynamically compresses old context using semantic drift detection with cascading compression.

Overview

Unlike the reactive Context Summarization Plugin (which only activates on context overflow), this plugin:

Proactive: Runs after every turn, continuously maintaining conversation state
Incremental: Updates cumulative state descriptions rather than re-summarizing
Multi-resolution: Maintains descriptions at multiple token levels (15, 50, 250, 1000)
Drift-aware: Uses LLM-based semantic drift detection to identify topic shifts
Cascading compression: Older content gets progressively more compressed

Cascading Compression Model

The plugin uses a sophisticated multi-level compression scheme:

Context sent to LLM:
┌────────────────────────────────────────────────────────┐
│ [DISTANT HISTORY - 15 tokens, turns 1-C15]             │  ← Oldest, most compressed
│ Brief summary of earliest conversation...              │
├────────────────────────────────────────────────────────┤
│ [OLDER HISTORY - 50 tokens, turns 1-C50]               │
│ Slightly more detailed summary...                      │
├────────────────────────────────────────────────────────┤
│ [PAST HISTORY - 250 tokens, turns 1-C250]              │
│ Moderate detail summary...                             │
├────────────────────────────────────────────────────────┤
│ [RECENT HISTORY - 1000 tokens, turns 1-C1000]          │
│ Detailed summary of recent past...                     │
├────────────────────────────────────────────────────────┤
│ [Raw messages from turn C1000+1 onwards]               │  ← Most recent, uncompressed
│ User: ...                                              │
│ Assistant: ...                                         │
└────────────────────────────────────────────────────────┘

When a new topic shift (drift) is detected:

Previous 1000-token compaction point → cascades to 250-token level
Previous 250-token compaction point → cascades to 50-token level
Previous 50-token compaction point → cascades to 15-token level
New drift turn → becomes new 1000-token compaction point

This provides increasingly detailed context as you approach the present conversation.

Configuration

Basic Configuration

plugins:
  - module: "codumentor.plugins.context_management"
    class: "ContextManagementPlugin"
    priority: 5
    args:
      enabled: true
      target_token_levels: [15, 50, 250, 1000]
      provide_retrieval_tool: true
      min_turns_before_compaction: 3
      compaction_token_threshold: 1000

Configuration Options

Option	Type	Default	Description
`enabled`	boolean	`true`	Enable/disable the plugin
`target_token_levels`	list	`[15, 50, 250, 1000]`	Token levels for state descriptions
`provide_retrieval_tool`	boolean	`true`	Inject history retrieval tool
`state_store_path`	string	`null`	Custom path for state storage
`state_generation_model`	string	`null`	Model for state generation (default: main model)
`drift_detection_model`	string	`null`	Model for drift detection (default: main model)
`min_turns_before_compaction`	integer	`3`	Minimum turns before compaction can occur
`compaction_token_threshold`	integer	`1000`	Token threshold to trigger compaction
`drift_window_size`	integer	`10`	Number of states to analyze for drift
`estimate_sufficient_tokens`	boolean	`true`	Estimate minimum tokens needed for adequate summary
`ui_verbosity`	string	`"normal"`	Controls which events show as UI cards (see below)

UI Verbosity Levels

The ui_verbosity setting controls which context management events appear as cards in the UI:

Level	Events Shown	Use Case
`"all"`	state_updated, drift_detected, context_compacted	Debugging, understanding plugin behavior
`"normal"`	drift_detected, context_compacted	Default - see significant events
`"minimal"`	drift_detected	Only major topic shifts
`"off"`	(none)	Silent operation

Example:

plugins:
  - module: "codumentor.plugins.context_management"
    class: "ContextManagementPlugin"
    args:
      ui_verbosity: "minimal"  # Only show topic shifts

Sufficient Tokens Estimation

When estimate_sufficient_tokens is enabled (default), the plugin asks the LLM to estimate the minimum tokens needed for an adequate summary of each turn.

The Key Insight

Different conversations have different information density:

A simple "fix this typo" conversation might be fully captured in 30 tokens
A complex architectural discussion might need 800 tokens

The sufficient tokens estimate answers: "What's the minimum size where a reader would understand the key points AND know what details they might want to retrieve?"

This is "retrieval-aware summarization" - the summary should be self-aware of its incompleteness.

How It's Used

Stored per turn: Each turn tracks its sufficient_tokens estimate
Smart level selection: When building cascading summaries, if a smaller level is adequate (based on sufficient_tokens), it's used instead - saving tokens while maintaining adequacy
Included in events: The state_updated event includes sufficientTokens for UI display

Example

Turn 1: "Help me fix a typo" → sufficient_tokens=25 → use 50-token summary
Turn 2: "Explain the auth system" → sufficient_tokens=180 → use 250-token summary
Turn 3: "Design a new microservice" → sufficient_tokens=600 → use 1000-token summary

Retrieval Hints

The plugin also extracts retrieval hints - short descriptions of what details are available but not included in compressed summaries.

Purpose

Hints help readers know what they can ask about. A summary might say "discussed authentication" but hints tell you: "JWT token expiry, refresh flow, error codes" - specific topics you can retrieve if needed.

How It Works

Extracted per turn: After generating summaries, the LLM identifies 2-5 retrievable details
Stored alongside states: Hints are persisted in the state store
Included in cascading summaries: When context is compacted, hints from all compaction points are collected and shown:

[CONVERSATION HISTORY - CASCADING SUMMARIES]
...summaries...

[RETRIEVABLE DETAILS: JWT token handling, error response format, rate limiting config]

[END SUMMARIES - Conversation continues below]

This makes the summary explicitly self-aware of its own incompleteness - the reader knows exactly what's been compressed away.

How It Works

1. Per-Turn State Generation

After each Q/A exchange, the plugin:

Captures the user message and assistant response
Loads previous cumulative state descriptions
Generates updated state descriptions at all token levels (in parallel)
Stores the new states for future reference

2. Drift Detection

The plugin analyzes the sequence of 15-token state descriptions to detect semantic shifts:

[1-5]: User discussing Python decorators and class design
[1-6]: User continuing Python decorators discussion
[1-7]: User now asking about database schema design  ← Drift detected!
[1-8]: User exploring PostgreSQL indexing strategies

When significant drift is detected, compaction points cascade down to compress older content.

3. Context Substitution

Before each LLM call, if compaction is beneficial:

Estimates tokens for messages up to the highest compaction point
If exceeds threshold, builds cascading summary header
Replaces prefix messages with multi-level summaries
Preserves raw messages for recent context

4. History Retrieval Tool

The plugin optionally injects a retrieve_conversation_history tool:

Tool: retrieve_conversation_history
Arguments:
  - start_turn: First turn to retrieve (1-indexed)
  - end_turn: Last turn to retrieve (inclusive)
  - detail_level: "brief" (50 tokens), "medium" (250), "full" (1000), or "raw"

This allows the agent to fetch more details about earlier conversation when needed.

Storage

State is persisted per session:

{state_store_path}/{session_id}/
├── context_state.json    # State descriptions and compaction points
└── raw_messages.jsonl    # Original messages for retrieval

State Format

{
  "session_id": "abc123",
  "compaction_points": {
    "15": 3,
    "50": 5,
    "250": 8,
    "1000": 12
  },
  "current_turn": 15,
  "states": {
    "1": {"15": "...", "50": "...", "250": "...", "1000": "..."},
    "2": {"15": "...", "50": "...", "250": "...", "1000": "..."}
  },
  "sufficient_tokens": {
    "1": 75,
    "2": 120
  },
  "retrieval_hints": {
    "1": ["initial setup", "config options"],
    "2": ["auth flow details", "error handling approach"]
  }
}

UI Events

The plugin emits the following UI events (filtered by ui_verbosity):

Event	Description	Verbosity
`plugin.context_management.state_updated`	State descriptions generated (includes `sufficientTokens`, `retrievalHints`)	`"all"` only
`plugin.context_management.drift_detected`	Semantic drift detected, compaction points cascaded	`"all"`, `"normal"`, `"minimal"`
`plugin.context_management.context_compacted`	Context was compacted before LLM call	`"all"`, `"normal"`

Examples

Example 1: Long Technical Conversation

A conversation that starts with Python, shifts to database design, then to deployment:

Turns 1-5: Python code review → compressed to 15 tokens
Turns 1-10: Python + early DB design → compressed to 50 tokens
Turns 1-15: Python + DB design + schema → compressed to 250 tokens
Turns 1-20: Recent DB + deployment planning → 1000-token summary
Turns 21-25: Current deployment discussion → raw messages

Example 2: Disabled Retrieval Tool

plugins:
  - module: "codumentor.plugins.context_management"
    class: "ContextManagementPlugin"
    args:
      enabled: true
      provide_retrieval_tool: false  # Don't inject tool

Example 3: Custom Token Levels

plugins:
  - module: "codumentor.plugins.context_management"
    class: "ContextManagementPlugin"
    args:
      target_token_levels: [10, 100, 500, 2000]  # Custom levels
      compaction_token_threshold: 2000  # Higher threshold

Comparison with Context Summarization

Feature	Context Summarization	Context Management
Trigger	Context overflow	Every turn
Approach	Reactive truncation	Proactive summarization
State	None	Persistent multi-level
Compression	Single level	Cascading multi-level
Drift detection	No	Yes, LLM-based
Retrieval	No	Yes, via tool
Adaptive sizing	No	Yes, via sufficient_tokens
Self-aware gaps	No	Yes, via retrieval hints

Use Context Summarization for simple overflow handling.
Use Context Management for sophisticated long-conversation support.

Testing

Run tests with:

python -m pytest tests/test_plugins/test_context_management_plugin.py -v

Context Management Plugin

Overview

Cascading Compression Model

Configuration

Basic Configuration

Configuration Options

UI Verbosity Levels

Sufficient Tokens Estimation

The Key Insight

How It's Used

Example

Retrieval Hints

Purpose

How It Works

How It Works

1. Per-Turn State Generation

2. Drift Detection

3. Context Substitution

4. History Retrieval Tool

Storage

State Format

UI Events

Examples

Example 1: Long Technical Conversation

Example 2: Disabled Retrieval Tool

Example 3: Custom Token Levels

Comparison with Context Summarization

Testing

See Also