Script Filter Plugin

The Script Filter plugin removes runs of non-Latin characters (CJK, kana,
hangul, fullwidth forms, …) that a model sometimes leaks into otherwise-Latin
output. It strips them live during streaming (replacing each run with a
placeholder) and, when the turn finishes, optionally **translates the removed
fragments** back into the surrounding language and revises the message in
place.

Configuration

plugins:
  - module: codumentor.plugins.script_filter
    class: ScriptFilterPlugin
    args:
      enabled: true
      scope: both              # both | content | thinking
      placeholder: "□"
      translate: true
      target_language: auto    # 'auto' = match the surrounding text
      request_timeout_seconds: 20.0
      defer_translation: true

Parameters

• enabled (optional): Strip characters at all (default: true)
• scope (optional): Where to strip — "both", "content" (visible answer), or "thinking" (reasoning). Any other value falls back to "both" (default: "both")
• placeholder (optional): Shown in place of each removed run during streaming. "" deletes with no marker and disables translation (no insertion point) (default: "□")
• translate (optional): Translate removed fragments when the turn finishes (default: true)
• target_language (optional): Language to translate into; "auto" detects the dominant surrounding language (default: "auto")
• model_role (optional): Model-profile role for the translation call. Unset = the global agent model
• temperature (optional): Sampling temperature for translation (default: 0.0)
• max_tokens (optional): Translation token budget floor; a large answer may scale above this per chunk. Values below 1 are raised to 1 (default: 1024)
• request_timeout_seconds (optional): Hard timeout per translation chunk. Values below 1 are raised to 1 (default: 20.0)
• coalesce_adjacent (optional): Merge back-to-back placeholders (a phrase split across stream chunks) into one translation unit (default: true)
• defer_translation (optional): Run finish-time translation as a background job (with retry) instead of blocking the turn. Falls back to synchronous translation if no job queue is available (default: true)
• translation_max_attempts (optional): Retry count for a deferred translation failure (defer_translation only). Values below 1 are raised to 1 (default: 3)
• translation_retry_backoff_seconds (optional): Linear backoff between deferred retries (default: 2.0)
• unicode_ranges (optional): List of [start, end] codepoint pairs (hex strings or ints) to strip. Default is the CJK / kana / hangul / fullwidth set.
• pattern (optional): Explicit regex for characters to strip; overrides unicode_ranges when set.

What you see

Settings → Script Filter lets each user toggle stripping and translation,
and set the placeholder and target language.

The placeholder briefly flashes during streaming; the translation lands
after the turn completes, as a post-hoc revision of the message.

Notes

• If translation fails (or placeholder is ""), the placeholders simply remain — the turn is never blocked or corrupted.
• Subagent output is left untouched.
• Legitimate non-Latin characters inside fenced code or string literals are also stripped. There is no code-block exception today.
• A heavily mixed answer is translated in concurrent chunks; a timeout on one chunk leaves that run as a placeholder instead of failing the whole translation.