I am not a native English speaker; this article was translated by AI.

With the Model Context Protocol (MCP) becoming widely adopted, attaching multiple local or remote MCP servers to an agent runtime is standard practice.

In real workloads, however, running multiple MCP servers quickly exposes a fragility issue:

You have five MCP servers in your configuration. Four core local tools for filesystem and terminal work are healthy, but an optional third-party translation or web search server times out or fails due to a local dependency mismatch.

The entire agent runtime panics during initialization. You cannot even ask basic questions or edit local files.

A non-essential auxiliary plugin crash shouldn’t take down the entire core session.

To resolve this, we introduced concurrent startup isolation and graceful degradation for MCP services.

flowchart TD
  subgraph Config[MCP Service Criticality]
    C1[Critical: Local Filesystem & Terminal]
    C2[Optional: Remote Knowledge & Search]
  end

  subgraph Startup[Concurrent Isolation Sandbox]
    C1 --> T1[Tokio Task 1: Critical Service, Strict Validation]
    C2 --> T2[Tokio Task 2: Isolated 3s Timeout]
    C2 --> T3[Tokio Task 3: Isolated 3s Timeout]
  end

  subgraph Outcome[Aggregation & Dynamic Degradation]
    T1 -->|Success| R[Dynamic Tool Registry]
    T2 -->|Timeout / Error| D[Log Diagnostic Warning, Don't Block]
    T3 -->|Success| R
    D -.->|Filter Unavailable Tools| R
    R --> S[Session Launches Cleanly / UI Notice: Degraded Mode]
  end

  Config --> Startup

1. What Was Wrong With the Legacy Flow?

#

Many MCP clients initialize servers via a simple sequential loop:

// Fragile serial loop
for server_config in mcp_servers {
    // If any connect or list_tools call fails, the entire setup returns Err
    let client = McpClient::connect(&server_config).await?;
    let tools = client.list_tools().await?;
    registered_tools.extend(tools);
}

The flaws are obvious:

1. Cumulative startup latency: Total startup time is the sum of every server’s handshake. A single slow server stalling for 5 seconds delays the whole agent by 5 seconds.
2. Missing fault isolation: Core filesystem tools and auxiliary search tools share the same lifecycle. An external timeout escalates into a fatal crash.

2. The Solution

#

The revised flow focuses on three changes:

1. Explicitly Distinguish Critical from Optional Services

#

We added a criticality flag to the configuration schema:

{
  "mcpServers": {
    "filesystem": {
      "command": "agent-mcp-fs",
      "required": true
    },
    "web_search": {
      "command": "uvx",
      "args": ["mcp-server-duckduckgo"],
      "required": false,
      "timeout_ms": 3000
    }
  }
}

• required: true (Critical): Tools the agent cannot function without. Failures will cleanly abort with a clear error.
• required: false (Optional, Default): Auxiliary enhancements. Failures trigger circuit-breaking without affecting the main session.

2. Concurrent Probing with Isolated Timeouts

#

Using Tokio, each MCP server is probed in a dedicated asynchronous task (tokio::spawn) wrapped in a tokio::time::timeout:

• All servers handshake concurrently. Cold startup latency is bounded by the slowest individual server rather than their cumulative sum.
• If an optional server fails to connect within 3 seconds or crashes, the error is caught and marked as Degraded rather than bubbling up.

3. Dynamic Tool Filtering and UI Awareness

#

Once all probing tasks settle:

1. Successfully initialized tools are registered into the session context.
2. Tools from degraded servers are filtered out, preventing the model from hallucinating broken calls.
3. A lightweight notification informs the frontend which optional plugin failed, while keeping the main chat fully operational.

3. Takeaway

#

With startup isolation in place, even if network access is spotty or an MCP plugin config is broken, the agent’s core capabilities launch in sub-seconds.

Plugin systems that depend on external environments must design for failure. An issue in an optional feature should never break core tool availability.

I am not a native English speaker; this article was translated by AI.

When building desktop or web-based agent clients, there is a recurring, annoying UX friction:

The model has finished streaming its last token on screen, but the input box stays disabled with a “Task in progress…” placeholder. You cannot focus the cursor. It takes several seconds to unlock, and switching away from the window mid-stream can sometimes freeze it in a loading state permanently.

This looks like a simple frontend state bug, but tracing through the stack reveals an issue of asynchronous disk persistence blocking the event dispatch pipeline.

flowchart TD
  subgraph Backend[Agent Host Process / Runtime]
    M[Receive Final Text Chunk] --> MC[Emit MessageComplete]
    MC --> P[Heavy Async Persistence: SQLite & Archiving]
    P --> AE[Emit AgentEnd Terminal Event]
  end

  subgraph Legacy[Legacy Serial Pattern]
    L1[Wait for Persistence to Finish] --> L2[Notify Frontend via Long IPC Queue]
    L2 --> L3[Clear isStreaming / Noticeable UI Freeze]
  end

  subgraph Optimized[Decoupled & Reconciled Pattern]
    MC -->|Optimistic: No Pending Tools| UI1[Unlock Input Box Instantly]
    AE -->|Bypass Fast Channel| UI1
    T[10s Periodic Stale Reconciliation] -.->|Handles Dropouts & Blur Edge Cases| UI1
  end

1. Why Does the Input Lock Up?

#

From model output to disk storage, messages pass through three layers:

1. Sampling Runtime: Consumes the SSE stream, producing text deltas and MessageComplete.
2. Host Process (Node.js / IPC Layer): Manages cross-process communication, SQLite writes, image materialization, and JSONL archiving.
3. Renderer Process (Frontend UI): Manages reactive states like isStreaming and input enablement in React/Vue.

Tracing the logs revealed three main bottlenecks:

Culprit 1: Terminal Events Blocked Behind Heavy I/O

#

The legacy implementation waited for all disk writes in the turn to finish before sending agent_end to the UI:

// Legacy flow: Heavy I/O blocks terminal notifications
await persistTurnToSqlite(turnData);      // 50~200ms
await materializeImagesToDisk(images);    // 500ms~2s
await appendSessionJsonl(largePayload);   // 100~500ms

// Only now is the UI notified
emitToRenderer('agent_end', session);

While the user is already reading the final response, the host process is still grinding through disk writes. In large sessions, this easily introduces noticeable delays.

Culprit 2: Head-of-Line Blocking in IPC Queues

#

To preserve message ordering, events pass through an ordered queue (Emit Chain). If the model emits diagnostic logs during finalization, agent_end gets queued behind them.

Culprit 3: Window Defocus Drops

#

When a user switches windows mid-stream, Chromium throttles background timers. If a cross-process packet drops during that transition, the UI permanently misses agent_end, leaving the input locked.

2. The Solution

#

We updated the IPC and frontend state management across three areas:

1. Optimistic Unlocking

#

Input availability should not wait on disk writes. Once the frontend receives MessageComplete and verifies:

• The reply is a genuine terminal turn (EndTurn);
• No background tools are currently executing;
• The user has not requested a stop.

It unlocks the input box immediately, dropping perceived recovery latency to zero.

2. Bypass Channel for Terminal Events

#

Lifecycle events (agent_end, error, cancelled) have higher priority than regular stream deltas. We introduced a fast-track bypass in the IPC bridge:

• Terminal events skip the standard ordered queue and dispatch directly to the renderer.
• Even if SQLite writes or archiving queues are backed up, the UI lifecycle updates without delay.

3. 10-Second Low-Overhead Reconciliation Loop

#

To handle orphaned states from window blur or dropped packets, the frontend runs a 10-second reconciliation check:

useEffect(() => {
  const timer = setInterval(() => {
    // Check for sessions with no activity for >10s still marked as streaming
    reconcileStaleStreamingSessions();
  }, 10_000);
  return () => clearInterval(timer);
}, []);

Any stuck sessions are idempotently cleaned up and synchronized.

3. Takeaway

#

With these changes in place, the input box is ready the moment generation finishes, and focus freezes from window switching are gone.

In rich-client apps, keep user interactions optimistic and fast, run heavy persistence asynchronously in the background, and use periodic reconciliation to ensure eventual consistency.

I am not a native English speaker; this article was translated by AI.

When running complex tasks, one of the worst states an agent can enter is a repetitive execution loop.

For example, if a regex finds nothing, the model might dispatch the exact same grep_search three or four times in a row. Or if a file does not exist, it repeatedly calls view_file on the same path. Each call returns an identical empty result or error, yet the model stubbornly keeps trying.

Legacy approaches to handling loops are usually blunt:

1. Let it run until max_turns exhausts: Wasting dozens of API calls and crashing with a generic “maximum turns exceeded” error.
2. Throw an exception immediately upon repetition: Aborting the task right away. But all the accumulated investigation context and progress vanish with it.

Neither approach works well. We added a Graded Loop Guard to our runtime using a three-tier escalation ladder.

flowchart TD
  subgraph Ingestion[Tool Call Ingestion]
    A[Receive Model Tool Call] --> B[Compute Canonical Args Hash]
    B --> C[Classify Side-Effects: Read-Only vs Mutating]
  end

  subgraph Ladder[Graded Escalation Ladder]
    C --> D{Sequential Duplicate Count}
    D -->|1st Duplicate Count=2| E[Warn: Execute Tool & Append Steering Guidance]
    D -->|2nd Duplicate Count=3| F[Block: Intercept Execution & Return Synthetic Error]
    D -->|3rd Duplicate Count=4| G[Halt: Abort Turn & Expose repeated_tool_calls]
  end

  subgraph Outcome[Outcome & Recovery]
    E --> H[Model Reads Hint & Self-Corrects]
    F --> I[Prevent Wasted IO & Force Strategy Shift]
    G --> J[Preserve Context & Provide Clear Diagnostics]
  end

  Ingestion --> Ladder

1. Why Brute-Force Aborts Fail

#

Determining whether a model is genuinely looping requires a few distinctions:

• Read-Only vs. Mutating Tools: Repeatedly inspecting the same file with read-only tools (grep, view_file) is often benign, so tolerance can be higher. But repeating mutating actions (write_to_file or running write commands) with identical arguments is dangerous.
• Models Can Self-Correct: Often, the model is simply stuck in a temporary rut. If you explicitly remind it in the tool result (“You have run this exact query twice with zero results; please adjust your approach”), modern reasoning models usually pivot autonomously in the next turn.

The guardrail’s design logic is straightforward: guide first, block physically second, halt last.

2. The Three-Tier Escalation Ladder

#

The guard checks invocations in the before_tool_call phase of the main loop:

#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum LoopGuardAction {
    /// Allow: Initial or normal execution
    Allow,
    /// Warn: Inject guidance into tool result, execute normally
    Warn,
    /// Block: Prevent physical execution, return synthetic error
    Block,
    /// Halt: Abort active turn
    Halt,
}

Tier 1: Warn (Steering Guidance Injection)

#

When the same tool with identical normalized arguments appears for the second time in a task:

• The tool executes normally.
• When the tool finishes, the runtime appends an automated notice to the end of the ToolResult:

  [System Notice] You have executed this tool repeatedly with identical arguments without progress. Do not repeat identical calls. Adjust your search pattern, change the target path, or switch tools.

• Most models immediately adjust their strategy on the next turn.

Tier 2: Block (Synthetic Interception)

#

If the model ignores the warning and calls the exact same tool a third time:

• The guard short-circuits the call, skipping physical file I/O or command execution entirely.
• A structured error (ToolResultContent::Error) is synthesized at the protocol layer, informing the model that repeated execution was blocked.
• LLM protocols mandate that every tool_use must have an accompanying tool_result. Returning a synthetic error preserves protocol parity while preventing wasted computation.

Tier 3: Halt (Safe Turn Abort)

#

If duplicate calls reach four (HALT_AFTER = 4), the model is stuck:

• The runtime ends the current turn and triggers RepeatedToolCalls.
• The UI and logs explicitly identify the offending tool name and arguments.
• Prior session history remains intact on disk, allowing the user to guide the agent with a follow-up prompt without re-running from scratch.

3. Canonical Argument Matching

#

To prevent false negatives from varying JSON key order, the guard sorts JSON object keys recursively before computing canonical_args_hash. This ensures {"a": 1, "b": 2} and {"b": 2, "a": 1} yield identical fingerprints.

Tools are also categorized as ReadOnly or Mutating, applying tighter thresholds to state-modifying actions.

4. Takeaway

#

In real workloads, most edge-case loops resolve autonomously during the Warn tier.

Replacing abrupt aborts with a progressive ladder of warnings, synthetic blocks, and clean halts preserves token budgets while meaningfully boosting task completion rates.

I am not a native English speaker; this article was translated by AI.

When integrating diverse models for coding agents, transport quirks are bound to happen.

One recurring issue is when a model, instead of returning structured function calls via the standard tool_calls payload, dumps raw XML markup directly into the text stream (content or output_text.delta).

In certain reasoning models (such as DeepSeek derivatives or proxied gateways), it often looks like this:

＜ＤＳＭＬ＜tool_calls＞
＜ＤＳＭＬ＜invoke name="read_file"＞
＜ＤＳＭＬ＜parameter name="path"＞"src/main.rs"＜／ＤＳＭＬ＜／parameter＞
＜／ＤＳＭＬ＜／invoke＞
＜／ＤＳＭＬ＜／tool_calls＞

If the transport layer simply passes these tokens through to the UI as plain text:

1. The agent loop receives zero tool_call events, stalling the task.
2. The user’s screen gets littered with unparsed full-width XML tags.

To handle this cleanly in our runtime, we built a cross-chunk streaming DSML recovery state machine.

flowchart TD
  subgraph Ingestion[Streaming Input Token Chunks]
    A[Chunk 1: Protocol Prefix] --> B[Chunk 2: invoke name=read_file]
    B --> C[Chunk 3: parameter name=path]
    C --> D[Chunk 4: Protocol Closing Tag]
  end

  subgraph StateMachine[DSML Streaming State Machine]
    S1[Detect Prefix: ＜ＤＳＭＬ＜] --> S2{Is Prompt Example?}
    S2 -->|Yes| S3[Disable Recovery / Stream as Text]
    S2 -->|No| S4[Capture Mode / Hold Text Output]
    S4 --> S5[Buffer Chunks & Assemble Tags]
    S5 --> S6{Is Markup Valid & Closed?}
    S6 -->|No or Over Limit| S7[Fail-Closed / Emit Safe Error]
    S6 -->|Yes| S8[Extract Tool Name & Parameter Pairs]
  end

  subgraph Dispatch[Protocol Conversion & Dispatch]
    S8 --> E1[Validate against Tool Schema & Deserialize]
    E1 --> E2[Synthesize ToolCall & ToolUse Events]
    E2 --> E3[Agent Loop Executes Real Tool]
  end

  Ingestion --> StateMachine

1. Where the Complexity Lies

#

If you receive a single, complete HTTP response body, extracting the tags via regex or an XML parser is straightforward. But agents require low-latency streaming text, which introduces several constraints:

1. Chunk Fragmentation: Output arrives token-by-token. A tag like ＜ＤＳＭＬ＜invoke might arrive fragmented as ["＜", "ＤＳ", "ＭＬ＜in", "voke"] across four separate network packets. Single-chunk regex matching is ineffective.
2. Hold Buffers: When receiving a partial prefix (like a standalone ＜), we cannot stream it immediately to the client (in case it turns out to be markup). But we also cannot hold it indefinitely; normal text must flush immediately once verified.
3. User Prompt Examples: If a user is explicitly discussing DSML syntax (e.g., “What does ＜ＤＳＭＬ＜ mean?”), the state machine must recognize this and disable recovery, rather than attempting to execute quoted examples as real system commands.
4. Native vs. DSML Conflicts: If a model returns both native structured tool calls and raw DSML text in the same turn, we fail closed to prevent duplicate executions.

2. State Machine Design & Intermediate Representation

#

To support multiple providers (OpenAI-compatible endpoints, Responses APIs), the state machine operates on a decoupled Intermediate Representation:

pub enum DsmlOutcome {
    /// Confirmed user-visible text, released for frontend streaming
    Text(String),
    /// Successfully captured and parsed complete tool invocations
    ToolCalls(Vec<DsmlToolCall>),
}

pub struct DsmlToolCall {
    pub id: String,
    pub name: String,
    pub arguments: String, // Normalized standard JSON string
}

The provider passes incoming text chunks into the state machine:

• It maintains an internal capture_buffer.
• Once ＜ＤＳＭＬ＜ is detected, it switches to capture mode, pausing downstream TextDelta emissions.
• A DSML_CAPTURE_LIMIT (256KB) guards against memory exhaustion from malformed output.

3. Schema Validation & Event Synthesis

#

When closed tags are parsed, and nodes are extracted:

1. Verify the tool name exists in the current registry.
2. Parse non-string parameters into valid JSON values.
3. Validate against the tool’s registered JSON Schema.
4. Synthesize native ToolCallStart, ToolInputDelta, and ToolUse events.

From the agent loop’s perspective, this is indistinguishable from standard provider tool calls, routing directly into normal tool execution.

4. Takeaway

#

When building agent runtimes against varied model endpoints, output formatting anomalies are inevitable. Absorbing these quirks in the transport adapter layer keeps higher-level agent state machines clean and dependable.

I am not a native English speaker; this article was translated by AI.

When running long agent tasks, context grows fast.

Refactoring code, checking logs, running test suites—after dozens of turns, tokens quickly approach their limit. The simplest legacy approach was single-pass compaction: when limits are crossed, stop the main loop, send the entire message history to an LLM for a global summary, and swap out old messages.

In practice, several problems surfaced quickly:

1. Main-loop pauses: Sending tens of kilobytes of history for an LLM summary often takes seconds or tens of seconds. The client input box just sits in a loading state.
2. Lost execution details: Once condensed into prose, exact error line numbers, file paths, and stray arguments vanish. When subsequent steps need them, the model either re-queries from scratch or hallucinates.
3. Broken KV caching: Cramming the summary into the System Prompt mutates the prompt prefix on provider servers, busting Prompt Caching completely and driving up both TTFT and API costs.
4. Wasted summaries when pruning would suffice: Often, context blows up simply because a tool dumped a massive git diff or build log. Truncating old tool outputs reclaims tens of thousands of tokens without needing an LLM summary at all.

I overhauled our agent runtime’s context management over the past three days. Here is what changed.

flowchart TD
  subgraph Prefire[Phase 1: Pass 1 Background Prefire]
    A[Tokens hit 90% threshold] --> B[Slice 95% prefix history]
    B --> C[Async background LLM summary]
    C --> D[Cache NOTE1 note and fingerprint]
  end

  subgraph Foreground[Foreground Main Loop]
    U[Normal turns and tool calls] --> E[Generate new turns]
  end

  subgraph Compaction[Phase 2: Pass 2 Compaction]
    F[Tokens cross 100% threshold] --> G{Fingerprint matches?}
    G -->|Match| H[Merge NOTE1 with tail delta]
    G -->|Drift or Failure| I[Fallback to full single-pass summary]
    H --> J[Generate final summary]
    I --> J
    J --> K[Inject standalone summary turn & JSONL pointer]
  end

  Prefire -.->|Runs in background| Foreground
  Foreground --> F

1. Two-Pass Prefire: Moving Heavy Summaries to the Background

#

To keep the UI responsive, the heaviest work must happen in the background before the hard limit is hit.

Phase 1: Prefix Prefire (Pass 1)

#

We added a safety margin (prefire_margin_tokens, defaulting to a 10% buffer).

When context usage reaches 90% of the threshold, the main loop keeps running while spawning a background task:

• Slice the prefix: Take the oldest 95% of messages. Slicing must strictly protect tool_use and tool_result boundaries without severing unclosed invocations.
• Generate NOTE1: The prefix is summarized into a structured note (NOTE1), and a hash fingerprint of that prefix is stored in cache.
• The foreground streams text normally without interruption.

Phase 2: Incremental Compaction (Pass 2)

#

When subsequent turns push total tokens past the 100% threshold, formal compaction kicks in:

• Verify the fingerprint. If the prefix hasn’t changed and the background task finished, grab the cached NOTE1.
• Instead of feeding the whole history to the model, we only send NOTE1 plus the few tail messages produced after Pass 1.
• If the fingerprint drifted or Pass 1 failed, it cleanly falls back to standard single-pass compaction.

This cuts the tokens sent during active compaction by over 80%, dropping main-thread wait times to milliseconds.

2. Compaction Is Not Deletion: Recovery Hints Pointing to Local JSONL

#

Compacting history shouldn’t mean destroying raw details.

1. Local append-only source of truth: All raw messages, arguments, and full outputs are continuously written to a local session JSONL file and are never deleted.
2. Recovery pointers: The block automatically includes a tag containing the session’s absolute JSONL path:

   History has been summarized. Full raw records remain in session.jsonl. If subsequent steps require exact line numbers, error traces, or command outputs, inspect this file directly with read tools.

3. Leave the System Prompt alone: Summaries are injected as standalone conversation turns. The System Prompt stays frozen, keeping server-side Prompt Caching hit rates high.

3. Prune First Before Calling an LLM Summary

#

Often, invoking an LLM for summarization is unnecessary.

Before running full compaction, we execute trim_old_tool_results to truncate verbose outputs from older turns into short previews. If pruning frees enough tokens, we set context_changed = true and retry immediately, saving an expensive summary call.

We also resolved an edge case: when a turn’s final assistant reply (EndTurn) pushes context over the threshold, compaction used to wait until the user’s next prompt. Now, the runtime compacts and persists checkpoints immediately between MessageComplete and TurnEnd, ensuring cold reboots wake up to clean context.

4. Real-World Telemetry

#

Testing over a continuous 47-hour session with 1,000 requests and 490 million input tokens yielded clear metrics:

Average per-turn tokens dropped by 40%, and context stabilized around 280K instead of ballooning uncontrollably. Beyond a minor ~20K cache miss right after compaction restarts, subsequent turns sustained 99% to 100% prompt cache hit rates.

Keeping long-running agents reliable comes down to getting these state transitions and caching boundaries right.