tests — streaming

Module: tests-streaming Cohesion: 0.80 Members: 0

tests — streaming

The streaming module provides a suite of utilities designed to manage and enhance asynchronous operations, particularly in contexts involving real-time data processing, long-running tasks, and interactions with external services. It focuses on providing robust solutions for chunking content, tracking progress, ensuring resilience, and standardizing event reporting for tool executions.

This documentation covers the core components within the src/streaming directory, drawing insights from their respective test files to accurately describe their functionality and API.

Module Overview

The streaming module addresses several key concerns:

Core Components

The streaming module is composed of several distinct, yet often complementary, sub-modules:

1. Markdown Chunker (markdown-chunker.js)

This module provides utilities for intelligently splitting Markdown text into smaller chunks, respecting structural elements like paragraphs and code blocks. This is particularly useful for processing large language model outputs or user inputs where content needs to be broken down for further analysis or display without breaking semantic units.

Key Components:

How it Works:

The MarkdownChunker maintains an internal buffer and a BlockState to track whether it's currently inside a code block. It processes text line by line, prioritizing splitting at paragraph breaks (\n\n). If preserveCodeBlocks is enabled, it avoids splitting within a code block unless the hardMaxChars limit is severely exceeded, in which case it might force a split. The softMaxChars and hardMaxChars options guide the chunking behavior, with softMaxChars indicating a preferred maximum and hardMaxChars an absolute maximum before a forced split.

2. Progress Tracker (progress-tracker.js)

This module provides a flexible way to track and report progress for operations that may involve multiple stages or iterations. It supports weighted stages and emits events for real-time updates.

Key Components:

How it Works:

ProgressTracker calculates overall progress by considering the weights of each stage. When updateProgress is called, it updates the progress for the active stage, and then recalculates the total progress based on the completed stages and the current stage's progress and weight. It can also track elapsed time and provide estimates for time remaining.

3. Retry Policy (retry-policy.js)

This module provides robust mechanisms for handling transient failures in asynchronous operations, combining retry logic with a circuit breaker pattern to prevent cascading failures.

Key Components:

How it Works:

The retry function catches errors, checks if they are retryable, and if so, waits for a calculated delay before re-executing the operation. The CircuitBreaker monitors failures over a time window. If failures exceed a threshold, it "opens" to prevent further requests, allowing the service to recover. After a resetTimeoutMs, it transitions to half-open, allowing a limited number of "probe" requests to determine if the service has recovered. The RetryManager orchestrates these patterns, providing a unified interface for resilient service interactions.

4. Tool Phases (tool-phases.js)

This module provides a standardized way to track and report the lifecycle and progress of individual tool executions, which is crucial for providing real-time feedback in interactive systems.

Key Components:

How it Works:

A ToolPhaseEmitter is instantiated for each individual tool call. It tracks the state and timing of that specific call. The ToolPhaseManager acts as a central hub, allowing the application to listen to all tool phase events from a single point, regardless of how many tools are executing concurrently. This provides a unified stream of updates for UI or logging purposes.

5. Tool Throttle (tool-throttle.js)

This module provides a generic throttling utility and a specialized throttler for ToolPhaseEvents, ensuring that event consumers are not overwhelmed by rapid updates.

Key Components:

How it Works:

The throttle utility is a classic debouncing/throttling pattern. ToolPhaseThrottler leverages this by maintaining a separate throttled function for each toolCallId. This allows independent throttling of update events for different concurrent tool calls, while ensuring that critical 'start' and 'result' events are always delivered immediately.

Architectural Relationships

The streaming module components are designed to work together, particularly in the context of managing tool executions and their progress. 

graph TD
    subgraph Tool Execution Flow
        A[Tool Call Logic] --> B(ToolPhaseEmitter)
        B -- Emits ToolPhaseEvent --> C(ToolPhaseThrottler)
        C -- Throttled Event --> D(ToolPhaseManager)
        D -- Aggregates & Forwards --> E[Application UI/Logging]
    end

    subgraph Resilience
        F[External Service Call] --> G(RetryManager)
        G -- Uses --> H(CircuitBreaker)
        G -- Uses --> I(Retry Logic)
        G --> F
    end

    subgraph Content Processing
        J[Raw Markdown Input] --> K(MarkdownChunker)
        K -- Emits ChunkResult --> L[Further Processing/Display]
    end

    subgraph Progress Reporting
        M[Long-running Operation] --> N(ProgressTracker)
        N -- Emits ProgressUpdate --> E
    end

    B -- Can use --> N

Key Interactions:

This modular design allows developers to pick and choose the specific streaming utilities they need, while also providing a clear path for integrating them into a cohesive system for managing complex asynchronous workflows.