src — offline

Module: src-offline Cohesion: 0.80 Members: 0

src — offline

The src/offline module provides Code Buddy with robust offline capabilities, allowing it to function and provide value even without an active internet connection. It achieves this through a combination of intelligent response caching, local Large Language Model (LLM) fallback, embedding storage for semantic search, and a resilient request queuing system.

The core of this module is the OfflineMode class, which manages all aspects of offline operation.

Core Concepts

The offline module is designed around several key features:

  1. Response Caching: Stores previous LLM responses to avoid re-querying for identical or highly similar prompts.
  2. Local LLM Fallback: Integrates with local LLM providers (Ollama, llama.cpp, node-llama-cpp, WebLLM) to generate responses when online models are unreachable.
  3. Embedding Cache: Stores vector embeddings of queries and responses, enabling semantic search for cached content.
  4. Request Queuing: When offline, requests that require internet connectivity (e.g., to remote LLMs or APIs) are queued and automatically processed once connectivity is restored.
  5. Automatic Sync: Monitors internet connectivity and triggers processing of queued requests when back online.
  6. Offline-Capable Tools: Provides the infrastructure for tools to leverage local LLMs and cached data.

Architecture Overview

The OfflineMode class acts as a central manager, orchestrating interactions between various internal components and external dependencies. It maintains its state (caches, queue, configuration) persistently on disk and actively monitors network status. 

graph TD
    subgraph OfflineMode Manager
        OM[OfflineMode Class] --> Cfg(OfflineConfig)
        OM --> RC(Response Cache: LRUCache<CachedResponse>)
        OM --> EC(Embedding Cache: LRUCache<CachedEmbedding>)
        OM --> RQ(Request Queue: QueuedRequest[])
        OM --> LLM(Local LLM Integration)
        OM --> Net(Internet Check)
        OM --> Stats(OfflineStats)
    end

    LLM --> LPM(LocalProviderManager)
    LLM --> Ax(Axios for Ollama/llama.cpp)
    RC --> FS(fs-extra: Disk Persistence)
    EC --> FS
    RQ --> FS
    Cfg --> FS
    Net --> Ax
    LPM --> LocalLLMProviders[Local LLM Providers (node-llama-cpp, WebLLM)]
    Ax --> ExternalAPIs[External APIs (Ollama, llama.cpp, Health Checks)]

    OM -- Emits Events --> EventBus[EventEmitter]
    MainApp[Main Application] -- Uses Singleton --> OM

Key Components

OfflineMode Class

The OfflineMode class (src/offline/offline-mode.ts) is the primary entry point and manager for all offline functionalities. It extends EventEmitter to broadcast important status changes and events.

Constructor and Initialization: The constructor new OfflineMode(config) initializes the module with a given configuration (or DEFAULT_CONFIG). It sets up data directories (~/.codebuddy/offline), initializes LRUCache instances for responses and embeddings, and then calls initialize().

The initialize() method performs critical setup:

  1. Ensures necessary directories exist (dataDir, cacheDir, cacheDir/responses, cacheDir/embeddings).
  2. Loads previously saved cache indexes (response-index.json, embedding-index.json) and the request queue (queue.json) from disk.
  3. Performs an initial internet connectivity check via checkInternet().
  4. Starts a periodic internet connectivity check using startInternetCheck().

Configuration (OfflineConfig)

The OfflineConfig interface defines the module's behavior:

Configuration can be updated at runtime using updateConfig(config: Partial), which also persists the changes to config.json.

Response Caching

The module uses an LRUCache (responseCache) to store LLM responses.

An LRUCache (embeddingCache) stores text embeddings.

  1. It first gets or computes the embedding for the input query.
  2. Then, it iterates through all cached responses. For each cached response, it retrieves its associated query embedding from embeddingCache.
  3. It calculates the cosineSimilarity() between the input query's embedding and the cached query's embedding.
  4. Responses exceeding the threshold are returned, sorted by similarity.

Request Queuing

When offline, requests that cannot be fulfilled locally are added to a queue.

Local LLM Integration

The module supports various local LLM providers.

Internet Connectivity Management

Data Persistence

All critical data (configuration, cache indexes, request queue) is persisted to disk within the user's home directory (~/.codebuddy/offline).

Statistics & Monitoring

The OfflineStats interface tracks various metrics:

The getStats() method returns the current statistics. formatStatus() provides a human-readable summary, useful for debugging or user interfaces.

Usage

The OfflineMode instance is typically accessed as a singleton: 

import { getOfflineMode, OfflineConfig } from &#39;./offline-mode.js&#39;;

class="hl-cmt">// Get the singleton instance, optionally with initial config
const offlineManager = getOfflineMode({
  localLLMProvider: &#39;ollama&#39;,
  localLLMModel: &#39;llama3&#39;,
  cacheMaxSize: 1024, class="hl-cmt">// 1GB
});

class="hl-cmt">// Check internet status
console.log(&#39;Is online:&#39;, offlineManager.getStats().isOnline);

class="hl-cmt">// Try to get a cached response
const cached = await offlineManager.getCachedResponse(&#39;What is Code Buddy?&#39;);
if (cached) {
  console.log(&#39;Cached response:&#39;, cached.response);
} else {
  class="hl-cmt">// If offline, queue a request or use local LLM
  if (!offlineManager.getStats().isOnline && offlineManager.getConfig().localLLMEnabled) {
    const localResponse = await offlineManager.callLocalLLM(&#39;Explain offline mode.&#39;);
    console.log(&#39;Local LLM response:&#39;, localResponse);
  } else if (!offlineManager.getStats().isOnline && offlineManager.getConfig().queueRequestsWhenOffline) {
    const requestId = offlineManager.queueRequest(&#39;chat&#39;, { prompt: &#39;What is Code Buddy?&#39; });
    console.log(&#39;Request queued:&#39;, requestId);
  }
}

class="hl-cmt">// Listen for events
offlineManager.on(&#39;online&#39;, () => console.log(&#39;Back online! Processing queue...&#39;));
offlineManager.on(&#39;request:processed&#39;, ({ request }) => console.log(`Processed queued request: ${request.id}`));

class="hl-cmt">// Clean up on application exit
class="hl-cmt">// offlineManager.dispose();

Integration Points

Events

The OfflineMode class extends EventEmitter and emits the following events:

Lifecycle