src — persistence

Module: src-persistence Cohesion: 0.80 Members: 0

src — persistence

The src/persistence module is responsible for managing the long-term storage, retrieval, and manipulation of conversation data within Code Buddy. It provides mechanisms for persisting chat sessions, handling conversation branches, exporting/replaying sessions, and ensuring data integrity through file locking.

Core Concepts: Sessions, Branches, and Exports

Before diving into the components, it's important to understand the distinct concepts of "sessions," "branches," and "exported sessions" as handled by this module:

  1. Sessions (Managed by SessionStore):

  1. Conversation Branches (Managed by ConversationBranchManager):

  1. Exported Sessions (Managed by SessionRecorder, SessionExporter, SessionPlayer):

Architecture Overview

The persistence module is composed of several distinct classes, each with a specialized role. SessionStore and ConversationBranchManager handle the live, mutable state of conversations, while SessionRecorder, SessionExporter, and SessionPlayer deal with immutable snapshots for export and replay. SessionLock provides a critical utility for SessionStore to prevent data corruption. 

graph TD
    subgraph Live Conversation Management
        SS[SessionStore] -->|Persists & Loads| SessionFiles(Session JSON Files)
        SS -->|Uses for concurrency| SL(SessionLock)
        SS -->|Partially writes to| DB(SQLite Database)
        CBM[ConversationBranchManager] -->|Persists & Loads| BranchFiles(Branch JSON Files)
    end

    subgraph Session Export & Replay
        SR[SessionRecorder] -->|Records messages into| ExportedSession(In-memory Session)
        ExportedSession --> SE[SessionExporter]
        ExportedSession --> SP[SessionPlayer]
        SE -->|Outputs| JSON(JSON String)
        SE -->|Outputs| MD(Markdown String)
        SE -->|Outputs| HTML(HTML String)
        SP -->|Consumes| JSON
    end

    subgraph Utilities
        SPicker[SessionPicker] -->|Reads from| SessionFiles
    end

    subgraph External Dependencies
        SS -- Calls --> GRT[generateConversationTitle (utils)]
        SE -- Calls --> DRE[DataRedactionEngine (security)]
        SL -- Calls --> Process(OS Process API)
    end

    style SS fill:#e0f7fa,stroke:#00796b,stroke-width:2px
    style CBM fill:#e0f7fa,stroke:#00796b,stroke-width:2px
    style SR fill:#fff3e0,stroke:#ff8f00,stroke-width:2px
    style SE fill:#fff3e0,stroke:#ff8f00,stroke-width:2px
    style SP fill:#fff3e0,stroke:#ff8f00,stroke-width:2px
    style SL fill:#e8f5e9,stroke:#4caf50,stroke-width:2px
    style SPicker fill:#f3e5f5,stroke:#9c27b0,stroke-width:2px
    style SessionFiles fill:#f5f5f5,stroke:#9e9e9e,stroke-dasharray: 5 5
    style BranchFiles fill:#f5f5f5,stroke:#9e9e9e,stroke-dasharray: 5 5
    style ExportedSession fill:#f5f5f5,stroke:#9e9e9e,stroke-dasharray: 5 5
    style DB fill:#f5f5f5,stroke:#9e9e9e,stroke-dasharray: 5 5

Key Components

SessionStore (src/persistence/session-store.ts)

The SessionStore is the primary component for managing the persistence of Code Buddy's main conversation sessions. It handles saving and loading session data, including messages and metadata, to disk.

Key Features:

Usage Example: 

import { getSessionStore } from './persistence/session-store.js';
import type { ChatEntry } from '../agent/types.js';

const sessionStore = getSessionStore();

async function manageSession() {
  class="hl-cmt">// Create a new session
  const newSession = await sessionStore.createSession("My New Project Chat");
  console.log(`Created session: ${newSession.id}`);

  class="hl-cmt">// Add a message
  const userMessage: ChatEntry = {
    type: 'user',
    content: 'Hello Code Buddy, how can I refactor this?',
    timestamp: new Date(),
  };
  await sessionStore.addMessageToCurrentSession(userMessage);

  class="hl-cmt">// Load a session
  const loadedSession = await sessionStore.loadSession(newSession.id);
  if (loadedSession) {
    console.log(`Loaded session "${loadedSession.name}" with ${loadedSession.messages.length} messages.`);
  }

  class="hl-cmt">// List recent sessions
  const recent = await sessionStore.getRecentSessions(5);
  console.log("Recent sessions:\n", sessionStore.formatSessionList());

  class="hl-cmt">// Export to Markdown
  const markdownPath = await sessionStore.exportSessionToFile(newSession.id, 'my-session.md');
  console.log(`Session exported to ${markdownPath}`);
}

manageSession();

ConversationBranchManager (src/persistence/conversation-branches.ts)

The ConversationBranchManager enables Git-like branching and merging of conversation histories. This is crucial for exploring different solutions or conversational paths without losing the original context.

Key Features:

Usage Example: 

import { getBranchManager } from './persistence/conversation-branches.js';
import { CodeBuddyMessage } from '../codebuddy/client.js';

const branchManager = getBranchManager();

async function manageBranches() {
  class="hl-cmt">// Ensure 'main' branch exists and is current
  branchManager.checkout("main");

  class="hl-cmt">// Add some initial messages
  branchManager.addMessage({ role: 'user', content: 'Initial query' } as CodeBuddyMessage);
  branchManager.addMessage({ role: 'assistant', content: 'Initial response' } as CodeBuddyMessage);

  class="hl-cmt">// Fork a new branch
  const featureBranch = branchManager.fork("feature-a");
  console.log(`Forked to branch: ${featureBranch.name} (${featureBranch.id})`);

  class="hl-cmt">// Add messages to the feature branch
  branchManager.addMessage({ role: 'user', content: 'On feature-a: new question' } as CodeBuddyMessage);
  branchManager.addMessage({ role: 'assistant', content: 'On feature-a: new answer' } as CodeBuddyMessage);

  class="hl-cmt">// Checkout back to main
  branchManager.checkout("main");
  branchManager.addMessage({ role: 'user', content: 'On main: another question' } as CodeBuddyMessage);

  class="hl-cmt">// Merge feature-a into main
  branchManager.merge(featureBranch.id, "append");
  console.log(`Merged ${featureBranch.name} into main.`);

  console.log(branchManager.formatBranches());
  console.log(branchManager.formatBranchTree());
}

manageBranches();

SessionRecorder, SessionExporter, SessionPlayer (src/persistence/session-export.ts)

These three classes work together to provide advanced capabilities for recording, exporting, and replaying conversation sessions. They operate on an ExportedSession data structure, which is a self-contained snapshot of a conversation.

SessionRecorder

Records messages, tool calls, and metadata into an in-memory ExportedSession object.

Key Features:

SessionExporter

Takes an ExportedSession and converts it into various output formats.

Key Features:

SessionPlayer

Loads an ExportedSession (typically from a JSON file) and replays it, simulating the original conversation flow.

Key Features:

Usage Example (Recorder/Exporter): 

import { getSessionRecorder, SessionExporter } from './persistence/session-export.js';

const recorder = getSessionRecorder(); class="hl-cmt">// Global singleton
recorder.start();

recorder.addUserMessage("What's the capital of France?");
recorder.addAssistantMessage("Paris.");
recorder.updateUsage(10, 0.0001);
recorder.createCheckpoint("Initial Q&A");

const sessionData = recorder.getSession();
recorder.stop();

const exporter = new SessionExporter();
const markdownOutput = exporter.export(sessionData, { format: 'markdown' });
console.log(markdownOutput);

class="hl-cmt">// To export to file:
class="hl-cmt">// await exporter.exportToFile(sessionData, 'exported-session.html', { format: 'html' });

Usage Example (Player): 

import { SessionPlayer } from './persistence/session-export.js';
import * as fs from 'fs/promises';

async function replayExample() {
  class="hl-cmt">// Assume 'exported-session.json' exists from a previous export
  const player = new SessionPlayer();
  await player.loadFromFile('exported-session.json');

  player.on('message', ({ message, index }) => {
    console.log(`[${index}] ${message.role}: ${message.content.slice(0, 50)}...`);
  });
  player.on('replay:ended', () => console.log('Replay finished.'));

  await player.replay({ speed: 2, pauseAtToolCalls: true }); class="hl-cmt">// 2x speed, pause on tools
  player.dispose();
}

class="hl-cmt">// replayExample();

SessionLock (src/persistence/session-lock.ts)

Provides a file-based locking mechanism to prevent multiple processes from concurrently writing to the same session file, which could lead to data corruption.

Key Features:

Usage Example (Internal to SessionStore): 

class="hl-cmt">// Inside SessionStore.saveSession:
import { withSessionLock } from './session-lock.js';

class="hl-cmt">// ...
const filePath = this.getSessionFilePath(session.id);
await withSessionLock(filePath, async () => {
  await fsPromises.writeFile(filePath, JSON.stringify(data, null, 2));
});
class="hl-cmt">// ...

SessionPicker (src/persistence/session-picker.ts)

A utility class for browsing and formatting session entries, primarily for CLI or UI display.

Key Features:

Usage Example: 

import { SessionPicker, SessionPickerEntry } from './persistence/session-picker.js';

const entries: SessionPickerEntry[] = [
  { id: 'session_abc123', name: 'Refactor Utility', branch: 'main', messageCount: 25, lastAccessed: Date.now() - 100000, tags: ['refactor'] },
  { id: 'session_def456', name: 'New Feature Idea', branch: 'feature-x', messageCount: 10, lastAccessed: Date.now() - 50000, tags: ['feature'] },
];

const picker = new SessionPicker(entries);

console.log(picker.formatTable(picker.getEntries()));
class="hl-cmt">// Output:
class="hl-cmt">// ID        Name                 Branch          Messages  Last Used
class="hl-cmt">// --------------------------------------------------
class="hl-cmt">// session_d New Feature Idea     feature-x       10        2023-10-27
class="hl-cmt">// session_a Refactor Utility     main            25        2023-10-27

Data Models

The persistence module defines several key interfaces to structure conversation data: 

    interface ConversationBranch {
      id: string;
      name: string;
      parentId?: string;
      parentMessageIndex?: number;
      messages: CodeBuddyMessage[]; class="hl-cmt">// From ../codebuddy/client.js
      createdAt: Date;
      updatedAt: Date;
      metadata?: BranchMetadata;
    }
    interface Session {
      id: string;
      name: string;
      workingDirectory: string;
      model: string;
      messages: SessionMessage[];
      createdAt: Date;
      lastAccessedAt: Date;
      metadata?: SessionMetadata;
    }
    interface SessionMessage {
      type: 'user' | 'assistant' | 'tool_result' | 'tool_call' | 'reasoning' | 'plan_progress' | 'steer' | 'diff_preview';
      content: string;
      timestamp: string;
      toolCallName?: string;
      toolCallSuccess?: boolean;
      taskState?: Record<string, unknown>; class="hl-cmt">// For cross-session continuity
    }
    interface ExportedSession {
      version: string;
      exportedAt: number;
      metadata: SessionMetadata;
      messages: SessionMessage[];
      checkpoints?: SessionCheckpoint[];
    }

Integration Points & Dependencies

The persistence module interacts with several other parts of the Code Buddy codebase:

Usage Patterns & Entry Points

Developers should primarily interact with this module through its singleton instances and factory functions:

Important Note on Exports: The src/persistence/index.ts file only re-exports conversation-branches.ts. This means that SessionStore, SessionRecorder, SessionExporter, SessionPlayer, SessionLock, and SessionPicker must be imported directly from their respective files (e.g., import { getSessionStore } from './persistence/session-store.js';).

Contribution Guidelines