src — scripting
src — scripting
The src/scripting module provides the core infrastructure for executing custom scripts within the Code Buddy environment. It unifies the capabilities of previous "FileCommander Script" (FCS) and "Buddy Script" systems, offering a rich set of built-in functions, AI integration, file system access, and shell command execution. Scripts written for this module typically use the .bs (primary) or .fcs (backward-compatible alias) extensions.
This module is designed to be extensible, allowing developers to define new language features, integrate with external tools, and provide a sandboxed environment for script execution.
Architecture Overview
The scripting module follows a classic compiler-interpreter pipeline:
- Lexer: Converts raw script source code into a stream of tokens.
- Parser: Transforms the token stream into an Abstract Syntax Tree (AST).
- Runtime: Executes the AST, interpreting statements and expressions within a defined context.
- Built-ins & Bindings: Provide the core functionality and external integrations available to scripts.
graph TD
A[Script Source Code] --> B(Lexer: src/scripting/lexer.ts);
B --> C{Tokens};
C --> D(Parser: src/scripting/parser.ts);
D --> E{Abstract Syntax Tree (AST)};
E --> F(Runtime: src/scripting/runtime.ts);
F --> G(Built-ins: src/scripting/builtins.ts);
F --> H(CodeBuddy Bindings: src/scripting/codebuddy-bindings.ts);
G & H --> F;
F --> I{ScriptResult};
Key Components
1. Lexer (src/scripting/lexer.ts)
The FCSLexer class is responsible for the lexical analysis phase. It takes the raw script source code as input and produces an array of Token objects.
Key Features:
- Tokenization: Identifies keywords, identifiers, literals (strings, numbers, booleans, null), operators, and punctuation.
- Indentation Handling: Tracks
IndentandDedenttokens, crucial for block-scoped languages that use whitespace for structure. - Shebang Support: Skips the initial
#!line common in executable scripts. - Extended Literals: Supports hex (
0x), binary (0b), and scientific notation for numbers, as well as template strings with interpolation (\${}). - Comments: Skips single-line (
//,#) and multi-line (/ ... /) comments. - Decorators: Recognizes
@for decorators.
The tokenize(source: string) function is the primary entry point for the lexer.
2. Parser (src/scripting/parser.ts)
The FCSParser class performs syntactic analysis, taking the token stream from the lexer and constructing an Abstract Syntax Tree (AST). The parser uses a recursive descent approach.
Key Features:
- AST Construction: Builds a hierarchical representation of the script's structure, defining nodes for programs, statements, and expressions (e.g.,
Program,VarDeclaration,CallExpr,BinaryExpr). - Language Constructs: Supports a wide range of statements and expressions, including:
- Declarations:
var,let,const,func/function,class,import,export,test. - Control Flow:
if/else,while,for(for-in style),forCStyle(C-stylefor (init; test; update)),return,break,continue,try/catch/finally,throw. - Expressions: Assignments (
=,+=, etc.), ternary (? :), pipeline (|>), logical (&&,||), equality (==,!=), comparison (<,>,<=,>=), arithmetic (+,-,,/,%,*), unary (!,-), postfix ((),[],.),await, literals, identifiers, arrays, dictionaries, and lambda/arrow functions. - String Interpolation: Parses expressions embedded within template strings.
- Recursion Guard: Includes
createLoopGuardto prevent stack overflows on deeply nested or malformed input, throwingLoopTimeoutErrorifMAX_RECURSION_DEPTHis exceeded.
The parse(tokens: Token[]) function is the primary entry point for the parser.
3. Runtime (src/scripting/runtime.ts)
The FCSRuntime class is the interpreter that executes the AST generated by the parser. It manages the script's execution context, handles control flow, and interacts with built-in functions and external bindings.
Key Features:
- Context Management: Uses a scope chain (
RuntimeContext) to manage variables and functions, allowing for lexical scoping. - Asynchronous Execution: Supports
async/awaitnatively, enabling scripts to perform non-blocking operations (e.g., AI calls, shell commands). - Statement & Expression Execution: Recursively traverses the AST, executing statements (
executeNode) and evaluating expressions (evaluate). - Control Flow Signals: Uses special
ReturnSignal,BreakSignal, andContinueSignalclasses to manage control flow within loops and functions. - Built-in & Binding Integration: Initializes the script's global context with functions from
createBuiltinsandcreateGrokBindings. - AST Normalization: Includes
normalizeNodeto handle backward compatibility with older "Buddy Script" AST formats, mapping old type and field names to the current FCS format. - Timeout Enforcement: Monitors script execution time and throws an error if
config.timeoutis exceeded. - Test Execution: Executes
TestDeclarationnodes and records results intestResults. - Output Capture: Collects all
printstatements into anoutputarray within theScriptResult.
The execute(program: Program) method is the main entry point for running an AST.
4. Built-in Functions (src/scripting/builtins.ts)
The createBuiltins function generates a comprehensive set of standard functions and objects available to all scripts. These are fundamental operations that don't require specific Code Buddy integrations.
Categories of Built-ins:
- Core I/O:
print,println,input. - Type Conversion:
int,float,num,str,bool,array. - Collection Functions:
len,range,enumerate,zip,map,filter,reduce,find,every,some,sort,reverse,slice,concat,push,pop,shift,unshift,includes,indexOf,join,split,keys,values,entries. - String Functions:
upper,lower,trim,ltrim,rtrim,startsWith,endsWith,contains,replace,replaceAll,substr,charAt,repeat,padStart,padEnd,format,match. - Math Functions:
abs,ceil,floor,round,sqrt,pow,min,max,sin,cos,tan,log,log10,exp,random,randomInt,sum,avg,PI,E. Also provides aMathnamespace. - Date/Time Functions:
now,date,time,datetime,timestamp,formatDate. Also provides aDatenamespace. - JSON Functions:
jsonEncode,jsonDecode, and ajson/JSONnamespace withparseandstringify. - Console:
console.log,console.info,console.warn,console.error. - File Operations: (Enabled by
config.enableFileOps)readFile,writeFile,appendFile,fileExists,isFile,isDir,mkdir,rmdir,remove,rename,copy,listDir,glob,fileSize,fileMtime,basename,dirname,extname,joinPath,resolvePath. Also provides afilenamespace with similar functions. These functions usefsandpathmodules and respectconfig.workdirandconfig.dryRun. - Shell/Bash Functions: (Enabled by
config.enableBash)exec,shell. Also provides abashnamespace withrun,exec,spawn. These functions usechild_processand respectconfig.workdirandconfig.dryRun. - AI Operations: (Enabled by
config.enableAI)ai.ask,ai.chat,ai.complete, and a flatgrokalias. These functions lazily load and interact with theCodeBuddyAgent(viagetOrCreateAgent) and requireGROK_API_KEY. - Environment & Config:
env.get,env.set,env.all,cwd. - Utility Functions:
typeof,isNull,isNumber,isString,isBool,isArray,isDict,isFunction,sleep,exit,assert,expect.
5. CodeBuddy Bindings (src/scripting/codebuddy-bindings.ts)
The createGrokBindings function provides specialized functions that integrate scripts directly with Code Buddy's internal capabilities, such as AI interaction, tool execution, and session management. These bindings are designed to give scripts powerful control over the Code Buddy environment.
Namespaces:
grok: AI integration (similar toaiin built-ins but with conversation history).ask(prompt: string): Sends a prompt to the AI for completion.chat(message: string): Engages in a conversational chat with the AI, maintaining history.generate(prompt: string, language?: string): Generates code based on a prompt.explain(code: string): Explains provided code.review(code: string): Reviews code for issues.clearHistory(),history(): Manage chat history.tool: Advanced file system and code search operations.read(filePath: string, startLine?: number, endLine?: number): Reads file content, optionally by line range.write(filePath: string, content: string): Writes content to a file.edit(filePath: string, oldText: string, newText: string): Replaces specific text in a file.multiEdit(filePath: string, edits: Array<{ old: string; new: string }>)`: Applies multiple text replacements.glob(pattern: string): Finds files matching a glob pattern.grep(pattern: string, filePattern?: string): Searches for patterns in files usinggrep.rg(pattern: string, filePattern?: string): Searches for patterns in files usingripgrep(falls back togrep).ls(dirPath?: string): Lists directory contents.stat(filePath: string): Gets file statistics (size, mtime, isDir).context: Manages files included in the AI's context.add(pattern: string): Adds files matching a pattern to the context.remove(pattern: string): Removes files from the context.clear(): Clears all context files.list(): Lists current context files.content(): Returns the concatenated content of all context files.size(): Returns the number of files in context.agent: High-level agent control functions.run(task: string): Instructs the agent to perform a task.parallel(tasks: string[]): Runs multiple tasks concurrently.securityReview(targetPath?: string): Initiates a security review.codeReview(filePath?: string): Initiates a code review (optionally on a specific file or git diff).generateTests(filePath: string): Generates tests for a given file.refactor(filePath: string, instructions: string): Refactors code based on instructions.mcp: Multi-tool Communication Protocol (MCP) integration.servers(): Lists available MCP servers.tools(server: string): Lists tools on a specific server.call(server: string, toolName: string, args: Record: Calls an MCP tool.) git: Git repository operations.status(),diff(file?: string),add(pattern: string),commit(message: string),log(count?: number),branch().session: Script session management.save(name?: string): Saves the current conversation history and context.list(): Lists saved sessions.load(name: string): Loads a previously saved session.
The setCodeBuddyClient and setMCPManager functions are used by the main application to inject the necessary client instances for these bindings to function.
6. Script Manager (src/scripting/index.ts)
This file serves as the public API for the entire scripting module, consolidating exports and providing high-level utility functions for script management.
Key Functions & Classes:
executeScript(source: string, config: Partial: Executes a script directly from its source code.) executeScriptFile(filePath: string, config: Partial: Reads a script from a file and executes it.) validateScript(source: string): Checks a script for lexical and parsing errors without execution.formatAST(ast: Program): Pretty-prints an AST (primarily for debugging).createScriptTemplate(name: string, description: string): Generates a boilerplate script file.isBuddyScript(filePath: string): Checks if a given file path corresponds to a Code Buddy script.parseScript(source: string): Tokenizes and parses a script, returning both tokens and the AST.CodeBuddyScriptManager: A singleton class (getScriptManager()) that provides:load(filePath: string): Loads and parses a script, caching its AST.execute(filePath: string, config: Partial: Executes a script file and records its history.) clearCache(): Clears cached ASTs.getHistory(): Retrieves execution history.listScripts(dir?: string): Lists Code Buddy scripts in a directory.
Execution Flow
When a script is executed (e.g., via executeScript or executeScriptFile):
- The script source is passed to
tokenize(fromlexer.ts) to produce a list ofTokenobjects. - These tokens are then passed to
parse(fromparser.ts) to construct anAstNoderepresenting theProgram. - A new
FCSRuntimeinstance is created, configured withCodeBuddyScriptConfig. - The
FCSRuntimeinitializes itsglobalContextby callingcreateBuiltinsandcreateGrokBindings, injecting all available functions and objects. - The
runtime.execute(program)method is called, which iterates through the AST's statements. - For each statement or expression,
executeNodeorevaluateis called, recursively traversing the AST and performing the corresponding operations. Asynchronous operations (like AI calls or shell commands) are handled usingawait. - The runtime captures all
printoutput and any errors. IfTestDeclarationnodes are present, their results are also collected. - Finally, a
ScriptResultobject is returned, containing the success status, output, return value, duration, and test results.
Integration Points
The src/scripting module is a central piece of the Code Buddy application, integrating with various other components:
- Command Handlers (
commands/handlers/script-handlers.ts): The primary interface for the application to interact with scripts, callingexecuteScriptAsync,validateScript,createScriptTemplate, andgetScriptManager. - AI Agent (
src/agent/codebuddy-agent.ts): ThegetOrCreateAgentfunction inbuiltins.tsdynamically imports and instantiatesCodeBuddyAgentto handle AI operations requested by scripts. - MCP Client (
src/mcp/mcp-client.ts): Themcp.callbinding incodebuddy-bindings.tsdirectly invokes thecallToolmethod of theMCPManagerInterface, enabling scripts to interact with external tools. - Workspace Synchronization (
src/scripting/sync-bindings.ts,src/sync/index.ts): While not directly part of the core Lexer/Parser/Runtime, thesync-bindings.tsmodule (re-exported byindex.ts) provides mechanisms for scripts to interact with the workspace state, which in turn usessrc/sync/index.tsfor state management. - Error Handling (
src/utils/errors.ts):createLoopGuardis used extensively in the Lexer, Parser, and Runtime to prevent infinite loops and excessive recursion, ensuring script stability. - Configuration (
src/config/model-defaults.ts): Used bygetOrCreateAgentto determine the default AI model. - Logging (
src/utils/logger.js): Used bygetOrCreateAgentfor warnings and errors related to AI agent creation. - Type Definitions (
src/types/index.js):getErrorMessageis used incodebuddy-bindings.tsfor consistent error reporting. - Native System Access (
fs,path,child_process): Thebuiltins.tsandcodebuddy-bindings.tsmodules directly leverage Node.js's file system and child process APIs for file operations and shell command execution, respecting theworkdiranddryRunconfigurations. - Text Editor Tools (
src/tools/text-editor.ts,src/tools/multi-edit.ts): These tools might useresolvePathfrombuiltins.tsfor path resolution.
Developer Guide
Extending Built-ins or Bindings
To add new functionality available to scripts:
- For general utilities: Add functions to
src/scripting/builtins.tswithin an appropriate category or create a new one. Ensure they handleCodeBuddyValuetypes correctly and respectconfig.dryRunif they perform external actions. - For Code Buddy-specific integrations: Add functions to
src/scripting/codebuddy-bindings.tsunder an existing namespace (e.g.,tool,agent) or create a new one. These typically interact withCodeBuddyClientInterfaceorMCPManagerInterface.
Modifying the Language
- New Tokens: If you need to introduce new syntax (e.g., a new operator or keyword), you'll need to modify
src/scripting/lexer.tsto recognize the new token type andsrc/scripting/types.tsto define it. - New Syntax/Statements/Expressions: After defining new tokens, you'll need to modify
src/scripting/parser.tsto define how these tokens form valid AST nodes. This involves adding new parsing rules (e.g.,parseNewStatement()). - Runtime Semantics: Once the AST can represent your new syntax, you'll need to modify
src/scripting/runtime.tsto define how these new AST nodes are executed or evaluated. This typically involves adding a newcaseto theswitchstatement inexecuteNodeorevaluate.
Debugging Scripts
- Output: Use
print()statements within scripts to trace execution. - AST Inspection: Use
parseScript(source)(fromindex.ts) to get the tokens and AST, which can be helpful for understanding how the parser interprets your script.formatASTcan pretty-print the AST. - Runtime Errors: The
ScriptResultobject returned byexecuteScriptwill contain anerrorfield if an unhandled exception occurs during execution. - Configuration: Pay attention to
CodeBuddyScriptConfigoptions likeverbose,dryRun,enableFileOps,enableBash,enableAI, andtimeoutas they significantly affect script behavior and debugging.