# Current Multi-Agent System: Deep Analysis

## 1. Complete Agent Inventory

### Overview
The system has **9 agent classes** organized into three categories:

| Agent | Purpose | File Location |
|-------|---------|---------------|
| **PortalAgent** | Initial triage, routing to specialists | Line 609 |
| **ServiceAgent** | Equipment repairs, breakdowns, emergencies | Line 615 |
| **PartsAgent** | Part orders, ETAs, availability | Line 632 |
| **BillingAgent** | Invoices, payments, disputes | Line 649 |
| **ProjectsAgent** | Bids, quotes, installations | Line 666 |
| **MaintenanceAgent** | PM scheduling, contracts | Line 683 |
| **ControlsAgent** | BAS, building automation | Line 700 |
| **GeneralAgent** | Catch-all for general inquiries | Line 717 |
| **ClosingAgent** | Wraps up calls, recaps tickets | Line 734 |

### Agent Class Structures

#### PortalAgent (Lines 609-613)
```python
class PortalAgent(Agent):
    """Initial triage agent - routes callers to specialists"""
    def __init__(self) -> None:
        super().__init__(instructions=PORTAL_INSTRUCTION)
```
- **Purpose**: Initial greeting, first-touch triage
- **Tools**: None (routing happens externally via event handlers)
- **on_enter()**: Not defined (inherited default)
- **Prompt**: `PORTAL_INSTRUCTION` (from prompts/portal.py)

#### Specialist Agents Pattern (Service, Parts, Billing, Projects, Maintenance, Controls, General)
All follow this identical pattern:
```python
class ServiceAgent(Agent):
    def __init__(self, call_data: CallData, tools: list = None) -> None:
        super().__init__(
            instructions=get_specialist_instruction("service", call_data),
            tools=tools or []
        )
        self._call_data = call_data

    async def on_enter(self) -> None:
        caller_name = self._call_data.caller_name if self._call_data else None
        if caller_name:
            self.session.say("What's happening with the equipment?", allow_interruptions=True)
        else:
            self.session.say("Can I get your name?", allow_interruptions=True)
```

**Key Pattern Elements**:
- Accept `call_data` and `tools` in constructor
- Instructions built via `get_specialist_instruction()` with caller context injection
- Store `self._call_data` for access
- `on_enter()` checks if caller name is known to personalize opening

#### ClosingAgent (Lines 734-790)
```python
class ClosingAgent(Agent):
    def __init__(self, call_data: CallData, tools: list = None) -> None:
        super().__init__(
            instructions=CLOSING_INSTRUCTION,
            tools=tools or []
        )
        self._call_data = call_data

    async def on_enter(self) -> None:
        # Complex recap logic based on departments_visited
        # Builds dynamic message: "I have created tickets for..."
```
- **Purpose**: Recap all tickets, ask for feedback, say goodbye
- **on_enter()**: Dynamically builds recap from `call_data.departments_visited`

---

## 2. Agent Lifecycle Diagram

```
?????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
???                           HVAC GRACE CALL LIFECYCLE                             ???
?????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????

                    ??????????????????????????????????????????????????????????????????????????????????????????
                    ???     CALL CONNECTED         ???
                    ???   CallData() created       ???
                    ???   room_name set            ???
                    ??????????????????????????????????????????????????????????????????????????????????????????
                                  ???
                    ??????????????????????????????????????????????????????????????????????????????????????????
                    ???      SESSION STARTS        ???
                    ???  session.start() with      ???
                    ???  PortalAgent()             ???
                    ??????????????????????????????????????????????????????????????????????????????????????????
                                  ???
                    ??????????????????????????????????????????????????????????????????????????????????????????
                    ???      PORTAL GREETING       ???
                    ???  generate_reply():         ???
                    ???  "Thanks for calling..."   ???
                    ??????????????????????????????????????????????????????????????????????????????????????????
                                  ???
              ???????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
              ???          INTENT DETECTION             ???
              ???  on_user_speech_for_routing()         ???
              ???  detect_intent(user_text)             ???
              ???  call_data.switch_department(intent)  ???
              ???????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
                                  ???
              ???????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
              ???         SILENT HANDOFF                ???
              ???  perform_transfer() async task        ???
              ???  await asyncio.sleep(1.0)             ???
              ???  new_agent = AgentClass(call_data,    ???
              ???                         tools)         ???
              ???  session.update_agent(new_agent)      ???
              ???????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
                                  ???
              ???????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
              ???        SPECIALIST on_enter()          ???
              ???  Checks if caller_name known          ???
              ???  session.say("contextual opening")    ???
              ???????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
                                  ???
        ???????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
        ???                         ???                         ???
        ???                         ???                         ???
  ????????????????????????????????????            ????????????????????????????????????????????????          ????????????????????????????????????????????????
  ??? Collect  ???            ??? Inter-Agent  ???          ??? Caller Done  ???
  ???   Info   ??????????????????????????????????????????   Routing    ???          ???  Detected    ???
  ???  (normal ???            ??? route_to_*() ???          ???              ???
  ???   flow)  ???            ???   tools      ???          ???              ???
  ????????????????????????????????????            ????????????????????????????????????????????????          ????????????????????????????????????????????????
                                                          ???
                                ????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
                                ???         route_to_closing()           ???
                                ???   call_data.switch_department(       ???
                                ???          "closing")                  ???
                                ???   return ClosingAgent(call_data,     ???
                                ???          tools)                      ???
                                ????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
                                                          ???
                                ????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
                                ???       CLOSING on_enter()             ???
                                ???   Builds ticket recap from           ???
                                ???   departments_visited[]              ???
                                ???   session.say(recap_message)         ???
                                ????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
                                                          ???
                                ????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
                                ???         CALL END                     ???
                                ???   participant_disconnected event     ???
                                ???   process_call_end(call_data)        ???
                                ???   - Save audio recording             ???
                                ???   - Extract tickets via LLM          ???
                                ???   - Post to n8n webhook              ???
                                ????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
```

---

## 3. CallData Schema and Flow

### CallData Class Definition (Lines 55-105)

```python
@dataclass
class CallData:
    """Shared state that persists across all agent handoffs"""
    
    # Call identification
    room_name: str = ""
    call_start: datetime = field(default_factory=datetime.now)
    
    # Caller info (carried between specialists)
    caller_name: str = ""
    caller_phone: str = ""
    caller_company: str = ""
    caller_site: str = ""
    caller_role: str = ""
    
    # Transcript (captured in real-time)
    transcript_lines: List[str] = field(default_factory=list)
    
    # Department tracking
    current_department: str = "portal"
    departments_visited: List[str] = field(default_factory=list)
    
    # Audio recording
    audio_frames: List[bytes] = field(default_factory=list)
    sample_rate: int = 48000
```

### Key Methods

| Method | Purpose |
|--------|---------|
| `add_transcript_line(speaker, text)` | Appends timestamped line to transcript |
| `switch_department(new_department)` | Updates current_department, appends old to visited |
| `get_full_transcript()` | Returns "\n".join(transcript_lines) |
| `get_caller_context()` | Returns known caller info for prompt injection |

### How CallData Flows Through the System

```
????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
???                        CallData Flow Diagram                             ???
????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????

   SESSION START
        ???
        ???
  ?????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
  ???  call_data = CallData(room_name=...)    ???
  ???  _current_call_data = call_data         ???  ??? Global reference
  ?????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
                     ???
   ???????????????????????????????????????????????????????????????????????????????????????????????????????????????
   ???                 ???                 ???
   ???                 ???                 ???
????????????????????????????????????   ????????????????????????????????????    ??????????????????????????????????????????????????????
??? Event:   ???   ??? routing_ ???    ??? Agents get     ???
??? user_    ???   ??? tools =  ???    ??? call_data in   ???
??? input_   ???   ??? create_  ???    ??? constructor    ???
??? trans-   ???   ??? routing_ ???    ???                ???
??? cribed   ???   ??? tools()  ???    ??? ServiceAgent(  ???
???          ???   ???          ???    ???   call_data,   ???
??? ???????????????????????? ???   ??? ???????????????????????? ???    ???   tools)       ???
??? add_     ???   ??? Closures ???    ???                ???
??? trans-   ???   ??? capture  ???    ??? self._call_data???
??? cript_   ???   ??? call_data???    ??? = call_data    ???
??? line()   ???   ???          ???    ???                ???
??? extract_ ???   ???          ???    ???                ???
??? caller_  ???   ???          ???    ???                ???
??? info()   ???   ???          ???    ???                ???
????????????????????????????????????   ????????????????????????????????????    ??????????????????????????????????????????????????????
                     ???
                     ???
         ???????????????????????????????????????????????????????????????????????????
         ??? At call end:          ???
         ??? process_call_end(     ???
         ???   call_data)          ???
         ???                       ???
         ??? Uses:                 ???
         ??? - transcript_lines    ???
         ??? - departments_visited ???
         ??? - audio_frames        ???
         ??? - caller_*            ???
         ???????????????????????????????????????????????????????????????????????????
```

---

## 4. Routing Mechanics Documentation

### How Routing Tools Are Created (Lines 802-870)

```python
def create_routing_tools(call_data: CallData):
    """Create routing tools that have access to call_data.
    
    The tools list is built first, then each tool closure references it.
    This allows agents to pass tools to subsequent agents during handoffs.
    """
    tools_container = []  # Mutable container for circular reference
    
    @function_tool()
    async def route_to_closing():
        """Route to closing agent when caller is done."""
        call_data.switch_department("closing")
        return ClosingAgent(call_data, tools=tools_container), ""
    
    @function_tool()
    async def route_to_service():
        """Route to service team for equipment repairs..."""
        call_data.switch_department("service")
        return ServiceAgent(call_data, tools=tools_container), ""
    
    # ... similar for all other specialists
    
    tools = [route_to_closing, route_to_service, route_to_billing, ...]
    tools_container.extend(tools)  # Populate the container
    return tools
```

### Available Routing Tools

| Tool | Target Agent | Docstring (what LLM sees) |
|------|--------------|---------------------------|
| `route_to_closing` | ClosingAgent | "Route to closing agent when caller is done. Call this when the caller says no to 'anything else?'" |
| `route_to_service` | ServiceAgent | "Route to service team for equipment repairs, breakdowns, emergencies" |
| `route_to_billing` | BillingAgent | "Route to billing team for invoice questions, payments, disputes" |
| `route_to_parts` | PartsAgent | "Route to parts team for part orders, ETAs, availability" |
| `route_to_projects` | ProjectsAgent | "Route to projects team for quotes, bids, installations" |
| `route_to_maintenance` | MaintenanceAgent | "Route to maintenance team for PM scheduling, contracts, agreements" |
| `route_to_controls` | ControlsAgent | "Route to controls team for BAS, thermostats, building automation" |
| `route_to_general` | GeneralAgent | "Route to office/general team for COI requests, certificates of insurance, feedback, complaints, salespeople, vendor calls, or general inquiries" |

### Initial Routing (Portal ??? Specialist)

The first routing happens via **intent detection**, not via tools:

```python
# In hvac_agent() session handler (Lines 1055-1075)
@session.on("user_input_transcribed")
def on_user_speech_for_routing(event):
    nonlocal has_transferred
    if has_transferred:
        return  # Only route once from portal
    
    user_text = event.transcript
    if len(user_text.strip()) < 5:
        return
    
    has_transferred = True
    intent = detect_intent(user_text)  # Keyword-based detection
    call_data.switch_department(intent)
    asyncio.create_task(perform_transfer(session, intent, user_text, call_data))
```

### perform_transfer() Implementation (Lines 1077-1095)

```python
async def perform_transfer(session: AgentSession, intent: str, context: str, call_data: CallData):
    """Transfer to specialist agent with proper handoff"""
    agent_map = {
        "service": ServiceAgent,
        "parts": PartsAgent,
        "billing": BillingAgent,
        # ... etc
    }
    
    agent_class = agent_map.get(intent, GeneralAgent)
    await asyncio.sleep(1.0)  # Brief pause for any portal speech
    new_agent = agent_class(call_data, tools=routing_tools)
    session.update_agent(new_agent)  # ??? Triggers on_enter()
```

---

## 5. Prompt Injection System

### How Specialists Get Context (Lines 280-308)

```python
def get_specialist_instruction(intent: str, call_data: CallData) -> str:
    """Get the appropriate specialist instruction with caller context"""
    base_instructions = {
        "service": SERVICE_INSTRUCTION,
        "parts": PARTS_INSTRUCTION,
        # ... etc
    }
    
    base = base_instructions.get(intent, OFFICE_INSTRUCTION)
    
    # Add caller context if we have any info
    caller_context = call_data.get_caller_context()
    if caller_context != "No caller info collected yet.":
        context_section = f"""

# KNOWN CALLER INFORMATION
The following information has already been collected. DO NOT ask for this again.
{caller_context}

Use this information to personalize your greeting (e.g., "Hi [name], I see you're calling about [site]...")
Skip any intake questions for information you already have.
"""
        base = base + context_section
    
    return base + CLOSING_SEQUENCE  # Adds consistent closing behavior
```

### get_caller_context() (Lines 107-116)

```python
def get_caller_context(self) -> str:
    """Get known caller info for specialist prompts"""
    parts = []
    if self.caller_name:
        parts.append(f"Caller name: {self.caller_name}")
    if self.caller_phone:
        parts.append(f"Callback number: {self.caller_phone}")
    if self.caller_company:
        parts.append(f"Company: {self.caller_company}")
    if self.caller_site:
        parts.append(f"Site: {self.caller_site}")
    return "\n".join(parts) if parts else "No caller info collected yet."
```

### What Gets Injected

**Example**: If caller gave name "John" and company "ABC Corp" to Portal, 
the ServiceAgent receives this instruction suffix:

```
# KNOWN CALLER INFORMATION
The following information has already been collected. DO NOT ask for this again.
Caller name: John
Company: ABC Corp

Use this information to personalize your greeting (e.g., "Hi [name], I see you're calling about [site]...")
Skip any intake questions for information you already have.

# WHEN CALLER SAYS THEY ARE DONE
If they say "no" or "that's it"...
```

---

## 6. What Works Well (PRESERVE THESE)

### ??? 1. Clean Separation of Concerns
- Each specialist has a **single-file prompt** (prompts/service.py, etc.)
- Prompts are modular and easy to update independently
- Agent classes are minimal - just wiring, not logic

### ??? 2. Consistent Agent Pattern
All specialists follow the **same constructor and on_enter() pattern**:
```python
def __init__(self, call_data: CallData, tools: list = None)
async def on_enter(self) -> None:  # Context-aware greeting
```

### ??? 3. Caller Context Injection
- `get_caller_context()` builds dynamic context
- `get_specialist_instruction()` appends it to base prompts
- Prevents re-asking for already-provided info

### ??? 4. Real-Time Transcript Capture
- Events `user_input_transcribed` and `conversation_item_added` 
- Captures both sides of conversation as it happens
- Used for ticket extraction at end

### ??? 5. Department Tracking
- `current_department` and `departments_visited[]` 
- Enables ClosingAgent to build accurate recap
- Used for multi-ticket extraction

### ??? 6. Tools Container Pattern
```python
tools_container = []
# ... define tools with closures referencing tools_container
tools_container.extend(tools)
```
This clever pattern lets routing tools be passed to new agents.

### ??? 7. Silent Handoff UX
- No "transferring you now" announcements
- `asyncio.sleep(1.0)` prevents overlap with portal speech
- `on_enter()` provides smooth contextual greeting

---

## 7. What Needs Improvement (FIX THESE)

### ??? 1. No Specialist-to-Closing Context Transfer
**Problem**: When caller finishes with ServiceAgent and goes to ClosingAgent, the specific ticket details aren't passed.

**Current**:
```python
async def on_enter(self) -> None:
    # ClosingAgent only knows departments_visited, not what was discussed
    for dept in self._call_data.departments_visited:
        if dept == "service":
            recap_parts.append("your service request")  # Generic!
```

**Gap**: Can't say "your RTU repair at Main Street" - only "your service request"

### ??? 2. Information Extraction is Post-Call Only
**Problem**: `extract_caller_info_from_text()` runs on user speech but only populates basic fields (name, phone, company). Equipment details, site names, urgency are NOT extracted until call ends.

**Impact**: Specialists can't access structured ticket data during the call.

### ??? 3. Intent Detection is Keyword-Only
**Problem**: `detect_intent()` uses simple keyword matching:
```python
INTENT_KEYWORDS = {
    "service": ["broken", "down", "not working", ...],
    "billing": ["invoice", "bill", "payment", ...],
}
```

**Gap**: "I need to check on something" ??? goes to "general" even if they have invoice questions

### ??? 4. No Inter-Specialist Context Passing
**Problem**: When routing from ServiceAgent ??? PartsAgent (e.g., "actually, I also need a part status"), the PartsAgent doesn't know what service issue was just discussed.

**Current Flow**:
```python
@function_tool()
async def route_to_parts():
    call_data.switch_department("parts")
    return PartsAgent(call_data, tools=tools_container), ""
    # ??? No context about the service call
```

### ??? 5. CallData Has No Ticket Structure
**Problem**: `CallData` stores raw transcript lines but no structured ticket data:
```python
@dataclass
class CallData:
    transcript_lines: List[str]  # Raw lines
    # No: tickets: List[TicketData]
```

**Impact**: Can't track multiple discrete issues during the call.

### ??? 6. ClosingAgent Recap is Hardcoded
**Problem**: Recap messages are hardcoded per department:
```python
if dept == "service":
    recap_parts.append("your service request")
elif dept == "billing":
    recap_parts.append("your invoice question")
```

**Gap**: No way to say "your RTU repair ticket" or "your Invoice #12345 inquiry"

### ??? 7. No Specialist Handoff Notes
**Problem**: When specialists route to each other, there's no way to include a note:
```python
# Desired:
route_to_parts(note="Caller waiting on compressor for ticket #5678")
# Actual:
route_to_parts()  # Just switches, no context
```

### ??? 8. Portal Can't Pre-Fill Ticket Details
**Problem**: Portal captures caller's first statement but doesn't extract ticket-relevant info from it:
```
Caller: "Hi, the RTU on the roof is making a loud noise"
```
Portal detects intent="service", but doesn't extract:
- equipment_type: "RTU"  
- equipment_location: "roof"
- symptom: "loud noise"

---

## Summary Table

| Aspect | Current State | Gap |
|--------|---------------|-----|
| **Agent Pattern** | ??? Clean, consistent | - |
| **Prompt Modularity** | ??? One file per specialist | - |
| **Basic Context** | ??? Name/phone/company passed | ??? Ticket details not passed |
| **Department Tracking** | ??? departments_visited[] | ??? No ticket-level detail |
| **Transcript Capture** | ??? Real-time | ??? Not structured until end |
| **Routing Tools** | ??? All specialists reachable | ??? No context in handoff |
| **Intent Detection** | ?????? Works for obvious cases | ??? Keyword-only, no LLM |
| **Closing Recap** | ?????? Lists departments | ??? No ticket summaries |
| **Ticket Extraction** | ?????? Post-call via LLM | ??? Not available during call |

---

## Architecture Diagram

```
?????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
???                     CURRENT MULTI-AGENT ARCHITECTURE                        ???
?????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????

                         ????????????????????????????????????????????????????????????
                         ???   AgentSession   ???
                         ???   (LiveKit)      ???
                         ????????????????????????????????????????????????????????????
                                  ??? session.update_agent()
                                  ???
    ???????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
    ???                             ???                             ???
    ???                             ???                             ???
?????????????????????????????????     ?????????????????????????????????????????????????????????????????????????????????????????????     ?????????????????????????????????????????????????????????
??? Portal  ?????????????????????      SPECIALIST POOL        ?????????????????????   Closing       ???
??? Agent   ???     ???                             ???     ???   Agent         ???
???         ???     ??? ?????????????????????????????????  ?????????????????????????????????   ???     ???                 ???
??? No tools???     ??? ??? Service ???  ???  Parts  ???   ???     ??? recap based on  ???
???         ???     ??? ?????????????????????????????????  ?????????????????????????????????   ???     ??? departments_    ???
?????????????????????????????????     ??? ?????????????????????????????????  ?????????????????????????????????   ???     ??? visited[]       ???
     ???          ??? ??? Billing ???  ???Projects ???   ???     ?????????????????????????????????????????????????????????
     ???          ??? ?????????????????????????????????  ?????????????????????????????????   ???
     ???          ??? ?????????????????????????????????  ?????????????????????????????????   ???
     ???          ??? ??? Maint.  ???  ???Controls ???   ???
     ???          ??? ?????????????????????????????????  ?????????????????????????????????   ???
     ???          ??? ?????????????????????????????????               ???
     ???          ??? ??? General ???  ??????????????????????????????????????????????????? All have routing tools
     ???          ??? ?????????????????????????????????               ???
     ???          ?????????????????????????????????????????????????????????????????????????????????????????????
     ???                        ???
     ???                        ??? route_to_*() tools
     ???                        ??? return (NewAgent, "")
     ???                        ???
     ???                        ???
????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
???                 CallData                      ???
??? ???????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????? ???
??? ??? caller_name, caller_phone, caller_company ??? ???
??? ??? current_department, departments_visited[] ??? ???
??? ??? transcript_lines[], audio_frames[]        ??? ???
??? ???????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????? ???
???                                              ???
??? ?????? No ticket structure                       ???
??? ?????? No handoff notes                          ???
??? ?????? No specialist context transfer            ???
????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
```

---

*Document created: 2026-02-02*
*Author: Research Subagent*
*Purpose: Foundation documentation for multi-agent enhancement project*
