---
title: "Introduction to asa: AI Search Agent for Large-Scale Research Automation"
author: "Connor Jerzak"
date: "`r Sys.Date()`"
output:
  rmarkdown::html_vignette:
    highlight: null
vignette: >
  %\VignetteIndexEntry{Introduction to asa: AI Search Agent for Large-Scale Research Automation}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  eval = FALSE
)
```

## Overview

The **asa** package provides an LLM-powered research agent for performing AI search tasks at large scales. It implements a ReAct (Reasoning + Acting) agent pattern that combines:
- **Reasoning**: The LLM thinks through what information it needs
- **Acting**: The agent uses tools (web search, Wikipedia) to gather information
- **Iteration**: The process repeats until the task is complete

This architecture enables complex research tasks that require multiple searches and synthesis of information from various sources.

### Key Features

- **Multiple LLM Backends**: Support for OpenAI, Groq, xAI, Gemini, Anthropic, Bedrock, OpenRouter, and Exo
- **Web Search Tools**: DuckDuckGo search with 4-tier fallback system and Wikipedia lookup
- **Memory Folding**: DeepAgent-style context compression for long conversations
- **Structured Outputs**: Extract JSON or specific fields from agent responses
- **Batch Processing**: Run multiple queries with optional parallelization
- **Privacy Support**: Optional Tor proxy for anonymous searching

## Installation

### Install the R Package

```{r install-package}
# Install from GitHub
devtools::install_github("cjerzak/asa-software/asa")

# Or install locally from the repository
devtools::install("path/to/asa-software/asa")
```

### Build the Python Backend

The package uses Python for the LangGraph agent infrastructure. Run this once to set up the environment:

```{r build-backend}
library(asa)

# Create conda environment with all Python dependencies
build_backend(conda_env = "asa_env")
```
This installs LangChain, LangGraph, search tools, and proxy support packages.

### API Keys

Set environment variables for your chosen LLM backend:

```{r api-keys}
# For OpenAI (recommended for best results)
Sys.setenv(OPENAI_API_KEY = "your-openai-api-key")

# For Groq (fast inference, free tier available)
Sys.setenv(GROQ_API_KEY = "your-groq-api-key")

# For xAI
Sys.setenv(XAI_API_KEY = "your-xai-api-key")

# Also supported:
# Sys.setenv(GOOGLE_API_KEY = "your-gemini-api-key")      # or GEMINI_API_KEY
# Sys.setenv(ANTHROPIC_API_KEY = "your-anthropic-api-key")
# Sys.setenv(OPENROUTER_API_KEY = "your-openrouter-api-key")
# Sys.setenv(AWS_ACCESS_KEY_ID = "...")
# Sys.setenv(AWS_SECRET_ACCESS_KEY = "...")
# Exo uses a local server and does not require an API key.
```

## Quick Start

### Basic Query

```{r basic-query}
library(asa)

# Initialize the agent
agent <- initialize_agent(
  backend = "openai",
  model = "gpt-4.1-mini",
  conda_env = "asa_env"
)

# Run a simple query
result <- run_task(
  prompt = "What is the current population of Tokyo, Japan?",
  output_format = "text",
  agent = agent
)

# View the response
print(result$message)
```

### Structured JSON Output

For queries where you need specific data fields:

```{r json-output}
result <- run_task(
  prompt = "Find information about Marie Curie and return JSON with these exact fields:
            birth_year, death_year, nationality, field_of_study, major_achievement",
  output_format = "json",
  agent = agent
)

# Access parsed data
print(result$parsed)
# $birth_year
# [1] "1867"
#
# $death_year
# [1] "1934"
#
# $nationality
# [1] "Polish-French"
# ...
```

### Extract Specific Fields

When you know exactly which fields you need:

```{r extract-fields}
result <- run_task(
  prompt = "What is the capital of Australia and its population?",
  output_format = c("capital", "population"),
  agent = agent
)

# Access extracted fields
result$parsed$capital     # "Canberra"
result$parsed$population  # "453,000" (approximate)
```

## Working with Prompts

### Building Prompts from Templates

Use `build_prompt()` to create dynamic prompts from templates:

```{r build-prompt}
# Define a reusable template
template <- "Find the {{metric}} of {{entity}} in {{year}} and return JSON
             with fields: value, source, confidence"

# Generate specific prompts
prompt1 <- build_prompt(template,
                        metric = "GDP",
                        entity = "Germany",
                        year = 2023)

prompt2 <- build_prompt(template,
                        metric = "population",
                        entity = "Brazil",
                        year = 2024)

# Run the queries
result1 <- run_task(prompt1, output_format = "json", agent = agent)
result2 <- run_task(prompt2, output_format = "json", agent = agent)
```

## Batch Processing

### Running Multiple Queries

Process multiple research questions efficiently:
```{r batch-simple}
# Define your queries
prompts <- c(
  "What is the population of Tokyo?",
  "What is the population of New York City?",
  "What is the population of London?",
  "What is the population of Paris?"
)

# Run all queries (sequential)
results <- run_task_batch(prompts, agent = agent, progress = TRUE)

# Extract the messages
responses <- sapply(results, function(r) r$message)
```

### Batch with Data Frames

For research workflows, use data frames:

```{r batch-dataframe}
# Create research tasks
research_df <- data.frame(
  country = c("France", "Germany", "Japan", "Brazil"),
  prompt = c(
    "What is the current head of state of France?",
    "What is the current head of state of Germany?",
    "What is the current head of state of Japan?",
    "What is the current head of state of Brazil?"
  )
)

# Run batch and get results merged back
results_df <- run_task_batch(
  research_df,
  output_format = "text",
  agent = agent,
  progress = TRUE
)

# Results are added as new columns
print(results_df[, c("country", "response", "status")])
```

### Parallel Processing

For large batches, enable parallelization:

```{r batch-parallel}
# Requires future.apply package
# install.packages("future.apply")

results <- run_task_batch(
  prompts,
  agent = agent,
  parallel = TRUE,
  workers = 4
)
```

**Note**: Parallel processing works best with multiple API keys or when rate limits allow concurrent requests.

## Agent Configuration

### LLM Backend Options

asa supports `openai`, `groq`, `xai`, `gemini`, `anthropic`, `bedrock`, `openrouter`, and `exo`. The examples below show a few common setups:

```{r backend-options}
# OpenAI - Best for complex reasoning tasks
agent_openai <- initialize_agent(
  backend = "openai",
  model = "gpt-4.1-mini"  # or "gpt-4.1", "gpt-4o"
)

# Groq - Fast inference, good for simple queries
agent_groq <- initialize_agent(
  backend = "groq",
  model = "llama-3.3-70b-versatile"
)

# xAI - Alternative provider
agent_xai <- initialize_agent(
  backend = "xai",
  model = "grok-3"
)

# OpenRouter - provider/model format
agent_openrouter <- initialize_agent(
  backend = "openrouter",
  model = "anthropic/claude-3-sonnet"
)
```

### Memory Folding

For long research sessions, memory folding compresses older messages:

```{r memory-folding}
# Enable memory folding (default)
agent <- initialize_agent(
  backend = "openai",
  model = "gpt-4.1-mini",
  use_memory_folding = TRUE,
  memory_threshold = 4,      # Fold after 4 messages
  memory_keep_recent = 2     # Keep 2 most recent exchanges intact
)

# Disable for short, independent queries
agent_no_fold <- initialize_agent(
  backend = "openai",
  model = "gpt-4.1-mini",
  use_memory_folding = FALSE
)
```

Memory folding is beneficial when:
- Running many sequential queries in one session
- Building on previous results
- Avoiding context length limits

### Rate Limiting

Control request frequency to avoid API limits:

```{r rate-limit}
agent <- initialize_agent(
  backend = "openai",
  model = "gpt-4.1-mini",
  rate_limit = 0.2,  # 0.2 requests per second (1 every 5 seconds)
  timeout = 120      # 2 minute timeout per request
)
```

## Privacy with Tor

### Setting Up Tor

For anonymous searching, configure a Tor proxy:

```bash
# macOS
brew install tor
brew services start tor

# Linux (Ubuntu/Debian)
sudo apt install tor
sudo systemctl start tor
```

### Using Tor with asa

```{r tor-setup}
# Check if Tor is running
if (is_tor_running()) {
  message("Tor is running!")

  # Initialize agent with Tor proxy
  agent <- initialize_agent(
    backend = "openai",
    model = "gpt-4.1-mini",
    proxy = "socks5h://127.0.0.1:9050"
  )

  # Verify your Tor IP
  ip <- get_tor_ip()
  message("Current Tor exit IP: ", ip)
} else {
  message("Tor is not running. Start with: brew services start tor")
}
```

### Rotating Tor Circuits

Get a new IP address for different searches:

```{r tor-rotate}
# Rotate to a new Tor circuit (new exit node)
rotate_tor_circuit(method = "brew")  # macOS
# rotate_tor_circuit(method = "systemctl")  # Linux

# Verify the new IP
new_ip <- get_tor_ip()
message("New Tor exit IP: ", new_ip)
```

## Extracting Data from Agent Traces

### Understanding Agent Output

Each agent response includes a text trace in `raw_output` and, when available,
a structured JSON trace in `trace_json`:

```{r trace-data}
# Run a query
result <- run_task(
  prompt = "Who won the 2024 Nobel Prize in Physics?",
  output_format = "text",
  agent = agent
)

# Prefer the structured trace when available
trace <- if (!is.null(result$trace_json) && nzchar(result$trace_json)) {
  result$trace_json
} else {
  result$raw_output
}
cat(substr(trace, 1, 500))  # First 500 characters
```

### Extracting Search Results

```{r extract-search}
result <- run_task(
  prompt = "What are the main causes of climate change?",
  agent = agent
)

# Extract structured data from the preferred trace representation
trace <- if (!is.null(result$trace_json) && nzchar(result$trace_json)) {
  result$trace_json
} else {
  result$raw_output
}
extracted <- extract_agent_results(trace)

# Search snippets from DuckDuckGo
print(extracted$search_snippets)

# URLs of sources used
print(extracted$search_urls)

# Wikipedia content retrieved
print(extracted$wikipedia_snippets)
```

### Batch Output Processing

Process multiple raw outputs efficiently:

```{r process-outputs}
# After running a batch on a data frame
results_df <- run_task_batch(
  data.frame(prompt = prompts),
  agent = agent
)

# raw_output is included automatically and can be processed directly
processed_df <- process_outputs(results_df)

# Now contains search_count, wiki_count, and any JSON fields
print(processed_df[, c("prompt", "search_count", "wiki_count")])
```

## Best Practices

### 1. Clear, Specific Prompts

```{r clear-prompts}
# Good: Specific and structured
good_prompt <- "Find the GDP of South Korea in 2023. Return JSON with fields:
                gdp_value (in USD), gdp_growth_rate (percentage), data_source"

# Bad: Vague
bad_prompt <- "Tell me about South Korea's economy"
```

### 2. Handle Errors Gracefully

```{r error-handling}
result <- run_task(prompt, agent = agent)

if (result$status == "success") {
  # Process result
  print(result$message)
} else {
  # Handle error
  warning("Query failed: ", result$raw_output)
}
```

### 3. Check Backend Status

```{r check-backend}
# Verify environment before running queries
status <- check_backend(conda_env = "asa_env")

if (!status$available) {
  message("Missing packages: ", paste(status$missing_packages, collapse = ", "))
  build_backend()  # Rebuild if needed
}
```

### 4. Manage Agent State

```{r agent-state}
# Get current agent (if initialized)
current_agent <- get_agent()

# Reset agent state between different tasks
reset_agent()

# Re-initialize with new settings
agent <- initialize_agent(backend = "groq", model = "llama-3.3-70b-versatile")
```

## Complete Research Workflow Example

Here's a complete example of a research workflow:

```{r complete-example}
library(asa)
library(dplyr)

# 1. Setup
build_backend(conda_env = "asa_env")

agent <- initialize_agent(
  backend = "openai",
  model = "gpt-4.1-mini",
  conda_env = "asa_env",
  use_memory_folding = TRUE
)

# 2. Define research questions
countries <- c("Norway", "Sweden", "Denmark", "Finland", "Iceland")

research_df <- data.frame(
  country = countries,
  prompt = sapply(countries, function(c) {
    build_prompt(
      "Research {{country}} and return JSON with:
       population (number), capital (city name),
       gdp_per_capita (USD), happiness_rank (global ranking)",
      country = c
    )
  })
)

# 3. Run batch research
results_df <- run_task_batch(
  research_df,
  output_format = "json",
  agent = agent,
  progress = TRUE
)

# 4. Analyze results
summary_df <- results_df %>%
  select(country, population, capital, gdp_per_capita, happiness_rank, status) %>%
  filter(status == "success")

print(summary_df)

# 5. Save results
write.csv(summary_df, "nordic_countries_research.csv", row.names = FALSE)
```

## Troubleshooting

### Common Issues

**"Agent not initialized"**
```{r troubleshoot-1}
# Always initialize before running tasks
agent <- initialize_agent(backend = "openai", model = "gpt-4.1-mini")
```

**"Environment not found"**
```{r troubleshoot-2}
# Rebuild the Python environment
build_backend(conda_env = "asa_env")
```

**Rate Limiting Errors**
```{r troubleshoot-3}
# Reduce request rate
agent <- initialize_agent(
  backend = "openai",
  model = "gpt-4.1-mini",
  rate_limit = 0.1  # 1 request per 10 seconds
)
```

**Empty or NA Responses**
```{r troubleshoot-4}
# Check the trace for error details
result <- run_task(prompt, agent = agent)
if (is.na(result$message)) {
  cat("Error trace:\n", result$raw_output)
}
```

## Further Resources

- **GitHub Repository**: https://github.com/cjerzak/asa-software
- **Bug Reports**: https://github.com/cjerzak/asa-software/issues
- **LangGraph Documentation**: https://langchain-ai.github.io/langgraph/
- **ReAct Paper**: Yao et al., "ReAct: Synergizing Reasoning and Acting in Language Models"
- **DeepAgent Paper**: Memory folding architecture for long-context agents
