AIR-Agent

Prerequisites

Option 1 — Docker Compose

The image bakes the frontend build into the API container, so one service serves both.

```bash git clone https://github.com/ShiYu0318/ScholaRAGent.git cd ScholaRAGent

cp backend/.env.example backend/.env

Deployment

The Docker build is multi-stage: the frontend compiles in a Node stage and its dist output is copied into the Python image, which FastAPI serves as static files behind the API routes. One container, one port.

Scaling notes. The default IndexFlatIP is exact and fine well past tens of thousands of papers; switch INDEX_TYPE=hnsw when approximate search becomes worth the recall tradeoff. The scheduler holds in-process state (dedupe sets, fired-reminder ids), so running multiple API replicas with SCHEDULER_ENABLED=1 would duplicate digests — run the scheduler in exactly one replica. Postgres + pgvector is the path to horizontal scaling, since SQLite+FAISS assumes a single writer and a local index file.

Quick start

Option 2 — Local development

```bash

Configuration

Settings live in backend/.env, which is never committed. The minimum working setup:

GROQ_API_KEY=your-groq-api-key       # required

JWT_SECRET=$(openssl rand -hex 32)   # recommended: ephemeral if unset
SCHEDULER_ENABLED=1                  # per-user digests and reminders
STORE_BACKEND=sqlite                 # or postgres, with DATABASE_URL

Everything else is optional and safely skipped when unset.

Notes. The arXiv, news, Hacker News, Reddit, and GitHub crawlers need no credentials. Telegram, Email, LINE, OAuth, and X/Twitter activate only once their keys are present. Changing EMBED_MODEL changes vector dimensionality; the store detects the mismatch and rebuilds the index automatically. Leaving JWT_SECRET unset generates a random secret at startup, which invalidates every existing token on restart — acceptable locally, not in production.

Set GROQ_API_KEY at minimum

docker compose up --build # SQLite + FAISS, data in ./backend/data docker compose --profile postgres up --build # Postgres + pgvector instead ```

Web UI and API: http://localhost:8000 · Interactive API docs: http://localhost:8000/docs

For the Postgres profile, also set STORE_BACKEND=postgres and DATABASE_URL in backend/.env.

Frontend — separate terminal, hot reload, proxies /api to :8000

cd frontend npm install npm run dev # http://localhost:5173 ```

API reference

Every endpoint is browsable live at /docs (Swagger UI) and /redoc. Unless marked public, endpoints require Authorization: Bearer <JWT> from register/login or OAuth. Streaming endpoints emit Server-Sent Events as data: {json}\n\n frames.

Library and papers — `/api`

Feeds and subscriptions — `/api`

Graph — `/api/graph`

Research and writing — `/api`

Insights — `/api`

Notifications, reminders, learning — `/api`

System — `/api`

Query pipeline

Retrieval depth is chosen per question rather than fixed. Cheap questions skip retrieval entirely; multi-faceted ones get query expansion and reranking.

Why Reciprocal Rank Fusion. Dense and lexical retrievers produce scores on incomparable scales — cosine similarity against BM25 term weights. Normalizing them into a common range requires calibration that shifts with corpus and query. RRF sidesteps this entirely by discarding scores and fusing on rank alone: each result contributes 1 / (k + rank) to its document (rank counted from 1), summed across retrievers.

The constant k = 60 deliberately flattens the curve. A document ranked tenth by both retrievers scores 2/70 = 0.029, which beats a document ranked first by only one of them at 1/61 = 0.016. Cross-retriever agreement is worth more than a top position in a single ranking — exactly the behavior you want when fusing two methods that fail in different ways: vector search misses exact identifiers, BM25 misses paraphrase, and a document both agree on is unlikely to be an artifact of either failure mode. The system fetches 20 candidates per retriever before fusion, then truncates to the requested result count.

Two orthogonal routers. classify_complexity decides retrieval depth (none / simple / complex); route_query decides retrieval scope (local neighborhood versus global community reports). Both prefer an LLM classification and fall back to keyword heuristics, which keeps the pipeline working — and deterministically testable — without a model.

Agentic workflows

Research workflow

- Literature review generation with identified research gaps. - Multi-paper method comparison tables. - Guided deep-read explanations of dense papers. - Credibility signals from citation data and reproducibility signals from linked code. - Reading kanban (to-read / reading / done) with drag-and-drop, plus topic subscriptions. - Exports: BibTeX with generated keys, CSV, and Obsidian-ready Markdown with frontmatter and wikilinks (compatible with Juggl and Dataview). - Writing assistance: LaTeX drafts, slide outlines, polishing, contribution extraction, review suggestions, and submission checklists.

Retrieval and question answering

• Hybrid dense + sparse retrieval fused by RRF, with optional HNSW indexing at scale.
• Query transformation (HyDE, multi-query, decomposition) and cross-encoder reranking.
• Parent-document retrieval, contextual chunk embedding, and a semantic answer cache.
• Traceable chunk-level citations resolving to paper, section, and link.

Q&A and conversations — `/api`

Troubleshooting