JAiRouter

JAiRouter

<p align="center"> <img src="logo/JAiRouterLogo.png" alt="JAiRouter — AI Model Gateway" width="180"> </p>

<p align="center"> <strong>One OpenAI-compatible endpoint in front of every model backend you run.</strong> </p>

<p align="center"> Ollama · vLLM · GPUStack · Xinference · LocalAI · OpenAI · Claude · Gemini<br> Routing · load balancing · rate limiting · circuit breaking · failover · RBAC · audit · visual console </p>

<p align="center"> <a href="https://github.com/Lincoln-cn/JAiRouter/releases"> <img src="https://img.shields.io/github/v/release/Lincoln-cn/JAiRouter?style=flat-square" alt="Latest release"> </a> <a href="https://hub.docker.com/r/sodlinken/jairouter"> <img src="https://img.shields.io/docker/pulls/sodlinken/jairouter?style=flat-square&logo=docker" alt="Docker Pulls"> </a> <a href="https://github.com/Lincoln-cn/JAiRouter/blob/master/LICENSE"> <img src="https://img.shields.io/github/license/Lincoln-cn/JAiRouter?style=flat-square" alt="License"> </a> </p>

<p align="center"> <a href="README-ZH.md">中文</a> • <a href="https://jairouter.com">Docs</a> • <a href="https://github.com/Lincoln-cn/JAiRouter/issues">Issues</a> • <a href="https://github.com/Lincoln-cn/JAiRouter/discussions">Discussions</a> </p>

This README is written to be read by people and AI agents alike: the facts block below and the Repository map / Commands / HTTP surfaces sections are the machine-actionable core. Where the code and this file disagree, the code wins — please open an issue so we can fix the file.

product:         JAiRouter
repository:      Lincoln-cn/JAiRouter
checkout_dir:    model-router            # the local/git directory name differs from the product name
artifact:        model-router            # Maven artifactId
project_version: 3.5.3                   # pom.xml
latest_release:  v3.5.3 (2026-10-25)
stack:
  backend:  Java 17 + Spring Boot 3.5.5 (WebFlux / reactive)
  frontend: Vue 3 + TypeScript + Vite, bilingual zh-CN / en-US
  build:    Maven wrapper — ./mvnw (Unix) · .\mvnw.cmd (Windows)
runtime:
  default_port: 8080
  console:      /admin/
  swagger_ui:   dev profile only
  required_env_in_prod: [JWT_SECRET, INITIAL_ADMIN_PASSWORD]
surfaces:
  - /v1/**        # client surface: OpenAI-native + Anthropic-compatible
  - /api/**       # management + legacy surface (RBAC-governed)
  - /actuator/**  # health, readiness, Prometheus
tests:
  command: ./mvnw test -o
  scale:   ~4,100 test methods across ~400 test classes
license: Apache-2.0

Feature inventory

Implemented and exercised by the test suite:

- Adapters — Ollama, vLLM, GPUStack, Xinference, LocalAI, OpenAI, Anthropic (native), Gemini (native), plus any OpenAI-compatible provider declared in adapter-definitions. - Load balancing — round-robin, weighted, least-connections, IP hash, consistent hash, EWMA latency-aware, random. - Rule engine — conditional routing on model, service type, header, client IP, weight and instance tags, with hit statistics; visual editing in the console. - Resilience — circuit breaker per instance, request-level failover to another healthy instance, rate limiting (token bucket / leaky bucket / sliding window), response cache for deterministic requests (opt-in, tenant-isolated keys). - Quota — multi-dimensional, multi-window ledger (MINUTE/HOUR/DAY/MONTH) with Redis distributed counting and local fallback (jairouter.quota.enabled, off by default). - Record governance — call history at three levels: metadata only, desensitized summary, or AES-256-GCM-encrypted full content; ADMIN-only decryption endpoint. - Observability — Prometheus metrics, OpenTelemetry tracing, slow-query analysis, error tracking, security alerts. Alert outlets are deliberately limited to Prometheus metrics + structured logs — the repository implements no push channels (email/webhook/IM); wire those up in Alertmanager or your log platform. - Persistence — H2 by default; PostgreSQL (with Flyway migrations) when DATABASE_URL points at it; Redis for shared state in multi-replica deployments. - Multi-replica — cross-replica event broadcast, shared rate-limit/quota counters, and a distributed lock for scheduled tasks (including the alert cooldown). - Console — Vue 3 SPA at /admin/: services, instances, adapters, rules, versions & rollback, quota, call history, tracing, alerts, API keys, RBAC, dark/light theme, zh-CN / en-US.

Deliberately not implemented: semantic (vector-similarity) cache and any in-repo notification delivery. See Status and recent work for the open ledger.