Architecture

Understanding AIRouter's system design and components

System Overview

AIRouter is built as a high-performance API gateway that provides unified access to multiple LLM providers with intelligent routing and caching.

┌─────────────────────────────────────────┐
│           Client Application            │
└───────────────┬─────────────────────────┘
                │
                ▼
┌─────────────────────────────────────────┐
│           AIRouter Gateway              │
│  ┌────────────────────────────────────┐ │
│  │  Authentication & Rate Limiting    │ │
│  └────────────────────────────────────┘ │
│  ┌────────────────────────────────────┐ │
│  │  Request Caching                   │ │
│  └────────────────────────────────────┘ │
│  ┌────────────────────────────────────┐ │
│  │  Routing Engine                    │ │
│  │  • Cost-based                      │ │
│  │  • Latency-based                   │ │
│  │  • Priority                        │ │
│  │  • Fallback                        │ │
│  └────────────────────────────────────┘ │
│  ┌────────────────────────────────────┐ │
│  │  Analytics & Billing               │ │
│  └────────────────────────────────────┘ │
└───────────────┬─────────────────────────┘
                │
        ┌───────┼───────┐
        ▼       ▼       ▼
    ┌─────┐ ┌─────┐ ┌─────┐
    │OpenAI│ │Claude│ │Other│
    └─────┘ └─────┘ └─────┘

Core Components

1. API Gateway (Fastify)

Request validation and routing
Authentication (JWT + API Keys)
Rate limiting
CORS handling
Error handling

2. Routing Engine

Cost-Based: Route to cheapest provider
Latency-Based: Route to fastest provider
Priority: Preferred provider with fallback
Fallback: Automatic retry chain

3. Provider Connectors

Unified interface for different LLM providers:

OpenAI (GPT-3.5, GPT-4)
Anthropic (Claude 3)
Mistral AI
Extensible for new providers

4. Caching Layer (Redis)

Intelligent cache key generation
Instant response for cached queries
$0 cost for cache hits
Automatic cache invalidation

5. Database (PostgreSQL)

User and organization management
API key storage (hashed)
Request logging and analytics
Billing and credit tracking

Technology Stack

Component	Technology
Backend	Node.js + TypeScript + Fastify
Frontend	Next.js 14 + React + Tailwind CSS
Database	PostgreSQL + Drizzle ORM
Cache	Redis
Authentication	JWT + API Keys + Bcrypt
Monitoring	Prometheus + Pino Logger

Security Features

🔐 API Key Hashing

Bcrypt with 10 rounds

🚦 Rate Limiting

Per API key limits

🛡️ CORS Protection

Configurable origins

📝 Audit Logging

Track all activities