SarmaLink-AI

Abstract

01Introduction & Motivation

Every major AI provider offers a free tier. Groq hosts GPT-OSS 120B and delivers first tokens in 41 milliseconds. SambaNova runs DeepSeek V3.2, a 685-billion-parameter Mixture-of-Experts model that beats GPT-4o on MATH-500 and HumanEval. Cerebras serves inference at 2,000 tokens per second on wafer-scale chips. Google Gemini has grounded Google Search built in at the token level. OpenRouter aggregates 100+ models behind a single OpenAI-compatible API, including a :free pool of 17+ community-hosted models. Cloudflare Workers AI runs FLUX.2 klein for image generation at the edge. Tavily provides structured search designed for LLM consumption.

Each of these, individually, is generous. Each, on its own, still hits rate limits. The moment a single provider returns a 429, an AI application breaks. Users see an error. Trust evaporates. The common workaround — paying for an upgrade — defeats the point of using free tiers in the first place, and lock users into a single vendor’s roadmap.

The problem with existing solutions

• LiteLLM is a library, not an application. You still have to write the app, the routing logic, the retry policy, the stream parser, the database schema.
• LangChain is a framework with a steep learning curve and heavy abstractions. Multi-provider failover is possible but manual.
• OpenRouter is one aggregator service — not seven redundant providers. When OpenRouter has an issue, you have an issue.
• ChatGPT Plus / Claude Pro / Gemini Advanced are rentals. No self-hosting, no vendor independence, no data ownership.

First-principles multi-provider failover

SarmaLink-AI treats every AI provider as a commodity. If Groq returns 429, SambaNova fires. If SambaNova is busy, Cerebras. Then Gemini. Then OpenRouter’s :free pool as the final safety net. Round-robin key rotation distributes load across keys within a provider. Round-robin across providers survives outages. Every step is logged to ai_events for observability.

Target audience

Small-to-medium businesses, indie developers, digital agencies, research teams — anyone who needs production-grade AI capability without a vendor rental fee, and who values code they can read, fork, and extend.

02System Architecture

SarmaLink-AI is a Next.js 14 application using the App Router, deployed on Vercel (or any Next.js-compatible host). The runtime is a thin HTTP layer over a modular lib/ directory. Every external dependency is behind an interface. Every piece of untrusted data is wrapped at a trust boundary before reaching the model.

Core modules

• app/api/ai-chat/route.ts — the HTTP entry point. ~45 lines after refactoring. Delegates all logic to lib.
• lib/providers/failover.ts — the failover runner (tryFailover). Orchestrates retries across steps and keys.
• lib/providers/registry.ts — declares every provider, endpoint, and key collection.
• lib/tools/registry.ts — plugin pattern for live tools (weather, FX, container tracking).
• lib/prompts/sanitize.ts — prompt injection defence at trust boundaries.
• lib/repositories/ — typed Supabase CRUD for sessions, usage, events, memories.
• lib/intent.ts — auto-router classifier (regex patterns, zero API calls).

Request lifecycle

Browser
   │ POST /api/ai-chat { message, model? }
   ▼
Route handler (~45 lines)
   │ 1. Supabase auth (cookie → user.id)
   │ 2. RLS enforces ownership
   ▼
Sanitizer
   │ wrapUntrusted(user_message)
   │ wrapMemories(user_memories)
   │ wrapToolResult(tool_outputs)
   ▼
Auto-router (lib/intent.ts)
   │ image intent? → image pipeline
   │ search intent? → Live mode
   │ default → Smart mode
   ▼
tryFailover (lib/providers/failover.ts)
   │ for each step in mode.failover:
   │   for each key in providerKeys(step):
   │     try stream → yield tokens
   │     catch 429/5xx → next key or step
   ▼
SSE stream → Browser

Deployment topology

Vercel (or any Node.js host) runs the route handlers. Supabase provides PostgreSQL + Auth + Row-Level Security. Cloudflare R2 stores binary attachments (images, PDFs) with 7-day signed URLs. Cloudflare Workers AI serves image generation. All other providers are accessed via their public OpenAI-compatible endpoints.

03The Failover Runner

tryFailover is the load-bearing module. It accepts a sequence of provider/model steps and iterates through them until one returns a successful stream.

Algorithm

async function tryFailover(steps, messages, opts) {
  for (const step of steps) {
    const keys = providerKeys(step.provider)
    const offset = Date.now() % keys.length  // round-robin
    for (let i = 0; i < keys.length; i++) {
      const key = keys[(offset + i) % keys.length]
      try {
        const stream = await callProvider(step, key, messages)
        return stream  // success
      } catch (err) {
        logEvent({ backend: step.label, status: err.status })
        if (err.status === 429 || err.status >= 500) continue
        throw err  // non-recoverable
      }
    }
  }
  throw new Error('All providers exhausted')
}

Uptime math

Handoff timing

Typical failover from 429 on one provider to first token on the next is under 50 milliseconds. The request parser reads the response headers, classifies the failure, rotates to the next key, and dispatches — all without user-visible interruption. Every step writes to ai_events with status, backend, latency_ms, and tokens_out.

04The Six Modes

Each mode is a different failover sequence, optimised for a specific task type.

05The Seven Providers

06Security Model

Trust boundaries

Three sources of untrusted text reach the model on every request:

• User messages — from the browser, potentially adversarial
• Tool results — from external APIs (Tavily, Open-Meteo, frankfurter.app) which may return manipulated content
• Saved memories — from the database, written by the memory extractor which may have laundered injection strings from past conversations

Each is wrapped by a dedicated sanitiser: wrapUntrusted, wrapToolResult, wrapMemories. Output is wrapped in explicit XML-style markers before reaching the model, and known jailbreak patterns ("ignore previous instructions", "system:" prefixes, role-switch attempts) are stripped.

Unit test coverage

11 unit tests in __tests__/sanitize.test.ts cover documented jailbreak categories. Defence is layered: even if strip misses a pattern, the wrapping ensures the model can never interpret untrusted text as a command.

Row-Level Security

Every table enforces per-user isolation at the PostgreSQL layer. Even if route logic has a bug, cross-user reads return zero rows.

CREATE POLICY "own_rows" ON ai_chat_sessions
  FOR ALL USING (auth.uid() = user_id);

The same policy is applied to ai_chat_usage, ai_events, and ai_user_memories. The service-role key bypasses RLS but is server-only — never in the client bundle, never in env vars exposed to browsers.

07Observability & Operations

The /api/admin/health endpoint returns per-provider success rates, p50/p95 latency, dead-model detection, and 24-hour request volume — all computed from the ai_events audit log.

Event schema

ai_events (
  id          uuid,
  user_id     uuid,
  event_type  text,      -- 'message' | 'tool' | 'error'
  backend     text,      -- 'Groq GPT-OSS 120B', etc.
  status      text,      -- 'success' | 'rate_limited' | 'error'
  latency_ms  integer,
  tokens_out  integer,
  created_at  timestamptz
);

Diagnostic queries

Per-backend p95 latency over 24 hours:

SELECT backend,
       percentile_cont(0.95) WITHIN GROUP (ORDER BY latency_ms) AS p95,
       COUNT(*) AS volume
FROM ai_events
WHERE event_type = 'message'
  AND created_at > now() - interval '24 hours'
GROUP BY backend
ORDER BY p95;

Dead model detection (always returns 404/429):

SELECT backend,
       COUNT(*) FILTER (WHERE status = 'success') AS ok,
       COUNT(*) FILTER (WHERE status != 'success') AS fail,
       ROUND(100.0 * COUNT(*) FILTER (WHERE status = 'success')
             / COUNT(*), 1) AS success_rate
FROM ai_events
WHERE created_at > now() - interval '1 hour'
GROUP BY backend
HAVING COUNT(*) > 10
ORDER BY success_rate;

Scaling up

The Gmail +alias trick multiplies capacity. Sign up at you+provider2@gmail.com, +provider3@gmail.com, etc. — each counts as a distinct account at most providers. Adding 8 keys per provider yields 8× daily capacity with zero code changes; the failover runner already rotates through all keys.

08Benchmarks & Performance

DeepSeek V3.2 — SarmaLink-AI’s primary engine — compared to commercial AI products. Published scores from DeepSeek technical reports, lmarena.ai, SWE-bench leaderboards, Arena-Hard, and the GPQA paper.

Capacity math

09Deployment Guide

Prerequisites

• Node.js 20 or later
• Git
• Supabase account (free tier)
• API keys from Groq (required), plus optional SambaNova, Cerebras, Gemini, OpenRouter, Cloudflare, Tavily
• GitHub account (for deployment via Vercel)
• Vercel account (free tier)

Fast path

git clone https://github.com/sarmakska/sarmalink-ai.git
cd sarmalink-ai
npm install
cp .env.example .env.local
# Fill in .env.local with your Supabase + provider keys
# Run supabase/migrations/001_sarmalink_ai.sql in Supabase SQL editor
npm run dev

Then push your repo to GitHub, import into Vercel, paste env vars into Vercel’s dashboard, and deploy. Full 45-minute walkthrough in the Complete Setup Guide.

Vercel Pro recommendation

Vercel Hobby (free) has a 10-second function timeout — adequate for most requests but can cut off long failover chains. Vercel Pro ($20/month) raises it to 60 seconds (300s with streaming) and is recommended for production.

10Extension Points

Adding a new provider

Any provider with an OpenAI-compatible chat completions endpoint can be added in ~10 lines across four files: lib/ai-models.ts (register type), lib/providers/registry.ts (endpoint + keys), lib/env/validate.ts (env collection), and the mode’s failover array.

Adding a new live tool

Implement the Tool<Args> interface (match, args, run) in lib/tools/, then add one line to the TOOLS array in lib/tools/registry.ts. The auto-router picks it up automatically.

Customising the system prompt

System prompts are per-mode in lib/ai-models.ts. Each mode has its own persona, tone, and constraint set. Version control history preserves prompt evolution.

11Comparison with Alternatives

12Cost Analysis

A typical AI-heavy individual pays for 3-5 separate subscriptions to get the capabilities SarmaLink-AI ships with by default.

For a 15-person team each paying ChatGPT Plus + Claude Pro alone: £5,700/year. SarmaLink-AI serves 15,000 daily requests across the same team at £0 recurring (optional £20/month for Vercel Pro if function timeouts become a constraint).

13Roadmap

Now (shipped)

• 7 providers, 36 engines, 6 specialised modes
• Automatic mode detection from message content
• Persistent cross-session memory (30-fact cap per user)
• 5 live tools: weather, exchange rates, container tracking, news search, URL summary

Next

• Per-mode prompt versioning with A/B testing
• Example chat UI in examples/ folder
• Streaming chunk replay for debugging
• Usage analytics dashboard

Soon

• Voice mode (Whisper + TTS via Groq)
• Video frame analysis via Gemini Vision
• Tool marketplace — community plugins
• One-click Vercel deploy template

Later

• Federated failover — share capacity across instances
• Model fine-tuning pipeline
• Mobile app with offline fallback to on-device LLM

14Governance & Licensing

SarmaLink-AI is released under the MIT License. Contributors retain copyright of their contributions. Pull requests are reviewed against CI (lint, typecheck, test, build) and CodeQL security scans; all must pass before merge.

Security vulnerabilities should be reported privately via the process documented in SECURITY.md. Community channels: GitHub Issues and Discussions.

15Conclusion

SarmaLink-AI demonstrates that production-grade AI capability doesn’t require per-user subscriptions, vendor lock-in, or proprietary infrastructure. A first-principles multi-provider failover architecture, built on open-source primitives and free tiers, delivers 99.9999% effective uptime with frontier model quality — at zero recurring cost. The codebase is small enough to read in an afternoon, documented in a 22-page wiki, and licensed under MIT for any use. Fork it, self-host it, extend it, ship it.

AGlossary

• 429 — HTTP status code for "Too Many Requests". Indicates rate limiting.
• Failover — a sequence of steps tried in order until one succeeds.
• LPU — Language Processing Unit. Groq’s custom silicon for LLM inference.
• MoE — Mixture of Experts. Large model where only a subset of parameters activate per token.
• RLS — Row-Level Security. PostgreSQL feature enforcing per-row access policies.
• SSE — Server-Sent Events. HTTP-native streaming protocol for one-directional server→client data.
• WSE — Wafer-Scale Engine. Cerebras’ single-chip architecture using entire silicon wafers.
• R2 — Cloudflare’s S3-compatible object storage with no egress fees.
• OKLCH — Perceptually uniform colour space used in the site’s design system.

BAPI Reference

Chat (streaming SSE)

curl -N https://your-deploy.vercel.app/api/ai-chat \
  -H "Content-Type: application/json" \
  -H "Cookie: <supabase-auth>" \
  -d '{"message":"Draft a follow-up email","model":"smart"}'
# Returns: SSE stream with {"type":"token","value":"..."} events

Image generation

curl -X POST https://your-deploy.vercel.app/api/images/generate \
  -H "Content-Type: application/json" \
  -d '{"prompt":"sunset over Himalayas"}'
# Returns: {url: "https://r2.../signed?..."}  (7-day URL)

Image editing

curl -X POST https://your-deploy.vercel.app/api/images/edit \
  -F "image=@original.jpg" \
  -F 'instruction=change sky to emerald green'

File upload

curl -X POST https://your-deploy.vercel.app/api/attachments/upload \
  -F "file=@contract.pdf"
# Extracted text stored and referenceable in next message

Health check

curl https://your-deploy.vercel.app/api/admin/health
# Returns: per-provider success rates, p95 latency, 24h volume

CEnvironment Variables

DDatabase Schema

CREATE TABLE ai_chat_sessions (
  id          uuid PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id     uuid NOT NULL REFERENCES auth.users(id) ON DELETE CASCADE,
  title       text,
  messages    jsonb NOT NULL DEFAULT '[]',
  updated_at  timestamptz DEFAULT now()
);

CREATE TABLE ai_chat_usage (
  user_id     uuid NOT NULL REFERENCES auth.users(id) ON DELETE CASCADE,
  day         date NOT NULL,
  count       integer NOT NULL DEFAULT 0,
  PRIMARY KEY (user_id, day)
);

CREATE TABLE ai_events (
  id          uuid PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id     uuid NOT NULL REFERENCES auth.users(id) ON DELETE CASCADE,
  event_type  text,
  backend     text,
  status      text,
  latency_ms  integer,
  tokens_out  integer,
  created_at  timestamptz DEFAULT now()
);

CREATE TABLE ai_user_memories (
  id          uuid PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id     uuid NOT NULL REFERENCES auth.users(id) ON DELETE CASCADE,
  fact        text NOT NULL,
  created_at  timestamptz DEFAULT now()
);

-- Row-Level Security on every table
ALTER TABLE ai_chat_sessions ENABLE ROW LEVEL SECURITY;
ALTER TABLE ai_chat_usage     ENABLE ROW LEVEL SECURITY;
ALTER TABLE ai_events         ENABLE ROW LEVEL SECURITY;
ALTER TABLE ai_user_memories  ENABLE ROW LEVEL SECURITY;

CREATE POLICY "own_rows" ON ai_chat_sessions
  FOR ALL USING (auth.uid() = user_id);
CREATE POLICY "own_rows" ON ai_chat_usage
  FOR ALL USING (auth.uid() = user_id);
CREATE POLICY "own_rows" ON ai_events
  FOR ALL USING (auth.uid() = user_id);
CREATE POLICY "own_rows" ON ai_user_memories
  FOR ALL USING (auth.uid() = user_id);

EReferences

• SarmaLink-AI repository
• SarmaLink-AI Wiki (22 pages)
• Groq Console · LPU inference
• SambaNova Cloud · DeepSeek V3.2
• Cerebras Cloud · WSE-3
• Google AI Studio · Gemini grounding
• OpenRouter · aggregator
• Cloudflare · Workers AI + R2
• Tavily · structured search
• Supabase · Postgres + Auth + RLS
• LMArena · benchmark leaderboards
• Sarma Linux · publisher