Docs — AIntOps

// Providers

Connect a Provider

AIntOps uses your LLM provider's Admin API key to pull usage and cost data. Keys are encrypted at rest with Fernet AES-128 and decrypted only when making API calls.

Navigation

Settings → Providers → Connect → Select provider → Paste key → Save

Once connected, AIntOps syncs usage data every hour via the background scheduler.

OpenAI

You need an Admin API key — not a standard API key. Find it in your OpenAI organization settings under API keys → Admin keys.

ℹ️

Admin keys have read access to usage data. AIntOps never makes inference calls with your key — it only queries the usage API.

Anthropic

Generate an Admin API key from your Anthropic Console under Settings → API Keys. The key must have the usage:read permission scope.

Mock Provider (development)

AIntOps ships with a mock OpenAI server for development and testing — no real API key needed.

Connection settings for the mock provider

{
          "provider": "openai",
          "credentials": "sk-admin-MOCK-VALID-KEY-1234567890",
          "base_url": "http://mock-openai:8001/v1"
        }

The mock server generates 30 days of realistic synthetic data, letting you test dashboards, alerts, and recommendations without an OpenAI account.

// Dashboards

Dashboards & Widgets

Dashboards are fully customizable with a drag-and-drop layout. Each organization gets a default dashboard on signup.

Available widgets

Widget	Description
`spend_over_time`	Daily/weekly cost chart per provider or model
`token_consumption`	Token volume over time with input/output breakdown
`cost_per_model`	Ranked list of models by spend
`request_volume`	API request count over time
`latency`	Average response latency per model
`budget_gauge`	Current spend vs budget threshold
`top_recommendations`	Latest AI cost-saving suggestions
`anomaly_feed`	Recent anomaly detections

Key metrics explained

Metric	Definition
MTD Spend	Total cost since the 1st of the current month
Cost / 1K tokens	Blended average cost across all models and providers
Token efficiency	Output tokens / total tokens — higher is better
Request latency	P50 and P95 response time per model

⚡

Dashboard queries scan up to 30 days of raw event data in under 50ms thanks to TimescaleDB continuous aggregates refreshed automatically every hour.

// Alerts & Budgets

Budget Setup

Budgets are set per organization and evaluated every 30 minutes by the background scheduler.

Creating a budget

Navigation

Settings → Budgets → New Budget

Field	Description
Period	Monthly or rolling 30-day window
Warning threshold	Sends a notification when spend reaches this amount
Critical threshold	Sends a critical alert — can trigger hard stop if configured
Scope	All providers, or a specific provider

Anomaly Detection

AIntOps uses a z-score baseline to detect unexpected cost spikes. The anomaly detector runs every 15 minutes and compares current spend rate against your historical baseline.

When an anomaly is detected, you receive:

A notification via your configured channels (Slack / email)
An entry in the anomaly_feed dashboard widget
An automatic recommendation if a model substitution can reduce the spike

Notifications

Configure notification channels under Settings → Notifications. Each channel can be toggled independently per alert type.

Channel	Setup
Slack	Paste a Slack Incoming Webhook URL
Email	Requires SMTP configuration in `.env`

// Recommendations

AI Cost Recommendations

The recommendation engine runs every 24 hours and analyzes your usage patterns to surface actionable savings — always with a quantified dollar amount.

🐜

"Switch gpt-4o → gpt-4o-mini on 73% of your workload and save $432/month." — that's the kind of insight AIntOps surfaces automatically.

Model substitution recommendations

AIntOps identifies requests where a cheaper model would produce equivalent quality output based on your usage patterns and request characteristics.

Recommendation type	Description
Model downgrade	Switch to a cheaper model for a subset of your workload
Prompt compression	Identified prompts that can be shortened without quality loss
Caching opportunities	Repeated identical requests that could be cached
Provider switch	Equivalent model available cheaper on another provider

Recommendations appear in the top_recommendations dashboard widget and are sent via your notification channels if configured.

// Teams & Security

Organizations & Roles

AIntOps is multi-tenant by design. Every piece of data is scoped by organization_id — there is zero cross-organization data leakage.

Roles

Role	Permissions
`owner` 1 minimum	Full access. Can manage members, providers, budgets, billing. Cannot remove themselves if they are the last owner.
`admin`	Can manage providers, dashboards, budgets, and members (except other owners).
`member`	Read-only access to dashboards and recommendations. Cannot modify settings.

Inviting members

Navigation

Settings → Members → Invite → Enter email → Choose role → Send

🔒

Accessing a resource that belongs to another organization returns 404, not 403 — preventing enumeration attacks.

Security

Mechanism	Implementation
Credentials at rest	Fernet symmetric encryption (AES-128 + HMAC-SHA256)
Passwords	bcrypt hashing — no plaintext storage
Authentication	Short-lived JWT access tokens + long-lived refresh tokens with rotation
Anti-stampede	Queue-based refresh token rotation to prevent race conditions
Multi-tenancy	Every query scoped by `organization_id` at the ORM level
Last-owner protection	Cannot remove the last owner of an organization

// Architecture

Tech Stack

Layer	Technology
Backend	FastAPI · SQLAlchemy 2 async · Pydantic 2 · Alembic
Database	PostgreSQL 16 + TimescaleDB (hypertables + continuous aggregates)
Cache / Queue	Redis 7
Scheduler	APScheduler (in-process, async)
Auth	JWT (access + refresh) · bcrypt · multi-org JWT enrichment
Encryption	Fernet symmetric encryption for credentials at rest
Frontend	React 18 · Vite · Tailwind CSS · Zustand · Recharts
Infrastructure	Docker Compose · Caddy · rolling migrations · health checks

Scheduler Jobs

Job	Interval	Description
Alert evaluation	Every 1 min	Check thresholds and fire notifications
Provider sync	Every 1 hour	Pull usage + cost data from provider APIs
Anomaly detection	Every 15 min	Z-score baseline analysis
Recommendations	Every 24 hours	Generate model substitution suggestions
Budget monitoring	Every 30 min	Evaluate spend against budget thresholds

Why TimescaleDB?

LLM cost events are time-series data by nature. TimescaleDB gives AIntOps:

Hypertables chunked by day — handles 100M+ events without slowdown
Continuous aggregates (hourly, daily) refreshed automatically — 30-day dashboard queries return in under 50ms
Compression on chunks older than 7 days — ~10× storage savings
Retention policies — raw events kept 90 days, aggregates kept forever