Architecture Overview
This document provides a high-level overview of the Heimdall platform architecture.
System Architecture
┌──────────────────────────────────────────────────────────────────────────────────┐
│ Internet │
├──────────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ Client │ │ Backend │ │ ID │ │ Policies │ │ Docs │ │
│ │ Apps │ │ (Next.js)│ │ (Next.js)│ │ (Next.js)│ │(Docusaur)│ │
│ │ (REST/WS)│ │ :3001 │ │ :3002 │ │ :3004 │ │ :3003 │ │
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ └──────────┘ └──────────┘ │
│ │ │ │ │
│ │ HTTPS │ HTTPS │ HTTPS │
│ │ WebSocket │ GraphQL │ OAuth │
│ │ │ REST │ │
│ │ │ │ │
│ ┌────┴─────────────┴─────────────┴──────────────────────────────────────┐ │
│ │ Load Balancer │ │
│ │ (nginx / cloud LB) │ │
│ └───────────────────────────────┬────────────────────────────────────────┘ │
│ │ │
│ ┌─────────────┐ ┌─────────────┐│ │
│ │ Discord Bot │ │ Twitch Bot ││ ◄── Bots (Rust, Poise/Serenity) │
│ │ (Rust) │ │ (Rust) ││ GraphQL, REST, WebSocket │
│ │ :3006 │ │ :3007 ││ │
│ └──────┬──────┘ └──────┬──────┘│ │
│ │ │ │ │
│ └────────────────┼───────┘ │
│ │ │
│ ┌───────────────────────┴────────────────────────────────────────────────┐ │
│ │ Rust API │ │
│ │ (Actix-web Server) │ │
│ │ │ │
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │
│ │ │ REST API │ │ GraphQL │ │ WebSocket │ │ OAuth │ │ │
│ │ │ Handlers │ │ Resolvers │ │ Handlers │ │ Provider │ │ │
│ │ └─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │
│ │ │ │
│ │ ┌───────────────────────────────────────────────────────────────┐ │ │
│ │ │ Middleware │ │ │
│ │ │ (Auth, Rate Limiting, CORS, Logging, 2FA) │ │ │
│ │ └───────────────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ ┌───────────────────────────────────────────────────────────────┐ │ │
│ │ │ Business Logic │ │ │
│ │ │ (Services, Validation, Permissions, GDPR) │ │ │
│ │ └───────────────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ └───────────────────────────────┬─────────────────────────────────────────┘ │
│ │ │
│ ┌───────────────────────────────┴─────────────────────────────────────────┐ │
│ │ Data Layer │ │
│ │ │ │
│ │ ┌─────────────────────┐ ┌─────────────────────┐ ┌─────────────────┐ │ │
│ │ │ PostgreSQL │ │ TimescaleDB │ │ Redis │ │ │
│ │ │ (Primary Store) │ │ (Time-Series DB) │ │ (Cache & Rate) │ │ │
│ │ │ │ │ │ │ │ │ │
│ │ │ - Users │ │ - AuditEvent │ │ - Session Cache │ │ │
│ │ │ - GPS Data │ │ (hypertable) │ │ - Rate Limiting │ │ │
│ │ │ - API Keys │ │ - Time-based │ │ - Pub/Sub │ │ │
│ │ │ - Roles/Permissions │ │ partitioning │ │ - Token Cache │ │ │
│ │ │ - OAuth Clients │ │ - Automatic data │ │ - 2FA Tokens │ │ │
│ │ │ - 2FA Secrets │ │ retention │ │ │ │ │
│ │ │ - Data Exports │ │ - GeoIP enrichment │ │ │ │ │
│ │ └─────────────────────┘ └─────────────────────┘ └─────────────────┘ │ │
│ │ │ │
│ └──────────────────────────────────────────────────────────────────────────┘ │
│ │
└───────────────────────────────────────────────────────────────────────────────────┘
Component Overview
Rust API (platform/api)
The core backend service, handling all API requests.
Technology Stack:
- Framework: Actix-web 4
- Database: SQLx with PostgreSQL (primary) + TimescaleDB (time-series)
- GraphQL: async-graphql
- OpenAPI: utoipa with Swagger UI
- Caching: Redis
- GeoIP: MaxMind GeoLite2
Responsibilities:
- REST API endpoints
- GraphQL queries and mutations
- WebSocket connections for real-time updates
- Authentication and authorization
- Rate limiting
- Data persistence
Backend (platform/backend)
Administrative interface for platform management.
Technology Stack:
- Framework: Next.js 16 (App Router)
- Language: TypeScript
- Styling: TailwindCSS 4
- Auth: NextAuth.js 5
- GraphQL: Apollo Client
Responsibilities:
- Admin dashboard
- GPS data visualization
- User management
- Role management
- API key management
- Developer tools (OAuth apps)
ID (platform/id)
Identity and authentication service with GDPR compliance.
Technology Stack:
- Framework: Next.js 16 (App Router)
- Language: TypeScript
- Styling: TailwindCSS 4
- Auth: NextAuth.js 5
- i18n: next-intl
Responsibilities:
- User registration and login
- Two-factor authentication (TOTP)
- OAuth 2.0 provider functionality
- Account management
- Account linking (multiple providers)
- Connected apps management
- GDPR data export
- Account deletion with grace period
- Session management
Policies (platform/policies)
Legal, privacy, and compliance content.
Technology Stack:
- Framework: Next.js 16 (App Router)
- Language: TypeScript
- Styling: TailwindCSS 4
- i18n: next-intl
Responsibilities:
- Privacy policy
- Terms of service
- Cookie policy
- Security information
- FAQ
Documentation (platform/docs)
This documentation site.
Technology Stack:
- Framework: Docusaurus 3
- Language: TypeScript/MDX
Discord Bot (platform/discord_bot)
Discord integration bot for the Heimdall platform.
Technology Stack:
- Framework: Poise (on Serenity-rs)
- Language: Rust 2024 edition
- API Communication: GraphQL, REST, WebSocket (Protobuf)
- i18n: rust-i18n
Responsibilities:
- Slash commands and prefix commands
- RBAC permission checking via Heimdall API
- Audit event logging to Heimdall
- Real-time event notifications via WebSocket
- Moderation commands (warn, kick, ban, mute)
- Admin settings management
Twitch Bot (platform/twitch_bot)
Twitch integration bot for the Heimdall platform.
Technology Stack:
- Framework: twitch-irc (Rust)
- Language: Rust 2024 edition
- API Communication: GraphQL, REST, WebSocket (Protobuf)
Responsibilities:
- Chat commands
- Stream event handling
- Audit event logging to Heimdall
- Real-time notifications
Data Flow
REST API Request Flow
Client Request
│
▼
┌─────────────┐
│ CORS │
│ Middleware │
└──────┬──────┘
│
▼
┌─────────────┐
│ Logger │
│ Middleware │
└──────┬──────┘
│
▼
┌─────────────┐
│ Auth │
│ Middleware │
└──────┬──────┘
│
▼
┌─────────────┐
│ Rate Limit │
│ Middleware │
└──────┬──────┘
│
▼
┌─────────────┐
│ Handler │
│ Function │
└──────┬──────┘
│
▼
┌─────────────┐
│ Service │
│ Layer │
└──────┬──────┘
│
▼
┌─────────────┐
│ Database │
│ Query │
└──────┬──────┘
│
▼
Response
WebSocket Connection Flow
Client Connect
│
▼
┌─────────────┐
│ WebSocket │
│ Upgrade │
└──────┬──────┘
│
▼
┌─────────────┐
│ State │
│ Manager │──────┐
└──────┬──────┘ │
│ │
▼ ▼
┌─────────────┐ ┌─────────────┐
│ Connection │ │ Channel │
│ Handler │ │ Manager │
└──────┬──────┘ └──────┬──────┘
│ │
▼ ▼
┌─────────────┐ ┌─────────────┐
│ Subscribe/ │ │ Publish │
│ Unsubscribe │ │ Updates │
└─────────────┘ └─────────────┘
Database Architecture
Heimdall uses a dual-database architecture:
- PostgreSQL (Primary) - Core application data (users, sessions, API keys, etc.)
- TimescaleDB (Time-Series) - Audit events stored in a hypertable with automatic time-based partitioning
This separation provides:
- Optimized time-series queries for audit logs with automatic compression
- Independent scaling for OLTP vs analytics workloads
- GeoIP enrichment for security monitoring (country, city, region)
- Automatic data retention policies for compliance
Core Tables (PostgreSQL)
-- Users (managed by Heimdall ID)
User
├── id (UUID)
├── twitch_id
├── username
├── email
├── avatar_url
├── created_at
└── updated_at
-- Sessions (NextAuth)
Session
├── id (UUID)
├── user_id → User
├── session_token
├── expires
└── created_at
-- API Keys
ApiKey
├── id (UUID)
├── key (hashed)
├── name
├── description
├── user_id → User (optional)
├── scopes[]
├── is_active
├── expires_at
├── last_used_at
└── created_at
-- Roles
Role
├── id (UUID)
├── name (unique)
├── description
├── is_system
└── created_at
-- Permissions
Permission
├── id (UUID)
├── resource
├── action
├── description
└── created_at
-- GPS Data
GpsData
├── id (UUID)
├── latitude
├── longitude
├── altitude
├── speed
├── timestamp
└── created_at
-- Settings
Setting
├── key (primary)
├── value (JSON)
├── is_public
└── updated_at
-- OAuth Clients
OAuthClient
├── id (UUID)
├── client_id
├── client_secret (hashed)
├── name
├── redirect_uris[]
├── user_id → User
└── created_at
Audit Events (TimescaleDB)
Audit events are stored in a TimescaleDB hypertable with automatic time-based partitioning and GeoIP enrichment:
-- Audit Events (TimescaleDB Hypertable)
AuditEvent
├── id (UUID)
├── user_id (optional, can be Heimdall UUID or external ID like Discord snowflake)
├── event_type (string)
├── resource_type (optional)
├── resource_id (optional)
├── actor_id (optional, for admin actions)
├── ip_address (optional)
├── user_agent (optional)
├── description (optional)
├── metadata (JSONB)
├── status (success/failure)
├── error_message (optional)
├── country_code (ISO 3166-1 alpha-2, auto-enriched from IP)
├── country_name (auto-enriched from IP)
├── city (auto-enriched from IP)
├── region (auto-enriched from IP)
├── source_service (optional)
├── is_reported (boolean, for suspicious activity reports)
└── created_at (partitioning key)
Key Features:
- Hypertable: Automatic time-based partitioning (7-day chunks)
- GeoIP Enrichment: Country, city, region automatically added from IP address
- Compression: Automatic compression after 30 days
- Retention: Configurable data retention policies
Audit Event Types
| Event Type | Description |
|---|---|
| Authentication | |
login | User signed in |
login_failed | Sign-in attempt failed |
logout | User signed out |
password_changed | Password was changed |
password_reset_requested | Password reset requested |
| Two-Factor Auth | |
2fa_enabled | 2FA was enabled |
2fa_disabled | 2FA was disabled |
2fa_verified | 2FA code verified |
2fa_backup_codes_regenerated | Backup codes regenerated |
| Sessions | |
session_created | New session created |
session_revoked | Session was revoked |
all_sessions_revoked | All other sessions revoked |
| Account | |
user_created | Account created |
user_updated | Profile updated |
user_deleted | Account deleted |
deletion_scheduled | Account deletion scheduled |
deletion_cancelled | Scheduled deletion cancelled |
| Account Links | |
account_linked | Platform account linked |
account_unlinked | Platform account unlinked |
primary_account_changed | Primary account changed |
| OAuth | |
consent_granted | OAuth consent given |
consent_revoked | OAuth consent revoked |
token_created | OAuth token created |
token_revoked | OAuth token revoked |
client_created | OAuth client created |
client_updated | OAuth client updated |
client_deleted | OAuth client deleted |
client_secret_regenerated | OAuth client secret regenerated |
| API Keys | |
api_key_created | API key was created |
api_key_updated | API key was updated |
api_key_revoked | API key was revoked |
| Admin | |
user_banned | User was banned |
user_unbanned | User was unbanned |
role_assigned | Role assigned to user |
role_removed | Role removed from user |
permission_changed | Permission changed |
| Data | |
data_exported | GDPR data export completed |
activity_reported | Suspicious activity reported |
| Bot Events | |
bot_command_executed | Bot command executed |
bot_config_changed | Bot configuration changed |
bot_moderation_action | Bot moderation action |
bot_guild_configured | Discord guild configured |
| Integrations | |
integration_connected | Platform integration connected |
integration_reconnected | Integration re-authorized |
integration_disconnected | Integration disconnected |
integration_token_refreshed | Token automatically refreshed |
integration_status_updated | Integration status updated |
Source Services
The source_service field tracks which application created the audit event:
| Source Service | Description |
|---|---|
api | Direct API access |
id | Heimdall ID webapp |
backend | Backend dashboard |
policies | Policies webapp |
discord_bot | Discord bot |
twitch_bot | Twitch bot |
Junction Tables
UserRole (user_id, role_id)
ApiKeyRole (api_key_id, role_id)
RolePermission (role_id, permission_id)
LinkedAccount (user_id, provider, provider_account_id)
Deployment Architecture
Development
┌───────────────────────────────────────────────────────────────────────────┐
│ Local Development │
├───────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ │
│ │ API │ │Backend │ │ ID │ │ Docs │ │Policies│ │
│ │ :3000 │ │ :3001 │ │ :3002 │ │ :3003 │ │ :3004 │ │
│ └───┬────┘ └───┬────┘ └───┬────┘ └────────┘ └────────┘ │
│ │ │ │ │
│ └───────────┼───────────┘ │
│ │ │
│ ┌───────────────┼───────────────────────────────────────┐ │
│ │ │ │ │
│ │ ┌────────────┴────────────┐ ┌────────────────────┐ │ │
│ │ │ Discord Bot │ │ Twitch Bot │ │ │
│ │ │ :3006 │ │ :3007 │ │ Rust Bots │
│ │ └────────────┬────────────┘ └─────────┬──────────┘ │ │
│ │ └──────────────┬──────────┘ │ │
│ └──────────────────────────────┼────────────────────────┘ │
│ │ │
│ ┌──────────────────────────────┴────────────────────────────────┐ │
│ │ Docker Compose │ │
│ │ ┌────────────┐ ┌────────────┐ ┌────────────┐ ┌─────────┐ │ │
│ │ │ PostgreSQL │ │TimescaleDB │ │ Redis │ │ MailDev │ │ │
│ │ │ :5432 │ │ :5433 │ │ :6379 │ │ :1080 │ │ │
│ │ └────────────┘ └────────────┘ └────────────┘ └─────────┘ │ │
│ └───────────────────────────────────────────────────────────────┘ │
│ │
└───────────────────────────────────────────────────────────────────────────┘
Production
┌───────────────────────────────────────────────────────────────────┐
│ Production Deployment │
├───────────────────────────────────────────────────────────────────┤
│ │
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ CDN / Edge Network │ │
│ │ (Cloudflare / Vercel Edge) │ │
│ └────────────────────────────────────────────────────────────┘ │
│ │ │
│ ┌───────────────────────────┴────────────────────────────────┐ │
│ │ Load Balancer │ │
│ │ (nginx / Cloud LB) │ │
│ └───────────────────────────┬────────────────────────────────┘ │
│ │ │
│ ┌───────────────────────────┼────────────────────────────────┐ │
│ │ ┌──────────┐ ┌───────┴────┐ ┌──────────┐ │ │
│ │ │ API Pod │ │ API Pod │ │ API Pod │ │ │
│ │ │ #1 │ │ #2 │ │ #3 │ │ │
│ │ └──────────┘ └────────────┘ └──────────┘ │ │
│ │ (Kubernetes) │ │
│ └───────────────────────────┬────────────────────────────────┘ │
│ │ │
│ ┌───────────────────────────┼────────────────────────────────┐ │
│ │ ┌─────────────────┐ │ ┌─────────────────┐ │ │
│ │ │ PostgreSQL │ │ │ Redis │ │ │
│ │ │ (Primary) │◄─────┴─────►│ (Cluster) │ │ │
│ │ └────────┬────────┘ └─────────────────┘ │ │
│ │ │ │ │
│ │ ┌────────┴────────┐ │ │
│ │ │ PostgreSQL │ │ │
│ │ │ (Replica) │ │ │
│ │ └─────────────────┘ │ │
│ │ (Managed Database) │ │
│ └────────────────────────────────────────────────────────────┘ │
│ │
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ Next.js Apps │ │
│ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │
│ │ │ Backend │ │ ID │ │ Policies │ │ Docs │ │ │
│ │ │ (Vercel) │ │ (Vercel) │ │ (Vercel) │ │ (Vercel) │ │ │
│ │ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │ │
│ └────────────────────────────────────────────────────────────┘ │
│ │
└───────────────────────────────────────────────────────────────────┘
Security Architecture
Authentication Layers
-
Session-based (Next.js apps via NextAuth)
- HTTP-only secure cookies
- JWT tokens with short expiry
- Automatic refresh
-
Token-based (API access)
- Bearer tokens in Authorization header
- API keys for programmatic access
- OAuth 2.0 for third-party apps
-
Two-Factor Authentication
- TOTP-based (RFC 6238)
- Backup codes for recovery
- Role-based 2FA requirements
Authorization Model
Request
│
▼
┌─────────────────────┐
│ Extract Auth Token │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Lookup Token │
│ (JWT or API Key) │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Get User/Key │
│ Roles │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Aggregate │
│ Permissions │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Check Permission │
│ (resource:action) │
└──────────┬──────────┘
│
┌─────┴─────┐
│ │
▼ ▼
Allow Deny
Next Steps
- Auth System - Detailed authentication architecture
- API Overview - API documentation
- Web Apps - Frontend applications
- Discord Bot - Discord bot documentation