Data Synchronization

The Discord bot synchronizes data from Discord to the Heimdall database via WebSocket request-response patterns. This enables the Heimdall backend to display and manage Discord server information.

Overview

What Gets Synced

Data Type	Description	Sync Trigger
Guilds	Server metadata (name, icon, member count, boost level, bot join date)	Manual or periodic
Channels	All channel types with hierarchy and settings	Manual or periodic
Roles	Role metadata with colors and permissions	Manual or periodic
Members	Full member list with roles, join dates, status	Manual or periodic

Sync Operations

The API provides granular control over what data to sync:

Operation	GraphQL Mutation	Description
Sync All	`syncDiscordGuilds`	Sync guilds + channels + roles
Sync Guilds Only	`syncDiscordGuildsOnly`	Only guild metadata
Sync Channels Only	`syncDiscordChannelsOnly(guildId)`	Only channels for a guild
Sync Roles Only	`syncDiscordRolesOnly(guildId)`	Only roles for a guild
Sync Members	`syncDiscordMembers(guildId)`	All members for a guild

Guild/Channel/Role Sync

Sync Flow

┌──────────────────┐        ┌──────────────────┐        ┌──────────────────┐
│  Backend Console │        │   Heimdall API   │        │   Discord Bot    │
└────────┬─────────┘        └────────┬─────────┘        └────────┬─────────┘
         │                           │                           │
         │  syncDiscordGuilds()      │                           │
         │─────────────────────────>│                           │
         │                           │                           │
         │                           │  SyncDiscordGuildsRequest │
         │                           │  (with sync_type)         │
         │                           │─────────────────────────>│
         │                           │                           │
         │                           │   (Bot fetches from      │
         │                           │    Discord API based     │
         │                           │    on sync_type)         │
         │                           │                           │
         │                           │  SyncDiscordGuildsResponse│
         │                           │<─────────────────────────│
         │                           │                           │
         │                           │  (API stores in database)│
         │                           │                           │
         │  { success, guildsSynced, │                           │
         │    channelsSynced,        │                           │
         │    rolesSynced }          │                           │
         │<─────────────────────────│                           │

Protocol Messages

Sync Type Enum

enum DiscordSyncType {
  DISCORD_SYNC_TYPE_ALL = 0;           // Sync guilds, channels, and roles
  DISCORD_SYNC_TYPE_GUILDS_ONLY = 1;   // Sync only guild metadata
  DISCORD_SYNC_TYPE_CHANNELS_ONLY = 2; // Sync only channels for a guild
  DISCORD_SYNC_TYPE_ROLES_ONLY = 3;    // Sync only roles for a guild
  DISCORD_SYNC_TYPE_MEMBERS_ONLY = 4;  // Sync only members for a guild
}

Request Message

message SyncDiscordGuildsRequest {
  optional string guild_id = 1;    // Optional: sync specific guild only
  string request_id = 2;           // Unique request ID for response matching
  DiscordSyncType sync_type = 3;   // Type of sync to perform
}

The sync_type field optimizes network traffic by telling the bot exactly what data to fetch:

Sync Type	Fetches	Use Case
`ALL`	Guilds + Channels + Roles + Members	Full sync, periodic sync
`GUILDS_ONLY`	Guild metadata only	Quick metadata refresh
`CHANNELS_ONLY`	Channels for specified guild	After channel changes
`ROLES_ONLY`	Roles for specified guild	After role changes
`MEMBERS_ONLY`	Members for specified guild	Targeted member refresh

Response Message

message SyncDiscordGuildsResponse {
  string request_id = 1;
  bool success = 2;
  optional string error = 3;
  repeated DiscordGuildData guilds = 4;
}

message DiscordGuildData {
  string guild_id = 1;               // Discord snowflake ID
  string name = 2;
  optional string icon = 3;          // Icon hash
  string owner_id = 4;
  optional int32 member_count = 5;
  repeated DiscordChannelData channels = 6;
  optional int32 boost_count = 7;
  optional int32 premium_tier = 8;   // 0=none, 1-3=boost tiers
  repeated DiscordRoleData roles = 9;
  repeated DiscordMemberData members = 10;  // Populated when sync_type=MEMBERS_ONLY
  optional string bot_joined_at = 11;       // When the bot joined this guild (RFC3339)
}

message DiscordChannelData {
  string channel_id = 1;
  string name = 2;
  int32 channel_type = 3;            // Discord channel type enum
  optional string parent_id = 4;     // Parent category ID
  int32 position = 5;
  optional string topic = 6;
  bool nsfw = 7;
}

message DiscordRoleData {
  string role_id = 1;
  string name = 2;
  int32 color = 3;                   // RGB color value
  int32 position = 4;
  string permissions = 5;            // Permission bitfield as string
  bool hoist = 6;                    // Displayed separately
  bool managed = 7;                  // Bot/integration managed
  bool mentionable = 8;
}

Channel Types

Type	Value	Description
Text	0	Text channel
DM	1	Direct message
Voice	2	Voice channel
Group DM	3	Group direct message
Category	4	Channel category
Announcement	5	News/announcement channel
Announcement Thread	10	Thread in announcement
Public Thread	11	Public thread
Private Thread	12	Private thread
Stage	13	Stage voice channel
Directory	14	Hub directory
Forum	15	Forum channel
Media	16	Media channel

Member Sync

Members are synced automatically as part of the ALL sync (including periodic sync). You can also sync members separately using sync_type = MEMBERS_ONLY for a targeted refresh without re-syncing channels and roles.

Sync Flow

┌──────────────────┐        ┌──────────────────┐        ┌──────────────────┐
│  Backend Console │        │   Heimdall API   │        │   Discord Bot    │
└────────┬─────────┘        └────────┬─────────┘        └────────┬─────────┘
         │                           │                           │
         │  syncDiscordMembers()     │                           │
         │─────────────────────────>│                           │
         │                           │                           │
         │                           │  SyncDiscordGuildsRequest │
         │                           │  (sync_type=MEMBERS_ONLY) │
         │                           │─────────────────────────>│
         │                           │                           │
         │                           │   (Bot fetches members   │
         │                           │    via Discord API)      │
         │                           │                           │
         │                           │  SyncDiscordGuildsResponse│
         │                           │  (with members data)      │
         │                           │<─────────────────────────│
         │                           │                           │
         │                           │  (API upserts members)   │
         │                           │                           │
         │  { success, membersSynced}│                           │
         │<─────────────────────────│                           │

Protocol Messages

Member sync uses the unified sync request with sync_type = DISCORD_SYNC_TYPE_MEMBERS_ONLY:

// Request - same as guild/channel/role sync
message SyncDiscordGuildsRequest {
  optional string guild_id = 1;    // Required for member sync
  string request_id = 2;
  DiscordSyncType sync_type = 3;   // DISCORD_SYNC_TYPE_MEMBERS_ONLY
}

// Response includes member data
message SyncDiscordGuildsResponse {
  string request_id = 1;
  bool success = 2;
  optional string error = 3;
  repeated DiscordGuildData guilds = 4;  // Contains members when sync_type=MEMBERS_ONLY
}

Member Data Structure

message DiscordMemberData {
  string user_id = 1;                           // Discord user ID (snowflake)
  string username = 2;
  optional string discriminator = 3;            // Legacy, may be "0"
  optional string global_name = 4;              // Global display name
  optional string nickname = 5;                 // Server nickname
  optional string avatar = 6;                   // Avatar hash
  bool bot = 7;
  bool system = 8;
  optional string joined_at = 9;                // ISO 8601
  optional string premium_since = 10;           // Boosting since
  bool deaf = 11;                               // Voice deafened
  bool mute = 12;                               // Voice muted
  bool pending = 13;                            // Membership screening
  optional string communication_disabled_until = 14;  // Timeout
  repeated string roles = 15;                   // Role IDs
}

Unified Sync Approach

All sync operations (guilds, channels, roles, members) use the same SyncDiscordGuildsRequest and SyncDiscordGuildsResponse messages. The sync_type field determines what data is fetched and returned.

Performance Considerations

Large guilds: Member sync can take 30-60 seconds for guilds with 10,000+ members
Rate limits: Discord API rate limits apply; the bot handles these automatically
Timeout: API has a 60-second timeout for member sync operations
Incremental updates: Currently does full sync; no incremental member updates

Database Schema

Guilds Table

CREATE TABLE "DiscordGuild" (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    guild_id TEXT NOT NULL UNIQUE,  -- Discord snowflake
    name TEXT NOT NULL,
    icon TEXT,
    owner_id TEXT NOT NULL,
    member_count INTEGER,
    boost_count INTEGER,
    premium_tier INTEGER,           -- 0=none, 1-3=boost tiers
    default_language TEXT,          -- Per-guild language override
    bot_joined_at TIMESTAMPTZ,
    last_synced_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

Channels Table

CREATE TABLE "DiscordChannel" (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    channel_id TEXT NOT NULL UNIQUE,  -- Discord snowflake
    guild_id TEXT NOT NULL,           -- Discord guild snowflake
    guild_uuid UUID NOT NULL REFERENCES "DiscordGuild"(id) ON DELETE CASCADE,  -- Internal FK
    name TEXT NOT NULL,
    channel_type SMALLINT NOT NULL,
    channel_type_raw SMALLINT NOT NULL,
    parent_id TEXT,                   -- Parent category ID
    position INTEGER NOT NULL DEFAULT 0,
    topic TEXT,
    nsfw BOOLEAN NOT NULL DEFAULT FALSE,
    is_text_based BOOLEAN NOT NULL DEFAULT FALSE,
    is_voice BOOLEAN NOT NULL DEFAULT FALSE,
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

Roles Table

CREATE TABLE "DiscordRole" (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    guild_id UUID NOT NULL REFERENCES "DiscordGuild"(id) ON DELETE CASCADE,
    role_id TEXT NOT NULL,            -- Discord snowflake
    name TEXT NOT NULL,
    color INTEGER NOT NULL DEFAULT 0,
    position INTEGER NOT NULL DEFAULT 0,
    permissions TEXT,
    hoist BOOLEAN NOT NULL DEFAULT FALSE,
    managed BOOLEAN NOT NULL DEFAULT FALSE,
    mentionable BOOLEAN NOT NULL DEFAULT FALSE,
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    UNIQUE (guild_id, role_id)
);

Members Table

CREATE TABLE "DiscordMember" (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    guild_id UUID NOT NULL REFERENCES "DiscordGuild"(id) ON DELETE CASCADE,
    user_id TEXT NOT NULL,            -- Discord snowflake
    username TEXT NOT NULL,
    discriminator TEXT,
    global_name TEXT,
    nickname TEXT,
    avatar TEXT,
    bot BOOLEAN NOT NULL DEFAULT FALSE,
    system BOOLEAN NOT NULL DEFAULT FALSE,
    joined_at TIMESTAMPTZ,
    premium_since TIMESTAMPTZ,
    deaf BOOLEAN NOT NULL DEFAULT FALSE,
    mute BOOLEAN NOT NULL DEFAULT FALSE,
    pending BOOLEAN NOT NULL DEFAULT FALSE,
    communication_disabled_until TIMESTAMPTZ,
    roles TEXT[] NOT NULL DEFAULT '{}',
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    UNIQUE (guild_id, user_id)
);

Periodic Sync

The bot supports automatic periodic syncing of guild data, controlled by settings that can be changed at runtime.

Configuration

Periodic sync is controlled via bot settings in the database. These settings can be modified from the Discord Settings page in the Heimdall Console.

Setting	Default	Description
`auto_sync_enabled`	`true`	Enable/disable periodic sync
`sync_interval_seconds`	`86400`	Interval between syncs (24 hours)
`initial_delay_seconds`	`300`	Delay after bot startup (5 minutes)

Runtime Settings

Settings are managed dynamically:

On Startup: The bot fetches global settings from the API via GraphQL (discordBotSettings query)
Runtime Updates: When settings are changed in the backend, a DiscordSettingsChanged WebSocket event is broadcast to all connected bots
Immediate Effect: The bot updates its internal state immediately - no restart required

┌──────────────────┐        ┌──────────────────┐        ┌──────────────────┐
│  Backend Console │        │   Heimdall API   │        │   Discord Bot    │
└────────┬─────────┘        └────────┬─────────┘        └────────┬─────────┘
         │                           │                           │
         │  Update auto_sync_enabled │                           │
         │─────────────────────────>│                           │
         │                           │                           │
         │                           │  DiscordSettingsChanged   │
         │                           │  (via WebSocket)          │
         │                           │─────────────────────────>│
         │                           │                           │
         │                           │   (Bot updates internal   │
         │                           │    sync_settings)         │
         │                           │                           │
         │                           │   (Periodic sync task     │
         │                           │    respects new settings) │

Behavior

When auto_sync_enabled = true:

Bot waits for initial_delay_seconds after startup
Performs full sync (guilds + channels + roles)
Waits for sync_interval_seconds
Repeats from step 2

When auto_sync_enabled = false:

Periodic sync is disabled
Bot checks every 60 seconds if auto sync was re-enabled
Guilds are only synced when manually triggered from the backend

tip

To sync guilds without automatic periodic sync, disable auto_sync_enabled and use the Sync Guilds button on the Discord Guilds page in the console.

note

Periodic sync includes members. For large guilds (10,000+ members), this may take additional time due to Discord API rate limits.

Audit Events

All sync operations are logged to the audit system:

Event	Description
Sync All	"Synced X guild(s), Y channel(s), and Z role(s) from Discord"
Sync Guilds Only	"Synced X guild(s) from Discord"
Sync Channels	"Synced X channel(s) from Discord for guild [name]"
Sync Roles	"Synced X role(s) from Discord for guild [name]"
Sync Members	"Synced X member(s) from Discord for guild [name]"

GraphQL API

Queries

# List all guilds
query {
  discordGuilds {
    id guildId name icon iconUrl memberCount boostCount premiumTier
  }
}

# Get guild with channels
query {
  discordGuildWithChannels(guildId: "123456789") {
    guild { id name memberCount }
    channels { id name channelType position }
  }
}

# List members with filtering
query {
  discordMembers(guildId: "123456789", filter: { search: "john", humansOnly: true }) {
    members { userId username displayName roles }
    totalCount hasMore
  }
}

Mutations

# Sync all
mutation { syncDiscordGuilds { success guildsSynced channelsSynced rolesSynced } }

# Sync guilds only
mutation { syncDiscordGuildsOnly { success guildsSynced } }

# Sync channels for a guild
mutation { syncDiscordChannelsOnly(guildId: "5543dd84-9fe3-4331-b071-f2e7e39c8bbe") { success channelsSynced } }

# Sync roles for a guild
mutation { syncDiscordRolesOnly(guildId: "5543dd84-9fe3-4331-b071-f2e7e39c8bbe") { success rolesSynced } }

# Sync members for a guild
mutation { syncDiscordMembers(guildId: "5543dd84-9fe3-4331-b071-f2e7e39c8bbe") { success membersSynced } }

Guild ID Format

All guildId parameters require the internal UUID (database ID), not the Discord snowflake ID. Example: "5543dd84-9fe3-4331-b071-f2e7e39c8bbe"

You can get the internal UUID from the guild list query or from the URL when viewing a guild in the console.

Permissions

Permission	Description
`discord:read`	Main Discord access (sidebar, guilds list, bot settings view)
`discord:edit`	Edit Discord bot settings
`discord:delete`	Delete Discord guilds from the database
`discord:sync`	Trigger sync operations to discover new guilds
`discord:guild.read`	View Discord guild data (channels, roles, members)
`discord:guild.edit`	Edit Discord guild data and member information
`discord:guild.sync`	Resync data for a specific guild (channels, roles, members)

Overview​

What Gets Synced​

Sync Operations​

Guild/Channel/Role Sync​

Sync Flow​

Protocol Messages​

Sync Type Enum​

Request Message​

Response Message​

Channel Types​

Member Sync​

Sync Flow​

Protocol Messages​

Member Data Structure​

Performance Considerations​

Database Schema​

Guilds Table​

Channels Table​

Roles Table​

Members Table​

Periodic Sync​

Configuration​

Runtime Settings​

Behavior​

Audit Events​

GraphQL API​

Queries​

Mutations​

Permissions​

Overview

What Gets Synced

Sync Operations

Guild/Channel/Role Sync

Sync Flow

Protocol Messages

Sync Type Enum

Request Message

Response Message

Channel Types

Member Sync

Sync Flow

Protocol Messages

Member Data Structure

Performance Considerations

Database Schema

Guilds Table

Channels Table

Roles Table

Members Table

Periodic Sync

Configuration

Runtime Settings

Behavior

Audit Events

GraphQL API

Queries

Mutations

Permissions