Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

How PolarBot works

This page explains PolarBot’s architecture: what runs, what talks to what, where data goes, and how load is kept under Discord’s limits. The scanner and the security model have their own pages: The scanner and Security model. Exact names, values and files are in the Reference.

The process

PolarBot is one Rust binary (polarbot) running as one systemd service. At startup it:

  1. reads the config file (unknown or secret-looking keys stop it here, exit status 1);
  2. reads the discord_token credential and opens the SQLite database, running any new migrations;
  3. asks Discord who it is (application and bot user), which also proves the token works;
  4. loads the cached threat lists and adult-site lists from the data directory (an empty list if there is no cache yet);
  5. loads the two OAuth credentials if both are present (otherwise command visibility and the web server stay off);
  6. connects to the Discord gateway with the shard count Discord recommends, and starts the background jobs.

After the first Ready from the gateway (Discord accepted the token and the intents) it registers the slash commands (globally, or in discord.dev_guild_id only), queues a command visibility sync for every authorized server, and posts the one-time “PolarBot updated to vX.Y.Z” notice if this version has not been announced yet. Doing this only after Ready means a start that Discord refuses never touches the registered commands.

On SIGTERM (systemctl stop) or Ctrl-C it closes the gateway connections and gives running handlers and jobs at most 10 seconds to finish.

Architecture

                    Discord gateway (one websocket per shard)
                                   | events
                                   v
+---------------------------- shard loop -----------------------------+
| 1. gateway_log::prepare  (cheap, never waits: is this event logged?) |
| 2. cache.update          (guilds, channels, roles, members, the last |
|                           200 messages per channel)                  |
| 3. start a handler task if a slot of its pool is free               |
+--------+-----------------------+--------------------------+----------+
         | interactions          | messages, deletes        | other logged events
         | (64 slots)            | (256 slots)              | (64 slots)
         v                       v                          v
  commands::dispatch      scanner pipeline            gateway_log --> log dispatcher
  card buttons            server/role/channel                        (per-server queue,
  /help autocomplete      deletes                                     batched posts)
         |                       |
         v                       v
  Discord HTTP API        scanner -> actions -> card queue --> log channel
         |                       |
         +------------> SQLite database <------------ background jobs
                                                      (list updater, sweeps, purges)

  OAuth callback web server (127.0.0.1:8787, behind nginx) --> database, visibility sync

Gateway and shards

PolarBot asks Discord for the recommended number of shards and runs one websocket connection for each. Each shard loop only receives events, updates the in-memory cache and starts handler tasks: it never waits on a handler, because it also drives the gateway heartbeat. When all handler slots of a kind are busy, the event is dropped (with a log line) instead of waiting. There are three separate pools, so one kind of load can never starve another:

PoolSizeGets
Interactions64Slash commands, message commands, card buttons, autocomplete
Handlers256New and edited messages (the scanner), server/channel/role deletes
Log handlers64Every other event a server logs (presence, typing, member updates, …)

If Discord closes a connection with a code that allows reconnecting, the shard reconnects by itself. A fatal close code (for example 4014: privileged intents not enabled, or 4004: bad token) stops the bot with exit status 78, which the systemd unit does not restart (see exit codes).

The cache

The in-memory cache holds the servers, channels, roles, members and users the bot can see, and the last 200 messages of each channel. It is used for permission and hierarchy checks and for the before/after of edits and deletes in the log. Presences are not cached. Memory grows with the size of the servers, not with time.

Handlers

  • Interactions go to commands::dispatch. In a server it first defers the reply (Discord drops interactions not answered within 3 seconds), loads the server’s settings, runs the matching command and fills in the deferred reply. Replies are ephemeral (only the caller sees them) and never ping anyone. In a DM only /help, /about and /privacy run.
  • Messages (new and edited) go to the scanner. An update that did not change the text (a link preview, a pin) is ignored.
  • Bot removed from a server (a GUILD_DELETE that is not an outage): every row of that server is deleted from the database and its caches are dropped.
  • Log channel deleted: the server’s log channel setting is cleared.
  • Role deleted: if it had a /perms level, the mapping is removed and command visibility is synced again.

Permission levels

PolarBot has its own four levels, checked by the bot on every command and button press, whatever Discord shows:

LevelWhoCan use
UserEveryone/help, /about, /privacy
ModeratorRoles given Moderator with /perms setModeration commands, /lookup, /scanner analyze, Analyze with PolarBot, card buttons. The scanner skips Moderators and above.
AdminRoles given AdminAlso /setup, /config, the /scanner settings, /perms list
OwnerThe server owner and every member with Discord’s Administrator permissionAlso /perms set, /perms remove, /perms authorize

A member gets the highest level of their roles. The exact level of every command and subcommand is in Commands. Moderators can only act on members below them both in Discord’s role order and in PolarBot’s levels (the server owner may act on anyone), and PolarBot’s own role must be above the target’s.

Command visibility

Every command except /about, /help and /privacy is registered with default_member_permissions = 0, so Discord shows it only to Administrators. After an Owner runs /perms authorize and approves the OAuth link, PolarBot stores an encrypted refresh token for that server and keeps Discord’s per-command permission overrides in sync with /perms: each role sees exactly the commands its level allows. A sync runs after every /perms change and role deletion and at startup, at most once per 10 seconds per server and two servers at a time, retried three times (after 10, 20 and 40 s) on failure. If the authorization is revoked, PolarBot deletes it and the next /perms reply asks for /perms authorize again; Discord keeps the last overrides, and the bot’s own level check applies either way. See the security model for the OAuth details.

The web server

Only when both OAuth credentials are present, PolarBot serves exactly two routes on [web] bind (default 127.0.0.1:8787):

  • GET /oauth/callback?code&state: the end of /perms authorize;
  • GET /healthz: answers ok.

Everything else is a 404. Every response has a 10 s deadline and strict security headers. nginx terminates TLS in front of it (Install: OAuth and nginx). If the web server cannot bind, the bot keeps running without it.

Background jobs

JobEveryDoes
List updater (one for threat lists, one for adult lists)scanner.refresh_hours (24 h) per list; a failed download is retried after 1 hDownloads each list (https only, size cap, 60 s timeout), keeps it only if it parses to at least one domain, writes the cache file atomically and swaps the in-memory index without blocking the scanner
Scanner sweep1 minDrops expired behaviour-tracker and cache entries
Purge1 h (the first right at startup)Deletes punishment history older than 180 days and scanner counters older than 30 days
Card and log queuesPer server, while there is workPost cards and log batches (below)

Data flows

DataComes fromGoes toStored?
Message textGateway MESSAGE_CREATE / MESSAGE_UPDATEScanner (memory); the card, if flaggedNo
Flag decisionScannerDiscord (delete, timeout, kick, ban, with an audit-log reason); the card; countersPunishments in mod_actions (180 days); daily counters
Moderator verdict (card button)Discord interactionThe card is updated; risky domains go on the block list; an anonymized training examplescan_domains, training_examples
Moderation commandDiscord interactionDiscord API; log channelmod_actions / warnings (180 days)
Logged eventsGatewayLog channel, batchedNo
OAuth grantDiscord, through nginx, to the web serverEncrypted refresh tokenguild_oauth
Domain listsThe configured https URLsCache files and in-memory indexCache files in the data directory

Everything that is stored, and for how long, is in Privacy and data and Database.

Rate limiting and batching

PolarBot never posts to Discord from inside an event handler, where a burst could pile up:

  • Log events go into a per-server queue of at most 500 events (the oldest are dropped when it is full, and the next post says how many). One worker per active server posts every 2 seconds, or as soon as 10 embeds are waiting, packing them into as few messages as fit Discord’s limits, at most one post per 1.5 seconds. Events in the log channel itself and the bot’s own actions are dropped. An idle worker exits after 5 minutes.
  • Mod cards go into a per-server queue of at most 50 cards, posted at most once per second. A delete-only card waits 3 seconds; further delete-only flags of the same author within 10 seconds are merged into it (“and N more messages”), so a spam wave becomes one card. Punishment cards go out at once.
  • Repeat offenders: within 2 minutes of a punishment, further flags of the same member only delete the message: one ping per incident.
  • Update notices go out one server at a time, 1.5 seconds apart, starting 30 seconds after the first Ready.
  • On top of that, every Discord request goes through twilight’s rate limiter, which waits out Discord’s 429 responses.

Caches and freshness

WhatHow fresh
A server’s scanner settings and listsUpdated at once by /scanner; otherwise at most 60 s old
A server’s log level and log channelUpdated at once by /config and /setup; otherwise at most 60 s old
A member’s punishment count (a scanner signal)At most 5 minutes old; updated at once after a scanner punishment
Domain listsEvery refresh_hours