Prepare for open source: license, contributing, self-hosting guide, fix Close() #29

Merged
FernandoJVideira merged 2 commits from ft/open-source into dev 2026-10-23 00:54:45 +00:00
11 changed files with 93 additions and 77 deletions
Showing only changes of commit 02b7db1813 - Show all commits

No files matched your search

+2 -2
View File
@@ -17,7 +17,7 @@ If you find a way to get around a permission check, make the bot act for someone
1. **Open an issue for anything big**: a new module, a change to how permissions or security work, a change to the database. It saves you from writing something that can't be merged. 1. **Open an issue for anything big**: a new module, a change to how permissions or security work, a change to the database. It saves you from writing something that can't be merged.
2. Read [Architecture](docs/architecture.md) so you know the layers, and the [Development guide](docs/development.md) for the recipe that matches your change. 2. Read [Architecture](docs/architecture.md) so you know the layers, and the [Development guide](docs/development.md) for the recipe that matches your change.
3. For anything that touches the database, read [Database](docs/database.md#changing-the-schema). Tables are shared with the dashboard, so a change has to be compatible with an older bot for a short while. 3. For anything that touches the database, read [Database](docs/database.md#changing-the-schema). A schema change has to keep a bot that is one version behind working for a short while.
## Making the change ## Making the change
@@ -45,7 +45,7 @@ These are the things a reviewer will check first. The reasons are in the [Develo
- **No SQL outside repos.** Always use query parameters (`$1`), never string formatting. - **No SQL outside repos.** Always use query parameters (`$1`), never string formatting.
- **A module that acts on a server starts off.** An admin turns it on. - **A module that acts on a server starts off.** An admin turns it on.
- **Escape user text and set `AllowedMentions`** so a message can't ping people. - **Escape user text and set `AllowedMentions`** so a message can't ping people.
- **Validate everything that comes from users or from the dashboard**: options, custom IDs, command payloads. - **Validate everything that comes from users or from other programs writing to the database**: options, custom IDs, command payloads.
- **No secrets** in code, logs, tests or commit messages. - **No secrets** in code, logs, tests or commit messages.
## Commit messages and pull requests ## Commit messages and pull requests
+5 -8
View File
@@ -2,12 +2,9 @@
Velox is a Discord bot written in Go, backed by PostgreSQL. It covers server security (anti-raid and anti-nuke), support tickets, moderation, leveling, music, meeting rooms, scheduling, Twitch alerts and a few games. Every module is off until a server turns it on. Velox is a Discord bot written in Go, backed by PostgreSQL. It covers server security (anti-raid and anti-nuke), support tickets, moderation, leveling, music, meeting rooms, scheduling, Twitch alerts and a few games. Every module is off until a server turns it on.
This repository is the bot. It has two companions that live in their own repositories: **This repository is the bot, and it is the only open source part of Velox.** The web dashboard and its API, which the hosted Velox uses, are separate proprietary software and are not part of this repository or its license. The bot does not need them: everything in the [command reference](docs/commands.md) works on its own, with PostgreSQL as its only dependency.
- **velox-dashboard-api**: a Go REST API behind the web dashboard. It owns most of the database schema and runs the migrations. The dashboard and the bot meet in the database. The dashboard writes settings and requests into tables, and the bot reads them (and is woken through Postgres `NOTIFY` where it has to react at once). Those tables are documented, so you can change the same settings with SQL, or build your own tool on top. See [Architecture](docs/architecture.md) and [Database](docs/database.md).
- **velox-dashboard-web**: the Vue dashboard and public website.
The bot and the API share one PostgreSQL database. They never call each other over HTTP. The dashboard writes settings and requests into tables, and the bot reads them (and is woken through Postgres `NOTIFY` where it has to react at once). See [Architecture](docs/architecture.md).
## Documentation ## Documentation
@@ -18,7 +15,7 @@ This README is the front page. The rest of the documentation lives in [`docs/`](
| [Getting started](docs/getting-started.md) | Requirements, creating the Discord application, configuration, database, running locally | | [Getting started](docs/getting-started.md) | Requirements, creating the Discord application, configuration, database, running locally |
| [Host your own bot](docs/self-hosting.md) | A step-by-step guide to running your own copy on a server, with Docker Compose | | [Host your own bot](docs/self-hosting.md) | A step-by-step guide to running your own copy on a server, with Docker Compose |
| [Deployment](docs/deployment.md) | Docker, Compose, upgrade order, backups | | [Deployment](docs/deployment.md) | Docker, Compose, upgrade order, backups |
| [Architecture](docs/architecture.md) | How the code is laid out, how an interaction flows through it, background loops, the link to the dashboard | | [Architecture](docs/architecture.md) | How the code is laid out, how an interaction flows through it, background loops, how a dashboard talks to the bot |
| [Features](docs/features.md) | What each module does and how it behaves | | [Features](docs/features.md) | What each module does and how it behaves |
| [Command reference](docs/commands.md) | Every slash command, its options and who may run it | | [Command reference](docs/commands.md) | Every slash command, its options and who may run it |
| [Database](docs/database.md) | Who owns the schema, the tables, the `NOTIFY` channels, how to change them | | [Database](docs/database.md) | Who owns the schema, the tables, the `NOTIFY` channels, how to change them |
@@ -47,7 +44,7 @@ On start the bot registers its slash commands in the server named by `GUILD_ID`
- **Tickets**: a panel with categories, a private thread per ticket, claim and close, transcripts to a log channel. - **Tickets**: a panel with categories, a private thread per ticket, claim and close, transcripts to a log channel.
- **Moderation**: purge, timeout, kick, ban, unban, slowmode, plus server logs for edits, deletes, joins, leaves, role changes, bans and mod actions. - **Moderation**: purge, timeout, kick, ban, unban, slowmode, plus server logs for edits, deletes, joins, leaves, role changes, bans and mod actions.
- **Leveling**: XP per message, level cards, leaderboard, role rewards. - **Leveling**: XP per message, level cards, leaderboard, role rewards.
- **Music**: Lavalink-powered playback with a queue, controlled from Discord or the dashboard. - **Music**: Lavalink-powered playback with a queue, controlled from Discord.
- **Meeting rooms**: join a lobby voice channel and get a temporary room you can lock and invite people to. - **Meeting rooms**: join a lobby voice channel and get a temporary room you can lock and invite people to.
- **Scheduling and projects**: 1:1 sessions with DM invites and reminders, and a simple project list. - **Scheduling and projects**: 1:1 sessions with DM invites and reminders, and a simple project list.
- **Twitch alerts, welcome messages, default role**: per-server setup through `/config`. - **Twitch alerts, welcome messages, default role**: per-server setup through `/config`.
@@ -87,4 +84,4 @@ Contributions are welcome: bug reports, fixes, documentation and new features.
## License ## License
Velox Bot is released under the [Apache License 2.0](LICENSE). Copyright 2026 Fernando Videira. See [NOTICE](NOTICE). The libraries it uses are under their own permissive licenses (see `go.mod`). Velox Bot, the code in this repository, is released under the [Apache License 2.0](LICENSE). Copyright 2026 Fernando Videira. See [NOTICE](NOTICE). The license does not cover the proprietary Velox dashboard and API, or the Velox name and logo as used by the hosted service. The libraries the bot uses are under their own permissive licenses (see `go.mod`).
+6 -6
View File
@@ -19,7 +19,7 @@ This page explains how the bot is put together, so you know where to look and wh
| | | | | |
+----------- services (rules) <------ repos (SQL) -----+ +----------- services (rules) <------ repos (SQL) -----+
| |
PostgreSQL <----> velox-dashboard-api <----> dashboard PostgreSQL <----> a dashboard (the hosted Velox has a proprietary one)
``` ```
The bot is one Go process. It keeps one websocket connection to Discord and one connection pool to PostgreSQL. It has no HTTP server of its own. The bot is one Go process. It keeps one websocket connection to Discord and one connection pool to PostgreSQL. It has no HTTP server of its own.
@@ -85,14 +85,14 @@ Some commands have a response that takes longer than three seconds (anything tha
| Raid watch | `security` | 30 s | Ends raid mode after a quiet period | | Raid watch | `security` | 30 s | Ends raid mode after a quiet period |
| Snapshots | `security` | 6 h | Saves roles and channels while anti-nuke is on (keeps three) | | Snapshots | `security` | 6 h | Saves roles and channels while anti-nuke is on (keeps three) |
| Beta settings | `bot/bot.go` (`WatchBeta`) | 30 s | Re-reads who the beta testers are and which servers opted in, and adds or removes beta commands to match | | Beta settings | `bot/bot.go` (`WatchBeta`) | 30 s | Re-reads who the beta testers are and which servers opted in, and adds or removes beta commands to match |
| Command listeners | music, tickets, security | on `NOTIFY` | Carry out dashboard requests | | Command listeners | music, tickets, security | on `NOTIFY` | Carry out requests written to the `*_commands` tables |
## The link to the dashboard ## How a dashboard talks to the bot
The bot and the dashboard API do not talk to each other directly. They meet in PostgreSQL, in two ways: The bot has no HTTP server and no API of its own. A web dashboard, such as the proprietary one the hosted Velox uses (not part of this repository), meets the bot in PostgreSQL, in two ways. This is the whole contract, so you can drive the bot the same way with SQL or write your own tool:
1. **Settings tables.** The dashboard writes settings (log channel, security thresholds, ticket categories, beta role). The bot reads them when it needs them. Hot paths cache them for a short time, so a raid of thousands of joins does not become thousands of queries. 1. **Settings tables.** A dashboard writes settings (log channel, security thresholds, ticket categories, beta role). The bot reads them when it needs them. Hot paths cache them for a short time, so a raid of thousands of joins does not become thousands of queries.
2. **Command tables with `NOTIFY`.** When the dashboard needs the bot to *do* something (skip a song, post the ticket panel, release a quarantined member), it inserts a row in `music_commands`, `ticket_commands` or `security_commands` and issues `NOTIFY` on that channel. The bot has a long-lived connection that runs `LISTEN`, picks up the row, validates it again and carries it out. Never trust a row just because it is in the table: the listeners re-check permissions themselves. 2. **Command tables with `NOTIFY`.** When a dashboard needs the bot to *do* something (skip a song, post the ticket panel, release a quarantined member), it inserts a row in `music_commands`, `ticket_commands` or `security_commands` and issues `NOTIFY` on that channel. The bot has a long-lived connection that runs `LISTEN`, picks up the row, validates it again and carries it out. Never trust a row just because it is in the table: the listeners re-check permissions themselves.
Both directions are described table by table in [Database](database.md). Both directions are described table by table in [Database](database.md).
+11 -15
View File
@@ -5,28 +5,27 @@ A beta feature is one that is still being tried out, so only chosen people can u
1. **The person is a beta tester**: a member of the beta server who holds the Beta Tester role. 1. **The person is a beta tester**: a member of the beta server who holds the Beta Tester role.
2. **Beta features are on in the server they use it in**: the beta server itself, or any other server whose owner opted in. 2. **Beta features are on in the server they use it in**: the beta server itself, or any other server whose owner opted in.
The same rule applies on the bot and on the dashboard. The rule is the same on the bot and on the hosted Velox dashboard. The dashboard is proprietary and not part of this repository, so this page is about the bot, and mentions the dashboard only where it explains how the settings get into the database.
## Who the testers are ## Who the testers are
- The **beta server** and the **Beta Tester role** are chosen by the bot's owner on the dashboard, under **Admin, Beta access**. Only the owner can see or change that page. - The **beta server** and the **Beta Tester role** live in the `beta_settings` table, one row. On the hosted Velox the owner sets them on the dashboard. If you host the bot yourself, set them with SQL (see [Settings without a dashboard](self-hosting.md#settings-without-a-dashboard)).
- Giving someone the role in Discord makes them a tester. Taking it away removes their access: within about 30 seconds on the bot (a few seconds if the bot has them cached), and within about 30 seconds on the dashboard API. - Giving someone the role in Discord makes them a tester. Taking it away removes their access, within about 30 seconds on the bot (a few seconds if the bot has them cached).
- Until a role is chosen, **nobody** has beta access. - Until a role is chosen, **nobody** has beta access.
- Being a tester is about the person, so it is always checked in the **beta server**, also when they use a feature somewhere else. Having a role with the same ID in another server means nothing. - Being a tester is about the person, so it is always checked in the **beta server**, also when they use a feature somewhere else. Having a role with the same ID in another server means nothing.
## Which servers have beta features ## Which servers have beta features
- The beta server always has them. - The beta server always has them.
- Any other server gets them only when **its owner** switches on **Beta features** on that server's page of the dashboard. Beta features can act on a whole server (anti-raid behaviour, for example), so the people who answer for the server decide. A server's admins who are not its owner cannot switch it on. - Any other server gets them only when it is in the `beta_guilds` table. Beta features can act on a whole server (anti-raid behaviour, for example), so a server should only be listed with its owner's agreement. On the hosted Velox, only the server's own owner can switch it on, from the dashboard. If you host the bot yourself, you add rows yourself.
- The bot's owner can take a server out again from **Admin, Beta access**, but cannot put one in. - The bot polls the table every 30 seconds, together with the beta settings.
- The list of opted-in servers is the `beta_guilds` table. The bot polls it every 30 seconds, together with the beta settings.
Beta never replaces a feature's own permission checks. A beta command that needs Manage Server still needs it, a music command still needs the DJ role, and anything for the owner or trusted users still needs that. Beta never replaces a feature's own permission checks. A beta command that needs Manage Server still needs it, a music command still needs the DJ role, and anything for the owner or trusted users still needs that.
## What a tester sees ## What a tester sees
- **Bot:** beta commands appear only in the beta server and in opted-in servers, and run only for testers. When a server opts in, the commands show up within about 30 seconds. When it opts out, they are removed. - **Bot:** beta commands appear only in the beta server and in opted-in servers, and run only for testers. When a server opts in, the commands show up within about 30 seconds. When it opts out, they are removed.
- **Dashboard:** beta pages and links show a **Beta** badge, and only to testers, and only for servers where beta features are on. - **On the hosted Velox dashboard:** beta pages and links show a **Beta** badge, only to testers and only for servers where beta features are on.
## On the bot ## On the bot
@@ -35,7 +34,7 @@ Code: `internal/beta` (the rules), `internal/commands/commands.go` (`RegisterBet
- Beta commands are registered in each server where beta features are on, and only there. `/beta` is only in the beta server. - Beta commands are registered in each server where beta features are on, and only there. `/beta` is only in the beta server.
- Each beta handler is wrapped in `beta.Gate`. It checks that beta is on in the server of the interaction and that the person is a tester, and otherwise answers "This feature is in beta and isn't available to you or this server yet." (Autocomplete requests get an empty list.) Hiding the command is never the only protection. - Each beta handler is wrapped in `beta.Gate`. It checks that beta is on in the server of the interaction and that the person is a tester, and otherwise answers "This feature is in beta and isn't available to you or this server yet." (Autocomplete requests get an empty list.) Hiding the command is never the only protection.
- For a person using a command in another server, the bot looks up their roles in the beta server: first in its gateway cache, otherwise it asks Discord (remembered for 30 seconds). If anything fails, the answer is no. - For a person using a command in another server, the bot looks up their roles in the beta server: first in its gateway cache, otherwise it asks Discord (remembered for 30 seconds). If anything fails, the answer is no.
- If the owner changes the beta server on the dashboard, the beta commands move within 30 seconds. In the beta server, commands this bot left there that no longer exist are removed too. If the beta server is also `GUILD_ID`, the regular commands are kept. - If the beta server changes in `beta_settings`, the beta commands move within 30 seconds. In the beta server, commands this bot left there that no longer exist are removed too. If the beta server is also `GUILD_ID`, the regular commands are kept.
- `/beta` shows the bot's owner how access is set up. Only the owner of the Discord application can run it, and it is hidden from everyone but administrators. - `/beta` shows the bot's owner how access is set up. Only the owner of the Discord application can run it, and it is hidden from everyone but administrators.
- The bot remembers the servers where it registered beta commands (`beta_registered` table). A server that opted out while the bot was **offline** is cleaned up on the next start, and a cleanup that fails is retried every 30 seconds. A server the bot has left is forgotten. - The bot remembers the servers where it registered beta commands (`beta_registered` table). A server that opted out while the bot was **offline** is cleaned up on the next start, and a cleanup that fails is retried every 30 seconds. A server the bot has left is forgotten.
@@ -67,17 +66,14 @@ Do not use this for anything that should only be for testers as people. It means
Move the definition into `AllCommands` and the handler into the `handlers` map, and delete the `RegisterBeta` call. On the next start the command appears everywhere, and the beta servers drop it from their own lists because it is now a regular command. Move the definition into `AllCommands` and the handler into the `handlers` map, and delete the `RegisterBeta` call. On the next start the command appears everywhere, and the beta servers drop it from their own lists because it is now a regular command.
## On the API and dashboard ## On the dashboard
These live in the other two repositories. The hosted Velox has a proprietary dashboard and API that apply the same rule to their own pages: a beta page is only shown, and its API routes only answer, for a tester in a server where beta features are on. Their code is not part of this repository. If you build your own tool on the same tables, apply the rule on your server side too: never rely on hiding a page.
- **API**: put the feature's routes under `/api/v1/beta` (not tied to a server), or add the `betaGuildOnly()` middleware to a route under `/api/v1/guilds/:guild_id/...` (tied to a server). The first asks Discord whether the signed-in user holds the role in the beta server (30-second memory); the second also checks that the server is the beta server or opted in. Both answer `403` otherwise. `GET /users/me` returns `is_beta`, and `GET /guilds/:guild_id/beta` tells the dashboard whether beta features are on for that server and whether the caller owns it.
- **Dashboard**: give a route `meta: { requiresBeta: true }` to keep its page away from non-testers, and mark navigation entries with `beta: true` so testers see them with a **Beta** badge (and nobody else sees them). This is only about display. The API checks access on every request, so a page that was shown by mistake still cannot read or change anything.
## Why it works this way ## Why it works this way
- The role in Discord is the single list of testers, so there is no second list to keep in step with it. - The role in Discord is the single list of testers, so there is no second list to keep in step with it.
- Servers choose to take part, because a beta feature can affect everyone in them. - Servers choose to take part, because a beta feature can affect everyone in them.
- Access is decided by the server (bot and API), never by the browser. - Access is decided on the server, never in the browser.
- Beta commands do not exist outside the beta server, and the gate checks again when they run. - Beta commands do not exist outside the beta server, and the gate checks again when they run.
- The settings are in the database, not in the environment, so the owner can change them without a deploy. - The settings are in the database, not in the environment, so they can change without a deploy.
+24 -18
View File
@@ -4,19 +4,26 @@ The bot uses PostgreSQL through `database/sql` with the pgx driver. All SQL live
## Who owns the schema ## Who owns the schema
The bot and the dashboard API share one database, so two files describe it: **`schema.sql`** in this repository is the schema. It describes every table the bot reads and writes, and it is enough to run the bot. It is safe to run more than once (`CREATE TABLE IF NOT EXISTS`). The bot never runs migrations itself.
- **`schema.sql`** in this repository: the tables the bot reads and writes. It is safe to run more than once (`CREATE TABLE IF NOT EXISTS`). It is enough to run the bot **on its own**. The hosted Velox also has a proprietary dashboard API that keeps its own migrations for the same tables. That is the maintainer's concern: if your change adds or alters a table, update `schema.sql` and say so in your pull request, and the maintainer ports it. You never need the API to develop or run the bot.
- **The API's migrations** (`velox-dashboard-api/cmd/db/migrations`): when the API is deployed, it creates and changes the tables, and runs its migrations on start. The bot never runs migrations.
Some tables are older and were created by the bot (levels, welcome, and so on). The API's migration `000005_bot_owned_tables` creates them for a fresh combined database. Newer modules (music, tickets, security, meeting lobbies, beta settings) were created by API migrations, and `schema.sql` carries a copy so the bot can run alone. Two things are not in `schema.sql`, because only a dashboard needs them: the `NOTIFY` triggers on the three `*_commands` tables (see below), and a migration history. If you write a tool that inserts command rows, add the trigger yourself:
**The rule:** a table must have the same shape in `schema.sql` and in the API migrations. When you change one, change the other in the same release. ```sql
CREATE OR REPLACE FUNCTION notify_music_command() RETURNS TRIGGER AS $$
BEGIN
PERFORM pg_notify('music_commands', NEW.id::text);
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
Two things only exist in the API's migrations, because the bot never inserts into those tables: CREATE TRIGGER music_commands_notify
AFTER INSERT ON music_commands
FOR EACH ROW EXECUTE FUNCTION notify_music_command();
```
- the `NOTIFY` triggers on the three `*_commands` tables (see below), The same works for `ticket_commands` and `security_commands`. Without the trigger the bot still picks the rows up, but only the next time it starts or is notified for another reason.
- the migration history.
## Tables ## Tables
@@ -30,7 +37,7 @@ Two things only exist in the API's migrations, because the bot never inserts int
| Meeting rooms | `meeting_lobbies`, `meeting_temp_channels`, `meeting_settings` | Lobbies (10 per server, enforced by a trigger), the temporary rooms and who created them. `meeting_settings.lobby_channel_id` is the old single lobby column and is no longer read | | Meeting rooms | `meeting_lobbies`, `meeting_temp_channels`, `meeting_settings` | Lobbies (10 per server, enforced by a trigger), the temporary rooms and who created them. `meeting_settings.lobby_channel_id` is the old single lobby column and is no longer read |
| Projects | `projects`, `project_members` | | | Projects | `projects`, `project_members` | |
| Scheduling | `schedules`, `schedule_messages`, `user_settings` | Sessions, the DM messages to update, and each member's timezone | | Scheduling | `schedules`, `schedule_messages`, `user_settings` | Sessions, the DM messages to update, and each member's timezone |
| Music | `music_settings`, `music_now_playing`, `music_queue`, `music_commands` | State for the dashboard's player, plus requests from the dashboard | | Music | `music_settings`, `music_now_playing`, `music_queue`, `music_commands` | State a dashboard can show, plus requests to the bot |
| Tickets | `ticket_settings`, `ticket_categories`, `tickets`, `ticket_commands` | | | Tickets | `ticket_settings`, `ticket_categories`, `tickets`, `ticket_commands` | |
| Security | `security_settings`, `security_trusted`, `security_quarantines`, `security_lockdowns`, `security_raids`, `security_raid_caught`, `security_snapshots`, `security_events`, `security_commands` | Settings, who is trusted, quarantines (with the roles to give back), locked channels (with the permission to restore), raids in progress, snapshots, an event history, and dashboard requests | | Security | `security_settings`, `security_trusted`, `security_quarantines`, `security_lockdowns`, `security_raids`, `security_raid_caught`, `security_snapshots`, `security_events`, `security_commands` | Settings, who is trusted, quarantines (with the roles to give back), locked channels (with the permission to restore), raids in progress, snapshots, an event history, and dashboard requests |
| Beta | `beta_settings`, `beta_guilds` | `beta_settings` is one row: the beta server and the Beta Tester role. `beta_guilds` lists servers whose owner opted in to beta features. Both are written by the dashboard and read by the bot. `beta_registered` is the bot's own note of where it registered beta commands | | Beta | `beta_settings`, `beta_guilds` | `beta_settings` is one row: the beta server and the Beta Tester role. `beta_guilds` lists servers whose owner opted in to beta features. Both are written by the dashboard and read by the bot. `beta_registered` is the bot's own note of where it registered beta commands |
@@ -39,27 +46,26 @@ Discord IDs are `BIGINT` (snowflakes fit in 64 bits). In Go they are `int64`; Di
## Dashboard to bot: settings and requests ## Dashboard to bot: settings and requests
There are two kinds of traffic from the dashboard, and the difference matters. There are two kinds of traffic from a dashboard to the bot, and the difference matters.
**Settings.** The dashboard writes a row, and the bot reads it when it next needs it. Nothing wakes the bot up. Hot paths cache settings for a short time. The bot's beta settings are polled every 30 seconds. **Settings.** A dashboard writes a row, and the bot reads it when it next needs it. Nothing wakes the bot up. Hot paths cache settings for a short time. The bot's beta settings are polled every 30 seconds.
**Requests that need the bot to act.** `music_commands`, `ticket_commands` and `security_commands` work as a small queue: **Requests that need the bot to act.** `music_commands`, `ticket_commands` and `security_commands` work as a small queue:
1. The dashboard API inserts a row (`command_type`, a JSON `payload`, `requested_by`, `processed_at` empty). 1. A dashboard inserts a row (`command_type`, a JSON `payload`, `requested_by`, `processed_at` empty).
2. A trigger in the database runs `pg_notify('<table>', id)`. 2. A trigger in the database runs `pg_notify('<table>', id)`.
3. The bot holds one connection that ran `LISTEN <table>`. It wakes, loads **every** unprocessed row, carries each out, and sets `processed_at`. 3. The bot holds one connection that ran `LISTEN <table>`. It wakes, loads **every** unprocessed row, carries each out, and sets `processed_at`.
Because the bot loads all unprocessed rows (also when it starts), nothing is lost if it was down when a request arrived. A command that fails is logged and marked processed anyway, since retrying would fail the same way. Because the bot loads all unprocessed rows (also when it starts), nothing is lost if it was down when a request arrived. A command that fails is logged and marked processed anyway, since retrying would fail the same way.
Every listener **validates the request again**. The API checks that the requester is allowed, and the bot checks too, because it should not trust a row just because it exists. The listeners live in `internal/music/listener.go`, `internal/tickets/listener.go` and `internal/security/listener.go`, and each reconnects after 10 seconds if its connection drops. Every listener **validates the request again**. The dashboard should check that the requester is allowed, and the bot checks too, because it should not trust a row just because it exists. The listeners live in `internal/music/listener.go`, `internal/tickets/listener.go` and `internal/security/listener.go`, and each reconnects after 10 seconds if its connection drops.
## Changing the schema ## Changing the schema
1. Decide who owns the table (see above). New tables for the dashboard's modules are created by an API migration. 1. Update `schema.sql`. Use `CREATE TABLE IF NOT EXISTS`, and avoid destructive changes: a bot that is a version behind (or a tool reading the same tables) must keep working against the new schema for a moment. Add a column or table first, and remove old ones in a later release.
2. Write the API migration, **and** update `schema.sql` to match. 2. Add or change the repo method in `internal/db/repos/<area>repo`.
3. Add or change the repo method in `internal/db/repos/<area>repo`. Use `IF NOT EXISTS` and avoid destructive changes: the bot and the dashboard may be deployed a few minutes apart, so the old bot must keep working against the new schema for a moment. Add a column or table first, and remove old ones in a later release. 3. If something outside the bot must make it do something, add a `command_type` to the `CHECK` in the right `*_commands` table and handle it in the listener.
4. If the dashboard must make the bot do something, add a `command_type` to the `CHECK` in the right `*_commands` table and handle it in the listener. 4. In your pull request, say that the schema changed and whether an existing table is affected, so the maintainer can port the change to the dashboard's migrations.
5. Say in the release notes if the API, bot and dashboard have to deploy together.
## Connection settings ## Connection settings
+2 -8
View File
@@ -27,15 +27,9 @@ Compose does not include Lavalink. Run it separately (or add a service) if you w
Change the default database password (`velox_pwd`) before exposing PostgreSQL to anything other than your own machine, and remove the published `5432` port if only the bot needs it. Change the default database password (`velox_pwd`) before exposing PostgreSQL to anything other than your own machine, and remove the published `5432` port if only the bot needs it.
## Running next to the dashboard ## Database changes
If you also run `velox-dashboard-api` and `velox-dashboard-web`: The bot does not run migrations. When a release adds tables, load `schema.sql` again before restarting the bot (it only creates what is missing, and is safe to repeat). A release that changes an existing table says so in its notes.
1. They all use the **same database**. Give each the same PostgreSQL, and point `DB_HOST` (bot) and the API's `DB_ADDR` at it.
2. The API runs the migrations when it starts. The bot does not run migrations.
3. **Deploy order for changes that touch the database:** API first (it applies the migration), then the bot and the dashboard.
Some changes need the API, the bot and the dashboard to go out together. For example, when a table the dashboard writes changes shape, an old dashboard talking to a new API fails. The release notes for such a change say so.
## Updating ## Updating
+5 -5
View File
@@ -154,7 +154,7 @@ Never trust the contents of a custom ID. A user cannot edit it, but old messages
The usual path is table, repo, service, command. The meeting-room lobbies are a complete small example (`/meeting add-lobby`). The usual path is table, repo, service, command. The meeting-room lobbies are a complete small example (`/meeting add-lobby`).
**1. The table.** Add it to `schema.sql`, and to an API migration if the dashboard will show it (see [Database](database.md#changing-the-schema)). **1. The table.** Add it to `schema.sql` (see [Database](database.md#changing-the-schema)).
**2. The repo**, in `internal/db/repos/<area>repo/repo.go`. SQL only, returning plain Go values: **2. The repo**, in `internal/db/repos/<area>repo/repo.go`. SQL only, returning plain Go values:
@@ -188,7 +188,7 @@ if services.Global == nil || services.Global.Meeting == nil {
added, err := services.Global.Meeting.AddLobby(ctx, guildID, channelID) added, err := services.Global.Meeting.AddLobby(ctx, guildID, channelID)
``` ```
**6. Enforce limits in the database when you can.** The 10-lobby limit is a trigger, so the bot and the dashboard cannot disagree and two simultaneous requests cannot both slip through. **6. Enforce limits in the database when you can.** The 10-lobby limit is a trigger, so the bot and anything else writing to the table cannot disagree and two simultaneous requests cannot both slip through.
**7. Default to off.** A module that does something to a server without being asked (posts messages, moderates, grants roles) must start disabled and be turned on by an admin. **7. Default to off.** A module that does something to a server without being asked (posts messages, moderates, grants roles) must start disabled and be turned on by an admin.
@@ -244,7 +244,7 @@ Call it at the end of `bot.Start()`. Log errors instead of returning them, and m
Never log a secret, and never put one in a message to a user. Never log a secret, and never put one in a message to a user.
Use an environment variable for deployment settings (tokens, hosts). Settings an admin or owner should change at runtime belong in the database and on the dashboard instead. That is how the beta settings work. Use an environment variable for deployment settings (tokens, hosts). Settings an admin or owner should change at runtime belong in the database instead (set with a command, or by a dashboard). That is how the beta settings work.
## Make a command beta ## Make a command beta
@@ -281,7 +281,7 @@ Good things to test: permission checks (does the wrong person get refused?), lim
- **One feature per package, SQL only in repos, rules in services.** Commands never contain SQL. - **One feature per package, SQL only in repos, rules in services.** Commands never contain SQL.
- **Match the code around you**: naming, comment density, error handling. Comments explain *why*, not *what*. - **Match the code around you**: naming, comment density, error handling. Comments explain *why*, not *what*.
- **Fail closed.** If you cannot tell whether someone is allowed, they are not. - **Fail closed.** If you cannot tell whether someone is allowed, they are not.
- **Validate everything that comes from a user or the dashboard**: option values, custom IDs, `payload` JSON in command rows. - **Validate everything that comes from a user or from another program writing to the database**: option values, custom IDs, `payload` JSON in command rows.
- **Escape user text** in messages, and set `AllowedMentions` so it cannot ping. - **Escape user text** in messages, and set `AllowedMentions` so it cannot ping.
- **Bound everything per server**: caches, lists, loops. One server must not be able to use unlimited memory. - **Bound everything per server**: caches, lists, loops. One server must not be able to use unlimited memory.
- **Do not leak internals** to users: reply with a short message, log the detail. - **Do not leak internals** to users: reply with a short message, log the detail.
@@ -292,7 +292,7 @@ Good things to test: permission checks (does the wrong person get refused?), lim
- [ ] `gofmt` on the files you touched, `go build ./...`, `go vet ./...`, `go test -race ./...` - [ ] `gofmt` on the files you touched, `go build ./...`, `go vet ./...`, `go test -race ./...`
- [ ] Commands: in `AllCommands` and `handlers`, permission check in the handler, ephemeral errors - [ ] Commands: in `AllCommands` and `handlers`, permission check in the handler, ephemeral errors
- [ ] New table: in `schema.sql` **and** in an API migration, with a note about deploy order - [ ] New table or column: in `schema.sql`, and mentioned in the pull request
- [ ] Module that acts on a server: off by default - [ ] Module that acts on a server: off by default
- [ ] `/help`, [Command reference](commands.md) and [Features](features.md) updated - [ ] `/help`, [Command reference](commands.md) and [Features](features.md) updated
- [ ] No secrets, tokens or personal IDs in code, logs or tests - [ ] No secrets, tokens or personal IDs in code, logs or tests
+2 -2
View File
@@ -29,7 +29,7 @@ Rules that hold everywhere in this module:
- Roles that are configured (such as the quarantine role) are checked when they are used, not only when saved. A quarantine role that gained dangerous permissions is not assigned. - Roles that are configured (such as the quarantine role) are checked when they are used, not only when saved. A quarantine role that gained dangerous permissions is not assigned.
- Messages ping only the configured alert role, and names are escaped. - Messages ping only the configured alert role, and names are escaped.
Thresholds, Join Gate filters and heat settings are set on the dashboard. The bot's role must be above the roles it protects against: it can only quarantine members whose highest role is below its own. Thresholds, Join Gate filters and heat settings are rows in the `security_settings` table (the hosted Velox edits them on its dashboard; see [Settings without a dashboard](self-hosting.md#settings-without-a-dashboard)). The bot's role must be above the roles it protects against: it can only quarantine members whose highest role is below its own.
## Tickets ## Tickets
@@ -69,7 +69,7 @@ Lavalink-powered playback. Code: `internal/music`, commands in `internal/command
- `/play` takes a name or a URL, `/queue` shows the queue, `/volume` sets 0 to 150 percent. These need a role called **DJ**. - `/play` takes a name or a URL, `/queue` shows the queue, `/volume` sets 0 to 150 percent. These need a role called **DJ**.
- The now-playing message has buttons for playback and repeat. Buttons also check for the DJ role. - The now-playing message has buttons for playback and repeat. Buttons also check for the DJ role.
- `/music toggle` (Manage Server) turns music on or off in a server. - `/music toggle` (Manage Server) turns music on or off in a server.
- The dashboard shows the queue and can control playback. It does so through the `music_commands` table and a `NOTIFY` (see [Database](database.md)). - The queue and now-playing state are kept in the database, and playback can be controlled by writing to the `music_commands` table. The hosted Velox dashboard uses this (see [Database](database.md)).
- Music needs a Lavalink server. Without `LAVALINK_HOST` the rest of the bot works and music does not. - Music needs a Lavalink server. Without `LAVALINK_HOST` the rest of the bot works and music does not.
## Meeting rooms ## Meeting rooms
+2 -2
View File
@@ -57,7 +57,7 @@ TWITCH_CLIENT_ID=unused
`.env` is ignored by git. Never commit it. `.env` is ignored by git. Never commit it.
Who the beta testers are is **not** an environment variable. It is set on the dashboard and stored in the database. See [Beta features](beta-features.md). Who the beta testers are is **not** an environment variable. It is stored in the database. See [Beta features](beta-features.md).
## 3. Set up the database ## 3. Set up the database
@@ -73,7 +73,7 @@ Then load the schema:
psql "postgres://velox:velox_pwd@localhost:5432/velox?sslmode=disable" -f schema.sql psql "postgres://velox:velox_pwd@localhost:5432/velox?sslmode=disable" -f schema.sql
``` ```
`schema.sql` is written with `CREATE TABLE IF NOT EXISTS`, so running it again is safe. It is a mirror of the tables the bot uses. If you run the dashboard API as well, the API's migrations create and change these tables, and you do not need `schema.sql` at all. Read [Database](database.md) before changing a table. `schema.sql` is written with `CREATE TABLE IF NOT EXISTS`, so running it again is safe. It is the schema of every table the bot uses. Read [Database](database.md) before changing a table.
## 4. Run ## 4. Run
+30 -7
View File
@@ -8,7 +8,7 @@ You will run:
- **PostgreSQL**, started by the same Compose file. - **PostgreSQL**, started by the same Compose file.
- Optionally **Lavalink**, if you want music. - Optionally **Lavalink**, if you want music.
The web dashboard lives in two other repositories (`velox-dashboard-api` and `velox-dashboard-web`) and is optional. See [What you get without the dashboard](#what-you-get-without-the-dashboard). The hosted Velox also has a web dashboard. It is proprietary and not part of this repository, so it is not something you can host. You do not need it. See [Settings without a dashboard](#settings-without-a-dashboard).
## Before you start ## Before you start
@@ -154,15 +154,38 @@ Run it on a schedule and keep copies off the server. To restore into an empty da
Discord requires you to **verify** your application once the bot is in 100 servers, and to be approved for the privileged intents (Server Members, Message Content). Plan for that if you intend to run a public bot. For a bot in a few servers you manage yourself, nothing is needed. Discord requires you to **verify** your application once the bot is in 100 servers, and to be approved for the privileged intents (Server Members, Message Content). Plan for that if you intend to run a public bot. For a bot in a few servers you manage yourself, nothing is needed.
## What you get without the dashboard ## Settings without a dashboard
The bot works on its own and every command in the [Command reference](commands.md) does. What needs the dashboard (the `velox-dashboard-api` and `velox-dashboard-web` repositories, which share this database): The bot works fully on its own, and every command in the [Command reference](commands.md) does. Most things are set with slash commands. A few settings that the hosted Velox sets on its dashboard have no command, so you set them in the database. They are plain rows, described in `schema.sql`.
- Security thresholds, Join Gate filters and heat settings. Without the dashboard the bot uses the defaults from the schema. **Security thresholds and filters.** The defaults are in `security_settings`. To change a server's settings, insert or update its row (the columns are documented in `schema.sql`, and invalid values are refused):
- Beta access: choosing the beta server and role, and servers opting in. Without the dashboard there is no way to set a role, so beta features stay off.
- The web view of the music queue and ticket panel, and the changelog.
If you do run the dashboard, it and the bot must use the **same PostgreSQL database**, and the API runs the migrations. Its own README explains its setup. When both are deployed, [deploy the API first](deployment.md#running-next-to-the-dashboard). ```sql
INSERT INTO security_settings (guild_id, raid_join_count, raid_join_seconds)
VALUES (123456789012345678, 15, 10)
ON CONFLICT (guild_id) DO UPDATE
SET raid_join_count = EXCLUDED.raid_join_count, raid_join_seconds = EXCLUDED.raid_join_seconds;
```
Protections are switched on with `/security toggle`, not with SQL.
**Beta features.** Beta features are off until you choose a beta server and a Beta Tester role. Run this once, with your own IDs (the bot reads it every 30 seconds):
```sql
INSERT INTO beta_settings (id, guild_id, role_id) VALUES (TRUE, 123456789012345678, 234567890123456789)
ON CONFLICT (id) DO UPDATE SET guild_id = EXCLUDED.guild_id, role_id = EXCLUDED.role_id;
```
To let another server's beta features on, add it (its owner's agreement is yours to arrange):
```sql
INSERT INTO beta_guilds (guild_id, enabled_by) VALUES (345678901234567890, 123456789012345678)
ON CONFLICT DO NOTHING;
```
See [Beta features](beta-features.md) for what this does.
**Queue and panel requests.** The `*_commands` tables let another program ask the bot to skip a song, post the ticket panel or end a raid. See [Database](database.md#dashboard-to-bot-settings-and-requests) if you want to build something on them.
## Making it your own ## Making it your own
+4 -4
View File
@@ -40,17 +40,17 @@ Most failures are about the bot's role:
They are off until enabled: `/slvl toggle`, `/config enablelogging` with `/config setlogchannel`, and `/config setwelcomechannel`. Leveling and spam detection read message text, so the **Message Content Intent** must be on. Welcome messages and security need the **Server Members Intent**. They are off until enabled: `/slvl toggle`, `/config enablelogging` with `/config setlogchannel`, and `/config setwelcomechannel`. Leveling and spam detection read message text, so the **Message Content Intent** must be on. Welcome messages and security need the **Server Members Intent**.
## Dashboard actions have no effect ## A dashboard or tool's requests have no effect
The dashboard talks to the bot through the database (see [Database](database.md#dashboard-to-bot-settings-and-requests)). A dashboard (the hosted Velox has a proprietary one) or any tool of yours talks to the bot through the database (see [Database](database.md#dashboard-to-bot-settings-and-requests)).
- Check that the bot and the API use the **same database**. - Check that the tool and the bot use the **same database**.
- For a request such as skipping a song or ending a raid, look in the matching `*_commands` table. A row with an empty `processed_at` is waiting. If many are waiting, the bot's listener is down: look for `command listener stopped` in its log. It reconnects every 10 seconds. - For a request such as skipping a song or ending a raid, look in the matching `*_commands` table. A row with an empty `processed_at` is waiting. If many are waiting, the bot's listener is down: look for `command listener stopped` in its log. It reconnects every 10 seconds.
- Settings are read when needed and may be cached briefly (the beta settings are polled every 30 seconds). - Settings are read when needed and may be cached briefly (the beta settings are polled every 30 seconds).
## Beta features are not available to a tester ## Beta features are not available to a tester
Go through [Beta features](beta-features.md): is the Beta Tester role chosen on the dashboard? Is the person in the **beta server**, and do they have the role there? Is the server where they use it the beta server, or has **its owner** switched on Beta features on the dashboard? Has 30 seconds passed? `/beta` (owner only) shows how access is set up. Go through [Beta features](beta-features.md): is the Beta Tester role set in `beta_settings`? Is the person in the **beta server**, and do they have the role there? Is the server where they use it the beta server, or is it listed in `beta_guilds`? Has 30 seconds passed? `/beta` (owner only) shows how access is set up.
## Getting more detail ## Getting more detail