Add /tldr: a private overview of what you missed in a channel

/tldr [since] summarizes the messages in the channel it is run in that the
caller has not been summarized yet, using a small local model on Ollama, and
replies ephemerally. The answer streams into the reply as it is written.

- Per-user, per-channel cursor (tldr_cursors) so each run picks up where the
  last left off; `since:` (e.g. 6h, 2d, max 7d) peeks without moving it.
  First use covers the last 24h, and a stale cursor never reaches back more
  than 7 days.
- Only summarizes channels both the caller and the bot can read.
- The transcript drops bots, system messages, reactions and filler, turns
  links, GIFs, images and files into short tags ([link: site], [image],
  [file: name], preview titles), marks replies, and writes a date marker
  where the chat paused for 3h or more so the model can keep topics apart.
- The prompt asks for one bullet per conversation, oldest first, with a
  worked example; it was tuned against a synthetic chat and a real export
  with qwen3:4b-instruct-2507 (the default model).
- One model call at a time across the bot, one /tldr per user.
- docker-compose: Ollama service (CPU only, 8g memory cap: 6g killed the
  runner mid-request), model pull on startup.

Needs the tldr_cursors migration (velox-dashboard-api 000016).

Co-Authored-By: Claude Opus 5.5 <[email protected]>

docker-compose.yml

@@ -26,11 +26,53 @@ services:
       DB_HOST: ${DB_HOST:-postgres://velox:velox_pwd@db:5432/velox?sslmode=disable}
       LAVALINK_HOST: ${LAVALINK_HOST:-ws://lavalink:2333}
       LAVALINK_PASSWORD: ${LAVALINK_PASSWORD:-youshallnotpass}
+      OLLAMA_HOST: ${OLLAMA_HOST:-http://ollama:11434}
+      TLDR_MODEL: ${TLDR_MODEL:-qwen3:4b-instruct-2507-q4_K_M}
+      TLDR_THREADS: ${TLDR_THREADS:-6}
     depends_on:
       db:
         condition: service_healthy
     networks:
       - default
 
+  # Local LLM behind /tldr. CPU only, so it's kept polite: low CPU priority
+  # (cpu_shares), a memory cap, and a thread cap the bot sends per request
+  # (TLDR_THREADS). Do NOT add a `cpus:` quota: Ollama would still start one
+  # busy-waiting thread per core, the quota throttles them, and inference
+  # drops to a fraction of a token per second.
+  ollama:
+    image: ollama/ollama:latest
+    container_name: velox-ollama
+    restart: unless-stopped
+    environment:
+      OLLAMA_KEEP_ALIVE: 5m # unload the model when idle to give the RAM back
+      OLLAMA_NUM_PARALLEL: "1"
+      OLLAMA_MAX_LOADED_MODELS: "1"
+      OLLAMA_CONTEXT_LENGTH: "8192"
+    volumes:
+      - ollama_models:/root/.ollama
+    # qwen3:4b measured 5.9 GB in use (model, 8k context cache, buffers and the
+    # file cache of the model), so 6g killed the runner mid-request. Leave room.
+    mem_limit: 8g
+    cpu_shares: 512
+    healthcheck:
+      test: ["CMD", "ollama", "list"]
+      interval: 15s
+      timeout: 5s
+      retries: 10
+
+  # One-shot: downloads the model into the volume, then exits.
+  ollama-init:
+    image: ollama/ollama:latest
+    container_name: velox-ollama-init
+    restart: "no"
+    depends_on:
+      ollama:
+        condition: service_healthy
+    environment:
+      OLLAMA_HOST: http://ollama:11434
+    entrypoint: ["/bin/sh", "-c", "ollama pull ${TLDR_MODEL:-qwen3:4b-instruct-2507-q4_K_M}"]
+
 volumes:
   velox_pgdata:
+  ollama_models: