Add a local benchmark for /tldr with a dashboard
`go test -tags bench ./internal/tldr -run TestBench` runs the real /tldr pipeline against one or more Ollama models and prompt variants on a chat dump from disk (text copied from Discord, "Author: message" lines, or a DiscordChatExporter JSON export) and writes a self-contained HTML dashboard plus a JSON of the results. Scoring uses an optional facts file (a fact passes only when one bullet holds all its details) and automatic checks: numbers not in the chat, copying the chat, an English intro, cut-off output. TestFactsSkeleton proposes candidate facts from the chat to prune by hand. Chat dumps and results stay local: bench-out/ and the private folders are git-ignored. See internal/tldr/tldrbench/README.md. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
1 parent
7ae1fa5747
commit
17548bbf7d
10 files changed
+1495
No files matched your search
@@ -0,0 +1,8 @@
|
||||
[
|
||||
{"name": "Hosting 80 to 120", "all": ["120", "80"]},
|
||||
{"name": "Quote by Monday", "all": ["orçamento|fornecedor", "segunda"]},
|
||||
{"name": "Dinner Friday 20h", "all": ["jantar", "sexta", "20h"]},
|
||||
{"name": "Dinner place and size", "all": ["cantinho", "9"]},
|
||||
{"name": "Certificate renewed", "all": ["certificado|tls", "renov"]},
|
||||
{"name": "14-day alert", "all": ["14 dias", "alerta|grafana"]}
|
||||
]
|
||||
Reference in new issue
Block a user