Reading the images a page paints only works when the page is one scanned image.
A hotel voucher whose text is drawn as vector outlines -- 2559 constructPath
operations and no text-showing operator at all -- has exactly one image on it,
the 180x91 logo, so OCR returned "SMART TOURS" and nothing else. Neither pdf.js
nor PDFium finds any text in that file; the only way to read it is to rasterize
the page.
Rendering the page covers every case pulling the images out did, and several it
could not: text drawn as outlines, a scan split across image strips, a page
whose /Rotate the images themselves know nothing about, and stamps or annotations
drawn over a scan. It also bounds the work, since a page becomes a fixed number
of pixels no matter what resolution it was scanned at -- the old path would have
allocated a 140 MB RGBA buffer for a 600 dpi scan.
It costs nothing to do so. End to end, PDF bytes to a PNG Tesseract can read, the
two paths measured 175-240 ms and 186-256 ms on the sample scan; PNG encoding
dominates both. Recognition is unchanged: 94 confidence and 718 characters from
the embedded image, 93 and 716 from the render, same text. So the image path goes
away entirely rather than staying on as a fast case, and with it the channel
conversion, its unsupported-channel error path, the minimum image dimension and
the per-image loop.
PDFium ships as WebAssembly, which sidesteps what a native rasterizer would have
cost: @napi-rs/canvas is 30 MB per platform (20 MB of Skia statically linked
against libpng, libjpeg-turbo, libavif, harfbuzz and an SVG renderer, plus 11 MB
of ICU data the desktop build already carries a copy of), and it would need an
esbuild external plus a per-platform binary in copyNodeModules. The wasm is 4 MB,
identical on every target, and bundles normally -- main.mjs carries no reference
to it and it loads from a lazy chunk on the first page that needs rasterizing.
Its loader resolves the wasm against `import.meta.url`, which in the split ESM
bundle is a hash-named file under chunks/, so the bytes are read from
RESOURCE_DIR and handed to init() instead -- the same two paths core_assets.ts
uses for schema.sql, copied beside the server's assets by the build and resolved
through node_modules when running from source.
Pages are rendered in colour rather than PDFium's grayscale. Its conversion
flattens coloured text into its background: on the demo document it swallows the
whole "Organize your thoughts" panel, 60 characters that colour rendering keeps
and that no increase in scale recovers. Scale 2 is the knee of the quality curve
-- confidence 82 at scale 1, 93 at 1.5, 94 at 2, and flat above it while the
pixels to encode and read double again.
The voucher now yields 3284 characters at 0.93 confidence across both pages,
including the guest name, the hotel, the booking reference and the room type.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The desktop main process moved to ESM with code splitting a while ago; the
server stayed on a single 13.6 MB CJS bundle, which V8 parsed in full at every
boot. That also made the backend lazy-loading pattern a no-op here: a dynamic
import() only moved code around inside one file that was loaded anyway.
Build src/main.ts with { format: "esm" }, and rename the entry everywhere that
launches it: the two Docker entrypoint scripts (all five Dockerfiles go through
them), the trilium.sh written into the Linux tarball, start-prod-no-dir (also
the Playwright webServer), the sync e2e harness, and the Nix wrapper. The image
worker stays CJS -- the pool spawns it by its .cjs path.
Measured on a demo-sized document, old and new bundles built from the same
source and benchmarked interleaved with a pristine data dir per boot (n=6):
loaded at startup 13.56 MB (1 file) -> 6.64 MB (98 of 313 files)
spawn -> first response 439 ms -> 328 ms
heap used after GC 56.9 MB -> 32.4 MB
RSS improved too but was too noisy on the test machine to quote. The largest
chunks that no longer load at boot are pdfjs (1.61 MB), the Anthropic agent SDK
(1.35 MB), express-openid-connect, undici and the share theme.
flake.nix also pointed the desktop wrapper at main.cjs, left behind by the
desktop ESM migration; it would have failed to launch. Fixed alongside, with
the stale main.cjs mentions in nearby comments.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Until February 2023 the healthcheck was thirteen lines of wget. It was
rewritten in node so it could read the configuration, and the cost of
starting a runtime every interval came with it. That cost, not the
bundle that grew around it later, is what the CPU spike reports were
measuring: the bundle only arrived in 2025 and made an existing problem
about five times worse.
Node is no longer needed to read the configuration, because the server
records where it is listening once it has bound. It writes the address
from httpServer.address() rather than the one that was configured, so
the probe follows what actually happened, and the file existing at all
means the server reached the point of listening.
The probe is then ten lines of shell with nothing to resolve. Measured
against a real instance in all three listening modes:
http http://127.0.0.1:18081/... up 0, killed 1
https https://127.0.0.1:18443/... up 0, killed 1
unix socket url and socket path apart up 0
cpu ~30 ms -> 9.5 ms per probe
curl rather than wget: neither GNU nor busybox wget can reach a unix
socket, which Trilium supports and which would have broken silently.
All five images therefore install curl, and all five now invoke the same
script — which incidentally fixes the rootless images, whose healthcheck
has called docker_healthcheck.js since May 2025 while the build emitted
.cjs, failing with MODULE_NOT_FOUND on every run.
Two defects surfaced only once the tests drove the real script. curl's
--fail treats a 3xx as success, where the node probe required exactly
200, so the status is now compared explicitly. And the TLS fixture had
to be reissued as RSA: the curl macOS ships links LibreSSL 3.3.6, which
cannot complete an EC handshake against node's TLS, so the certificate
test passed in CI and failed on every mac.
The suite keeps its shape. Six unit tests cover the publisher, and seven
drive the shipped script itself against real servers, so the exit codes
asserted are the ones docker reads. They skip on Windows, which costs
nothing while every CI suite runs on ubuntu. healthcheck.ts is at 100%
lines and branches.
Not verified here, for want of a container: the package installs, the
probe running under gosu against files the server owns, and the image
size. CI covers the first two on Dockerfile and Dockerfile.alpine only.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The Dockerfiles set only --start-period, so the interval and timeout came
from Docker's defaults. The 30s interval spends 2880 cold node processes a
day on machines that are otherwise idle, and the 30s timeout was dead
config: probeHealth() caps itself at 2s, so Docker's own timer could never
fire. Vaultwarden, Outline and Uptime Kuma all settled on 60s; the projects
still at 30s got there by inheriting the default rather than choosing it.
--start-period goes to 30s so a first boot that creates the schema, runs a
migration or checks consistency is not reported unhealthy, and so the CI
step that waits 50s for a healthy container keeps its slack now that a
missed probe retries a minute later instead of 30s later. A successful
probe ends the window immediately, so a normal boot pays none of it.
Worst case time to unhealthy moves from ~100s to ~210s. Nothing acts on
that signal on its own, so the cost is a human reading docker ps later.
assertHealthcheckStaysSmall() justifies its bundle ceiling with the probe's
cadence, so its comment moves to 60 seconds too.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two defects in the docker healthcheck, in one module because the second
rewrote the file the first touches.
The probe passed `timeout: 2000` to http.request, which only arms the
socket idle timer and emits a "timeout" event. Nothing listened for it
and nothing destroyed the request, so the option never had any effect:
against a server that completes the TCP handshake and then goes quiet,
the probe waited without end. That is the exact failure a healthcheck
exists to catch, and it was the one case the probe could not report.
Docker's own 30 s --timeout, which no Dockerfile overrides, was all that
ended these attempts, each of them pinning a node process for the full
30 s first. Handle the event: log the deadline, destroy the request,
report unhealthy.
The bundle was 3 MB because docker_healthcheck.ts imported config.ts,
which reaches @triliumnext/core for envToBoolean and stringToInt.
That import is a namespace re-export, so the namespace object keeps
every export of utils/index.ts live, and with it sanitize-html, cheerio,
parse5, domino, mime-db and iconv-lite. Weight was not the whole
problem: loading that chain also created the data directory, wrote a
config sample when none existed, and ran the process.exit(1) in
resource_dir.ts, so the verdict depended on filesystem layout rather
than on the server. No bundler setting removes that. Marking both
packages sideEffects-free still left 5.36 MB, because the chain is
genuinely reachable and genuinely side-effectful.
resolveHealthcheckTarget() reads the same environment variables and the
same [Network] section of config.ini instead, read-only, importing
nothing. Measured against the built artifact:
bundle 3002 KB -> 5.3 KB
cpu 154 ms -> ~30 ms per probe (bare node is ~27 ms)
data dir created -> untouched
Behavior is unchanged across 200, 500, unix socket, refused connection
and https=true; the silent server now exits 1 after 2032 ms.
Duplicating the resolution is the cost, so the tests load config.ts,
port.ts and host.ts over six environments and assert both readings
agree, rather than checking hand-written expectations. build.ts fails
the build past 100 KB, verified by lowering the limit until it fired;
that guard runs on every image build, unlike spec/build-checks, which
no workflow invokes.
healthcheck.ts is at 100% lines, branches and functions.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The system prompt tells the model it MUST call load_skill before writing
Trilium code or a search query, because guessing the API produces code that
does not run. In the browser build that tool was never registered: its four
markdown sheets were read off disk, so the whole registry stayed behind with
the server, and the instruction pointed at nothing.
The sheets move to core, beside the stack that owns them, and each host
reaches them the way its toolchain already reaches core's schema.sql — the
Node builds copy them next to the bundle and read them from RESOURCE_DIR
(falling back to the workspace symlink in dev), the browser build inlines
them with ?raw. Core keeps the catalog and the tool and asks a
host-registered reader for the bytes, the same shape as the doc-note reader
beside it.
The reader seam is a module of its own so that registering one costs a host
nothing else: the tool reaches the AI SDK through the tool registry, which
the standalone build keeps out of its startup path deliberately, and its
registration is a dynamic import for the same reason.
load_skill is now listed in allToolRegistries rather than contributed from
outside, since every host can answer for it. The help tools stay
host-registered: they read a User Guide the browser build does not carry.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Porting the LLM stack to @triliumnext/core moved model_prices.json with it,
but update-model-prices kept writing to the old
apps/server/src/services/llm/providers/ path. That directory still exists --
it holds the Claude Agent provider -- so the write succeeded, the script
printed "Wrote ..." and exited 0, and the committed table went stale with
nothing to notice. The weekly workflow failed the same way but worse: its
add-paths named the same dead location, and create-pull-request restricted
to a path that only ever holds an untracked file sees no change, so it has
quietly opened no PR since the port.
Nothing was wrong with the generation itself -- a run against current
upstream data reproduces the committed table byte for byte -- so this is
only the wiring. The script moves beside the file it generates, which makes
both of its relative paths correct without editing either, and the npm
script and the workflow's paths follow it.
Two guards, because the breakage was silent in both directions:
- The script asserts its output already exists. It regenerates a committed
file, never creates one, so a missing target means the table has moved
again; that is now an error rather than a successful write into nowhere.
- scripts/ gets typechecked, via a tsconfig.scripts.json referenced from
core's solution file. It cannot join tsconfig.lib.json, whose rootDir is
src and whose types are deliberately empty. The script's `import type`
of base_provider.js would have flagged the port the day it happened, but
tsx erases type imports and scripts/ sat outside every project, so the
one signal that existed never ran.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Since v13 the package bundles a prebuilt binary for all eight platforms
it supports (~17 MB) on top of the SQLite amalgamation needed to compile
from scratch (~10 MB). A packaged artifact loads exactly one binary and
never compiles, so all of that but ~2 MB was dead weight:
v0.104.1 (v12) 11.93 MB deps/ + a compiled build/Release binary
v13, untrimmed 27.0 MB deps/ + eight prebuilds
server, trimmed 4.5 MB lib/ + linux-x64 + linuxmusl-x64
desktop, trimmed 2.2 MB lib/ + linux-x64
That is ~6 MB off each compressed Linux artifact, and leaves us below
what v0.104.1 shipped rather than 15 MB above it. lib/ plus one prebuild
is the complete runtime set -- verified by loading a copy pruned to just
those files and running a query against it.
Two things the trim has to get right, both covered:
Linux keeps the musl build alongside the glibc one. The server's dist is
produced once on a glibc runner and then consumed by both the Debian and
the Alpine images, and better-sqlite3 resolves linuxmusl-* at runtime on
the latter, so dropping it would break the amd64 image at startup. The
Electron artifacts opt out, since Electron ships no musl builds.
The platform and architecture are the ones being built *for*, not the
host's: the macOS runners are arm64 and also package darwin-x64, so
trimming by host arch would strip the binary that build needs.
TARGET_ARCH / MATRIX_ARCH already carry the target in CI.
An unsupported platform/arch throws rather than silently producing an
artifact with no native addon.
Also drops the better-sqlite3 cleanup from flake.nix's server install
phase: it removed deps/sqlite3 and the build/ scaffolding that held
build-time store paths, all of which the build now trims itself.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
v13 moves the addon onto N-API and ships prebuilt binaries for every
supported platform/arch inside the package itself. Three parts of the
build assumed the old prebuild-install/node-gyp layout.
Dropped `bindings` and `file-uri-to-path` from copyNodeModules: v13 no
longer depends on either, so the copy aborted the build outright with
"Unable to find any of the paths: .../node_modules/bindings".
Removed the Electron native rebuild. v13's binding.gyp resolves both of
its targets to `type: none` whenever a prebuild for the host exists, so
electron-rebuild.mts produced a stamp file and no addon on every install,
leaving BETTERSQLITE3_NATIVE_PATH pointing at a build/Release binary that
is never created -- `pnpm desktop:start` died on MODULE_NOT_FOUND. The
prebuilds are ABI-stable and load unchanged under Electron 42 (Node-API
10), so the rebuild step, its @electron/rebuild and prebuild-install
devDependencies, and the ELECTRON_NODEDIR plumbing in flake.nix all go.
Rebuilt the Docker images around the same short-circuit: `pnpm rebuild`
in the builder stages was also a no-op, so the images would have shipped
the upstream prebuild, which needs glibc >= 2.34 (x64) / >= 2.38 (arm64)
against bullseye's 2.31. The glibc images move to trixie and lose their
builder stage; the Alpine pair lose theirs too, since the linuxmusl-*
prebuilds only need libstdc++, which node:*-alpine already provides.
Alpine stays pinned to 3.23 for the musl 1.2.5 reason in #10627.
Dockerfile.legacy must keep compiling from source -- 32-bit ARM has no
prebuild at all, and linux/arm/v8 normalizes to arm64, whose prebuild
outruns that image's bullseye glibc. npm_config_force_build=1 restores
the real compile and link rules.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
DeepSeek speaks the OpenAI API, so the custom endpoint card already
reached it — but that card can name no provider, and pricing comes from
`MODEL_PRICES[this.name]`, so its models arrived with no cost and no
context window. A card of its own fixes exactly that: DeepSeek publishes
bare ids that the committed table carries verbatim.
The price table gains a `deepseek` branch in the update script, so the
weekly refresh keeps it current (4 models: the v4 pair at $0.14/$0.28
and $0.435/$0.87, plus the older chat/reasoner pair at $0.28/$0.42).
Model defaults follow the listing rather than a hardcoded alias, as the
self-hosted provider already does: an account provisioned for the v4 line
lists neither `deepseek-chat` nor `deepseek-reasoner`, so titling with a
baked-in id would call a model the account does not have. Titles go to
the cheapest listed model, conversation to the dearest, both taking the
first of a tied group — chat and reasoner cost the same per token, but
the reasoner spends far more of them.
Closes#8822
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
model_listing.ts had accreted three unrelated things over the feature's
seven commits. Only one of them was genuinely shared; the rest were
single-provider rules parked in a shared module for testability, which
the provider specs already cover through their mocked endpoints.
- OpenAI's chat-model filter, display-name derivation and newest-
generation rule move into openai.ts; Gemini's chat-model filter into
google.ts; the Claude per-family rule into anthropic.ts.
- recommendedModelIds becomes a method on LlmProvider, defaulting to the
generic non-preview/non-legacy rule in BaseProvider. listProviderModels
now calls instance.recommendedModelIds(models) instead of dispatching
on a user-controlled provider string.
- claude-agent shares Anthropic's id shape, so it delegates to the
helper exported from anthropic.ts rather than duplicating the regex —
it implements LlmProvider directly and cannot inherit the override.
- mergeModelLists, RemoteModel and the price-table types have two
independent consumers (BaseProvider and ClaudeAgentProvider) and no
single owner, so they move to base_provider.ts, which claude_agent.ts
already imported for buildModelList. update-model-prices.ts follows;
its import stays type-only so the script pulls in no runtime deps.
The module's tests move with it, each into the spec of the provider that
now owns the rule.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the hand-curated per-provider model arrays with a committed,
pruned LiteLLM price/context table (model_prices.json) as the single
source of truth for pricing and context windows. The provider endpoints
stay authoritative for which models exist and their display names;
recommendations and the default model are derived from id shape.
- Prune LiteLLM to the bare ids our endpoints return, grouped by
provider, via scripts/update-model-prices.ts; refresh weekly through a
scheduled workflow that opens a PR only when the table changes.
- Drop the per-provider costMultiplier (only comparable within one
provider) in favour of showing real $/Mtok in the model pickers.
- The Claude subscription provider keeps its own zero-priced catalog;
metered LiteLLM prices do not apply to it.
Also surface provider model-listing failures (bad API key, unreachable
endpoint, Claude Code not installed) as actionable errors in the
add/edit-provider screen instead of masking them with fallback defaults.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
