mirror of
https://github.com/zadam/trilium.git
synced 2026-09-14 02:56:04 +05:00
2faa9407c5
Reading the images a page paints only works when the page is one scanned image. A hotel voucher whose text is drawn as vector outlines -- 2559 constructPath operations and no text-showing operator at all -- has exactly one image on it, the 180x91 logo, so OCR returned "SMART TOURS" and nothing else. Neither pdf.js nor PDFium finds any text in that file; the only way to read it is to rasterize the page. Rendering the page covers every case pulling the images out did, and several it could not: text drawn as outlines, a scan split across image strips, a page whose /Rotate the images themselves know nothing about, and stamps or annotations drawn over a scan. It also bounds the work, since a page becomes a fixed number of pixels no matter what resolution it was scanned at -- the old path would have allocated a 140 MB RGBA buffer for a 600 dpi scan. It costs nothing to do so. End to end, PDF bytes to a PNG Tesseract can read, the two paths measured 175-240 ms and 186-256 ms on the sample scan; PNG encoding dominates both. Recognition is unchanged: 94 confidence and 718 characters from the embedded image, 93 and 716 from the render, same text. So the image path goes away entirely rather than staying on as a fast case, and with it the channel conversion, its unsupported-channel error path, the minimum image dimension and the per-image loop. PDFium ships as WebAssembly, which sidesteps what a native rasterizer would have cost: @napi-rs/canvas is 30 MB per platform (20 MB of Skia statically linked against libpng, libjpeg-turbo, libavif, harfbuzz and an SVG renderer, plus 11 MB of ICU data the desktop build already carries a copy of), and it would need an esbuild external plus a per-platform binary in copyNodeModules. The wasm is 4 MB, identical on every target, and bundles normally -- main.mjs carries no reference to it and it loads from a lazy chunk on the first page that needs rasterizing. Its loader resolves the wasm against `import.meta.url`, which in the split ESM bundle is a hash-named file under chunks/, so the bytes are read from RESOURCE_DIR and handed to init() instead -- the same two paths core_assets.ts uses for schema.sql, copied beside the server's assets by the build and resolved through node_modules when running from source. Pages are rendered in colour rather than PDFium's grayscale. Its conversion flattens coloured text into its background: on the demo document it swallows the whole "Organize your thoughts" panel, 60 characters that colour rendering keeps and that no increase in scale recovers. Scale 2 is the knee of the quality curve -- confidence 82 at scale 1, 93 at 1.5, 94 at 2, and flat above it while the pixels to encode and read double again. The voucher now yields 3284 characters at 0.93 confidence across both pages, including the guest name, the hotel, the booking reference and the room type. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
42 lines
2.2 KiB
TypeScript
42 lines
2.2 KiB
TypeScript
import BuildHelper from "../../../scripts/build-utils";
|
|
|
|
const build = new BuildHelper("apps/server");
|
|
|
|
async function main() {
|
|
// ESM so dynamic-import boundaries split into chunks/ that only load on first use; the
|
|
// image worker below is spawned by its .cjs path and stays CJS.
|
|
await build.buildBackend([ "src/main.ts" ], { format: "esm" });
|
|
// Its own call so it lands beside the bundle rather than under a `services/` path: the pool
|
|
// looks for it next to whatever is running, and desktop builds it the same way.
|
|
await build.buildBackend([ "src/services/image_worker.ts" ]);
|
|
|
|
// Copy assets
|
|
build.copy("src/assets", "assets/");
|
|
// The healthcheck docker runs; the Dockerfiles point at it inside the image.
|
|
build.copy("docker_healthcheck.sh", "docker_healthcheck.sh");
|
|
// schema.sql lives in trilium-core but is loaded at server startup. The
|
|
// bundled main.mjs can't `require.resolve("@triliumnext/core/...")` in
|
|
// Docker (no workspace symlinks in the image), so we copy the file
|
|
// alongside the server's own assets and read it via RESOURCE_DIR at
|
|
// runtime. See main.ts.
|
|
build.copy("/packages/trilium-core/src/assets/schema.sql", "assets/schema.sql");
|
|
// Same story for the LLM skill sheets: core owns them, the server reads them
|
|
// from RESOURCE_DIR at runtime. See core_assets.ts.
|
|
build.copy("/packages/trilium-core/src/assets/llm/skills", "assets/llm/skills/");
|
|
// PDFium rasterizes scanned PDF pages for OCR. Its own loader resolves the wasm relative to
|
|
// `import.meta.url`, which in a split bundle is a hash-named file under chunks/, so the bytes
|
|
// are handed to it explicitly from here instead. See pdf_renderer.ts.
|
|
build.copy("/node_modules/@hyzyla/pdfium/dist/pdfium.wasm", "assets/pdfium.wasm");
|
|
build.triggerBuildAndCopyTo("packages/share-theme", "share-theme/assets/");
|
|
build.copy("/packages/share-theme/src/templates", "share-theme/templates/");
|
|
|
|
// Copy node modules dependencies
|
|
build.copyNodeModules([ "better-sqlite3" ]);
|
|
build.trimBetterSqlite3();
|
|
build.copy("/node_modules/ckeditor5/dist/ckeditor5-content.css", "ckeditor5-content.css");
|
|
|
|
build.buildFrontend();
|
|
}
|
|
|
|
main();
|