Commit Graph

496 Commits

Author SHA1 Message Date
Nick Sweeting
8a06218852
Allow stable ArchiveBox releases only from main
Some checks are pending
CI / Linters (push) Waiting to run
CI / Install and CLI platform compatibility (push) Waiting to run
CI / Discovered test matrix (push) Waiting to run
CI / Documentation and root tests (push) Waiting to run
CI / CodeQL (push) Waiting to run
CI / Tested Python artifacts (push) Waiting to run
CI / Tested Docker images (push) Waiting to run
CI / All required CI lanes (push) Blocked by required conditions
Deploy Publicsite to GitHub Pages / deploy (push) Waiting to run
2026-07-27 23:35:57 -07:00
Nick Sweeting
227a2a46a7
Require prerelease versions outside ArchiveBox main 2026-07-27 21:57:08 -07:00
Nick Sweeting
f6edf69d54
Resolve CI tools through normal abxpkg paths 2026-07-27 14:54:27 -07:00
Nick Sweeting
f58301b70c Improve setup wizard sections 2026-07-26 12:23:13 -07:00
Nick Sweeting
412bd144f0
Stabilize rewritten preview screenshots 2026-07-25 22:54:35 -07:00
Nick Sweeting
7781f306f3
Install OpenCode before screenshot capture 2026-07-25 16:20:07 -07:00
Nick Sweeting
61d3aae6c6
Enable OpenCode for screenshot gallery 2026-07-25 15:51:03 -07:00
Nick Sweeting
b9b0e0e808
Accept live progress placeholder in screenshots
Some checks are pending
CI / Linters (push) Waiting to run
CI / Install and CLI platform compatibility (push) Waiting to run
CI / Discovered test matrix (push) Waiting to run
CI / Documentation and root tests (push) Waiting to run
CI / CodeQL (push) Waiting to run
CI / Tested Python artifacts (push) Waiting to run
CI / Tested Docker images (push) Waiting to run
CI / All required CI lanes (push) Blocked by required conditions
Deploy Publicsite to GitHub Pages / deploy (push) Waiting to run
2026-07-25 15:34:59 -07:00
Nick Sweeting
28d3392291
Fix screenshot deploy sequencing 2026-07-25 15:26:08 -07:00
Nick Sweeting
002e348672
Disable sandbox for UI setup screenshots without display 2026-07-25 15:09:35 -07:00
Nick Sweeting
32031b17f7
Use projected Chrome for UI screenshots 2026-07-25 15:04:29 -07:00
Nick Sweeting
05350846ab
Sync ArchiveBox UI and setup updates 2026-07-25 14:50:15 -07:00
Nick Sweeting
c89f6b0542
Fix ArchiveBox CI runtime regressions 2026-07-25 12:26:20 -07:00
Claude
f7c182d76c
Merge remote-tracking branch 'origin/dev' into claude/archivebox-postgresql-support-xrlgim
# Conflicts:
#	.github/workflows/test-parallel.yml
#	.github/workflows/test.yml
#	uv.lock
2026-07-24 10:30:55 +00:00
Nick Sweeting
eacfae5d6c
Run all ArchiveBox tests through unified CI 2026-07-24 00:50:40 -07:00
Claude
c2191f9e74
Fix lint: settings import placement and benchmark script exec bit
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019YHSjZM6TstSAMN2PhgfUg
2026-07-23 17:18:54 +00:00
Claude
7167c5dd8a
Add native PostgreSQL support alongside SQLite
Add DATABASE_ENGINE=postgres (plus DATABASE_HOST/PORT/USER/PASSWORD/NAME)
config and centralize all sqlite-vs-postgres branching in
archivebox.misc.db:

- get_database_settings() builds DATABASES for either backend; the sqlite
  path is unchanged (custom lock-retry backend, same PRAGMAs).
- database_exists()/ensure_database_ready() replace index.sqlite3 file
  checks; init auto-creates the postgres database when missing.
- approximate_row_counts() serves admin index counts from sqlite_stat1 or
  pg_class.reltuples; missing-table detection covers both vendors.
- rebuild_models_from_migration_state() lets historical sqlite-only raw
  SQL migrations resync postgres schema from Django migration state at
  every divergence point (postgres can never hold legacy data, so
  affected tables are empty when these run). All raw-DDL and PRAGMA
  migrations are now vendor-gated with sqlite behavior byte-for-byte
  unchanged.
- A pre_save clamp truncates CharField values to max_length: sqlite
  never enforced varchar(n) but postgres does (e.g. long crawl labels).
- Collation-sensitive URL range scans branch to escaped LIKE on postgres
  (with a text_pattern_ops index) since linguistic collations break
  bytewise range tricks; the crawl-config JSON search wave gets a
  jsonb-text implementation.

Verified on real PostgreSQL 16: fresh init applies the entire migration
graph, schema matches models exactly (column-level parity check +
makemigrations --check), and add/run/list/search/status/remove all work
end-to-end. New test_postgres_backend.py suite boots a real throwaway
postgres cluster (initdb + pg_ctl); CI workflows install postgres server
binaries.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019YHSjZM6TstSAMN2PhgfUg
2026-07-23 17:08:47 +00:00
Nick Sweeting
a3c5e13ff8 Release ArchiveBox through complete CI coverage
Some checks failed
CI / Linters (push) Waiting to run
CI / Complete test suite (push) Waiting to run
CI / Sharded and plugin tests (push) Waiting to run
CI / Documentation and root tests (push) Waiting to run
CI / CodeQL (push) Waiting to run
CI / Tested Python artifacts (push) Waiting to run
CI / Tested Docker images (push) Waiting to run
CI / All required CI lanes (push) Blocked by required conditions
Deploy Publicsite to GitHub Pages / deploy (push) Has been cancelled
2026-07-23 07:00:05 -07:00
Nick Sweeting
c36d3c1575
Make release publication atomic and CI-only 2026-07-21 19:26:42 -07:00
Nick Sweeting
6592af0d19
Exercise ArchiveBox through unified runtime lifecycles 2026-07-21 19:26:42 -07:00
Nick Sweeting
582499be86
Deduplicate ArchiveBox release CI 2026-07-20 15:19:32 -07:00
Nick Sweeting
01f5666dbc
Route runtime binaries through abxpkg 2026-07-20 12:50:14 -07:00
Nick Sweeting
11d962406d
Update setup script for dev uv installs 2026-06-24 09:28:32 -07:00
Nick Sweeting
095c0e5112
fix admin snapshot changelist preview summaries 2026-06-21 21:33:14 -07:00
Nick Sweeting
8b57085827
chore: commit local archivebox changes 2026-06-14 11:44:38 -07:00
Nick Sweeting
f3c8460767
ci: use pypi version for archivebox rc bumps 2026-06-14 11:43:38 -07:00
Nick Sweeting
e63f24dc71
ci: require trusted publishing for pypi 2026-06-14 11:19:03 -07:00
Nick Sweeting
cb5c822eb4
ci: use pypi trusted publishing in release state 2026-06-14 10:47:44 -07:00
Nick Sweeting
b768e1767e
release: archivebox 0.9.35rc34 2026-06-14 08:41:37 -07:00
Nick Sweeting
9a6d1b0410
fix: wait for abx-dl docker base in dev release 2026-06-14 01:43:20 -07:00
Nick Sweeting
d3ee9512e5
fix: pin archivebox docker base to abx-dl dependency 2026-06-14 01:07:24 -07:00
Nick Sweeting
18e3580dcd
fix: harden reflected metadata rendering 2026-06-14 00:06:35 -07:00
Nick Sweeting
6ae56b7366
release: archivebox 0.9.35rc28 2026-06-13 23:53:45 -07:00
Nick Sweeting
73ec255855
fix: harden admin metadata surfaces 2026-06-13 23:44:59 -07:00
Nick Sweeting
426e23f33c
Prevent deploy exec from consuming script input
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Docker image / build ${{ matrix.platform }} (digest-linux-amd64, docker-amd64, linux/amd64, ubuntu-24.04) (push) Waiting to run
Build Docker image / build ${{ matrix.platform }} (digest-linux-arm64, docker-arm64, linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build Docker image / publish multiarch tags (push) Blocked by required conditions
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Release State / release-state (push) Waiting to run
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Parallel Tests / ${{ matrix.plugin.name }} (push) Blocked by required conditions
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
Update Homebrew tap / dispatch (push) Waiting to run
2026-06-11 01:02:09 -07:00
Nick Sweeting
e6091cf50e
Capture deploy version warnings 2026-06-11 00:59:55 -07:00
Nick Sweeting
77500bfac8
Fix dev deploy image selection 2026-06-11 00:56:28 -07:00
Nick Sweeting
bb598b9710
release: archivebox 0.9.35rc23 2026-06-11 00:41:34 -07:00
Nick Sweeting
9d5ed1bc42
release: archivebox 0.9.35rc21 2026-06-11 00:08:54 -07:00
Nick Sweeting
585c951aa7
release: archivebox 0.9.35rc20 2026-06-11 00:00:21 -07:00
Nick Sweeting
635e318647
Use PyPI version endpoint in release wait 2026-06-10 23:45:16 -07:00
Nick Sweeting
1da4b79e9d
Make dev stack release resumable after PyPI lag 2026-06-10 23:43:27 -07:00
Nick Sweeting
303e99258c
Simplify Docker data ownership detection 2026-06-10 09:13:02 -07:00
Nick Sweeting
9c29724efa
Allow selecting Docker release registries 2026-06-09 23:31:07 -07:00
Nick Sweeting
98aecfaf91
release: archivebox 0.9.35rc8 2026-06-09 23:12:22 -07:00
Nick Sweeting
a5e419ca3f
release: archivebox 0.9.35rc6 2026-06-09 22:45:25 -07:00
Nick Sweeting
2b71254fe9
Restore Docker lib root under opt archivebox 2026-06-08 04:44:40 -07:00
Nick Sweeting
1bb51ca9ee
Repair abxbus cache ownership at Docker startup 2026-06-08 04:30:12 -07:00
Nick Sweeting
4fa90e484a
release: archivebox 0.9.34rc71 2026-06-07 20:51:28 -07:00
Nick Sweeting
87b518314a
release: archivebox 0.9.34rc67 2026-06-07 04:06:45 -07:00
Nick Sweeting
0a3c89e73c
release: archivebox 0.9.34rc63 2026-06-06 23:27:54 -07:00
Nick Sweeting
466238d7dd
Use published abx-dl image for Docker builds 2026-06-05 00:18:11 -07:00
Nick Sweeting
364efdf409
fix release pypi retry loop 2026-06-04 09:17:34 -07:00
Nick Sweeting
caf0acdec8
release: archivebox 0.9.34rc54 2026-06-04 00:48:47 -07:00
Nick Sweeting
c0fb8eb532
release: archivebox 0.9.34rc39 2026-06-03 17:19:48 -07:00
Nick Sweeting
96437e1ffd
Publish local ArchiveBox changes 2026-06-02 02:25:52 -07:00
Nick Sweeting
cab05eb1c6
Refactor plugins search progress and config flows 2026-06-01 00:08:27 -07:00
Nick Sweeting
211c3e234d
release: archivebox 0.9.33rc77 2026-05-31 18:24:37 -07:00
Nick Sweeting
c1e792fee6
release: v0.9.33rc52 2026-05-31 01:24:43 -07:00
Nick Sweeting
3521f21c36
fix: self-heal legacy crawls_crawl NOT NULL columns + Puppeteer timeout
Pre-0.9.0 crawl tables had ``max_urls``, ``crawl_max_size``, and
``snapshot_max_size`` as NOT NULL ``IntegerField`` columns. The 0011
migration RemoveField'd them, but on long-lived DBs where a historical
0011 ran with different semantics the columns stayed on disk while
being absent from the model. Every Crawl.objects.create() through the
ORM (e.g. the /add/ form view) skips them, and SQLite refuses the
insert with "NOT NULL constraint failed: crawls_crawl.max_urls" —
an HTTP 500 on /add/. Cabbage hit this on the rc51 UI test.

The new 0017 migration introspects the live table, copies any
pre-existing values into ``config`` under the canonical
CRAWL_MAX_URLS / CRAWL_MAX_SIZE / SNAPSHOT_MAX_SIZE keys so nothing
gets silently dropped, then ALTER TABLE DROP COLUMNs each legacy
field. Fresh installs already have these columns removed and the
migration is a no-op there. Requires SQLite 3.35+ for DROP COLUMN.

Plus: bin/take_screenshot.js bumps Puppeteer's CDP protocolTimeout
from the 30s default to 5 minutes — admin pages with many DB-backed
partials (snapshot grid w/ 23K rows, progress monitor, etc.) blow
past 30s on the screenshot capture under load.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-30 23:14:35 -07:00
Nick Sweeting
5a38193f56
release: archivebox 0.9.33rc50 2026-05-30 22:27:28 -07:00
Nick Sweeting
031e956080
release: archivebox 0.9.33rc29
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Debian package / build (amd64) (push) Waiting to run
Build Debian package / build (arm64) (push) Waiting to run
Build Debian package / test (amd64, ubuntu-24.04) (push) Blocked by required conditions
Build Debian package / test (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
Build Debian package / release (push) Blocked by required conditions
Build Docker image / build ${{ matrix.platform }} (digest-linux-amd64, docker-amd64, linux/amd64, ubuntu-24.04) (push) Waiting to run
Build Docker image / build ${{ matrix.platform }} (digest-linux-arm64, docker-arm64, linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build Docker image / publish multiarch tags (push) Blocked by required conditions
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Release State / release-state (push) Waiting to run
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Parallel Tests / ${{ matrix.plugin.name }} (push) Blocked by required conditions
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2026-05-28 15:35:33 -07:00
Nick Sweeting
f60b190182
fix: centralize ctrl-c rich logging 2026-05-28 08:49:17 -07:00
Nick Sweeting
145168621c
chore: checkpoint search loop changes 2026-05-28 08:04:03 -07:00
Nick Sweeting
e15fc42e43
chore: checkpoint deploy loop updates 2026-05-28 07:15:20 -07:00
Nick Sweeting
fe0ba18faa
release: archivebox 0.9.33rc16 2026-05-28 06:48:51 -07:00
Nick Sweeting
fe09f5a517
chore: apply release hook fixes 2026-05-28 06:42:53 -07:00
Nick Sweeting
b2fa839788
chore: checkpoint dev stack deploy changes 2026-05-28 06:39:06 -07:00
Nick Sweeting
f09c3a8e0d
release: archivebox 0.9.33rc9 2026-05-28 05:50:28 -07:00
Nick Sweeting
fbcc972441
backup: save in-progress dev changes 2026-05-28 05:20:52 -07:00
Nick Sweeting
3f31223ba8
release: archivebox 0.9.32rc36
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Debian package / build (amd64) (push) Waiting to run
Build Debian package / build (arm64) (push) Waiting to run
Build Debian package / test (amd64, ubuntu-24.04) (push) Blocked by required conditions
Build Debian package / test (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
Build Debian package / release (push) Blocked by required conditions
Build Docker image / buildx (push) Waiting to run
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Release State / release-state (push) Waiting to run
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Parallel Tests / ${{ matrix.plugin.name }} (push) Blocked by required conditions
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2026-05-27 16:24:09 -07:00
Nick Sweeting
a21765f366
release: v0.9.32rc25 2026-05-27 13:00:12 -07:00
Nick Sweeting
12ad336e37
release: archivebox 0.9.32rc25 2026-05-27 12:50:44 -07:00
Nick Sweeting
5d6df4b310
fix: skip CI waits without precheck job 2026-05-27 11:08:43 -07:00
Nick Sweeting
5fbd6383d8
fix: wait for exact PyPI release 2026-05-27 11:02:40 -07:00
Nick Sweeting
5256e5cd33
fix: improve crawl progress metadata 2026-05-27 10:58:54 -07:00
Nick Sweeting
28b890a2c7 Fix multi-platform Docker release scripts 2026-05-25 06:20:58 +00:00
Nick Sweeting
30c841d477
Update dev docs and MHTML preview handling 2026-05-24 22:41:16 -07:00
Nick Sweeting
ec4eded38a
release: archivebox 0.9.31rc39 2026-05-24 13:25:16 -07:00
Nick Sweeting
fb9f4a724b
release: archivebox 0.9.31rc20
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Debian package / build (amd64) (push) Waiting to run
Build Debian package / build (arm64) (push) Waiting to run
Build Debian package / test (amd64, ubuntu-24.04) (push) Blocked by required conditions
Build Debian package / test (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
Build Debian package / release (push) Blocked by required conditions
Build Docker image / buildx (push) Waiting to run
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Release State / release-state (push) Waiting to run
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Parallel Tests / ${{ matrix.plugin.name }} (push) Blocked by required conditions
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2026-05-23 19:36:09 -07:00
Nick Sweeting
3b8ce3f467
release: archivebox 0.9.31rc18 2026-05-23 17:32:58 -07:00
Nick Sweeting
f9512d5ea2
release: archivebox 0.9.31rc17 2026-05-23 17:14:50 -07:00
Nick Sweeting
ebf597b898
release: archivebox 0.9.31rc16 2026-05-23 17:10:35 -07:00
Nick Sweeting
8f2ca42550
Avoid committing release dist artifacts 2026-05-23 17:00:57 -07:00
Nick Sweeting
e5dd4a0fad
Add dev stack release helper 2026-05-23 17:00:34 -07:00
Nick Sweeting
d263a611cc
Allow root with warnings 2026-05-23 16:49:20 -07:00
Nick Sweeting
009d338dad
Simplify standalone setup dependencies 2026-05-23 16:03:29 -07:00
Nick Sweeting
943d4ee64b
Simplify Docker runtime dependency install
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Debian package / build (amd64) (push) Waiting to run
Build Debian package / build (arm64) (push) Waiting to run
Build Debian package / test (amd64, ubuntu-24.04) (push) Blocked by required conditions
Build Debian package / test (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
Build Debian package / release (push) Blocked by required conditions
Build Docker image / buildx (push) Waiting to run
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Release State / release-state (push) Waiting to run
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Parallel Tests / ${{ matrix.plugin.name }} (push) Blocked by required conditions
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2026-05-23 15:51:53 -07:00
Nick Sweeting
5f69ec656d
Move Docker runtime lib dir 2026-05-23 15:44:03 -07:00
Nick Sweeting
0e527df929
Bake plugin runtime deps into Docker image 2026-05-23 15:37:42 -07:00
Nick Sweeting
811ef03982
Bump rc and fix demo deploy image override 2026-05-23 14:41:47 -07:00
Nick Sweeting
880aaa13c1
Drop stale docker mirror tag 2026-05-22 20:17:34 -07:00
Nick Sweeting
c1e9106b9a
Fix docker release platform default 2026-05-22 20:16:32 -07:00
Nick Sweeting
92339e08f0
Prepare dev demo deploy 2026-05-22 20:12:48 -07:00
Nick Sweeting
fa15e1edfa
Fix docker runtime and real crawl flows 2026-05-14 21:24:32 -07:00
Nick Sweeting
5f8098c632
fix recursive crawls and docker runtime 2026-05-13 08:13:33 -07:00
Nick Sweeting
5d60098618
fix: stabilize dockerized crawl runtime
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Debian package / build (amd64) (push) Waiting to run
Build Debian package / build (arm64) (push) Waiting to run
Build Debian package / test (amd64, ubuntu-24.04) (push) Blocked by required conditions
Build Debian package / test (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
Build Debian package / release (push) Blocked by required conditions
Build Docker image / buildx (push) Waiting to run
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Release State / release-state (push) Waiting to run
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Parallel Tests / Plugin tests (push) Waiting to run
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2026-05-13 02:05:07 -07:00
Nick Sweeting
7d8c468659
rename abx-pkg to abxpkg
Some checks failed
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Has been cancelled
Build Debian package / build (amd64) (push) Has been cancelled
Build Debian package / build (arm64) (push) Has been cancelled
Build Docker image / buildx (push) Has been cancelled
Deploy static content to Pages / deploy (push) Has been cancelled
Build GitHub Pages website / build (push) Has been cancelled
Run linters / lint (push) Has been cancelled
Build Pip package / build (push) Has been cancelled
Release State / release-state (push) Has been cancelled
Parallel Tests / Discover test files (push) Has been cancelled
Parallel Tests / Plugin tests (push) Has been cancelled
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Has been cancelled
Run tests / docker_tests (push) Has been cancelled
Build Debian package / test (amd64, ubuntu-24.04) (push) Has been cancelled
Build Debian package / test (arm64, ubuntu-24.04-arm) (push) Has been cancelled
Build Debian package / release (push) Has been cancelled
Build GitHub Pages website / deploy (push) Has been cancelled
Parallel Tests / ${{ matrix.test.name }} (push) Has been cancelled
2026-04-17 10:48:40 -07:00
Nick Sweeting
ee5685353b
rename abx-pkg to abxpkg 2026-04-17 10:37:10 -07:00
Nick Sweeting
f1287510ff
rename abxpkg 2026-04-17 10:36:19 -07:00
Nick Sweeting
f126c6e628
symlink lock_pkgs to setup monorepo script 2026-04-07 20:21:41 -07:00
Nick Sweeting
3e7b83ac91
bump versions
Some checks failed
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Has been cancelled
Build Debian package / build (amd64) (push) Has been cancelled
Build Debian package / build (arm64) (push) Has been cancelled
Build Docker image / buildx (push) Has been cancelled
Deploy static content to Pages / deploy (push) Has been cancelled
Build GitHub Pages website / build (push) Has been cancelled
Run linters / lint (push) Has been cancelled
Build Pip package / build (push) Has been cancelled
Release State / release-state (push) Has been cancelled
Parallel Tests / Discover test files (push) Has been cancelled
Parallel Tests / Plugin tests (push) Has been cancelled
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Has been cancelled
Run tests / docker_tests (push) Has been cancelled
Build Debian package / test (amd64, ubuntu-24.04) (push) Has been cancelled
Build Debian package / test (arm64, ubuntu-24.04-arm) (push) Has been cancelled
Build Debian package / release (push) Has been cancelled
Build GitHub Pages website / deploy (push) Has been cancelled
Parallel Tests / ${{ matrix.test.name }} (push) Has been cancelled
2026-04-04 23:10:12 -07:00
Nick Sweeting
f2c81142e1
tweak release script
Some checks failed
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Deploy static content to Pages / deploy (push) Waiting to run
Build GitHub Pages website / build (push) Waiting to run
Build GitHub Pages website / deploy (push) Blocked by required conditions
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Parallel Tests / Plugin tests (push) Waiting to run
Build Debian package / build (amd64) (push) Has been cancelled
Build Debian package / build (arm64) (push) Has been cancelled
Build Docker image / buildx (push) Has been cancelled
Run linters / lint (push) Has been cancelled
Build Pip package / build (push) Has been cancelled
Release State / release-state (push) Has been cancelled
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Has been cancelled
Run tests / docker_tests (push) Has been cancelled
Build Debian package / test (amd64, ubuntu-24.04) (push) Has been cancelled
Build Debian package / test (arm64, ubuntu-24.04-arm) (push) Has been cancelled
Build Debian package / release (push) Has been cancelled
2026-03-23 04:21:12 -07:00
Nick Sweeting
1d94645abd
test fixes 2026-03-23 04:12:31 -07:00
Nick Sweeting
b749b26c5d
wip 2026-03-23 03:58:32 -07:00
Nick Sweeting
f400a2cd67
WIP: checkpoint working tree before rebasing onto dev 2026-03-22 20:25:18 -07:00
Nick Sweeting
9de084da65
bump package versions 2026-03-15 20:47:28 -07:00
Nick Sweeting
4756697a17
Use ruff pyright and ty for linting 2026-03-15 19:43:59 -07:00
Nick Sweeting
934e02695b
fix lint 2026-03-15 18:45:29 -07:00
Nick Sweeting
70c9358cf9
Improve scheduling, runtime paths, and API behavior 2026-03-15 18:31:56 -07:00
Nick Sweeting
7d42c6c8b5
bump versions and fix docs 2026-03-15 17:43:07 -07:00
Nick Sweeting
9ef8a1b23a
Stabilize secret-backed plugin CI 2026-03-15 08:37:41 -07:00
Nick Sweeting
4fa701fafe
Update abx dependencies and plugin test harness 2026-03-15 04:37:32 -07:00
Nick Sweeting
c64e14c656
Merge branch 'dev' into claude/restore-package-managers-GlgBJ 2026-03-14 22:43:15 -07:00
Claude
36b4055304
Add caveats block to Homebrew formula showing data directory
The post_install initializes var/archivebox as the data directory,
but users need to know where it is for subsequent commands. The
caveats block is shown after brew install/upgrade to guide users.

https://claude.ai/code/session_01Vx1EsNrNySgsc8Y67dGzCn
2026-03-15 04:56:25 +00:00
Claude
fbde2dee03
Fix remaining PR review comments across packaging files
- preremove.sh: Stop the systemd service during upgrades (not just
  remove/purge) so the running process doesn't use stale venv binaries
  while postinstall replaces them. Only disable + remove venv on full
  removal.

- debian.yml: Fail loudly when release lookup fails during a release
  event (exit 1), but still skip gracefully for workflow_dispatch
  manual testing (exit 0). Prevents silently broken .deb publication.

- archivebox.rb + build_brew.sh + homebrew.yml: Add post_install that
  initializes ArchiveBox in var/archivebox with DATA_DIR set, using
  correct Homebrew system() syntax (no env hash as first arg).

https://claude.ai/code/session_01Vx1EsNrNySgsc8Y67dGzCn
2026-03-15 04:42:01 +00:00
Claude
82932812ae
Simplify deb/brew packages to thin wrappers around pip install
Remove all non-essential dependencies from both package formats.
The .deb now only depends on python3, pip, and venv. The brew
formula only depends on python@3.13. All other runtime deps
(node, chrome, yt-dlp, wget, ripgrep, etc.) are installed
on-demand by `archivebox install` at runtime.

Removed from brew formula:
- 6 depends_on entries (node, git, wget, curl, ripgrep, yt-dlp)
- on_linux block (pkg-config, openssl, libffi)
- post_install hook (was running archivebox install)
- service block (users run archivebox server directly)

Removed from .deb:
- 6 depends entries (nodejs, npm, git, wget, curl, ripgrep)
- recommends section (yt-dlp, ffmpeg, chromium)

Net: -126 lines across packaging files.

https://claude.ai/code/session_01Vx1EsNrNySgsc8Y67dGzCn
2026-03-15 03:56:39 +00:00
Claude
496b54a5e1
Fix remaining PR review comments: release ordering, verification, README
- Move .deb upload to GitHub Release into a separate job that runs after tests pass
- Fix workflow_call event propagation so release jobs run when called from release.yml
- Fix setup.sh post-install verification to check `which archivebox` first (works for brew/deb)
- Fix README.md: detect architecture with dpkg instead of hardcoding amd64
- Fix README.md: remove --setup flag from apt install instructions

https://claude.ai/code/session_01Vx1EsNrNySgsc8Y67dGzCn
2026-03-15 03:20:32 +00:00
Claude
7c7a9ee599
Fix PR review comments: service flags, DATA_DIR, version pinning, upgrade safety
- Remove --setup flag from systemd service and CI (not valid in 0.9.x)
- Remove release triggers from debian/homebrew workflows (handled by release.yml)
- Fix brew post_install to set DATA_DIR so it initializes in var/archivebox
- Add PATH export to deb wrapper script for bundled console scripts
- Remove pip install fallback in install.sh (strict version pinning)
- Guard preremove.sh cleanup to only run on remove/purge, not upgrade
- Initialize SDIST_URL/SDIST_SHA256 in build_brew.sh (nounset safety)
- Pin awalsh128/cache-apt-pkgs-action to v1.6.0 (supply chain safety)

https://claude.ai/code/session_01Vx1EsNrNySgsc8Y67dGzCn
2026-03-15 03:12:37 +00:00
Claude
4c113f8eb9
Fix CI: create tests/out dir, fix archivebox add cmd, revert setup.sh
- test-parallel.yml: mkdir -p tests/out before pytest --basetemp
  (fixes FileNotFoundError in chrome test fixture)
- debian.yml: fix archivebox add command (--parser url_list removed
  in 0.9.x), remove || true so failures are caught
- setup.sh: revert apt section to always use pip install, not .deb

https://claude.ai/code/session_01Vx1EsNrNySgsc8Y67dGzCn
2026-03-15 03:02:47 +00:00
Claude
c8f562ee37
Wire up GitHub Actions for deb/brew build, test, and release
- Fix debian.yml: pin nfpm version, add permissions, improve test job
  with user creation, init test, and status check
- Fix homebrew.yml: use PyPI JSON API (macOS-compatible, no grep -oP),
  wait for PyPI availability on release, use generated formula not template,
  add Linux (Linuxbrew) test job alongside macOS
- Add release.yml orchestrator: pip → deb + brew + docker in order
- Add workflow_call triggers to pip.yml and docker.yml
- Fix build_brew.sh: replace grep -oP with Python-based PyPI API,
  add on_linux deps (pkg-config, openssl, libffi)
- Fix setup.sh: use GitHub API to find correct .deb download URL
  (filename includes version number)
- Fix postinstall.sh: create archivebox system user, pin version from
  package, check for systemd before daemon-reload
- Fix preremove.sh: stop service before removal, check for systemd
- Fix install.sh: fallback to latest if pinned version not on PyPI
- Add on_linux deps to brew formula for Linuxbrew support
- Tested: .deb builds, installs, creates user, runs archivebox init

https://claude.ai/code/session_01Vx1EsNrNySgsc8Y67dGzCn
2026-03-15 02:50:14 +00:00
Claude
f3fcc1584c
Restore Homebrew and Debian package manager support
- Add Homebrew formula (brew_dist/archivebox.rb) using virtualenv pattern
  with auto-generation via homebrew-pypi-poet in bin/build_brew.sh
- Add Debian packaging via nFPM (pkg/debian/) with thin .deb that pip-installs
  archivebox into /opt/archivebox/venv on postinstall
- Add build/release scripts: bin/{build,release}_{brew,deb}.sh
- Update CI workflows to build packages on release and test them
- Update README apt/brew install instructions with working commands
- Update bin/setup.sh to use .deb download instead of old Launchpad PPA

https://claude.ai/code/session_01Vx1EsNrNySgsc8Y67dGzCn
2026-03-15 00:19:50 +00:00
Claude
5e6ba0bfa5
Update Dockerfile, docker-compose.yml, and README for v0.9.0 plugin system overhaul
- Dockerfile: Fix Python version refs (3.14->3.13), update uv 0.5->0.6,
  fix double GPG dearmor for NodeSource key, fix trailing whitespace in
  playwright install, fix HEALTHCHECK to use localhost instead of
  admin.archivebox.localhost, fix multi-arch build missing space,
  remove stale GLOBAL_VENV comments, re-enable archivebox version check,
  update example FROM python:3.13-slim and pip install archivebox>=0.9.0
- docker-compose.yml: Remove deprecated SAVE_ARCHIVEDOTORG and
  LISTEN_HOST config, update CSRF_TRUSTED_ORIGINS to localhost,
  fix docker-compose -> docker compose in comments
- docker_entrypoint.sh: Fix unquoted PUID variable that could fail
  when unset (use ${PUID:-})
- README.md: Replace --setup with --install (matching actual CLI flag),
  update Python >=3.10 -> >=3.13, Node >=18 -> >=22, remove deprecated
  SAVE_* config options (SAVE_ARCHIVEDOTORG, SAVE_FAVICON, SAVE_WGET,
  SAVE_DOM), update build tool refs (pdm->uv), update job queue ref
  (Huey->orchestrator+supervisord), fix Django version refs (5.1->6.0),
  fix daphne link typo, fix archivebox setup -> install, simplify pip
  install instructions

https://claude.ai/code/session_01X2H7XLawCzLGnrxMArXtVZ
2026-03-15 00:17:10 +00:00
Nick Sweeting
7ceaeae2d9
rename archive_org to archivedotorg, add BinaryWorker, fix config pass-through 2026-01-04 22:38:15 -08:00
Nick Sweeting
456aaee287
more migration id/uuid and config propagation fixes
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Debian package / build (push) Waiting to run
Deploy static content to Pages / deploy (push) Waiting to run
Build Homebrew package / build (push) Waiting to run
Build GitHub Pages website / build (push) Waiting to run
Build GitHub Pages website / deploy (push) Blocked by required conditions
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2026-01-04 16:16:26 -08:00
Nick Sweeting
65ee09ceab
move tests into subfolder, add missing install hooks 2026-01-02 00:22:07 -08:00
Nick Sweeting
60422adc87
fix orchestrator statemachine and Process from archiveresult migrations 2026-01-01 16:43:02 -08:00
Nick Sweeting
a04e4a7345
cleanup migrations, json, jsonl 2025-12-31 15:36:43 -08:00
Nick Sweeting
9487f8a0de
add ci for parallel tests 2025-12-29 02:39:24 -08:00
Nick Sweeting
f0aa19fa7d
wip
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Debian package / build (push) Waiting to run
Deploy static content to Pages / deploy (push) Waiting to run
Build Homebrew package / build (push) Waiting to run
Build GitHub Pages website / build (push) Waiting to run
Build GitHub Pages website / deploy (push) Blocked by required conditions
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2025-12-28 17:51:54 -08:00
Nick Sweeting
4ccb0863bb
continue renaming extractor to plugin, add plan for hook concurrency, add chrome kill helper script 2025-12-28 05:29:24 -08:00
Claude
ae2ab5b273
Add Python 3.13 support with uuid7 backport compatibility
- Create uuid_compat.py module that provides uuid7 for Python <3.14
  using uuid_extensions package, and native uuid.uuid7 for Python 3.14+
- Update all model files and migrations to use archivebox.uuid_compat
- Add uuid7 conditional dependency in pyproject.toml for Python <3.14
- Update requires-python to >=3.13 (from >=3.14)
- Update GitHub workflows, lock_pkgs.sh to use Python 3.13
- Update tool configs (ruff, pyright, uv) for Python 3.13

This enables running ArchiveBox on Python 3.13 while maintaining
forward compatibility with Python 3.14's native uuid7 support.
2025-12-27 01:07:30 +00:00
Claude
cff4077c23
Bump Python version requirement from 3.11 to 3.14
Update all references to Python 3.11 to use Python 3.14 to match
the pyproject.toml requires-python = ">=3.14" setting:

- bin/lock_pkgs.sh: uv venv --python 3.14
- .github/workflows/test.yml: python matrix and PDM version
- .github/workflows/pip.yml: PYTHON_VERSION env var
- Dockerfile: comment and example FROM line
- Issue templates: example version output
2025-12-27 00:30:27 +00:00
Philip Crockett
ba6a8c2da5 support XDG standard, search for chrome and chromium DBs 2025-02-18 21:38:52 +01:00
Philip Crockett
639aa7242b fix typo 2025-02-18 21:22:52 +01:00
Phil Crockett
9fbc2d3818 fix chrome browser history export on Linux 2025-02-18 21:08:56 +01:00
Phil Crockett
58bf8d07e1 feat(export_browser_history): add linux support for firefox 2025-02-16 10:24:37 +01:00
Phil Crockett
feded9e3d4 fix(export_browser_history): fix sqlite quote syntax error 2025-02-16 10:24:13 +01:00
Phil Crockett
2e1ac0409d feat(export_browser_history): fail script when errors occur 2025-02-16 08:34:41 +01:00
Phil Crockett
2ff3fc434e feat(export_browser_history): basic arg parsing error message 2025-02-16 08:31:21 +01:00
Phil Crockett
0043b59bc8 fix(export_browser_history): tilde doesn't expand in quotes 2025-02-16 08:22:17 +01:00
Nick Sweeting
839016b4b1
get docker image down to 630mb 2024-12-18 05:03:52 -08:00
Nick Sweeting
54d4d7f640
bring image back down to 700mb 2024-12-18 04:43:53 -08:00
Nick Sweeting
098573705b
clean up Dockerfile 2024-12-18 02:24:21 -08:00
Nick Sweeting
0ad1bdaf93
remove old deprecated bin/archive entrypoint 2024-12-18 01:31:38 -05:00
Nick Sweeting
6b3e297db8
fix lock_pkgs.sh version parsing and python version 2024-12-12 21:41:45 -08:00
Nick Sweeting
5d3c2a8a99
Update docker_entrypoint.sh 2024-11-03 17:02:36 -08:00
Nick Sweeting
99ed97836f
Prevent accidentally mounting home folder as DATA_DIR 2024-11-03 16:40:23 -08:00
Nick Sweeting
fd89de5b91
Update setup.sh 2024-11-03 16:15:09 -08:00
Nick Sweeting
63bf902f35
chown TMP_DIR and LIB_DIR recursively inside docker_entrypoint to handle custom PUID 2024-10-23 23:28:11 -07:00