Commit Graph

5574 Commits

Author SHA1 Message Date
Nick Sweeting
24c123e331
ci: validate archivebox docker image metadata 2026-06-14 07:58:06 -07:00
Nick Sweeting
3596325733
fix: keep docker init cache out of archivebox home 2026-06-14 02:51:51 -07:00
Nick Sweeting
8baec62daa
fix: keep docker build cache out of archivebox home 2026-06-14 02:47:00 -07:00
Nick Sweeting
27d39749bf
release: archivebox 0.9.35rc32
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Docker image / build ${{ matrix.platform }} (digest-linux-amd64, docker-amd64, linux/amd64, ubuntu-24.04) (push) Waiting to run
Build Docker image / build ${{ matrix.platform }} (digest-linux-arm64, docker-arm64, linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build Docker image / publish multiarch tags (push) Blocked by required conditions
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Release State / release-state (push) Waiting to run
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Parallel Tests / ${{ matrix.plugin.name }} (push) Blocked by required conditions
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
Update Homebrew tap / dispatch (push) Waiting to run
2026-06-14 02:07:03 -07:00
Nick Sweeting
9a6d1b0410
fix: wait for abx-dl docker base in dev release 2026-06-14 01:43:20 -07:00
Nick Sweeting
3adb214b42
release: archivebox 0.9.35rc31 2026-06-14 01:39:23 -07:00
Nick Sweeting
d3ee9512e5
fix: pin archivebox docker base to abx-dl dependency 2026-06-14 01:07:24 -07:00
Nick Sweeting
12deb40daa
fix: keep docker install caches out of runtime image 2026-06-14 00:52:58 -07:00
Nick Sweeting
fa5c918c2f
release: archivebox 0.9.35rc30 2026-06-14 00:43:10 -07:00
Nick Sweeting
11c9f8c163
fix: improve snapshot preview rendering 2026-06-14 00:16:21 -07:00
Nick Sweeting
18e3580dcd
fix: harden reflected metadata rendering 2026-06-14 00:06:35 -07:00
Nick Sweeting
6ae56b7366
release: archivebox 0.9.35rc28 2026-06-13 23:53:45 -07:00
Nick Sweeting
73ec255855
fix: harden admin metadata surfaces 2026-06-13 23:44:59 -07:00
Nick Sweeting
faf2109642
test: tighten tag field form assertions 2026-06-13 23:21:24 -07:00
Nick Sweeting
1dbde4776f
release: archivebox 0.9.35rc27 2026-06-13 23:12:36 -07:00
Nick Sweeting
c19b8de5d6
Update test_cli_recursive_crawl depth assertions to depth=0 input convention 2026-06-13 21:05:17 -07:00
Nick Sweeting
87dbff486d
Update direct URL snapshot depth assertion to depth=0 2026-06-13 20:43:39 -07:00
Nick Sweeting
a48ce6871b
Make per-snapshot crawl seal idempotent against the run_snapshot race
run_snapshot calls load_snapshot_payload which replaces self.crawl with
a fresh Crawl model instance every time. When N snapshot tasks finish
near-simultaneously their independent self.crawl/SM pairs all see
'STARTED + no open snapshots' and race on sm.seal(). The first task
drives the SM to a final state (engine.running becomes False), but the
loser's current_state still reads STARTED off its stale model field,
so python-statemachine then raises 'Can't Seal when in Started.' and
fails the whole snapshot task even though the crawl is already sealed.

Refresh the row right before the call so the guard sees the committed
status, and treat TransitionNotAllowed as the expected idempotent
no-op when another task already drove the transition.
2026-06-13 20:17:30 -07:00
Nick Sweeting
8ba658b4dc
Fix two more depth assertions missed in earlier alignment 2026-06-13 19:12:30 -07:00
Nick Sweeting
824af25371
Align remaining tests with depth=0 input-layer convention
Reverts 6a635d3c's piecewise depth-shift in test_recursive_crawl.py
plus matching test_cli_add tagged-URL, test_api_v1_cli_workflow,
test_config_MAX_limits assertions. Direct URLs land at snapshot.depth=0
and max_depth==depth as the single user-facing contract; only stdin/
import text adds the +1 hop for the synthetic archivebox://internal
root.
2026-06-13 18:50:54 -07:00
Nick Sweeting
cfdd1293e9
Use archivebox install in the Dockerfile so archivebox-only plugins ship preinstalled
abx-dl install defaults to ABX_RUNTIME=abx-dl, which makes
discover_plugins() filter out any plugin whose config.json declares
"x-runtimes": ["archivebox"] (opencode, search_backend_sonic,
search_backend_sqlite). Running 'archivebox install' here uses
discover_plugins(runtime="archivebox") so those plugin binaries
(notably opencode) land in LIB_DIR at image build time instead of
trying to install on the first /admin/agent request.
2026-06-13 18:23:36 -07:00
Nick Sweeting
54dae36aec
release: archivebox 0.9.35rc26
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Docker image / build ${{ matrix.platform }} (digest-linux-amd64, docker-amd64, linux/amd64, ubuntu-24.04) (push) Waiting to run
Build Docker image / build ${{ matrix.platform }} (digest-linux-arm64, docker-arm64, linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build Docker image / publish multiarch tags (push) Blocked by required conditions
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Release State / release-state (push) Waiting to run
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Parallel Tests / ${{ matrix.plugin.name }} (push) Blocked by required conditions
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
Update Homebrew tap / dispatch (push) Waiting to run
2026-06-13 18:11:04 -07:00
Nick Sweeting
e547abbf27
Align direct URL Crawl flow with historical depth=0 convention
archivebox add and other entry points now seed Crawl.urls as
CrawlSeed JSONL at depth=0 (the input layer) with max_depth=depth
for direct URLs and depth+1 only for stdin/import text where the
synthetic archivebox://internal root lives at depth=0. The runner
also accepts one plain URL per line for ORM/crawl-create/schedule
callers so every Crawl row goes through the same expansion path
without scattering CrawlSeed knowledge across the codebase.

Tests updated to match restored convention.
2026-06-13 18:04:45 -07:00
Nick Sweeting
22453418fb
release: archivebox 0.9.35rc25 2026-06-13 17:19:18 -07:00
Nick Sweeting
c9e63ccffd
Centralize synthetic root snapshot creation in CrawlRunner
Direct URL inputs from CLI/UI/API now seed Crawl.urls as explicit
{type:CrawlSeed,url,depth} JSONL rows; raw stdin/UI/API import text
stays verbatim. The runner's create_initial_snapshots() is now the
single place that either expands seed rows or creates the synthetic
archivebox://internal root + staticfile/stdin.txt, so add paths no
longer perform DB/FS side effects and the parser hooks run through
the same Snapshot lifecycle as every other extractor.
2026-06-13 17:10:06 -07:00
Nick Sweeting
6b6ac24307
Specify version in installation command for ArchiveBox
Some checks failed
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Has been cancelled
Build Docker image / build ${{ matrix.platform }} (digest-linux-amd64, docker-amd64, linux/amd64, ubuntu-24.04) (push) Has been cancelled
Build Docker image / build ${{ matrix.platform }} (digest-linux-arm64, docker-arm64, linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Run linters / lint (push) Has been cancelled
Build Pip package / build (push) Has been cancelled
Release State / release-state (push) Has been cancelled
Parallel Tests / Discover test files (push) Has been cancelled
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Has been cancelled
Run tests / docker_tests (push) Has been cancelled
Update Homebrew tap / dispatch (push) Has been cancelled
Build Docker image / publish multiarch tags (push) Has been cancelled
Parallel Tests / ${{ matrix.test.name }} (push) Has been cancelled
Parallel Tests / ${{ matrix.plugin.name }} (push) Has been cancelled
Updated installation command to specify version 0.9.5rc1.
2026-06-12 11:36:56 -04:00
Nick Sweeting
6a635d3cd6
Fix UI add direct URL runner flow
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Docker image / build ${{ matrix.platform }} (digest-linux-amd64, docker-amd64, linux/amd64, ubuntu-24.04) (push) Waiting to run
Build Docker image / build ${{ matrix.platform }} (digest-linux-arm64, docker-arm64, linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build Docker image / publish multiarch tags (push) Blocked by required conditions
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Release State / release-state (push) Waiting to run
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Parallel Tests / ${{ matrix.plugin.name }} (push) Blocked by required conditions
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
Update Homebrew tap / dispatch (push) Waiting to run
2026-06-11 09:28:28 -07:00
Nick Sweeting
7e4dd2ac2d
Preserve queued hook validation for maintenance
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Docker image / build ${{ matrix.platform }} (digest-linux-amd64, docker-amd64, linux/amd64, ubuntu-24.04) (push) Waiting to run
Build Docker image / build ${{ matrix.platform }} (digest-linux-arm64, docker-arm64, linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build Docker image / publish multiarch tags (push) Blocked by required conditions
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Release State / release-state (push) Waiting to run
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Parallel Tests / ${{ matrix.plugin.name }} (push) Blocked by required conditions
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
Update Homebrew tap / dispatch (push) Waiting to run
2026-06-11 08:45:57 -07:00
Nick Sweeting
29cf3fe4ee
Align limit tests with direct snapshot depth 2026-06-11 08:18:36 -07:00
Nick Sweeting
14d43c88f5
Repair stale binaries in version output 2026-06-11 07:52:05 -07:00
Nick Sweeting
33436ae646
Run queued maintenance plugins explicitly 2026-06-11 07:26:00 -07:00
Nick Sweeting
6e0f2a41a3
Fix add input routing and plugin dependencies 2026-06-11 07:06:41 -07:00
Nick Sweeting
b6921d5e03
Restore direct URL add snapshots 2026-06-11 06:21:46 -07:00
Nick Sweeting
f6c98b67d7
Limit stale binary checks to version output 2026-06-11 01:32:30 -07:00
Nick Sweeting
24caacdd1d
Invalidate stale binary paths after lib moves 2026-06-11 01:16:02 -07:00
Nick Sweeting
426e23f33c
Prevent deploy exec from consuming script input
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Docker image / build ${{ matrix.platform }} (digest-linux-amd64, docker-amd64, linux/amd64, ubuntu-24.04) (push) Waiting to run
Build Docker image / build ${{ matrix.platform }} (digest-linux-arm64, docker-arm64, linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build Docker image / publish multiarch tags (push) Blocked by required conditions
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Release State / release-state (push) Waiting to run
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Parallel Tests / ${{ matrix.plugin.name }} (push) Blocked by required conditions
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
Update Homebrew tap / dispatch (push) Waiting to run
2026-06-11 01:02:09 -07:00
Nick Sweeting
e6091cf50e
Capture deploy version warnings 2026-06-11 00:59:55 -07:00
Nick Sweeting
77500bfac8
Fix dev deploy image selection 2026-06-11 00:56:28 -07:00
Nick Sweeting
2e9be6751e
release: archivebox 0.9.35rc24 2026-06-11 00:48:08 -07:00
Nick Sweeting
bb598b9710
release: archivebox 0.9.35rc23 2026-06-11 00:41:34 -07:00
Nick Sweeting
283f0afdb5
release: archivebox 0.9.35rc22 2026-06-11 00:15:52 -07:00
Nick Sweeting
9d5ed1bc42
release: archivebox 0.9.35rc21 2026-06-11 00:08:54 -07:00
Nick Sweeting
585c951aa7
release: archivebox 0.9.35rc20 2026-06-11 00:00:21 -07:00
Nick Sweeting
635e318647
Use PyPI version endpoint in release wait 2026-06-10 23:45:16 -07:00
Nick Sweeting
1da4b79e9d
Make dev stack release resumable after PyPI lag 2026-06-10 23:43:27 -07:00
Nick Sweeting
e63552ad1d
release: archivebox 0.9.35rc19 2026-06-10 23:02:37 -07:00
Nick Sweeting
dfc0c66fc0
release: archivebox 0.9.35rc18 2026-06-10 22:19:59 -07:00
Nick Sweeting
cd959844ca
release: archivebox 0.9.35rc17 2026-06-10 21:56:32 -07:00
Nick Sweeting
0920431b53
Fix discovered snapshot plugin selection 2026-06-10 21:34:44 -07:00
Nick Sweeting
7c51542335
release: archivebox 0.9.35rc16 2026-06-10 21:11:33 -07:00