Nick Sweeting
9ee0dcf165
fix: address orchestration review findings
2026-09-02 12:56:23 -07:00
Nick Sweeting
2bba758314
refactor: simplify snapshot maintenance orchestration
2026-09-02 12:05:09 -07:00
Nick Sweeting
663017924b
refactor: consume canonical abx-dl execution plans
2026-09-02 05:23:11 -07:00
Nick Sweeting
e7b3f1c485
Adopt canonical abx-dl execution plan API
2026-09-02 04:44:55 -07:00
Nick Sweeting
aaea35f9fd
Preserve lifecycle compatibility after runtime unification
2026-09-01 12:39:58 -07:00
Nick Sweeting
f2ce574a1c
Exit foreground add after server finishes crawl
2026-08-19 02:23:53 -07:00
Nick Sweeting
43d2a8b106
Reject crawls before low-memory startup
2026-08-01 12:38:36 -07:00
Nick Sweeting
43f7af8799
Claim foreground crawl ownership before enqueue
2026-07-29 20:29:11 -07:00
Nick Sweeting
eacfae5d6c
Run all ArchiveBox tests through unified CI
2026-07-24 00:50:40 -07:00
Nick Sweeting
6592af0d19
Exercise ArchiveBox through unified runtime lifecycles
2026-07-21 19:26:42 -07:00
Nick Sweeting
8b57085827
chore: commit local archivebox changes
2026-06-14 11:44:38 -07:00
Nick Sweeting
1dbde4776f
release: archivebox 0.9.35rc27
2026-06-13 23:12:36 -07:00
Nick Sweeting
e547abbf27
Align direct URL Crawl flow with historical depth=0 convention
...
archivebox add and other entry points now seed Crawl.urls as
CrawlSeed JSONL at depth=0 (the input layer) with max_depth=depth
for direct URLs and depth+1 only for stdin/import text where the
synthetic archivebox://internal root lives at depth=0. The runner
also accepts one plain URL per line for ORM/crawl-create/schedule
callers so every Crawl row goes through the same expansion path
without scattering CrawlSeed knowledge across the codebase.
Tests updated to match restored convention.
2026-06-13 18:04:45 -07:00
Nick Sweeting
c9e63ccffd
Centralize synthetic root snapshot creation in CrawlRunner
...
Direct URL inputs from CLI/UI/API now seed Crawl.urls as explicit
{type:CrawlSeed,url,depth} JSONL rows; raw stdin/UI/API import text
stays verbatim. The runner's create_initial_snapshots() is now the
single place that either expands seed rows or creates the synthetic
archivebox://internal root + staticfile/stdin.txt, so add paths no
longer perform DB/FS side effects and the parser hooks run through
the same Snapshot lifecycle as every other extractor.
2026-06-13 17:10:06 -07:00
Nick Sweeting
6a635d3cd6
Fix UI add direct URL runner flow
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Docker image / build ${{ matrix.platform }} (digest-linux-amd64, docker-amd64, linux/amd64, ubuntu-24.04) (push) Waiting to run
Build Docker image / build ${{ matrix.platform }} (digest-linux-arm64, docker-arm64, linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build Docker image / publish multiarch tags (push) Blocked by required conditions
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Release State / release-state (push) Waiting to run
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Parallel Tests / ${{ matrix.plugin.name }} (push) Blocked by required conditions
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
Update Homebrew tap / dispatch (push) Waiting to run
2026-06-11 09:28:28 -07:00
Nick Sweeting
b6921d5e03
Restore direct URL add snapshots
2026-06-11 06:21:46 -07:00
Nick Sweeting
2bfb3ad4eb
release: archivebox 0.9.35rc5
2026-06-09 22:09:28 -07:00
Nick Sweeting
a1449c2822
Use plugin URL patterns for source imports
2026-06-08 23:40:25 -07:00
Nick Sweeting
5adec53b4c
Expand add flow runtime handling
2026-06-08 23:27:08 -07:00
Nick Sweeting
4fa90e484a
release: archivebox 0.9.34rc71
2026-06-07 20:51:28 -07:00
Nick Sweeting
e4ec848da8
fix: keep background add crawls runnable
2026-06-02 18:53:59 -07:00
Nick Sweeting
39eac65ed0
Stabilize frozen config CLI test flows
...
(cherry picked from commit 2bca869e32
2026-06-02 18:44:57 -07:00
Nick Sweeting
96437e1ffd
Publish local ArchiveBox changes
2026-06-02 02:25:52 -07:00
Nick Sweeting
c075d654d8
Consolidate runtime config handling
2026-06-01 15:03:40 -07:00
Nick Sweeting
453d998e7d
fix: schedule background admin crawls
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Debian package / build (amd64) (push) Waiting to run
Build Debian package / build (arm64) (push) Waiting to run
Build Debian package / test (amd64, ubuntu-24.04) (push) Blocked by required conditions
Build Debian package / test (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
Build Debian package / release (push) Blocked by required conditions
Build Docker image / build ${{ matrix.platform }} (digest-linux-amd64, docker-amd64, linux/amd64, ubuntu-24.04) (push) Waiting to run
Build Docker image / build ${{ matrix.platform }} (digest-linux-arm64, docker-arm64, linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build Docker image / publish multiarch tags (push) Blocked by required conditions
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Release State / release-state (push) Waiting to run
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Parallel Tests / ${{ matrix.plugin.name }} (push) Blocked by required conditions
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2026-06-01 10:44:21 -07:00
Nick Sweeting
cab05eb1c6
Refactor plugins search progress and config flows
2026-06-01 00:08:27 -07:00
Nick Sweeting
83d5161b3e
release: v0.9.33rc51
2026-05-31 01:14:40 -07:00
Nick Sweeting
5a38193f56
release: archivebox 0.9.33rc50
2026-05-30 22:27:28 -07:00
Nick Sweeting
6ce2555dfd
fix: rename utils.py → util.py across modules, fix add --index-only, misc cleanups
...
Renames (no functional change, just consistency with the rest of the codebase):
- cli/cli_utils.py → cli/cli_util.py
- core/host_utils.py → core/host_util.py
- core/tag_utils.py → core/tag_util.py
- crawls/schedule_utils.py → crawls/schedule_util.py
- machine/env_utils.py → machine/env_util.py
Functional fixes:
- archivebox add --index-only now materializes Snapshot rows synchronously
via crawl.create_snapshots_from_urls() instead of just queueing the Crawl
and leaving the index empty. The previous behavior broke every test that
expected --index-only to populate the index, since the runner is never
started in index-only mode.
- config/collection.py: add _coerce_from_str_dict as the inverse of
_coerce_to_str_dict so JSON-encoded INI values are decoded back to native
dict/list types when mirrored into Machine.config (a JSONField). Without
this, downstream consumers like MachineEvent / abx-dl get raw JSON
strings where they expect dicts.
Plus matching admin / middleware / model touch-ups, the registration
password_change_form template, and assorted small cleanups the user
worked through while validating the deploy path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-30 14:30:33 -07:00
Nick Sweeting
b0a47e8bf5
wip: snapshot live progress, universal --init, runner perms, supervisord SIGINT
...
- Snapshot detail page: embed scoped live-progress monitor (same-origin
/progress.json on whichever host the page is served from); hide admin
action buttons when scoped; per-snapshot perms via can_view_snapshot.
- crawl_file API: respect crawl-level permissions; PUBLIC/UNLISTED served
to guests, PRIVATE returns 404 for non-admin/non-owner.
- CrawlRunner: replace allow_paused_snapshot_maintenance with
allow_maintenance_on_inactive_crawl so SEALED crawls don't short-circuit
the cancellation guard for legitimate maintenance hooks (search backend
backfill, fs migration, etc.). Fixes infinite STARTED loop on snapshots
with queued search_backend results.
- Universal `--init` flag: works on any subcommand (server, update, add,
shell, install, ...). Detected at module load, stripped from argv, and
consumed in the dispatcher so subprocesses inherit a clean env.
- supervisord_util.run_runner_worker: route Ctrl+C through
supervisor.signalProcess(name, "SIGINT") instead of raw os.kill on a
cached pid, gated on statename=RUNNING. Prevents killing unrelated
processes when the worker's pid has been reused by the OS.
- Login page: remove non-functional password-reset links; add
has_real_admin_users template tag to gate the bootstrap hint.
- Add page: hide underline on the "Get the extension" link.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-30 04:45:15 -07:00
Nick Sweeting
c750084c3c
release: archivebox 0.9.33rc37
2026-05-29 02:50:04 -07:00
Nick Sweeting
881a858383
release: archivebox 0.9.33rc35
2026-05-28 20:23:01 -07:00
Nick Sweeting
031e956080
release: archivebox 0.9.33rc29
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Debian package / build (amd64) (push) Waiting to run
Build Debian package / build (arm64) (push) Waiting to run
Build Debian package / test (amd64, ubuntu-24.04) (push) Blocked by required conditions
Build Debian package / test (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
Build Debian package / release (push) Blocked by required conditions
Build Docker image / build ${{ matrix.platform }} (digest-linux-amd64, docker-amd64, linux/amd64, ubuntu-24.04) (push) Waiting to run
Build Docker image / build ${{ matrix.platform }} (digest-linux-arm64, docker-arm64, linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build Docker image / publish multiarch tags (push) Blocked by required conditions
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Release State / release-state (push) Waiting to run
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Parallel Tests / ${{ matrix.plugin.name }} (push) Blocked by required conditions
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2026-05-28 15:35:33 -07:00
Nick Sweeting
804be3075f
chore: checkpoint deploy loop changes
2026-05-28 07:41:56 -07:00
Nick Sweeting
6c5a64800b
chore: checkpoint runner deploy changes
2026-05-28 07:27:35 -07:00
Nick Sweeting
cef1bafb6a
chore: checkpoint dev stack changes
2026-05-28 06:58:30 -07:00
Nick Sweeting
6c926c6335
chore: apply rc15 release formatting
2026-05-28 06:46:07 -07:00
Nick Sweeting
fe09f5a517
chore: apply release hook fixes
2026-05-28 06:42:53 -07:00
Nick Sweeting
b2fa839788
chore: checkpoint dev stack deploy changes
2026-05-28 06:39:06 -07:00
Nick Sweeting
f09c3a8e0d
release: archivebox 0.9.33rc9
2026-05-28 05:50:28 -07:00
Nick Sweeting
fbcc972441
backup: save in-progress dev changes
2026-05-28 05:20:52 -07:00
Nick Sweeting
89a3119f2a
release: v0.9.32rc24
2026-05-27 12:40:49 -07:00
Nick Sweeting
69a6bb360d
release: archivebox 0.9.31rc42
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Debian package / build (amd64) (push) Waiting to run
Build Debian package / build (arm64) (push) Waiting to run
Build Debian package / test (amd64, ubuntu-24.04) (push) Blocked by required conditions
Build Debian package / test (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
Build Debian package / release (push) Blocked by required conditions
Build Docker image / buildx (push) Waiting to run
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Release State / release-state (push) Waiting to run
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Parallel Tests / ${{ matrix.plugin.name }} (push) Blocked by required conditions
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2026-05-24 15:40:51 -07:00
Nick Sweeting
ec4eded38a
release: archivebox 0.9.31rc39
2026-05-24 13:25:16 -07:00
Nick Sweeting
cadd3f517d
release: archivebox 0.9.31rc30
2026-05-24 02:05:13 -07:00
Nick Sweeting
c5d11c3224
Prevent foreground add runner races
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Debian package / build (amd64) (push) Waiting to run
Build Debian package / build (arm64) (push) Waiting to run
Build Debian package / test (amd64, ubuntu-24.04) (push) Blocked by required conditions
Build Debian package / test (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
Build Debian package / release (push) Blocked by required conditions
Build Docker image / buildx (push) Waiting to run
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Release State / release-state (push) Waiting to run
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Parallel Tests / ${{ matrix.plugin.name }} (push) Blocked by required conditions
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2026-05-18 09:39:27 -07:00
Nick Sweeting
0cc4251c8a
Update ArchiveBox for abx dependency releases
2026-05-16 23:50:16 -07:00
Nick Sweeting
b749b26c5d
wip
2026-03-23 03:58:32 -07:00
Nick Sweeting
f400a2cd67
WIP: checkpoint working tree before rebasing onto dev
2026-03-22 20:25:18 -07:00
Nick Sweeting
c87079aa0a
Refactor ArchiveBox onto abx-dl bus runner
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Has been cancelled
Build Debian package / build (amd64) (push) Has been cancelled
Build Debian package / build (arm64) (push) Has been cancelled
Build Docker image / buildx (push) Has been cancelled
Deploy static content to Pages / deploy (push) Has been cancelled
Build GitHub Pages website / build (push) Has been cancelled
Run linters / lint (push) Has been cancelled
Build Pip package / build (push) Has been cancelled
Parallel Tests / Discover test files (push) Has been cancelled
Parallel Tests / Plugin tests (push) Has been cancelled
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Has been cancelled
Run tests / docker_tests (push) Has been cancelled
Build Debian package / test (amd64, ubuntu-24.04) (push) Has been cancelled
Build Debian package / test (arm64, ubuntu-24.04-arm) (push) Has been cancelled
Build Debian package / release (push) Has been cancelled
Build GitHub Pages website / deploy (push) Has been cancelled
Parallel Tests / ${{ matrix.test.name }} (push) Has been cancelled
2026-03-21 11:47:57 -07:00
