Commit Graph

238 Commits

Author SHA1 Message Date
Nick Sweeting
73ec255855
fix: harden admin metadata surfaces 2026-06-13 23:44:59 -07:00
Nick Sweeting
98aecfaf91
release: archivebox 0.9.35rc8 2026-06-09 23:12:22 -07:00
Nick Sweeting
4fa90e484a
release: archivebox 0.9.34rc71 2026-06-07 20:51:28 -07:00
Nick Sweeting
232de59e95
Fix ArchiveBox Docker runtime layering 2026-06-05 02:03:12 -07:00
Nick Sweeting
7dd738b5b7
release: archivebox 0.9.34rc37
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Debian package / build (amd64) (push) Waiting to run
Build Debian package / build (arm64) (push) Waiting to run
Build Debian package / test (amd64, ubuntu-24.04) (push) Blocked by required conditions
Build Debian package / test (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
Build Debian package / release (push) Blocked by required conditions
Build Docker image / build ${{ matrix.platform }} (digest-linux-amd64, docker-amd64, linux/amd64, ubuntu-24.04) (push) Waiting to run
Build Docker image / build ${{ matrix.platform }} (digest-linux-arm64, docker-arm64, linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build Docker image / publish multiarch tags (push) Blocked by required conditions
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Release State / release-state (push) Waiting to run
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Parallel Tests / ${{ matrix.plugin.name }} (push) Blocked by required conditions
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2026-06-01 21:44:23 -07:00
Nick Sweeting
bcfb8c12a5
release: archivebox 0.9.34rc35 2026-06-01 19:35:35 -07:00
Nick Sweeting
e1c84e7d26
release: archivebox 0.9.34rc32 2026-06-01 18:10:44 -07:00
Nick Sweeting
62fb2c8659
release: archivebox 0.9.34rc30 2026-06-01 17:36:52 -07:00
Nick Sweeting
5c3161a5c1
release: archivebox 0.9.34rc29 2026-06-01 17:22:59 -07:00
Nick Sweeting
c075d654d8
Consolidate runtime config handling 2026-06-01 15:03:40 -07:00
Nick Sweeting
cab05eb1c6
Refactor plugins search progress and config flows 2026-06-01 00:08:27 -07:00
Nick Sweeting
7062f9f2e5
fix: derive binary provider runtime paths 2026-05-31 13:27:22 -07:00
Nick Sweeting
9bcba41b58
release: archivebox 0.9.33rc58 2026-05-31 04:38:45 -07:00
Nick Sweeting
deb0940f5f
fix(snapshots): honor SNAPSHOTS_PER_PAGE without silent clamping; bump default to 50
admin_snapshots.py:571 had min(max(50, SNAPSHOTS_PER_PAGE), 500), and
admin_archiveresults.py:501 had min(max(5, SNAPSHOTS_PER_PAGE), 5000).
Both clamps silently overrode the configured value — a documented
default of 40 was inaccessible in the Snapshot admin, and the
ArchiveResult admin also reused the same setting without being mentioned
in the docs.

- Drop both clamps; admin changelists now use SNAPSHOTS_PER_PAGE as-is.
- Bump the default in common.py from 40 to 50 (matches what users were
  actually seeing in the admin under the old floor).
- Add ge=1 validation so non-positive values are rejected at config
  parse time instead of producing broken pagination.
- Update Configuration.md: new default 50, clarify the option drives
  both Snapshot and ArchiveResult admin changelists plus the public
  index, and that it must be >= 1.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-31 03:02:34 -07:00
Nick Sweeting
76c8ca4ca8
refactor: remove ENABLED_PLUGINS config, use PLUGINS as the single plugin selector
ENABLED_PLUGINS and PLUGINS were two near-identical config keys: PLUGINS
was the CLI/per-run whitelist (--plugins flag, runner config), while
ENABLED_PLUGINS was the UI/API "persisted enabled set" — but both ended
up steering the same plugin resolution. Consolidating on PLUGINS as the
single source of truth.

- archivebox/config/common.py: drop the ENABLED_PLUGINS Field entirely
  (no alias, no compat shim — the migration is one-shot).
- archivebox/hooks.py:get_enabled_plugins(): read PLUGINS instead of
  ENABLED_PLUGINS. Function name kept (describes the return value).
- archivebox/templates/core/add.html: admin "Add" form JS now writes to
  PLUGINS; help text updated to reference PLUGINS.

views.py:1302 and runner.py:585 already read/wrote PLUGINS; they're now
consistent with the resolver.

abx-dl is unaffected — it receives selected_plugins as a Python list
argument and never reads either config key.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-31 02:49:53 -07:00
Nick Sweeting
c1e792fee6
release: v0.9.33rc52 2026-05-31 01:24:43 -07:00
Nick Sweeting
b08f70e510
release: archivebox 0.9.33rc51 2026-05-30 22:37:06 -07:00
Nick Sweeting
5a38193f56
release: archivebox 0.9.33rc50 2026-05-30 22:27:28 -07:00
Nick Sweeting
6ce2555dfd
fix: rename utils.py → util.py across modules, fix add --index-only, misc cleanups
Renames (no functional change, just consistency with the rest of the codebase):
- cli/cli_utils.py → cli/cli_util.py
- core/host_utils.py → core/host_util.py
- core/tag_utils.py → core/tag_util.py
- crawls/schedule_utils.py → crawls/schedule_util.py
- machine/env_utils.py → machine/env_util.py

Functional fixes:
- archivebox add --index-only now materializes Snapshot rows synchronously
  via crawl.create_snapshots_from_urls() instead of just queueing the Crawl
  and leaving the index empty. The previous behavior broke every test that
  expected --index-only to populate the index, since the runner is never
  started in index-only mode.
- config/collection.py: add _coerce_from_str_dict as the inverse of
  _coerce_to_str_dict so JSON-encoded INI values are decoded back to native
  dict/list types when mirrored into Machine.config (a JSONField). Without
  this, downstream consumers like MachineEvent / abx-dl get raw JSON
  strings where they expect dicts.

Plus matching admin / middleware / model touch-ups, the registration
password_change_form template, and assorted small cleanups the user
worked through while validating the deploy path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-30 14:30:33 -07:00
Nick Sweeting
0373f73fff
release: archivebox 0.9.33rc47
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Debian package / build (amd64) (push) Waiting to run
Build Debian package / build (arm64) (push) Waiting to run
Build Debian package / test (amd64, ubuntu-24.04) (push) Blocked by required conditions
Build Debian package / test (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
Build Debian package / release (push) Blocked by required conditions
Build Docker image / build ${{ matrix.platform }} (digest-linux-amd64, docker-amd64, linux/amd64, ubuntu-24.04) (push) Waiting to run
Build Docker image / build ${{ matrix.platform }} (digest-linux-arm64, docker-arm64, linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build Docker image / publish multiarch tags (push) Blocked by required conditions
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Release State / release-state (push) Waiting to run
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Parallel Tests / ${{ matrix.plugin.name }} (push) Blocked by required conditions
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2026-05-30 06:08:53 -07:00
Nick Sweeting
a0d47808c5
release: archivebox 0.9.33rc46 2026-05-30 05:58:42 -07:00
Nick Sweeting
383e4b5c6e
release: archivebox 0.9.33rc45 2026-05-30 05:42:53 -07:00
Nick Sweeting
b0a47e8bf5
wip: snapshot live progress, universal --init, runner perms, supervisord SIGINT
- Snapshot detail page: embed scoped live-progress monitor (same-origin
  /progress.json on whichever host the page is served from); hide admin
  action buttons when scoped; per-snapshot perms via can_view_snapshot.
- crawl_file API: respect crawl-level permissions; PUBLIC/UNLISTED served
  to guests, PRIVATE returns 404 for non-admin/non-owner.
- CrawlRunner: replace allow_paused_snapshot_maintenance with
  allow_maintenance_on_inactive_crawl so SEALED crawls don't short-circuit
  the cancellation guard for legitimate maintenance hooks (search backend
  backfill, fs migration, etc.). Fixes infinite STARTED loop on snapshots
  with queued search_backend results.
- Universal `--init` flag: works on any subcommand (server, update, add,
  shell, install, ...). Detected at module load, stripped from argv, and
  consumed in the dispatcher so subprocesses inherit a clean env.
- supervisord_util.run_runner_worker: route Ctrl+C through
  supervisor.signalProcess(name, "SIGINT") instead of raw os.kill on a
  cached pid, gated on statename=RUNNING. Prevents killing unrelated
  processes when the worker's pid has been reused by the OS.
- Login page: remove non-functional password-reset links; add
  has_real_admin_users template tag to gate the bootstrap hint.
- Add page: hide underline on the "Get the extension" link.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-30 04:45:15 -07:00
Nick Sweeting
c750084c3c
release: archivebox 0.9.33rc37 2026-05-29 02:50:04 -07:00
Nick Sweeting
881a858383
release: archivebox 0.9.33rc35 2026-05-28 20:23:01 -07:00
Nick Sweeting
f60b190182
fix: centralize ctrl-c rich logging 2026-05-28 08:49:17 -07:00
Nick Sweeting
66e5e06c7f
chore: checkpoint deploy loop changes 2026-05-28 07:12:56 -07:00
Nick Sweeting
cef1bafb6a
chore: checkpoint dev stack changes 2026-05-28 06:58:30 -07:00
Nick Sweeting
b1fa3de5a5
release: archivebox 0.9.33rc13 2026-05-28 06:27:57 -07:00
Nick Sweeting
f09c3a8e0d
release: archivebox 0.9.33rc9 2026-05-28 05:50:28 -07:00
Nick Sweeting
fbcc972441
backup: save in-progress dev changes 2026-05-28 05:20:52 -07:00
Nick Sweeting
09f7c8bba0
release: archivebox 0.9.32rc32 2026-05-27 14:50:49 -07:00
Nick Sweeting
e8bb2351d9
release: archivebox 0.9.32rc26 2026-05-27 13:23:23 -07:00
Nick Sweeting
fc539673ca
fix: use shared lib dir for docker installs
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Debian package / build (amd64) (push) Waiting to run
Build Debian package / build (arm64) (push) Waiting to run
Build Debian package / test (amd64, ubuntu-24.04) (push) Blocked by required conditions
Build Debian package / test (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
Build Debian package / release (push) Blocked by required conditions
Build Docker image / buildx (push) Waiting to run
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Release State / release-state (push) Waiting to run
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Parallel Tests / ${{ matrix.plugin.name }} (push) Blocked by required conditions
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2026-05-27 05:17:22 -07:00
Nick Sweeting
a34abb5c52
release: archivebox 0.9.32rc4 2026-05-27 03:03:04 -07:00
Nick Sweeting
30c841d477
Update dev docs and MHTML preview handling 2026-05-24 22:41:16 -07:00
Nick Sweeting
69a6bb360d
release: archivebox 0.9.31rc42
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Debian package / build (amd64) (push) Waiting to run
Build Debian package / build (arm64) (push) Waiting to run
Build Debian package / test (amd64, ubuntu-24.04) (push) Blocked by required conditions
Build Debian package / test (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
Build Debian package / release (push) Blocked by required conditions
Build Docker image / buildx (push) Waiting to run
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Release State / release-state (push) Waiting to run
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Parallel Tests / ${{ matrix.plugin.name }} (push) Blocked by required conditions
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2026-05-24 15:40:51 -07:00
Nick Sweeting
ec4eded38a
release: archivebox 0.9.31rc39 2026-05-24 13:25:16 -07:00
Nick Sweeting
687c27ab23
release: archivebox 0.9.31rc31 2026-05-24 03:07:01 -07:00
Nick Sweeting
3b8ce3f467
release: archivebox 0.9.31rc18 2026-05-23 17:32:58 -07:00
Nick Sweeting
ea4ce5b641
Prepare ArchiveBox 0.9.30rc81 demo release 2026-05-18 01:51:55 -07:00
Nick Sweeting
0cc4251c8a
Update ArchiveBox for abx dependency releases 2026-05-16 23:50:16 -07:00
Nick Sweeting
f3622d8cd3 update working changes
Some checks failed
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Has been cancelled
Build Debian package / build (amd64) (push) Has been cancelled
Build Debian package / build (arm64) (push) Has been cancelled
Build Docker image / buildx (push) Has been cancelled
Deploy static content to Pages / deploy (push) Has been cancelled
Build GitHub Pages website / build (push) Has been cancelled
Run linters / lint (push) Has been cancelled
Build Pip package / build (push) Has been cancelled
Release State / release-state (push) Has been cancelled
Parallel Tests / Discover test files (push) Has been cancelled
Parallel Tests / Plugin tests (push) Has been cancelled
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Has been cancelled
Run tests / docker_tests (push) Has been cancelled
Build Debian package / test (amd64, ubuntu-24.04) (push) Has been cancelled
Build Debian package / test (arm64, ubuntu-24.04-arm) (push) Has been cancelled
Build Debian package / release (push) Has been cancelled
Build GitHub Pages website / deploy (push) Has been cancelled
Parallel Tests / ${{ matrix.test.name }} (push) Has been cancelled
2026-03-25 05:36:07 -07:00
Nick Sweeting
ed1ddbc95e
Fix CI workflows and migration tests 2026-03-24 13:37:02 -07:00
Nick Sweeting
50286d3c38 Reuse cached binaries in archivebox runtime
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Docker image / buildx (push) Waiting to run
Deploy static content to Pages / deploy (push) Waiting to run
Build GitHub Pages website / build (push) Waiting to run
Build GitHub Pages website / deploy (push) Blocked by required conditions
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Release State / release-state (push) Waiting to run
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Parallel Tests / Plugin tests (push) Waiting to run
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2026-03-24 11:03:43 -07:00
Nick Sweeting
39450111dd
Update CI uv handling and runner changes
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Debian package / build (amd64) (push) Waiting to run
Build Debian package / build (arm64) (push) Waiting to run
Build Debian package / test (amd64, ubuntu-24.04) (push) Blocked by required conditions
Build Debian package / test (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
Build Debian package / release (push) Blocked by required conditions
Build Docker image / buildx (push) Waiting to run
Deploy static content to Pages / deploy (push) Waiting to run
Build GitHub Pages website / build (push) Waiting to run
Build GitHub Pages website / deploy (push) Blocked by required conditions
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Release State / release-state (push) Waiting to run
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Parallel Tests / Plugin tests (push) Waiting to run
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2026-03-23 13:27:23 -07:00
Nick Sweeting
b749b26c5d
wip 2026-03-23 03:58:32 -07:00
Nick Sweeting
268856bcfb Preserve common config console handling after rebase 2026-03-22 20:25:53 -07:00
Nick Sweeting
f400a2cd67
WIP: checkpoint working tree before rebasing onto dev 2026-03-22 20:25:18 -07:00
Nick Sweeting
ad41b15581
Add configurable server security modes 2026-03-15 23:34:40 -07:00