Commit Graph

317 Commits

Author SHA1 Message Date
Nick Sweeting
fb9f4a724b
release: archivebox 0.9.31rc20
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Debian package / build (amd64) (push) Waiting to run
Build Debian package / build (arm64) (push) Waiting to run
Build Debian package / test (amd64, ubuntu-24.04) (push) Blocked by required conditions
Build Debian package / test (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
Build Debian package / release (push) Blocked by required conditions
Build Docker image / buildx (push) Waiting to run
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Release State / release-state (push) Waiting to run
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Parallel Tests / ${{ matrix.plugin.name }} (push) Blocked by required conditions
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2026-05-23 19:36:09 -07:00
Nick Sweeting
3b8ce3f467
release: archivebox 0.9.31rc18 2026-05-23 17:32:58 -07:00
Nick Sweeting
f9512d5ea2
release: archivebox 0.9.31rc17 2026-05-23 17:14:50 -07:00
Nick Sweeting
ebf597b898
release: archivebox 0.9.31rc16 2026-05-23 17:10:35 -07:00
Nick Sweeting
8f2ca42550
Avoid committing release dist artifacts 2026-05-23 17:00:57 -07:00
Nick Sweeting
e5dd4a0fad
Add dev stack release helper 2026-05-23 17:00:34 -07:00
Nick Sweeting
d263a611cc
Allow root with warnings 2026-05-23 16:49:20 -07:00
Nick Sweeting
009d338dad
Simplify standalone setup dependencies 2026-05-23 16:03:29 -07:00
Nick Sweeting
943d4ee64b
Simplify Docker runtime dependency install
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Debian package / build (amd64) (push) Waiting to run
Build Debian package / build (arm64) (push) Waiting to run
Build Debian package / test (amd64, ubuntu-24.04) (push) Blocked by required conditions
Build Debian package / test (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
Build Debian package / release (push) Blocked by required conditions
Build Docker image / buildx (push) Waiting to run
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Release State / release-state (push) Waiting to run
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Parallel Tests / ${{ matrix.plugin.name }} (push) Blocked by required conditions
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2026-05-23 15:51:53 -07:00
Nick Sweeting
5f69ec656d
Move Docker runtime lib dir 2026-05-23 15:44:03 -07:00
Nick Sweeting
0e527df929
Bake plugin runtime deps into Docker image 2026-05-23 15:37:42 -07:00
Nick Sweeting
811ef03982
Bump rc and fix demo deploy image override 2026-05-23 14:41:47 -07:00
Nick Sweeting
880aaa13c1
Drop stale docker mirror tag 2026-05-22 20:17:34 -07:00
Nick Sweeting
c1e9106b9a
Fix docker release platform default 2026-05-22 20:16:32 -07:00
Nick Sweeting
92339e08f0
Prepare dev demo deploy 2026-05-22 20:12:48 -07:00
Nick Sweeting
fa15e1edfa
Fix docker runtime and real crawl flows 2026-05-14 21:24:32 -07:00
Nick Sweeting
5f8098c632
fix recursive crawls and docker runtime 2026-05-13 08:13:33 -07:00
Nick Sweeting
5d60098618
fix: stabilize dockerized crawl runtime
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Debian package / build (amd64) (push) Waiting to run
Build Debian package / build (arm64) (push) Waiting to run
Build Debian package / test (amd64, ubuntu-24.04) (push) Blocked by required conditions
Build Debian package / test (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
Build Debian package / release (push) Blocked by required conditions
Build Docker image / buildx (push) Waiting to run
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Release State / release-state (push) Waiting to run
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Parallel Tests / Plugin tests (push) Waiting to run
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2026-05-13 02:05:07 -07:00
Nick Sweeting
7d8c468659
rename abx-pkg to abxpkg
Some checks failed
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Has been cancelled
Build Debian package / build (amd64) (push) Has been cancelled
Build Debian package / build (arm64) (push) Has been cancelled
Build Docker image / buildx (push) Has been cancelled
Deploy static content to Pages / deploy (push) Has been cancelled
Build GitHub Pages website / build (push) Has been cancelled
Run linters / lint (push) Has been cancelled
Build Pip package / build (push) Has been cancelled
Release State / release-state (push) Has been cancelled
Parallel Tests / Discover test files (push) Has been cancelled
Parallel Tests / Plugin tests (push) Has been cancelled
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Has been cancelled
Run tests / docker_tests (push) Has been cancelled
Build Debian package / test (amd64, ubuntu-24.04) (push) Has been cancelled
Build Debian package / test (arm64, ubuntu-24.04-arm) (push) Has been cancelled
Build Debian package / release (push) Has been cancelled
Build GitHub Pages website / deploy (push) Has been cancelled
Parallel Tests / ${{ matrix.test.name }} (push) Has been cancelled
2026-04-17 10:48:40 -07:00
Nick Sweeting
ee5685353b
rename abx-pkg to abxpkg 2026-04-17 10:37:10 -07:00
Nick Sweeting
f1287510ff
rename abxpkg 2026-04-17 10:36:19 -07:00
Nick Sweeting
f126c6e628
symlink lock_pkgs to setup monorepo script 2026-04-07 20:21:41 -07:00
Nick Sweeting
3e7b83ac91
bump versions
Some checks failed
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Has been cancelled
Build Debian package / build (amd64) (push) Has been cancelled
Build Debian package / build (arm64) (push) Has been cancelled
Build Docker image / buildx (push) Has been cancelled
Deploy static content to Pages / deploy (push) Has been cancelled
Build GitHub Pages website / build (push) Has been cancelled
Run linters / lint (push) Has been cancelled
Build Pip package / build (push) Has been cancelled
Release State / release-state (push) Has been cancelled
Parallel Tests / Discover test files (push) Has been cancelled
Parallel Tests / Plugin tests (push) Has been cancelled
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Has been cancelled
Run tests / docker_tests (push) Has been cancelled
Build Debian package / test (amd64, ubuntu-24.04) (push) Has been cancelled
Build Debian package / test (arm64, ubuntu-24.04-arm) (push) Has been cancelled
Build Debian package / release (push) Has been cancelled
Build GitHub Pages website / deploy (push) Has been cancelled
Parallel Tests / ${{ matrix.test.name }} (push) Has been cancelled
2026-04-04 23:10:12 -07:00
Nick Sweeting
f2c81142e1
tweak release script
Some checks failed
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Deploy static content to Pages / deploy (push) Waiting to run
Build GitHub Pages website / build (push) Waiting to run
Build GitHub Pages website / deploy (push) Blocked by required conditions
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Parallel Tests / Plugin tests (push) Waiting to run
Build Debian package / build (amd64) (push) Has been cancelled
Build Debian package / build (arm64) (push) Has been cancelled
Build Docker image / buildx (push) Has been cancelled
Run linters / lint (push) Has been cancelled
Build Pip package / build (push) Has been cancelled
Release State / release-state (push) Has been cancelled
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Has been cancelled
Run tests / docker_tests (push) Has been cancelled
Build Debian package / test (amd64, ubuntu-24.04) (push) Has been cancelled
Build Debian package / test (arm64, ubuntu-24.04-arm) (push) Has been cancelled
Build Debian package / release (push) Has been cancelled
2026-03-23 04:21:12 -07:00
Nick Sweeting
1d94645abd
test fixes 2026-03-23 04:12:31 -07:00
Nick Sweeting
b749b26c5d
wip 2026-03-23 03:58:32 -07:00
Nick Sweeting
f400a2cd67
WIP: checkpoint working tree before rebasing onto dev 2026-03-22 20:25:18 -07:00
Nick Sweeting
9de084da65
bump package versions 2026-03-15 20:47:28 -07:00
Nick Sweeting
4756697a17
Use ruff pyright and ty for linting 2026-03-15 19:43:59 -07:00
Nick Sweeting
934e02695b
fix lint 2026-03-15 18:45:29 -07:00
Nick Sweeting
70c9358cf9
Improve scheduling, runtime paths, and API behavior 2026-03-15 18:31:56 -07:00
Nick Sweeting
7d42c6c8b5
bump versions and fix docs 2026-03-15 17:43:07 -07:00
Nick Sweeting
9ef8a1b23a
Stabilize secret-backed plugin CI 2026-03-15 08:37:41 -07:00
Nick Sweeting
4fa701fafe
Update abx dependencies and plugin test harness 2026-03-15 04:37:32 -07:00
Nick Sweeting
c64e14c656
Merge branch 'dev' into claude/restore-package-managers-GlgBJ 2026-03-14 22:43:15 -07:00
Claude
36b4055304
Add caveats block to Homebrew formula showing data directory
The post_install initializes var/archivebox as the data directory,
but users need to know where it is for subsequent commands. The
caveats block is shown after brew install/upgrade to guide users.

https://claude.ai/code/session_01Vx1EsNrNySgsc8Y67dGzCn
2026-03-15 04:56:25 +00:00
Claude
fbde2dee03
Fix remaining PR review comments across packaging files
- preremove.sh: Stop the systemd service during upgrades (not just
  remove/purge) so the running process doesn't use stale venv binaries
  while postinstall replaces them. Only disable + remove venv on full
  removal.

- debian.yml: Fail loudly when release lookup fails during a release
  event (exit 1), but still skip gracefully for workflow_dispatch
  manual testing (exit 0). Prevents silently broken .deb publication.

- archivebox.rb + build_brew.sh + homebrew.yml: Add post_install that
  initializes ArchiveBox in var/archivebox with DATA_DIR set, using
  correct Homebrew system() syntax (no env hash as first arg).

https://claude.ai/code/session_01Vx1EsNrNySgsc8Y67dGzCn
2026-03-15 04:42:01 +00:00
Claude
82932812ae
Simplify deb/brew packages to thin wrappers around pip install
Remove all non-essential dependencies from both package formats.
The .deb now only depends on python3, pip, and venv. The brew
formula only depends on python@3.13. All other runtime deps
(node, chrome, yt-dlp, wget, ripgrep, etc.) are installed
on-demand by `archivebox install` at runtime.

Removed from brew formula:
- 6 depends_on entries (node, git, wget, curl, ripgrep, yt-dlp)
- on_linux block (pkg-config, openssl, libffi)
- post_install hook (was running archivebox install)
- service block (users run archivebox server directly)

Removed from .deb:
- 6 depends entries (nodejs, npm, git, wget, curl, ripgrep)
- recommends section (yt-dlp, ffmpeg, chromium)

Net: -126 lines across packaging files.

https://claude.ai/code/session_01Vx1EsNrNySgsc8Y67dGzCn
2026-03-15 03:56:39 +00:00
Claude
496b54a5e1
Fix remaining PR review comments: release ordering, verification, README
- Move .deb upload to GitHub Release into a separate job that runs after tests pass
- Fix workflow_call event propagation so release jobs run when called from release.yml
- Fix setup.sh post-install verification to check `which archivebox` first (works for brew/deb)
- Fix README.md: detect architecture with dpkg instead of hardcoding amd64
- Fix README.md: remove --setup flag from apt install instructions

https://claude.ai/code/session_01Vx1EsNrNySgsc8Y67dGzCn
2026-03-15 03:20:32 +00:00
Claude
7c7a9ee599
Fix PR review comments: service flags, DATA_DIR, version pinning, upgrade safety
- Remove --setup flag from systemd service and CI (not valid in 0.9.x)
- Remove release triggers from debian/homebrew workflows (handled by release.yml)
- Fix brew post_install to set DATA_DIR so it initializes in var/archivebox
- Add PATH export to deb wrapper script for bundled console scripts
- Remove pip install fallback in install.sh (strict version pinning)
- Guard preremove.sh cleanup to only run on remove/purge, not upgrade
- Initialize SDIST_URL/SDIST_SHA256 in build_brew.sh (nounset safety)
- Pin awalsh128/cache-apt-pkgs-action to v1.6.0 (supply chain safety)

https://claude.ai/code/session_01Vx1EsNrNySgsc8Y67dGzCn
2026-03-15 03:12:37 +00:00
Claude
4c113f8eb9
Fix CI: create tests/out dir, fix archivebox add cmd, revert setup.sh
- test-parallel.yml: mkdir -p tests/out before pytest --basetemp
  (fixes FileNotFoundError in chrome test fixture)
- debian.yml: fix archivebox add command (--parser url_list removed
  in 0.9.x), remove || true so failures are caught
- setup.sh: revert apt section to always use pip install, not .deb

https://claude.ai/code/session_01Vx1EsNrNySgsc8Y67dGzCn
2026-03-15 03:02:47 +00:00
Claude
c8f562ee37
Wire up GitHub Actions for deb/brew build, test, and release
- Fix debian.yml: pin nfpm version, add permissions, improve test job
  with user creation, init test, and status check
- Fix homebrew.yml: use PyPI JSON API (macOS-compatible, no grep -oP),
  wait for PyPI availability on release, use generated formula not template,
  add Linux (Linuxbrew) test job alongside macOS
- Add release.yml orchestrator: pip → deb + brew + docker in order
- Add workflow_call triggers to pip.yml and docker.yml
- Fix build_brew.sh: replace grep -oP with Python-based PyPI API,
  add on_linux deps (pkg-config, openssl, libffi)
- Fix setup.sh: use GitHub API to find correct .deb download URL
  (filename includes version number)
- Fix postinstall.sh: create archivebox system user, pin version from
  package, check for systemd before daemon-reload
- Fix preremove.sh: stop service before removal, check for systemd
- Fix install.sh: fallback to latest if pinned version not on PyPI
- Add on_linux deps to brew formula for Linuxbrew support
- Tested: .deb builds, installs, creates user, runs archivebox init

https://claude.ai/code/session_01Vx1EsNrNySgsc8Y67dGzCn
2026-03-15 02:50:14 +00:00
Claude
f3fcc1584c
Restore Homebrew and Debian package manager support
- Add Homebrew formula (brew_dist/archivebox.rb) using virtualenv pattern
  with auto-generation via homebrew-pypi-poet in bin/build_brew.sh
- Add Debian packaging via nFPM (pkg/debian/) with thin .deb that pip-installs
  archivebox into /opt/archivebox/venv on postinstall
- Add build/release scripts: bin/{build,release}_{brew,deb}.sh
- Update CI workflows to build packages on release and test them
- Update README apt/brew install instructions with working commands
- Update bin/setup.sh to use .deb download instead of old Launchpad PPA

https://claude.ai/code/session_01Vx1EsNrNySgsc8Y67dGzCn
2026-03-15 00:19:50 +00:00
Claude
5e6ba0bfa5
Update Dockerfile, docker-compose.yml, and README for v0.9.0 plugin system overhaul
- Dockerfile: Fix Python version refs (3.14->3.13), update uv 0.5->0.6,
  fix double GPG dearmor for NodeSource key, fix trailing whitespace in
  playwright install, fix HEALTHCHECK to use localhost instead of
  admin.archivebox.localhost, fix multi-arch build missing space,
  remove stale GLOBAL_VENV comments, re-enable archivebox version check,
  update example FROM python:3.13-slim and pip install archivebox>=0.9.0
- docker-compose.yml: Remove deprecated SAVE_ARCHIVEDOTORG and
  LISTEN_HOST config, update CSRF_TRUSTED_ORIGINS to localhost,
  fix docker-compose -> docker compose in comments
- docker_entrypoint.sh: Fix unquoted PUID variable that could fail
  when unset (use ${PUID:-})
- README.md: Replace --setup with --install (matching actual CLI flag),
  update Python >=3.10 -> >=3.13, Node >=18 -> >=22, remove deprecated
  SAVE_* config options (SAVE_ARCHIVEDOTORG, SAVE_FAVICON, SAVE_WGET,
  SAVE_DOM), update build tool refs (pdm->uv), update job queue ref
  (Huey->orchestrator+supervisord), fix Django version refs (5.1->6.0),
  fix daphne link typo, fix archivebox setup -> install, simplify pip
  install instructions

https://claude.ai/code/session_01X2H7XLawCzLGnrxMArXtVZ
2026-03-15 00:17:10 +00:00
Nick Sweeting
7ceaeae2d9
rename archive_org to archivedotorg, add BinaryWorker, fix config pass-through 2026-01-04 22:38:15 -08:00
Nick Sweeting
456aaee287
more migration id/uuid and config propagation fixes
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Debian package / build (push) Waiting to run
Deploy static content to Pages / deploy (push) Waiting to run
Build Homebrew package / build (push) Waiting to run
Build GitHub Pages website / build (push) Waiting to run
Build GitHub Pages website / deploy (push) Blocked by required conditions
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2026-01-04 16:16:26 -08:00
Nick Sweeting
65ee09ceab
move tests into subfolder, add missing install hooks 2026-01-02 00:22:07 -08:00
Nick Sweeting
60422adc87
fix orchestrator statemachine and Process from archiveresult migrations 2026-01-01 16:43:02 -08:00
Nick Sweeting
a04e4a7345
cleanup migrations, json, jsonl 2025-12-31 15:36:43 -08:00
Nick Sweeting
9487f8a0de
add ci for parallel tests 2025-12-29 02:39:24 -08:00