Commit Graph

286 Commits

Author SHA1 Message Date
Nick Sweeting
ec4b8ef2ef
fix: preserve unbounded maintenance pagination 2026-09-02 14:43:13 -07:00
Nick Sweeting
e878007b6a
fix: bound snapshot scheduling ownership 2026-09-02 14:35:57 -07:00
Nick Sweeting
35e014ae90
fix: correlate snapshot completion leases 2026-09-02 14:18:00 -07:00
Nick Sweeting
8bc9f40e9f
fix: harden concurrent snapshot retries 2026-09-02 13:52:42 -07:00
Nick Sweeting
a85d417d0f
fix: preserve queued snapshot retry work 2026-09-02 13:33:21 -07:00
Nick Sweeting
9ee0dcf165
fix: address orchestration review findings 2026-09-02 12:56:23 -07:00
Nick Sweeting
2bba758314
refactor: simplify snapshot maintenance orchestration 2026-09-02 12:05:09 -07:00
Nick Sweeting
3e58261cd5
Expose yt-dlp media metadata for snapshot cards (#1819)
Supersedes #1818.

This keeps yt-dlp-specific player rendering out of ArchiveBox core. Core
now:

- exposes media type metadata to plugin card templates: `is_video`,
`is_audio`, `is_browser_playable`
- sorts yt-dlp media with browser-playable video/audio first
- prefers browser-playable yt-dlp outputs before thumbnails and
non-browser containers when choosing the primary card output
- avoids pinning resumed snapshots to stale selected hook maps, fixing a
pause/resume runner regression surfaced by CI
- makes `update --index-only` discover the configured search backend
even when `PLUGINS` is restricted for test/runtime isolation
- stabilizes CI-only recursive/takeover/public-UI tests that were
failing due external/heavy plugin coverage, hard-kill recovery races,
and transient full-server `/add/` startup responses

The actual click-to-load player UI lives in the companion plugin PR:
ArchiveBox/abx-plugins#34.

Verification:
```text
pytest archivebox/tests/test_ui_admin_snapshot.py -q
48 passed

pytest archivebox/tests/test_cli_update_reindex_snapshots.py -q
9 passed

pytest -xvs archivebox/tests/test_api_v1_crawls_crawl_crawl_id.py --basetemp=tests/out --ignore=archivebox/pkgs
7 passed

pytest -xvs archivebox/tests/test_takeover_util.py --basetemp=tests/out --ignore=archivebox/pkgs
19 passed

pytest -xvs archivebox/tests/test_takeover_util.py::test_live_add_update_jobs_survive_server_and_cli_owner_exits --basetemp=tests/out --ignore=archivebox/pkgs
1 passed

pytest -xvs archivebox/tests/test_ui_public_snapshot.py --basetemp=tests/out --ignore=archivebox/pkgs
6 passed

pytest -q archivebox/tests/test_recursive_crawl.py::test_recursive_crawl_depth_two_all_plugins_runs_snapshots_in_parallel --basetemp=tests/out --ignore=archivebox/pkgs
1 passed
```
2026-09-02 04:14:15 -07:00
Nick Sweeting
fbff88fb0e
Merge remote-tracking branch 'origin/dev' into audit-pr1846
# Conflicts:
#	archivebox/cli/archivebox_extract.py
#	archivebox/core/recovery_util.py
#	archivebox/services/archive_result_service.py
#	archivebox/services/runner.py
2026-09-01 15:43:48 -07:00
Nick Sweeting
032c20bdf5
cleanup: delete obsolete model lifecycle helpers 2026-09-01 15:06:48 -07:00
Nick Sweeting
c2ce82964d
Expose yt-dlp media metadata for snapshot cards 2026-09-01 11:49:16 -07:00
Nick Sweeting
c817da7af6
Merge remote-tracking branch 'origin/dev' into audit-pr1846 2026-09-01 11:38:24 -07:00
Nick Sweeting
3fc67705a5
Unify plugin runtime and queue lifecycle boundaries 2026-09-01 01:07:04 -07:00
Nick Sweeting
c498fc174c
Keep static manifest schema consistent 2026-08-30 16:11:48 -07:00
Nick Sweeting
e874ee4201
Write linked static JSONL manifest 2026-08-30 16:02:27 -07:00
Nick Sweeting
a1b763974e
Handle unarchived static snapshot details 2026-08-30 13:01:34 -07:00
Nick Sweeting
920826b1cb
Fix complete static exports and crawl progress 2026-08-30 12:39:54 -07:00
Nick Sweeting
2c901aaa44
fix: rename Archive Results admin label 2026-08-30 11:22:49 -07:00
Nick Sweeting
c8ef576235
fix: secure snapshot output deletion handoff 2026-08-29 23:03:08 -07:00
Nick Sweeting
6906edc076
fix: make portable exports and screenshot deploys reliable 2026-08-29 16:03:29 -07:00
Nick Sweeting
97486882bd
Address Cubic review feedback 2026-08-28 21:56:23 -07:00
Nick Sweeting
c062a3c54f
Eliminate request-side SQLite write transactions 2026-08-28 21:56:18 -07:00
Nick Sweeting
bdc236f076
Revert "fix: project one archive result per plugin" 2026-08-28 21:56:13 -07:00
Nick Sweeting
de14346a08
fix: project one archive result per plugin 2026-08-28 19:47:15 -07:00
Nick Sweeting
056a02a06e
Restore ArchiveResult hook identity 2026-08-28 11:52:36 -07:00
Nick Sweeting
43c2dc9896
Enforce one ArchiveResult per plugin 2026-08-28 08:56:07 -07:00
Nick Sweeting
f092db3255
Preserve shared ArchiveResult outputs 2026-08-28 00:23:49 -07:00
Nick Sweeting
e548608149
feat: add snapshot output batch deletion 2026-08-27 21:34:48 -07:00
Nick Sweeting
3bb5bb48bc
remove snapshot status and icon caches 2026-08-27 15:54:25 -07:00
Nick Sweeting
f04433e9ad
eliminate snapshot detail query overhead 2026-08-27 15:24:04 -07:00
Nick Sweeting
134be3a1cd
speed up snapshot detail rendering 2026-08-27 14:21:59 -07:00
Nick Sweeting
3a7354d230
fix unified browser extension snapshots
Some checks are pending
CI / Linters (push) Waiting to run
CI / Install and CLI platform compatibility (push) Waiting to run
CI / Discovered test matrix (push) Waiting to run
CI / Documentation and root tests (push) Waiting to run
CI / CodeQL (push) Waiting to run
CI / Tested Python artifacts (push) Waiting to run
CI / Tested Docker images (push) Waiting to run
CI / All required CI lanes (push) Blocked by required conditions
Deploy Publicsite to GitHub Pages / deploy (push) Waiting to run
2026-08-26 18:24:08 -07:00
Nick Sweeting
1d07b0775e
fix escaped entities in snapshot titles 2026-08-26 16:30:38 -07:00
Nick Sweeting
ba2bc83809
Preserve outputs during filesystem migration 2026-08-12 22:07:34 -07:00
Nick Sweeting
1740dbc218
Clean up verified migrations owned by legacy users 2026-08-09 14:26:48 -04:00
Nick Sweeting
3980dff46d
Preserve interrupted 0.8.5 filesystem migrations 2026-08-02 01:32:24 -07:00
Nick Sweeting
2c451ed3e4
Support 0.8.5 filesystem upgrades 2026-08-02 01:12:30 -07:00
Nick Sweeting
7611a57917
Preserve complete legacy collections during upgrade 2026-08-01 14:25:57 -07:00
Nick Sweeting
6b08efd799
Preserve legacy ArchiveResult history 2026-08-01 14:25:57 -07:00
Nick Sweeting
54b1c04750
Preserve legacy plugin outputs during migration 2026-08-01 13:07:32 -07:00
Nick Sweeting
7dfe9465b4
Retry only failed archive results 2026-08-01 11:49:07 -07:00
Nick Sweeting
6c87629b39
Make API pause win over runner leases 2026-07-31 23:50:40 -07:00
Nick Sweeting
3835344855
Isolate sharded tests without losing coverage 2026-07-25 19:05:42 -07:00
Nick Sweeting
05350846ab
Sync ArchiveBox UI and setup updates 2026-07-25 14:50:15 -07:00
Claude
6a8062090b
Slim DB helpers and address PR review feedback
Helper reduction:
- Move the Django DATABASES/SQLITE_CONNECTION_OPTIONS assembly into
  core/settings.py, dropping get_database_settings() and
  get_sqlite_connection_options() from misc/db.py.
- Inline the single-use migration_table_columns() into its one migration.

Review fixes:
- search: match only scalar JSON *values* on postgres (jsonb_path_query
  over '$.**' scalar leaves), mirroring SQLite json_tree.atom so config
  keys no longer match.
- CharField clamp now also runs in SnapshotQuerySet.bulk_create (bulk paths
  bypass the pre_save signal); truncate_overlong_charfields is dual-use.
- Restore reverse-migration parity on postgres: crawls/machine/api initial
  migrations drop their rebuilt tables on reverse via drop_models_on_postgres.
- docs: give DATABASE_NAME its own section so the anchor resolves correctly.
- CI: only install postgres binaries on the shard that runs the postgres test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019YHSjZM6TstSAMN2PhgfUg
2026-07-24 07:57:09 +00:00
Nick Sweeting
6592af0d19
Exercise ArchiveBox through unified runtime lifecycles 2026-07-21 19:26:42 -07:00
Nick Sweeting
6e915c11aa
Restore complete deterministic ArchiveBox CI 2026-07-19 20:08:29 -07:00
Nick Sweeting
ee0e3eb43c
Enforce the sealed search maintenance boundary 2026-07-19 08:54:32 -07:00
Nick Sweeting
18e3580dcd
fix: harden reflected metadata rendering 2026-06-14 00:06:35 -07:00
Nick Sweeting
1dbde4776f
release: archivebox 0.9.35rc27 2026-06-13 23:12:36 -07:00