143 Commits

Author	SHA1	Message	Date
Nick Sweeting	8c761cb424	Pass query data to SavePageNow form	2026-05-23 15:29:10 -07:00
Nick Sweeting	92339e08f0	Prepare dev demo deploy	2026-05-22 20:12:48 -07:00
Nick Sweeting	c1593fe55b	Report supervisord runner in live progress	2026-05-18 02:10:45 -07:00
Nick Sweeting	ea4ce5b641	Prepare ArchiveBox 0.9.30rc81 demo release	2026-05-18 01:51:55 -07:00
Nick Sweeting	0e2ae425ce	Start Sonic worker under supervisord	2026-05-17 21:24:02 -07:00
Nick Sweeting	0cc4251c8a	Update ArchiveBox for abx dependency releases	2026-05-16 23:50:16 -07:00
Nick Sweeting	f3622d8cd3	update working changes	2026-03-25 05:36:07 -07:00
Nick Sweeting	50286d3c38	Reuse cached binaries in archivebox runtime	2026-03-24 11:03:43 -07:00
Nick Sweeting	25f935b9d1	split CrawlSetup into Install phase with new Binary + BinaryRequest events	2026-03-23 13:15:41 -07:00
Nick Sweeting	b749b26c5d	wip	2026-03-23 03:58:32 -07:00
Nick Sweeting	f400a2cd67	WIP: checkpoint working tree before rebasing onto dev	2026-03-22 20:25:18 -07:00
Nick Sweeting	a6548df8d0	Add configurable server security modes (#1773) ... Fixes https://github.com/ArchiveBox/ArchiveBox/issues/239 ## Summary - add `SERVER_SECURITY_MODE` presets for safe subdomain replay, safe one-domain no-JS replay, unsafe one-domain no-admin, and dangerous one-domain full replay - make host routing, replay URLs, static serving, and control-plane access mode-aware - add strict routing/header coverage plus a browser-backed Chrome/Puppeteer test that verifies real same-origin behavior in all four modes ## Testing - `uv run pytest archivebox/tests/test_urls.py -v` - `uv run pytest archivebox/tests/test_admin_views.py -v` - `uv run pytest archivebox/tests/test_server_security_browser.py -v` <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/archivebox/archivebox/pull/1773" target="_blank"> <picture> <source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1"> <img src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1" alt="Open with Devin"> </picture> </a> <!-- devin-review-badge-end --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Adds configurable server security modes to isolate admin/API from archived content, with a safe subdomain default and single-domain fallbacks. Routing, replay endpoints, headers, and middleware are mode-aware, with browser tests validating same-origin behavior. - New Features - Introduced SERVER_SECURITY_MODE with presets: safe-subdomains-fullreplay (default), safe-onedomain-nojsreplay, unsafe-onedomain-noadmin, danger-onedomain-fullreplay. - Mode-aware routing and base URLs; one-domain modes use path-based replay: /snapshot/<id>/... and /original/<domain>/.... - Control plane gate: block admin/API and non-GET methods in unsafe-onedomain-noadmin; allow full access in danger-onedomain-fullreplay. - Safer replay: detect risky HTML/SVG and apply CSP sandbox (no scripts) in safe-onedomain-nojsreplay; add X-ArchiveBox-Security-Mode and X-Content-Type-Options: nosniff on replay responses. - Middleware and serving: added ServerSecurityModeMiddleware, improved HostRouting, and static server byte-range/CSP handling. - Tests: added Chrome/Puppeteer browser tests and stricter URL routing tests covering all modes. - Migration - Default requires wildcard subdomains for full isolation (admin., web., api., and snapshot-id.<base>). - To run on one domain, set SERVER_SECURITY_MODE to a one-domain preset; URLs switch to /snapshot/<id>/ and /original/<domain>/ paths. - For production, prefer safe-subdomains-fullreplay; lower-security modes print a startup warning. <sup>Written for commit ad41b15581. Summary will update on new commits.</sup> <!-- End of auto-generated description by cubic. -->	2026-03-22 20:17:21 -07:00
Nick Sweeting	c87079aa0a	Refactor ArchiveBox onto abx-dl bus runner	2026-03-21 11:47:57 -07:00
Nick Sweeting	ad41b15581	Add configurable server security modes	2026-03-15 23:34:40 -07:00
Nick Sweeting	9de084da65	bump package versions	2026-03-15 20:47:28 -07:00
Nick Sweeting	934e02695b	fix lint	2026-03-15 18:45:29 -07:00
Nick Sweeting	ecb1764590	switch to external plugins	2026-03-15 03:46:23 -07:00
Nick Sweeting	07dc880d0b	Harden AddView config overrides to admin-only	2026-03-15 03:45:57 -07:00
Nick Sweeting	ec4b27056e	wip	2026-01-21 03:19:56 -08:00
Nick Sweeting	86e7973334	cleanup tui, startup, card templtes, and more	2026-01-19 14:33:20 -08:00
Nick Sweeting	c7b2217cd6	tons of fixes with codex	2026-01-19 01:00:53 -08:00
Claude	17029ba8b8	Add thumbnail strip to live progress monitor ... Show small thumbnails of recently completed ArchiveResult content in the progress header. The thumbnail strip appears below the stats bar and shows the last 20 successfully archived items with embeddable content (screenshots, favicons, DOM snapshots, etc.). Features: - API returns recent_thumbnails with embed paths for succeeded results - Thumbnails display with plugin-specific icons as fallback - New thumbnails animate in with a pop effect - Clicking a thumbnail navigates to the snapshot admin page - Horizontal scrollable strip with custom scrollbar styling	2025-12-31 20:38:55 +00:00
Nick Sweeting	2e350d317d	fix initial migrtaions	2025-12-29 21:27:31 -08:00
Nick Sweeting	30c60eef76	much better tests and add page ui	2025-12-29 04:02:11 -08:00
Nick Sweeting	f4e7820533	use full dotted paths for all archivebox imports, add migrations and more fixes	2025-12-29 00:47:08 -08:00
Nick Sweeting	f0aa19fa7d	wip	2025-12-28 17:51:54 -08:00
Nick Sweeting	4ccb0863bb	continue renaming extractor to plugin, add plan for hook concurrency, add chrome kill helper script	2025-12-28 05:29:24 -08:00
Nick Sweeting	bd265c0083	rename extractor to plugin everywhere	2025-12-28 04:43:15 -08:00
Nick Sweeting	4fd7fcdbcf	new gallerydl plugin and more	2025-12-26 11:55:03 -08:00
Nick Sweeting	9838d7ba02	tons of ui fixes and plugin fixes	2025-12-25 03:59:51 -08:00
Nick Sweeting	bb53228ebf	remove Seed model in favor of Crawl as template	2025-12-25 01:52:41 -08:00
Nick Sweeting	866f993f26	logging and admin ui improvements	2025-12-25 01:10:41 -08:00
Nick Sweeting	d95f0dc186	remove huey	2025-12-24 23:40:18 -08:00
Nick Sweeting	1915333b81	wip major changes	2025-12-24 20:10:38 -08:00
Nick Sweeting	c1335fed37	Remove ABID system and KVTag model - use UUIDv7 IDs exclusively ... This commit completes the simplification of the ID system by: - Removing the ABID (ArchiveBox ID) system entirely - Removing the base_models/abid.py file - Removing KVTag model in favor of the existing Tag model in core/models.py - Simplifying all models to use standard UUIDv7 primary keys - Removing ABID-related admin functionality - Cleaning up commented-out ABID code from views and statemachines - Deleting migration files for ABID field removal (no longer needed) All models now use simple UUIDv7 ids via `id = models.UUIDField(primary_key=True, default=uuid7)` Note: Old migrations containing ABID references are preserved for database migration history compatibility. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>	2025-12-24 06:13:49 -08:00
Nick Sweeting	f6d22a3cc4	tweak worker updated logic and add output_dir_template and symlinks logic	2024-12-13 06:03:52 -08:00
Nick Sweeting	b948e49013	add urls log to Crawl model	2024-11-19 06:32:33 -08:00
Nick Sweeting	4a5d607296	move logging_util into archivebox.misc subfolder	2024-11-18 19:08:49 -08:00
Nick Sweeting	e469c5a344	merge queues and actors apps into new workers app	2024-11-18 18:52:48 -08:00
Nick Sweeting	c8e186f21b	fix plugin loading order, admin, abx-pkg	2024-11-16 06:44:12 -08:00
Nick Sweeting	70926f1d9f	replace os.access with os.path.isdir	2024-10-28 19:06:38 -07:00
Nick Sweeting	d93aa46949	fix django.forms.JSONField does not exist 500 error	2024-10-28 18:47:45 -07:00
Nick Sweeting	b3c1cb716e	move abx plugins inside vendor dir	2024-10-28 04:07:35 -07:00
Nick Sweeting	80d8a6b667	split archivebox.use into archivebox.reads and archivebox.writes	2024-10-15 01:03:01 -07:00
Nick Sweeting	01ba6d49d3	new vastly simplified plugin spec without pydantic	2024-10-14 21:50:47 -07:00
Nick Sweeting	59b669691f	fix Admin data view for Config to render both sections and individual values	2024-10-14 17:39:14 -07:00
Nick Sweeting	9a04ed7c76	move serve_static and shell_welcome_message into misc	2024-10-14 17:35:28 -07:00
Nick Sweeting	5ac941cf2e	resolve circular import in core/views	2024-10-14 15:43:53 -07:00
Nick Sweeting	de2ab43f7f	switch .is_dir and .exists for os.access to avoid PermissionError on startup	2024-10-08 03:02:34 -07:00
Nick Sweeting	cf1ea8f80f	improve config loading of TMP_DIR, LIB_DIR, move to separate files	2024-10-07 23:45:11 -07:00