Nick Sweeting
5f1f2a95fb
Point init hint to canonical web setup
2026-08-19 01:03:17 -07:00
Nick Sweeting
e6c3dc8a64
Fix docs root install ownership and live progress rows
2026-07-26 20:59:08 -07:00
Nick Sweeting
d4ff0d033f
Merge remote-tracking branch 'origin/dev' into codex/all-tests-ci
2026-07-25 11:51:21 -07:00
Nick Sweeting
7658df9208
Fix ArchiveBox CI runtime paths and docs validation
2026-07-25 11:51:15 -07:00
Claude
7167c5dd8a
Add native PostgreSQL support alongside SQLite
...
Add DATABASE_ENGINE=postgres (plus DATABASE_HOST/PORT/USER/PASSWORD/NAME)
config and centralize all sqlite-vs-postgres branching in
archivebox.misc.db:
- get_database_settings() builds DATABASES for either backend; the sqlite
path is unchanged (custom lock-retry backend, same PRAGMAs).
- database_exists()/ensure_database_ready() replace index.sqlite3 file
checks; init auto-creates the postgres database when missing.
- approximate_row_counts() serves admin index counts from sqlite_stat1 or
pg_class.reltuples; missing-table detection covers both vendors.
- rebuild_models_from_migration_state() lets historical sqlite-only raw
SQL migrations resync postgres schema from Django migration state at
every divergence point (postgres can never hold legacy data, so
affected tables are empty when these run). All raw-DDL and PRAGMA
migrations are now vendor-gated with sqlite behavior byte-for-byte
unchanged.
- A pre_save clamp truncates CharField values to max_length: sqlite
never enforced varchar(n) but postgres does (e.g. long crawl labels).
- Collation-sensitive URL range scans branch to escaped LIKE on postgres
(with a text_pattern_ops index) since linguistic collations break
bytewise range tricks; the crawl-config JSON search wave gets a
jsonb-text implementation.
Verified on real PostgreSQL 16: fresh init applies the entire migration
graph, schema matches models exactly (column-level parity check +
makemigrations --check), and add/run/list/search/status/remove all work
end-to-end. New test_postgres_backend.py suite boots a real throwaway
postgres cluster (initdb + pg_ctl); CI workflows install postgres server
binaries.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019YHSjZM6TstSAMN2PhgfUg
2026-07-23 17:08:47 +00:00
Nick Sweeting
6592af0d19
Exercise ArchiveBox through unified runtime lifecycles
2026-07-21 19:26:42 -07:00
Nick Sweeting
8b57085827
chore: commit local archivebox changes
2026-06-14 11:44:38 -07:00
Nick Sweeting
4fa90e484a
release: archivebox 0.9.34rc71
2026-06-07 20:51:28 -07:00
Nick Sweeting
3669133a05
fix: allow install to initialize collections
2026-06-02 21:25:20 -07:00
Nick Sweeting
c075d654d8
Consolidate runtime config handling
2026-06-01 15:03:40 -07:00
Nick Sweeting
cab05eb1c6
Refactor plugins search progress and config flows
2026-06-01 00:08:27 -07:00
Nick Sweeting
65dc252189
fix: detect newer database migrations
2026-05-28 13:52:25 -07:00
Nick Sweeting
fc539673ca
fix: use shared lib dir for docker installs
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Debian package / build (amd64) (push) Waiting to run
Build Debian package / build (arm64) (push) Waiting to run
Build Debian package / test (amd64, ubuntu-24.04) (push) Blocked by required conditions
Build Debian package / test (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
Build Debian package / release (push) Blocked by required conditions
Build Docker image / buildx (push) Waiting to run
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Release State / release-state (push) Waiting to run
Parallel Tests / Discover test files (push) Waiting to run
Parallel Tests / ${{ matrix.test.name }} (push) Blocked by required conditions
Parallel Tests / ${{ matrix.plugin.name }} (push) Blocked by required conditions
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2026-05-27 05:17:22 -07:00
Nick Sweeting
0cc4251c8a
Update ArchiveBox for abx dependency releases
2026-05-16 23:50:16 -07:00
Nick Sweeting
b749b26c5d
wip
2026-03-23 03:58:32 -07:00
Nick Sweeting
4756697a17
Use ruff pyright and ty for linting
2026-03-15 19:43:59 -07:00
Nick Sweeting
49436af869
Tighten CLI and admin typing
2026-03-15 19:33:15 -07:00
Nick Sweeting
70c9358cf9
Improve scheduling, runtime paths, and API behavior
2026-03-15 18:31:56 -07:00
Nick Sweeting
dd77511026
unified Process source of truth and better screenshot tests
2026-01-02 04:20:34 -08:00
Nick Sweeting
dd2302ad92
new jsonl cli interface
2025-12-30 16:12:53 -08:00
Nick Sweeting
f4e7820533
use full dotted paths for all archivebox imports, add migrations and more fixes
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Debian package / build (push) Waiting to run
Deploy static content to Pages / deploy (push) Waiting to run
Build Homebrew package / build (push) Waiting to run
Build GitHub Pages website / build (push) Waiting to run
Build GitHub Pages website / deploy (push) Blocked by required conditions
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2025-12-29 00:47:08 -08:00
Nick Sweeting
f0aa19fa7d
wip
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
Build Debian package / build (push) Waiting to run
Deploy static content to Pages / deploy (push) Waiting to run
Build Homebrew package / build (push) Waiting to run
Build GitHub Pages website / build (push) Waiting to run
Build GitHub Pages website / deploy (push) Blocked by required conditions
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Run tests / python_tests (ubuntu-22.04, 3.13) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2025-12-28 17:51:54 -08:00
Nick Sweeting
50e527ec65
way better plugin hooks system wip
2025-12-28 03:39:59 -08:00
Nick Sweeting
1915333b81
wip major changes
2025-12-24 20:10:38 -08:00
Nick Sweeting
a0edf218e8
fix archivebox init and archivebox install CLI commands
2024-11-19 01:05:49 -08:00
Nick Sweeting
328eb98a38
move main funcs into cli files and switch to using click for CLI
2024-11-19 00:18:51 -08:00
Nick Sweeting
4a5d607296
move logging_util into archivebox.misc subfolder
2024-11-18 19:08:49 -08:00
Nick Sweeting
8c3342afe5
rename archivebox setup to archivebox install
2024-09-30 23:19:46 -07:00
Nick Sweeting
b913e6f426
rename OUTPUT_DIR to DATA_DIR
2024-09-30 17:44:18 -07:00
Nick Sweeting
363a499289
move util.py into misc folder
2024-09-30 17:25:15 -07:00
Nick Sweeting
3e5b6ddeae
move config into dedicated global app
2024-09-30 15:59:05 -07:00
Nick Sweeting
8b4f84959a
new setup flag on init command to autosetup on init
2021-04-05 21:15:32 -04:00
Nick Sweeting
3c3bae02d2
add quick-init option to skip reimporting all snapshot dirs on init
2021-02-15 14:52:10 -05:00
Cristian
6006b4f93b
refactor: Organize code to remove flake8 issues
2020-07-24 12:25:25 -05:00
Cristian
a5550b2105
fix: Rename logging folder to avoid naming conflicts (and circular import issues)
2020-07-22 11:02:13 -05:00
Cristian
f4d1b5121e
refactor: Move logging.py to main module to avoid circular import issues
2020-07-17 18:00:04 -05:00
Nick Sweeting
ad3898add6
allow forcing init in dirty directory
2019-05-01 02:27:50 -04:00
Nick Sweeting
158f145d9a
move docstrings to main.py out of cli files
2019-04-30 23:10:48 -04:00
Nick Sweeting
1b8abc0961
move everything out of legacy folder
2019-04-27 17:26:24 -04:00
Nick Sweeting
39a0ab3013
add pipenv, schedule cmd, logs dir, and lots more
2019-04-18 21:10:00 -04:00
Nick Sweeting
21174da014
better data folder checking on startup
2019-04-17 02:22:19 -04:00
Nick Sweeting
4ca9a0beac
colorized and better command sorting in help msg
2019-04-11 06:59:30 -04:00
Nick Sweeting
718e25c973
better dependency checking system and guards
2019-04-11 03:42:35 -04:00
Nick Sweeting
fd80275856
print import instructions on first run
2019-04-03 01:54:15 -04:00
Nick Sweeting
51ae634ec9
working argparse based CLI with most commands implemented
2019-04-03 00:27:37 -04:00
