mirror of
https://github.com/ArchiveBox/ArchiveBox.git
synced 2026-09-14 11:06:13 +05:00
3521f21c36
Pre-0.9.0 crawl tables had ``max_urls``, ``crawl_max_size``, and ``snapshot_max_size`` as NOT NULL ``IntegerField`` columns. The 0011 migration RemoveField'd them, but on long-lived DBs where a historical 0011 ran with different semantics the columns stayed on disk while being absent from the model. Every Crawl.objects.create() through the ORM (e.g. the /add/ form view) skips them, and SQLite refuses the insert with "NOT NULL constraint failed: crawls_crawl.max_urls" — an HTTP 500 on /add/. Cabbage hit this on the rc51 UI test. The new 0017 migration introspects the live table, copies any pre-existing values into ``config`` under the canonical CRAWL_MAX_URLS / CRAWL_MAX_SIZE / SNAPSHOT_MAX_SIZE keys so nothing gets silently dropped, then ALTER TABLE DROP COLUMNs each legacy field. Fresh installs already have these columns removed and the migration is a no-op there. Requires SQLite 3.35+ for DROP COLUMN. Plus: bin/take_screenshot.js bumps Puppeteer's CDP protocolTimeout from the 30s default to 5 minutes — admin pages with many DB-backed partials (snapshot grid w/ 23K rows, progress monitor, etc.) blow past 30s on the screenshot capture under load. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| migrations | ||
| __init__.py | ||
| admin.py | ||
| apps.py | ||
| models.py | ||
| schedule_util.py | ||