Pre-0.9.0 crawl tables had ``max_urls``, ``crawl_max_size``, and
``snapshot_max_size`` as NOT NULL ``IntegerField`` columns. The 0011
migration RemoveField'd them, but on long-lived DBs where a historical
0011 ran with different semantics the columns stayed on disk while
being absent from the model. Every Crawl.objects.create() through the
ORM (e.g. the /add/ form view) skips them, and SQLite refuses the
insert with "NOT NULL constraint failed: crawls_crawl.max_urls" —
an HTTP 500 on /add/. Cabbage hit this on the rc51 UI test.
The new 0017 migration introspects the live table, copies any
pre-existing values into ``config`` under the canonical
CRAWL_MAX_URLS / CRAWL_MAX_SIZE / SNAPSHOT_MAX_SIZE keys so nothing
gets silently dropped, then ALTER TABLE DROP COLUMNs each legacy
field. Fresh installs already have these columns removed and the
migration is a no-op there. Requires SQLite 3.35+ for DROP COLUMN.
Plus: bin/take_screenshot.js bumps Puppeteer's CDP protocolTimeout
from the 30s default to 5 minutes — admin pages with many DB-backed
partials (snapshot grid w/ 23K rows, progress monitor, etc.) blow
past 30s on the screenshot capture under load.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
