ArchiveBox/archivebox/core
Claude b74861ef1e
Support URLs up to 8000 chars via variable-length indexed TextField
Snapshot.url was a CharField(max_length=65535) which reserves a fixed-width
column and is too long to index or constrain on real DB backends. Store it as a
variable-length TextField instead, so short URLs don't waste space and very long
URLs (up to MAX_URL_LENGTH=8000) are supported, while keeping a normal index on
the field so exact, prefix, and substring (icontains) URL lookups all stay fast.

- misc/util.py: MAX_URL_LENGTH 65535 -> 8000 (the practical web-server limit)
- core/models.py: Snapshot.url CharField -> TextField(db_index=True)
- migration 0049_alter_snapshot_url
- tests covering 8000-char persistence, exact/prefix/substring lookups,
  over-length rejection, and per-crawl uniqueness for long URLs

https://claude.ai/code/session_01BLnGTL5GSoouD4ihaYp55n
2026-06-04 22:35:27 +00:00
..
management release: archivebox 0.9.33rc50 2026-05-30 22:27:28 -07:00
migrations Support URLs up to 8000 chars via variable-length indexed TextField 2026-06-04 22:35:27 +00:00
sqlite_backend Refactor plugins search progress and config flows 2026-06-01 00:08:27 -07:00
templatetags Consolidate runtime config handling 2026-06-01 15:03:40 -07:00
__init__.py Update ArchiveBox for abx dependency releases 2026-05-16 23:50:16 -07:00
admin_archiveresults.py release: archivebox 0.9.34rc39 2026-06-03 17:19:48 -07:00
admin_site.py release: archivebox 0.9.32rc4 2026-05-27 03:03:04 -07:00
admin_snapshots.py Update snapshot admin selection behavior 2026-06-04 08:00:25 -07:00
admin_tags.py release: archivebox 0.9.34rc39 2026-06-03 17:19:48 -07:00
admin_users.py Consolidate runtime config handling 2026-06-01 15:03:40 -07:00
admin.py wip 2026-03-23 03:58:32 -07:00
apps.py release: archivebox 0.9.33rc37 2026-05-29 02:50:04 -07:00
asgi.py fix: stop ThreadSensitiveContext.__aexit__ from blocking daphne's event loop 2026-05-30 17:53:17 -07:00
context_processors.py wip: snapshot live progress, universal --init, runner perms, supervisord SIGINT 2026-05-30 04:45:15 -07:00
forms.py Consolidate runtime config handling 2026-06-01 15:03:40 -07:00
middleware.py fix private snapshot replay auth 2026-06-04 09:12:56 -07:00
models.py Support URLs up to 8000 chars via variable-length indexed TextField 2026-06-04 22:35:27 +00:00
permissions.py release: archivebox 0.9.33rc37 2026-05-29 02:50:04 -07:00
recovery_util.py Publish local ArchiveBox changes 2026-06-02 02:25:52 -07:00
routes_util.py Refactor plugins search progress and config flows 2026-06-01 00:08:27 -07:00
settings_logging.py fix: support replay assets without base url 2026-06-01 10:38:45 -07:00
settings.py release: archivebox 0.9.34rc39 2026-06-03 17:19:48 -07:00
shutdown_util.py Consolidate runtime config handling 2026-06-01 15:03:40 -07:00
snapshot_status.py Publish local ArchiveBox changes 2026-06-02 02:25:52 -07:00
tag_util.py release: archivebox 0.9.34rc39 2026-06-03 17:19:48 -07:00
takeover_util.py Publish local ArchiveBox changes 2026-06-02 02:25:52 -07:00
urls.py fix private snapshot replay auth 2026-06-04 09:12:56 -07:00
views.py fix private snapshot replay auth 2026-06-04 09:12:56 -07:00
widgets.py Consolidate runtime config handling 2026-06-01 15:03:40 -07:00
wsgi.py wip 2026-03-23 03:58:32 -07:00