diff --git a/docs/Configuration.md b/docs/Configuration.md
index abdde245..26a51b62 100644
--- a/docs/Configuration.md
+++ b/docs/Configuration.md
@@ -4,11 +4,11 @@ Configuration of ArchiveBox is done by using the `archivebox config` command, mo
 
 *Some equivalent examples of setting some configuration options:*
 ```bash
-archivebox config --set CHROME_BINARY=google-chrome-stable
+archivebox config --set TIMEOUT=120
 # OR
-echo "CHROME_BINARY=google-chrome-stable" >> ArchiveBox.conf
+echo "TIMEOUT=120" >> ArchiveBox.conf
 # OR
-env CHROME_BINARY=google-chrome-stable archivebox add ~/Downloads/bookmarks_export.html
+env TIMEOUT=120 archivebox add ~/Downloads/bookmarks_export.html
 ```
 
 Environment variables take precedence over the config file, which is useful if you only want to use a certain option temporarily during a single run. For more examples see [Usage: Configuration](Usage#run-archivebox-with-configuration-options)...
@@ -18,12 +18,13 @@ Environment variables take precedence over the config file, which is useful if y
 <img src="https://imgur.zervice.io/EUeQbiZ.png" width="200px" align="right"/>
 
 **Available Configuration Options:**
- - [General Settings:](#general-settings) Archiving process, output format, and timing.
- - [Server Settings:](#server-settings) Web UI, authentication, and reverse proxy options.
- - [Storage Settings:](#storage-settings) File layout, permissions, and temp directories.
- - [Search Settings:](#search-settings) Full-text search backend configuration.
+ - [General Settings:](#general-settings) Archiving process, output format, crawl limits, and retention.
+ - [Server Settings:](#server-settings) Web UI, authentication, subdomain routing, and reverse proxy options.
+ - [Storage Settings:](#storage-settings) File layout, permissions, and temp/lib directories.
+ - [Database Settings:](#database-settings) SQLite tuning and lock-retry behavior.
+ - [Search Settings:](#search-settings) Full-text search backend selection.
  - [Shell Options:](#shell-options) Format & behavior of CLI output.
- - [Plugin Settings:](#plugin-settings) Per-plugin configuration options.
+ - [Plugin Configuration:](#plugin-configuration) Per-plugin options (now documented separately).
 
 <br/>
 
@@ -36,7 +37,7 @@ Environment variables take precedence over the config file, which is useful if y
 
 ## General Settings
 
-*General options around the archiving process, output format, and timing.*
+*General options around the archiving process, output format, retention, and concurrency limits.*
 
 ---
 #### `ONLY_NEW`
@@ -45,204 +46,453 @@ Toggle whether or not to attempt rechecking old links when adding new ones, or l
 
 By default, ArchiveBox will only archive new links on each import. If you want it to go back through all links in the index and download any missing files on every run, set this to `False`.
 
-*Note: Regardless of how this is set, ArchiveBox will never re-download sites that have already succeeded previously. When this is `False` it only attempts to fix previous pages have *missing* archive extractor outputs, it does not re-archive pages that have already been successfully archived.*
-
----
-#### `OVERWRITE`
-**Possible Values:** [`False`]/`True`
-When set to `True`, ArchiveBox will re-archive URLs even if they have already been successfully archived before, overwriting any existing output.
+*Note: Regardless of how this is set, ArchiveBox will never re-download sites that have already succeeded previously. When this is `False` it only attempts to fix previous pages that have *missing* archive extractor outputs, it does not re-archive pages that have already been successfully archived.*
 
 ---
 #### `TIMEOUT`
 **Possible Values:** [`60`]/`120`/...
-Maximum allowed download time per archive method for each link in seconds.  If you have a slow network connection or are seeing frequent timeout errors, you can raise this value.
+Maximum allowed runtime **per-extractor, per-Snapshot** in seconds. If you have a slow network connection or are seeing frequent timeout errors, you can raise this value.
 
-*Note: Do not set this to anything less than `5` seconds as it will cause Chrome to hang indefinitely and many sites to fail completely.*
+This is a *plugin-shared* setting — each individual extractor can override it with its own `<EXTRACTOR>_TIMEOUT` (e.g. [`WGET_TIMEOUT`](https://archivebox.github.io/abx-plugins/#wget), [`CHROME_TIMEOUT`](https://archivebox.github.io/abx-plugins/#chrome), [`YTDLP_TIMEOUT`](https://archivebox.github.io/abx-plugins/#ytdlp)). See the [per-plugin docs](https://archivebox.github.io/abx-plugins/) for the full list.
 
----
-#### `MAX_URL_ATTEMPTS`
-**Possible Values:** [`50`]/`100`/...
-Maximum number of times ArchiveBox will attempt to archive a URL before giving up. Useful for handling transient failures.
+> [!NOTE]
+> `TIMEOUT` only caps a single extractor invocation. To bound the *total* wall-clock runtime of an entire crawl, use [`CRAWL_TIMEOUT`](#crawl_timeout) instead.
+
+> [!WARNING]
+> Do not set this to anything less than `5` seconds — Chrome will hang indefinitely and many sites will fail completely. Anywhere between `30` and `3000` is the recommended range.
+
+*Related options:*
+[`CRAWL_TIMEOUT`](#crawl_timeout), [`CRAWL_MAX_URLS`](#crawl_max_urls), [`SNAPSHOT_MAX_SIZE`](#snapshot_max_size)
 
 ---
 #### `RESOLUTION`
 **Possible Values:** [`1440,2000`]/`1024,768`/...
-Default screenshot/PDF resolution in pixels width,height. Used as the fallback for `SCREENSHOT_RESOLUTION`, `PDF_RESOLUTION`, and `CHROME_RESOLUTION`.
+Default screenshot/PDF viewport resolution in `width,height` pixels. Used as the fallback for `SCREENSHOT_RESOLUTION`, `PDF_RESOLUTION`, and `CHROME_RESOLUTION`.
+
+This is a *plugin-shared* setting — individual extractors override it via `<EXTRACTOR>_RESOLUTION` (e.g. [`SCREENSHOT_RESOLUTION`](https://archivebox.github.io/abx-plugins/#screenshot), [`PDF_RESOLUTION`](https://archivebox.github.io/abx-plugins/#pdf), [`CHROME_RESOLUTION`](https://archivebox.github.io/abx-plugins/#chrome)). See the [per-plugin docs](https://archivebox.github.io/abx-plugins/) for plugin-specific overrides.
 
 ---
 #### `CHECK_SSL_VALIDITY`
 **Possible Values:** [`True`]/`False`
-Whether to enforce HTTPS certificate and HSTS chain of trust when archiving sites.  Set this to `False` if you want to archive pages even if they have expired or invalid certificates.  Be aware that when `False` you cannot guarantee that you have not been man-in-the-middle'd while archiving content, so the content cannot be verified to be what's on the original site.
+Whether to enforce HTTPS certificate validity and HSTS chain of trust when archiving sites. Set this to `False` if you want to archive pages even if they have expired or invalid certificates.
+
+This is a *plugin-shared* setting — every HTTP-fetching extractor ([`wget`](https://archivebox.github.io/abx-plugins/#wget), [`yt-dlp`](https://archivebox.github.io/abx-plugins/#ytdlp), [`gallery-dl`](https://archivebox.github.io/abx-plugins/#gallerydl), [`chrome`](https://archivebox.github.io/abx-plugins/#chrome), etc.) honors it, and individual extractors can override with `<EXTRACTOR>_CHECK_SSL_VALIDITY`. See the [per-plugin docs](https://archivebox.github.io/abx-plugins/).
+
+> [!WARNING]
+> When `False`, ArchiveBox cannot guarantee that the captured content matches the real site — a man-in-the-middle could substitute responses. Only disable for trusted networks or for archiving legacy/internal sites with expired certs.
 
 ---
 #### `USER_AGENT`
 **Possible Values:** [`Mozilla/5.0 ... ArchiveBox/{VERSION} ...`]/`"Mozilla/5.0 ..."`/...
-The default user agent string used during archiving. Individual extractors (wget, Chrome, curl, etc.) can override this with their own `*_USER_AGENT` settings, or fall back to this value.
+The default `User-Agent` string sent during archiving. The built-in default identifies ArchiveBox and links back to the GitHub repo so site operators can identify and contact archivers if needed.
+
+This is a *plugin-shared* setting — each extractor ([`wget`](https://archivebox.github.io/abx-plugins/#wget), [`chrome`](https://archivebox.github.io/abx-plugins/#chrome), [`yt-dlp`](https://archivebox.github.io/abx-plugins/#ytdlp), [`singlefile`](https://archivebox.github.io/abx-plugins/#singlefile), …) can override it with its own `<EXTRACTOR>_USER_AGENT`, otherwise it falls back to this value. See the [per-plugin docs](https://archivebox.github.io/abx-plugins/) for per-extractor specifics.
+
+> [!NOTE]
+> Some sites block requests that look like bots or that don't match a real browser. If you're getting 403s or empty responses, try setting this to a current Chrome/Firefox UA string.
 
 ---
 #### `COOKIES_FILE`
 **Possible Values:** [`None`]/`/path/to/cookies.txt`/...
 
-Cookies file to pass to `wget`, `curl`, `yt-dlp` and other extractors that don't use Chrome (with its `CHROME_USER_DATA_DIR`) for authentication.  To capture sites that require a user to be logged in, you configure this option to point to a [netscape-format `cookies.txt`](http://www.cookiecentral.com/faq/#3.5) file containing all the cookies you want to use during archiving.
+> [!TIP]
+> **Prefer [personas](#default_persona) over `COOKIES_FILE` for authentication.** A persona bundles a `cookies.txt`, a Chrome user-data-dir, a user-agent, and any other per-identity state into one named profile that's swappable per-crawl and automatically scoped across every extractor. `COOKIES_FILE` (and the per-extractor `<EXTRACTOR>_COOKIES_FILE` overrides) is a low-level escape hatch for when you specifically need to point at a hand-rolled cookies file outside the persona system — most users should ignore it and configure auth through `archivebox persona create` instead.
 
-You can generate this `cookies.txt` file by using a number of different [browser extensions](https://chromewebstore.google.com/detail/get-cookiestxt-locally/cclelndahbckbenkjhflpdbgdldlbecc) that can export your cookies in this format, or by using `wget` on the command line with `--save-cookies` + `--user=... --password=...`.
+Path to a [Netscape-format `cookies.txt`](http://www.cookiecentral.com/faq/#3.5) file passed to `wget`, `curl`, `yt-dlp`, and other non-Chrome extractors for authentication. Required when archiving sites behind a login (paywalls, social media feeds, members-only forums, etc.) **if you're not using a persona**.
 
-Alternatively, you can create a persona and import cookies directly from your browser profile:
+This is a *plugin-shared* setting — each extractor can override it with `<EXTRACTOR>_COOKIES_FILE` (e.g. [`WGET_COOKIES_FILE`](https://archivebox.github.io/abx-plugins/#wget), [`YTDLP_COOKIES_FILE`](https://archivebox.github.io/abx-plugins/#ytdlp), [`GALLERYDL_COOKIES_FILE`](https://archivebox.github.io/abx-plugins/#gallerydl)). [Chrome](https://archivebox.github.io/abx-plugins/#chrome)-based extractors instead read auth state from the persona's `CHROME_USER_DATA_DIR`. See the [per-plugin docs](https://archivebox.github.io/abx-plugins/) for per-extractor variants.
+
+You can generate a `cookies.txt` using a [browser extension](https://chromewebstore.google.com/detail/get-cookiestxt-locally/cclelndahbckbenkjhflpdbgdldlbecc), or with `wget --save-cookies` + `--user=... --password=...`.
+
+The recommended path is to create a persona and let it manage cookies + Chrome profile state for you:
 
 ```bash
 archivebox persona create --import=chrome personal
+archivebox add --persona=personal https://members.example.com/feed
 ```
 
 > [!WARNING]
-> **Make sure you use separate burner credentials dedicated to archiving,** e.g. don't re-use your normal daily Facebook/Instagram/Youtube/etc. account cookies as server responses often contain your name/email/PII, session tokens, etc. which then get preserved in your snapshots!
+> **Use separate burner credentials dedicated to archiving** — don't re-use your normal daily Facebook/Instagram/Youtube/etc. account cookies as server responses often contain your name/email/PII and session tokens, which then get preserved in your snapshots forever!
 
 *Related options:*
-[`CHROME_USER_DATA_DIR`](#chrome_user_data_dir), [`DEFAULT_PERSONA`](#default_persona)
+[`DEFAULT_PERSONA`](#default_persona), [`ACTIVE_PERSONA`](#active_persona), [`CHROME_USER_DATA_DIR`](https://archivebox.github.io/abx-plugins/#chrome)
 
 ---
 #### `DEFAULT_PERSONA`
 **Possible Values:** [`Default`]/`personal`/`work`/...
-The persona profile to use by default when archiving. Personas allow you to have separate sets of cookies, Chrome profiles, and user agent strings for different archiving contexts.
+The persona profile used when no explicit persona is selected for a crawl. Personas bundle a Chrome user-data-dir, a `cookies.txt`, auth state, a user-agent, and any other per-identity config into a single named profile, letting you swap between archiving contexts (logged-out vs. signed-into-work-account vs. signed-into-personal-account) without manually juggling files.
+
+ArchiveBox auto-creates the named persona on disk if it doesn't already exist. See the [Personas wiki page](https://github.com/ArchiveBox/ArchiveBox/wiki/Personas) for the full directory layout.
+
+*Related options:*
+[`ACTIVE_PERSONA`](#active_persona), [`COOKIES_FILE`](#cookies_file)
 
 ---
+#### `ACTIVE_PERSONA`
+**Possible Values:** *auto-set, read-only at runtime*
+The name of the persona actually being used for the *current* crawl/snapshot. Where [`DEFAULT_PERSONA`](#default_persona) is the user-configured *fallback*, `ACTIVE_PERSONA` is **derived** — ArchiveBox sets it automatically based on the resolved persona for each Snapshot (explicit selection on the Crawl > persona on the URL > `DEFAULT_PERSONA`).
+
+You generally read this rather than write it. Plugins and templates can inspect `ACTIVE_PERSONA` to render persona-specific UI or pick persona-scoped paths. Setting it manually in `ArchiveBox.conf` has no effect — it will be overwritten on every run by the persona resolver.
+
+*Related options:*
+[`DEFAULT_PERSONA`](#default_persona)
+
+---
+<a id="url_blacklist"></a>
 #### `URL_DENYLIST`
 **Possible Values:** [`\.(css|js|otf|ttf|woff|woff2|gstatic\.com|googleapis\.com/css)(\?.*)?$`]/`.+\.exe$`/...
 
-A regex expression used to exclude certain URLs from archiving.
+Regex pattern matched against every URL discovered during a crawl. Any matching URL is **excluded** from archiving — useful for blocking tracking pixels, ad networks, CDN-hosted CSS/fonts, or arbitrary file extensions you don't want to capture.
+
+The default skips common static assets (CSS, fonts, Google Fonts CDN) so they aren't re-fetched as separate Snapshots during recursive crawls — the parent page's `singlefile`/`dom` output already inlines them.
+
+*Note: This option is also recognized under its legacy alias `URL_BLACKLIST`.*
 
 *Related options:*
-[`URL_ALLOWLIST`](#url_allowlist), [`SAVE_ALLOWLIST`](#save_allowlist), [`SAVE_DENYLIST`](#save_denylist)
+[`URL_ALLOWLIST`](#url_allowlist)
 
 ---
+<a id="url_whitelist"></a>
 #### `URL_ALLOWLIST`
 **Possible Values:** [`None`]/`^http(s)?:\/\/(.+)?example\.com\/?.*$`/...
 
-A regex expression used to exclude all URLs that don't match the given pattern from archiving. Useful for recursive crawling within a single domain.
+Regex pattern matched against every URL discovered during a crawl. When set, any URL that does **not** match is excluded. Useful for recursive crawling scoped to a single domain or path prefix (e.g. only follow links within `docs.example.com/v2/`).
 
----
-#### `SAVE_ALLOWLIST`
-**Possible Values:** [`{}`]/`{".*example\\.com.*": ["screenshot", "pdf"]}`/...
-A JSON dictionary mapping URL regex patterns to lists of archive methods. Only the specified methods will be used for URLs matching each pattern.
+When both are set, `URL_DENYLIST` takes precedence over `URL_ALLOWLIST`.
 
----
-#### `SAVE_DENYLIST`
-**Possible Values:** [`{}`]/`{".*\\.pdf$": ["screenshot", "dom"]}`/...
-A JSON dictionary mapping URL regex patterns to lists of archive methods to *skip*.
+*Note: This option is also recognized under its legacy alias `URL_WHITELIST`.*
+
+*Related options:*
+[`URL_DENYLIST`](#url_denylist)
 
 ---
 #### `TAG_SEPARATOR_PATTERN`
-**Possible Values:** [`[,]`]/`[,;]`/...
-Regex pattern used to split tag strings into individual tags.
+**Possible Values:** [`[,]`]/`[,;]`/`[,;\s]`/...
+Regex character class used to split tag strings (e.g. `news,politics; longform`) into individual tags when importing URLs. The default splits on commas only; widen it if you paste in tags separated by semicolons, spaces, or other delimiters.
+
+---
+#### `CRAWL_MAX_URLS`
+**Possible Values:** [`0`]/`50`/`500`/...
+Maximum number of unique URLs (Snapshots) a single crawl is allowed to produce. `0` means unlimited. Counts both seed URLs you submitted and URLs discovered by recursive crawlers (`parse_dom_outlinks`, `parse_html_urls`, etc.).
+
+Once the cap is reached, recursive crawlers stop emitting new Snapshots and the crawl is marked with `stop_reason = "crawl_max_urls"`. **Raising the cap later and re-queuing the crawl will resume discovery** — the limit state is persisted in `<crawl_dir>/.abx-dl/limits.json` and re-evaluated each tick.
+
+> [!NOTE]
+> Use this as a safety net for recursive crawls (`--depth=N`) that could otherwise blow up to thousands of pages on link-heavy sites.
+
+*Related options:*
+[`CRAWL_MAX_SIZE`](#crawl_max_size), [`CRAWL_TIMEOUT`](#crawl_timeout), [`CRAWL_MAX_CONCURRENT_SNAPSHOTS`](#crawl_max_concurrent_snapshots), [`SNAPSHOT_MAX_SIZE`](#snapshot_max_size)
+
+---
+#### `CRAWL_MAX_SIZE`
+**Possible Values:** [`0`]/`50MB`/`5GB`/`104857600`/...
+Maximum cumulative output size (in bytes) a single crawl is allowed to produce across all of its Snapshots. `0` means unlimited.
+
+Accepts a raw byte count (`104857600`) or a unit-suffixed string (`100MB`, `5GB`, `1TiB`). Sizes are accumulated by the extractor service as each `ArchiveResult` writes its outputs to disk; once the cap is exceeded, in-flight Snapshots finish but no new ones are admitted and the crawl stops with `stop_reason = "crawl_max_size"`.
+
+> [!NOTE]
+> Bounds the **disk footprint** of a crawl, not the wire transfer — a 2MB HTML page can produce 50MB of screenshots, PDFs, SingleFile bundles, and media downloads, and this cap applies to the on-disk total.
+
+*Related options:*
+[`SNAPSHOT_MAX_SIZE`](#snapshot_max_size), [`CRAWL_MAX_URLS`](#crawl_max_urls), [`CRAWL_TIMEOUT`](#crawl_timeout)
+
+---
+#### `CRAWL_TIMEOUT`
+**Possible Values:** [`0`]/`300`/`3600`/...
+Maximum total wall-clock runtime for a single crawl in seconds. `0` means unlimited.
+
+Distinct from [`TIMEOUT`](#timeout): `TIMEOUT` caps one extractor invocation on one Snapshot; `CRAWL_TIMEOUT` caps the *entire crawl* — all Snapshots, all extractors, all retries, all recursive discovery passes — together. Once exceeded the crawl is marked `stop_reason = "crawl_timeout"` and queued Snapshots are skipped.
+
+> [!NOTE]
+> Useful as a hard ceiling for unattended/scheduled crawls (e.g. "spend at most 1 hour archiving Hacker News tonight"). Pair with `CRAWL_MAX_URLS` and `CRAWL_MAX_SIZE` for belt-and-suspenders bounds.
+
+*Related options:*
+[`TIMEOUT`](#timeout), [`CRAWL_MAX_URLS`](#crawl_max_urls), [`CRAWL_MAX_SIZE`](#crawl_max_size)
+
+---
+#### `CRAWL_MAX_CONCURRENT_SNAPSHOTS`
+**Possible Values:** [`4`]/`1`/`8`/`16`/...
+How many Snapshots within a single crawl ArchiveBox will archive in parallel. The runner schedules up to this many extractor pipelines at once, then waits for one to finish before starting the next.
+
+Raising this speeds up large crawls on beefy hardware, but each concurrent Snapshot launches its own Chrome instance (when Chrome-based extractors are enabled) — RAM and CPU pressure scale roughly linearly. On a typical laptop, `2-4` is sane; on a dedicated server with 32GB+ RAM, `8-16` can be reasonable.
+
+> [!NOTE]
+> This is **per-crawl** concurrency. If you run multiple crawls simultaneously, each one independently gets up to `CRAWL_MAX_CONCURRENT_SNAPSHOTS` parallel Snapshots.
+
+*Related options:*
+[`CRAWL_MAX_URLS`](#crawl_max_urls), [`TIMEOUT`](#timeout)
+
+---
+#### `SNAPSHOT_MAX_SIZE`
+**Possible Values:** [`0`]/`10MB`/`500MB`/...
+Maximum cumulative output size (in bytes) **per individual Snapshot**. `0` means unlimited. Same unit-suffix parsing as `CRAWL_MAX_SIZE` (`10MB`, `2GB`, raw bytes, etc.).
+
+Where `CRAWL_MAX_SIZE` is a *crawl-wide* budget, `SNAPSHOT_MAX_SIZE` puts a ceiling on any *one* page's output. Once a Snapshot's outputs exceed the cap, remaining extractors for that Snapshot are skipped and the Snapshot is tagged with `stop_reason = "snapshot_max_size"` — but the rest of the crawl continues normally.
+
+> [!NOTE]
+> Particularly useful when crawling sites with occasional huge pages (e.g. a forum where most threads are small but a few are 500MB media galleries) — it caps the outliers without throttling the whole crawl.
+
+*Related options:*
+[`CRAWL_MAX_SIZE`](#crawl_max_size), [`CRAWL_MAX_URLS`](#crawl_max_urls)
+
+---
+#### `DELETE_AFTER`
+**Possible Values:** [`0`]/`24h`/`7d`/`4w`/`6mo`/`1y`/...
+Retention policy: automatically delete Crawls, Snapshots, ArchiveResults, and Process rows (and their on-disk outputs) after this duration has elapsed. `0`, `""`, or `None` disables auto-deletion (the default — ArchiveBox never deletes anything unless you ask).
+
+Accepted units: `h`/`hr`/`hour`, `d`/`day`, `w`/`week`, `mo`/`month`, `y`/`yr`/`year`. The minimum non-zero duration is `1h`. Examples:
+
+```bash
+archivebox config --set DELETE_AFTER=24h     # daily rolling buffer
+archivebox config --set DELETE_AFTER=30d     # 30-day retention
+archivebox config --set DELETE_AFTER=6mo     # 6 months
+```
+
+`DELETE_AFTER` can be set globally, per-persona, per-crawl, or per-snapshot — the most-specific value wins. When a Snapshot is created, its `delete_at` timestamp is computed from the effective `DELETE_AFTER` and persisted; the retention sweeper then deletes rows whose `delete_at` is in the past.
+
+> [!WARNING]
+> Deletion is **destructive and irreversible**. Files in the snapshot's output directory are removed from disk. Use with care on important archives — and *never* set this on the global config if you have legacy snapshots you don't want garbage-collected.
+
+*Related options:*
+[`PERMISSIONS`](#permissions)
+
+---
+<a id="public_snapshots"></a>
+<a id="public_index--public_snapshots--public_add_view"></a>
+#### `PERMISSIONS`
+**Possible Values:** [`public`]/`unlisted`/`private`
+Default visibility for newly created Snapshots. Inherited by every Snapshot in a Crawl unless explicitly overridden at the Crawl or Snapshot level.
+
+- **`public`** — Snapshot appears in the public index *and* its content is directly accessible without login.
+- **`unlisted`** — Snapshot content is accessible via direct link, but it is **not** listed in the public index. Equivalent to a "secret URL."
+- **`private`** — Snapshot is hidden from the public index *and* its content requires admin login.
+
+This option supersedes the removed `PUBLIC_SNAPSHOTS` boolean and is also driven by the still-current [`PUBLIC_INDEX`](#public_index) flag — both are interpreted as a coarse mapping onto `PERMISSIONS` for backwards compatibility (`PUBLIC_SNAPSHOTS=False` ⇒ `private`, `PUBLIC_INDEX=False` ⇒ `unlisted`, either set to `True` ⇒ `public`). Setting `PERMISSIONS` directly wins over either legacy flag.
+
+> [!NOTE]
+> `PERMISSIONS` controls **per-Snapshot** visibility. Server-wide auth (whether the whole UI requires login, whether the add-view is open) is still controlled by [`PUBLIC_INDEX`](#public_index) and [`PUBLIC_ADD_VIEW`](#public_add_view) under Server Settings.
+
+*Related options:*
+[`PUBLIC_INDEX`](#public_index), [`PUBLIC_ADD_VIEW`](#public_add_view), [`DELETE_AFTER`](#delete_after)
+
+---
+#### `PLUGINS`
+**Possible Values:** [`""`]/`wget,favicon,screenshot`/`chrome,singlefile,dom`/...
+Comma-separated **whitelist** of plugins to load and run for this archiving run. When empty (the default), ArchiveBox uses the installed/enabled plugin set — i.e. every plugin whose `<PLUGIN>_ENABLED` config evaluates true.
+
+When set, only the listed plugins (plus any plugins they declare as `required_plugins` in their `config.json` — e.g. picking `singlefile` automatically pulls in `chrome`) participate in the run. Equivalent to the CLI flag:
+
+```bash
+archivebox add --plugins=wget,favicon,screenshot https://example.com
+```
+
+Useful for one-off runs ("just grab a screenshot and skip everything else") or for reproducible per-crawl pipelines stored on the Crawl row.
+
+*Related options:*
+[`ENABLED_PLUGINS`](#enabled_plugins)
+
+---
+#### `ENABLED_PLUGINS`
+**Possible Values:** [`""`]/`wget,chrome,singlefile`/...
+Comma-separated **override** of the enabled plugin set, used primarily by the admin UI and REST API to express "these are the plugins I want enabled for this Crawl/Snapshot/Persona" without having to flip every individual `<PLUGIN>_ENABLED` flag.
+
+The distinction vs. [`PLUGINS`](#plugins):
+- `PLUGINS` is the **run-time selector** (what to actually execute on this `add` invocation, with transitive dependency expansion).
+- `ENABLED_PLUGINS` is the **persisted enabled set** (what the UI/API thinks should be on for this scope, used to compute per-plugin `<PLUGIN>_ENABLED` defaults).
+
+When both are set, `PLUGINS` wins for the actual run; `ENABLED_PLUGINS` remains as the stored default for future runs at the same scope.
+
+*Related options:*
+[`PLUGINS`](#plugins)
 
 ---
 
 ## Server Settings
 
-*Options for the web UI, authentication, and reverse proxy configuration.*
+*Options for the web UI, authentication, subdomain routing, and reverse proxy configuration.*
 
 ---
+<a id="admin_username"></a>
+<a id="admin_password"></a>
 #### `ADMIN_USERNAME` / `ADMIN_PASSWORD`
 **Possible Values:** [`None`]/`"admin"`/...
 
-Only used on first run / initial setup in Docker. ArchiveBox will create an admin user with the specified username and password when these options are found in the environment.
+Only used on first run / initial setup in Docker. ArchiveBox will create an admin superuser with the specified username and password when both options are present in the environment at startup. After the user exists, changing these values has no effect — use `archivebox manage changepassword <username>` or the Django admin UI instead.
+
+> [!WARNING]
+> Setting `ADMIN_PASSWORD` via environment variable bakes the secret into your shell history, Docker inspect output, and process listings. For long-lived deployments, set it once during provisioning, create the user, then unset the variable.
 
 More info:
 - https://github.com/ArchiveBox/ArchiveBox/wiki/Setting-up-Authentication
 
+*Related options:*
+[`LDAP_ENABLED`](#ldap_enabled), [`REVERSE_PROXY_USER_HEADER`](#reverse_proxy_user_header)
+
 ---
-#### `PUBLIC_INDEX` / `PUBLIC_SNAPSHOTS` / `PUBLIC_ADD_VIEW`
-**Possible Values:** [`True`]/`False`
-Configure whether or not login is required to use each area of ArchiveBox.
+<a id="public_index"></a>
+<a id="public_add_view"></a>
+#### `PUBLIC_INDEX` / `PUBLIC_ADD_VIEW`
+**Possible Values:** [`True`]/`False` (for `PUBLIC_INDEX`), [`False`]/`True` (for `PUBLIC_ADD_VIEW`)
+
+Server-wide toggles for whether login is required to use each public area of ArchiveBox.
 
 ```bash
-archivebox config --set PUBLIC_INDEX=True        # allow viewing snapshots list without login
-archivebox config --set PUBLIC_SNAPSHOTS=True    # allow viewing snapshot content without login
-archivebox config --set PUBLIC_ADD_VIEW=False    # allow submitting new URLs without login
+archivebox config --set PUBLIC_INDEX=True        # allow viewing the snapshot index without login
+archivebox config --set PUBLIC_ADD_VIEW=False    # require login to submit new URLs via the web UI
 ```
 
+- `PUBLIC_INDEX` (default `True`) — when on, anonymous visitors can browse the snapshot list page. Individual snapshot visibility is still gated by each Snapshot's own [`PERMISSIONS`](#permissions) field.
+- `PUBLIC_ADD_VIEW` (default `False`) — when on, anonymous visitors can submit new URLs to be archived via the `/add` form. Leave this off on any internet-exposed instance unless you actively want a public submission endpoint.
+
+> [!NOTE]
+> **`PUBLIC_SNAPSHOTS` has been removed as a global toggle.** Snapshot visibility is now decided per-Snapshot via the [`PERMISSIONS`](#permissions) field (`public` / `unlisted` / `private`) under General Settings. The old anchors are preserved on `PERMISSIONS` so existing links keep working.
+
+*Related options:*
+[`PERMISSIONS`](#permissions), [`SERVER_SECURITY_MODE`](#server_security_mode), [`ADMIN_USERNAME`](#admin_username--admin_password)
+
 ---
 #### `SECRET_KEY`
-**Possible Values:** *auto-generated random string*
-Django's secret key for cryptographic signing (sessions, CSRF tokens, etc.). Automatically generated on first run.
+**Possible Values:** *auto-generated 50-character random string*
+
+Django's secret key, used for cryptographic signing of sessions, CSRF tokens, password reset links, and other signed payloads. Auto-generated on first server start and persisted to `ArchiveBox.conf` so it survives restarts. If the config file isn't writable (read-only mount, mid-init race), an in-memory random key is used and all users are logged out on the next boot.
+
+> [!WARNING]
+> Treat this value like a password. Anyone with the `SECRET_KEY` can forge sessions and CSRF tokens for your instance. Don't commit `ArchiveBox.conf` to public repos, and rotate it (forcing all users to log in again) if you suspect it's been exposed.
 
 ---
 #### `BIND_ADDR`
-**Possible Values:** [`127.0.0.1:8000`]/`0.0.0.0:8000`/...
-Address and port for the ArchiveBox web server to listen on.
+**Possible Values:** [`127.0.0.1:8000`]/`0.0.0.0:8000`/`[::]:8000`/`0.0.0.0:80`/...
+
+The `host:port` socket the ArchiveBox web server actually listens on. **This is the local bind socket, not the public URL** — for the public URL clients see, set [`BASE_URL`](#base_url).
+
+- `127.0.0.1:8000` (default) — listen only on the loopback interface. Safest when you're running a reverse proxy on the same host and don't want the server reachable directly from the network.
+- `0.0.0.0:8000` — listen on **all** IPv4 interfaces. Required when running in Docker without `--network=host`, or when you want the server reachable from other machines on your LAN without a reverse proxy.
+- `[::]:8000` — listen on all IPv6 interfaces (most modern OSes will accept v4-mapped connections too).
+- `unix:/path/to/archivebox.sock` — bind to a Unix socket instead of a TCP port (useful for nginx/Caddy on the same host).
+
+IPv6 literal addresses must be bracketed: `[::1]:8000`, not `::1:8000`.
+
+> [!NOTE]
+> Inside Docker, binding to `127.0.0.1` means the server is unreachable from outside the container — use `0.0.0.0:8000` and let Docker handle the port-forwarding, or publish the port with `-p 127.0.0.1:8000:8000` on the host side instead.
+
+*Related options:*
+[`BASE_URL`](#base_url), [`SERVER_SECURITY_MODE`](#server_security_mode)
 
 ---
-#### `LISTEN_HOST`
-**Possible Values:** [`archivebox.localhost:8000`]/`archive.example.com:443`/...
-The public hostname and port that ArchiveBox is accessible at.
+<a id="allowed_hosts"></a>
+<a id="csrf_trusted_origins"></a>
+#### `BASE_URL`
+**Possible Values:** [`""`]/`https://archive.example.com`/`http://archivebox.localhost:8000`/...
+
+The canonical public URL of your ArchiveBox instance. Used to build absolute links in templates, redirects (`/admin/login/?next=...`), admin notification emails, OG/meta tags, and — in subdomain security mode — to derive the `admin.`, `web.`, `api.`, `public.`, and per-snapshot `snap-<id>.` subdomains.
+
+**When `BASE_URL` is set explicitly**, ArchiveBox treats it as the source of truth and ignores the incoming `Host` header for URL building. In `safe-subdomains-fullreplay` mode this is **required for redirects to work** — without an explicit base, the middleware can't safely emit `admin.<host>` redirects (they'd compound onto whatever subdomain the request already arrived on).
+
+**When `BASE_URL` is empty**, the value is resolved at request time from the incoming request's `Host` header (with any leading `admin.` / `web.` / `api.` / `public.` / `snap-*.` label stripped to recover the canonical base). Loopback hostnames (`localhost`, `127.0.0.1`, `0.0.0.0`, `::`) are rewritten to `archivebox.localhost` so subdomain routing works without `/etc/hosts` edits. If there's no live request, [`BIND_ADDR`](#bind_addr) is used as a last resort.
+
+The scheme is taken from the explicit `BASE_URL` if set, otherwise from the request (so put a reverse proxy in front for HTTPS and trust `X-Forwarded-Proto`).
+
+ArchiveBox automatically derives the underlying Django `ALLOWED_HOSTS` and `CSRF_TRUSTED_ORIGINS` settings from `BASE_URL` + [`SERVER_SECURITY_MODE`](#server_security_mode), so you do **not** set those directly — the system widens them as needed to admit the admin/web/api/public subdomains.
+
+> [!NOTE]
+> **In `safe-subdomains-fullreplay` mode, pin `BASE_URL` explicitly.** Without it, the misconfig banner will surface in the rendered page and host-based redirects (`/admin` → `admin.<host>`) are suppressed to avoid the `admin.admin.admin.<host>` compounding bug.
+
+> [!NOTE]
+> **Legacy upgrade path (0.7.3 → 0.9):** older deployments that set `CSRF_TRUSTED_ORIGINS=https://archive.example.com` for their reverse-proxy login but never set `BASE_URL` still work — when exactly one CSRF origin is present and `BASE_URL` is empty, ArchiveBox uses that origin as the implicit base URL. New installs should set `BASE_URL` directly; `CSRF_TRUSTED_ORIGINS` is no longer a user-settable knob.
+
+*Related options:*
+[`SERVER_SECURITY_MODE`](#server_security_mode), [`BIND_ADDR`](#bind_addr)
 
 ---
-#### `ALLOWED_HOSTS`
-**Possible Values:** [`*`]/`archive.example.com,localhost`/...
-Comma-separated list of allowed HTTP Host header values. Set this to your domain name(s) in production.
+#### `SERVER_SECURITY_MODE`
+**Possible Values:** [`safe-subdomains-fullreplay`]/`safe-onedomain-nojsreplay`/`unsafe-onedomain-noadmin`/`danger-onedomain-fullreplay`
 
----
-#### `CSRF_TRUSTED_ORIGINS`
-**Possible Values:** [`http://admin.archivebox.localhost:8000`]/`https://archive.example.com`/...
-Comma-separated list of trusted origins for CSRF validation. Must include the scheme (http/https).
+The top-level security posture of the server. Controls how archived content is served, whether the admin/API control plane is reachable, and which host(s) the UI is split across. **This is the most important security knob** — pick the most restrictive mode that still works for your use case.
 
----
-#### `ADMIN_BASE_URL`
-**Possible Values:** [`""`]/`/admin/`/...
-Base URL path for the Django admin interface.
+ArchiveBox splits its surfaces across four logical hosts: `admin.*` (Django admin + session cookies, the entire control plane), `web.*` (logged-in browsing UI), `api.*` (REST/JSON endpoints), and `public.*` (unauthenticated browsing of `PERMISSIONS=public` snapshots). In subdomain mode each gets its own host derived from [`BASE_URL`](#base_url); session/CSRF cookies are scoped to `admin.*` only, so a compromised replay page on `snap-<id>.*` can't read admin auth.
 
----
-#### `ARCHIVE_BASE_URL`
-**Possible Values:** [`""`]/`/archive/`/...
-Base URL path for serving archived content.
+| Mode | Host layout | JS replay | Control plane | Use when |
+|---|---|---|---|---|
+| **`safe-subdomains-fullreplay`** *(default, recommended)* | admin/web/api/public/snap-* on separate subdomains | Full JS replay enabled | Enabled on `admin.*` only | You have wildcard DNS (`*.archive.example.com`) and a TLS cert that covers it. Archived JS runs sandboxed away from the admin origin. |
+| **`safe-onedomain-nojsreplay`** | Everything on one host | JS in replays is neutered (served as `text/plain` or stripped) | Enabled | You can't get wildcard DNS. Trades replay fidelity for same-origin safety — archived pages won't execute scripts. |
+| **`unsafe-onedomain-noadmin`** | Everything on one host | Full JS replay enabled | **Disabled** — `/admin`, `/accounts`, `/api`, `/add`, `/web` return 403; only GET/HEAD/OPTIONS allowed | Read-only public archive on a single host. Operate the instance via CLI only; the web admin is unreachable. |
+| **`danger-onedomain-fullreplay`** | Everything on one host | Full JS replay enabled | Enabled | Local dev / trusted-network only. Archived JS runs on the **same origin as the admin UI** — a malicious archived page can call admin endpoints with your session. **Do not expose this mode to the internet.** |
+
+> [!WARNING]
+> Switching to any mode whose name starts with `unsafe-` or `danger-` is logged at startup and surfaces a banner in the UI. **Don't use these modes on a public hostname** — archived JavaScript will run on the same origin as your admin session.
+
+> [!NOTE]
+> Subdomain mode requires both wildcard DNS (`*.archive.example.com`) and (if using TLS) a wildcard certificate. Without those, fall back to `safe-onedomain-nojsreplay`.
+
+*Related options:*
+[`BASE_URL`](#base_url), [`PERMISSIONS`](#permissions)
+
+More info:
+- [Security Overview](Security-Overview)
 
 ---
 #### `SNAPSHOTS_PER_PAGE`
 **Possible Values:** [`40`]/`100`/...
-Maximum number of Snapshots to show per page on Snapshot list pages.
 
----
-#### `PREVIEW_ORIGINALS`
-**Possible Values:** [`True`]/`False`
-Whether to show inline previews of the original URL on snapshot detail pages.
+Maximum number of Snapshots to render per page on the snapshot list views (both the admin index and the public index). Larger values speed up bulk browsing at the cost of heavier per-request rendering.
 
 ---
 #### `FOOTER_INFO`
 **Possible Values:** [`Content is hosted for personal archiving purposes only.  Contact server owner for any takedown requests.`]/...
-Text to display in the footer of the archive index.
+
+Free-form text rendered in the footer of every archive page. Useful for adding a takedown contact, an org disclaimer, or attribution. Plain text — no HTML.
 
 ---
 #### `CUSTOM_TEMPLATES_DIR`
 **Possible Values:** [`data/custom_templates`]/`/path/to/custom_templates`/...
-Path to a directory containing custom html/css/images for overriding the default UI styling.
+
+Path to a directory containing custom HTML / CSS / image overrides for the default ArchiveBox templates. Files placed here shadow the built-in templates of the same path, letting you rebrand the UI without forking. See the Django template loader docs for the resolution order.
 
 ---
 #### `REVERSE_PROXY_USER_HEADER`
-**Possible Values:** [`Remote-User`]/`X-Remote-User`/...
-HTTP header containing user name from authenticated proxy.
+**Possible Values:** [`Remote-User`]/`X-Remote-User`/`X-Forwarded-User`/...
+
+HTTP header your reverse proxy (Authelia, oauth2-proxy, Authentik, nginx `auth_request`, etc.) sets to the authenticated username. ArchiveBox's `ReverseProxyAuthMiddleware` reads this header **only when the request's source IP is inside [`REVERSE_PROXY_WHITELIST`](#reverse_proxy_whitelist)** — otherwise the header is ignored to prevent direct-connect spoofing.
+
+The header name is matched case-insensitively and normalized to the `HTTP_*` form Django exposes (e.g. `Remote-User` → `HTTP_REMOTE_USER`).
 
 *Related options:*
 [`REVERSE_PROXY_WHITELIST`](#reverse_proxy_whitelist), [`LOGOUT_REDIRECT_URL`](#logout_redirect_url)
 
 ---
 #### `REVERSE_PROXY_WHITELIST`
-**Possible Values:** [`<empty string>`]/`172.16.0.0/16`/...
-Comma separated list of IP CIDRs which are allowed to use reverse proxy authentication.
+**Possible Values:** [`""`]/`172.16.0.0/16`/`10.0.0.5/32,fd00::/8`/...
+
+Comma-separated list of IPv4 / IPv6 addresses or CIDR networks that are trusted to set [`REVERSE_PROXY_USER_HEADER`](#reverse_proxy_user_header). When empty (the default), reverse-proxy auth is **completely disabled** — the header is never consulted no matter who set it.
+
+When non-empty, only requests whose `REMOTE_ADDR` falls inside one of the listed networks have the header honored. Anything else falls back to standard session auth. The CIDR list is validated on every request; an invalid entry raises `ImproperlyConfigured` and breaks the server, so test changes carefully.
+
+> [!WARNING]
+> **Set this to the actual IP of your reverse proxy, never `0.0.0.0/0` or a public network.** With a wide-open whitelist, anyone who can reach the server directly can forge any username they like via the header.
+
+*Related options:*
+[`REVERSE_PROXY_USER_HEADER`](#reverse_proxy_user_header), [`LOGOUT_REDIRECT_URL`](#logout_redirect_url)
 
 ---
 #### `LOGOUT_REDIRECT_URL`
-**Possible Values:** [`/`]/`https://example.com/some/other/app`/...
-URL to redirect users back to on logout when using reverse proxy authentication.
+**Possible Values:** [`/`]/`https://example.com/some/other/app`/`/accounts/logout-landing/`/...
+
+URL users are redirected to after logging out. The default `/` keeps users on ArchiveBox; set this to an external URL when using reverse-proxy SSO so logout terminates the upstream session too (e.g. `https://auth.example.com/logout`).
+
+*Related options:*
+[`REVERSE_PROXY_USER_HEADER`](#reverse_proxy_user_header), [`REVERSE_PROXY_WHITELIST`](#reverse_proxy_whitelist)
 
 ---
 
 ### LDAP Settings
 
-*Options for LDAP/Active Directory authentication. Requires `pip install archivebox[ldap]`.*
+*Options for LDAP / Active Directory authentication via [django-auth-ldap](https://github.com/django-auth-ldap/django-auth-ldap). Requires `pip install archivebox[ldap]` (which also pulls in the system `libldap` / `libsasl` headers).*
 
 ---
 #### `LDAP_ENABLED`
 **Possible Values:** [`False`]/`True`
-Whether to use an external LDAP server for authentication.
+
+Master switch for LDAP authentication. When `True`, ArchiveBox loads the `django-auth-ldap` backend and validates that `LDAP_SERVER_URI`, `LDAP_BIND_DN`, `LDAP_BIND_PASSWORD`, and `LDAP_USER_BASE` are all set — startup fails fast otherwise.
 
 ```bash
 pip install archivebox[ldap]
@@ -267,104 +517,304 @@ More info:
 - https://github.com/ArchiveBox/ArchiveBox/wiki/Setting-up-Authentication
 - https://github.com/django-auth-ldap/django-auth-ldap#example-configuration
 
+*Related options:*
+[`ADMIN_USERNAME`](#admin_username--admin_password), [`REVERSE_PROXY_USER_HEADER`](#reverse_proxy_user_header)
+
 ---
 #### `LDAP_SERVER_URI`
-**Default:** [`None`]
-LDAP server URI (e.g. `ldap://ldap.example.com:389`).
+**Possible Values:** [`None`]/`ldap://ldap.example.com:389`/`ldaps://ldap.example.com:636`/...
+
+URI of the LDAP server to bind against. Use `ldaps://` for TLS or `ldap://` for plaintext (plus optional StartTLS at the protocol level). Required when [`LDAP_ENABLED`](#ldap_enabled) is `True`.
 
 ---
 #### `LDAP_BIND_DN`
-**Default:** [`None`]
-DN to bind for searching.
+**Possible Values:** [`None`]/`cn=archivebox,ou=services,dc=example,dc=com`/...
+
+Distinguished name of the service account used to perform user searches. This account only needs read access to the user subtree under [`LDAP_USER_BASE`](#ldap_user_base). Required when [`LDAP_ENABLED`](#ldap_enabled) is `True`.
 
 ---
 #### `LDAP_BIND_PASSWORD`
-**Default:** [`None`]
-Password for bind DN.
+**Possible Values:** [`None`]/`<bind-service-password>`/...
+
+Password for the [`LDAP_BIND_DN`](#ldap_bind_dn) service account. Required when [`LDAP_ENABLED`](#ldap_enabled) is `True`.
+
+> [!WARNING]
+> Treat this like any other service credential — keep it out of shell history and version control. Prefer setting it via the config file (which has owner-only permissions) over environment variables.
 
 ---
 #### `LDAP_USER_BASE`
-**Default:** [`None`]
-Base DN for user searches.
+**Possible Values:** [`None`]/`ou=users,dc=example,dc=com`/...
+
+Base DN under which to search for user entries. Required when [`LDAP_ENABLED`](#ldap_enabled) is `True`. The search is performed as `LDAP_BIND_DN` with the filter from [`LDAP_USER_FILTER`](#ldap_user_filter).
 
 ---
 #### `LDAP_USER_FILTER`
-**Default:** [`(uid=%(user)s)`]
-LDAP search filter for users.
+**Possible Values:** [`(uid=%(user)s)`]/`(sAMAccountName=%(user)s)`/`(&(objectClass=person)(mail=%(user)s))`/...
+
+LDAP search filter used to find a user entry at login. The literal token `%(user)s` is replaced with the username the user typed into the login form. Common values:
+- `(uid=%(user)s)` — OpenLDAP-style
+- `(sAMAccountName=%(user)s)` — Active Directory
+- `(mail=%(user)s)` — match by email
 
 ---
 #### `LDAP_USERNAME_ATTR`
-**Default:** [`username`]
-LDAP attribute for username.
+**Possible Values:** [`username`]/`uid`/`sAMAccountName`/...
+
+LDAP attribute on the user entry that becomes the local Django `username`. Must be unique within the directory.
 
 ---
 #### `LDAP_FIRSTNAME_ATTR`
-**Default:** [`givenName`]
-LDAP attribute for first name.
+**Possible Values:** [`givenName`]/...
+
+LDAP attribute mapped to Django's `User.first_name`.
 
 ---
 #### `LDAP_LASTNAME_ATTR`
-**Default:** [`sn`]
-LDAP attribute for last name.
+**Possible Values:** [`sn`]/...
+
+LDAP attribute mapped to Django's `User.last_name`.
 
 ---
 #### `LDAP_EMAIL_ATTR`
-**Default:** [`mail`]
-LDAP attribute for email.
+**Possible Values:** [`mail`]/`userPrincipalName`/...
+
+LDAP attribute mapped to Django's `User.email`.
 
 ---
 #### `LDAP_CREATE_SUPERUSER`
-**Default:** [`False`]
-Auto-create superuser accounts for LDAP users.
+**Possible Values:** [`False`]/`True`
+
+When `True`, every LDAP user who successfully authenticates is auto-promoted to Django superuser. **Off by default** — leave it off unless your directory's user base is already restricted to operators, since superusers can modify config, delete snapshots, and run server commands.
+
+> [!WARNING]
+> Combining `LDAP_CREATE_SUPERUSER=True` with a broad [`LDAP_USER_BASE`](#ldap_user_base) (e.g. an entire company OU) effectively grants admin to every employee. Scope the user base or use group-based access control via `django-auth-ldap`'s `AUTH_LDAP_USER_FLAGS_BY_GROUP` (configured in custom `settings.py`) instead.
 
 ---
 
 ## Storage Settings
 
-*Options for file layout, permissions, and temp/lib directories.*
+*Options for the on-disk layout, file permissions, and temp/lib directories that ArchiveBox reads and writes during archiving.*
 
 ---
+<a id="dir_output_permissions"></a>
 #### `OUTPUT_PERMISSIONS`
 **Possible Values:** [`644`]/`755`/...
-Permissions to set output files to.
+Permissions to set on output files written into the [`ARCHIVE_DIR`](#archive_dir). The directory mode is derived from this by OR-ing in the execute bits (so `644` files imply `755` dirs), which subsumes the legacy `DIR_OUTPUT_PERMISSIONS` option (formerly a separate `755`-default field) — directory mode is no longer settable on its own.
+
+> [!NOTE]
+> Set this to `600` if you want archives to be readable only by the ArchiveBox user, or `664`/`775` if you need a shared group to read/write the data dir.
 
 *Related options:*
-[`PUID` / `PGID`](#puid--pgid)
+[`PUID` / `PGID`](#puid--pgid), [`ENFORCE_ATOMIC_WRITES`](#enforce_atomic_writes)
 
 ---
+<a id="puid"></a>
+<a id="pgid"></a>
 #### `PUID` / `PGID`
 **Possible Values:** [`911`]/`1000`/...
-*Note: Only applicable for Docker users, settable via environment variables only.*
-User and Group ID that the data directory should be owned by.
+*Note: These are Docker-only environment variables — they only take effect when set on the Docker entrypoint at container startup. Setting them in `ArchiveBox.conf` or via `archivebox config --set` has no effect. Outside Docker the UID/GID is auto-detected from the ownership of the data directory (or the running user) and cannot be overridden.*
+
+The UID/GID that the ArchiveBox process should run as (and that all files in the data dir should be owned by). Honored by the Docker entrypoint, which `chown`s the data dir and drops privileges before running ArchiveBox. Outside Docker, ArchiveBox refuses to run as root and instead drops to the user that owns the data dir.
 
 *Learn more:*
 - https://docs.linuxserver.io/general/understanding-puid-and-pgid/
 - https://github.com/ArchiveBox/ArchiveBox/wiki/Troubleshooting#docker-permissions-issues
 
----
-#### `RESTRICT_FILE_NAMES`
-**Possible Values:** [`windows`]/`unix`/`ascii`/...
-Restrict output filenames to be compatible with the given filesystem type.
-
 ---
 #### `ENFORCE_ATOMIC_WRITES`
 **Possible Values:** [`True`]/`False`
-Whether to use atomic writes when saving files.
+Whether to write output files atomically (write to a tempfile + `rename()` into place) so that a crash or `kill -9` mid-write can never leave a partial file in the archive. Disable only if you are debugging a filesystem that doesn't support atomic renames (some FUSE mounts).
 
 ---
 #### `TMP_DIR`
-**Possible Values:** [`data/tmp/<machine_id>`]/`/tmp/archivebox/abc5d851`/...
-Path for temporary files, unix sockets, and supervisor config. Must be a local, fast, short-path directory.
+**Possible Values:** [`<DATA_DIR>/tmp/<machine_id>`]/`/tmp/archivebox/abc5d851`/...
+Path for temporary files, the supervisord unix socket, and generated supervisor config. The default is a per-machine subdirectory under the data dir (`tmp/<machine_id>`) so multiple machines sharing the same data dir (e.g., over NFS) don't collide on socket files.
+
+> [!WARNING]
+> `TMP_DIR` *must* be a short, local path readable/writable by the ArchiveBox user. Unix socket paths have a hard ~96-character limit, so a deeply nested `TMP_DIR` will silently break the supervisor. It also must live on a real local filesystem (tmpfs/SSD) — FUSE, network mounts, and Docker bind mounts on macOS often cannot host unix sockets at all (see [`ALLOW_NO_UNIX_SOCKETS`](#allow_no_unix_sockets)).
+
+If ArchiveBox detects the configured `TMP_DIR` is unwritable or too long, it will auto-fall-back to `/tmp/archivebox/<collection_id>` at startup.
+
+*Related options:*
+[`LIB_DIR`](#lib_dir), [`ALLOW_NO_UNIX_SOCKETS`](#allow_no_unix_sockets)
 
 ---
 #### `LIB_DIR`
-**Possible Values:** [`data/lib/<arch>-<os>`]/`/usr/local/share/archivebox/abc5`/...
-Path for installed binary dependencies.
+**Possible Values:** [`<DATA_DIR>/lib/<arch>-<os>`]/`/opt/archivebox/lib`/`~/.config/abx/lib`/...
+Path for installed binary dependencies (`chromium`, `single-file`, `yt-dlp`, `ripgrep`, etc.) managed by `abxpkg`. The default is namespaced by architecture/OS (e.g. `arm64-darwin`, `x86_64-linux-docker`) so the same data dir can be safely mounted into containers with different CPU architectures without re-downloading binaries.
+
+> [!NOTE]
+> `LIB_DIR` can grow to several GB. Put it on a fast local disk — running extractors off a network-mounted `LIB_DIR` will be painfully slow.
+
+*Related options:*
+[`LIB_BIN_DIR`](#lib_bin_dir), [`TMP_DIR`](#tmp_dir)
 
 ---
 #### `LIB_BIN_DIR`
-**Possible Values:** [`LIB_DIR/bin`]
-Path where installed binaries are symlinked for easy PATH management.
+**Possible Values:** [`<LIB_DIR>/bin`]
+Path where installed binaries are symlinked for a flat, shared lookup `PATH`. Both `abxpkg` and `abx-dl` build the executable-resolution environment from this directory at exec time, so anything dropped (or symlinked) here becomes available to all extractor hooks.
+
+Almost no one needs to change this — it tracks [`LIB_DIR`](#lib_dir) automatically when `LIB_DIR` is overridden.
+
+---
+#### `DATA_DIR`
+**Possible Values:** [`<cwd>`]/`/data`/`~/archivebox-data`/...
+The root of an ArchiveBox collection. Holds `index.sqlite3`, `ArchiveBox.conf`, the [`ARCHIVE_DIR`](#archive_dir), [`PERSONAS_DIR`](#personas_dir), `sources/`, `logs/`, `cache/`, etc.
+
+Normally you do *not* set this explicitly — instead you `cd` into the data folder and run `archivebox` there, and `DATA_DIR` defaults to the current working directory. The `DATA_DIR` environment variable is available as an override (used internally by the test suite and some wrappers), but if it's set it must match the cwd or ArchiveBox will refuse to start — this is a guardrail against accidentally pointing two different processes at different roots.
+
+> [!WARNING]
+> ArchiveBox refuses to run as root, refuses to run from an unwritable directory, and refuses to run when `DATA_DIR` disagrees with the current working directory. Always `cd` into your data folder first.
+
+---
+#### `ARCHIVE_DIR`
+**Possible Values:** [`<DATA_DIR>/archive`]
+Where Snapshot output directories are written. This is the heavy directory — every archived URL gets a subtree here. Override it when you want index/config to live on a small fast disk but snapshot data on bulk storage:
+
+```bash
+archivebox config --set ARCHIVE_DIR=/mnt/bulk/archivebox/archive
+```
+
+Relative paths are resolved against [`DATA_DIR`](#data_dir).
+
+*Related options:*
+[`USERS_DIR`](#users_dir), [`DATA_DIR`](#data_dir)
+
+---
+#### `USERS_DIR`
+**Possible Values:** [`<ARCHIVE_DIR>/users`]
+Root of the per-user namespace inside the archive. Each ArchiveBox user gets a subdir (`users/<username>/crawls/...` and `users/<username>/snapshots/...`) so multiple users sharing one collection do not collide on output paths, and per-user retention/permission policies are easy to enforce at the filesystem level.
+
+Relative paths are resolved against [`ARCHIVE_DIR`](#archive_dir).
+
+---
+#### `PERSONAS_DIR`
+**Possible Values:** [`<DATA_DIR>/personas`]
+Where persona state lives — Chrome user-data-dirs, cookie jars, sessionstorage, and any other auth/profile state that should follow a "persona" across snapshots. Each persona owns a subdirectory here that gets bind-mounted (or pointed at via `CHROME_USER_DATA_DIR`) when extractors run on its behalf.
+
+> [!WARNING]
+> `PERSONAS_DIR` typically contains plaintext cookies and logged-in browser sessions. Treat it as secret material — set restrictive [`OUTPUT_PERMISSIONS`](#output_permissions) (e.g. `600`) and never commit it to git or include it in shared backups without encryption.
+
+---
+#### `CRAWL_DIR`
+**Possible Values:** *runtime-injected, default `None`*
+The output directory of the *currently running crawl* (e.g. `<USERS_DIR>/<username>/crawls/YYYYMMDD/<domain>/<crawl-id>/`). Crawl-level extractors (chrome launcher, parsers, etc.) write here.
+
+You almost never set this yourself — the snapshot/crawl orchestrator injects it into the per-call config and passes it through to plugin hooks via the `CRAWL_DIR` environment variable. It is documented here for plugin authors who need to read `config.CRAWL_DIR` from inside a hook to locate sibling crawl-level outputs.
+
+*Related options:*
+[`SNAP_DIR`](#snap_dir), [`USERS_DIR`](#users_dir)
+
+---
+#### `SNAP_DIR`
+**Possible Values:** *runtime-injected, default `None`*
+The output directory of the *currently running snapshot* (e.g. `<USERS_DIR>/<username>/snapshots/YYYYMMDD/<domain>/<snapshot-uuid>/`). Snapshot-level extractors (screenshot, pdf, dom, singlefile, etc.) write their output into per-plugin subdirectories of this path.
+
+Like [`CRAWL_DIR`](#crawl_dir), this is set per-call by the orchestrator and passed to hooks via the `SNAP_DIR` environment variable — it is not something users configure. Documented only so plugin authors know which config key to read inside a hook.
+
+*Related options:*
+[`CRAWL_DIR`](#crawl_dir), [`ARCHIVE_DIR`](#archive_dir)
+
+---
+#### `ALLOW_NO_UNIX_SOCKETS`
+**Possible Values:** [`False`]/`True`
+**Alias:** `ARCHIVEBOX_ALLOW_NO_UNIX_SOCKETS`
+
+Skip the startup check that verifies [`TMP_DIR`](#tmp_dir) can host unix-domain sockets (a real `bind()` on a `.sock` file). Set to `True` only when running ArchiveBox on a filesystem that cannot back unix sockets — most commonly Docker Desktop on macOS with a host bind-mounted `TMP_DIR`, where the osxfs/virtiofs layer rejects `bind()` calls.
+
+> [!WARNING]
+> This disables a real safety check, not a cosmetic one. When unix sockets are unavailable some plugins that talk to long-lived helpers over `.sock` files (supervisord control socket, browser launcher RPC) may behave unpredictably. Prefer fixing [`TMP_DIR`](#tmp_dir) to point at a tmpfs/SSD inside the container; reach for `ALLOW_NO_UNIX_SOCKETS` only when that's genuinely not possible.
+
+*Related options:*
+[`TMP_DIR`](#tmp_dir)
+
+---
+
+## Database Settings
+
+*Options for tuning the SQLite index database that backs ArchiveBox's snapshot, tag, and crawl metadata.*
+
+ArchiveBox stores all of its index metadata in a single SQLite database file (`index.sqlite3` inside your data directory). The defaults are tuned for nearly all users — the knobs below mostly govern **lock-contention behavior**, which matters when multiple workers touch the database concurrently (e.g. supervised orchestrators, parallel `archivebox add` runs, container restarts that race against an in-flight write, or long-running web/admin processes alongside CLI commands).
+
+> [!NOTE]
+> These are advanced operator tuning options. If you are not actively diagnosing `database is locked` errors or planning a non-default storage layout, you can safely leave everything in this section at its default.
+
+*Learn more:*
+- https://github.com/ArchiveBox/ArchiveBox/wiki/Troubleshooting#sqlite-database-is-locked
+- https://www.sqlite.org/wal.html
+- https://www.sqlite.org/pragma.html
+
+---
+#### `DATABASE_NAME`
+**Possible Values:** [`<DATA_DIR>/index.sqlite3`]/`/absolute/path/to/index.sqlite3`/...
+Absolute filesystem path to the SQLite index database file. Settable as the environment variable `ARCHIVEBOX_DATABASE_NAME`.
+
+By default this resolves to `index.sqlite3` inside your data directory and you should not need to change it. Override only when you have a specific reason — e.g. pointing a temporary process at a snapshot of the DB for testing, running multiple ArchiveBox instances out of the same data directory against separate indexes, or relocating the index file onto a different volume.
+
+> [!WARNING]
+> The data directory layout (snapshots, tags, archive folders on disk) is keyed off the index database. Pointing `DATABASE_NAME` at a database that does not match the surrounding data directory will produce broken references and missing archive folders.
+
+---
+#### `SQLITE_JOURNAL_MODE`
+**Possible Values:** [`WAL`]/`DELETE`/`TRUNCATE`/`PERSIST`/`MEMORY`/`OFF`
+SQLite [journal mode](https://www.sqlite.org/pragma.html#pragma_journal_mode), applied via `PRAGMA journal_mode = ...` on every new connection. Settable as `ARCHIVEBOX_SQLITE_JOURNAL_MODE`.
+
+The default `WAL` (Write-Ahead Logging) lets readers and a single writer operate concurrently without blocking each other — readers see a stable snapshot while a write is in progress, instead of being serialized behind it. This is a substantial win for ArchiveBox, where the web UI, admin, and CLI workers frequently read the index while an extractor is writing.
+
+> [!WARNING]
+> Do not change this unless you have a specific reason. `DELETE` and `TRUNCATE` serialize all readers against any writer (much worse concurrency). `MEMORY` and `OFF` disable durable journaling and can corrupt the database on crash or power loss. `WAL` requires the database to live on a real local filesystem — it does not work correctly over network filesystems like NFS or SMB.
+
+---
+#### `SQLITE_MMAP_SIZE`
+**Possible Values:** [`134217728`] (128 MiB) on bare-metal, [`0`] (disabled) inside Docker / `0` / `268435456` / ...
+Maximum number of bytes of the database file SQLite is allowed to map into memory via `mmap()`, applied via `PRAGMA mmap_size = ...`. Settable as `ARCHIVEBOX_SQLITE_MMAP_SIZE`.
+
+When mmap is enabled, SQLite reads pages directly from the OS page cache instead of issuing `read()` syscalls and copying into a userspace buffer — meaningfully faster page reads on large databases when there is RAM available to cache them. Setting this to `0` disables memory-mapped I/O entirely and falls back to regular `read()` calls.
+
+*Note: The default is `0` (disabled) inside Docker, because the container's reported memory limits often do not reflect the host page cache and large mmap regions can interact poorly with `cgroup` accounting. On bare-metal installs the default is `134217728` (128 MiB).*
+
+---
+#### `SQLITE_TIMEOUT`
+**Possible Values:** [`30.0`]/`5.0`/`60.0`/... (seconds, float)
+Python `sqlite3` connection-level busy timeout in **seconds**, passed as the `timeout=` argument when the Django backend opens a connection. Settable as `ARCHIVEBOX_SQLITE_TIMEOUT`.
+
+This is the maximum amount of time the underlying Python driver will wait on a contended lock before raising `OperationalError: database is locked`. Raise it if you see spurious lock errors under sustained write contention and you would rather block than fail; lower it if you want callers to fail fast.
+
+*Related options:*
+[`SQLITE_BUSY_TIMEOUT`](#sqlite_busy_timeout), [`SQLITE_LOCK_RETRY_TIMEOUT`](#sqlite_lock_retry_timeout)
+
+---
+#### `SQLITE_BUSY_TIMEOUT`
+**Possible Values:** [`30000`]/`5000`/`60000`/... (milliseconds, integer)
+SQLite-internal busy-wait timeout in **milliseconds**, applied via `PRAGMA busy_timeout = ...` on every new connection. Settable as `ARCHIVEBOX_SQLITE_BUSY_TIMEOUT`.
+
+This is SQLite's own retry-on-busy loop, sitting one layer below [`SQLITE_TIMEOUT`](#sqlite_timeout): when a statement encounters a write lock, SQLite will sleep and retry internally for up to this many milliseconds before returning `SQLITE_BUSY` to the Python driver. The default (`30000` = 30 seconds) is deliberately matched to [`SQLITE_TIMEOUT`](#sqlite_timeout).
+
+> [!WARNING]
+> Easy to confuse with [`SQLITE_TIMEOUT`](#sqlite_timeout): this one is in **milliseconds**, that one is in **seconds**. Keep them aligned in real time when adjusting either.
+
+---
+#### `SQLITE_LOCK_RETRY_TIMEOUT`
+**Possible Values:** [`60.0`]/`0`/`120.0`/... (seconds, float)
+Total wall-clock budget in **seconds** that ArchiveBox's own retry loop will spend re-attempting a single locked statement before aborting it. Settable as `ARCHIVEBOX_SQLITE_LOCK_RETRY_TIMEOUT`.
+
+When the SQLite driver eventually surfaces a `database is locked` error (after [`SQLITE_BUSY_TIMEOUT`](#sqlite_busy_timeout) / [`SQLITE_TIMEOUT`](#sqlite_timeout) have already elapsed), ArchiveBox wraps the cursor in a higher-level retry loop that logs the locking holders and re-issues the statement. This is the maximum total time spent in that outer loop, across all retries, before giving up and raising. Set to `0` to disable the cap and retry indefinitely.
+
+> [!NOTE]
+> The outer retry only applies to statements that are *not* inside an explicit `transaction.atomic()` block. Statements inside an explicit transaction propagate the error to the caller immediately, since silently retrying would re-execute statements the caller already considered committed.
+
+*Related options:*
+[`SQLITE_LOCK_RETRY_INTERVAL`](#sqlite_lock_retry_interval)
+
+---
+#### `SQLITE_LOCK_RETRY_INTERVAL`
+**Possible Values:** [`5.0`]/`1.0`/`10.0`/... (seconds, float, must be `> 0`)
+Sleep duration in **seconds** between successive attempts inside the ArchiveBox lock-retry loop. Settable as `ARCHIVEBOX_SQLITE_LOCK_RETRY_INTERVAL`.
+
+Lower values retry more aggressively (useful if you expect locks to clear quickly and want to minimize end-to-end latency); higher values reduce log noise and wasted CPU when locks are typically held for a long time. Must be strictly greater than `0`.
+
+*Related options:*
+[`SQLITE_LOCK_RETRY_TIMEOUT`](#sqlite_lock_retry_timeout)
 
 ---
 
@@ -372,1089 +822,354 @@ Path where installed binaries are symlinked for easy PATH management.
 
 *Options for full-text search backend configuration.*
 
----
-#### `USE_INDEXING_BACKEND`
-**Possible Values:** [`True`]/`False`
-Enable the search indexing backend.
+ArchiveBox can index Snapshot text/HTML output into a searchable index that powers the search bar in the Web UI and the `archivebox search <query>` CLI command. Multiple backend engines are supported — pick the one that best matches your collection size, available system resources, and tolerance for extra moving parts.
 
----
-#### `USE_SEARCHING_BACKEND`
-**Possible Values:** [`True`]/`False`
-Enable the search querying backend.
+> [!NOTE]
+> Each backend has its own tuning knobs (e.g. [Sonic](https://archivebox.github.io/abx-plugins/#search_backend_sonic) host/port, [ripgrep](https://archivebox.github.io/abx-plugins/#search_backend_ripgrep) flags, [SQLite FTS](https://archivebox.github.io/abx-plugins/#search_backend_sqlite) database path). Those backend-specific options now live with the plugin that implements them — see the [abx-plugins docs](https://archivebox.github.io/abx-plugins/) for the full per-backend schema.
 
 ---
 #### `SEARCH_BACKEND_ENGINE`
 **Possible Values:** [`ripgrep`]/`sqlite`/`sonic`
-Which search backend engine to use. `ripgrep` (default) requires no setup. `sqlite` uses FTS5. `sonic` requires a running Sonic instance.
 
----
-#### `SEARCH_PROCESS_HTML`
-**Possible Values:** [`True`]/`False`
-Whether to strip HTML tags before indexing content for search.
+Which search backend engine to use when running `archivebox search` and rendering the Web UI search bar.
+
+- **`ripgrep`** *(default)* — Pure filesystem grep across each Snapshot's archived output (HTML, text, metadata) via the [`search_backend_ripgrep`](https://archivebox.github.io/abx-plugins/#search_backend_ripgrep) plugin. No extra daemon, no extra database to maintain — just install `rg` and it works. Slow on very large collections (each query re-scans the disk) but always 100% correct: results reflect what's actually on disk *right now*, no stale index. Best choice for small-to-medium collections (≲50k snapshots) and for users who don't want to run extra services.
+
+- **`sonic`** — Fast, suggest-style fuzzy search via a running [Sonic](https://github.com/valeriansaliou/sonic) daemon (configured via the [`search_backend_sonic`](https://archivebox.github.io/abx-plugins/#search_backend_sonic) plugin). ArchiveBox pushes text into Sonic at index time and queries it at search time. Sub-millisecond queries even at very large scale, but you have to run and maintain the Sonic process (Docker compose has it built in). Best choice for large collections (≳100k snapshots) when query latency matters.
+
+- **`sqlite`** — FTS5 full-text index stored alongside ArchiveBox's main `index.sqlite3`, configured via the [`search_backend_sqlite`](https://archivebox.github.io/abx-plugins/#search_backend_sqlite) plugin. No extra processes, no extra binary — uses the SQLite already shipped with Python. Faster than `ripgrep` on large collections, slightly slower than `sonic`, but no daemon to babysit. Good middle ground for users who want a real index without operational overhead.
+
+*Note: Backend-specific tuning ([Sonic](https://archivebox.github.io/abx-plugins/#search_backend_sonic) host/port/password, [ripgrep](https://archivebox.github.io/abx-plugins/#search_backend_ripgrep) flag overrides, [SQLite FTS](https://archivebox.github.io/abx-plugins/#search_backend_sqlite) database path, indexer batch size, etc.) lives in each search-backend plugin's own config schema — see the [abx-plugins docs](https://archivebox.github.io/abx-plugins/) for the full per-backend option list.*
 
 ---
 
 ## Shell Options
 
-*Options around the format of the CLI output.*
+*Options around the format & behavior of CLI output.*
+
+Most of the values in this section are auto-detected from your terminal at startup, but each can be overridden explicitly via env var, `ArchiveBox.conf`, or `archivebox config --set` — useful for CI logs, cron jobs, log files, and Docker stdout where the auto-detection isn't what you want.
 
 ---
 #### `DEBUG`
 **Possible Values:** [`False`]/`True`
-Enable debug mode. Automatically set to `True` if `--debug` is passed on the command line.
 
----
-#### `IS_TTY`
-**Possible Values:** *auto-detected*
-Whether stdout is a TTY (interactive terminal).
+Enable verbose debug mode for the entire ArchiveBox process. Automatically set to `True` when `--debug` is passed on the command line; otherwise honors the env var / config value.
+
+When enabled this turns on:
+- Full Python tracebacks (instead of the trimmed friendly version) on any error
+- Django SQL query logging to stderr
+- Template auto-reload (no caching) for the web UI
+- Verbose plugin / hook lifecycle logging
+- Extra detail in `archivebox version`, `archivebox status`, and crash reports
+
+> [!WARNING]
+> **Do not leave `DEBUG=True` enabled on a production / publicly-reachable server.** It exposes tracebacks with file paths, SQL queries, and environment details that can leak sensitive info to anyone who triggers an error page.
+
+*Related options:* [`USE_COLOR`](#use_color), [`SHOW_PROGRESS`](#show_progress)
 
 ---
 #### `USE_COLOR`
-**Possible Values:** [`True`]/`False`
-Colorize console output. Defaults to `True` if stdin is a TTY.
+**Possible Values:** [`True` *(auto-detected)*]/`False`
+
+Whether to colorize console output with ANSI escape codes. Defaults to `True` when stdout is a TTY (interactive terminal) and `False` otherwise.
+
+Override to **force-off** when piping `archivebox` output into a log file or cron-mail wrapper that doesn't strip ANSI codes (otherwise you'll see `^[[31m...^[[0m` litter throughout your logs). Override to **force-on** for tools like `script(1)` or some CI runners that don't report as a TTY but *do* render ANSI correctly.
+
+```bash
+USE_COLOR=False archivebox add https://example.com >> archive.log
+```
+
+*Related options:* [`SHOW_PROGRESS`](#show_progress), [`DEBUG`](#debug)
 
 ---
 #### `SHOW_PROGRESS`
-**Possible Values:** [`True`]/`False`
-Show real-time progress bar in console output. Defaults to `True` if stdin is a TTY.
+**Possible Values:** [`True` *(auto-detected)*]/`False`
 
----
-#### `IN_DOCKER`
-**Possible Values:** [`False`]/`True`
-Whether ArchiveBox is running inside a Docker container.
+Whether to render live progress bars during long-running operations (archiving, indexing, migrations). Defaults to `True` when stdout is a TTY, `False` otherwise.
 
----
-#### `IN_QEMU`
-**Possible Values:** [`False`]/`True`
-Whether ArchiveBox is running inside QEMU emulation.
-
----
-
-## Plugin Settings
-
-ArchiveBox uses a plugin system where each extractor defines its own configuration via `config.json` files. All plugin config options can be set the same way as core options — via environment variables, `ArchiveBox.conf`, or `archivebox config --set`.
+Override to **force-off** in environments where the auto-detection is fooled into thinking it has a TTY (some Docker setups, Kubernetes log collectors, `tmux`/`screen` pipes) but the redrawing carriage-return output ends up as garbage in your logs.
 
 ```bash
-archivebox config                              # see all available config options
-archivebox config --set SCREENSHOT_TIMEOUT=120  # set a plugin option
+SHOW_PROGRESS=False archivebox add < urls.txt
 ```
 
-For the full list of plugins and their config schemas, see the [abx-plugins repository](https://github.com/ArchiveBox/abx-plugins).
-
-### Title Settings
-
-#### `TITLE_ENABLED`
-**Default:** [`True`]
-Enable title extraction
-
----
-#### `TITLE_TIMEOUT`
-**Default:** [`30`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for title extraction in seconds
-
-
-### Favicon Settings
-
-#### `FAVICON_ENABLED`
-**Default:** [`True`]
-Enable favicon downloading
-
----
-#### `FAVICON_TIMEOUT`
-**Default:** [`30`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for favicon fetch in seconds
-
----
-#### `FAVICON_USER_AGENT`
-**Default:** [`""`] *(falls back to [`USER_AGENT`](#user_agent))*
-User agent string
-
-
-### Wget Settings
-
-#### `WGET_ARGS`
-**Default:** [*see defaults*]
-Default wget arguments
-
----
-#### `WGET_ARGS_EXTRA`
-**Default:** [`[]`]
-Extra arguments to append to wget command
-
----
-#### `WGET_BINARY`
-**Default:** [`wget`]
-Path to wget binary
-
----
-#### `WGET_CHECK_SSL_VALIDITY`
-**Default:** [`True`] *(falls back to [`CHECK_SSL_VALIDITY`](#check_ssl_validity))*
-Whether to verify SSL certificates
-
----
-#### `WGET_COOKIES_FILE`
-**Default:** [`""`] *(falls back to [`COOKIES_FILE`](#cookies_file))*
-Path to cookies file
-
----
-#### `WGET_ENABLED`
-**Default:** [`True`]
-Enable wget archiving
-
----
-#### `WGET_TIMEOUT`
-**Default:** [`60`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for wget in seconds
-
----
-#### `WGET_USER_AGENT`
-**Default:** [`""`] *(falls back to [`USER_AGENT`](#user_agent))*
-User agent string for wget
-
----
-#### `WGET_WARC_ENABLED`
-**Default:** [`True`]
-Save WARC archive file
-
-
-### Screenshot Settings
-
-#### `SCREENSHOT_ENABLED`
-**Default:** [`True`]
-Enable screenshot capture
-
----
-#### `SCREENSHOT_RESOLUTION`
-**Default:** [`1440,2000`] *(falls back to [`RESOLUTION`](#resolution))*
-Screenshot resolution (width,height)
-
----
-#### `SCREENSHOT_TIMEOUT`
-**Default:** [`60`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for screenshot capture in seconds
-
-
-### PDF Settings
-
-#### `PDF_ENABLED`
-**Default:** [`True`]
-Enable PDF generation
-
----
-#### `PDF_RESOLUTION`
-**Default:** [`1440,2000`] *(falls back to [`RESOLUTION`](#resolution))*
-PDF page resolution (width,height)
-
----
-#### `PDF_TIMEOUT`
-**Default:** [`60`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for PDF generation in seconds
-
-
-### DOM Settings
-
-#### `DOM_ENABLED`
-**Default:** [`True`]
-Enable DOM capture
-
----
-#### `DOM_TIMEOUT`
-**Default:** [`60`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for DOM capture in seconds
-
-
-### SingleFile Settings
-
-#### `SINGLEFILE_ARGS`
-**Default:** [`['--browser-headless']`]
-Default single-file arguments
-
----
-#### `SINGLEFILE_ARGS_EXTRA`
-**Default:** [`[]`]
-Extra arguments to append to single-file command
-
----
-#### `SINGLEFILE_BINARY`
-**Default:** [`single-file`]
-Path to single-file binary
-
----
-#### `SINGLEFILE_CHECK_SSL_VALIDITY`
-**Default:** [`True`] *(falls back to [`CHECK_SSL_VALIDITY`](#check_ssl_validity))*
-Whether to verify SSL certificates
-
----
-#### `SINGLEFILE_CHROME_ARGS`
-**Default:** [`[]`] *(falls back to [`CHROME_ARGS`](#chrome_args))*
-Chrome command-line arguments for SingleFile
-
-#### `SINGLEFILE_COOKIES_FILE`
-**Default:** [`""`] *(falls back to [`COOKIES_FILE`](#cookies_file))*
-Path to cookies file
-
----
-#### `SINGLEFILE_ENABLED`
-**Default:** [`True`]
-Enable SingleFile archiving
-
-#### `SINGLEFILE_TIMEOUT`
-**Default:** [`60`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for SingleFile in seconds
-
----
-#### `SINGLEFILE_USER_AGENT`
-**Default:** [`""`] *(falls back to [`USER_AGENT`](#user_agent))*
-User agent string
-
-
-### Readability Settings
-
-#### `READABILITY_ARGS`
-**Default:** [`[]`]
-Default Readability arguments
-
----
-#### `READABILITY_ARGS_EXTRA`
-**Default:** [`[]`]
-Extra arguments to append to Readability command
-
----
-#### `READABILITY_BINARY`
-**Default:** [`readability-extractor`]
-Path to readability-extractor binary
-
----
-#### `READABILITY_ENABLED`
-**Default:** [`True`]
-Enable Readability text extraction
-
----
-#### `READABILITY_TIMEOUT`
-**Default:** [`30`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for Readability in seconds
-
-
-### Mercury Settings
-
-#### `MERCURY_ARGS`
-**Default:** [`[]`]
-Default Mercury parser arguments
-
----
-#### `MERCURY_ARGS_EXTRA`
-**Default:** [`[]`]
-Extra arguments to append to Mercury parser command
-
----
-#### `MERCURY_BINARY`
-**Default:** [`postlight-parser`]
-Path to Mercury/Postlight parser binary
-
----
-#### `MERCURY_ENABLED`
-**Default:** [`True`]
-Enable Mercury text extraction
-
----
-#### `MERCURY_TIMEOUT`
-**Default:** [`30`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for Mercury in seconds
-
-
-### Defuddle Settings
-
-#### `DEFUDDLE_ARGS`
-**Default:** [`[]`]
-Default Defuddle arguments
-
----
-#### `DEFUDDLE_ARGS_EXTRA`
-**Default:** [`[]`]
-Extra arguments to append to Defuddle command
-
----
-#### `DEFUDDLE_BINARY`
-**Default:** [`defuddle`]
-Path to defuddle binary
-
----
-#### `DEFUDDLE_ENABLED`
-**Default:** [`True`]
-Enable Defuddle text extraction
-
----
-#### `DEFUDDLE_TIMEOUT`
-**Default:** [`30`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for Defuddle in seconds
-
-
-### HTML to Text Settings
-
-#### `HTMLTOTEXT_ENABLED`
-**Default:** [`True`]
-Enable HTML to text conversion
-
----
-#### `HTMLTOTEXT_TIMEOUT`
-**Default:** [`30`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for HTML to text conversion in seconds
-
-
-### Trafilatura Settings
-
-#### `TRAFILATURA_BINARY`
-**Default:** [`trafilatura`]
-Path to trafilatura binary
-
----
-#### `TRAFILATURA_ENABLED`
-**Default:** [`True`]
-Enable Trafilatura extraction
-
----
-#### `TRAFILATURA_OUTPUT_CSV`
-**Default:** [`False`]
-Write CSV output (content.csv)
-
----
-#### `TRAFILATURA_OUTPUT_HTML`
-**Default:** [`True`]
-Write HTML output (content.html)
-
----
-#### `TRAFILATURA_OUTPUT_JSON`
-**Default:** [`False`]
-Write JSON output (content.json)
-
----
-#### `TRAFILATURA_OUTPUT_MARKDOWN`
-**Default:** [`True`]
-Write markdown output (content.md)
-
----
-#### `TRAFILATURA_OUTPUT_TXT`
-**Default:** [`True`]
-Write plain text output (content.txt)
-
----
-#### `TRAFILATURA_OUTPUT_XML`
-**Default:** [`False`]
-Write XML output (content.xml)
-
----
-#### `TRAFILATURA_OUTPUT_XMLTEI`
-**Default:** [`False`]
-Write XML TEI output (content.xmltei)
-
----
-#### `TRAFILATURA_TIMEOUT`
-**Default:** [`30`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for Trafilatura in seconds
-
-
-### Git Settings
-
-#### `GIT_ARGS`
-**Default:** [`['clone', '--depth=1', '--recursive']`]
-Default git arguments
-
----
-#### `GIT_ARGS_EXTRA`
-**Default:** [`[]`]
-Extra arguments to append to git command
-
----
-#### `GIT_BINARY`
-**Default:** [`git`]
-Path to git binary
-
----
-#### `GIT_DOMAINS`
-**Default:** [*see defaults*]
-Comma-separated list of domains to treat as git repositories
-
----
-#### `GIT_ENABLED`
-**Default:** [`True`]
-Enable git repository cloning
-
----
-#### `GIT_TIMEOUT`
-**Default:** [`120`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for git operations in seconds
-
-
-### yt-dlp Settings
-
-#### `YTDLP_ARGS`
-**Default:** [*see defaults*]
-Default yt-dlp arguments
-
----
-#### `YTDLP_ARGS_EXTRA`
-**Default:** [`[]`]
-Extra arguments to append to yt-dlp command
-
----
-#### `YTDLP_BINARY`
-**Default:** [`yt-dlp`]
-Path to yt-dlp binary
-
----
-#### `YTDLP_CHECK_SSL_VALIDITY`
-**Default:** [`True`] *(falls back to [`CHECK_SSL_VALIDITY`](#check_ssl_validity))*
-Whether to verify SSL certificates
-
----
-#### `YTDLP_COOKIES_FILE`
-**Default:** [`""`] *(falls back to [`COOKIES_FILE`](#cookies_file))*
-Path to cookies file
-
----
-#### `YTDLP_ENABLED`
-**Default:** [`True`]
-Enable video/audio downloading with yt-dlp
-
----
-#### `YTDLP_MAX_SIZE`
-**Default:** [`750m`]
-Maximum file size for yt-dlp downloads
-
-#### `YTDLP_TIMEOUT`
-**Default:** [`3600`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for yt-dlp downloads in seconds
-
-
-### gallery-dl Settings
-
-#### `GALLERYDL_ARGS`
-**Default:** [`['--write-metadata', '--write-info-json']`]
-Default gallery-dl arguments
-
----
-#### `GALLERYDL_ARGS_EXTRA`
-**Default:** [`[]`]
-Extra arguments to append to gallery-dl command
-
----
-#### `GALLERYDL_BINARY`
-**Default:** [`gallery-dl`]
-Path to gallery-dl binary
-
----
-#### `GALLERYDL_CHECK_SSL_VALIDITY`
-**Default:** [`True`] *(falls back to [`CHECK_SSL_VALIDITY`](#check_ssl_validity))*
-Whether to verify SSL certificates
-
----
-#### `GALLERYDL_COOKIES_FILE`
-**Default:** [`""`] *(falls back to [`COOKIES_FILE`](#cookies_file))*
-Path to cookies file
-
----
-#### `GALLERYDL_ENABLED`
-**Default:** [`True`]
-Enable gallery downloading with gallery-dl
-
----
-#### `GALLERYDL_TIMEOUT`
-**Default:** [`3600`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for gallery downloads in seconds
-
-
-### forum-dl Settings
-
-#### `FORUMDL_ARGS`
-**Default:** [`[]`]
-Default forum-dl arguments
-
----
-#### `FORUMDL_ARGS_EXTRA`
-**Default:** [`[]`]
-Extra arguments to append to forum-dl command
-
----
-#### `FORUMDL_BINARY`
-**Default:** [`forum-dl`]
-Path to forum-dl binary
-
----
-#### `FORUMDL_ENABLED`
-**Default:** [`True`]
-Enable forum downloading with forum-dl
-
----
-#### `FORUMDL_OUTPUT_FORMAT`
-**Default:** [`jsonl`]
-Output format for forum downloads
-
----
-#### `FORUMDL_TIMEOUT`
-**Default:** [`3600`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for forum downloads in seconds
-
-
-### papers-dl Settings
-
-#### `PAPERSDL_ARGS`
-**Default:** [`['fetch']`]
-Default papers-dl arguments
-
----
-#### `PAPERSDL_ARGS_EXTRA`
-**Default:** [`[]`]
-Extra arguments to append to papers-dl command
-
----
-#### `PAPERSDL_BINARY`
-**Default:** [`papers-dl`]
-Path to papers-dl binary
-
----
-#### `PAPERSDL_ENABLED`
-**Default:** [`True`]
-Enable paper downloading with papers-dl
-
----
-#### `PAPERSDL_TIMEOUT`
-**Default:** [`300`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for paper downloads in seconds
-
-
-### Archive.org Settings
-
-#### `ARCHIVEDOTORG_ENABLED`
-**Default:** [`True`]
-Submit URLs to archive.org Wayback Machine
-
----
-#### `ARCHIVEDOTORG_TIMEOUT`
-**Default:** [`60`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for archive.org submission in seconds
-
----
-#### `ARCHIVEDOTORG_USER_AGENT`
-**Default:** [`""`] *(falls back to [`USER_AGENT`](#user_agent))*
-User agent string
-
-
-### Chrome Settings
-
-#### `CHROME_ARGS`
-**Default:** [*see defaults*]
-Default Chrome command-line arguments (static flags only, dynamic args like --user-data-dir are added at runtime)
-
----
-#### `CHROME_ARGS_EXTRA`
-**Default:** [`[]`]
-Extra arguments to append to Chrome command (for user customization)
-
----
-#### `CHROME_BINARY`
-**Default:** [`chromium`]
-Path to Chromium binary
-
----
-#### `CHROME_CHECK_SSL_VALIDITY`
-**Default:** [`True`] *(falls back to [`CHECK_SSL_VALIDITY`](#check_ssl_validity))*
-Whether to verify SSL certificates (disable for self-signed certs)
-
----
-#### `CHROME_DELAY_AFTER_LOAD`
-**Default:** [`0`]
-Extra delay in seconds after page load completes before archiving (useful for JS-heavy SPAs)
-
----
-#### `CHROME_ENABLED`
-**Default:** [`True`]
-Enable Chromium browser integration for archiving
-
----
-#### `CHROME_HEADLESS`
-**Default:** [`True`]
-Run Chrome in headless mode
-
-#### `CHROME_PAGELOAD_TIMEOUT`
-**Default:** [`60`] *(falls back to [`CHROME_TIMEOUT`](#chrome_timeout))*
-Timeout for page navigation/load in seconds
-
----
-#### `CHROME_RESOLUTION`
-**Default:** [`1440,2000`] *(falls back to [`RESOLUTION`](#resolution))*
-Browser viewport resolution (width,height)
-
----
-#### `CHROME_SANDBOX`
-**Default:** [`True`]
-Enable Chrome sandbox (disable in Docker with --no-sandbox)
-
----
-#### `CHROME_TIMEOUT`
-**Default:** [`60`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for Chrome operations in seconds
-
----
-#### `CHROME_USER_AGENT`
-**Default:** [`""`] *(falls back to [`USER_AGENT`](#user_agent))*
-User agent string for Chrome
-
----
-#### `CHROME_USER_DATA_DIR`
-**Default:** [`""`]
-Path to Chrome user data directory for persistent sessions (derived from ACTIVE_PERSONA if not set)
-
----
-#### `CHROME_WAIT_FOR`
-**Default:** [`networkidle2`]
-Page load completion condition (domcontentloaded, load, networkidle0, networkidle2)
-
-
-### DNS Settings
-
-#### `DNS_ENABLED`
-**Default:** [`True`]
-Enable DNS traffic recording during page load
-
----
-#### `DNS_TIMEOUT`
-**Default:** [`30`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for DNS recording in seconds
-
-
-### SSL Settings
-
-#### `SSL_ENABLED`
-**Default:** [`True`]
-Enable SSL certificate capture
-
----
-#### `SSL_TIMEOUT`
-**Default:** [`30`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for SSL capture in seconds
-
-
-### Headers Settings
-
-#### `HEADERS_ENABLED`
-**Default:** [`True`]
-Enable HTTP headers capture
-
----
-#### `HEADERS_TIMEOUT`
-**Default:** [`30`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for headers capture in seconds
-
-
-### Redirects Settings
-
-#### `REDIRECTS_ENABLED`
-**Default:** [`True`]
-Enable redirect chain capture
-
----
-#### `REDIRECTS_TIMEOUT`
-**Default:** [`30`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for redirect capture in seconds
-
-
-### Responses Settings
-
-#### `RESPONSES_ENABLED`
-**Default:** [`True`]
-Enable HTTP response capture
-
----
-#### `RESPONSES_TIMEOUT`
-**Default:** [`30`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for response capture in seconds
-
-
-### Console Log Settings
-
-#### `CONSOLELOG_ENABLED`
-**Default:** [`True`]
-Enable console log capture
-
----
-#### `CONSOLELOG_TIMEOUT`
-**Default:** [`30`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for console log capture in seconds
-
-
-### Accessibility Settings
-
-#### `ACCESSIBILITY_ENABLED`
-**Default:** [`True`]
-Enable accessibility tree capture
-
----
-#### `ACCESSIBILITY_TIMEOUT`
-**Default:** [`30`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for accessibility capture in seconds
-
-
-### SEO Settings
-
-#### `SEO_ENABLED`
-**Default:** [`True`]
-Enable SEO metadata capture
-
----
-#### `SEO_TIMEOUT`
-**Default:** [`30`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for SEO capture in seconds
-
-
-### Hashes Settings
-
-#### `HASHES_ENABLED`
-**Default:** [`True`]
-Enable merkle tree hash generation
-
----
-#### `HASHES_TIMEOUT`
-**Default:** [`30`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for merkle tree generation in seconds
-
-
-### Static File Settings
-
-#### `STATICFILE_ENABLED`
-**Default:** [`True`]
-Enable static file detection
-
----
-#### `STATICFILE_TIMEOUT`
-**Default:** [`30`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for static file detection in seconds
-
-
-### uBlock Origin Settings
-
-#### `UBLOCK_ENABLED`
-**Default:** [`True`]
-Enable uBlock Origin browser extension for ad blocking
-
-
-### I Still Don't Care About Cookies Settings
-
-#### `ISTILLDONTCAREABOUTCOOKIES_ENABLED`
-**Default:** [`True`]
-Enable I Still Don't Care About Cookies browser extension
-
-
-### 2captcha Settings
-
-#### `TWOCAPTCHA_API_KEY`
-**Default:** [`""`]
-2captcha API key for CAPTCHA solving service (get from https://2captcha.com)
-
----
-#### `TWOCAPTCHA_AUTO_SUBMIT`
-**Default:** [`False`]
-Automatically submit forms after CAPTCHA is solved
-
----
-#### `TWOCAPTCHA_ENABLED`
-**Default:** [`True`]
-Enable 2captcha browser extension for automatic CAPTCHA solving
-
----
-#### `TWOCAPTCHA_RETRY_COUNT`
-**Default:** [`3`]
-Number of times to retry CAPTCHA solving on error
-
----
-#### `TWOCAPTCHA_RETRY_DELAY`
-**Default:** [`5`]
-Delay in seconds between CAPTCHA solving retries
-
----
-#### `TWOCAPTCHA_TIMEOUT`
-**Default:** [`60`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for CAPTCHA solving in seconds
-
-
-### Modal Closer Settings
-
-#### `MODALCLOSER_ENABLED`
-**Default:** [`True`]
-Enable automatic modal and dialog closing
-
----
-#### `MODALCLOSER_POLL_INTERVAL`
-**Default:** [`500`]
-How often to check for CSS modals (ms)
-
----
-#### `MODALCLOSER_TIMEOUT`
-**Default:** [`1250`]
-Delay before auto-closing dialogs (ms)
-
-
-### Infinite Scroll Settings
-
-#### `INFINISCROLL_ENABLED`
-**Default:** [`True`]
-Enable infinite scroll page expansion
-
----
-#### `INFINISCROLL_EXPAND_DETAILS`
-**Default:** [`True`]
-Expand <details> elements and click 'load more' buttons for comments
-
----
-#### `INFINISCROLL_MIN_HEIGHT`
-**Default:** [`16000`]
-Minimum page height to scroll to in pixels
-
----
-#### `INFINISCROLL_SCROLL_DELAY`
-**Default:** [`2000`]
-Delay between scrolls in milliseconds
-
----
-#### `INFINISCROLL_SCROLL_DISTANCE`
-**Default:** [`1600`]
-Distance to scroll per step in pixels
-
----
-#### `INFINISCROLL_SCROLL_LIMIT`
-**Default:** [`10`]
-Maximum number of scroll steps
-
----
-#### `INFINISCROLL_TIMEOUT`
-**Default:** [`120`] *(falls back to [`TIMEOUT`](#timeout))*
-Maximum timeout for scrolling in seconds
-
-
-### DOM Outlinks Parser Settings
-
-#### `PARSE_DOM_OUTLINKS_ENABLED`
-**Default:** [`True`]
-Enable DOM outlinks parsing from archived pages
-
----
-#### `PARSE_DOM_OUTLINKS_TIMEOUT`
-**Default:** [`30`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for DOM outlinks parsing in seconds
-
-
-### HTML URL Parser Settings
-
-#### `PARSE_HTML_URLS_ENABLED`
-**Default:** [`True`]
-Enable HTML URL parsing
-
-
-### JSONL URL Parser Settings
-
-#### `PARSE_JSONL_URLS_ENABLED`
-**Default:** [`True`]
-Enable JSON Lines URL parsing
-
-
-### Netscape URL Parser Settings
-
-#### `PARSE_NETSCAPE_URLS_ENABLED`
-**Default:** [`True`]
-Enable Netscape bookmarks HTML URL parsing
-
-
-### Text URL Parser Settings
-
-#### `PARSE_TXT_URLS_ENABLED`
-**Default:** [`True`]
-Enable plain text URL parsing
-
-
-### RSS URL Parser Settings
-
-#### `PARSE_RSS_URLS_ENABLED`
-**Default:** [`True`]
-Enable RSS/Atom feed URL parsing
-
-
-### Claude Code Settings
-
-#### `ANTHROPIC_API_KEY`
-**Default:** [`""`]
-Anthropic API key for Claude Code authentication
-
----
-#### `CLAUDECODE_BINARY`
-**Default:** [`claude`]
-Path to Claude Code CLI binary
-
----
-#### `CLAUDECODE_ENABLED`
-**Default:** [`False`]
-Enable Claude Code AI agent integration. Controls whether the claudecode plugin participates in crawl-time extraction; child plugins still need the claudecode plugin installed and a working Claude binary.
-
----
-#### `CLAUDECODE_MAX_TURNS`
-**Default:** [`10`]
-Maximum number of agentic turns per invocation
-
----
-#### `CLAUDECODE_MODEL`
-**Default:** [`sonnet`]
-Claude model to use (e.g. sonnet, opus, haiku)
-
----
-#### `CLAUDECODE_TIMEOUT`
-**Default:** [`120`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for Claude Code operations in seconds
-
-
-### Claude Chrome Settings
-
-#### `CLAUDECHROME_ENABLED`
-**Default:** [`False`]
-Enable Claude for Chrome browser extension for AI-driven page interaction
-
----
-#### `CLAUDECHROME_MAX_ACTIONS`
-**Default:** [`15`]
-Maximum number of agentic loop iterations (screenshots + actions) per page
-
----
-#### `CLAUDECHROME_MODEL`
-**Default:** [`sonnet`]
-Claude model to use (e.g. sonnet, opus, haiku). Availability depends on your plan.
-
----
-#### `CLAUDECHROME_PROMPT`
-**Default:** [*see defaults*]
-Prompt for Claude to execute on the page. Claude can click buttons, fill forms, download files, and interact with any page element.
-
----
-#### `CLAUDECHROME_TIMEOUT`
-**Default:** [`120`] *(falls back to [`TIMEOUT`](#timeout))*
-Timeout for Claude for Chrome operations in seconds
-
-
-### Claude Code Extract Settings
-
-#### `CLAUDECODEEXTRACT_ENABLED`
-**Default:** [`False`]
-Enable Claude Code AI extraction
-
----
-#### `CLAUDECODEEXTRACT_MAX_TURNS`
-**Default:** [`10`] *(falls back to [`CLAUDECODE_MAX_TURNS`](#claudecode_max_turns))*
-Maximum number of agentic turns for extraction
-
----
-#### `CLAUDECODEEXTRACT_MODEL`
-**Default:** [`sonnet`] *(falls back to [`CLAUDECODE_MODEL`](#claudecode_model))*
-Claude model to use for extraction (e.g. sonnet, opus, haiku)
-
----
-#### `CLAUDECODEEXTRACT_PROMPT`
-**Default:** [*see defaults*]
-Custom prompt for Claude Code extraction. Use this to define what Claude should extract or generate from the snapshot.
-
----
-#### `CLAUDECODEEXTRACT_TIMEOUT`
-**Default:** [`120`] *(falls back to [`CLAUDECODE_TIMEOUT`](#claudecode_timeout))*
-Timeout for Claude Code extraction in seconds
-
-
-### Claude Code Cleanup Settings
-
-#### `CLAUDECODECLEANUP_ENABLED`
-**Default:** [`False`]
-Enable Claude Code AI cleanup of snapshot files
-
----
-#### `CLAUDECODECLEANUP_MAX_TURNS`
-**Default:** [`15`] *(falls back to [`CLAUDECODE_MAX_TURNS`](#claudecode_max_turns))*
-Maximum number of agentic turns for cleanup
-
----
-#### `CLAUDECODECLEANUP_MODEL`
-**Default:** [`sonnet`] *(falls back to [`CLAUDECODE_MODEL`](#claudecode_model))*
-Claude model to use for cleanup (e.g. sonnet, opus, haiku)
-
----
-#### `CLAUDECODECLEANUP_PROMPT`
-**Default:** [*see defaults*]
-Custom prompt for Claude Code cleanup. Defines what Claude should clean up and how to determine which duplicates to keep.
-
----
-#### `CLAUDECODECLEANUP_TIMEOUT`
-**Default:** [`120`] *(falls back to [`CLAUDECODE_TIMEOUT`](#claudecode_timeout))*
-Timeout for Claude Code cleanup in seconds
-
-
-### Ripgrep Search Settings
-
-#### `RIPGREP_ARGS`
-**Default:** [`['--files-with-matches', '--no-messages', '--ignore-case']`]
-Default ripgrep arguments
-
----
-#### `RIPGREP_ARGS_EXTRA`
-**Default:** [`[]`]
-Extra arguments to append to ripgrep command
-
----
-#### `RIPGREP_BINARY`
-**Default:** [`rg`]
-Path to ripgrep binary
-
----
-#### `RIPGREP_TIMEOUT`
-**Default:** [`90`] *(falls back to [`TIMEOUT`](#timeout))*
-Search timeout in seconds
-
-
-### Sonic Search Settings
-
-#### `SEARCH_BACKEND_SONIC_BUCKET`
-**Default:** [`snapshots`]
-Sonic bucket name
-
----
-#### `SEARCH_BACKEND_SONIC_COLLECTION`
-**Default:** [`archivebox`]
-Sonic collection name
-
----
-#### `SEARCH_BACKEND_SONIC_HOST_NAME`
-**Default:** [`127.0.0.1`]
-Sonic server hostname
-
----
-#### `SEARCH_BACKEND_SONIC_PASSWORD`
-**Default:** [`SecretPassword`]
-Sonic server password
-
----
-#### `SEARCH_BACKEND_SONIC_PORT`
-**Default:** [`1491`]
-Sonic server port
-
-
-### SQLite FTS Search Settings
-
-#### `SEARCH_BACKEND_SQLITE_DB`
-**Default:** [`search.sqlite3`]
-SQLite FTS database filename
-
----
-#### `SEARCH_BACKEND_SQLITE_SEPARATE_DATABASE`
-**Default:** [`True`]
-Use separate database file for FTS index
-
----
-#### `SEARCH_BACKEND_SQLITE_TOKENIZERS`
-**Default:** [`porter unicode61 remove_diacritics 2`]
-FTS5 tokenizer configuration
-
-
-
-<img src="https://github.com/ArchiveBox/ArchiveBox/assets/511499/5a4dd576-387a-4a1f-9dfa-407eac87078c" width="100%"/>
+*Related options:* [`USE_COLOR`](#use_color)
+
+---
+
+## Plugin Configuration
+
+<!-- Backward-compatibility anchors for old links to plugin sections/options that have moved to the per-plugin documentation site. -->
+<a id="2captcha-settings"></a>
+<a id="accessibility-settings"></a>
+<a id="accessibility_enabled"></a>
+<a id="accessibility_timeout"></a>
+<a id="anthropic_api_key"></a>
+<a id="archive.org-settings"></a>
+<a id="archivedotorg_enabled"></a>
+<a id="archivedotorg_timeout"></a>
+<a id="chrome-settings"></a>
+<a id="chrome_args"></a>
+<a id="chrome_args_extra"></a>
+<a id="chrome_binary"></a>
+<a id="chrome_check_ssl_validity"></a>
+<a id="chrome_delay_after_load"></a>
+<a id="chrome_enabled"></a>
+<a id="chrome_headless"></a>
+<a id="chrome_pageload_timeout"></a>
+<a id="chrome_resolution"></a>
+<a id="chrome_sandbox"></a>
+<a id="chrome_timeout"></a>
+<a id="chrome_user_agent"></a>
+<a id="chrome_user_data_dir"></a>
+<a id="chrome_wait_for"></a>
+<a id="claude-chrome-settings"></a>
+<a id="claude-code-cleanup-settings"></a>
+<a id="claude-code-extract-settings"></a>
+<a id="claude-code-settings"></a>
+<a id="claudechrome_enabled"></a>
+<a id="claudechrome_max_actions"></a>
+<a id="claudechrome_model"></a>
+<a id="claudechrome_prompt"></a>
+<a id="claudechrome_timeout"></a>
+<a id="claudecode_binary"></a>
+<a id="claudecode_enabled"></a>
+<a id="claudecode_max_turns"></a>
+<a id="claudecode_model"></a>
+<a id="claudecode_timeout"></a>
+<a id="claudecodecleanup_enabled"></a>
+<a id="claudecodecleanup_max_turns"></a>
+<a id="claudecodecleanup_model"></a>
+<a id="claudecodecleanup_prompt"></a>
+<a id="claudecodecleanup_timeout"></a>
+<a id="claudecodeextract_enabled"></a>
+<a id="claudecodeextract_max_turns"></a>
+<a id="claudecodeextract_model"></a>
+<a id="claudecodeextract_prompt"></a>
+<a id="claudecodeextract_timeout"></a>
+<a id="console-log-settings"></a>
+<a id="consolelog_enabled"></a>
+<a id="consolelog_timeout"></a>
+<a id="defuddle-settings"></a>
+<a id="defuddle_args"></a>
+<a id="defuddle_args_extra"></a>
+<a id="defuddle_binary"></a>
+<a id="defuddle_enabled"></a>
+<a id="defuddle_timeout"></a>
+<a id="dns-settings"></a>
+<a id="dns_enabled"></a>
+<a id="dns_timeout"></a>
+<a id="dom-outlinks-parser-settings"></a>
+<a id="dom-settings"></a>
+<a id="dom_enabled"></a>
+<a id="dom_timeout"></a>
+<a id="favicon-settings"></a>
+<a id="favicon_enabled"></a>
+<a id="favicon_timeout"></a>
+<a id="forum-dl-settings"></a>
+<a id="forumdl_args"></a>
+<a id="forumdl_args_extra"></a>
+<a id="forumdl_binary"></a>
+<a id="forumdl_enabled"></a>
+<a id="forumdl_output_format"></a>
+<a id="forumdl_timeout"></a>
+<a id="gallery-dl-settings"></a>
+<a id="gallerydl_args"></a>
+<a id="gallerydl_args_extra"></a>
+<a id="gallerydl_binary"></a>
+<a id="gallerydl_check_ssl_validity"></a>
+<a id="gallerydl_cookies_file"></a>
+<a id="gallerydl_enabled"></a>
+<a id="gallerydl_timeout"></a>
+<a id="git-settings"></a>
+<a id="git_args"></a>
+<a id="git_args_extra"></a>
+<a id="git_binary"></a>
+<a id="git_domains"></a>
+<a id="git_enabled"></a>
+<a id="git_timeout"></a>
+<a id="hashes-settings"></a>
+<a id="hashes_enabled"></a>
+<a id="hashes_timeout"></a>
+<a id="headers-settings"></a>
+<a id="headers_enabled"></a>
+<a id="headers_timeout"></a>
+<a id="html-to-text-settings"></a>
+<a id="html-url-parser-settings"></a>
+<a id="htmltotext_enabled"></a>
+<a id="htmltotext_timeout"></a>
+<a id="i-still-dont-care-about-cookies-settings"></a>
+<a id="infinite-scroll-settings"></a>
+<a id="infiniscroll_enabled"></a>
+<a id="infiniscroll_expand_details"></a>
+<a id="infiniscroll_min_height"></a>
+<a id="infiniscroll_scroll_delay"></a>
+<a id="infiniscroll_scroll_distance"></a>
+<a id="infiniscroll_scroll_limit"></a>
+<a id="infiniscroll_timeout"></a>
+<a id="istilldontcareaboutcookies_enabled"></a>
+<a id="jsonl-url-parser-settings"></a>
+<a id="mercury-settings"></a>
+<a id="mercury_args"></a>
+<a id="mercury_args_extra"></a>
+<a id="mercury_binary"></a>
+<a id="mercury_enabled"></a>
+<a id="mercury_timeout"></a>
+<a id="modal-closer-settings"></a>
+<a id="modalcloser_enabled"></a>
+<a id="modalcloser_poll_interval"></a>
+<a id="modalcloser_timeout"></a>
+<a id="netscape-url-parser-settings"></a>
+<a id="papers-dl-settings"></a>
+<a id="papersdl_args"></a>
+<a id="papersdl_args_extra"></a>
+<a id="papersdl_binary"></a>
+<a id="papersdl_enabled"></a>
+<a id="papersdl_timeout"></a>
+<a id="parse_dom_outlinks_enabled"></a>
+<a id="parse_dom_outlinks_timeout"></a>
+<a id="parse_html_urls_enabled"></a>
+<a id="parse_jsonl_urls_enabled"></a>
+<a id="parse_netscape_urls_enabled"></a>
+<a id="parse_rss_urls_enabled"></a>
+<a id="parse_txt_urls_enabled"></a>
+<a id="pdf-settings"></a>
+<a id="pdf_enabled"></a>
+<a id="pdf_resolution"></a>
+<a id="pdf_timeout"></a>
+<a id="readability-settings"></a>
+<a id="readability_args"></a>
+<a id="readability_args_extra"></a>
+<a id="readability_binary"></a>
+<a id="readability_enabled"></a>
+<a id="readability_timeout"></a>
+<a id="redirects-settings"></a>
+<a id="redirects_enabled"></a>
+<a id="redirects_timeout"></a>
+<a id="responses-settings"></a>
+<a id="responses_enabled"></a>
+<a id="responses_timeout"></a>
+<a id="ripgrep-search-settings"></a>
+<a id="ripgrep_args"></a>
+<a id="ripgrep_args_extra"></a>
+<a id="ripgrep_binary"></a>
+<a id="ripgrep_timeout"></a>
+<a id="rss-url-parser-settings"></a>
+<a id="screenshot-settings"></a>
+<a id="screenshot_enabled"></a>
+<a id="screenshot_resolution"></a>
+<a id="screenshot_timeout"></a>
+<a id="search_backend_sonic_bucket"></a>
+<a id="search_backend_sonic_collection"></a>
+<a id="search_backend_sonic_host_name"></a>
+<a id="search_backend_sonic_password"></a>
+<a id="search_backend_sonic_port"></a>
+<a id="search_backend_sqlite_db"></a>
+<a id="search_backend_sqlite_separate_database"></a>
+<a id="search_backend_sqlite_tokenizers"></a>
+<a id="seo-settings"></a>
+<a id="seo_enabled"></a>
+<a id="seo_timeout"></a>
+<a id="singlefile-settings"></a>
+<a id="singlefile_args"></a>
+<a id="singlefile_args_extra"></a>
+<a id="singlefile_binary"></a>
+<a id="singlefile_check_ssl_validity"></a>
+<a id="singlefile_chrome_args"></a>
+<a id="singlefile_cookies_file"></a>
+<a id="singlefile_enabled"></a>
+<a id="singlefile_timeout"></a>
+<a id="singlefile_user_agent"></a>
+<a id="sonic-search-settings"></a>
+<a id="sqlite-fts-search-settings"></a>
+<a id="ssl-settings"></a>
+<a id="ssl_enabled"></a>
+<a id="ssl_timeout"></a>
+<a id="static-file-settings"></a>
+<a id="staticfile_enabled"></a>
+<a id="staticfile_timeout"></a>
+<a id="text-url-parser-settings"></a>
+<a id="title-settings"></a>
+<a id="title_enabled"></a>
+<a id="title_timeout"></a>
+<a id="trafilatura-settings"></a>
+<a id="trafilatura_binary"></a>
+<a id="trafilatura_enabled"></a>
+<a id="trafilatura_output_csv"></a>
+<a id="trafilatura_output_html"></a>
+<a id="trafilatura_output_json"></a>
+<a id="trafilatura_output_markdown"></a>
+<a id="trafilatura_output_txt"></a>
+<a id="trafilatura_output_xml"></a>
+<a id="trafilatura_output_xmltei"></a>
+<a id="trafilatura_timeout"></a>
+<a id="twocaptcha_api_key"></a>
+<a id="twocaptcha_auto_submit"></a>
+<a id="twocaptcha_enabled"></a>
+<a id="twocaptcha_retry_count"></a>
+<a id="twocaptcha_retry_delay"></a>
+<a id="twocaptcha_timeout"></a>
+<a id="ublock-origin-settings"></a>
+<a id="wget-settings"></a>
+<a id="wget_args"></a>
+<a id="wget_args_extra"></a>
+<a id="wget_binary"></a>
+<a id="wget_check_ssl_validity"></a>
+<a id="wget_cookies_file"></a>
+<a id="wget_enabled"></a>
+<a id="wget_timeout"></a>
+<a id="wget_user_agent"></a>
+<a id="wget_warc_enabled"></a>
+<a id="yt-dlp-settings"></a>
+<a id="ytdlp_args"></a>
+<a id="ytdlp_args_extra"></a>
+<a id="ytdlp_binary"></a>
+<a id="ytdlp_check_ssl_validity"></a>
+<a id="ytdlp_cookies_file"></a>
+<a id="ytdlp_enabled"></a>
+<a id="ytdlp_max_size"></a>
+<a id="ytdlp_timeout"></a>
+
+> [!IMPORTANT]
+> **Per-plugin configuration has moved to its own documentation site.**
+> This `Configuration.md` doc covers only ArchiveBox's *core* settings. For everything that lives inside a plugin — extractor toggles, binary paths, timeouts, args, user agents, cookies, persona scoping, etc. — see:
+>
+> ## ➡️ **<https://archivebox.github.io/abx-plugins/>**
+
+That site is regenerated from each plugin's `config.json` schema on every release, so it stays in sync with the code. Looking for [`WGET_ARGS`](https://archivebox.github.io/abx-plugins/#wget), [`CHROME_USER_DATA_DIR`](https://archivebox.github.io/abx-plugins/#chrome), [`SCREENSHOT_RESOLUTION`](https://archivebox.github.io/abx-plugins/#screenshot), [`YTDLP_EXTRA_ARGS`](https://archivebox.github.io/abx-plugins/#ytdlp), [`SINGLEFILE_*`](https://archivebox.github.io/abx-plugins/#singlefile), [`SONIC_HOST`](https://archivebox.github.io/abx-plugins/#search_backend_sonic), etc.? They all live there now.
+
+### Shared core options that plugins fall back to
+
+A handful of *core* options (documented above on this page) act as the **fallback default** for every plugin that has a matching per-extractor override. If you set the core option, every plugin honors it; if you also set the plugin-specific override, that wins for just that one plugin.
+
+| Core option (this doc) | Plugin-level overrides (see [abx-plugins](https://archivebox.github.io/abx-plugins/)) |
+|---|---|
+| [`TIMEOUT`](#timeout) | [`WGET_TIMEOUT`](https://archivebox.github.io/abx-plugins/#wget), [`CHROME_TIMEOUT`](https://archivebox.github.io/abx-plugins/#chrome), [`YTDLP_TIMEOUT`](https://archivebox.github.io/abx-plugins/#ytdlp), [`SINGLEFILE_TIMEOUT`](https://archivebox.github.io/abx-plugins/#singlefile), [`TITLE_TIMEOUT`](https://archivebox.github.io/abx-plugins/#title), [`FAVICON_TIMEOUT`](https://archivebox.github.io/abx-plugins/#favicon), ... |
+| [`CHECK_SSL_VALIDITY`](#check_ssl_validity) | [`WGET_CHECK_SSL_VALIDITY`](https://archivebox.github.io/abx-plugins/#wget), [`YTDLP_CHECK_SSL_VALIDITY`](https://archivebox.github.io/abx-plugins/#ytdlp), [`GALLERYDL_CHECK_SSL_VALIDITY`](https://archivebox.github.io/abx-plugins/#gallerydl), [`CHROME_CHECK_SSL_VALIDITY`](https://archivebox.github.io/abx-plugins/#chrome), ... |
+| [`USER_AGENT`](#user_agent) | [`WGET_USER_AGENT`](https://archivebox.github.io/abx-plugins/#wget), [`CHROME_USER_AGENT`](https://archivebox.github.io/abx-plugins/#chrome), [`SINGLEFILE_USER_AGENT`](https://archivebox.github.io/abx-plugins/#singlefile), ... |
+| [`COOKIES_FILE`](#cookies_file) | [`WGET_COOKIES_FILE`](https://archivebox.github.io/abx-plugins/#wget), [`YTDLP_COOKIES_FILE`](https://archivebox.github.io/abx-plugins/#ytdlp), [`GALLERYDL_COOKIES_FILE`](https://archivebox.github.io/abx-plugins/#gallerydl), [`SINGLEFILE_COOKIES_FILE`](https://archivebox.github.io/abx-plugins/#singlefile), ... |
+| [`RESOLUTION`](#resolution) | [`SCREENSHOT_RESOLUTION`](https://archivebox.github.io/abx-plugins/#screenshot), [`PDF_RESOLUTION`](https://archivebox.github.io/abx-plugins/#pdf), [`CHROME_RESOLUTION`](https://archivebox.github.io/abx-plugins/#chrome) |
+| [`DEFAULT_PERSONA`](#default_persona) | per-plugin persona scoping (browser profile / cookie jar selection) |
+
+> [!TIP]
+> The resolution order for any plugin-tunable option is always:
+> **1.** `<PLUGIN>_<OPTION>` (explicit per-plugin override) →
+> **2.** the matching shared core option above →
+> **3.** the plugin's own hardcoded default.
+>
+> So setting `TIMEOUT=120` once at the top of your `ArchiveBox.conf` raises the timeout for *every* extractor at once; setting `CHROME_TIMEOUT=300` on top of that lifts it further for just Chrome.
+
+### Listing & setting plugin options
+
+All plugin options can be set via the same three mechanisms as core options — env var, `ArchiveBox.conf`, or `archivebox config --set` — and inspected with `archivebox config`:
+
+```bash
+archivebox config                                  # show every option (core + every installed plugin)
+archivebox config --get SCREENSHOT_RESOLUTION      # read one value
+archivebox config --set SCREENSHOT_RESOLUTION=1920,1080
+archivebox config --search wget                    # search options by name/description
+```
+
+### Why is plugin config documented separately?
+
+Plugin schemas evolve on their own release cadence — new extractors ship between ArchiveBox releases, options are added/renamed as hooks revise, and pinning their docs to the core release schedule produced unavoidable drift. The per-plugin doc is auto-generated from each plugin's `config.json` schema at build time, so it never lags behind the code.