The_Great_Archive/ArchiveBox
1
0
Fork 0
mirror of https://github.com/ArchiveBox/ArchiveBox.git synced 2026-09-12 19:50:57 +05:00
Code Issues Actions 4 Packages Projects Releases Wiki Activity

Restore complete deterministic CI coverage

Browse Source
This commit is contained in:
Nick Sweeting 2026-07-18 21:22:53 -07:00
parent d132fe5b71
commit 0843fc67e4
No known key found for this signature in database
27 changed files with 109 additions and 227 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
.github/workflows/test-parallel.yml vendored
View File

@ -234,15 +234,7 @@ jobs:
- name: Run test - ${{ matrix.test.name }}
run: |
mkdir -p tests/out
set +e
uv run --no-sync --no-sources pytest -xvs "${{ matrix.test.path }}" --basetemp=tests/out --ignore=archivebox/pkgs
status=$?
set -e
if [ "$status" -eq 5 ]; then
echo "No tests collected from ${{ matrix.test.path }}; treating as an empty test module."
exit 0
fi
exit "$status"
uv run --no-sync --no-sources pytest -xvs "${{ matrix.test.path }}" --basetemp=tests/out
plugin-tests:
name: ${{ matrix.plugin.name }}

12
.github/workflows/test.yml vendored
View File

@ -21,7 +21,6 @@ jobs:
strategy:
matrix:
os: [ubuntu-22.04]
# os: [ubuntu-22.04, macos-latest, windows-latest]
python: ["3.13"]
steps:
@ -92,14 +91,14 @@ jobs:
uv run --directory "$DATA_DIR" --no-sync --no-sources archivebox status
- name: Test built package with pytest
# TODO: remove this exception for windows once we get tests passing on that platform
if: ${{ !contains(matrix.os, 'windows') }}
run: |
mkdir -p tests/out
uv run --no-sync --no-sources pytest -s archivebox/tests --basetemp=tests/out --ignore=archivebox/pkgs
uv run --no-sync --no-sources pytest -s archivebox/tests --basetemp=tests/out
- name: Run documentation code blocks
run: uv run --no-sync --no-sources pytest -vv --tb=long README.md AGENTS.md skills docs
- name: Run plugin tests
if: ${{ !contains(matrix.os, 'windows') }}
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
TWOCAPTCHA_API_KEY: ${{ secrets.TWOCAPTCHA_API_KEY }}
@ -116,9 +115,6 @@ jobs:
submodules: true
fetch-depth: 1
# TODO: as of 2020-11 this helper layer broke, upgrade and re-enable this once it's usable again
# - uses: satackey/action-docker-layer-caching@v0.0.8
- name: Build image
run: |
docker build . -t "$DOCKER_IMAGE"

6
AGENTS.md
View File

@ -73,15 +73,13 @@ archivebox run
Use targeted tests for focused work:
<!--pytest.mark.skip(reason="pytest invocation")-->
```bash
```console
uv run pytest archivebox/tests/test_cli_add.py -q
uv run prek run --all-files
```
Use the full release/deploy loop only when requested:
<!--pytest.mark.skip(reason="release/deploy script")-->
```bash
```console
./bin/release_dev_stack.sh
```

33
README.md
View File

@ -491,8 +491,7 @@ test -d archive
rm -rf "$__archivebox_docs_home"
```
-->
<!--pytest.mark.skip(reason='Mixed host/Docker alternatives include interactive Docker commands')-->
```bash
```console
mkdir -p ~/archivebox/data # create a new data dir anywhere
cd ~/archivebox/data # IMPORTANT: cd into the directory
@ -1404,8 +1403,7 @@ For low hanging fruit / easy first tickets, see: <a href="https://github.com/Arc
First make sure you have `uv` installed: https://docs.astral.sh/uv/getting-started/installation/
<!--pytest.mark.skip(reason='Development workflow snippets are not part of docs codeblock tests')-->
```bash
```console
git clone https://github.com/ArchiveBox/monorepo
cd monorepo
./bin/setup.sh
@ -1426,8 +1424,7 @@ Repos included in monorepo setup:
#### 2. Option A: Install the Python, JS, and system dependencies directly on your machine
<!--pytest.mark.skip(reason='Development workflow snippets are not part of docs codeblock tests')-->
```bash
```console
# Install ArchiveBox runtime dependencies
mkdir -p data && cd data
archivebox init
@ -1442,8 +1439,7 @@ archivebox server 0.0.0.0:8000
#### 2. Option B: Build the docker container and use that for development instead
<!--pytest.mark.skip(reason='Development workflow snippets are not part of docs codeblock tests')-->
```bash
```console
# Optional: develop via docker by mounting the code dir into the container
# if you edit e.g. ./archivebox/core/models.py on the docker host, runserver
# inside the container will reload and pick up your changes
@ -1473,8 +1469,7 @@ You can also run all these in Docker. For more examples see the GitHub Actions C
<details><summary><i>Click to expand...</i></summary>
<!--pytest.mark.skip(reason='Development workflow snippets are not part of docs codeblock tests')-->
```bash
```console
# set up persistent DEBUG=True for all runs
archivebox config --set DEBUG=True
@ -1498,8 +1493,7 @@ https://stackoverflow.com/questions/1074212/how-can-i-see-the-raw-sql-queries-dj
If you're looking for the latest `dev` Docker image, it's often available pre-built on Docker Hub, simply pull and use `archivebox/archivebox:dev`.
<!--pytest.mark.skip(reason='Development workflow snippets are not part of docs codeblock tests')-->
```bash
```console
docker pull archivebox/archivebox:dev
docker run archivebox/archivebox:dev version
# verify the BUILD_TIME and COMMIT_HASH in the output are recent
@ -1509,8 +1503,7 @@ docker run archivebox/archivebox:dev version
You can also build and run any branch yourself from source, for example to build & use `dev` locally:
<!--pytest.mark.skip(reason='Development workflow snippets are not part of docs codeblock tests')-->
```bash
```console
# docker-compose.yml:
services:
archivebox:
@ -1534,8 +1527,7 @@ archivebox install
<details><summary><i>Click to expand...</i></summary>
<!--pytest.mark.skip(reason='Development workflow snippets are not part of docs codeblock tests')-->
```bash
```console
./bin/lint.sh
./bin/test.sh
```
@ -1548,8 +1540,7 @@ archivebox install
<details><summary><i>Click to expand...</i></summary>
<!--pytest.mark.skip(reason='Development workflow snippets are not part of docs codeblock tests')-->
```bash
```console
# generate the database migrations after changes to models.py
cd archivebox/
./manage.py makemigrations
@ -1613,8 +1604,7 @@ Copy a similar plugin as a template to modify, then open a new PR to add it in t
<details><summary><i>Click to expand...</i></summary>
(Normally CI takes care of this, but these scripts can be run to do it manually)
<!--pytest.mark.skip(reason='Development workflow snippets are not part of docs codeblock tests')-->
```bash
```console
./bin/build.sh
# or individually:
@ -1630,8 +1620,7 @@ Copy a similar plugin as a template to modify, then open a new PR to add it in t
<details><summary><i>Click to expand...</i></summary>
(Normally CI takes care of this, but these scripts can be run to do it manually)
<!--pytest.mark.skip(reason='Development workflow snippets are not part of docs codeblock tests')-->
```bash
```console
./bin/release.sh
# or individually:

10
archivebox/services/runner.py
View File

@ -1853,16 +1853,18 @@ def run_due_snapshot(snapshot, *, lock_seconds: int, interactive_interrupts: boo
return False
snapshot.refresh_from_db()
if snapshot.status == Snapshot.StatusChoices.QUEUED:
if snapshot.archiveresult_set.exists() and snapshot.is_finished_processing():
has_results = snapshot.archiveresult_set.exists()
has_extraction_results = snapshot.archiveresult_set.exclude(plugin__startswith="search_backend_").exists()
if has_results and has_extraction_results and snapshot.is_finished_processing():
snapshot.sm.tick()
snapshot.refresh_from_db()
if snapshot.status == Snapshot.StatusChoices.SEALED:
_runner_console_line(crawl_id=snapshot.crawl_id, snapshot=snapshot, status="SEALED")
return True
# The runner owns queued Snapshot setup. Create missing enabled hook
# rows before ticking so maintenance-only final rows, e.g. search
# backfill on a paused snapshot, cannot make queued -> sealed skip the
# real extraction work after resume.
# rows before ticking when the only existing rows are search
# maintenance. Otherwise a search backfill on a paused Snapshot can
# make queued -> sealed skip the real extraction work after resume.
snapshot.create_pending_archiveresults(hooks=snapshot_hooks_for_pending_archiveresults(snapshot))
snapshot.sm.tick()
snapshot.refresh_from_db()

1
archivebox/tests/test_api_archiveresult.py
View File

@ -1 +0,0 @@
# Tests moved to test_api_v1_core_archiveresults.py and test_api_v1_core_archiveresult_archiveresult_id.py.

1
archivebox/tests/test_api_cli.py
View File

@ -1 +0,0 @@
# CLI endpoint tests moved to test_api_v1_cli_add.py and test_api_v1_cli_update.py.

1
archivebox/tests/test_api_cli_schedule.py
View File

@ -1 +0,0 @@
# CLI schedule endpoint tests moved to test_api_v1_cli_schedule.py.

1
archivebox/tests/test_api_crawl.py
View File

@ -1 +0,0 @@
# Tests moved to test_api_v1_crawls_crawl_crawl_id.py.

1
archivebox/tests/test_api_crud.py
View File

@ -1 +0,0 @@
# Tests moved to test_api_v1_workflow_core_token_auth_side_effects.py and exact endpoint files under test_api_v1_core_*.

1
archivebox/tests/test_api_delete_paths.py
View File

@ -1 +0,0 @@
# Tests moved to test_api_v1_core_snapshot_snapshot_id.py and test_api_v1_crawls_crawl_crawl_id.py.

1
archivebox/tests/test_api_personas.py
View File

@ -1 +0,0 @@
# Tests moved to test_api_v1_personas_sync.py and test_api_v1_personas_personas.py.

1
archivebox/tests/test_api_remove.py
View File

@ -1 +0,0 @@
# CLI remove endpoint tests moved to test_api_v1_cli_remove.py.

1
archivebox/tests/test_api_rss.py
View File

@ -1 +0,0 @@
# Tests moved to test_api_v1_core_snapshots_rss.py and test_api_v1_crawls_crawl_crawl_id.py.

1
archivebox/tests/test_api_search.py
View File

@ -1 +0,0 @@
# Tests moved to test_api_v1_core_snapshots.py.

51
archivebox/tests/test_api_v1_core_snapshot_snapshot_id.py
View File

@ -389,54 +389,51 @@ def test_targeted_extract_retries_one_failed_archiveresult_while_snapshot_stays_
@pytest.mark.timeout(240)
def test_paused_snapshot_survives_server_restart_and_resumes_via_api(tmp_path, recursive_test_site):
def test_paused_snapshot_survives_server_restart_and_resumes_via_api(client, tmp_path, recursive_test_site):
init_archive(tmp_path)
port = get_free_port()
env = cli_env(port=port, server=True, PLUGINS="wget", SAVE_WGET="True")
api_token = create_admin_and_token(tmp_path)
try:
start_archivebox_server(tmp_path, env=env, port=port)
wait_for_live_api(port)
crawl_response = live_api_request(
port,
with use_archivebox_db(tmp_path):
crawl_response = api_client_request(
client,
"post",
"/api/v1/crawls/crawls",
api_token=api_token,
json={
payload={
"urls": [recursive_test_site["root_url"]],
"max_depth": 0,
"tags": ["snapshot-pause-restart-e2e"],
"config": {"PLUGINS": "wget", "URL_ALLOWLIST": r"127\.0\.0\.1[:/].*"},
},
timeout=10,
)
assert crawl_response.status_code == 200, crawl_response.text
crawl_id = crawl_response.json()["id"]
crawl_state = _wait_for_crawl_snapshot_rows(tmp_path, crawl_id)
snapshot_id = crawl_state["snapshots"][0]["id"]
pause_response = live_api_request(
port,
assert crawl_response.status_code == 200, crawl_response.content.decode()
crawl_id = json.loads(crawl_response.content.decode())["id"]
crawl = Crawl.objects.get(id=crawl_id)
snapshot = Snapshot.objects.create(
url=recursive_test_site["root_url"],
crawl=crawl,
status=Snapshot.StatusChoices.QUEUED,
retry_at=timezone.now(),
)
snapshot_id = str(snapshot.id)
pause_response = api_client_request(
client,
"patch",
f"/api/v1/crawls/crawl/{crawl_id}",
f"/api/v1/core/snapshot/{snapshot_id}",
api_token=api_token,
json={"action": "pause"},
timeout=10,
payload={"action": "pause"},
)
assert pause_response.status_code == 200, pause_response.text
assert pause_response.status_code == 200, pause_response.content.decode()
current_state = _paused_snapshot_state(tmp_path, snapshot_id)
if current_state["status"] == Snapshot.StatusChoices.SEALED:
assert current_state["succeeded_results"] > 0
return
try:
start_archivebox_server(tmp_path, env=env, port=port)
wait_for_live_api(port)
paused_state = _wait_for_paused_scheduler_marker(tmp_path, snapshot_id)
if paused_state["status"] == Snapshot.StatusChoices.SEALED:
assert paused_state["succeeded_results"] > 0
return
assert paused_state["status"] == Snapshot.StatusChoices.PAUSED
assert paused_state["succeeded_results"] == 0
assert not list((paused_state["snapshot_dir"] / "wget").rglob("*.html"))

146
archivebox/tests/test_api_v1_crawls_crawl_crawl_id.py
View File

@ -146,6 +146,44 @@ def wait_for_sqlite_index_result(cwd, crawl_id, timeout=45):
raise AssertionError(f"timed out waiting for sqlite index result for crawl {crawl_id}: {latest_state}")
def seed_paused_crawl(client, cwd: Path, api_token: str, url: str, tag: str) -> tuple[str, str]:
from archivebox.services.runner import run_due_snapshot
with use_archivebox_db(cwd):
response = api_client_request(
client,
"post",
"/api/v1/crawls/crawls",
api_token=api_token,
payload={
"urls": [url],
"max_depth": 0,
"tags": [tag],
"config": {"PLUGINS": "wget", "URL_ALLOWLIST": r"127\.0\.0\.1[:/].*"},
},
)
assert response.status_code == 200, response.content.decode()
crawl_id = json.loads(response.content.decode())["id"]
crawl = Crawl.objects.get(id=crawl_id)
snapshot = Snapshot.objects.create(url=url, crawl=crawl, status=Snapshot.StatusChoices.QUEUED, retry_at=timezone.now())
pause_response = api_client_request(
client,
"patch",
f"/api/v1/crawls/crawl/{crawl_id}",
api_token=api_token,
payload={"action": "pause"},
)
assert pause_response.status_code == 200, pause_response.content.decode()
assert run_due_snapshot(snapshot, lock_seconds=60)
crawl.refresh_from_db()
snapshot.refresh_from_db()
assert crawl.status == Crawl.StatusChoices.PAUSED
assert crawl.retry_at == RETRY_AT_MAX
assert snapshot.status == Snapshot.StatusChoices.PAUSED
assert snapshot.retry_at == RETRY_AT_MAX
return str(crawl_id), str(snapshot.id)
def make_snapshot(*, user, url: str, title: str, bookmarked_at: datetime):
crawl = Crawl.objects.create(urls=url, created_by=user)
snapshot = Snapshot.objects.create(
@ -341,55 +379,24 @@ def test_crawl_pause_resume_api_cascades_archiveresults_and_leaves_finished_snap
@pytest.mark.timeout(240)
def test_crawl_pause_resume_api_survives_server_restart_and_processes_after_resume(tmp_path, recursive_test_site):
def test_crawl_pause_resume_api_survives_server_restart_and_processes_after_resume(client, tmp_path, recursive_test_site):
init_archive(tmp_path)
port = get_free_port()
env = cli_env(port=port, server=True, PLUGINS="wget", SAVE_WGET="True")
api_token = create_admin_and_token(tmp_path)
crawl_id, _snapshot_id = seed_paused_crawl(client, tmp_path, api_token, recursive_test_site["root_url"], "pause-resume-e2e")
try:
start_archivebox_server(tmp_path, env=env, port=port)
wait_for_live_api(port)
crawl_response = live_api_request(
port,
"post",
"/api/v1/crawls/crawls",
api_token=api_token,
json={
"urls": [recursive_test_site["root_url"]],
"max_depth": 0,
"tags": ["pause-resume-e2e"],
"config": {"PLUGINS": "wget", "URL_ALLOWLIST": r"127\.0\.0\.1[:/].*"},
},
timeout=10,
)
assert crawl_response.status_code == 200, crawl_response.text
crawl_id = crawl_response.json()["id"]
wait_for_crawl_snapshot_rows(tmp_path, crawl_id)
pause_response = live_api_request(
port,
"patch",
f"/api/v1/crawls/crawl/{crawl_id}",
api_token=api_token,
json={"action": "pause"},
timeout=10,
)
assert pause_response.status_code == 200, pause_response.text
assert pause_response.json()["status"] == "paused"
paused_state = wait_for_crawl_child_snapshots_paused_or_sealed(tmp_path, crawl_id)
assert paused_state["crawl_status"] == "paused"
assert paused_state["crawl_retry_at"] == paused_state["retry_at_max"]
assert len(paused_state["snapshots"]) == 1
snapshot_finished_before_pause = paused_state["snapshots"][0]["status"] == "sealed"
if snapshot_finished_before_pause:
assert any(result["status"] == "succeeded" for result in paused_state["results"])
else:
assert paused_state["snapshots"][0]["status"] == "paused"
assert paused_state["snapshots"][0]["retry_at"] == paused_state["retry_at_max"]
assert paused_state["snapshots"][0]["status"] == "paused"
assert paused_state["snapshots"][0]["retry_at"] == paused_state["retry_at_max"]
stop_server(tmp_path)
start_archivebox_server(tmp_path, env=env, port=port)
@ -398,10 +405,6 @@ def test_crawl_pause_resume_api_survives_server_restart_and_processes_after_resu
restarted_state = get_crawl_runtime_state(tmp_path, crawl_id)
assert restarted_state["crawl_status"] == "paused"
assert restarted_state["crawl_retry_at"] == restarted_state["retry_at_max"]
if snapshot_finished_before_pause:
assert restarted_state["snapshots"][0]["status"] == "sealed"
assert any(result["status"] == "succeeded" for result in restarted_state["results"])
return
assert restarted_state["snapshots"][0]["status"] == "paused"
assert restarted_state["snapshots"][0]["retry_at"] == restarted_state["retry_at_max"]
assert not any(result["status"] == "succeeded" for result in restarted_state["results"])
@ -431,62 +434,13 @@ def test_crawl_pause_resume_api_survives_server_restart_and_processes_after_resu
@pytest.mark.timeout(420)
def test_update_index_only_runs_paused_search_rows_and_resume_later_runs_crawl(tmp_path, recursive_test_site):
def test_update_index_only_runs_paused_search_rows_and_resume_later_runs_crawl(client, tmp_path, recursive_test_site):
init_archive(tmp_path)
port = get_free_port()
env = cli_env(port=port, server=True, PLUGINS="wget", SAVE_WGET="True")
api_token = create_admin_and_token(tmp_path)
try:
start_archivebox_server(tmp_path, env=env, port=port)
wait_for_live_api(port)
crawl_response = live_api_request(
port,
"post",
"/api/v1/crawls/crawls",
api_token=api_token,
json={
"urls": [recursive_test_site["root_url"]],
"max_depth": 0,
"tags": ["paused-index-e2e"],
"config": {"PLUGINS": "wget", "URL_ALLOWLIST": r"127\.0\.0\.1[:/].*"},
},
timeout=10,
)
assert crawl_response.status_code == 200, crawl_response.text
crawl_id = crawl_response.json()["id"]
wait_for_crawl_snapshot_rows(tmp_path, crawl_id)
pause_response = live_api_request(
port,
"patch",
f"/api/v1/crawls/crawl/{crawl_id}",
api_token=api_token,
json={"action": "pause"},
timeout=10,
)
assert pause_response.status_code == 200, pause_response.text
paused_state = wait_for_crawl_child_snapshots_paused_or_sealed(tmp_path, crawl_id)
snapshot_finished_before_pause = paused_state["snapshots"][0]["status"] == "sealed"
finally:
stop_server(tmp_path)
if snapshot_finished_before_pause:
indexed_state = get_crawl_runtime_state(tmp_path, crawl_id)
assert indexed_state["snapshots"][0]["status"] == "sealed"
wget_results = [result for result in indexed_state["results"] if result["plugin"] == "wget"]
assert any(result["status"] == "succeeded" and result["output_size"] > 0 for result in wget_results)
if indexed_state["crawl_status"] == "paused":
assert indexed_state["crawl_retry_at"] == indexed_state["retry_at_max"]
else:
assert indexed_state["crawl_status"] == "sealed"
assert all(result["status"] not in {"queued", "started", "paused"} for result in indexed_state["results"])
captured_text = wait_for_snapshot_capture(tmp_path, recursive_test_site["root_url"], timeout=60)
assert "Root" in captured_text
assert "About" in captured_text
return
crawl_id, _snapshot_id = seed_paused_crawl(client, tmp_path, api_token, recursive_test_site["root_url"], "paused-index-e2e")
update_env = cli_env(
port=port,
@ -543,14 +497,10 @@ def test_update_index_only_runs_paused_search_rows_and_resume_later_runs_crawl(t
assert resumed_state["snapshots"][0]["status"] == "sealed"
wget_results = [result for result in resumed_state["results"] if result["plugin"] == "wget"]
wget_succeeded = any(result["status"] == "succeeded" and result["output_size"] > 0 for result in wget_results)
if wget_succeeded:
captured_text = wait_for_snapshot_capture(tmp_path, recursive_test_site["root_url"], timeout=60)
assert "Root" in captured_text
assert "About" in captured_text
else:
assert resumed_state["crawl_status"] == "sealed"
assert all(result["status"] not in {"queued", "started", "paused"} for result in resumed_state["results"])
assert any(result["status"] == "succeeded" and result["output_size"] > 0 for result in wget_results)
captured_text = wait_for_snapshot_capture(tmp_path, recursive_test_site["root_url"], timeout=60)
assert "Root" in captured_text
assert "About" in captured_text
finally:
stop_server(tmp_path)

23
archivebox/tests/test_api_v1_workflow_core_token_auth_side_effects.py
View File

@ -1,5 +1,3 @@
import time
import pytest
import requests
@ -21,17 +19,6 @@ from archivebox.tests.test_orm_helpers import use_archivebox_db
pytestmark = pytest.mark.django_db(transaction=True)
def live_api_request_retrying_sqlite_lock(*args, attempts: int = 12, delay: float = 2.0, **kwargs):
response = None
for _attempt in range(attempts):
response = live_api_request(*args, **kwargs)
if response.status_code != 503 or "database is locked" not in response.text:
return response
time.sleep(delay)
assert response is not None
return response
@pytest.mark.timeout(180)
def test_core_api_workflow_uses_token_auth_and_persists_side_effects_over_server(tmp_path, recursive_test_site):
init_archive(tmp_path)
@ -113,7 +100,7 @@ def test_core_api_workflow_uses_token_auth_and_persists_side_effects_over_server
assert snapshot_payload["url"] == recursive_test_site["child_urls"][0]
assert snapshot_payload["tags"] == ["api-child"]
patch_snapshot = live_api_request_retrying_sqlite_lock(
patch_snapshot = live_api_request(
port,
"patch",
f"/api/v1/core/snapshot/{snapshot_id}",
@ -125,7 +112,7 @@ def test_core_api_workflow_uses_token_auth_and_persists_side_effects_over_server
assert patch_snapshot.json()["status"] == "sealed"
assert set(patch_snapshot.json()["tags"]) == {"api-child", "api-patched"}
tag_create = live_api_request_retrying_sqlite_lock(
tag_create = live_api_request(
port,
"post",
"/api/v1/core/tags/create/",
@ -136,7 +123,7 @@ def test_core_api_workflow_uses_token_auth_and_persists_side_effects_over_server
assert tag_create.status_code == 200, tag_create.text
tag_id = tag_create.json()["tag_id"]
add_tag = live_api_request_retrying_sqlite_lock(
add_tag = live_api_request(
port,
"post",
"/api/v1/core/tags/add-to-snapshot/",
@ -145,7 +132,7 @@ def test_core_api_workflow_uses_token_auth_and_persists_side_effects_over_server
timeout=10,
)
assert add_tag.status_code == 200, add_tag.text
remove_tag = live_api_request_retrying_sqlite_lock(
remove_tag = live_api_request(
port,
"post",
"/api/v1/core/tags/remove-from-snapshot/",
@ -155,7 +142,7 @@ def test_core_api_workflow_uses_token_auth_and_persists_side_effects_over_server
)
assert remove_tag.status_code == 200, remove_tag.text
crawl_patch = live_api_request_retrying_sqlite_lock(
crawl_patch = live_api_request(
port,
"patch",
f"/api/v1/crawls/crawl/{crawl_id}",

2
archivebox/tests/test_archiveresult_pause.py
View File

@ -1,2 +0,0 @@
# test_crawl_pause_resume_api_cascades_archiveresults_and_leaves_finished_snapshot_results_alone moved to test_api_v1_crawls_crawl_crawl_id.py.
# test_targeted_extract_retries_one_failed_archiveresult_while_snapshot_stays_paused moved to test_api_v1_core_snapshot_snapshot_id.py.

4
archivebox/tests/test_crawl_pause.py
View File

@ -6,7 +6,3 @@ from archivebox.workers.models import RETRY_AT_MAX
def test_retry_at_max_is_safe_for_admin_timezone_localization():
with timezone.override("Pacific/Kiritimati"):
assert timezone.localtime(RETRY_AT_MAX).year == 9999
# test_crawl_pause_resume_api_survives_server_restart_and_processes_after_resume moved to test_api_v1_crawls_crawl_crawl_id.py.
# test_update_index_only_runs_paused_search_rows_and_resume_later_runs_crawl moved to test_api_v1_crawls_crawl_crawl_id.py.

3
archivebox/tests/test_crawl_runner.py
View File

@ -752,9 +752,6 @@ def test_sealed_crawl_does_not_create_discovered_snapshots():
assert crawl.snapshot_set.count() == 1
# test_create_crawl_api_queues_crawl_without_spawning_runner moved to test_api_v1_crawls_crawls.py.
def test_wait_for_snapshot_tasks_surfaces_already_failed_task():
from archivebox.base_models.models import get_or_create_system_user_pk
from archivebox.crawls.models import Crawl

3
archivebox/tests/test_frozen_crawl_config.py
View File

@ -269,9 +269,6 @@ def test_crawl_config_projections_stay_under_hot_path_budget():
assert average_seconds < max_average_seconds, f"{name} averaged {average_seconds * 1000:.3f}ms"
# test_api_create_and_cli_add_store_full_frozen_config moved to test_api_v1_workflow_frozen_crawl_config_sources.py.
def test_schedule_enqueue_refreezes_using_current_template_persona_defaults(archivebox_db):
from archivebox.crawls.models import Crawl, CrawlSchedule

6
archivebox/tests/test_orm_helpers.py
View File

@ -11,6 +11,12 @@ def archivebox_db_path(path: str | Path = ".") -> Path:
return path if path.name == "index.sqlite3" else path / "index.sqlite3"
def test_archivebox_db_path_accepts_collection_or_database_path(tmp_path: Path) -> None:
database_path = tmp_path / "index.sqlite3"
assert archivebox_db_path(tmp_path) == database_path
assert archivebox_db_path(database_path) == database_path
@contextmanager
def use_archivebox_db(path: str | Path = ".") -> Iterator[None]:
connection = connections["default"]

1
archivebox/tests/test_snapshot_pause.py
View File

@ -1 +0,0 @@
# test_snapshot_pause_resume_api_cascades_active_archiveresults_and_preserves_finished_rows moved to test_api_v1_core_snapshot_snapshot_id.py.

3
archivebox/tests/test_snapshot_service.py
View File

@ -80,6 +80,3 @@ def test_snapshot_service_cli_add_seals_snapshot_and_writes_indexes(tmp_path, re
assert wget_files
assert any("Root" in path.read_text(encoding="utf-8", errors="ignore") for path in wget_files if path.suffix in (".html", ".txt"))
assert any(result["plugin"] == "wget" and result["status"] == ArchiveResult.StatusChoices.SUCCEEDED for result in state["results"])
# test_paused_snapshot_survives_server_restart_and_resumes_via_api moved to test_api_v1_core_snapshot_snapshot_id.py.

7
archivebox/tests/test_ui_admin_tag.py
View File

@ -26,10 +26,3 @@ def test_tag_admin_add_view_renders_similar_tag_reference(admin_client):
assert response.status_code == 200
assert b"Similar Tags" in response.content
assert b'data-tag-name-input="1"' in response.content
# test_tag_search_api_returns_card_payload moved to test_api_v1_core_tags_search.py.
# test_tag_search_api_respects_sort_and_filters moved to test_api_v1_core_tags_search.py.
# test_tag_rename_api_updates_name moved to test_api_v1_core_tag_tag_id_rename.py.
# test_tag_snapshots_export_returns_jsonl moved to test_api_v1_core_tag_tag_id_snapshots_jsonl.py.
# test_tag_urls_export_returns_plain_text_urls moved to test_api_v1_core_tag_tag_id_urls_txt.py.

6
skills/archivebox/SKILL.md
View File

@ -67,15 +67,13 @@ archivebox run
## Verification
<!--pytest.mark.skip(reason="pytest invocation")-->
```bash
```console
uv run pytest archivebox/tests/test_cli_add.py -q
uv run prek run --all-files
```
Use the full release/deploy loop only when requested:
<!--pytest.mark.skip(reason="release/deploy script")-->
```bash
```console
./bin/release_dev_stack.sh
```