The_Great_Archive/ArchiveBox
1
0
Fork 0
mirror of https://github.com/ArchiveBox/ArchiveBox.git synced 2026-09-14 11:06:13 +05:00
Code Issues Actions Packages Projects Releases Wiki Activity

Make per-snapshot crawl seal idempotent against the run_snapshot race

Browse Source 
run_snapshot calls load_snapshot_payload which replaces self.crawl with
a fresh Crawl model instance every time. When N snapshot tasks finish
near-simultaneously their independent self.crawl/SM pairs all see
'STARTED + no open snapshots' and race on sm.seal(). The first task
drives the SM to a final state (engine.running becomes False), but the
loser's current_state still reads STARTED off its stale model field,
so python-statemachine then raises 'Can't Seal when in Started.' and
fails the whole snapshot task even though the crawl is already sealed.

Refresh the row right before the call so the guard sees the committed
status, and treat TransitionNotAllowed as the expected idempotent
no-op when another task already drove the transition.
This commit is contained in:
Nick Sweeting 2026-06-13 20:17:30 -07:00
parent 8ba658b4dc
commit a48ce6871b
No known key found for this signature in database
1 changed files with 30 additions and 11 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

41
archivebox/services/runner.py
View File

@ -1239,17 +1239,36 @@ class CrawlRunner:
await sync_to_async(run_snapshot_maintenance, thread_sensitive=True)(snapshot_id, output_dir=output_dir)
return
await self.enqueue_discovered_snapshots_from_outputs(snapshot)
await sync_to_async(
lambda: (
self.crawl.sm.seal()
if self.crawl.status == self.crawl.StatusChoices.STARTED
and not self.crawl.snapshot_set.filter(
status__in=self.crawl.snapshot_set.model.OPEN_STATES,
).exists()
else None
),
thread_sensitive=True,
)()
def _seal_when_last_snapshot_finished() -> None:
# run_snapshot replaces self.crawl per-snapshot, so multiple
# concurrent tasks each load a fresh Crawl/SM pointing at the
# same DB row. The "no open snapshots" check is non-atomic
# with sm.seal(), so two tasks racing to finish the last
# snapshot can both pass the guard. The first call drives
# the SM to a final state (engine.running=False); the loser
# then raises TransitionNotAllowed even though current_state
# still reads STARTED off its stale model field. Re-read the
# row right before the call and swallow the race so the
# task that lost the lap doesn't fail the whole snapshot.
from statemachine.exceptions import TransitionNotAllowed
crawl = self.crawl
crawl.refresh_from_db(fields=["status"])
if crawl.status != crawl.StatusChoices.STARTED:
return
if crawl.snapshot_set.filter(
status__in=crawl.snapshot_set.model.OPEN_STATES,
).exists():
return
try:
crawl.sm.seal()
except TransitionNotAllowed:
# Another task sealed it between our refresh and the
# SM call. Idempotent by design.
pass
await sync_to_async(_seal_when_last_snapshot_finished, thread_sensitive=True)()
finally:
snapshot_service.close()