lab5 уровень 4: jobctl (реестр failed, replay, reprocess), тесты окон отказа и demo

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

jobctl: replay только failed с живым источником (новое задание replay_of + аудит),
reprocess только ready и только более новой версии; решения под той же блокировкой
строки, что и уборка. test_pipeline_flow: все окна отказа attachment-states.md, retry с
jitter, retry_exhausted -> replay, постоянные ошибки, лимиты памяти/времени, DLQ, зависший
worker (docker pause), graceful stop, prefetch, уборка vs replay, кейсы из рецензий.
This commit is contained in:
Alexey Bodanyuk
2026-10-31 16:48:26 +03:00
parent 1d37fd0dab
commit 962adbe03b
4 changed files with 865 additions and 4 deletions
+15 -4
View File
@@ -1,5 +1,6 @@
#!/usr/bin/env python3
"""Defense scenario, steps 1-2 (README): upload the fixture, send a message with attachment_ids,
"""Lab 4 defense scenario (kept in lab 5; upload is now 202 queued, the download waits for ready):
upload the fixture, send a message with attachment_ids,
download as the second member and compare SHA-256; the outsider gets 404; an unauthenticated
request straight to S3 (from inside the storage network) is refused, the signed one works.
@@ -67,6 +68,16 @@ def user(c: Client, label: str) -> dict:
return {"id": session["user"]["id"], "token": session["token"], "name": name}
def wait_ready(c: Client, actor: dict, attachment_id: str, timeout: float = 60):
import time
deadline = time.monotonic() + timeout
while True:
st, h, b = c.call("GET", f"/api/v1/attachments/{attachment_id}", token=actor["token"])
if st != 200 or json.loads(b)["status"] in ("ready", "failed") or time.monotonic() > deadline:
return st, h, b
time.sleep(0.3)
def main() -> int:
p = argparse.ArgumentParser()
p.add_argument("--base-url", default="http://127.0.0.1:8080")
@@ -77,7 +88,7 @@ def main() -> int:
alice, bob, eve = user(c, "alice"), user(c, "bob"), user(c, "eve")
print(f"users: alice, bob (members), eve (outsider)")
status, _, body = c.call("POST", "/api/v1/chats", token=alice["token"],
body={"title": "Демо lab4", "member_ids": [alice["id"], bob["id"]]})
body={"title": "Демо lab5", "member_ids": [alice["id"], bob["id"]]})
chat = json.loads(body)
print("\n1. upload -> message -> download by the second member")
@@ -88,8 +99,8 @@ def main() -> int:
st, h, b = c.call("POST", f"/api/v1/chats/{chat['id']}/messages", token=alice["token"],
body={"text": "", "attachment_ids": [att["id"]]}, headers={"Idempotency-Key": uuid.uuid4().hex})
show("POST /chats/{id}/messages text='' attachment_ids=[id]", st, h, b)
st, h, b = c.call("GET", f"/api/v1/attachments/{att['id']}", token=bob["token"])
show("GET /attachments/{id} (bob)", st, h, b)
st, h, b = wait_ready(c, bob, att["id"])
show("GET /attachments/{id} (bob) until ready", st, h, b)
st, h, got = c.call("GET", f"/api/v1/attachments/{att['id']}/content", token=bob["token"])
show("GET /attachments/{id}/content (bob)", st, h, got, binary=True)
same = hashlib.sha256(got).hexdigest() == hashlib.sha256(content).hexdigest() == att["sha256"]