lab3: fix reviewer feedback — idempotency atomicity, UUID validation, error headers, instance-id, migrate limits

This commit is contained in:
2026-10-10 08:56:12 +03:00
parent eacab162c9
commit 11bdc0bf62
6 changed files with 149 additions and 106 deletions
+30
View File
@@ -184,6 +184,36 @@ PASS: parallel idempotency — exactly one message created
| Параллельные POST с одним ключом | Одно сообщение | 1 сообщение в истории | — |
| Rolling update под нагрузкой | ≤ нескольких % ошибок | 0.28% (9 из 3206) | мгновенно |
## Исправления по замечаниям ревью
### Уровень 3
1. **Небезопасный повтор после обрыва POST.** Ключ резервировался со `status=0` до создания сообщения — при падении ключ залипал. Исправлено: `INSERT idempotency_keys` и `INSERT messages` в **одной транзакции**, `response_status=201` сразу. При падении — ROLLBACK откатывает всё.
2. **Конкурентная идемпотентность.** В `except UniqueViolation` возвращался `409` вместо перечитывания. Исправлено: retry loop (5 попыток по 50 мс). При одинаковых запросах все потоки получают **тот же 201** и **то же тело**.
Проверка: `test_parallel_idempotency.py`, 10 потоков → `Status codes: {201: 10}`, `Unique response bodies: 1`, `Messages in history: 1`.
3. **ACL при повторе.** Проверка ключа шла до проверки доступа → посторонний видел `409`. Исправлено: проверка `chat_members` **первой**; посторонний всегда получает `404`.
4. **`X-Instance-Id: unknown`.** В `compose.yaml` было `INSTANCE_ID: ${HOSTNAME:-unknown}` — переменная WSL, не контейнера. Исправлено: строка удалена, `src/instance.py` использует `socket.gethostname()` — у каждой реплики уникальный ID.
5. **Ошибки контракта.**
- Неверный UUID → `400`: функция `valid_uuid` в `main.py`, применяется в роутах с `{chat_id}`.
- 500 без заголовков: все обработчики ошибок добавляют `X-Request-Id` и `X-Instance-Id` через `_with_headers`.
- Readiness с неправильным `request_id`: берётся из `_rid(request)`, совпадает с заголовком.
### Уровень 4
1. **Конкурентная идемпотентность** — см. п. 2 выше.
2. **Migrate без лимитов** — добавлен `deploy.resources.limits` (cpus: 0.5, memory: 256M).
3. **Healthcheck в отчёте.** Синхронизировано:
- `postgres` — `pg_isready` каждые 5s.
- `cache` — `valkey-cli ping` каждые 5s.
- `api` — `urllib /health/live` каждые 5s, `start_period: 15s`.
- `migrate` — healthcheck **нет** (разовый сервис).
- `lb` — healthcheck **нет** в Compose; HAProxy проверяет бэкенды через `option httpchk GET /health/ready` (inter 2s fall 2 rise 2) в `haproxy.cfg`.
## Ограничения и следующие шаги
- **SPOF:** LB (HAProxy) и Postgres — единственные экземпляры. Падение LB → сервис недоступен. Падение Postgres → 503. Это **осознанное** ограничение lab3: HA для БД и LB — тема lab5+ или трека «Отказ узла».
+5 -1
View File
@@ -48,6 +48,11 @@ services:
condition: service_healthy
environment:
DATABASE_URL: postgresql://${POSTGRES_USER:-messenger}:${POSTGRES_PASSWORD:-changeme}@postgres:5432/${POSTGRES_DB:-messenger}
deploy:
resources:
limits:
cpus: "0.5"
memory: 256M
restart: "no"
api:
@@ -67,7 +72,6 @@ services:
SESSION_TTL_SECONDS: ${SESSION_TTL_SECONDS:-3600}
DATABASE_URL: postgresql://${POSTGRES_USER:-messenger}:${POSTGRES_PASSWORD:-changeme}@postgres:5432/${POSTGRES_DB:-messenger}
CACHE_URL: redis://cache:6379/0
INSTANCE_ID: ${HOSTNAME:-unknown}
deploy:
replicas: 2
resources:
+22 -13
View File
@@ -7,9 +7,8 @@ from fastapi import Request
from fastapi.responses import JSONResponse
from starlette.middleware.base import BaseHTTPMiddleware
from .store import ApiError
from .instance import INSTANCE_ID
from .store import ApiError
REQUEST_ID_HEADER = "X-Request-Id"
@@ -33,13 +32,11 @@ def _rid(request: Request) -> str:
return getattr(request.state, "request_id", None) or uuid.uuid4().hex
async def api_error_handler(request: Request, exc: Exception) -> JSONResponse:
assert isinstance(exc, ApiError)
response = JSONResponse(
status_code=exc.status,
content=_envelope(exc.code, exc.message, _rid(request)),
)
if exc.status == 401:
def _with_headers(response: JSONResponse, request: Request, status: int) -> JSONResponse:
rid = _rid(request)
response.headers[REQUEST_ID_HEADER] = rid
response.headers["X-Instance-Id"] = INSTANCE_ID
if status == 401:
scheme = "Bearer"
if request.url.path == "/api/v1/auth/sessions":
scheme = "Basic"
@@ -47,22 +44,34 @@ async def api_error_handler(request: Request, exc: Exception) -> JSONResponse:
return response
async def api_error_handler(request: Request, exc: Exception) -> JSONResponse:
assert isinstance(exc, ApiError)
response = JSONResponse(
status_code=exc.status,
content=_envelope(exc.code, exc.message, _rid(request)),
)
return _with_headers(response, request, exc.status)
async def validation_handler(request: Request, exc: Exception) -> JSONResponse:
return JSONResponse(
response = JSONResponse(
status_code=400,
content=_envelope("invalid_request", "Request validation failed", _rid(request)),
)
return _with_headers(response, request, 400)
async def json_decode_handler(request: Request, exc: Exception) -> JSONResponse:
return JSONResponse(
response = JSONResponse(
status_code=400,
content=_envelope("invalid_request", "Malformed JSON", _rid(request)),
)
return _with_headers(response, request, 400)
async def unhandled_handler(request: Request, exc: Exception) -> JSONResponse:
return JSONResponse(
response = JSONResponse(
status_code=500,
content=_envelope("internal_error", "Internal error", _rid(request)),
)
)
return _with_headers(response, request, 500)
+21 -5
View File
@@ -5,7 +5,8 @@ import os
import re
from contextlib import asynccontextmanager
from typing import Optional
import uuid
import uuid
from uuid import UUID
from fastapi import Depends, FastAPI, Header, Query, Request, Response
from fastapi.exceptions import RequestValidationError
@@ -76,11 +77,19 @@ async def _http_exc(request: Request, exc: StarletteHTTPException):
content=_envelope(code, str(exc.detail), _rid(request)),
)
# ---------- утилиты валидации ----------
def valid_uuid(value: str, name: str = "id") -> str:
"""Проверяет, что value — валидный UUID. Иначе бросает BadRequest (400)."""
try:
UUID(value)
except (ValueError, TypeError, AttributeError):
raise BadRequest(f"Invalid {name}")
return value
# ---------- зависимости ----------
def _parse_basic(header_value: str) -> tuple[str, str]:
"""Разбирает `Basic base64(username:password)`. Бросает Unauthorized."""
import base64
if not header_value.lower().startswith("basic "):
raise Unauthorized("Basic authentication required")
@@ -125,17 +134,21 @@ def liveness():
@app.get("/health/ready", response_model=Health)
def readiness():
def readiness(request: Request):
try:
from .db import transaction
with transaction() as conn:
with conn.cursor() as cur:
cur.execute("SELECT 1")
except Exception:
return JSONResponse(
rid = _rid(request)
response = JSONResponse(
status_code=503,
content=_envelope("unavailable", "Database unavailable", "readiness"),
content=_envelope("unavailable", "Database unavailable", rid),
)
response.headers["X-Request-Id"] = rid
response.headers["X-Instance-Id"] = INSTANCE_ID
return response
return {"status": "ok"}
@@ -204,6 +217,7 @@ def list_chats(
@app.get("/api/v1/chats/{chat_id}", response_model=Chat)
def get_chat(chat_id: str, user_id: str = Depends(current_user_id)):
chat_id = valid_uuid(chat_id, "chat_id")
chat = store.get_chat(chat_id, user_id)
if chat is None:
raise NotFound()
@@ -219,6 +233,7 @@ def create_message(
user_id: str = Depends(current_user_id),
idempotency_key: Optional[str] = Header(default=None, alias="Idempotency-Key"),
):
chat_id = valid_uuid(chat_id, "chat_id")
if not idempotency_key:
raise BadRequest("Idempotency-Key header is required")
if not re.match(r"^[A-Za-z0-9._:-]{1,128}$", idempotency_key):
@@ -237,6 +252,7 @@ def list_messages(
cursor: Optional[str] = Query(default=None, min_length=1),
user_id: str = Depends(current_user_id),
):
chat_id = valid_uuid(chat_id, "chat_id")
items, next_cursor = store.list_messages(chat_id, user_id, limit, cursor)
return {"items": items, "next_cursor": next_cursor}
+61 -86
View File
@@ -270,96 +270,71 @@ class Store:
path = f"/api/v1/chats/{chat_id}/messages"
method = "POST"
with transaction() as conn:
with conn.cursor() as cur:
cur.execute(
"""
SELECT request_hash, response_status, response_body
FROM idempotency_keys
WHERE user_id = %s AND method = %s AND path = %s AND key = %s
""",
(sender_id, method, path, idempotency_key),
)
row = cur.fetchone()
if row is not None:
if row["request_hash"] != request_hash:
raise Conflict("idempotency key reused with different body")
return row["response_body"], row["response_status"]
import time as _time
for attempt in range(5):
try:
with transaction() as conn:
with conn.cursor() as cur:
cur.execute(
"SELECT 1 FROM chat_members WHERE chat_id = %s AND user_id = %s",
(chat_id, sender_id),
)
if cur.fetchone() is None:
raise NotFound()
try:
with transaction() as conn:
with conn.cursor() as cur:
cur.execute(
"""
INSERT INTO idempotency_keys
(user_id, method, path, key, request_hash,
response_status, response_body, created_at)
VALUES (%s, %s, %s, %s, %s, 0, '{}'::jsonb, %s)
""",
(sender_id, method, path, idempotency_key, request_hash, _now()),
)
except UniqueViolation:
with transaction() as conn:
with conn.cursor() as cur:
cur.execute(
"""
SELECT request_hash, response_status, response_body
FROM idempotency_keys
WHERE user_id = %s AND method = %s AND path = %s AND key = %s
""",
(sender_id, method, path, idempotency_key),
)
row = cur.fetchone()
if row is None:
raise Conflict("idempotency key race, please retry")
if row["request_hash"] != request_hash:
raise Conflict("idempotency key reused with different body")
if row["response_status"] == 0:
raise Conflict("idempotency key in progress, please retry")
return row["response_body"], row["response_status"]
cur.execute(
"""
SELECT request_hash, response_status, response_body
FROM idempotency_keys
WHERE user_id = %s AND method = %s AND path = %s AND key = %s
""",
(sender_id, method, path, idempotency_key),
)
row = cur.fetchone()
if row is not None:
if row["request_hash"] != request_hash:
raise Conflict(
"idempotency key reused with different body"
)
return row["response_body"], row["response_status"]
with transaction() as conn:
with conn.cursor() as cur:
cur.execute(
"SELECT 1 FROM chat_members WHERE chat_id = %s AND user_id = %s",
(chat_id, sender_id),
)
if cur.fetchone() is None:
cur.execute(
"DELETE FROM idempotency_keys WHERE user_id = %s AND method = %s AND path = %s AND key = %s",
(sender_id, method, path, idempotency_key),
)
raise NotFound()
mid = str(uuid.uuid4())
now = _now()
cur.execute(
"""
INSERT INTO messages (id, chat_id, sender_id, text, created_at)
VALUES (%s, %s, %s, %s, %s)
""",
(mid, chat_id, sender_id, text, now),
)
message = {
"id": mid,
"chat_id": chat_id,
"sender_id": sender_id,
"text": text,
"created_at": iso(now),
}
mid = str(uuid.uuid4())
now = _now()
cur.execute(
"""
INSERT INTO messages (id, chat_id, sender_id, text, created_at)
VALUES (%s, %s, %s, %s, %s)
""",
(mid, chat_id, sender_id, text, now),
)
cur.execute(
"""
INSERT INTO idempotency_keys
(user_id, method, path, key, request_hash,
response_status, response_body, created_at)
VALUES (%s, %s, %s, %s, %s, 201, %s, %s)
""",
(
sender_id, method, path, idempotency_key,
request_hash,
_json.dumps(message, ensure_ascii=False),
now,
),
)
return message, 201
except UniqueViolation:
_time.sleep(0.05)
continue
message = {
"id": mid,
"chat_id": chat_id,
"sender_id": sender_id,
"text": text,
"created_at": iso(now),
}
cur.execute(
"""
UPDATE idempotency_keys
SET response_status = 201, response_body = %s
WHERE user_id = %s AND method = %s AND path = %s AND key = %s
""",
(_json.dumps(message, ensure_ascii=False),
sender_id, method, path, idempotency_key),
)
return message, 201
raise Conflict("idempotency key in progress, please retry")
def list_messages(
self,
+10 -1
View File
@@ -76,7 +76,7 @@ def main():
"Authorization": "Bearer " + token,
"Idempotency-Key": idem_key,
},
body={"text": f"parallel-message-{i}"}, # ← ВАЖНО: разные тексты!
body={"text": "parallel-message-same"}, # ← ОДИНАКОВЫЙ текст!
)
with results_lock:
results.append((code, body))
@@ -101,6 +101,15 @@ def main():
assert len(messages) == 1, f"expected exactly 1 message, got {len(messages)}"
print("PASS: parallel idempotency — exactly one message created")
# Проверка: все ответы должны быть одинаковыми
codes = [c for c, _ in results]
bodies = [json.dumps(b, sort_keys=True) for _, b in results if b is not None]
unique_bodies = set(bodies)
print(f"Status codes: {dict((c, codes.count(c)) for c in set(codes))}")
print(f"Unique response bodies: {len(unique_bodies)}")
assert codes.count(201) == THREADS, f"expected all {THREADS} to be 201, got {codes.count(201)}"
assert len(unique_bodies) == 1, f"expected same response body, got {len(unique_bodies)}"
if __name__ == "__main__":
main()