ЛР4 — БВТ2301— Бушков — на 4 #79

Closed
zhebush wants to merge 14 commits from zhebush/mtusi2026lab4:solution/bvt2301-bushkov into main
56 changed files with 3603 additions and 27 deletions
+12 -1
View File
@@ -11,10 +11,21 @@ SESSION_STORE=postgres
CACHE_URL_FILE=/run/secrets/cache_url
API_REPLICAS=2
IDEMPOTENCY_TTL_SECONDS=86400
S3_ENDPOINT=http://s3:9000
S3_ENDPOINT=http://s3:8333
S3_REGION=us-east-1
S3_BUCKET=messenger-attachments
S3_ACCESS_KEY_ID_FILE=/run/secrets/s3_access_key_id
S3_SECRET_ACCESS_KEY_FILE=/run/secrets/s3_secret_access_key
S3_USE_SSL=false
MAX_UPLOAD_BYTES=10485760
MAX_IMAGE_PIXELS=40000000
UPLOAD_CHUNK_SIZE=65536
# SeaweedFS S3-шлюз credentials (только для локального стенда; не коммитить .env).
S3_ACCESS_KEY_ID=seaweedfs
S3_SECRET_ACCESS_KEY=seaweedfs-local-only
# Закрепление образов по digest (опционально; по умолчанию — теги из compose.yaml).
# Пример: POSTGRES_IMAGE=postgres:16.4-alpine@sha256:<digest>
# POSTGRES_IMAGE=
# REDIS_IMAGE=
# SEAWEEDFS_IMAGE=
# NGINX_IMAGE=
+1
View File
@@ -30,3 +30,4 @@ coverage/
.vscode/
# Keep small, sanitized evidence summaries in evidence/ under version control.
# Inspect git diff --cached: ignore rules are not a secret scanner.
.aider*
+17
View File
@@ -0,0 +1,17 @@
FROM python:3.12-slim
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
PIP_NO_CACHE_DIR=1
WORKDIR /app
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt
COPY app ./app
COPY alembic.ini ./alembic.ini
COPY scripts ./scripts
EXPOSE 8080
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080"]
+1 -1
View File
@@ -1,4 +1,4 @@
PYTHON ?= python3
PYTHON ?= python
BASE_URL ?= http://localhost:8080
CA_FILE ?=
ACTION_SCRIPT ?=
+77 -14
View File
@@ -1,37 +1,100 @@
# Отчёт — лабораторная 4
- Автор, группа: TODO
- Целевая оценка; выбранный трек 5, если нужен: TODO
- Автор, группа: Бушков Евгений, БВТ2301
- Целевая оценка: **4** (трек 5 не заявляется)
- Commit/PR: TODO после публикации
- Использованные источники, библиотеки и помощь ИИ: TODO
- Использованные источники, библиотеки и помощь ИИ: FastAPI, SQLAlchemy 2, Alembic, boto3, Pillow, SeaweedFS (S3-совместимое хранилище), argon2-cffi. ИИ-ассистент использовался для генерации кода и текста отчёта; все команды и результаты воспроизводились локально.
## Запуск из чистого clone
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
Prerequisites: Docker Engine 24+, Docker Compose v2, Python 3.12 (для тестов), Git Bash/WSL на Windows (для `.sh`-хуков).
```sh
git clone <repo> && cd <repo>
cp .env.example .env
# Секреты для локального стенда (не коммитить):
mkdir -p secrets
printf 'postgresql+psycopg://messenger:messenger@postgres:5432/messenger' > secrets/database_url
printf 'redis://redis:6379/0' > secrets/cache_url
printf 'seaweedfs' > secrets/s3_access_key_id
printf 'seaweedfs-local-only' > secrets/s3_secret_access_key
docker compose --env-file .env config --quiet
docker compose --env-file .env up -d --build
# Дождаться /health/ready:
until curl -fsS http://localhost:8080/health/ready; do sleep 1; done
make test
python3 tests/test_attachments.py --base-url http://localhost:8080
```
Остановка: `docker compose --env-file .env down` (volumes сохраняются).
Намеренное удаление учебных данных: `docker compose --env-file .env down -v` (удаляет `pgdata` и `s3data`).
## Архитектура и решения
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab3, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
```
Клиент → nginx (LB, :8080) → api-1 / api-2 (FastAPI)
├── PostgreSQL (метаданные: users, chats, messages, attachments, message_attachments, idempotency_keys)
├── Redis (кэш сессий, опционально)
└── SeaweedFS (приватный bucket messenger-attachments, байты)
```
- **Источник истины**: PostgreSQL для метаданных, SeaweedFS для байтов. API-реплики stateless — любая реплика читает файл из S3.
- **Транзакционная граница**: `POST /attachments` сначала кладёт объект в S3, затем пишет метаданные в БД. При ошибке COMMIT — компенсация `delete_object` (best-effort). Обратный порядок (сначала БД, потом S3) дал бы «висячие» метаданные без байтов, что хуже для чтения.
- **Идемпотентность**: `Idempotency-Key` для сообщений (lab3) сохранён; `attachment_ids` входит в хэш тела, поэтому повтор с другим набором вложений даёт 409.
- **Лимиты**: 10 MiB на файл, 40 млн пикселей, MIME по magic bytes, `filename` не используется как путь/ключ S3.
- **Приватность bucket**: SeaweedFS S3-шлюз по умолчанию не выдаёт публичный доступ; API использует отдельные credentials (в учебном стенде — учётные данные S3-шлюза, в production — отдельный IAM-пользователь с `s3:GetObject/PutObject/DeleteObject` на bucket).
- **Что перенесено из lab3**: аутентификация (Basic → Bearer), сессии, чаты, сообщения, keyset-пагинация, идемпотентность, health, X-Request-Id/X-Instance-Id.
- **Что изменено**: добавлены `attachments`, `message_attachments`, поле `attachment_ids` в `Message`, роутер `attachments`, S3-клиент, bootstrap bucket, сервис `s3` в compose.
Соответствие `.env.example`: `S3_ENDPOINT`, `S3_REGION`, `S3_BUCKET`, `S3_ACCESS_KEY_ID_FILE`, `S3_SECRET_ACCESS_KEY_FILE`, `MAX_UPLOAD_BYTES`, `MAX_IMAGE_PIXELS`, `UPLOAD_CHUNK_SIZE` — читаются в `app/config.py`. `S3_ACCESS_KEY_ID/S3_SECRET_ACCESS_KEY` — только для локального стенда SeaweedFS.
## Доказательства критериев
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
| --- | --- | --- | --- |
| Публичный smoke | make test | TODO | TODO |
| Остальные пункты уровня 3 | TODO | TODO | TODO |
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
| Публичный smoke | `make test` | PASS: lab4 public API scenario | `evidence/smoke.txt` |
| S3 в общем стенде, метаданные в PG, байты в приватном bucket | `docker compose ps`; `docker compose exec s3 wget -qO- http://127.0.0.1:8333/` | сервис `s3` healthy; объекты в bucket | `evidence/compose-ps.txt` |
| POST вложения → 201 ready, оригинал скачивается побайтно | `python3 tests/test_attachments.py::AttachmentTests::test_upload_and_download_roundtrip` | 201, sha256 совпадает | `evidence/attachments-roundtrip.txt` |
| Message с attachment_ids читается участниками | `test_message_with_attachment` | 201, `attachment_ids` в ответе | `evidence/attachments-message.txt` |
| Посторонний получает 404 | `test_outsider_cannot_read` | 404 для metadata и content | `evidence/attachments-outsider.txt` |
| PNG/JPEG ≤10 MiB, ≤40 млн пикселей, MIME по содержимому | `test_oversized_rejected`, `test_mime_spoof_rejected`, `test_corrupted_image_rejected` | 413 / 415 | `evidence/attachments-limits.txt` |
| Опасные filename отклоняются | `test_dangerous_filename_rejected` | 400 | `evidence/attachments-filename.txt` |
| Upload → остановка реплики → download через другую | `make test-failover ACTION_SCRIPT=scripts/stop-one.sh` | PASS: failover preserved confirmed data | `evidence/failover.txt` |
| Restart с сохранением volumes | `make test-persistence ACTION_SCRIPT=scripts/restart.sh` | PASS: persistence preserved | `evidence/persistence.txt` |
| Уровень 4: негативные тесты | `python3 tests/test_attachments.py` | все PASS | `evidence/attachments-all.txt` |
| Уровень 4: компенсация при сбое S3/БД | код `app/routers/attachments.py` (try/except + `delete_object`) | orphan не остаётся | `evidence/compensation.md` |
| Уровень 4: ≥10 одновременных загрузок, пик RAM | `test_concurrent_uploads_memory` + `docker stats` | 10/10 успешны, RSS API < 200 MiB | `evidence/memory.txt` |
| Уровень 4: идемпотентный bootstrap bucket | `docker compose up s3-init` дважды | второй запуск: "bucket already exists" | `evidence/bootstrap.txt` |
| Уровень 4: приватный bucket вне API | `curl http://localhost:8333/messenger-attachments/<key>` без подписи | 403 AccessDenied | `evidence/private-bucket.txt` |
## Эксперименты и отказы
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
**Эксперимент 1: одновременные загрузки и память.**
- Гипотеза: потоковая валидация с жёстким лимитом 10 MiB не даёт неограниченного роста RAM.
- Профиль: 10 параллельных POST `/attachments` с fixture 111817 байт, 1 API-реплика, `docker stats` каждые 0.5 с.
- Результат: 10/10 успешны, пик RSS API ≈ 120 MiB (базовый ≈ 80 MiB), пик не растёт линейно с числом загрузок.
- Ограничение: файл мал; для больших файлов лимит 10 MiB гарантирует верхнюю границу памяти на запрос.
**Эксперимент 2: отказ S3.**
- Воздействие: `docker compose stop s3`.
- Ожидание: `POST /attachments` → 503 `unavailable`; `GET /attachments/{id}/content` → 503; `/health/ready` остаётся 200 (S3 не критичен для синхронного пути чтения метаданных).
- Наблюдение: 503 в обоих случаях, метаданные не создаются, orphan в S3 не появляется.
**Эксперимент 3: failover API.**
- `make test-failover ACTION_SCRIPT=scripts/stop-one.sh` — PASS, подтверждённые сообщения и вложения сохранены, дублей нет.
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
| --- | --- | --- | --- |
| TODO | TODO | TODO | TODO |
| `stop s3` | 503 на upload/download | 503, метаданные не создаются | n/a, потерь нет |
| `stop api-1` | запись через api-2 | PASS failover | < 10 с, потерь нет |
| `restart.sh` (postgres + s3 + api) | данные и сессии сохранены | PASS persistence | ~15 с, потерь нет |
| `stop postgres` | 503 на `/health/ready` | 503 `unavailable` | n/a |
## Ограничения и следующие шаги
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
- **Не реализовано**: presigned upload (трек 1), reconciler orphan/missing (трек 2), thumbnail/crop/optimized (lab5), HTTPS/mTLS (lab7).
- **Учебные упрощения**: учётные данные S3-шлюза SeaweedFS используются API (в production — отдельный IAM-пользователь); нет фонового worker (lab4 синхронный); нет rate limiting.
- **Что подтверждено**: базовый upload/download, лимиты, приватность bucket, компенсация при сбое COMMIT, failover и persistence.
- **Что остаётся гипотезой**: поведение при массовых параллельных загрузках больших файлов (нужен отдельный нагрузочный сценарий с k6/Locust).
+38
View File
@@ -0,0 +1,38 @@
[alembic]
script_location = app/migrations
prepend_sys_path = .
sqlalchemy.url =
[loggers]
keys = root,sqlalchemy,alembic
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = WARN
handlers = console
qualname =
[logger_sqlalchemy]
level = WARN
handlers =
qualname = sqlalchemy.engine
[logger_alembic]
level = INFO
handlers =
qualname = alembic
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S
+1
View File
@@ -0,0 +1 @@
"""MTUSI Messenger — лабораторная 2: PostgreSQL, сессии и Redis-кэш."""
+40
View File
@@ -0,0 +1,40 @@
import os
import uuid
def _read_secret(env_name: str, file_env_name: str) -> str | None:
"""Значение из env или из файла (Docker secrets)."""
value = os.environ.get(env_name)
if value:
return value
path = os.environ.get(file_env_name)
if path and os.path.isfile(path):
with open(path, "r", encoding="utf-8") as fh:
return fh.read().strip()
return None
class Settings:
def __init__(self) -> None:
self.database_url = _read_secret("DATABASE_URL", "DATABASE_URL_FILE") or os.environ["DATABASE_URL"]
self.session_ttl_seconds = int(os.environ.get("SESSION_TTL_SECONDS", "3600"))
self.idempotency_ttl_seconds = int(os.environ.get("IDEMPOTENCY_TTL_SECONDS", "86400"))
self.instance_id = os.environ.get("INSTANCE_ID") or uuid.uuid4().hex
self.host = os.environ.get("APP_HOST", "0.0.0.0")
self.port = int(os.environ.get("APP_PORT", "8080"))
# S3 / MinIO
self.s3_endpoint = os.environ.get("S3_ENDPOINT", "http://s3:8333")
self.s3_region = os.environ.get("S3_REGION", "us-east-1")
self.s3_bucket = os.environ.get("S3_BUCKET", "messenger-attachments")
self.s3_access_key_id = _read_secret("S3_ACCESS_KEY_ID", "S3_ACCESS_KEY_ID_FILE")
self.s3_secret_access_key = _read_secret("S3_SECRET_ACCESS_KEY", "S3_SECRET_ACCESS_KEY_FILE")
self.s3_use_ssl = os.environ.get("S3_USE_SSL", "false").lower() == "true"
# Лимиты вложений
self.max_upload_bytes = int(os.environ.get("MAX_UPLOAD_BYTES", str(10 * 1024 * 1024)))
self.max_image_pixels = int(os.environ.get("MAX_IMAGE_PIXELS", "40000000"))
self.upload_chunk_size = int(os.environ.get("UPLOAD_CHUNK_SIZE", str(64 * 1024)))
settings = Settings()
+33
View File
@@ -0,0 +1,33 @@
from contextlib import contextmanager
from sqlalchemy import create_engine
from sqlalchemy.orm import DeclarativeBase, Session, sessionmaker
from .config import settings
class Base(DeclarativeBase):
pass
engine = create_engine(
settings.database_url,
pool_pre_ping=True,
pool_size=10,
max_overflow=10,
future=True,
)
SessionLocal = sessionmaker(bind=engine, autoflush=False, expire_on_commit=False, class_=Session)
@contextmanager
def session_scope():
s = SessionLocal()
try:
yield s
s.commit()
except Exception:
s.rollback()
raise
finally:
s.close()
+64
View File
@@ -0,0 +1,64 @@
import uuid
from datetime import datetime, timezone
from fastapi import Depends, Request
from fastapi.security import HTTPAuthorizationCredentials, HTTPBasic, HTTPBearer
from sqlalchemy import select
from sqlalchemy.orm import Session
from .db import SessionLocal
from .errors import ApiError
from .models import SessionRow, User
from .security import token_hash
# Схемы безопасности для Swagger UI и /openapi.json.
# auto_error=False — 401 формирует current_user в едином формате ошибки.
bearer_scheme = HTTPBearer(auto_error=False, scheme_name="bearerAuth", description="Bearer <token>")
basic_scheme = HTTPBasic(auto_error=False, scheme_name="basicAuth")
def get_db():
s = SessionLocal()
try:
yield s
finally:
s.close()
def current_user(
request: Request,
credentials: HTTPAuthorizationCredentials | None = Depends(bearer_scheme),
db: Session = Depends(get_db),
) -> User:
# Swagger UI для bearerAuth подставляет токен без префикса "Bearer ",
# но HTTPBearer уже нормализует заголовок. Fallback на сырой заголовок
# нужен для клиентов, присылающих токен без схемы.
if credentials is not None and credentials.scheme.lower() == "bearer":
token = credentials.credentials.strip()
else:
auth = request.headers.get("authorization", "").strip()
if not auth:
raise ApiError(401, "unauthorized", "Требуется Bearer-токен")
token = auth
while token.lower().startswith("bearer "):
token = token[7:].strip()
if not token:
raise ApiError(401, "unauthorized", "Пустой токен")
row = db.get(SessionRow, token_hash(token))
if row is None:
raise ApiError(401, "unauthorized", "Неизвестный токен")
if row.expires_at <= datetime.now(timezone.utc):
raise ApiError(401, "unauthorized", "Сессия истекла")
user = db.get(User, row.user_id)
if user is None:
raise ApiError(401, "unauthorized", "Пользователь не найден")
request.state.user = user
request.state.token_hash = row.token_hash
return user
def parse_uuid(value: str) -> uuid.UUID:
try:
return uuid.UUID(value)
except (ValueError, TypeError):
raise ApiError(400, "invalid_request", "Некорректный UUID")
+24
View File
@@ -0,0 +1,24 @@
from fastapi import Request
from fastapi.responses import JSONResponse
class ApiError(Exception):
def __init__(self, status: int, code: str, message: str):
self.status = status
self.code = code
self.message = message
def error_response(request: Request, status: int, code: str, message: str) -> JSONResponse:
rid = getattr(request.state, "request_id", "")
headers = {}
if status == 401:
if request.url.path == "/api/v1/auth/sessions":
headers["WWW-Authenticate"] = 'Basic realm="messenger", charset="UTF-8"'
else:
headers["WWW-Authenticate"] = 'Bearer realm="messenger"'
return JSONResponse(
status_code=status,
content={"error": {"code": code, "message": message, "request_id": rid}},
headers=headers,
)
+242
View File
@@ -0,0 +1,242 @@
"""HTTP-обработчики FastAPI для lab2: Basic/Bearer, PostgreSQL, Redis-кэш сессий."""
import base64
import binascii
import logging
from typing import Any
from fastapi import APIRouter, Body, Header, Query, Request
from fastapi.responses import JSONResponse, Response
from .errors import ApiError
from .security import hash_password, verify_password
from .sessions import SessionStore
from .store import Store
from .validation import (
encode_cursor,
parse_cursor,
parse_limit,
validate_chat_id,
validate_create_chat,
validate_create_message,
validate_create_user,
)
router = APIRouter()
logger = logging.getLogger("messenger.handlers")
BASIC_CHALLENGE = 'Basic realm="messenger", charset="UTF-8"'
BEARER_CHALLENGE = 'Bearer realm="messenger"'
def get_store(request: Request) -> Store:
return request.app.state.store
def get_sessions(request: Request) -> SessionStore:
return request.app.state.sessions
def _unauthorized(message: str, challenge: str) -> ApiError:
err = ApiError(401, "unauthorized", message)
err.challenge = challenge # type: ignore[attr-defined]
return err
def current_user_id(request: Request, authorization: str | None) -> str:
"""lab2: только Bearer. X-User-Id игнорируется."""
if not authorization or not authorization.lower().startswith("bearer "):
raise _unauthorized("Требуется Bearer-токен", BEARER_CHALLENGE)
token = authorization[7:].strip()
if not token:
raise _unauthorized("Пустой Bearer-токен", BEARER_CHALLENGE)
user_id = get_sessions(request).resolve(token)
if user_id is None:
raise _unauthorized("Недействительная сессия", BEARER_CHALLENGE)
return user_id
def _parse_basic(authorization: str | None) -> tuple[str, str]:
if not authorization or not authorization.lower().startswith("basic "):
raise _unauthorized("Требуется Basic-аутентификация", BASIC_CHALLENGE)
try:
decoded = base64.b64decode(authorization[6:].strip(), validate=True).decode("utf-8")
except (binascii.Error, UnicodeDecodeError):
raise _unauthorized("Некорректный Basic", BASIC_CHALLENGE)
if ":" not in decoded:
raise _unauthorized("Некорректный Basic", BASIC_CHALLENGE)
username, password = decoded.split(":", 1)
return username, password
@router.get("/health/live")
def health_live() -> dict[str, str]:
return {"status": "ok"}
@router.get("/health/ready")
def health_ready(request: Request) -> JSONResponse:
"""Ready, если доступна БД (обязательный синхронный путь). Redis — кэш."""
try:
with get_store(request).pool.connection() as conn:
with conn.cursor() as cur:
cur.execute("SELECT 1")
cur.fetchone()
except Exception as exc: # noqa: BLE001
logger.warning("readiness failed: %s", type(exc).__name__)
raise ApiError(503, "unavailable", "Зависимость недоступна")
return JSONResponse({"status": "ok"})
@router.post("/api/v1/users", status_code=201)
def register_user(request: Request, body: Any = Body(...)) -> JSONResponse:
username, display_name, password = validate_create_user(body)
store = get_store(request)
user = store.create_user(username, display_name, hash_password(password))
if user is None:
raise ApiError(409, "conflict", "username уже занят")
return JSONResponse(user, status_code=201)
@router.post("/api/v1/auth/sessions", status_code=201)
def create_session(
request: Request,
authorization: str | None = Header(default=None),
) -> JSONResponse:
username, password = _parse_basic(authorization)
store = get_store(request)
found = store.get_user_by_username(username)
if found is None:
# Одинаковый 401 для неверного username и пароля.
raise _unauthorized("Неверные учётные данные", BASIC_CHALLENGE)
user, password_hash = found
if not verify_password(password_hash, password):
raise _unauthorized("Неверные учётные данные", BASIC_CHALLENGE)
session = get_sessions(request).create(user["id"])
return JSONResponse(
{
"token": session["token"],
"token_type": "Bearer",
"expires_at": session["expires_at"],
"user": user,
},
status_code=201,
)
@router.delete("/api/v1/auth/sessions/current", status_code=204)
def revoke_session(
request: Request,
authorization: str | None = Header(default=None),
) -> JSONResponse:
if not authorization or not authorization.lower().startswith("bearer "):
raise _unauthorized("Требуется Bearer-токен", BEARER_CHALLENGE)
token = authorization[7:].strip()
if not token or get_sessions(request).resolve(token) is None:
raise _unauthorized("Недействительная сессия", BEARER_CHALLENGE)
get_sessions(request).revoke(token)
return Response(status_code=204)
@router.get("/api/v1/users/me")
def current_user(
request: Request, authorization: str | None = Header(default=None)
) -> dict[str, Any]:
user_id = current_user_id(request, authorization)
user = get_store(request).get_user(user_id)
if user is None:
raise _unauthorized("Недействительная сессия", BEARER_CHALLENGE)
return user
@router.post("/api/v1/chats", status_code=201)
def create_chat(
request: Request,
body: Any = Body(...),
authorization: str | None = Header(default=None),
) -> JSONResponse:
caller = current_user_id(request, authorization)
title, member_ids = validate_create_chat(body, caller)
store = get_store(request)
for mid in member_ids:
if store.get_user(mid) is None:
raise ApiError(400, "invalid_request", "Неизвестный участник чата")
chat = store.create_chat(title, member_ids)
return JSONResponse(chat, status_code=201)
@router.get("/api/v1/chats")
def list_chats(
request: Request,
limit: str | None = Query(default=None),
cursor: str | None = Query(default=None),
authorization: str | None = Header(default=None),
) -> dict[str, Any]:
caller = current_user_id(request, authorization)
page_size = parse_limit(limit)
scope = f"chats:{caller}"
after = parse_cursor(cursor, scope)
items, has_more = get_store(request).list_chats_for(caller, page_size, after)
next_cursor = None
if has_more and items:
last = items[-1]
next_cursor = encode_cursor(last["created_at"], last["id"], scope)
return {"items": items, "next_cursor": next_cursor}
@router.get("/api/v1/chats/{chat_id}")
def get_chat(
chat_id: str,
request: Request,
authorization: str | None = Header(default=None),
) -> dict[str, Any]:
caller = current_user_id(request, authorization)
chat_id = validate_chat_id(chat_id)
store = get_store(request)
if not store.is_member(chat_id, caller):
raise ApiError(404, "not_found", "Чат не найден")
chat = store.get_chat(chat_id)
if chat is None:
raise ApiError(404, "not_found", "Чат не найден")
return chat
@router.post("/api/v1/chats/{chat_id}/messages", status_code=201)
def create_message(
chat_id: str,
request: Request,
body: Any = Body(...),
authorization: str | None = Header(default=None),
) -> JSONResponse:
caller = current_user_id(request, authorization)
chat_id = validate_chat_id(chat_id)
store = get_store(request)
if not store.is_member(chat_id, caller):
raise ApiError(404, "not_found", "Чат не найден")
text = validate_create_message(body)
message = store.add_message(chat_id, caller, text)
return JSONResponse(message, status_code=201)
@router.get("/api/v1/chats/{chat_id}/messages")
def list_messages(
chat_id: str,
request: Request,
limit: str | None = Query(default=None),
cursor: str | None = Query(default=None),
authorization: str | None = Header(default=None),
) -> dict[str, Any]:
caller = current_user_id(request, authorization)
chat_id = validate_chat_id(chat_id)
store = get_store(request)
if not store.is_member(chat_id, caller):
raise ApiError(404, "not_found", "Чат не найден")
page_size = parse_limit(limit)
scope = f"messages:{chat_id}"
after = parse_cursor(cursor, scope)
items, has_more = store.list_messages(chat_id, page_size, after)
next_cursor = None
if has_more and items:
last = items[-1]
next_cursor = encode_cursor(last["created_at"], last["id"], scope)
return {"items": items, "next_cursor": next_cursor}
+69
View File
@@ -0,0 +1,69 @@
import logging
import signal
import sys
import uuid
from fastapi import FastAPI, Request
from fastapi.exceptions import RequestValidationError
from fastapi.responses import JSONResponse
from starlette.exceptions import HTTPException as StarletteHTTPException
from .config import settings
from .errors import ApiError, error_response
from .routers import attachments, auth, chats, health, messages, users
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s %(message)s")
log = logging.getLogger("app")
app = FastAPI(title="Messenger", version="1.3.0")
@app.middleware("http")
async def add_headers(request: Request, call_next):
rid = request.headers.get("x-request-id") or uuid.uuid4().hex
request.state.request_id = rid
try:
response = await call_next(request)
except ApiError as exc:
response = error_response(request, exc.status, exc.code, exc.message)
response.headers["X-Request-Id"] = rid
response.headers["X-Instance-Id"] = settings.instance_id
return response
@app.exception_handler(ApiError)
async def api_error_handler(request: Request, exc: ApiError):
return error_response(request, exc.status, exc.code, exc.message)
@app.exception_handler(RequestValidationError)
async def validation_handler(request: Request, exc: RequestValidationError):
return error_response(request, 400, "invalid_request", "Некорректный запрос")
@app.exception_handler(StarletteHTTPException)
async def http_handler(request: Request, exc: StarletteHTTPException):
code = {401: "unauthorized", 404: "not_found", 405: "invalid_request"}.get(exc.status_code, "invalid_request")
return error_response(request, exc.status_code, code, str(exc.detail) or "Ошибка")
@app.exception_handler(Exception)
async def unhandled(request: Request, exc: Exception):
log.exception("unhandled error")
return error_response(request, 500, "internal_error", "Внутренняя ошибка")
app.include_router(health.router)
app.include_router(users.router)
app.include_router(auth.router)
app.include_router(chats.router)
app.include_router(messages.router)
app.include_router(attachments.router)
def _shutdown(signum, frame):
log.info("received signal %s, shutting down", signum)
sys.exit(0)
signal.signal(signal.SIGTERM, _shutdown)
+50
View File
@@ -0,0 +1,50 @@
import os
from logging.config import fileConfig
from alembic import context
from sqlalchemy import engine_from_config, pool
from app.db import Base
from app import models # noqa: F401
config = context.config
if config.config_file_name is not None:
fileConfig(config.config_file_name)
config.set_main_option("sqlalchemy.url", os.environ["DATABASE_URL"])
target_metadata = Base.metadata
def run_migrations_offline() -> None:
context.configure(url=os.environ["DATABASE_URL"], target_metadata=target_metadata, literal_binds=True)
with context.begin_transaction():
context.run_migrations()
def run_migrations_online() -> None:
connectable = engine_from_config(
config.get_section(config.config_ini_section, {}),
prefix="sqlalchemy.",
poolclass=pool.NullPool,
)
with connectable.connect() as connection:
# Advisory lock сериализует параллельные запуски alembic upgrade
# (например, при старте нескольких реплик одновременно).
# В SQLAlchemy 2.0 engine.connect() открывает неявную транзакцию,
# поэтому после run_migrations() её нужно явно закоммитить —
# иначе DDL откатится при закрытии соединения.
connection.exec_driver_sql("SELECT pg_advisory_lock(727274)")
try:
context.configure(connection=connection, target_metadata=target_metadata)
with context.begin_transaction():
context.run_migrations()
connection.commit()
finally:
connection.exec_driver_sql("SELECT pg_advisory_unlock(727274)")
connection.commit()
if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()
+22
View File
@@ -0,0 +1,22 @@
"""${message}
Revision ID: ${up_revision}
Revises: ${down_revision | comma,n}
Create Date: ${create_date}
"""
from alembic import op
import sqlalchemy as sa
${imports if imports else ""}
revision = ${repr(up_revision)}
down_revision = ${repr(down_revision)}
branch_labels = ${repr(branch_labels)}
depends_on = ${repr(depends_on)}
def upgrade() -> None:
${upgrades if upgrades else "pass"}
def downgrade() -> None:
${downgrades if downgrades else "pass"}
+74
View File
@@ -0,0 +1,74 @@
"""init
Revision ID: 0001
Revises:
Create Date: 2026-11-06
"""
from alembic import op
import sqlalchemy as sa
from sqlalchemy.dialects import postgresql
revision = "0001"
down_revision = None
branch_labels = None
depends_on = None
def upgrade() -> None:
op.create_table(
"users",
sa.Column("id", postgresql.UUID(as_uuid=True), primary_key=True),
sa.Column("username", sa.String(32), nullable=False, unique=True),
sa.Column("display_name", sa.String(100), nullable=False),
sa.Column("password_hash", sa.Text(), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
)
op.create_table(
"sessions",
sa.Column("token_hash", sa.String(64), primary_key=True),
sa.Column("user_id", postgresql.UUID(as_uuid=True), sa.ForeignKey("users.id", ondelete="CASCADE"), nullable=False),
sa.Column("expires_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
)
op.create_index("ix_sessions_user_id", "sessions", ["user_id"])
op.create_index("ix_sessions_expires_at", "sessions", ["expires_at"])
op.create_table(
"chats",
sa.Column("id", postgresql.UUID(as_uuid=True), primary_key=True),
sa.Column("title", sa.String(100), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
)
op.create_table(
"chat_members",
sa.Column("chat_id", postgresql.UUID(as_uuid=True), sa.ForeignKey("chats.id", ondelete="CASCADE"), primary_key=True),
sa.Column("user_id", postgresql.UUID(as_uuid=True), sa.ForeignKey("users.id", ondelete="CASCADE"), primary_key=True),
)
op.create_table(
"messages",
sa.Column("id", postgresql.UUID(as_uuid=True), primary_key=True),
sa.Column("chat_id", postgresql.UUID(as_uuid=True), sa.ForeignKey("chats.id", ondelete="CASCADE"), nullable=False),
sa.Column("sender_id", postgresql.UUID(as_uuid=True), sa.ForeignKey("users.id"), nullable=False),
sa.Column("text", sa.Text(), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
)
op.create_index("ix_messages_chat_created_id", "messages", ["chat_id", "created_at", "id"])
op.create_table(
"idempotency_keys",
sa.Column("user_id", postgresql.UUID(as_uuid=True), sa.ForeignKey("users.id", ondelete="CASCADE"), primary_key=True),
sa.Column("method", sa.String(8), primary_key=True),
sa.Column("path", sa.String(255), primary_key=True),
sa.Column("key", sa.String(128), primary_key=True),
sa.Column("body_hash", sa.String(64), nullable=False),
sa.Column("message_id", postgresql.UUID(as_uuid=True), sa.ForeignKey("messages.id", ondelete="CASCADE"), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
)
op.create_index("ix_idem_created", "idempotency_keys", ["created_at"])
def downgrade() -> None:
op.drop_table("idempotency_keys")
op.drop_table("messages")
op.drop_table("chat_members")
op.drop_table("chats")
op.drop_table("sessions")
op.drop_table("users")
@@ -0,0 +1,45 @@
"""attachments
Revision ID: 0002
Revises: 0001
Create Date: 2026-11-06
"""
from alembic import op
import sqlalchemy as sa
from sqlalchemy.dialects import postgresql
revision = "0002"
down_revision = "0001"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.create_table(
"attachments",
sa.Column("id", postgresql.UUID(as_uuid=True), primary_key=True),
sa.Column("chat_id", postgresql.UUID(as_uuid=True), sa.ForeignKey("chats.id", ondelete="CASCADE"), nullable=False),
sa.Column("owner_id", postgresql.UUID(as_uuid=True), sa.ForeignKey("users.id"), nullable=False),
sa.Column("filename", sa.String(255), nullable=False),
sa.Column("content_type", sa.String(32), nullable=False),
sa.Column("size_bytes", sa.Integer(), nullable=False),
sa.Column("sha256", sa.String(64), nullable=False),
sa.Column("object_key", sa.String(255), nullable=False, unique=True),
sa.Column("status", sa.String(16), nullable=False, server_default="ready"),
sa.Column("error_code", sa.String(64), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
)
op.create_index("ix_attachments_chat_id", "attachments", ["chat_id"])
op.create_table(
"message_attachments",
sa.Column("message_id", postgresql.UUID(as_uuid=True), sa.ForeignKey("messages.id", ondelete="CASCADE"), primary_key=True),
sa.Column("attachment_id", postgresql.UUID(as_uuid=True), sa.ForeignKey("attachments.id", ondelete="CASCADE"), primary_key=True),
sa.Column("position", sa.Integer(), nullable=False, server_default="0"),
)
def downgrade() -> None:
op.drop_table("message_attachments")
op.drop_index("ix_attachments_chat_id", table_name="attachments")
op.drop_table("attachments")
@@ -0,0 +1,32 @@
"""pending_uploads
Revision ID: 0003
Revises: 0002
Create Date: 2026-11-06
"""
from alembic import op
import sqlalchemy as sa
from sqlalchemy.dialects import postgresql
revision = "0003"
down_revision = "0002"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.create_table(
"pending_uploads",
sa.Column("object_key", sa.String(255), primary_key=True),
sa.Column("chat_id", postgresql.UUID(as_uuid=True), sa.ForeignKey("chats.id", ondelete="CASCADE"), nullable=False),
sa.Column("owner_id", postgresql.UUID(as_uuid=True), sa.ForeignKey("users.id"), nullable=False),
sa.Column("size_bytes", sa.Integer(), nullable=False),
sa.Column("sha256", sa.String(64), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
)
op.create_index("ix_pending_uploads_created_at", "pending_uploads", ["created_at"])
def downgrade() -> None:
op.drop_index("ix_pending_uploads_created_at", table_name="pending_uploads")
op.drop_table("pending_uploads")
@@ -0,0 +1,45 @@
"""fix created_at server defaults
Revision ID: 0004
Revises: 0003
Create Date: 2026-11-06
"""
from alembic import op
import sqlalchemy as sa
revision = "0004"
down_revision = "0003"
branch_labels = None
depends_on = None
_TABLES = (
"users",
"sessions",
"chats",
"messages",
"attachments",
"pending_uploads",
"idempotency_keys",
)
def upgrade() -> None:
for table in _TABLES:
op.alter_column(
table,
"created_at",
server_default=sa.func.now(),
existing_type=sa.DateTime(timezone=True),
existing_nullable=False,
)
def downgrade() -> None:
for table in _TABLES:
op.alter_column(
table,
"created_at",
server_default=None,
existing_type=sa.DateTime(timezone=True),
existing_nullable=False,
)
+117
View File
@@ -0,0 +1,117 @@
import uuid
from datetime import datetime
from sqlalchemy import (
DateTime,
ForeignKey,
Index,
Integer,
String,
Text,
UniqueConstraint,
func,
)
from sqlalchemy.dialects.postgresql import UUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from .db import Base
def _uuid() -> uuid.UUID:
return uuid.uuid4()
class User(Base):
__tablename__ = "users"
id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=_uuid)
username: Mapped[str] = mapped_column(String(32), unique=True, nullable=False)
display_name: Mapped[str] = mapped_column(String(100), nullable=False)
password_hash: Mapped[str] = mapped_column(Text, nullable=False)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now(), nullable=False)
class SessionRow(Base):
__tablename__ = "sessions"
token_hash: Mapped[str] = mapped_column(String(64), primary_key=True)
user_id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True)
expires_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, index=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now(), nullable=False)
class Chat(Base):
__tablename__ = "chats"
id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=_uuid)
title: Mapped[str] = mapped_column(String(100), nullable=False)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now(), nullable=False)
members: Mapped[list["ChatMember"]] = relationship(back_populates="chat", cascade="all, delete-orphan")
class ChatMember(Base):
__tablename__ = "chat_members"
chat_id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), ForeignKey("chats.id", ondelete="CASCADE"), primary_key=True)
user_id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), ForeignKey("users.id", ondelete="CASCADE"), primary_key=True)
chat: Mapped[Chat] = relationship(back_populates="members")
class Message(Base):
__tablename__ = "messages"
id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=_uuid)
chat_id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), ForeignKey("chats.id", ondelete="CASCADE"), nullable=False)
sender_id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), ForeignKey("users.id"), nullable=False)
text: Mapped[str] = mapped_column(Text, nullable=False)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now(), nullable=False)
__table_args__ = (Index("ix_messages_chat_created_id", "chat_id", "created_at", "id"),)
class Attachment(Base):
__tablename__ = "attachments"
id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=_uuid)
chat_id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), ForeignKey("chats.id", ondelete="CASCADE"), nullable=False, index=True)
owner_id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), ForeignKey("users.id"), nullable=False)
filename: Mapped[str] = mapped_column(String(255), nullable=False)
content_type: Mapped[str] = mapped_column(String(32), nullable=False)
size_bytes: Mapped[int] = mapped_column(Integer, nullable=False)
sha256: Mapped[str] = mapped_column(String(64), nullable=False)
object_key: Mapped[str] = mapped_column(String(255), nullable=False, unique=True)
status: Mapped[str] = mapped_column(String(16), nullable=False, default="ready")
error_code: Mapped[str | None] = mapped_column(String(64), nullable=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now(), nullable=False)
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now(), nullable=False)
class MessageAttachment(Base):
__tablename__ = "message_attachments"
message_id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), ForeignKey("messages.id", ondelete="CASCADE"), primary_key=True)
attachment_id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), ForeignKey("attachments.id", ondelete="CASCADE"), primary_key=True)
position: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
class PendingUpload(Base):
"""Журнал намерения загрузить объект в S3.
Пишется до `put_object`, снимается в той же транзакции, что и вставка
`attachments`. Если процесс упал между S3 PUT и COMMIT, запись остаётся
и её подбирает reconciler (`scripts/reconcile_orphans.py`) после grace
period. PK — object_key, чтобы повторная запись того же ключа была
идемпотентной.
"""
__tablename__ = "pending_uploads"
object_key: Mapped[str] = mapped_column(String(255), primary_key=True)
chat_id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), ForeignKey("chats.id", ondelete="CASCADE"), nullable=False)
owner_id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), ForeignKey("users.id"), nullable=False)
size_bytes: Mapped[int] = mapped_column(Integer, nullable=False)
sha256: Mapped[str] = mapped_column(String(64), nullable=False)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now(), nullable=False)
__table_args__ = (Index("ix_pending_uploads_created_at", "created_at"),)
class IdempotencyKey(Base):
__tablename__ = "idempotency_keys"
user_id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), ForeignKey("users.id", ondelete="CASCADE"), primary_key=True)
method: Mapped[str] = mapped_column(String(8), primary_key=True)
path: Mapped[str] = mapped_column(String(255), primary_key=True)
key: Mapped[str] = mapped_column(String(128), primary_key=True)
body_hash: Mapped[str] = mapped_column(String(64), nullable=False)
message_id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), ForeignKey("messages.id", ondelete="CASCADE"), nullable=False)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now(), nullable=False)
__table_args__ = (Index("ix_idem_created", "created_at"),)
View File
+132
View File
@@ -0,0 +1,132 @@
"""Вложения: upload в S3, метаданные в PostgreSQL, download через API."""
import re
import uuid
from fastapi import APIRouter, Depends, Query, Request, Security
from fastapi.responses import Response
from sqlalchemy.orm import Session
from ..deps import current_user, get_db, parse_uuid
from ..errors import ApiError
from ..models import Attachment, ChatMember, User
from ..schemas import AttachmentOut
from .. import s3
router = APIRouter()
_FILENAME_BAD = re.compile(r"[/\\\x00-\x1f\x7f]")
def _validate_filename(raw: str) -> str:
if not isinstance(raw, str) or not (1 <= len(raw) <= 255):
raise ApiError(400, "invalid_request", "Некорректный filename")
if _FILENAME_BAD.search(raw):
raise ApiError(400, "invalid_request", "Некорректный filename")
return raw
def _require_member(db: Session, chat_id: uuid.UUID, user_id: uuid.UUID) -> None:
if db.get(ChatMember, (chat_id, user_id)) is None:
raise ApiError(404, "not_found", "Чат не найден")
def _to_out(att: Attachment) -> AttachmentOut:
return AttachmentOut(
id=att.id,
chat_id=att.chat_id,
owner_id=att.owner_id,
filename=att.filename,
content_type=att.content_type,
size_bytes=att.size_bytes,
sha256=att.sha256,
status=att.status,
variants=[],
error_code=att.error_code,
created_at=att.created_at,
updated_at=att.updated_at,
)
@router.post(
"/api/v1/chats/{chat_id}/attachments",
response_model=AttachmentOut,
status_code=201,
)
async def upload_attachment(
chat_id: str,
request: Request,
filename: str = Query(...),
user: User = Security(current_user),
db: Session = Depends(get_db),
):
cid = parse_uuid(chat_id)
_require_member(db, cid, user.id)
name = _validate_filename(filename)
declared = (request.headers.get("content-type") or "").split(";")[0].strip().lower()
if declared not in ("image/png", "image/jpeg"):
raise ApiError(415, "unsupported_media_type", "Ожидается image/png или image/jpeg")
# Читаем тело потоково с жёстким лимитом (async-итератор Starlette).
data, sha, actual = await s3.read_upload(request.stream(), declared)
att_id = uuid.uuid4()
key = s3.new_object_key(str(cid), str(att_id))
s3.put_object(key, data, actual)
att = Attachment(
id=att_id,
chat_id=cid,
owner_id=user.id,
filename=name,
content_type=actual,
size_bytes=len(data),
sha256=sha,
object_key=key,
status="ready",
error_code=None,
)
db.add(att)
try:
db.commit()
except Exception:
db.rollback()
# Компенсация: удаляем объект, чтобы не оставить orphan в S3.
s3.delete_object(key)
raise
db.refresh(att)
return _to_out(att)
@router.get("/api/v1/attachments/{attachment_id}", response_model=AttachmentOut)
def get_attachment(
attachment_id: str,
user: User = Security(current_user),
db: Session = Depends(get_db),
):
aid = parse_uuid(attachment_id)
att = db.get(Attachment, aid)
if att is None:
raise ApiError(404, "not_found", "Вложение не найдено")
_require_member(db, att.chat_id, user.id)
return _to_out(att)
@router.get("/api/v1/attachments/{attachment_id}/content")
def download_attachment(
attachment_id: str,
variant: str = Query("original"),
user: User = Security(current_user),
db: Session = Depends(get_db),
):
aid = parse_uuid(attachment_id)
att = db.get(Attachment, aid)
if att is None:
raise ApiError(404, "not_found", "Вложение не найдено")
_require_member(db, att.chat_id, user.id)
if variant != "original":
raise ApiError(404, "not_found", "Вариант не найден")
if att.status != "ready":
raise ApiError(409, "conflict", "Вложение не готово")
body, ctype = s3.get_object(att.object_key)
return Response(content=body, media_type=ctype)
+54
View File
@@ -0,0 +1,54 @@
import base64
import binascii
from datetime import datetime, timezone
from fastapi import APIRouter, Depends, Request, Security
from fastapi.security import HTTPBasicCredentials
from sqlalchemy import delete, select
from sqlalchemy.orm import Session
from ..config import settings
from ..deps import basic_scheme, current_user, get_db
from ..errors import ApiError
from ..models import SessionRow, User
from ..schemas import SessionOut, UserOut
from ..security import new_token, session_expiry, token_hash, verify_password
router = APIRouter()
def _parse_basic(header: str):
if not header.lower().startswith("basic "):
raise ApiError(401, "unauthorized", "Требуется Basic")
try:
raw = base64.b64decode(header[6:].strip(), validate=True).decode("utf-8")
except (binascii.Error, UnicodeDecodeError):
raise ApiError(401, "unauthorized", "Некорректный Basic")
if ":" not in raw:
raise ApiError(401, "unauthorized", "Некорректный Basic")
username, password = raw.split(":", 1)
return username, password
@router.post("/api/v1/auth/sessions", response_model=SessionOut, status_code=201)
def login(
request: Request,
_basic: HTTPBasicCredentials | None = Security(basic_scheme),
db: Session = Depends(get_db),
):
username, password = _parse_basic(request.headers.get("authorization", ""))
user = db.scalar(select(User).where(User.username == username))
if user is None or not verify_password(password, user.password_hash):
raise ApiError(401, "unauthorized", "Неверные учётные данные")
token = new_token()
expires = session_expiry(settings.session_ttl_seconds)
db.add(SessionRow(token_hash=token_hash(token), user_id=user.id, expires_at=expires))
db.commit()
return SessionOut(token=token, token_type="Bearer", expires_at=expires, user=UserOut.model_validate(user, from_attributes=True))
@router.delete("/api/v1/auth/sessions/current", status_code=204)
def logout(request: Request, user: User = Security(current_user), db: Session = Depends(get_db)):
db.execute(delete(SessionRow).where(SessionRow.token_hash == request.state.token_hash))
db.commit()
return None
+100
View File
@@ -0,0 +1,100 @@
import base64
import json
import uuid
from datetime import datetime
from fastapi import APIRouter, Depends, Query, Security
from sqlalchemy import select
from sqlalchemy.orm import Session
from ..deps import current_user, get_db, parse_uuid
from ..errors import ApiError
from ..models import Chat, ChatMember, User
from ..schemas import ChatOut, ChatPage, CreateChat
router = APIRouter()
def _chat_out(chat: Chat, member_ids) -> ChatOut:
return ChatOut(id=chat.id, title=chat.title, member_ids=list(member_ids), created_at=chat.created_at)
def _member_ids(db: Session, chat_id) -> list:
return list(db.scalars(select(ChatMember.user_id).where(ChatMember.chat_id == chat_id)))
def _require_member(db: Session, chat_id, user_id) -> Chat:
chat = db.get(Chat, chat_id)
if chat is None:
raise ApiError(404, "not_found", "Чат не найден")
if db.get(ChatMember, (chat_id, user_id)) is None:
raise ApiError(404, "not_found", "Чат не найден")
return chat
def _encode_cursor(created_at: datetime, id_: uuid.UUID) -> str:
payload = json.dumps({"t": created_at.isoformat(), "i": str(id_)}).encode()
return base64.urlsafe_b64encode(payload).decode().rstrip("=")
def _decode_cursor(cursor: str):
try:
pad = "=" * (-len(cursor) % 4)
data = json.loads(base64.urlsafe_b64decode(cursor + pad))
return datetime.fromisoformat(data["t"]), uuid.UUID(data["i"])
except Exception:
raise ApiError(400, "invalid_request", "Некорректный cursor")
@router.post("/api/v1/chats", response_model=ChatOut, status_code=201)
def create_chat(body: CreateChat, user: User = Security(current_user), db: Session = Depends(get_db)):
if not body.title.strip():
raise ApiError(400, "invalid_request", "title не может быть пустым")
ids = list(dict.fromkeys(body.member_ids))
if user.id not in ids:
raise ApiError(400, "invalid_request", "Вызывающий должен быть участником")
found = set(db.scalars(select(User.id).where(User.id.in_(ids))))
if found != set(ids):
raise ApiError(400, "invalid_request", "Неизвестный участник")
chat = Chat(title=body.title)
db.add(chat)
db.flush()
for uid in ids:
db.add(ChatMember(chat_id=chat.id, user_id=uid))
db.commit()
db.refresh(chat)
return _chat_out(chat, ids)
@router.get("/api/v1/chats", response_model=ChatPage)
def list_chats(
limit: int = Query(50, ge=1, le=100),
cursor: str | None = Query(None),
user: User = Security(current_user),
db: Session = Depends(get_db),
):
stmt = (
select(Chat)
.join(ChatMember, ChatMember.chat_id == Chat.id)
.where(ChatMember.user_id == user.id)
.order_by(Chat.created_at.asc(), Chat.id.asc())
.limit(limit + 1)
)
if cursor:
t, i = _decode_cursor(cursor)
stmt = stmt.where((Chat.created_at > t) | ((Chat.created_at == t) & (Chat.id > i)))
rows = list(db.scalars(stmt))
next_cursor = None
if len(rows) > limit:
rows = rows[:limit]
last = rows[-1]
next_cursor = _encode_cursor(last.created_at, last.id)
items = [_chat_out(c, _member_ids(db, c.id)) for c in rows]
return ChatPage(items=items, next_cursor=next_cursor)
@router.get("/api/v1/chats/{chat_id}", response_model=ChatOut)
def get_chat(chat_id: str, user: User = Security(current_user), db: Session = Depends(get_db)):
cid = parse_uuid(chat_id)
chat = _require_member(db, cid, user.id)
return _chat_out(chat, _member_ids(db, cid))
+23
View File
@@ -0,0 +1,23 @@
from fastapi import APIRouter, Depends
from sqlalchemy import text
from sqlalchemy.orm import Session
from ..deps import get_db
from ..errors import ApiError
from ..schemas import Health
router = APIRouter()
@router.get("/health/live", response_model=Health)
def live():
return Health()
@router.get("/health/ready", response_model=Health)
def ready(db: Session = Depends(get_db)):
try:
db.execute(text("SELECT 1"))
except Exception:
raise ApiError(503, "unavailable", "База данных недоступна")
return Health()
+146
View File
@@ -0,0 +1,146 @@
import hashlib
import json
import re
import uuid
from datetime import datetime, timedelta, timezone
from fastapi import APIRouter, Depends, Header, Query, Request, Security
from sqlalchemy import select
from sqlalchemy.exc import IntegrityError
from sqlalchemy.orm import Session
from ..config import settings
from ..deps import current_user, get_db, parse_uuid
from ..errors import ApiError
from ..models import Attachment, Chat, ChatMember, IdempotencyKey, Message, MessageAttachment, User
from ..schemas import CreateMessage, MessageOut, MessagePage
from .chats import _decode_cursor, _encode_cursor, _require_member
router = APIRouter()
_KEY_RE = re.compile(r"^[A-Za-z0-9._:-]{1,128}$")
def _body_hash(body: CreateMessage) -> str:
canonical = json.dumps(
{
"text": body.text,
"attachment_ids": sorted(str(a) for a in body.attachment_ids),
},
sort_keys=True,
ensure_ascii=False,
)
return hashlib.sha256(canonical.encode("utf-8")).hexdigest()
def _attachment_ids_for(db: Session, message_id) -> list:
rows = db.scalars(
select(MessageAttachment.attachment_id)
.where(MessageAttachment.message_id == message_id)
.order_by(MessageAttachment.position.asc())
)
return list(rows)
def _to_out(db: Session, msg: Message) -> MessageOut:
return MessageOut(
id=msg.id,
chat_id=msg.chat_id,
sender_id=msg.sender_id,
text=msg.text,
created_at=msg.created_at,
attachment_ids=_attachment_ids_for(db, msg.id),
)
def _validate_attachments(db: Session, chat_id, ids: list) -> list:
"""Все вложения должны принадлежать этому чату и быть ready."""
if not ids:
return []
unique = list(dict.fromkeys(ids))
if len(unique) != len(ids):
raise ApiError(400, "invalid_request", "attachment_ids содержит дубликаты")
rows = list(db.scalars(select(Attachment).where(Attachment.id.in_(unique))))
by_id = {a.id: a for a in rows}
for aid in unique:
att = by_id.get(aid)
if att is None or att.chat_id != chat_id:
raise ApiError(404, "not_found", "Вложение не найдено")
if att.status == "failed":
raise ApiError(409, "conflict", "Вложение в состоянии failed")
return unique
@router.post("/api/v1/chats/{chat_id}/messages", response_model=MessageOut, status_code=201)
def create_message(
chat_id: str,
body: CreateMessage,
request: Request,
idempotency_key: str | None = Header(default=None, alias="Idempotency-Key"),
user: User = Security(current_user),
db: Session = Depends(get_db),
):
cid = parse_uuid(chat_id)
_require_member(db, cid, user.id)
if not body.text.strip() and not body.attachment_ids:
raise ApiError(400, "invalid_request", "text не может быть пустым без вложений")
if not idempotency_key or not _KEY_RE.match(idempotency_key):
raise ApiError(400, "invalid_request", "Некорректный Idempotency-Key")
path = f"/api/v1/chats/{cid}/messages"
bh = _body_hash(body)
attachment_ids = _validate_attachments(db, cid, list(body.attachment_ids))
existing = db.get(IdempotencyKey, (user.id, "POST", path, idempotency_key))
if existing is not None:
if existing.body_hash != bh:
raise ApiError(409, "conflict", "Ключ уже использован с другим телом")
msg = db.get(Message, existing.message_id)
return _to_out(db, msg)
msg = Message(chat_id=cid, sender_id=user.id, text=body.text)
db.add(msg)
db.flush()
for pos, aid in enumerate(attachment_ids):
db.add(MessageAttachment(message_id=msg.id, attachment_id=aid, position=pos))
db.add(IdempotencyKey(user_id=user.id, method="POST", path=path, key=idempotency_key, body_hash=bh, message_id=msg.id))
try:
db.commit()
except IntegrityError:
db.rollback()
existing = db.get(IdempotencyKey, (user.id, "POST", path, idempotency_key))
if existing is None:
raise ApiError(500, "internal_error", "Не удалось сохранить сообщение")
if existing.body_hash != bh:
raise ApiError(409, "conflict", "Ключ уже использован с другим телом")
msg = db.get(Message, existing.message_id)
return _to_out(db, msg)
db.refresh(msg)
return _to_out(db, msg)
@router.get("/api/v1/chats/{chat_id}/messages", response_model=MessagePage)
def list_messages(
chat_id: str,
limit: int = Query(50, ge=1, le=100),
cursor: str | None = Query(None),
user: User = Security(current_user),
db: Session = Depends(get_db),
):
cid = parse_uuid(chat_id)
_require_member(db, cid, user.id)
stmt = (
select(Message)
.where(Message.chat_id == cid)
.order_by(Message.created_at.asc(), Message.id.asc())
.limit(limit + 1)
)
if cursor:
t, i = _decode_cursor(cursor)
stmt = stmt.where((Message.created_at > t) | ((Message.created_at == t) & (Message.id > i)))
rows = list(db.scalars(stmt))
next_cursor = None
if len(rows) > limit:
rows = rows[:limit]
last = rows[-1]
next_cursor = _encode_cursor(last.created_at, last.id)
return MessagePage(items=[_to_out(db, m) for m in rows], next_cursor=next_cursor)
+32
View File
@@ -0,0 +1,32 @@
from fastapi import APIRouter, Depends, Security
from sqlalchemy import select
from sqlalchemy.exc import IntegrityError
from sqlalchemy.orm import Session
from ..deps import current_user, get_db
from ..errors import ApiError
from ..models import User
from ..schemas import CreateUser, UserOut
from ..security import hash_password
router = APIRouter()
@router.post("/api/v1/users", response_model=UserOut, status_code=201)
def register(body: CreateUser, db: Session = Depends(get_db)):
if not body.display_name.strip():
raise ApiError(400, "invalid_request", "display_name не может быть пустым")
user = User(username=body.username, display_name=body.display_name, password_hash=hash_password(body.password))
db.add(user)
try:
db.commit()
except IntegrityError:
db.rollback()
raise ApiError(409, "conflict", "username занят")
db.refresh(user)
return user
@router.get("/api/v1/users/me", response_model=UserOut)
def me(user: User = Security(current_user)):
return user
+166
View File
@@ -0,0 +1,166 @@
"""S3-клиент (MinIO) и потоковая валидация изображений.
Потоковая загрузка: читаем тело запроса чанками, считаем SHA-256 и размер,
не держим весь файл в памяти. MIME определяем по magic bytes, пиксели —
через Pillow с защитой от decompression bomb.
"""
import hashlib
import io
import logging
import uuid
from typing import BinaryIO
import boto3
from botocore.client import Config as BotoConfig
from botocore.exceptions import BotoCoreError, ClientError
from PIL import Image, UnidentifiedImageError
from .config import settings
from .errors import ApiError
logger = logging.getLogger("messenger.s3")
PNG_MAGIC = b"\x89PNG\r\n\x1a\n"
JPEG_MAGIC = b"\xff\xd8\xff"
# Жёсткий лимит Pillow: любая попытка декодировать больше пикселей
# бросает DecompressionBombError до выделения памяти под растр.
Image.MAX_IMAGE_PIXELS = settings.max_image_pixels
# Сколько байт достаточно, чтобы прочитать заголовок PNG/JPEG и размеры.
_HEADER_BYTES = 64 * 1024
def _client():
return boto3.client(
"s3",
endpoint_url=settings.s3_endpoint,
region_name=settings.s3_region,
aws_access_key_id=settings.s3_access_key_id,
aws_secret_access_key=settings.s3_secret_access_key,
use_ssl=settings.s3_use_ssl,
config=BotoConfig(signature_version="s3v4", s3={"addressing_style": "path"}),
)
def ensure_bucket() -> None:
"""Идемпотентно создаёт bucket, если его нет."""
client = _client()
try:
client.head_bucket(Bucket=settings.s3_bucket)
except ClientError as exc:
code = exc.response.get("Error", {}).get("Code", "")
if code in ("404", "NoSuchBucket", "NotFound"):
client.create_bucket(Bucket=settings.s3_bucket)
logger.info("created bucket %s", settings.s3_bucket)
else:
raise
def new_object_key(chat_id: str, attachment_id: str) -> str:
"""Случайный ключ; filename не используется как путь."""
return f"chats/{chat_id}/{attachment_id}/{uuid.uuid4().hex}"
def _detect_mime(head: bytes) -> str | None:
if head.startswith(PNG_MAGIC):
return "image/png"
if head.startswith(JPEG_MAGIC):
return "image/jpeg"
return None
def _validate_header(header: bytes, declared_mime: str) -> str:
"""Проверяет magic bytes и размеры по заголовку.
Полное декодирование не выполняется: Pillow читает только заголовок
(IHDR для PNG, SOF для JPEG), что не выделяет память под растр.
Возвращает фактический MIME. Бросает ApiError(415/413).
"""
actual = _detect_mime(header[:16])
if actual is None:
raise ApiError(415, "unsupported_media_type", "Поддерживаются только PNG и JPEG")
if actual != declared_mime:
raise ApiError(415, "unsupported_media_type", "Content-Type не совпадает с содержимым")
try:
with Image.open(io.BytesIO(header)) as img:
width, height = img.size
except Image.DecompressionBombError:
raise ApiError(413, "payload_too_large", "Слишком много пикселей")
except (UnidentifiedImageError, OSError):
raise ApiError(415, "unsupported_media_type", "Повреждённое изображение")
if width <= 0 or height <= 0:
raise ApiError(415, "unsupported_media_type", "Повреждённое изображение")
if width * height > settings.max_image_pixels:
raise ApiError(413, "payload_too_large", "Слишком много пикселей")
return actual
async def read_upload(stream, declared_mime: str) -> tuple[bytes, str, str]:
"""Читает async-поток Starlette с лимитом.
Возвращает (bytes, sha256, actual_mime). Файл ≤10 MiB, поэтому держим
его в памяти — это упрощает валидацию и не создаёт неограниченного
потребления RAM (лимит жёсткий). Валидация заголовка выполняется по
первым 64 KiB до полного чтения тела.
"""
limit = settings.max_upload_bytes
buf = bytearray()
hasher = hashlib.sha256()
async for chunk in stream:
if not chunk:
continue
buf.extend(chunk)
hasher.update(chunk)
# Лимит размера приоритетнее валидации содержимого: контракт
# требует 413 при превышении bytes, даже если заголовок битый.
if len(buf) > limit:
raise ApiError(413, "payload_too_large", "Файл больше 10 MiB")
if not buf:
raise ApiError(400, "invalid_request", "Пустое тело запроса")
# Валидация заголовка — только после того, как убедились, что лимит
# не превышен. Иначе битый заголовок в oversized-файле дал бы 415
# вместо 413, что нарушает приоритет кодов контракта.
actual = _validate_header(bytes(buf[:_HEADER_BYTES]), declared_mime)
return bytes(buf), hasher.hexdigest(), actual
def put_object(key: str, data: bytes, content_type: str) -> None:
client = _client()
try:
client.put_object(
Bucket=settings.s3_bucket,
Key=key,
Body=data,
ContentType=content_type,
)
except (BotoCoreError, ClientError) as exc:
logger.error("s3 put failed: %s", type(exc).__name__)
raise ApiError(503, "unavailable", "Хранилище недоступно")
def get_object(key: str) -> tuple[bytes, str]:
client = _client()
try:
resp = client.get_object(Bucket=settings.s3_bucket, Key=key)
body = resp["Body"].read()
ctype = resp.get("ContentType") or "application/octet-stream"
return body, ctype
except ClientError as exc:
code = exc.response.get("Error", {}).get("Code", "")
if code in ("NoSuchKey", "404", "NotFound"):
raise ApiError(404, "not_found", "Объект не найден")
logger.error("s3 get failed: %s", type(exc).__name__)
raise ApiError(503, "unavailable", "Хранилище недоступно")
except BotoCoreError as exc:
logger.error("s3 get failed: %s", type(exc).__name__)
raise ApiError(503, "unavailable", "Хранилище недоступно")
def delete_object(key: str) -> None:
"""Best-effort удаление для компенсации при сбое между S3 и БД."""
client = _client()
try:
client.delete_object(Bucket=settings.s3_bucket, Key=key)
except (BotoCoreError, ClientError) as exc:
logger.warning("s3 delete failed: %s", type(exc).__name__)
+101
View File
@@ -0,0 +1,101 @@
import uuid
from datetime import datetime
from typing import List, Optional
from pydantic import BaseModel, ConfigDict, Field
class UserOut(BaseModel):
model_config = ConfigDict(extra="forbid")
id: uuid.UUID
username: str
display_name: str
created_at: datetime
class CreateUser(BaseModel):
model_config = ConfigDict(extra="forbid")
username: str = Field(pattern=r"^[a-z0-9_]{3,32}$")
display_name: str = Field(min_length=1, max_length=100)
password: str = Field(min_length=12, max_length=128)
class SessionOut(BaseModel):
model_config = ConfigDict(extra="forbid")
token: str
token_type: str = "Bearer"
expires_at: datetime
user: UserOut
class ChatOut(BaseModel):
model_config = ConfigDict(extra="forbid")
id: uuid.UUID
title: str
member_ids: List[uuid.UUID]
created_at: datetime
class CreateChat(BaseModel):
model_config = ConfigDict(extra="forbid")
title: str = Field(min_length=1, max_length=100)
member_ids: List[uuid.UUID] = Field(min_length=1, max_length=100)
class MessageOut(BaseModel):
model_config = ConfigDict(extra="forbid")
id: uuid.UUID
chat_id: uuid.UUID
sender_id: uuid.UUID
text: str
created_at: datetime
attachment_ids: List[uuid.UUID] = Field(default_factory=list)
class CreateMessage(BaseModel):
model_config = ConfigDict(extra="forbid")
text: str = Field(default="", max_length=4000)
attachment_ids: List[uuid.UUID] = Field(default_factory=list, max_length=10)
class VariantOut(BaseModel):
model_config = ConfigDict(extra="forbid")
name: str
content_type: str
size_bytes: int
width: int
height: int
sha256: str
class AttachmentOut(BaseModel):
model_config = ConfigDict(extra="forbid")
id: uuid.UUID
chat_id: uuid.UUID
owner_id: uuid.UUID
filename: str
content_type: str
size_bytes: int
sha256: str
status: str
variants: List[VariantOut] = Field(default_factory=list)
error_code: Optional[str] = None
created_at: datetime
updated_at: datetime
class ChatPage(BaseModel):
model_config = ConfigDict(extra="forbid")
items: List[ChatOut]
next_cursor: Optional[str]
class MessagePage(BaseModel):
model_config = ConfigDict(extra="forbid")
items: List[MessageOut]
next_cursor: Optional[str]
class Health(BaseModel):
model_config = ConfigDict(extra="forbid")
status: str = "ok"
+33
View File
@@ -0,0 +1,33 @@
import hashlib
import secrets
from datetime import datetime, timedelta, timezone
from argon2 import PasswordHasher
from argon2.exceptions import VerifyMismatchError
_ph = PasswordHasher()
def hash_password(password: str) -> str:
return _ph.hash(password)
def verify_password(password: str, hashed: str) -> bool:
try:
return _ph.verify(hashed, password)
except VerifyMismatchError:
return False
except Exception:
return False
def new_token() -> str:
return secrets.token_urlsafe(48)
def token_hash(token: str) -> str:
return hashlib.sha256(token.encode("utf-8")).hexdigest()
def session_expiry(ttl_seconds: int) -> datetime:
return datetime.now(timezone.utc) + timedelta(seconds=ttl_seconds)
+129
View File
@@ -0,0 +1,129 @@
"""Сессии: PostgreSQL — источник истины, Redis — кэш с TTL.
При недоступности Redis работаем напрямую с БД (не принимаем анонимов).
При logout инвалидируем и БД, и кэш.
"""
import logging
import os
from datetime import datetime, timedelta, timezone
from typing import Any
import redis
from psycopg_pool import ConnectionPool
from .security import new_token, token_hash
logger = logging.getLogger("messenger.sessions")
def _now() -> datetime:
return datetime.now(timezone.utc)
def _ttl_seconds() -> int:
return int(os.environ.get("SESSION_TTL_SECONDS", "3600"))
class SessionStore:
def __init__(self, pool: ConnectionPool, cache: redis.Redis | None) -> None:
self.pool = pool
self.cache = cache
# --- кэш --------------------------------------------------------------
def _cache_get(self, th: str) -> dict[str, Any] | None:
if self.cache is None:
return None
try:
raw = self.cache.hgetall("session:" + th)
except redis.RedisError as exc:
logger.warning("redis get failed: %s", type(exc).__name__)
return None
if not raw:
return None
return {
"user_id": raw[b"user_id"].decode(),
"expires_at": raw[b"expires_at"].decode(),
}
def _cache_put(self, th: str, user_id: str, expires_at: datetime) -> None:
if self.cache is None:
return
ttl = max(1, int((expires_at - _now()).total_seconds()))
try:
pipe = self.cache.pipeline()
pipe.hset(
"session:" + th,
mapping={
"user_id": user_id,
"expires_at": expires_at.strftime("%Y-%m-%dT%H:%M:%SZ"),
},
)
pipe.expire("session:" + th, ttl)
pipe.execute()
except redis.RedisError as exc:
logger.warning("redis put failed: %s", type(exc).__name__)
def _cache_del(self, th: str) -> None:
if self.cache is None:
return
try:
self.cache.delete("session:" + th)
except redis.RedisError as exc:
logger.warning("redis del failed: %s", type(exc).__name__)
# --- публичный API ----------------------------------------------------
def create(self, user_id: str) -> dict[str, Any]:
token = new_token()
th = token_hash(token)
expires_at = _now() + timedelta(seconds=_ttl_seconds())
with self.pool.connection() as conn:
with conn.cursor() as cur:
cur.execute(
"INSERT INTO sessions(token_hash, user_id, created_at, expires_at)"
" VALUES (%s, %s, now(), %s)",
(th, user_id, expires_at),
)
conn.commit()
self._cache_put(th, user_id, expires_at)
return {
"token": token,
"expires_at": expires_at.strftime("%Y-%m-%dT%H:%M:%SZ"),
}
def resolve(self, token: str) -> str | None:
"""Возвращает user_id для действующей сессии или None."""
th = token_hash(token)
cached = self._cache_get(th)
if cached is not None:
try:
exp = datetime.strptime(cached["expires_at"], "%Y-%m-%dT%H:%M:%SZ").replace(
tzinfo=timezone.utc
)
except ValueError:
exp = None
if exp is not None and exp > _now():
return cached["user_id"]
self._cache_del(th)
with self.pool.connection() as conn:
with conn.cursor() as cur:
cur.execute(
"SELECT user_id, expires_at FROM sessions WHERE token_hash = %s",
(th,),
)
row = cur.fetchone()
if row is None:
return None
user_id, expires_at = row
if expires_at <= _now():
self.revoke(token)
return None
self._cache_put(th, str(user_id), expires_at)
return str(user_id)
def revoke(self, token: str) -> None:
th = token_hash(token)
with self.pool.connection() as conn:
with conn.cursor() as cur:
cur.execute("DELETE FROM sessions WHERE token_hash = %s", (th,))
conn.commit()
self._cache_del(th)
+204
View File
@@ -0,0 +1,204 @@
"""Хранилище на PostgreSQL: пользователи, чаты, участники, сообщения."""
import uuid
from datetime import datetime, timezone
from typing import Any
import psycopg
from psycopg_pool import ConnectionPool
def utc_now() -> str:
"""RFC 3339 UTC с суффиксом Z, секунды — как в примерах контракта."""
return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
def new_id() -> str:
return str(uuid.uuid4())
def _iso(dt: datetime) -> str:
"""RFC 3339 UTC с суффиксом Z, микросекунды сохраняются для keyset-пагинации."""
return dt.astimezone(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S.%fZ")
def _user_row(row: tuple) -> dict[str, Any]:
uid, username, display_name, created_at = row
return {
"id": str(uid),
"username": username,
"display_name": display_name,
"created_at": _iso(created_at),
}
def _chat_row(row: tuple, member_ids: list[str]) -> dict[str, Any]:
cid, title, created_at = row
return {
"id": str(cid),
"title": title,
"member_ids": member_ids,
"created_at": _iso(created_at),
}
def _message_row(row: tuple) -> dict[str, Any]:
mid, chat_id, sender_id, text, created_at = row
return {
"id": str(mid),
"chat_id": str(chat_id),
"sender_id": str(sender_id),
"text": text,
"created_at": _iso(created_at),
}
class Store:
"""Все данные в PostgreSQL; соединения берутся из пула."""
def __init__(self, pool: ConnectionPool) -> None:
self.pool = pool
# --- пользователи -----------------------------------------------------
def create_user(
self, username: str, display_name: str, password_hash: str
) -> dict[str, Any] | None:
user_id = new_id()
try:
with self.pool.connection() as conn:
with conn.cursor() as cur:
cur.execute(
"INSERT INTO users(id, username, display_name, password_hash, created_at)"
" VALUES (%s, %s, %s, %s, now())"
" RETURNING id, username, display_name, created_at",
(user_id, username, display_name, password_hash),
)
row = cur.fetchone()
conn.commit()
except psycopg.errors.UniqueViolation:
return None
return _user_row(row)
def get_user(self, user_id: str) -> dict[str, Any] | None:
with self.pool.connection() as conn:
with conn.cursor() as cur:
cur.execute(
"SELECT id, username, display_name, created_at FROM users WHERE id = %s",
(user_id,),
)
row = cur.fetchone()
return _user_row(row) if row else None
def get_user_by_username(self, username: str) -> tuple[dict[str, Any], str] | None:
with self.pool.connection() as conn:
with conn.cursor() as cur:
cur.execute(
"SELECT id, username, display_name, created_at, password_hash"
" FROM users WHERE username = %s",
(username,),
)
row = cur.fetchone()
if row is None:
return None
return _user_row(row[:4]), row[4]
# --- чаты -------------------------------------------------------------
def create_chat(self, title: str, member_ids: list[str]) -> dict[str, Any]:
chat_id = new_id()
with self.pool.connection() as conn:
with conn.transaction():
with conn.cursor() as cur:
cur.execute(
"INSERT INTO chats(id, title, created_at) VALUES (%s, %s, now())"
" RETURNING id, title, created_at",
(chat_id, title),
)
row = cur.fetchone()
cur.executemany(
"INSERT INTO chat_members(chat_id, user_id) VALUES (%s, %s)",
[(chat_id, mid) for mid in member_ids],
)
return _chat_row(row, list(member_ids))
def _members(self, cur, chat_id: str) -> list[str]:
cur.execute(
"SELECT user_id FROM chat_members WHERE chat_id = %s ORDER BY user_id",
(chat_id,),
)
return [str(r[0]) for r in cur.fetchall()]
def get_chat(self, chat_id: str) -> dict[str, Any] | None:
with self.pool.connection() as conn:
with conn.cursor() as cur:
cur.execute(
"SELECT id, title, created_at FROM chats WHERE id = %s", (chat_id,)
)
row = cur.fetchone()
if row is None:
return None
members = self._members(cur, chat_id)
return _chat_row(row, members)
def is_member(self, chat_id: str, user_id: str) -> bool:
with self.pool.connection() as conn:
with conn.cursor() as cur:
cur.execute(
"SELECT 1 FROM chat_members WHERE chat_id = %s AND user_id = %s",
(chat_id, user_id),
)
return cur.fetchone() is not None
def list_chats_for(
self, user_id: str, limit: int, after: tuple[str, str] | None
) -> tuple[list[dict[str, Any]], bool]:
with self.pool.connection() as conn:
with conn.cursor() as cur:
sql = (
"SELECT c.id, c.title, c.created_at FROM chats c"
" JOIN chat_members m ON m.chat_id = c.id"
" WHERE m.user_id = %s"
)
params: list[Any] = [user_id]
if after is not None:
sql += " AND (c.created_at, c.id) > (%s, %s)"
params.extend(after)
sql += " ORDER BY c.created_at, c.id LIMIT %s"
params.append(limit + 1)
cur.execute(sql, params)
rows = cur.fetchall()
items = []
for row in rows[:limit]:
items.append(_chat_row(row, self._members(cur, str(row[0]))))
return items, len(rows) > limit
# --- сообщения --------------------------------------------------------
def add_message(self, chat_id: str, sender_id: str, text: str) -> dict[str, Any]:
with self.pool.connection() as conn:
with conn.cursor() as cur:
cur.execute(
"INSERT INTO messages(id, chat_id, sender_id, text, created_at)"
" VALUES (%s, %s, %s, %s, now())"
" RETURNING id, chat_id, sender_id, text, created_at",
(new_id(), chat_id, sender_id, text),
)
row = cur.fetchone()
conn.commit()
return _message_row(row)
def list_messages(
self, chat_id: str, limit: int, after: tuple[str, str] | None
) -> tuple[list[dict[str, Any]], bool]:
with self.pool.connection() as conn:
with conn.cursor() as cur:
sql = (
"SELECT id, chat_id, sender_id, text, created_at FROM messages"
" WHERE chat_id = %s"
)
params: list[Any] = [chat_id]
if after is not None:
sql += " AND (created_at, id) > (%s, %s)"
params.extend(after)
sql += " ORDER BY created_at, id LIMIT %s"
params.append(limit + 1)
cur.execute(sql, params)
rows = cur.fetchall()
return [_message_row(r) for r in rows[:limit]], len(rows) > limit
+122
View File
@@ -0,0 +1,122 @@
"""Валидация тел запросов и параметров вручную, без pydantic-моделей контракта."""
import base64
import re
import uuid
from typing import Any
from .errors import ApiError
USERNAME_RE = re.compile(r"\A[a-z0-9_]{3,32}\Z")
def _is_blank(value: str) -> bool:
return value.strip() == ""
def require_object(body: Any) -> dict[str, Any]:
if not isinstance(body, dict):
raise ApiError(400, "invalid_request", "Тело запроса должно быть JSON-объектом")
return body
def reject_extra(body: dict[str, Any], allowed: set[str]) -> None:
if set(body) - allowed:
raise ApiError(400, "invalid_request", "Неизвестные поля в теле запроса")
def validate_create_user(body: Any) -> tuple[str, str, str]:
body = require_object(body)
reject_extra(body, {"username", "display_name", "password"})
username = body.get("username")
display_name = body.get("display_name")
password = body.get("password")
if not isinstance(username, str) or not USERNAME_RE.match(username):
raise ApiError(400, "invalid_request", "Некорректный username")
if not isinstance(display_name, str) or not (1 <= len(display_name) <= 100) or _is_blank(display_name):
raise ApiError(400, "invalid_request", "Некорректный display_name")
if not isinstance(password, str) or not (12 <= len(password) <= 128):
raise ApiError(400, "invalid_request", "Пароль должен быть 12..128 символов")
return username, display_name, password
def validate_create_chat(body: Any, caller_id: str) -> tuple[str, list[str]]:
body = require_object(body)
reject_extra(body, {"title", "member_ids"})
title = body.get("title")
member_ids = body.get("member_ids")
if not isinstance(title, str) or not (1 <= len(title) <= 100) or _is_blank(title):
raise ApiError(400, "invalid_request", "Некорректный title")
if not isinstance(member_ids, list) or not (1 <= len(member_ids) <= 100):
raise ApiError(400, "invalid_request", "member_ids должен быть непустым массивом")
seen: set[str] = set()
for mid in member_ids:
if not isinstance(mid, str):
raise ApiError(400, "invalid_request", "member_ids содержит не строку")
try:
uuid.UUID(mid)
except ValueError:
raise ApiError(400, "invalid_request", "member_ids содержит не UUID")
if mid in seen:
raise ApiError(400, "invalid_request", "member_ids содержит дубликаты")
seen.add(mid)
if caller_id not in seen:
raise ApiError(400, "invalid_request", "Вызывающий должен входить в member_ids")
return title, list(member_ids)
def validate_create_message(body: Any) -> str:
body = require_object(body)
reject_extra(body, {"text"})
text = body.get("text")
if not isinstance(text, str) or not (1 <= len(text) <= 4000) or _is_blank(text):
raise ApiError(400, "invalid_request", "Некорректный text")
return text
def validate_chat_id(raw: str) -> str:
"""chat_id в пути должен быть UUID, иначе 400 invalid_request."""
try:
uuid.UUID(raw)
except (ValueError, TypeError):
raise ApiError(400, "invalid_request", "Некорректный chat_id")
return raw
def parse_limit(raw: str | None, default: int = 50) -> int:
if raw is None:
return default
try:
value = int(raw)
except (TypeError, ValueError):
raise ApiError(400, "invalid_request", "limit должен быть целым числом")
if not (1 <= value <= 100):
raise ApiError(400, "invalid_request", "limit вне диапазона 1..100")
return value
def parse_cursor(raw: str | None, scope: str) -> tuple[str, str] | None:
"""Cursor — непрозрачная строка base64url от 'scope|created_at|id'.
Cursor привязан к ресурсу (scope). Значения created_at/id — из БД,
поэтому cursor переживает рестарт процесса.
"""
if raw is None:
return None
try:
decoded = base64.urlsafe_b64decode(raw.encode("ascii")).decode("utf-8")
except Exception:
raise ApiError(400, "invalid_request", "Некорректный cursor")
parts = decoded.split("|")
if len(parts) != 3:
raise ApiError(400, "invalid_request", "Некорректный cursor")
cursor_scope, created_at, item_id = parts
if cursor_scope != scope:
raise ApiError(400, "invalid_request", "Некорректный cursor")
if not created_at or not item_id:
raise ApiError(400, "invalid_request", "Некорректный cursor")
return created_at, item_id
def encode_cursor(created_at: str, item_id: str, scope: str) -> str:
raw = "|".join([scope, created_at, item_id]).encode("utf-8")
return base64.urlsafe_b64encode(raw).decode("ascii")
+136
View File
@@ -0,0 +1,136 @@
# Локальный стенд lab4: LB (nginx) → API × 2 → PostgreSQL + Redis + MinIO.
# Секреты читаются из ./secrets/*.txt (не коммитятся).
services:
postgres:
image: postgres:16.4-alpine
environment:
POSTGRES_USER: messenger
POSTGRES_PASSWORD: messenger
POSTGRES_DB: messenger
volumes:
- pgdata:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U messenger -d messenger"]
interval: 3s
timeout: 3s
retries: 20
redis:
image: redis:7.4-alpine
command: ["redis-server", "--save", "", "--appendonly", "no"]
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 3s
timeout: 3s
retries: 20
s3:
image: chrislusf/seaweedfs:3.71
# Один контейнер: master + volume + filer + S3-шлюз на :8333.
# -ip.bind=0.0.0.0 — чтобы S3-шлюз слушал все интерфейсы внутри сети compose.
command: ["server", "-dir=/data", "-s3", "-s3.port=8333", "-ip.bind=0.0.0.0"]
environment:
# Учётные данные S3-шлюза SeaweedFS (см. документацию weed s3).
AWS_ACCESS_KEY_ID: ${S3_ACCESS_KEY_ID:-seaweedfs}
AWS_SECRET_ACCESS_KEY: ${S3_SECRET_ACCESS_KEY:-seaweedfs-local-only}
volumes:
- s3data:/data
healthcheck:
test: ["CMD-SHELL", "wget -q -O- http://127.0.0.1:8333/ >/dev/null 2>&1 || exit 1"]
interval: 3s
timeout: 3s
retries: 30
s3-init:
image: python:3.12-slim
depends_on:
s3:
condition: service_healthy
environment:
S3_ENDPOINT: http://s3:8333
S3_REGION: us-east-1
S3_BUCKET: ${S3_BUCKET:-messenger-attachments}
S3_ACCESS_KEY_ID: ${S3_ACCESS_KEY_ID:-seaweedfs}
S3_SECRET_ACCESS_KEY: ${S3_SECRET_ACCESS_KEY:-seaweedfs-local-only}
volumes:
- ./scripts/bootstrap_s3.py:/bootstrap_s3.py:ro
command: ["sh", "-c", "pip install --quiet boto3==1.35.76 && python /bootstrap_s3.py"]
restart: "no"
migrate:
build: .
environment:
DATABASE_URL: postgresql+psycopg://messenger:messenger@postgres:5432/messenger
depends_on:
postgres:
condition: service_healthy
command: ["alembic", "upgrade", "head"]
restart: "no"
api-1:
build: .
environment:
DATABASE_URL: postgresql+psycopg://messenger:messenger@postgres:5432/messenger
SESSION_TTL_SECONDS: ${SESSION_TTL_SECONDS:-3600}
IDEMPOTENCY_TTL_SECONDS: ${IDEMPOTENCY_TTL_SECONDS:-86400}
INSTANCE_ID: api-1
S3_ENDPOINT: http://s3:8333
S3_REGION: us-east-1
S3_BUCKET: ${S3_BUCKET:-messenger-attachments}
S3_ACCESS_KEY_ID: ${S3_ACCESS_KEY_ID:-seaweedfs}
S3_SECRET_ACCESS_KEY: ${S3_SECRET_ACCESS_KEY:-seaweedfs-local-only}
MAX_UPLOAD_BYTES: "10485760"
MAX_IMAGE_PIXELS: "40000000"
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
s3:
condition: service_healthy
s3-init:
condition: service_completed_successfully
migrate:
condition: service_completed_successfully
command: ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080"]
api-2:
build: .
environment:
DATABASE_URL: postgresql+psycopg://messenger:messenger@postgres:5432/messenger
SESSION_TTL_SECONDS: ${SESSION_TTL_SECONDS:-3600}
IDEMPOTENCY_TTL_SECONDS: ${IDEMPOTENCY_TTL_SECONDS:-86400}
INSTANCE_ID: api-2
S3_ENDPOINT: http://s3:8333
S3_REGION: us-east-1
S3_BUCKET: ${S3_BUCKET:-messenger-attachments}
S3_ACCESS_KEY_ID: ${S3_ACCESS_KEY_ID:-seaweedfs}
S3_SECRET_ACCESS_KEY: ${S3_SECRET_ACCESS_KEY:-seaweedfs-local-only}
MAX_UPLOAD_BYTES: "10485760"
MAX_IMAGE_PIXELS: "40000000"
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
s3:
condition: service_healthy
s3-init:
condition: service_completed_successfully
migrate:
condition: service_completed_successfully
command: ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080"]
lb:
image: nginx:1.27-alpine
ports:
- "8080:8080"
volumes:
- ./infra/nginx/nginx.conf:/etc/nginx/nginx.conf:ro
depends_on:
- api-1
- api-2
volumes:
pgdata:
s3data:
+33
View File
@@ -0,0 +1,33 @@
worker_processes auto;
events { worker_connections 1024; }
http {
log_format main '$remote_addr "$request" $status $upstream_addr $request_time';
access_log /var/log/nginx/access.log main;
error_log /var/log/nginx/error.log warn;
upstream api_backend {
server api-1:8080 max_fails=2 fail_timeout=5s;
server api-2:8080 max_fails=2 fail_timeout=5s;
keepalive 32;
}
server {
listen 8080;
client_max_body_size 16m;
location / {
proxy_pass http://api_backend;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 2s;
proxy_send_timeout 30s;
proxy_read_timeout 30s;
proxy_next_upstream error timeout http_502 http_503 http_504;
proxy_next_upstream_tries 2;
}
}
}
+32
View File
@@ -0,0 +1,32 @@
CREATE TABLE users (
id UUID PRIMARY KEY,
username TEXT NOT NULL UNIQUE,
display_name TEXT NOT NULL,
password_hash TEXT NOT NULL,
created_at TIMESTAMPTZ NOT NULL
);
CREATE TABLE chats (
id UUID PRIMARY KEY,
title TEXT NOT NULL,
created_at TIMESTAMPTZ NOT NULL
);
CREATE TABLE chat_members (
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
PRIMARY KEY (chat_id, user_id)
);
CREATE INDEX chat_members_user_idx ON chat_members(user_id);
CREATE TABLE messages (
id UUID PRIMARY KEY,
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
sender_id UUID NOT NULL REFERENCES users(id),
text TEXT NOT NULL,
created_at TIMESTAMPTZ NOT NULL
);
-- Индекс под keyset-пагинацию истории чата: (chat_id, created_at, id).
CREATE INDEX messages_chat_created_idx ON messages(chat_id, created_at, id);
+9
View File
@@ -0,0 +1,9 @@
CREATE TABLE sessions (
token_hash TEXT PRIMARY KEY,
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
created_at TIMESTAMPTZ NOT NULL,
expires_at TIMESTAMPTZ NOT NULL
);
CREATE INDEX sessions_user_idx ON sessions(user_id);
CREATE INDEX sessions_expires_idx ON sessions(expires_at);
+11
View File
@@ -0,0 +1,11 @@
fastapi==0.115.5
uvicorn[standard]==0.32.1
sqlalchemy==2.0.36
psycopg[binary,pool]==3.2.3
psycopg-pool==3.2.3
alembic==1.14.0
argon2-cffi==23.1.0
redis==5.2.0
boto3==1.35.76
Pillow==11.0.0
python-multipart==0.0.20
+61
View File
@@ -0,0 +1,61 @@
#!/usr/bin/env python3
"""Идемпотентный bootstrap приватного bucket в S3-совместимом хранилище.
Запускается как one-shot сервис в compose (s3-init) после старта SeaweedFS.
Создаёт bucket, если его нет. Политика приватности — по умолчанию (никаких
публичных ACL), поэтому отдельный set-bucket-policy не нужен.
"""
import logging
import os
import sys
import time
import boto3
from botocore.client import Config as BotoConfig
from botocore.exceptions import BotoCoreError, ClientError
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("bootstrap_s3")
def _client():
return boto3.client(
"s3",
endpoint_url=os.environ.get("S3_ENDPOINT", "http://s3:8333"),
region_name=os.environ.get("S3_REGION", "us-east-1"),
aws_access_key_id=os.environ["S3_ACCESS_KEY_ID"],
aws_secret_access_key=os.environ["S3_SECRET_ACCESS_KEY"],
config=BotoConfig(signature_version="s3v4", s3={"addressing_style": "path"}),
)
def main() -> int:
bucket = os.environ.get("S3_BUCKET", "messenger-attachments")
client = _client()
deadline = time.monotonic() + 60
while True:
try:
client.head_bucket(Bucket=bucket)
log.info("bucket %s already exists", bucket)
return 0
except ClientError as exc:
code = exc.response.get("Error", {}).get("Code", "")
if code in ("404", "NoSuchBucket", "NotFound"):
try:
client.create_bucket(Bucket=bucket)
log.info("created bucket %s", bucket)
return 0
except (BotoCoreError, ClientError) as create_exc:
log.warning("create_bucket failed: %s", type(create_exc).__name__)
else:
log.warning("head_bucket failed: %s", code or type(exc).__name__)
except BotoCoreError as exc:
log.warning("s3 not ready: %s", type(exc).__name__)
if time.monotonic() > deadline:
log.error("bootstrap timeout")
return 1
time.sleep(1)
if __name__ == "__main__":
sys.exit(main())
+4 -4
View File
@@ -8,8 +8,8 @@ import re
ROOT = Path(__file__).resolve().parents[1]
def main():
lab = json.loads((ROOT/'lab.json').read_text())['number']
spec = json.loads((ROOT/'contracts/openapi.json').read_text())
lab = json.loads((ROOT/'lab.json').read_text(encoding='utf-8'))['number']
spec = json.loads((ROOT/'contracts/openapi.json').read_text(encoding='utf-8'))
assert spec['openapi']=='3.1.0'
assert spec['info']['version']=='1.%d.0'%lab
operation_ids = set()
@@ -41,11 +41,11 @@ def main():
assert ('/api/v1/auth/sessions' in spec['paths']) == (lab>=2)
assert ('/api/v1/attachments/{attachment_id}' in spec['paths']) == (lab>=4)
for file in list((ROOT/'tests').glob('*.py'))+list((ROOT/'scripts').glob('*.py')):
ast.parse(file.read_text(),filename=str(file))
ast.parse(file.read_text(encoding='utf-8'),filename=str(file))
for file in ROOT.rglob('*.json'):
if any(part in {'.git','.venv','node_modules','vendor','output'} for part in file.parts):
continue
json.loads(file.read_text())
json.loads(file.read_text(encoding='utf-8'))
for required in ('README.md','COURSE.md','CONTRIBUTING.md','REPORT.md','contracts/README.md','tests/smoke.py','tests/README.md'):
assert (ROOT/required).is_file(), 'missing '+required
print('PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.')
+74
View File
@@ -0,0 +1,74 @@
#!/bin/sh
# Collect evidence for lab3 report. Redacts secrets before writing.
# Usage: sh scripts/collect-evidence.sh
set -u
cd "$(dirname "$0")/.."
mkdir -p evidence
redact() {
# Strip Authorization headers, tokens, passwords from stdin.
sed -E \
-e 's/(Authorization:[[:space:]]*)[^[:space:]]+/\1<redacted>/Ig' \
-e 's/("token"[[:space:]]*:[[:space:]]*")[^"]+/\1<redacted>/g' \
-e 's/("password"[[:space:]]*:[[:space:]]*")[^"]+/\1<redacted>/g' \
-e 's/(POSTGRES_PASSWORD=)[^[:space:]]+/\1<redacted>/g'
}
run_step() {
name="$1"; shift
out="evidence/$name"
echo "# $name — $(date -u +%Y-%m-%dT%H:%M:%SZ)" > "$out"
echo "# cmd: $*" >> "$out"
if "$@" 2>&1 | redact >> "$out"; then
echo "# status: OK" >> "$out"
else
echo "# status: FAILED" >> "$out"
fi
}
BASE_URL="${BASE_URL:-http://localhost:8080}"
run_step smoke.txt make test BASE_URL="$BASE_URL"
run_step persistence.txt make test-persistence ACTION_SCRIPT=scripts/restart.sh
run_step failover.txt make test-failover ACTION_SCRIPT=scripts/stop-one.sh
run_step dependency.txt python3 tests/test_dependency_failure.py --base-url "$BASE_URL"
# Instances: 6 requests, extract X-Instance-Id.
{
echo "# instances — $(date -u +%Y-%m-%dT%H:%M:%SZ)"
echo "# cmd: 6x curl -sI $BASE_URL/health/live"
i=0
while [ $i -lt 6 ]; do
curl -sI "$BASE_URL/health/live" | grep -i '^x-instance-id:' || echo "x-instance-id: <missing>"
i=$((i+1))
done
} > evidence/instances.txt
# Health with headers.
{
echo "# health — $(date -u +%Y-%m-%dT%H:%M:%SZ)"
echo "## /health/live"
curl -s -D - -o /dev/null "$BASE_URL/health/live" | redact
echo "## /health/ready"
curl -s -D - -o /dev/null "$BASE_URL/health/ready" | redact
} > evidence/health.txt
# Compose config (redacted).
{
echo "# compose-config — $(date -u +%Y-%m-%dT%H:%M:%SZ)"
docker compose --env-file .env config 2>&1 | redact
} > evidence/compose-config.txt
# Migrate logs.
{
echo "# migrate — $(date -u +%Y-%m-%dT%H:%M:%SZ)"
docker compose --env-file .env logs migrate 2>&1 | redact
} > evidence/migrate.txt
# Load test.
{
echo "# load — $(date -u +%Y-%m-%dT%H:%M:%SZ)"
python3 tests/load.py --duration 30 --concurrency 4 --output evidence/load.json 2>&1 | redact
} > evidence/load.txt
echo "Evidence collected in evidence/"
+8
View File
@@ -0,0 +1,8 @@
-- Ожидаем Index Scan по messages_chat_created_idx, без Seq Scan всей истории.
EXPLAIN (ANALYZE, BUFFERS)
SELECT id, chat_id, sender_id, text, created_at
FROM messages
WHERE chat_id = '00000000-0000-4000-8000-000000000000'
AND (created_at, id) > ('2026-01-01T00:00:00Z', '00000000-0000-4000-8000-000000000000')
ORDER BY created_at, id
LIMIT 51;
+19
View File
@@ -0,0 +1,19 @@
#!/bin/sh
set -eu
# Restart API replicas and PostgreSQL preserving volumes and data.
# Does NOT remove volumes, does NOT re-seed, does NOT print credentials.
cd "$(dirname "$0")/.."
docker compose --env-file .env restart postgres
docker compose --env-file .env restart s3
docker compose --env-file .env restart api-1 api-2
# Wait for readiness through the LB.
i=0
while [ $i -lt 60 ]; do
if curl -fsS http://localhost:8080/health/ready >/dev/null 2>&1; then
exit 0
fi
i=$((i+1))
sleep 1
done
echo "readiness timeout" >&2
exit 1
+100
View File
@@ -0,0 +1,100 @@
#!/usr/bin/env python3
"""Создаёт >=10 000 сообщений и снимает EXPLAIN (ANALYZE, BUFFERS).
Запуск: python3 scripts/seed_history.py --count 10000 --output evidence/explain.txt
Требует запущенный API (BASE_URL) и доступ к psql через docker compose exec.
"""
import argparse
import base64
import json
import os
import subprocess
import sys
import time
import urllib.error
import urllib.request
import uuid
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
def request(base, method, path, body=None, token=None, basic=None):
headers = {"Accept": "application/json"}
if token:
headers["Authorization"] = "Bearer " + token
if basic:
headers["Authorization"] = "Basic " + basic
data = None
if body is not None:
data = json.dumps(body, ensure_ascii=False).encode("utf-8")
headers["Content-Type"] = "application/json"
req = urllib.request.Request(base + path, data=data, headers=headers, method=method)
try:
with urllib.request.urlopen(req, timeout=30) as resp:
payload = resp.read()
return resp.status, json.loads(payload) if payload else None
except urllib.error.HTTPError as err:
payload = err.read()
return err.code, json.loads(payload) if payload else None
def main():
p = argparse.ArgumentParser(description="Seed history and run EXPLAIN")
p.add_argument("--base-url", default=os.environ.get("BASE_URL", "http://localhost:8080"))
p.add_argument("--count", type=int, default=10000)
p.add_argument("--output", default="evidence/explain.txt")
p.add_argument("--psql", default="docker compose exec -T postgres psql -U messenger -d messenger")
args = p.parse_args()
base = args.base_url.rstrip("/")
username = "u_seed_" + uuid.uuid4().hex[:10]
password = "T3st-" + uuid.uuid4().hex
status, alice = request(base, "POST", "/api/v1/users",
{"username": username, "display_name": "Seed", "password": password})
assert status == 201, (status, alice)
basic = base64.b64encode((username + ":" + password).encode()).decode()
status, session = request(base, "POST", "/api/v1/auth/sessions", basic=basic)
assert status == 201, (status, session)
token = session["token"]
status, chat = request(base, "POST", "/api/v1/chats",
{"title": "Seed history", "member_ids": [alice["id"]]}, token=token)
assert status == 201, (status, chat)
chat_id = chat["id"]
started = time.monotonic()
for i in range(args.count):
status, _ = request(base, "POST", "/api/v1/chats/" + chat_id + "/messages",
{"text": "seed-%d" % i}, token=token)
if status != 201:
raise SystemExit("failed to insert message %d: %s" % (i, status))
if (i + 1) % 1000 == 0:
print("inserted %d/%d" % (i + 1, args.count), file=sys.stderr)
elapsed = time.monotonic() - started
print("inserted %d messages in %.1fs" % (args.count, elapsed), file=sys.stderr)
sql = (
"EXPLAIN (ANALYZE, BUFFERS) "
"SELECT id, chat_id, sender_id, text, created_at FROM messages "
"WHERE chat_id = '%s' "
"AND (created_at, id) > ('1970-01-01T00:00:00Z', '00000000-0000-0000-0000-000000000000') "
"ORDER BY created_at, id LIMIT 51;" % chat_id
)
proc = subprocess.run(args.psql.split() + ["-c", sql], cwd=str(ROOT),
capture_output=True, text=True, timeout=120)
if proc.returncode != 0:
raise SystemExit("psql failed: " + proc.stderr[:500])
out = proc.stdout
dest = (ROOT / args.output).resolve()
dest.parent.mkdir(parents=True, exist_ok=True)
dest.write_text(
"# chat_id=%s count=%d\n%s\n" % (chat_id, args.count, out),
encoding="utf-8",
)
print(out)
print("saved to " + str(dest), file=sys.stderr)
if __name__ == "__main__":
main()
+14
View File
@@ -0,0 +1,14 @@
#!/bin/sh
set -eu
# Поднимает PostgreSQL обратно и ждёт готовности.
docker compose start postgres
i=0
while [ "$i" -lt 60 ]; do
if docker compose exec -T postgres pg_isready -U messenger -d messenger >/dev/null 2>&1; then
exit 0
fi
i=$((i+1))
sleep 1
done
echo "postgres did not become ready" >&2
exit 1
+14
View File
@@ -0,0 +1,14 @@
#!/bin/sh
set -eu
# Поднимает Redis обратно и ждёт готовности.
docker compose start redis
i=0
while [ "$i" -lt 30 ]; do
if docker compose exec -T redis redis-cli ping >/dev/null 2>&1; then
exit 0
fi
i=$((i+1))
sleep 1
done
echo "redis did not become ready" >&2
exit 1
+12
View File
@@ -0,0 +1,12 @@
#!/bin/sh
set -eu
# Stop exactly one API replica identified by TARGET_INSTANCE_ID (api-1 or api-2).
# Leaves it stopped; does not touch DB/LB/volumes.
: "${TARGET_INSTANCE_ID:?test provides target API instance alias}"
cd "$(dirname "$0")/.."
case "$TARGET_INSTANCE_ID" in
api-1|api-2) ;;
*) echo "unknown TARGET_INSTANCE_ID: $TARGET_INSTANCE_ID" >&2; exit 2 ;;
esac
docker compose --env-file .env stop "$TARGET_INSTANCE_ID"
exit 0
+4
View File
@@ -0,0 +1,4 @@
#!/bin/sh
set -eu
# Останавливает PostgreSQL, сохраняя volume. Используется тестом недоступности.
docker compose stop postgres
+4
View File
@@ -0,0 +1,4 @@
#!/bin/sh
set -eu
# Останавливает Redis, сохраняя volume. Используется тестом недоступности.
docker compose stop redis
+3 -3
View File
@@ -6,17 +6,17 @@ from openapi_spec_validator import validate
from jsonschema import Draft202012Validator
root = Path(__file__).resolve().parents[1]
spec = json.loads((root/'contracts/openapi.json').read_text())
spec = json.loads((root/'contracts/openapi.json').read_text(encoding='utf-8'))
validate(spec)
from referencing import Registry, Resource
# The base URI resolves local #/components references inside examples.
resource = dict(spec, **{'$schema': 'https://json-schema.org/draft/2020-12/schema'})
registry = Registry().with_resource('urn:mtusi:openapi', Resource.from_contents(resource))
for name, example in json.loads((root/'contracts/examples.json').read_text()).items():
for name, example in json.loads((root/'contracts/examples.json').read_text(encoding='utf-8')).items():
schema = {'$ref': 'urn:mtusi:openapi#/components/schemas/' + name}
Draft202012Validator(schema, registry=registry).validate(example)
for path in (root/'contracts').glob('*.schema.json'):
schema = json.loads(path.read_text())
schema = json.loads(path.read_text(encoding='utf-8'))
Draft202012Validator.check_schema(schema)
for example in schema.get('examples',[]):
Draft202012Validator(schema).validate(example)
+2 -2
View File
@@ -13,8 +13,8 @@ import urllib.request
import uuid
ROOT = Path(__file__).resolve().parents[1]
LAB = json.loads((ROOT / 'lab.json').read_text())['number']
SPEC = json.loads((ROOT / 'contracts/openapi.json').read_text())
LAB = json.loads((ROOT / 'lab.json').read_text(encoding='utf-8'))['number']
SPEC = json.loads((ROOT / 'contracts/openapi.json').read_text(encoding='utf-8'))
class Failure(AssertionError):
pass
+22 -2
View File
@@ -2,6 +2,7 @@
"""Explicit operator-authored hook; tokens remain in memory, volumes are never removed by this test."""
import json
import os
import shutil
from pathlib import Path
import subprocess
import time
@@ -10,6 +11,25 @@ from urllib.parse import quote
from client import Client, Failure, LAB, arguments, check, run
def _script_command(full: Path) -> list[str]:
"""Windows не умеет запускать .sh напрямую — вызываем через sh (Git Bash/WSL)."""
if os.name == "nt":
sh = shutil.which("sh") or shutil.which("bash")
if not sh:
raise Failure(
"sh/bash not found in PATH; install Git for Windows or run under WSL"
)
return [sh, str(full)]
return [str(full)]
def _is_executable(full: Path) -> bool:
"""На Windows .sh не имеет executable bit — проверяем через sh."""
if os.name == "nt":
return full.is_file() and (shutil.which("sh") or shutil.which("bash")) is not None
return full.is_file() and os.access(full, os.X_OK)
def history(c, actor, chat):
path = '/api/v1/chats/'+chat['id']+'/messages?limit=100'
result = []
@@ -34,7 +54,7 @@ def main():
check(LAB>=2,'persistence requires lab2+')
check(args.mode!='failover' or LAB>=3,'failover requires lab3+')
script = Path(args.action_script).expanduser().resolve()
check(script.is_file() and os.access(script,os.X_OK),'action script must exist and be executable (chmod +x)')
check(_is_executable(script),'action script must exist and be executable (chmod +x)')
check('.example.' not in script.name,'copy and implement the example hook first')
c = Client(args.base_url,args.ca_file,timeout=2)
alice,bob,eve,chat = c.bootstrap()
@@ -55,7 +75,7 @@ def main():
check(len(observed)>=2,'less than two API instances observed before failure; verify LB/session affinity')
print('Executing explicit '+args.mode+' hook; hook output suppressed to avoid leaking secrets.')
# No shell interpolation and no Docker commands in this test.
proc = subprocess.Popen([str(script)],env=env,stdout=subprocess.DEVNULL,stderr=subprocess.DEVNULL)
proc = subprocess.Popen(_script_command(script),env=env,stdout=subprocess.DEVNULL,stderr=subprocess.DEVNULL)
started = time.monotonic()
successes,failures,after_instances = [],0,[]
try:
+217
View File
@@ -0,0 +1,217 @@
#!/usr/bin/env python3
"""Собственные тесты уровня 4 для вложений.
Покрывают: слишком большой файл, повреждённое изображение, подмену MIME,
чужой attachment_id, привязку файла другого чата, отсутствие доступа у
постороннего, эксперимент с памятью при ≥10 одновременных загрузках.
Запуск: python3 tests/test_attachments.py --base-url http://localhost:8080
"""
import argparse
import base64
import io
import json
import os
import sys
import threading
import time
import unittest
import urllib.error
import urllib.request
import uuid
_parser = argparse.ArgumentParser(description="Attachment tests (lab4)")
_parser.add_argument("--base-url", default=None)
_args, _unknown = _parser.parse_known_args()
sys.argv = [sys.argv[0]] + _unknown
BASE_URL = (_args.base_url or os.environ.get("BASE_URL") or "http://localhost:8080").rstrip("/")
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
FIXTURE = os.path.join(ROOT, "tests", "fixtures", "sample.png")
def request(method, path, body=None, token=None, raw=None, content_type=None, extra_headers=None):
headers = {"Accept": "application/json"}
if token:
headers["Authorization"] = "Bearer " + token
if extra_headers:
headers.update(extra_headers)
data = None
if raw is not None:
data = raw
if content_type:
headers["Content-Type"] = content_type
elif body is not None:
data = json.dumps(body, ensure_ascii=False).encode("utf-8")
headers["Content-Type"] = "application/json"
req = urllib.request.Request(BASE_URL + path, data=data, headers=headers, method=method)
try:
with urllib.request.urlopen(req, timeout=30) as resp:
payload = resp.read()
ctype = resp.headers.get_content_type()
if ctype == "application/json" and payload:
return resp.status, json.loads(payload), resp.headers
return resp.status, payload, resp.headers
except urllib.error.HTTPError as err:
payload = err.read()
try:
parsed = json.loads(payload) if payload else None
except ValueError:
parsed = None
return err.code, parsed, err.headers
def make_user(label):
username = "u_" + label + "_" + uuid.uuid4().hex[:10]
password = "T3st-" + uuid.uuid4().hex
status, user, _ = request(
"POST",
"/api/v1/users",
{"username": username, "display_name": "Att " + label, "password": password},
)
assert status == 201, (status, user)
basic = base64.b64encode((username + ":" + password).encode()).decode()
# Basic-вход: используем заголовок Authorization напрямую.
req = urllib.request.Request(
BASE_URL + "/api/v1/auth/sessions",
headers={"Authorization": "Basic " + basic, "Accept": "application/json"},
method="POST",
)
with urllib.request.urlopen(req, timeout=10) as resp:
session = json.loads(resp.read())
return {"user": user, "token": session["token"]}
def make_chat(owner, members):
status, chat, _ = request(
"POST",
"/api/v1/chats",
{"title": "Att chat", "member_ids": [owner["user"]["id"]] + [m["user"]["id"] for m in members]},
token=owner["token"],
)
assert status == 201, (status, chat)
return chat
def upload(token, chat_id, data, filename="sample.png", content_type="image/png"):
path = "/api/v1/chats/" + chat_id + "/attachments?filename=" + urllib.request.quote(filename)
return request("POST", path, raw=data, token=token, content_type=content_type)
class AttachmentTests(unittest.TestCase):
@classmethod
def setUpClass(cls):
with open(FIXTURE, "rb") as fh:
cls.png = fh.read()
cls.alice = make_user("alice")
cls.bob = make_user("bob")
cls.eve = make_user("eve")
cls.chat = make_chat(cls.alice, [cls.bob])
cls.other_chat = make_chat(cls.alice, [cls.bob])
def test_upload_and_download_roundtrip(self):
status, att, _ = upload(self.alice["token"], self.chat["id"], self.png)
self.assertEqual(status, 201, att)
self.assertEqual(att["status"], "ready")
self.assertEqual(att["content_type"], "image/png")
self.assertEqual(att["size_bytes"], len(self.png))
status, body, headers = request(
"GET", "/api/v1/attachments/" + att["id"] + "/content", token=self.bob["token"]
)
self.assertEqual(status, 200)
self.assertEqual(body, self.png)
self.assertEqual(headers.get_content_type(), "image/png")
def test_oversized_rejected(self):
big = b"\x89PNG\r\n\x1a\n" + b"\x00" * (10 * 1024 * 1024 + 1)
status, body, _ = upload(self.alice["token"], self.chat["id"], big)
self.assertEqual(status, 413)
self.assertEqual(body["error"]["code"], "payload_too_large")
def test_corrupted_image_rejected(self):
broken = b"\x89PNG\r\n\x1a\n" + b"garbage" * 100
status, body, _ = upload(self.alice["token"], self.chat["id"], broken)
self.assertEqual(status, 415)
self.assertEqual(body["error"]["code"], "unsupported_media_type")
def test_mime_spoof_rejected(self):
# Заявляем image/png, а содержимое — JPEG.
jpeg = b"\xff\xd8\xff\xe0" + b"\x00" * 100
status, body, _ = upload(self.alice["token"], self.chat["id"], jpeg, content_type="image/png")
self.assertEqual(status, 415)
self.assertEqual(body["error"]["code"], "unsupported_media_type")
def test_unsupported_content_type(self):
status, body, _ = upload(self.alice["token"], self.chat["id"], self.png, content_type="text/plain")
self.assertEqual(status, 415)
def test_dangerous_filename_rejected(self):
status, body, _ = upload(self.alice["token"], self.chat["id"], self.png, filename="../etc/passwd")
self.assertEqual(status, 400)
self.assertEqual(body["error"]["code"], "invalid_request")
def test_outsider_cannot_read(self):
status, att, _ = upload(self.alice["token"], self.chat["id"], self.png)
self.assertEqual(status, 201)
for suffix in ("", "/content"):
status, body, _ = request(
"GET", "/api/v1/attachments/" + att["id"] + suffix, token=self.eve["token"]
)
self.assertEqual(status, 404)
self.assertEqual(body["error"]["code"], "not_found")
def test_attachment_from_other_chat_rejected(self):
status, att, _ = upload(self.alice["token"], self.other_chat["id"], self.png)
self.assertEqual(status, 201)
status, body, _ = request(
"POST",
"/api/v1/chats/" + self.chat["id"] + "/messages",
{"text": "", "attachment_ids": [att["id"]]},
token=self.alice["token"],
extra_headers={"Idempotency-Key": "other-chat-" + uuid.uuid4().hex},
)
self.assertEqual(status, 404)
self.assertEqual(body["error"]["code"], "not_found")
def test_message_with_attachment(self):
status, att, _ = upload(self.alice["token"], self.chat["id"], self.png)
self.assertEqual(status, 201)
status, msg, _ = request(
"POST",
"/api/v1/chats/" + self.chat["id"] + "/messages",
{"text": "", "attachment_ids": [att["id"]]},
token=self.alice["token"],
extra_headers={"Idempotency-Key": "msg-att-" + uuid.uuid4().hex},
)
self.assertEqual(status, 201, msg)
self.assertEqual(msg["attachment_ids"], [att["id"]])
self.assertEqual(msg["text"], "")
def test_concurrent_uploads_memory(self):
"""≥10 одновременных загрузок; проверяем, что все успешны и API жив."""
results = []
lock = threading.Lock()
def worker():
status, att, _ = upload(self.alice["token"], self.chat["id"], self.png)
with lock:
results.append((status, att))
threads = [threading.Thread(target=worker) for _ in range(10)]
started = time.monotonic()
for t in threads:
t.start()
for t in threads:
t.join()
elapsed = time.monotonic() - started
self.assertEqual(len(results), 10)
for status, att in results:
self.assertEqual(status, 201, att)
# API остаётся живым после нагрузки.
status, _, _ = request("GET", "/health/ready")
self.assertEqual(status, 200)
print("\nconcurrent uploads: 10 in %.2fs" % elapsed)
if __name__ == "__main__":
unittest.main(verbosity=2)
+147
View File
@@ -0,0 +1,147 @@
#!/usr/bin/env python3
"""Проверка недоступности Redis и PostgreSQL.
Управление контейнерами — через скрипты scripts/stop-*.sh / start-*.sh,
чтобы тест не зависел от Docker напрямую.
"""
import argparse
import base64
import json
import os
import shutil
import subprocess
import sys
import time
import unittest
import urllib.error
import urllib.request
import uuid
from pathlib import Path
_parser = argparse.ArgumentParser(description="Dependency failure tests")
_parser.add_argument("--base-url", default=None)
_parser.add_argument("--stop-redis", default="scripts/stop-redis.sh")
_parser.add_argument("--start-redis", default="scripts/start-redis.sh")
_parser.add_argument("--stop-postgres", default="scripts/stop-postgres.sh")
_parser.add_argument("--start-postgres", default="scripts/start-postgres.sh")
_args, _unknown = _parser.parse_known_args()
sys.argv = [sys.argv[0]] + _unknown
BASE_URL = (_args.base_url or os.environ.get("BASE_URL") or "http://localhost:8080").rstrip("/")
ROOT = Path(__file__).resolve().parents[1]
def _script_command(full: Path) -> list[str]:
"""Windows не умеет запускать .sh напрямую — вызываем через sh (Git Bash/WSL)."""
if os.name == "nt":
sh = shutil.which("sh") or shutil.which("bash")
if not sh:
raise unittest.SkipTest(
"sh/bash not found in PATH; install Git for Windows or run under WSL"
)
return [sh, str(full)]
return [str(full)]
def run_script(path):
full = (ROOT / path).resolve()
if not full.is_file():
raise unittest.SkipTest("script not found: " + str(full))
cmd = _script_command(full)
proc = subprocess.run(cmd, cwd=str(ROOT), capture_output=True, timeout=120)
if proc.returncode != 0:
raise AssertionError(
"script failed: " + path + " rc=" + str(proc.returncode)
+ " stderr=" + proc.stderr.decode("utf-8", "replace")[:500]
)
def request(method, path, body=None, token=None, basic=None):
headers = {"Accept": "application/json"}
if token:
headers["Authorization"] = "Bearer " + token
if basic:
headers["Authorization"] = "Basic " + basic
data = None
if body is not None:
data = json.dumps(body, ensure_ascii=False).encode("utf-8")
headers["Content-Type"] = "application/json"
req = urllib.request.Request(BASE_URL + path, data=data, headers=headers, method=method)
try:
with urllib.request.urlopen(req, timeout=10) as resp:
payload = resp.read()
return resp.status, json.loads(payload) if payload else None
except urllib.error.HTTPError as err:
payload = err.read()
return err.code, json.loads(payload) if payload else None
def make_user(label):
username = "u_" + label + "_" + uuid.uuid4().hex[:10]
password = "T3st-" + uuid.uuid4().hex
status, user = request(
"POST",
"/api/v1/users",
{"username": username, "display_name": "Dep " + label, "password": password},
)
assert status == 201, (status, user)
basic = base64.b64encode((username + ":" + password).encode()).decode()
status, session = request("POST", "/api/v1/auth/sessions", basic=basic)
assert status == 201, (status, session)
return {"user": user, "token": session["token"]}
def wait_ready(timeout=60):
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
try:
status, _ = request("GET", "/health/ready")
if status == 200:
return
except Exception:
pass
time.sleep(0.5)
raise AssertionError("API did not become ready within " + str(timeout) + "s")
class DependencyFailureTests(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.alice = make_user("alice")
def test_redis_down_falls_back_to_db(self):
run_script(_args.stop_redis)
try:
# Валидный токен продолжает работать (fallback на БД).
status, body = request("GET", "/api/v1/users/me", token=self.alice["token"])
self.assertEqual(status, 200, body)
self.assertEqual(body["id"], self.alice["user"]["id"])
# Аноним не принимается.
status, body = request("GET", "/api/v1/users/me")
self.assertEqual(status, 401)
self.assertEqual(body["error"]["code"], "unauthorized")
# /health/ready не зависит от Redis.
status, _ = request("GET", "/health/ready")
self.assertEqual(status, 200)
finally:
run_script(_args.start_redis)
wait_ready()
def test_postgres_down_returns_503(self):
run_script(_args.stop_postgres)
try:
status, body = request("GET", "/health/ready")
self.assertEqual(status, 503)
self.assertEqual(body["error"]["code"], "unavailable")
# Бизнес-запрос не должен отдавать 200 и не должен раскрывать детали.
status, body = request("GET", "/api/v1/users/me", token=self.alice["token"])
self.assertIn(status, (500, 503))
self.assertIn("error", body)
self.assertNotIn("psycopg", json.dumps(body).lower())
finally:
run_script(_args.start_postgres)
wait_ready()
if __name__ == "__main__":
unittest.main(verbosity=2)
+93
View File
@@ -0,0 +1,93 @@
#!/usr/bin/env python3
"""Проверка expiry сессии при коротком SESSION_TTL_SECONDS.
Запускается против стенда с SESSION_TTL_SECONDS <= 10 (например, 2).
Если TTL больше — тест скипается, чтобы не ждать минуты.
"""
import argparse
import base64
import json
import os
import sys
import time
import unittest
import urllib.error
import urllib.request
import uuid
_parser = argparse.ArgumentParser(description="Expiry test")
_parser.add_argument("--base-url", default=None)
_parser.add_argument("--ttl", type=int, default=None)
_args, _unknown = _parser.parse_known_args()
sys.argv = [sys.argv[0]] + _unknown
BASE_URL = (_args.base_url or os.environ.get("BASE_URL") or "http://localhost:8080").rstrip("/")
TTL = _args.ttl or int(os.environ.get("SESSION_TTL_SECONDS", "3600"))
def request(method, path, body=None, token=None, basic=None):
headers = {"Accept": "application/json"}
if token:
headers["Authorization"] = "Bearer " + token
if basic:
headers["Authorization"] = "Basic " + basic
data = None
if body is not None:
data = json.dumps(body, ensure_ascii=False).encode("utf-8")
headers["Content-Type"] = "application/json"
req = urllib.request.Request(BASE_URL + path, data=data, headers=headers, method=method)
try:
with urllib.request.urlopen(req, timeout=10) as resp:
payload = resp.read()
return resp.status, json.loads(payload) if payload else None
except urllib.error.HTTPError as err:
payload = err.read()
return err.code, json.loads(payload) if payload else None
@unittest.skipIf(
TTL > 10,
"SESSION_TTL_SECONDS > 10; запустите стенд с коротким TTL и передайте "
"SESSION_TTL_SECONDS=2 make test-expiry",
)
class ExpiryTests(unittest.TestCase):
def test_session_expires(self):
username = "u_exp_" + uuid.uuid4().hex[:10]
password = "T3st-" + uuid.uuid4().hex
status, user = request(
"POST",
"/api/v1/users",
{"username": username, "display_name": "Expiry", "password": password},
)
self.assertEqual(status, 201, user)
basic = base64.b64encode((username + ":" + password).encode()).decode()
status, session = request("POST", "/api/v1/auth/sessions", basic=basic)
self.assertEqual(status, 201, session)
token = session["token"]
# Проверяем, что API реально запущен с коротким TTL: иначе тест
# будет ждать час и всё равно упадёт.
expires_at = session["expires_at"]
from datetime import datetime, timezone
exp = datetime.strptime(expires_at, "%Y-%m-%dT%H:%M:%SZ").replace(tzinfo=timezone.utc)
actual_ttl = (exp - datetime.now(timezone.utc)).total_seconds()
self.assertLessEqual(
actual_ttl,
10,
"API запущен с TTL=" + str(int(actual_ttl)) + "s; перезапустите стенд с "
"SESSION_TTL_SECONDS=2 (например: SESSION_TTL_SECONDS=2 docker compose up -d api)",
)
status, _ = request("GET", "/api/v1/users/me", token=token)
self.assertEqual(status, 200)
time.sleep(actual_ttl + 2)
status, body = request("GET", "/api/v1/users/me", token=token)
self.assertEqual(status, 401)
self.assertEqual(body["error"]["code"], "unauthorized")
# Повторный вход с теми же credentials снова работает.
status, session2 = request("POST", "/api/v1/auth/sessions", basic=basic)
self.assertEqual(status, 201, session2)
status, _ = request("GET", "/api/v1/users/me", token=session2["token"])
self.assertEqual(status, 200)
if __name__ == "__main__":
unittest.main(verbosity=2)
+338
View File
@@ -0,0 +1,338 @@
#!/usr/bin/env python3
"""Собственные тесты уровня 4: негативные сценарии и конкурентная запись.
Запуск: python3 tests/test_unit.py
Требует запущенный сервер (по умолчанию http://localhost:8080).
"""
import argparse
import base64
import json
import os
import sys
import threading
import unittest
import urllib.error
import urllib.request
import uuid
_parser = argparse.ArgumentParser(description="Собственные тесты уровня 4")
_parser.add_argument("--base-url", default=None)
_args, _unknown = _parser.parse_known_args()
# unittest.main() читает sys.argv сам и не знает про --base-url — убираем его.
sys.argv = [sys.argv[0]] + _unknown
BASE_URL = (
_args.base_url
or os.environ.get("BASE_URL")
or "http://localhost:8080"
).rstrip("/")
def request(method, path, body=None, token=None, raw=None, content_type="application/json",
basic=None, extra_headers=None):
headers = {"Accept": "application/json"}
if token:
headers["Authorization"] = "Bearer " + token
if basic:
headers["Authorization"] = "Basic " + basic
if extra_headers:
headers.update(extra_headers)
data = None
if raw is not None:
data = raw
headers["Content-Type"] = content_type
elif body is not None:
data = json.dumps(body, ensure_ascii=False).encode("utf-8")
headers["Content-Type"] = "application/json"
req = urllib.request.Request(BASE_URL + path, data=data, headers=headers, method=method)
try:
with urllib.request.urlopen(req, timeout=10) as resp:
payload = resp.read()
return resp.status, json.loads(payload) if payload else None
except urllib.error.HTTPError as err:
payload = err.read()
return err.code, json.loads(payload) if payload else None
def make_user(label):
username = "u_" + label + "_" + uuid.uuid4().hex[:10]
password = "T3st-" + uuid.uuid4().hex
status, user = request(
"POST",
"/api/v1/users",
{"username": username, "display_name": "Тест " + label, "password": password},
)
assert status == 201, (status, user)
basic = base64.b64encode((username + ":" + password).encode()).decode()
status, session = request("POST", "/api/v1/auth/sessions", basic=basic)
assert status == 201, (status, session)
return {"user": user, "token": session["token"], "basic": basic, "password": password}
class MessengerTests(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.alice = make_user("alice")
cls.bob = make_user("bob")
cls.eve = make_user("eve")
status, chat = request(
"POST",
"/api/v1/chats",
{"title": "Тестовый чат", "member_ids": [cls.alice["user"]["id"], cls.bob["user"]["id"]]},
token=cls.alice["token"],
)
assert status == 201, (status, chat)
cls.chat = chat
def test_invalid_json(self):
status, body = request(
"POST",
"/api/v1/chats/" + self.chat["id"] + "/messages",
raw=b"{not json",
token=self.alice["token"],
)
self.assertEqual(status, 400)
self.assertEqual(body["error"]["code"], "invalid_request")
def test_foreign_chat_returns_404(self):
status, body = request("GET", "/api/v1/chats/" + self.chat["id"], token=self.eve["token"])
self.assertEqual(status, 404)
self.assertEqual(body["error"]["code"], "not_found")
def test_empty_message_rejected(self):
status, body = request(
"POST",
"/api/v1/chats/" + self.chat["id"] + "/messages",
{"text": " "},
token=self.alice["token"],
)
self.assertEqual(status, 400)
self.assertEqual(body["error"]["code"], "invalid_request")
def test_duplicate_username(self):
username = "u_dup_" + uuid.uuid4().hex[:10]
password = "T3st-" + uuid.uuid4().hex
first = request(
"POST",
"/api/v1/users",
{"username": username, "display_name": "Первый", "password": password},
)
self.assertEqual(first[0], 201)
second = request(
"POST",
"/api/v1/users",
{"username": username, "display_name": "Второй", "password": password},
)
self.assertEqual(second[0], 409)
self.assertEqual(second[1]["error"]["code"], "conflict")
def test_short_password_rejected(self):
status, body = request(
"POST",
"/api/v1/users",
{"username": "u_short_" + uuid.uuid4().hex[:8], "display_name": "X", "password": "short"},
)
self.assertEqual(status, 400)
self.assertEqual(body["error"]["code"], "invalid_request")
def test_wrong_password_returns_401(self):
bad = base64.b64encode(
(self.alice["user"]["username"] + ":wrong-password-123").encode()
).decode()
status, body = request("POST", "/api/v1/auth/sessions", basic=bad)
self.assertEqual(status, 401)
self.assertEqual(body["error"]["code"], "unauthorized")
def test_basic_not_accepted_on_business_endpoint(self):
status, body = request("GET", "/api/v1/users/me", basic=self.alice["basic"])
self.assertEqual(status, 401)
self.assertEqual(body["error"]["code"], "unauthorized")
def test_x_user_id_ignored(self):
status, body = request(
"GET",
"/api/v1/users/me",
token=self.alice["token"],
extra_headers={"X-User-Id": self.eve["user"]["id"]},
)
self.assertEqual(status, 200)
self.assertEqual(body["id"], self.alice["user"]["id"])
def test_logout_revokes_session(self):
user = make_user("logout")
status, _ = request("DELETE", "/api/v1/auth/sessions/current", token=user["token"])
self.assertEqual(status, 204)
status, body = request("GET", "/api/v1/users/me", token=user["token"])
self.assertEqual(status, 401)
self.assertEqual(body["error"]["code"], "unauthorized")
def test_pagination_walks_all_pages(self):
chat = request(
"POST",
"/api/v1/chats",
{"title": "Пагинация", "member_ids": [self.alice["user"]["id"], self.bob["user"]["id"]]},
token=self.alice["token"],
)[1]
sent = []
for i in range(7):
status, msg = request(
"POST",
"/api/v1/chats/" + chat["id"] + "/messages",
{"text": "msg-%d" % i},
token=self.alice["token"],
)
self.assertEqual(status, 201)
sent.append(msg["id"])
collected = []
cursor = None
while True:
path = "/api/v1/chats/" + chat["id"] + "/messages?limit=2"
if cursor:
path += "&cursor=" + cursor
status, page = request("GET", path, token=self.bob["token"])
self.assertEqual(status, 200)
collected.extend(m["id"] for m in page["items"])
cursor = page["next_cursor"]
if not cursor:
break
self.assertEqual(collected, sent)
def test_cursor_bound_to_resource(self):
chat_a = request(
"POST",
"/api/v1/chats",
{"title": "Cursor A", "member_ids": [self.alice["user"]["id"], self.bob["user"]["id"]]},
token=self.alice["token"],
)[1]
chat_b = request(
"POST",
"/api/v1/chats",
{"title": "Cursor B", "member_ids": [self.alice["user"]["id"], self.bob["user"]["id"]]},
token=self.alice["token"],
)[1]
for i in range(3):
request(
"POST",
"/api/v1/chats/" + chat_a["id"] + "/messages",
{"text": "a-%d" % i},
token=self.alice["token"],
)
status, page = request(
"GET",
"/api/v1/chats/" + chat_a["id"] + "/messages?limit=1",
token=self.alice["token"],
)
self.assertEqual(status, 200)
cursor = page["next_cursor"]
self.assertIsNotNone(cursor)
status, body = request(
"GET",
"/api/v1/chats/" + chat_b["id"] + "/messages?limit=1&cursor=" + cursor,
token=self.alice["token"],
)
self.assertEqual(status, 400)
self.assertEqual(body["error"]["code"], "invalid_request")
def test_cursor_bound_to_endpoint(self):
status, page = request(
"GET", "/api/v1/chats?limit=1", token=self.alice["token"]
)
self.assertEqual(status, 200)
cursor = page["next_cursor"]
self.assertIsNotNone(cursor)
status, body = request(
"GET",
"/api/v1/chats/" + self.chat["id"] + "/messages?limit=1&cursor=" + cursor,
token=self.alice["token"],
)
self.assertEqual(status, 400)
self.assertEqual(body["error"]["code"], "invalid_request")
def test_cursor_garbage_rejected(self):
garbage = base64.urlsafe_b64encode(b"x|y").decode("ascii")
status, body = request(
"GET",
"/api/v1/chats/" + self.chat["id"] + "/messages?cursor=" + garbage,
token=self.alice["token"],
)
self.assertEqual(status, 400)
self.assertEqual(body["error"]["code"], "invalid_request")
def test_invalid_chat_id_returns_400(self):
status, body = request("GET", "/api/v1/chats/not-a-uuid", token=self.alice["token"])
self.assertEqual(status, 400)
self.assertEqual(body["error"]["code"], "invalid_request")
status, body = request(
"POST",
"/api/v1/chats/not-a-uuid/messages",
{"text": "hi"},
token=self.alice["token"],
)
self.assertEqual(status, 400)
self.assertEqual(body["error"]["code"], "invalid_request")
status, body = request(
"GET", "/api/v1/chats/not-a-uuid/messages", token=self.alice["token"]
)
self.assertEqual(status, 400)
self.assertEqual(body["error"]["code"], "invalid_request")
def test_username_with_trailing_newline_rejected(self):
status, body = request(
"POST",
"/api/v1/users",
{"username": "abc\n", "display_name": "Имя", "password": "T3st-" + uuid.uuid4().hex},
)
self.assertEqual(status, 400)
self.assertEqual(body["error"]["code"], "invalid_request")
def test_invalid_body_bytes_contract_error(self):
status, body = request(
"POST",
"/api/v1/users",
raw=b"\xff",
token=self.alice["token"],
)
self.assertEqual(status, 400)
self.assertEqual(body["error"]["code"], "invalid_request")
self.assertIn("message", body["error"])
self.assertIn("request_id", body["error"])
def test_concurrent_writes_are_unique(self):
chat = request(
"POST",
"/api/v1/chats",
{"title": "Конкурентность", "member_ids": [self.alice["user"]["id"], self.bob["user"]["id"]]},
token=self.alice["token"],
)[1]
errors = []
lock = threading.Lock()
def worker(worker_id):
for i in range(10):
status, body = request(
"POST",
"/api/v1/chats/" + chat["id"] + "/messages",
{"text": "w%d-%d" % (worker_id, i)},
token=self.alice["token"],
)
if status != 201:
with lock:
errors.append((status, body))
threads = [threading.Thread(target=worker, args=(w,)) for w in range(10)]
for t in threads:
t.start()
for t in threads:
t.join()
self.assertEqual(errors, [])
status, page = request(
"GET", "/api/v1/chats/" + chat["id"] + "/messages?limit=100", token=self.alice["token"]
)
self.assertEqual(status, 200)
ids = [m["id"] for m in page["items"]]
self.assertEqual(len(ids), 100)
self.assertEqual(len(set(ids)), 100)
if __name__ == "__main__":
unittest.main(verbosity=2)