выполнение лр2 на 4
Skeleton contracts / contracts (pull_request) Canceled after 0s

This commit is contained in:
Ilya
2026-10-07 18:28:12 +03:00
parent 65003b6119
commit e914e384fc
32 changed files with 2054 additions and 15 deletions
+6
View File
@@ -9,3 +9,9 @@ DATABASE_URL_FILE=/run/secrets/database_url
# Optional on 3, required use on 4 in lab2; Redis OR Valkey:
SESSION_STORE=postgres
CACHE_URL_FILE=/run/secrets/cache_url
# --- Actual lab2 implementation (see REPORT.md) ---
# APP_PORT is also the host port published on 127.0.0.1 by compose.yaml.
# SESSION_STORE=postgres: sessions live in PostgreSQL; Valkey (CACHE_URL_FILE) caches lookups.
SESSION_CACHE_MAX_SECONDS=300
DB_POOL_MAX=10
SHORT_TTL_PORT=8081
+23
View File
@@ -0,0 +1,23 @@
FROM python:3.12.8-slim-bookworm
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY app ./app
COPY migrations ./migrations
RUN useradd --system --no-create-home --uid 10001 appuser
USER appuser
ENV APP_HOST=0.0.0.0 \
APP_PORT=8080 \
PYTHONUNBUFFERED=1
EXPOSE 8080
HEALTHCHECK --interval=10s --timeout=3s --start-period=5s --retries=3 \
CMD python -c "import os,urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:'+os.environ.get('APP_PORT','8080')+'/health/ready',timeout=2).status==200 else 1)"
CMD ["sh", "-c", "exec uvicorn app.main:app --host ${APP_HOST} --port ${APP_PORT} --timeout-graceful-shutdown 8"]
+17 -1
View File
@@ -4,7 +4,7 @@ CA_FILE ?=
ACTION_SCRIPT ?=
TLS_ARGS = $(if $(CA_FILE),--ca-file "$(CA_FILE)",)
.PHONY: help check validate test load test-persistence
.PHONY: help check validate test load test-persistence secrets up down test-student explain
help:
@echo "check: offline structure; validate: full OpenAPI (requirements-dev.txt); test: running API; load: synthetic writes"
@@ -24,3 +24,19 @@ load:
test-persistence:
@test -n "$(ACTION_SCRIPT)" || (echo "Set ACTION_SCRIPT to your implemented executable restart hook"; exit 2)
$(PYTHON) tests/resilience.py --mode persistence --action-script "$(ACTION_SCRIPT)" --base-url "$(BASE_URL)" $(TLS_ARGS)
# --- Student implementation helpers (see REPORT.md) ---
secrets:
sh scripts/init-secrets.sh
up: secrets
docker compose up -d --build --wait
down:
docker compose --profile ttl-test down
test-student:
BASE_URL="$(BASE_URL)" sh student-tests/run_all.sh
explain:
sh student-tests/explain_history.sh
+119 -14
View File
@@ -1,37 +1,142 @@
# Отчёт — лабораторная 2
- Автор, группа: TODO
- Целевая оценка; выбранный трек 5, если нужен: TODO
- Целевая оценка: **4** (трек 5 не заявлен)
- Commit/PR: TODO после публикации
- Использованные источники, библиотеки и помощь ИИ: TODO
- Использованные источники, библиотеки и помощь ИИ: FastAPI, uvicorn, pydantic, asyncpg, redis-py (клиент к Valkey), argon2-cffi; документация PostgreSQL (transactions, EXPLAIN), Valkey persistence, RFC 7617. Код lab1 перенесён из собственного решения. При переносе на PostgreSQL, написании тестов и отчёта использовался ИИ-ассистент Claude (Anthropic); решения проверены и объяснимы автором.
## Запуск из чистого clone
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
Требования: Docker Engine с Compose v2 (проверено: Docker 28.0.1, Compose 2.33.1), Python 3.9+ для тестов (проверено на 3.14), `make`.
| Роль | Образ | Версия |
| --- | --- | --- |
| api / migrate | `mtusi-messenger:lab2`, сборка из `Dockerfile` | `python:3.12.8-slim-bookworm`, зависимости закреплены в `requirements.txt` |
| postgres | `postgres:17.6-alpine3.22` | PostgreSQL 17.6 |
| valkey | `valkey/valkey:8.1.4-alpine3.22` | Valkey 8.1.4 |
```sh
cp .env.example .env # при занятом 8080 поменять APP_PORT (у автора 18080)
make secrets # scripts/init-secrets.sh: случайные пароли в ./secrets (в .gitignore)
docker compose --env-file .env config --quiet
make up # docker compose up -d --build --wait: postgres -> migrate -> api; ждёт healthcheck /health/ready
make check
make test BASE_URL=http://localhost:8080
make test-persistence ACTION_SCRIPT=scripts/restart.sh BASE_URL=http://localhost:8080
make test-student BASE_URL=http://localhost:8080 # собственные тесты; останавливает/запускает контейнеры
make explain # засев 70 000 синтетических сообщений + EXPLAIN
```
- Миграции: отдельный одноразовый сервис `migrate` (`python -m app.migrate`). API стартует только после его успешного завершения. Повторный запуск безопасен (`schema_migrations` + `pg_advisory_lock`).
- Безопасная остановка без потери данных: `make down` (= `docker compose --profile ttl-test down`, volume `pgdata` сохраняется).
- **Намеренное удаление учебных данных:** `docker compose --profile ttl-test down -v` (удаляет volume `messenger-lab2_pgdata`; необратимо).
- Действия оператора: только `make secrets` перед первым запуском. Секреты в образ и Git не попадают.
## Архитектура и решения
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab1, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
```
клиент ──HTTP──> api (FastAPI/uvicorn, 1 процесс)
├──> postgres : источник истины — users, chats, chat_members, messages, sessions
└──> valkey : кэш поиска сессии (только ускорение, может пропасть в любой момент)
migrate (one-shot) ──> postgres : применение migrations/*.sql
```
**Перенесено из lab1:** модели pydantic, формат ошибок и `X-Request-Id`, keyset-курсор, привязанный к ресурсу, проверки контракта (пустой текст, дубликаты участников, 404 на чужой чат), Dockerfile (non-root, graceful SIGTERM). In-memory `Store` заменён на репозиторий PostgreSQL (`app/storage.py`).
**Изменено по контракту lab2:** `password` при регистрации, `POST /auth/sessions` через Basic, Bearer на всех бизнес-endpoint, `DELETE /auth/sessions/current`. `X-User-Id` не читается вообще.
### Схема БД (`migrations/0001_init.sql`)
| Таблица | Ключи и ограничения | Индексы |
| --- | --- | --- |
| users | PK id; `UNIQUE(username)`; CHECK формата username и длины display_name | уникальный по username |
| chats | PK id; CHECK длины title | — |
| chat_members | PK (chat_id, user_id); FK на chats и users; `position` хранит порядок member_ids | `(user_id, chat_id)` для списка чатов пользователя |
| messages | PK id; **FK (chat_id, sender_id) → chat_members**: отправитель обязан быть участником на уровне БД; CHECK длины текста | `(chat_id, created_at, id)` для истории |
| sessions | PK token_hash (sha256 токена, сам токен не хранится); FK user_id; expires_at, revoked_at | `(expires_at)` |
Время (`created_at`, `expires_at`) назначает PostgreSQL (`clock_timestamp()`), в API выдаётся RFC 3339 UTC с микросекундами. Порядок `(created_at, id)`; сравнение uuid в PostgreSQL побайтовое и совпадает с порядком UUID-int в тестах.
### Транзакционные границы
- **Создание чата** — одна транзакция: `INSERT chats` + `INSERT chat_members … FROM unnest($2::uuid[])`. Неизвестный участник отлавливает FK внутри транзакции → `ROLLBACK` → 400. Предварительного SELECT нет, поэтому откат реально выполняется (тест `db_integrity_test.py`).
- **Регистрация** — один `INSERT`; гонку одинаковых username решает `UNIQUE` (`UniqueViolationError` → 409), а не проверка «SELECT, потом INSERT».
- **Отправка сообщения** — одна инструкция `INSERT … SELECT … WHERE EXISTS(участник)`: проверка доступа и запись атомарны.
- **Logout** — `UPDATE sessions SET revoked_at` (истина), затем надгробие в Valkey.
- `COMMIT` подтверждает запись в WAL; при настройках по умолчанию (`fsync=on`, `synchronous_commit=on`) подтверждённое сообщение переживает падение процесса PostgreSQL. Ответ 201 отправляется только после COMMIT.
### Аутентификация и сессии
- Пароль: **Argon2id** (argon2-cffi, профиль RFC 9106 low-memory: t=3, m=64 MiB, p=4). Хеширование в thread pool, не более 4 одновременно (`PASSWORD_HASH_CONCURRENCY`), чтобы ограничить RAM. Для несуществующего username проверяется фиктивный хеш, поэтому время ответа одинаковое.
- Basic разбирается строго: base64 → UTF-8 → разделение по **первому** `:` (двоеточие в пароле допустимо), username только ASCII, пароль 12–128 code points. Любая ошибка даёт одинаковый 401 с `WWW-Authenticate: Basic realm="messenger", charset="UTF-8"`.
- Токен — `secrets.token_urlsafe(32)` (256 бит). В БД и кэше хранится только `sha256(token)`. Быстрый хеш здесь уместен: токен случайный, перебирать его бессмысленно, в отличие от человеческого пароля, которому нужен медленный KDF.
- Срок жизни `SESSION_TTL_SECONDS` (по умолчанию 3600) задаётся в БД как `expires_at`, поэтому рестарт его не продлевает.
### Роль Valkey (уровень 4): кэш сессий
- **Источник истины — PostgreSQL.** Valkey хранит `session:<sha256 hex>` → `{user, expires_at}`, чтобы не ходить в БД на каждом запросе; `/users/me` обслуживается вообще без БД.
- **Expiry:** TTL записи = `min(остаток сессии, SESSION_CACHE_MAX_SECONDS=300)`, плюс при чтении из кэша сравнивается `expires_at`. Просроченная сессия не принимается ни из кэша, ни из БД.
- **Инвалидация:** при logout сначала `revoked_at` в PostgreSQL, затем в Valkey пишется **надгробие** `revoked` (а не DEL). Заполнение кэша идёт через `SET NX`, поэтому параллельный запрос, прочитавший БД до отзыва, не сможет вернуть сессию в кэш.
- **Persistence не нужна и выключена** (`save ""`, `appendonly no`): после flush или рестарта Valkey первая проверка идёт в PostgreSQL. Потеря кэша ничего не ломает, это подтверждает тест сохранности, где Valkey тоже перезапускается.
- **Отказ Valkey:** таймаут 0.3 с, затем circuit breaker на 5 с, в течение которых кэш не используется. Readiness при этом остаётся 200: Valkey не критичен.
- Конфигурация с `requirepass` лежит в секрете `secrets/valkey.conf`, порт наружу не публикуется.
### Устойчивость к отказам
- asyncpg pool: `max_size=10`, таймауты подключения 3 с, захвата соединения 3 с, команды 5 с. `min_size=0` позволяет API стартовать при недоступной БД (тогда readiness = 503).
- Недоступность или рестарт PostgreSQL (`OSError`, `PostgresConnectionError`, `InterfaceError`, `TooManyConnections`, admin shutdown, statement timeout) превращается в `503 unavailable` без деталей. Без токена по-прежнему 401. Токен, который нельзя проверить (нет в кэше, БД недоступна), получает 503 и **никогда** не считается аутентифицированным.
- Readiness: 200 только при успешном `SELECT 1` в PostgreSQL. Liveness всегда 200.
- Все SQL-запросы параметризованы (`$1…`), строк, склеенных из пользовательского ввода, нет.
### Переменные `.env.example` → реальная конфигурация
| Переменная | Где используется |
| --- | --- |
| APP_HOST / APP_PORT | uvicorn внутри контейнера слушает 0.0.0.0:8080; `APP_PORT` в `.env` — порт хоста (`127.0.0.1:${APP_PORT}`) |
| APP_ENV | передаётся в контейнер, логики не меняет |
| SESSION_TTL_SECONDS | срок сессии; у `api-short-ttl` = 2 |
| DATABASE_URL_FILE | `/run/secrets/database_url` (compose secret из `secrets/database_url`) |
| SESSION_STORE=postgres | сессии хранятся в PostgreSQL |
| CACHE_URL_FILE | `/run/secrets/cache_url`, URL Valkey; без него кэш отключён |
| SESSION_CACHE_MAX_SECONDS, DB_POOL_MAX, SHORT_TTL_PORT | добавлены этой работой (см. выше) |
## Доказательства критериев
Все файлы в `evidence/` сняты 2026-10-07 на MacBook (Apple Silicon, Docker Desktop, без CPU/RAM-лимитов в compose). Порт стенда автора — 18080, потому что 8080 занят другим проектом.
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
| --- | --- | --- | --- |
| Публичный smoke | make test | TODO | TODO |
| Остальные пункты уровня 3 | TODO | TODO | TODO |
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
| Публичный smoke | `make check`, `make test` | PASS обоих | `evidence/smoke.txt` |
| 3: данные и сессии переживают рестарт API **и хранилищ** | `make test-persistence ACTION_SCRIPT=scripts/restart.sh` | PASS; `StartedAt` api/postgres/valkey изменились, volume тот же, тот же токен читает данные | `evidence/persistence.txt` |
| 3: миграции с нуля и повторно | `docker compose -p lab2-clean up` на пустом volume, затем ещё `run migrate` | `1 applied now, 0 before`, затем `0 applied now, 1 before`; smoke PASS на новом стенде | `evidence/fresh_migrations.txt` |
| 3: PK/FK, UNIQUE username, атомарное создание чата | `student-tests/db_integrity_test.py` | чат с неизвестным участником → 400, в `chats` 0 строк с этим title и 0 чатов без участников | `evidence/student_tests.txt` |
| 3: Basic только на /auth/sessions, Bearer, X-User-Id игнорируется, logout, KDF | `auth_test.py`, smoke, `db_integrity_test.py` | Basic/X-User-Id на бизнес-endpoint → 401; повторный logout → 401; hash начинается с `$argon2id$` | `evidence/student_tests.txt` |
| 3: secrets/volumes не в Git | `git status`, `.gitignore` (`secrets/`, `.env`) | секреты только в `./secrets`, в репозитории нет | `.gitignore` |
| 4: Valkey реально используется, TTL, инвалидация | `dependency_test.py` | ключи `session:*` с TTL ≤ 300, после logout значение `revoked` | `evidence/student_tests.txt` |
| 4: параметризованные запросы, ограниченный pool, таймауты | код `app/storage.py`, `app/config.py` | `$n`-параметры, `max_size=10`, таймауты 3/3/5 с | — |
| 4: гонка регистрации | `db_integrity_test.py` | 20 параллельных запросов → 1×201, 19×409, 1 строка | `evidence/student_tests.txt` |
| 4: rollback составной операции | `db_integrity_test.py` | см. строку «атомарное создание чата» | `evidence/student_tests.txt` |
| 4: индекс истории + EXPLAIN на ≥10 000 | `make explain` | 70 000 сообщений, в чате 20 000; с индексом Index Scan, 4 буфера, 0.03 мс на первой и глубокой (#15000) странице; без индекса Seq Scan + Sort, 940 буферов, 3–4 мс | `evidence/explain_history.txt` |
| 4: expiry при коротком TTL | `auth_test.py` против `api-short-ttl` (TTL=2) | через 3 с → 401 (и из кэша) | `evidence/student_tests.txt` |
| 4: logout, неверный пароль | `auth_test.py` | неверный пароль и неизвестный user дают одинаковый 401 | `evidence/student_tests.txt` |
| 4: недоступность Redis/БД | `dependency_test.py` | Valkey stop: ready=200, запросы ~4 мс; PostgreSQL stop: live=200, ready и бизнес-запросы 503 мгновенно, непроверенный токен → 503, без токена 401; после start всё восстанавливается без рестарта API | `evidence/student_tests.txt` |
| Graceful shutdown (из lab1) | `sigterm_test.sh` | `docker compose stop api` → exit 0 за < 1 с | `evidence/student_tests.txt` |
## Эксперименты и отказы
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
| --- | --- | --- | --- |
| TODO | TODO | TODO | TODO |
| `scripts/restart.sh`: stop api, valkey, postgres → up | данные и сессия на месте, токен тот же | PASS, кэш пустой, сессии берутся из PostgreSQL | ~11 с весь hook / потерь нет |
| `docker compose stop valkey` | API работает без кэша | 200, ~4 мс на запрос (breaker), revoked-токен 401 | мгновенно / потерь нет (кэш) |
| `docker compose stop postgres` | 503 без зависаний, анонимный доступ не появляется | ready 503, бизнес 503, live 200, всё за < 0.1 с | API восстанавливается сам после `start postgres`, без своего рестарта; подтверждённые данные целы |
| SIGTERM api | graceful exit 0 | exit 0, «Application shutdown complete» | `docker compose up -d --wait api` |
## Ограничения и следующие шаги
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
- Один Docker-host, по одному PostgreSQL и Valkey, без репликации: потеря host или диска = потеря сервиса (бэкапы не настроены, это трек 2 пятёрки).
- Если logout пришёлся на момент сетевой недоступности Valkey, а сам Valkey при этом не перезапускался, старая запись в кэше может жить до `SESSION_CACHE_MAX_SECONDS` (≤ 5 мин). Это осознанная граница: TTL кэша ограничен именно поэтому.
- Просроченные строки `sessions` не удаляются автоматически (есть индекс `expires_at` для будущей периодической очистки).
- `/health/ready` не проверяет Valkey намеренно.
- Тело запроса проверяется до аутентификации: на неавторизованный запрос с битым JSON вернётся 400, а не 401 (особенность FastAPI). Это не раскрывает данных.
- `make test` создаёт синтетических пользователей при каждом запуске; удаления в API нет.
- Нагрузочное измерение (`make load`) не делалось: для уровня 4 не требуется.
View File
+122
View File
@@ -0,0 +1,122 @@
"""Passwords (Argon2id), Basic login and opaque Bearer sessions."""
from __future__ import annotations
import asyncio
import base64
import binascii
import hashlib
import secrets
from datetime import datetime, timezone
from typing import Optional, Tuple
from argon2 import PasswordHasher
from argon2.exceptions import InvalidHashError, VerificationError
from starlette.concurrency import run_in_threadpool
from app import config
from app.cache import REVOKED, SessionCache
from app.errors import BASIC_CHALLENGE, unauthorized
from app.storage import Store, fmt_ts, user_row
# argon2-cffi defaults = RFC 9106 low-memory profile (Argon2id, t=3, m=64 MiB, p=4).
_hasher = PasswordHasher()
_hash_slots = asyncio.Semaphore(config.PASSWORD_HASH_CONCURRENCY)
# Verified when the username does not exist, so both failures cost the same time.
_DUMMY_HASH = _hasher.hash("dummy-password-for-timing")
async def hash_password(password: str) -> str:
async with _hash_slots:
return await run_in_threadpool(_hasher.hash, password)
async def verify_password(password_hash: str, password: str) -> bool:
def verify() -> bool:
try:
return _hasher.verify(password_hash, password)
except (VerificationError, InvalidHashError):
return False
async with _hash_slots:
return await run_in_threadpool(verify)
def token_digest(token: str) -> bytes:
# A fast hash is enough here: the token already carries 256 random bits,
# unlike a human password that needs a slow KDF against guessing.
return hashlib.sha256(token.encode("ascii")).digest()
def parse_basic(header: Optional[str]) -> Tuple[str, str]:
bad = unauthorized("invalid credentials", BASIC_CHALLENGE)
if not header:
raise bad
scheme, _, value = header.partition(" ")
if scheme.lower() != "basic" or not value.strip():
raise bad
try:
decoded = base64.b64decode(value.strip(), validate=True).decode("utf-8")
except (binascii.Error, UnicodeDecodeError):
raise bad from None
username, sep, password = decoded.partition(":") # the password may contain ':'
if not sep or not username.isascii() or not 12 <= len(password) <= 128:
raise bad
return username, password
def parse_bearer(header: Optional[str]) -> str:
if not header:
raise unauthorized("bearer token required")
scheme, _, token = header.partition(" ")
token = token.strip()
if scheme.lower() != "bearer" or not token or not token.isascii() or len(token) > 256:
raise unauthorized("bearer token required")
return token
async def login(store: Store, header: Optional[str]) -> dict:
username, password = parse_basic(header)
row = await store.get_credentials(username)
if row is None:
await verify_password(_DUMMY_HASH, password)
raise unauthorized("invalid credentials", BASIC_CHALLENGE)
if not await verify_password(row["password_hash"], password):
raise unauthorized("invalid credentials", BASIC_CHALLENGE)
token = secrets.token_urlsafe(32) # 256 bits of entropy
expires_at = await store.create_session(token_digest(token), row["id"], config.SESSION_TTL_SECONDS)
return {"token": token, "token_type": "Bearer", "expires_at": fmt_ts(expires_at), "user": user_row(row)}
async def authenticate(store: Store, cache: SessionCache, header: Optional[str]) -> dict:
"""Return the session's user or raise 401. Cache first, PostgreSQL on a miss.
If PostgreSQL is down and the cache has no answer the error propagates as 503:
an unverifiable token is never treated as authenticated."""
digest = token_digest(parse_bearer(header))
cached = await cache.get(digest)
if cached == REVOKED:
raise unauthorized("session expired or revoked")
if isinstance(cached, dict):
expires_at = datetime.fromisoformat(cached["expires_at"].replace("Z", "+00:00"))
if expires_at > datetime.now(timezone.utc):
return cached["user"]
raise unauthorized("session expired or revoked")
row = await store.find_session(digest)
if row is None:
raise unauthorized("session expired or revoked")
user = user_row(row)
remaining = int((row["expires_at"] - datetime.now(timezone.utc)).total_seconds())
await cache.put(digest, {"user": user, "expires_at": fmt_ts(row["expires_at"])}, remaining)
return user
async def logout(store: Store, cache: SessionCache, header: Optional[str]) -> None:
# Authenticate first so an already revoked token gets 401, as the contract requires.
await authenticate(store, cache, header)
digest = token_digest(parse_bearer(header))
expires_at = await store.revoke_session(digest) # PostgreSQL first: it is the truth
if expires_at is None:
raise unauthorized("session expired or revoked")
remaining = int((expires_at - datetime.now(timezone.utc)).total_seconds()) + 1
await cache.revoke(digest, remaining)
+102
View File
@@ -0,0 +1,102 @@
"""Valkey session cache. Never the source of truth: every failure degrades to a miss.
Keys: session:<sha256(token) hex>
Value: JSON {"user": {...}, "expires_at": "..."} - a valid session
"revoked" - tombstone written by logout
Entry TTL = min(time left in the session, SESSION_CACHE_MAX_SECONDS).
"""
from __future__ import annotations
import json
import logging
import time
from typing import Optional, Union
import redis.asyncio as redis
from redis.exceptions import RedisError
from app import config
logger = logging.getLogger("messenger.cache")
REVOKED = "revoked"
class SessionCache:
def __init__(self, url: Optional[str]):
self._client: Optional[redis.Redis] = None
self._down_until = 0.0
if url:
pool = redis.BlockingConnectionPool.from_url(
url,
max_connections=config.CACHE_POOL_MAX,
timeout=config.CACHE_TIMEOUT,
socket_timeout=config.CACHE_TIMEOUT,
socket_connect_timeout=config.CACHE_TIMEOUT,
decode_responses=True,
)
self._client = redis.Redis(connection_pool=pool)
@property
def enabled(self) -> bool:
return self._client is not None
def _usable(self) -> bool:
return self._client is not None and time.monotonic() >= self._down_until
def _failed(self, op: str, exc: Exception) -> None:
# Circuit breaker: one slow failure, then skip the cache for a few seconds.
self._down_until = time.monotonic() + config.CACHE_RETRY_AFTER
logger.warning("cache %s failed (%s); bypassing cache for %.0fs", op, type(exc).__name__, config.CACHE_RETRY_AFTER)
@staticmethod
def _key(token_hash: bytes) -> str:
return "session:" + token_hash.hex()
async def get(self, token_hash: bytes) -> Union[dict, str, None]:
if not self._usable():
return None
try:
raw = await self._client.get(self._key(token_hash))
except (RedisError, OSError) as exc:
self._failed("get", exc)
return None
if raw is None or raw == REVOKED:
return raw
try:
return json.loads(raw)
except ValueError:
return None
async def put(self, token_hash: bytes, entry: dict, ttl_seconds: int) -> None:
"""Fill after a PostgreSQL read. NX: never overwrite a logout tombstone."""
ttl = min(ttl_seconds, config.SESSION_CACHE_MAX_SECONDS)
if ttl <= 0 or not self._usable():
return
try:
await self._client.set(self._key(token_hash), json.dumps(entry), ex=ttl, nx=True)
except (RedisError, OSError) as exc:
self._failed("set", exc)
async def revoke(self, token_hash: bytes, ttl_seconds: int) -> None:
"""Tombstone instead of DEL: a concurrent fill that read PostgreSQL before the
revoke cannot re-insert the session afterwards (its SET NX loses).
Tried even while the breaker is open: logout is rare and correctness matters more."""
if self._client is None:
return
try:
await self._client.set(self._key(token_hash), REVOKED, ex=max(1, min(ttl_seconds, config.SESSION_CACHE_MAX_SECONDS)))
except (RedisError, OSError) as exc:
self._failed("revoke", exc)
async def ping(self) -> bool:
if self._client is None:
return False
try:
return bool(await self._client.ping())
except (RedisError, OSError):
return False
async def close(self) -> None:
if self._client is not None:
await self._client.aclose()
+46
View File
@@ -0,0 +1,46 @@
"""Runtime configuration from environment; secrets are read from *_FILE paths."""
from __future__ import annotations
import os
from typing import Optional
def _secret(name: str) -> Optional[str]:
"""NAME wins over NAME_FILE; NAME_FILE keeps the value out of `docker inspect`."""
value = os.environ.get(name)
if value:
return value.strip()
path = os.environ.get(name + "_FILE")
if path and os.path.exists(path):
with open(path, encoding="utf-8") as fh:
return fh.read().strip() or None
return None
def _int(name: str, default: int) -> int:
return int(os.environ.get(name, default))
def _float(name: str, default: float) -> float:
return float(os.environ.get(name, default))
DATABASE_URL = _secret("DATABASE_URL")
CACHE_URL = _secret("CACHE_URL")
SESSION_TTL_SECONDS = _int("SESSION_TTL_SECONDS", 3600)
# Upper bound for one cached session entry; PostgreSQL is re-checked after it.
SESSION_CACHE_MAX_SECONDS = _int("SESSION_CACHE_MAX_SECONDS", 300)
DB_POOL_MAX = _int("DB_POOL_MAX", 10)
DB_CONNECT_TIMEOUT = _float("DB_CONNECT_TIMEOUT", 3)
DB_ACQUIRE_TIMEOUT = _float("DB_ACQUIRE_TIMEOUT", 3)
DB_COMMAND_TIMEOUT = _float("DB_COMMAND_TIMEOUT", 5)
CACHE_TIMEOUT = _float("CACHE_TIMEOUT", 0.3)
CACHE_POOL_MAX = _int("CACHE_POOL_MAX", 20)
# After a cache error, skip the cache for this long instead of paying the timeout per request.
CACHE_RETRY_AFTER = _float("CACHE_RETRY_AFTER", 5)
# Argon2id hashing is CPU/RAM heavy: bound how many run at once.
PASSWORD_HASH_CONCURRENCY = _int("PASSWORD_HASH_CONCURRENCY", 4)
+24
View File
@@ -0,0 +1,24 @@
"""API error type shared by all layers."""
from __future__ import annotations
from typing import Dict, Optional
BEARER_CHALLENGE = 'Bearer realm="messenger"'
BASIC_CHALLENGE = 'Basic realm="messenger", charset="UTF-8"'
class ApiError(Exception):
def __init__(self, status_code: int, code: str, message: str, headers: Optional[Dict[str, str]] = None):
super().__init__(message)
self.status_code = status_code
self.code = code
self.message = message
self.headers = headers or {}
def unauthorized(message: str, challenge: str = BEARER_CHALLENGE) -> ApiError:
return ApiError(401, "unauthorized", message, {"WWW-Authenticate": challenge})
def unavailable() -> ApiError:
return ApiError(503, "unavailable", "service temporarily unavailable")
+234
View File
@@ -0,0 +1,234 @@
"""Lab 2: messenger API on PostgreSQL with Bearer sessions and a Valkey session cache."""
from __future__ import annotations
import logging
import uuid
from contextlib import asynccontextmanager
from typing import Optional
import asyncpg
from fastapi import Depends, FastAPI, Query, Request, Response
from fastapi.exceptions import RequestValidationError
from fastapi.responses import JSONResponse
from starlette.exceptions import HTTPException as StarletteHTTPException
from app import auth, config
from app.cache import SessionCache
from app.errors import ApiError
from app.models import Chat, ChatPage, CreateChat, CreateMessage, CreateUser, Health, Message, MessagePage, Session, User
from app.storage import Store
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger("messenger")
store = Store()
cache = SessionCache(config.CACHE_URL)
@asynccontextmanager
async def lifespan(_app: FastAPI):
if not config.DATABASE_URL:
raise RuntimeError("DATABASE_URL or DATABASE_URL_FILE is required")
await store.open()
logger.info("session cache: %s", "valkey" if cache.enabled else "disabled")
yield
await cache.close()
await store.close()
app = FastAPI(title="MTUSI Messenger — lab2", lifespan=lifespan)
# ---------------------------------------------------------------------------
# Cross-cutting: request id + uniform error envelope
# ---------------------------------------------------------------------------
# PostgreSQL unreachable, restarting or pool exhausted -> bounded 503, never a hang.
DB_UNAVAILABLE = (
OSError, # includes ConnectionRefusedError and TimeoutError (pool acquire / connect)
asyncpg.PostgresConnectionError,
asyncpg.InterfaceError,
asyncpg.TooManyConnectionsError,
asyncpg.exceptions.OperatorInterventionError, # admin shutdown, cannot connect now
asyncpg.exceptions.QueryCanceledError, # statement timeout
)
def error_body(code: str, message: str, request_id: str) -> dict:
return {"error": {"code": code, "message": message, "request_id": request_id}}
def error_response(request: Request, status: int, code: str, message: str, headers: Optional[dict] = None) -> JSONResponse:
request_id = getattr(request.state, "request_id", "")
return JSONResponse(status_code=status, content=error_body(code, message, request_id), headers=headers)
@app.middleware("http")
async def request_id_middleware(request: Request, call_next):
request_id = uuid.uuid4().hex
request.state.request_id = request_id
try:
response = await call_next(request)
except Exception:
# Unhandled errors must be answered here: Starlette's own 500 handler runs
# outside this middleware and would drop the X-Request-Id header.
logger.exception("unhandled error for request_id=%s", request_id)
response = JSONResponse(status_code=500, content=error_body("internal_error", "internal error", request_id))
response.headers["X-Request-Id"] = request_id
return response
@app.exception_handler(ApiError)
async def api_error_handler(request: Request, exc: ApiError):
return error_response(request, exc.status_code, exc.code, exc.message, exc.headers)
@app.exception_handler(RequestValidationError)
async def validation_error_handler(request: Request, exc: RequestValidationError):
return error_response(request, 400, "invalid_request", "request failed validation")
@app.exception_handler(StarletteHTTPException)
async def http_exception_handler(request: Request, exc: StarletteHTTPException):
code = {404: "not_found", 405: "invalid_request", 415: "unsupported_media_type"}.get(exc.status_code, "invalid_request")
message = exc.detail if isinstance(exc.detail, str) else "request failed"
return error_response(request, exc.status_code, code, message)
async def db_unavailable_handler(request: Request, exc: Exception):
logger.warning("database unavailable for request_id=%s: %s", request.state.request_id, type(exc).__name__)
return error_response(request, 503, "unavailable", "service temporarily unavailable")
for _exc in DB_UNAVAILABLE:
app.add_exception_handler(_exc, db_unavailable_handler)
# ---------------------------------------------------------------------------
# Auth: Basic only on POST /auth/sessions, Bearer everywhere else.
# X-User-Id is ignored since lab2.
# ---------------------------------------------------------------------------
async def current_user(request: Request) -> dict:
return await auth.authenticate(store, cache, request.headers.get("Authorization"))
def parse_uuid(value: str, field: str) -> uuid.UUID:
try:
return uuid.UUID(value)
except ValueError:
raise ApiError(400, "invalid_request", f"{field} is not a valid UUID") from None
def not_found() -> ApiError:
return ApiError(404, "not_found", "chat not found")
# ---------------------------------------------------------------------------
# Health
# ---------------------------------------------------------------------------
@app.get("/health/live", response_model=Health)
async def liveness():
return Health()
@app.get("/health/ready", response_model=Health)
async def readiness():
# PostgreSQL is critical; Valkey is not (the API bypasses the cache without it).
try:
await store.ping()
except DB_UNAVAILABLE:
raise ApiError(503, "unavailable", "database is not available") from None
return Health()
# ---------------------------------------------------------------------------
# Users and sessions
# ---------------------------------------------------------------------------
@app.post("/api/v1/users", response_model=User, status_code=201)
async def register_user(body: CreateUser):
if not body.display_name.strip() or not body.password.strip():
raise ApiError(400, "invalid_request", "display_name and password must not be blank")
password_hash = await auth.hash_password(body.password)
return await store.create_user(body.username, body.display_name, password_hash)
@app.get("/api/v1/users/me", response_model=User)
async def current_user_endpoint(user: dict = Depends(current_user)):
return user
@app.post("/api/v1/auth/sessions", response_model=Session, status_code=201)
async def create_session(request: Request):
return await auth.login(store, request.headers.get("Authorization"))
@app.delete("/api/v1/auth/sessions/current", status_code=204)
async def delete_session(request: Request):
await auth.logout(store, cache, request.headers.get("Authorization"))
return Response(status_code=204)
# ---------------------------------------------------------------------------
# Chats
# ---------------------------------------------------------------------------
@app.post("/api/v1/chats", response_model=Chat, status_code=201)
async def create_chat(body: CreateChat, user: dict = Depends(current_user)):
if not body.title.strip():
raise ApiError(400, "invalid_request", "title must not be blank")
if len(set(body.member_ids)) != len(body.member_ids):
raise ApiError(400, "invalid_request", "member_ids must not contain duplicates")
if uuid.UUID(user["id"]) not in body.member_ids:
raise ApiError(400, "invalid_request", "caller must be a member of the chat")
return await store.create_chat(body.title, list(body.member_ids))
@app.get("/api/v1/chats", response_model=ChatPage)
async def list_chats(
user: dict = Depends(current_user),
limit: int = Query(default=50, ge=1, le=100),
cursor: Optional[str] = Query(default=None, min_length=1),
):
return await store.list_chats(uuid.UUID(user["id"]), limit, cursor)
@app.get("/api/v1/chats/{chat_id}", response_model=Chat)
async def get_chat(chat_id: str, user: dict = Depends(current_user)):
chat = await store.get_chat_for_member(parse_uuid(chat_id, "chat_id"), uuid.UUID(user["id"]))
if chat is None:
raise not_found()
return chat
# ---------------------------------------------------------------------------
# Messages
# ---------------------------------------------------------------------------
@app.post("/api/v1/chats/{chat_id}/messages", response_model=Message, status_code=201)
async def create_message(chat_id: str, body: CreateMessage, user: dict = Depends(current_user)):
chat_uuid = parse_uuid(chat_id, "chat_id")
if not body.text.strip():
# Access is checked before content: an outsider gets 404 even for a blank text.
if not await store.is_member(chat_uuid, uuid.UUID(user["id"])):
raise not_found()
raise ApiError(400, "invalid_request", "text must not be blank")
message = await store.create_message(chat_uuid, uuid.UUID(user["id"]), body.text)
if message is None:
raise not_found()
return message
@app.get("/api/v1/chats/{chat_id}/messages", response_model=MessagePage)
async def list_messages(
chat_id: str,
user: dict = Depends(current_user),
limit: int = Query(default=50, ge=1, le=100),
cursor: Optional[str] = Query(default=None, min_length=1),
):
chat_uuid = parse_uuid(chat_id, "chat_id")
if not await store.is_member(chat_uuid, uuid.UUID(user["id"])):
raise not_found()
return await store.list_messages(chat_uuid, limit, cursor)
+68
View File
@@ -0,0 +1,68 @@
"""Apply versioned SQL migrations from migrations/; safe to run repeatedly and concurrently.
Usage: python -m app.migrate
"""
from __future__ import annotations
import asyncio
import logging
import sys
from pathlib import Path
import asyncpg
from app import config
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(name)s: %(message)s")
logger = logging.getLogger("migrate")
MIGRATIONS_DIR = Path(__file__).resolve().parents[1] / "migrations"
# Arbitrary constant: serialises concurrent runners (e.g. several API replicas in lab3).
LOCK_ID = 2026_0002
async def connect_with_retry(attempts: int = 30) -> asyncpg.Connection:
for attempt in range(1, attempts + 1):
try:
return await asyncpg.connect(config.DATABASE_URL, timeout=config.DB_CONNECT_TIMEOUT)
except (OSError, asyncpg.PostgresError, asyncpg.InterfaceError) as exc:
logger.warning("database not reachable (attempt %d/%d): %s", attempt, attempts, type(exc).__name__)
await asyncio.sleep(1)
raise SystemExit("database is not reachable")
async def migrate() -> int:
if not config.DATABASE_URL:
raise SystemExit("DATABASE_URL or DATABASE_URL_FILE is required")
conn = await connect_with_retry()
applied_now = 0
try:
await conn.execute("SELECT pg_advisory_lock($1)", LOCK_ID)
await conn.execute(
"""
CREATE TABLE IF NOT EXISTS schema_migrations (
version text PRIMARY KEY,
applied_at timestamptz NOT NULL DEFAULT now()
)
"""
)
done = {r["version"] for r in await conn.fetch("SELECT version FROM schema_migrations")}
for path in sorted(MIGRATIONS_DIR.glob("*.sql")):
version = path.stem
if version in done:
continue
logger.info("applying %s", version)
async with conn.transaction():
await conn.execute(path.read_text(encoding="utf-8"))
await conn.execute("INSERT INTO schema_migrations (version) VALUES ($1)", version)
applied_now += 1
logger.info("migrations up to date: %d applied now, %d before", applied_now, len(done))
finally:
await conn.execute("SELECT pg_advisory_unlock($1)", LOCK_ID)
await conn.close()
return applied_now
if __name__ == "__main__":
asyncio.run(migrate())
sys.exit(0)
+73
View File
@@ -0,0 +1,73 @@
"""Pydantic request/response schemas mirroring contracts/openapi.json."""
from __future__ import annotations
import uuid
from typing import List, Optional
from pydantic import BaseModel, ConfigDict, Field
USERNAME_PATTERN = r"^[a-z0-9_]{3,32}$"
class CreateUser(BaseModel):
model_config = ConfigDict(extra="forbid")
username: str = Field(pattern=USERNAME_PATTERN)
display_name: str = Field(min_length=1, max_length=100)
password: str = Field(min_length=12, max_length=128)
class User(BaseModel):
id: uuid.UUID
username: str
display_name: str
created_at: str
class CreateChat(BaseModel):
model_config = ConfigDict(extra="forbid")
title: str = Field(min_length=1, max_length=100)
member_ids: List[uuid.UUID] = Field(min_length=1, max_length=100)
class Chat(BaseModel):
id: uuid.UUID
title: str
member_ids: List[uuid.UUID]
created_at: str
class ChatPage(BaseModel):
items: List[Chat]
next_cursor: Optional[str] = None
class CreateMessage(BaseModel):
model_config = ConfigDict(extra="forbid")
text: str = Field(max_length=4000)
class Message(BaseModel):
id: uuid.UUID
chat_id: uuid.UUID
sender_id: uuid.UUID
text: str
created_at: str
class MessagePage(BaseModel):
items: List[Message]
next_cursor: Optional[str] = None
class Session(BaseModel):
token: str
token_type: str = "Bearer"
expires_at: str
user: User
class Health(BaseModel):
status: str = "ok"
+288
View File
@@ -0,0 +1,288 @@
"""PostgreSQL repository. Every query is parameterised; PostgreSQL is the source of truth."""
from __future__ import annotations
import base64
import json
import uuid
from datetime import datetime, timezone
from typing import List, Optional, Tuple
import asyncpg
from app import config
from app.errors import ApiError
def fmt_ts(value: datetime) -> str:
# Always 6 fractional digits: exact round trip with PostgreSQL microseconds.
return value.astimezone(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S.%fZ")
def user_row(row: asyncpg.Record) -> dict:
return {
"id": str(row["id"]),
"username": row["username"],
"display_name": row["display_name"],
"created_at": fmt_ts(row["created_at"]),
}
def chat_row(row: asyncpg.Record) -> dict:
return {
"id": str(row["id"]),
"title": row["title"],
"member_ids": [str(m) for m in row["member_ids"]],
"created_at": fmt_ts(row["created_at"]),
}
def message_row(row: asyncpg.Record) -> dict:
return {
"id": str(row["id"]),
"chat_id": str(row["chat_id"]),
"sender_id": str(row["sender_id"]),
"text": row["text"],
"created_at": fmt_ts(row["created_at"]),
}
# ---------------------------------------------------------------------------
# Opaque keyset cursor bound to one resource
# ---------------------------------------------------------------------------
def encode_cursor(resource: str, created_at: str, item_id: str) -> str:
payload = json.dumps({"r": resource, "t": created_at, "i": item_id}, separators=(",", ":"))
return base64.urlsafe_b64encode(payload.encode("utf-8")).decode("ascii").rstrip("=")
def decode_cursor(resource: str, cursor: str) -> Tuple[datetime, uuid.UUID]:
try:
padded = cursor + "=" * (-len(cursor) % 4)
payload = json.loads(base64.urlsafe_b64decode(padded.encode("ascii")).decode("utf-8"))
if payload.get("r") != resource:
raise ValueError("cursor belongs to a different resource")
created_at = datetime.fromisoformat(payload["t"].replace("Z", "+00:00"))
if created_at.tzinfo is None:
raise ValueError("cursor time has no timezone")
return created_at, uuid.UUID(payload["i"])
except Exception as exc: # noqa: BLE001 - any malformed cursor is a 400
raise ApiError(400, "invalid_request", "invalid cursor") from exc
def page_of(items: List[dict], resource: str, limit: int) -> dict:
"""Queries fetch limit+1 rows: the extra row only tells whether a next page exists."""
page = items[:limit]
next_cursor = None
if len(items) > limit:
last = page[-1]
next_cursor = encode_cursor(resource, last["created_at"], last["id"])
return {"items": page, "next_cursor": next_cursor}
# ---------------------------------------------------------------------------
# Repository
# ---------------------------------------------------------------------------
class Store:
def __init__(self):
self.pool: Optional[asyncpg.Pool] = None
async def open(self) -> None:
# min_size=0: connections open on demand, so the API starts even while
# PostgreSQL is down and readiness reports 503 until it is back.
self.pool = await asyncpg.create_pool(
config.DATABASE_URL,
min_size=0,
max_size=config.DB_POOL_MAX,
timeout=config.DB_CONNECT_TIMEOUT,
command_timeout=config.DB_COMMAND_TIMEOUT,
max_inactive_connection_lifetime=60,
)
async def close(self) -> None:
if self.pool is not None:
await self.pool.close()
def acquire(self):
return self.pool.acquire(timeout=config.DB_ACQUIRE_TIMEOUT)
async def ping(self) -> bool:
async with self.acquire() as conn:
return await conn.fetchval("SELECT 1") == 1
# -- users --------------------------------------------------------------
async def create_user(self, username: str, display_name: str, password_hash: str) -> dict:
async with self.acquire() as conn:
try:
row = await conn.fetchrow(
"""
INSERT INTO users (id, username, display_name, password_hash)
VALUES ($1, $2, $3, $4)
RETURNING id, username, display_name, created_at
""",
uuid.uuid4(), username, display_name, password_hash,
)
except asyncpg.UniqueViolationError:
# The UNIQUE constraint, not a prior SELECT, decides concurrent registrations.
raise ApiError(409, "conflict", "username already taken") from None
return user_row(row)
async def get_credentials(self, username: str) -> Optional[asyncpg.Record]:
async with self.acquire() as conn:
return await conn.fetchrow(
"SELECT id, username, display_name, created_at, password_hash FROM users WHERE username = $1",
username,
)
# -- sessions -----------------------------------------------------------
async def create_session(self, token_hash: bytes, user_id: uuid.UUID, ttl_seconds: int) -> datetime:
async with self.acquire() as conn:
return await conn.fetchval(
"""
INSERT INTO sessions (token_hash, user_id, expires_at)
VALUES ($1, $2, clock_timestamp() + make_interval(secs => $3))
RETURNING expires_at
""",
token_hash, user_id, float(ttl_seconds),
)
async def find_session(self, token_hash: bytes) -> Optional[asyncpg.Record]:
"""Only a live session: not revoked and not expired by the database clock."""
async with self.acquire() as conn:
return await conn.fetchrow(
"""
SELECT s.expires_at, u.id, u.username, u.display_name, u.created_at
FROM sessions s JOIN users u ON u.id = s.user_id
WHERE s.token_hash = $1 AND s.revoked_at IS NULL AND s.expires_at > clock_timestamp()
""",
token_hash,
)
async def revoke_session(self, token_hash: bytes) -> Optional[datetime]:
async with self.acquire() as conn:
return await conn.fetchval(
"""
UPDATE sessions SET revoked_at = clock_timestamp()
WHERE token_hash = $1 AND revoked_at IS NULL AND expires_at > clock_timestamp()
RETURNING expires_at
""",
token_hash,
)
# -- chats --------------------------------------------------------------
async def create_chat(self, title: str, member_ids: List[uuid.UUID]) -> dict:
"""Chat and all memberships commit together or not at all.
Unknown members are detected by the foreign key inside the transaction, so a
failure after the chat row was inserted really exercises the rollback."""
chat_id = uuid.uuid4()
async with self.acquire() as conn:
try:
async with conn.transaction():
row = await conn.fetchrow(
"INSERT INTO chats (id, title) VALUES ($1, $2) RETURNING id, title, created_at",
chat_id, title,
)
await conn.execute(
"""
INSERT INTO chat_members (chat_id, user_id, position)
SELECT $1, m.user_id, m.ord - 1
FROM unnest($2::uuid[]) WITH ORDINALITY AS m(user_id, ord)
""",
chat_id, member_ids,
)
except asyncpg.ForeignKeyViolationError:
raise ApiError(400, "invalid_request", "unknown member in member_ids") from None
return {
"id": str(row["id"]),
"title": row["title"],
"member_ids": [str(m) for m in member_ids],
"created_at": fmt_ts(row["created_at"]),
}
async def get_chat_for_member(self, chat_id: uuid.UUID, user_id: uuid.UUID) -> Optional[dict]:
async with self.acquire() as conn:
row = await conn.fetchrow(
"""
SELECT c.id, c.title, c.created_at,
array_agg(m.user_id ORDER BY m.position) AS member_ids
FROM chats c JOIN chat_members m ON m.chat_id = c.id
WHERE c.id = $1
AND EXISTS (SELECT 1 FROM chat_members me WHERE me.chat_id = c.id AND me.user_id = $2)
GROUP BY c.id
""",
chat_id, user_id,
)
return chat_row(row) if row else None
async def list_chats(self, user_id: uuid.UUID, limit: int, cursor: Optional[str]) -> dict:
after_t, after_i = decode_cursor("chats", cursor) if cursor else (None, None)
async with self.acquire() as conn:
rows = await conn.fetch(
"""
SELECT c.id, c.title, c.created_at,
array_agg(m.user_id ORDER BY m.position) AS member_ids
FROM chat_members me
JOIN chats c ON c.id = me.chat_id
JOIN chat_members m ON m.chat_id = c.id
WHERE me.user_id = $1
AND ($2::timestamptz IS NULL OR (c.created_at, c.id) > ($2, $3::uuid))
GROUP BY c.id
ORDER BY c.created_at, c.id
LIMIT $4
""",
user_id, after_t, after_i, limit + 1,
)
return page_of([chat_row(r) for r in rows], "chats", limit)
async def is_member(self, chat_id: uuid.UUID, user_id: uuid.UUID) -> bool:
async with self.acquire() as conn:
return bool(await conn.fetchval(
"SELECT EXISTS (SELECT 1 FROM chat_members WHERE chat_id = $1 AND user_id = $2)",
chat_id, user_id,
))
# -- messages -----------------------------------------------------------
async def create_message(self, chat_id: uuid.UUID, sender_id: uuid.UUID, text: str) -> Optional[dict]:
"""Membership check and insert in one statement; None means "no such chat for you"."""
async with self.acquire() as conn:
row = await conn.fetchrow(
"""
INSERT INTO messages (id, chat_id, sender_id, text)
SELECT $1, $2, $3, $4
WHERE EXISTS (SELECT 1 FROM chat_members WHERE chat_id = $2 AND user_id = $3)
RETURNING id, chat_id, sender_id, text, created_at
""",
uuid.uuid4(), chat_id, sender_id, text,
)
return message_row(row) if row else None
async def list_messages(self, chat_id: uuid.UUID, limit: int, cursor: Optional[str]) -> dict:
resource = str(chat_id)
async with self.acquire() as conn:
if cursor:
after_t, after_i = decode_cursor(resource, cursor)
rows = await conn.fetch(
"""
SELECT id, chat_id, sender_id, text, created_at FROM messages
WHERE chat_id = $1 AND (created_at, id) > ($2, $3)
ORDER BY created_at, id
LIMIT $4
""",
chat_id, after_t, after_i, limit + 1,
)
else:
rows = await conn.fetch(
"""
SELECT id, chat_id, sender_id, text, created_at FROM messages
WHERE chat_id = $1
ORDER BY created_at, id
LIMIT $2
""",
chat_id, limit + 1,
)
return page_of([message_row(r) for r in rows], resource, limit)
+100
View File
@@ -0,0 +1,100 @@
# Lab 2 stack: api -> postgres (source of truth) + valkey (session cache).
# Prepare secrets first: scripts/init-secrets.sh. Start: docker compose up -d --build --wait
name: messenger-lab2
x-api: &api
build: .
image: mtusi-messenger:lab2
environment: &api-env
APP_HOST: 0.0.0.0
APP_PORT: "8080"
APP_ENV: ${APP_ENV:-lab}
SESSION_TTL_SECONDS: ${SESSION_TTL_SECONDS:-3600}
SESSION_STORE: postgres
SESSION_CACHE_MAX_SECONDS: ${SESSION_CACHE_MAX_SECONDS:-300}
DATABASE_URL_FILE: /run/secrets/database_url
CACHE_URL_FILE: /run/secrets/cache_url
DB_POOL_MAX: ${DB_POOL_MAX:-10}
secrets: [database_url, cache_url]
depends_on:
postgres:
condition: service_healthy
valkey:
# The cache is optional for correctness: only wait for the container to start.
condition: service_started
migrate:
condition: service_completed_successfully
stop_grace_period: 10s
restart: unless-stopped
services:
postgres:
image: postgres:17.6-alpine3.22
environment:
POSTGRES_DB: messenger
POSTGRES_USER: messenger
POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password
secrets: [postgres_password]
volumes:
- pgdata:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U messenger -d messenger"]
interval: 2s
timeout: 3s
retries: 30
stop_grace_period: 30s
restart: unless-stopped
valkey:
image: valkey/valkey:8.1.4-alpine3.22
# requirepass and cache-only settings live in the secret config file.
command: ["valkey-server", "/run/secrets/valkey_conf"]
secrets: [valkey_conf]
healthcheck:
# NOAUTH also proves the server answers; the password stays out of the process list.
test: ["CMD-SHELL", "valkey-cli ping 2>&1 | grep -qE 'PONG|NOAUTH'"]
interval: 2s
timeout: 3s
retries: 30
restart: unless-stopped
migrate:
build: .
image: mtusi-messenger:lab2
command: ["python", "-m", "app.migrate"]
environment:
DATABASE_URL_FILE: /run/secrets/database_url
secrets: [database_url]
depends_on:
postgres:
condition: service_healthy
restart: "no"
api:
<<: *api
ports:
- "127.0.0.1:${APP_PORT:-8080}:8080"
# Same image with a 2-second TTL, only for the expiry test:
# docker compose --profile ttl-test up -d --wait api-short-ttl
api-short-ttl:
<<: *api
profiles: [ttl-test]
environment:
<<: *api-env
SESSION_TTL_SECONDS: "2"
ports:
- "127.0.0.1:${SHORT_TTL_PORT:-8081}:8080"
volumes:
pgdata:
secrets:
postgres_password:
file: ./secrets/postgres_password
database_url:
file: ./secrets/database_url
cache_url:
file: ./secrets/cache_url
valkey_conf:
file: ./secrets/valkey.conf
+74
View File
@@ -0,0 +1,74 @@
# EXPLAIN evidence for message history pagination
# date: 2026-10-07T14:17:55Z commit: 65003b6
# postgres: 17.6
# messages total: 70112, in measured chat: 20000
PREPARE
PREPARE
=== 1. WITH index messages_chat_history_idx: first page (limit 50 + 1) ===
QUERY PLAN
---------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=0.42..8.60 rows=51 width=78) (actual time=0.014..0.019 rows=51 loops=1)
Buffers: shared hit=4
-> Index Scan using messages_chat_history_idx on messages (cost=0.42..3227.11 rows=20122 width=78) (actual time=0.013..0.015 rows=51 loops=1)
Index Cond: (chat_id = '00000000-0000-4000-8000-000000000100'::uuid)
Buffers: shared hit=4
Planning:
Buffers: shared hit=106
Planning Time: 0.272 ms
Execution Time: 0.031 ms
(9 rows)
=== 2. WITH index: deep page, cursor after message #15000 of 20000 ===
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=0.42..55.74 rows=51 width=78) (actual time=0.012..0.017 rows=51 loops=1)
Buffers: shared hit=4
-> Index Scan using messages_chat_history_idx on messages (cost=0.42..1590.70 rows=1466 width=78) (actual time=0.012..0.014 rows=51 loops=1)
Index Cond: ((chat_id = '00000000-0000-4000-8000-000000000100'::uuid) AND (ROW(created_at, id) > ROW('2026-01-01 04:10:00+00'::timestamp with time zone, '47ba2a9c-d238-4807-2008-6ea72cb0f95c'::uuid)))
Buffers: shared hit=4
Planning:
Buffers: shared hit=42
Planning Time: 0.276 ms
Execution Time: 0.035 ms
(9 rows)
BEGIN
DROP INDEX
=== 3. WITHOUT index: first page ===
QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------
Limit (cost=2487.71..2487.84 rows=51 width=78) (actual time=4.235..4.239 rows=51 loops=1)
Buffers: shared hit=946
-> Sort (cost=2487.71..2538.02 rows=20122 width=78) (actual time=4.234..4.236 rows=51 loops=1)
Sort Key: created_at, id
Sort Method: top-N heapsort Memory: 31kB
Buffers: shared hit=946
-> Seq Scan on messages (cost=0.00..1816.40 rows=20122 width=78) (actual time=0.011..3.325 rows=20000 loops=1)
Filter: (chat_id = '00000000-0000-4000-8000-000000000100'::uuid)
Rows Removed by Filter: 50112
Buffers: shared hit=940
Planning:
Buffers: shared hit=16
Planning Time: 0.098 ms
Execution Time: 4.249 ms
(14 rows)
=== 4. WITHOUT index: deep page ===
QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=2215.87..2216.00 rows=51 width=78) (actual time=3.117..3.121 rows=51 loops=1)
Buffers: shared hit=940
-> Sort (cost=2215.87..2219.53 rows=1466 width=78) (actual time=3.116..3.118 rows=51 loops=1)
Sort Key: created_at, id
Sort Method: top-N heapsort Memory: 31kB
Buffers: shared hit=940
-> Seq Scan on messages (cost=0.00..2166.96 rows=1466 width=78) (actual time=0.605..2.853 rows=5000 loops=1)
Filter: ((chat_id = '00000000-0000-4000-8000-000000000100'::uuid) AND (ROW(created_at, id) > ROW('2026-01-01 04:10:00+00'::timestamp with time zone, '47ba2a9c-d238-4807-2008-6ea72cb0f95c'::uuid)))
Rows Removed by Filter: 65112
Buffers: shared hit=940
Planning Time: 0.069 ms
Execution Time: 3.133 ms
(12 rows)
ROLLBACK
+20
View File
@@ -0,0 +1,20 @@
# Fresh volume: migrations from scratch + public smoke
# date: 2026-10-07T14:18:43Z
$ APP_PORT=18090 docker compose -p lab2-clean up -d --wait
Container lab2-clean-api-1 Created
Container lab2-clean-api-1 Healthy
Container lab2-clean-migrate-1 Created
Container lab2-clean-migrate-1 Exited
Container lab2-clean-postgres-1 Created
Container lab2-clean-postgres-1 Healthy
Container lab2-clean-valkey-1 Created
Container lab2-clean-valkey-1 Healthy
Network lab2-clean_default Created
Volume "lab2-clean_pgdata" Created
$ docker compose -p lab2-clean logs migrate
INFO migrate: applying 0001_init
INFO migrate: migrations up to date: 1 applied now, 0 before
$ make test BASE_URL=http://localhost:18090
PASS: lab2 public API scenario; infrastructure and grading evidence remain separate.
$ docker compose -p lab2-clean run --rm migrate # second run
INFO migrate: migrations up to date: 0 applied now, 1 before
+20
View File
@@ -0,0 +1,20 @@
# Persistence: API + PostgreSQL + Valkey restarted, volume kept, same token
# date: 2026-10-07T14:19:03Z
## containers BEFORE
api 2026-10-07T14:18:25.262485419Z running restarts=0
postgres 2026-10-07T14:18:22.306271835Z running restarts=0
valkey 2026-10-07T14:18:21.888265501Z running restarts=0
volume messenger-lab2_pgdata
$ make test-persistence ACTION_SCRIPT=scripts/restart.sh
Executing explicit persistence hook; hook output suppressed to avoid leaking secrets.
PASS: persistence preserved confirmed data and existing session. Hook scope needs operator evidence.
## containers AFTER (StartedAt changed for all three)
api 2026-10-07T14:19:08.428103009Z running restarts=0
postgres 2026-10-07T14:19:05.240867716Z running restarts=0
valkey 2026-10-07T14:19:05.240850591Z running restarts=0
volume messenger-lab2_pgdata
$ docker compose logs migrate | tail -1
INFO migrate: migrations up to date: 0 applied now, 1 before
+5
View File
@@ -0,0 +1,5 @@
# date: 2026-10-07T14:19:14Z
$ make check
PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.
$ make test
PASS: lab2 public API scenario; infrastructure and grading evidence remain separate.
+44
View File
@@ -0,0 +1,44 @@
# date: 2026-10-07T14:19:14Z
$ make test-student
BASE_URL="http://localhost:18080" sh student-tests/run_all.sh
== own_tests.py
ok - malformed JSON -> 400 invalid_request
ok - foreign chat access -> 404 (no leak)
ok - empty/whitespace message text -> 400
ok - duplicate username -> 409, original user intact
ok - multi-page pagination traversal: 4 pages, 7 messages, no dup/loss
ok - cursor from another chat -> 400 (cursor is bound to its resource)
PASS: 6/6 student checks
== concurrency_test.py
PASS: 100 unique messages written and read back from 10 concurrent clients
== auth_test.py
ok - wrong password / unknown user -> identical 401 Basic
ok - missing / malformed / non-Basic login -> 401
ok - UTF-8 password containing ':' -> 201
ok - password shorter than 12 / missing -> 400
ok - Basic / X-User-Id / unknown token on business endpoint -> 401
ok - logout revokes only the current token; repeat logout -> 401
ok - session with TTL=2s is rejected after 3s (cache entry included)
PASS: 7 checks
== db_integrity_test.py
ok - 20 concurrent registrations of one username -> 1x201, 19x409, 1 row
ok - chat with unknown member -> 400 and transaction rolled back (no orphan chat)
ok - password stored as $argon2id$..., sessions keep only sha256(token)
ok - migrations run twice more: schema_migrations and data unchanged (1:49)
PASS: 4 checks
== dependency_test.py
ok - Valkey caches sessions with TTL<=300s and keeps logout tombstones
ok - Valkey stopped: ready=200, reads/writes OK, revoked token 401, 4 ms/request
ok - PostgreSQL stopped: live=200, ready/business=503 in 0.0s total, unverified token not trusted
ok - PostgreSQL started again: API recovers without restart, data intact
PASS: 4 checks
== sigterm_test.sh
api stopped in 0s, exit code 0 (0 = graceful, 137 = SIGKILL)
PASS: graceful SIGTERM shutdown, api restarted
ALL STUDENT TESTS PASSED
+58
View File
@@ -0,0 +1,58 @@
-- Initial schema for lab2. Applied once by app/migrate.py inside one transaction.
CREATE TABLE users (
id uuid PRIMARY KEY,
username text NOT NULL,
display_name text NOT NULL,
password_hash text NOT NULL,
created_at timestamptz NOT NULL DEFAULT clock_timestamp(),
CONSTRAINT users_username_key UNIQUE (username),
CONSTRAINT users_username_format CHECK (username ~ '^[a-z0-9_]{3,32}$'),
CONSTRAINT users_display_name_len CHECK (char_length(display_name) BETWEEN 1 AND 100)
);
CREATE TABLE chats (
id uuid PRIMARY KEY,
title text NOT NULL,
created_at timestamptz NOT NULL DEFAULT clock_timestamp(),
CONSTRAINT chats_title_len CHECK (char_length(title) BETWEEN 1 AND 100)
);
-- position keeps member_ids in the order the creator sent them.
CREATE TABLE chat_members (
chat_id uuid NOT NULL REFERENCES chats (id),
user_id uuid NOT NULL REFERENCES users (id),
position smallint NOT NULL,
PRIMARY KEY (chat_id, user_id)
);
-- "Chats of a user": GET /chats starts from the caller's memberships.
CREATE INDEX chat_members_user_idx ON chat_members (user_id, chat_id);
CREATE TABLE messages (
id uuid PRIMARY KEY,
chat_id uuid NOT NULL,
sender_id uuid NOT NULL,
text text NOT NULL,
created_at timestamptz NOT NULL DEFAULT clock_timestamp(),
-- The sender must be a member of the chat, enforced by the database itself.
CONSTRAINT messages_sender_member_fk FOREIGN KEY (chat_id, sender_id)
REFERENCES chat_members (chat_id, user_id),
CONSTRAINT messages_text_len CHECK (char_length(text) BETWEEN 1 AND 4000)
);
-- Keyset pagination of history: WHERE chat_id = $1 AND (created_at, id) > ($2, $3)
-- ORDER BY created_at, id LIMIT n. Justified by EXPLAIN in evidence/explain_history.txt.
CREATE INDEX messages_chat_history_idx ON messages (chat_id, created_at, id);
-- Sessions: PostgreSQL is the source of truth; Valkey only caches lookups.
-- token_hash = sha256(token); the raw token is never stored.
CREATE TABLE sessions (
token_hash bytea PRIMARY KEY,
user_id uuid NOT NULL REFERENCES users (id),
created_at timestamptz NOT NULL DEFAULT clock_timestamp(),
expires_at timestamptz NOT NULL,
revoked_at timestamptz
);
CREATE INDEX sessions_expires_idx ON sessions (expires_at);
+6
View File
@@ -0,0 +1,6 @@
fastapi==0.142.2
uvicorn[standard]==0.54.0
pydantic==2.13.5
asyncpg==0.32.0
redis==8.1.0
argon2-cffi==25.1.0
+32
View File
@@ -0,0 +1,32 @@
#!/bin/sh
# Generate local synthetic credentials into ./secrets (git-ignored). Idempotent:
# existing files are kept, so a running stack keeps working. Prints no secret values.
set -eu
cd "$(dirname "$0")/.."
mkdir -p secrets
gen() { python3 -c 'import secrets; print(secrets.token_hex(24))'; }
if [ ! -f secrets/postgres_password ]; then
gen > secrets/postgres_password
echo "created secrets/postgres_password"
fi
PG_PASS=$(cat secrets/postgres_password)
printf 'postgresql://messenger:%s@postgres:5432/messenger\n' "$PG_PASS" > secrets/database_url
if [ ! -f secrets/valkey.conf ]; then
{
printf 'requirepass %s\n' "$(gen)"
# Pure cache: PostgreSQL holds the sessions, so no RDB/AOF is needed.
printf 'save ""\nappendonly no\n'
printf 'maxmemory 64mb\nmaxmemory-policy volatile-lru\n'
} > secrets/valkey.conf
echo "created secrets/valkey.conf"
fi
VK_PASS=$(sed -n 's/^requirepass //p' secrets/valkey.conf)
printf 'redis://:%s@valkey:6379/0\n' "$VK_PASS" > secrets/cache_url
# Containers run as non-root users that do not own these files on Linux.
chmod 644 secrets/*
echo "secrets ready in ./secrets (not tracked by git)"
+12
View File
@@ -0,0 +1,12 @@
#!/bin/sh
# Restart the API and ALL state stores (PostgreSQL, Valkey), keeping volumes.
# Never `down -v`, never flush, never re-seed. Migrations run again (idempotent).
set -eu
cd "$(dirname "$0")/.."
docker compose stop api
docker compose stop valkey postgres
# depends_on brings postgres, valkey and migrate up first; --wait = api healthcheck (/health/ready).
docker compose up -d --no-build --wait api
docker compose ps --all --format 'table {{.Service}}\t{{.Status}}'
+70
View File
@@ -0,0 +1,70 @@
#!/usr/bin/env python3
"""Sessions: wrong password, Basic/Bearer separation, X-User-Id ignored, logout, expiry.
Expiry runs against the api-short-ttl service (SESSION_TTL_SECONDS=2):
docker compose --profile ttl-test up -d --wait api-short-ttl
"""
import base64
import time
import uuid
from stack import BASE_URL, SHORT_TTL_URL, Client, check, main_wrapper, ok
def basic(username, password):
return {"Authorization": "Basic " + base64.b64encode((username + ":" + password).encode("utf-8")).decode()}
def main():
c = Client(BASE_URL)
alice = c.user("auth")
reg = alice["registration"]
# Wrong password and unknown user: the same 401 with a Basic challenge.
r1 = c.call("POST", "/api/v1/auth/sessions", 401, headers=basic(reg["username"], "wrong-password-123"))
r2 = c.call("POST", "/api/v1/auth/sessions", 401, headers=basic("nobody_" + uuid.uuid4().hex[:8], "wrong-password-123"))
check(r1["json"]["error"]["message"] == r2["json"]["error"]["message"], "unknown user must look like a wrong password")
ok("wrong password / unknown user -> identical 401 Basic")
c.call("POST", "/api/v1/auth/sessions", 401)
c.call("POST", "/api/v1/auth/sessions", 401, headers={"Authorization": "Basic !!!not-base64"})
c.call("POST", "/api/v1/auth/sessions", 401, headers={"Authorization": "Bearer " + alice["token"]})
ok("missing / malformed / non-Basic login -> 401")
# A password with ':' and non-ASCII characters survives Basic encoding.
tricky = "пароль:с:двоеточием-" + uuid.uuid4().hex[:6]
user = c.call("POST", "/api/v1/users", 201, body={"username": "u_colon_" + uuid.uuid4().hex[:8], "display_name": "Двоеточие", "password": tricky}, schema="User")["json"]
c.call("POST", "/api/v1/auth/sessions", 201, headers=basic(user["username"], tricky), schema="Session")
ok("UTF-8 password containing ':' -> 201")
c.call("POST", "/api/v1/users", 400, body={"username": "u_short_" + uuid.uuid4().hex[:8], "display_name": "x", "password": "short-11ch"})
c.call("POST", "/api/v1/users", 400, body={"username": "u_nopass_" + uuid.uuid4().hex[:8], "display_name": "x"})
ok("password shorter than 12 / missing -> 400")
me = c.call("GET", "/api/v1/users/me", 401, headers=basic(reg["username"], reg["password"]))
check(me["headers"].get("WWW-Authenticate", "").startswith("Bearer"), "Bearer challenge expected")
c.call("GET", "/api/v1/users/me", 401, headers={"X-User-Id": alice["id"]})
c.call("GET", "/api/v1/users/me", 401, headers={"Authorization": "Bearer " + uuid.uuid4().hex * 2})
ok("Basic / X-User-Id / unknown token on business endpoint -> 401")
session = c.call("POST", "/api/v1/auth/sessions", 201, headers=basic(reg["username"], reg["password"]), schema="Session")["json"]
check(session["token"] != alice["token"], "every login must issue a new token")
second = dict(alice, token=session["token"])
c.call("DELETE", "/api/v1/auth/sessions/current", 204, actor=alice)
c.call("GET", "/api/v1/users/me", 401, actor=alice)
c.call("DELETE", "/api/v1/auth/sessions/current", 401, actor=alice)
c.call("GET", "/api/v1/users/me", 200, actor=second, schema="User")
ok("logout revokes only the current token; repeat logout -> 401")
# Expiry: separate instance of the same image with SESSION_TTL_SECONDS=2.
s = Client(SHORT_TTL_URL)
bob = s.user("ttl")
s.call("GET", "/api/v1/users/me", 200, actor=bob, schema="User")
time.sleep(3)
s.call("GET", "/api/v1/users/me", 401, actor=bob)
s.call("DELETE", "/api/v1/auth/sessions/current", 401, actor=bob)
ok("session with TTL=2s is rejected after 3s (cache entry included)")
if __name__ == "__main__":
main_wrapper(main)
+61
View File
@@ -0,0 +1,61 @@
#!/usr/bin/env python3
"""Level-4 race-protection evidence: 10 clients send 100 messages concurrently;
expect exactly 100 unique messages, with no loss/duplication under pagination.
"""
import sys
import urllib.parse
import concurrent.futures
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "tests"))
from client import Client, check # noqa: E402
def main():
base_url = sys.argv[1] if len(sys.argv) > 1 else "http://localhost:8080"
n_messages = 100
n_clients = 10
c = Client(base_url)
users = [c.user("race%d" % i) for i in range(n_clients)]
chat = c.call(
"POST", "/api/v1/chats", 201, actor=users[0],
body={"title": "race", "member_ids": [u["id"] for u in users]},
schema="Chat",
)["json"]
path = "/api/v1/chats/" + chat["id"] + "/messages"
def send(i):
actor = users[i % n_clients]
return c.message(actor, chat, "race-%d" % i)["json"]["id"]
with concurrent.futures.ThreadPoolExecutor(max_workers=n_clients) as pool:
ids = list(pool.map(send, range(n_messages)))
check(len(ids) == n_messages, "expected %d responses" % n_messages)
check(len(set(ids)) == n_messages, "expected %d unique message ids, got %d" % (n_messages, len(set(ids))))
# Walk the full history back through the API to confirm the store itself
# (not just the HTTP responses) holds exactly n_messages distinct rows.
seen = []
cursor = None
while True:
qs = "?limit=25" + ("&cursor=" + urllib.parse.quote(cursor, safe="") if cursor else "")
page = c.call("GET", path + qs, actor=users[0], schema="MessagePage")["json"]
seen.extend(page["items"])
cursor = page["next_cursor"]
if not cursor:
break
check(len(seen) == n_messages, "expected to read back %d messages, got %d" % (n_messages, len(seen)))
check(len({m["id"] for m in seen}) == n_messages, "readback contains duplicates")
print("PASS: %d unique messages written and read back from %d concurrent clients" % (n_messages, n_clients))
if __name__ == "__main__":
try:
main()
except AssertionError as exc:
print("FAIL:", exc)
raise SystemExit(1)
+53
View File
@@ -0,0 +1,53 @@
#!/usr/bin/env python3
"""PostgreSQL integrity: registration race, rollback of chat creation, repeatable migrations,
password storage. Needs the compose stack (reads tables via `docker compose exec postgres`)."""
import concurrent.futures
import uuid
from stack import BASE_URL, Client, check, compose, main_wrapper, ok, psql
def main():
c = Client(BASE_URL)
# 1. Registration race: 20 parallel requests for one username.
username = "u_race_" + uuid.uuid4().hex[:10]
body = {"username": username, "display_name": "Гонка", "password": "Race-password-123"}
def register(_):
return c.call("POST", "/api/v1/users", [201, 409], body=body)["status"]
with concurrent.futures.ThreadPoolExecutor(max_workers=20) as pool:
statuses = list(pool.map(register, range(20)))
check(statuses.count(201) == 1, "exactly one registration must win, got %s" % statuses)
check(statuses.count(409) == 19, "all others must get 409, got %s" % statuses)
check(psql("SELECT count(*) FROM users WHERE username = '%s'" % username) == "1", "exactly one row expected")
ok("20 concurrent registrations of one username -> 1x201, 19x409, 1 row")
# 2. Rollback: the chat row is inserted, then the member insert hits the FK.
alice = c.user("rb")
title = "rollback-" + uuid.uuid4().hex
c.call("POST", "/api/v1/chats", 400, actor=alice, body={"title": title, "member_ids": [alice["id"], str(uuid.uuid4())]})
check(psql("SELECT count(*) FROM chats WHERE title = '%s'" % title) == "0", "failed chat creation left a chat row")
check(psql("SELECT count(*) FROM chats c WHERE NOT EXISTS (SELECT 1 FROM chat_members m WHERE m.chat_id = c.id)") == "0",
"chat without members exists")
ok("chat with unknown member -> 400 and transaction rolled back (no orphan chat)")
# 3. Passwords are stored as Argon2id hashes, never plaintext.
stored = psql("SELECT password_hash FROM users WHERE username = '%s'" % username)
check(stored.startswith("$argon2id$") and "Race-password-123" not in stored, "password must be an Argon2id hash")
check(psql("SELECT count(*) FROM information_schema.columns WHERE table_name = 'sessions' AND column_name = 'token'") == "0",
"raw tokens must not be stored")
ok("password stored as $argon2id$..., sessions keep only sha256(token)")
# 4. Migrations re-run without duplicates or data changes.
before = psql("SELECT (SELECT count(*) FROM schema_migrations) || ':' || (SELECT count(*) FROM users)")
compose("run", "--rm", "--no-deps", "migrate")
compose("run", "--rm", "--no-deps", "migrate")
after = psql("SELECT (SELECT count(*) FROM schema_migrations) || ':' || (SELECT count(*) FROM users)")
check(before == after, "migrations changed data on re-run: %s -> %s" % (before, after))
ok("migrations run twice more: schema_migrations and data unchanged (%s)" % after)
if __name__ == "__main__":
main_wrapper(main)
+86
View File
@@ -0,0 +1,86 @@
#!/usr/bin/env python3
"""Valkey's role and failures of each dependency. STOPS AND STARTS containers of this stack.
- Valkey really caches sessions and holds logout tombstones.
- Valkey down: API keeps working (cache bypassed), readiness stays 200.
- PostgreSQL down: bounded 503 quickly, readiness 503, liveness 200; an unverifiable
token is never accepted; no token is still 401. Recovery after start.
"""
import time
from stack import BASE_URL, Client, Failure, check, compose, main_wrapper, ok, valkey
def wait_ready(c, seconds=60):
deadline = time.monotonic() + seconds
while time.monotonic() < deadline:
try:
c.call("GET", "/health/ready", 200)
return
except Failure:
time.sleep(0.5)
raise Failure("API did not become ready")
def main():
c = Client(BASE_URL, timeout=10)
alice, bob, _eve, chat = c.bootstrap()
history = "/api/v1/chats/" + chat["id"] + "/messages"
# Role of Valkey: a session lookup fills the cache, logout leaves a tombstone.
before = int(valkey("DBSIZE"))
c.call("GET", "/api/v1/users/me", actor=alice, schema="User")
keys = valkey("--scan", "--pattern", "session:*").split()
check(len(keys) >= 1 and int(valkey("DBSIZE")) >= before, "session lookups must be cached in Valkey")
ttl = int(valkey("TTL", keys[0]))
check(0 < ttl <= 300, "cache entries must expire (TTL %d)" % ttl)
temp = c.user("tomb")
c.call("GET", "/api/v1/users/me", actor=temp, schema="User")
c.call("DELETE", "/api/v1/auth/sessions/current", 204, actor=temp)
tombstones = [k for k in valkey("--scan", "--pattern", "session:*").split() if valkey("GET", k) == "revoked"]
check(tombstones, "logout must leave a 'revoked' tombstone in the cache")
ok("Valkey caches sessions with TTL<=300s and keeps logout tombstones")
# Valkey down -> the API bypasses the cache, PostgreSQL stays the source of truth.
try:
compose("stop", "valkey")
c.call("GET", "/health/ready", 200)
c.call("GET", "/api/v1/users/me", actor=alice, schema="User")
c.message(bob, chat, "без кэша")
c.call("GET", "/api/v1/users/me", 401, actor=temp) # revoked in PostgreSQL too
start = time.monotonic()
for _ in range(10):
c.call("GET", history + "?limit=1", actor=alice, schema="MessagePage")
per_request = (time.monotonic() - start) / 10
check(per_request < 0.5, "cache bypass must be fast (%.3fs per request)" % per_request)
ok("Valkey stopped: ready=200, reads/writes OK, revoked token 401, %.0f ms/request" % (per_request * 1000))
finally:
compose("start", "valkey")
# PostgreSQL down -> bounded 503, never anonymous access.
fresh = c.user("dbdown") # token never looked up yet: not in the cache
try:
compose("stop", "postgres")
c.call("GET", "/health/live", 200)
start = time.monotonic()
r = c.call("GET", "/health/ready", 503)
check(r["json"]["error"]["code"] == "unavailable", "readiness must report unavailable")
c.call("GET", "/api/v1/chats", 503, actor=alice)
c.call("POST", history, 503, actor=alice, body={"text": "в никуда"})
c.call("GET", "/api/v1/users/me", 503, actor=fresh) # cannot verify -> not trusted
c.call("GET", "/api/v1/users/me", 401) # no credentials at all
elapsed = time.monotonic() - start
check(elapsed < 20, "errors must be bounded in time (%.1fs)" % elapsed)
ok("PostgreSQL stopped: live=200, ready/business=503 in %.1fs total, unverified token not trusted" % elapsed)
finally:
compose("start", "postgres")
wait_ready(c)
c.call("GET", "/api/v1/users/me", actor=fresh, schema="User")
page = c.call("GET", history + "?limit=100", actor=alice, schema="MessagePage")["json"]
check(any(m["text"] == "без кэша" for m in page["items"]), "message written without cache must persist")
ok("PostgreSQL started again: API recovers without restart, data intact")
if __name__ == "__main__":
main_wrapper(main)
+81
View File
@@ -0,0 +1,81 @@
#!/bin/sh
# Seed synthetic history (20 000 messages in one chat + 50 000 in 50 other chats) and
# record EXPLAIN (ANALYZE, BUFFERS) of the exact keyset queries used by the API,
# with and without messages_chat_history_idx. Output: evidence/explain_history.txt
# Seeding is idempotent (fixed IDs, ON CONFLICT DO NOTHING); data is synthetic.
set -eu
cd "$(dirname "$0")/.."
OUT="${OUT:-evidence/explain_history.txt}"
PSQL="docker compose exec -T postgres psql -U messenger -d messenger -X -v ON_ERROR_STOP=1"
$PSQL -q <<'SQL'
INSERT INTO users (id, username, display_name, password_hash) VALUES
('00000000-0000-4000-8000-000000000001', 'seed_explain_a', 'Seed A', 'not-a-login'),
('00000000-0000-4000-8000-000000000002', 'seed_explain_b', 'Seed B', 'not-a-login')
ON CONFLICT DO NOTHING;
-- chat ...0100 is the measured one; ...0101-0150 are background noise.
INSERT INTO chats (id, title, created_at)
SELECT ('00000000-0000-4000-8000-' || lpad((100 + g)::text, 12, '0'))::uuid, 'explain ' || g, now()
FROM generate_series(0, 50) g
ON CONFLICT DO NOTHING;
INSERT INTO chat_members (chat_id, user_id, position)
SELECT c.id, u.id, u.pos
FROM chats c
CROSS JOIN (VALUES ('00000000-0000-4000-8000-000000000001'::uuid, 0),
('00000000-0000-4000-8000-000000000002'::uuid, 1)) AS u(id, pos)
WHERE c.title LIKE 'explain %'
ON CONFLICT DO NOTHING;
INSERT INTO messages (id, chat_id, sender_id, text, created_at)
SELECT md5('explain-' || c.n || '-' || g)::uuid,
('00000000-0000-4000-8000-' || lpad((100 + c.n)::text, 12, '0'))::uuid,
CASE WHEN g % 2 = 0 THEN '00000000-0000-4000-8000-000000000001'::uuid
ELSE '00000000-0000-4000-8000-000000000002'::uuid END,
'synthetic message ' || g,
timestamptz '2026-01-01' + (g * interval '1 second') + (c.n * interval '1 millisecond')
FROM (SELECT n, CASE WHEN n = 0 THEN 20000 ELSE 1000 END AS cnt FROM generate_series(0, 50) n) c
CROSS JOIN LATERAL generate_series(1, c.cnt) g
ON CONFLICT DO NOTHING;
ANALYZE users; ANALYZE chats; ANALYZE chat_members; ANALYZE messages;
SQL
{
echo "# EXPLAIN evidence for message history pagination"
echo "# date: $(date -u +%Y-%m-%dT%H:%M:%SZ) commit: $(git rev-parse --short HEAD 2>/dev/null || echo n/a)"
echo "# postgres: $($PSQL -tA -c 'SHOW server_version')"
$PSQL -tA -c "SELECT '# messages total: ' || count(*) || ', in measured chat: ' || count(*) FILTER (WHERE chat_id = '00000000-0000-4000-8000-000000000100') FROM messages"
echo
$PSQL <<'SQL'
-- The same statements as app/storage.py (Store.list_messages), as prepared statements.
PREPARE first_page(uuid, int) AS
SELECT id, chat_id, sender_id, text, created_at FROM messages
WHERE chat_id = $1 ORDER BY created_at, id LIMIT $2;
PREPARE next_page(uuid, timestamptz, uuid, int) AS
SELECT id, chat_id, sender_id, text, created_at FROM messages
WHERE chat_id = $1 AND (created_at, id) > ($2, $3) ORDER BY created_at, id LIMIT $4;
\echo '=== 1. WITH index messages_chat_history_idx: first page (limit 50 + 1) ==='
EXPLAIN (ANALYZE, BUFFERS) EXECUTE first_page('00000000-0000-4000-8000-000000000100', 51);
\echo '=== 2. WITH index: deep page, cursor after message #15000 of 20000 ==='
EXPLAIN (ANALYZE, BUFFERS) EXECUTE next_page('00000000-0000-4000-8000-000000000100',
timestamptz '2026-01-01' + interval '15000 seconds', md5('explain-0-15000')::uuid, 51);
-- Same queries without the index, inside a transaction that is rolled back.
BEGIN;
DROP INDEX messages_chat_history_idx;
\echo '=== 3. WITHOUT index: first page ==='
EXPLAIN (ANALYZE, BUFFERS) EXECUTE first_page('00000000-0000-4000-8000-000000000100', 51);
\echo '=== 4. WITHOUT index: deep page ==='
EXPLAIN (ANALYZE, BUFFERS) EXECUTE next_page('00000000-0000-4000-8000-000000000100',
timestamptz '2026-01-01' + interval '15000 seconds', md5('explain-0-15000')::uuid, 51);
ROLLBACK;
SQL
} > "$OUT"
echo "written $OUT"
grep -E '^===|Index|Seq Scan|Sort|Buffers|Execution Time' "$OUT"
+120
View File
@@ -0,0 +1,120 @@
#!/usr/bin/env python3
"""Student-written negative/edge-case tests, additional to the public tests/smoke.py.
Covers level-4 requirements from README.md:
- malformed JSON
- foreign chat access
- empty message text
- duplicate username
- multi-page pagination traversal
"""
import json
import sys
import urllib.error
import urllib.parse
import urllib.request
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "tests"))
from client import Client, check # noqa: E402
PASSED = 0
def ok(label):
global PASSED
PASSED += 1
print("ok -", label)
def raw_call(base, method, path, data=None, headers=None):
h = {"Accept": "application/json"}
h.update(headers or {})
req = urllib.request.Request(base + path, data=data, headers=h, method=method)
try:
resp = urllib.request.urlopen(req, timeout=10)
except urllib.error.HTTPError as exc:
resp = exc
with resp:
return resp.code, resp.read()
def main():
base_url = sys.argv[1] if len(sys.argv) > 1 else "http://localhost:8080"
c = Client(base_url)
alice, bob, eve, chat = c.bootstrap()
path = "/api/v1/chats/" + chat["id"]
# 1. Malformed JSON body -> 400 invalid_request, not a 500.
status, body = raw_call(
base_url, "POST", "/api/v1/users",
data=b'{"username": "broken", "display_name": ',
headers={"Content-Type": "application/json", "X-User-Id": alice["id"]},
)
check(status == 400, "malformed JSON must yield 400, got %d" % status)
parsed = json.loads(body)
check(parsed["error"]["code"] == "invalid_request", "malformed JSON must report invalid_request")
ok("malformed JSON -> 400 invalid_request")
# 2. Foreign chat: eve is not a member -> 404, not 403 (no existence leak).
c.call("GET", path, 404, actor=eve)
c.call("GET", path + "/messages", 404, actor=eve)
ok("foreign chat access -> 404 (no leak)")
# 3. Empty / whitespace-only message text -> 400.
c.call("POST", path + "/messages", 400, actor=alice, body={"text": ""})
c.call("POST", path + "/messages", 400, actor=alice, body={"text": " "})
ok("empty/whitespace message text -> 400")
# 4. Duplicate username -> 409 conflict, original user untouched.
c.call("POST", "/api/v1/users", 409, body=alice["registration"])
me = c.call("GET", "/api/v1/users/me", actor=alice, schema="User")["json"]
check(me == alice["user"], "existing user must be unaffected by a rejected duplicate registration")
ok("duplicate username -> 409, original user intact")
# 5. Multi-page pagination traversal: write N messages, walk every page with a small
# limit and confirm the full, ordered, duplicate-free set is recovered.
sent = [c.message(alice, chat, "page-msg-%d" % i)["json"] for i in range(7)]
collected = []
cursor = None
pages = 0
while True:
qs = "?limit=2" + ("&cursor=" + urllib.parse.quote(cursor, safe="") if cursor else "")
page = c.call("GET", path + "/messages" + qs, actor=bob, schema="MessagePage")["json"]
pages += 1
check(len(page["items"]) <= 2, "page must respect limit")
collected.extend(page["items"])
cursor = page["next_cursor"]
if not cursor:
break
check(pages < 50, "pagination did not terminate")
ids_seen = [m["id"] for m in collected]
check(len(ids_seen) == len(set(ids_seen)), "pagination must not repeat messages")
check({m["id"] for m in sent}.issubset(set(ids_seen)), "pagination must not drop messages")
tail = [m for m in collected if m["id"] in {s["id"] for s in sent}]
check(tail == sorted(tail, key=lambda m: (m["created_at"], m["id"])), "pagination must stay ordered")
ok("multi-page pagination traversal: %d pages, %d messages, no dup/loss" % (pages, len(collected)))
# 6. A cursor is bound to the chat it was issued for: reusing it on a
# different chat the same actor can also read must be rejected.
first_page = c.call("GET", path + "/messages?limit=1", actor=alice, schema="MessagePage")["json"]
check(first_page["next_cursor"], "need a non-null cursor to test cross-chat reuse")
chat2 = c.call(
"POST", "/api/v1/chats", 201, actor=alice,
body={"title": "second chat", "member_ids": [alice["id"], bob["id"]]}, schema="Chat",
)["json"]
c.message(alice, chat2, "unrelated message")
foreign_cursor = urllib.parse.quote(first_page["next_cursor"], safe="")
c.call("GET", "/api/v1/chats/" + chat2["id"] + "/messages?limit=1&cursor=" + foreign_cursor, 400, actor=alice)
ok("cursor from another chat -> 400 (cursor is bound to its resource)")
print("PASS: %d/%d student checks" % (PASSED, 6))
if __name__ == "__main__":
try:
main()
except AssertionError as exc:
print("FAIL:", exc)
raise SystemExit(1)
+24
View File
@@ -0,0 +1,24 @@
#!/bin/sh
# All student tests in one command. Needs the running compose stack; starts the
# short-TTL instance itself. dependency_test/sigterm_test STOP AND START containers.
# Usage: BASE_URL=http://localhost:8080 student-tests/run_all.sh
set -eu
cd "$(dirname "$0")/.."
PY="${PYTHON:-python3}"
export BASE_URL="${BASE_URL:-http://localhost:8080}"
export SHORT_TTL_URL="${SHORT_TTL_URL:-http://localhost:8081}"
docker compose --profile ttl-test up -d --no-build --wait api-short-ttl >/dev/null 2>&1
for t in own_tests.py concurrency_test.py auth_test.py db_integrity_test.py dependency_test.py; do
echo "== $t"
"$PY" "student-tests/$t" "$BASE_URL"
echo
done
echo "== sigterm_test.sh"
sh student-tests/sigterm_test.sh
docker compose --profile ttl-test stop api-short-ttl >/dev/null 2>&1
echo
echo "ALL STUDENT TESTS PASSED"
+19
View File
@@ -0,0 +1,19 @@
#!/bin/sh
# Graceful shutdown of the compose api service: SIGTERM -> exit 0 within 10s
# (not SIGKILL 137), then the service is started again. Volumes are untouched.
set -eu
cd "$(dirname "$0")/.."
CID=$(docker compose ps -q api)
[ -n "$CID" ] || { echo "FAIL: api is not running"; exit 1; }
START=$(date +%s)
docker compose stop api >/dev/null 2>&1
ELAPSED=$(( $(date +%s) - START ))
EXIT_CODE=$(docker inspect -f '{{.State.ExitCode}}' "$CID")
echo "api stopped in ${ELAPSED}s, exit code ${EXIT_CODE} (0 = graceful, 137 = SIGKILL)"
docker compose logs --since 30s api 2>&1 | grep -q "Application shutdown complete" || { echo "FAIL: no graceful shutdown line"; exit 1; }
[ "$ELAPSED" -le 10 ] && [ "$EXIT_CODE" -eq 0 ] || { docker compose up -d --wait api; echo "FAIL: not graceful"; exit 1; }
docker compose up -d --no-build --wait api >/dev/null 2>&1
echo "PASS: graceful SIGTERM shutdown, api restarted"
+47
View File
@@ -0,0 +1,47 @@
"""Shared helpers for student tests: public client + direct access to the compose stack."""
import os
import subprocess
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "tests"))
from client import Client, Failure, check # noqa: E402,F401
BASE_URL = os.environ.get("BASE_URL", "http://localhost:8080")
SHORT_TTL_URL = os.environ.get("SHORT_TTL_URL", "http://localhost:8081")
PASSED = []
def ok(label):
PASSED.append(label)
print("ok -", label)
def compose(*args, check_rc=True):
"""docker compose in the repository root; output is captured, never printed raw."""
proc = subprocess.run(["docker", "compose", *args], cwd=ROOT, capture_output=True, text=True)
if check_rc and proc.returncode != 0:
raise Failure("docker compose %s failed: %s" % (args[0], proc.stderr.strip()[-300:]))
return proc.stdout
def psql(sql):
"""Run SQL inside the postgres container (local socket, no password on the wire)."""
return compose("exec", "-T", "postgres", "psql", "-U", "messenger", "-d", "messenger", "-tAX", "-c", sql).strip()
def valkey(*args):
"""valkey-cli with the password read inside the container, never on this host's argv."""
cmd = 'valkey-cli --no-auth-warning -a "$(sed -n "s/^requirepass //p" /run/secrets/valkey_conf)" "$@"'
return compose("exec", "-T", "valkey", "sh", "-c", cmd, "valkey-cli", *args).strip()
def main_wrapper(main):
try:
main()
except (AssertionError, Failure) as exc:
print("FAIL:", exc)
raise SystemExit(1)
print("PASS: %d checks" % len(PASSED))