ЛР 2 — БВТ2301 — Александров— на 4 #140

Open
t-d-aleksandrov-edu-mtuci-ru wants to merge 3 commits from t-d-aleksandrov-edu-mtuci-ru/mtusi2026lab2:lab2-implementation into master
49 changed files with 3863 additions and 31 deletions
+14
View File
@@ -1,3 +1,4 @@
[TEMPLATE]
# Copy to .env for your LOCAL synthetic test environment; never commit .env.
# These are logical interface suggestions; map actual framework names in REPORT.md.
APP_HOST=0.0.0.0
@@ -9,3 +10,16 @@ DATABASE_URL_FILE=/run/secrets/database_url
# Optional on 3, required use on 4 in lab2; Redis OR Valkey:
SESSION_STORE=postgres
CACHE_URL_FILE=/run/secrets/cache_url
# --- lab 2 implementation knobs (mapped in REPORT.md) ---
# Valkey entry lifetime is additionally capped by the session's remaining life.
CACHE_TTL_SECONDS=300
# Bounded PostgreSQL pool and explicit timeouts.
DB_POOL_MIN=1
DB_POOL_MAX=10
DB_POOL_TIMEOUT_SECONDS=2
DB_CONNECT_TIMEOUT_SECONDS=2
DB_STATEMENT_TIMEOUT_MS=5000
# Valkey operations fail over to PostgreSQL after this per-call timeout.
CACHE_TIMEOUT_SECONDS=0.5
# Host port used by compose; the TTL stand uses 8081.
API_PUBLISH_PORT=8080
+48 -14
View File
@@ -1,18 +1,52 @@
## Лабораторная 2
Автор / группа: TODO
Целевая оценка (3/4/5): TODO
Трек 5, если заявлен: TODO
Автор / группа: Тимофей Александров, БВТ2301
Целевая оценка (3/4/5): 5
Трек 5: трек 2 «План запроса и восстановление»
Что реализовано и что перенесено: TODO
Команды запуска из чистого clone: TODO
Результаты проверок и ссылка на REPORT.md: TODO
Ограничения / известные ошибки: TODO
Источники и использование ИИ: TODO
Что реализовано и что перенесено:
- Перенесено из lab1: контракт API, keyset-пагинация, health-эндпоинты, регистрация пользователей.
- Новое в lab2: PostgreSQL источник истины (миграции 001-003), Valkey кэш сессий, Argon2id пароли, Basic только на POST /auth/sessions, Bearer на остальных эндпоинтах, транзакционное создание чата, idempotency-ключи, circuit breaker на Valkey, graceful degradation при отказе кэша/БД, bounded connection pool с таймаутами.
- Исправлено по ревью: Valkey больше не авторизует cache-hit самостоятельно. Каждая Bearer-сессия подтверждается в PostgreSQL, поэтому токен, отозванный при outage кэша, остаётся 401 после рестарта API и возврата старого Valkey-тома.
- [ ] Весь обязательный минимум текущей и предыдущих работ сохранён с объявленными переходами.
- [ ] Контракт и публичные тесты не ослаблены.
- [ ] make check и make test выполнены; приложен реальный вывод.
- [ ] Сценарии отказов/безопасности своего уровня воспроизведены.
- [ ] Все секреты и приватные ключи находятся вне Git; отчёт не содержит tokens.
- [ ] README задания сохранён, REPORT.md содержит инструкцию именно моего решения.
Команды запуска из чистого clone:
```sh
sh scripts/bootstrap-secrets.sh
docker compose --env-file .env config --quiet
docker compose --env-file .env up -d --build --wait --wait-timeout 180
python3 scripts/check_contract.py
python3 scripts/validate_openapi.py
python3 tests/smoke.py --base-url http://127.0.0.1:8080
python3 tests/resilience.py --mode persistence --action-script scripts/restart.sh --base-url http://127.0.0.1:8080
cd student-tests && BASE_URL=http://127.0.0.1:8080 python3 -u -B -m unittest -v test_api test_race test_degradation
sh scripts/test-ttl.sh
sh scripts/explain-history.sh
sh scripts/research-history-backup.sh
```
Результаты проверок и ссылка на REPORT.md:
- Smoke: PASS: lab2 public API scenario
- check_contract: PASS: offline skeleton structure, local refs and Python syntax
- validate_openapi: PASS: full OpenAPI 3.1 and JSON Schema validation
- Student tests: 26/26 OK; degradation 2/2 OK, включая durable logout после cache outage/API restart
- Persistence: PASS: persistence preserved confirmed data and existing session
- TTL: 4/4 OK, 6.1 с (стенд SESSION_TTL_SECONDS=2, :8081)
- Degradation: OK — cached_p50=16 мс, cache_down_p50=39 мс, cache_down_p95=78 мс
- EXPLAIN: 12 000 сообщений, Index Scan, 4 буфера, 0.208 мс
- Трек 5: 100 000 сообщений в 101 чате (90 000/100); composite index 0.128 мс / 6 buffers, chat-only 89.728 мс / 1525 buffers; backup/restore aggregates совпали.
- Нагрузка: 25.5 rps, 523 запроса, 0 ошибок, p50 125 мс, p95 328 мс
- Подробности: REPORT.md
Ограничения / известные ошибки:
- Один процесс api; репликация API — область следующей лабораторной.
- Нагрузочный тест закрытого цикла содержит coordinated omission.
- make в среде отсутствует, команды выполнялись напрямую.
Источники и использование ИИ: README/COURSE/CONTRIBUTING этого skeleton, contracts/README.md, tests/README.md; документация PostgreSQL, Valkey; RFC 7617; OWASP Password Storage Cheat Sheet. ИИ (Codex) использовался для изучения задания, проектирования, написания исходников и тестов, проверки и подготовки отчёта. Автор проверяет результат и должен уметь объяснить и изменить его на защите.
- [x] Весь обязательный минимум текущей и предыдущих работ сохранён с объявленными переходами.
- [x] Контракт и публичные тесты не ослаблены.
- [x] make check и make test выполнены; приложен реальный вывод.
- [x] Сценарии отказов/безопасности своего уровня воспроизведены.
- [x] Все секреты и приватные ключи находятся вне Git; отчёт не содержит tokens.
- [x] README задания сохранён, REPORT.md содержит инструкцию именно моего решения.
+32
View File
@@ -0,0 +1,32 @@
# syntax=docker/dockerfile:1
# Runtime image is pinned by digest, dependencies by version in requirements.txt.
FROM python:3.12.7-slim-bookworm@sha256:60d9996b6a8a3689d36db740b49f4327be3be09a21122bd02fb8895abb38b50d
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
APP_HOST=0.0.0.0 \
APP_PORT=8080
WORKDIR /app
# Unprivileged runtime user; the API never needs to write to the image.
RUN groupadd --system --gid 10001 messenger \
&& useradd --system --uid 10001 --gid messenger --no-create-home messenger
COPY requirements.txt /app/requirements.txt
RUN pip install --no-cache-dir --requirement /app/requirements.txt
COPY --chown=messenger:messenger src/ /app/src/
COPY --chown=messenger:messenger contracts/ /app/contracts/
COPY --chown=messenger:messenger --chmod=0555 entrypoint.sh /app/entrypoint.sh
USER messenger
EXPOSE 8080
STOPSIGNAL SIGTERM
# Readiness (not just liveness) so an unhealthy container is visible to Compose.
HEALTHCHECK --interval=5s --timeout=5s --start-period=20s --retries=3 \
CMD ["python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/health/ready', timeout=2).read()"]
# Migrations are idempotent; the entrypoint execs the server so it is PID 1.
ENTRYPOINT ["/app/entrypoint.sh"]
+38 -2
View File
@@ -1,13 +1,20 @@
PYTHON ?= python3
BASE_URL ?= http://localhost:8080
# 127.0.0.1, not localhost: compose publishes the API on IPv4 only, and a
# dual-stack localhost would add a ~2 s IPv6 retry to every request.
BASE_URL ?= http://127.0.0.1:8080
CA_FILE ?=
ACTION_SCRIPT ?=
TLS_ARGS = $(if $(CA_FILE),--ca-file "$(CA_FILE)",)
STUDENT_DIR = student-tests
.PHONY: help check validate test load test-persistence
.PHONY: help check validate test load test-persistence run bootstrap up down test-student test-ttl test-degradation explain research-history-backup
help:
@echo "check: offline structure; validate: full OpenAPI (requirements-dev.txt); test: running API; load: synthetic writes"
@echo "bootstrap/up/down: local Compose stand; run: API without Docker (needs DATABASE_URL/CACHE_URL)"
@echo "test-student: own functional+race tests; test-ttl: expiry on the short-TTL stand; test-degradation: cache/DB outages"
@echo "explain: seed 12k messages and capture EXPLAIN (ANALYZE, BUFFERS) into evidence/"
@echo "research-history-backup: grade-5 100k index comparison and PostgreSQL backup/restore"
check:
$(PYTHON) scripts/check_contract.py
@@ -24,3 +31,32 @@ load:
test-persistence:
@test -n "$(ACTION_SCRIPT)" || (echo "Set ACTION_SCRIPT to your implemented executable restart hook"; exit 2)
$(PYTHON) tests/resilience.py --mode persistence --action-script "$(ACTION_SCRIPT)" --base-url "$(BASE_URL)" $(TLS_ARGS)
# --- lab 2 -----------------------------------------------------------------
run:
$(PYTHON) -B src/app.py
bootstrap:
sh scripts/bootstrap-secrets.sh
up: bootstrap
docker compose --env-file .env up -d --build --wait --wait-timeout 180
docker compose --env-file .env ps
down:
docker compose --env-file .env down
test-student:
cd $(STUDENT_DIR) && BASE_URL="$(BASE_URL)" $(PYTHON) -B -m unittest -v test_api test_race
test-ttl:
sh scripts/test-ttl.sh
test-degradation:
cd $(STUDENT_DIR) && BASE_URL="$(BASE_URL)" STUDENT_DOCKER_TESTS=1 $(PYTHON) -B -m unittest -v test_degradation
explain:
sh scripts/explain-history.sh
research-history-backup:
sh scripts/research-history-backup.sh
+88 -13
View File
@@ -1,37 +1,112 @@
# Отчёт — лабораторная 2
- Автор, группа: TODO
- Целевая оценка; выбранный трек 5, если нужен: TODO
- Автор, группа: Тимофей Александров, БВТ2301
- Целевая оценка; выбранный трек 5, если нужен: **5**, трек 2 «План запроса и восстановление»
- Commit/PR: TODO после публикации
- Использованные источники, библиотеки и помощь ИИ: TODO
- Использованные источники, библиблиотеки и помощь ИИ: README/COURSE/CONTRIBUTING этого skeleton, `contracts/README.md` и `tests/README.md`; документация PostgreSQL (transactions, EXPLAIN), Valkey (persistence), RFC 7617 (Basic), OWASP Password Storage Cheat Sheet (Argon2id). Библиотеки: FastAPI/Starlette/uvicorn, psycopg 3 + psycopg-pool, redis-py, argon2-cffi (списки в `requirements.txt`).
## Запуск из чистого clone
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
Версии, на которых всё воспроизведено: Docker Engine **29.9.2**, Docker Compose **v5.6.1**, на хосте Python **3.11.5** (нужен только для скриптов и тестов), внутри контейнера Python **3.12.7** (`python:3.12.7-slim-bookworm`), PostgreSQL **17.6**, Valkey **8.1.3** — образы закреплены по digest в `compose.yaml`. Аппаратура стенда: Intel Pentium Silver N5030 @ 1.10 GHz, 4 логических ядра, ~7.6 GiB RAM, лимитов CPU/RAM для контейнеров не задано.
Нужны: Git с POSIX-shell (на Windows — Git Bash, скрипты `scripts/*.sh`) и Docker с Compose. `make` в этой среде отсутствует, поэтому ниже приведены те же команды напрямую (сам `Makefile` сохранён и использует те же пути).
```sh
# 1) секреты и .env: генерирует ./secrets (mode 0600, в .gitignore) и копирует .env
sh scripts/bootstrap-secrets.sh # на Windows: sh из Git Bash
# 2) проверка конфигурации и подъём стенда (миграции применяет entrypoint контейнера api)
docker compose --env-file .env config --quiet
docker compose --env-file .env up -d --build --wait --wait-timeout 180
docker compose --env-file .env ps
docker compose --env-file .env logs api | grep migrations # migrations_applied / migrations_up_to_date
# 3) проверки
python3 scripts/check_contract.py # = make check
python3 scripts/validate_openapi.py # = make validate (нужен requirements-dev.txt)
python3 tests/smoke.py --base-url http://127.0.0.1:8080 # = make test
python3 tests/resilience.py --mode persistence \
--action-script scripts/restart.cmd --base-url http://127.0.0.1:8080 # = make test-persistence
sh scripts/test-ttl.sh # = make test-ttl (отдельный стенд на :8081, SESSION_TTL_SECONDS=2)
cd student-tests && STUDENT_DOCKER_TESTS=1 BASE_URL=http://127.0.0.1:8080 \
python3 -u -B -m unittest -v test_api test_race test_degradation
sh scripts/explain-history.sh # = make explain (EXPLAIN ANALYZE на 12 000 сообщений)
sh scripts/research-history-backup.sh # трек 5: 100 000 сообщений, сравнение индексов, backup/restore
```
Безопасная остановка с сохранением данных: `docker compose --env-file .env down` (volumes `pgdata`/`valkeydata` не удаляются, сессии и сообщения переживают рестарт). Намеренное удаление учебных данных: `docker compose --env-file .env down -v`. На Windows эквивалент restart-hook для `tests/resilience.py` — `scripts/restart.cmd` (тот же сценарий, что и `scripts/restart.sh`: `down` без `-v`, затем `up -d --wait`; Python не умеет запускать `.sh` напрямую, ошибка `WinError 193`).
URL вида `http://127.0.0.1:8080`, а не `localhost`: Compose публикует только IPv4-порт, а на dual-stack хосте `localhost` сначала идёт в `::1` и добавляет ~2 с TCP-retry к каждому запросу (это и было причиной первоначальных «медленных» тестов).
## Архитектура и решения
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab1, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
**Процессы/хранилища.** Клиент → `api` (FastAPI + uvicorn, один процесс) → `postgres` (источник истины) и `valkey` (кэш сессий). Сеть `backend` внутренняя, наружу публикуется только `127.0.0.1:${API_PUBLISH_PORT:-8080}:8080`. Логика в `src/`, SQL-миграции в `src/migrations/`.
**Что перенесено из lab1.** Тот же контракт и та же логика keyset-пагинации, но хранилище заменено с in-memory dict на PostgreSQL; `X-User-Id` больше не используется нигде (входящее поле игнорируется, проверено тестом); вместо Basic-входа на каждый запрос — Basic только на `POST /api/v1/auth/sessions` и Bearer-токен на остальных.
**Источник истины и кэш.** Сессии живут в таблице `sessions` (хранится только SHA-256 токена). Valkey хранит публичный снимок пользователя под `session:<sha256>` максимум `CACHE_TTL_SECONDS=300` с и одновременно не дольше остатка жизни сессии. Кэш инвалидируется логаутом через tombstone `revoked`. PostgreSQL подтверждает, что сессия не отозвана и не истекла **на каждом Bearer-запросе**: cache-hit никогда сам по себе не авторизует. Поэтому недоставленный tombstone при остановленном Valkey, рестарт API и восстановление старого кэша не могут воскресить токен. Если Valkey недоступен — запрос всё равно работает через PostgreSQL; кэш-ошибка никогда не означает «авторизован». Плюс 5-секундный circuit breaker.
**Транзакционные границы.** Создание чата (`chats` + строки `chat_members`) — одна транзакция: ошибка хотя бы одного участника откатывает всё (проверено: в таблицах остаётся 0 строк). Регистрация и отправка сообщения — по одной транзакции. Миграции — версионированный раннер `src/migrate.py`: одна транзакция + `pg_advisory_xact_lock`, таблица `schema_migrations`, повторный запуск печатает `migrations_up_to_date` и не дублирует данные; запуск идемпотентен и защищён от параллельных контейнеров.
**Ограничения и таймауты.** Пул `DB_POOL_MIN=1 … DB_POOL_MAX=10` c `DB_POOL_TIMEOUT_SECONDS=2`, `DB_CONNECT_TIMEOUT_SECONDS=2`, серверные `statement_timeout=5000`, `lock_timeout=2000`, `idle_in_transaction_session_timeout=10000`, `timezone=UTC`. Все запросы параметризованные (psycopg3, никакой конкатенации SQL). Кэш-операции ограничены `CACHE_TIMEOUT_SECONDS=0.5`.
**Аутентификация.** Пароли — Argon2id через `argon2-cffi` (19 MiB, 2 итерации, 1 lane; для неизвестного пользователя проверяется dummy-hash, чтобы путь отказа не выдавал существование логина). Токен — `secrets.token_urlsafe(48)` = 384 бита, в БД только его SHA-256. `SESSION_TTL_SECONDS` (3600 по умолчанию), отзыв через `DELETE /api/v1/auth/sessions/current`. `WWW-Authenticate` различает Bearer и Basic.
**Секреты.** `scripts/bootstrap-secrets.sh` генерирует `./secrets/*` (mode 0600, каталог в `.gitignore`), compose монтирует их как файлы (`DATABASE_URL_FILE`, `POSTGRES_PASSWORD_FILE`, …), значения не попадают в `compose.yaml`, `docker inspect`, логи или отчёт.
**Соответствие `.env.example` ↔ `src/config.py`.** `APP_HOST/APP_PORT` → `load_address()`; `SESSION_TTL_SECONDS`, `CACHE_TTL_SECONDS`, `DB_POOL_*`, `DB_*_TIMEOUT*`, `CACHE_TIMEOUT_SECONDS` → одноимённые поля `Settings`; `DATABASE_URL_FILE`/`CACHE_URL_FILE` → `_secret()`, с падением на старт-апе при нечитаемом файле; `SESSION_STORE` допускает только `postgres` (иное значение — явная ошибка конфигурации, а не тихий обход).
## Доказательства критериев
Все выводы сохранены в `evidence/` (без токенов и секретов); файлы сгенерированы 2026-10-30, commit указывается в PR.
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
| --- | --- | --- | --- |
| Публичный smoke | make test | TODO | TODO |
| Остальные пункты уровня 3 | TODO | TODO | TODO |
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
| Публичный smoke | `python3 tests/smoke.py --base-url http://127.0.0.1:8080` | `PASS: lab2 public API scenario` | `evidence/smoke.txt` |
| make check: структура/контракт | `python3 scripts/check_contract.py` | `PASS: offline skeleton structure, local refs and Python syntax` | строка в PR |
| Полная валидация OpenAPI | `python3 scripts/validate_openapi.py` | `PASS: full OpenAPI 3.1 and JSON Schema validation` | строка в PR |
| Уровень 3: рестарт сохраняет данные и сессию | `python3 tests/resilience.py --mode persistence --action-script scripts/restart.cmd …` | `PASS: persistence preserved confirmed data and existing session` | `evidence/persistence.txt` |
| Миграции, транзакции, гонка username, откат создания чата | `python3 -m unittest -v test_api test_race` | **26/26 OK, 14.8 с**, включая `duplicate_username_race` (ровно 1 пользователь) и `chat_creation_rolls_back_on_unknown_member` | `evidence/student-tests.txt` |
| Ограниченный пул и параллельные чтения | там же, `bounded_pool_serves_parallel_reads` | OK; 100 записей от 10 клиентов за **3.08 с**, ровно 100 строк | `evidence/student-tests.txt` |
| Индекс истории, EXPLAIN ≥ 10 000 строк, без OFFSET | `sh scripts/explain-history.sh` | 12 000 сообщений: страница глубоко в истории — **Index Scan, 4 буфера, 0.208 мс**; вариант `OFFSET` (не используется) — **Seq Scan + Sort, 205 буферов, 14.0 мс** | `evidence/explain-history.txt` |
| Автопроверка expiry при коротком TTL, logout, неверный пароль | `sh scripts/test-ttl.sh` (стенд `SESSION_TTL_SECONDS=2`, :8081) | **4/4 OK, 6.1 с**; кэш не продлевает жизнь токену, logout мгновенен и при закэшированной сессии | `evidence/test-ttl.txt` |
| Недоступность Redis/Valkey и БД | `STUDENT_DOCKER_TESTS=1 python3 -m unittest -v test_degradation` | **OK, 21.4 с**: `cached_p50=16 мс`, `cache_down_p50=39 мс`, `cache_down_p95=78 мс`, `/health/ready` остаётся 200 при падении кэша и становится 503 `unavailable` при падении БД, запись отказывает за **2.05 с**, восстановление **0.58 с**, после восстановления та же сессия и те же сообщения | `evidence/test-degradation.txt` |
| Durable logout при cache outage + API restart | `test_logout_remains_revoked_after_cache_outage_and_api_restart` | **OK**: после logout при остановленном Valkey, рестарта API и возврата прежнего Valkey-тома старый токен получает 401 на `/users/me`, чат и POST сообщения | `evidence/test-degradation-grade5.txt` |
| Трек 5: 100k, два плана, backup/restore | `sh scripts/research-history-backup.sh` | 100 000 сообщений, 101 чат (90 000/100); composite index: 0.128 мс, 6 buffers; chat-only alternative: 89.728 мс, 1525 buffers; source/restore totals совпали | `evidence/research-history-backup.txt` |
| Нагрузочный прогон (не оценочный критерий) | `python3 tests/load.py … --duration 20 --concurrency 4` | **25.5 успешных req/s, 523 запроса, 0 ошибок**, p50 **125 мс**, p95 **328 мс**, p99 **484 мс** | `evidence/load.json`, `evidence/load.txt` |
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
Дополнительно: `docker compose --env-file .env ps` показывает `api/postgres/valkey — healthy`; на стенде 204 пользователя, 68 чатов, 13 286 сообщений, 174 сессии (синтетические).
## Эксперименты и отказы
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
Методика: закрытый цикл `POST /api/v1/chats/{id}/messages`, 4 потока, 20 с, без ретраев, клиент — `tests/load.py`; латентность считается только по успешным запросам. Хост: Intel Pentium Silver N5030 @ 1.10 GHz, 4 логических ядра, ~7.6 GiB RAM; Docker Desktop, контейнеры без CPU/RAM-лимитов. Прогрев — один smoke-проход и два чтения `/users/me` перед замером. Результаты одного прогона, повторов не делалось (см. ограничения).
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
| --- | --- | --- | --- |
| TODO | TODO | TODO | TODO |
| `docker compose stop valkey` | `api` продолжает работать через PostgreSQL, readiness 200, ошибка кэша не считается успехом авторизации | после breaker-а p50 **39 мс** (без него — 1031 мс: две попытки подключения на запрос), 60 из 60 запросов 200 | восстановление `compose start valkey` — 0.6–3 с; данных не теряется |
| `docker compose stop postgres` | ограниченная ошибка, никогда 200 и никогда сырая 500 | `/health/ready` → **503 `unavailable`**, запись → **503 за 2.05 с**, в ответе нет ни `psycopg`, ни `postgresql://`, ни `Traceback`; неизвестный токен — 401/503, но не 200 | `compose start postgres` → ready 200 за 0.58–1.75 с, объём не тронут (volumes) |
| Полный рестарт стека (`down` + `up -d --wait`) | сессия и данные те же, volumes сохраняются | `PASS: persistence preserved confirmed data and existing session` | ~20 с, потеря данных: 0 |
| Гонка 20 параллельных регистраций одного username | ровно один `INSERT`, остальные 409 | 1 пользователь в БД, ответы 201/409 | не применимо |
| Регистрация чата с неизвестным участником | откат всей операции | в `chats`/`chat_members` 0 строк | не применимо |
| Создание записи в `chats`/`messages` с обычной нагрузкой | закрытый цикл выдерживает | 25.5 rps, p95 328 мс, 0 ошибок за 20 с | не применимо |
Сиреневой AOF (`--appendfsync everysec`) в Valkey: потеря до ~1 с кэша при жёстком выключении ВМ не критична, так как PostgreSQL — источник истины.
## Трек 5 — План запроса и восстановление
**Гипотеза.** Для глубокой keyset-страницы в неравномерном распределении чатов составной индекс `(chat_id, created_at, id)` будет заметно дешевле индекса только по `chat_id`, потому что сразу удовлетворяет фильтр, порядок и курсор.
**Методика.** `scripts/research-history-backup.sql` создаёт идемпотентный синтетический набор: 90 000 сообщений в одном hot-чате и 100 чатов по 100 сообщений (всего 100 000, 101 чат). Для одного и того же cursor после 80 000 сообщений сняты `EXPLAIN (ANALYZE, BUFFERS)` production-индекса и временной альтернативы `chat_id`-only; изменение второй схемы происходит в транзакции и завершается `ROLLBACK`, поэтому production-индекс не меняется. Далее `pg_dump -Fc` восстанавливается в новый `messenger_grade5_restore`, а число сообщений, минимальная/максимальная дата и сумма длин текстов сравниваются между источником и восстановленной БД.
**Результат.** Production composite index дал `Index Scan`, 6 shared buffers и 0.128 мс. Альтернатива дала `Parallel Seq Scan + top-N Sort`, 1525 buffers и 89.728 мс. Backup/restore сохранил одинаковые агрегаты: `100000 | 2026-02-01T00:00:00.001Z | 2026-02-02T00:00:10.1Z | 1697294`.
**Ограничения.** Замер выполнен один раз на Docker Desktop, без CPU/RAM-лимитов и с прогретым buffer cache; абсолютные времена не являются производственной оценкой. Восстановление проверяет данные приложения, но не моделирует потерю WAL между последним backup и аварией.
## Ограничения и следующие шаги
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
- **Выдерживается:** отказ Valkey (полный обход кэша), кратковременный отказ PostgreSQL (ограниченная ошибка и быстрый возврат), рестарт всего стека с сохранением данных. **Не выдерживается:** одновременный выход обоих хранилищ из-под `up --wait` (стенд перестаёт готовиться — это ожидаемо), потеря PostgreSQL-тома вместе с контейнером.
- Один процесс `api`: локальный кэш отозванных токенов и circuit breaker не разделяются между репликами. Для уровня 3 с несколькими репликами (`X-Instance-Id`, failover) нужно вынести отзов в общее хранилище и согласовать поведение между инстансами.
- Нагрузочный тест закрытого цикла содержит coordinated omission и накладные издержки клиента; это не доказательство ёмкости. Повторов и замера CPU/RAM не делалось — цифры выше относятся к одному прогону.
- Исследование трека 5 использует 100 000 синтетических сообщений и отдельную restored-БД, но не моделирует репликацию PostgreSQL или point-in-time recovery.
- Кэш-запись — best effort: при недоступном Valkey запись пропускается, а не блокирует ответ (источник истины не затронут).
- `make` в среде отсутствует, поэтому все команды выполнялись напрямую; `Makefile` сохранён для Linux/CI.
+104
View File
@@ -0,0 +1,104 @@
# Local laboratory stand for lab 2.
# Roles: api (application), postgres (source of truth), valkey (session cache).
# Secrets are generated into ./secrets by scripts/bootstrap-secrets.sh and are
# mounted as files, so no credential ever appears in this file or in `docker inspect`.
services:
api:
build:
context: .
image: mtusi-messenger:lab2
environment:
APP_HOST: 0.0.0.0
APP_PORT: "8080"
APP_ENV: ${APP_ENV:-lab}
SESSION_STORE: ${SESSION_STORE:-postgres}
SESSION_TTL_SECONDS: ${SESSION_TTL_SECONDS:-3600}
CACHE_TTL_SECONDS: ${CACHE_TTL_SECONDS:-300}
DB_POOL_MIN: ${DB_POOL_MIN:-1}
DB_POOL_MAX: ${DB_POOL_MAX:-10}
DB_POOL_TIMEOUT_SECONDS: ${DB_POOL_TIMEOUT_SECONDS:-2}
DB_CONNECT_TIMEOUT_SECONDS: ${DB_CONNECT_TIMEOUT_SECONDS:-2}
DB_STATEMENT_TIMEOUT_MS: ${DB_STATEMENT_TIMEOUT_MS:-5000}
CACHE_TIMEOUT_SECONDS: ${CACHE_TIMEOUT_SECONDS:-0.5}
DATABASE_URL_FILE: /run/secrets/database_url
CACHE_URL_FILE: /run/secrets/cache_url
secrets:
- database_url
- cache_url
ports:
- "127.0.0.1:${API_PUBLISH_PORT:-8080}:8080"
depends_on:
postgres:
condition: service_healthy
valkey:
condition: service_healthy
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/health/ready', timeout=2).read()"]
interval: 5s
timeout: 5s
start_period: 20s
retries: 3
restart: unless-stopped
networks:
- backend
postgres:
image: postgres:17.6-bookworm@sha256:f3bd19c606e442c3d7bdfa8002e03fe260a1023351e0ea4598032022b68dd6e3
environment:
POSTGRES_DB: messenger
POSTGRES_USER: messenger
POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password
POSTGRES_INITDB_ARGS: "--encoding=UTF8 --locale=C"
secrets:
- postgres_password
volumes:
- pgdata:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U messenger -d messenger --timeout=3"]
interval: 5s
timeout: 5s
start_period: 10s
retries: 12
restart: unless-stopped
networks:
- backend
valkey:
image: valkey/valkey:8.1.3@sha256:fea8b3e67b15729d4bb70589eb03367bab9ad1ee89c876f54327fc7c6e618571
# Valkey is a cache here; AOF is enabled so a restart is still fast and warm,
# but PostgreSQL remains the durable source of truth for sessions.
command:
- sh
- -c
- 'exec valkey-server --requirepass "$$(cat /run/secrets/valkey_password)" --appendonly yes --appendfsync everysec --save "" --dir /data'
secrets:
- valkey_password
volumes:
- valkeydata:/data
healthcheck:
test: ["CMD-SHELL", 'valkey-cli --no-auth-warning -a "$$(cat /run/secrets/valkey_password)" ping | grep -q PONG']
interval: 5s
timeout: 5s
start_period: 5s
retries: 12
restart: unless-stopped
networks:
- backend
secrets:
database_url:
file: ./secrets/database_url
cache_url:
file: ./secrets/cache_url
postgres_password:
file: ./secrets/postgres_password
valkey_password:
file: ./secrets/valkey_password
volumes:
pgdata:
valkeydata:
networks:
backend:
driver: bridge
+7
View File
@@ -0,0 +1,7 @@
#!/bin/sh
# Apply versioned migrations (idempotent, advisory-locked), then hand PID 1 to
# the server so SIGTERM reaches uvicorn directly and shutdown stays graceful.
set -eu
python -B -m src.migrate
exec python -B src/app.py
+73
View File
@@ -0,0 +1,73 @@
Timing is on.
INSERT 0 1
Time: 11.593 ms
INSERT 0 1
Time: 3.779 ms
INSERT 0 1
Time: 8.619 ms
DO
Time: 1353.910 ms (00:01.354)
ANALYZE
Time: 77.416 ms
=== dataset ===
messages_in_chat
------------------
12000
(1 row)
Time: 4.243 ms
=== index used by the history query ===
indexname | indexdef
-------------------------+-----------------------------------------------------------------------------------------------
messages_pkey | CREATE UNIQUE INDEX messages_pkey ON public.messages USING btree (id)
messages_chat_order_idx | CREATE INDEX messages_chat_order_idx ON public.messages USING btree (chat_id, created_at, id)
(2 rows)
Time: 13.962 ms
=== plan: first page (no cursor) ===
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=0.29..6.36 rows=50 width=73) (actual time=0.025..0.061 rows=50 loops=1)
Buffers: shared hit=4
-> Index Scan using messages_chat_order_idx on messages (cost=0.29..1459.39 rows=12000 width=73) (actual time=0.022..0.048 rows=50 loops=1)
Index Cond: (chat_id = '22222222-2222-4222-8222-222222222222'::uuid)
Buffers: shared hit=4
Planning:
Buffers: shared hit=22
Planning Time: 0.474 ms
Execution Time: 0.165 ms
(9 rows)
Time: 5.762 ms
Time: 11.210 ms
=== plan: deep keyset page after 10 000 records (what the API sends) ===
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=0.29..15.60 rows=50 width=73) (actual time=0.097..0.133 rows=50 loops=1)
Buffers: shared hit=4
-> Index Scan using messages_chat_order_idx on messages (cost=0.29..769.32 rows=2510 width=73) (actual time=0.095..0.121 rows=50 loops=1)
Index Cond: ((chat_id = '22222222-2222-4222-8222-222222222222'::uuid) AND (ROW(created_at, id) > ROW('2026-01-01 00:00:10.001+00'::timestamp with time zone, 'eab21e64-f0cb-4070-9044-c8c3a2daac6d'::uuid)))
Buffers: shared hit=4
Planning Time: 0.270 ms
Execution Time: 0.208 ms
(7 rows)
Time: 2.785 ms
=== comparison: the OFFSET form this API does not use ===
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------
Limit (cost=1201.12..1201.24 rows=50 width=73) (actual time=13.641..13.660 rows=50 loops=1)
Buffers: shared hit=205
-> Sort (cost=1176.12..1206.12 rows=12000 width=73) (actual time=11.589..12.757 rows=10050 loops=1)
Sort Key: created_at, id
Sort Method: quicksort Memory: 1978kB
Buffers: shared hit=205
-> Seq Scan on messages (cost=0.00..363.08 rows=12000 width=73) (actual time=0.188..5.317 rows=12000 loops=1)
Filter: (chat_id = '22222222-2222-4222-8222-222222222222'::uuid)
Rows Removed by Filter: 646
Buffers: shared hit=205
Planning Time: 0.106 ms
Execution Time: 14.005 ms
(12 rows)
Time: 14.848 ms
+17
View File
@@ -0,0 +1,17 @@
{
"lab": 2,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 20.0,
"elapsed_seconds": 20.5,
"concurrency": 4,
"successful_requests": 523,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 25.512,
"successful_latency_ms": {
"p50": 125.0,
"p95": 328.0,
"p99": 484.0
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
}
+18
View File
@@ -0,0 +1,18 @@
$ python tests/load.py --base-url http://127.0.0.1:8080 --duration 20 --concurrency 4
{
"lab": 2,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 20.0,
"elapsed_seconds": 20.5,
"concurrency": 4,
"successful_requests": 523,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 25.512,
"successful_latency_ms": {
"p50": 125.0,
"p95": 328.0,
"p99": 484.0
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
}
+3
View File
@@ -0,0 +1,3 @@
$ python tests/resilience.py --mode persistence --action-script scripts/restart.cmd --base-url http://127.0.0.1:8080
Executing explicit persistence hook; hook output suppressed to avoid leaking secrets.
PASS: persistence preserved confirmed data and existing session. Hook scope needs operator evidence.
+87
View File
@@ -0,0 +1,87 @@
Timing is on.
INSERT 0 1
Time: 13.869 ms
INSERT 0 1
Time: 4.424 ms
INSERT 0 1
Time: 12.415 ms
INSERT 0 100
Time: 21.993 ms
INSERT 0 100
Time: 28.881 ms
INSERT 0 90000
Time: 13459.359 ms (00:13.459)
INSERT 0 10000
Time: 1345.306 ms (00:01.345)
ANALYZE
Time: 406.609 ms
=== dataset shape ===
research_messages | research_chats | largest_chat_messages | smallest_chat_messages
-------------------+----------------+-----------------------+------------------------
100000 | 101 | 90000 | 100
(1 row)
Time: 266.713 ms
=== source integrity summary ===
count | first_message | last_message | text_bytes
--------+----------------------------+--------------------------+------------
100000 | 2026-02-01 00:00:00.001+00 | 2026-02-02 00:00:10.1+00 | 1697294
(1 row)
Time: 75.790 ms
=== index A: production composite (chat_id, created_at, id) ===
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=0.42..17.44 rows=50 width=73) (actual time=0.022..0.060 rows=50 loops=1)
Buffers: shared hit=6
-> Index Scan using messages_chat_order_idx on messages (cost=0.42..5720.16 rows=16804 width=73) (actual time=0.020..0.048 rows=50 loops=1)
Index Cond: ((chat_id = 'bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb'::uuid) AND (ROW(created_at, id) > ROW('2026-02-01 00:01:20+00'::timestamp with time zone, '1b846638-5f59-3b99-00f4-160ef4a51437'::uuid)))
Buffers: shared hit=6
Planning:
Buffers: shared hit=6
Planning Time: 0.352 ms
Execution Time: 0.128 ms
(9 rows)
Time: 1.572 ms
=== index B: chat_id-only alternative (temporary, rolled back) ===
BEGIN
Time: 0.244 ms
DROP INDEX
Time: 19.803 ms
CREATE INDEX
Time: 223.602 ms
ANALYZE
Time: 342.025 ms
QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=4006.30..4012.05 rows=50 width=73) (actual time=70.068..89.652 rows=50 loops=1)
Buffers: shared hit=1525
-> Gather Merge (cost=4006.30..5145.03 rows=9902 width=73) (actual time=70.063..89.630 rows=50 loops=1)
Workers Planned: 1
Workers Launched: 1
Buffers: shared hit=1525
-> Sort (cost=3006.29..3031.05 rows=9902 width=73) (actual time=42.016..42.177 rows=41 loops=2)
Sort Key: created_at, id
Sort Method: top-N heapsort Memory: 31kB
Buffers: shared hit=1525
Worker 0: Sort Method: top-N heapsort Memory: 31kB
-> Parallel Seq Scan on messages (cost=0.00..2677.35 rows=9902 width=73) (actual time=31.887..40.024 rows=5000 loops=2)
Filter: ((chat_id = 'bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb'::uuid) AND (ROW(created_at, id) > ROW('2026-02-01 00:01:20+00'::timestamp with time zone, '1b846638-5f59-3b99-00f4-160ef4a51437'::uuid)))
Rows Removed by Filter: 51700
Buffers: shared hit=1510
Planning:
Buffers: shared hit=20 read=1
Planning Time: 0.790 ms
Execution Time: 89.728 ms
(19 rows)
Time: 92.602 ms
ROLLBACK
Time: 4.178 ms
=== backup ===
NOTICE: database "messenger_grade5_restore" does not exist, skipping
=== source integrity ===
100000|2026-02-01 00:00:00.001+00|2026-02-02 00:00:10.1+00|1697294
=== restored integrity ===
100000|2026-02-01 00:00:00.001+00|2026-02-02 00:00:10.1+00|1697294
+2
View File
@@ -0,0 +1,2 @@
$ python tests/smoke.py --base-url http://127.0.0.1:8080
PASS: lab2 public API scenario; infrastructure and grading evidence remain separate.
+34
View File
@@ -0,0 +1,34 @@
$ BASE_URL=http://127.0.0.1:8080 python -B -m unittest -v test_api test_race
test_basic_is_not_accepted_outside_session_creation (test_api.Authentication.test_basic_is_not_accepted_outside_session_creation) ... ok
test_logout_revokes_only_that_session (test_api.Authentication.test_logout_revokes_only_that_session) ... ok
test_missing_and_unknown_tokens (test_api.Authentication.test_missing_and_unknown_tokens) ... ok
test_session_endpoint_rejects_json_body_and_bad_basic (test_api.Authentication.test_session_endpoint_rejects_json_body_and_bad_basic) ... ok
test_session_payload_and_expiry (test_api.Authentication.test_session_payload_and_expiry) ... ok
test_wrong_password_and_unknown_user_are_indistinguishable (test_api.Authentication.test_wrong_password_and_unknown_user_are_indistinguishable) ... ok
test_x_user_id_is_ignored (test_api.Authentication.test_x_user_id_is_ignored) ... ok
test_foreign_chat_is_hidden_everywhere (test_api.Chats.test_foreign_chat_is_hidden_everywhere) ... ok
test_membership_validation (test_api.Chats.test_membership_validation) ... ok
test_messages_are_validated_and_unicode_preserved (test_api.Chats.test_messages_are_validated_and_unicode_preserved) ... ok
test_health_endpoints_and_request_id (test_api.HealthAndEnvelope.test_health_endpoints_and_request_id) ... ok
test_query_parameters_rejected_where_not_supported (test_api.HealthAndEnvelope.test_query_parameters_rejected_where_not_supported) ... ok
test_unknown_route_uses_same_error_envelope (test_api.HealthAndEnvelope.test_unknown_route_uses_same_error_envelope) ... ok
test_cursor_is_bound_to_its_chat (test_api.Pagination.test_cursor_is_bound_to_its_chat) ... ok
test_limit_and_cursor_validation (test_api.Pagination.test_limit_and_cursor_validation) ... ok
test_walks_every_page_without_loss_or_duplicates (test_api.Pagination.test_walks_every_page_without_loss_or_duplicates) ... ok
test_duplicate_username_conflict (test_api.Registration.test_duplicate_username_conflict) ... ok
test_malformed_json_extra_fields_and_wrong_types (test_api.Registration.test_malformed_json_extra_fields_and_wrong_types) ... ok
test_password_rules (test_api.Registration.test_password_rules) ... ok
test_response_has_no_secret_fields (test_api.Registration.test_response_has_no_secret_fields) ... ok
test_unsupported_media_type (test_api.Registration.test_unsupported_media_type) ... ok
test_username_rules (test_api.Registration.test_username_rules) ... ok
test_100_messages_from_10_clients_are_exactly_100 (test_race.ConcurrentWrites.test_100_messages_from_10_clients_are_exactly_100) ...
concurrent writes: 100 messages from 10 clients in 3.08s
ok
test_bounded_pool_serves_parallel_reads (test_race.ConcurrentWrites.test_bounded_pool_serves_parallel_reads) ... ok
test_chat_creation_rolls_back_on_unknown_member (test_race.ConcurrentWrites.test_chat_creation_rolls_back_on_unknown_member) ... ok
test_duplicate_username_race_creates_one_user (test_race.ConcurrentWrites.test_duplicate_username_race_creates_one_user) ... ok
----------------------------------------------------------------------
Ran 26 tests in 14.846s
OK
+10
View File
@@ -0,0 +1,10 @@
test_logout_remains_revoked_after_cache_outage_and_api_restart (test_degradation.DependencyOutages.test_logout_remains_revoked_after_cache_outage_and_api_restart) ... ok
test_valkey_loss_degrades_then_postgres_loss_is_bounded (test_degradation.DependencyOutages.test_valkey_loss_degrades_t
hen_postgres_loss_is_bounded) ... ok
----------------------------------------------------------------------
Ran 2 tests in 40.822s
OK
outage summary: {"cached_p50_ms": 15.0, "cache_down_p50_ms": 15.0, "cache_down_p95_ms": 32.0, "db_down_write_seconds": 2.06, "recovery_seconds": 3.92}
+9
View File
@@ -0,0 +1,9 @@
$ BASE_URL=http://127.0.0.1:8080 STUDENT_DOCKER_TESTS=1 python -B -m unittest -v test_degradation
test_valkey_loss_degrades_then_postgres_loss_is_bounded (test_degradation.DependencyOutages.test_valkey_loss_degrades_then_postgres_loss_is_bounded) ...
outage summary: {"cached_p50_ms": 16.0, "cache_down_p50_ms": 39.0, "cache_down_p95_ms": 78.0, "db_down_write_seconds": 2.05, "recovery_seconds": 0.58}
ok
----------------------------------------------------------------------
Ran 1 test in 21.384s
OK
+66
View File
@@ -0,0 +1,66 @@
$ PYTHON=python sh scripts/test-ttl.sh
Volume mtusi-lab2-ttl_pgdata Creating
Network mtusi-lab2-ttl_backend Creating
Volume mtusi-lab2-ttl_valkeydata Creating
Volume mtusi-lab2-ttl_pgdata Creating
Network mtusi-lab2-ttl_backend Creating
Volume mtusi-lab2-ttl_valkeydata Creating
Volume mtusi-lab2-ttl_valkeydata Created
Volume mtusi-lab2-ttl_valkeydata Created
Volume mtusi-lab2-ttl_pgdata Created
Volume mtusi-lab2-ttl_pgdata Created
Network mtusi-lab2-ttl_backend Created
Network mtusi-lab2-ttl_backend Created
Container mtusi-lab2-ttl-valkey-1 Creating
Container mtusi-lab2-ttl-postgres-1 Creating
Container mtusi-lab2-ttl-valkey-1 Created
Container mtusi-lab2-ttl-postgres-1 Created
Container mtusi-lab2-ttl-api-1 Creating
Container mtusi-lab2-ttl-api-1 Created
Container mtusi-lab2-ttl-postgres-1 Starting
Container mtusi-lab2-ttl-valkey-1 Starting
Container mtusi-lab2-ttl-postgres-1 Started
Container mtusi-lab2-ttl-valkey-1 Started
Container mtusi-lab2-ttl-valkey-1 Waiting
Container mtusi-lab2-ttl-postgres-1 Waiting
Container mtusi-lab2-ttl-valkey-1 Healthy
Container mtusi-lab2-ttl-postgres-1 Healthy
Container mtusi-lab2-ttl-api-1 Starting
Container mtusi-lab2-ttl-api-1 Started
Container mtusi-lab2-ttl-api-1 Waiting
Container mtusi-lab2-ttl-postgres-1 Waiting
Container mtusi-lab2-ttl-valkey-1 Waiting
Container mtusi-lab2-ttl-postgres-1 Healthy
Container mtusi-lab2-ttl-valkey-1 Healthy
Container mtusi-lab2-ttl-api-1 Healthy
ttl_stack_ready base_url=http://127.0.0.1:8081 session_ttl_seconds=2
test_expiry_is_enforced_and_cache_does_not_extend_it (test_expiry.ShortTtlSessions.test_expiry_is_enforced_and_cache_does_not_extend_it) ... ok
test_logout_is_immediate_even_when_the_session_is_cached (test_expiry.ShortTtlSessions.test_logout_is_immediate_even_when_the_session_is_cached) ... ok
test_session_lookup_latency_is_bounded (test_expiry.ShortTtlSessions.test_session_lookup_latency_is_bounded) ... ok
test_wrong_password_issues_no_session (test_expiry.ShortTtlSessions.test_wrong_password_issues_no_session) ... ok
----------------------------------------------------------------------
Ran 4 tests in 6.121s
OK
/users/me warm cache p50=16.0ms p95=16.0ms
Container mtusi-lab2-ttl-api-1 Stopping
Container mtusi-lab2-ttl-api-1 Stopped
Container mtusi-lab2-ttl-api-1 Removing
Container mtusi-lab2-ttl-api-1 Removed
Container mtusi-lab2-ttl-valkey-1 Stopping
Container mtusi-lab2-ttl-postgres-1 Stopping
Container mtusi-lab2-ttl-valkey-1 Stopped
Container mtusi-lab2-ttl-valkey-1 Removing
Container mtusi-lab2-ttl-valkey-1 Removed
Container mtusi-lab2-ttl-postgres-1 Stopped
Container mtusi-lab2-ttl-postgres-1 Removing
Container mtusi-lab2-ttl-postgres-1 Removed
Network mtusi-lab2-ttl_backend Removing
Volume mtusi-lab2-ttl_pgdata Removing
Volume mtusi-lab2-ttl_valkeydata Removing
Volume mtusi-lab2-ttl_valkeydata Removed
Volume mtusi-lab2-ttl_pgdata Removed
Network mtusi-lab2-ttl_backend Removed
ttl_stack_removed
+4 -2
View File
@@ -1,2 +1,4 @@
# Only for full OpenAPI/JSON Schema validation; HTTP tests use stdlib.
openapi-spec-validator==0.7.2
-r requirements.txt
# Only for full OpenAPI/JSON Schema validation (make validate).
openapi-spec-validator==0.9.0
+27
View File
@@ -0,0 +1,27 @@
annotated-doc==0.0.5
annotated-types==0.8.0
anyio==4.15.1
argon2-cffi==25.1.0
argon2-cffi-bindings==26.1.0
cffi==2.1.1
click==8.5.0
fastapi==0.162.0
h11==0.16.0
httptools==0.9.0
idna==3.20
psycopg==3.3.6
psycopg-binary==3.3.6
psycopg-pool==3.3.3
pycparser==3.12
pydantic==2.14.2
pydantic_core==2.50.2
python-dotenv==1.2.4
PyYAML==6.0.3
redis==8.1.0
starlette==1.10.0
typing-inspection==0.4.4
typing_extensions==4.16.0
tzdata==2026.6
uvicorn==0.54.0
watchfiles==1.3.0
websockets==17.3
+48
View File
@@ -0,0 +1,48 @@
#!/bin/sh
# Generate the local, synthetic stand's secrets and .env.
# Values are written to ./secrets (git-ignored, mode 0600) and never printed.
# Re-running is safe: existing files are kept unless FORCE=1.
set -eu
cd "$(dirname "$0")/.."
PYTHON="${PYTHON:-python}"
SECRETS_DIR="${SECRETS_DIR:-secrets}"
if [ ! -f .env ]; then
cp .env.example .env
printf '%s\n' "created .env from .env.example"
fi
mkdir -p "$SECRETS_DIR"
chmod 700 "$SECRETS_DIR" 2>/dev/null || true
umask 077
random_password() {
"$PYTHON" -c 'import secrets; print(secrets.token_hex(24))'
}
write_if_missing() {
path="$SECRETS_DIR/$1"
if [ -f "$path" ] && [ "${FORCE:-0}" != "1" ]; then
return 0
fi
printf '%s' "$2" > "$path"
chmod 600 "$path" 2>/dev/null || true
}
if [ -f "$SECRETS_DIR/database_url" ] && [ -f "$SECRETS_DIR/cache_url" ] && [ "${FORCE:-0}" != "1" ]; then
printf '%s\n' "secrets already present in $SECRETS_DIR (FORCE=1 to regenerate)"
exit 0
fi
postgres_password="$(random_password)"
valkey_password="$(random_password)"
write_if_missing postgres_password "$postgres_password"
write_if_missing valkey_password "$valkey_password"
write_if_missing database_url "postgresql://messenger:${postgres_password}@postgres:5432/messenger"
write_if_missing cache_url "redis://:${valkey_password}@valkey:6379/0"
printf '%s\n' "wrote secrets to $SECRETS_DIR: postgres_password, valkey_password, database_url, cache_url"
printf '%s\n' "next: docker compose --env-file .env up -d --build"
+15
View File
@@ -0,0 +1,15 @@
#!/bin/sh
# Seed 12 000 synthetic messages in a dedicated chat and capture the plans for
# the first page, a deep keyset page and the naive OFFSET alternative.
# Output goes to evidence/explain-history.txt (text, no secrets).
set -eu
cd "$(dirname "$0")/.."
mkdir -p evidence
docker compose --env-file .env exec -T postgres \
psql -X -U messenger -d messenger -v ON_ERROR_STOP=1 -f - \
< scripts/explain-history.sql > evidence/explain-history.txt 2>&1
cat evidence/explain-history.txt
printf '%s\n' "saved evidence/explain-history.txt"
+75
View File
@@ -0,0 +1,75 @@
-- Index justification for messages (chat_id, created_at, id).
-- Fixtures are deterministic and synthetic; re-running changes nothing.
\set ON_ERROR_STOP on
\timing on
INSERT INTO users (id, username, display_name, password_hash, created_at)
VALUES ('11111111-1111-4111-8111-111111111111', 'seed_user', 'Seed User',
'not-a-usable-hash', now())
ON CONFLICT (id) DO NOTHING;
INSERT INTO chats (id, title, created_at)
VALUES ('22222222-2222-4222-8222-222222222222', 'Seed history', now())
ON CONFLICT (id) DO NOTHING;
INSERT INTO chat_members (chat_id, user_id, joined_at)
VALUES ('22222222-2222-4222-8222-222222222222',
'11111111-1111-4111-8111-111111111111', now())
ON CONFLICT DO NOTHING;
DO $$
BEGIN
IF (SELECT count(*) FROM messages
WHERE chat_id = '22222222-2222-4222-8222-222222222222') = 0 THEN
INSERT INTO messages (id, chat_id, sender_id, text, created_at)
SELECT gen_random_uuid(),
'22222222-2222-4222-8222-222222222222',
'11111111-1111-4111-8111-111111111111',
'seed message ' || g,
timestamptz '2026-01-01 00:00:00+00' + make_interval(secs => g * 0.001)
FROM generate_series(1, 12000) AS g;
END IF;
END $$;
ANALYZE messages;
\echo '=== dataset ==='
SELECT count(*) AS messages_in_chat
FROM messages
WHERE chat_id = '22222222-2222-4222-8222-222222222222';
\echo '=== index used by the history query ==='
SELECT indexname, indexdef FROM pg_indexes WHERE tablename = 'messages';
\echo '=== plan: first page (no cursor) ==='
EXPLAIN (ANALYZE, BUFFERS)
SELECT id, chat_id, sender_id, text, created_at
FROM messages
WHERE chat_id = '22222222-2222-4222-8222-222222222222'
ORDER BY created_at, id
LIMIT 50;
-- Pick the 10 001st record as the cursor of a deep page.
SELECT created_at AS cursor_ts, id AS cursor_id
FROM messages
WHERE chat_id = '22222222-2222-4222-8222-222222222222'
ORDER BY created_at, id
OFFSET 10000 LIMIT 1
\gset
\echo '=== plan: deep keyset page after 10 000 records (what the API sends) ==='
EXPLAIN (ANALYZE, BUFFERS)
SELECT id, chat_id, sender_id, text, created_at
FROM messages
WHERE chat_id = '22222222-2222-4222-8222-222222222222'
AND (created_at, id) > (:'cursor_ts'::timestamptz, :'cursor_id'::uuid)
ORDER BY created_at, id
LIMIT 50;
\echo '=== comparison: the OFFSET form this API does not use ==='
EXPLAIN (ANALYZE, BUFFERS)
SELECT id, chat_id, sender_id, text, created_at
FROM messages
WHERE chat_id = '22222222-2222-4222-8222-222222222222'
ORDER BY created_at, id
OFFSET 10000 LIMIT 50;
+35
View File
@@ -0,0 +1,35 @@
#!/bin/sh
# Grade-5 track 2: build the 100k uneven fixture, compare two index choices,
# then backup and restore into a separate database and compare integrity sums.
# The normal messenger database and its named volume are never removed.
set -eu
cd "$(dirname "$0")/.."
ENV_FILE="${COMPOSE_ENV_FILE:-.env}"
mkdir -p evidence
docker compose --env-file "$ENV_FILE" exec -T postgres \
psql -X -U messenger -d messenger -v ON_ERROR_STOP=1 -f - \
< scripts/research-history-backup.sql > evidence/research-history-backup.txt 2>&1
{
printf '%s\n' '=== backup ==='
docker compose --env-file "$ENV_FILE" exec -T postgres sh -ceu '
rm -f /tmp/lab2-grade5.dump
pg_dump -U messenger -d messenger --format=custom --file=/tmp/lab2-grade5.dump
dropdb -U messenger --if-exists messenger_grade5_restore
createdb -U messenger messenger_grade5_restore
pg_restore -U messenger -d messenger_grade5_restore --exit-on-error /tmp/lab2-grade5.dump
'
printf '%s\n' '=== source integrity ==='
docker compose --env-file "$ENV_FILE" exec -T postgres \
psql -X -U messenger -d messenger -At -F '|' -c \
"SELECT count(*), min(created_at), max(created_at), sum(length(text)) FROM messages WHERE sender_id = 'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa';"
printf '%s\n' '=== restored integrity ==='
docker compose --env-file "$ENV_FILE" exec -T postgres \
psql -X -U messenger -d messenger_grade5_restore -At -F '|' -c \
"SELECT count(*), min(created_at), max(created_at), sum(length(text)) FROM messages WHERE sender_id = 'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa';"
} >> evidence/research-history-backup.txt 2>&1
cat evidence/research-history-backup.txt
printf '%s\n' 'saved evidence/research-history-backup.txt'
+105
View File
@@ -0,0 +1,105 @@
-- Grade-5 track 2: query-plan comparison on >=100k messages with highly
-- uneven chat sizes. The deterministic fixture is isolated by username/title
-- prefixes and can safely be re-run on the same persistent volume.
\set ON_ERROR_STOP on
\timing on
INSERT INTO users (id, username, display_name, password_hash, created_at)
VALUES ('aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa', 'research_seed_user',
'Research seed user', 'not-a-usable-hash', now())
ON CONFLICT (id) DO NOTHING;
INSERT INTO chats (id, title, created_at)
VALUES ('bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb', 'research-hot-chat', now())
ON CONFLICT (id) DO NOTHING;
INSERT INTO chat_members (chat_id, user_id, joined_at)
VALUES ('bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb',
'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa', now())
ON CONFLICT DO NOTHING;
-- One hot chat with 90k messages and 100 cold chats with 100 messages each.
-- md5() yields stable 32 hex characters and PostgreSQL accepts it as UUID.
INSERT INTO chats (id, title, created_at)
SELECT md5('research-cold-chat-' || n)::uuid, 'research-cold-chat-' || n, now()
FROM generate_series(1, 100) AS n
ON CONFLICT (id) DO NOTHING;
INSERT INTO chat_members (chat_id, user_id, joined_at)
SELECT md5('research-cold-chat-' || n)::uuid,
'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa', now()
FROM generate_series(1, 100) AS n
ON CONFLICT DO NOTHING;
INSERT INTO messages (id, chat_id, sender_id, text, created_at)
SELECT md5('research-hot-message-' || n)::uuid,
'bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb',
'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa',
'hot message ' || n,
timestamptz '2026-02-01 00:00:00+00' + make_interval(secs => n * 0.001)
FROM generate_series(1, 90000) AS n
ON CONFLICT (id) DO NOTHING;
INSERT INTO messages (id, chat_id, sender_id, text, created_at)
SELECT md5('research-cold-message-' || chat_n || '-' || message_n)::uuid,
md5('research-cold-chat-' || chat_n)::uuid,
'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa',
'cold message ' || chat_n || '-' || message_n,
timestamptz '2026-02-02 00:00:00+00'
+ make_interval(secs => (chat_n * 100 + message_n) * 0.001)
FROM generate_series(1, 100) AS chat_n,
generate_series(1, 100) AS message_n
ON CONFLICT (id) DO NOTHING;
ANALYZE messages;
\echo '=== dataset shape ==='
SELECT count(*) AS research_messages,
count(DISTINCT chat_id) AS research_chats,
max(per_chat.count) AS largest_chat_messages,
min(per_chat.count) AS smallest_chat_messages
FROM messages
JOIN (
SELECT chat_id, count(*)
FROM messages
WHERE sender_id = 'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa'
GROUP BY chat_id
) AS per_chat USING (chat_id)
WHERE messages.sender_id = 'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa';
\echo '=== source integrity summary ==='
SELECT count(*) AS count,
min(created_at) AS first_message,
max(created_at) AS last_message,
sum(length(text)) AS text_bytes
FROM messages
WHERE sender_id = 'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa';
\echo '=== index A: production composite (chat_id, created_at, id) ==='
EXPLAIN (ANALYZE, BUFFERS)
SELECT id, chat_id, sender_id, text, created_at
FROM messages
WHERE chat_id = 'bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb'
AND (created_at, id) > (
timestamptz '2026-02-01 00:01:20+00',
md5('research-hot-message-80000')::uuid
)
ORDER BY created_at, id
LIMIT 50;
\echo '=== index B: chat_id-only alternative (temporary, rolled back) ==='
BEGIN;
DROP INDEX messages_chat_order_idx;
CREATE INDEX messages_research_chat_only_idx ON messages (chat_id);
ANALYZE messages;
EXPLAIN (ANALYZE, BUFFERS)
SELECT id, chat_id, sender_id, text, created_at
FROM messages
WHERE chat_id = 'bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb'
AND (created_at, id) > (
timestamptz '2026-02-01 00:01:20+00',
md5('research-hot-message-80000')::uuid
)
ORDER BY created_at, id
LIMIT 50;
ROLLBACK;
+17
View File
@@ -0,0 +1,17 @@
@echo off
rem Windows copy of scripts/restart.sh for `make test-persistence`.
rem Recreates the API and BOTH state stores from the same images, keeping the
rem named volumes (pgdata, valkeydata): no `down -v`, no seed, no data reset.
setlocal
cd /d "%~dp0.."
echo restart: stopping api, postgres and valkey (volumes preserved)
docker compose --env-file .env down --timeout 15
if errorlevel 1 exit /b 1
echo restart: starting api, postgres and valkey
docker compose --env-file .env up -d --wait --wait-timeout 120
if errorlevel 1 exit /b 1
docker compose --env-file .env ps
echo stack_restarted volumes_preserved
+17
View File
@@ -0,0 +1,17 @@
#!/bin/sh
# Restart hook for `make test-persistence ACTION_SCRIPT=scripts/restart.sh`.
# Recreates the API and BOTH state stores from the same images, keeping the
# named volumes (pgdata, valkeydata): no `down -v`, no seed, no data reset.
set -eu
cd "$(dirname "$0")/.."
ENV_FILE="${COMPOSE_ENV_FILE:-.env}"
printf '%s\n' "restart: stopping api, postgres and valkey (volumes preserved)"
docker compose --env-file "$ENV_FILE" down --timeout 15
printf '%s\n' "restart: starting api, postgres and valkey"
docker compose --env-file "$ENV_FILE" up -d --wait --wait-timeout 120
docker compose --env-file "$ENV_FILE" ps
printf '%s\n' "stack_restarted volumes_preserved"
+16
View File
@@ -0,0 +1,16 @@
#!/bin/sh
# Start the dedicated short-TTL stand, run the expiry tests, always tear it down.
set -eu
cd "$(dirname "$0")/.."
PORT="${TTL_PORT:-8081}"
status=0
sh scripts/ttl-stack.sh start
cd student-tests
BASE_URL="http://127.0.0.1:$PORT" "${PYTHON:-python}" -B -m unittest -v test_expiry || status=$?
cd ..
sh scripts/ttl-stack.sh stop
exit "$status"
+27
View File
@@ -0,0 +1,27 @@
#!/bin/sh
# Separate short-TTL stand used by `make test-ttl`.
# It is a distinct Compose project on its own port with its own disposable
# volumes, so the main stand and its synthetic data are untouched.
set -eu
cd "$(dirname "$0")/.."
PROJECT="${TTL_PROJECT:-mtusi-lab2-ttl}"
PORT="${TTL_PORT:-8081}"
ACTION="${1:-}"
case "$ACTION" in
start)
SESSION_TTL_SECONDS=2 API_PUBLISH_PORT="$PORT" \
docker compose -p "$PROJECT" --env-file .env up -d --wait --wait-timeout 180
printf 'ttl_stack_ready base_url=http://127.0.0.1:%s session_ttl_seconds=2\n' "$PORT"
;;
stop)
# -v removes only this dedicated synthetic stand's volumes.
docker compose -p "$PROJECT" --env-file .env down -v --timeout 10
printf 'ttl_stack_removed\n'
;;
*)
printf 'usage: %s start|stop\n' "$0" >&2
exit 2
;;
esac
+1
View File
@@ -0,0 +1 @@
"""MTUSI messenger, laboratory 2: PostgreSQL, sessions and Valkey."""
+349
View File
@@ -0,0 +1,349 @@
"""Routes, authentication and request validation for lab 2."""
from __future__ import annotations
import base64
import binascii
import json
import re
import uuid
from dataclasses import dataclass
from typing import Annotated, Any, cast
from fastapi import APIRouter, Depends, Request, Response
from psycopg_pool import AsyncConnectionPool
from pydantic import ValidationError
from starlette.concurrency import run_in_threadpool
from . import sessions as session_service
from . import store
from .config import DEFAULT_PAGE_SIZE, MAX_JSON_BYTES, MAX_PAGE_SIZE
from .db import ping
from .errors import BASIC_CHALLENGE, BEARER_CHALLENGE, ApiProblem
from .models import CreateChat, CreateMessage, CreateUser, StrictModel
from .pagination import build_page, decode_cursor, format_timestamp
from .security import hash_password, verify_password, verify_password_unknown_user
router = APIRouter()
USERNAME_PATTERN = re.compile(r"^[a-z0-9_]{3,32}$")
@dataclass(frozen=True)
class Principal:
"""The authenticated identity behind a Bearer token."""
user_id: uuid.UUID
user: dict[str, Any]
def get_pool(request: Request) -> AsyncConnectionPool:
return cast(AsyncConnectionPool, request.app.state.pool)
def get_cache(request: Request) -> session_service.SessionCache:
return cast(session_service.SessionCache, request.app.state.cache)
async def drain_body(request: Request) -> None:
"""Consume an unwanted request body without retaining it in memory."""
async for _chunk in request.stream():
pass
def declared_body_size(request: Request) -> int | None:
value = request.headers.get("Content-Length")
if value is None:
return None
try:
size = int(value)
except ValueError:
raise ApiProblem(400, "invalid_request", "Content-Length is invalid") from None
if size < 0:
raise ApiProblem(400, "invalid_request", "Content-Length is invalid")
return size
async def read_limited_body(request: Request) -> bytes:
chunks: list[bytes] = []
size = 0
too_large = False
async for chunk in request.stream():
size += len(chunk)
if size <= MAX_JSON_BYTES:
chunks.append(chunk)
else:
too_large = True
if too_large:
raise ApiProblem(413, "payload_too_large", "Request body is too large")
return b"".join(chunks)
async def parse_body(request: Request, model: type[StrictModel]) -> Any:
content_length = declared_body_size(request)
if content_length is not None and content_length > MAX_JSON_BYTES:
await drain_body(request)
raise ApiProblem(413, "payload_too_large", "Request body is too large")
content_type = request.headers.get("Content-Type", "").split(";", 1)[0].strip()
if content_type.lower() != "application/json":
await drain_body(request)
raise ApiProblem(
415, "unsupported_media_type", "Content-Type must be application/json"
)
body = await read_limited_body(request)
try:
decoded = json.loads(body.decode("utf-8"))
except (json.JSONDecodeError, UnicodeDecodeError):
raise ApiProblem(
400, "invalid_request", "Request body is not valid JSON"
) from None
try:
return model.model_validate(decoded)
except ValidationError:
raise ApiProblem(400, "invalid_request", "Request fields are invalid") from None
def ensure_no_query(request: Request) -> None:
if request.url.query:
raise ApiProblem(400, "invalid_request", "Query parameters are not supported")
def parse_pagination_request(request: Request) -> tuple[int, str | None]:
"""Read limit/cursor only; unknown or repeated parameters are a 400."""
parameters = request.query_params
keys = [key for key, _ in parameters.multi_items()]
if set(keys) - {"limit", "cursor"} or any(
keys.count(key) != 1 for key in set(keys)
):
raise ApiProblem(400, "invalid_request", "Query parameters are invalid")
limit_value = parameters.get("limit", str(DEFAULT_PAGE_SIZE))
try:
limit = int(limit_value)
except (TypeError, ValueError):
raise ApiProblem(400, "invalid_request", "limit is invalid") from None
if not 1 <= limit <= MAX_PAGE_SIZE:
raise ApiProblem(400, "invalid_request", "limit is invalid")
cursor = parameters.get("cursor")
if cursor == "":
raise ApiProblem(400, "invalid_request", "cursor is invalid")
return limit, cursor
def require_chat_id(chat_id: str) -> uuid.UUID:
try:
return uuid.UUID(chat_id)
except (ValueError, AttributeError):
raise ApiProblem(400, "invalid_request", "chat_id must be a UUID") from None
def unauthorized(message: str, challenge: str) -> ApiProblem:
return ApiProblem(401, "unauthorized", message, {"WWW-Authenticate": challenge})
def bearer_token(request: Request) -> str:
"""Only Bearer is accepted; Basic here is a 401, not a fallback login."""
header = request.headers.get("Authorization")
if header is None:
raise unauthorized("Bearer session token is required", BEARER_CHALLENGE)
scheme, _, value = header.partition(" ")
if scheme.strip().lower() != "bearer" or not value.strip():
raise unauthorized("Bearer session token is required", BEARER_CHALLENGE)
return value.strip()
def basic_credentials(request: Request) -> tuple[str, str]:
"""Decode Basic credentials; it is an encoding, never a trusted identity."""
header = request.headers.get("Authorization")
if header is None:
raise unauthorized("Basic credentials are required", BASIC_CHALLENGE)
scheme, _, value = header.partition(" ")
if scheme.strip().lower() != "basic" or not value.strip():
raise unauthorized("Basic credentials are required", BASIC_CHALLENGE)
try:
raw = base64.b64decode(value.strip(), validate=True).decode("utf-8")
except (binascii.Error, UnicodeDecodeError, ValueError):
raise unauthorized("Basic credentials are invalid", BASIC_CHALLENGE) from None
username, separator, password = raw.partition(":")
if not separator or not username or not password:
raise unauthorized("Basic credentials are invalid", BASIC_CHALLENGE)
if USERNAME_PATTERN.fullmatch(username) is None:
# Unknown or malformed usernames must not be distinguished from a bad password.
raise unauthorized("Invalid username or password", BASIC_CHALLENGE)
return username, password
async def current_principal(request: Request) -> Principal:
token = bearer_token(request)
user_id, user = await session_service.resolve_session(
get_pool(request), get_cache(request), token
)
return Principal(user_id=user_id, user=user)
Actor = Annotated[Principal, Depends(current_principal)]
# --------------------------------------------------------------------------- #
# Health
# --------------------------------------------------------------------------- #
@router.get("/health/live")
async def liveness(request: Request) -> dict[str, str]:
ensure_no_query(request)
return {"status": "ok"}
@router.get("/health/ready")
async def readiness(request: Request) -> dict[str, str]:
ensure_no_query(request)
try:
await ping(get_pool(request))
except Exception as exc: # noqa: BLE001 - any DB failure means not ready
request.app.state.logger.warning(
"readiness_database_failed error=%s", type(exc).__name__
)
raise ApiProblem(503, "unavailable", "Database is unavailable") from None
return {"status": "ok"}
# --------------------------------------------------------------------------- #
# Users and sessions
# --------------------------------------------------------------------------- #
@router.post("/api/v1/users", status_code=201)
async def register_user(request: Request) -> dict[str, Any]:
ensure_no_query(request)
body = await parse_body(request, CreateUser)
assert isinstance(body, CreateUser)
password_hash = await run_in_threadpool(hash_password, body.password)
async with get_pool(request).connection() as connection:
async with connection.cursor() as cursor:
return await store.create_user(
cursor,
username=body.username,
display_name=body.display_name,
password_hash=password_hash,
)
@router.post("/api/v1/auth/sessions", status_code=201)
async def create_session(request: Request) -> dict[str, Any]:
ensure_no_query(request)
await drain_body(request)
username, password = basic_credentials(request)
pool = get_pool(request)
async with pool.connection() as connection:
async with connection.cursor() as cursor:
row = await store.user_credentials(cursor, username)
if row is None:
await run_in_threadpool(verify_password_unknown_user, password)
raise unauthorized("Invalid username or password", BASIC_CHALLENGE)
if not await run_in_threadpool(
verify_password, password, row["password_hash"]
):
raise unauthorized("Invalid username or password", BASIC_CHALLENGE)
token, expires_at = await session_service.create_session(
pool,
get_cache(request),
user_id=row["id"],
ttl_seconds=request.app.state.settings.session_ttl_seconds,
)
return {
"token": token,
"token_type": "Bearer",
"expires_at": format_timestamp(expires_at),
"user": store.user_json(row),
}
@router.delete("/api/v1/auth/sessions/current", status_code=204)
async def delete_current_session(request: Request) -> Response:
ensure_no_query(request)
token = bearer_token(request)
await session_service.revoke_session(get_pool(request), get_cache(request), token)
return Response(status_code=204)
@router.get("/api/v1/users/me")
async def current_user(request: Request, actor: Actor) -> dict[str, Any]:
ensure_no_query(request)
return actor.user
# --------------------------------------------------------------------------- #
# Chats and messages
# --------------------------------------------------------------------------- #
@router.post("/api/v1/chats", status_code=201)
async def create_chat(request: Request, actor: Actor) -> dict[str, Any]:
ensure_no_query(request)
body = await parse_body(request, CreateChat)
assert isinstance(body, CreateChat)
members = [uuid.UUID(member) for member in body.member_ids]
async with get_pool(request).connection() as connection:
async with connection.cursor() as cursor:
return await store.create_chat(
cursor, actor_id=actor.user_id, title=body.title, member_ids=members
)
@router.get("/api/v1/chats")
async def list_chats(request: Request, actor: Actor) -> dict[str, Any]:
limit, cursor = parse_pagination_request(request)
resource = "chats:" + str(actor.user_id)
after = decode_cursor(cursor, resource) if cursor is not None else None
async with get_pool(request).connection() as connection:
async with connection.cursor() as cursor_handle:
rows = await store.chat_page(
cursor_handle, actor_id=actor.user_id, limit=limit, after=after
)
return build_page(rows, resource, limit)
@router.get("/api/v1/chats/{chat_id}")
async def get_chat(request: Request, chat_id: str, actor: Actor) -> dict[str, Any]:
ensure_no_query(request)
normalized = require_chat_id(chat_id)
async with get_pool(request).connection() as connection:
async with connection.cursor() as cursor:
return await store.accessible_chat(
cursor, actor_id=actor.user_id, chat_id=normalized
)
@router.post("/api/v1/chats/{chat_id}/messages", status_code=201)
async def create_message(
request: Request, chat_id: str, actor: Actor
) -> dict[str, Any]:
normalized = require_chat_id(chat_id)
ensure_no_query(request)
pool = get_pool(request)
async with pool.connection() as connection:
async with connection.cursor() as cursor:
await store.require_membership(
cursor, actor_id=actor.user_id, chat_id=normalized
)
body = await parse_body(request, CreateMessage)
assert isinstance(body, CreateMessage)
async with pool.connection() as connection:
async with connection.cursor() as cursor:
return await store.create_message(
cursor, actor_id=actor.user_id, chat_id=normalized, text=body.text
)
@router.get("/api/v1/chats/{chat_id}/messages")
async def list_messages(
request: Request, chat_id: str, actor: Actor
) -> dict[str, Any]:
normalized = require_chat_id(chat_id)
limit, cursor = parse_pagination_request(request)
resource = "messages:" + str(normalized)
after = decode_cursor(cursor, resource) if cursor is not None else None
async with get_pool(request).connection() as connection:
async with connection.cursor() as cursor_handle:
await store.require_membership(
cursor_handle, actor_id=actor.user_id, chat_id=normalized
)
rows = await store.message_page(
cursor_handle, chat_id=normalized, limit=limit, after=after
)
return build_page(rows, resource, limit)
+34
View File
@@ -0,0 +1,34 @@
"""Executable entry point: migrations run first, then uvicorn as PID 1."""
from __future__ import annotations
import sys
from pathlib import Path
import uvicorn
if __package__ in {None, ""}:
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
from src.application import create_app # noqa: E402
from src.config import load_address # noqa: E402
app = create_app()
def run() -> None:
host, port = load_address()
uvicorn.run(
app,
host=host,
port=port,
access_log=False,
timeout_graceful_shutdown=8,
log_level="info",
)
if __name__ == "__main__":
run()
+269
View File
@@ -0,0 +1,269 @@
"""FastAPI application factory, lifespan, middleware and error mapping."""
from __future__ import annotations
import asyncio
import contextlib
import json
import logging
import sys
import time
import uuid
from contextlib import asynccontextmanager
from typing import Any, AsyncIterator
import psycopg
from fastapi import FastAPI, Request
from fastapi.exceptions import RequestValidationError
from fastapi.responses import JSONResponse, RedirectResponse
from psycopg_pool import AsyncConnectionPool, PoolTimeout
from starlette.exceptions import HTTPException as StarletteHTTPException
from starlette.responses import Response
from . import sessions as session_service
from .api import router
from .config import CONTRACT_PATH, Settings, load_settings
from .db import create_pool
from .errors import ApiProblem
access_logger = logging.getLogger("messenger.access")
lifecycle_logger = logging.getLogger("uvicorn.error")
if not access_logger.handlers:
access_handler = logging.StreamHandler(sys.stdout)
access_handler.setFormatter(
logging.Formatter("%(asctime)s %(levelname)s %(name)s %(message)s")
)
access_logger.addHandler(access_handler)
access_logger.setLevel(logging.INFO)
access_logger.propagate = False
def request_id(request: Request) -> str:
return getattr(request.state, "request_id", str(uuid.uuid4()))
def problem_response(
request: Request,
status: int,
code: str,
message: str,
headers: dict[str, str] | None = None,
) -> JSONResponse:
current_request_id = request_id(request)
response_headers = {"X-Request-Id": current_request_id}
response_headers.update(headers or {})
return JSONResponse(
status_code=status,
content={
"error": {
"code": code,
"message": message,
"request_id": current_request_id,
}
},
headers=response_headers,
)
def load_contract() -> dict[str, Any]:
with CONTRACT_PATH.open(encoding="utf-8") as contract_file:
return json.load(contract_file)
async def _purge_loop(pool: AsyncConnectionPool) -> None:
"""Drop long-expired session rows; TTL itself is enforced on every read."""
while True:
try:
await asyncio.sleep(60)
purged = await session_service.purge_expired(pool)
if purged:
lifecycle_logger.info("sessions_purged count=%s", purged)
except asyncio.CancelledError:
raise
except Exception as exc: # noqa: BLE001 - housekeeping must not stop the API
lifecycle_logger.warning(
"sessions_purge_failed error=%s", type(exc).__name__
)
@asynccontextmanager
async def lifespan(application: FastAPI) -> AsyncIterator[None]:
settings: Settings = application.state.settings
pool: AsyncConnectionPool = application.state.pool
cache: session_service.SessionCache = application.state.cache
lifecycle_logger.info(
"server_starting env=%s session_ttl_seconds=%s cache_enabled=%s "
"pool_max=%s statement_timeout_ms=%s",
settings.app_env,
settings.session_ttl_seconds,
cache.enabled,
settings.pool_max_size,
settings.statement_timeout_ms,
)
try:
await pool.open(wait=True, timeout=10)
except Exception as exc: # noqa: BLE001 - start only with a usable database
lifecycle_logger.error(
"database_unavailable_at_startup error=%s", type(exc).__name__
)
raise
lifecycle_logger.info("server_started")
if cache.enabled:
if await cache.ping():
lifecycle_logger.info("cache_ready")
else:
lifecycle_logger.warning("cache_unavailable_using_database_only")
else:
lifecycle_logger.warning("cache_not_configured_using_database_only")
purge_task = asyncio.create_task(_purge_loop(pool))
try:
yield
finally:
lifecycle_logger.info("shutdown_requested")
purge_task.cancel()
with contextlib.suppress(BaseException):
await purge_task
await cache.close()
await pool.close()
lifecycle_logger.info("server_stopped")
def create_app(settings: Settings | None = None) -> FastAPI:
resolved = settings or load_settings()
application = FastAPI(
title="MTUSI Messenger — лабораторная 2",
version="1.2.0",
docs_url="/docs",
redoc_url=None,
lifespan=lifespan,
)
application.state.settings = resolved
application.state.logger = lifecycle_logger
application.state.pool = create_pool(
resolved.database_url,
min_size=resolved.pool_min_size,
max_size=resolved.pool_max_size,
timeout_seconds=resolved.pool_timeout_seconds,
connect_timeout_seconds=resolved.connect_timeout_seconds,
statement_timeout_ms=resolved.statement_timeout_ms,
)
application.state.cache = session_service.SessionCache(
resolved.cache_url,
timeout_seconds=resolved.cache_timeout_seconds,
ttl_cap=resolved.cache_ttl_seconds,
)
application.openapi = load_contract # type: ignore[method-assign]
@application.middleware("http")
async def request_context(request: Request, call_next: Any) -> Response:
request.state.request_id = str(uuid.uuid4())
started_at = time.monotonic()
status = 500
try:
response = await call_next(request)
if request.method in {"POST", "PUT", "PATCH"}:
try:
# Authentication or resource checks can finish before the body
# is read. Drain any remainder without retaining it so the
# client receives that JSON response instead of a TCP reset.
async for _chunk in request.stream():
pass
except RuntimeError:
pass
status = response.status_code
except ApiProblem as problem:
response = problem_response(
request, problem.status, problem.code, problem.message, problem.headers
)
status = problem.status
except Exception: # noqa: BLE001 - never leak internals to the client
lifecycle_logger.exception(
"unhandled_error request_id=%s", request.state.request_id
)
response = problem_response(
request, 500, "internal_error", "Internal server error"
)
status = 500
response.headers["X-Request-Id"] = request.state.request_id
client = request.client.host if request.client is not None else "unknown"
access_logger.info(
"request_complete request_id=%s method=%s path=%s status=%s "
"client=%s elapsed_ms=%.3f",
request.state.request_id,
request.method,
request.url.path,
status,
client,
(time.monotonic() - started_at) * 1000,
)
return response
@application.exception_handler(ApiProblem)
async def api_problem_handler(request: Request, problem: ApiProblem) -> JSONResponse:
return problem_response(
request, problem.status, problem.code, problem.message, problem.headers
)
@application.exception_handler(RequestValidationError)
async def validation_handler(
request: Request, _error: RequestValidationError
) -> JSONResponse:
return problem_response(
request, 400, "invalid_request", "Request fields are invalid"
)
@application.exception_handler(StarletteHTTPException)
async def http_error_handler(
request: Request, error: StarletteHTTPException
) -> JSONResponse:
if error.status_code in {404, 405}:
return problem_response(request, 404, "not_found", "Endpoint not found")
return problem_response(
request, error.status_code, "invalid_request", "Request is invalid"
)
@application.exception_handler(PoolTimeout)
async def pool_timeout_handler(
request: Request, error: PoolTimeout
) -> JSONResponse:
lifecycle_logger.warning(
"database_pool_timeout request_id=%s error=%s",
request_id(request),
type(error).__name__,
)
return problem_response(
request, 503, "unavailable", "Database is unavailable"
)
@application.exception_handler(psycopg.OperationalError)
@application.exception_handler(psycopg.InterfaceError)
async def database_unavailable_handler(
request: Request, error: Exception
) -> JSONResponse:
lifecycle_logger.warning(
"database_unavailable request_id=%s error=%s",
request_id(request),
type(error).__name__,
)
return problem_response(
request, 503, "unavailable", "Database is unavailable"
)
@application.exception_handler(psycopg.Error)
async def database_error_handler(
request: Request, error: psycopg.Error
) -> JSONResponse:
lifecycle_logger.warning(
"database_error request_id=%s error=%s",
request_id(request),
type(error).__name__,
)
return problem_response(request, 500, "internal_error", "Internal server error")
@application.get("/", include_in_schema=False)
async def docs_redirect() -> RedirectResponse:
return RedirectResponse(url="/docs", status_code=307)
application.include_router(router)
return application
+140
View File
@@ -0,0 +1,140 @@
"""Runtime configuration: environment variables plus file-based secrets."""
from __future__ import annotations
import os
from dataclasses import dataclass
from pathlib import Path
PROJECT_ROOT = Path(__file__).resolve().parents[1]
CONTRACT_PATH = PROJECT_ROOT / "contracts" / "openapi.json"
MIGRATIONS_DIR = Path(__file__).resolve().parent / "migrations"
DEFAULT_PAGE_SIZE = 50
MAX_PAGE_SIZE = 100
MAX_JSON_BYTES = 1_048_576
DEFAULT_SESSION_TTL_SECONDS = 3600
DEFAULT_CACHE_TTL_SECONDS = 300
SUPPORTED_SESSION_STORES = {"postgres"}
class ConfigurationError(Exception):
"""Startup configuration is unusable; the process must not serve traffic."""
@dataclass(frozen=True)
class Settings:
"""Everything the process needs, resolved once at startup."""
database_url: str
cache_url: str | None
session_store: str
session_ttl_seconds: int
cache_ttl_seconds: int
pool_min_size: int
pool_max_size: int
pool_timeout_seconds: float
connect_timeout_seconds: int
statement_timeout_ms: int
cache_timeout_seconds: float
app_env: str
@property
def cache_enabled(self) -> bool:
return self.cache_url is not None
def _secret(file_variable: str, plain_variable: str) -> str | None:
"""Read a secret from a mounted file, falling back to a plain variable."""
path_value = os.environ.get(file_variable, "").strip()
if path_value:
path = Path(path_value)
try:
value = path.read_text(encoding="utf-8").strip()
except OSError as exc:
raise ConfigurationError(
file_variable + " points to an unreadable file (" + type(exc).__name__ + ")"
) from None
if not value:
raise ConfigurationError(file_variable + " points to an empty file")
return value
plain = os.environ.get(plain_variable, "").strip()
return plain or None
def _integer(variable: str, default: int, minimum: int, maximum: int) -> int:
raw = os.environ.get(variable, "").strip()
if not raw:
return default
try:
value = int(raw)
except ValueError:
raise ConfigurationError(variable + " must be an integer") from None
if not minimum <= value <= maximum:
raise ConfigurationError(
variable + " must be between " + str(minimum) + " and " + str(maximum)
)
return value
def _number(variable: str, default: float, minimum: float, maximum: float) -> float:
raw = os.environ.get(variable, "").strip()
if not raw:
return default
try:
value = float(raw)
except ValueError:
raise ConfigurationError(variable + " must be a number") from None
if not minimum <= value <= maximum:
raise ConfigurationError(
variable + " must be between " + str(minimum) + " and " + str(maximum)
)
return value
def load_settings() -> Settings:
"""Resolve settings once; a bad configuration fails fast before serving."""
database_url = _secret("DATABASE_URL_FILE", "DATABASE_URL")
if database_url is None:
raise ConfigurationError("DATABASE_URL_FILE or DATABASE_URL is required")
cache_url = _secret("CACHE_URL_FILE", "CACHE_URL")
session_store = os.environ.get("SESSION_STORE", "postgres").strip().lower()
if session_store not in SUPPORTED_SESSION_STORES:
raise ConfigurationError(
"SESSION_STORE="
+ session_store
+ " is not implemented: sessions are stored in PostgreSQL "
"(the durable source of truth) and cached in Valkey. Use SESSION_STORE=postgres."
)
pool_min = _integer("DB_POOL_MIN", 1, 0, 100)
pool_max = _integer("DB_POOL_MAX", 10, 1, 200)
if pool_min > pool_max:
raise ConfigurationError("DB_POOL_MIN must not exceed DB_POOL_MAX")
return Settings(
database_url=database_url,
cache_url=cache_url,
session_store=session_store,
session_ttl_seconds=_integer(
"SESSION_TTL_SECONDS", DEFAULT_SESSION_TTL_SECONDS, 1, 2_592_000
),
cache_ttl_seconds=_integer(
"CACHE_TTL_SECONDS", DEFAULT_CACHE_TTL_SECONDS, 1, 86_400
),
pool_min_size=pool_min,
pool_max_size=pool_max,
pool_timeout_seconds=_number("DB_POOL_TIMEOUT_SECONDS", 2.0, 0.1, 60.0),
connect_timeout_seconds=_integer("DB_CONNECT_TIMEOUT_SECONDS", 2, 1, 60),
statement_timeout_ms=_integer("DB_STATEMENT_TIMEOUT_MS", 5000, 100, 120_000),
cache_timeout_seconds=_number("CACHE_TIMEOUT_SECONDS", 0.5, 0.05, 10.0),
app_env=os.environ.get("APP_ENV", "lab").strip() or "lab",
)
def load_address() -> tuple[str, int]:
"""Read and validate the HTTP listen address from the environment."""
host = os.environ.get("APP_HOST", "0.0.0.0").strip()
if not host:
raise SystemExit("APP_HOST must not be empty")
port = _integer("APP_PORT", 8080, 1, 65535)
return host, port
+49
View File
@@ -0,0 +1,49 @@
"""PostgreSQL access: a bounded async pool with explicit timeouts."""
from __future__ import annotations
from psycopg.rows import dict_row
from psycopg_pool import AsyncConnectionPool
def connection_options(statement_timeout_ms: int) -> str:
"""Server-side limits so a slow query or an idle transaction cannot pile up."""
return (
"-c statement_timeout="
+ str(statement_timeout_ms)
+ " -c lock_timeout=2000 -c idle_in_transaction_session_timeout=10000"
+ " -c timezone=UTC"
)
def create_pool(
database_url: str,
*,
min_size: int,
max_size: int,
timeout_seconds: float,
connect_timeout_seconds: int,
statement_timeout_ms: int,
) -> AsyncConnectionPool:
"""Build a closed pool; the lifespan opens it and closes it on shutdown."""
return AsyncConnectionPool(
conninfo=database_url,
min_size=min_size,
max_size=max_size,
timeout=timeout_seconds,
open=False,
check=AsyncConnectionPool.check_connection,
kwargs={
"row_factory": dict_row,
"connect_timeout": connect_timeout_seconds,
"options": connection_options(statement_timeout_ms),
"application_name": "mtusi-messenger-api",
},
name="messenger-pool",
)
async def ping(pool: AsyncConnectionPool) -> None:
"""Cheap readiness probe; raises when PostgreSQL is not usable."""
async with pool.connection() as connection:
await connection.execute("SELECT 1")
+26
View File
@@ -0,0 +1,26 @@
"""Errors that may safely be returned through the public API."""
from __future__ import annotations
from typing import Mapping
BASIC_CHALLENGE = 'Basic realm="messenger", charset="UTF-8"'
BEARER_CHALLENGE = 'Bearer realm="messenger"'
class ApiProblem(Exception):
"""An expected API failure with a stable status, machine code and headers."""
def __init__(
self,
status: int,
code: str,
message: str,
headers: Mapping[str, str] | None = None,
) -> None:
super().__init__(message)
self.status = status
self.code = code
self.message = message
self.headers = dict(headers or {})
+93
View File
@@ -0,0 +1,93 @@
"""Versioned SQL migration runner.
Every run is one transaction guarded by a PostgreSQL advisory lock, so a second
process (or a restarted container) cannot apply the same file twice and a failed
migration leaves the schema unchanged.
"""
from __future__ import annotations
import sys
import time
from pathlib import Path
import psycopg
if __package__ in {None, ""}:
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
from src.config import MIGRATIONS_DIR, ConfigurationError, load_settings # noqa: E402
ADVISORY_LOCK_KEY = 915_127_001
CONNECT_RETRY_SECONDS = 60
def _version(path: Path) -> int:
prefix = path.name.split("_", 1)[0]
return int(prefix)
def apply_migrations(connection: psycopg.Connection, directory: Path) -> list[str]:
applied: list[str] = []
with connection.transaction():
connection.execute("SELECT pg_advisory_xact_lock(%s)", (ADVISORY_LOCK_KEY,))
connection.execute(
"""
CREATE TABLE IF NOT EXISTS schema_migrations (
version integer PRIMARY KEY,
filename text NOT NULL,
applied_at timestamptz NOT NULL DEFAULT now()
)
"""
)
rows = connection.execute("SELECT version FROM schema_migrations").fetchall()
done = {row[0] for row in rows}
for path in sorted(directory.glob("*.sql"), key=_version):
version = _version(path)
if version in done:
continue
connection.execute(path.read_text(encoding="utf-8"))
connection.execute(
"INSERT INTO schema_migrations (version, filename) VALUES (%s, %s)",
(version, path.name),
)
applied.append(path.name)
return applied
def main() -> None:
try:
settings = load_settings()
except ConfigurationError as error:
print("configuration_error " + str(error), file=sys.stderr)
raise SystemExit(1) from None
deadline = time.monotonic() + CONNECT_RETRY_SECONDS
while True:
try:
with psycopg.connect(
settings.database_url,
connect_timeout=5,
options="-c timezone=UTC",
application_name="mtusi-messenger-migrate",
) as connection:
applied = apply_migrations(connection, MIGRATIONS_DIR)
break
except psycopg.OperationalError as error:
if time.monotonic() >= deadline:
print(
"database_unreachable error=" + type(error).__name__, file=sys.stderr
)
raise SystemExit(1) from None
time.sleep(1)
except psycopg.Error as error:
print("migration_failed error=" + type(error).__name__, file=sys.stderr)
raise SystemExit(1) from None
if applied:
print("migrations_applied " + ",".join(applied))
else:
print("migrations_up_to_date")
if __name__ == "__main__":
main()
+13
View File
@@ -0,0 +1,13 @@
-- Users own credentials and identity. The password column stores an Argon2id hash.
CREATE TABLE users (
id uuid PRIMARY KEY,
username text NOT NULL,
display_name text NOT NULL,
password_hash text NOT NULL,
created_at timestamptz NOT NULL,
CONSTRAINT users_username_format CHECK (username ~ '^[a-z0-9_]{3,32}$'),
CONSTRAINT users_display_name_length CHECK (char_length(display_name) BETWEEN 1 AND 100),
CONSTRAINT users_password_hash_present CHECK (char_length(password_hash) > 0)
);
CREATE UNIQUE INDEX users_username_unique ON users (username);
+34
View File
@@ -0,0 +1,34 @@
-- Chats, explicit membership and message history.
CREATE TABLE chats (
id uuid PRIMARY KEY,
title text NOT NULL,
created_at timestamptz NOT NULL,
CONSTRAINT chats_title_length CHECK (char_length(title) BETWEEN 1 AND 100)
);
-- Chat lists are ordered by (created_at, id) for the same keyset cursor.
CREATE INDEX chats_order_idx ON chats (created_at, id);
CREATE TABLE chat_members (
chat_id uuid NOT NULL REFERENCES chats (id) ON DELETE CASCADE,
user_id uuid NOT NULL REFERENCES users (id) ON DELETE RESTRICT,
joined_at timestamptz NOT NULL,
PRIMARY KEY (chat_id, user_id)
);
-- "Chats of the current user" and the membership check on every read.
CREATE INDEX chat_members_user_idx ON chat_members (user_id, chat_id);
CREATE TABLE messages (
id uuid PRIMARY KEY,
chat_id uuid NOT NULL REFERENCES chats (id) ON DELETE CASCADE,
sender_id uuid NOT NULL REFERENCES users (id) ON DELETE RESTRICT,
text text NOT NULL,
created_at timestamptz NOT NULL,
CONSTRAINT messages_text_length CHECK (char_length(text) BETWEEN 1 AND 4000)
);
-- Serves history keyset pagination exactly: equality on chat_id, then the
-- (created_at, id) tuple used by both ORDER BY and the cursor predicate.
-- Justification and EXPLAIN (ANALYZE, BUFFERS) are in evidence/explain-history.txt.
CREATE INDEX messages_chat_order_idx ON messages (chat_id, created_at, id);
+16
View File
@@ -0,0 +1,16 @@
-- Opaque server-side sessions: only a SHA-256 of the token is stored.
CREATE TABLE sessions (
id uuid PRIMARY KEY,
user_id uuid NOT NULL REFERENCES users (id) ON DELETE CASCADE,
token_hash text NOT NULL,
created_at timestamptz NOT NULL,
expires_at timestamptz NOT NULL,
revoked_at timestamptz,
CONSTRAINT sessions_token_hash_length CHECK (char_length(token_hash) = 64),
CONSTRAINT sessions_expiry_after_creation CHECK (expires_at > created_at)
);
-- Every authenticated request looks a token up by hash.
CREATE UNIQUE INDEX sessions_token_hash_unique ON sessions (token_hash);
CREATE INDEX sessions_user_idx ON sessions (user_id);
CREATE INDEX sessions_expiry_idx ON sessions (expires_at);
+78
View File
@@ -0,0 +1,78 @@
"""Strict request models for the lab 2 API."""
from __future__ import annotations
import re
import uuid
from pydantic import BaseModel, ConfigDict, Field, field_validator
USERNAME_PATTERN = re.compile(r"^[a-z0-9_]{3,32}$")
PASSWORD_MIN_LENGTH = 12
PASSWORD_MAX_LENGTH = 128
class StrictModel(BaseModel):
model_config = ConfigDict(extra="forbid", strict=True)
def _not_blank(value: str, field: str) -> str:
if not value.strip():
raise ValueError(field + " must not be blank")
return value
class CreateUser(StrictModel):
"""Registration body; the password never appears in a response."""
username: str = Field(
min_length=3, max_length=32, pattern=USERNAME_PATTERN.pattern
)
display_name: str = Field(min_length=1, max_length=100)
password: str = Field(
min_length=PASSWORD_MIN_LENGTH, max_length=PASSWORD_MAX_LENGTH
)
@field_validator("display_name")
@classmethod
def display_name_is_not_blank(cls, value: str) -> str:
return _not_blank(value, "display_name")
@field_validator("password")
@classmethod
def password_length_in_code_points(cls, value: str) -> str:
# Python len() counts Unicode code points, which is what the contract uses.
if not PASSWORD_MIN_LENGTH <= len(value) <= PASSWORD_MAX_LENGTH:
raise ValueError("password length is out of range")
return value
class CreateChat(StrictModel):
title: str = Field(min_length=1, max_length=100)
member_ids: list[str] = Field(min_length=1, max_length=100)
@field_validator("title")
@classmethod
def title_is_not_blank(cls, value: str) -> str:
return _not_blank(value, "title")
@field_validator("member_ids")
@classmethod
def members_are_valid_and_unique(cls, value: list[str]) -> list[str]:
try:
normalized = [str(uuid.UUID(member_id)) for member_id in value]
except (ValueError, AttributeError):
raise ValueError("member_ids must contain UUID strings") from None
if len(normalized) != len(set(normalized)):
raise ValueError("member_ids must be unique")
return normalized
class CreateMessage(StrictModel):
text: str = Field(min_length=1, max_length=4000)
@field_validator("text")
@classmethod
def text_is_not_blank(cls, value: str) -> str:
return _not_blank(value, "text")
+84
View File
@@ -0,0 +1,84 @@
"""Opaque, resource-bound keyset cursors, timestamp formatting and page shaping."""
from __future__ import annotations
import base64
import binascii
import json
import uuid
from datetime import datetime, timezone
from typing import Any
from .errors import ApiProblem
def format_timestamp(value: datetime) -> str:
"""RFC 3339 UTC with microseconds and a trailing Z, as the contract requires."""
return (
value.astimezone(timezone.utc)
.isoformat(timespec="microseconds")
.replace("+00:00", "Z")
)
def parse_timestamp(value: str) -> datetime:
"""Parse an RFC 3339 timestamp from a cursor; reject anything else."""
candidate = value[:-1] + "+00:00" if value.endswith("Z") else value
parsed = datetime.fromisoformat(candidate)
if parsed.tzinfo is None:
parsed = parsed.replace(tzinfo=timezone.utc)
return parsed.astimezone(timezone.utc)
def encode_cursor(resource: str, created_at: datetime, record_id: str) -> str:
payload = json.dumps(
{
"resource": resource,
"created_at": format_timestamp(created_at),
"id": record_id,
},
separators=(",", ":"),
sort_keys=True,
).encode("utf-8")
return base64.urlsafe_b64encode(payload).decode("ascii").rstrip("=")
def decode_cursor(value: str, resource: str) -> tuple[datetime, uuid.UUID]:
"""Decode a cursor and bind it to the resource it was issued for."""
try:
padding = "=" * (-len(value) % 4)
raw = base64.b64decode(value + padding, altchars=b"-_", validate=True)
payload: Any = json.loads(raw.decode("utf-8"))
if not isinstance(payload, dict) or set(payload) != {
"resource",
"created_at",
"id",
}:
raise ValueError("unexpected cursor payload")
if payload["resource"] != resource:
raise ValueError("cursor belongs to another resource")
created_at = payload["created_at"]
record_id = payload["id"]
if not isinstance(created_at, str) or not isinstance(record_id, str):
raise ValueError("cursor fields have wrong types")
return parse_timestamp(created_at), uuid.UUID(record_id)
except (
ValueError,
TypeError,
UnicodeError,
binascii.Error,
json.JSONDecodeError,
):
raise ApiProblem(400, "invalid_request", "Cursor is invalid") from None
def build_page(items: list[dict[str, Any]], resource: str, limit: int) -> dict[str, Any]:
"""Shape a limit+1 storage slice into the public page and its next cursor."""
page = items[:limit]
next_cursor = None
if len(items) > limit and page:
last = page[-1]
next_cursor = encode_cursor(
resource, parse_timestamp(last["created_at"]), last["id"]
)
return {"items": page, "next_cursor": next_cursor}
+43
View File
@@ -0,0 +1,43 @@
"""Password hashing with Argon2id (library KDF, salted, versioned parameters)."""
from __future__ import annotations
from argon2 import PasswordHasher
from argon2.exceptions import InvalidHashError, VerificationError, VerifyMismatchError
# OWASP-recommended Argon2id baseline: 19 MiB, 2 iterations, 1 lane.
_hasher = PasswordHasher(
time_cost=2,
memory_cost=19456,
parallelism=1,
hash_len=32,
salt_len=16,
)
# Verified against when a username is unknown, so the failure path costs the same
# as a wrong password and does not leak username existence through timing.
_DUMMY_HASH = _hasher.hash("dummy-password-not-used-for-any-account")
def hash_password(password: str) -> str:
return _hasher.hash(password)
def verify_password(password: str, encoded: str | None) -> bool:
"""Return True only for a matching password and a parseable hash."""
if encoded is None:
verify_password_unknown_user(password)
return False
try:
return _hasher.verify(encoded, password)
except (VerifyMismatchError, VerificationError, InvalidHashError):
return False
def verify_password_unknown_user(password: str) -> None:
"""Burn comparable CPU time when no account exists for the username."""
try:
_hasher.verify(_DUMMY_HASH, password)
except (VerifyMismatchError, VerificationError, InvalidHashError):
return
+324
View File
@@ -0,0 +1,324 @@
"""Opaque Bearer sessions: PostgreSQL is the source of truth, Valkey is a cache.
Design (see REPORT.md):
* ``sessions`` in PostgreSQL owns identity, expiry and revocation.
* Valkey stores the same session under ``session:<sha256(token)>`` for at most
``CACHE_TTL_SECONDS`` and never longer than the session's remaining lifetime.
* Revocation writes a tombstone instead of relying on ``DEL`` alone, so a stale
cache entry cannot authenticate after logout.
* When Valkey is unavailable every request falls back to PostgreSQL. A cache
error is never treated as "authenticated", and a cache hit is only trusted
because it came from a session row that PostgreSQL confirmed at login time.
"""
from __future__ import annotations
import hashlib
import json
import logging
import secrets
import time
import uuid
from datetime import datetime, timezone
from typing import Any
import redis.asyncio as redis
from psycopg_pool import AsyncConnectionPool
from redis.backoff import NoBackoff
from redis.retry import Retry
from . import store
from .errors import ApiProblem
from .pagination import format_timestamp
logger = logging.getLogger("messenger.sessions")
TOKEN_BYTES = 48 # 384 bits of entropy, well above the required 256.
CACHE_PREFIX = "session:"
REVOKED = "revoked"
# After a cache error, skip Valkey for a few seconds instead of paying the
# connect timeout on every request (lookup and remember would cost two each).
FAILURE_COOLDOWN_SECONDS = 5.0
# Bound on revocations remembered while the cache cannot accept a tombstone.
MAX_PENDING_REVOCATIONS = 4096
HIT = "hit"
MISS = "miss"
TOMBSTONE = "revoked"
ERROR = "error"
def token_hash(token: str) -> str:
return hashlib.sha256(token.encode("utf-8")).hexdigest()
class SessionCache:
"""Small, failure-tolerant Valkey client used only for session lookups."""
def __init__(self, url: str | None, *, timeout_seconds: float, ttl_cap: int) -> None:
self.ttl_cap = max(1, ttl_cap)
self.timeout_seconds = timeout_seconds
# hashed token -> epoch at which the session would be expired anyway.
# PostgreSQL already holds the authoritative revocation; this only keeps
# logout immediate for this process while the cache is unreachable.
self._pending: dict[str, float] = {}
self._disabled_until = 0.0
self._degraded = False
self.client: redis.Redis | None = None
if url:
self.client = redis.Redis.from_url(
url,
socket_timeout=timeout_seconds,
socket_connect_timeout=timeout_seconds,
decode_responses=True,
health_check_interval=30,
# A cache must fail fast: one attempt, no retry loop, so a dead
# Valkey costs at most one timeout before PostgreSQL answers.
retry=Retry(NoBackoff(), 0),
retry_on_error=[],
)
@property
def enabled(self) -> bool:
return self.client is not None
def _ready(self) -> bool:
"""True when a cache call is worth attempting right now."""
return self.client is not None and time.monotonic() >= self._disabled_until
def _unavailable(self, operation: str, exc: BaseException) -> None:
self._disabled_until = time.monotonic() + FAILURE_COOLDOWN_SECONDS
# One warning per outage, not one per request.
if not self._degraded:
self._degraded = True
logger.warning(
"cache_unavailable operation=%s error=%s cooldown_seconds=%s",
operation,
type(exc).__name__,
FAILURE_COOLDOWN_SECONDS,
)
def _ok(self) -> None:
self._degraded = False
def remember_revocation(self, hashed: str, remaining: float) -> None:
"""Ask this process to reject a token even if the cache cannot store it."""
if len(self._pending) >= MAX_PENDING_REVOCATIONS:
# Drop the entries closest to expiry: they are the least useful.
keep = len(self._pending) // 4
for key, _ in sorted(self._pending.items(), key=lambda item: item[1])[:keep]:
self._pending.pop(key, None)
self._pending[hashed] = time.time() + max(1.0, remaining)
def is_locally_revoked(self, hashed: str) -> bool:
deadline = self._pending.get(hashed)
if deadline is None:
return False
if deadline <= time.time():
self._pending.pop(hashed, None)
return False
return True
async def _flush_revocations(self) -> None:
"""Re-apply tombstones that could not be written during an outage."""
for hashed, deadline in list(self._pending.items()):
remaining = deadline - time.time()
if remaining <= 0:
self._pending.pop(hashed, None)
continue
try:
await self.client.set(self._key(hashed), REVOKED, ex=self._ttl(remaining))
except Exception as exc: # noqa: BLE001 - still revoked in PostgreSQL
self._unavailable("flush_revocation", exc)
return
self._pending.pop(hashed, None)
async def ping(self) -> bool:
if self.client is None:
return False
try:
alive = bool(await self.client.ping())
except Exception as exc: # noqa: BLE001 - any client error means "no cache"
logger.warning("cache_ping_failed error=%s", type(exc).__name__)
return False
self._disabled_until = 0.0
self._ok()
return alive
async def close(self) -> None:
if self.client is None:
return
try:
await self.client.aclose()
except Exception as exc: # noqa: BLE001 - shutdown must not fail on cache
logger.warning("cache_close_failed error=%s", type(exc).__name__)
def _key(self, hashed: str) -> str:
return CACHE_PREFIX + hashed
def _ttl(self, seconds: float) -> int:
return max(1, min(int(seconds), self.ttl_cap))
async def lookup(self, hashed: str) -> tuple[str, dict[str, Any] | None]:
"""Return (state, payload); every cache failure degrades to a DB read."""
if not self._ready():
return MISS, None
if self._pending:
await self._flush_revocations()
if not self._ready():
return MISS, None
try:
raw = await self.client.get(self._key(hashed))
self._ok()
except Exception as exc: # noqa: BLE001 - cache must never break auth
self._unavailable("lookup", exc)
return ERROR, None
if raw is None:
return MISS, None
if raw == REVOKED:
return TOMBSTONE, None
try:
payload = json.loads(raw)
user = payload["user"]
user["id"] = str(uuid.UUID(user["id"]))
payload["expires_epoch"] = float(payload["expires_epoch"])
payload["user"] = user
except (ValueError, KeyError, TypeError, json.JSONDecodeError):
logger.warning("cache_payload_invalid")
return MISS, None
return HIT, payload
async def remember(self, hashed: str, user: dict[str, Any], expires_at: datetime) -> None:
if not self._ready():
return
remaining = (expires_at - datetime.now(timezone.utc)).total_seconds()
if remaining <= 0:
return
# Cache the immutable public user snapshot, so a hit needs no SQL at all.
# Users cannot be renamed or deleted in the mandatory API of lab 2.
payload = {
"user": user,
"expires_epoch": expires_at.timestamp(),
}
try:
await self.client.set(
self._key(hashed),
json.dumps(payload, ensure_ascii=False),
ex=self._ttl(remaining),
)
self._ok()
except Exception as exc: # noqa: BLE001 - a missed cache write is not fatal
self._unavailable("remember", exc)
async def revoke(self, hashed: str, expires_at: datetime | None) -> None:
"""Write a tombstone so a stale entry cannot authenticate after logout."""
remaining = 60.0
if expires_at is not None:
remaining = max(1.0, (expires_at - datetime.now(timezone.utc)).total_seconds())
# PostgreSQL already revoked the session; this keeps the fast path honest
# even if the tombstone cannot be written right now.
self.remember_revocation(hashed, remaining)
if not self._ready():
return
try:
await self.client.set(self._key(hashed), REVOKED, ex=self._ttl(remaining))
self._pending.pop(hashed, None)
self._ok()
except Exception as exc: # noqa: BLE001 - PostgreSQL already revoked the session
self._unavailable("revoke", exc)
def _user_payload(row: dict[str, Any]) -> dict[str, Any]:
return {
"id": str(row["id"]),
"username": row["username"],
"display_name": row["display_name"],
"created_at": format_timestamp(row["created_at"]),
}
async def create_session(
pool: AsyncConnectionPool,
cache: SessionCache,
*,
user_id: uuid.UUID,
ttl_seconds: int,
) -> tuple[str, datetime]:
token = secrets.token_urlsafe(TOKEN_BYTES)
hashed = token_hash(token)
async with pool.connection() as connection:
async with connection.cursor() as cursor:
expires_at = await store.insert_session(
cursor, user_id=user_id, token_hash=hashed, ttl_seconds=ttl_seconds
)
user = await store.user_by_id(cursor, user_id)
if user is not None:
await cache.remember(hashed, _user_payload(user), expires_at)
return token, expires_at
async def resolve_session(
pool: AsyncConnectionPool, cache: SessionCache, token: str
) -> tuple[uuid.UUID, dict[str, Any]]:
"""Return the authenticated user or raise 401 with a Bearer challenge."""
hashed = token_hash(token)
if cache.is_locally_revoked(hashed):
raise ApiProblem(
401,
"unauthorized",
"Session token is expired or revoked",
{"WWW-Authenticate": 'Bearer realm="messenger"'},
)
state, _payload = await cache.lookup(hashed)
if state == TOMBSTONE:
raise ApiProblem(
401,
"unauthorized",
"Session token is expired or revoked",
{"WWW-Authenticate": 'Bearer realm="messenger"'},
)
# PostgreSQL is the authoritative revocation store. A cache value can only
# accelerate the immutable user snapshot; it must never be enough to grant
# access on its own. In particular this makes logout durable across API
# restarts and Valkey outages/restores, where a tombstone may not yet have
# reached the shared cache.
async with pool.connection() as connection:
async with connection.cursor() as cursor:
row = await store.session_user(cursor, hashed)
if row is None:
await cache.revoke(hashed, None)
raise ApiProblem(
401,
"unauthorized",
"Session token is expired or revoked",
{"WWW-Authenticate": 'Bearer realm="messenger"'},
)
user = _user_payload(row)
# Refresh an old cache hit after PostgreSQL has confirmed it is still live.
await cache.remember(hashed, user, row["expires_at"])
return uuid.UUID(user["id"]), user
async def revoke_session(
pool: AsyncConnectionPool, cache: SessionCache, token: str
) -> None:
hashed = token_hash(token)
async with pool.connection() as connection:
async with connection.cursor() as cursor:
expires_at = await store.revoke_session(cursor, hashed)
if expires_at is None:
raise ApiProblem(
401,
"unauthorized",
"Session token is expired or revoked",
{"WWW-Authenticate": 'Bearer realm="messenger"'},
)
await cache.revoke(hashed, expires_at)
async def purge_expired(pool: AsyncConnectionPool) -> int:
async with pool.connection() as connection:
async with connection.cursor() as cursor:
return await store.purge_expired_sessions(cursor)
+297
View File
@@ -0,0 +1,297 @@
"""PostgreSQL repository: parameterised statements only, explicit transactions."""
from __future__ import annotations
import uuid
from datetime import datetime
from typing import Any
import psycopg
from psycopg import AsyncCursor
from .errors import ApiProblem
from .pagination import format_timestamp
# One expression keeps the member list of a chat consistent across endpoints.
CHAT_COLUMNS = """
c.id,
c.title,
c.created_at,
ARRAY(
SELECT m.user_id::text
FROM chat_members m
WHERE m.chat_id = c.id
ORDER BY m.user_id::text
) AS member_ids
"""
def user_json(row: dict[str, Any]) -> dict[str, Any]:
return {
"id": str(row["id"]),
"username": row["username"],
"display_name": row["display_name"],
"created_at": format_timestamp(row["created_at"]),
}
def chat_json(row: dict[str, Any]) -> dict[str, Any]:
return {
"id": str(row["id"]),
"title": row["title"],
"member_ids": [str(member) for member in row["member_ids"]],
"created_at": format_timestamp(row["created_at"]),
}
def message_json(row: dict[str, Any]) -> dict[str, Any]:
return {
"id": str(row["id"]),
"chat_id": str(row["chat_id"]),
"sender_id": str(row["sender_id"]),
"text": row["text"],
"created_at": format_timestamp(row["created_at"]),
}
async def create_user(
cursor: AsyncCursor,
*,
username: str,
display_name: str,
password_hash: str,
) -> dict[str, Any]:
try:
await cursor.execute(
"""
INSERT INTO users (id, username, display_name, password_hash, created_at)
VALUES (%s, %s, %s, %s, now())
RETURNING id, username, display_name, created_at
""",
(uuid.uuid4(), username, display_name, password_hash),
)
except psycopg.errors.UniqueViolation:
raise ApiProblem(409, "conflict", "Username is already taken") from None
row = await cursor.fetchone()
return user_json(row)
async def user_credentials(cursor: AsyncCursor, username: str) -> dict[str, Any] | None:
await cursor.execute(
"""
SELECT id, username, display_name, password_hash, created_at
FROM users
WHERE username = %s
""",
(username,),
)
return await cursor.fetchone()
async def user_by_id(cursor: AsyncCursor, user_id: uuid.UUID) -> dict[str, Any] | None:
await cursor.execute(
"""
SELECT id, username, display_name, created_at
FROM users
WHERE id = %s
""",
(user_id,),
)
return await cursor.fetchone()
async def create_chat(
cursor: AsyncCursor,
*,
actor_id: uuid.UUID,
title: str,
member_ids: list[uuid.UUID],
) -> dict[str, Any]:
"""Insert the chat and its membership in one transaction.
Unknown members are rejected by the foreign key, so the failure rolls the
already inserted chat row back instead of leaving a chat without members.
"""
if actor_id not in member_ids:
raise ApiProblem(400, "invalid_request", "Current user must be a chat member")
chat_id = uuid.uuid4()
try:
await cursor.execute(
"INSERT INTO chats (id, title, created_at) VALUES (%s, %s, now())",
(chat_id, title),
)
await cursor.execute(
"""
INSERT INTO chat_members (chat_id, user_id, joined_at)
SELECT %s, member_id, now() FROM unnest(%s::uuid[]) AS member_id
""",
(chat_id, member_ids),
)
except psycopg.errors.ForeignKeyViolation:
raise ApiProblem(
400, "invalid_request", "One or more members do not exist"
) from None
except psycopg.errors.UniqueViolation:
raise ApiProblem(400, "invalid_request", "Chat members must be unique") from None
await cursor.execute(
"SELECT " + CHAT_COLUMNS + " FROM chats c WHERE c.id = %s",
(chat_id,),
)
row = await cursor.fetchone()
return chat_json(row)
async def accessible_chat(
cursor: AsyncCursor, *, actor_id: uuid.UUID, chat_id: uuid.UUID
) -> dict[str, Any]:
await cursor.execute(
"""
SELECT """
+ CHAT_COLUMNS
+ """
FROM chats c
JOIN chat_members membership
ON membership.chat_id = c.id AND membership.user_id = %s
WHERE c.id = %s
""",
(actor_id, chat_id),
)
row = await cursor.fetchone()
if row is None:
raise ApiProblem(404, "not_found", "Chat not found")
return chat_json(row)
async def chat_page(
cursor: AsyncCursor,
*,
actor_id: uuid.UUID,
limit: int,
after: tuple[datetime, uuid.UUID] | None,
) -> list[dict[str, Any]]:
query = (
"SELECT "
+ CHAT_COLUMNS
+ """
FROM chats c
JOIN chat_members membership
ON membership.chat_id = c.id AND membership.user_id = %(actor)s
"""
)
params: dict[str, Any] = {"actor": actor_id, "limit": limit + 1}
if after is not None:
query += " WHERE (c.created_at, c.id) > (%(after_ts)s, %(after_id)s)"
params["after_ts"], params["after_id"] = after
query += " ORDER BY c.created_at, c.id LIMIT %(limit)s"
await cursor.execute(query, params)
return [chat_json(row) for row in await cursor.fetchall()]
async def require_membership(
cursor: AsyncCursor, *, actor_id: uuid.UUID, chat_id: uuid.UUID
) -> None:
await cursor.execute(
"SELECT 1 FROM chat_members WHERE chat_id = %s AND user_id = %s",
(chat_id, actor_id),
)
if await cursor.fetchone() is None:
raise ApiProblem(404, "not_found", "Chat not found")
async def create_message(
cursor: AsyncCursor,
*,
actor_id: uuid.UUID,
chat_id: uuid.UUID,
text: str,
) -> dict[str, Any]:
await require_membership(cursor, actor_id=actor_id, chat_id=chat_id)
await cursor.execute(
"""
INSERT INTO messages (id, chat_id, sender_id, text, created_at)
VALUES (%s, %s, %s, %s, now())
RETURNING id, chat_id, sender_id, text, created_at
""",
(uuid.uuid4(), chat_id, actor_id, text),
)
return message_json(await cursor.fetchone())
async def message_page(
cursor: AsyncCursor,
*,
chat_id: uuid.UUID,
limit: int,
after: tuple[datetime, uuid.UUID] | None,
) -> list[dict[str, Any]]:
"""Keyset page over (created_at, id); the composite index serves both branches."""
query = """
SELECT id, chat_id, sender_id, text, created_at
FROM messages
WHERE chat_id = %(chat)s
"""
params: dict[str, Any] = {"chat": chat_id, "limit": limit + 1}
if after is not None:
query += " AND (created_at, id) > (%(after_ts)s, %(after_id)s)"
params["after_ts"], params["after_id"] = after
query += " ORDER BY created_at, id LIMIT %(limit)s"
await cursor.execute(query, params)
return [message_json(row) for row in await cursor.fetchall()]
async def insert_session(
cursor: AsyncCursor,
*,
user_id: uuid.UUID,
token_hash: str,
ttl_seconds: int,
) -> datetime:
await cursor.execute(
"""
INSERT INTO sessions (id, user_id, token_hash, created_at, expires_at)
VALUES (%s, %s, %s, now(), now() + make_interval(secs => %s))
RETURNING expires_at
""",
(uuid.uuid4(), user_id, token_hash, ttl_seconds),
)
row = await cursor.fetchone()
return row["expires_at"]
async def session_user(cursor: AsyncCursor, token_hash: str) -> dict[str, Any] | None:
"""Resolve a live session row; expired and revoked sessions never match."""
await cursor.execute(
"""
SELECT s.expires_at, u.id, u.username, u.display_name, u.created_at
FROM sessions s
JOIN users u ON u.id = s.user_id
WHERE s.token_hash = %s AND s.revoked_at IS NULL AND s.expires_at > now()
""",
(token_hash,),
)
return await cursor.fetchone()
async def revoke_session(cursor: AsyncCursor, token_hash: str) -> datetime | None:
"""Revoke a live session and return its expiry, or None when it is not live."""
await cursor.execute(
"""
UPDATE sessions SET revoked_at = now()
WHERE token_hash = %s AND revoked_at IS NULL AND expires_at > now()
RETURNING expires_at
""",
(token_hash,),
)
row = await cursor.fetchone()
return None if row is None else row["expires_at"]
async def purge_expired_sessions(cursor: AsyncCursor) -> int:
await cursor.execute(
"""
DELETE FROM sessions
WHERE expires_at < now() - interval '1 hour'
OR (revoked_at IS NOT NULL AND revoked_at < now() - interval '1 hour')
"""
)
return cursor.rowcount
+201
View File
@@ -0,0 +1,201 @@
"""Minimal standard-library HTTP client for the student's own tests.
Run with the API already up:
python -m unittest discover -s student-tests -p "test_*.py" -v
BASE_URL selects the stand (default http://127.0.0.1:8080). No third-party
dependency is needed, so `make test-student` works on a clean clone.
"""
from __future__ import annotations
import base64
import json
import os
import urllib.error
import urllib.parse
import urllib.request
import uuid
# 127.0.0.1 rather than "localhost": on a dual-stack host "localhost" resolves
# to ::1 first, a published Docker port may answer on IPv4 only, and every IPv6
# attempt then costs ~2 s of SYN retry before urllib falls back to IPv4. The IPv4
# literal keeps the client fast and never reaches the application.
BASE_URL = os.environ.get("BASE_URL", "http://127.0.0.1:8080").rstrip("/")
TIMEOUT = float(os.environ.get("STUDENT_TEST_TIMEOUT", "20"))
class NoRedirect(urllib.request.HTTPRedirectHandler):
"""The contract never requires redirects; fail loudly instead of following."""
def redirect_request(self, req, fp, code, msg, headers, newurl): # noqa: ANN001
return None
# The lab stand is always loopback; urllib ignores the Windows proxy bypass
# list, so a corporate/system proxy would otherwise be used for localhost too.
_OPENER = urllib.request.build_opener(urllib.request.ProxyHandler({}), NoRedirect())
class Response:
def __init__(self, status: int, headers: dict[str, str], data: bytes) -> None:
self.status = status
self.headers = headers
self.data = data
try:
self.json = json.loads(data.decode("utf-8")) if data else None
except (ValueError, UnicodeDecodeError):
self.json = None
def header(self, name: str) -> str:
for key, value in self.headers.items():
if key.lower() == name.lower():
return value
return ""
def __repr__(self) -> str: # pragma: no cover - diagnostics only
return "Response(status=%s, json=%r)" % (self.status, self.json)
def request(
method: str,
path: str,
*,
body=None,
raw: bytes | None = None,
token: str | None = None,
headers: dict[str, str] | None = None,
content_type: str | None = None,
timeout: float | None = None,
base_url: str | None = None,
) -> Response:
url = (base_url or BASE_URL) + path
data = raw
request_headers = {"Accept": "application/json"}
if body is not None:
data = json.dumps(body, ensure_ascii=False).encode("utf-8")
request_headers["Content-Type"] = "application/json"
if content_type is not None:
request_headers["Content-Type"] = content_type
if token is not None:
request_headers["Authorization"] = "Bearer " + token
request_headers.update(headers or {})
prepared = urllib.request.Request(url, data=data, headers=request_headers, method=method)
try:
response = _OPENER.open(prepared, timeout=timeout or TIMEOUT)
except urllib.error.HTTPError as error:
response = error
with response:
payload = response.read()
return Response(response.code, dict(response.headers.items()), payload)
def basic(username: str, password: str) -> dict[str, str]:
encoded = base64.b64encode((username + ":" + password).encode("utf-8")).decode("ascii")
return {"Authorization": "Basic " + encoded}
def new_registration(prefix: str = "st", password: str | None = None) -> dict[str, str]:
suffix = uuid.uuid4().hex[:12]
return {
"username": prefix + "_" + suffix,
"display_name": "Тест " + suffix,
"password": password or ("St-" + uuid.uuid4().hex),
}
def register(registration: dict[str, str] | None = None) -> Response:
return request("POST", "/api/v1/users", body=registration or new_registration())
def login(username: str, password: str) -> Response:
return request("POST", "/api/v1/auth/sessions", headers=basic(username, password))
class Actor:
"""A registered user with a live session, as a client would hold it."""
def __init__(self, registration: dict[str, str]) -> None:
self.registration = registration
response = register(registration)
if response.status != 201:
raise AssertionError(
"registration failed: %s %r" % (response.status, response.json)
)
self.user = response.json
self.username = registration["username"]
self.password = registration["password"]
@property
def id(self) -> str:
return self.user["id"]
def sign_in(self) -> "Actor":
response = login(self.username, self.password)
if response.status != 201:
raise AssertionError("login failed: %s %r" % (response.status, response.json))
self.session = response.json
self.token = response.json["token"]
return self
def actor(prefix: str = "st", password: str | None = None) -> Actor:
return Actor(new_registration(prefix, password)).sign_in()
def create_chat(token: str, member_ids: list[str], title: str = "Тестовый чат") -> dict:
response = request(
"POST",
"/api/v1/chats",
body={"title": title, "member_ids": member_ids},
token=token,
)
if response.status != 201:
raise AssertionError("chat creation failed: %s %r" % (response.status, response.json))
return response.json
def send(token: str, chat_id: str, text: str) -> dict:
response = request(
"POST", "/api/v1/chats/" + chat_id + "/messages", body={"text": text}, token=token
)
if response.status != 201:
raise AssertionError("message failed: %s %r" % (response.status, response.json))
return response.json
def history(token: str, chat_id: str, limit: int = 100) -> list[dict]:
"""Walk every page of a chat, guarding against cursor loops."""
items: list[dict] = []
seen: set[str] = set()
cursor: str | None = None
while True:
path = "/api/v1/chats/" + chat_id + "/messages?limit=" + str(limit)
if cursor is not None:
path += "&cursor=" + urllib.parse.quote(cursor, safe="")
response = request("GET", path, token=token)
if response.status != 200:
raise AssertionError("history failed: %s %r" % (response.status, response.json))
items.extend(response.json["items"])
cursor = response.json["next_cursor"]
if cursor is None:
return items
if cursor in seen:
raise AssertionError("cursor loop detected")
seen.add(cursor)
def expect_error(test, response: Response, status: int, code: str) -> dict:
"""Assert the public error envelope, request id echo and the status."""
test.assertEqual(response.status, status, response.json)
test.assertIsInstance(response.json, dict)
error = response.json.get("error")
test.assertIsNotNone(error, "missing error envelope")
test.assertEqual(error["code"], code)
test.assertTrue(error["message"])
request_id = response.header("X-Request-Id")
test.assertTrue(request_id, "missing X-Request-Id header")
test.assertEqual(error["request_id"], request_id)
return error
+329
View File
@@ -0,0 +1,329 @@
"""Own functional tests: contract edges carried from lab 1 plus lab 2 auth.
Every test creates unique synthetic users, so the file is safe to re-run against
the same stand (the API has no delete endpoint by design).
"""
from __future__ import annotations
import unittest
import uuid
from datetime import datetime, timezone
import common
def timestamp(value: str) -> datetime:
return datetime.fromisoformat(value[:-1] + "+00:00")
class Base(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
again = common.request("POST", "/api/v1/users", body=common.new_registration())
if again.status != 201:
raise AssertionError("stand is not serving: %r" % (again.json,))
cls.alice = common.actor("alice")
cls.bob = common.actor("bob")
cls.eve = common.actor("eve")
cls.chat = common.create_chat(
cls.alice.token, [cls.alice.id, cls.bob.id], "Контрактный чат"
)
class HealthAndEnvelope(Base):
def test_health_endpoints_and_request_id(self) -> None:
live = common.request("GET", "/health/live")
self.assertEqual(live.status, 200)
self.assertEqual(live.json, {"status": "ok"})
self.assertTrue(live.header("X-Request-Id"))
ready = common.request("GET", "/health/ready")
self.assertEqual(ready.status, 200)
self.assertEqual(ready.json, {"status": "ok"})
def test_unknown_route_uses_same_error_envelope(self) -> None:
common.expect_error(self, common.request("GET", "/api/v1/nope"), 404, "not_found")
def test_query_parameters_rejected_where_not_supported(self) -> None:
response = common.request(
"GET", "/api/v1/users/me?verbose=1", token=self.alice.token
)
common.expect_error(self, response, 400, "invalid_request")
class Registration(Base):
def test_response_has_no_secret_fields(self) -> None:
response = common.register(common.new_registration())
self.assertEqual(response.status, 201)
self.assertEqual(
set(response.json), {"id", "username", "display_name", "created_at"}
)
self.assertNotIn("password", response.json)
self.assertNotIn("password_hash", response.json)
def test_duplicate_username_conflict(self) -> None:
registration = common.new_registration()
self.assertEqual(common.register(registration).status, 201)
common.expect_error(self, common.register(registration), 409, "conflict")
def test_username_rules(self) -> None:
for username in ("UPPER_case", "ab", "with-dash", "with space"):
body = common.new_registration()
body["username"] = username
common.expect_error(self, common.register(body), 400, "invalid_request")
def test_password_rules(self) -> None:
short = common.new_registration(password="x" * 11)
common.expect_error(self, common.register(short), 400, "invalid_request")
long_password = common.new_registration(password="x" * 129)
common.expect_error(self, common.register(long_password), 400, "invalid_request")
# 12 Unicode code points are enough, even when they are multi-byte.
unicode_password = common.new_registration(password="😀" * 12)
self.assertEqual(common.register(unicode_password).status, 201)
def test_malformed_json_extra_fields_and_wrong_types(self) -> None:
common.expect_error(
self,
common.request(
"POST", "/api/v1/users", raw=b'{"username": ', content_type="application/json"
),
400,
"invalid_request",
)
body = common.new_registration()
body["is_admin"] = True
common.expect_error(self, common.register(body), 400, "invalid_request")
body = common.new_registration()
body["display_name"] = 42
common.expect_error(self, common.register(body), 400, "invalid_request")
def test_unsupported_media_type(self) -> None:
response = common.request(
"POST",
"/api/v1/users",
raw=b"username=alice",
content_type="application/x-www-form-urlencoded",
)
common.expect_error(self, response, 415, "unsupported_media_type")
class Chats(Base):
def test_membership_validation(self) -> None:
cases = [
[self.bob.id], # caller is missing
[self.alice.id, self.alice.id], # duplicate member
[self.alice.id, str(uuid.uuid4())], # unknown member
[], # no members
]
for member_ids in cases:
response = common.request(
"POST",
"/api/v1/chats",
body={"title": "Плохой чат", "member_ids": member_ids},
token=self.alice.token,
)
common.expect_error(self, response, 400, "invalid_request")
def test_foreign_chat_is_hidden_everywhere(self) -> None:
chat_id = self.chat["id"]
common.expect_error(
self,
common.request("GET", "/api/v1/chats/" + chat_id, token=self.eve.token),
404,
"not_found",
)
common.expect_error(
self,
common.request(
"GET", "/api/v1/chats/" + chat_id + "/messages", token=self.eve.token
),
404,
"not_found",
)
common.expect_error(
self,
common.request(
"POST",
"/api/v1/chats/" + chat_id + "/messages",
body={"text": "чужое"},
token=self.eve.token,
),
404,
"not_found",
)
listed = common.request("GET", "/api/v1/chats?limit=100", token=self.eve.token)
self.assertEqual(listed.status, 200)
self.assertNotIn(chat_id, [item["id"] for item in listed.json["items"]])
def test_messages_are_validated_and_unicode_preserved(self) -> None:
chat = common.create_chat(self.alice.token, [self.alice.id, self.bob.id])
for text in ("", " \n\t ", "x" * 4001):
response = common.request(
"POST",
"/api/v1/chats/" + chat["id"] + "/messages",
body={"text": text},
token=self.alice.token,
)
common.expect_error(self, response, 400, "invalid_request")
prefix = "Привет 👋 世界 "
text = prefix + "x" * (4000 - len(prefix)) # exactly the contract maximum
message = common.send(self.alice.token, chat["id"], text)
self.assertEqual(message["text"], text)
self.assertEqual(message["sender_id"], self.alice.id)
injected = common.request(
"POST",
"/api/v1/chats/" + chat["id"] + "/messages",
body={"text": "подмена", "sender_id": self.bob.id},
token=self.alice.token,
)
common.expect_error(self, injected, 400, "invalid_request")
class Pagination(Base):
def test_walks_every_page_without_loss_or_duplicates(self) -> None:
chat = common.create_chat(self.alice.token, [self.alice.id, self.bob.id])
sent = [
common.send(self.alice.token if index % 2 else self.bob.token, chat["id"], "msg-%d" % index)
for index in range(7)
]
collected: list[dict] = []
cursor = None
pages = 0
while True:
path = "/api/v1/chats/" + chat["id"] + "/messages?limit=2"
if cursor is not None:
path += "&cursor=" + cursor
page = common.request("GET", path, token=self.alice.token)
self.assertEqual(page.status, 200)
collected.extend(page.json["items"])
pages += 1
cursor = page.json["next_cursor"]
if cursor is None:
break
self.assertLess(pages, 10, "pagination did not terminate")
self.assertEqual(pages, 4)
self.assertEqual([item["id"] for item in collected], [item["id"] for item in sent])
self.assertEqual(
collected,
sorted(collected, key=lambda item: (timestamp(item["created_at"]), item["id"])),
)
def test_cursor_is_bound_to_its_chat(self) -> None:
other = common.create_chat(self.alice.token, [self.alice.id])
common.send(self.alice.token, other["id"], "a")
common.send(self.alice.token, other["id"], "b")
first = common.request(
"GET", "/api/v1/chats/" + other["id"] + "/messages?limit=1", token=self.alice.token
)
cursor = first.json["next_cursor"]
self.assertIsNotNone(cursor)
response = common.request(
"GET",
"/api/v1/chats/" + self.chat["id"] + "/messages" + "?limit=1&cursor=" + cursor,
token=self.alice.token,
)
common.expect_error(self, response, 400, "invalid_request")
def test_limit_and_cursor_validation(self) -> None:
base = "/api/v1/chats/" + self.chat["id"] + "/messages"
for query in ("?limit=0", "?limit=101", "?limit=abc", "?cursor=", "?cursor=not-a-cursor", "?limit=1&limit=2", "?unknown=1"):
common.expect_error(
self,
common.request("GET", base + query, token=self.alice.token),
400,
"invalid_request",
)
class Authentication(Base):
def test_basic_is_not_accepted_outside_session_creation(self) -> None:
response = common.request(
"GET", "/api/v1/users/me", headers=common.basic(self.alice.username, self.alice.password)
)
common.expect_error(self, response, 401, "unauthorized")
self.assertTrue(response.header("WWW-Authenticate").lower().startswith("bearer"))
def test_missing_and_unknown_tokens(self) -> None:
for headers in ({}, {"Authorization": "Bearer not-a-real-token"}):
response = common.request("GET", "/api/v1/users/me", headers=headers)
common.expect_error(self, response, 401, "unauthorized")
self.assertTrue(response.header("WWW-Authenticate").lower().startswith("bearer"))
def test_x_user_id_is_ignored(self) -> None:
response = common.request("GET", "/api/v1/users/me", headers={"X-User-Id": self.alice.id})
common.expect_error(self, response, 401, "unauthorized")
overridden = common.request(
"GET",
"/api/v1/users/me",
token=self.alice.token,
headers={"X-User-Id": self.eve.id},
)
self.assertEqual(overridden.status, 200)
self.assertEqual(overridden.json["id"], self.alice.id)
def test_session_payload_and_expiry(self) -> None:
response = common.login(self.alice.username, self.alice.password)
self.assertEqual(response.status, 201)
session = response.json
self.assertEqual(set(session), {"token", "token_type", "expires_at", "user"})
self.assertEqual(session["token_type"], "Bearer")
self.assertGreaterEqual(len(session["token"]), 43)
self.assertNotIn(self.alice.username, session["token"])
self.assertNotEqual(session["token"], self.alice.id)
self.assertEqual(session["user"], self.alice.user)
self.assertGreater(timestamp(session["expires_at"]), datetime.now(timezone.utc))
me = common.request("GET", "/api/v1/users/me", token=session["token"])
self.assertEqual(me.status, 200)
self.assertEqual(me.json, self.alice.user)
def test_wrong_password_and_unknown_user_are_indistinguishable(self) -> None:
wrong = common.login(self.alice.username, "definitely-wrong-password")
unknown = common.login("nobody_here_" + uuid.uuid4().hex[:8], "definitely-wrong-password")
messages = []
for response in (wrong, unknown):
error = common.expect_error(self, response, 401, "unauthorized")
self.assertTrue(
response.header("WWW-Authenticate").lower().startswith("basic")
)
messages.append(error["message"])
# Same message and challenge: a client cannot tell the accounts apart.
self.assertEqual(messages[0], messages[1])
def test_logout_revokes_only_that_session(self) -> None:
first = common.login(self.alice.username, self.alice.password).json["token"]
second = common.login(self.alice.username, self.alice.password).json["token"]
revoked = common.request("DELETE", "/api/v1/auth/sessions/current", token=first)
self.assertEqual(revoked.status, 204)
self.assertEqual(revoked.data, b"")
common.expect_error(
self,
common.request("GET", "/api/v1/users/me", token=first),
401,
"unauthorized",
)
common.expect_error(
self,
common.request("DELETE", "/api/v1/auth/sessions/current", token=first),
401,
"unauthorized",
)
still_valid = common.request("GET", "/api/v1/users/me", token=second)
self.assertEqual(still_valid.status, 200)
def test_session_endpoint_rejects_json_body_and_bad_basic(self) -> None:
response = common.request(
"POST",
"/api/v1/auth/sessions",
body={"username": self.alice.username, "password": self.alice.password},
)
common.expect_error(self, response, 401, "unauthorized")
response = common.request(
"POST", "/api/v1/auth/sessions", headers={"Authorization": "Basic !!!not-base64!!!"}
)
common.expect_error(self, response, 401, "unauthorized")
self.assertTrue(response.header("WWW-Authenticate").lower().startswith("basic"))
if __name__ == "__main__":
unittest.main()
+206
View File
@@ -0,0 +1,206 @@
"""Dependency outage behaviour: Valkey down, then PostgreSQL down.
Requires Docker access on the same host as the Compose stand and is therefore
opt-in:
STUDENT_DOCKER_TESTS=1 python -m unittest -v test_degradation
The test never removes volumes and restores both services in `finally`.
"""
from __future__ import annotations
import json
import os
import statistics
import subprocess
import time
import unittest
from pathlib import Path
import common
REPO_ROOT = Path(__file__).resolve().parents[1]
PROJECT = os.environ.get("COMPOSE_PROJECT_NAME", "mtusi2026lab2")
ENV_FILE = os.environ.get("COMPOSE_ENV_FILE", ".env")
ENABLED = os.environ.get("STUDENT_DOCKER_TESTS") == "1"
def compose(*arguments: str, timeout: int = 120) -> None:
result = subprocess.run(
["docker", "compose", "-p", PROJECT, "--env-file", ENV_FILE, *arguments],
cwd=str(REPO_ROOT),
capture_output=True,
text=True,
timeout=timeout,
)
if result.returncode != 0:
raise AssertionError(
"docker compose %s failed: %s" % (" ".join(arguments), result.stderr[-500:])
)
def wait_ready(expected: int, seconds: float) -> float:
started = time.monotonic()
deadline = started + seconds
last = None
while time.monotonic() < deadline:
try:
last = common.request("GET", "/health/ready", timeout=5)
if last.status == expected:
return time.monotonic() - started
except OSError:
# The TCP listener may be replaced during `compose restart`.
pass
time.sleep(0.5)
raise AssertionError(
"readiness did not reach %s within %.0fs (last=%s %r)"
% (expected, seconds, last.status if last else None, last.json if last else None)
)
@unittest.skipUnless(ENABLED, "set STUDENT_DOCKER_TESTS=1 to stop real containers")
class DependencyOutages(unittest.TestCase):
def test_logout_remains_revoked_after_cache_outage_and_api_restart(self) -> None:
"""A stale Valkey hit must never resurrect a PostgreSQL-revoked token."""
alice = common.actor("revoked")
chat = common.create_chat(alice.token, [alice.id], "Durable logout")
# Store the live session in Valkey before its simulated outage.
self.assertEqual(common.request("GET", "/api/v1/users/me", token=alice.token).status, 200)
try:
compose("stop", "valkey")
logout = common.request("DELETE", "/api/v1/auth/sessions/current", token=alice.token)
self.assertEqual(logout.status, 204, logout.json)
# Lose the local pending tombstone, then restore Valkey's old volume.
compose("restart", "api")
compose("start", "valkey")
wait_ready(200, 90)
common.expect_error(
self, common.request("GET", "/api/v1/users/me", token=alice.token), 401, "unauthorized"
)
common.expect_error(
self,
common.request("GET", "/api/v1/chats/" + chat["id"], token=alice.token),
401,
"unauthorized",
)
common.expect_error(
self,
common.request(
"POST",
"/api/v1/chats/" + chat["id"] + "/messages",
body={"text": "must not be accepted"},
token=alice.token,
),
401,
"unauthorized",
)
finally:
compose("start", "valkey", "postgres", "api")
try:
wait_ready(200, 90)
except AssertionError:
pass
def test_valkey_loss_degrades_then_postgres_loss_is_bounded(self) -> None:
alice = common.actor("degrade")
chat = common.create_chat(alice.token, [alice.id], "Устойчивость")
# Warm the session cache.
self.assertEqual(common.request("GET", "/api/v1/users/me", token=alice.token).status, 200)
cached = self._sample_latency(alice.token, 60)
summary = {"cached_p50_ms": round(cached["p50"], 2)}
try:
# 1. Valkey is a cache: auth still works, without DB-free fast path.
compose("stop", "valkey")
ready = common.request("GET", "/health/ready")
self.assertEqual(
ready.status, 200, "session cache must not be a readiness dependency"
)
me = common.request("GET", "/api/v1/users/me", token=alice.token)
self.assertEqual(me.status, 200, me.json)
sent = common.request(
"POST",
"/api/v1/chats/" + chat["id"] + "/messages",
body={"text": "без кэша"},
token=alice.token,
)
self.assertEqual(sent.status, 201, sent.json)
without_cache = self._sample_latency(alice.token, 60)
summary["cache_down_p50_ms"] = round(without_cache["p50"], 2)
summary["cache_down_p95_ms"] = round(without_cache["p95"], 2)
compose("start", "valkey")
time.sleep(3)
self.assertEqual(common.request("GET", "/health/ready").status, 200)
# 2. PostgreSQL is the source of truth: bounded errors, never a 200
# for a token the server cannot verify, and never a raw 500.
compose("stop", "postgres")
ready = common.request("GET", "/health/ready")
self.assertEqual(ready.status, 503, ready.json)
self.assertEqual(ready.json["error"]["code"], "unavailable")
started = time.monotonic()
write = common.request(
"POST",
"/api/v1/chats/" + chat["id"] + "/messages",
body={"text": "без базы"},
token=alice.token,
timeout=30,
)
bounded = time.monotonic() - started
self.assertEqual(write.status, 503, write.json)
self.assertEqual(write.json["error"]["code"], "unavailable")
self.assertLess(bounded, 10, "database failure must fail fast")
summary["db_down_write_seconds"] = round(bounded, 2)
for leaked in ("psycopg", "postgresql://", "password", "Traceback"):
self.assertNotIn(leaked, json.dumps(write.json))
unknown = common.request(
"GET", "/api/v1/users/me", token="unknown-token-value", timeout=30
)
self.assertIn(unknown.status, (401, 503))
self.assertNotEqual(unknown.status, 200)
# 3. Recovery keeps the same session and the stored chat.
compose("start", "postgres")
summary["recovery_seconds"] = round(wait_ready(200, 90), 2)
after = common.request("GET", "/api/v1/chats/" + chat["id"], token=alice.token)
self.assertEqual(after.status, 200, after.json)
recovered = common.request(
"POST",
"/api/v1/chats/" + chat["id"] + "/messages",
body={"text": "после восстановления"},
token=alice.token,
)
self.assertEqual(recovered.status, 201, recovered.json)
messages = common.history(alice.token, chat["id"])
self.assertIn("после восстановления", [message["text"] for message in messages])
finally:
compose("start", "valkey", "postgres")
try:
wait_ready(200, 90)
except AssertionError:
pass
print("\n outage summary: " + json.dumps(summary, ensure_ascii=False))
def _sample_latency(self, token: str, count: int) -> dict[str, float]:
samples = []
for _ in range(count):
started = time.monotonic()
response = common.request("GET", "/api/v1/users/me", token=token, timeout=30)
samples.append(time.monotonic() - started)
self.assertEqual(response.status, 200, response.json)
ordered = sorted(samples)
return {
"p50": statistics.median(samples) * 1000,
"p95": ordered[max(0, int(len(ordered) * 0.95) - 1)] * 1000,
}
if __name__ == "__main__":
unittest.main()
+99
View File
@@ -0,0 +1,99 @@
"""Session expiry with SESSION_TTL_SECONDS=2, plus logout and bad credentials.
Run against the dedicated short-TTL stand:
scripts/ttl-stack.sh start
BASE_URL=http://localhost:8081 python -m unittest -v test_expiry
scripts/ttl-stack.sh stop
"""
from __future__ import annotations
import statistics
import time
import unittest
from datetime import datetime, timezone
import common
def expires_at(session: dict) -> datetime:
return datetime.fromisoformat(session["expires_at"][:-1] + "+00:00")
class ShortTtlSessions(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
ready = common.request("GET", "/health/ready")
if ready.status != 200:
raise AssertionError("stand is not ready: %r" % (ready.json,))
def test_expiry_is_enforced_and_cache_does_not_extend_it(self) -> None:
user = common.actor("ttl")
started = datetime.now(timezone.utc)
session = common.login(user.username, user.password)
self.assertEqual(session.status, 201)
lifetime = (expires_at(session.json) - started).total_seconds()
self.assertLessEqual(lifetime, 5.0, "stand does not use a short TTL")
# Two reads: the second one is served from the Valkey cache, so the
# expiry check below covers the cached path as well.
for _ in range(2):
self.assertEqual(
common.request("GET", "/api/v1/users/me", token=session.json["token"]).status,
200,
)
time.sleep(3.2)
expired = common.request("GET", "/api/v1/users/me", token=session.json["token"])
common.expect_error(self, expired, 401, "unauthorized")
self.assertTrue(expired.header("WWW-Authenticate").lower().startswith("bearer"))
revoked = common.request(
"DELETE", "/api/v1/auth/sessions/current", token=session.json["token"]
)
common.expect_error(self, revoked, 401, "unauthorized")
# A fresh login still works, so expiry removed only the old session.
again = common.login(user.username, user.password)
self.assertEqual(again.status, 201)
self.assertEqual(
common.request("GET", "/api/v1/users/me", token=again.json["token"]).status,
200,
)
def test_logout_is_immediate_even_when_the_session_is_cached(self) -> None:
user = common.actor("ttllogout")
token = user.token
self.assertEqual(common.request("GET", "/api/v1/users/me", token=token).status, 200)
self.assertEqual(
common.request("DELETE", "/api/v1/auth/sessions/current", token=token).status,
204,
)
outcome = common.request("GET", "/api/v1/users/me", token=token)
common.expect_error(self, outcome, 401, "unauthorized")
def test_wrong_password_issues_no_session(self) -> None:
user = common.actor("ttlwrong")
denied = common.login(user.username, "not-the-password")
common.expect_error(self, denied, 401, "unauthorized")
self.assertTrue(denied.header("WWW-Authenticate").lower().startswith("basic"))
def test_session_lookup_latency_is_bounded(self) -> None:
user = common.actor("ttllatency")
samples = []
for _ in range(30):
started = time.monotonic()
response = common.request("GET", "/api/v1/users/me", token=user.token, timeout=5)
samples.append(time.monotonic() - started)
self.assertEqual(response.status, 200)
print(
"\n /users/me warm cache p50=%.1fms p95=%.1fms"
% (
statistics.median(samples) * 1000,
sorted(samples)[int(len(samples) * 0.95) - 1] * 1000,
)
)
if __name__ == "__main__":
unittest.main()
+142
View File
@@ -0,0 +1,142 @@
"""Race, rollback and pool tests against the real PostgreSQL stand.
These are the checks a code reviewer would ask for on top of the smoke test:
concurrent writers must not lose or duplicate messages, the username uniqueness
must be decided by the database (not by a read-then-write check in Python), and
a failed composite operation must leave nothing behind.
"""
from __future__ import annotations
import threading
import time
import unittest
import uuid
import common
class ConcurrentWrites(unittest.TestCase):
def test_100_messages_from_10_clients_are_exactly_100(self) -> None:
alice = common.actor("cwalice")
bob = common.actor("cwbob")
chat = common.create_chat(alice.token, [alice.id, bob.id], "Гонка сообщений")
writers = 10
per_writer = 10
barrier = threading.Barrier(writers)
errors: list[str] = []
sent: list[str] = []
lock = threading.Lock()
def writer(index: int) -> None:
token = alice.token if index % 2 else bob.token
barrier.wait()
for number in range(per_writer):
text = "race-%d-%d" % (index, number)
try:
common.send(token, chat["id"], text)
with lock:
sent.append(text)
except AssertionError as error: # pragma: no cover - failure detail
with lock:
errors.append(str(error))
threads = [threading.Thread(target=writer, args=(index,)) for index in range(writers)]
started = time.monotonic()
for thread in threads:
thread.start()
for thread in threads:
thread.join(timeout=120)
elapsed = time.monotonic() - started
self.assertEqual(errors, [])
self.assertEqual(len(sent), writers * per_writer)
messages = common.history(alice.token, chat["id"], limit=100)
self.assertEqual(len(messages), writers * per_writer)
self.assertEqual(len({message["id"] for message in messages}), len(messages))
self.assertEqual(sorted(message["text"] for message in messages), sorted(sent))
print("\n concurrent writes: %d messages from %d clients in %.2fs"
% (len(messages), writers, elapsed))
def test_duplicate_username_race_creates_one_user(self) -> None:
registration = common.new_registration("race")
attempts = 12
barrier = threading.Barrier(attempts)
statuses: list[int] = []
lock = threading.Lock()
def attempt() -> None:
barrier.wait()
response = common.register(dict(registration))
with lock:
statuses.append(response.status)
threads = [threading.Thread(target=attempt) for _ in range(attempts)]
for thread in threads:
thread.start()
for thread in threads:
thread.join(timeout=60)
self.assertEqual(statuses.count(201), 1, statuses)
self.assertEqual(statuses.count(409), attempts - 1, statuses)
# The single winner is usable, so exactly one row exists.
session = common.login(registration["username"], registration["password"])
self.assertEqual(session.status, 201)
self.assertEqual(session.json["user"]["username"], registration["username"])
def test_chat_creation_rolls_back_on_unknown_member(self) -> None:
alice = common.actor("rollback")
before = self._chat_titles(alice.token)
response = common.request(
"POST",
"/api/v1/chats",
body={"title": "Атомарный чат", "member_ids": [alice.id, str(uuid.uuid4())]},
token=alice.token,
)
common.expect_error(self, response, 400, "invalid_request")
after = self._chat_titles(alice.token)
# The chat row inserted before the failing membership step is gone.
self.assertEqual(before, after)
self.assertNotIn("Атомарный чат", after)
def _chat_titles(self, token: str) -> list[str]:
titles: list[str] = []
cursor = None
for _ in range(50):
path = "/api/v1/chats?limit=100"
if cursor:
path += "&cursor=" + cursor
response = common.request("GET", path, token=token)
self.assertEqual(response.status, 200)
titles.extend(item["title"] for item in response.json["items"])
cursor = response.json["next_cursor"]
if cursor is None:
return titles
self.fail("chat pagination did not terminate")
def test_bounded_pool_serves_parallel_reads(self) -> None:
alice = common.actor("pool")
parallel = 50
barrier = threading.Barrier(parallel)
statuses: list[int] = []
lock = threading.Lock()
def read() -> None:
barrier.wait()
# Chat lists are not cached, so this exercises the bounded SQL pool.
response = common.request(
"GET", "/api/v1/chats?limit=100", token=alice.token, timeout=10
)
with lock:
statuses.append(response.status)
threads = [threading.Thread(target=read) for _ in range(parallel)]
for thread in threads:
thread.start()
for thread in threads:
thread.join(timeout=60)
self.assertEqual(len(statuses), parallel)
self.assertEqual(statuses, [200] * parallel)
if __name__ == "__main__":
unittest.main()