ЛР 3 — БВТ2301 — Александров— на 4 #81

Open
t-d-aleksandrov-edu-mtuci-ru wants to merge 3 commits from t-d-aleksandrov-edu-mtuci-ru/mtusi2026lab3:lab3-implementation into main
56 changed files with 4185 additions and 43 deletions
+18
View File
@@ -0,0 +1,18 @@
# Shell scripts and configs must stay LF so they run in Linux containers.
*.sh text eol=lf
*.conf text eol=lf
nginx.conf text eol=lf
Dockerfile text eol=lf
dockerignore text eol=lf
compose.yaml text eol=lf
compose.example.yaml text eol=lf
entrypoint.sh text eol=lf
# Python and text files are fine with platform-native line endings.
*.py text
*.md text
*.json text
*.txt text
*.sql text
.env.example text
Makefile text
+47 -14
View File
@@ -1,18 +1,51 @@
## Лабораторная 3
Автор / группа: TODO
Целевая оценка (3/4/5): TODO
Трек 5, если заявлен: TODO
Автор / группа: Тимофей Александров, БВТ2301
Целевая оценка (3/4/5): 5
Трек 5: трек 1 «Цена масштабирования»
Что реализовано и что перенесено: TODO
Команды запуска из чистого clone: TODO
Результаты проверок и ссылка на REPORT.md: TODO
Ограничения / известные ошибки: TODO
Источники и использование ИИ: TODO
Что реализовано и что перенесено:
- Перенесено из lab2: PostgreSQL источник истины, Valkey кэш сессий, Argon2id пароли, Bearer-сессии, транзакционное создание чата, keyset-пагинация, circuit breaker на Valkey.
- Новое в lab3: 2 API-реплики (api-1, api-2) с уникальным INSTANCE_ID; Nginx load balancer с health-check и автоматическим failover; Idempotency-Key в заголовке (replay возвращает исходное сообщение); X-Instance-Id в каждом ответе; /version и /info эндпоинты; housekeeping-loop для очистки истёкших сессий и ключей; таблица idempotency_keys с миграциями 004–007.
- Исправлено по ревью: scope idempotency — `(actor_id, chat_id, key)`; атомарное резервирование ключа не даёт 500 при 32 concurrent replays; Bearer-сессия проверяет PostgreSQL на каждой реплике, поэтому старый cache-hit не может воскресить отозванный токен.
- Исправлены импорты config/models/api/store/sessions (убраны несуществующие `config`, добавлены `IDEMPOTENCY_KEY_PATTERN`, `__version__`).
- Добавлены скрипты stop-one.sh/cmd (failover) и restart.sh/cmd (persistence).
- [ ] Весь обязательный минимум текущей и предыдущих работ сохранён с объявленными переходами.
- [ ] Контракт и публичные тесты не ослаблены.
- [ ] make check и make test выполнены; приложен реальный вывод.
- [ ] Сценарии отказов/безопасности своего уровня воспроизведены.
- [ ] Все секреты и приватные ключи находятся вне Git; отчёт не содержит tokens.
- [ ] README задания сохранён, REPORT.md содержит инструкцию именно моего решения.
Команды запуска из чистого clone:
```sh
sh scripts/bootstrap-secrets.sh
docker compose --env-file .env config --quiet
docker compose --env-file .env up -d --build --wait --wait-timeout 180
python3 scripts/check_contract.py
python3 scripts/validate_openapi.py
python3 tests/smoke.py --base-url http://127.0.0.1:8081
python3 tests/resilience.py --mode persistence --action-script scripts/restart.sh --base-url http://127.0.0.1:8081
python3 tests/resilience.py --mode failover --action-script scripts/stop-one.sh --base-url http://127.0.0.1:8081
cd student-tests && BASE_URL=http://127.0.0.1:8081 python3 -u -B -m unittest -v test_api test_race test_idempotency test_degradation
cd .. && python3 scripts/research-scale.py --base-url http://127.0.0.1:8081 --duration 10 --concurrency 4
```
Результаты проверок и ссылка на REPORT.md:
- Smoke: PASS: lab3 public API scenario
- check_contract: PASS: offline skeleton structure, local refs and Python syntax
- validate_openapi: PASS: full OpenAPI 3.1 and JSON Schema validation
- Student tests: 28/28 OK; idempotency 2/2 OK (32 concurrent replays); degradation 2/2 OK (durable cross-replica logout)
- Failover: PASS — 79 успешных записей из 83, восстановление за 2.6 с
- Persistence: PASS — данные и сессия сохранены после рестарта
- Degradation: OK — cache_down_p50=47 мс, db_down_write=2.05 с, recovery=1.97 с
- Трек 5: fixed total RPS 1/2/4 = 36.9 / 49.4 / 36.2; growing resources = 42.4 / 45.2 / 57.0. Raw JSON: `evidence/scale-*.json`.
- Подробности: REPORT.md
Ограничения / известные ошибки:
- На Windows .sh-скрипты не запускаются напрямую (WinError 193); созданы эквиваленты .cmd.
- PostgreSQL и Nginx остаются одиночными узлами; трек не заявляет выживание их отказа.
- Нагрузочный тест закрытого цикла содержит coordinated omission.
Источники и использование ИИ: README/COURSE/CONTRIBUTING этого skeleton, contracts/README.md, tests/README.md; документация PostgreSQL, Valkey, Nginx; RFC 7617; OWASP Password Storage Cheat Sheet. ИИ (Codex) использовался для изучения задания, проектирования, написания исходников и тестов, проверки и подготовки отчёта. Автор проверяет результат и должен уметь объяснить и изменить его на защите.
- [x] Весь обязательный минимум текущей и предыдущих работ сохранён с объявленными переходами.
- [x] Контракт и публичные тесты не ослаблены.
- [x] make check и make test выполнены; приложен реальный вывод.
- [x] Сценарии отказов/безопасности своего уровня воспроизведены.
- [x] Все секреты и приватные ключи находятся вне Git; отчёт не содержит tokens.
- [x] README задания сохранён, REPORT.md содержит инструкцию именно моего решения.
+35
View File
@@ -0,0 +1,35 @@
# syntax=docker/dockerfile:1
# Runtime image is pinned by digest, dependencies by version in requirements.txt.
FROM python:3.12.7-slim-bookworm@sha256:60d9996b6a8a3689d36db740b49f4327be3be09a21122bd02fb8895abb38b50d
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
APP_HOST=0.0.0.0 \
APP_PORT=8080
WORKDIR /app
# Unprivileged runtime user; the API never needs to write to the image.
RUN groupadd --system --gid 10001 messenger \
&& useradd --system --uid 10001 --gid messenger --no-create-home messenger
COPY requirements.txt /app/requirements.txt
RUN pip install --no-cache-dir --requirement /app/requirements.txt
COPY --chown=messenger:messenger src/ /app/src/
COPY --chown=messenger:messenger contracts/ /app/contracts/
COPY --chown=messenger:messenger --chmod=0555 entrypoint.sh /app/entrypoint.sh
# Windows hosts may store CRLF; Python and /bin/sh need LF.
RUN find /app/src -name '*.py' -print0 | xargs -0 sed -i 's/\r$//' \
&& sed -i 's/\r$//' /app/entrypoint.sh
USER messenger
EXPOSE 8080
STOPSIGNAL SIGTERM
# Readiness (not just liveness) so an unhealthy container is visible to Compose.
HEALTHCHECK --interval=5s --timeout=5s --start-period=20s --retries=3 \
CMD ["python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/health/ready', timeout=2).read()"]
# Migrations are idempotent; the entrypoint execs the server so it is PID 1.
ENTRYPOINT ["/app/entrypoint.sh"]
+37 -4
View File
@@ -1,13 +1,20 @@
PYTHON ?= python3
BASE_URL ?= http://localhost:8080
# 127.0.0.1, not localhost: compose publishes the API on IPv4 only, and a
# dual-stack localhost would add a ~2 s IPv6 retry to every request.
BASE_URL ?= http://127.0.0.1:8080
CA_FILE ?=
ACTION_SCRIPT ?=
TLS_ARGS = $(if $(CA_FILE),--ca-file "$(CA_FILE)",)
TARGET_INSTANCE_ID ?= api-1
STUDENT_DIR = student-tests
.PHONY: help check validate test load test-persistence test-failover
.PHONY: help check validate test load test-persistence test-failover run bootstrap up down test-student test-degradation explain research-scale
help:
@echo "check: offline structure; validate: full OpenAPI (requirements-dev.txt); test: running API; load: synthetic writes"
@echo "bootstrap/up/down: local Compose stand; run: API without Docker (needs DATABASE_URL/CACHE_URL)"
@echo "test-student: own functional+race tests; test-persistence: restart stack; test-failover: stop one API replica"
@echo "explain: seed 12k messages and capture EXPLAIN (ANALYZE, BUFFERS) into evidence/"
@echo "research-scale: grade-5 1/2/4 replica experiment; set BASE_URL, DURATION and CONCURRENCY as needed"
check:
$(PYTHON) scripts/check_contract.py
@@ -27,4 +34,30 @@ test-persistence:
test-failover:
@test -n "$(ACTION_SCRIPT)" || (echo "Set ACTION_SCRIPT to your implemented executable stop-one hook"; exit 2)
$(PYTHON) tests/resilience.py --mode failover --action-script "$(ACTION_SCRIPT)" --base-url "$(BASE_URL)" $(TLS_ARGS)
TARGET_INSTANCE_ID="$(TARGET_INSTANCE_ID)" $(PYTHON) tests/resilience.py --mode failover --action-script "$(ACTION_SCRIPT)" --base-url "$(BASE_URL)" $(TLS_ARGS)
# --- lab 3 ----------------------------------------------------------------
run:
$(PYTHON) -B src/app.py
bootstrap:
sh scripts/bootstrap-secrets.sh
up: bootstrap
docker compose --env-file .env up -d --build --wait --wait-timeout 180
docker compose --env-file .env ps
down:
docker compose --env-file .env down
test-student:
cd $(STUDENT_DIR) && BASE_URL="$(BASE_URL)" $(PYTHON) -B -m unittest -v test_api test_race test_idempotency
test-degradation:
cd $(STUDENT_DIR) && BASE_URL="$(BASE_URL)" STUDENT_DOCKER_TESTS=1 $(PYTHON) -B -m unittest -v test_degradation
explain:
sh scripts/explain-history.sh
research-scale:
$(PYTHON) scripts/research-scale.py --base-url "$(BASE_URL)"
+92 -14
View File
@@ -1,37 +1,115 @@
# Отчёт — лабораторная 3
- Автор, группа: TODO
- Целевая оценка; выбранный трек 5, если нужен: TODO
- Commit/PR: TODO после публикации
- Использованные источники, библиотеки и помощь ИИ: TODO
- Автор, группа: Тимофей Александров, БВТ2301
- Целевая оценка; выбранный трек 5, если нужен: **5**, трек 1 «Цена масштабирования»
- Commit/PR: подготовлен к публикации
- Использованные источники, библиотеки и помощь ИИ: README/COURSE/CONTRIBUTING этого skeleton, `contracts/README.md` и `tests/README.md`; документация PostgreSQL (transactions, EXPLAIN), Valkey (persistence), Nginx (upstream, health checks), RFC 7617 (Basic), OWASP Password Storage Cheat Sheet (Argon2id). Библиотеки: FastAPI/Starlette/uvicorn, psycopg 3 + psycopg-pool, redis-py, argon2-cffi, nginx 1.27. Лабораторная 3 построена на кодовой базе лабораторной 2.
## Запуск из чистого clone
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
Версии, на которых всё воспроизведено: Docker Engine **29.9.2**, Docker Compose **v5.6.1**, на хосте Python **3.11.5** (нужен только для скриптов и тестов), внутри контейнера Python **3.12.7** (`python:3.12.7-slim-bookworm`), PostgreSQL **17.6**, Valkey **8.1.3**, Nginx **1.27-alpine** — образы закреплены (postgres и valkey по digest, nginx без digest). Аппаратура стенда: Intel Pentium Silver N5030 @ 1.10 GHz, 4 логических ядра, ~7.6 GiB RAM, лимитов CPU/RAM для контейнеров не задано.
Нужны: Git с POSIX-shell (на Windows — Git Bash, скрипты `scripts/*.sh`) и Docker с Compose. `make` в этой среде отсутствует, поэтому ниже приведены те же команды напрямую (сам `Makefile` сохранён и использует те же пути).
```sh
# 1) секреты и .env: генерирует ./secrets (mode 0600, в .gitignore) и копирует .env
sh scripts/bootstrap-secrets.sh # на Windows: sh из Git Bash
# 2) проверка конфигурации и подъём стенда (миграции применяет entrypoint контейнера api)
docker compose --env-file .env config --quiet
docker compose --env-file .env up -d --build --wait --wait-timeout 180
docker compose --env-file .env ps
# 3) проверки
python3 scripts/check_contract.py # = make check
python3 scripts/validate_openapi.py # = make validate (нужен requirements-dev.txt)
python3 tests/smoke.py --base-url http://127.0.0.1:8081 # = make test (порт 8081)
python3 tests/resilience.py --mode persistence \
--action-script scripts/restart.sh --base-url http://127.0.0.1:8081 # = make test-persistence
python3 tests/resilience.py --mode failover \
--action-script scripts/stop-one.sh --base-url http://127.0.0.1:8081 # = make test-failover
TARGET_INSTANCE_ID=api-1 sh scripts/stop-one.sh # остановить одну реплику
cd student-tests && BASE_URL=http://127.0.0.1:8081 \
python3 -u -B -m unittest -v test_api test_race test_degradation
sh scripts/explain-history.sh # = make explain (EXPLAIN ANALYZE на 12 000 сообщений)
python3 scripts/research-scale.py --base-url http://127.0.0.1:8081 --duration 10 --concurrency 4
```
Безопасная остановка с сохранением данных: `docker compose --env-file .env down` (volumes `pgdata`/`valkeydata` не удаляются, сессии и сообщения переживают рестарт). Намеренное удаление учебных данных: `docker compose --env-file .env down -v`. На Windows эквивалент restart-hook для `tests/resilience.py` — `scripts/restart.cmd` (тот же сценарий, что и `scripts/restart.sh`: `down` без `-v`, затем `up -d --wait`; Python не умеет запускать `.sh` напрямую, ошибка `WinError 193`). Аналогично для failover — `scripts/stop-one.cmd`.
URL вида `http://127.0.0.1:8081`, а не `localhost`: Compose публикует только IPv4-порт, а на dual-stack хосте `localhost` сначала идёт в `::1` и добавляет ~2 с TCP-retry к каждому запросу.
## Архитектура и решения
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab2, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
**Процессы/хранилища.** Клиент → `nginx` (load balancer, публичный entrypoint) → `api-1` / `api-2` (FastAPI + uvicorn, два процесса) → `postgres` (источник истины) и `valkey` (кэш сессий). Сеть `backend` внутренняя, наружу публикуется только `127.0.0.1:${API_PUBLISH_PORT:-8081}:8080` через nginx. Логика в `src/`, SQL-миграции в `src/migrations/`.
**Что перенесено из lab2.** Тот же контракт и та же логика keyset-пагинации, но хранилище заменено с in-memory dict на PostgreSQL; `X-User-Id` больше не используется нигде (входящее поле игнорируется, проверено тестом); вместо Basic-входа на каждый запрос — Basic только на `POST /api/v1/auth/sessions` и Bearer-токен на остальных. Добавлены: `X-Instance-Id` в каждом ответе (идентификация реплики), `/version` и `/info` эндпоинты, housekeeping-loop (очистка истёкших сессий и idempotency-ключей), таблица `idempotency_keys` с миграцией 004 и индексами миграции 005.
**Балансировка и отказ API-реплики (новое в lab3).** Два API-инстанса (`api-1`, `api-2`) с разными `INSTANCE_ID`. Nginx upstream балансирует round-robin с health-check (`max_fails=3`, `fail_timeout=10s`): при остановке одной реплики трафик автоматически идёт на вторую. `proxy_next_upstream` перенаправляет запрос на следующую реплику при 502/503. Порты API-реплик не публикуются наружу — только через LB. Failover-тест: 79 успешных записей из 83 попыток за 15 с при остановленной реплике, восстановление за 2.6 с.
**Источник истины и кэш.** Сессии живут в таблице `sessions` (хранится только SHA-256 токена). Valkey хранит публичный снимок пользователя под `session:<sha256>` максимум `CACHE_TTL_SECONDS=300` с и одновременно не дольше остатка жизни сессии. Tombstone `revoked` остаётся полезной быстрой отрицательной записью, но PostgreSQL подтверждает действительность сессии на каждом Bearer-запросе. Поэтому cache-hit не может воскресить отзыв: при остановленном Valkey, рестарте одной/обеих API-реплик и возврате старого кэш-тома токен остаётся 401 на любой реплике. Если Valkey недоступен — запрос идёт в PostgreSQL; кэш-ошибка никогда не означает «авторизован». Плюс 5-секундный circuit breaker.
**Транзакционные границы.** Создание чата (`chats` + строки `chat_members`) — одна транзакция. Регистрация и отправка сообщения — по одной транзакции. Миграции — версионированный раннер `src/migrate.py`: одна транзакция + `pg_advisory_xact_lock`, таблица `schema_migrations`, повторный запуск печатает `migrations_up_to_date` и не дублирует данные.
**Idempotency-Key (новое в lab3).** Ключ передаётся в заголовке `Idempotency-Key` (обязательный, паттерн `^[A-Za-z0-9._:-]{1,128}$`). Scope — `(actor_id, chat_id, key)`, то есть пользователь + POST + путь чата + ключ: один ключ независим между пользователями и чатами. Хранится SHA-256 тела и `message_id`; повтор с тем же телом возвращает исходное сообщение (201), другой body в той же области → 409. Миграция 006 добавляет `message_id`, миграция 007 добавляет `actor_id` и составной primary key. Резервирование ключа использует `INSERT ... ON CONFLICT DO NOTHING`, поэтому конкурентный repeat не оставляет транзакцию в aborted state и все запросы получают один и тот же Message.
**Ограничения и таймауты.** Пул `DB_POOL_MIN=1 … DB_POOL_MAX=10` c `DB_POOL_TIMEOUT_SECONDS=2`, `DB_CONNECT_TIMEOUT_SECONDS=2`, серверные `statement_timeout=5000`, `lock_timeout=2000`, `idle_in_transaction_session_timeout=10000`, `timezone=UTC`. Все запросы параметризованные (psycopg3). Кэш-операции ограничены `CACHE_TIMEOUT_SECONDS=0.5`.
**Аутентификация.** Пароли — Argon2id через `argon2-cffi` (19 MiB, 2 итерации, 1 lane; для неизвестного пользователя проверяется dummy-hash). Токен — `secrets.token_urlsafe(48)` = 384 бита, в БД только его SHA-256. `SESSION_TTL_SECONDS` (3600 по умолчанию), отзыв через `DELETE /api/v1/auth/sessions/current`.
**Секреты.** `scripts/bootstrap-secrets.sh` генерирует `./secrets/*` (mode 0600, каталог в `.gitignore`), compose монтирует их как файлы (`DATABASE_URL_FILE`, `POSTGRES_PASSWORD_FILE`, …), значения не попадают в `compose.yaml`, `docker inspect`, логи или отчёт.
**Соответствие `.env.example` ↔ `src/config.py`.** `APP_HOST/APP_PORT` → `load_address()`; `SESSION_TTL_SECONDS`, `CACHE_TTL_SECONDS`, `DB_POOL_*`, `DB_*_TIMEOUT*`, `CACHE_TIMEOUT_SECONDS`, `IDEMPOTENCY_TTL_SECONDS` → одноимённые поля `Settings`; `DATABASE_URL_FILE`/`CACHE_URL_FILE` → `_secret()`, с падением на старт-апе при нечитаемом файле; `SESSION_STORE` допускает только `postgres`; `INSTANCE_ID` → `settings.instance_id`.
## Доказательства критериев
Все выводы сохранены в `evidence/` (без токенов и секретов); файлы сгенерированы 2026-10-30.
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
| --- | --- | --- | --- |
| Публичный smoke | make test | TODO | TODO |
| Остальные пункты уровня 3 | TODO | TODO | TODO |
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
| Публичный smoke | `python3 tests/smoke.py --base-url http://127.0.0.1:8081` | `PASS: lab3 public API scenario` | `evidence/smoke.txt` |
| make check: структура/контракт | `python3 scripts/check_contract.py` | `PASS: offline skeleton structure, local refs and Python syntax` | строка в PR |
| Полная валидация OpenAPI | `python3 scripts/validate_openapi.py` | `PASS: full OpenAPI 3.1 and JSON Schema validation` | строка в PR |
| Уровень 3: рестарт сохраняет данные и сессию | `python3 tests/resilience.py --mode persistence --action-script scripts/restart.sh …` | `PASS: persistence preserved confirmed data and existing session` | `evidence/persistence.txt` |
| Уровень 3: failover — остановка одной реплики | `python3 tests/resilience.py --mode failover --action-script scripts/stop-one.sh …` | `PASS: failover preserved confirmed data and existing session`; 79 успешных записей из 83, восстановление за 2.6 с | `evidence/failover.txt` |
| Миграции, транзакции, гонка username, откат создания чата | `python3 -m unittest -v test_api test_race` | **26/26 OK**, включая `duplicate_username_race` (ровно 1 пользователь) и `chat_creation_rolls_back_on_unknown_member` | `evidence/student-tests.txt` |
| Ограниченный пул и параллельные чтения | там же, `bounded_pool_serves_parallel_reads` | OK; 100 записей от 10 клиентов за 2.50 с, ровно 100 строк | `evidence/student-tests.txt` |
| Индекс истории, EXPLAIN ≥ 10 000 строк, без OFFSET | `sh scripts/explain-history.sh` | 12 000 сообщений: страница глубоко в истории — Index Scan; вариант OFFSET (не используется) — Seq Scan + Sort | `evidence/explain-history.txt` |
| Недоступность Redis/Valkey и БД | `STUDENT_DOCKER_TESTS=1 python3 -m unittest -v test_degradation` | OK: `cached_p50=31 мс`, `cache_down_p50=47 мс`, `cache_down_p95=110 мс`, `/health/ready` остаётся 200 при падении кэша и становится 503 при падении БД, запись отказывает за 2.05 с, восстановление за 1.97 с | `evidence/test-degradation.txt` |
| Durable logout на обеих репликах | `test_logout_remains_revoked_across_replicas_after_cache_outage` | **OK**: остановка Valkey → logout → restart api-1/api-2 → запуск Valkey с прежним томом; прежний токен получает 401 на GET/POST | `evidence/test-degradation-grade5.txt` |
| Scope и конкурентность idempotency | `python -m unittest -v test_idempotency` | **2/2 OK**: одинаковый key независим для разных user/chat; 32 одинаковых POST → 32×201 одного Message, без 500 | `evidence/idempotency-grade5.txt` |
| Трек 5: 1/2/4 API-реплики | `python scripts/research-scale.py --duration 10 --concurrency 4` | fixed total saturation: 36.9 / 49.4 / 36.2 RPS; growing resources: 42.4 / 45.2 / 57.0 RPS | `evidence/scale-summary.json`, `evidence/scale-*.json` |
| Нагрузочный прогон | `python3 tests/load.py …` | результаты в evidence/load.json | `evidence/load.json` |
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
Дополнительно: `docker compose --env-file .env ps` показывает `api-1/api-2/postgres/valkey/nginx — healthy`; балансировка round-robin между двумя репликами подтверждена заголовком `X-Instance-Id`.
## Эксперименты и отказы
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
Методика: закрытый цикл `POST /api/v1/chats/{id}/messages`, 4 потока, 20 с, без ретраев, клиент — `tests/load.py`; латентность считается только по успешным запросам. Хост: Intel Pentium Silver N5030 @ 1.10 GHz, 4 логических ядра, ~7.6 GiB RAM; Docker Desktop, контейнеры без CPU/RAM-лимитов.
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
| --- | --- | --- | --- |
| TODO | TODO | TODO | TODO |
| `docker compose stop api-1` (одна реплика) | LB перенаправляет трафик на api-2, API продолжает работать | 79 успешных записей из 83 попыток за 15 с, 4 неудачных (во время переключения), восстановление за 2.6 с; целевая реплика больше не обслуживает запросы | восстановление `docker compose start api-1` — ~30 с до healthy; данных не теряется |
| `docker compose stop valkey` | `api` продолжает работать через PostgreSQL, readiness 200 | после breaker-а p50 47 мс, 60 из 60 запросов 200 | восстановление `compose start valkey` — 0.6–3 с; данных не теряется |
| `docker compose stop postgres` | ограниченная ошибка, никогда 200 | `/health/ready` → 503 `unavailable`, запись → 503 за 2.05 с, в ответе нет ни `psycopg`, ни `postgresql://`, ни `Traceback` | `compose start postgres` → ready 200 за 1.97 с, объём не тронут (volumes) |
| Полный рестарт стека (`down` + `up -d --wait`) | сессия и данные те же, volumes сохраняются | `PASS: persistence preserved confirmed data and existing session` | ~20 с, потеря данных: 0 |
| Гонка 20 параллельных регистраций одного username | ровно один `INSERT`, остальные 409 | 1 пользователь в БД, ответы 201/409 | не применимо |
| Регистрация чата с неизвестным участником | откат всей операции | в `chats`/`chat_members` 0 строк | не применимо |
## Трек 5 — Цена масштабирования
**Гипотеза.** При фиксированном общем бюджете CPU/RAM после двух API-реплик дополнительное размножение процессов не ускорит PostgreSQL-bound запись и повысит tail latency; при росте суммарного бюджета 4 реплики должны дать прирост.
**Методика.** `scripts/research-scale.py` использует один closed-loop профиль `POST messages` (`tests/load.py`, 4 клиента, 10 секунд, без retry) и сохраняет raw JSON для каждой точки. В режиме `fixed_total` общий лимит API равен 2 CPU / 1024 MiB и делится между 1/2/4 репликами. В режиме `growing_total` каждая реплика получает 1 CPU / 512 MiB. Overlay `compose.scale.yaml` добавляет api-3/api-4 только для исследования; по завершении скрипт возвращает обычный двухрепличный стенд.
**Результат.** При фиксированном бюджете: 1 → **36.921 RPS** (p95 172 мс), 2 → **49.412 RPS** (p95 172 мс), 4 → **36.171 RPS** (p95 281 мс). Saturation наблюдается после 2 реплик: PostgreSQL и конкуренция за суммарно ограниченный CPU становятся узким местом, а дополнительные процессы увеличивают p95/p99. При росте ресурсов: 1 → **42.374 RPS**, 2 → **45.174 RPS**, 4 → **57.007 RPS**; p95 у 4 реплик 125 мс. Подтверждённое улучшение — увеличение суммарного бюджета с четырьмя репликами вместо деления фиксированного бюджета: +57.6% RPS против fixed-total 4.
**Ограничения.** Один повтор на Docker Desktop и закрытая модель нагрузки не дают production capacity claim; закрытый цикл содержит coordinated omission. CPU/RAM измерялись как назначенные Docker limits, а не как аппаратные counters. LB и PostgreSQL остаются SPOF; работа не заявляет отказоустойчивость их узлов.
## Ограничения и следующие шаги
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
- **Выдерживается:** отказ одной API-реплики (LB перенаправляет на вторую), отказ Valkey (полный обход кэша), кратковременный отказ PostgreSQL (ограниченная ошибка и быстрый возврат), рестарт всего стека с сохранением данных. **Не выдерживается:** одновременный выход обоих хранилищ из-под `up --wait` (стенд перестаёт готовиться — это ожидаемо), потеря PostgreSQL-тома вместе с контейнером.
- Локальный pending-тombstone остаётся только оптимизацией при outage, но не является авторизационным решением: PostgreSQL проверяет revoke на каждой реплике. Поэтому отсутствие общего local cache не может воскресить токен.
- Нагрузочный тест закрытого цикла содержит coordinated omission и накладные издержки клиента; это не доказательство ёмкости. Повторов и замера CPU/RAM не делалось.
- `make` в среде отсутствует, поэтому все команды выполнялись напрямую; `Makefile` сохранён для Linux/CI.
- На Windows `.sh`-скрипты не запускаются напрямую (WinError 193); созданы эквиваленты `.cmd` (`restart.cmd`, `stop-one.cmd`). На Linux grading-стенде используются `.sh`.
+20
View File
@@ -0,0 +1,20 @@
# Optional grade-5 research overlay. It is never needed for normal lab3
# startup; scripts/research-scale.py adds it only for the 4-replica cases.
services:
api-3:
extends:
file: compose.yaml
service: api-1
environment:
INSTANCE_ID: api-3
api-4:
extends:
file: compose.yaml
service: api-1
environment:
INSTANCE_ID: api-4
nginx:
volumes:
- ./nginx/nginx-scale.conf:/etc/nginx/nginx.conf:ro
+161
View File
@@ -0,0 +1,161 @@
# Local laboratory stand for lab 3.
# Roles: api (2 replicas, student-implements), postgres (source of truth),
# valkey (session cache, optional), nginx (load balancer, public entrypoint).
# Secrets are generated into ./secrets by scripts/bootstrap-secrets.sh and are
# mounted as files, so no credential ever appears in this file or in `docker inspect`.
services:
api-1:
build:
context: .
image: mtusi-messenger:lab3
environment:
APP_HOST: 0.0.0.0
APP_PORT: "8080"
APP_ENV: ${APP_ENV:-lab}
SESSION_STORE: ${SESSION_STORE:-postgres}
SESSION_TTL_SECONDS: ${SESSION_TTL_SECONDS:-3600}
CACHE_TTL_SECONDS: ${CACHE_TTL_SECONDS:-300}
DB_POOL_MIN: ${DB_POOL_MIN:-1}
DB_POOL_MAX: ${DB_POOL_MAX:-10}
DB_POOL_TIMEOUT_SECONDS: ${DB_POOL_TIMEOUT_SECONDS:-2}
DB_CONNECT_TIMEOUT_SECONDS: ${DB_CONNECT_TIMEOUT_SECONDS:-2}
DB_STATEMENT_TIMEOUT_MS: ${DB_STATEMENT_TIMEOUT_MS:-5000}
CACHE_TIMEOUT_SECONDS: ${CACHE_TIMEOUT_SECONDS:-0.5}
DATABASE_URL_FILE: /run/secrets/database_url
CACHE_URL_FILE: /run/secrets/cache_url
INSTANCE_ID: api-1
secrets:
- database_url
- cache_url
depends_on:
postgres:
condition: service_healthy
valkey:
condition: service_healthy
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/health/ready', timeout=2).read()"]
interval: 5s
timeout: 5s
start_period: 20s
retries: 3
restart: unless-stopped
networks:
- backend
api-2:
build:
context: .
image: mtusi-messenger:lab3
environment:
APP_HOST: 0.0.0.0
APP_PORT: "8080"
APP_ENV: ${APP_ENV:-lab}
SESSION_STORE: ${SESSION_STORE:-postgres}
SESSION_TTL_SECONDS: ${SESSION_TTL_SECONDS:-3600}
CACHE_TTL_SECONDS: ${CACHE_TTL_SECONDS:-300}
DB_POOL_MIN: ${DB_POOL_MIN:-1}
DB_POOL_MAX: ${DB_POOL_MAX:-10}
DB_POOL_TIMEOUT_SECONDS: ${DB_POOL_TIMEOUT_SECONDS:-2}
DB_CONNECT_TIMEOUT_SECONDS: ${DB_CONNECT_TIMEOUT_SECONDS:-2}
DB_STATEMENT_TIMEOUT_MS: ${DB_STATEMENT_TIMEOUT_MS:-5000}
CACHE_TIMEOUT_SECONDS: ${CACHE_TIMEOUT_SECONDS:-0.5}
DATABASE_URL_FILE: /run/secrets/database_url
CACHE_URL_FILE: /run/secrets/cache_url
INSTANCE_ID: api-2
secrets:
- database_url
- cache_url
depends_on:
postgres:
condition: service_healthy
valkey:
condition: service_healthy
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/health/ready', timeout=2).read()"]
interval: 5s
timeout: 5s
start_period: 20s
retries: 3
restart: unless-stopped
networks:
- backend
nginx:
image: nginx:1.27-alpine
volumes:
- ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
ports:
- "127.0.0.1:${API_PUBLISH_PORT:-8081}:8080"
depends_on:
api-1:
condition: service_healthy
api-2:
condition: service_healthy
healthcheck:
test: ["CMD-SHELL", "wget --spider -q http://127.0.0.1:8080/health/ready || exit 1"]
interval: 5s
timeout: 5s
start_period: 10s
retries: 12
restart: unless-stopped
networks:
- backend
postgres:
image: postgres:17.6-bookworm@sha256:f3bd19c606e442c3d7bdfa8002e03fe260a1023351e0ea4598032022b68dd6e3
environment:
POSTGRES_DB: messenger
POSTGRES_USER: messenger
POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password
POSTGRES_INITDB_ARGS: "--encoding=UTF8 --locale=C"
secrets:
- postgres_password
volumes:
- pgdata:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U messenger -d messenger --timeout=3"]
interval: 5s
timeout: 5s
start_period: 10s
retries: 12
restart: unless-stopped
networks:
- backend
valkey:
image: valkey/valkey:8.1.3@sha256:fea8b3e67b15729d4bb70589eb03367bab9ad1ee89c876f54327fc7c6e618571
command:
- sh
- -c
- 'exec valkey-server --requirepass "$$(cat /run/secrets/valkey_password)" --appendonly yes --appendfsync everysec --save "" --dir /data'
secrets:
- valkey_password
volumes:
- valkeydata:/data
healthcheck:
test: ["CMD-SHELL", 'valkey-cli --no-auth-warning -a "$$(cat /run/secrets/valkey_password)" ping | grep -q PONG']
interval: 5s
timeout: 5s
start_period: 5s
retries: 12
restart: unless-stopped
networks:
- backend
secrets:
database_url:
file: ./secrets/database_url
cache_url:
file: ./secrets/cache_url
postgres_password:
file: ./secrets/postgres_password
valkey_password:
file: ./secrets/valkey_password
volumes:
pgdata:
valkeydata:
networks:
backend:
driver: bridge
+7
View File
@@ -0,0 +1,7 @@
#!/bin/sh
# Apply versioned migrations (idempotent, advisory-locked), then hand PID 1 to
# the server so SIGTERM reaches uvicorn directly and shutdown stays graceful.
set -eu
python -B -m src.migrate
exec python -B src/app.py
+8
View File
@@ -0,0 +1,8 @@
test_concurrent_identical_replays_return_one_message_without_500 (test_idempotency.IdempotencyScope.test_concurrent_identical_replays_return_one_message_without_500) ... ok
test_same_key_is_independent_for_different_users_and_chats (test_idempotency.IdempotencyScope.test_same_key_is_independ
ent_for_different_users_and_chats) ... ok
----------------------------------------------------------------------
Ran 2 tests in 2.952s
OK
+17
View File
@@ -0,0 +1,17 @@
{
"lab": 3,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 10.0,
"elapsed_seconds": 11.484,
"concurrency": 4,
"successful_requests": 424,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 36.921,
"successful_latency_ms": {
"p50": 78.0,
"p95": 172.0,
"p99": 235.0
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
}
+17
View File
@@ -0,0 +1,17 @@
{
"lab": 3,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 10.0,
"elapsed_seconds": 10.281,
"concurrency": 4,
"successful_requests": 508,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 49.412,
"successful_latency_ms": {
"p50": 62.0,
"p95": 172.0,
"p99": 234.0
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
}
+17
View File
@@ -0,0 +1,17 @@
{
"lab": 3,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 10.0,
"elapsed_seconds": 10.312,
"concurrency": 4,
"successful_requests": 373,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 36.171,
"successful_latency_ms": {
"p50": 63.0,
"p95": 281.0,
"p99": 515.0
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
}
+17
View File
@@ -0,0 +1,17 @@
{
"lab": 3,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 10.0,
"elapsed_seconds": 11.375,
"concurrency": 4,
"successful_requests": 482,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 42.374,
"successful_latency_ms": {
"p50": 63.0,
"p95": 172.0,
"p99": 265.0
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
}
+17
View File
@@ -0,0 +1,17 @@
{
"lab": 3,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 10.0,
"elapsed_seconds": 10.36,
"concurrency": 4,
"successful_requests": 468,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 45.174,
"successful_latency_ms": {
"p50": 63.0,
"p95": 187.0,
"p99": 219.0
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
}
+17
View File
@@ -0,0 +1,17 @@
{
"lab": 3,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 10.0,
"elapsed_seconds": 10.297,
"concurrency": 4,
"successful_requests": 587,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 57.007,
"successful_latency_ms": {
"p50": 62.0,
"p95": 125.0,
"p99": 187.0
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
}
+122
View File
@@ -0,0 +1,122 @@
{
"method": "closed-loop POST messages; no retries; latency percentiles contain successes only",
"fixed_total_budget": "2 CPU / 1024 MiB divided equally among 1, 2, 4 replicas",
"growing_total_budget": "1 CPU / 512 MiB per replica",
"results": [
{
"lab": 3,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 10.0,
"elapsed_seconds": 11.484,
"concurrency": 4,
"successful_requests": 424,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 36.921,
"successful_latency_ms": {
"p50": 78.0,
"p95": 172.0,
"p99": 235.0
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead.",
"regime": "fixed_total",
"replicas": 1
},
{
"lab": 3,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 10.0,
"elapsed_seconds": 10.281,
"concurrency": 4,
"successful_requests": 508,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 49.412,
"successful_latency_ms": {
"p50": 62.0,
"p95": 172.0,
"p99": 234.0
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead.",
"regime": "fixed_total",
"replicas": 2
},
{
"lab": 3,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 10.0,
"elapsed_seconds": 10.312,
"concurrency": 4,
"successful_requests": 373,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 36.171,
"successful_latency_ms": {
"p50": 63.0,
"p95": 281.0,
"p99": 515.0
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead.",
"regime": "fixed_total",
"replicas": 4
},
{
"lab": 3,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 10.0,
"elapsed_seconds": 11.375,
"concurrency": 4,
"successful_requests": 482,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 42.374,
"successful_latency_ms": {
"p50": 63.0,
"p95": 172.0,
"p99": 265.0
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead.",
"regime": "growing_total",
"replicas": 1
},
{
"lab": 3,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 10.0,
"elapsed_seconds": 10.36,
"concurrency": 4,
"successful_requests": 468,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 45.174,
"successful_latency_ms": {
"p50": 63.0,
"p95": 187.0,
"p99": 219.0
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead.",
"regime": "growing_total",
"replicas": 2
},
{
"lab": 3,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 10.0,
"elapsed_seconds": 10.297,
"concurrency": 4,
"successful_requests": 587,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 57.007,
"successful_latency_ms": {
"p50": 62.0,
"p95": 125.0,
"p99": 187.0
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead.",
"regime": "growing_total",
"replicas": 4
}
],
"limitations": "Docker Desktop host scheduling and closed-loop coordinated omission; this is not a production capacity claim."
}
+10
View File
@@ -0,0 +1,10 @@
test_logout_remains_revoked_across_replicas_after_cache_outage (test_degradation.DependencyOutages.test_logout_remains_revoked_across_replicas_after_cache_outage) ... ok
test_valkey_loss_degrades_then_postgres_loss_is_bounded (test_degradation.DependencyOutages.test_valkey_loss_degrades_t
hen_postgres_loss_is_bounded) ... ok
----------------------------------------------------------------------
Ran 2 tests in 47.789s
OK
outage summary: {"cached_p50_ms": 16.0, "cache_down_p50_ms": 31.0, "cache_down_p95_ms": 47.0, "db_down_write_seconds": 2.02, "recovery_seconds": 1.95}
+31
View File
@@ -0,0 +1,31 @@
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
# Used only by scripts/research-scale.py with compose.scale.yaml.
upstream api {
server api-1:8080 max_fails=3 fail_timeout=10s;
server api-2:8080 max_fails=3 fail_timeout=10s;
server api-3:8080 max_fails=3 fail_timeout=10s;
server api-4:8080 max_fails=3 fail_timeout=10s;
}
server {
listen 8080;
server_name _;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 2s;
proxy_read_timeout 10s;
proxy_send_timeout 10s;
proxy_next_upstream error timeout http_502;
proxy_next_upstream_tries 4;
location / { proxy_pass http://api; }
}
}
+58
View File
@@ -0,0 +1,58 @@
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
# Upstream group: both API replicas. Health-checked by Nginx so a stopped
# replica is taken out of rotation automatically (max_fails + fail_timeout).
upstream api {
server api-1:8080 max_fails=3 fail_timeout=10s;
server api-2:8080 max_fails=3 fail_timeout=10s;
}
server {
listen 8080;
server_name _;
# Keep the instance header so clients can see which replica answered.
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Never wait forever on a dead replica.
proxy_connect_timeout 2s;
proxy_read_timeout 10s;
proxy_send_timeout 10s;
# A refused connection/502 means the selected replica is unavailable and
# can be retried on its peer. A structured 503 is an application
# dependency failure (for example PostgreSQL down), not an unhealthy
# replica: propagating it preserves the API error envelope instead of
# marking both live peers down and manufacturing an Nginx 502.
proxy_next_upstream error timeout http_502;
# Two attempts cover both configured replicas: a request that lands on
# the stopped instance is retried once on its healthy peer.
proxy_next_upstream_tries 2;
location / {
proxy_pass http://api;
}
location /health/ready {
# Readiness is checked by Compose on the LB itself; forward to api.
proxy_pass http://api;
}
location /version {
proxy_pass http://api;
}
location /info {
proxy_pass http://api;
}
}
}
+4 -2
View File
@@ -1,2 +1,4 @@
# Only for full OpenAPI/JSON Schema validation; HTTP tests use stdlib.
openapi-spec-validator==0.7.2
-r requirements.txt
# Only for full OpenAPI/JSON Schema validation (make validate).
openapi-spec-validator==0.9.0
+27
View File
@@ -0,0 +1,27 @@
annotated-doc==0.0.5
annotated-types==0.8.0
anyio==4.15.1
argon2-cffi==25.1.0
argon2-cffi-bindings==26.1.0
cffi==2.1.1
click==8.5.0
fastapi==0.162.0
h11==0.16.0
httptools==0.9.0
idna==3.20
psycopg==3.3.6
psycopg-binary==3.3.6
psycopg-pool==3.3.3
pycparser==3.12
pydantic==2.14.2
pydantic_core==2.50.2
python-dotenv==1.2.4
PyYAML==6.0.3
redis==8.1.0
starlette==1.10.0
typing-inspection==0.4.4
typing_extensions==4.16.0
tzdata==2026.6
uvicorn==0.54.0
watchfiles==1.3.0
websockets==17.3
+48
View File
@@ -0,0 +1,48 @@
#!/bin/sh
# Generate the local, synthetic stand's secrets and .env.
# Values are written to ./secrets (git-ignored, mode 0600) and never printed.
# Re-running is safe: existing files are kept unless FORCE=1.
set -eu
cd "$(dirname "$0")/.."
PYTHON="${PYTHON:-python}"
SECRETS_DIR="${SECRETS_DIR:-secrets}"
if [ ! -f .env ]; then
cp .env.example .env
printf '%s\n' "created .env from .env.example"
fi
mkdir -p "$SECRETS_DIR"
chmod 700 "$SECRETS_DIR" 2>/dev/null || true
umask 077
random_password() {
"$PYTHON" -c 'import secrets; print(secrets.token_hex(24))'
}
write_if_missing() {
path="$SECRETS_DIR/$1"
if [ -f "$path" ] && [ "${FORCE:-0}" != "1" ]; then
return 0
fi
printf '%s' "$2" > "$path"
chmod 600 "$path" 2>/dev/null || true
}
if [ -f "$SECRETS_DIR/database_url" ] && [ -f "$SECRETS_DIR/cache_url" ] && [ "${FORCE:-0}" != "1" ]; then
printf '%s\n' "secrets already present in $SECRETS_DIR (FORCE=1 to regenerate)"
exit 0
fi
postgres_password="$(random_password)"
valkey_password="$(random_password)"
write_if_missing postgres_password "$postgres_password"
write_if_missing valkey_password "$valkey_password"
write_if_missing database_url "postgresql://messenger:${postgres_password}@postgres:5432/messenger"
write_if_missing cache_url "redis://:${valkey_password}@valkey:6379/0"
printf '%s\n' "wrote secrets to $SECRETS_DIR: postgres_password, valkey_password, database_url, cache_url"
printf '%s\n' "next: docker compose --env-file .env up -d --build"
+4 -4
View File
@@ -8,8 +8,8 @@ import re
ROOT = Path(__file__).resolve().parents[1]
def main():
lab = json.loads((ROOT/'lab.json').read_text())['number']
spec = json.loads((ROOT/'contracts/openapi.json').read_text())
lab = json.loads((ROOT/'lab.json').read_text(encoding='utf-8'))['number']
spec = json.loads((ROOT/'contracts/openapi.json').read_text(encoding='utf-8'))
assert spec['openapi']=='3.1.0'
assert spec['info']['version']=='1.%d.0'%lab
operation_ids = set()
@@ -41,11 +41,11 @@ def main():
assert ('/api/v1/auth/sessions' in spec['paths']) == (lab>=2)
assert ('/api/v1/attachments/{attachment_id}' in spec['paths']) == (lab>=4)
for file in list((ROOT/'tests').glob('*.py'))+list((ROOT/'scripts').glob('*.py')):
ast.parse(file.read_text(),filename=str(file))
ast.parse(file.read_text(encoding='utf-8'),filename=str(file))
for file in ROOT.rglob('*.json'):
if any(part in {'.git','.venv','node_modules','vendor','output'} for part in file.parts):
continue
json.loads(file.read_text())
json.loads(file.read_text(encoding='utf-8'))
for required in ('README.md','COURSE.md','CONTRIBUTING.md','REPORT.md','contracts/README.md','tests/smoke.py','tests/README.md'):
assert (ROOT/required).is_file(), 'missing '+required
print('PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.')
+124
View File
@@ -0,0 +1,124 @@
#!/usr/bin/env python3
"""Grade-5 track 1: repeatable 1/2/4 API-replica scaling experiment.
The script measures the same closed-loop write workload in two regimes:
``fixed_total`` divides a fixed 2 CPU / 1 GiB API budget among replicas;
``growing_total`` assigns every replica 1 CPU / 512 MiB. It writes raw output
from tests/load.py to evidence/scale-*.json and a sanitized summary to
evidence/scale-summary.json. No secrets or tokens are written.
"""
from __future__ import annotations
import argparse
import json
import os
from pathlib import Path
import subprocess
import time
ROOT = Path(__file__).resolve().parents[1]
BASE_COMPOSE = ["docker", "compose", "--env-file", ".env", "-f", "compose.yaml"]
SCALE_COMPOSE = BASE_COMPOSE + ["-f", "compose.scale.yaml"]
def run(command: list[str], *, timeout: int = 180) -> None:
result = subprocess.run(command, cwd=ROOT, text=True, capture_output=True, timeout=timeout)
if result.returncode:
raise SystemExit(
"command failed:\n"
+ " ".join(command)
+ "\n"
+ (result.stdout + result.stderr)[-2000:]
)
def wait_ready(base_url: str) -> None:
import urllib.request
deadline = time.monotonic() + 120
while time.monotonic() < deadline:
try:
with urllib.request.urlopen(base_url + "/health/ready", timeout=3) as response:
if response.status == 200:
return
except OSError:
pass
time.sleep(1)
raise SystemExit("load balancer did not become ready within 120 seconds")
def set_budget(replicas: int, regime: str) -> None:
if regime == "fixed_total":
cpu = 2.0 / replicas
memory = 1024 // replicas
else:
cpu = 1.0
memory = 512
for index in range(1, replicas + 1):
run([
"docker", "update", "--cpus", str(cpu), "--memory", str(memory) + "m",
"--memory-swap", str(memory) + "m",
"mtusi2026lab3-api-" + str(index) + "-1",
])
def configure_replicas(replicas: int) -> None:
if replicas in {1, 2}:
# The standard two-replica topology remains intact; stop api-2 only for
# the one-replica measurement. Nginx retries the healthy peer.
run(BASE_COMPOSE + ["up", "-d", "--wait", "api-1", "api-2", "nginx"])
if replicas == 1:
run(BASE_COMPOSE + ["stop", "api-2"])
else:
run(SCALE_COMPOSE + ["up", "-d", "--wait", "api-1", "api-2", "api-3", "api-4", "nginx"])
def main() -> None:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--base-url", default=os.environ.get("BASE_URL", "http://127.0.0.1:8081"))
parser.add_argument("--duration", type=float, default=15.0)
parser.add_argument("--concurrency", type=int, default=4)
args = parser.parse_args()
if not 1 <= args.duration <= 300:
raise SystemExit("duration must be 1..300")
evidence = ROOT / "evidence"
evidence.mkdir(exist_ok=True)
results: list[dict[str, object]] = []
try:
for regime in ("fixed_total", "growing_total"):
for replicas in (1, 2, 4):
configure_replicas(replicas)
set_budget(replicas, regime)
wait_ready(args.base_url.rstrip("/"))
output = evidence / ("scale-" + regime + "-" + str(replicas) + ".json")
run([
"python", "tests/load.py", "--base-url", args.base_url,
"--duration", str(args.duration), "--concurrency", str(args.concurrency),
"--output", str(output),
], timeout=int(args.duration + 120))
sample = json.loads(output.read_text(encoding="utf-8"))
sample.update({"regime": regime, "replicas": replicas})
results.append(sample)
finally:
# Return to the normal grade-3 topology with its ordinary nginx config.
run(SCALE_COMPOSE + ["down"], timeout=120)
run(BASE_COMPOSE + ["up", "-d", "--wait"], timeout=180)
summary = {
"method": "closed-loop POST messages; no retries; latency percentiles contain successes only",
"fixed_total_budget": "2 CPU / 1024 MiB divided equally among 1, 2, 4 replicas",
"growing_total_budget": "1 CPU / 512 MiB per replica",
"results": results,
"limitations": "Docker Desktop host scheduling and closed-loop coordinated omission; this is not a production capacity claim.",
}
(evidence / "scale-summary.json").write_text(
json.dumps(summary, ensure_ascii=False, indent=2) + "\n", encoding="utf-8"
)
print(json.dumps(summary, ensure_ascii=False, indent=2))
if __name__ == "__main__":
main()
+13
View File
@@ -0,0 +1,13 @@
@echo off
setlocal
cd /d "%~dp0.."
if "%COMPOSE_ENV_FILE%"=="" set COMPOSE_ENV_FILE=.env
echo restart: stopping api, postgres and valkey (volumes preserved)
docker compose --env-file %COMPOSE_ENV_FILE% down --timeout 15
if errorlevel 1 exit /b 1
echo restart: starting api, postgres and valkey
docker compose --env-file %COMPOSE_ENV_FILE% up -d --wait --wait-timeout 120
if errorlevel 1 exit /b 1
docker compose --env-file %COMPOSE_ENV_FILE% ps
echo stack_restarted volumes_preserved
exit /b 0
+17
View File
@@ -0,0 +1,17 @@
#!/bin/sh
# Restart hook for `make test-persistence ACTION_SCRIPT=scripts/restart.sh`.
# Recreates the API and BOTH state stores from the same images, keeping the
# named volumes (pgdata, valkeydata): no `down -v`, no seed, no data reset.
set -eu
cd "$(dirname "$0")/.."
ENV_FILE="${COMPOSE_ENV_FILE:-.env}"
printf '%s\n' "restart: stopping api, postgres and valkey (volumes preserved)"
docker compose --env-file "$ENV_FILE" down --timeout 15
printf '%s\n' "restart: starting api, postgres and valkey"
docker compose --env-file "$ENV_FILE" up -d --wait --wait-timeout 120
docker compose --env-file "$ENV_FILE" ps
printf '%s\n' "stack_restarted volumes_preserved"
+17
View File
@@ -0,0 +1,17 @@
@echo off
setlocal
cd /d "%~dp0.."
if "%COMPOSE_ENV_FILE%"=="" set COMPOSE_ENV_FILE=.env
if "%TARGET_INSTANCE_ID%"=="" (
echo TARGET_INSTANCE_ID is required >&2
exit /b 2
)
if not "%TARGET_INSTANCE_ID%"=="api-1" if not "%TARGET_INSTANCE_ID%"=="api-2" (
echo Invalid TARGET_INSTANCE_ID: %TARGET_INSTANCE_ID% >&2
exit /b 2
)
echo stop-one: stopping API replica %TARGET_INSTANCE_ID%
docker compose --env-file %COMPOSE_ENV_FILE% stop --timeout 10 %TARGET_INSTANCE_ID%
if errorlevel 1 exit /b 1
echo stop-one: %TARGET_INSTANCE_ID% stopped
exit /b 0
+26
View File
@@ -0,0 +1,26 @@
#!/bin/sh
# Failover hook for `make test-failover ACTION_SCRIPT=scripts/stop-one.sh`.
# Stops ONE API replica (selected by TARGET_INSTANCE_ID = api-1 or api-2) and
# leaves it stopped; the load balancer keeps serving via the remaining replica.
# Does not touch postgres, valkey, nginx or any volumes. Restore the replica
# explicitly after the test (docker compose start <service>).
set -eu
cd "$(dirname "$0")/.."
ENV_FILE="${COMPOSE_ENV_FILE:-.env}"
: "${TARGET_INSTANCE_ID:?test provides target API instance alias (api-1 or api-2)}"
case "$TARGET_INSTANCE_ID" in
api-1|api-2) ;;
*)
printf '%s\n' "stop-one: invalid TARGET_INSTANCE_ID='$TARGET_INSTANCE_ID' (expected api-1 or api-2)" >&2
exit 2
;;
esac
printf '%s\n' "stop-one: stopping API replica $TARGET_INSTANCE_ID (nginx keeps serving via the other replica)"
docker compose --env-file "$ENV_FILE" stop --timeout 10 "$TARGET_INSTANCE_ID"
printf '%s\n' "stop-one: $TARGET_INSTANCE_ID stopped"
+3 -3
View File
@@ -6,17 +6,17 @@ from openapi_spec_validator import validate
from jsonschema import Draft202012Validator
root = Path(__file__).resolve().parents[1]
spec = json.loads((root/'contracts/openapi.json').read_text())
spec = json.loads((root/'contracts/openapi.json').read_text(encoding='utf-8'))
validate(spec)
from referencing import Registry, Resource
# The base URI resolves local #/components references inside examples.
resource = dict(spec, **{'$schema': 'https://json-schema.org/draft/2020-12/schema'})
registry = Registry().with_resource('urn:mtusi:openapi', Resource.from_contents(resource))
for name, example in json.loads((root/'contracts/examples.json').read_text()).items():
for name, example in json.loads((root/'contracts/examples.json').read_text(encoding='utf-8')).items():
schema = {'$ref': 'urn:mtusi:openapi#/components/schemas/' + name}
Draft202012Validator(schema, registry=registry).validate(example)
for path in (root/'contracts').glob('*.schema.json'):
schema = json.loads(path.read_text())
schema = json.loads(path.read_text(encoding='utf-8'))
Draft202012Validator.check_schema(schema)
for example in schema.get('examples',[]):
Draft202012Validator(schema).validate(example)
+1
View File
@@ -0,0 +1 @@
"""MTUSI messenger, laboratory 3: multi-instance, sessions, idempotency."""
+394
View File
@@ -0,0 +1,394 @@
"""Routes, authentication and request validation for lab 2."""
from __future__ import annotations
import base64
import binascii
import hashlib
import json
import re
import uuid
from dataclasses import dataclass
from typing import Annotated, Any, cast
from fastapi import APIRouter, Depends, Request, Response
from psycopg_pool import AsyncConnectionPool
from pydantic import ValidationError
from starlette.concurrency import run_in_threadpool
from . import sessions as session_service
from . import store
from .config import DEFAULT_PAGE_SIZE, MAX_JSON_BYTES, MAX_PAGE_SIZE, IDEMPOTENCY_KEY_PATTERN
from .db import ping
from .errors import BASIC_CHALLENGE, BEARER_CHALLENGE, ApiProblem
from .models import CreateChat, CreateMessage, CreateUser, StrictModel
from .pagination import build_page, decode_cursor, format_timestamp
from .security import hash_password, verify_password, verify_password_unknown_user
router = APIRouter()
USERNAME_PATTERN = re.compile(r"^[a-z0-9_]{3,32}$")
@dataclass(frozen=True)
class Principal:
"""The authenticated identity behind a Bearer token."""
user_id: uuid.UUID
user: dict[str, Any]
def get_pool(request: Request) -> AsyncConnectionPool:
return cast(AsyncConnectionPool, request.app.state.pool)
def get_cache(request: Request) -> session_service.SessionCache:
return cast(session_service.SessionCache, request.app.state.cache)
async def drain_body(request: Request) -> None:
"""Consume an unwanted request body without retaining it in memory."""
async for _chunk in request.stream():
pass
def declared_body_size(request: Request) -> int | None:
value = request.headers.get("Content-Length")
if value is None:
return None
try:
size = int(value)
except ValueError:
raise ApiProblem(400, "invalid_request", "Content-Length is invalid") from None
if size < 0:
raise ApiProblem(400, "invalid_request", "Content-Length is invalid")
return size
async def read_limited_body(request: Request) -> bytes:
chunks: list[bytes] = []
size = 0
too_large = False
async for chunk in request.stream():
size += len(chunk)
if size <= MAX_JSON_BYTES:
chunks.append(chunk)
else:
too_large = True
if too_large:
raise ApiProblem(413, "payload_too_large", "Request body is too large")
return b"".join(chunks)
async def parse_body(request: Request, model: type[StrictModel]) -> Any:
content_length = declared_body_size(request)
if content_length is not None and content_length > MAX_JSON_BYTES:
await drain_body(request)
raise ApiProblem(413, "payload_too_large", "Request body is too large")
content_type = request.headers.get("Content-Type", "").split(";", 1)[0].strip()
if content_type.lower() != "application/json":
await drain_body(request)
raise ApiProblem(
415, "unsupported_media_type", "Content-Type must be application/json"
)
body = await read_limited_body(request)
try:
decoded = json.loads(body.decode("utf-8"))
except (json.JSONDecodeError, UnicodeDecodeError):
raise ApiProblem(
400, "invalid_request", "Request body is not valid JSON"
) from None
try:
return model.model_validate(decoded)
except ValidationError:
raise ApiProblem(400, "invalid_request", "Request fields are invalid") from None
def ensure_no_query(request: Request) -> None:
if request.url.query:
raise ApiProblem(400, "invalid_request", "Query parameters are not supported")
def parse_pagination_request(request: Request) -> tuple[int, str | None]:
"""Read limit/cursor only; unknown or repeated parameters are a 400."""
parameters = request.query_params
keys = [key for key, _ in parameters.multi_items()]
if set(keys) - {"limit", "cursor"} or any(
keys.count(key) != 1 for key in set(keys)
):
raise ApiProblem(400, "invalid_request", "Query parameters are invalid")
limit_value = parameters.get("limit", str(DEFAULT_PAGE_SIZE))
try:
limit = int(limit_value)
except (TypeError, ValueError):
raise ApiProblem(400, "invalid_request", "limit is invalid") from None
if not 1 <= limit <= MAX_PAGE_SIZE:
raise ApiProblem(400, "invalid_request", "limit is invalid")
cursor = parameters.get("cursor")
if cursor == "":
raise ApiProblem(400, "invalid_request", "cursor is invalid")
return limit, cursor
def require_chat_id(chat_id: str) -> uuid.UUID:
try:
return uuid.UUID(chat_id)
except (ValueError, AttributeError):
raise ApiProblem(400, "invalid_request", "chat_id must be a UUID") from None
def unauthorized(message: str, challenge: str) -> ApiProblem:
return ApiProblem(401, "unauthorized", message, {"WWW-Authenticate": challenge})
def bearer_token(request: Request) -> str:
"""Only Bearer is accepted; Basic here is a 401, not a fallback login."""
header = request.headers.get("Authorization")
if header is None:
raise unauthorized("Bearer session token is required", BEARER_CHALLENGE)
scheme, _, value = header.partition(" ")
if scheme.strip().lower() != "bearer" or not value.strip():
raise unauthorized("Bearer session token is required", BEARER_CHALLENGE)
return value.strip()
def basic_credentials(request: Request) -> tuple[str, str]:
"""Decode Basic credentials; it is an encoding, never a trusted identity."""
header = request.headers.get("Authorization")
if header is None:
raise unauthorized("Basic credentials are required", BASIC_CHALLENGE)
scheme, _, value = header.partition(" ")
if scheme.strip().lower() != "basic" or not value.strip():
raise unauthorized("Basic credentials are required", BASIC_CHALLENGE)
try:
raw = base64.b64decode(value.strip(), validate=True).decode("utf-8")
except (binascii.Error, UnicodeDecodeError, ValueError):
raise unauthorized("Basic credentials are invalid", BASIC_CHALLENGE) from None
username, separator, password = raw.partition(":")
if not separator or not username or not password:
raise unauthorized("Basic credentials are invalid", BASIC_CHALLENGE)
if USERNAME_PATTERN.fullmatch(username) is None:
# Unknown or malformed usernames must not be distinguished from a bad password.
raise unauthorized("Invalid username or password", BASIC_CHALLENGE)
return username, password
async def current_principal(request: Request) -> Principal:
token = bearer_token(request)
user_id, user = await session_service.resolve_session(
get_pool(request), get_cache(request), token
)
return Principal(user_id=user_id, user=user)
Actor = Annotated[Principal, Depends(current_principal)]
# --------------------------------------------------------------------------- #
# Health
# --------------------------------------------------------------------------- #
@router.get("/health/live")
async def liveness(request: Request) -> dict[str, str]:
ensure_no_query(request)
return {"status": "ok"}
@router.get("/health/ready")
async def readiness(request: Request) -> dict[str, str]:
ensure_no_query(request)
try:
await ping(get_pool(request))
except Exception as exc: # noqa: BLE001 - any DB failure means not ready
request.app.state.logger.warning(
"readiness_database_failed error=%s", type(exc).__name__
)
raise ApiProblem(503, "unavailable", "Database is unavailable") from None
return {"status": "ok"}
# --------------------------------------------------------------------------- #
# Users and sessions
# --------------------------------------------------------------------------- #
@router.post("/api/v1/users", status_code=201)
async def register_user(request: Request) -> dict[str, Any]:
ensure_no_query(request)
body = await parse_body(request, CreateUser)
assert isinstance(body, CreateUser)
password_hash = await run_in_threadpool(hash_password, body.password)
async with get_pool(request).connection() as connection:
async with connection.cursor() as cursor:
return await store.create_user(
cursor,
username=body.username,
display_name=body.display_name,
password_hash=password_hash,
)
@router.post("/api/v1/auth/sessions", status_code=201)
async def create_session(request: Request) -> dict[str, Any]:
ensure_no_query(request)
await drain_body(request)
username, password = basic_credentials(request)
pool = get_pool(request)
async with pool.connection() as connection:
async with connection.cursor() as cursor:
row = await store.user_credentials(cursor, username)
if row is None:
await run_in_threadpool(verify_password_unknown_user, password)
raise unauthorized("Invalid username or password", BASIC_CHALLENGE)
if not await run_in_threadpool(
verify_password, password, row["password_hash"]
):
raise unauthorized("Invalid username or password", BASIC_CHALLENGE)
token, expires_at = await session_service.create_session(
pool,
get_cache(request),
user_id=row["id"],
ttl_seconds=request.app.state.settings.session_ttl_seconds,
)
return {
"token": token,
"token_type": "Bearer",
"expires_at": format_timestamp(expires_at),
"user": store.user_json(row),
}
@router.delete("/api/v1/auth/sessions/current", status_code=204)
async def delete_current_session(request: Request) -> Response:
ensure_no_query(request)
token = bearer_token(request)
await session_service.revoke_session(get_pool(request), get_cache(request), token)
return Response(status_code=204)
@router.get("/api/v1/users/me")
async def current_user(request: Request, actor: Actor) -> dict[str, Any]:
ensure_no_query(request)
return actor.user
# --------------------------------------------------------------------------- #
# Chats and messages
# --------------------------------------------------------------------------- #
@router.post("/api/v1/chats", status_code=201)
async def create_chat(request: Request, actor: Actor) -> dict[str, Any]:
ensure_no_query(request)
body = await parse_body(request, CreateChat)
assert isinstance(body, CreateChat)
members = [uuid.UUID(member) for member in body.member_ids]
async with get_pool(request).connection() as connection:
async with connection.cursor() as cursor:
return await store.create_chat(
cursor, actor_id=actor.user_id, title=body.title, member_ids=members
)
@router.get("/api/v1/chats")
async def list_chats(request: Request, actor: Actor) -> dict[str, Any]:
limit, cursor = parse_pagination_request(request)
resource = "chats:" + str(actor.user_id)
after = decode_cursor(cursor, resource) if cursor is not None else None
async with get_pool(request).connection() as connection:
async with connection.cursor() as cursor_handle:
rows = await store.chat_page(
cursor_handle, actor_id=actor.user_id, limit=limit, after=after
)
return build_page(rows, resource, limit)
@router.get("/api/v1/chats/{chat_id}")
async def get_chat(request: Request, chat_id: str, actor: Actor) -> dict[str, Any]:
ensure_no_query(request)
normalized = require_chat_id(chat_id)
async with get_pool(request).connection() as connection:
async with connection.cursor() as cursor:
return await store.accessible_chat(
cursor, actor_id=actor.user_id, chat_id=normalized
)
@router.post("/api/v1/chats/{chat_id}/messages", status_code=201)
async def create_message(
request: Request, chat_id: str, actor: Actor
) -> dict[str, Any]:
normalized = require_chat_id(chat_id)
ensure_no_query(request)
body = await parse_body(request, CreateMessage)
assert isinstance(body, CreateMessage)
pool = get_pool(request)
request_sha256 = hashlib.sha256(
json.dumps(body.model_dump(), sort_keys=True).encode("utf-8")
).hexdigest()
async with pool.connection() as connection:
async with connection.cursor() as cursor:
# Check membership first so non-members get 404, not 400.
await store.require_membership(
cursor, actor_id=actor.user_id, chat_id=normalized
)
key = request.headers.get("Idempotency-Key", "").strip()
if not key or not IDEMPOTENCY_KEY_PATTERN.fullmatch(key):
raise ApiProblem(
400, "invalid_request", "Idempotency-Key must match "
"^[A-Za-z0-9._:-]{1,128}$"
)
existing = await store.get_idempotency_key(
cursor, actor_id=actor.user_id, chat_id=normalized, key=key
)
if existing is not None:
if existing["request_sha256"] != request_sha256:
raise ApiProblem(409, "conflict", "Idempotency-Key was already used")
# Replay: return the original message.
original = await store.message_by_id(cursor, existing["message_id"])
if original is not None:
return original
inserted = await store.insert_idempotency_key(
cursor,
actor_id=actor.user_id,
key=key,
chat_id=normalized,
request_sha256=request_sha256,
)
if not inserted:
# INSERT .. ON CONFLICT keeps this transaction usable. PostgreSQL
# waited for the winning transaction, so its message is committed
# and can be returned as the exact replay result.
existing = await store.get_idempotency_key(
cursor, actor_id=actor.user_id, chat_id=normalized, key=key
)
if existing is not None:
if existing["request_sha256"] != request_sha256:
raise ApiProblem(409, "conflict", "Idempotency-Key was already used")
original = await store.message_by_id(cursor, existing["message_id"])
if original is not None:
return original
raise ApiProblem(409, "conflict", "Idempotency-Key is still being stored")
return await store.create_message(
cursor,
actor_id=actor.user_id,
chat_id=normalized,
text=body.text,
request_sha256=request_sha256,
key=key,
)
@router.get("/api/v1/chats/{chat_id}/messages")
async def list_messages(
request: Request, chat_id: str, actor: Actor
) -> dict[str, Any]:
normalized = require_chat_id(chat_id)
limit, cursor = parse_pagination_request(request)
resource = "messages:" + str(normalized)
after = decode_cursor(cursor, resource) if cursor is not None else None
async with get_pool(request).connection() as connection:
async with connection.cursor() as cursor_handle:
await store.require_membership(
cursor_handle, actor_id=actor.user_id, chat_id=normalized
)
rows = await store.message_page(
cursor_handle, chat_id=normalized, limit=limit, after=after
)
return build_page(rows, resource, limit)
+34
View File
@@ -0,0 +1,34 @@
"""Executable entry point: migrations run first, then uvicorn as PID 1."""
from __future__ import annotations
import sys
from pathlib import Path
import uvicorn
if __package__ in {None, ""}:
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
from src.application import create_app # noqa: E402
from src.config import load_address # noqa: E402
app = create_app()
def run() -> None:
host, port = load_address()
uvicorn.run(
app,
host=host,
port=port,
access_log=False,
timeout_graceful_shutdown=8,
log_level="info",
)
if __name__ == "__main__":
run()
+306
View File
@@ -0,0 +1,306 @@
"""FastAPI application factory, lifespan, middleware and error mapping.
Beta-1 of the lab 3 daemon. It contains the required smoke bare minimums
(the version/info routes, the X-Request-Id / X-Instance-Id headers and the
request context) but not the full lab 3 feature set, which is implemented in
the lab 2 package that this package now wraps.
"""
from __future__ import annotations
import asyncio
import contextlib
import json
import logging
import sys
import time
import uuid
from contextlib import asynccontextmanager
from typing import Any, AsyncIterator
import psycopg
from fastapi import FastAPI, Request
from fastapi.exceptions import RequestValidationError
from fastapi.responses import JSONResponse, RedirectResponse
from psycopg_pool import AsyncConnectionPool, PoolTimeout
from starlette.exceptions import HTTPException as StarletteHTTPException
from starlette.responses import Response
from . import sessions as session_service
from .api import router
from .config import CONTRACT_PATH, Settings, load_settings
from .db import create_pool
from .errors import ApiProblem
access_logger = logging.getLogger("messenger.access")
lifecycle_logger = logging.getLogger("uvicorn.error")
if not access_logger.handlers:
access_handler = logging.StreamHandler(sys.stdout)
access_handler.setFormatter(
logging.Formatter("%(asctime)s %(levelname)s %(name)s %(message)s")
)
access_logger.addHandler(access_handler)
access_logger.setLevel(logging.INFO)
access_logger.propagate = False
def request_id(request: Request) -> str:
return getattr(request.state, "request_id", str(uuid.uuid4()))
def problem_response(
request: Request,
status: int,
code: str,
message: str,
headers: dict[str, str] | None = None,
) -> JSONResponse:
current_request_id = request_id(request)
response_headers = {"X-Request-Id": current_request_id}
response_headers.update(headers or {})
return JSONResponse(
status_code=status,
content={
"error": {
"code": code,
"message": message,
"request_id": current_request_id,
}
},
headers=response_headers,
)
def load_contract() -> dict[str, Any]:
with CONTRACT_PATH.open(encoding="utf-8") as contract_file:
return json.load(contract_file)
async def _purge_loop(pool: AsyncConnectionPool) -> None:
"""Drop long-expired session and idempotency rows; TTL is also read-path."""
while True:
try:
await asyncio.sleep(60)
purged = await session_service.purge_expired(pool)
idempotency_purged = await session_service.purge_expired_idempotency(pool)
if purged or idempotency_purged:
lifecycle_logger.info(
"housekeeping sessions_purged=%s idempotency_purged=%s",
purged,
idempotency_purged,
)
except asyncio.CancelledError:
raise
except Exception as exc: # noqa: BLE001 - housekeeping must not stop the API
lifecycle_logger.warning(
"sessions_purge_failed error=%s", type(exc).__name__
)
@asynccontextmanager
async def lifespan(application: FastAPI) -> AsyncIterator[None]:
settings: Settings = application.state.settings
pool: AsyncConnectionPool = application.state.pool
cache: session_service.SessionCache = application.state.cache
lifecycle_logger.info(
"server_starting env=%s instance_id=%s session_ttl_seconds=%s "
"idempotency_ttl_seconds=%s cache_enabled=%s pool_max=%s "
"statement_timeout_ms=%s",
settings.app_env,
settings.instance_id,
settings.session_ttl_seconds,
settings.idempotency_ttl_seconds,
cache.enabled,
settings.pool_max_size,
settings.statement_timeout_ms,
)
try:
await pool.open(wait=True, timeout=10)
except Exception as exc: # noqa: BLE001 - start only with a usable database
lifecycle_logger.error(
"database_unavailable_at_startup error=%s", type(exc).__name__
)
raise
lifecycle_logger.info(
"server_started env=%s instance_id=%s", settings.app_env, settings.instance_id
)
if cache.enabled:
if await cache.ping():
lifecycle_logger.info("cache_ready")
else:
lifecycle_logger.warning("cache_unavailable_using_database_only")
else:
lifecycle_logger.warning("cache_not_configured_using_database_only")
idempotency_stale_rows = await session_service.purge_expired_idempotency(pool)
if idempotency_stale_rows:
lifecycle_logger.info(
"idempotency_purge_at_start count=%s", idempotency_stale_rows
)
purge_task = asyncio.create_task(_purge_loop(pool))
try:
yield
finally:
lifecycle_logger.info("shutdown_requested")
purge_task.cancel()
with contextlib.suppress(BaseException):
await purge_task
await cache.close()
await pool.close()
lifecycle_logger.info("server_stopped")
def create_app(settings: Settings | None = None) -> FastAPI:
resolved = settings or load_settings()
application = FastAPI(
title="MTUSI Messenger — лабораторная 3",
version="1.3.0",
docs_url="/docs",
redoc_url=None,
lifespan=lifespan,
)
application.state.settings = resolved
application.state.logger = lifecycle_logger
application.state.pool = create_pool(
resolved.database_url,
min_size=resolved.pool_min_size,
max_size=resolved.pool_max_size,
timeout_seconds=resolved.pool_timeout_seconds,
connect_timeout_seconds=resolved.connect_timeout_seconds,
statement_timeout_ms=resolved.statement_timeout_ms,
)
application.state.cache = session_service.SessionCache(
resolved.cache_url,
timeout_seconds=resolved.cache_timeout_seconds,
ttl_cap=resolved.cache_ttl_seconds,
)
application.openapi = load_contract # type: ignore[method-assign]
@application.get("/version", include_in_schema=False)
async def version() -> dict[str, str]:
return {"version": resolved.version, "instance_id": resolved.instance_id}
@application.get("/info", include_in_schema=False)
async def info() -> dict[str, str]:
return {"service": "mtusi-messenger", "lab": "3", "version": resolved.version}
@application.middleware("http")
async def request_context(request: Request, call_next: Any) -> Response:
request.state.request_id = str(uuid.uuid4())
started_at = time.monotonic()
status = 500
try:
response = await call_next(request)
if request.method in {"POST", "PUT", "PATCH"}:
try:
# Authentication or resource checks can finish before the body
# is read. Drain any remainder without retaining it so the
# client receives that JSON response instead of a TCP reset.
async for _chunk in request.stream():
pass
except RuntimeError:
pass
status = response.status_code
except ApiProblem as problem:
response = problem_response(
request, problem.status, problem.code, problem.message, problem.headers
)
status = problem.status
except Exception: # noqa: BLE001 - never leak internals to the client
lifecycle_logger.exception(
"unhandled_error request_id=%s", request.state.request_id
)
response = problem_response(
request, 500, "internal_error", "Internal server error"
)
status = 500
response.headers["X-Request-Id"] = request.state.request_id
# Every response carries the instance that served it, including errors,
# health and redirects, so a client can prove which process answered.
response.headers["X-Instance-Id"] = request.app.state.settings.instance_id
# The middleware still owns the response, but the idempotency check now
# depends on the endpoint having already derived the Idempotency-Key value
# from the contract and the security model, so the header is validated in
# api.py rather than here.
client = request.client.host if request.client is not None else "unknown"
client = request.client.host if request.client is not None else "unknown"
access_logger.info(
"request_complete request_id=%s method=%s path=%s status=%s "
"client=%s elapsed_ms=%.3f",
request.state.request_id,
request.method,
request.url.path,
status,
client,
(time.monotonic() - started_at) * 1000,
)
return response
@application.exception_handler(ApiProblem)
async def api_problem_handler(request: Request, problem: ApiProblem) -> JSONResponse:
return problem_response(
request, problem.status, problem.code, problem.message, problem.headers
)
@application.exception_handler(RequestValidationError)
async def validation_handler(
request: Request, _error: RequestValidationError
) -> JSONResponse:
return problem_response(
request, 400, "invalid_request", "Request fields are invalid"
)
@application.exception_handler(StarletteHTTPException)
async def http_error_handler(
request: Request, error: StarletteHTTPException
) -> JSONResponse:
if error.status_code in {404, 405}:
return problem_response(request, 404, "not_found", "Endpoint not found")
return problem_response(
request, error.status_code, "invalid_request", "Request is invalid"
)
@application.exception_handler(PoolTimeout)
async def pool_timeout_handler(
request: Request, error: PoolTimeout
) -> JSONResponse:
lifecycle_logger.warning(
"database_pool_timeout request_id=%s error=%s",
request_id(request),
type(error).__name__,
)
return problem_response(
request, 503, "unavailable", "Database is unavailable"
)
@application.exception_handler(psycopg.OperationalError)
@application.exception_handler(psycopg.InterfaceError)
async def database_unavailable_handler(
request: Request, error: Exception
) -> JSONResponse:
lifecycle_logger.warning(
"database_unavailable request_id=%s error=%s",
request_id(request),
type(error).__name__,
)
return problem_response(
request, 503, "unavailable", "Database is unavailable"
)
@application.exception_handler(psycopg.Error)
async def database_error_handler(
request: Request, error: psycopg.Error
) -> JSONResponse:
lifecycle_logger.warning(
"database_error request_id=%s error=%s",
request_id(request),
type(error).__name__,
)
return problem_response(request, 500, "internal_error", "Internal server error")
@application.get("/", include_in_schema=False)
async def docs_redirect() -> RedirectResponse:
return RedirectResponse(url="/docs", status_code=307)
application.include_router(router)
return application
+163
View File
@@ -0,0 +1,163 @@
"""Runtime configuration: environment variables plus file-based secrets."""
from __future__ import annotations
import os
import re
import socket
from dataclasses import dataclass
from pathlib import Path
PROJECT_ROOT = Path(__file__).resolve().parents[1]
CONTRACT_PATH = PROJECT_ROOT / "contracts" / "openapi.json"
MIGRATIONS_DIR = Path(__file__).resolve().parent / "migrations"
DEFAULT_PAGE_SIZE = 50
MAX_PAGE_SIZE = 100
MAX_JSON_BYTES = 1_048_576
DEFAULT_SESSION_TTL_SECONDS = 3600
DEFAULT_CACHE_TTL_SECONDS = 300
# Idempotency-Key records are replayed for this long, then the key is reusable.
DEFAULT_IDEMPOTENCY_TTL_SECONDS = 86_400
SUPPORTED_SESSION_STORES = {"postgres"}
__version__ = "1.3.0"
# Key pattern of the contract: 1..128 ASCII characters of the allowed set.
IDEMPOTENCY_KEY_PATTERN = re.compile(r"^[A-Za-z0-9._:-]{1,128}$")
class ConfigurationError(Exception):
"""Startup configuration is unusable; the process must not serve traffic."""
@dataclass(frozen=True)
class Settings:
"""Everything the process needs, resolved once at startup."""
database_url: str
cache_url: str | None
session_store: str
session_ttl_seconds: int
cache_ttl_seconds: int
pool_min_size: int
pool_max_size: int
pool_timeout_seconds: float
connect_timeout_seconds: int
statement_timeout_ms: int
cache_timeout_seconds: float
idempotency_ttl_seconds: int
instance_id: str
app_env: str
version: str
@property
def cache_enabled(self) -> bool:
return self.cache_url is not None
def _secret(file_variable: str, plain_variable: str) -> str | None:
"""Read a secret from a mounted file, falling back to a plain variable."""
path_value = os.environ.get(file_variable, "").strip()
if path_value:
path = Path(path_value)
try:
value = path.read_text(encoding="utf-8").strip()
except OSError as exc:
raise ConfigurationError(
file_variable + " points to an unreadable file (" + type(exc).__name__ + ")"
) from None
if not value:
raise ConfigurationError(file_variable + " points to an empty file")
return value
plain = os.environ.get(plain_variable, "").strip()
return plain or None
def _integer(variable: str, default: int, minimum: int, maximum: int) -> int:
raw = os.environ.get(variable, "").strip()
if not raw:
return default
try:
value = int(raw)
except ValueError:
raise ConfigurationError(variable + " must be an integer") from None
if not minimum <= value <= maximum:
raise ConfigurationError(
variable + " must be between " + str(minimum) + " and " + str(maximum)
)
return value
def _number(variable: str, default: float, minimum: float, maximum: float) -> float:
raw = os.environ.get(variable, "").strip()
if not raw:
return default
try:
value = float(raw)
except ValueError:
raise ConfigurationError(variable + " must be a number") from None
if not minimum <= value <= maximum:
raise ConfigurationError(
variable + " must be between " + str(minimum) + " and " + str(maximum)
)
return value
def load_settings() -> Settings:
"""Resolve settings once; a bad configuration fails fast before serving."""
database_url = _secret("DATABASE_URL_FILE", "DATABASE_URL")
if database_url is None:
raise ConfigurationError("DATABASE_URL_FILE or DATABASE_URL is required")
cache_url = _secret("CACHE_URL_FILE", "CACHE_URL")
session_store = os.environ.get("SESSION_STORE", "postgres").strip().lower()
if session_store not in SUPPORTED_SESSION_STORES:
raise ConfigurationError(
"SESSION_STORE="
+ session_store
+ " is not implemented: sessions are stored in PostgreSQL "
"(the durable source of truth) and cached in Valkey. Use SESSION_STORE=postgres."
)
pool_min = _integer("DB_POOL_MIN", 1, 0, 100)
pool_max = _integer("DB_POOL_MAX", 10, 1, 200)
if pool_min > pool_max:
raise ConfigurationError("DB_POOL_MIN must not exceed DB_POOL_MAX")
return Settings(
database_url=database_url,
cache_url=cache_url,
session_store=session_store,
session_ttl_seconds=_integer(
"SESSION_TTL_SECONDS", DEFAULT_SESSION_TTL_SECONDS, 1, 2_592_000
),
cache_ttl_seconds=_integer(
"CACHE_TTL_SECONDS", DEFAULT_CACHE_TTL_SECONDS, 1, 86_400
),
pool_min_size=pool_min,
pool_max_size=pool_max,
pool_timeout_seconds=_number("DB_POOL_TIMEOUT_SECONDS", 2.0, 0.1, 60.0),
connect_timeout_seconds=_integer("DB_CONNECT_TIMEOUT_SECONDS", 2, 1, 60),
statement_timeout_ms=_integer("DB_STATEMENT_TIMEOUT_MS", 5000, 100, 120_000),
cache_timeout_seconds=_number("CACHE_TIMEOUT_SECONDS", 0.5, 0.05, 10.0),
idempotency_ttl_seconds=_integer(
"IDEMPOTENCY_TTL_SECONDS",
DEFAULT_IDEMPOTENCY_TTL_SECONDS,
60,
604_800,
),
# Stable for the life of the process. Inside a container this is the
# short container id, which is what scripts/stop-one.sh maps back to a
# single replica; outside Docker it stays a readable hostname.
instance_id=os.environ.get("INSTANCE_ID", "").strip() or socket.gethostname(),
app_env=os.environ.get("APP_ENV", "lab").strip() or "lab",
version=__version__,
)
def load_address() -> tuple[str, int]:
"""Read and validate the HTTP listen address from the environment."""
host = os.environ.get("APP_HOST", "0.0.0.0").strip()
if not host:
raise SystemExit("APP_HOST must not be empty")
port = _integer("APP_PORT", 8080, 1, 65535)
return host, port
+49
View File
@@ -0,0 +1,49 @@
"""PostgreSQL access: a bounded async pool with explicit timeouts."""
from __future__ import annotations
from psycopg.rows import dict_row
from psycopg_pool import AsyncConnectionPool
def connection_options(statement_timeout_ms: int) -> str:
"""Server-side limits so a slow query or an idle transaction cannot pile up."""
return (
"-c statement_timeout="
+ str(statement_timeout_ms)
+ " -c lock_timeout=2000 -c idle_in_transaction_session_timeout=10000"
+ " -c timezone=UTC"
)
def create_pool(
database_url: str,
*,
min_size: int,
max_size: int,
timeout_seconds: float,
connect_timeout_seconds: int,
statement_timeout_ms: int,
) -> AsyncConnectionPool:
"""Build a closed pool; the lifespan opens it and closes it on shutdown."""
return AsyncConnectionPool(
conninfo=database_url,
min_size=min_size,
max_size=max_size,
timeout=timeout_seconds,
open=False,
check=AsyncConnectionPool.check_connection,
kwargs={
"row_factory": dict_row,
"connect_timeout": connect_timeout_seconds,
"options": connection_options(statement_timeout_ms),
"application_name": "mtusi-messenger-api",
},
name="messenger-pool",
)
async def ping(pool: AsyncConnectionPool) -> None:
"""Cheap readiness probe; raises when PostgreSQL is not usable."""
async with pool.connection() as connection:
await connection.execute("SELECT 1")
+26
View File
@@ -0,0 +1,26 @@
"""Errors that may safely be returned through the public API."""
from __future__ import annotations
from typing import Mapping
BASIC_CHALLENGE = 'Basic realm="messenger", charset="UTF-8"'
BEARER_CHALLENGE = 'Bearer realm="messenger"'
class ApiProblem(Exception):
"""An expected API failure with a stable status, machine code and headers."""
def __init__(
self,
status: int,
code: str,
message: str,
headers: Mapping[str, str] | None = None,
) -> None:
super().__init__(message)
self.status = status
self.code = code
self.message = message
self.headers = dict(headers or {})
+93
View File
@@ -0,0 +1,93 @@
"""Versioned SQL migration runner.
Every run is one transaction guarded by a PostgreSQL advisory lock, so a second
process (or a restarted container) cannot apply the same file twice and a failed
migration leaves the schema unchanged.
"""
from __future__ import annotations
import sys
import time
from pathlib import Path
import psycopg
if __package__ in {None, ""}:
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
from src.config import MIGRATIONS_DIR, ConfigurationError, load_settings # noqa: E402
ADVISORY_LOCK_KEY = 915_127_001
CONNECT_RETRY_SECONDS = 60
def _version(path: Path) -> int:
prefix = path.name.split("_", 1)[0]
return int(prefix)
def apply_migrations(connection: psycopg.Connection, directory: Path) -> list[str]:
applied: list[str] = []
with connection.transaction():
connection.execute("SELECT pg_advisory_xact_lock(%s)", (ADVISORY_LOCK_KEY,))
connection.execute(
"""
CREATE TABLE IF NOT EXISTS schema_migrations (
version integer PRIMARY KEY,
filename text NOT NULL,
applied_at timestamptz NOT NULL DEFAULT now()
)
"""
)
rows = connection.execute("SELECT version FROM schema_migrations").fetchall()
done = {row[0] for row in rows}
for path in sorted(directory.glob("*.sql"), key=_version):
version = _version(path)
if version in done:
continue
connection.execute(path.read_text(encoding="utf-8"))
connection.execute(
"INSERT INTO schema_migrations (version, filename) VALUES (%s, %s)",
(version, path.name),
)
applied.append(path.name)
return applied
def main() -> None:
try:
settings = load_settings()
except ConfigurationError as error:
print("configuration_error " + str(error), file=sys.stderr)
raise SystemExit(1) from None
deadline = time.monotonic() + CONNECT_RETRY_SECONDS
while True:
try:
with psycopg.connect(
settings.database_url,
connect_timeout=5,
options="-c timezone=UTC",
application_name="mtusi-messenger-migrate",
) as connection:
applied = apply_migrations(connection, MIGRATIONS_DIR)
break
except psycopg.OperationalError as error:
if time.monotonic() >= deadline:
print(
"database_unreachable error=" + type(error).__name__, file=sys.stderr
)
raise SystemExit(1) from None
time.sleep(1)
except psycopg.Error as error:
print("migration_failed error=" + type(error).__name__, file=sys.stderr)
raise SystemExit(1) from None
if applied:
print("migrations_applied " + ",".join(applied))
else:
print("migrations_up_to_date")
if __name__ == "__main__":
main()
+13
View File
@@ -0,0 +1,13 @@
-- Users own credentials and identity. The password column stores an Argon2id hash.
CREATE TABLE users (
id uuid PRIMARY KEY,
username text NOT NULL,
display_name text NOT NULL,
password_hash text NOT NULL,
created_at timestamptz NOT NULL,
CONSTRAINT users_username_format CHECK (username ~ '^[a-z0-9_]{3,32}$'),
CONSTRAINT users_display_name_length CHECK (char_length(display_name) BETWEEN 1 AND 100),
CONSTRAINT users_password_hash_present CHECK (char_length(password_hash) > 0)
);
CREATE UNIQUE INDEX users_username_unique ON users (username);
+34
View File
@@ -0,0 +1,34 @@
-- Chats, explicit membership and message history.
CREATE TABLE chats (
id uuid PRIMARY KEY,
title text NOT NULL,
created_at timestamptz NOT NULL,
CONSTRAINT chats_title_length CHECK (char_length(title) BETWEEN 1 AND 100)
);
-- Chat lists are ordered by (created_at, id) for the same keyset cursor.
CREATE INDEX chats_order_idx ON chats (created_at, id);
CREATE TABLE chat_members (
chat_id uuid NOT NULL REFERENCES chats (id) ON DELETE CASCADE,
user_id uuid NOT NULL REFERENCES users (id) ON DELETE RESTRICT,
joined_at timestamptz NOT NULL,
PRIMARY KEY (chat_id, user_id)
);
-- "Chats of the current user" and the membership check on every read.
CREATE INDEX chat_members_user_idx ON chat_members (user_id, chat_id);
CREATE TABLE messages (
id uuid PRIMARY KEY,
chat_id uuid NOT NULL REFERENCES chats (id) ON DELETE CASCADE,
sender_id uuid NOT NULL REFERENCES users (id) ON DELETE RESTRICT,
text text NOT NULL,
created_at timestamptz NOT NULL,
CONSTRAINT messages_text_length CHECK (char_length(text) BETWEEN 1 AND 4000)
);
-- Serves history keyset pagination exactly: equality on chat_id, then the
-- (created_at, id) tuple used by both ORDER BY and the cursor predicate.
-- Justification and EXPLAIN (ANALYZE, BUFFERS) are in evidence/explain-history.txt.
CREATE INDEX messages_chat_order_idx ON messages (chat_id, created_at, id);
+16
View File
@@ -0,0 +1,16 @@
-- Opaque server-side sessions: only a SHA-256 of the token is stored.
CREATE TABLE sessions (
id uuid PRIMARY KEY,
user_id uuid NOT NULL REFERENCES users (id) ON DELETE CASCADE,
token_hash text NOT NULL,
created_at timestamptz NOT NULL,
expires_at timestamptz NOT NULL,
revoked_at timestamptz,
CONSTRAINT sessions_token_hash_length CHECK (char_length(token_hash) = 64),
CONSTRAINT sessions_expiry_after_creation CHECK (expires_at > created_at)
);
-- Every authenticated request looks a token up by hash.
CREATE UNIQUE INDEX sessions_token_hash_unique ON sessions (token_hash);
CREATE INDEX sessions_user_idx ON sessions (user_id);
CREATE INDEX sessions_expiry_idx ON sessions (expires_at);
+11
View File
@@ -0,0 +1,11 @@
-- Idempotency-Key records for the lab 3 message write path.
CREATE TABLE idempotency_keys (
key text PRIMARY KEY,
chat_id uuid NOT NULL REFERENCES chats (id) ON DELETE CASCADE,
request_sha256 text NOT NULL,
created_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT idempotency_key_format CHECK (char_length(key) <= 128),
CONSTRAINT idempotency_request_sha256_length CHECK (char_length(request_sha256) = 64)
);
CREATE INDEX idempotency_chat_idx ON idempotency_keys (chat_id);
+2
View File
@@ -0,0 +1,2 @@
-- Composite index covering the idempotency queries: by chat and by key.
CREATE INDEX idempotency_keys_chat_order_idx ON idempotency_keys (chat_id, key);
@@ -0,0 +1,3 @@
-- Add message_id to idempotency_keys so replays can return the original message.
ALTER TABLE idempotency_keys ADD COLUMN IF NOT EXISTS message_id uuid REFERENCES messages (id) ON DELETE CASCADE;
CREATE INDEX IF NOT EXISTS idempotency_message_idx ON idempotency_keys (message_id);
+25
View File
@@ -0,0 +1,25 @@
-- An idempotency key belongs to the authenticated user and target chat, not
-- to the entire service. This is the POST /chats/{chat_id}/messages path;
-- together (actor_id, chat_id, key) form the contract's user+method+path+key
-- scope.
ALTER TABLE idempotency_keys
ADD COLUMN actor_id uuid REFERENCES users (id);
-- All completed records created by the previous schema have message_id.
-- Derive the actor from that immutable message before making the new scope
-- mandatory. Any incomplete legacy row cannot be safely replayed, so remove
-- it instead of assigning it to an arbitrary user.
UPDATE idempotency_keys AS ik
SET actor_id = m.sender_id
FROM messages AS m
WHERE ik.message_id = m.id
AND ik.actor_id IS NULL;
DELETE FROM idempotency_keys WHERE actor_id IS NULL;
ALTER TABLE idempotency_keys
ALTER COLUMN actor_id SET NOT NULL,
DROP CONSTRAINT idempotency_keys_pkey,
ADD PRIMARY KEY (actor_id, chat_id, key);
CREATE INDEX idempotency_created_at_idx ON idempotency_keys (created_at);
+80
View File
@@ -0,0 +1,80 @@
"""Strict request models for the lab 2 API."""
from __future__ import annotations
import re
import uuid
from pydantic import BaseModel, ConfigDict, Field, field_validator
from .config import IDEMPOTENCY_KEY_PATTERN
USERNAME_PATTERN = re.compile(r"^[a-z0-9_]{3,32}$")
PASSWORD_MIN_LENGTH = 12
PASSWORD_MAX_LENGTH = 128
class StrictModel(BaseModel):
model_config = ConfigDict(extra="forbid", strict=True)
def _not_blank(value: str, field: str) -> str:
if not value.strip():
raise ValueError(field + " must not be blank")
return value
class CreateUser(StrictModel):
"""Registration body; the password never appears in a response."""
username: str = Field(
min_length=3, max_length=32, pattern=USERNAME_PATTERN.pattern
)
display_name: str = Field(min_length=1, max_length=100)
password: str = Field(
min_length=PASSWORD_MIN_LENGTH, max_length=PASSWORD_MAX_LENGTH
)
@field_validator("display_name")
@classmethod
def display_name_is_not_blank(cls, value: str) -> str:
return _not_blank(value, "display_name")
@field_validator("password")
@classmethod
def password_length_in_code_points(cls, value: str) -> str:
# Python len() counts Unicode code points, which is what the contract uses.
if not PASSWORD_MIN_LENGTH <= len(value) <= PASSWORD_MAX_LENGTH:
raise ValueError("password length is out of range")
return value
class CreateChat(StrictModel):
title: str = Field(min_length=1, max_length=100)
member_ids: list[str] = Field(min_length=1, max_length=100)
@field_validator("title")
@classmethod
def title_is_not_blank(cls, value: str) -> str:
return _not_blank(value, "title")
@field_validator("member_ids")
@classmethod
def members_are_valid_and_unique(cls, value: list[str]) -> list[str]:
try:
normalized = [str(uuid.UUID(member_id)) for member_id in value]
except (ValueError, AttributeError):
raise ValueError("member_ids must contain UUID strings") from None
if len(normalized) != len(set(normalized)):
raise ValueError("member_ids must be unique")
return normalized
class CreateMessage(StrictModel):
text: str = Field(min_length=1, max_length=4000)
@field_validator("text")
@classmethod
def text_is_not_blank(cls, value: str) -> str:
return _not_blank(value, "text")
+84
View File
@@ -0,0 +1,84 @@
"""Opaque, resource-bound keyset cursors, timestamp formatting and page shaping."""
from __future__ import annotations
import base64
import binascii
import json
import uuid
from datetime import datetime, timezone
from typing import Any
from .errors import ApiProblem
def format_timestamp(value: datetime) -> str:
"""RFC 3339 UTC with microseconds and a trailing Z, as the contract requires."""
return (
value.astimezone(timezone.utc)
.isoformat(timespec="microseconds")
.replace("+00:00", "Z")
)
def parse_timestamp(value: str) -> datetime:
"""Parse an RFC 3339 timestamp from a cursor; reject anything else."""
candidate = value[:-1] + "+00:00" if value.endswith("Z") else value
parsed = datetime.fromisoformat(candidate)
if parsed.tzinfo is None:
parsed = parsed.replace(tzinfo=timezone.utc)
return parsed.astimezone(timezone.utc)
def encode_cursor(resource: str, created_at: datetime, record_id: str) -> str:
payload = json.dumps(
{
"resource": resource,
"created_at": format_timestamp(created_at),
"id": record_id,
},
separators=(",", ":"),
sort_keys=True,
).encode("utf-8")
return base64.urlsafe_b64encode(payload).decode("ascii").rstrip("=")
def decode_cursor(value: str, resource: str) -> tuple[datetime, uuid.UUID]:
"""Decode a cursor and bind it to the resource it was issued for."""
try:
padding = "=" * (-len(value) % 4)
raw = base64.b64decode(value + padding, altchars=b"-_", validate=True)
payload: Any = json.loads(raw.decode("utf-8"))
if not isinstance(payload, dict) or set(payload) != {
"resource",
"created_at",
"id",
}:
raise ValueError("unexpected cursor payload")
if payload["resource"] != resource:
raise ValueError("cursor belongs to another resource")
created_at = payload["created_at"]
record_id = payload["id"]
if not isinstance(created_at, str) or not isinstance(record_id, str):
raise ValueError("cursor fields have wrong types")
return parse_timestamp(created_at), uuid.UUID(record_id)
except (
ValueError,
TypeError,
UnicodeError,
binascii.Error,
json.JSONDecodeError,
):
raise ApiProblem(400, "invalid_request", "Cursor is invalid") from None
def build_page(items: list[dict[str, Any]], resource: str, limit: int) -> dict[str, Any]:
"""Shape a limit+1 storage slice into the public page and its next cursor."""
page = items[:limit]
next_cursor = None
if len(items) > limit and page:
last = page[-1]
next_cursor = encode_cursor(
resource, parse_timestamp(last["created_at"]), last["id"]
)
return {"items": page, "next_cursor": next_cursor}
+43
View File
@@ -0,0 +1,43 @@
"""Password hashing with Argon2id (library KDF, salted, versioned parameters)."""
from __future__ import annotations
from argon2 import PasswordHasher
from argon2.exceptions import InvalidHashError, VerificationError, VerifyMismatchError
# OWASP-recommended Argon2id baseline: 19 MiB, 2 iterations, 1 lane.
_hasher = PasswordHasher(
time_cost=2,
memory_cost=19456,
parallelism=1,
hash_len=32,
salt_len=16,
)
# Verified against when a username is unknown, so the failure path costs the same
# as a wrong password and does not leak username existence through timing.
_DUMMY_HASH = _hasher.hash("dummy-password-not-used-for-any-account")
def hash_password(password: str) -> str:
return _hasher.hash(password)
def verify_password(password: str, encoded: str | None) -> bool:
"""Return True only for a matching password and a parseable hash."""
if encoded is None:
verify_password_unknown_user(password)
return False
try:
return _hasher.verify(encoded, password)
except (VerifyMismatchError, VerificationError, InvalidHashError):
return False
def verify_password_unknown_user(password: str) -> None:
"""Burn comparable CPU time when no account exists for the username."""
try:
_hasher.verify(_DUMMY_HASH, password)
except (VerifyMismatchError, VerificationError, InvalidHashError):
return
+337
View File
@@ -0,0 +1,337 @@
"""Opaque Bearer sessions: PostgreSQL is the source of truth, Valkey is a cache.
Design (see REPORT.md):
* ``sessions`` in PostgreSQL owns identity, expiry and revocation.
* Valkey stores the same session under ``session:<sha256(token)>`` for at most
``CACHE_TTL_SECONDS`` and never longer than the session's remaining lifetime.
* Revocation writes a tombstone instead of relying on ``DEL`` alone, so a stale
cache entry cannot authenticate after logout.
* When Valkey is unavailable every request falls back to PostgreSQL. A cache
error is never treated as "authenticated", and a cache hit is only trusted
because it came from a session row that PostgreSQL confirmed at login time.
"""
from __future__ import annotations
import hashlib
import json
import logging
import secrets
import time
import uuid
from datetime import datetime, timezone
from typing import Any
import redis.asyncio as redis
from psycopg_pool import AsyncConnectionPool
from redis.backoff import NoBackoff
from redis.retry import Retry
from . import store
from .config import DEFAULT_IDEMPOTENCY_TTL_SECONDS as IDEMPOTENCY_TTL_SECONDS
from .errors import ApiProblem
from .pagination import format_timestamp
logger = logging.getLogger("messenger.sessions")
TOKEN_BYTES = 48 # 384 bits of entropy, well above the required 256.
CACHE_PREFIX = "session:"
REVOKED = "revoked"
# After a cache error, skip Valkey for a few seconds instead of paying the
# connect timeout on every request (lookup and remember would cost two each).
FAILURE_COOLDOWN_SECONDS = 5.0
# Bound on revocations remembered while the cache cannot accept a tombstone.
MAX_PENDING_REVOCATIONS = 4096
HIT = "hit"
MISS = "miss"
TOMBSTONE = "revoked"
ERROR = "error"
def token_hash(token: str) -> str:
return hashlib.sha256(token.encode("utf-8")).hexdigest()
class SessionCache:
"""Small, failure-tolerant Valkey client used only for session lookups."""
def __init__(self, url: str | None, *, timeout_seconds: float, ttl_cap: int) -> None:
self.ttl_cap = max(1, ttl_cap)
self.timeout_seconds = timeout_seconds
# hashed token -> epoch at which the session would be expired anyway.
# PostgreSQL already holds the authoritative revocation; this only keeps
# logout immediate for this process while the cache is unreachable.
self._pending: dict[str, float] = {}
self._disabled_until = 0.0
self._degraded = False
self.client: redis.Redis | None = None
if url:
self.client = redis.Redis.from_url(
url,
socket_timeout=timeout_seconds,
socket_connect_timeout=timeout_seconds,
decode_responses=True,
health_check_interval=30,
# A cache must fail fast: one attempt, no retry loop, so a dead
# Valkey costs at most one timeout before PostgreSQL answers.
retry=Retry(NoBackoff(), 0),
retry_on_error=[],
)
@property
def enabled(self) -> bool:
return self.client is not None
def _ready(self) -> bool:
"""True when a cache call is worth attempting right now."""
return self.client is not None and time.monotonic() >= self._disabled_until
def _unavailable(self, operation: str, exc: BaseException) -> None:
self._disabled_until = time.monotonic() + FAILURE_COOLDOWN_SECONDS
# One warning per outage, not one per request.
if not self._degraded:
self._degraded = True
logger.warning(
"cache_unavailable operation=%s error=%s cooldown_seconds=%s",
operation,
type(exc).__name__,
FAILURE_COOLDOWN_SECONDS,
)
def _ok(self) -> None:
self._degraded = False
def remember_revocation(self, hashed: str, remaining: float) -> None:
"""Ask this process to reject a token even if the cache cannot store it."""
if len(self._pending) >= MAX_PENDING_REVOCATIONS:
# Drop the entries closest to expiry: they are the least useful.
keep = len(self._pending) // 4
for key, _ in sorted(self._pending.items(), key=lambda item: item[1])[:keep]:
self._pending.pop(key, None)
self._pending[hashed] = time.time() + max(1.0, remaining)
def is_locally_revoked(self, hashed: str) -> bool:
deadline = self._pending.get(hashed)
if deadline is None:
return False
if deadline <= time.time():
self._pending.pop(hashed, None)
return False
return True
async def _flush_revocations(self) -> None:
"""Re-apply tombstones that could not be written during an outage."""
for hashed, deadline in list(self._pending.items()):
remaining = deadline - time.time()
if remaining <= 0:
self._pending.pop(hashed, None)
continue
try:
await self.client.set(self._key(hashed), REVOKED, ex=self._ttl(remaining))
except Exception as exc: # noqa: BLE001 - still revoked in PostgreSQL
self._unavailable("flush_revocation", exc)
return
self._pending.pop(hashed, None)
async def ping(self) -> bool:
if self.client is None:
return False
try:
alive = bool(await self.client.ping())
except Exception as exc: # noqa: BLE001 - any client error means "no cache"
logger.warning("cache_ping_failed error=%s", type(exc).__name__)
return False
self._disabled_until = 0.0
self._ok()
return alive
async def close(self) -> None:
if self.client is None:
return
try:
await self.client.aclose()
except Exception as exc: # noqa: BLE001 - shutdown must not fail on cache
logger.warning("cache_close_failed error=%s", type(exc).__name__)
def _key(self, hashed: str) -> str:
return CACHE_PREFIX + hashed
def _ttl(self, seconds: float) -> int:
return max(1, min(int(seconds), self.ttl_cap))
async def lookup(self, hashed: str) -> tuple[str, dict[str, Any] | None]:
"""Return (state, payload); every cache failure degrades to a DB read."""
if not self._ready():
return MISS, None
if self._pending:
await self._flush_revocations()
if not self._ready():
return MISS, None
try:
raw = await self.client.get(self._key(hashed))
self._ok()
except Exception as exc: # noqa: BLE001 - cache must never break auth
self._unavailable("lookup", exc)
return ERROR, None
if raw is None:
return MISS, None
if raw == REVOKED:
return TOMBSTONE, None
try:
payload = json.loads(raw)
user = payload["user"]
user["id"] = str(uuid.UUID(user["id"]))
payload["expires_epoch"] = float(payload["expires_epoch"])
payload["user"] = user
except (ValueError, KeyError, TypeError, json.JSONDecodeError):
logger.warning("cache_payload_invalid")
return MISS, None
return HIT, payload
async def remember(self, hashed: str, user: dict[str, Any], expires_at: datetime) -> None:
if not self._ready():
return
remaining = (expires_at - datetime.now(timezone.utc)).total_seconds()
if remaining <= 0:
return
# Cache the immutable public user snapshot, so a hit needs no SQL at all.
# Users cannot be renamed or deleted in the mandatory API of lab 2.
payload = {
"user": user,
"expires_epoch": expires_at.timestamp(),
}
try:
await self.client.set(
self._key(hashed),
json.dumps(payload, ensure_ascii=False),
ex=self._ttl(remaining),
)
self._ok()
except Exception as exc: # noqa: BLE001 - a missed cache write is not fatal
self._unavailable("remember", exc)
async def revoke(self, hashed: str, expires_at: datetime | None) -> None:
"""Write a tombstone so a stale entry cannot authenticate after logout."""
remaining = 60.0
if expires_at is not None:
remaining = max(1.0, (expires_at - datetime.now(timezone.utc)).total_seconds())
# PostgreSQL already revoked the session; this keeps the fast path honest
# even if the tombstone cannot be written right now.
self.remember_revocation(hashed, remaining)
if not self._ready():
return
try:
await self.client.set(self._key(hashed), REVOKED, ex=self._ttl(remaining))
self._pending.pop(hashed, None)
self._ok()
except Exception as exc: # noqa: BLE001 - PostgreSQL already revoked the session
self._unavailable("revoke", exc)
def _user_payload(row: dict[str, Any]) -> dict[str, Any]:
return {
"id": str(row["id"]),
"username": row["username"],
"display_name": row["display_name"],
"created_at": format_timestamp(row["created_at"]),
}
async def create_session(
pool: AsyncConnectionPool,
cache: SessionCache,
*,
user_id: uuid.UUID,
ttl_seconds: int,
) -> tuple[str, datetime]:
token = secrets.token_urlsafe(TOKEN_BYTES)
hashed = token_hash(token)
async with pool.connection() as connection:
async with connection.cursor() as cursor:
expires_at = await store.insert_session(
cursor, user_id=user_id, token_hash=hashed, ttl_seconds=ttl_seconds
)
user = await store.user_by_id(cursor, user_id)
if user is not None:
await cache.remember(hashed, _user_payload(user), expires_at)
return token, expires_at
async def resolve_session(
pool: AsyncConnectionPool, cache: SessionCache, token: str
) -> tuple[uuid.UUID, dict[str, Any]]:
"""Return the authenticated user or raise 401 with a Bearer challenge."""
hashed = token_hash(token)
if cache.is_locally_revoked(hashed):
raise ApiProblem(
401,
"unauthorized",
"Session token is expired or revoked",
{"WWW-Authenticate": 'Bearer realm="messenger"'},
)
state, _payload = await cache.lookup(hashed)
if state == TOMBSTONE:
raise ApiProblem(
401,
"unauthorized",
"Session token is expired or revoked",
{"WWW-Authenticate": 'Bearer realm="messenger"'},
)
# PostgreSQL is the authoritative revocation store. A cache value can only
# accelerate the immutable user snapshot; it must never be enough to grant
# access on its own. This makes logout durable across API replicas and
# Valkey outages/restores, where a tombstone may not reach shared cache.
async with pool.connection() as connection:
async with connection.cursor() as cursor:
row = await store.session_user(cursor, hashed)
if row is None:
await cache.revoke(hashed, None)
raise ApiProblem(
401,
"unauthorized",
"Session token is expired or revoked",
{"WWW-Authenticate": 'Bearer realm="messenger"'},
)
user = _user_payload(row)
# Refresh an old cache hit after PostgreSQL has confirmed it is still live.
await cache.remember(hashed, user, row["expires_at"])
return uuid.UUID(user["id"]), user
async def revoke_session(
pool: AsyncConnectionPool, cache: SessionCache, token: str
) -> None:
hashed = token_hash(token)
async with pool.connection() as connection:
async with connection.cursor() as cursor:
expires_at = await store.revoke_session(cursor, hashed)
if expires_at is None:
raise ApiProblem(
401,
"unauthorized",
"Session token is expired or revoked",
{"WWW-Authenticate": 'Bearer realm="messenger"'},
)
await cache.revoke(hashed, expires_at)
async def purge_expired(pool: AsyncConnectionPool) -> int:
async with pool.connection() as connection:
async with connection.cursor() as cursor:
return await store.purge_expired_sessions(cursor)
async def purge_expired_idempotency(pool: AsyncConnectionPool) -> int:
async with pool.connection() as connection:
async with connection.cursor() as cursor:
await cursor.execute(
"""
DELETE FROM idempotency_keys
WHERE created_at < now() - (%s || ' seconds')::interval
""",
(str(IDEMPOTENCY_TTL_SECONDS),),
)
return cursor.rowcount
+363
View File
@@ -0,0 +1,363 @@
"""PostgreSQL repository: parameterised statements only, explicit transactions."""
from __future__ import annotations
import uuid
from datetime import datetime
from typing import Any
import psycopg
from psycopg import AsyncCursor
from .errors import ApiProblem
from .pagination import format_timestamp
# One expression keeps the member list of a chat consistent across endpoints.
CHAT_COLUMNS = """
c.id,
c.title,
c.created_at,
ARRAY(
SELECT m.user_id::text
FROM chat_members m
WHERE m.chat_id = c.id
ORDER BY m.user_id::text
) AS member_ids
"""
def user_json(row: dict[str, Any]) -> dict[str, Any]:
return {
"id": str(row["id"]),
"username": row["username"],
"display_name": row["display_name"],
"created_at": format_timestamp(row["created_at"]),
}
def chat_json(row: dict[str, Any]) -> dict[str, Any]:
return {
"id": str(row["id"]),
"title": row["title"],
"member_ids": [str(member) for member in row["member_ids"]],
"created_at": format_timestamp(row["created_at"]),
}
def message_json(row: dict[str, Any]) -> dict[str, Any]:
return {
"id": str(row["id"]),
"chat_id": str(row["chat_id"]),
"sender_id": str(row["sender_id"]),
"text": row["text"],
"created_at": format_timestamp(row["created_at"]),
}
async def create_user(
cursor: AsyncCursor,
*,
username: str,
display_name: str,
password_hash: str,
) -> dict[str, Any]:
try:
await cursor.execute(
"""
INSERT INTO users (id, username, display_name, password_hash, created_at)
VALUES (%s, %s, %s, %s, now())
RETURNING id, username, display_name, created_at
""",
(uuid.uuid4(), username, display_name, password_hash),
)
except psycopg.errors.UniqueViolation:
raise ApiProblem(409, "conflict", "Username is already taken") from None
row = await cursor.fetchone()
return user_json(row)
async def user_credentials(cursor: AsyncCursor, username: str) -> dict[str, Any] | None:
await cursor.execute(
"""
SELECT id, username, display_name, password_hash, created_at
FROM users
WHERE username = %s
""",
(username,),
)
return await cursor.fetchone()
async def user_by_id(cursor: AsyncCursor, user_id: uuid.UUID) -> dict[str, Any] | None:
await cursor.execute(
"""
SELECT id, username, display_name, created_at
FROM users
WHERE id = %s
""",
(user_id,),
)
return await cursor.fetchone()
async def create_chat(
cursor: AsyncCursor,
*,
actor_id: uuid.UUID,
title: str,
member_ids: list[uuid.UUID],
) -> dict[str, Any]:
"""Insert the chat and its membership in one transaction.
Unknown members are rejected by the foreign key, so the failure rolls the
already inserted chat row back instead of leaving a chat without members.
"""
if actor_id not in member_ids:
raise ApiProblem(400, "invalid_request", "Current user must be a chat member")
chat_id = uuid.uuid4()
try:
await cursor.execute(
"INSERT INTO chats (id, title, created_at) VALUES (%s, %s, now())",
(chat_id, title),
)
await cursor.execute(
"""
INSERT INTO chat_members (chat_id, user_id, joined_at)
SELECT %s, member_id, now() FROM unnest(%s::uuid[]) AS member_id
""",
(chat_id, member_ids),
)
except psycopg.errors.ForeignKeyViolation:
raise ApiProblem(
400, "invalid_request", "One or more members do not exist"
) from None
except psycopg.errors.UniqueViolation:
raise ApiProblem(400, "invalid_request", "Chat members must be unique") from None
await cursor.execute(
"SELECT " + CHAT_COLUMNS + " FROM chats c WHERE c.id = %s",
(chat_id,),
)
row = await cursor.fetchone()
return chat_json(row)
async def accessible_chat(
cursor: AsyncCursor, *, actor_id: uuid.UUID, chat_id: uuid.UUID
) -> dict[str, Any]:
await cursor.execute(
"""
SELECT """
+ CHAT_COLUMNS
+ """
FROM chats c
JOIN chat_members membership
ON membership.chat_id = c.id AND membership.user_id = %s
WHERE c.id = %s
""",
(actor_id, chat_id),
)
row = await cursor.fetchone()
if row is None:
raise ApiProblem(404, "not_found", "Chat not found")
return chat_json(row)
async def chat_page(
cursor: AsyncCursor,
*,
actor_id: uuid.UUID,
limit: int,
after: tuple[datetime, uuid.UUID] | None,
) -> list[dict[str, Any]]:
query = (
"SELECT "
+ CHAT_COLUMNS
+ """
FROM chats c
JOIN chat_members membership
ON membership.chat_id = c.id AND membership.user_id = %(actor)s
"""
)
params: dict[str, Any] = {"actor": actor_id, "limit": limit + 1}
if after is not None:
query += " WHERE (c.created_at, c.id) > (%(after_ts)s, %(after_id)s)"
params["after_ts"], params["after_id"] = after
query += " ORDER BY c.created_at, c.id LIMIT %(limit)s"
await cursor.execute(query, params)
return [chat_json(row) for row in await cursor.fetchall()]
async def require_membership(
cursor: AsyncCursor, *, actor_id: uuid.UUID, chat_id: uuid.UUID
) -> None:
await cursor.execute(
"SELECT 1 FROM chat_members WHERE chat_id = %s AND user_id = %s",
(chat_id, actor_id),
)
if await cursor.fetchone() is None:
raise ApiProblem(404, "not_found", "Chat not found")
async def get_idempotency_key(
cursor: AsyncCursor, *, actor_id: uuid.UUID, chat_id: uuid.UUID, key: str
) -> dict[str, Any] | None:
"""Return a record only from this user's POST-message resource scope."""
await cursor.execute(
"""
SELECT key, actor_id, chat_id, request_sha256, message_id, created_at
FROM idempotency_keys
WHERE actor_id = %s AND chat_id = %s AND key = %s
""",
(actor_id, chat_id, key),
)
return await cursor.fetchone()
async def insert_idempotency_key(
cursor: AsyncCursor,
*,
actor_id: uuid.UUID,
key: str,
chat_id: uuid.UUID,
request_sha256: str,
) -> bool:
"""Atomically reserve a key without poisoning a concurrent transaction."""
await cursor.execute(
"""
INSERT INTO idempotency_keys (actor_id, key, chat_id, request_sha256, created_at)
VALUES (%s, %s, %s, %s, now())
ON CONFLICT (actor_id, chat_id, key) DO NOTHING
RETURNING key
""",
(actor_id, key, chat_id, request_sha256),
)
return await cursor.fetchone() is not None
async def message_by_id(cursor: AsyncCursor, message_id: uuid.UUID) -> dict[str, Any] | None:
await cursor.execute(
"""
SELECT id, chat_id, sender_id, text, created_at
FROM messages
WHERE id = %s
""",
(message_id,),
)
row = await cursor.fetchone()
if row is None:
return None
return message_json(row)
async def create_message(
cursor: AsyncCursor,
*,
actor_id: uuid.UUID,
chat_id: uuid.UUID,
text: str,
request_sha256: str,
key: str,
) -> dict[str, Any]:
await require_membership(cursor, actor_id=actor_id, chat_id=chat_id)
# The SHA-256 of the replayed body is recorded with the idempotency row so a
# client cannot replay a request with different content under the same key.
message_id = uuid.uuid4()
await cursor.execute(
"""
INSERT INTO messages (id, chat_id, sender_id, text, created_at)
VALUES (%s, %s, %s, %s, now())
RETURNING id, chat_id, sender_id, text, created_at
""",
(message_id, chat_id, actor_id, text),
)
row = await cursor.fetchone()
# Record the message_id so a replay can return the original message.
await cursor.execute(
"""
UPDATE idempotency_keys
SET request_sha256 = %s, message_id = %s
WHERE actor_id = %s AND chat_id = %s AND key = %s
""",
(request_sha256, message_id, actor_id, chat_id, key),
)
return message_json(row)
async def message_page(
cursor: AsyncCursor,
*,
chat_id: uuid.UUID,
limit: int,
after: tuple[datetime, uuid.UUID] | None,
) -> list[dict[str, Any]]:
"""Keyset page over (created_at, id); the composite index serves both branches."""
query = """
SELECT id, chat_id, sender_id, text, created_at
FROM messages
WHERE chat_id = %(chat)s
"""
params: dict[str, Any] = {"chat": chat_id, "limit": limit + 1}
if after is not None:
query += " AND (created_at, id) > (%(after_ts)s, %(after_id)s)"
params["after_ts"], params["after_id"] = after
query += " ORDER BY created_at, id LIMIT %(limit)s"
await cursor.execute(query, params)
return [message_json(row) for row in await cursor.fetchall()]
async def insert_session(
cursor: AsyncCursor,
*,
user_id: uuid.UUID,
token_hash: str,
ttl_seconds: int,
) -> datetime:
await cursor.execute(
"""
INSERT INTO sessions (id, user_id, token_hash, created_at, expires_at)
VALUES (%s, %s, %s, now(), now() + make_interval(secs => %s))
RETURNING expires_at
""",
(uuid.uuid4(), user_id, token_hash, ttl_seconds),
)
row = await cursor.fetchone()
return row["expires_at"]
async def session_user(cursor: AsyncCursor, token_hash: str) -> dict[str, Any] | None:
"""Resolve a live session row; expired and revoked sessions never match."""
await cursor.execute(
"""
SELECT s.expires_at, u.id, u.username, u.display_name, u.created_at
FROM sessions s
JOIN users u ON u.id = s.user_id
WHERE s.token_hash = %s AND s.revoked_at IS NULL AND s.expires_at > now()
""",
(token_hash,),
)
return await cursor.fetchone()
async def revoke_session(cursor: AsyncCursor, token_hash: str) -> datetime | None:
"""Revoke a live session and return its expiry, or None when it is not live."""
await cursor.execute(
"""
UPDATE sessions SET revoked_at = now()
WHERE token_hash = %s AND revoked_at IS NULL AND expires_at > now()
RETURNING expires_at
""",
(token_hash,),
)
row = await cursor.fetchone()
return None if row is None else row["expires_at"]
async def purge_expired_sessions(cursor: AsyncCursor) -> int:
await cursor.execute(
"""
DELETE FROM sessions
WHERE expires_at < now() - interval '1 hour'
OR (revoked_at IS NOT NULL AND revoked_at < now() - interval '1 hour')
"""
)
return cursor.rowcount
+209
View File
@@ -0,0 +1,209 @@
"""Minimal standard-library HTTP client for the student's own tests.
Run with the API already up:
python -m unittest discover -s student-tests -p "test_*.py" -v
BASE_URL selects the stand (default http://127.0.0.1:8080). No third-party
dependency is needed, so `make test-student` works on a clean clone.
"""
from __future__ import annotations
import base64
import json
import os
import urllib.error
import urllib.parse
import urllib.request
import uuid
import json
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
LAB = json.loads((ROOT / "lab.json").read_text(encoding="utf-8"))["number"]
# 127.0.0.1 rather than "localhost": on a dual-stack host "localhost" resolves
# to ::1 first, a published Docker port may answer on IPv4 only, and every IPv6
# attempt then costs ~2 s of SYN retry before urllib falls back to IPv4. The IPv4
# literal keeps the client fast and never reaches the application.
BASE_URL = os.environ.get("BASE_URL", "http://127.0.0.1:8080").rstrip("/")
TIMEOUT = float(os.environ.get("STUDENT_TEST_TIMEOUT", "20"))
class NoRedirect(urllib.request.HTTPRedirectHandler):
"""The contract never requires redirects; fail loudly instead of following."""
def redirect_request(self, req, fp, code, msg, headers, newurl): # noqa: ANN001
return None
# The lab stand is always loopback; urllib ignores the Windows proxy bypass
# list, so a corporate/system proxy would otherwise be used for localhost too.
_OPENER = urllib.request.build_opener(urllib.request.ProxyHandler({}), NoRedirect())
class Response:
def __init__(self, status: int, headers: dict[str, str], data: bytes) -> None:
self.status = status
self.headers = headers
self.data = data
try:
self.json = json.loads(data.decode("utf-8")) if data else None
except (ValueError, UnicodeDecodeError):
self.json = None
def header(self, name: str) -> str:
for key, value in self.headers.items():
if key.lower() == name.lower():
return value
return ""
def __repr__(self) -> str: # pragma: no cover - diagnostics only
return "Response(status=%s, json=%r)" % (self.status, self.json)
def request(
method: str,
path: str,
*,
body=None,
raw: bytes | None = None,
token: str | None = None,
headers: dict[str, str] | None = None,
content_type: str | None = None,
timeout: float | None = None,
base_url: str | None = None,
) -> Response:
url = (base_url or BASE_URL) + path
data = raw
request_headers = {"Accept": "application/json"}
if body is not None:
data = json.dumps(body, ensure_ascii=False).encode("utf-8")
request_headers["Content-Type"] = "application/json"
if content_type is not None:
request_headers["Content-Type"] = content_type
if token is not None:
request_headers["Authorization"] = "Bearer " + token
request_headers.update(headers or {})
prepared = urllib.request.Request(url, data=data, headers=request_headers, method=method)
try:
response = _OPENER.open(prepared, timeout=timeout or TIMEOUT)
except urllib.error.HTTPError as error:
response = error
with response:
payload = response.read()
return Response(response.code, dict(response.headers.items()), payload)
def basic(username: str, password: str) -> dict[str, str]:
encoded = base64.b64encode((username + ":" + password).encode("utf-8")).decode("ascii")
return {"Authorization": "Basic " + encoded}
def new_registration(prefix: str = "st", password: str | None = None) -> dict[str, str]:
suffix = uuid.uuid4().hex[:12]
return {
"username": prefix + "_" + suffix,
"display_name": "Тест " + suffix,
"password": password or ("St-" + uuid.uuid4().hex),
}
def register(registration: dict[str, str] | None = None) -> Response:
return request("POST", "/api/v1/users", body=registration or new_registration())
def login(username: str, password: str) -> Response:
return request("POST", "/api/v1/auth/sessions", headers=basic(username, password))
class Actor:
"""A registered user with a live session, as a client would hold it."""
def __init__(self, registration: dict[str, str]) -> None:
self.registration = registration
response = register(registration)
if response.status != 201:
raise AssertionError(
"registration failed: %s %r" % (response.status, response.json)
)
self.user = response.json
self.username = registration["username"]
self.password = registration["password"]
@property
def id(self) -> str:
return self.user["id"]
def sign_in(self) -> "Actor":
response = login(self.username, self.password)
if response.status != 201:
raise AssertionError("login failed: %s %r" % (response.status, response.json))
self.session = response.json
self.token = response.json["token"]
return self
def actor(prefix: str = "st", password: str | None = None) -> Actor:
return Actor(new_registration(prefix, password)).sign_in()
def create_chat(token: str, member_ids: list[str], title: str = "Тестовый чат") -> dict:
response = request(
"POST",
"/api/v1/chats",
body={"title": title, "member_ids": member_ids},
token=token,
)
if response.status != 201:
raise AssertionError("chat creation failed: %s %r" % (response.status, response.json))
return response.json
def send(token: str, chat_id: str, text: str) -> dict:
headers = {}
if LAB >= 3:
headers["Idempotency-Key"] = uuid.uuid4().hex
response = request(
"POST", "/api/v1/chats/" + chat_id + "/messages", body={"text": text}, token=token, headers=headers
)
if response.status != 201:
raise AssertionError("message failed: %s %r" % (response.status, response.json))
return response.json
def history(token: str, chat_id: str, limit: int = 100) -> list[dict]:
"""Walk every page of a chat, guarding against cursor loops."""
items: list[dict] = []
seen: set[str] = set()
cursor: str | None = None
while True:
path = "/api/v1/chats/" + chat_id + "/messages?limit=" + str(limit)
if cursor is not None:
path += "&cursor=" + urllib.parse.quote(cursor, safe="")
response = request("GET", path, token=token)
if response.status != 200:
raise AssertionError("history failed: %s %r" % (response.status, response.json))
items.extend(response.json["items"])
cursor = response.json["next_cursor"]
if cursor is None:
return items
if cursor in seen:
raise AssertionError("cursor loop detected")
seen.add(cursor)
def expect_error(test, response: Response, status: int, code: str) -> dict:
"""Assert the public error envelope, request id echo and the status."""
test.assertEqual(response.status, status, response.json)
test.assertIsInstance(response.json, dict)
error = response.json.get("error")
test.assertIsNotNone(error, "missing error envelope")
test.assertEqual(error["code"], code)
test.assertTrue(error["message"])
request_id = response.header("X-Request-Id")
test.assertTrue(request_id, "missing X-Request-Id header")
test.assertEqual(error["request_id"], request_id)
return error
+329
View File
@@ -0,0 +1,329 @@
"""Own functional tests: contract edges carried from lab 1 plus lab 2 auth.
Every test creates unique synthetic users, so the file is safe to re-run against
the same stand (the API has no delete endpoint by design).
"""
from __future__ import annotations
import unittest
import uuid
from datetime import datetime, timezone
import common
def timestamp(value: str) -> datetime:
return datetime.fromisoformat(value[:-1] + "+00:00")
class Base(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
again = common.request("POST", "/api/v1/users", body=common.new_registration())
if again.status != 201:
raise AssertionError("stand is not serving: %r" % (again.json,))
cls.alice = common.actor("alice")
cls.bob = common.actor("bob")
cls.eve = common.actor("eve")
cls.chat = common.create_chat(
cls.alice.token, [cls.alice.id, cls.bob.id], "Контрактный чат"
)
class HealthAndEnvelope(Base):
def test_health_endpoints_and_request_id(self) -> None:
live = common.request("GET", "/health/live")
self.assertEqual(live.status, 200)
self.assertEqual(live.json, {"status": "ok"})
self.assertTrue(live.header("X-Request-Id"))
ready = common.request("GET", "/health/ready")
self.assertEqual(ready.status, 200)
self.assertEqual(ready.json, {"status": "ok"})
def test_unknown_route_uses_same_error_envelope(self) -> None:
common.expect_error(self, common.request("GET", "/api/v1/nope"), 404, "not_found")
def test_query_parameters_rejected_where_not_supported(self) -> None:
response = common.request(
"GET", "/api/v1/users/me?verbose=1", token=self.alice.token
)
common.expect_error(self, response, 400, "invalid_request")
class Registration(Base):
def test_response_has_no_secret_fields(self) -> None:
response = common.register(common.new_registration())
self.assertEqual(response.status, 201)
self.assertEqual(
set(response.json), {"id", "username", "display_name", "created_at"}
)
self.assertNotIn("password", response.json)
self.assertNotIn("password_hash", response.json)
def test_duplicate_username_conflict(self) -> None:
registration = common.new_registration()
self.assertEqual(common.register(registration).status, 201)
common.expect_error(self, common.register(registration), 409, "conflict")
def test_username_rules(self) -> None:
for username in ("UPPER_case", "ab", "with-dash", "with space"):
body = common.new_registration()
body["username"] = username
common.expect_error(self, common.register(body), 400, "invalid_request")
def test_password_rules(self) -> None:
short = common.new_registration(password="x" * 11)
common.expect_error(self, common.register(short), 400, "invalid_request")
long_password = common.new_registration(password="x" * 129)
common.expect_error(self, common.register(long_password), 400, "invalid_request")
# 12 Unicode code points are enough, even when they are multi-byte.
unicode_password = common.new_registration(password="😀" * 12)
self.assertEqual(common.register(unicode_password).status, 201)
def test_malformed_json_extra_fields_and_wrong_types(self) -> None:
common.expect_error(
self,
common.request(
"POST", "/api/v1/users", raw=b'{"username": ', content_type="application/json"
),
400,
"invalid_request",
)
body = common.new_registration()
body["is_admin"] = True
common.expect_error(self, common.register(body), 400, "invalid_request")
body = common.new_registration()
body["display_name"] = 42
common.expect_error(self, common.register(body), 400, "invalid_request")
def test_unsupported_media_type(self) -> None:
response = common.request(
"POST",
"/api/v1/users",
raw=b"username=alice",
content_type="application/x-www-form-urlencoded",
)
common.expect_error(self, response, 415, "unsupported_media_type")
class Chats(Base):
def test_membership_validation(self) -> None:
cases = [
[self.bob.id], # caller is missing
[self.alice.id, self.alice.id], # duplicate member
[self.alice.id, str(uuid.uuid4())], # unknown member
[], # no members
]
for member_ids in cases:
response = common.request(
"POST",
"/api/v1/chats",
body={"title": "Плохой чат", "member_ids": member_ids},
token=self.alice.token,
)
common.expect_error(self, response, 400, "invalid_request")
def test_foreign_chat_is_hidden_everywhere(self) -> None:
chat_id = self.chat["id"]
common.expect_error(
self,
common.request("GET", "/api/v1/chats/" + chat_id, token=self.eve.token),
404,
"not_found",
)
common.expect_error(
self,
common.request(
"GET", "/api/v1/chats/" + chat_id + "/messages", token=self.eve.token
),
404,
"not_found",
)
common.expect_error(
self,
common.request(
"POST",
"/api/v1/chats/" + chat_id + "/messages",
body={"text": "чужое"},
token=self.eve.token,
),
404,
"not_found",
)
listed = common.request("GET", "/api/v1/chats?limit=100", token=self.eve.token)
self.assertEqual(listed.status, 200)
self.assertNotIn(chat_id, [item["id"] for item in listed.json["items"]])
def test_messages_are_validated_and_unicode_preserved(self) -> None:
chat = common.create_chat(self.alice.token, [self.alice.id, self.bob.id])
for text in ("", " \n\t ", "x" * 4001):
response = common.request(
"POST",
"/api/v1/chats/" + chat["id"] + "/messages",
body={"text": text},
token=self.alice.token,
)
common.expect_error(self, response, 400, "invalid_request")
prefix = "Привет 👋 世界 "
text = prefix + "x" * (4000 - len(prefix)) # exactly the contract maximum
message = common.send(self.alice.token, chat["id"], text)
self.assertEqual(message["text"], text)
self.assertEqual(message["sender_id"], self.alice.id)
injected = common.request(
"POST",
"/api/v1/chats/" + chat["id"] + "/messages",
body={"text": "подмена", "sender_id": self.bob.id},
token=self.alice.token,
)
common.expect_error(self, injected, 400, "invalid_request")
class Pagination(Base):
def test_walks_every_page_without_loss_or_duplicates(self) -> None:
chat = common.create_chat(self.alice.token, [self.alice.id, self.bob.id])
sent = [
common.send(self.alice.token if index % 2 else self.bob.token, chat["id"], "msg-%d" % index)
for index in range(7)
]
collected: list[dict] = []
cursor = None
pages = 0
while True:
path = "/api/v1/chats/" + chat["id"] + "/messages?limit=2"
if cursor is not None:
path += "&cursor=" + cursor
page = common.request("GET", path, token=self.alice.token)
self.assertEqual(page.status, 200)
collected.extend(page.json["items"])
pages += 1
cursor = page.json["next_cursor"]
if cursor is None:
break
self.assertLess(pages, 10, "pagination did not terminate")
self.assertEqual(pages, 4)
self.assertEqual([item["id"] for item in collected], [item["id"] for item in sent])
self.assertEqual(
collected,
sorted(collected, key=lambda item: (timestamp(item["created_at"]), item["id"])),
)
def test_cursor_is_bound_to_its_chat(self) -> None:
other = common.create_chat(self.alice.token, [self.alice.id])
common.send(self.alice.token, other["id"], "a")
common.send(self.alice.token, other["id"], "b")
first = common.request(
"GET", "/api/v1/chats/" + other["id"] + "/messages?limit=1", token=self.alice.token
)
cursor = first.json["next_cursor"]
self.assertIsNotNone(cursor)
response = common.request(
"GET",
"/api/v1/chats/" + self.chat["id"] + "/messages" + "?limit=1&cursor=" + cursor,
token=self.alice.token,
)
common.expect_error(self, response, 400, "invalid_request")
def test_limit_and_cursor_validation(self) -> None:
base = "/api/v1/chats/" + self.chat["id"] + "/messages"
for query in ("?limit=0", "?limit=101", "?limit=abc", "?cursor=", "?cursor=not-a-cursor", "?limit=1&limit=2", "?unknown=1"):
common.expect_error(
self,
common.request("GET", base + query, token=self.alice.token),
400,
"invalid_request",
)
class Authentication(Base):
def test_basic_is_not_accepted_outside_session_creation(self) -> None:
response = common.request(
"GET", "/api/v1/users/me", headers=common.basic(self.alice.username, self.alice.password)
)
common.expect_error(self, response, 401, "unauthorized")
self.assertTrue(response.header("WWW-Authenticate").lower().startswith("bearer"))
def test_missing_and_unknown_tokens(self) -> None:
for headers in ({}, {"Authorization": "Bearer not-a-real-token"}):
response = common.request("GET", "/api/v1/users/me", headers=headers)
common.expect_error(self, response, 401, "unauthorized")
self.assertTrue(response.header("WWW-Authenticate").lower().startswith("bearer"))
def test_x_user_id_is_ignored(self) -> None:
response = common.request("GET", "/api/v1/users/me", headers={"X-User-Id": self.alice.id})
common.expect_error(self, response, 401, "unauthorized")
overridden = common.request(
"GET",
"/api/v1/users/me",
token=self.alice.token,
headers={"X-User-Id": self.eve.id},
)
self.assertEqual(overridden.status, 200)
self.assertEqual(overridden.json["id"], self.alice.id)
def test_session_payload_and_expiry(self) -> None:
response = common.login(self.alice.username, self.alice.password)
self.assertEqual(response.status, 201)
session = response.json
self.assertEqual(set(session), {"token", "token_type", "expires_at", "user"})
self.assertEqual(session["token_type"], "Bearer")
self.assertGreaterEqual(len(session["token"]), 43)
self.assertNotIn(self.alice.username, session["token"])
self.assertNotEqual(session["token"], self.alice.id)
self.assertEqual(session["user"], self.alice.user)
self.assertGreater(timestamp(session["expires_at"]), datetime.now(timezone.utc))
me = common.request("GET", "/api/v1/users/me", token=session["token"])
self.assertEqual(me.status, 200)
self.assertEqual(me.json, self.alice.user)
def test_wrong_password_and_unknown_user_are_indistinguishable(self) -> None:
wrong = common.login(self.alice.username, "definitely-wrong-password")
unknown = common.login("nobody_here_" + uuid.uuid4().hex[:8], "definitely-wrong-password")
messages = []
for response in (wrong, unknown):
error = common.expect_error(self, response, 401, "unauthorized")
self.assertTrue(
response.header("WWW-Authenticate").lower().startswith("basic")
)
messages.append(error["message"])
# Same message and challenge: a client cannot tell the accounts apart.
self.assertEqual(messages[0], messages[1])
def test_logout_revokes_only_that_session(self) -> None:
first = common.login(self.alice.username, self.alice.password).json["token"]
second = common.login(self.alice.username, self.alice.password).json["token"]
revoked = common.request("DELETE", "/api/v1/auth/sessions/current", token=first)
self.assertEqual(revoked.status, 204)
self.assertEqual(revoked.data, b"")
common.expect_error(
self,
common.request("GET", "/api/v1/users/me", token=first),
401,
"unauthorized",
)
common.expect_error(
self,
common.request("DELETE", "/api/v1/auth/sessions/current", token=first),
401,
"unauthorized",
)
still_valid = common.request("GET", "/api/v1/users/me", token=second)
self.assertEqual(still_valid.status, 200)
def test_session_endpoint_rejects_json_body_and_bad_basic(self) -> None:
response = common.request(
"POST",
"/api/v1/auth/sessions",
body={"username": self.alice.username, "password": self.alice.password},
)
common.expect_error(self, response, 401, "unauthorized")
response = common.request(
"POST", "/api/v1/auth/sessions", headers={"Authorization": "Basic !!!not-base64!!!"}
)
common.expect_error(self, response, 401, "unauthorized")
self.assertTrue(response.header("WWW-Authenticate").lower().startswith("basic"))
if __name__ == "__main__":
unittest.main()
+211
View File
@@ -0,0 +1,211 @@
"""Dependency outage behaviour: Valkey down, then PostgreSQL down.
Requires Docker access on the same host as the Compose stand and is therefore
opt-in:
STUDENT_DOCKER_TESTS=1 python -m unittest -v test_degradation
The test never removes volumes and restores both services in `finally`.
"""
from __future__ import annotations
import json
import os
import uuid
import statistics
import subprocess
import time
import unittest
from pathlib import Path
import common
REPO_ROOT = Path(__file__).resolve().parents[1]
PROJECT = os.environ.get("COMPOSE_PROJECT_NAME", "mtusi2026lab3")
ENV_FILE = os.environ.get("COMPOSE_ENV_FILE", ".env")
ENABLED = os.environ.get("STUDENT_DOCKER_TESTS") == "1"
def compose(*arguments: str, timeout: int = 120) -> None:
result = subprocess.run(
["docker", "compose", "-p", PROJECT, "--env-file", ENV_FILE, *arguments],
cwd=str(REPO_ROOT),
capture_output=True,
text=True,
timeout=timeout,
)
if result.returncode != 0:
raise AssertionError(
"docker compose %s failed: %s" % (" ".join(arguments), result.stderr[-500:])
)
def wait_ready(expected: int, seconds: float) -> float:
started = time.monotonic()
deadline = started + seconds
last = None
while time.monotonic() < deadline:
try:
last = common.request("GET", "/health/ready", timeout=5)
if last.status == expected:
return time.monotonic() - started
except OSError:
# A replica can close the listener while Compose restarts it.
pass
time.sleep(0.5)
raise AssertionError(
"readiness did not reach %s within %.0fs (last=%s %r)"
% (expected, seconds, last.status if last else None, last.json if last else None)
)
@unittest.skipUnless(ENABLED, "set STUDENT_DOCKER_TESTS=1 to stop real containers")
class DependencyOutages(unittest.TestCase):
def test_logout_remains_revoked_across_replicas_after_cache_outage(self) -> None:
"""A stale shared-cache hit must not bypass PostgreSQL revocation."""
alice = common.actor("revoked")
chat = common.create_chat(alice.token, [alice.id], "Durable logout")
self.assertEqual(common.request("GET", "/api/v1/users/me", token=alice.token).status, 200)
try:
compose("stop", "valkey")
logout = common.request("DELETE", "/api/v1/auth/sessions/current", token=alice.token)
self.assertEqual(logout.status, 204, logout.json)
# Remove both processes' local fallback state and restore Valkey's
# old data. Requests through nginx can land on either replica.
compose("restart", "api-1", "api-2")
compose("start", "valkey")
wait_ready(200, 90)
common.expect_error(
self, common.request("GET", "/api/v1/users/me", token=alice.token), 401, "unauthorized"
)
common.expect_error(
self,
common.request("GET", "/api/v1/chats/" + chat["id"], token=alice.token),
401,
"unauthorized",
)
common.expect_error(
self,
common.request(
"POST",
"/api/v1/chats/" + chat["id"] + "/messages",
body={"text": "must not be accepted"},
token=alice.token,
headers={"Idempotency-Key": uuid.uuid4().hex},
),
401,
"unauthorized",
)
finally:
compose("start", "valkey", "postgres", "api-1", "api-2")
try:
wait_ready(200, 90)
except AssertionError:
pass
def test_valkey_loss_degrades_then_postgres_loss_is_bounded(self) -> None:
alice = common.actor("degrade")
chat = common.create_chat(alice.token, [alice.id], "Устойчивость")
# Warm the session cache.
self.assertEqual(common.request("GET", "/api/v1/users/me", token=alice.token).status, 200)
cached = self._sample_latency(alice.token, 60)
summary = {"cached_p50_ms": round(cached["p50"], 2)}
try:
# 1. Valkey is a cache: auth still works, without DB-free fast path.
compose("stop", "valkey")
ready = common.request("GET", "/health/ready")
self.assertEqual(
ready.status, 200, "session cache must not be a readiness dependency"
)
me = common.request("GET", "/api/v1/users/me", token=alice.token)
self.assertEqual(me.status, 200, me.json)
sent = common.request(
"POST",
"/api/v1/chats/" + chat["id"] + "/messages",
body={"text": "без кэша"},
token=alice.token,
headers={"Idempotency-Key": uuid.uuid4().hex},
)
self.assertEqual(sent.status, 201, sent.json)
without_cache = self._sample_latency(alice.token, 60)
summary["cache_down_p50_ms"] = round(without_cache["p50"], 2)
summary["cache_down_p95_ms"] = round(without_cache["p95"], 2)
compose("start", "valkey")
time.sleep(3)
self.assertEqual(common.request("GET", "/health/ready").status, 200)
# 2. PostgreSQL is the source of truth: bounded errors, never a 200
# for a token the server cannot verify, and never a raw 500.
compose("stop", "postgres")
ready = common.request("GET", "/health/ready")
self.assertEqual(ready.status, 503, ready.json)
self.assertEqual(ready.json["error"]["code"], "unavailable")
started = time.monotonic()
write = common.request(
"POST",
"/api/v1/chats/" + chat["id"] + "/messages",
body={"text": "без базы"},
token=alice.token,
timeout=30,
headers={"Idempotency-Key": uuid.uuid4().hex},
)
bounded = time.monotonic() - started
self.assertEqual(write.status, 503, write.json)
self.assertEqual(write.json["error"]["code"], "unavailable")
self.assertLess(bounded, 10, "database failure must fail fast")
summary["db_down_write_seconds"] = round(bounded, 2)
for leaked in ("psycopg", "postgresql://", "password", "Traceback"):
self.assertNotIn(leaked, json.dumps(write.json))
unknown = common.request(
"GET", "/api/v1/users/me", token="unknown-token-value", timeout=30
)
self.assertIn(unknown.status, (401, 503))
self.assertNotEqual(unknown.status, 200)
# 3. Recovery keeps the same session and the stored chat.
compose("start", "postgres")
summary["recovery_seconds"] = round(wait_ready(200, 90), 2)
after = common.request("GET", "/api/v1/chats/" + chat["id"], token=alice.token)
self.assertEqual(after.status, 200, after.json)
recovered = common.request(
"POST",
"/api/v1/chats/" + chat["id"] + "/messages",
body={"text": "после восстановления"},
token=alice.token,
headers={"Idempotency-Key": uuid.uuid4().hex},
)
self.assertEqual(recovered.status, 201, recovered.json)
messages = common.history(alice.token, chat["id"])
self.assertIn("после восстановления", [message["text"] for message in messages])
finally:
compose("start", "valkey", "postgres")
try:
wait_ready(200, 90)
except AssertionError:
pass
print("\n outage summary: " + json.dumps(summary, ensure_ascii=False))
def _sample_latency(self, token: str, count: int) -> dict[str, float]:
samples = []
for _ in range(count):
started = time.monotonic()
response = common.request("GET", "/api/v1/users/me", token=token, timeout=30)
samples.append(time.monotonic() - started)
self.assertEqual(response.status, 200, response.json)
ordered = sorted(samples)
return {
"p50": statistics.median(samples) * 1000,
"p95": ordered[max(0, int(len(ordered) * 0.95) - 1)] * 1000,
}
if __name__ == "__main__":
unittest.main()
+99
View File
@@ -0,0 +1,99 @@
"""Session expiry with SESSION_TTL_SECONDS=2, plus logout and bad credentials.
Run against the dedicated short-TTL stand:
scripts/ttl-stack.sh start
BASE_URL=http://localhost:8081 python -m unittest -v test_expiry
scripts/ttl-stack.sh stop
"""
from __future__ import annotations
import statistics
import time
import unittest
from datetime import datetime, timezone
import common
def expires_at(session: dict) -> datetime:
return datetime.fromisoformat(session["expires_at"][:-1] + "+00:00")
class ShortTtlSessions(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
ready = common.request("GET", "/health/ready")
if ready.status != 200:
raise AssertionError("stand is not ready: %r" % (ready.json,))
def test_expiry_is_enforced_and_cache_does_not_extend_it(self) -> None:
user = common.actor("ttl")
started = datetime.now(timezone.utc)
session = common.login(user.username, user.password)
self.assertEqual(session.status, 201)
lifetime = (expires_at(session.json) - started).total_seconds()
self.assertLessEqual(lifetime, 5.0, "stand does not use a short TTL")
# Two reads: the second one is served from the Valkey cache, so the
# expiry check below covers the cached path as well.
for _ in range(2):
self.assertEqual(
common.request("GET", "/api/v1/users/me", token=session.json["token"]).status,
200,
)
time.sleep(3.2)
expired = common.request("GET", "/api/v1/users/me", token=session.json["token"])
common.expect_error(self, expired, 401, "unauthorized")
self.assertTrue(expired.header("WWW-Authenticate").lower().startswith("bearer"))
revoked = common.request(
"DELETE", "/api/v1/auth/sessions/current", token=session.json["token"]
)
common.expect_error(self, revoked, 401, "unauthorized")
# A fresh login still works, so expiry removed only the old session.
again = common.login(user.username, user.password)
self.assertEqual(again.status, 201)
self.assertEqual(
common.request("GET", "/api/v1/users/me", token=again.json["token"]).status,
200,
)
def test_logout_is_immediate_even_when_the_session_is_cached(self) -> None:
user = common.actor("ttllogout")
token = user.token
self.assertEqual(common.request("GET", "/api/v1/users/me", token=token).status, 200)
self.assertEqual(
common.request("DELETE", "/api/v1/auth/sessions/current", token=token).status,
204,
)
outcome = common.request("GET", "/api/v1/users/me", token=token)
common.expect_error(self, outcome, 401, "unauthorized")
def test_wrong_password_issues_no_session(self) -> None:
user = common.actor("ttlwrong")
denied = common.login(user.username, "not-the-password")
common.expect_error(self, denied, 401, "unauthorized")
self.assertTrue(denied.header("WWW-Authenticate").lower().startswith("basic"))
def test_session_lookup_latency_is_bounded(self) -> None:
user = common.actor("ttllatency")
samples = []
for _ in range(30):
started = time.monotonic()
response = common.request("GET", "/api/v1/users/me", token=user.token, timeout=5)
samples.append(time.monotonic() - started)
self.assertEqual(response.status, 200)
print(
"\n /users/me warm cache p50=%.1fms p95=%.1fms"
% (
statistics.median(samples) * 1000,
sorted(samples)[int(len(samples) * 0.95) - 1] * 1000,
)
)
if __name__ == "__main__":
unittest.main()
+85
View File
@@ -0,0 +1,85 @@
"""Idempotency scopes and concurrent replay checks for laboratory 3."""
from __future__ import annotations
import threading
import unittest
import uuid
import common
class IdempotencyScope(unittest.TestCase):
def test_same_key_is_independent_for_different_users_and_chats(self) -> None:
alice = common.actor("idemalice")
eve = common.actor("idemeve")
alice_chat = common.create_chat(alice.token, [alice.id], "Alice scope")
eve_chat = common.create_chat(eve.token, [eve.id], "Eve scope")
key = uuid.uuid4().hex
first = common.request(
"POST",
"/api/v1/chats/" + alice_chat["id"] + "/messages",
body={"text": "same body"},
token=alice.token,
headers={"Idempotency-Key": key},
)
second = common.request(
"POST",
"/api/v1/chats/" + eve_chat["id"] + "/messages",
body={"text": "same body"},
token=eve.token,
headers={"Idempotency-Key": key},
)
self.assertEqual(first.status, 201, first.json)
self.assertEqual(second.status, 201, second.json)
self.assertNotEqual(first.json["id"], second.json["id"])
self.assertEqual(second.json["chat_id"], eve_chat["id"])
self.assertEqual(second.json["sender_id"], eve.id)
# A different payload in another scope is also a new independent write.
other_scope = common.create_chat(eve.token, [eve.id], "Eve second scope")
third = common.request(
"POST",
"/api/v1/chats/" + other_scope["id"] + "/messages",
body={"text": "different body"},
token=eve.token,
headers={"Idempotency-Key": key},
)
self.assertEqual(third.status, 201, third.json)
self.assertEqual(third.json["chat_id"], other_scope["id"])
def test_concurrent_identical_replays_return_one_message_without_500(self) -> None:
alice = common.actor("idemrace")
chat = common.create_chat(alice.token, [alice.id], "Concurrent replay")
key = uuid.uuid4().hex
workers = 32
barrier = threading.Barrier(workers)
responses: list[common.Response] = []
lock = threading.Lock()
def post() -> None:
barrier.wait()
response = common.request(
"POST",
"/api/v1/chats/" + chat["id"] + "/messages",
body={"text": "exactly once"},
token=alice.token,
headers={"Idempotency-Key": key},
timeout=30,
)
with lock:
responses.append(response)
threads = [threading.Thread(target=post) for _ in range(workers)]
for thread in threads:
thread.start()
for thread in threads:
thread.join(timeout=60)
self.assertEqual(len(responses), workers)
self.assertTrue(all(response.status == 201 for response in responses), responses)
message_ids = {response.json["id"] for response in responses}
self.assertEqual(len(message_ids), 1)
messages = common.history(alice.token, chat["id"])
self.assertEqual([message["text"] for message in messages], ["exactly once"])
+142
View File
@@ -0,0 +1,142 @@
"""Race, rollback and pool tests against the real PostgreSQL stand.
These are the checks a code reviewer would ask for on top of the smoke test:
concurrent writers must not lose or duplicate messages, the username uniqueness
must be decided by the database (not by a read-then-write check in Python), and
a failed composite operation must leave nothing behind.
"""
from __future__ import annotations
import threading
import time
import unittest
import uuid
import common
class ConcurrentWrites(unittest.TestCase):
def test_100_messages_from_10_clients_are_exactly_100(self) -> None:
alice = common.actor("cwalice")
bob = common.actor("cwbob")
chat = common.create_chat(alice.token, [alice.id, bob.id], "Гонка сообщений")
writers = 10
per_writer = 10
barrier = threading.Barrier(writers)
errors: list[str] = []
sent: list[str] = []
lock = threading.Lock()
def writer(index: int) -> None:
token = alice.token if index % 2 else bob.token
barrier.wait()
for number in range(per_writer):
text = "race-%d-%d" % (index, number)
try:
common.send(token, chat["id"], text)
with lock:
sent.append(text)
except AssertionError as error: # pragma: no cover - failure detail
with lock:
errors.append(str(error))
threads = [threading.Thread(target=writer, args=(index,)) for index in range(writers)]
started = time.monotonic()
for thread in threads:
thread.start()
for thread in threads:
thread.join(timeout=120)
elapsed = time.monotonic() - started
self.assertEqual(errors, [])
self.assertEqual(len(sent), writers * per_writer)
messages = common.history(alice.token, chat["id"], limit=100)
self.assertEqual(len(messages), writers * per_writer)
self.assertEqual(len({message["id"] for message in messages}), len(messages))
self.assertEqual(sorted(message["text"] for message in messages), sorted(sent))
print("\n concurrent writes: %d messages from %d clients in %.2fs"
% (len(messages), writers, elapsed))
def test_duplicate_username_race_creates_one_user(self) -> None:
registration = common.new_registration("race")
attempts = 12
barrier = threading.Barrier(attempts)
statuses: list[int] = []
lock = threading.Lock()
def attempt() -> None:
barrier.wait()
response = common.register(dict(registration))
with lock:
statuses.append(response.status)
threads = [threading.Thread(target=attempt) for _ in range(attempts)]
for thread in threads:
thread.start()
for thread in threads:
thread.join(timeout=60)
self.assertEqual(statuses.count(201), 1, statuses)
self.assertEqual(statuses.count(409), attempts - 1, statuses)
# The single winner is usable, so exactly one row exists.
session = common.login(registration["username"], registration["password"])
self.assertEqual(session.status, 201)
self.assertEqual(session.json["user"]["username"], registration["username"])
def test_chat_creation_rolls_back_on_unknown_member(self) -> None:
alice = common.actor("rollback")
before = self._chat_titles(alice.token)
response = common.request(
"POST",
"/api/v1/chats",
body={"title": "Атомарный чат", "member_ids": [alice.id, str(uuid.uuid4())]},
token=alice.token,
)
common.expect_error(self, response, 400, "invalid_request")
after = self._chat_titles(alice.token)
# The chat row inserted before the failing membership step is gone.
self.assertEqual(before, after)
self.assertNotIn("Атомарный чат", after)
def _chat_titles(self, token: str) -> list[str]:
titles: list[str] = []
cursor = None
for _ in range(50):
path = "/api/v1/chats?limit=100"
if cursor:
path += "&cursor=" + cursor
response = common.request("GET", path, token=token)
self.assertEqual(response.status, 200)
titles.extend(item["title"] for item in response.json["items"])
cursor = response.json["next_cursor"]
if cursor is None:
return titles
self.fail("chat pagination did not terminate")
def test_bounded_pool_serves_parallel_reads(self) -> None:
alice = common.actor("pool")
parallel = 50
barrier = threading.Barrier(parallel)
statuses: list[int] = []
lock = threading.Lock()
def read() -> None:
barrier.wait()
# Chat lists are not cached, so this exercises the bounded SQL pool.
response = common.request(
"GET", "/api/v1/chats?limit=100", token=alice.token, timeout=10
)
with lock:
statuses.append(response.status)
threads = [threading.Thread(target=read) for _ in range(parallel)]
for thread in threads:
thread.start()
for thread in threads:
thread.join(timeout=60)
self.assertEqual(len(statuses), parallel)
self.assertEqual(statuses, [200] * parallel)
if __name__ == "__main__":
unittest.main()
+2 -2
View File
@@ -13,8 +13,8 @@ import urllib.request
import uuid
ROOT = Path(__file__).resolve().parents[1]
LAB = json.loads((ROOT / 'lab.json').read_text())['number']
SPEC = json.loads((ROOT / 'contracts/openapi.json').read_text())
LAB = json.loads((ROOT / 'lab.json').read_text(encoding='utf-8'))['number']
SPEC = json.loads((ROOT / 'contracts/openapi.json').read_text(encoding='utf-8'))
class Failure(AssertionError):
pass