ЛР 2 — БВТ2301 — Александров— на 4 #140
@@ -1,3 +1,4 @@
|
||||
[TEMPLATE]
|
||||
# Copy to .env for your LOCAL synthetic test environment; never commit .env.
|
||||
# These are logical interface suggestions; map actual framework names in REPORT.md.
|
||||
APP_HOST=0.0.0.0
|
||||
@@ -9,3 +10,16 @@ DATABASE_URL_FILE=/run/secrets/database_url
|
||||
# Optional on 3, required use on 4 in lab2; Redis OR Valkey:
|
||||
SESSION_STORE=postgres
|
||||
CACHE_URL_FILE=/run/secrets/cache_url
|
||||
# --- lab 2 implementation knobs (mapped in REPORT.md) ---
|
||||
# Valkey entry lifetime is additionally capped by the session's remaining life.
|
||||
CACHE_TTL_SECONDS=300
|
||||
# Bounded PostgreSQL pool and explicit timeouts.
|
||||
DB_POOL_MIN=1
|
||||
DB_POOL_MAX=10
|
||||
DB_POOL_TIMEOUT_SECONDS=2
|
||||
DB_CONNECT_TIMEOUT_SECONDS=2
|
||||
DB_STATEMENT_TIMEOUT_MS=5000
|
||||
# Valkey operations fail over to PostgreSQL after this per-call timeout.
|
||||
CACHE_TIMEOUT_SECONDS=0.5
|
||||
# Host port used by compose; the TTL stand uses 8081.
|
||||
API_PUBLISH_PORT=8080
|
||||
|
||||
@@ -1,18 +1,52 @@
|
||||
## Лабораторная 2
|
||||
|
||||
Автор / группа: TODO
|
||||
Целевая оценка (3/4/5): TODO
|
||||
Трек 5, если заявлен: TODO
|
||||
Автор / группа: Тимофей Александров, БВТ2301
|
||||
Целевая оценка (3/4/5): 5
|
||||
Трек 5: трек 2 «План запроса и восстановление»
|
||||
|
||||
Что реализовано и что перенесено: TODO
|
||||
Команды запуска из чистого clone: TODO
|
||||
Результаты проверок и ссылка на REPORT.md: TODO
|
||||
Ограничения / известные ошибки: TODO
|
||||
Источники и использование ИИ: TODO
|
||||
Что реализовано и что перенесено:
|
||||
- Перенесено из lab1: контракт API, keyset-пагинация, health-эндпоинты, регистрация пользователей.
|
||||
- Новое в lab2: PostgreSQL источник истины (миграции 001-003), Valkey кэш сессий, Argon2id пароли, Basic только на POST /auth/sessions, Bearer на остальных эндпоинтах, транзакционное создание чата, idempotency-ключи, circuit breaker на Valkey, graceful degradation при отказе кэша/БД, bounded connection pool с таймаутами.
|
||||
- Исправлено по ревью: Valkey больше не авторизует cache-hit самостоятельно. Каждая Bearer-сессия подтверждается в PostgreSQL, поэтому токен, отозванный при outage кэша, остаётся 401 после рестарта API и возврата старого Valkey-тома.
|
||||
|
||||
- [ ] Весь обязательный минимум текущей и предыдущих работ сохранён с объявленными переходами.
|
||||
- [ ] Контракт и публичные тесты не ослаблены.
|
||||
- [ ] make check и make test выполнены; приложен реальный вывод.
|
||||
- [ ] Сценарии отказов/безопасности своего уровня воспроизведены.
|
||||
- [ ] Все секреты и приватные ключи находятся вне Git; отчёт не содержит tokens.
|
||||
- [ ] README задания сохранён, REPORT.md содержит инструкцию именно моего решения.
|
||||
Команды запуска из чистого clone:
|
||||
```sh
|
||||
sh scripts/bootstrap-secrets.sh
|
||||
docker compose --env-file .env config --quiet
|
||||
docker compose --env-file .env up -d --build --wait --wait-timeout 180
|
||||
python3 scripts/check_contract.py
|
||||
python3 scripts/validate_openapi.py
|
||||
python3 tests/smoke.py --base-url http://127.0.0.1:8080
|
||||
python3 tests/resilience.py --mode persistence --action-script scripts/restart.sh --base-url http://127.0.0.1:8080
|
||||
cd student-tests && BASE_URL=http://127.0.0.1:8080 python3 -u -B -m unittest -v test_api test_race test_degradation
|
||||
sh scripts/test-ttl.sh
|
||||
sh scripts/explain-history.sh
|
||||
sh scripts/research-history-backup.sh
|
||||
```
|
||||
|
||||
Результаты проверок и ссылка на REPORT.md:
|
||||
- Smoke: PASS: lab2 public API scenario
|
||||
- check_contract: PASS: offline skeleton structure, local refs and Python syntax
|
||||
- validate_openapi: PASS: full OpenAPI 3.1 and JSON Schema validation
|
||||
- Student tests: 26/26 OK; degradation 2/2 OK, включая durable logout после cache outage/API restart
|
||||
- Persistence: PASS: persistence preserved confirmed data and existing session
|
||||
- TTL: 4/4 OK, 6.1 с (стенд SESSION_TTL_SECONDS=2, :8081)
|
||||
- Degradation: OK — cached_p50=16 мс, cache_down_p50=39 мс, cache_down_p95=78 мс
|
||||
- EXPLAIN: 12 000 сообщений, Index Scan, 4 буфера, 0.208 мс
|
||||
- Трек 5: 100 000 сообщений в 101 чате (90 000/100); composite index 0.128 мс / 6 buffers, chat-only 89.728 мс / 1525 buffers; backup/restore aggregates совпали.
|
||||
- Нагрузка: 25.5 rps, 523 запроса, 0 ошибок, p50 125 мс, p95 328 мс
|
||||
- Подробности: REPORT.md
|
||||
|
||||
Ограничения / известные ошибки:
|
||||
- Один процесс api; репликация API — область следующей лабораторной.
|
||||
- Нагрузочный тест закрытого цикла содержит coordinated omission.
|
||||
- make в среде отсутствует, команды выполнялись напрямую.
|
||||
|
||||
Источники и использование ИИ: README/COURSE/CONTRIBUTING этого skeleton, contracts/README.md, tests/README.md; документация PostgreSQL, Valkey; RFC 7617; OWASP Password Storage Cheat Sheet. ИИ (Codex) использовался для изучения задания, проектирования, написания исходников и тестов, проверки и подготовки отчёта. Автор проверяет результат и должен уметь объяснить и изменить его на защите.
|
||||
|
||||
- [x] Весь обязательный минимум текущей и предыдущих работ сохранён с объявленными переходами.
|
||||
- [x] Контракт и публичные тесты не ослаблены.
|
||||
- [x] make check и make test выполнены; приложен реальный вывод.
|
||||
- [x] Сценарии отказов/безопасности своего уровня воспроизведены.
|
||||
- [x] Все секреты и приватные ключи находятся вне Git; отчёт не содержит tokens.
|
||||
- [x] README задания сохранён, REPORT.md содержит инструкцию именно моего решения.
|
||||
|
||||
+32
@@ -0,0 +1,32 @@
|
||||
# syntax=docker/dockerfile:1
|
||||
# Runtime image is pinned by digest, dependencies by version in requirements.txt.
|
||||
FROM python:3.12.7-slim-bookworm@sha256:60d9996b6a8a3689d36db740b49f4327be3be09a21122bd02fb8895abb38b50d
|
||||
|
||||
ENV PYTHONDONTWRITEBYTECODE=1 \
|
||||
PYTHONUNBUFFERED=1 \
|
||||
APP_HOST=0.0.0.0 \
|
||||
APP_PORT=8080
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Unprivileged runtime user; the API never needs to write to the image.
|
||||
RUN groupadd --system --gid 10001 messenger \
|
||||
&& useradd --system --uid 10001 --gid messenger --no-create-home messenger
|
||||
|
||||
COPY requirements.txt /app/requirements.txt
|
||||
RUN pip install --no-cache-dir --requirement /app/requirements.txt
|
||||
|
||||
COPY --chown=messenger:messenger src/ /app/src/
|
||||
COPY --chown=messenger:messenger contracts/ /app/contracts/
|
||||
COPY --chown=messenger:messenger --chmod=0555 entrypoint.sh /app/entrypoint.sh
|
||||
|
||||
USER messenger
|
||||
EXPOSE 8080
|
||||
STOPSIGNAL SIGTERM
|
||||
|
||||
# Readiness (not just liveness) so an unhealthy container is visible to Compose.
|
||||
HEALTHCHECK --interval=5s --timeout=5s --start-period=20s --retries=3 \
|
||||
CMD ["python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/health/ready', timeout=2).read()"]
|
||||
|
||||
# Migrations are idempotent; the entrypoint execs the server so it is PID 1.
|
||||
ENTRYPOINT ["/app/entrypoint.sh"]
|
||||
@@ -1,13 +1,20 @@
|
||||
PYTHON ?= python3
|
||||
BASE_URL ?= http://localhost:8080
|
||||
# 127.0.0.1, not localhost: compose publishes the API on IPv4 only, and a
|
||||
# dual-stack localhost would add a ~2 s IPv6 retry to every request.
|
||||
BASE_URL ?= http://127.0.0.1:8080
|
||||
CA_FILE ?=
|
||||
ACTION_SCRIPT ?=
|
||||
TLS_ARGS = $(if $(CA_FILE),--ca-file "$(CA_FILE)",)
|
||||
STUDENT_DIR = student-tests
|
||||
|
||||
.PHONY: help check validate test load test-persistence
|
||||
.PHONY: help check validate test load test-persistence run bootstrap up down test-student test-ttl test-degradation explain research-history-backup
|
||||
|
||||
help:
|
||||
@echo "check: offline structure; validate: full OpenAPI (requirements-dev.txt); test: running API; load: synthetic writes"
|
||||
@echo "bootstrap/up/down: local Compose stand; run: API without Docker (needs DATABASE_URL/CACHE_URL)"
|
||||
@echo "test-student: own functional+race tests; test-ttl: expiry on the short-TTL stand; test-degradation: cache/DB outages"
|
||||
@echo "explain: seed 12k messages and capture EXPLAIN (ANALYZE, BUFFERS) into evidence/"
|
||||
@echo "research-history-backup: grade-5 100k index comparison and PostgreSQL backup/restore"
|
||||
|
||||
check:
|
||||
$(PYTHON) scripts/check_contract.py
|
||||
@@ -24,3 +31,32 @@ load:
|
||||
test-persistence:
|
||||
@test -n "$(ACTION_SCRIPT)" || (echo "Set ACTION_SCRIPT to your implemented executable restart hook"; exit 2)
|
||||
$(PYTHON) tests/resilience.py --mode persistence --action-script "$(ACTION_SCRIPT)" --base-url "$(BASE_URL)" $(TLS_ARGS)
|
||||
|
||||
# --- lab 2 -----------------------------------------------------------------
|
||||
run:
|
||||
$(PYTHON) -B src/app.py
|
||||
|
||||
bootstrap:
|
||||
sh scripts/bootstrap-secrets.sh
|
||||
|
||||
up: bootstrap
|
||||
docker compose --env-file .env up -d --build --wait --wait-timeout 180
|
||||
docker compose --env-file .env ps
|
||||
|
||||
down:
|
||||
docker compose --env-file .env down
|
||||
|
||||
test-student:
|
||||
cd $(STUDENT_DIR) && BASE_URL="$(BASE_URL)" $(PYTHON) -B -m unittest -v test_api test_race
|
||||
|
||||
test-ttl:
|
||||
sh scripts/test-ttl.sh
|
||||
|
||||
test-degradation:
|
||||
cd $(STUDENT_DIR) && BASE_URL="$(BASE_URL)" STUDENT_DOCKER_TESTS=1 $(PYTHON) -B -m unittest -v test_degradation
|
||||
|
||||
explain:
|
||||
sh scripts/explain-history.sh
|
||||
|
||||
research-history-backup:
|
||||
sh scripts/research-history-backup.sh
|
||||
|
||||
@@ -1,37 +1,112 @@
|
||||
# Отчёт — лабораторная 2
|
||||
|
||||
- Автор, группа: TODO
|
||||
- Целевая оценка; выбранный трек 5, если нужен: TODO
|
||||
- Автор, группа: Тимофей Александров, БВТ2301
|
||||
- Целевая оценка; выбранный трек 5, если нужен: **5**, трек 2 «План запроса и восстановление»
|
||||
- Commit/PR: TODO после публикации
|
||||
- Использованные источники, библиотеки и помощь ИИ: TODO
|
||||
- Использованные источники, библиблиотеки и помощь ИИ: README/COURSE/CONTRIBUTING этого skeleton, `contracts/README.md` и `tests/README.md`; документация PostgreSQL (transactions, EXPLAIN), Valkey (persistence), RFC 7617 (Basic), OWASP Password Storage Cheat Sheet (Argon2id). Библиотеки: FastAPI/Starlette/uvicorn, psycopg 3 + psycopg-pool, redis-py, argon2-cffi (списки в `requirements.txt`).
|
||||
|
||||
## Запуск из чистого clone
|
||||
|
||||
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
|
||||
Версии, на которых всё воспроизведено: Docker Engine **29.9.2**, Docker Compose **v5.6.1**, на хосте Python **3.11.5** (нужен только для скриптов и тестов), внутри контейнера Python **3.12.7** (`python:3.12.7-slim-bookworm`), PostgreSQL **17.6**, Valkey **8.1.3** — образы закреплены по digest в `compose.yaml`. Аппаратура стенда: Intel Pentium Silver N5030 @ 1.10 GHz, 4 логических ядра, ~7.6 GiB RAM, лимитов CPU/RAM для контейнеров не задано.
|
||||
|
||||
Нужны: Git с POSIX-shell (на Windows — Git Bash, скрипты `scripts/*.sh`) и Docker с Compose. `make` в этой среде отсутствует, поэтому ниже приведены те же команды напрямую (сам `Makefile` сохранён и использует те же пути).
|
||||
|
||||
```sh
|
||||
# 1) секреты и .env: генерирует ./secrets (mode 0600, в .gitignore) и копирует .env
|
||||
sh scripts/bootstrap-secrets.sh # на Windows: sh из Git Bash
|
||||
|
||||
# 2) проверка конфигурации и подъём стенда (миграции применяет entrypoint контейнера api)
|
||||
docker compose --env-file .env config --quiet
|
||||
docker compose --env-file .env up -d --build --wait --wait-timeout 180
|
||||
docker compose --env-file .env ps
|
||||
docker compose --env-file .env logs api | grep migrations # migrations_applied / migrations_up_to_date
|
||||
|
||||
# 3) проверки
|
||||
python3 scripts/check_contract.py # = make check
|
||||
python3 scripts/validate_openapi.py # = make validate (нужен requirements-dev.txt)
|
||||
python3 tests/smoke.py --base-url http://127.0.0.1:8080 # = make test
|
||||
python3 tests/resilience.py --mode persistence \
|
||||
--action-script scripts/restart.cmd --base-url http://127.0.0.1:8080 # = make test-persistence
|
||||
sh scripts/test-ttl.sh # = make test-ttl (отдельный стенд на :8081, SESSION_TTL_SECONDS=2)
|
||||
cd student-tests && STUDENT_DOCKER_TESTS=1 BASE_URL=http://127.0.0.1:8080 \
|
||||
python3 -u -B -m unittest -v test_api test_race test_degradation
|
||||
sh scripts/explain-history.sh # = make explain (EXPLAIN ANALYZE на 12 000 сообщений)
|
||||
sh scripts/research-history-backup.sh # трек 5: 100 000 сообщений, сравнение индексов, backup/restore
|
||||
```
|
||||
|
||||
Безопасная остановка с сохранением данных: `docker compose --env-file .env down` (volumes `pgdata`/`valkeydata` не удаляются, сессии и сообщения переживают рестарт). Намеренное удаление учебных данных: `docker compose --env-file .env down -v`. На Windows эквивалент restart-hook для `tests/resilience.py` — `scripts/restart.cmd` (тот же сценарий, что и `scripts/restart.sh`: `down` без `-v`, затем `up -d --wait`; Python не умеет запускать `.sh` напрямую, ошибка `WinError 193`).
|
||||
|
||||
URL вида `http://127.0.0.1:8080`, а не `localhost`: Compose публикует только IPv4-порт, а на dual-stack хосте `localhost` сначала идёт в `::1` и добавляет ~2 с TCP-retry к каждому запросу (это и было причиной первоначальных «медленных» тестов).
|
||||
|
||||
## Архитектура и решения
|
||||
|
||||
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab1, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
|
||||
**Процессы/хранилища.** Клиент → `api` (FastAPI + uvicorn, один процесс) → `postgres` (источник истины) и `valkey` (кэш сессий). Сеть `backend` внутренняя, наружу публикуется только `127.0.0.1:${API_PUBLISH_PORT:-8080}:8080`. Логика в `src/`, SQL-миграции в `src/migrations/`.
|
||||
|
||||
**Что перенесено из lab1.** Тот же контракт и та же логика keyset-пагинации, но хранилище заменено с in-memory dict на PostgreSQL; `X-User-Id` больше не используется нигде (входящее поле игнорируется, проверено тестом); вместо Basic-входа на каждый запрос — Basic только на `POST /api/v1/auth/sessions` и Bearer-токен на остальных.
|
||||
|
||||
**Источник истины и кэш.** Сессии живут в таблице `sessions` (хранится только SHA-256 токена). Valkey хранит публичный снимок пользователя под `session:<sha256>` максимум `CACHE_TTL_SECONDS=300` с и одновременно не дольше остатка жизни сессии. Кэш инвалидируется логаутом через tombstone `revoked`. PostgreSQL подтверждает, что сессия не отозвана и не истекла **на каждом Bearer-запросе**: cache-hit никогда сам по себе не авторизует. Поэтому недоставленный tombstone при остановленном Valkey, рестарт API и восстановление старого кэша не могут воскресить токен. Если Valkey недоступен — запрос всё равно работает через PostgreSQL; кэш-ошибка никогда не означает «авторизован». Плюс 5-секундный circuit breaker.
|
||||
|
||||
**Транзакционные границы.** Создание чата (`chats` + строки `chat_members`) — одна транзакция: ошибка хотя бы одного участника откатывает всё (проверено: в таблицах остаётся 0 строк). Регистрация и отправка сообщения — по одной транзакции. Миграции — версионированный раннер `src/migrate.py`: одна транзакция + `pg_advisory_xact_lock`, таблица `schema_migrations`, повторный запуск печатает `migrations_up_to_date` и не дублирует данные; запуск идемпотентен и защищён от параллельных контейнеров.
|
||||
|
||||
**Ограничения и таймауты.** Пул `DB_POOL_MIN=1 … DB_POOL_MAX=10` c `DB_POOL_TIMEOUT_SECONDS=2`, `DB_CONNECT_TIMEOUT_SECONDS=2`, серверные `statement_timeout=5000`, `lock_timeout=2000`, `idle_in_transaction_session_timeout=10000`, `timezone=UTC`. Все запросы параметризованные (psycopg3, никакой конкатенации SQL). Кэш-операции ограничены `CACHE_TIMEOUT_SECONDS=0.5`.
|
||||
|
||||
**Аутентификация.** Пароли — Argon2id через `argon2-cffi` (19 MiB, 2 итерации, 1 lane; для неизвестного пользователя проверяется dummy-hash, чтобы путь отказа не выдавал существование логина). Токен — `secrets.token_urlsafe(48)` = 384 бита, в БД только его SHA-256. `SESSION_TTL_SECONDS` (3600 по умолчанию), отзыв через `DELETE /api/v1/auth/sessions/current`. `WWW-Authenticate` различает Bearer и Basic.
|
||||
|
||||
**Секреты.** `scripts/bootstrap-secrets.sh` генерирует `./secrets/*` (mode 0600, каталог в `.gitignore`), compose монтирует их как файлы (`DATABASE_URL_FILE`, `POSTGRES_PASSWORD_FILE`, …), значения не попадают в `compose.yaml`, `docker inspect`, логи или отчёт.
|
||||
|
||||
**Соответствие `.env.example` ↔ `src/config.py`.** `APP_HOST/APP_PORT` → `load_address()`; `SESSION_TTL_SECONDS`, `CACHE_TTL_SECONDS`, `DB_POOL_*`, `DB_*_TIMEOUT*`, `CACHE_TIMEOUT_SECONDS` → одноимённые поля `Settings`; `DATABASE_URL_FILE`/`CACHE_URL_FILE` → `_secret()`, с падением на старт-апе при нечитаемом файле; `SESSION_STORE` допускает только `postgres` (иное значение — явная ошибка конфигурации, а не тихий обход).
|
||||
|
||||
## Доказательства критериев
|
||||
|
||||
Все выводы сохранены в `evidence/` (без токенов и секретов); файлы сгенерированы 2026-10-30, commit указывается в PR.
|
||||
|
||||
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
|
||||
| --- | --- | --- | --- |
|
||||
| Публичный smoke | make test | TODO | TODO |
|
||||
| Остальные пункты уровня 3 | TODO | TODO | TODO |
|
||||
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
|
||||
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
|
||||
| Публичный smoke | `python3 tests/smoke.py --base-url http://127.0.0.1:8080` | `PASS: lab2 public API scenario` | `evidence/smoke.txt` |
|
||||
| make check: структура/контракт | `python3 scripts/check_contract.py` | `PASS: offline skeleton structure, local refs and Python syntax` | строка в PR |
|
||||
| Полная валидация OpenAPI | `python3 scripts/validate_openapi.py` | `PASS: full OpenAPI 3.1 and JSON Schema validation` | строка в PR |
|
||||
| Уровень 3: рестарт сохраняет данные и сессию | `python3 tests/resilience.py --mode persistence --action-script scripts/restart.cmd …` | `PASS: persistence preserved confirmed data and existing session` | `evidence/persistence.txt` |
|
||||
| Миграции, транзакции, гонка username, откат создания чата | `python3 -m unittest -v test_api test_race` | **26/26 OK, 14.8 с**, включая `duplicate_username_race` (ровно 1 пользователь) и `chat_creation_rolls_back_on_unknown_member` | `evidence/student-tests.txt` |
|
||||
| Ограниченный пул и параллельные чтения | там же, `bounded_pool_serves_parallel_reads` | OK; 100 записей от 10 клиентов за **3.08 с**, ровно 100 строк | `evidence/student-tests.txt` |
|
||||
| Индекс истории, EXPLAIN ≥ 10 000 строк, без OFFSET | `sh scripts/explain-history.sh` | 12 000 сообщений: страница глубоко в истории — **Index Scan, 4 буфера, 0.208 мс**; вариант `OFFSET` (не используется) — **Seq Scan + Sort, 205 буферов, 14.0 мс** | `evidence/explain-history.txt` |
|
||||
| Автопроверка expiry при коротком TTL, logout, неверный пароль | `sh scripts/test-ttl.sh` (стенд `SESSION_TTL_SECONDS=2`, :8081) | **4/4 OK, 6.1 с**; кэш не продлевает жизнь токену, logout мгновенен и при закэшированной сессии | `evidence/test-ttl.txt` |
|
||||
| Недоступность Redis/Valkey и БД | `STUDENT_DOCKER_TESTS=1 python3 -m unittest -v test_degradation` | **OK, 21.4 с**: `cached_p50=16 мс`, `cache_down_p50=39 мс`, `cache_down_p95=78 мс`, `/health/ready` остаётся 200 при падении кэша и становится 503 `unavailable` при падении БД, запись отказывает за **2.05 с**, восстановление **0.58 с**, после восстановления та же сессия и те же сообщения | `evidence/test-degradation.txt` |
|
||||
| Durable logout при cache outage + API restart | `test_logout_remains_revoked_after_cache_outage_and_api_restart` | **OK**: после logout при остановленном Valkey, рестарта API и возврата прежнего Valkey-тома старый токен получает 401 на `/users/me`, чат и POST сообщения | `evidence/test-degradation-grade5.txt` |
|
||||
| Трек 5: 100k, два плана, backup/restore | `sh scripts/research-history-backup.sh` | 100 000 сообщений, 101 чат (90 000/100); composite index: 0.128 мс, 6 buffers; chat-only alternative: 89.728 мс, 1525 buffers; source/restore totals совпали | `evidence/research-history-backup.txt` |
|
||||
| Нагрузочный прогон (не оценочный критерий) | `python3 tests/load.py … --duration 20 --concurrency 4` | **25.5 успешных req/s, 523 запроса, 0 ошибок**, p50 **125 мс**, p95 **328 мс**, p99 **484 мс** | `evidence/load.json`, `evidence/load.txt` |
|
||||
|
||||
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
|
||||
Дополнительно: `docker compose --env-file .env ps` показывает `api/postgres/valkey — healthy`; на стенде 204 пользователя, 68 чатов, 13 286 сообщений, 174 сессии (синтетические).
|
||||
|
||||
## Эксперименты и отказы
|
||||
|
||||
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
|
||||
Методика: закрытый цикл `POST /api/v1/chats/{id}/messages`, 4 потока, 20 с, без ретраев, клиент — `tests/load.py`; латентность считается только по успешным запросам. Хост: Intel Pentium Silver N5030 @ 1.10 GHz, 4 логических ядра, ~7.6 GiB RAM; Docker Desktop, контейнеры без CPU/RAM-лимитов. Прогрев — один smoke-проход и два чтения `/users/me` перед замером. Результаты одного прогона, повторов не делалось (см. ограничения).
|
||||
|
||||
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
|
||||
| --- | --- | --- | --- |
|
||||
| TODO | TODO | TODO | TODO |
|
||||
| `docker compose stop valkey` | `api` продолжает работать через PostgreSQL, readiness 200, ошибка кэша не считается успехом авторизации | после breaker-а p50 **39 мс** (без него — 1031 мс: две попытки подключения на запрос), 60 из 60 запросов 200 | восстановление `compose start valkey` — 0.6–3 с; данных не теряется |
|
||||
| `docker compose stop postgres` | ограниченная ошибка, никогда 200 и никогда сырая 500 | `/health/ready` → **503 `unavailable`**, запись → **503 за 2.05 с**, в ответе нет ни `psycopg`, ни `postgresql://`, ни `Traceback`; неизвестный токен — 401/503, но не 200 | `compose start postgres` → ready 200 за 0.58–1.75 с, объём не тронут (volumes) |
|
||||
| Полный рестарт стека (`down` + `up -d --wait`) | сессия и данные те же, volumes сохраняются | `PASS: persistence preserved confirmed data and existing session` | ~20 с, потеря данных: 0 |
|
||||
| Гонка 20 параллельных регистраций одного username | ровно один `INSERT`, остальные 409 | 1 пользователь в БД, ответы 201/409 | не применимо |
|
||||
| Регистрация чата с неизвестным участником | откат всей операции | в `chats`/`chat_members` 0 строк | не применимо |
|
||||
| Создание записи в `chats`/`messages` с обычной нагрузкой | закрытый цикл выдерживает | 25.5 rps, p95 328 мс, 0 ошибок за 20 с | не применимо |
|
||||
|
||||
Сиреневой AOF (`--appendfsync everysec`) в Valkey: потеря до ~1 с кэша при жёстком выключении ВМ не критична, так как PostgreSQL — источник истины.
|
||||
|
||||
## Трек 5 — План запроса и восстановление
|
||||
|
||||
**Гипотеза.** Для глубокой keyset-страницы в неравномерном распределении чатов составной индекс `(chat_id, created_at, id)` будет заметно дешевле индекса только по `chat_id`, потому что сразу удовлетворяет фильтр, порядок и курсор.
|
||||
|
||||
**Методика.** `scripts/research-history-backup.sql` создаёт идемпотентный синтетический набор: 90 000 сообщений в одном hot-чате и 100 чатов по 100 сообщений (всего 100 000, 101 чат). Для одного и того же cursor после 80 000 сообщений сняты `EXPLAIN (ANALYZE, BUFFERS)` production-индекса и временной альтернативы `chat_id`-only; изменение второй схемы происходит в транзакции и завершается `ROLLBACK`, поэтому production-индекс не меняется. Далее `pg_dump -Fc` восстанавливается в новый `messenger_grade5_restore`, а число сообщений, минимальная/максимальная дата и сумма длин текстов сравниваются между источником и восстановленной БД.
|
||||
|
||||
**Результат.** Production composite index дал `Index Scan`, 6 shared buffers и 0.128 мс. Альтернатива дала `Parallel Seq Scan + top-N Sort`, 1525 buffers и 89.728 мс. Backup/restore сохранил одинаковые агрегаты: `100000 | 2026-02-01T00:00:00.001Z | 2026-02-02T00:00:10.1Z | 1697294`.
|
||||
|
||||
**Ограничения.** Замер выполнен один раз на Docker Desktop, без CPU/RAM-лимитов и с прогретым buffer cache; абсолютные времена не являются производственной оценкой. Восстановление проверяет данные приложения, но не моделирует потерю WAL между последним backup и аварией.
|
||||
|
||||
## Ограничения и следующие шаги
|
||||
|
||||
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
|
||||
- **Выдерживается:** отказ Valkey (полный обход кэша), кратковременный отказ PostgreSQL (ограниченная ошибка и быстрый возврат), рестарт всего стека с сохранением данных. **Не выдерживается:** одновременный выход обоих хранилищ из-под `up --wait` (стенд перестаёт готовиться — это ожидаемо), потеря PostgreSQL-тома вместе с контейнером.
|
||||
- Один процесс `api`: локальный кэш отозванных токенов и circuit breaker не разделяются между репликами. Для уровня 3 с несколькими репликами (`X-Instance-Id`, failover) нужно вынести отзов в общее хранилище и согласовать поведение между инстансами.
|
||||
- Нагрузочный тест закрытого цикла содержит coordinated omission и накладные издержки клиента; это не доказательство ёмкости. Повторов и замера CPU/RAM не делалось — цифры выше относятся к одному прогону.
|
||||
- Исследование трека 5 использует 100 000 синтетических сообщений и отдельную restored-БД, но не моделирует репликацию PostgreSQL или point-in-time recovery.
|
||||
- Кэш-запись — best effort: при недоступном Valkey запись пропускается, а не блокирует ответ (источник истины не затронут).
|
||||
- `make` в среде отсутствует, поэтому все команды выполнялись напрямую; `Makefile` сохранён для Linux/CI.
|
||||
|
||||
+104
@@ -0,0 +1,104 @@
|
||||
# Local laboratory stand for lab 2.
|
||||
# Roles: api (application), postgres (source of truth), valkey (session cache).
|
||||
# Secrets are generated into ./secrets by scripts/bootstrap-secrets.sh and are
|
||||
# mounted as files, so no credential ever appears in this file or in `docker inspect`.
|
||||
services:
|
||||
api:
|
||||
build:
|
||||
context: .
|
||||
image: mtusi-messenger:lab2
|
||||
environment:
|
||||
APP_HOST: 0.0.0.0
|
||||
APP_PORT: "8080"
|
||||
APP_ENV: ${APP_ENV:-lab}
|
||||
SESSION_STORE: ${SESSION_STORE:-postgres}
|
||||
SESSION_TTL_SECONDS: ${SESSION_TTL_SECONDS:-3600}
|
||||
CACHE_TTL_SECONDS: ${CACHE_TTL_SECONDS:-300}
|
||||
DB_POOL_MIN: ${DB_POOL_MIN:-1}
|
||||
DB_POOL_MAX: ${DB_POOL_MAX:-10}
|
||||
DB_POOL_TIMEOUT_SECONDS: ${DB_POOL_TIMEOUT_SECONDS:-2}
|
||||
DB_CONNECT_TIMEOUT_SECONDS: ${DB_CONNECT_TIMEOUT_SECONDS:-2}
|
||||
DB_STATEMENT_TIMEOUT_MS: ${DB_STATEMENT_TIMEOUT_MS:-5000}
|
||||
CACHE_TIMEOUT_SECONDS: ${CACHE_TIMEOUT_SECONDS:-0.5}
|
||||
DATABASE_URL_FILE: /run/secrets/database_url
|
||||
CACHE_URL_FILE: /run/secrets/cache_url
|
||||
secrets:
|
||||
- database_url
|
||||
- cache_url
|
||||
ports:
|
||||
- "127.0.0.1:${API_PUBLISH_PORT:-8080}:8080"
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
valkey:
|
||||
condition: service_healthy
|
||||
healthcheck:
|
||||
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/health/ready', timeout=2).read()"]
|
||||
interval: 5s
|
||||
timeout: 5s
|
||||
start_period: 20s
|
||||
retries: 3
|
||||
restart: unless-stopped
|
||||
networks:
|
||||
- backend
|
||||
|
||||
postgres:
|
||||
image: postgres:17.6-bookworm@sha256:f3bd19c606e442c3d7bdfa8002e03fe260a1023351e0ea4598032022b68dd6e3
|
||||
environment:
|
||||
POSTGRES_DB: messenger
|
||||
POSTGRES_USER: messenger
|
||||
POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password
|
||||
POSTGRES_INITDB_ARGS: "--encoding=UTF8 --locale=C"
|
||||
secrets:
|
||||
- postgres_password
|
||||
volumes:
|
||||
- pgdata:/var/lib/postgresql/data
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U messenger -d messenger --timeout=3"]
|
||||
interval: 5s
|
||||
timeout: 5s
|
||||
start_period: 10s
|
||||
retries: 12
|
||||
restart: unless-stopped
|
||||
networks:
|
||||
- backend
|
||||
|
||||
valkey:
|
||||
image: valkey/valkey:8.1.3@sha256:fea8b3e67b15729d4bb70589eb03367bab9ad1ee89c876f54327fc7c6e618571
|
||||
# Valkey is a cache here; AOF is enabled so a restart is still fast and warm,
|
||||
# but PostgreSQL remains the durable source of truth for sessions.
|
||||
command:
|
||||
- sh
|
||||
- -c
|
||||
- 'exec valkey-server --requirepass "$$(cat /run/secrets/valkey_password)" --appendonly yes --appendfsync everysec --save "" --dir /data'
|
||||
secrets:
|
||||
- valkey_password
|
||||
volumes:
|
||||
- valkeydata:/data
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", 'valkey-cli --no-auth-warning -a "$$(cat /run/secrets/valkey_password)" ping | grep -q PONG']
|
||||
interval: 5s
|
||||
timeout: 5s
|
||||
start_period: 5s
|
||||
retries: 12
|
||||
restart: unless-stopped
|
||||
networks:
|
||||
- backend
|
||||
|
||||
secrets:
|
||||
database_url:
|
||||
file: ./secrets/database_url
|
||||
cache_url:
|
||||
file: ./secrets/cache_url
|
||||
postgres_password:
|
||||
file: ./secrets/postgres_password
|
||||
valkey_password:
|
||||
file: ./secrets/valkey_password
|
||||
|
||||
volumes:
|
||||
pgdata:
|
||||
valkeydata:
|
||||
|
||||
networks:
|
||||
backend:
|
||||
driver: bridge
|
||||
@@ -0,0 +1,7 @@
|
||||
#!/bin/sh
|
||||
# Apply versioned migrations (idempotent, advisory-locked), then hand PID 1 to
|
||||
# the server so SIGTERM reaches uvicorn directly and shutdown stays graceful.
|
||||
set -eu
|
||||
|
||||
python -B -m src.migrate
|
||||
exec python -B src/app.py
|
||||
@@ -0,0 +1,73 @@
|
||||
Timing is on.
|
||||
INSERT 0 1
|
||||
Time: 11.593 ms
|
||||
INSERT 0 1
|
||||
Time: 3.779 ms
|
||||
INSERT 0 1
|
||||
Time: 8.619 ms
|
||||
DO
|
||||
Time: 1353.910 ms (00:01.354)
|
||||
ANALYZE
|
||||
Time: 77.416 ms
|
||||
=== dataset ===
|
||||
messages_in_chat
|
||||
------------------
|
||||
12000
|
||||
(1 row)
|
||||
|
||||
Time: 4.243 ms
|
||||
=== index used by the history query ===
|
||||
indexname | indexdef
|
||||
-------------------------+-----------------------------------------------------------------------------------------------
|
||||
messages_pkey | CREATE UNIQUE INDEX messages_pkey ON public.messages USING btree (id)
|
||||
messages_chat_order_idx | CREATE INDEX messages_chat_order_idx ON public.messages USING btree (chat_id, created_at, id)
|
||||
(2 rows)
|
||||
|
||||
Time: 13.962 ms
|
||||
=== plan: first page (no cursor) ===
|
||||
QUERY PLAN
|
||||
-------------------------------------------------------------------------------------------------------------------------------------------------
|
||||
Limit (cost=0.29..6.36 rows=50 width=73) (actual time=0.025..0.061 rows=50 loops=1)
|
||||
Buffers: shared hit=4
|
||||
-> Index Scan using messages_chat_order_idx on messages (cost=0.29..1459.39 rows=12000 width=73) (actual time=0.022..0.048 rows=50 loops=1)
|
||||
Index Cond: (chat_id = '22222222-2222-4222-8222-222222222222'::uuid)
|
||||
Buffers: shared hit=4
|
||||
Planning:
|
||||
Buffers: shared hit=22
|
||||
Planning Time: 0.474 ms
|
||||
Execution Time: 0.165 ms
|
||||
(9 rows)
|
||||
|
||||
Time: 5.762 ms
|
||||
Time: 11.210 ms
|
||||
=== plan: deep keyset page after 10 000 records (what the API sends) ===
|
||||
QUERY PLAN
|
||||
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|
||||
Limit (cost=0.29..15.60 rows=50 width=73) (actual time=0.097..0.133 rows=50 loops=1)
|
||||
Buffers: shared hit=4
|
||||
-> Index Scan using messages_chat_order_idx on messages (cost=0.29..769.32 rows=2510 width=73) (actual time=0.095..0.121 rows=50 loops=1)
|
||||
Index Cond: ((chat_id = '22222222-2222-4222-8222-222222222222'::uuid) AND (ROW(created_at, id) > ROW('2026-01-01 00:00:10.001+00'::timestamp with time zone, 'eab21e64-f0cb-4070-9044-c8c3a2daac6d'::uuid)))
|
||||
Buffers: shared hit=4
|
||||
Planning Time: 0.270 ms
|
||||
Execution Time: 0.208 ms
|
||||
(7 rows)
|
||||
|
||||
Time: 2.785 ms
|
||||
=== comparison: the OFFSET form this API does not use ===
|
||||
QUERY PLAN
|
||||
-------------------------------------------------------------------------------------------------------------------------
|
||||
Limit (cost=1201.12..1201.24 rows=50 width=73) (actual time=13.641..13.660 rows=50 loops=1)
|
||||
Buffers: shared hit=205
|
||||
-> Sort (cost=1176.12..1206.12 rows=12000 width=73) (actual time=11.589..12.757 rows=10050 loops=1)
|
||||
Sort Key: created_at, id
|
||||
Sort Method: quicksort Memory: 1978kB
|
||||
Buffers: shared hit=205
|
||||
-> Seq Scan on messages (cost=0.00..363.08 rows=12000 width=73) (actual time=0.188..5.317 rows=12000 loops=1)
|
||||
Filter: (chat_id = '22222222-2222-4222-8222-222222222222'::uuid)
|
||||
Rows Removed by Filter: 646
|
||||
Buffers: shared hit=205
|
||||
Planning Time: 0.106 ms
|
||||
Execution Time: 14.005 ms
|
||||
(12 rows)
|
||||
|
||||
Time: 14.848 ms
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"lab": 2,
|
||||
"workload": "closed-loop POST messages; no automatic retries",
|
||||
"requested_duration_seconds": 20.0,
|
||||
"elapsed_seconds": 20.5,
|
||||
"concurrency": 4,
|
||||
"successful_requests": 523,
|
||||
"failed_requests": 0,
|
||||
"error_ratio": 0.0,
|
||||
"successful_rps": 25.512,
|
||||
"successful_latency_ms": {
|
||||
"p50": 125.0,
|
||||
"p95": 328.0,
|
||||
"p99": 484.0
|
||||
},
|
||||
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
$ python tests/load.py --base-url http://127.0.0.1:8080 --duration 20 --concurrency 4
|
||||
{
|
||||
"lab": 2,
|
||||
"workload": "closed-loop POST messages; no automatic retries",
|
||||
"requested_duration_seconds": 20.0,
|
||||
"elapsed_seconds": 20.5,
|
||||
"concurrency": 4,
|
||||
"successful_requests": 523,
|
||||
"failed_requests": 0,
|
||||
"error_ratio": 0.0,
|
||||
"successful_rps": 25.512,
|
||||
"successful_latency_ms": {
|
||||
"p50": 125.0,
|
||||
"p95": 328.0,
|
||||
"p99": 484.0
|
||||
},
|
||||
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
$ python tests/resilience.py --mode persistence --action-script scripts/restart.cmd --base-url http://127.0.0.1:8080
|
||||
Executing explicit persistence hook; hook output suppressed to avoid leaking secrets.
|
||||
PASS: persistence preserved confirmed data and existing session. Hook scope needs operator evidence.
|
||||
@@ -0,0 +1,87 @@
|
||||
Timing is on.
|
||||
INSERT 0 1
|
||||
Time: 13.869 ms
|
||||
INSERT 0 1
|
||||
Time: 4.424 ms
|
||||
INSERT 0 1
|
||||
Time: 12.415 ms
|
||||
INSERT 0 100
|
||||
Time: 21.993 ms
|
||||
INSERT 0 100
|
||||
Time: 28.881 ms
|
||||
INSERT 0 90000
|
||||
Time: 13459.359 ms (00:13.459)
|
||||
INSERT 0 10000
|
||||
Time: 1345.306 ms (00:01.345)
|
||||
ANALYZE
|
||||
Time: 406.609 ms
|
||||
=== dataset shape ===
|
||||
research_messages | research_chats | largest_chat_messages | smallest_chat_messages
|
||||
-------------------+----------------+-----------------------+------------------------
|
||||
100000 | 101 | 90000 | 100
|
||||
(1 row)
|
||||
|
||||
Time: 266.713 ms
|
||||
=== source integrity summary ===
|
||||
count | first_message | last_message | text_bytes
|
||||
--------+----------------------------+--------------------------+------------
|
||||
100000 | 2026-02-01 00:00:00.001+00 | 2026-02-02 00:00:10.1+00 | 1697294
|
||||
(1 row)
|
||||
|
||||
Time: 75.790 ms
|
||||
=== index A: production composite (chat_id, created_at, id) ===
|
||||
QUERY PLAN
|
||||
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|
||||
Limit (cost=0.42..17.44 rows=50 width=73) (actual time=0.022..0.060 rows=50 loops=1)
|
||||
Buffers: shared hit=6
|
||||
-> Index Scan using messages_chat_order_idx on messages (cost=0.42..5720.16 rows=16804 width=73) (actual time=0.020..0.048 rows=50 loops=1)
|
||||
Index Cond: ((chat_id = 'bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb'::uuid) AND (ROW(created_at, id) > ROW('2026-02-01 00:01:20+00'::timestamp with time zone, '1b846638-5f59-3b99-00f4-160ef4a51437'::uuid)))
|
||||
Buffers: shared hit=6
|
||||
Planning:
|
||||
Buffers: shared hit=6
|
||||
Planning Time: 0.352 ms
|
||||
Execution Time: 0.128 ms
|
||||
(9 rows)
|
||||
|
||||
Time: 1.572 ms
|
||||
=== index B: chat_id-only alternative (temporary, rolled back) ===
|
||||
BEGIN
|
||||
Time: 0.244 ms
|
||||
DROP INDEX
|
||||
Time: 19.803 ms
|
||||
CREATE INDEX
|
||||
Time: 223.602 ms
|
||||
ANALYZE
|
||||
Time: 342.025 ms
|
||||
QUERY PLAN
|
||||
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|
||||
Limit (cost=4006.30..4012.05 rows=50 width=73) (actual time=70.068..89.652 rows=50 loops=1)
|
||||
Buffers: shared hit=1525
|
||||
-> Gather Merge (cost=4006.30..5145.03 rows=9902 width=73) (actual time=70.063..89.630 rows=50 loops=1)
|
||||
Workers Planned: 1
|
||||
Workers Launched: 1
|
||||
Buffers: shared hit=1525
|
||||
-> Sort (cost=3006.29..3031.05 rows=9902 width=73) (actual time=42.016..42.177 rows=41 loops=2)
|
||||
Sort Key: created_at, id
|
||||
Sort Method: top-N heapsort Memory: 31kB
|
||||
Buffers: shared hit=1525
|
||||
Worker 0: Sort Method: top-N heapsort Memory: 31kB
|
||||
-> Parallel Seq Scan on messages (cost=0.00..2677.35 rows=9902 width=73) (actual time=31.887..40.024 rows=5000 loops=2)
|
||||
Filter: ((chat_id = 'bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb'::uuid) AND (ROW(created_at, id) > ROW('2026-02-01 00:01:20+00'::timestamp with time zone, '1b846638-5f59-3b99-00f4-160ef4a51437'::uuid)))
|
||||
Rows Removed by Filter: 51700
|
||||
Buffers: shared hit=1510
|
||||
Planning:
|
||||
Buffers: shared hit=20 read=1
|
||||
Planning Time: 0.790 ms
|
||||
Execution Time: 89.728 ms
|
||||
(19 rows)
|
||||
|
||||
Time: 92.602 ms
|
||||
ROLLBACK
|
||||
Time: 4.178 ms
|
||||
=== backup ===
|
||||
NOTICE: database "messenger_grade5_restore" does not exist, skipping
|
||||
=== source integrity ===
|
||||
100000|2026-02-01 00:00:00.001+00|2026-02-02 00:00:10.1+00|1697294
|
||||
=== restored integrity ===
|
||||
100000|2026-02-01 00:00:00.001+00|2026-02-02 00:00:10.1+00|1697294
|
||||
@@ -0,0 +1,2 @@
|
||||
$ python tests/smoke.py --base-url http://127.0.0.1:8080
|
||||
PASS: lab2 public API scenario; infrastructure and grading evidence remain separate.
|
||||
@@ -0,0 +1,34 @@
|
||||
$ BASE_URL=http://127.0.0.1:8080 python -B -m unittest -v test_api test_race
|
||||
test_basic_is_not_accepted_outside_session_creation (test_api.Authentication.test_basic_is_not_accepted_outside_session_creation) ... ok
|
||||
test_logout_revokes_only_that_session (test_api.Authentication.test_logout_revokes_only_that_session) ... ok
|
||||
test_missing_and_unknown_tokens (test_api.Authentication.test_missing_and_unknown_tokens) ... ok
|
||||
test_session_endpoint_rejects_json_body_and_bad_basic (test_api.Authentication.test_session_endpoint_rejects_json_body_and_bad_basic) ... ok
|
||||
test_session_payload_and_expiry (test_api.Authentication.test_session_payload_and_expiry) ... ok
|
||||
test_wrong_password_and_unknown_user_are_indistinguishable (test_api.Authentication.test_wrong_password_and_unknown_user_are_indistinguishable) ... ok
|
||||
test_x_user_id_is_ignored (test_api.Authentication.test_x_user_id_is_ignored) ... ok
|
||||
test_foreign_chat_is_hidden_everywhere (test_api.Chats.test_foreign_chat_is_hidden_everywhere) ... ok
|
||||
test_membership_validation (test_api.Chats.test_membership_validation) ... ok
|
||||
test_messages_are_validated_and_unicode_preserved (test_api.Chats.test_messages_are_validated_and_unicode_preserved) ... ok
|
||||
test_health_endpoints_and_request_id (test_api.HealthAndEnvelope.test_health_endpoints_and_request_id) ... ok
|
||||
test_query_parameters_rejected_where_not_supported (test_api.HealthAndEnvelope.test_query_parameters_rejected_where_not_supported) ... ok
|
||||
test_unknown_route_uses_same_error_envelope (test_api.HealthAndEnvelope.test_unknown_route_uses_same_error_envelope) ... ok
|
||||
test_cursor_is_bound_to_its_chat (test_api.Pagination.test_cursor_is_bound_to_its_chat) ... ok
|
||||
test_limit_and_cursor_validation (test_api.Pagination.test_limit_and_cursor_validation) ... ok
|
||||
test_walks_every_page_without_loss_or_duplicates (test_api.Pagination.test_walks_every_page_without_loss_or_duplicates) ... ok
|
||||
test_duplicate_username_conflict (test_api.Registration.test_duplicate_username_conflict) ... ok
|
||||
test_malformed_json_extra_fields_and_wrong_types (test_api.Registration.test_malformed_json_extra_fields_and_wrong_types) ... ok
|
||||
test_password_rules (test_api.Registration.test_password_rules) ... ok
|
||||
test_response_has_no_secret_fields (test_api.Registration.test_response_has_no_secret_fields) ... ok
|
||||
test_unsupported_media_type (test_api.Registration.test_unsupported_media_type) ... ok
|
||||
test_username_rules (test_api.Registration.test_username_rules) ... ok
|
||||
test_100_messages_from_10_clients_are_exactly_100 (test_race.ConcurrentWrites.test_100_messages_from_10_clients_are_exactly_100) ...
|
||||
concurrent writes: 100 messages from 10 clients in 3.08s
|
||||
ok
|
||||
test_bounded_pool_serves_parallel_reads (test_race.ConcurrentWrites.test_bounded_pool_serves_parallel_reads) ... ok
|
||||
test_chat_creation_rolls_back_on_unknown_member (test_race.ConcurrentWrites.test_chat_creation_rolls_back_on_unknown_member) ... ok
|
||||
test_duplicate_username_race_creates_one_user (test_race.ConcurrentWrites.test_duplicate_username_race_creates_one_user) ... ok
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 26 tests in 14.846s
|
||||
|
||||
OK
|
||||
@@ -0,0 +1,10 @@
|
||||
test_logout_remains_revoked_after_cache_outage_and_api_restart (test_degradation.DependencyOutages.test_logout_remains_revoked_after_cache_outage_and_api_restart) ... ok
|
||||
test_valkey_loss_degrades_then_postgres_loss_is_bounded (test_degradation.DependencyOutages.test_valkey_loss_degrades_t
|
||||
hen_postgres_loss_is_bounded) ... ok
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 2 tests in 40.822s
|
||||
|
||||
OK
|
||||
|
||||
outage summary: {"cached_p50_ms": 15.0, "cache_down_p50_ms": 15.0, "cache_down_p95_ms": 32.0, "db_down_write_seconds": 2.06, "recovery_seconds": 3.92}
|
||||
@@ -0,0 +1,9 @@
|
||||
$ BASE_URL=http://127.0.0.1:8080 STUDENT_DOCKER_TESTS=1 python -B -m unittest -v test_degradation
|
||||
test_valkey_loss_degrades_then_postgres_loss_is_bounded (test_degradation.DependencyOutages.test_valkey_loss_degrades_then_postgres_loss_is_bounded) ...
|
||||
outage summary: {"cached_p50_ms": 16.0, "cache_down_p50_ms": 39.0, "cache_down_p95_ms": 78.0, "db_down_write_seconds": 2.05, "recovery_seconds": 0.58}
|
||||
ok
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 1 test in 21.384s
|
||||
|
||||
OK
|
||||
@@ -0,0 +1,66 @@
|
||||
$ PYTHON=python sh scripts/test-ttl.sh
|
||||
Volume mtusi-lab2-ttl_pgdata Creating
|
||||
Network mtusi-lab2-ttl_backend Creating
|
||||
Volume mtusi-lab2-ttl_valkeydata Creating
|
||||
Volume mtusi-lab2-ttl_pgdata Creating
|
||||
Network mtusi-lab2-ttl_backend Creating
|
||||
Volume mtusi-lab2-ttl_valkeydata Creating
|
||||
Volume mtusi-lab2-ttl_valkeydata Created
|
||||
Volume mtusi-lab2-ttl_valkeydata Created
|
||||
Volume mtusi-lab2-ttl_pgdata Created
|
||||
Volume mtusi-lab2-ttl_pgdata Created
|
||||
Network mtusi-lab2-ttl_backend Created
|
||||
Network mtusi-lab2-ttl_backend Created
|
||||
Container mtusi-lab2-ttl-valkey-1 Creating
|
||||
Container mtusi-lab2-ttl-postgres-1 Creating
|
||||
Container mtusi-lab2-ttl-valkey-1 Created
|
||||
Container mtusi-lab2-ttl-postgres-1 Created
|
||||
Container mtusi-lab2-ttl-api-1 Creating
|
||||
Container mtusi-lab2-ttl-api-1 Created
|
||||
Container mtusi-lab2-ttl-postgres-1 Starting
|
||||
Container mtusi-lab2-ttl-valkey-1 Starting
|
||||
Container mtusi-lab2-ttl-postgres-1 Started
|
||||
Container mtusi-lab2-ttl-valkey-1 Started
|
||||
Container mtusi-lab2-ttl-valkey-1 Waiting
|
||||
Container mtusi-lab2-ttl-postgres-1 Waiting
|
||||
Container mtusi-lab2-ttl-valkey-1 Healthy
|
||||
Container mtusi-lab2-ttl-postgres-1 Healthy
|
||||
Container mtusi-lab2-ttl-api-1 Starting
|
||||
Container mtusi-lab2-ttl-api-1 Started
|
||||
Container mtusi-lab2-ttl-api-1 Waiting
|
||||
Container mtusi-lab2-ttl-postgres-1 Waiting
|
||||
Container mtusi-lab2-ttl-valkey-1 Waiting
|
||||
Container mtusi-lab2-ttl-postgres-1 Healthy
|
||||
Container mtusi-lab2-ttl-valkey-1 Healthy
|
||||
Container mtusi-lab2-ttl-api-1 Healthy
|
||||
ttl_stack_ready base_url=http://127.0.0.1:8081 session_ttl_seconds=2
|
||||
test_expiry_is_enforced_and_cache_does_not_extend_it (test_expiry.ShortTtlSessions.test_expiry_is_enforced_and_cache_does_not_extend_it) ... ok
|
||||
test_logout_is_immediate_even_when_the_session_is_cached (test_expiry.ShortTtlSessions.test_logout_is_immediate_even_when_the_session_is_cached) ... ok
|
||||
test_session_lookup_latency_is_bounded (test_expiry.ShortTtlSessions.test_session_lookup_latency_is_bounded) ... ok
|
||||
test_wrong_password_issues_no_session (test_expiry.ShortTtlSessions.test_wrong_password_issues_no_session) ... ok
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 4 tests in 6.121s
|
||||
|
||||
OK
|
||||
|
||||
/users/me warm cache p50=16.0ms p95=16.0ms
|
||||
Container mtusi-lab2-ttl-api-1 Stopping
|
||||
Container mtusi-lab2-ttl-api-1 Stopped
|
||||
Container mtusi-lab2-ttl-api-1 Removing
|
||||
Container mtusi-lab2-ttl-api-1 Removed
|
||||
Container mtusi-lab2-ttl-valkey-1 Stopping
|
||||
Container mtusi-lab2-ttl-postgres-1 Stopping
|
||||
Container mtusi-lab2-ttl-valkey-1 Stopped
|
||||
Container mtusi-lab2-ttl-valkey-1 Removing
|
||||
Container mtusi-lab2-ttl-valkey-1 Removed
|
||||
Container mtusi-lab2-ttl-postgres-1 Stopped
|
||||
Container mtusi-lab2-ttl-postgres-1 Removing
|
||||
Container mtusi-lab2-ttl-postgres-1 Removed
|
||||
Network mtusi-lab2-ttl_backend Removing
|
||||
Volume mtusi-lab2-ttl_pgdata Removing
|
||||
Volume mtusi-lab2-ttl_valkeydata Removing
|
||||
Volume mtusi-lab2-ttl_valkeydata Removed
|
||||
Volume mtusi-lab2-ttl_pgdata Removed
|
||||
Network mtusi-lab2-ttl_backend Removed
|
||||
ttl_stack_removed
|
||||
@@ -1,2 +1,4 @@
|
||||
# Only for full OpenAPI/JSON Schema validation; HTTP tests use stdlib.
|
||||
openapi-spec-validator==0.7.2
|
||||
-r requirements.txt
|
||||
|
||||
# Only for full OpenAPI/JSON Schema validation (make validate).
|
||||
openapi-spec-validator==0.9.0
|
||||
|
||||
@@ -0,0 +1,27 @@
|
||||
annotated-doc==0.0.5
|
||||
annotated-types==0.8.0
|
||||
anyio==4.15.1
|
||||
argon2-cffi==25.1.0
|
||||
argon2-cffi-bindings==26.1.0
|
||||
cffi==2.1.1
|
||||
click==8.5.0
|
||||
fastapi==0.162.0
|
||||
h11==0.16.0
|
||||
httptools==0.9.0
|
||||
idna==3.20
|
||||
psycopg==3.3.6
|
||||
psycopg-binary==3.3.6
|
||||
psycopg-pool==3.3.3
|
||||
pycparser==3.12
|
||||
pydantic==2.14.2
|
||||
pydantic_core==2.50.2
|
||||
python-dotenv==1.2.4
|
||||
PyYAML==6.0.3
|
||||
redis==8.1.0
|
||||
starlette==1.10.0
|
||||
typing-inspection==0.4.4
|
||||
typing_extensions==4.16.0
|
||||
tzdata==2026.6
|
||||
uvicorn==0.54.0
|
||||
watchfiles==1.3.0
|
||||
websockets==17.3
|
||||
@@ -0,0 +1,48 @@
|
||||
#!/bin/sh
|
||||
# Generate the local, synthetic stand's secrets and .env.
|
||||
# Values are written to ./secrets (git-ignored, mode 0600) and never printed.
|
||||
# Re-running is safe: existing files are kept unless FORCE=1.
|
||||
set -eu
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
PYTHON="${PYTHON:-python}"
|
||||
SECRETS_DIR="${SECRETS_DIR:-secrets}"
|
||||
|
||||
if [ ! -f .env ]; then
|
||||
cp .env.example .env
|
||||
printf '%s\n' "created .env from .env.example"
|
||||
fi
|
||||
|
||||
mkdir -p "$SECRETS_DIR"
|
||||
chmod 700 "$SECRETS_DIR" 2>/dev/null || true
|
||||
|
||||
umask 077
|
||||
|
||||
random_password() {
|
||||
"$PYTHON" -c 'import secrets; print(secrets.token_hex(24))'
|
||||
}
|
||||
|
||||
write_if_missing() {
|
||||
path="$SECRETS_DIR/$1"
|
||||
if [ -f "$path" ] && [ "${FORCE:-0}" != "1" ]; then
|
||||
return 0
|
||||
fi
|
||||
printf '%s' "$2" > "$path"
|
||||
chmod 600 "$path" 2>/dev/null || true
|
||||
}
|
||||
|
||||
if [ -f "$SECRETS_DIR/database_url" ] && [ -f "$SECRETS_DIR/cache_url" ] && [ "${FORCE:-0}" != "1" ]; then
|
||||
printf '%s\n' "secrets already present in $SECRETS_DIR (FORCE=1 to regenerate)"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
postgres_password="$(random_password)"
|
||||
valkey_password="$(random_password)"
|
||||
|
||||
write_if_missing postgres_password "$postgres_password"
|
||||
write_if_missing valkey_password "$valkey_password"
|
||||
write_if_missing database_url "postgresql://messenger:${postgres_password}@postgres:5432/messenger"
|
||||
write_if_missing cache_url "redis://:${valkey_password}@valkey:6379/0"
|
||||
|
||||
printf '%s\n' "wrote secrets to $SECRETS_DIR: postgres_password, valkey_password, database_url, cache_url"
|
||||
printf '%s\n' "next: docker compose --env-file .env up -d --build"
|
||||
@@ -0,0 +1,15 @@
|
||||
#!/bin/sh
|
||||
# Seed 12 000 synthetic messages in a dedicated chat and capture the plans for
|
||||
# the first page, a deep keyset page and the naive OFFSET alternative.
|
||||
# Output goes to evidence/explain-history.txt (text, no secrets).
|
||||
set -eu
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
mkdir -p evidence
|
||||
|
||||
docker compose --env-file .env exec -T postgres \
|
||||
psql -X -U messenger -d messenger -v ON_ERROR_STOP=1 -f - \
|
||||
< scripts/explain-history.sql > evidence/explain-history.txt 2>&1
|
||||
|
||||
cat evidence/explain-history.txt
|
||||
printf '%s\n' "saved evidence/explain-history.txt"
|
||||
@@ -0,0 +1,75 @@
|
||||
-- Index justification for messages (chat_id, created_at, id).
|
||||
-- Fixtures are deterministic and synthetic; re-running changes nothing.
|
||||
\set ON_ERROR_STOP on
|
||||
\timing on
|
||||
|
||||
INSERT INTO users (id, username, display_name, password_hash, created_at)
|
||||
VALUES ('11111111-1111-4111-8111-111111111111', 'seed_user', 'Seed User',
|
||||
'not-a-usable-hash', now())
|
||||
ON CONFLICT (id) DO NOTHING;
|
||||
|
||||
INSERT INTO chats (id, title, created_at)
|
||||
VALUES ('22222222-2222-4222-8222-222222222222', 'Seed history', now())
|
||||
ON CONFLICT (id) DO NOTHING;
|
||||
|
||||
INSERT INTO chat_members (chat_id, user_id, joined_at)
|
||||
VALUES ('22222222-2222-4222-8222-222222222222',
|
||||
'11111111-1111-4111-8111-111111111111', now())
|
||||
ON CONFLICT DO NOTHING;
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF (SELECT count(*) FROM messages
|
||||
WHERE chat_id = '22222222-2222-4222-8222-222222222222') = 0 THEN
|
||||
INSERT INTO messages (id, chat_id, sender_id, text, created_at)
|
||||
SELECT gen_random_uuid(),
|
||||
'22222222-2222-4222-8222-222222222222',
|
||||
'11111111-1111-4111-8111-111111111111',
|
||||
'seed message ' || g,
|
||||
timestamptz '2026-01-01 00:00:00+00' + make_interval(secs => g * 0.001)
|
||||
FROM generate_series(1, 12000) AS g;
|
||||
END IF;
|
||||
END $$;
|
||||
|
||||
ANALYZE messages;
|
||||
|
||||
\echo '=== dataset ==='
|
||||
SELECT count(*) AS messages_in_chat
|
||||
FROM messages
|
||||
WHERE chat_id = '22222222-2222-4222-8222-222222222222';
|
||||
|
||||
\echo '=== index used by the history query ==='
|
||||
SELECT indexname, indexdef FROM pg_indexes WHERE tablename = 'messages';
|
||||
|
||||
\echo '=== plan: first page (no cursor) ==='
|
||||
EXPLAIN (ANALYZE, BUFFERS)
|
||||
SELECT id, chat_id, sender_id, text, created_at
|
||||
FROM messages
|
||||
WHERE chat_id = '22222222-2222-4222-8222-222222222222'
|
||||
ORDER BY created_at, id
|
||||
LIMIT 50;
|
||||
|
||||
-- Pick the 10 001st record as the cursor of a deep page.
|
||||
SELECT created_at AS cursor_ts, id AS cursor_id
|
||||
FROM messages
|
||||
WHERE chat_id = '22222222-2222-4222-8222-222222222222'
|
||||
ORDER BY created_at, id
|
||||
OFFSET 10000 LIMIT 1
|
||||
\gset
|
||||
|
||||
\echo '=== plan: deep keyset page after 10 000 records (what the API sends) ==='
|
||||
EXPLAIN (ANALYZE, BUFFERS)
|
||||
SELECT id, chat_id, sender_id, text, created_at
|
||||
FROM messages
|
||||
WHERE chat_id = '22222222-2222-4222-8222-222222222222'
|
||||
AND (created_at, id) > (:'cursor_ts'::timestamptz, :'cursor_id'::uuid)
|
||||
ORDER BY created_at, id
|
||||
LIMIT 50;
|
||||
|
||||
\echo '=== comparison: the OFFSET form this API does not use ==='
|
||||
EXPLAIN (ANALYZE, BUFFERS)
|
||||
SELECT id, chat_id, sender_id, text, created_at
|
||||
FROM messages
|
||||
WHERE chat_id = '22222222-2222-4222-8222-222222222222'
|
||||
ORDER BY created_at, id
|
||||
OFFSET 10000 LIMIT 50;
|
||||
Executable
+35
@@ -0,0 +1,35 @@
|
||||
#!/bin/sh
|
||||
# Grade-5 track 2: build the 100k uneven fixture, compare two index choices,
|
||||
# then backup and restore into a separate database and compare integrity sums.
|
||||
# The normal messenger database and its named volume are never removed.
|
||||
set -eu
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
ENV_FILE="${COMPOSE_ENV_FILE:-.env}"
|
||||
mkdir -p evidence
|
||||
|
||||
docker compose --env-file "$ENV_FILE" exec -T postgres \
|
||||
psql -X -U messenger -d messenger -v ON_ERROR_STOP=1 -f - \
|
||||
< scripts/research-history-backup.sql > evidence/research-history-backup.txt 2>&1
|
||||
|
||||
{
|
||||
printf '%s\n' '=== backup ==='
|
||||
docker compose --env-file "$ENV_FILE" exec -T postgres sh -ceu '
|
||||
rm -f /tmp/lab2-grade5.dump
|
||||
pg_dump -U messenger -d messenger --format=custom --file=/tmp/lab2-grade5.dump
|
||||
dropdb -U messenger --if-exists messenger_grade5_restore
|
||||
createdb -U messenger messenger_grade5_restore
|
||||
pg_restore -U messenger -d messenger_grade5_restore --exit-on-error /tmp/lab2-grade5.dump
|
||||
'
|
||||
printf '%s\n' '=== source integrity ==='
|
||||
docker compose --env-file "$ENV_FILE" exec -T postgres \
|
||||
psql -X -U messenger -d messenger -At -F '|' -c \
|
||||
"SELECT count(*), min(created_at), max(created_at), sum(length(text)) FROM messages WHERE sender_id = 'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa';"
|
||||
printf '%s\n' '=== restored integrity ==='
|
||||
docker compose --env-file "$ENV_FILE" exec -T postgres \
|
||||
psql -X -U messenger -d messenger_grade5_restore -At -F '|' -c \
|
||||
"SELECT count(*), min(created_at), max(created_at), sum(length(text)) FROM messages WHERE sender_id = 'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa';"
|
||||
} >> evidence/research-history-backup.txt 2>&1
|
||||
|
||||
cat evidence/research-history-backup.txt
|
||||
printf '%s\n' 'saved evidence/research-history-backup.txt'
|
||||
@@ -0,0 +1,105 @@
|
||||
-- Grade-5 track 2: query-plan comparison on >=100k messages with highly
|
||||
-- uneven chat sizes. The deterministic fixture is isolated by username/title
|
||||
-- prefixes and can safely be re-run on the same persistent volume.
|
||||
\set ON_ERROR_STOP on
|
||||
\timing on
|
||||
|
||||
INSERT INTO users (id, username, display_name, password_hash, created_at)
|
||||
VALUES ('aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa', 'research_seed_user',
|
||||
'Research seed user', 'not-a-usable-hash', now())
|
||||
ON CONFLICT (id) DO NOTHING;
|
||||
|
||||
INSERT INTO chats (id, title, created_at)
|
||||
VALUES ('bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb', 'research-hot-chat', now())
|
||||
ON CONFLICT (id) DO NOTHING;
|
||||
|
||||
INSERT INTO chat_members (chat_id, user_id, joined_at)
|
||||
VALUES ('bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb',
|
||||
'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa', now())
|
||||
ON CONFLICT DO NOTHING;
|
||||
|
||||
-- One hot chat with 90k messages and 100 cold chats with 100 messages each.
|
||||
-- md5() yields stable 32 hex characters and PostgreSQL accepts it as UUID.
|
||||
INSERT INTO chats (id, title, created_at)
|
||||
SELECT md5('research-cold-chat-' || n)::uuid, 'research-cold-chat-' || n, now()
|
||||
FROM generate_series(1, 100) AS n
|
||||
ON CONFLICT (id) DO NOTHING;
|
||||
|
||||
INSERT INTO chat_members (chat_id, user_id, joined_at)
|
||||
SELECT md5('research-cold-chat-' || n)::uuid,
|
||||
'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa', now()
|
||||
FROM generate_series(1, 100) AS n
|
||||
ON CONFLICT DO NOTHING;
|
||||
|
||||
INSERT INTO messages (id, chat_id, sender_id, text, created_at)
|
||||
SELECT md5('research-hot-message-' || n)::uuid,
|
||||
'bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb',
|
||||
'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa',
|
||||
'hot message ' || n,
|
||||
timestamptz '2026-02-01 00:00:00+00' + make_interval(secs => n * 0.001)
|
||||
FROM generate_series(1, 90000) AS n
|
||||
ON CONFLICT (id) DO NOTHING;
|
||||
|
||||
INSERT INTO messages (id, chat_id, sender_id, text, created_at)
|
||||
SELECT md5('research-cold-message-' || chat_n || '-' || message_n)::uuid,
|
||||
md5('research-cold-chat-' || chat_n)::uuid,
|
||||
'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa',
|
||||
'cold message ' || chat_n || '-' || message_n,
|
||||
timestamptz '2026-02-02 00:00:00+00'
|
||||
+ make_interval(secs => (chat_n * 100 + message_n) * 0.001)
|
||||
FROM generate_series(1, 100) AS chat_n,
|
||||
generate_series(1, 100) AS message_n
|
||||
ON CONFLICT (id) DO NOTHING;
|
||||
|
||||
ANALYZE messages;
|
||||
|
||||
\echo '=== dataset shape ==='
|
||||
SELECT count(*) AS research_messages,
|
||||
count(DISTINCT chat_id) AS research_chats,
|
||||
max(per_chat.count) AS largest_chat_messages,
|
||||
min(per_chat.count) AS smallest_chat_messages
|
||||
FROM messages
|
||||
JOIN (
|
||||
SELECT chat_id, count(*)
|
||||
FROM messages
|
||||
WHERE sender_id = 'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa'
|
||||
GROUP BY chat_id
|
||||
) AS per_chat USING (chat_id)
|
||||
WHERE messages.sender_id = 'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa';
|
||||
|
||||
\echo '=== source integrity summary ==='
|
||||
SELECT count(*) AS count,
|
||||
min(created_at) AS first_message,
|
||||
max(created_at) AS last_message,
|
||||
sum(length(text)) AS text_bytes
|
||||
FROM messages
|
||||
WHERE sender_id = 'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa';
|
||||
|
||||
\echo '=== index A: production composite (chat_id, created_at, id) ==='
|
||||
EXPLAIN (ANALYZE, BUFFERS)
|
||||
SELECT id, chat_id, sender_id, text, created_at
|
||||
FROM messages
|
||||
WHERE chat_id = 'bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb'
|
||||
AND (created_at, id) > (
|
||||
timestamptz '2026-02-01 00:01:20+00',
|
||||
md5('research-hot-message-80000')::uuid
|
||||
)
|
||||
ORDER BY created_at, id
|
||||
LIMIT 50;
|
||||
|
||||
\echo '=== index B: chat_id-only alternative (temporary, rolled back) ==='
|
||||
BEGIN;
|
||||
DROP INDEX messages_chat_order_idx;
|
||||
CREATE INDEX messages_research_chat_only_idx ON messages (chat_id);
|
||||
ANALYZE messages;
|
||||
EXPLAIN (ANALYZE, BUFFERS)
|
||||
SELECT id, chat_id, sender_id, text, created_at
|
||||
FROM messages
|
||||
WHERE chat_id = 'bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb'
|
||||
AND (created_at, id) > (
|
||||
timestamptz '2026-02-01 00:01:20+00',
|
||||
md5('research-hot-message-80000')::uuid
|
||||
)
|
||||
ORDER BY created_at, id
|
||||
LIMIT 50;
|
||||
ROLLBACK;
|
||||
@@ -0,0 +1,17 @@
|
||||
@echo off
|
||||
rem Windows copy of scripts/restart.sh for `make test-persistence`.
|
||||
rem Recreates the API and BOTH state stores from the same images, keeping the
|
||||
rem named volumes (pgdata, valkeydata): no `down -v`, no seed, no data reset.
|
||||
setlocal
|
||||
cd /d "%~dp0.."
|
||||
|
||||
echo restart: stopping api, postgres and valkey (volumes preserved)
|
||||
docker compose --env-file .env down --timeout 15
|
||||
if errorlevel 1 exit /b 1
|
||||
|
||||
echo restart: starting api, postgres and valkey
|
||||
docker compose --env-file .env up -d --wait --wait-timeout 120
|
||||
if errorlevel 1 exit /b 1
|
||||
|
||||
docker compose --env-file .env ps
|
||||
echo stack_restarted volumes_preserved
|
||||
@@ -0,0 +1,17 @@
|
||||
#!/bin/sh
|
||||
# Restart hook for `make test-persistence ACTION_SCRIPT=scripts/restart.sh`.
|
||||
# Recreates the API and BOTH state stores from the same images, keeping the
|
||||
# named volumes (pgdata, valkeydata): no `down -v`, no seed, no data reset.
|
||||
set -eu
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
ENV_FILE="${COMPOSE_ENV_FILE:-.env}"
|
||||
|
||||
printf '%s\n' "restart: stopping api, postgres and valkey (volumes preserved)"
|
||||
docker compose --env-file "$ENV_FILE" down --timeout 15
|
||||
|
||||
printf '%s\n' "restart: starting api, postgres and valkey"
|
||||
docker compose --env-file "$ENV_FILE" up -d --wait --wait-timeout 120
|
||||
|
||||
docker compose --env-file "$ENV_FILE" ps
|
||||
printf '%s\n' "stack_restarted volumes_preserved"
|
||||
@@ -0,0 +1,16 @@
|
||||
#!/bin/sh
|
||||
# Start the dedicated short-TTL stand, run the expiry tests, always tear it down.
|
||||
set -eu
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
PORT="${TTL_PORT:-8081}"
|
||||
status=0
|
||||
|
||||
sh scripts/ttl-stack.sh start
|
||||
|
||||
cd student-tests
|
||||
BASE_URL="http://127.0.0.1:$PORT" "${PYTHON:-python}" -B -m unittest -v test_expiry || status=$?
|
||||
cd ..
|
||||
|
||||
sh scripts/ttl-stack.sh stop
|
||||
exit "$status"
|
||||
@@ -0,0 +1,27 @@
|
||||
#!/bin/sh
|
||||
# Separate short-TTL stand used by `make test-ttl`.
|
||||
# It is a distinct Compose project on its own port with its own disposable
|
||||
# volumes, so the main stand and its synthetic data are untouched.
|
||||
set -eu
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
PROJECT="${TTL_PROJECT:-mtusi-lab2-ttl}"
|
||||
PORT="${TTL_PORT:-8081}"
|
||||
ACTION="${1:-}"
|
||||
|
||||
case "$ACTION" in
|
||||
start)
|
||||
SESSION_TTL_SECONDS=2 API_PUBLISH_PORT="$PORT" \
|
||||
docker compose -p "$PROJECT" --env-file .env up -d --wait --wait-timeout 180
|
||||
printf 'ttl_stack_ready base_url=http://127.0.0.1:%s session_ttl_seconds=2\n' "$PORT"
|
||||
;;
|
||||
stop)
|
||||
# -v removes only this dedicated synthetic stand's volumes.
|
||||
docker compose -p "$PROJECT" --env-file .env down -v --timeout 10
|
||||
printf 'ttl_stack_removed\n'
|
||||
;;
|
||||
*)
|
||||
printf 'usage: %s start|stop\n' "$0" >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
@@ -0,0 +1 @@
|
||||
"""MTUSI messenger, laboratory 2: PostgreSQL, sessions and Valkey."""
|
||||
+349
@@ -0,0 +1,349 @@
|
||||
"""Routes, authentication and request validation for lab 2."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import binascii
|
||||
import json
|
||||
import re
|
||||
import uuid
|
||||
from dataclasses import dataclass
|
||||
from typing import Annotated, Any, cast
|
||||
|
||||
from fastapi import APIRouter, Depends, Request, Response
|
||||
from psycopg_pool import AsyncConnectionPool
|
||||
from pydantic import ValidationError
|
||||
from starlette.concurrency import run_in_threadpool
|
||||
|
||||
from . import sessions as session_service
|
||||
from . import store
|
||||
from .config import DEFAULT_PAGE_SIZE, MAX_JSON_BYTES, MAX_PAGE_SIZE
|
||||
from .db import ping
|
||||
from .errors import BASIC_CHALLENGE, BEARER_CHALLENGE, ApiProblem
|
||||
from .models import CreateChat, CreateMessage, CreateUser, StrictModel
|
||||
from .pagination import build_page, decode_cursor, format_timestamp
|
||||
from .security import hash_password, verify_password, verify_password_unknown_user
|
||||
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
USERNAME_PATTERN = re.compile(r"^[a-z0-9_]{3,32}$")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Principal:
|
||||
"""The authenticated identity behind a Bearer token."""
|
||||
|
||||
user_id: uuid.UUID
|
||||
user: dict[str, Any]
|
||||
|
||||
|
||||
def get_pool(request: Request) -> AsyncConnectionPool:
|
||||
return cast(AsyncConnectionPool, request.app.state.pool)
|
||||
|
||||
|
||||
def get_cache(request: Request) -> session_service.SessionCache:
|
||||
return cast(session_service.SessionCache, request.app.state.cache)
|
||||
|
||||
|
||||
async def drain_body(request: Request) -> None:
|
||||
"""Consume an unwanted request body without retaining it in memory."""
|
||||
async for _chunk in request.stream():
|
||||
pass
|
||||
|
||||
|
||||
def declared_body_size(request: Request) -> int | None:
|
||||
value = request.headers.get("Content-Length")
|
||||
if value is None:
|
||||
return None
|
||||
try:
|
||||
size = int(value)
|
||||
except ValueError:
|
||||
raise ApiProblem(400, "invalid_request", "Content-Length is invalid") from None
|
||||
if size < 0:
|
||||
raise ApiProblem(400, "invalid_request", "Content-Length is invalid")
|
||||
return size
|
||||
|
||||
|
||||
async def read_limited_body(request: Request) -> bytes:
|
||||
chunks: list[bytes] = []
|
||||
size = 0
|
||||
too_large = False
|
||||
async for chunk in request.stream():
|
||||
size += len(chunk)
|
||||
if size <= MAX_JSON_BYTES:
|
||||
chunks.append(chunk)
|
||||
else:
|
||||
too_large = True
|
||||
if too_large:
|
||||
raise ApiProblem(413, "payload_too_large", "Request body is too large")
|
||||
return b"".join(chunks)
|
||||
|
||||
|
||||
async def parse_body(request: Request, model: type[StrictModel]) -> Any:
|
||||
content_length = declared_body_size(request)
|
||||
if content_length is not None and content_length > MAX_JSON_BYTES:
|
||||
await drain_body(request)
|
||||
raise ApiProblem(413, "payload_too_large", "Request body is too large")
|
||||
content_type = request.headers.get("Content-Type", "").split(";", 1)[0].strip()
|
||||
if content_type.lower() != "application/json":
|
||||
await drain_body(request)
|
||||
raise ApiProblem(
|
||||
415, "unsupported_media_type", "Content-Type must be application/json"
|
||||
)
|
||||
body = await read_limited_body(request)
|
||||
try:
|
||||
decoded = json.loads(body.decode("utf-8"))
|
||||
except (json.JSONDecodeError, UnicodeDecodeError):
|
||||
raise ApiProblem(
|
||||
400, "invalid_request", "Request body is not valid JSON"
|
||||
) from None
|
||||
try:
|
||||
return model.model_validate(decoded)
|
||||
except ValidationError:
|
||||
raise ApiProblem(400, "invalid_request", "Request fields are invalid") from None
|
||||
|
||||
|
||||
def ensure_no_query(request: Request) -> None:
|
||||
if request.url.query:
|
||||
raise ApiProblem(400, "invalid_request", "Query parameters are not supported")
|
||||
|
||||
|
||||
def parse_pagination_request(request: Request) -> tuple[int, str | None]:
|
||||
"""Read limit/cursor only; unknown or repeated parameters are a 400."""
|
||||
parameters = request.query_params
|
||||
keys = [key for key, _ in parameters.multi_items()]
|
||||
if set(keys) - {"limit", "cursor"} or any(
|
||||
keys.count(key) != 1 for key in set(keys)
|
||||
):
|
||||
raise ApiProblem(400, "invalid_request", "Query parameters are invalid")
|
||||
limit_value = parameters.get("limit", str(DEFAULT_PAGE_SIZE))
|
||||
try:
|
||||
limit = int(limit_value)
|
||||
except (TypeError, ValueError):
|
||||
raise ApiProblem(400, "invalid_request", "limit is invalid") from None
|
||||
if not 1 <= limit <= MAX_PAGE_SIZE:
|
||||
raise ApiProblem(400, "invalid_request", "limit is invalid")
|
||||
cursor = parameters.get("cursor")
|
||||
if cursor == "":
|
||||
raise ApiProblem(400, "invalid_request", "cursor is invalid")
|
||||
return limit, cursor
|
||||
|
||||
|
||||
def require_chat_id(chat_id: str) -> uuid.UUID:
|
||||
try:
|
||||
return uuid.UUID(chat_id)
|
||||
except (ValueError, AttributeError):
|
||||
raise ApiProblem(400, "invalid_request", "chat_id must be a UUID") from None
|
||||
|
||||
|
||||
def unauthorized(message: str, challenge: str) -> ApiProblem:
|
||||
return ApiProblem(401, "unauthorized", message, {"WWW-Authenticate": challenge})
|
||||
|
||||
|
||||
def bearer_token(request: Request) -> str:
|
||||
"""Only Bearer is accepted; Basic here is a 401, not a fallback login."""
|
||||
header = request.headers.get("Authorization")
|
||||
if header is None:
|
||||
raise unauthorized("Bearer session token is required", BEARER_CHALLENGE)
|
||||
scheme, _, value = header.partition(" ")
|
||||
if scheme.strip().lower() != "bearer" or not value.strip():
|
||||
raise unauthorized("Bearer session token is required", BEARER_CHALLENGE)
|
||||
return value.strip()
|
||||
|
||||
|
||||
def basic_credentials(request: Request) -> tuple[str, str]:
|
||||
"""Decode Basic credentials; it is an encoding, never a trusted identity."""
|
||||
header = request.headers.get("Authorization")
|
||||
if header is None:
|
||||
raise unauthorized("Basic credentials are required", BASIC_CHALLENGE)
|
||||
scheme, _, value = header.partition(" ")
|
||||
if scheme.strip().lower() != "basic" or not value.strip():
|
||||
raise unauthorized("Basic credentials are required", BASIC_CHALLENGE)
|
||||
try:
|
||||
raw = base64.b64decode(value.strip(), validate=True).decode("utf-8")
|
||||
except (binascii.Error, UnicodeDecodeError, ValueError):
|
||||
raise unauthorized("Basic credentials are invalid", BASIC_CHALLENGE) from None
|
||||
username, separator, password = raw.partition(":")
|
||||
if not separator or not username or not password:
|
||||
raise unauthorized("Basic credentials are invalid", BASIC_CHALLENGE)
|
||||
if USERNAME_PATTERN.fullmatch(username) is None:
|
||||
# Unknown or malformed usernames must not be distinguished from a bad password.
|
||||
raise unauthorized("Invalid username or password", BASIC_CHALLENGE)
|
||||
return username, password
|
||||
|
||||
|
||||
async def current_principal(request: Request) -> Principal:
|
||||
token = bearer_token(request)
|
||||
user_id, user = await session_service.resolve_session(
|
||||
get_pool(request), get_cache(request), token
|
||||
)
|
||||
return Principal(user_id=user_id, user=user)
|
||||
|
||||
|
||||
Actor = Annotated[Principal, Depends(current_principal)]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Health
|
||||
# --------------------------------------------------------------------------- #
|
||||
@router.get("/health/live")
|
||||
async def liveness(request: Request) -> dict[str, str]:
|
||||
ensure_no_query(request)
|
||||
return {"status": "ok"}
|
||||
|
||||
|
||||
@router.get("/health/ready")
|
||||
async def readiness(request: Request) -> dict[str, str]:
|
||||
ensure_no_query(request)
|
||||
try:
|
||||
await ping(get_pool(request))
|
||||
except Exception as exc: # noqa: BLE001 - any DB failure means not ready
|
||||
request.app.state.logger.warning(
|
||||
"readiness_database_failed error=%s", type(exc).__name__
|
||||
)
|
||||
raise ApiProblem(503, "unavailable", "Database is unavailable") from None
|
||||
return {"status": "ok"}
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Users and sessions
|
||||
# --------------------------------------------------------------------------- #
|
||||
@router.post("/api/v1/users", status_code=201)
|
||||
async def register_user(request: Request) -> dict[str, Any]:
|
||||
ensure_no_query(request)
|
||||
body = await parse_body(request, CreateUser)
|
||||
assert isinstance(body, CreateUser)
|
||||
password_hash = await run_in_threadpool(hash_password, body.password)
|
||||
async with get_pool(request).connection() as connection:
|
||||
async with connection.cursor() as cursor:
|
||||
return await store.create_user(
|
||||
cursor,
|
||||
username=body.username,
|
||||
display_name=body.display_name,
|
||||
password_hash=password_hash,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/api/v1/auth/sessions", status_code=201)
|
||||
async def create_session(request: Request) -> dict[str, Any]:
|
||||
ensure_no_query(request)
|
||||
await drain_body(request)
|
||||
username, password = basic_credentials(request)
|
||||
pool = get_pool(request)
|
||||
async with pool.connection() as connection:
|
||||
async with connection.cursor() as cursor:
|
||||
row = await store.user_credentials(cursor, username)
|
||||
if row is None:
|
||||
await run_in_threadpool(verify_password_unknown_user, password)
|
||||
raise unauthorized("Invalid username or password", BASIC_CHALLENGE)
|
||||
if not await run_in_threadpool(
|
||||
verify_password, password, row["password_hash"]
|
||||
):
|
||||
raise unauthorized("Invalid username or password", BASIC_CHALLENGE)
|
||||
token, expires_at = await session_service.create_session(
|
||||
pool,
|
||||
get_cache(request),
|
||||
user_id=row["id"],
|
||||
ttl_seconds=request.app.state.settings.session_ttl_seconds,
|
||||
)
|
||||
return {
|
||||
"token": token,
|
||||
"token_type": "Bearer",
|
||||
"expires_at": format_timestamp(expires_at),
|
||||
"user": store.user_json(row),
|
||||
}
|
||||
|
||||
|
||||
@router.delete("/api/v1/auth/sessions/current", status_code=204)
|
||||
async def delete_current_session(request: Request) -> Response:
|
||||
ensure_no_query(request)
|
||||
token = bearer_token(request)
|
||||
await session_service.revoke_session(get_pool(request), get_cache(request), token)
|
||||
return Response(status_code=204)
|
||||
|
||||
|
||||
@router.get("/api/v1/users/me")
|
||||
async def current_user(request: Request, actor: Actor) -> dict[str, Any]:
|
||||
ensure_no_query(request)
|
||||
return actor.user
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Chats and messages
|
||||
# --------------------------------------------------------------------------- #
|
||||
@router.post("/api/v1/chats", status_code=201)
|
||||
async def create_chat(request: Request, actor: Actor) -> dict[str, Any]:
|
||||
ensure_no_query(request)
|
||||
body = await parse_body(request, CreateChat)
|
||||
assert isinstance(body, CreateChat)
|
||||
members = [uuid.UUID(member) for member in body.member_ids]
|
||||
async with get_pool(request).connection() as connection:
|
||||
async with connection.cursor() as cursor:
|
||||
return await store.create_chat(
|
||||
cursor, actor_id=actor.user_id, title=body.title, member_ids=members
|
||||
)
|
||||
|
||||
|
||||
@router.get("/api/v1/chats")
|
||||
async def list_chats(request: Request, actor: Actor) -> dict[str, Any]:
|
||||
limit, cursor = parse_pagination_request(request)
|
||||
resource = "chats:" + str(actor.user_id)
|
||||
after = decode_cursor(cursor, resource) if cursor is not None else None
|
||||
async with get_pool(request).connection() as connection:
|
||||
async with connection.cursor() as cursor_handle:
|
||||
rows = await store.chat_page(
|
||||
cursor_handle, actor_id=actor.user_id, limit=limit, after=after
|
||||
)
|
||||
return build_page(rows, resource, limit)
|
||||
|
||||
|
||||
@router.get("/api/v1/chats/{chat_id}")
|
||||
async def get_chat(request: Request, chat_id: str, actor: Actor) -> dict[str, Any]:
|
||||
ensure_no_query(request)
|
||||
normalized = require_chat_id(chat_id)
|
||||
async with get_pool(request).connection() as connection:
|
||||
async with connection.cursor() as cursor:
|
||||
return await store.accessible_chat(
|
||||
cursor, actor_id=actor.user_id, chat_id=normalized
|
||||
)
|
||||
|
||||
|
||||
@router.post("/api/v1/chats/{chat_id}/messages", status_code=201)
|
||||
async def create_message(
|
||||
request: Request, chat_id: str, actor: Actor
|
||||
) -> dict[str, Any]:
|
||||
normalized = require_chat_id(chat_id)
|
||||
ensure_no_query(request)
|
||||
pool = get_pool(request)
|
||||
async with pool.connection() as connection:
|
||||
async with connection.cursor() as cursor:
|
||||
await store.require_membership(
|
||||
cursor, actor_id=actor.user_id, chat_id=normalized
|
||||
)
|
||||
body = await parse_body(request, CreateMessage)
|
||||
assert isinstance(body, CreateMessage)
|
||||
async with pool.connection() as connection:
|
||||
async with connection.cursor() as cursor:
|
||||
return await store.create_message(
|
||||
cursor, actor_id=actor.user_id, chat_id=normalized, text=body.text
|
||||
)
|
||||
|
||||
|
||||
@router.get("/api/v1/chats/{chat_id}/messages")
|
||||
async def list_messages(
|
||||
request: Request, chat_id: str, actor: Actor
|
||||
) -> dict[str, Any]:
|
||||
normalized = require_chat_id(chat_id)
|
||||
limit, cursor = parse_pagination_request(request)
|
||||
resource = "messages:" + str(normalized)
|
||||
after = decode_cursor(cursor, resource) if cursor is not None else None
|
||||
async with get_pool(request).connection() as connection:
|
||||
async with connection.cursor() as cursor_handle:
|
||||
await store.require_membership(
|
||||
cursor_handle, actor_id=actor.user_id, chat_id=normalized
|
||||
)
|
||||
rows = await store.message_page(
|
||||
cursor_handle, chat_id=normalized, limit=limit, after=after
|
||||
)
|
||||
return build_page(rows, resource, limit)
|
||||
+34
@@ -0,0 +1,34 @@
|
||||
"""Executable entry point: migrations run first, then uvicorn as PID 1."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import uvicorn
|
||||
|
||||
|
||||
if __package__ in {None, ""}:
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||
|
||||
from src.application import create_app # noqa: E402
|
||||
from src.config import load_address # noqa: E402
|
||||
|
||||
|
||||
app = create_app()
|
||||
|
||||
|
||||
def run() -> None:
|
||||
host, port = load_address()
|
||||
uvicorn.run(
|
||||
app,
|
||||
host=host,
|
||||
port=port,
|
||||
access_log=False,
|
||||
timeout_graceful_shutdown=8,
|
||||
log_level="info",
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
run()
|
||||
@@ -0,0 +1,269 @@
|
||||
"""FastAPI application factory, lifespan, middleware and error mapping."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import contextlib
|
||||
import json
|
||||
import logging
|
||||
import sys
|
||||
import time
|
||||
import uuid
|
||||
from contextlib import asynccontextmanager
|
||||
from typing import Any, AsyncIterator
|
||||
|
||||
import psycopg
|
||||
from fastapi import FastAPI, Request
|
||||
from fastapi.exceptions import RequestValidationError
|
||||
from fastapi.responses import JSONResponse, RedirectResponse
|
||||
from psycopg_pool import AsyncConnectionPool, PoolTimeout
|
||||
from starlette.exceptions import HTTPException as StarletteHTTPException
|
||||
from starlette.responses import Response
|
||||
|
||||
from . import sessions as session_service
|
||||
from .api import router
|
||||
from .config import CONTRACT_PATH, Settings, load_settings
|
||||
from .db import create_pool
|
||||
from .errors import ApiProblem
|
||||
|
||||
|
||||
access_logger = logging.getLogger("messenger.access")
|
||||
lifecycle_logger = logging.getLogger("uvicorn.error")
|
||||
if not access_logger.handlers:
|
||||
access_handler = logging.StreamHandler(sys.stdout)
|
||||
access_handler.setFormatter(
|
||||
logging.Formatter("%(asctime)s %(levelname)s %(name)s %(message)s")
|
||||
)
|
||||
access_logger.addHandler(access_handler)
|
||||
access_logger.setLevel(logging.INFO)
|
||||
access_logger.propagate = False
|
||||
|
||||
|
||||
def request_id(request: Request) -> str:
|
||||
return getattr(request.state, "request_id", str(uuid.uuid4()))
|
||||
|
||||
|
||||
def problem_response(
|
||||
request: Request,
|
||||
status: int,
|
||||
code: str,
|
||||
message: str,
|
||||
headers: dict[str, str] | None = None,
|
||||
) -> JSONResponse:
|
||||
current_request_id = request_id(request)
|
||||
response_headers = {"X-Request-Id": current_request_id}
|
||||
response_headers.update(headers or {})
|
||||
return JSONResponse(
|
||||
status_code=status,
|
||||
content={
|
||||
"error": {
|
||||
"code": code,
|
||||
"message": message,
|
||||
"request_id": current_request_id,
|
||||
}
|
||||
},
|
||||
headers=response_headers,
|
||||
)
|
||||
|
||||
|
||||
def load_contract() -> dict[str, Any]:
|
||||
with CONTRACT_PATH.open(encoding="utf-8") as contract_file:
|
||||
return json.load(contract_file)
|
||||
|
||||
|
||||
async def _purge_loop(pool: AsyncConnectionPool) -> None:
|
||||
"""Drop long-expired session rows; TTL itself is enforced on every read."""
|
||||
while True:
|
||||
try:
|
||||
await asyncio.sleep(60)
|
||||
purged = await session_service.purge_expired(pool)
|
||||
if purged:
|
||||
lifecycle_logger.info("sessions_purged count=%s", purged)
|
||||
except asyncio.CancelledError:
|
||||
raise
|
||||
except Exception as exc: # noqa: BLE001 - housekeeping must not stop the API
|
||||
lifecycle_logger.warning(
|
||||
"sessions_purge_failed error=%s", type(exc).__name__
|
||||
)
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(application: FastAPI) -> AsyncIterator[None]:
|
||||
settings: Settings = application.state.settings
|
||||
pool: AsyncConnectionPool = application.state.pool
|
||||
cache: session_service.SessionCache = application.state.cache
|
||||
lifecycle_logger.info(
|
||||
"server_starting env=%s session_ttl_seconds=%s cache_enabled=%s "
|
||||
"pool_max=%s statement_timeout_ms=%s",
|
||||
settings.app_env,
|
||||
settings.session_ttl_seconds,
|
||||
cache.enabled,
|
||||
settings.pool_max_size,
|
||||
settings.statement_timeout_ms,
|
||||
)
|
||||
try:
|
||||
await pool.open(wait=True, timeout=10)
|
||||
except Exception as exc: # noqa: BLE001 - start only with a usable database
|
||||
lifecycle_logger.error(
|
||||
"database_unavailable_at_startup error=%s", type(exc).__name__
|
||||
)
|
||||
raise
|
||||
lifecycle_logger.info("server_started")
|
||||
if cache.enabled:
|
||||
if await cache.ping():
|
||||
lifecycle_logger.info("cache_ready")
|
||||
else:
|
||||
lifecycle_logger.warning("cache_unavailable_using_database_only")
|
||||
else:
|
||||
lifecycle_logger.warning("cache_not_configured_using_database_only")
|
||||
purge_task = asyncio.create_task(_purge_loop(pool))
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
lifecycle_logger.info("shutdown_requested")
|
||||
purge_task.cancel()
|
||||
with contextlib.suppress(BaseException):
|
||||
await purge_task
|
||||
await cache.close()
|
||||
await pool.close()
|
||||
lifecycle_logger.info("server_stopped")
|
||||
|
||||
|
||||
def create_app(settings: Settings | None = None) -> FastAPI:
|
||||
resolved = settings or load_settings()
|
||||
application = FastAPI(
|
||||
title="MTUSI Messenger — лабораторная 2",
|
||||
version="1.2.0",
|
||||
docs_url="/docs",
|
||||
redoc_url=None,
|
||||
lifespan=lifespan,
|
||||
)
|
||||
application.state.settings = resolved
|
||||
application.state.logger = lifecycle_logger
|
||||
application.state.pool = create_pool(
|
||||
resolved.database_url,
|
||||
min_size=resolved.pool_min_size,
|
||||
max_size=resolved.pool_max_size,
|
||||
timeout_seconds=resolved.pool_timeout_seconds,
|
||||
connect_timeout_seconds=resolved.connect_timeout_seconds,
|
||||
statement_timeout_ms=resolved.statement_timeout_ms,
|
||||
)
|
||||
application.state.cache = session_service.SessionCache(
|
||||
resolved.cache_url,
|
||||
timeout_seconds=resolved.cache_timeout_seconds,
|
||||
ttl_cap=resolved.cache_ttl_seconds,
|
||||
)
|
||||
application.openapi = load_contract # type: ignore[method-assign]
|
||||
|
||||
@application.middleware("http")
|
||||
async def request_context(request: Request, call_next: Any) -> Response:
|
||||
request.state.request_id = str(uuid.uuid4())
|
||||
started_at = time.monotonic()
|
||||
status = 500
|
||||
try:
|
||||
response = await call_next(request)
|
||||
if request.method in {"POST", "PUT", "PATCH"}:
|
||||
try:
|
||||
# Authentication or resource checks can finish before the body
|
||||
# is read. Drain any remainder without retaining it so the
|
||||
# client receives that JSON response instead of a TCP reset.
|
||||
async for _chunk in request.stream():
|
||||
pass
|
||||
except RuntimeError:
|
||||
pass
|
||||
status = response.status_code
|
||||
except ApiProblem as problem:
|
||||
response = problem_response(
|
||||
request, problem.status, problem.code, problem.message, problem.headers
|
||||
)
|
||||
status = problem.status
|
||||
except Exception: # noqa: BLE001 - never leak internals to the client
|
||||
lifecycle_logger.exception(
|
||||
"unhandled_error request_id=%s", request.state.request_id
|
||||
)
|
||||
response = problem_response(
|
||||
request, 500, "internal_error", "Internal server error"
|
||||
)
|
||||
status = 500
|
||||
response.headers["X-Request-Id"] = request.state.request_id
|
||||
client = request.client.host if request.client is not None else "unknown"
|
||||
access_logger.info(
|
||||
"request_complete request_id=%s method=%s path=%s status=%s "
|
||||
"client=%s elapsed_ms=%.3f",
|
||||
request.state.request_id,
|
||||
request.method,
|
||||
request.url.path,
|
||||
status,
|
||||
client,
|
||||
(time.monotonic() - started_at) * 1000,
|
||||
)
|
||||
return response
|
||||
|
||||
@application.exception_handler(ApiProblem)
|
||||
async def api_problem_handler(request: Request, problem: ApiProblem) -> JSONResponse:
|
||||
return problem_response(
|
||||
request, problem.status, problem.code, problem.message, problem.headers
|
||||
)
|
||||
|
||||
@application.exception_handler(RequestValidationError)
|
||||
async def validation_handler(
|
||||
request: Request, _error: RequestValidationError
|
||||
) -> JSONResponse:
|
||||
return problem_response(
|
||||
request, 400, "invalid_request", "Request fields are invalid"
|
||||
)
|
||||
|
||||
@application.exception_handler(StarletteHTTPException)
|
||||
async def http_error_handler(
|
||||
request: Request, error: StarletteHTTPException
|
||||
) -> JSONResponse:
|
||||
if error.status_code in {404, 405}:
|
||||
return problem_response(request, 404, "not_found", "Endpoint not found")
|
||||
return problem_response(
|
||||
request, error.status_code, "invalid_request", "Request is invalid"
|
||||
)
|
||||
|
||||
@application.exception_handler(PoolTimeout)
|
||||
async def pool_timeout_handler(
|
||||
request: Request, error: PoolTimeout
|
||||
) -> JSONResponse:
|
||||
lifecycle_logger.warning(
|
||||
"database_pool_timeout request_id=%s error=%s",
|
||||
request_id(request),
|
||||
type(error).__name__,
|
||||
)
|
||||
return problem_response(
|
||||
request, 503, "unavailable", "Database is unavailable"
|
||||
)
|
||||
|
||||
@application.exception_handler(psycopg.OperationalError)
|
||||
@application.exception_handler(psycopg.InterfaceError)
|
||||
async def database_unavailable_handler(
|
||||
request: Request, error: Exception
|
||||
) -> JSONResponse:
|
||||
lifecycle_logger.warning(
|
||||
"database_unavailable request_id=%s error=%s",
|
||||
request_id(request),
|
||||
type(error).__name__,
|
||||
)
|
||||
return problem_response(
|
||||
request, 503, "unavailable", "Database is unavailable"
|
||||
)
|
||||
|
||||
@application.exception_handler(psycopg.Error)
|
||||
async def database_error_handler(
|
||||
request: Request, error: psycopg.Error
|
||||
) -> JSONResponse:
|
||||
lifecycle_logger.warning(
|
||||
"database_error request_id=%s error=%s",
|
||||
request_id(request),
|
||||
type(error).__name__,
|
||||
)
|
||||
return problem_response(request, 500, "internal_error", "Internal server error")
|
||||
|
||||
@application.get("/", include_in_schema=False)
|
||||
async def docs_redirect() -> RedirectResponse:
|
||||
return RedirectResponse(url="/docs", status_code=307)
|
||||
|
||||
application.include_router(router)
|
||||
return application
|
||||
+140
@@ -0,0 +1,140 @@
|
||||
"""Runtime configuration: environment variables plus file-based secrets."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parents[1]
|
||||
CONTRACT_PATH = PROJECT_ROOT / "contracts" / "openapi.json"
|
||||
MIGRATIONS_DIR = Path(__file__).resolve().parent / "migrations"
|
||||
|
||||
DEFAULT_PAGE_SIZE = 50
|
||||
MAX_PAGE_SIZE = 100
|
||||
MAX_JSON_BYTES = 1_048_576
|
||||
DEFAULT_SESSION_TTL_SECONDS = 3600
|
||||
DEFAULT_CACHE_TTL_SECONDS = 300
|
||||
SUPPORTED_SESSION_STORES = {"postgres"}
|
||||
|
||||
|
||||
class ConfigurationError(Exception):
|
||||
"""Startup configuration is unusable; the process must not serve traffic."""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Settings:
|
||||
"""Everything the process needs, resolved once at startup."""
|
||||
|
||||
database_url: str
|
||||
cache_url: str | None
|
||||
session_store: str
|
||||
session_ttl_seconds: int
|
||||
cache_ttl_seconds: int
|
||||
pool_min_size: int
|
||||
pool_max_size: int
|
||||
pool_timeout_seconds: float
|
||||
connect_timeout_seconds: int
|
||||
statement_timeout_ms: int
|
||||
cache_timeout_seconds: float
|
||||
app_env: str
|
||||
|
||||
@property
|
||||
def cache_enabled(self) -> bool:
|
||||
return self.cache_url is not None
|
||||
|
||||
|
||||
def _secret(file_variable: str, plain_variable: str) -> str | None:
|
||||
"""Read a secret from a mounted file, falling back to a plain variable."""
|
||||
path_value = os.environ.get(file_variable, "").strip()
|
||||
if path_value:
|
||||
path = Path(path_value)
|
||||
try:
|
||||
value = path.read_text(encoding="utf-8").strip()
|
||||
except OSError as exc:
|
||||
raise ConfigurationError(
|
||||
file_variable + " points to an unreadable file (" + type(exc).__name__ + ")"
|
||||
) from None
|
||||
if not value:
|
||||
raise ConfigurationError(file_variable + " points to an empty file")
|
||||
return value
|
||||
plain = os.environ.get(plain_variable, "").strip()
|
||||
return plain or None
|
||||
|
||||
|
||||
def _integer(variable: str, default: int, minimum: int, maximum: int) -> int:
|
||||
raw = os.environ.get(variable, "").strip()
|
||||
if not raw:
|
||||
return default
|
||||
try:
|
||||
value = int(raw)
|
||||
except ValueError:
|
||||
raise ConfigurationError(variable + " must be an integer") from None
|
||||
if not minimum <= value <= maximum:
|
||||
raise ConfigurationError(
|
||||
variable + " must be between " + str(minimum) + " and " + str(maximum)
|
||||
)
|
||||
return value
|
||||
|
||||
|
||||
def _number(variable: str, default: float, minimum: float, maximum: float) -> float:
|
||||
raw = os.environ.get(variable, "").strip()
|
||||
if not raw:
|
||||
return default
|
||||
try:
|
||||
value = float(raw)
|
||||
except ValueError:
|
||||
raise ConfigurationError(variable + " must be a number") from None
|
||||
if not minimum <= value <= maximum:
|
||||
raise ConfigurationError(
|
||||
variable + " must be between " + str(minimum) + " and " + str(maximum)
|
||||
)
|
||||
return value
|
||||
|
||||
|
||||
def load_settings() -> Settings:
|
||||
"""Resolve settings once; a bad configuration fails fast before serving."""
|
||||
database_url = _secret("DATABASE_URL_FILE", "DATABASE_URL")
|
||||
if database_url is None:
|
||||
raise ConfigurationError("DATABASE_URL_FILE or DATABASE_URL is required")
|
||||
cache_url = _secret("CACHE_URL_FILE", "CACHE_URL")
|
||||
session_store = os.environ.get("SESSION_STORE", "postgres").strip().lower()
|
||||
if session_store not in SUPPORTED_SESSION_STORES:
|
||||
raise ConfigurationError(
|
||||
"SESSION_STORE="
|
||||
+ session_store
|
||||
+ " is not implemented: sessions are stored in PostgreSQL "
|
||||
"(the durable source of truth) and cached in Valkey. Use SESSION_STORE=postgres."
|
||||
)
|
||||
pool_min = _integer("DB_POOL_MIN", 1, 0, 100)
|
||||
pool_max = _integer("DB_POOL_MAX", 10, 1, 200)
|
||||
if pool_min > pool_max:
|
||||
raise ConfigurationError("DB_POOL_MIN must not exceed DB_POOL_MAX")
|
||||
return Settings(
|
||||
database_url=database_url,
|
||||
cache_url=cache_url,
|
||||
session_store=session_store,
|
||||
session_ttl_seconds=_integer(
|
||||
"SESSION_TTL_SECONDS", DEFAULT_SESSION_TTL_SECONDS, 1, 2_592_000
|
||||
),
|
||||
cache_ttl_seconds=_integer(
|
||||
"CACHE_TTL_SECONDS", DEFAULT_CACHE_TTL_SECONDS, 1, 86_400
|
||||
),
|
||||
pool_min_size=pool_min,
|
||||
pool_max_size=pool_max,
|
||||
pool_timeout_seconds=_number("DB_POOL_TIMEOUT_SECONDS", 2.0, 0.1, 60.0),
|
||||
connect_timeout_seconds=_integer("DB_CONNECT_TIMEOUT_SECONDS", 2, 1, 60),
|
||||
statement_timeout_ms=_integer("DB_STATEMENT_TIMEOUT_MS", 5000, 100, 120_000),
|
||||
cache_timeout_seconds=_number("CACHE_TIMEOUT_SECONDS", 0.5, 0.05, 10.0),
|
||||
app_env=os.environ.get("APP_ENV", "lab").strip() or "lab",
|
||||
)
|
||||
|
||||
|
||||
def load_address() -> tuple[str, int]:
|
||||
"""Read and validate the HTTP listen address from the environment."""
|
||||
host = os.environ.get("APP_HOST", "0.0.0.0").strip()
|
||||
if not host:
|
||||
raise SystemExit("APP_HOST must not be empty")
|
||||
port = _integer("APP_PORT", 8080, 1, 65535)
|
||||
return host, port
|
||||
@@ -0,0 +1,49 @@
|
||||
"""PostgreSQL access: a bounded async pool with explicit timeouts."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from psycopg.rows import dict_row
|
||||
from psycopg_pool import AsyncConnectionPool
|
||||
|
||||
|
||||
def connection_options(statement_timeout_ms: int) -> str:
|
||||
"""Server-side limits so a slow query or an idle transaction cannot pile up."""
|
||||
return (
|
||||
"-c statement_timeout="
|
||||
+ str(statement_timeout_ms)
|
||||
+ " -c lock_timeout=2000 -c idle_in_transaction_session_timeout=10000"
|
||||
+ " -c timezone=UTC"
|
||||
)
|
||||
|
||||
|
||||
def create_pool(
|
||||
database_url: str,
|
||||
*,
|
||||
min_size: int,
|
||||
max_size: int,
|
||||
timeout_seconds: float,
|
||||
connect_timeout_seconds: int,
|
||||
statement_timeout_ms: int,
|
||||
) -> AsyncConnectionPool:
|
||||
"""Build a closed pool; the lifespan opens it and closes it on shutdown."""
|
||||
return AsyncConnectionPool(
|
||||
conninfo=database_url,
|
||||
min_size=min_size,
|
||||
max_size=max_size,
|
||||
timeout=timeout_seconds,
|
||||
open=False,
|
||||
check=AsyncConnectionPool.check_connection,
|
||||
kwargs={
|
||||
"row_factory": dict_row,
|
||||
"connect_timeout": connect_timeout_seconds,
|
||||
"options": connection_options(statement_timeout_ms),
|
||||
"application_name": "mtusi-messenger-api",
|
||||
},
|
||||
name="messenger-pool",
|
||||
)
|
||||
|
||||
|
||||
async def ping(pool: AsyncConnectionPool) -> None:
|
||||
"""Cheap readiness probe; raises when PostgreSQL is not usable."""
|
||||
async with pool.connection() as connection:
|
||||
await connection.execute("SELECT 1")
|
||||
@@ -0,0 +1,26 @@
|
||||
"""Errors that may safely be returned through the public API."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Mapping
|
||||
|
||||
|
||||
BASIC_CHALLENGE = 'Basic realm="messenger", charset="UTF-8"'
|
||||
BEARER_CHALLENGE = 'Bearer realm="messenger"'
|
||||
|
||||
|
||||
class ApiProblem(Exception):
|
||||
"""An expected API failure with a stable status, machine code and headers."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
status: int,
|
||||
code: str,
|
||||
message: str,
|
||||
headers: Mapping[str, str] | None = None,
|
||||
) -> None:
|
||||
super().__init__(message)
|
||||
self.status = status
|
||||
self.code = code
|
||||
self.message = message
|
||||
self.headers = dict(headers or {})
|
||||
@@ -0,0 +1,93 @@
|
||||
"""Versioned SQL migration runner.
|
||||
|
||||
Every run is one transaction guarded by a PostgreSQL advisory lock, so a second
|
||||
process (or a restarted container) cannot apply the same file twice and a failed
|
||||
migration leaves the schema unchanged.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
import psycopg
|
||||
|
||||
if __package__ in {None, ""}:
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||
|
||||
from src.config import MIGRATIONS_DIR, ConfigurationError, load_settings # noqa: E402
|
||||
|
||||
|
||||
ADVISORY_LOCK_KEY = 915_127_001
|
||||
CONNECT_RETRY_SECONDS = 60
|
||||
|
||||
|
||||
def _version(path: Path) -> int:
|
||||
prefix = path.name.split("_", 1)[0]
|
||||
return int(prefix)
|
||||
|
||||
|
||||
def apply_migrations(connection: psycopg.Connection, directory: Path) -> list[str]:
|
||||
applied: list[str] = []
|
||||
with connection.transaction():
|
||||
connection.execute("SELECT pg_advisory_xact_lock(%s)", (ADVISORY_LOCK_KEY,))
|
||||
connection.execute(
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS schema_migrations (
|
||||
version integer PRIMARY KEY,
|
||||
filename text NOT NULL,
|
||||
applied_at timestamptz NOT NULL DEFAULT now()
|
||||
)
|
||||
"""
|
||||
)
|
||||
rows = connection.execute("SELECT version FROM schema_migrations").fetchall()
|
||||
done = {row[0] for row in rows}
|
||||
for path in sorted(directory.glob("*.sql"), key=_version):
|
||||
version = _version(path)
|
||||
if version in done:
|
||||
continue
|
||||
connection.execute(path.read_text(encoding="utf-8"))
|
||||
connection.execute(
|
||||
"INSERT INTO schema_migrations (version, filename) VALUES (%s, %s)",
|
||||
(version, path.name),
|
||||
)
|
||||
applied.append(path.name)
|
||||
return applied
|
||||
|
||||
|
||||
def main() -> None:
|
||||
try:
|
||||
settings = load_settings()
|
||||
except ConfigurationError as error:
|
||||
print("configuration_error " + str(error), file=sys.stderr)
|
||||
raise SystemExit(1) from None
|
||||
deadline = time.monotonic() + CONNECT_RETRY_SECONDS
|
||||
while True:
|
||||
try:
|
||||
with psycopg.connect(
|
||||
settings.database_url,
|
||||
connect_timeout=5,
|
||||
options="-c timezone=UTC",
|
||||
application_name="mtusi-messenger-migrate",
|
||||
) as connection:
|
||||
applied = apply_migrations(connection, MIGRATIONS_DIR)
|
||||
break
|
||||
except psycopg.OperationalError as error:
|
||||
if time.monotonic() >= deadline:
|
||||
print(
|
||||
"database_unreachable error=" + type(error).__name__, file=sys.stderr
|
||||
)
|
||||
raise SystemExit(1) from None
|
||||
time.sleep(1)
|
||||
except psycopg.Error as error:
|
||||
print("migration_failed error=" + type(error).__name__, file=sys.stderr)
|
||||
raise SystemExit(1) from None
|
||||
if applied:
|
||||
print("migrations_applied " + ",".join(applied))
|
||||
else:
|
||||
print("migrations_up_to_date")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,13 @@
|
||||
-- Users own credentials and identity. The password column stores an Argon2id hash.
|
||||
CREATE TABLE users (
|
||||
id uuid PRIMARY KEY,
|
||||
username text NOT NULL,
|
||||
display_name text NOT NULL,
|
||||
password_hash text NOT NULL,
|
||||
created_at timestamptz NOT NULL,
|
||||
CONSTRAINT users_username_format CHECK (username ~ '^[a-z0-9_]{3,32}$'),
|
||||
CONSTRAINT users_display_name_length CHECK (char_length(display_name) BETWEEN 1 AND 100),
|
||||
CONSTRAINT users_password_hash_present CHECK (char_length(password_hash) > 0)
|
||||
);
|
||||
|
||||
CREATE UNIQUE INDEX users_username_unique ON users (username);
|
||||
@@ -0,0 +1,34 @@
|
||||
-- Chats, explicit membership and message history.
|
||||
CREATE TABLE chats (
|
||||
id uuid PRIMARY KEY,
|
||||
title text NOT NULL,
|
||||
created_at timestamptz NOT NULL,
|
||||
CONSTRAINT chats_title_length CHECK (char_length(title) BETWEEN 1 AND 100)
|
||||
);
|
||||
|
||||
-- Chat lists are ordered by (created_at, id) for the same keyset cursor.
|
||||
CREATE INDEX chats_order_idx ON chats (created_at, id);
|
||||
|
||||
CREATE TABLE chat_members (
|
||||
chat_id uuid NOT NULL REFERENCES chats (id) ON DELETE CASCADE,
|
||||
user_id uuid NOT NULL REFERENCES users (id) ON DELETE RESTRICT,
|
||||
joined_at timestamptz NOT NULL,
|
||||
PRIMARY KEY (chat_id, user_id)
|
||||
);
|
||||
|
||||
-- "Chats of the current user" and the membership check on every read.
|
||||
CREATE INDEX chat_members_user_idx ON chat_members (user_id, chat_id);
|
||||
|
||||
CREATE TABLE messages (
|
||||
id uuid PRIMARY KEY,
|
||||
chat_id uuid NOT NULL REFERENCES chats (id) ON DELETE CASCADE,
|
||||
sender_id uuid NOT NULL REFERENCES users (id) ON DELETE RESTRICT,
|
||||
text text NOT NULL,
|
||||
created_at timestamptz NOT NULL,
|
||||
CONSTRAINT messages_text_length CHECK (char_length(text) BETWEEN 1 AND 4000)
|
||||
);
|
||||
|
||||
-- Serves history keyset pagination exactly: equality on chat_id, then the
|
||||
-- (created_at, id) tuple used by both ORDER BY and the cursor predicate.
|
||||
-- Justification and EXPLAIN (ANALYZE, BUFFERS) are in evidence/explain-history.txt.
|
||||
CREATE INDEX messages_chat_order_idx ON messages (chat_id, created_at, id);
|
||||
@@ -0,0 +1,16 @@
|
||||
-- Opaque server-side sessions: only a SHA-256 of the token is stored.
|
||||
CREATE TABLE sessions (
|
||||
id uuid PRIMARY KEY,
|
||||
user_id uuid NOT NULL REFERENCES users (id) ON DELETE CASCADE,
|
||||
token_hash text NOT NULL,
|
||||
created_at timestamptz NOT NULL,
|
||||
expires_at timestamptz NOT NULL,
|
||||
revoked_at timestamptz,
|
||||
CONSTRAINT sessions_token_hash_length CHECK (char_length(token_hash) = 64),
|
||||
CONSTRAINT sessions_expiry_after_creation CHECK (expires_at > created_at)
|
||||
);
|
||||
|
||||
-- Every authenticated request looks a token up by hash.
|
||||
CREATE UNIQUE INDEX sessions_token_hash_unique ON sessions (token_hash);
|
||||
CREATE INDEX sessions_user_idx ON sessions (user_id);
|
||||
CREATE INDEX sessions_expiry_idx ON sessions (expires_at);
|
||||
@@ -0,0 +1,78 @@
|
||||
"""Strict request models for the lab 2 API."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import uuid
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, field_validator
|
||||
|
||||
|
||||
USERNAME_PATTERN = re.compile(r"^[a-z0-9_]{3,32}$")
|
||||
PASSWORD_MIN_LENGTH = 12
|
||||
PASSWORD_MAX_LENGTH = 128
|
||||
|
||||
|
||||
class StrictModel(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid", strict=True)
|
||||
|
||||
|
||||
def _not_blank(value: str, field: str) -> str:
|
||||
if not value.strip():
|
||||
raise ValueError(field + " must not be blank")
|
||||
return value
|
||||
|
||||
|
||||
class CreateUser(StrictModel):
|
||||
"""Registration body; the password never appears in a response."""
|
||||
|
||||
username: str = Field(
|
||||
min_length=3, max_length=32, pattern=USERNAME_PATTERN.pattern
|
||||
)
|
||||
display_name: str = Field(min_length=1, max_length=100)
|
||||
password: str = Field(
|
||||
min_length=PASSWORD_MIN_LENGTH, max_length=PASSWORD_MAX_LENGTH
|
||||
)
|
||||
|
||||
@field_validator("display_name")
|
||||
@classmethod
|
||||
def display_name_is_not_blank(cls, value: str) -> str:
|
||||
return _not_blank(value, "display_name")
|
||||
|
||||
@field_validator("password")
|
||||
@classmethod
|
||||
def password_length_in_code_points(cls, value: str) -> str:
|
||||
# Python len() counts Unicode code points, which is what the contract uses.
|
||||
if not PASSWORD_MIN_LENGTH <= len(value) <= PASSWORD_MAX_LENGTH:
|
||||
raise ValueError("password length is out of range")
|
||||
return value
|
||||
|
||||
|
||||
class CreateChat(StrictModel):
|
||||
title: str = Field(min_length=1, max_length=100)
|
||||
member_ids: list[str] = Field(min_length=1, max_length=100)
|
||||
|
||||
@field_validator("title")
|
||||
@classmethod
|
||||
def title_is_not_blank(cls, value: str) -> str:
|
||||
return _not_blank(value, "title")
|
||||
|
||||
@field_validator("member_ids")
|
||||
@classmethod
|
||||
def members_are_valid_and_unique(cls, value: list[str]) -> list[str]:
|
||||
try:
|
||||
normalized = [str(uuid.UUID(member_id)) for member_id in value]
|
||||
except (ValueError, AttributeError):
|
||||
raise ValueError("member_ids must contain UUID strings") from None
|
||||
if len(normalized) != len(set(normalized)):
|
||||
raise ValueError("member_ids must be unique")
|
||||
return normalized
|
||||
|
||||
|
||||
class CreateMessage(StrictModel):
|
||||
text: str = Field(min_length=1, max_length=4000)
|
||||
|
||||
@field_validator("text")
|
||||
@classmethod
|
||||
def text_is_not_blank(cls, value: str) -> str:
|
||||
return _not_blank(value, "text")
|
||||
@@ -0,0 +1,84 @@
|
||||
"""Opaque, resource-bound keyset cursors, timestamp formatting and page shaping."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import binascii
|
||||
import json
|
||||
import uuid
|
||||
from datetime import datetime, timezone
|
||||
from typing import Any
|
||||
|
||||
from .errors import ApiProblem
|
||||
|
||||
|
||||
def format_timestamp(value: datetime) -> str:
|
||||
"""RFC 3339 UTC with microseconds and a trailing Z, as the contract requires."""
|
||||
return (
|
||||
value.astimezone(timezone.utc)
|
||||
.isoformat(timespec="microseconds")
|
||||
.replace("+00:00", "Z")
|
||||
)
|
||||
|
||||
|
||||
def parse_timestamp(value: str) -> datetime:
|
||||
"""Parse an RFC 3339 timestamp from a cursor; reject anything else."""
|
||||
candidate = value[:-1] + "+00:00" if value.endswith("Z") else value
|
||||
parsed = datetime.fromisoformat(candidate)
|
||||
if parsed.tzinfo is None:
|
||||
parsed = parsed.replace(tzinfo=timezone.utc)
|
||||
return parsed.astimezone(timezone.utc)
|
||||
|
||||
|
||||
def encode_cursor(resource: str, created_at: datetime, record_id: str) -> str:
|
||||
payload = json.dumps(
|
||||
{
|
||||
"resource": resource,
|
||||
"created_at": format_timestamp(created_at),
|
||||
"id": record_id,
|
||||
},
|
||||
separators=(",", ":"),
|
||||
sort_keys=True,
|
||||
).encode("utf-8")
|
||||
return base64.urlsafe_b64encode(payload).decode("ascii").rstrip("=")
|
||||
|
||||
|
||||
def decode_cursor(value: str, resource: str) -> tuple[datetime, uuid.UUID]:
|
||||
"""Decode a cursor and bind it to the resource it was issued for."""
|
||||
try:
|
||||
padding = "=" * (-len(value) % 4)
|
||||
raw = base64.b64decode(value + padding, altchars=b"-_", validate=True)
|
||||
payload: Any = json.loads(raw.decode("utf-8"))
|
||||
if not isinstance(payload, dict) or set(payload) != {
|
||||
"resource",
|
||||
"created_at",
|
||||
"id",
|
||||
}:
|
||||
raise ValueError("unexpected cursor payload")
|
||||
if payload["resource"] != resource:
|
||||
raise ValueError("cursor belongs to another resource")
|
||||
created_at = payload["created_at"]
|
||||
record_id = payload["id"]
|
||||
if not isinstance(created_at, str) or not isinstance(record_id, str):
|
||||
raise ValueError("cursor fields have wrong types")
|
||||
return parse_timestamp(created_at), uuid.UUID(record_id)
|
||||
except (
|
||||
ValueError,
|
||||
TypeError,
|
||||
UnicodeError,
|
||||
binascii.Error,
|
||||
json.JSONDecodeError,
|
||||
):
|
||||
raise ApiProblem(400, "invalid_request", "Cursor is invalid") from None
|
||||
|
||||
|
||||
def build_page(items: list[dict[str, Any]], resource: str, limit: int) -> dict[str, Any]:
|
||||
"""Shape a limit+1 storage slice into the public page and its next cursor."""
|
||||
page = items[:limit]
|
||||
next_cursor = None
|
||||
if len(items) > limit and page:
|
||||
last = page[-1]
|
||||
next_cursor = encode_cursor(
|
||||
resource, parse_timestamp(last["created_at"]), last["id"]
|
||||
)
|
||||
return {"items": page, "next_cursor": next_cursor}
|
||||
@@ -0,0 +1,43 @@
|
||||
"""Password hashing with Argon2id (library KDF, salted, versioned parameters)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from argon2 import PasswordHasher
|
||||
from argon2.exceptions import InvalidHashError, VerificationError, VerifyMismatchError
|
||||
|
||||
|
||||
# OWASP-recommended Argon2id baseline: 19 MiB, 2 iterations, 1 lane.
|
||||
_hasher = PasswordHasher(
|
||||
time_cost=2,
|
||||
memory_cost=19456,
|
||||
parallelism=1,
|
||||
hash_len=32,
|
||||
salt_len=16,
|
||||
)
|
||||
|
||||
# Verified against when a username is unknown, so the failure path costs the same
|
||||
# as a wrong password and does not leak username existence through timing.
|
||||
_DUMMY_HASH = _hasher.hash("dummy-password-not-used-for-any-account")
|
||||
|
||||
|
||||
def hash_password(password: str) -> str:
|
||||
return _hasher.hash(password)
|
||||
|
||||
|
||||
def verify_password(password: str, encoded: str | None) -> bool:
|
||||
"""Return True only for a matching password and a parseable hash."""
|
||||
if encoded is None:
|
||||
verify_password_unknown_user(password)
|
||||
return False
|
||||
try:
|
||||
return _hasher.verify(encoded, password)
|
||||
except (VerifyMismatchError, VerificationError, InvalidHashError):
|
||||
return False
|
||||
|
||||
|
||||
def verify_password_unknown_user(password: str) -> None:
|
||||
"""Burn comparable CPU time when no account exists for the username."""
|
||||
try:
|
||||
_hasher.verify(_DUMMY_HASH, password)
|
||||
except (VerifyMismatchError, VerificationError, InvalidHashError):
|
||||
return
|
||||
+324
@@ -0,0 +1,324 @@
|
||||
"""Opaque Bearer sessions: PostgreSQL is the source of truth, Valkey is a cache.
|
||||
|
||||
Design (see REPORT.md):
|
||||
|
||||
* ``sessions`` in PostgreSQL owns identity, expiry and revocation.
|
||||
* Valkey stores the same session under ``session:<sha256(token)>`` for at most
|
||||
``CACHE_TTL_SECONDS`` and never longer than the session's remaining lifetime.
|
||||
* Revocation writes a tombstone instead of relying on ``DEL`` alone, so a stale
|
||||
cache entry cannot authenticate after logout.
|
||||
* When Valkey is unavailable every request falls back to PostgreSQL. A cache
|
||||
error is never treated as "authenticated", and a cache hit is only trusted
|
||||
because it came from a session row that PostgreSQL confirmed at login time.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import logging
|
||||
import secrets
|
||||
import time
|
||||
import uuid
|
||||
from datetime import datetime, timezone
|
||||
from typing import Any
|
||||
|
||||
import redis.asyncio as redis
|
||||
from psycopg_pool import AsyncConnectionPool
|
||||
from redis.backoff import NoBackoff
|
||||
from redis.retry import Retry
|
||||
|
||||
from . import store
|
||||
from .errors import ApiProblem
|
||||
from .pagination import format_timestamp
|
||||
|
||||
logger = logging.getLogger("messenger.sessions")
|
||||
|
||||
TOKEN_BYTES = 48 # 384 bits of entropy, well above the required 256.
|
||||
CACHE_PREFIX = "session:"
|
||||
REVOKED = "revoked"
|
||||
# After a cache error, skip Valkey for a few seconds instead of paying the
|
||||
# connect timeout on every request (lookup and remember would cost two each).
|
||||
FAILURE_COOLDOWN_SECONDS = 5.0
|
||||
# Bound on revocations remembered while the cache cannot accept a tombstone.
|
||||
MAX_PENDING_REVOCATIONS = 4096
|
||||
|
||||
HIT = "hit"
|
||||
MISS = "miss"
|
||||
TOMBSTONE = "revoked"
|
||||
ERROR = "error"
|
||||
|
||||
|
||||
def token_hash(token: str) -> str:
|
||||
return hashlib.sha256(token.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
class SessionCache:
|
||||
"""Small, failure-tolerant Valkey client used only for session lookups."""
|
||||
|
||||
def __init__(self, url: str | None, *, timeout_seconds: float, ttl_cap: int) -> None:
|
||||
self.ttl_cap = max(1, ttl_cap)
|
||||
self.timeout_seconds = timeout_seconds
|
||||
# hashed token -> epoch at which the session would be expired anyway.
|
||||
# PostgreSQL already holds the authoritative revocation; this only keeps
|
||||
# logout immediate for this process while the cache is unreachable.
|
||||
self._pending: dict[str, float] = {}
|
||||
self._disabled_until = 0.0
|
||||
self._degraded = False
|
||||
self.client: redis.Redis | None = None
|
||||
if url:
|
||||
self.client = redis.Redis.from_url(
|
||||
url,
|
||||
socket_timeout=timeout_seconds,
|
||||
socket_connect_timeout=timeout_seconds,
|
||||
decode_responses=True,
|
||||
health_check_interval=30,
|
||||
# A cache must fail fast: one attempt, no retry loop, so a dead
|
||||
# Valkey costs at most one timeout before PostgreSQL answers.
|
||||
retry=Retry(NoBackoff(), 0),
|
||||
retry_on_error=[],
|
||||
)
|
||||
|
||||
@property
|
||||
def enabled(self) -> bool:
|
||||
return self.client is not None
|
||||
|
||||
def _ready(self) -> bool:
|
||||
"""True when a cache call is worth attempting right now."""
|
||||
return self.client is not None and time.monotonic() >= self._disabled_until
|
||||
|
||||
def _unavailable(self, operation: str, exc: BaseException) -> None:
|
||||
self._disabled_until = time.monotonic() + FAILURE_COOLDOWN_SECONDS
|
||||
# One warning per outage, not one per request.
|
||||
if not self._degraded:
|
||||
self._degraded = True
|
||||
logger.warning(
|
||||
"cache_unavailable operation=%s error=%s cooldown_seconds=%s",
|
||||
operation,
|
||||
type(exc).__name__,
|
||||
FAILURE_COOLDOWN_SECONDS,
|
||||
)
|
||||
|
||||
def _ok(self) -> None:
|
||||
self._degraded = False
|
||||
|
||||
def remember_revocation(self, hashed: str, remaining: float) -> None:
|
||||
"""Ask this process to reject a token even if the cache cannot store it."""
|
||||
if len(self._pending) >= MAX_PENDING_REVOCATIONS:
|
||||
# Drop the entries closest to expiry: they are the least useful.
|
||||
keep = len(self._pending) // 4
|
||||
for key, _ in sorted(self._pending.items(), key=lambda item: item[1])[:keep]:
|
||||
self._pending.pop(key, None)
|
||||
self._pending[hashed] = time.time() + max(1.0, remaining)
|
||||
|
||||
def is_locally_revoked(self, hashed: str) -> bool:
|
||||
deadline = self._pending.get(hashed)
|
||||
if deadline is None:
|
||||
return False
|
||||
if deadline <= time.time():
|
||||
self._pending.pop(hashed, None)
|
||||
return False
|
||||
return True
|
||||
|
||||
async def _flush_revocations(self) -> None:
|
||||
"""Re-apply tombstones that could not be written during an outage."""
|
||||
for hashed, deadline in list(self._pending.items()):
|
||||
remaining = deadline - time.time()
|
||||
if remaining <= 0:
|
||||
self._pending.pop(hashed, None)
|
||||
continue
|
||||
try:
|
||||
await self.client.set(self._key(hashed), REVOKED, ex=self._ttl(remaining))
|
||||
except Exception as exc: # noqa: BLE001 - still revoked in PostgreSQL
|
||||
self._unavailable("flush_revocation", exc)
|
||||
return
|
||||
self._pending.pop(hashed, None)
|
||||
|
||||
async def ping(self) -> bool:
|
||||
if self.client is None:
|
||||
return False
|
||||
try:
|
||||
alive = bool(await self.client.ping())
|
||||
except Exception as exc: # noqa: BLE001 - any client error means "no cache"
|
||||
logger.warning("cache_ping_failed error=%s", type(exc).__name__)
|
||||
return False
|
||||
self._disabled_until = 0.0
|
||||
self._ok()
|
||||
return alive
|
||||
|
||||
async def close(self) -> None:
|
||||
if self.client is None:
|
||||
return
|
||||
try:
|
||||
await self.client.aclose()
|
||||
except Exception as exc: # noqa: BLE001 - shutdown must not fail on cache
|
||||
logger.warning("cache_close_failed error=%s", type(exc).__name__)
|
||||
|
||||
def _key(self, hashed: str) -> str:
|
||||
return CACHE_PREFIX + hashed
|
||||
|
||||
def _ttl(self, seconds: float) -> int:
|
||||
return max(1, min(int(seconds), self.ttl_cap))
|
||||
|
||||
async def lookup(self, hashed: str) -> tuple[str, dict[str, Any] | None]:
|
||||
"""Return (state, payload); every cache failure degrades to a DB read."""
|
||||
if not self._ready():
|
||||
return MISS, None
|
||||
if self._pending:
|
||||
await self._flush_revocations()
|
||||
if not self._ready():
|
||||
return MISS, None
|
||||
try:
|
||||
raw = await self.client.get(self._key(hashed))
|
||||
self._ok()
|
||||
except Exception as exc: # noqa: BLE001 - cache must never break auth
|
||||
self._unavailable("lookup", exc)
|
||||
return ERROR, None
|
||||
if raw is None:
|
||||
return MISS, None
|
||||
if raw == REVOKED:
|
||||
return TOMBSTONE, None
|
||||
try:
|
||||
payload = json.loads(raw)
|
||||
user = payload["user"]
|
||||
user["id"] = str(uuid.UUID(user["id"]))
|
||||
payload["expires_epoch"] = float(payload["expires_epoch"])
|
||||
payload["user"] = user
|
||||
except (ValueError, KeyError, TypeError, json.JSONDecodeError):
|
||||
logger.warning("cache_payload_invalid")
|
||||
return MISS, None
|
||||
return HIT, payload
|
||||
|
||||
async def remember(self, hashed: str, user: dict[str, Any], expires_at: datetime) -> None:
|
||||
if not self._ready():
|
||||
return
|
||||
remaining = (expires_at - datetime.now(timezone.utc)).total_seconds()
|
||||
if remaining <= 0:
|
||||
return
|
||||
# Cache the immutable public user snapshot, so a hit needs no SQL at all.
|
||||
# Users cannot be renamed or deleted in the mandatory API of lab 2.
|
||||
payload = {
|
||||
"user": user,
|
||||
"expires_epoch": expires_at.timestamp(),
|
||||
}
|
||||
try:
|
||||
await self.client.set(
|
||||
self._key(hashed),
|
||||
json.dumps(payload, ensure_ascii=False),
|
||||
ex=self._ttl(remaining),
|
||||
)
|
||||
self._ok()
|
||||
except Exception as exc: # noqa: BLE001 - a missed cache write is not fatal
|
||||
self._unavailable("remember", exc)
|
||||
|
||||
async def revoke(self, hashed: str, expires_at: datetime | None) -> None:
|
||||
"""Write a tombstone so a stale entry cannot authenticate after logout."""
|
||||
remaining = 60.0
|
||||
if expires_at is not None:
|
||||
remaining = max(1.0, (expires_at - datetime.now(timezone.utc)).total_seconds())
|
||||
# PostgreSQL already revoked the session; this keeps the fast path honest
|
||||
# even if the tombstone cannot be written right now.
|
||||
self.remember_revocation(hashed, remaining)
|
||||
if not self._ready():
|
||||
return
|
||||
try:
|
||||
await self.client.set(self._key(hashed), REVOKED, ex=self._ttl(remaining))
|
||||
self._pending.pop(hashed, None)
|
||||
self._ok()
|
||||
except Exception as exc: # noqa: BLE001 - PostgreSQL already revoked the session
|
||||
self._unavailable("revoke", exc)
|
||||
|
||||
|
||||
def _user_payload(row: dict[str, Any]) -> dict[str, Any]:
|
||||
return {
|
||||
"id": str(row["id"]),
|
||||
"username": row["username"],
|
||||
"display_name": row["display_name"],
|
||||
"created_at": format_timestamp(row["created_at"]),
|
||||
}
|
||||
|
||||
|
||||
async def create_session(
|
||||
pool: AsyncConnectionPool,
|
||||
cache: SessionCache,
|
||||
*,
|
||||
user_id: uuid.UUID,
|
||||
ttl_seconds: int,
|
||||
) -> tuple[str, datetime]:
|
||||
token = secrets.token_urlsafe(TOKEN_BYTES)
|
||||
hashed = token_hash(token)
|
||||
async with pool.connection() as connection:
|
||||
async with connection.cursor() as cursor:
|
||||
expires_at = await store.insert_session(
|
||||
cursor, user_id=user_id, token_hash=hashed, ttl_seconds=ttl_seconds
|
||||
)
|
||||
user = await store.user_by_id(cursor, user_id)
|
||||
if user is not None:
|
||||
await cache.remember(hashed, _user_payload(user), expires_at)
|
||||
return token, expires_at
|
||||
|
||||
|
||||
async def resolve_session(
|
||||
pool: AsyncConnectionPool, cache: SessionCache, token: str
|
||||
) -> tuple[uuid.UUID, dict[str, Any]]:
|
||||
"""Return the authenticated user or raise 401 with a Bearer challenge."""
|
||||
hashed = token_hash(token)
|
||||
if cache.is_locally_revoked(hashed):
|
||||
raise ApiProblem(
|
||||
401,
|
||||
"unauthorized",
|
||||
"Session token is expired or revoked",
|
||||
{"WWW-Authenticate": 'Bearer realm="messenger"'},
|
||||
)
|
||||
state, _payload = await cache.lookup(hashed)
|
||||
if state == TOMBSTONE:
|
||||
raise ApiProblem(
|
||||
401,
|
||||
"unauthorized",
|
||||
"Session token is expired or revoked",
|
||||
{"WWW-Authenticate": 'Bearer realm="messenger"'},
|
||||
)
|
||||
|
||||
# PostgreSQL is the authoritative revocation store. A cache value can only
|
||||
# accelerate the immutable user snapshot; it must never be enough to grant
|
||||
# access on its own. In particular this makes logout durable across API
|
||||
# restarts and Valkey outages/restores, where a tombstone may not yet have
|
||||
# reached the shared cache.
|
||||
async with pool.connection() as connection:
|
||||
async with connection.cursor() as cursor:
|
||||
row = await store.session_user(cursor, hashed)
|
||||
if row is None:
|
||||
await cache.revoke(hashed, None)
|
||||
raise ApiProblem(
|
||||
401,
|
||||
"unauthorized",
|
||||
"Session token is expired or revoked",
|
||||
{"WWW-Authenticate": 'Bearer realm="messenger"'},
|
||||
)
|
||||
user = _user_payload(row)
|
||||
# Refresh an old cache hit after PostgreSQL has confirmed it is still live.
|
||||
await cache.remember(hashed, user, row["expires_at"])
|
||||
return uuid.UUID(user["id"]), user
|
||||
|
||||
|
||||
async def revoke_session(
|
||||
pool: AsyncConnectionPool, cache: SessionCache, token: str
|
||||
) -> None:
|
||||
hashed = token_hash(token)
|
||||
async with pool.connection() as connection:
|
||||
async with connection.cursor() as cursor:
|
||||
expires_at = await store.revoke_session(cursor, hashed)
|
||||
if expires_at is None:
|
||||
raise ApiProblem(
|
||||
401,
|
||||
"unauthorized",
|
||||
"Session token is expired or revoked",
|
||||
{"WWW-Authenticate": 'Bearer realm="messenger"'},
|
||||
)
|
||||
await cache.revoke(hashed, expires_at)
|
||||
|
||||
|
||||
async def purge_expired(pool: AsyncConnectionPool) -> int:
|
||||
async with pool.connection() as connection:
|
||||
async with connection.cursor() as cursor:
|
||||
return await store.purge_expired_sessions(cursor)
|
||||
+297
@@ -0,0 +1,297 @@
|
||||
"""PostgreSQL repository: parameterised statements only, explicit transactions."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
import psycopg
|
||||
from psycopg import AsyncCursor
|
||||
|
||||
from .errors import ApiProblem
|
||||
from .pagination import format_timestamp
|
||||
|
||||
# One expression keeps the member list of a chat consistent across endpoints.
|
||||
CHAT_COLUMNS = """
|
||||
c.id,
|
||||
c.title,
|
||||
c.created_at,
|
||||
ARRAY(
|
||||
SELECT m.user_id::text
|
||||
FROM chat_members m
|
||||
WHERE m.chat_id = c.id
|
||||
ORDER BY m.user_id::text
|
||||
) AS member_ids
|
||||
"""
|
||||
|
||||
|
||||
def user_json(row: dict[str, Any]) -> dict[str, Any]:
|
||||
return {
|
||||
"id": str(row["id"]),
|
||||
"username": row["username"],
|
||||
"display_name": row["display_name"],
|
||||
"created_at": format_timestamp(row["created_at"]),
|
||||
}
|
||||
|
||||
|
||||
def chat_json(row: dict[str, Any]) -> dict[str, Any]:
|
||||
return {
|
||||
"id": str(row["id"]),
|
||||
"title": row["title"],
|
||||
"member_ids": [str(member) for member in row["member_ids"]],
|
||||
"created_at": format_timestamp(row["created_at"]),
|
||||
}
|
||||
|
||||
|
||||
def message_json(row: dict[str, Any]) -> dict[str, Any]:
|
||||
return {
|
||||
"id": str(row["id"]),
|
||||
"chat_id": str(row["chat_id"]),
|
||||
"sender_id": str(row["sender_id"]),
|
||||
"text": row["text"],
|
||||
"created_at": format_timestamp(row["created_at"]),
|
||||
}
|
||||
|
||||
|
||||
async def create_user(
|
||||
cursor: AsyncCursor,
|
||||
*,
|
||||
username: str,
|
||||
display_name: str,
|
||||
password_hash: str,
|
||||
) -> dict[str, Any]:
|
||||
try:
|
||||
await cursor.execute(
|
||||
"""
|
||||
INSERT INTO users (id, username, display_name, password_hash, created_at)
|
||||
VALUES (%s, %s, %s, %s, now())
|
||||
RETURNING id, username, display_name, created_at
|
||||
""",
|
||||
(uuid.uuid4(), username, display_name, password_hash),
|
||||
)
|
||||
except psycopg.errors.UniqueViolation:
|
||||
raise ApiProblem(409, "conflict", "Username is already taken") from None
|
||||
row = await cursor.fetchone()
|
||||
return user_json(row)
|
||||
|
||||
|
||||
async def user_credentials(cursor: AsyncCursor, username: str) -> dict[str, Any] | None:
|
||||
await cursor.execute(
|
||||
"""
|
||||
SELECT id, username, display_name, password_hash, created_at
|
||||
FROM users
|
||||
WHERE username = %s
|
||||
""",
|
||||
(username,),
|
||||
)
|
||||
return await cursor.fetchone()
|
||||
|
||||
|
||||
async def user_by_id(cursor: AsyncCursor, user_id: uuid.UUID) -> dict[str, Any] | None:
|
||||
await cursor.execute(
|
||||
"""
|
||||
SELECT id, username, display_name, created_at
|
||||
FROM users
|
||||
WHERE id = %s
|
||||
""",
|
||||
(user_id,),
|
||||
)
|
||||
return await cursor.fetchone()
|
||||
|
||||
|
||||
async def create_chat(
|
||||
cursor: AsyncCursor,
|
||||
*,
|
||||
actor_id: uuid.UUID,
|
||||
title: str,
|
||||
member_ids: list[uuid.UUID],
|
||||
) -> dict[str, Any]:
|
||||
"""Insert the chat and its membership in one transaction.
|
||||
|
||||
Unknown members are rejected by the foreign key, so the failure rolls the
|
||||
already inserted chat row back instead of leaving a chat without members.
|
||||
"""
|
||||
if actor_id not in member_ids:
|
||||
raise ApiProblem(400, "invalid_request", "Current user must be a chat member")
|
||||
chat_id = uuid.uuid4()
|
||||
try:
|
||||
await cursor.execute(
|
||||
"INSERT INTO chats (id, title, created_at) VALUES (%s, %s, now())",
|
||||
(chat_id, title),
|
||||
)
|
||||
await cursor.execute(
|
||||
"""
|
||||
INSERT INTO chat_members (chat_id, user_id, joined_at)
|
||||
SELECT %s, member_id, now() FROM unnest(%s::uuid[]) AS member_id
|
||||
""",
|
||||
(chat_id, member_ids),
|
||||
)
|
||||
except psycopg.errors.ForeignKeyViolation:
|
||||
raise ApiProblem(
|
||||
400, "invalid_request", "One or more members do not exist"
|
||||
) from None
|
||||
except psycopg.errors.UniqueViolation:
|
||||
raise ApiProblem(400, "invalid_request", "Chat members must be unique") from None
|
||||
await cursor.execute(
|
||||
"SELECT " + CHAT_COLUMNS + " FROM chats c WHERE c.id = %s",
|
||||
(chat_id,),
|
||||
)
|
||||
row = await cursor.fetchone()
|
||||
return chat_json(row)
|
||||
|
||||
|
||||
async def accessible_chat(
|
||||
cursor: AsyncCursor, *, actor_id: uuid.UUID, chat_id: uuid.UUID
|
||||
) -> dict[str, Any]:
|
||||
await cursor.execute(
|
||||
"""
|
||||
SELECT """
|
||||
+ CHAT_COLUMNS
|
||||
+ """
|
||||
FROM chats c
|
||||
JOIN chat_members membership
|
||||
ON membership.chat_id = c.id AND membership.user_id = %s
|
||||
WHERE c.id = %s
|
||||
""",
|
||||
(actor_id, chat_id),
|
||||
)
|
||||
row = await cursor.fetchone()
|
||||
if row is None:
|
||||
raise ApiProblem(404, "not_found", "Chat not found")
|
||||
return chat_json(row)
|
||||
|
||||
|
||||
async def chat_page(
|
||||
cursor: AsyncCursor,
|
||||
*,
|
||||
actor_id: uuid.UUID,
|
||||
limit: int,
|
||||
after: tuple[datetime, uuid.UUID] | None,
|
||||
) -> list[dict[str, Any]]:
|
||||
query = (
|
||||
"SELECT "
|
||||
+ CHAT_COLUMNS
|
||||
+ """
|
||||
FROM chats c
|
||||
JOIN chat_members membership
|
||||
ON membership.chat_id = c.id AND membership.user_id = %(actor)s
|
||||
"""
|
||||
)
|
||||
params: dict[str, Any] = {"actor": actor_id, "limit": limit + 1}
|
||||
if after is not None:
|
||||
query += " WHERE (c.created_at, c.id) > (%(after_ts)s, %(after_id)s)"
|
||||
params["after_ts"], params["after_id"] = after
|
||||
query += " ORDER BY c.created_at, c.id LIMIT %(limit)s"
|
||||
await cursor.execute(query, params)
|
||||
return [chat_json(row) for row in await cursor.fetchall()]
|
||||
|
||||
|
||||
async def require_membership(
|
||||
cursor: AsyncCursor, *, actor_id: uuid.UUID, chat_id: uuid.UUID
|
||||
) -> None:
|
||||
await cursor.execute(
|
||||
"SELECT 1 FROM chat_members WHERE chat_id = %s AND user_id = %s",
|
||||
(chat_id, actor_id),
|
||||
)
|
||||
if await cursor.fetchone() is None:
|
||||
raise ApiProblem(404, "not_found", "Chat not found")
|
||||
|
||||
|
||||
async def create_message(
|
||||
cursor: AsyncCursor,
|
||||
*,
|
||||
actor_id: uuid.UUID,
|
||||
chat_id: uuid.UUID,
|
||||
text: str,
|
||||
) -> dict[str, Any]:
|
||||
await require_membership(cursor, actor_id=actor_id, chat_id=chat_id)
|
||||
await cursor.execute(
|
||||
"""
|
||||
INSERT INTO messages (id, chat_id, sender_id, text, created_at)
|
||||
VALUES (%s, %s, %s, %s, now())
|
||||
RETURNING id, chat_id, sender_id, text, created_at
|
||||
""",
|
||||
(uuid.uuid4(), chat_id, actor_id, text),
|
||||
)
|
||||
return message_json(await cursor.fetchone())
|
||||
|
||||
|
||||
async def message_page(
|
||||
cursor: AsyncCursor,
|
||||
*,
|
||||
chat_id: uuid.UUID,
|
||||
limit: int,
|
||||
after: tuple[datetime, uuid.UUID] | None,
|
||||
) -> list[dict[str, Any]]:
|
||||
"""Keyset page over (created_at, id); the composite index serves both branches."""
|
||||
query = """
|
||||
SELECT id, chat_id, sender_id, text, created_at
|
||||
FROM messages
|
||||
WHERE chat_id = %(chat)s
|
||||
"""
|
||||
params: dict[str, Any] = {"chat": chat_id, "limit": limit + 1}
|
||||
if after is not None:
|
||||
query += " AND (created_at, id) > (%(after_ts)s, %(after_id)s)"
|
||||
params["after_ts"], params["after_id"] = after
|
||||
query += " ORDER BY created_at, id LIMIT %(limit)s"
|
||||
await cursor.execute(query, params)
|
||||
return [message_json(row) for row in await cursor.fetchall()]
|
||||
|
||||
|
||||
async def insert_session(
|
||||
cursor: AsyncCursor,
|
||||
*,
|
||||
user_id: uuid.UUID,
|
||||
token_hash: str,
|
||||
ttl_seconds: int,
|
||||
) -> datetime:
|
||||
await cursor.execute(
|
||||
"""
|
||||
INSERT INTO sessions (id, user_id, token_hash, created_at, expires_at)
|
||||
VALUES (%s, %s, %s, now(), now() + make_interval(secs => %s))
|
||||
RETURNING expires_at
|
||||
""",
|
||||
(uuid.uuid4(), user_id, token_hash, ttl_seconds),
|
||||
)
|
||||
row = await cursor.fetchone()
|
||||
return row["expires_at"]
|
||||
|
||||
|
||||
async def session_user(cursor: AsyncCursor, token_hash: str) -> dict[str, Any] | None:
|
||||
"""Resolve a live session row; expired and revoked sessions never match."""
|
||||
await cursor.execute(
|
||||
"""
|
||||
SELECT s.expires_at, u.id, u.username, u.display_name, u.created_at
|
||||
FROM sessions s
|
||||
JOIN users u ON u.id = s.user_id
|
||||
WHERE s.token_hash = %s AND s.revoked_at IS NULL AND s.expires_at > now()
|
||||
""",
|
||||
(token_hash,),
|
||||
)
|
||||
return await cursor.fetchone()
|
||||
|
||||
|
||||
async def revoke_session(cursor: AsyncCursor, token_hash: str) -> datetime | None:
|
||||
"""Revoke a live session and return its expiry, or None when it is not live."""
|
||||
await cursor.execute(
|
||||
"""
|
||||
UPDATE sessions SET revoked_at = now()
|
||||
WHERE token_hash = %s AND revoked_at IS NULL AND expires_at > now()
|
||||
RETURNING expires_at
|
||||
""",
|
||||
(token_hash,),
|
||||
)
|
||||
row = await cursor.fetchone()
|
||||
return None if row is None else row["expires_at"]
|
||||
|
||||
|
||||
async def purge_expired_sessions(cursor: AsyncCursor) -> int:
|
||||
await cursor.execute(
|
||||
"""
|
||||
DELETE FROM sessions
|
||||
WHERE expires_at < now() - interval '1 hour'
|
||||
OR (revoked_at IS NOT NULL AND revoked_at < now() - interval '1 hour')
|
||||
"""
|
||||
)
|
||||
return cursor.rowcount
|
||||
@@ -0,0 +1,201 @@
|
||||
"""Minimal standard-library HTTP client for the student's own tests.
|
||||
|
||||
Run with the API already up:
|
||||
|
||||
python -m unittest discover -s student-tests -p "test_*.py" -v
|
||||
|
||||
BASE_URL selects the stand (default http://127.0.0.1:8080). No third-party
|
||||
dependency is needed, so `make test-student` works on a clean clone.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
import uuid
|
||||
|
||||
|
||||
# 127.0.0.1 rather than "localhost": on a dual-stack host "localhost" resolves
|
||||
# to ::1 first, a published Docker port may answer on IPv4 only, and every IPv6
|
||||
# attempt then costs ~2 s of SYN retry before urllib falls back to IPv4. The IPv4
|
||||
# literal keeps the client fast and never reaches the application.
|
||||
BASE_URL = os.environ.get("BASE_URL", "http://127.0.0.1:8080").rstrip("/")
|
||||
TIMEOUT = float(os.environ.get("STUDENT_TEST_TIMEOUT", "20"))
|
||||
|
||||
|
||||
class NoRedirect(urllib.request.HTTPRedirectHandler):
|
||||
"""The contract never requires redirects; fail loudly instead of following."""
|
||||
|
||||
def redirect_request(self, req, fp, code, msg, headers, newurl): # noqa: ANN001
|
||||
return None
|
||||
|
||||
|
||||
# The lab stand is always loopback; urllib ignores the Windows proxy bypass
|
||||
# list, so a corporate/system proxy would otherwise be used for localhost too.
|
||||
_OPENER = urllib.request.build_opener(urllib.request.ProxyHandler({}), NoRedirect())
|
||||
|
||||
|
||||
class Response:
|
||||
def __init__(self, status: int, headers: dict[str, str], data: bytes) -> None:
|
||||
self.status = status
|
||||
self.headers = headers
|
||||
self.data = data
|
||||
try:
|
||||
self.json = json.loads(data.decode("utf-8")) if data else None
|
||||
except (ValueError, UnicodeDecodeError):
|
||||
self.json = None
|
||||
|
||||
def header(self, name: str) -> str:
|
||||
for key, value in self.headers.items():
|
||||
if key.lower() == name.lower():
|
||||
return value
|
||||
return ""
|
||||
|
||||
def __repr__(self) -> str: # pragma: no cover - diagnostics only
|
||||
return "Response(status=%s, json=%r)" % (self.status, self.json)
|
||||
|
||||
|
||||
def request(
|
||||
method: str,
|
||||
path: str,
|
||||
*,
|
||||
body=None,
|
||||
raw: bytes | None = None,
|
||||
token: str | None = None,
|
||||
headers: dict[str, str] | None = None,
|
||||
content_type: str | None = None,
|
||||
timeout: float | None = None,
|
||||
base_url: str | None = None,
|
||||
) -> Response:
|
||||
url = (base_url or BASE_URL) + path
|
||||
data = raw
|
||||
request_headers = {"Accept": "application/json"}
|
||||
if body is not None:
|
||||
data = json.dumps(body, ensure_ascii=False).encode("utf-8")
|
||||
request_headers["Content-Type"] = "application/json"
|
||||
if content_type is not None:
|
||||
request_headers["Content-Type"] = content_type
|
||||
if token is not None:
|
||||
request_headers["Authorization"] = "Bearer " + token
|
||||
request_headers.update(headers or {})
|
||||
prepared = urllib.request.Request(url, data=data, headers=request_headers, method=method)
|
||||
try:
|
||||
response = _OPENER.open(prepared, timeout=timeout or TIMEOUT)
|
||||
except urllib.error.HTTPError as error:
|
||||
response = error
|
||||
with response:
|
||||
payload = response.read()
|
||||
return Response(response.code, dict(response.headers.items()), payload)
|
||||
|
||||
|
||||
def basic(username: str, password: str) -> dict[str, str]:
|
||||
encoded = base64.b64encode((username + ":" + password).encode("utf-8")).decode("ascii")
|
||||
return {"Authorization": "Basic " + encoded}
|
||||
|
||||
|
||||
def new_registration(prefix: str = "st", password: str | None = None) -> dict[str, str]:
|
||||
suffix = uuid.uuid4().hex[:12]
|
||||
return {
|
||||
"username": prefix + "_" + suffix,
|
||||
"display_name": "Тест " + suffix,
|
||||
"password": password or ("St-" + uuid.uuid4().hex),
|
||||
}
|
||||
|
||||
|
||||
def register(registration: dict[str, str] | None = None) -> Response:
|
||||
return request("POST", "/api/v1/users", body=registration or new_registration())
|
||||
|
||||
|
||||
def login(username: str, password: str) -> Response:
|
||||
return request("POST", "/api/v1/auth/sessions", headers=basic(username, password))
|
||||
|
||||
|
||||
class Actor:
|
||||
"""A registered user with a live session, as a client would hold it."""
|
||||
|
||||
def __init__(self, registration: dict[str, str]) -> None:
|
||||
self.registration = registration
|
||||
response = register(registration)
|
||||
if response.status != 201:
|
||||
raise AssertionError(
|
||||
"registration failed: %s %r" % (response.status, response.json)
|
||||
)
|
||||
self.user = response.json
|
||||
self.username = registration["username"]
|
||||
self.password = registration["password"]
|
||||
|
||||
@property
|
||||
def id(self) -> str:
|
||||
return self.user["id"]
|
||||
|
||||
def sign_in(self) -> "Actor":
|
||||
response = login(self.username, self.password)
|
||||
if response.status != 201:
|
||||
raise AssertionError("login failed: %s %r" % (response.status, response.json))
|
||||
self.session = response.json
|
||||
self.token = response.json["token"]
|
||||
return self
|
||||
|
||||
|
||||
def actor(prefix: str = "st", password: str | None = None) -> Actor:
|
||||
return Actor(new_registration(prefix, password)).sign_in()
|
||||
|
||||
|
||||
def create_chat(token: str, member_ids: list[str], title: str = "Тестовый чат") -> dict:
|
||||
response = request(
|
||||
"POST",
|
||||
"/api/v1/chats",
|
||||
body={"title": title, "member_ids": member_ids},
|
||||
token=token,
|
||||
)
|
||||
if response.status != 201:
|
||||
raise AssertionError("chat creation failed: %s %r" % (response.status, response.json))
|
||||
return response.json
|
||||
|
||||
|
||||
def send(token: str, chat_id: str, text: str) -> dict:
|
||||
response = request(
|
||||
"POST", "/api/v1/chats/" + chat_id + "/messages", body={"text": text}, token=token
|
||||
)
|
||||
if response.status != 201:
|
||||
raise AssertionError("message failed: %s %r" % (response.status, response.json))
|
||||
return response.json
|
||||
|
||||
|
||||
def history(token: str, chat_id: str, limit: int = 100) -> list[dict]:
|
||||
"""Walk every page of a chat, guarding against cursor loops."""
|
||||
items: list[dict] = []
|
||||
seen: set[str] = set()
|
||||
cursor: str | None = None
|
||||
while True:
|
||||
path = "/api/v1/chats/" + chat_id + "/messages?limit=" + str(limit)
|
||||
if cursor is not None:
|
||||
path += "&cursor=" + urllib.parse.quote(cursor, safe="")
|
||||
response = request("GET", path, token=token)
|
||||
if response.status != 200:
|
||||
raise AssertionError("history failed: %s %r" % (response.status, response.json))
|
||||
items.extend(response.json["items"])
|
||||
cursor = response.json["next_cursor"]
|
||||
if cursor is None:
|
||||
return items
|
||||
if cursor in seen:
|
||||
raise AssertionError("cursor loop detected")
|
||||
seen.add(cursor)
|
||||
|
||||
|
||||
def expect_error(test, response: Response, status: int, code: str) -> dict:
|
||||
"""Assert the public error envelope, request id echo and the status."""
|
||||
test.assertEqual(response.status, status, response.json)
|
||||
test.assertIsInstance(response.json, dict)
|
||||
error = response.json.get("error")
|
||||
test.assertIsNotNone(error, "missing error envelope")
|
||||
test.assertEqual(error["code"], code)
|
||||
test.assertTrue(error["message"])
|
||||
request_id = response.header("X-Request-Id")
|
||||
test.assertTrue(request_id, "missing X-Request-Id header")
|
||||
test.assertEqual(error["request_id"], request_id)
|
||||
return error
|
||||
@@ -0,0 +1,329 @@
|
||||
"""Own functional tests: contract edges carried from lab 1 plus lab 2 auth.
|
||||
|
||||
Every test creates unique synthetic users, so the file is safe to re-run against
|
||||
the same stand (the API has no delete endpoint by design).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
import uuid
|
||||
from datetime import datetime, timezone
|
||||
|
||||
import common
|
||||
|
||||
|
||||
def timestamp(value: str) -> datetime:
|
||||
return datetime.fromisoformat(value[:-1] + "+00:00")
|
||||
|
||||
|
||||
class Base(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls) -> None:
|
||||
again = common.request("POST", "/api/v1/users", body=common.new_registration())
|
||||
if again.status != 201:
|
||||
raise AssertionError("stand is not serving: %r" % (again.json,))
|
||||
cls.alice = common.actor("alice")
|
||||
cls.bob = common.actor("bob")
|
||||
cls.eve = common.actor("eve")
|
||||
cls.chat = common.create_chat(
|
||||
cls.alice.token, [cls.alice.id, cls.bob.id], "Контрактный чат"
|
||||
)
|
||||
|
||||
|
||||
class HealthAndEnvelope(Base):
|
||||
def test_health_endpoints_and_request_id(self) -> None:
|
||||
live = common.request("GET", "/health/live")
|
||||
self.assertEqual(live.status, 200)
|
||||
self.assertEqual(live.json, {"status": "ok"})
|
||||
self.assertTrue(live.header("X-Request-Id"))
|
||||
ready = common.request("GET", "/health/ready")
|
||||
self.assertEqual(ready.status, 200)
|
||||
self.assertEqual(ready.json, {"status": "ok"})
|
||||
|
||||
def test_unknown_route_uses_same_error_envelope(self) -> None:
|
||||
common.expect_error(self, common.request("GET", "/api/v1/nope"), 404, "not_found")
|
||||
|
||||
def test_query_parameters_rejected_where_not_supported(self) -> None:
|
||||
response = common.request(
|
||||
"GET", "/api/v1/users/me?verbose=1", token=self.alice.token
|
||||
)
|
||||
common.expect_error(self, response, 400, "invalid_request")
|
||||
|
||||
|
||||
class Registration(Base):
|
||||
def test_response_has_no_secret_fields(self) -> None:
|
||||
response = common.register(common.new_registration())
|
||||
self.assertEqual(response.status, 201)
|
||||
self.assertEqual(
|
||||
set(response.json), {"id", "username", "display_name", "created_at"}
|
||||
)
|
||||
self.assertNotIn("password", response.json)
|
||||
self.assertNotIn("password_hash", response.json)
|
||||
|
||||
def test_duplicate_username_conflict(self) -> None:
|
||||
registration = common.new_registration()
|
||||
self.assertEqual(common.register(registration).status, 201)
|
||||
common.expect_error(self, common.register(registration), 409, "conflict")
|
||||
|
||||
def test_username_rules(self) -> None:
|
||||
for username in ("UPPER_case", "ab", "with-dash", "with space"):
|
||||
body = common.new_registration()
|
||||
body["username"] = username
|
||||
common.expect_error(self, common.register(body), 400, "invalid_request")
|
||||
|
||||
def test_password_rules(self) -> None:
|
||||
short = common.new_registration(password="x" * 11)
|
||||
common.expect_error(self, common.register(short), 400, "invalid_request")
|
||||
long_password = common.new_registration(password="x" * 129)
|
||||
common.expect_error(self, common.register(long_password), 400, "invalid_request")
|
||||
# 12 Unicode code points are enough, even when they are multi-byte.
|
||||
unicode_password = common.new_registration(password="😀" * 12)
|
||||
self.assertEqual(common.register(unicode_password).status, 201)
|
||||
|
||||
def test_malformed_json_extra_fields_and_wrong_types(self) -> None:
|
||||
common.expect_error(
|
||||
self,
|
||||
common.request(
|
||||
"POST", "/api/v1/users", raw=b'{"username": ', content_type="application/json"
|
||||
),
|
||||
400,
|
||||
"invalid_request",
|
||||
)
|
||||
body = common.new_registration()
|
||||
body["is_admin"] = True
|
||||
common.expect_error(self, common.register(body), 400, "invalid_request")
|
||||
body = common.new_registration()
|
||||
body["display_name"] = 42
|
||||
common.expect_error(self, common.register(body), 400, "invalid_request")
|
||||
|
||||
def test_unsupported_media_type(self) -> None:
|
||||
response = common.request(
|
||||
"POST",
|
||||
"/api/v1/users",
|
||||
raw=b"username=alice",
|
||||
content_type="application/x-www-form-urlencoded",
|
||||
)
|
||||
common.expect_error(self, response, 415, "unsupported_media_type")
|
||||
|
||||
|
||||
class Chats(Base):
|
||||
def test_membership_validation(self) -> None:
|
||||
cases = [
|
||||
[self.bob.id], # caller is missing
|
||||
[self.alice.id, self.alice.id], # duplicate member
|
||||
[self.alice.id, str(uuid.uuid4())], # unknown member
|
||||
[], # no members
|
||||
]
|
||||
for member_ids in cases:
|
||||
response = common.request(
|
||||
"POST",
|
||||
"/api/v1/chats",
|
||||
body={"title": "Плохой чат", "member_ids": member_ids},
|
||||
token=self.alice.token,
|
||||
)
|
||||
common.expect_error(self, response, 400, "invalid_request")
|
||||
|
||||
def test_foreign_chat_is_hidden_everywhere(self) -> None:
|
||||
chat_id = self.chat["id"]
|
||||
common.expect_error(
|
||||
self,
|
||||
common.request("GET", "/api/v1/chats/" + chat_id, token=self.eve.token),
|
||||
404,
|
||||
"not_found",
|
||||
)
|
||||
common.expect_error(
|
||||
self,
|
||||
common.request(
|
||||
"GET", "/api/v1/chats/" + chat_id + "/messages", token=self.eve.token
|
||||
),
|
||||
404,
|
||||
"not_found",
|
||||
)
|
||||
common.expect_error(
|
||||
self,
|
||||
common.request(
|
||||
"POST",
|
||||
"/api/v1/chats/" + chat_id + "/messages",
|
||||
body={"text": "чужое"},
|
||||
token=self.eve.token,
|
||||
),
|
||||
404,
|
||||
"not_found",
|
||||
)
|
||||
listed = common.request("GET", "/api/v1/chats?limit=100", token=self.eve.token)
|
||||
self.assertEqual(listed.status, 200)
|
||||
self.assertNotIn(chat_id, [item["id"] for item in listed.json["items"]])
|
||||
|
||||
def test_messages_are_validated_and_unicode_preserved(self) -> None:
|
||||
chat = common.create_chat(self.alice.token, [self.alice.id, self.bob.id])
|
||||
for text in ("", " \n\t ", "x" * 4001):
|
||||
response = common.request(
|
||||
"POST",
|
||||
"/api/v1/chats/" + chat["id"] + "/messages",
|
||||
body={"text": text},
|
||||
token=self.alice.token,
|
||||
)
|
||||
common.expect_error(self, response, 400, "invalid_request")
|
||||
prefix = "Привет 👋 世界 "
|
||||
text = prefix + "x" * (4000 - len(prefix)) # exactly the contract maximum
|
||||
message = common.send(self.alice.token, chat["id"], text)
|
||||
self.assertEqual(message["text"], text)
|
||||
self.assertEqual(message["sender_id"], self.alice.id)
|
||||
|
||||
injected = common.request(
|
||||
"POST",
|
||||
"/api/v1/chats/" + chat["id"] + "/messages",
|
||||
body={"text": "подмена", "sender_id": self.bob.id},
|
||||
token=self.alice.token,
|
||||
)
|
||||
common.expect_error(self, injected, 400, "invalid_request")
|
||||
|
||||
|
||||
class Pagination(Base):
|
||||
def test_walks_every_page_without_loss_or_duplicates(self) -> None:
|
||||
chat = common.create_chat(self.alice.token, [self.alice.id, self.bob.id])
|
||||
sent = [
|
||||
common.send(self.alice.token if index % 2 else self.bob.token, chat["id"], "msg-%d" % index)
|
||||
for index in range(7)
|
||||
]
|
||||
collected: list[dict] = []
|
||||
cursor = None
|
||||
pages = 0
|
||||
while True:
|
||||
path = "/api/v1/chats/" + chat["id"] + "/messages?limit=2"
|
||||
if cursor is not None:
|
||||
path += "&cursor=" + cursor
|
||||
page = common.request("GET", path, token=self.alice.token)
|
||||
self.assertEqual(page.status, 200)
|
||||
collected.extend(page.json["items"])
|
||||
pages += 1
|
||||
cursor = page.json["next_cursor"]
|
||||
if cursor is None:
|
||||
break
|
||||
self.assertLess(pages, 10, "pagination did not terminate")
|
||||
self.assertEqual(pages, 4)
|
||||
self.assertEqual([item["id"] for item in collected], [item["id"] for item in sent])
|
||||
self.assertEqual(
|
||||
collected,
|
||||
sorted(collected, key=lambda item: (timestamp(item["created_at"]), item["id"])),
|
||||
)
|
||||
|
||||
def test_cursor_is_bound_to_its_chat(self) -> None:
|
||||
other = common.create_chat(self.alice.token, [self.alice.id])
|
||||
common.send(self.alice.token, other["id"], "a")
|
||||
common.send(self.alice.token, other["id"], "b")
|
||||
first = common.request(
|
||||
"GET", "/api/v1/chats/" + other["id"] + "/messages?limit=1", token=self.alice.token
|
||||
)
|
||||
cursor = first.json["next_cursor"]
|
||||
self.assertIsNotNone(cursor)
|
||||
response = common.request(
|
||||
"GET",
|
||||
"/api/v1/chats/" + self.chat["id"] + "/messages" + "?limit=1&cursor=" + cursor,
|
||||
token=self.alice.token,
|
||||
)
|
||||
common.expect_error(self, response, 400, "invalid_request")
|
||||
|
||||
def test_limit_and_cursor_validation(self) -> None:
|
||||
base = "/api/v1/chats/" + self.chat["id"] + "/messages"
|
||||
for query in ("?limit=0", "?limit=101", "?limit=abc", "?cursor=", "?cursor=not-a-cursor", "?limit=1&limit=2", "?unknown=1"):
|
||||
common.expect_error(
|
||||
self,
|
||||
common.request("GET", base + query, token=self.alice.token),
|
||||
400,
|
||||
"invalid_request",
|
||||
)
|
||||
|
||||
|
||||
class Authentication(Base):
|
||||
def test_basic_is_not_accepted_outside_session_creation(self) -> None:
|
||||
response = common.request(
|
||||
"GET", "/api/v1/users/me", headers=common.basic(self.alice.username, self.alice.password)
|
||||
)
|
||||
common.expect_error(self, response, 401, "unauthorized")
|
||||
self.assertTrue(response.header("WWW-Authenticate").lower().startswith("bearer"))
|
||||
|
||||
def test_missing_and_unknown_tokens(self) -> None:
|
||||
for headers in ({}, {"Authorization": "Bearer not-a-real-token"}):
|
||||
response = common.request("GET", "/api/v1/users/me", headers=headers)
|
||||
common.expect_error(self, response, 401, "unauthorized")
|
||||
self.assertTrue(response.header("WWW-Authenticate").lower().startswith("bearer"))
|
||||
|
||||
def test_x_user_id_is_ignored(self) -> None:
|
||||
response = common.request("GET", "/api/v1/users/me", headers={"X-User-Id": self.alice.id})
|
||||
common.expect_error(self, response, 401, "unauthorized")
|
||||
overridden = common.request(
|
||||
"GET",
|
||||
"/api/v1/users/me",
|
||||
token=self.alice.token,
|
||||
headers={"X-User-Id": self.eve.id},
|
||||
)
|
||||
self.assertEqual(overridden.status, 200)
|
||||
self.assertEqual(overridden.json["id"], self.alice.id)
|
||||
|
||||
def test_session_payload_and_expiry(self) -> None:
|
||||
response = common.login(self.alice.username, self.alice.password)
|
||||
self.assertEqual(response.status, 201)
|
||||
session = response.json
|
||||
self.assertEqual(set(session), {"token", "token_type", "expires_at", "user"})
|
||||
self.assertEqual(session["token_type"], "Bearer")
|
||||
self.assertGreaterEqual(len(session["token"]), 43)
|
||||
self.assertNotIn(self.alice.username, session["token"])
|
||||
self.assertNotEqual(session["token"], self.alice.id)
|
||||
self.assertEqual(session["user"], self.alice.user)
|
||||
self.assertGreater(timestamp(session["expires_at"]), datetime.now(timezone.utc))
|
||||
me = common.request("GET", "/api/v1/users/me", token=session["token"])
|
||||
self.assertEqual(me.status, 200)
|
||||
self.assertEqual(me.json, self.alice.user)
|
||||
|
||||
def test_wrong_password_and_unknown_user_are_indistinguishable(self) -> None:
|
||||
wrong = common.login(self.alice.username, "definitely-wrong-password")
|
||||
unknown = common.login("nobody_here_" + uuid.uuid4().hex[:8], "definitely-wrong-password")
|
||||
messages = []
|
||||
for response in (wrong, unknown):
|
||||
error = common.expect_error(self, response, 401, "unauthorized")
|
||||
self.assertTrue(
|
||||
response.header("WWW-Authenticate").lower().startswith("basic")
|
||||
)
|
||||
messages.append(error["message"])
|
||||
# Same message and challenge: a client cannot tell the accounts apart.
|
||||
self.assertEqual(messages[0], messages[1])
|
||||
|
||||
def test_logout_revokes_only_that_session(self) -> None:
|
||||
first = common.login(self.alice.username, self.alice.password).json["token"]
|
||||
second = common.login(self.alice.username, self.alice.password).json["token"]
|
||||
revoked = common.request("DELETE", "/api/v1/auth/sessions/current", token=first)
|
||||
self.assertEqual(revoked.status, 204)
|
||||
self.assertEqual(revoked.data, b"")
|
||||
common.expect_error(
|
||||
self,
|
||||
common.request("GET", "/api/v1/users/me", token=first),
|
||||
401,
|
||||
"unauthorized",
|
||||
)
|
||||
common.expect_error(
|
||||
self,
|
||||
common.request("DELETE", "/api/v1/auth/sessions/current", token=first),
|
||||
401,
|
||||
"unauthorized",
|
||||
)
|
||||
still_valid = common.request("GET", "/api/v1/users/me", token=second)
|
||||
self.assertEqual(still_valid.status, 200)
|
||||
|
||||
def test_session_endpoint_rejects_json_body_and_bad_basic(self) -> None:
|
||||
response = common.request(
|
||||
"POST",
|
||||
"/api/v1/auth/sessions",
|
||||
body={"username": self.alice.username, "password": self.alice.password},
|
||||
)
|
||||
common.expect_error(self, response, 401, "unauthorized")
|
||||
response = common.request(
|
||||
"POST", "/api/v1/auth/sessions", headers={"Authorization": "Basic !!!not-base64!!!"}
|
||||
)
|
||||
common.expect_error(self, response, 401, "unauthorized")
|
||||
self.assertTrue(response.header("WWW-Authenticate").lower().startswith("basic"))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,206 @@
|
||||
"""Dependency outage behaviour: Valkey down, then PostgreSQL down.
|
||||
|
||||
Requires Docker access on the same host as the Compose stand and is therefore
|
||||
opt-in:
|
||||
|
||||
STUDENT_DOCKER_TESTS=1 python -m unittest -v test_degradation
|
||||
|
||||
The test never removes volumes and restores both services in `finally`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import statistics
|
||||
import subprocess
|
||||
import time
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
import common
|
||||
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||
PROJECT = os.environ.get("COMPOSE_PROJECT_NAME", "mtusi2026lab2")
|
||||
ENV_FILE = os.environ.get("COMPOSE_ENV_FILE", ".env")
|
||||
ENABLED = os.environ.get("STUDENT_DOCKER_TESTS") == "1"
|
||||
|
||||
|
||||
def compose(*arguments: str, timeout: int = 120) -> None:
|
||||
result = subprocess.run(
|
||||
["docker", "compose", "-p", PROJECT, "--env-file", ENV_FILE, *arguments],
|
||||
cwd=str(REPO_ROOT),
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=timeout,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
raise AssertionError(
|
||||
"docker compose %s failed: %s" % (" ".join(arguments), result.stderr[-500:])
|
||||
)
|
||||
|
||||
|
||||
def wait_ready(expected: int, seconds: float) -> float:
|
||||
started = time.monotonic()
|
||||
deadline = started + seconds
|
||||
last = None
|
||||
while time.monotonic() < deadline:
|
||||
try:
|
||||
last = common.request("GET", "/health/ready", timeout=5)
|
||||
if last.status == expected:
|
||||
return time.monotonic() - started
|
||||
except OSError:
|
||||
# The TCP listener may be replaced during `compose restart`.
|
||||
pass
|
||||
time.sleep(0.5)
|
||||
raise AssertionError(
|
||||
"readiness did not reach %s within %.0fs (last=%s %r)"
|
||||
% (expected, seconds, last.status if last else None, last.json if last else None)
|
||||
)
|
||||
|
||||
|
||||
@unittest.skipUnless(ENABLED, "set STUDENT_DOCKER_TESTS=1 to stop real containers")
|
||||
class DependencyOutages(unittest.TestCase):
|
||||
def test_logout_remains_revoked_after_cache_outage_and_api_restart(self) -> None:
|
||||
"""A stale Valkey hit must never resurrect a PostgreSQL-revoked token."""
|
||||
alice = common.actor("revoked")
|
||||
chat = common.create_chat(alice.token, [alice.id], "Durable logout")
|
||||
# Store the live session in Valkey before its simulated outage.
|
||||
self.assertEqual(common.request("GET", "/api/v1/users/me", token=alice.token).status, 200)
|
||||
try:
|
||||
compose("stop", "valkey")
|
||||
logout = common.request("DELETE", "/api/v1/auth/sessions/current", token=alice.token)
|
||||
self.assertEqual(logout.status, 204, logout.json)
|
||||
|
||||
# Lose the local pending tombstone, then restore Valkey's old volume.
|
||||
compose("restart", "api")
|
||||
compose("start", "valkey")
|
||||
wait_ready(200, 90)
|
||||
|
||||
common.expect_error(
|
||||
self, common.request("GET", "/api/v1/users/me", token=alice.token), 401, "unauthorized"
|
||||
)
|
||||
common.expect_error(
|
||||
self,
|
||||
common.request("GET", "/api/v1/chats/" + chat["id"], token=alice.token),
|
||||
401,
|
||||
"unauthorized",
|
||||
)
|
||||
common.expect_error(
|
||||
self,
|
||||
common.request(
|
||||
"POST",
|
||||
"/api/v1/chats/" + chat["id"] + "/messages",
|
||||
body={"text": "must not be accepted"},
|
||||
token=alice.token,
|
||||
),
|
||||
401,
|
||||
"unauthorized",
|
||||
)
|
||||
finally:
|
||||
compose("start", "valkey", "postgres", "api")
|
||||
try:
|
||||
wait_ready(200, 90)
|
||||
except AssertionError:
|
||||
pass
|
||||
|
||||
def test_valkey_loss_degrades_then_postgres_loss_is_bounded(self) -> None:
|
||||
alice = common.actor("degrade")
|
||||
chat = common.create_chat(alice.token, [alice.id], "Устойчивость")
|
||||
# Warm the session cache.
|
||||
self.assertEqual(common.request("GET", "/api/v1/users/me", token=alice.token).status, 200)
|
||||
|
||||
cached = self._sample_latency(alice.token, 60)
|
||||
summary = {"cached_p50_ms": round(cached["p50"], 2)}
|
||||
try:
|
||||
# 1. Valkey is a cache: auth still works, without DB-free fast path.
|
||||
compose("stop", "valkey")
|
||||
ready = common.request("GET", "/health/ready")
|
||||
self.assertEqual(
|
||||
ready.status, 200, "session cache must not be a readiness dependency"
|
||||
)
|
||||
me = common.request("GET", "/api/v1/users/me", token=alice.token)
|
||||
self.assertEqual(me.status, 200, me.json)
|
||||
sent = common.request(
|
||||
"POST",
|
||||
"/api/v1/chats/" + chat["id"] + "/messages",
|
||||
body={"text": "без кэша"},
|
||||
token=alice.token,
|
||||
)
|
||||
self.assertEqual(sent.status, 201, sent.json)
|
||||
without_cache = self._sample_latency(alice.token, 60)
|
||||
summary["cache_down_p50_ms"] = round(without_cache["p50"], 2)
|
||||
summary["cache_down_p95_ms"] = round(without_cache["p95"], 2)
|
||||
|
||||
compose("start", "valkey")
|
||||
time.sleep(3)
|
||||
self.assertEqual(common.request("GET", "/health/ready").status, 200)
|
||||
|
||||
# 2. PostgreSQL is the source of truth: bounded errors, never a 200
|
||||
# for a token the server cannot verify, and never a raw 500.
|
||||
compose("stop", "postgres")
|
||||
ready = common.request("GET", "/health/ready")
|
||||
self.assertEqual(ready.status, 503, ready.json)
|
||||
self.assertEqual(ready.json["error"]["code"], "unavailable")
|
||||
|
||||
started = time.monotonic()
|
||||
write = common.request(
|
||||
"POST",
|
||||
"/api/v1/chats/" + chat["id"] + "/messages",
|
||||
body={"text": "без базы"},
|
||||
token=alice.token,
|
||||
timeout=30,
|
||||
)
|
||||
bounded = time.monotonic() - started
|
||||
self.assertEqual(write.status, 503, write.json)
|
||||
self.assertEqual(write.json["error"]["code"], "unavailable")
|
||||
self.assertLess(bounded, 10, "database failure must fail fast")
|
||||
summary["db_down_write_seconds"] = round(bounded, 2)
|
||||
|
||||
for leaked in ("psycopg", "postgresql://", "password", "Traceback"):
|
||||
self.assertNotIn(leaked, json.dumps(write.json))
|
||||
unknown = common.request(
|
||||
"GET", "/api/v1/users/me", token="unknown-token-value", timeout=30
|
||||
)
|
||||
self.assertIn(unknown.status, (401, 503))
|
||||
self.assertNotEqual(unknown.status, 200)
|
||||
|
||||
# 3. Recovery keeps the same session and the stored chat.
|
||||
compose("start", "postgres")
|
||||
summary["recovery_seconds"] = round(wait_ready(200, 90), 2)
|
||||
after = common.request("GET", "/api/v1/chats/" + chat["id"], token=alice.token)
|
||||
self.assertEqual(after.status, 200, after.json)
|
||||
recovered = common.request(
|
||||
"POST",
|
||||
"/api/v1/chats/" + chat["id"] + "/messages",
|
||||
body={"text": "после восстановления"},
|
||||
token=alice.token,
|
||||
)
|
||||
self.assertEqual(recovered.status, 201, recovered.json)
|
||||
messages = common.history(alice.token, chat["id"])
|
||||
self.assertIn("после восстановления", [message["text"] for message in messages])
|
||||
finally:
|
||||
compose("start", "valkey", "postgres")
|
||||
try:
|
||||
wait_ready(200, 90)
|
||||
except AssertionError:
|
||||
pass
|
||||
print("\n outage summary: " + json.dumps(summary, ensure_ascii=False))
|
||||
|
||||
def _sample_latency(self, token: str, count: int) -> dict[str, float]:
|
||||
samples = []
|
||||
for _ in range(count):
|
||||
started = time.monotonic()
|
||||
response = common.request("GET", "/api/v1/users/me", token=token, timeout=30)
|
||||
samples.append(time.monotonic() - started)
|
||||
self.assertEqual(response.status, 200, response.json)
|
||||
ordered = sorted(samples)
|
||||
return {
|
||||
"p50": statistics.median(samples) * 1000,
|
||||
"p95": ordered[max(0, int(len(ordered) * 0.95) - 1)] * 1000,
|
||||
}
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,99 @@
|
||||
"""Session expiry with SESSION_TTL_SECONDS=2, plus logout and bad credentials.
|
||||
|
||||
Run against the dedicated short-TTL stand:
|
||||
|
||||
scripts/ttl-stack.sh start
|
||||
BASE_URL=http://localhost:8081 python -m unittest -v test_expiry
|
||||
scripts/ttl-stack.sh stop
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import statistics
|
||||
import time
|
||||
import unittest
|
||||
from datetime import datetime, timezone
|
||||
|
||||
import common
|
||||
|
||||
|
||||
def expires_at(session: dict) -> datetime:
|
||||
return datetime.fromisoformat(session["expires_at"][:-1] + "+00:00")
|
||||
|
||||
|
||||
class ShortTtlSessions(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls) -> None:
|
||||
ready = common.request("GET", "/health/ready")
|
||||
if ready.status != 200:
|
||||
raise AssertionError("stand is not ready: %r" % (ready.json,))
|
||||
|
||||
def test_expiry_is_enforced_and_cache_does_not_extend_it(self) -> None:
|
||||
user = common.actor("ttl")
|
||||
started = datetime.now(timezone.utc)
|
||||
session = common.login(user.username, user.password)
|
||||
self.assertEqual(session.status, 201)
|
||||
lifetime = (expires_at(session.json) - started).total_seconds()
|
||||
self.assertLessEqual(lifetime, 5.0, "stand does not use a short TTL")
|
||||
|
||||
# Two reads: the second one is served from the Valkey cache, so the
|
||||
# expiry check below covers the cached path as well.
|
||||
for _ in range(2):
|
||||
self.assertEqual(
|
||||
common.request("GET", "/api/v1/users/me", token=session.json["token"]).status,
|
||||
200,
|
||||
)
|
||||
|
||||
time.sleep(3.2)
|
||||
expired = common.request("GET", "/api/v1/users/me", token=session.json["token"])
|
||||
common.expect_error(self, expired, 401, "unauthorized")
|
||||
self.assertTrue(expired.header("WWW-Authenticate").lower().startswith("bearer"))
|
||||
revoked = common.request(
|
||||
"DELETE", "/api/v1/auth/sessions/current", token=session.json["token"]
|
||||
)
|
||||
common.expect_error(self, revoked, 401, "unauthorized")
|
||||
|
||||
# A fresh login still works, so expiry removed only the old session.
|
||||
again = common.login(user.username, user.password)
|
||||
self.assertEqual(again.status, 201)
|
||||
self.assertEqual(
|
||||
common.request("GET", "/api/v1/users/me", token=again.json["token"]).status,
|
||||
200,
|
||||
)
|
||||
|
||||
def test_logout_is_immediate_even_when_the_session_is_cached(self) -> None:
|
||||
user = common.actor("ttllogout")
|
||||
token = user.token
|
||||
self.assertEqual(common.request("GET", "/api/v1/users/me", token=token).status, 200)
|
||||
self.assertEqual(
|
||||
common.request("DELETE", "/api/v1/auth/sessions/current", token=token).status,
|
||||
204,
|
||||
)
|
||||
outcome = common.request("GET", "/api/v1/users/me", token=token)
|
||||
common.expect_error(self, outcome, 401, "unauthorized")
|
||||
|
||||
def test_wrong_password_issues_no_session(self) -> None:
|
||||
user = common.actor("ttlwrong")
|
||||
denied = common.login(user.username, "not-the-password")
|
||||
common.expect_error(self, denied, 401, "unauthorized")
|
||||
self.assertTrue(denied.header("WWW-Authenticate").lower().startswith("basic"))
|
||||
|
||||
def test_session_lookup_latency_is_bounded(self) -> None:
|
||||
user = common.actor("ttllatency")
|
||||
samples = []
|
||||
for _ in range(30):
|
||||
started = time.monotonic()
|
||||
response = common.request("GET", "/api/v1/users/me", token=user.token, timeout=5)
|
||||
samples.append(time.monotonic() - started)
|
||||
self.assertEqual(response.status, 200)
|
||||
print(
|
||||
"\n /users/me warm cache p50=%.1fms p95=%.1fms"
|
||||
% (
|
||||
statistics.median(samples) * 1000,
|
||||
sorted(samples)[int(len(samples) * 0.95) - 1] * 1000,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,142 @@
|
||||
"""Race, rollback and pool tests against the real PostgreSQL stand.
|
||||
|
||||
These are the checks a code reviewer would ask for on top of the smoke test:
|
||||
concurrent writers must not lose or duplicate messages, the username uniqueness
|
||||
must be decided by the database (not by a read-then-write check in Python), and
|
||||
a failed composite operation must leave nothing behind.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import threading
|
||||
import time
|
||||
import unittest
|
||||
import uuid
|
||||
|
||||
import common
|
||||
|
||||
|
||||
class ConcurrentWrites(unittest.TestCase):
|
||||
def test_100_messages_from_10_clients_are_exactly_100(self) -> None:
|
||||
alice = common.actor("cwalice")
|
||||
bob = common.actor("cwbob")
|
||||
chat = common.create_chat(alice.token, [alice.id, bob.id], "Гонка сообщений")
|
||||
writers = 10
|
||||
per_writer = 10
|
||||
barrier = threading.Barrier(writers)
|
||||
errors: list[str] = []
|
||||
sent: list[str] = []
|
||||
lock = threading.Lock()
|
||||
|
||||
def writer(index: int) -> None:
|
||||
token = alice.token if index % 2 else bob.token
|
||||
barrier.wait()
|
||||
for number in range(per_writer):
|
||||
text = "race-%d-%d" % (index, number)
|
||||
try:
|
||||
common.send(token, chat["id"], text)
|
||||
with lock:
|
||||
sent.append(text)
|
||||
except AssertionError as error: # pragma: no cover - failure detail
|
||||
with lock:
|
||||
errors.append(str(error))
|
||||
|
||||
threads = [threading.Thread(target=writer, args=(index,)) for index in range(writers)]
|
||||
started = time.monotonic()
|
||||
for thread in threads:
|
||||
thread.start()
|
||||
for thread in threads:
|
||||
thread.join(timeout=120)
|
||||
elapsed = time.monotonic() - started
|
||||
self.assertEqual(errors, [])
|
||||
self.assertEqual(len(sent), writers * per_writer)
|
||||
|
||||
messages = common.history(alice.token, chat["id"], limit=100)
|
||||
self.assertEqual(len(messages), writers * per_writer)
|
||||
self.assertEqual(len({message["id"] for message in messages}), len(messages))
|
||||
self.assertEqual(sorted(message["text"] for message in messages), sorted(sent))
|
||||
print("\n concurrent writes: %d messages from %d clients in %.2fs"
|
||||
% (len(messages), writers, elapsed))
|
||||
|
||||
def test_duplicate_username_race_creates_one_user(self) -> None:
|
||||
registration = common.new_registration("race")
|
||||
attempts = 12
|
||||
barrier = threading.Barrier(attempts)
|
||||
statuses: list[int] = []
|
||||
lock = threading.Lock()
|
||||
|
||||
def attempt() -> None:
|
||||
barrier.wait()
|
||||
response = common.register(dict(registration))
|
||||
with lock:
|
||||
statuses.append(response.status)
|
||||
|
||||
threads = [threading.Thread(target=attempt) for _ in range(attempts)]
|
||||
for thread in threads:
|
||||
thread.start()
|
||||
for thread in threads:
|
||||
thread.join(timeout=60)
|
||||
self.assertEqual(statuses.count(201), 1, statuses)
|
||||
self.assertEqual(statuses.count(409), attempts - 1, statuses)
|
||||
# The single winner is usable, so exactly one row exists.
|
||||
session = common.login(registration["username"], registration["password"])
|
||||
self.assertEqual(session.status, 201)
|
||||
self.assertEqual(session.json["user"]["username"], registration["username"])
|
||||
|
||||
def test_chat_creation_rolls_back_on_unknown_member(self) -> None:
|
||||
alice = common.actor("rollback")
|
||||
before = self._chat_titles(alice.token)
|
||||
response = common.request(
|
||||
"POST",
|
||||
"/api/v1/chats",
|
||||
body={"title": "Атомарный чат", "member_ids": [alice.id, str(uuid.uuid4())]},
|
||||
token=alice.token,
|
||||
)
|
||||
common.expect_error(self, response, 400, "invalid_request")
|
||||
after = self._chat_titles(alice.token)
|
||||
# The chat row inserted before the failing membership step is gone.
|
||||
self.assertEqual(before, after)
|
||||
self.assertNotIn("Атомарный чат", after)
|
||||
|
||||
def _chat_titles(self, token: str) -> list[str]:
|
||||
titles: list[str] = []
|
||||
cursor = None
|
||||
for _ in range(50):
|
||||
path = "/api/v1/chats?limit=100"
|
||||
if cursor:
|
||||
path += "&cursor=" + cursor
|
||||
response = common.request("GET", path, token=token)
|
||||
self.assertEqual(response.status, 200)
|
||||
titles.extend(item["title"] for item in response.json["items"])
|
||||
cursor = response.json["next_cursor"]
|
||||
if cursor is None:
|
||||
return titles
|
||||
self.fail("chat pagination did not terminate")
|
||||
|
||||
def test_bounded_pool_serves_parallel_reads(self) -> None:
|
||||
alice = common.actor("pool")
|
||||
parallel = 50
|
||||
barrier = threading.Barrier(parallel)
|
||||
statuses: list[int] = []
|
||||
lock = threading.Lock()
|
||||
|
||||
def read() -> None:
|
||||
barrier.wait()
|
||||
# Chat lists are not cached, so this exercises the bounded SQL pool.
|
||||
response = common.request(
|
||||
"GET", "/api/v1/chats?limit=100", token=alice.token, timeout=10
|
||||
)
|
||||
with lock:
|
||||
statuses.append(response.status)
|
||||
|
||||
threads = [threading.Thread(target=read) for _ in range(parallel)]
|
||||
for thread in threads:
|
||||
thread.start()
|
||||
for thread in threads:
|
||||
thread.join(timeout=60)
|
||||
self.assertEqual(len(statuses), parallel)
|
||||
self.assertEqual(statuses, [200] * parallel)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in New Issue
Block a user