This commit is contained in:
@@ -9,3 +9,9 @@ DATABASE_URL_FILE=/run/secrets/database_url
|
||||
# Optional on 3, required use on 4 in lab2; Redis OR Valkey:
|
||||
SESSION_STORE=postgres
|
||||
CACHE_URL_FILE=/run/secrets/cache_url
|
||||
# --- Actual lab2 implementation (see REPORT.md) ---
|
||||
# APP_PORT is also the host port published on 127.0.0.1 by compose.yaml.
|
||||
# SESSION_STORE=postgres: sessions live in PostgreSQL; Valkey (CACHE_URL_FILE) caches lookups.
|
||||
SESSION_CACHE_MAX_SECONDS=300
|
||||
DB_POOL_MAX=10
|
||||
SHORT_TTL_PORT=8081
|
||||
|
||||
+23
@@ -0,0 +1,23 @@
|
||||
FROM python:3.12.8-slim-bookworm
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY requirements.txt .
|
||||
RUN pip install --no-cache-dir -r requirements.txt
|
||||
|
||||
COPY app ./app
|
||||
COPY migrations ./migrations
|
||||
|
||||
RUN useradd --system --no-create-home --uid 10001 appuser
|
||||
USER appuser
|
||||
|
||||
ENV APP_HOST=0.0.0.0 \
|
||||
APP_PORT=8080 \
|
||||
PYTHONUNBUFFERED=1
|
||||
|
||||
EXPOSE 8080
|
||||
|
||||
HEALTHCHECK --interval=10s --timeout=3s --start-period=5s --retries=3 \
|
||||
CMD python -c "import os,urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:'+os.environ.get('APP_PORT','8080')+'/health/ready',timeout=2).status==200 else 1)"
|
||||
|
||||
CMD ["sh", "-c", "exec uvicorn app.main:app --host ${APP_HOST} --port ${APP_PORT} --timeout-graceful-shutdown 8"]
|
||||
@@ -4,7 +4,7 @@ CA_FILE ?=
|
||||
ACTION_SCRIPT ?=
|
||||
TLS_ARGS = $(if $(CA_FILE),--ca-file "$(CA_FILE)",)
|
||||
|
||||
.PHONY: help check validate test load test-persistence
|
||||
.PHONY: help check validate test load test-persistence secrets up down test-student explain
|
||||
|
||||
help:
|
||||
@echo "check: offline structure; validate: full OpenAPI (requirements-dev.txt); test: running API; load: synthetic writes"
|
||||
@@ -24,3 +24,19 @@ load:
|
||||
test-persistence:
|
||||
@test -n "$(ACTION_SCRIPT)" || (echo "Set ACTION_SCRIPT to your implemented executable restart hook"; exit 2)
|
||||
$(PYTHON) tests/resilience.py --mode persistence --action-script "$(ACTION_SCRIPT)" --base-url "$(BASE_URL)" $(TLS_ARGS)
|
||||
|
||||
# --- Student implementation helpers (see REPORT.md) ---
|
||||
secrets:
|
||||
sh scripts/init-secrets.sh
|
||||
|
||||
up: secrets
|
||||
docker compose up -d --build --wait
|
||||
|
||||
down:
|
||||
docker compose --profile ttl-test down
|
||||
|
||||
test-student:
|
||||
BASE_URL="$(BASE_URL)" sh student-tests/run_all.sh
|
||||
|
||||
explain:
|
||||
sh student-tests/explain_history.sh
|
||||
|
||||
@@ -1,37 +1,142 @@
|
||||
# Отчёт — лабораторная 2
|
||||
|
||||
- Автор, группа: TODO
|
||||
- Целевая оценка; выбранный трек 5, если нужен: TODO
|
||||
- Целевая оценка: **4** (трек 5 не заявлен)
|
||||
- Commit/PR: TODO после публикации
|
||||
- Использованные источники, библиотеки и помощь ИИ: TODO
|
||||
- Использованные источники, библиотеки и помощь ИИ: FastAPI, uvicorn, pydantic, asyncpg, redis-py (клиент к Valkey), argon2-cffi; документация PostgreSQL (transactions, EXPLAIN), Valkey persistence, RFC 7617. Код lab1 перенесён из собственного решения. При переносе на PostgreSQL, написании тестов и отчёта использовался ИИ-ассистент Claude (Anthropic); решения проверены и объяснимы автором.
|
||||
|
||||
## Запуск из чистого clone
|
||||
|
||||
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
|
||||
Требования: Docker Engine с Compose v2 (проверено: Docker 28.0.1, Compose 2.33.1), Python 3.9+ для тестов (проверено на 3.14), `make`.
|
||||
|
||||
| Роль | Образ | Версия |
|
||||
| --- | --- | --- |
|
||||
| api / migrate | `mtusi-messenger:lab2`, сборка из `Dockerfile` | `python:3.12.8-slim-bookworm`, зависимости закреплены в `requirements.txt` |
|
||||
| postgres | `postgres:17.6-alpine3.22` | PostgreSQL 17.6 |
|
||||
| valkey | `valkey/valkey:8.1.4-alpine3.22` | Valkey 8.1.4 |
|
||||
|
||||
```sh
|
||||
cp .env.example .env # при занятом 8080 поменять APP_PORT (у автора 18080)
|
||||
make secrets # scripts/init-secrets.sh: случайные пароли в ./secrets (в .gitignore)
|
||||
docker compose --env-file .env config --quiet
|
||||
make up # docker compose up -d --build --wait: postgres -> migrate -> api; ждёт healthcheck /health/ready
|
||||
make check
|
||||
make test BASE_URL=http://localhost:8080
|
||||
make test-persistence ACTION_SCRIPT=scripts/restart.sh BASE_URL=http://localhost:8080
|
||||
make test-student BASE_URL=http://localhost:8080 # собственные тесты; останавливает/запускает контейнеры
|
||||
make explain # засев 70 000 синтетических сообщений + EXPLAIN
|
||||
```
|
||||
|
||||
- Миграции: отдельный одноразовый сервис `migrate` (`python -m app.migrate`). API стартует только после его успешного завершения. Повторный запуск безопасен (`schema_migrations` + `pg_advisory_lock`).
|
||||
- Безопасная остановка без потери данных: `make down` (= `docker compose --profile ttl-test down`, volume `pgdata` сохраняется).
|
||||
- **Намеренное удаление учебных данных:** `docker compose --profile ttl-test down -v` (удаляет volume `messenger-lab2_pgdata`; необратимо).
|
||||
- Действия оператора: только `make secrets` перед первым запуском. Секреты в образ и Git не попадают.
|
||||
|
||||
## Архитектура и решения
|
||||
|
||||
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab1, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
|
||||
```
|
||||
клиент ──HTTP──> api (FastAPI/uvicorn, 1 процесс)
|
||||
├──> postgres : источник истины — users, chats, chat_members, messages, sessions
|
||||
└──> valkey : кэш поиска сессии (только ускорение, может пропасть в любой момент)
|
||||
migrate (one-shot) ──> postgres : применение migrations/*.sql
|
||||
```
|
||||
|
||||
**Перенесено из lab1:** модели pydantic, формат ошибок и `X-Request-Id`, keyset-курсор, привязанный к ресурсу, проверки контракта (пустой текст, дубликаты участников, 404 на чужой чат), Dockerfile (non-root, graceful SIGTERM). In-memory `Store` заменён на репозиторий PostgreSQL (`app/storage.py`).
|
||||
|
||||
**Изменено по контракту lab2:** `password` при регистрации, `POST /auth/sessions` через Basic, Bearer на всех бизнес-endpoint, `DELETE /auth/sessions/current`. `X-User-Id` не читается вообще.
|
||||
|
||||
### Схема БД (`migrations/0001_init.sql`)
|
||||
|
||||
| Таблица | Ключи и ограничения | Индексы |
|
||||
| --- | --- | --- |
|
||||
| users | PK id; `UNIQUE(username)`; CHECK формата username и длины display_name | уникальный по username |
|
||||
| chats | PK id; CHECK длины title | — |
|
||||
| chat_members | PK (chat_id, user_id); FK на chats и users; `position` хранит порядок member_ids | `(user_id, chat_id)` для списка чатов пользователя |
|
||||
| messages | PK id; **FK (chat_id, sender_id) → chat_members**: отправитель обязан быть участником на уровне БД; CHECK длины текста | `(chat_id, created_at, id)` для истории |
|
||||
| sessions | PK token_hash (sha256 токена, сам токен не хранится); FK user_id; expires_at, revoked_at | `(expires_at)` |
|
||||
|
||||
Время (`created_at`, `expires_at`) назначает PostgreSQL (`clock_timestamp()`), в API выдаётся RFC 3339 UTC с микросекундами. Порядок `(created_at, id)`; сравнение uuid в PostgreSQL побайтовое и совпадает с порядком UUID-int в тестах.
|
||||
|
||||
### Транзакционные границы
|
||||
|
||||
- **Создание чата** — одна транзакция: `INSERT chats` + `INSERT chat_members … FROM unnest($2::uuid[])`. Неизвестный участник отлавливает FK внутри транзакции → `ROLLBACK` → 400. Предварительного SELECT нет, поэтому откат реально выполняется (тест `db_integrity_test.py`).
|
||||
- **Регистрация** — один `INSERT`; гонку одинаковых username решает `UNIQUE` (`UniqueViolationError` → 409), а не проверка «SELECT, потом INSERT».
|
||||
- **Отправка сообщения** — одна инструкция `INSERT … SELECT … WHERE EXISTS(участник)`: проверка доступа и запись атомарны.
|
||||
- **Logout** — `UPDATE sessions SET revoked_at` (истина), затем надгробие в Valkey.
|
||||
- `COMMIT` подтверждает запись в WAL; при настройках по умолчанию (`fsync=on`, `synchronous_commit=on`) подтверждённое сообщение переживает падение процесса PostgreSQL. Ответ 201 отправляется только после COMMIT.
|
||||
|
||||
### Аутентификация и сессии
|
||||
|
||||
- Пароль: **Argon2id** (argon2-cffi, профиль RFC 9106 low-memory: t=3, m=64 MiB, p=4). Хеширование в thread pool, не более 4 одновременно (`PASSWORD_HASH_CONCURRENCY`), чтобы ограничить RAM. Для несуществующего username проверяется фиктивный хеш, поэтому время ответа одинаковое.
|
||||
- Basic разбирается строго: base64 → UTF-8 → разделение по **первому** `:` (двоеточие в пароле допустимо), username только ASCII, пароль 12–128 code points. Любая ошибка даёт одинаковый 401 с `WWW-Authenticate: Basic realm="messenger", charset="UTF-8"`.
|
||||
- Токен — `secrets.token_urlsafe(32)` (256 бит). В БД и кэше хранится только `sha256(token)`. Быстрый хеш здесь уместен: токен случайный, перебирать его бессмысленно, в отличие от человеческого пароля, которому нужен медленный KDF.
|
||||
- Срок жизни `SESSION_TTL_SECONDS` (по умолчанию 3600) задаётся в БД как `expires_at`, поэтому рестарт его не продлевает.
|
||||
|
||||
### Роль Valkey (уровень 4): кэш сессий
|
||||
|
||||
- **Источник истины — PostgreSQL.** Valkey хранит `session:<sha256 hex>` → `{user, expires_at}`, чтобы не ходить в БД на каждом запросе; `/users/me` обслуживается вообще без БД.
|
||||
- **Expiry:** TTL записи = `min(остаток сессии, SESSION_CACHE_MAX_SECONDS=300)`, плюс при чтении из кэша сравнивается `expires_at`. Просроченная сессия не принимается ни из кэша, ни из БД.
|
||||
- **Инвалидация:** при logout сначала `revoked_at` в PostgreSQL, затем в Valkey пишется **надгробие** `revoked` (а не DEL). Заполнение кэша идёт через `SET NX`, поэтому параллельный запрос, прочитавший БД до отзыва, не сможет вернуть сессию в кэш.
|
||||
- **Persistence не нужна и выключена** (`save ""`, `appendonly no`): после flush или рестарта Valkey первая проверка идёт в PostgreSQL. Потеря кэша ничего не ломает, это подтверждает тест сохранности, где Valkey тоже перезапускается.
|
||||
- **Отказ Valkey:** таймаут 0.3 с, затем circuit breaker на 5 с, в течение которых кэш не используется. Readiness при этом остаётся 200: Valkey не критичен.
|
||||
- Конфигурация с `requirepass` лежит в секрете `secrets/valkey.conf`, порт наружу не публикуется.
|
||||
|
||||
### Устойчивость к отказам
|
||||
|
||||
- asyncpg pool: `max_size=10`, таймауты подключения 3 с, захвата соединения 3 с, команды 5 с. `min_size=0` позволяет API стартовать при недоступной БД (тогда readiness = 503).
|
||||
- Недоступность или рестарт PostgreSQL (`OSError`, `PostgresConnectionError`, `InterfaceError`, `TooManyConnections`, admin shutdown, statement timeout) превращается в `503 unavailable` без деталей. Без токена по-прежнему 401. Токен, который нельзя проверить (нет в кэше, БД недоступна), получает 503 и **никогда** не считается аутентифицированным.
|
||||
- Readiness: 200 только при успешном `SELECT 1` в PostgreSQL. Liveness всегда 200.
|
||||
- Все SQL-запросы параметризованы (`$1…`), строк, склеенных из пользовательского ввода, нет.
|
||||
|
||||
### Переменные `.env.example` → реальная конфигурация
|
||||
|
||||
| Переменная | Где используется |
|
||||
| --- | --- |
|
||||
| APP_HOST / APP_PORT | uvicorn внутри контейнера слушает 0.0.0.0:8080; `APP_PORT` в `.env` — порт хоста (`127.0.0.1:${APP_PORT}`) |
|
||||
| APP_ENV | передаётся в контейнер, логики не меняет |
|
||||
| SESSION_TTL_SECONDS | срок сессии; у `api-short-ttl` = 2 |
|
||||
| DATABASE_URL_FILE | `/run/secrets/database_url` (compose secret из `secrets/database_url`) |
|
||||
| SESSION_STORE=postgres | сессии хранятся в PostgreSQL |
|
||||
| CACHE_URL_FILE | `/run/secrets/cache_url`, URL Valkey; без него кэш отключён |
|
||||
| SESSION_CACHE_MAX_SECONDS, DB_POOL_MAX, SHORT_TTL_PORT | добавлены этой работой (см. выше) |
|
||||
|
||||
## Доказательства критериев
|
||||
|
||||
Все файлы в `evidence/` сняты 2026-10-07 на MacBook (Apple Silicon, Docker Desktop, без CPU/RAM-лимитов в compose). Порт стенда автора — 18080, потому что 8080 занят другим проектом.
|
||||
|
||||
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
|
||||
| --- | --- | --- | --- |
|
||||
| Публичный smoke | make test | TODO | TODO |
|
||||
| Остальные пункты уровня 3 | TODO | TODO | TODO |
|
||||
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
|
||||
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
|
||||
|
||||
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
|
||||
| Публичный smoke | `make check`, `make test` | PASS обоих | `evidence/smoke.txt` |
|
||||
| 3: данные и сессии переживают рестарт API **и хранилищ** | `make test-persistence ACTION_SCRIPT=scripts/restart.sh` | PASS; `StartedAt` api/postgres/valkey изменились, volume тот же, тот же токен читает данные | `evidence/persistence.txt` |
|
||||
| 3: миграции с нуля и повторно | `docker compose -p lab2-clean up` на пустом volume, затем ещё `run migrate` | `1 applied now, 0 before`, затем `0 applied now, 1 before`; smoke PASS на новом стенде | `evidence/fresh_migrations.txt` |
|
||||
| 3: PK/FK, UNIQUE username, атомарное создание чата | `student-tests/db_integrity_test.py` | чат с неизвестным участником → 400, в `chats` 0 строк с этим title и 0 чатов без участников | `evidence/student_tests.txt` |
|
||||
| 3: Basic только на /auth/sessions, Bearer, X-User-Id игнорируется, logout, KDF | `auth_test.py`, smoke, `db_integrity_test.py` | Basic/X-User-Id на бизнес-endpoint → 401; повторный logout → 401; hash начинается с `$argon2id$` | `evidence/student_tests.txt` |
|
||||
| 3: secrets/volumes не в Git | `git status`, `.gitignore` (`secrets/`, `.env`) | секреты только в `./secrets`, в репозитории нет | `.gitignore` |
|
||||
| 4: Valkey реально используется, TTL, инвалидация | `dependency_test.py` | ключи `session:*` с TTL ≤ 300, после logout значение `revoked` | `evidence/student_tests.txt` |
|
||||
| 4: параметризованные запросы, ограниченный pool, таймауты | код `app/storage.py`, `app/config.py` | `$n`-параметры, `max_size=10`, таймауты 3/3/5 с | — |
|
||||
| 4: гонка регистрации | `db_integrity_test.py` | 20 параллельных запросов → 1×201, 19×409, 1 строка | `evidence/student_tests.txt` |
|
||||
| 4: rollback составной операции | `db_integrity_test.py` | см. строку «атомарное создание чата» | `evidence/student_tests.txt` |
|
||||
| 4: индекс истории + EXPLAIN на ≥10 000 | `make explain` | 70 000 сообщений, в чате 20 000; с индексом Index Scan, 4 буфера, 0.03 мс на первой и глубокой (#15000) странице; без индекса Seq Scan + Sort, 940 буферов, 3–4 мс | `evidence/explain_history.txt` |
|
||||
| 4: expiry при коротком TTL | `auth_test.py` против `api-short-ttl` (TTL=2) | через 3 с → 401 (и из кэша) | `evidence/student_tests.txt` |
|
||||
| 4: logout, неверный пароль | `auth_test.py` | неверный пароль и неизвестный user дают одинаковый 401 | `evidence/student_tests.txt` |
|
||||
| 4: недоступность Redis/БД | `dependency_test.py` | Valkey stop: ready=200, запросы ~4 мс; PostgreSQL stop: live=200, ready и бизнес-запросы 503 мгновенно, непроверенный токен → 503, без токена 401; после start всё восстанавливается без рестарта API | `evidence/student_tests.txt` |
|
||||
| Graceful shutdown (из lab1) | `sigterm_test.sh` | `docker compose stop api` → exit 0 за < 1 с | `evidence/student_tests.txt` |
|
||||
|
||||
## Эксперименты и отказы
|
||||
|
||||
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
|
||||
|
||||
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
|
||||
| --- | --- | --- | --- |
|
||||
| TODO | TODO | TODO | TODO |
|
||||
| `scripts/restart.sh`: stop api, valkey, postgres → up | данные и сессия на месте, токен тот же | PASS, кэш пустой, сессии берутся из PostgreSQL | ~11 с весь hook / потерь нет |
|
||||
| `docker compose stop valkey` | API работает без кэша | 200, ~4 мс на запрос (breaker), revoked-токен 401 | мгновенно / потерь нет (кэш) |
|
||||
| `docker compose stop postgres` | 503 без зависаний, анонимный доступ не появляется | ready 503, бизнес 503, live 200, всё за < 0.1 с | API восстанавливается сам после `start postgres`, без своего рестарта; подтверждённые данные целы |
|
||||
| SIGTERM api | graceful exit 0 | exit 0, «Application shutdown complete» | `docker compose up -d --wait api` |
|
||||
|
||||
## Ограничения и следующие шаги
|
||||
|
||||
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
|
||||
- Один Docker-host, по одному PostgreSQL и Valkey, без репликации: потеря host или диска = потеря сервиса (бэкапы не настроены, это трек 2 пятёрки).
|
||||
- Если logout пришёлся на момент сетевой недоступности Valkey, а сам Valkey при этом не перезапускался, старая запись в кэше может жить до `SESSION_CACHE_MAX_SECONDS` (≤ 5 мин). Это осознанная граница: TTL кэша ограничен именно поэтому.
|
||||
- Просроченные строки `sessions` не удаляются автоматически (есть индекс `expires_at` для будущей периодической очистки).
|
||||
- `/health/ready` не проверяет Valkey намеренно.
|
||||
- Тело запроса проверяется до аутентификации: на неавторизованный запрос с битым JSON вернётся 400, а не 401 (особенность FastAPI). Это не раскрывает данных.
|
||||
- `make test` создаёт синтетических пользователей при каждом запуске; удаления в API нет.
|
||||
- Нагрузочное измерение (`make load`) не делалось: для уровня 4 не требуется.
|
||||
|
||||
+122
@@ -0,0 +1,122 @@
|
||||
"""Passwords (Argon2id), Basic login and opaque Bearer sessions."""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import base64
|
||||
import binascii
|
||||
import hashlib
|
||||
import secrets
|
||||
from datetime import datetime, timezone
|
||||
from typing import Optional, Tuple
|
||||
|
||||
from argon2 import PasswordHasher
|
||||
from argon2.exceptions import InvalidHashError, VerificationError
|
||||
from starlette.concurrency import run_in_threadpool
|
||||
|
||||
from app import config
|
||||
from app.cache import REVOKED, SessionCache
|
||||
from app.errors import BASIC_CHALLENGE, unauthorized
|
||||
from app.storage import Store, fmt_ts, user_row
|
||||
|
||||
# argon2-cffi defaults = RFC 9106 low-memory profile (Argon2id, t=3, m=64 MiB, p=4).
|
||||
_hasher = PasswordHasher()
|
||||
_hash_slots = asyncio.Semaphore(config.PASSWORD_HASH_CONCURRENCY)
|
||||
# Verified when the username does not exist, so both failures cost the same time.
|
||||
_DUMMY_HASH = _hasher.hash("dummy-password-for-timing")
|
||||
|
||||
|
||||
async def hash_password(password: str) -> str:
|
||||
async with _hash_slots:
|
||||
return await run_in_threadpool(_hasher.hash, password)
|
||||
|
||||
|
||||
async def verify_password(password_hash: str, password: str) -> bool:
|
||||
def verify() -> bool:
|
||||
try:
|
||||
return _hasher.verify(password_hash, password)
|
||||
except (VerificationError, InvalidHashError):
|
||||
return False
|
||||
|
||||
async with _hash_slots:
|
||||
return await run_in_threadpool(verify)
|
||||
|
||||
|
||||
def token_digest(token: str) -> bytes:
|
||||
# A fast hash is enough here: the token already carries 256 random bits,
|
||||
# unlike a human password that needs a slow KDF against guessing.
|
||||
return hashlib.sha256(token.encode("ascii")).digest()
|
||||
|
||||
|
||||
def parse_basic(header: Optional[str]) -> Tuple[str, str]:
|
||||
bad = unauthorized("invalid credentials", BASIC_CHALLENGE)
|
||||
if not header:
|
||||
raise bad
|
||||
scheme, _, value = header.partition(" ")
|
||||
if scheme.lower() != "basic" or not value.strip():
|
||||
raise bad
|
||||
try:
|
||||
decoded = base64.b64decode(value.strip(), validate=True).decode("utf-8")
|
||||
except (binascii.Error, UnicodeDecodeError):
|
||||
raise bad from None
|
||||
username, sep, password = decoded.partition(":") # the password may contain ':'
|
||||
if not sep or not username.isascii() or not 12 <= len(password) <= 128:
|
||||
raise bad
|
||||
return username, password
|
||||
|
||||
|
||||
def parse_bearer(header: Optional[str]) -> str:
|
||||
if not header:
|
||||
raise unauthorized("bearer token required")
|
||||
scheme, _, token = header.partition(" ")
|
||||
token = token.strip()
|
||||
if scheme.lower() != "bearer" or not token or not token.isascii() or len(token) > 256:
|
||||
raise unauthorized("bearer token required")
|
||||
return token
|
||||
|
||||
|
||||
async def login(store: Store, header: Optional[str]) -> dict:
|
||||
username, password = parse_basic(header)
|
||||
row = await store.get_credentials(username)
|
||||
if row is None:
|
||||
await verify_password(_DUMMY_HASH, password)
|
||||
raise unauthorized("invalid credentials", BASIC_CHALLENGE)
|
||||
if not await verify_password(row["password_hash"], password):
|
||||
raise unauthorized("invalid credentials", BASIC_CHALLENGE)
|
||||
token = secrets.token_urlsafe(32) # 256 bits of entropy
|
||||
expires_at = await store.create_session(token_digest(token), row["id"], config.SESSION_TTL_SECONDS)
|
||||
return {"token": token, "token_type": "Bearer", "expires_at": fmt_ts(expires_at), "user": user_row(row)}
|
||||
|
||||
|
||||
async def authenticate(store: Store, cache: SessionCache, header: Optional[str]) -> dict:
|
||||
"""Return the session's user or raise 401. Cache first, PostgreSQL on a miss.
|
||||
|
||||
If PostgreSQL is down and the cache has no answer the error propagates as 503:
|
||||
an unverifiable token is never treated as authenticated."""
|
||||
digest = token_digest(parse_bearer(header))
|
||||
cached = await cache.get(digest)
|
||||
if cached == REVOKED:
|
||||
raise unauthorized("session expired or revoked")
|
||||
if isinstance(cached, dict):
|
||||
expires_at = datetime.fromisoformat(cached["expires_at"].replace("Z", "+00:00"))
|
||||
if expires_at > datetime.now(timezone.utc):
|
||||
return cached["user"]
|
||||
raise unauthorized("session expired or revoked")
|
||||
|
||||
row = await store.find_session(digest)
|
||||
if row is None:
|
||||
raise unauthorized("session expired or revoked")
|
||||
user = user_row(row)
|
||||
remaining = int((row["expires_at"] - datetime.now(timezone.utc)).total_seconds())
|
||||
await cache.put(digest, {"user": user, "expires_at": fmt_ts(row["expires_at"])}, remaining)
|
||||
return user
|
||||
|
||||
|
||||
async def logout(store: Store, cache: SessionCache, header: Optional[str]) -> None:
|
||||
# Authenticate first so an already revoked token gets 401, as the contract requires.
|
||||
await authenticate(store, cache, header)
|
||||
digest = token_digest(parse_bearer(header))
|
||||
expires_at = await store.revoke_session(digest) # PostgreSQL first: it is the truth
|
||||
if expires_at is None:
|
||||
raise unauthorized("session expired or revoked")
|
||||
remaining = int((expires_at - datetime.now(timezone.utc)).total_seconds()) + 1
|
||||
await cache.revoke(digest, remaining)
|
||||
+102
@@ -0,0 +1,102 @@
|
||||
"""Valkey session cache. Never the source of truth: every failure degrades to a miss.
|
||||
|
||||
Keys: session:<sha256(token) hex>
|
||||
Value: JSON {"user": {...}, "expires_at": "..."} - a valid session
|
||||
"revoked" - tombstone written by logout
|
||||
Entry TTL = min(time left in the session, SESSION_CACHE_MAX_SECONDS).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import time
|
||||
from typing import Optional, Union
|
||||
|
||||
import redis.asyncio as redis
|
||||
from redis.exceptions import RedisError
|
||||
|
||||
from app import config
|
||||
|
||||
logger = logging.getLogger("messenger.cache")
|
||||
|
||||
REVOKED = "revoked"
|
||||
|
||||
|
||||
class SessionCache:
|
||||
def __init__(self, url: Optional[str]):
|
||||
self._client: Optional[redis.Redis] = None
|
||||
self._down_until = 0.0
|
||||
if url:
|
||||
pool = redis.BlockingConnectionPool.from_url(
|
||||
url,
|
||||
max_connections=config.CACHE_POOL_MAX,
|
||||
timeout=config.CACHE_TIMEOUT,
|
||||
socket_timeout=config.CACHE_TIMEOUT,
|
||||
socket_connect_timeout=config.CACHE_TIMEOUT,
|
||||
decode_responses=True,
|
||||
)
|
||||
self._client = redis.Redis(connection_pool=pool)
|
||||
|
||||
@property
|
||||
def enabled(self) -> bool:
|
||||
return self._client is not None
|
||||
|
||||
def _usable(self) -> bool:
|
||||
return self._client is not None and time.monotonic() >= self._down_until
|
||||
|
||||
def _failed(self, op: str, exc: Exception) -> None:
|
||||
# Circuit breaker: one slow failure, then skip the cache for a few seconds.
|
||||
self._down_until = time.monotonic() + config.CACHE_RETRY_AFTER
|
||||
logger.warning("cache %s failed (%s); bypassing cache for %.0fs", op, type(exc).__name__, config.CACHE_RETRY_AFTER)
|
||||
|
||||
@staticmethod
|
||||
def _key(token_hash: bytes) -> str:
|
||||
return "session:" + token_hash.hex()
|
||||
|
||||
async def get(self, token_hash: bytes) -> Union[dict, str, None]:
|
||||
if not self._usable():
|
||||
return None
|
||||
try:
|
||||
raw = await self._client.get(self._key(token_hash))
|
||||
except (RedisError, OSError) as exc:
|
||||
self._failed("get", exc)
|
||||
return None
|
||||
if raw is None or raw == REVOKED:
|
||||
return raw
|
||||
try:
|
||||
return json.loads(raw)
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
async def put(self, token_hash: bytes, entry: dict, ttl_seconds: int) -> None:
|
||||
"""Fill after a PostgreSQL read. NX: never overwrite a logout tombstone."""
|
||||
ttl = min(ttl_seconds, config.SESSION_CACHE_MAX_SECONDS)
|
||||
if ttl <= 0 or not self._usable():
|
||||
return
|
||||
try:
|
||||
await self._client.set(self._key(token_hash), json.dumps(entry), ex=ttl, nx=True)
|
||||
except (RedisError, OSError) as exc:
|
||||
self._failed("set", exc)
|
||||
|
||||
async def revoke(self, token_hash: bytes, ttl_seconds: int) -> None:
|
||||
"""Tombstone instead of DEL: a concurrent fill that read PostgreSQL before the
|
||||
revoke cannot re-insert the session afterwards (its SET NX loses).
|
||||
Tried even while the breaker is open: logout is rare and correctness matters more."""
|
||||
if self._client is None:
|
||||
return
|
||||
try:
|
||||
await self._client.set(self._key(token_hash), REVOKED, ex=max(1, min(ttl_seconds, config.SESSION_CACHE_MAX_SECONDS)))
|
||||
except (RedisError, OSError) as exc:
|
||||
self._failed("revoke", exc)
|
||||
|
||||
async def ping(self) -> bool:
|
||||
if self._client is None:
|
||||
return False
|
||||
try:
|
||||
return bool(await self._client.ping())
|
||||
except (RedisError, OSError):
|
||||
return False
|
||||
|
||||
async def close(self) -> None:
|
||||
if self._client is not None:
|
||||
await self._client.aclose()
|
||||
@@ -0,0 +1,46 @@
|
||||
"""Runtime configuration from environment; secrets are read from *_FILE paths."""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from typing import Optional
|
||||
|
||||
|
||||
def _secret(name: str) -> Optional[str]:
|
||||
"""NAME wins over NAME_FILE; NAME_FILE keeps the value out of `docker inspect`."""
|
||||
value = os.environ.get(name)
|
||||
if value:
|
||||
return value.strip()
|
||||
path = os.environ.get(name + "_FILE")
|
||||
if path and os.path.exists(path):
|
||||
with open(path, encoding="utf-8") as fh:
|
||||
return fh.read().strip() or None
|
||||
return None
|
||||
|
||||
|
||||
def _int(name: str, default: int) -> int:
|
||||
return int(os.environ.get(name, default))
|
||||
|
||||
|
||||
def _float(name: str, default: float) -> float:
|
||||
return float(os.environ.get(name, default))
|
||||
|
||||
|
||||
DATABASE_URL = _secret("DATABASE_URL")
|
||||
CACHE_URL = _secret("CACHE_URL")
|
||||
|
||||
SESSION_TTL_SECONDS = _int("SESSION_TTL_SECONDS", 3600)
|
||||
# Upper bound for one cached session entry; PostgreSQL is re-checked after it.
|
||||
SESSION_CACHE_MAX_SECONDS = _int("SESSION_CACHE_MAX_SECONDS", 300)
|
||||
|
||||
DB_POOL_MAX = _int("DB_POOL_MAX", 10)
|
||||
DB_CONNECT_TIMEOUT = _float("DB_CONNECT_TIMEOUT", 3)
|
||||
DB_ACQUIRE_TIMEOUT = _float("DB_ACQUIRE_TIMEOUT", 3)
|
||||
DB_COMMAND_TIMEOUT = _float("DB_COMMAND_TIMEOUT", 5)
|
||||
|
||||
CACHE_TIMEOUT = _float("CACHE_TIMEOUT", 0.3)
|
||||
CACHE_POOL_MAX = _int("CACHE_POOL_MAX", 20)
|
||||
# After a cache error, skip the cache for this long instead of paying the timeout per request.
|
||||
CACHE_RETRY_AFTER = _float("CACHE_RETRY_AFTER", 5)
|
||||
|
||||
# Argon2id hashing is CPU/RAM heavy: bound how many run at once.
|
||||
PASSWORD_HASH_CONCURRENCY = _int("PASSWORD_HASH_CONCURRENCY", 4)
|
||||
@@ -0,0 +1,24 @@
|
||||
"""API error type shared by all layers."""
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Dict, Optional
|
||||
|
||||
BEARER_CHALLENGE = 'Bearer realm="messenger"'
|
||||
BASIC_CHALLENGE = 'Basic realm="messenger", charset="UTF-8"'
|
||||
|
||||
|
||||
class ApiError(Exception):
|
||||
def __init__(self, status_code: int, code: str, message: str, headers: Optional[Dict[str, str]] = None):
|
||||
super().__init__(message)
|
||||
self.status_code = status_code
|
||||
self.code = code
|
||||
self.message = message
|
||||
self.headers = headers or {}
|
||||
|
||||
|
||||
def unauthorized(message: str, challenge: str = BEARER_CHALLENGE) -> ApiError:
|
||||
return ApiError(401, "unauthorized", message, {"WWW-Authenticate": challenge})
|
||||
|
||||
|
||||
def unavailable() -> ApiError:
|
||||
return ApiError(503, "unavailable", "service temporarily unavailable")
|
||||
+234
@@ -0,0 +1,234 @@
|
||||
"""Lab 2: messenger API on PostgreSQL with Bearer sessions and a Valkey session cache."""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import uuid
|
||||
from contextlib import asynccontextmanager
|
||||
from typing import Optional
|
||||
|
||||
import asyncpg
|
||||
from fastapi import Depends, FastAPI, Query, Request, Response
|
||||
from fastapi.exceptions import RequestValidationError
|
||||
from fastapi.responses import JSONResponse
|
||||
from starlette.exceptions import HTTPException as StarletteHTTPException
|
||||
|
||||
from app import auth, config
|
||||
from app.cache import SessionCache
|
||||
from app.errors import ApiError
|
||||
from app.models import Chat, ChatPage, CreateChat, CreateMessage, CreateUser, Health, Message, MessagePage, Session, User
|
||||
from app.storage import Store
|
||||
|
||||
logging.basicConfig(level=logging.INFO)
|
||||
logger = logging.getLogger("messenger")
|
||||
|
||||
store = Store()
|
||||
cache = SessionCache(config.CACHE_URL)
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(_app: FastAPI):
|
||||
if not config.DATABASE_URL:
|
||||
raise RuntimeError("DATABASE_URL or DATABASE_URL_FILE is required")
|
||||
await store.open()
|
||||
logger.info("session cache: %s", "valkey" if cache.enabled else "disabled")
|
||||
yield
|
||||
await cache.close()
|
||||
await store.close()
|
||||
|
||||
|
||||
app = FastAPI(title="MTUSI Messenger — lab2", lifespan=lifespan)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Cross-cutting: request id + uniform error envelope
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# PostgreSQL unreachable, restarting or pool exhausted -> bounded 503, never a hang.
|
||||
DB_UNAVAILABLE = (
|
||||
OSError, # includes ConnectionRefusedError and TimeoutError (pool acquire / connect)
|
||||
asyncpg.PostgresConnectionError,
|
||||
asyncpg.InterfaceError,
|
||||
asyncpg.TooManyConnectionsError,
|
||||
asyncpg.exceptions.OperatorInterventionError, # admin shutdown, cannot connect now
|
||||
asyncpg.exceptions.QueryCanceledError, # statement timeout
|
||||
)
|
||||
|
||||
|
||||
def error_body(code: str, message: str, request_id: str) -> dict:
|
||||
return {"error": {"code": code, "message": message, "request_id": request_id}}
|
||||
|
||||
|
||||
def error_response(request: Request, status: int, code: str, message: str, headers: Optional[dict] = None) -> JSONResponse:
|
||||
request_id = getattr(request.state, "request_id", "")
|
||||
return JSONResponse(status_code=status, content=error_body(code, message, request_id), headers=headers)
|
||||
|
||||
|
||||
@app.middleware("http")
|
||||
async def request_id_middleware(request: Request, call_next):
|
||||
request_id = uuid.uuid4().hex
|
||||
request.state.request_id = request_id
|
||||
try:
|
||||
response = await call_next(request)
|
||||
except Exception:
|
||||
# Unhandled errors must be answered here: Starlette's own 500 handler runs
|
||||
# outside this middleware and would drop the X-Request-Id header.
|
||||
logger.exception("unhandled error for request_id=%s", request_id)
|
||||
response = JSONResponse(status_code=500, content=error_body("internal_error", "internal error", request_id))
|
||||
response.headers["X-Request-Id"] = request_id
|
||||
return response
|
||||
|
||||
|
||||
@app.exception_handler(ApiError)
|
||||
async def api_error_handler(request: Request, exc: ApiError):
|
||||
return error_response(request, exc.status_code, exc.code, exc.message, exc.headers)
|
||||
|
||||
|
||||
@app.exception_handler(RequestValidationError)
|
||||
async def validation_error_handler(request: Request, exc: RequestValidationError):
|
||||
return error_response(request, 400, "invalid_request", "request failed validation")
|
||||
|
||||
|
||||
@app.exception_handler(StarletteHTTPException)
|
||||
async def http_exception_handler(request: Request, exc: StarletteHTTPException):
|
||||
code = {404: "not_found", 405: "invalid_request", 415: "unsupported_media_type"}.get(exc.status_code, "invalid_request")
|
||||
message = exc.detail if isinstance(exc.detail, str) else "request failed"
|
||||
return error_response(request, exc.status_code, code, message)
|
||||
|
||||
|
||||
async def db_unavailable_handler(request: Request, exc: Exception):
|
||||
logger.warning("database unavailable for request_id=%s: %s", request.state.request_id, type(exc).__name__)
|
||||
return error_response(request, 503, "unavailable", "service temporarily unavailable")
|
||||
|
||||
|
||||
for _exc in DB_UNAVAILABLE:
|
||||
app.add_exception_handler(_exc, db_unavailable_handler)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Auth: Basic only on POST /auth/sessions, Bearer everywhere else.
|
||||
# X-User-Id is ignored since lab2.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
async def current_user(request: Request) -> dict:
|
||||
return await auth.authenticate(store, cache, request.headers.get("Authorization"))
|
||||
|
||||
|
||||
def parse_uuid(value: str, field: str) -> uuid.UUID:
|
||||
try:
|
||||
return uuid.UUID(value)
|
||||
except ValueError:
|
||||
raise ApiError(400, "invalid_request", f"{field} is not a valid UUID") from None
|
||||
|
||||
|
||||
def not_found() -> ApiError:
|
||||
return ApiError(404, "not_found", "chat not found")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Health
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@app.get("/health/live", response_model=Health)
|
||||
async def liveness():
|
||||
return Health()
|
||||
|
||||
|
||||
@app.get("/health/ready", response_model=Health)
|
||||
async def readiness():
|
||||
# PostgreSQL is critical; Valkey is not (the API bypasses the cache without it).
|
||||
try:
|
||||
await store.ping()
|
||||
except DB_UNAVAILABLE:
|
||||
raise ApiError(503, "unavailable", "database is not available") from None
|
||||
return Health()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Users and sessions
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@app.post("/api/v1/users", response_model=User, status_code=201)
|
||||
async def register_user(body: CreateUser):
|
||||
if not body.display_name.strip() or not body.password.strip():
|
||||
raise ApiError(400, "invalid_request", "display_name and password must not be blank")
|
||||
password_hash = await auth.hash_password(body.password)
|
||||
return await store.create_user(body.username, body.display_name, password_hash)
|
||||
|
||||
|
||||
@app.get("/api/v1/users/me", response_model=User)
|
||||
async def current_user_endpoint(user: dict = Depends(current_user)):
|
||||
return user
|
||||
|
||||
|
||||
@app.post("/api/v1/auth/sessions", response_model=Session, status_code=201)
|
||||
async def create_session(request: Request):
|
||||
return await auth.login(store, request.headers.get("Authorization"))
|
||||
|
||||
|
||||
@app.delete("/api/v1/auth/sessions/current", status_code=204)
|
||||
async def delete_session(request: Request):
|
||||
await auth.logout(store, cache, request.headers.get("Authorization"))
|
||||
return Response(status_code=204)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Chats
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@app.post("/api/v1/chats", response_model=Chat, status_code=201)
|
||||
async def create_chat(body: CreateChat, user: dict = Depends(current_user)):
|
||||
if not body.title.strip():
|
||||
raise ApiError(400, "invalid_request", "title must not be blank")
|
||||
if len(set(body.member_ids)) != len(body.member_ids):
|
||||
raise ApiError(400, "invalid_request", "member_ids must not contain duplicates")
|
||||
if uuid.UUID(user["id"]) not in body.member_ids:
|
||||
raise ApiError(400, "invalid_request", "caller must be a member of the chat")
|
||||
return await store.create_chat(body.title, list(body.member_ids))
|
||||
|
||||
|
||||
@app.get("/api/v1/chats", response_model=ChatPage)
|
||||
async def list_chats(
|
||||
user: dict = Depends(current_user),
|
||||
limit: int = Query(default=50, ge=1, le=100),
|
||||
cursor: Optional[str] = Query(default=None, min_length=1),
|
||||
):
|
||||
return await store.list_chats(uuid.UUID(user["id"]), limit, cursor)
|
||||
|
||||
|
||||
@app.get("/api/v1/chats/{chat_id}", response_model=Chat)
|
||||
async def get_chat(chat_id: str, user: dict = Depends(current_user)):
|
||||
chat = await store.get_chat_for_member(parse_uuid(chat_id, "chat_id"), uuid.UUID(user["id"]))
|
||||
if chat is None:
|
||||
raise not_found()
|
||||
return chat
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Messages
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@app.post("/api/v1/chats/{chat_id}/messages", response_model=Message, status_code=201)
|
||||
async def create_message(chat_id: str, body: CreateMessage, user: dict = Depends(current_user)):
|
||||
chat_uuid = parse_uuid(chat_id, "chat_id")
|
||||
if not body.text.strip():
|
||||
# Access is checked before content: an outsider gets 404 even for a blank text.
|
||||
if not await store.is_member(chat_uuid, uuid.UUID(user["id"])):
|
||||
raise not_found()
|
||||
raise ApiError(400, "invalid_request", "text must not be blank")
|
||||
message = await store.create_message(chat_uuid, uuid.UUID(user["id"]), body.text)
|
||||
if message is None:
|
||||
raise not_found()
|
||||
return message
|
||||
|
||||
|
||||
@app.get("/api/v1/chats/{chat_id}/messages", response_model=MessagePage)
|
||||
async def list_messages(
|
||||
chat_id: str,
|
||||
user: dict = Depends(current_user),
|
||||
limit: int = Query(default=50, ge=1, le=100),
|
||||
cursor: Optional[str] = Query(default=None, min_length=1),
|
||||
):
|
||||
chat_uuid = parse_uuid(chat_id, "chat_id")
|
||||
if not await store.is_member(chat_uuid, uuid.UUID(user["id"])):
|
||||
raise not_found()
|
||||
return await store.list_messages(chat_uuid, limit, cursor)
|
||||
@@ -0,0 +1,68 @@
|
||||
"""Apply versioned SQL migrations from migrations/; safe to run repeatedly and concurrently.
|
||||
|
||||
Usage: python -m app.migrate
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import asyncpg
|
||||
|
||||
from app import config
|
||||
|
||||
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(name)s: %(message)s")
|
||||
logger = logging.getLogger("migrate")
|
||||
|
||||
MIGRATIONS_DIR = Path(__file__).resolve().parents[1] / "migrations"
|
||||
# Arbitrary constant: serialises concurrent runners (e.g. several API replicas in lab3).
|
||||
LOCK_ID = 2026_0002
|
||||
|
||||
|
||||
async def connect_with_retry(attempts: int = 30) -> asyncpg.Connection:
|
||||
for attempt in range(1, attempts + 1):
|
||||
try:
|
||||
return await asyncpg.connect(config.DATABASE_URL, timeout=config.DB_CONNECT_TIMEOUT)
|
||||
except (OSError, asyncpg.PostgresError, asyncpg.InterfaceError) as exc:
|
||||
logger.warning("database not reachable (attempt %d/%d): %s", attempt, attempts, type(exc).__name__)
|
||||
await asyncio.sleep(1)
|
||||
raise SystemExit("database is not reachable")
|
||||
|
||||
|
||||
async def migrate() -> int:
|
||||
if not config.DATABASE_URL:
|
||||
raise SystemExit("DATABASE_URL or DATABASE_URL_FILE is required")
|
||||
conn = await connect_with_retry()
|
||||
applied_now = 0
|
||||
try:
|
||||
await conn.execute("SELECT pg_advisory_lock($1)", LOCK_ID)
|
||||
await conn.execute(
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS schema_migrations (
|
||||
version text PRIMARY KEY,
|
||||
applied_at timestamptz NOT NULL DEFAULT now()
|
||||
)
|
||||
"""
|
||||
)
|
||||
done = {r["version"] for r in await conn.fetch("SELECT version FROM schema_migrations")}
|
||||
for path in sorted(MIGRATIONS_DIR.glob("*.sql")):
|
||||
version = path.stem
|
||||
if version in done:
|
||||
continue
|
||||
logger.info("applying %s", version)
|
||||
async with conn.transaction():
|
||||
await conn.execute(path.read_text(encoding="utf-8"))
|
||||
await conn.execute("INSERT INTO schema_migrations (version) VALUES ($1)", version)
|
||||
applied_now += 1
|
||||
logger.info("migrations up to date: %d applied now, %d before", applied_now, len(done))
|
||||
finally:
|
||||
await conn.execute("SELECT pg_advisory_unlock($1)", LOCK_ID)
|
||||
await conn.close()
|
||||
return applied_now
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(migrate())
|
||||
sys.exit(0)
|
||||
@@ -0,0 +1,73 @@
|
||||
"""Pydantic request/response schemas mirroring contracts/openapi.json."""
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
from typing import List, Optional
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
USERNAME_PATTERN = r"^[a-z0-9_]{3,32}$"
|
||||
|
||||
|
||||
class CreateUser(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
username: str = Field(pattern=USERNAME_PATTERN)
|
||||
display_name: str = Field(min_length=1, max_length=100)
|
||||
password: str = Field(min_length=12, max_length=128)
|
||||
|
||||
|
||||
class User(BaseModel):
|
||||
id: uuid.UUID
|
||||
username: str
|
||||
display_name: str
|
||||
created_at: str
|
||||
|
||||
|
||||
class CreateChat(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
title: str = Field(min_length=1, max_length=100)
|
||||
member_ids: List[uuid.UUID] = Field(min_length=1, max_length=100)
|
||||
|
||||
|
||||
class Chat(BaseModel):
|
||||
id: uuid.UUID
|
||||
title: str
|
||||
member_ids: List[uuid.UUID]
|
||||
created_at: str
|
||||
|
||||
|
||||
class ChatPage(BaseModel):
|
||||
items: List[Chat]
|
||||
next_cursor: Optional[str] = None
|
||||
|
||||
|
||||
class CreateMessage(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
text: str = Field(max_length=4000)
|
||||
|
||||
|
||||
class Message(BaseModel):
|
||||
id: uuid.UUID
|
||||
chat_id: uuid.UUID
|
||||
sender_id: uuid.UUID
|
||||
text: str
|
||||
created_at: str
|
||||
|
||||
|
||||
class MessagePage(BaseModel):
|
||||
items: List[Message]
|
||||
next_cursor: Optional[str] = None
|
||||
|
||||
|
||||
class Session(BaseModel):
|
||||
token: str
|
||||
token_type: str = "Bearer"
|
||||
expires_at: str
|
||||
user: User
|
||||
|
||||
|
||||
class Health(BaseModel):
|
||||
status: str = "ok"
|
||||
+288
@@ -0,0 +1,288 @@
|
||||
"""PostgreSQL repository. Every query is parameterised; PostgreSQL is the source of truth."""
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import json
|
||||
import uuid
|
||||
from datetime import datetime, timezone
|
||||
from typing import List, Optional, Tuple
|
||||
|
||||
import asyncpg
|
||||
|
||||
from app import config
|
||||
from app.errors import ApiError
|
||||
|
||||
|
||||
def fmt_ts(value: datetime) -> str:
|
||||
# Always 6 fractional digits: exact round trip with PostgreSQL microseconds.
|
||||
return value.astimezone(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S.%fZ")
|
||||
|
||||
|
||||
def user_row(row: asyncpg.Record) -> dict:
|
||||
return {
|
||||
"id": str(row["id"]),
|
||||
"username": row["username"],
|
||||
"display_name": row["display_name"],
|
||||
"created_at": fmt_ts(row["created_at"]),
|
||||
}
|
||||
|
||||
|
||||
def chat_row(row: asyncpg.Record) -> dict:
|
||||
return {
|
||||
"id": str(row["id"]),
|
||||
"title": row["title"],
|
||||
"member_ids": [str(m) for m in row["member_ids"]],
|
||||
"created_at": fmt_ts(row["created_at"]),
|
||||
}
|
||||
|
||||
|
||||
def message_row(row: asyncpg.Record) -> dict:
|
||||
return {
|
||||
"id": str(row["id"]),
|
||||
"chat_id": str(row["chat_id"]),
|
||||
"sender_id": str(row["sender_id"]),
|
||||
"text": row["text"],
|
||||
"created_at": fmt_ts(row["created_at"]),
|
||||
}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Opaque keyset cursor bound to one resource
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def encode_cursor(resource: str, created_at: str, item_id: str) -> str:
|
||||
payload = json.dumps({"r": resource, "t": created_at, "i": item_id}, separators=(",", ":"))
|
||||
return base64.urlsafe_b64encode(payload.encode("utf-8")).decode("ascii").rstrip("=")
|
||||
|
||||
|
||||
def decode_cursor(resource: str, cursor: str) -> Tuple[datetime, uuid.UUID]:
|
||||
try:
|
||||
padded = cursor + "=" * (-len(cursor) % 4)
|
||||
payload = json.loads(base64.urlsafe_b64decode(padded.encode("ascii")).decode("utf-8"))
|
||||
if payload.get("r") != resource:
|
||||
raise ValueError("cursor belongs to a different resource")
|
||||
created_at = datetime.fromisoformat(payload["t"].replace("Z", "+00:00"))
|
||||
if created_at.tzinfo is None:
|
||||
raise ValueError("cursor time has no timezone")
|
||||
return created_at, uuid.UUID(payload["i"])
|
||||
except Exception as exc: # noqa: BLE001 - any malformed cursor is a 400
|
||||
raise ApiError(400, "invalid_request", "invalid cursor") from exc
|
||||
|
||||
|
||||
def page_of(items: List[dict], resource: str, limit: int) -> dict:
|
||||
"""Queries fetch limit+1 rows: the extra row only tells whether a next page exists."""
|
||||
page = items[:limit]
|
||||
next_cursor = None
|
||||
if len(items) > limit:
|
||||
last = page[-1]
|
||||
next_cursor = encode_cursor(resource, last["created_at"], last["id"])
|
||||
return {"items": page, "next_cursor": next_cursor}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Repository
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class Store:
|
||||
def __init__(self):
|
||||
self.pool: Optional[asyncpg.Pool] = None
|
||||
|
||||
async def open(self) -> None:
|
||||
# min_size=0: connections open on demand, so the API starts even while
|
||||
# PostgreSQL is down and readiness reports 503 until it is back.
|
||||
self.pool = await asyncpg.create_pool(
|
||||
config.DATABASE_URL,
|
||||
min_size=0,
|
||||
max_size=config.DB_POOL_MAX,
|
||||
timeout=config.DB_CONNECT_TIMEOUT,
|
||||
command_timeout=config.DB_COMMAND_TIMEOUT,
|
||||
max_inactive_connection_lifetime=60,
|
||||
)
|
||||
|
||||
async def close(self) -> None:
|
||||
if self.pool is not None:
|
||||
await self.pool.close()
|
||||
|
||||
def acquire(self):
|
||||
return self.pool.acquire(timeout=config.DB_ACQUIRE_TIMEOUT)
|
||||
|
||||
async def ping(self) -> bool:
|
||||
async with self.acquire() as conn:
|
||||
return await conn.fetchval("SELECT 1") == 1
|
||||
|
||||
# -- users --------------------------------------------------------------
|
||||
|
||||
async def create_user(self, username: str, display_name: str, password_hash: str) -> dict:
|
||||
async with self.acquire() as conn:
|
||||
try:
|
||||
row = await conn.fetchrow(
|
||||
"""
|
||||
INSERT INTO users (id, username, display_name, password_hash)
|
||||
VALUES ($1, $2, $3, $4)
|
||||
RETURNING id, username, display_name, created_at
|
||||
""",
|
||||
uuid.uuid4(), username, display_name, password_hash,
|
||||
)
|
||||
except asyncpg.UniqueViolationError:
|
||||
# The UNIQUE constraint, not a prior SELECT, decides concurrent registrations.
|
||||
raise ApiError(409, "conflict", "username already taken") from None
|
||||
return user_row(row)
|
||||
|
||||
async def get_credentials(self, username: str) -> Optional[asyncpg.Record]:
|
||||
async with self.acquire() as conn:
|
||||
return await conn.fetchrow(
|
||||
"SELECT id, username, display_name, created_at, password_hash FROM users WHERE username = $1",
|
||||
username,
|
||||
)
|
||||
|
||||
# -- sessions -----------------------------------------------------------
|
||||
|
||||
async def create_session(self, token_hash: bytes, user_id: uuid.UUID, ttl_seconds: int) -> datetime:
|
||||
async with self.acquire() as conn:
|
||||
return await conn.fetchval(
|
||||
"""
|
||||
INSERT INTO sessions (token_hash, user_id, expires_at)
|
||||
VALUES ($1, $2, clock_timestamp() + make_interval(secs => $3))
|
||||
RETURNING expires_at
|
||||
""",
|
||||
token_hash, user_id, float(ttl_seconds),
|
||||
)
|
||||
|
||||
async def find_session(self, token_hash: bytes) -> Optional[asyncpg.Record]:
|
||||
"""Only a live session: not revoked and not expired by the database clock."""
|
||||
async with self.acquire() as conn:
|
||||
return await conn.fetchrow(
|
||||
"""
|
||||
SELECT s.expires_at, u.id, u.username, u.display_name, u.created_at
|
||||
FROM sessions s JOIN users u ON u.id = s.user_id
|
||||
WHERE s.token_hash = $1 AND s.revoked_at IS NULL AND s.expires_at > clock_timestamp()
|
||||
""",
|
||||
token_hash,
|
||||
)
|
||||
|
||||
async def revoke_session(self, token_hash: bytes) -> Optional[datetime]:
|
||||
async with self.acquire() as conn:
|
||||
return await conn.fetchval(
|
||||
"""
|
||||
UPDATE sessions SET revoked_at = clock_timestamp()
|
||||
WHERE token_hash = $1 AND revoked_at IS NULL AND expires_at > clock_timestamp()
|
||||
RETURNING expires_at
|
||||
""",
|
||||
token_hash,
|
||||
)
|
||||
|
||||
# -- chats --------------------------------------------------------------
|
||||
|
||||
async def create_chat(self, title: str, member_ids: List[uuid.UUID]) -> dict:
|
||||
"""Chat and all memberships commit together or not at all.
|
||||
|
||||
Unknown members are detected by the foreign key inside the transaction, so a
|
||||
failure after the chat row was inserted really exercises the rollback."""
|
||||
chat_id = uuid.uuid4()
|
||||
async with self.acquire() as conn:
|
||||
try:
|
||||
async with conn.transaction():
|
||||
row = await conn.fetchrow(
|
||||
"INSERT INTO chats (id, title) VALUES ($1, $2) RETURNING id, title, created_at",
|
||||
chat_id, title,
|
||||
)
|
||||
await conn.execute(
|
||||
"""
|
||||
INSERT INTO chat_members (chat_id, user_id, position)
|
||||
SELECT $1, m.user_id, m.ord - 1
|
||||
FROM unnest($2::uuid[]) WITH ORDINALITY AS m(user_id, ord)
|
||||
""",
|
||||
chat_id, member_ids,
|
||||
)
|
||||
except asyncpg.ForeignKeyViolationError:
|
||||
raise ApiError(400, "invalid_request", "unknown member in member_ids") from None
|
||||
return {
|
||||
"id": str(row["id"]),
|
||||
"title": row["title"],
|
||||
"member_ids": [str(m) for m in member_ids],
|
||||
"created_at": fmt_ts(row["created_at"]),
|
||||
}
|
||||
|
||||
async def get_chat_for_member(self, chat_id: uuid.UUID, user_id: uuid.UUID) -> Optional[dict]:
|
||||
async with self.acquire() as conn:
|
||||
row = await conn.fetchrow(
|
||||
"""
|
||||
SELECT c.id, c.title, c.created_at,
|
||||
array_agg(m.user_id ORDER BY m.position) AS member_ids
|
||||
FROM chats c JOIN chat_members m ON m.chat_id = c.id
|
||||
WHERE c.id = $1
|
||||
AND EXISTS (SELECT 1 FROM chat_members me WHERE me.chat_id = c.id AND me.user_id = $2)
|
||||
GROUP BY c.id
|
||||
""",
|
||||
chat_id, user_id,
|
||||
)
|
||||
return chat_row(row) if row else None
|
||||
|
||||
async def list_chats(self, user_id: uuid.UUID, limit: int, cursor: Optional[str]) -> dict:
|
||||
after_t, after_i = decode_cursor("chats", cursor) if cursor else (None, None)
|
||||
async with self.acquire() as conn:
|
||||
rows = await conn.fetch(
|
||||
"""
|
||||
SELECT c.id, c.title, c.created_at,
|
||||
array_agg(m.user_id ORDER BY m.position) AS member_ids
|
||||
FROM chat_members me
|
||||
JOIN chats c ON c.id = me.chat_id
|
||||
JOIN chat_members m ON m.chat_id = c.id
|
||||
WHERE me.user_id = $1
|
||||
AND ($2::timestamptz IS NULL OR (c.created_at, c.id) > ($2, $3::uuid))
|
||||
GROUP BY c.id
|
||||
ORDER BY c.created_at, c.id
|
||||
LIMIT $4
|
||||
""",
|
||||
user_id, after_t, after_i, limit + 1,
|
||||
)
|
||||
return page_of([chat_row(r) for r in rows], "chats", limit)
|
||||
|
||||
async def is_member(self, chat_id: uuid.UUID, user_id: uuid.UUID) -> bool:
|
||||
async with self.acquire() as conn:
|
||||
return bool(await conn.fetchval(
|
||||
"SELECT EXISTS (SELECT 1 FROM chat_members WHERE chat_id = $1 AND user_id = $2)",
|
||||
chat_id, user_id,
|
||||
))
|
||||
|
||||
# -- messages -----------------------------------------------------------
|
||||
|
||||
async def create_message(self, chat_id: uuid.UUID, sender_id: uuid.UUID, text: str) -> Optional[dict]:
|
||||
"""Membership check and insert in one statement; None means "no such chat for you"."""
|
||||
async with self.acquire() as conn:
|
||||
row = await conn.fetchrow(
|
||||
"""
|
||||
INSERT INTO messages (id, chat_id, sender_id, text)
|
||||
SELECT $1, $2, $3, $4
|
||||
WHERE EXISTS (SELECT 1 FROM chat_members WHERE chat_id = $2 AND user_id = $3)
|
||||
RETURNING id, chat_id, sender_id, text, created_at
|
||||
""",
|
||||
uuid.uuid4(), chat_id, sender_id, text,
|
||||
)
|
||||
return message_row(row) if row else None
|
||||
|
||||
async def list_messages(self, chat_id: uuid.UUID, limit: int, cursor: Optional[str]) -> dict:
|
||||
resource = str(chat_id)
|
||||
async with self.acquire() as conn:
|
||||
if cursor:
|
||||
after_t, after_i = decode_cursor(resource, cursor)
|
||||
rows = await conn.fetch(
|
||||
"""
|
||||
SELECT id, chat_id, sender_id, text, created_at FROM messages
|
||||
WHERE chat_id = $1 AND (created_at, id) > ($2, $3)
|
||||
ORDER BY created_at, id
|
||||
LIMIT $4
|
||||
""",
|
||||
chat_id, after_t, after_i, limit + 1,
|
||||
)
|
||||
else:
|
||||
rows = await conn.fetch(
|
||||
"""
|
||||
SELECT id, chat_id, sender_id, text, created_at FROM messages
|
||||
WHERE chat_id = $1
|
||||
ORDER BY created_at, id
|
||||
LIMIT $2
|
||||
""",
|
||||
chat_id, limit + 1,
|
||||
)
|
||||
return page_of([message_row(r) for r in rows], resource, limit)
|
||||
+100
@@ -0,0 +1,100 @@
|
||||
# Lab 2 stack: api -> postgres (source of truth) + valkey (session cache).
|
||||
# Prepare secrets first: scripts/init-secrets.sh. Start: docker compose up -d --build --wait
|
||||
name: messenger-lab2
|
||||
|
||||
x-api: &api
|
||||
build: .
|
||||
image: mtusi-messenger:lab2
|
||||
environment: &api-env
|
||||
APP_HOST: 0.0.0.0
|
||||
APP_PORT: "8080"
|
||||
APP_ENV: ${APP_ENV:-lab}
|
||||
SESSION_TTL_SECONDS: ${SESSION_TTL_SECONDS:-3600}
|
||||
SESSION_STORE: postgres
|
||||
SESSION_CACHE_MAX_SECONDS: ${SESSION_CACHE_MAX_SECONDS:-300}
|
||||
DATABASE_URL_FILE: /run/secrets/database_url
|
||||
CACHE_URL_FILE: /run/secrets/cache_url
|
||||
DB_POOL_MAX: ${DB_POOL_MAX:-10}
|
||||
secrets: [database_url, cache_url]
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
valkey:
|
||||
# The cache is optional for correctness: only wait for the container to start.
|
||||
condition: service_started
|
||||
migrate:
|
||||
condition: service_completed_successfully
|
||||
stop_grace_period: 10s
|
||||
restart: unless-stopped
|
||||
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:17.6-alpine3.22
|
||||
environment:
|
||||
POSTGRES_DB: messenger
|
||||
POSTGRES_USER: messenger
|
||||
POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password
|
||||
secrets: [postgres_password]
|
||||
volumes:
|
||||
- pgdata:/var/lib/postgresql/data
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U messenger -d messenger"]
|
||||
interval: 2s
|
||||
timeout: 3s
|
||||
retries: 30
|
||||
stop_grace_period: 30s
|
||||
restart: unless-stopped
|
||||
|
||||
valkey:
|
||||
image: valkey/valkey:8.1.4-alpine3.22
|
||||
# requirepass and cache-only settings live in the secret config file.
|
||||
command: ["valkey-server", "/run/secrets/valkey_conf"]
|
||||
secrets: [valkey_conf]
|
||||
healthcheck:
|
||||
# NOAUTH also proves the server answers; the password stays out of the process list.
|
||||
test: ["CMD-SHELL", "valkey-cli ping 2>&1 | grep -qE 'PONG|NOAUTH'"]
|
||||
interval: 2s
|
||||
timeout: 3s
|
||||
retries: 30
|
||||
restart: unless-stopped
|
||||
|
||||
migrate:
|
||||
build: .
|
||||
image: mtusi-messenger:lab2
|
||||
command: ["python", "-m", "app.migrate"]
|
||||
environment:
|
||||
DATABASE_URL_FILE: /run/secrets/database_url
|
||||
secrets: [database_url]
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
restart: "no"
|
||||
|
||||
api:
|
||||
<<: *api
|
||||
ports:
|
||||
- "127.0.0.1:${APP_PORT:-8080}:8080"
|
||||
|
||||
# Same image with a 2-second TTL, only for the expiry test:
|
||||
# docker compose --profile ttl-test up -d --wait api-short-ttl
|
||||
api-short-ttl:
|
||||
<<: *api
|
||||
profiles: [ttl-test]
|
||||
environment:
|
||||
<<: *api-env
|
||||
SESSION_TTL_SECONDS: "2"
|
||||
ports:
|
||||
- "127.0.0.1:${SHORT_TTL_PORT:-8081}:8080"
|
||||
|
||||
volumes:
|
||||
pgdata:
|
||||
|
||||
secrets:
|
||||
postgres_password:
|
||||
file: ./secrets/postgres_password
|
||||
database_url:
|
||||
file: ./secrets/database_url
|
||||
cache_url:
|
||||
file: ./secrets/cache_url
|
||||
valkey_conf:
|
||||
file: ./secrets/valkey.conf
|
||||
@@ -0,0 +1,74 @@
|
||||
# EXPLAIN evidence for message history pagination
|
||||
# date: 2026-10-07T14:17:55Z commit: 65003b6
|
||||
# postgres: 17.6
|
||||
# messages total: 70112, in measured chat: 20000
|
||||
|
||||
PREPARE
|
||||
PREPARE
|
||||
=== 1. WITH index messages_chat_history_idx: first page (limit 50 + 1) ===
|
||||
QUERY PLAN
|
||||
---------------------------------------------------------------------------------------------------------------------------------------------------
|
||||
Limit (cost=0.42..8.60 rows=51 width=78) (actual time=0.014..0.019 rows=51 loops=1)
|
||||
Buffers: shared hit=4
|
||||
-> Index Scan using messages_chat_history_idx on messages (cost=0.42..3227.11 rows=20122 width=78) (actual time=0.013..0.015 rows=51 loops=1)
|
||||
Index Cond: (chat_id = '00000000-0000-4000-8000-000000000100'::uuid)
|
||||
Buffers: shared hit=4
|
||||
Planning:
|
||||
Buffers: shared hit=106
|
||||
Planning Time: 0.272 ms
|
||||
Execution Time: 0.031 ms
|
||||
(9 rows)
|
||||
|
||||
=== 2. WITH index: deep page, cursor after message #15000 of 20000 ===
|
||||
QUERY PLAN
|
||||
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|
||||
Limit (cost=0.42..55.74 rows=51 width=78) (actual time=0.012..0.017 rows=51 loops=1)
|
||||
Buffers: shared hit=4
|
||||
-> Index Scan using messages_chat_history_idx on messages (cost=0.42..1590.70 rows=1466 width=78) (actual time=0.012..0.014 rows=51 loops=1)
|
||||
Index Cond: ((chat_id = '00000000-0000-4000-8000-000000000100'::uuid) AND (ROW(created_at, id) > ROW('2026-01-01 04:10:00+00'::timestamp with time zone, '47ba2a9c-d238-4807-2008-6ea72cb0f95c'::uuid)))
|
||||
Buffers: shared hit=4
|
||||
Planning:
|
||||
Buffers: shared hit=42
|
||||
Planning Time: 0.276 ms
|
||||
Execution Time: 0.035 ms
|
||||
(9 rows)
|
||||
|
||||
BEGIN
|
||||
DROP INDEX
|
||||
=== 3. WITHOUT index: first page ===
|
||||
QUERY PLAN
|
||||
--------------------------------------------------------------------------------------------------------------------------
|
||||
Limit (cost=2487.71..2487.84 rows=51 width=78) (actual time=4.235..4.239 rows=51 loops=1)
|
||||
Buffers: shared hit=946
|
||||
-> Sort (cost=2487.71..2538.02 rows=20122 width=78) (actual time=4.234..4.236 rows=51 loops=1)
|
||||
Sort Key: created_at, id
|
||||
Sort Method: top-N heapsort Memory: 31kB
|
||||
Buffers: shared hit=946
|
||||
-> Seq Scan on messages (cost=0.00..1816.40 rows=20122 width=78) (actual time=0.011..3.325 rows=20000 loops=1)
|
||||
Filter: (chat_id = '00000000-0000-4000-8000-000000000100'::uuid)
|
||||
Rows Removed by Filter: 50112
|
||||
Buffers: shared hit=940
|
||||
Planning:
|
||||
Buffers: shared hit=16
|
||||
Planning Time: 0.098 ms
|
||||
Execution Time: 4.249 ms
|
||||
(14 rows)
|
||||
|
||||
=== 4. WITHOUT index: deep page ===
|
||||
QUERY PLAN
|
||||
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|
||||
Limit (cost=2215.87..2216.00 rows=51 width=78) (actual time=3.117..3.121 rows=51 loops=1)
|
||||
Buffers: shared hit=940
|
||||
-> Sort (cost=2215.87..2219.53 rows=1466 width=78) (actual time=3.116..3.118 rows=51 loops=1)
|
||||
Sort Key: created_at, id
|
||||
Sort Method: top-N heapsort Memory: 31kB
|
||||
Buffers: shared hit=940
|
||||
-> Seq Scan on messages (cost=0.00..2166.96 rows=1466 width=78) (actual time=0.605..2.853 rows=5000 loops=1)
|
||||
Filter: ((chat_id = '00000000-0000-4000-8000-000000000100'::uuid) AND (ROW(created_at, id) > ROW('2026-01-01 04:10:00+00'::timestamp with time zone, '47ba2a9c-d238-4807-2008-6ea72cb0f95c'::uuid)))
|
||||
Rows Removed by Filter: 65112
|
||||
Buffers: shared hit=940
|
||||
Planning Time: 0.069 ms
|
||||
Execution Time: 3.133 ms
|
||||
(12 rows)
|
||||
|
||||
ROLLBACK
|
||||
@@ -0,0 +1,20 @@
|
||||
# Fresh volume: migrations from scratch + public smoke
|
||||
# date: 2026-10-07T14:18:43Z
|
||||
$ APP_PORT=18090 docker compose -p lab2-clean up -d --wait
|
||||
Container lab2-clean-api-1 Created
|
||||
Container lab2-clean-api-1 Healthy
|
||||
Container lab2-clean-migrate-1 Created
|
||||
Container lab2-clean-migrate-1 Exited
|
||||
Container lab2-clean-postgres-1 Created
|
||||
Container lab2-clean-postgres-1 Healthy
|
||||
Container lab2-clean-valkey-1 Created
|
||||
Container lab2-clean-valkey-1 Healthy
|
||||
Network lab2-clean_default Created
|
||||
Volume "lab2-clean_pgdata" Created
|
||||
$ docker compose -p lab2-clean logs migrate
|
||||
INFO migrate: applying 0001_init
|
||||
INFO migrate: migrations up to date: 1 applied now, 0 before
|
||||
$ make test BASE_URL=http://localhost:18090
|
||||
PASS: lab2 public API scenario; infrastructure and grading evidence remain separate.
|
||||
$ docker compose -p lab2-clean run --rm migrate # second run
|
||||
INFO migrate: migrations up to date: 0 applied now, 1 before
|
||||
@@ -0,0 +1,20 @@
|
||||
# Persistence: API + PostgreSQL + Valkey restarted, volume kept, same token
|
||||
# date: 2026-10-07T14:19:03Z
|
||||
## containers BEFORE
|
||||
api 2026-10-07T14:18:25.262485419Z running restarts=0
|
||||
postgres 2026-10-07T14:18:22.306271835Z running restarts=0
|
||||
valkey 2026-10-07T14:18:21.888265501Z running restarts=0
|
||||
volume messenger-lab2_pgdata
|
||||
|
||||
$ make test-persistence ACTION_SCRIPT=scripts/restart.sh
|
||||
Executing explicit persistence hook; hook output suppressed to avoid leaking secrets.
|
||||
PASS: persistence preserved confirmed data and existing session. Hook scope needs operator evidence.
|
||||
|
||||
## containers AFTER (StartedAt changed for all three)
|
||||
api 2026-10-07T14:19:08.428103009Z running restarts=0
|
||||
postgres 2026-10-07T14:19:05.240867716Z running restarts=0
|
||||
valkey 2026-10-07T14:19:05.240850591Z running restarts=0
|
||||
volume messenger-lab2_pgdata
|
||||
|
||||
$ docker compose logs migrate | tail -1
|
||||
INFO migrate: migrations up to date: 0 applied now, 1 before
|
||||
@@ -0,0 +1,5 @@
|
||||
# date: 2026-10-07T14:19:14Z
|
||||
$ make check
|
||||
PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.
|
||||
$ make test
|
||||
PASS: lab2 public API scenario; infrastructure and grading evidence remain separate.
|
||||
@@ -0,0 +1,44 @@
|
||||
# date: 2026-10-07T14:19:14Z
|
||||
$ make test-student
|
||||
BASE_URL="http://localhost:18080" sh student-tests/run_all.sh
|
||||
== own_tests.py
|
||||
ok - malformed JSON -> 400 invalid_request
|
||||
ok - foreign chat access -> 404 (no leak)
|
||||
ok - empty/whitespace message text -> 400
|
||||
ok - duplicate username -> 409, original user intact
|
||||
ok - multi-page pagination traversal: 4 pages, 7 messages, no dup/loss
|
||||
ok - cursor from another chat -> 400 (cursor is bound to its resource)
|
||||
PASS: 6/6 student checks
|
||||
|
||||
== concurrency_test.py
|
||||
PASS: 100 unique messages written and read back from 10 concurrent clients
|
||||
|
||||
== auth_test.py
|
||||
ok - wrong password / unknown user -> identical 401 Basic
|
||||
ok - missing / malformed / non-Basic login -> 401
|
||||
ok - UTF-8 password containing ':' -> 201
|
||||
ok - password shorter than 12 / missing -> 400
|
||||
ok - Basic / X-User-Id / unknown token on business endpoint -> 401
|
||||
ok - logout revokes only the current token; repeat logout -> 401
|
||||
ok - session with TTL=2s is rejected after 3s (cache entry included)
|
||||
PASS: 7 checks
|
||||
|
||||
== db_integrity_test.py
|
||||
ok - 20 concurrent registrations of one username -> 1x201, 19x409, 1 row
|
||||
ok - chat with unknown member -> 400 and transaction rolled back (no orphan chat)
|
||||
ok - password stored as $argon2id$..., sessions keep only sha256(token)
|
||||
ok - migrations run twice more: schema_migrations and data unchanged (1:49)
|
||||
PASS: 4 checks
|
||||
|
||||
== dependency_test.py
|
||||
ok - Valkey caches sessions with TTL<=300s and keeps logout tombstones
|
||||
ok - Valkey stopped: ready=200, reads/writes OK, revoked token 401, 4 ms/request
|
||||
ok - PostgreSQL stopped: live=200, ready/business=503 in 0.0s total, unverified token not trusted
|
||||
ok - PostgreSQL started again: API recovers without restart, data intact
|
||||
PASS: 4 checks
|
||||
|
||||
== sigterm_test.sh
|
||||
api stopped in 0s, exit code 0 (0 = graceful, 137 = SIGKILL)
|
||||
PASS: graceful SIGTERM shutdown, api restarted
|
||||
|
||||
ALL STUDENT TESTS PASSED
|
||||
@@ -0,0 +1,58 @@
|
||||
-- Initial schema for lab2. Applied once by app/migrate.py inside one transaction.
|
||||
|
||||
CREATE TABLE users (
|
||||
id uuid PRIMARY KEY,
|
||||
username text NOT NULL,
|
||||
display_name text NOT NULL,
|
||||
password_hash text NOT NULL,
|
||||
created_at timestamptz NOT NULL DEFAULT clock_timestamp(),
|
||||
CONSTRAINT users_username_key UNIQUE (username),
|
||||
CONSTRAINT users_username_format CHECK (username ~ '^[a-z0-9_]{3,32}$'),
|
||||
CONSTRAINT users_display_name_len CHECK (char_length(display_name) BETWEEN 1 AND 100)
|
||||
);
|
||||
|
||||
CREATE TABLE chats (
|
||||
id uuid PRIMARY KEY,
|
||||
title text NOT NULL,
|
||||
created_at timestamptz NOT NULL DEFAULT clock_timestamp(),
|
||||
CONSTRAINT chats_title_len CHECK (char_length(title) BETWEEN 1 AND 100)
|
||||
);
|
||||
|
||||
-- position keeps member_ids in the order the creator sent them.
|
||||
CREATE TABLE chat_members (
|
||||
chat_id uuid NOT NULL REFERENCES chats (id),
|
||||
user_id uuid NOT NULL REFERENCES users (id),
|
||||
position smallint NOT NULL,
|
||||
PRIMARY KEY (chat_id, user_id)
|
||||
);
|
||||
|
||||
-- "Chats of a user": GET /chats starts from the caller's memberships.
|
||||
CREATE INDEX chat_members_user_idx ON chat_members (user_id, chat_id);
|
||||
|
||||
CREATE TABLE messages (
|
||||
id uuid PRIMARY KEY,
|
||||
chat_id uuid NOT NULL,
|
||||
sender_id uuid NOT NULL,
|
||||
text text NOT NULL,
|
||||
created_at timestamptz NOT NULL DEFAULT clock_timestamp(),
|
||||
-- The sender must be a member of the chat, enforced by the database itself.
|
||||
CONSTRAINT messages_sender_member_fk FOREIGN KEY (chat_id, sender_id)
|
||||
REFERENCES chat_members (chat_id, user_id),
|
||||
CONSTRAINT messages_text_len CHECK (char_length(text) BETWEEN 1 AND 4000)
|
||||
);
|
||||
|
||||
-- Keyset pagination of history: WHERE chat_id = $1 AND (created_at, id) > ($2, $3)
|
||||
-- ORDER BY created_at, id LIMIT n. Justified by EXPLAIN in evidence/explain_history.txt.
|
||||
CREATE INDEX messages_chat_history_idx ON messages (chat_id, created_at, id);
|
||||
|
||||
-- Sessions: PostgreSQL is the source of truth; Valkey only caches lookups.
|
||||
-- token_hash = sha256(token); the raw token is never stored.
|
||||
CREATE TABLE sessions (
|
||||
token_hash bytea PRIMARY KEY,
|
||||
user_id uuid NOT NULL REFERENCES users (id),
|
||||
created_at timestamptz NOT NULL DEFAULT clock_timestamp(),
|
||||
expires_at timestamptz NOT NULL,
|
||||
revoked_at timestamptz
|
||||
);
|
||||
|
||||
CREATE INDEX sessions_expires_idx ON sessions (expires_at);
|
||||
@@ -0,0 +1,6 @@
|
||||
fastapi==0.142.2
|
||||
uvicorn[standard]==0.54.0
|
||||
pydantic==2.13.5
|
||||
asyncpg==0.32.0
|
||||
redis==8.1.0
|
||||
argon2-cffi==25.1.0
|
||||
Executable
+32
@@ -0,0 +1,32 @@
|
||||
#!/bin/sh
|
||||
# Generate local synthetic credentials into ./secrets (git-ignored). Idempotent:
|
||||
# existing files are kept, so a running stack keeps working. Prints no secret values.
|
||||
set -eu
|
||||
|
||||
cd "$(dirname "$0")/.."
|
||||
mkdir -p secrets
|
||||
|
||||
gen() { python3 -c 'import secrets; print(secrets.token_hex(24))'; }
|
||||
|
||||
if [ ! -f secrets/postgres_password ]; then
|
||||
gen > secrets/postgres_password
|
||||
echo "created secrets/postgres_password"
|
||||
fi
|
||||
PG_PASS=$(cat secrets/postgres_password)
|
||||
printf 'postgresql://messenger:%s@postgres:5432/messenger\n' "$PG_PASS" > secrets/database_url
|
||||
|
||||
if [ ! -f secrets/valkey.conf ]; then
|
||||
{
|
||||
printf 'requirepass %s\n' "$(gen)"
|
||||
# Pure cache: PostgreSQL holds the sessions, so no RDB/AOF is needed.
|
||||
printf 'save ""\nappendonly no\n'
|
||||
printf 'maxmemory 64mb\nmaxmemory-policy volatile-lru\n'
|
||||
} > secrets/valkey.conf
|
||||
echo "created secrets/valkey.conf"
|
||||
fi
|
||||
VK_PASS=$(sed -n 's/^requirepass //p' secrets/valkey.conf)
|
||||
printf 'redis://:%s@valkey:6379/0\n' "$VK_PASS" > secrets/cache_url
|
||||
|
||||
# Containers run as non-root users that do not own these files on Linux.
|
||||
chmod 644 secrets/*
|
||||
echo "secrets ready in ./secrets (not tracked by git)"
|
||||
Executable
+12
@@ -0,0 +1,12 @@
|
||||
#!/bin/sh
|
||||
# Restart the API and ALL state stores (PostgreSQL, Valkey), keeping volumes.
|
||||
# Never `down -v`, never flush, never re-seed. Migrations run again (idempotent).
|
||||
set -eu
|
||||
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
docker compose stop api
|
||||
docker compose stop valkey postgres
|
||||
# depends_on brings postgres, valkey and migrate up first; --wait = api healthcheck (/health/ready).
|
||||
docker compose up -d --no-build --wait api
|
||||
docker compose ps --all --format 'table {{.Service}}\t{{.Status}}'
|
||||
Executable
+70
@@ -0,0 +1,70 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Sessions: wrong password, Basic/Bearer separation, X-User-Id ignored, logout, expiry.
|
||||
|
||||
Expiry runs against the api-short-ttl service (SESSION_TTL_SECONDS=2):
|
||||
docker compose --profile ttl-test up -d --wait api-short-ttl
|
||||
"""
|
||||
import base64
|
||||
import time
|
||||
import uuid
|
||||
|
||||
from stack import BASE_URL, SHORT_TTL_URL, Client, check, main_wrapper, ok
|
||||
|
||||
|
||||
def basic(username, password):
|
||||
return {"Authorization": "Basic " + base64.b64encode((username + ":" + password).encode("utf-8")).decode()}
|
||||
|
||||
|
||||
def main():
|
||||
c = Client(BASE_URL)
|
||||
alice = c.user("auth")
|
||||
reg = alice["registration"]
|
||||
|
||||
# Wrong password and unknown user: the same 401 with a Basic challenge.
|
||||
r1 = c.call("POST", "/api/v1/auth/sessions", 401, headers=basic(reg["username"], "wrong-password-123"))
|
||||
r2 = c.call("POST", "/api/v1/auth/sessions", 401, headers=basic("nobody_" + uuid.uuid4().hex[:8], "wrong-password-123"))
|
||||
check(r1["json"]["error"]["message"] == r2["json"]["error"]["message"], "unknown user must look like a wrong password")
|
||||
ok("wrong password / unknown user -> identical 401 Basic")
|
||||
|
||||
c.call("POST", "/api/v1/auth/sessions", 401)
|
||||
c.call("POST", "/api/v1/auth/sessions", 401, headers={"Authorization": "Basic !!!not-base64"})
|
||||
c.call("POST", "/api/v1/auth/sessions", 401, headers={"Authorization": "Bearer " + alice["token"]})
|
||||
ok("missing / malformed / non-Basic login -> 401")
|
||||
|
||||
# A password with ':' and non-ASCII characters survives Basic encoding.
|
||||
tricky = "пароль:с:двоеточием-" + uuid.uuid4().hex[:6]
|
||||
user = c.call("POST", "/api/v1/users", 201, body={"username": "u_colon_" + uuid.uuid4().hex[:8], "display_name": "Двоеточие", "password": tricky}, schema="User")["json"]
|
||||
c.call("POST", "/api/v1/auth/sessions", 201, headers=basic(user["username"], tricky), schema="Session")
|
||||
ok("UTF-8 password containing ':' -> 201")
|
||||
|
||||
c.call("POST", "/api/v1/users", 400, body={"username": "u_short_" + uuid.uuid4().hex[:8], "display_name": "x", "password": "short-11ch"})
|
||||
c.call("POST", "/api/v1/users", 400, body={"username": "u_nopass_" + uuid.uuid4().hex[:8], "display_name": "x"})
|
||||
ok("password shorter than 12 / missing -> 400")
|
||||
|
||||
me = c.call("GET", "/api/v1/users/me", 401, headers=basic(reg["username"], reg["password"]))
|
||||
check(me["headers"].get("WWW-Authenticate", "").startswith("Bearer"), "Bearer challenge expected")
|
||||
c.call("GET", "/api/v1/users/me", 401, headers={"X-User-Id": alice["id"]})
|
||||
c.call("GET", "/api/v1/users/me", 401, headers={"Authorization": "Bearer " + uuid.uuid4().hex * 2})
|
||||
ok("Basic / X-User-Id / unknown token on business endpoint -> 401")
|
||||
|
||||
session = c.call("POST", "/api/v1/auth/sessions", 201, headers=basic(reg["username"], reg["password"]), schema="Session")["json"]
|
||||
check(session["token"] != alice["token"], "every login must issue a new token")
|
||||
second = dict(alice, token=session["token"])
|
||||
c.call("DELETE", "/api/v1/auth/sessions/current", 204, actor=alice)
|
||||
c.call("GET", "/api/v1/users/me", 401, actor=alice)
|
||||
c.call("DELETE", "/api/v1/auth/sessions/current", 401, actor=alice)
|
||||
c.call("GET", "/api/v1/users/me", 200, actor=second, schema="User")
|
||||
ok("logout revokes only the current token; repeat logout -> 401")
|
||||
|
||||
# Expiry: separate instance of the same image with SESSION_TTL_SECONDS=2.
|
||||
s = Client(SHORT_TTL_URL)
|
||||
bob = s.user("ttl")
|
||||
s.call("GET", "/api/v1/users/me", 200, actor=bob, schema="User")
|
||||
time.sleep(3)
|
||||
s.call("GET", "/api/v1/users/me", 401, actor=bob)
|
||||
s.call("DELETE", "/api/v1/auth/sessions/current", 401, actor=bob)
|
||||
ok("session with TTL=2s is rejected after 3s (cache entry included)")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main_wrapper(main)
|
||||
Executable
+61
@@ -0,0 +1,61 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Level-4 race-protection evidence: 10 clients send 100 messages concurrently;
|
||||
expect exactly 100 unique messages, with no loss/duplication under pagination.
|
||||
"""
|
||||
import sys
|
||||
import urllib.parse
|
||||
import concurrent.futures
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT / "tests"))
|
||||
from client import Client, check # noqa: E402
|
||||
|
||||
|
||||
def main():
|
||||
base_url = sys.argv[1] if len(sys.argv) > 1 else "http://localhost:8080"
|
||||
n_messages = 100
|
||||
n_clients = 10
|
||||
|
||||
c = Client(base_url)
|
||||
users = [c.user("race%d" % i) for i in range(n_clients)]
|
||||
chat = c.call(
|
||||
"POST", "/api/v1/chats", 201, actor=users[0],
|
||||
body={"title": "race", "member_ids": [u["id"] for u in users]},
|
||||
schema="Chat",
|
||||
)["json"]
|
||||
path = "/api/v1/chats/" + chat["id"] + "/messages"
|
||||
|
||||
def send(i):
|
||||
actor = users[i % n_clients]
|
||||
return c.message(actor, chat, "race-%d" % i)["json"]["id"]
|
||||
|
||||
with concurrent.futures.ThreadPoolExecutor(max_workers=n_clients) as pool:
|
||||
ids = list(pool.map(send, range(n_messages)))
|
||||
|
||||
check(len(ids) == n_messages, "expected %d responses" % n_messages)
|
||||
check(len(set(ids)) == n_messages, "expected %d unique message ids, got %d" % (n_messages, len(set(ids))))
|
||||
|
||||
# Walk the full history back through the API to confirm the store itself
|
||||
# (not just the HTTP responses) holds exactly n_messages distinct rows.
|
||||
seen = []
|
||||
cursor = None
|
||||
while True:
|
||||
qs = "?limit=25" + ("&cursor=" + urllib.parse.quote(cursor, safe="") if cursor else "")
|
||||
page = c.call("GET", path + qs, actor=users[0], schema="MessagePage")["json"]
|
||||
seen.extend(page["items"])
|
||||
cursor = page["next_cursor"]
|
||||
if not cursor:
|
||||
break
|
||||
check(len(seen) == n_messages, "expected to read back %d messages, got %d" % (n_messages, len(seen)))
|
||||
check(len({m["id"] for m in seen}) == n_messages, "readback contains duplicates")
|
||||
|
||||
print("PASS: %d unique messages written and read back from %d concurrent clients" % (n_messages, n_clients))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
try:
|
||||
main()
|
||||
except AssertionError as exc:
|
||||
print("FAIL:", exc)
|
||||
raise SystemExit(1)
|
||||
Executable
+53
@@ -0,0 +1,53 @@
|
||||
#!/usr/bin/env python3
|
||||
"""PostgreSQL integrity: registration race, rollback of chat creation, repeatable migrations,
|
||||
password storage. Needs the compose stack (reads tables via `docker compose exec postgres`)."""
|
||||
import concurrent.futures
|
||||
import uuid
|
||||
|
||||
from stack import BASE_URL, Client, check, compose, main_wrapper, ok, psql
|
||||
|
||||
|
||||
def main():
|
||||
c = Client(BASE_URL)
|
||||
|
||||
# 1. Registration race: 20 parallel requests for one username.
|
||||
username = "u_race_" + uuid.uuid4().hex[:10]
|
||||
body = {"username": username, "display_name": "Гонка", "password": "Race-password-123"}
|
||||
|
||||
def register(_):
|
||||
return c.call("POST", "/api/v1/users", [201, 409], body=body)["status"]
|
||||
|
||||
with concurrent.futures.ThreadPoolExecutor(max_workers=20) as pool:
|
||||
statuses = list(pool.map(register, range(20)))
|
||||
check(statuses.count(201) == 1, "exactly one registration must win, got %s" % statuses)
|
||||
check(statuses.count(409) == 19, "all others must get 409, got %s" % statuses)
|
||||
check(psql("SELECT count(*) FROM users WHERE username = '%s'" % username) == "1", "exactly one row expected")
|
||||
ok("20 concurrent registrations of one username -> 1x201, 19x409, 1 row")
|
||||
|
||||
# 2. Rollback: the chat row is inserted, then the member insert hits the FK.
|
||||
alice = c.user("rb")
|
||||
title = "rollback-" + uuid.uuid4().hex
|
||||
c.call("POST", "/api/v1/chats", 400, actor=alice, body={"title": title, "member_ids": [alice["id"], str(uuid.uuid4())]})
|
||||
check(psql("SELECT count(*) FROM chats WHERE title = '%s'" % title) == "0", "failed chat creation left a chat row")
|
||||
check(psql("SELECT count(*) FROM chats c WHERE NOT EXISTS (SELECT 1 FROM chat_members m WHERE m.chat_id = c.id)") == "0",
|
||||
"chat without members exists")
|
||||
ok("chat with unknown member -> 400 and transaction rolled back (no orphan chat)")
|
||||
|
||||
# 3. Passwords are stored as Argon2id hashes, never plaintext.
|
||||
stored = psql("SELECT password_hash FROM users WHERE username = '%s'" % username)
|
||||
check(stored.startswith("$argon2id$") and "Race-password-123" not in stored, "password must be an Argon2id hash")
|
||||
check(psql("SELECT count(*) FROM information_schema.columns WHERE table_name = 'sessions' AND column_name = 'token'") == "0",
|
||||
"raw tokens must not be stored")
|
||||
ok("password stored as $argon2id$..., sessions keep only sha256(token)")
|
||||
|
||||
# 4. Migrations re-run without duplicates or data changes.
|
||||
before = psql("SELECT (SELECT count(*) FROM schema_migrations) || ':' || (SELECT count(*) FROM users)")
|
||||
compose("run", "--rm", "--no-deps", "migrate")
|
||||
compose("run", "--rm", "--no-deps", "migrate")
|
||||
after = psql("SELECT (SELECT count(*) FROM schema_migrations) || ':' || (SELECT count(*) FROM users)")
|
||||
check(before == after, "migrations changed data on re-run: %s -> %s" % (before, after))
|
||||
ok("migrations run twice more: schema_migrations and data unchanged (%s)" % after)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main_wrapper(main)
|
||||
Executable
+86
@@ -0,0 +1,86 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Valkey's role and failures of each dependency. STOPS AND STARTS containers of this stack.
|
||||
|
||||
- Valkey really caches sessions and holds logout tombstones.
|
||||
- Valkey down: API keeps working (cache bypassed), readiness stays 200.
|
||||
- PostgreSQL down: bounded 503 quickly, readiness 503, liveness 200; an unverifiable
|
||||
token is never accepted; no token is still 401. Recovery after start.
|
||||
"""
|
||||
import time
|
||||
|
||||
from stack import BASE_URL, Client, Failure, check, compose, main_wrapper, ok, valkey
|
||||
|
||||
|
||||
def wait_ready(c, seconds=60):
|
||||
deadline = time.monotonic() + seconds
|
||||
while time.monotonic() < deadline:
|
||||
try:
|
||||
c.call("GET", "/health/ready", 200)
|
||||
return
|
||||
except Failure:
|
||||
time.sleep(0.5)
|
||||
raise Failure("API did not become ready")
|
||||
|
||||
|
||||
def main():
|
||||
c = Client(BASE_URL, timeout=10)
|
||||
alice, bob, _eve, chat = c.bootstrap()
|
||||
history = "/api/v1/chats/" + chat["id"] + "/messages"
|
||||
|
||||
# Role of Valkey: a session lookup fills the cache, logout leaves a tombstone.
|
||||
before = int(valkey("DBSIZE"))
|
||||
c.call("GET", "/api/v1/users/me", actor=alice, schema="User")
|
||||
keys = valkey("--scan", "--pattern", "session:*").split()
|
||||
check(len(keys) >= 1 and int(valkey("DBSIZE")) >= before, "session lookups must be cached in Valkey")
|
||||
ttl = int(valkey("TTL", keys[0]))
|
||||
check(0 < ttl <= 300, "cache entries must expire (TTL %d)" % ttl)
|
||||
temp = c.user("tomb")
|
||||
c.call("GET", "/api/v1/users/me", actor=temp, schema="User")
|
||||
c.call("DELETE", "/api/v1/auth/sessions/current", 204, actor=temp)
|
||||
tombstones = [k for k in valkey("--scan", "--pattern", "session:*").split() if valkey("GET", k) == "revoked"]
|
||||
check(tombstones, "logout must leave a 'revoked' tombstone in the cache")
|
||||
ok("Valkey caches sessions with TTL<=300s and keeps logout tombstones")
|
||||
|
||||
# Valkey down -> the API bypasses the cache, PostgreSQL stays the source of truth.
|
||||
try:
|
||||
compose("stop", "valkey")
|
||||
c.call("GET", "/health/ready", 200)
|
||||
c.call("GET", "/api/v1/users/me", actor=alice, schema="User")
|
||||
c.message(bob, chat, "без кэша")
|
||||
c.call("GET", "/api/v1/users/me", 401, actor=temp) # revoked in PostgreSQL too
|
||||
start = time.monotonic()
|
||||
for _ in range(10):
|
||||
c.call("GET", history + "?limit=1", actor=alice, schema="MessagePage")
|
||||
per_request = (time.monotonic() - start) / 10
|
||||
check(per_request < 0.5, "cache bypass must be fast (%.3fs per request)" % per_request)
|
||||
ok("Valkey stopped: ready=200, reads/writes OK, revoked token 401, %.0f ms/request" % (per_request * 1000))
|
||||
finally:
|
||||
compose("start", "valkey")
|
||||
|
||||
# PostgreSQL down -> bounded 503, never anonymous access.
|
||||
fresh = c.user("dbdown") # token never looked up yet: not in the cache
|
||||
try:
|
||||
compose("stop", "postgres")
|
||||
c.call("GET", "/health/live", 200)
|
||||
start = time.monotonic()
|
||||
r = c.call("GET", "/health/ready", 503)
|
||||
check(r["json"]["error"]["code"] == "unavailable", "readiness must report unavailable")
|
||||
c.call("GET", "/api/v1/chats", 503, actor=alice)
|
||||
c.call("POST", history, 503, actor=alice, body={"text": "в никуда"})
|
||||
c.call("GET", "/api/v1/users/me", 503, actor=fresh) # cannot verify -> not trusted
|
||||
c.call("GET", "/api/v1/users/me", 401) # no credentials at all
|
||||
elapsed = time.monotonic() - start
|
||||
check(elapsed < 20, "errors must be bounded in time (%.1fs)" % elapsed)
|
||||
ok("PostgreSQL stopped: live=200, ready/business=503 in %.1fs total, unverified token not trusted" % elapsed)
|
||||
finally:
|
||||
compose("start", "postgres")
|
||||
|
||||
wait_ready(c)
|
||||
c.call("GET", "/api/v1/users/me", actor=fresh, schema="User")
|
||||
page = c.call("GET", history + "?limit=100", actor=alice, schema="MessagePage")["json"]
|
||||
check(any(m["text"] == "без кэша" for m in page["items"]), "message written without cache must persist")
|
||||
ok("PostgreSQL started again: API recovers without restart, data intact")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main_wrapper(main)
|
||||
Executable
+81
@@ -0,0 +1,81 @@
|
||||
#!/bin/sh
|
||||
# Seed synthetic history (20 000 messages in one chat + 50 000 in 50 other chats) and
|
||||
# record EXPLAIN (ANALYZE, BUFFERS) of the exact keyset queries used by the API,
|
||||
# with and without messages_chat_history_idx. Output: evidence/explain_history.txt
|
||||
# Seeding is idempotent (fixed IDs, ON CONFLICT DO NOTHING); data is synthetic.
|
||||
set -eu
|
||||
|
||||
cd "$(dirname "$0")/.."
|
||||
OUT="${OUT:-evidence/explain_history.txt}"
|
||||
PSQL="docker compose exec -T postgres psql -U messenger -d messenger -X -v ON_ERROR_STOP=1"
|
||||
|
||||
$PSQL -q <<'SQL'
|
||||
INSERT INTO users (id, username, display_name, password_hash) VALUES
|
||||
('00000000-0000-4000-8000-000000000001', 'seed_explain_a', 'Seed A', 'not-a-login'),
|
||||
('00000000-0000-4000-8000-000000000002', 'seed_explain_b', 'Seed B', 'not-a-login')
|
||||
ON CONFLICT DO NOTHING;
|
||||
|
||||
-- chat ...0100 is the measured one; ...0101-0150 are background noise.
|
||||
INSERT INTO chats (id, title, created_at)
|
||||
SELECT ('00000000-0000-4000-8000-' || lpad((100 + g)::text, 12, '0'))::uuid, 'explain ' || g, now()
|
||||
FROM generate_series(0, 50) g
|
||||
ON CONFLICT DO NOTHING;
|
||||
|
||||
INSERT INTO chat_members (chat_id, user_id, position)
|
||||
SELECT c.id, u.id, u.pos
|
||||
FROM chats c
|
||||
CROSS JOIN (VALUES ('00000000-0000-4000-8000-000000000001'::uuid, 0),
|
||||
('00000000-0000-4000-8000-000000000002'::uuid, 1)) AS u(id, pos)
|
||||
WHERE c.title LIKE 'explain %'
|
||||
ON CONFLICT DO NOTHING;
|
||||
|
||||
INSERT INTO messages (id, chat_id, sender_id, text, created_at)
|
||||
SELECT md5('explain-' || c.n || '-' || g)::uuid,
|
||||
('00000000-0000-4000-8000-' || lpad((100 + c.n)::text, 12, '0'))::uuid,
|
||||
CASE WHEN g % 2 = 0 THEN '00000000-0000-4000-8000-000000000001'::uuid
|
||||
ELSE '00000000-0000-4000-8000-000000000002'::uuid END,
|
||||
'synthetic message ' || g,
|
||||
timestamptz '2026-01-01' + (g * interval '1 second') + (c.n * interval '1 millisecond')
|
||||
FROM (SELECT n, CASE WHEN n = 0 THEN 20000 ELSE 1000 END AS cnt FROM generate_series(0, 50) n) c
|
||||
CROSS JOIN LATERAL generate_series(1, c.cnt) g
|
||||
ON CONFLICT DO NOTHING;
|
||||
|
||||
ANALYZE users; ANALYZE chats; ANALYZE chat_members; ANALYZE messages;
|
||||
SQL
|
||||
|
||||
{
|
||||
echo "# EXPLAIN evidence for message history pagination"
|
||||
echo "# date: $(date -u +%Y-%m-%dT%H:%M:%SZ) commit: $(git rev-parse --short HEAD 2>/dev/null || echo n/a)"
|
||||
echo "# postgres: $($PSQL -tA -c 'SHOW server_version')"
|
||||
$PSQL -tA -c "SELECT '# messages total: ' || count(*) || ', in measured chat: ' || count(*) FILTER (WHERE chat_id = '00000000-0000-4000-8000-000000000100') FROM messages"
|
||||
echo
|
||||
$PSQL <<'SQL'
|
||||
-- The same statements as app/storage.py (Store.list_messages), as prepared statements.
|
||||
PREPARE first_page(uuid, int) AS
|
||||
SELECT id, chat_id, sender_id, text, created_at FROM messages
|
||||
WHERE chat_id = $1 ORDER BY created_at, id LIMIT $2;
|
||||
PREPARE next_page(uuid, timestamptz, uuid, int) AS
|
||||
SELECT id, chat_id, sender_id, text, created_at FROM messages
|
||||
WHERE chat_id = $1 AND (created_at, id) > ($2, $3) ORDER BY created_at, id LIMIT $4;
|
||||
|
||||
\echo '=== 1. WITH index messages_chat_history_idx: first page (limit 50 + 1) ==='
|
||||
EXPLAIN (ANALYZE, BUFFERS) EXECUTE first_page('00000000-0000-4000-8000-000000000100', 51);
|
||||
|
||||
\echo '=== 2. WITH index: deep page, cursor after message #15000 of 20000 ==='
|
||||
EXPLAIN (ANALYZE, BUFFERS) EXECUTE next_page('00000000-0000-4000-8000-000000000100',
|
||||
timestamptz '2026-01-01' + interval '15000 seconds', md5('explain-0-15000')::uuid, 51);
|
||||
|
||||
-- Same queries without the index, inside a transaction that is rolled back.
|
||||
BEGIN;
|
||||
DROP INDEX messages_chat_history_idx;
|
||||
\echo '=== 3. WITHOUT index: first page ==='
|
||||
EXPLAIN (ANALYZE, BUFFERS) EXECUTE first_page('00000000-0000-4000-8000-000000000100', 51);
|
||||
\echo '=== 4. WITHOUT index: deep page ==='
|
||||
EXPLAIN (ANALYZE, BUFFERS) EXECUTE next_page('00000000-0000-4000-8000-000000000100',
|
||||
timestamptz '2026-01-01' + interval '15000 seconds', md5('explain-0-15000')::uuid, 51);
|
||||
ROLLBACK;
|
||||
SQL
|
||||
} > "$OUT"
|
||||
|
||||
echo "written $OUT"
|
||||
grep -E '^===|Index|Seq Scan|Sort|Buffers|Execution Time' "$OUT"
|
||||
Executable
+120
@@ -0,0 +1,120 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Student-written negative/edge-case tests, additional to the public tests/smoke.py.
|
||||
|
||||
Covers level-4 requirements from README.md:
|
||||
- malformed JSON
|
||||
- foreign chat access
|
||||
- empty message text
|
||||
- duplicate username
|
||||
- multi-page pagination traversal
|
||||
"""
|
||||
import json
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT / "tests"))
|
||||
from client import Client, check # noqa: E402
|
||||
|
||||
PASSED = 0
|
||||
|
||||
|
||||
def ok(label):
|
||||
global PASSED
|
||||
PASSED += 1
|
||||
print("ok -", label)
|
||||
|
||||
|
||||
def raw_call(base, method, path, data=None, headers=None):
|
||||
h = {"Accept": "application/json"}
|
||||
h.update(headers or {})
|
||||
req = urllib.request.Request(base + path, data=data, headers=h, method=method)
|
||||
try:
|
||||
resp = urllib.request.urlopen(req, timeout=10)
|
||||
except urllib.error.HTTPError as exc:
|
||||
resp = exc
|
||||
with resp:
|
||||
return resp.code, resp.read()
|
||||
|
||||
|
||||
def main():
|
||||
base_url = sys.argv[1] if len(sys.argv) > 1 else "http://localhost:8080"
|
||||
c = Client(base_url)
|
||||
alice, bob, eve, chat = c.bootstrap()
|
||||
path = "/api/v1/chats/" + chat["id"]
|
||||
|
||||
# 1. Malformed JSON body -> 400 invalid_request, not a 500.
|
||||
status, body = raw_call(
|
||||
base_url, "POST", "/api/v1/users",
|
||||
data=b'{"username": "broken", "display_name": ',
|
||||
headers={"Content-Type": "application/json", "X-User-Id": alice["id"]},
|
||||
)
|
||||
check(status == 400, "malformed JSON must yield 400, got %d" % status)
|
||||
parsed = json.loads(body)
|
||||
check(parsed["error"]["code"] == "invalid_request", "malformed JSON must report invalid_request")
|
||||
ok("malformed JSON -> 400 invalid_request")
|
||||
|
||||
# 2. Foreign chat: eve is not a member -> 404, not 403 (no existence leak).
|
||||
c.call("GET", path, 404, actor=eve)
|
||||
c.call("GET", path + "/messages", 404, actor=eve)
|
||||
ok("foreign chat access -> 404 (no leak)")
|
||||
|
||||
# 3. Empty / whitespace-only message text -> 400.
|
||||
c.call("POST", path + "/messages", 400, actor=alice, body={"text": ""})
|
||||
c.call("POST", path + "/messages", 400, actor=alice, body={"text": " "})
|
||||
ok("empty/whitespace message text -> 400")
|
||||
|
||||
# 4. Duplicate username -> 409 conflict, original user untouched.
|
||||
c.call("POST", "/api/v1/users", 409, body=alice["registration"])
|
||||
me = c.call("GET", "/api/v1/users/me", actor=alice, schema="User")["json"]
|
||||
check(me == alice["user"], "existing user must be unaffected by a rejected duplicate registration")
|
||||
ok("duplicate username -> 409, original user intact")
|
||||
|
||||
# 5. Multi-page pagination traversal: write N messages, walk every page with a small
|
||||
# limit and confirm the full, ordered, duplicate-free set is recovered.
|
||||
sent = [c.message(alice, chat, "page-msg-%d" % i)["json"] for i in range(7)]
|
||||
collected = []
|
||||
cursor = None
|
||||
pages = 0
|
||||
while True:
|
||||
qs = "?limit=2" + ("&cursor=" + urllib.parse.quote(cursor, safe="") if cursor else "")
|
||||
page = c.call("GET", path + "/messages" + qs, actor=bob, schema="MessagePage")["json"]
|
||||
pages += 1
|
||||
check(len(page["items"]) <= 2, "page must respect limit")
|
||||
collected.extend(page["items"])
|
||||
cursor = page["next_cursor"]
|
||||
if not cursor:
|
||||
break
|
||||
check(pages < 50, "pagination did not terminate")
|
||||
ids_seen = [m["id"] for m in collected]
|
||||
check(len(ids_seen) == len(set(ids_seen)), "pagination must not repeat messages")
|
||||
check({m["id"] for m in sent}.issubset(set(ids_seen)), "pagination must not drop messages")
|
||||
tail = [m for m in collected if m["id"] in {s["id"] for s in sent}]
|
||||
check(tail == sorted(tail, key=lambda m: (m["created_at"], m["id"])), "pagination must stay ordered")
|
||||
ok("multi-page pagination traversal: %d pages, %d messages, no dup/loss" % (pages, len(collected)))
|
||||
|
||||
# 6. A cursor is bound to the chat it was issued for: reusing it on a
|
||||
# different chat the same actor can also read must be rejected.
|
||||
first_page = c.call("GET", path + "/messages?limit=1", actor=alice, schema="MessagePage")["json"]
|
||||
check(first_page["next_cursor"], "need a non-null cursor to test cross-chat reuse")
|
||||
chat2 = c.call(
|
||||
"POST", "/api/v1/chats", 201, actor=alice,
|
||||
body={"title": "second chat", "member_ids": [alice["id"], bob["id"]]}, schema="Chat",
|
||||
)["json"]
|
||||
c.message(alice, chat2, "unrelated message")
|
||||
foreign_cursor = urllib.parse.quote(first_page["next_cursor"], safe="")
|
||||
c.call("GET", "/api/v1/chats/" + chat2["id"] + "/messages?limit=1&cursor=" + foreign_cursor, 400, actor=alice)
|
||||
ok("cursor from another chat -> 400 (cursor is bound to its resource)")
|
||||
|
||||
print("PASS: %d/%d student checks" % (PASSED, 6))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
try:
|
||||
main()
|
||||
except AssertionError as exc:
|
||||
print("FAIL:", exc)
|
||||
raise SystemExit(1)
|
||||
Executable
+24
@@ -0,0 +1,24 @@
|
||||
#!/bin/sh
|
||||
# All student tests in one command. Needs the running compose stack; starts the
|
||||
# short-TTL instance itself. dependency_test/sigterm_test STOP AND START containers.
|
||||
# Usage: BASE_URL=http://localhost:8080 student-tests/run_all.sh
|
||||
set -eu
|
||||
|
||||
cd "$(dirname "$0")/.."
|
||||
PY="${PYTHON:-python3}"
|
||||
export BASE_URL="${BASE_URL:-http://localhost:8080}"
|
||||
export SHORT_TTL_URL="${SHORT_TTL_URL:-http://localhost:8081}"
|
||||
|
||||
docker compose --profile ttl-test up -d --no-build --wait api-short-ttl >/dev/null 2>&1
|
||||
|
||||
for t in own_tests.py concurrency_test.py auth_test.py db_integrity_test.py dependency_test.py; do
|
||||
echo "== $t"
|
||||
"$PY" "student-tests/$t" "$BASE_URL"
|
||||
echo
|
||||
done
|
||||
echo "== sigterm_test.sh"
|
||||
sh student-tests/sigterm_test.sh
|
||||
|
||||
docker compose --profile ttl-test stop api-short-ttl >/dev/null 2>&1
|
||||
echo
|
||||
echo "ALL STUDENT TESTS PASSED"
|
||||
Executable
+19
@@ -0,0 +1,19 @@
|
||||
#!/bin/sh
|
||||
# Graceful shutdown of the compose api service: SIGTERM -> exit 0 within 10s
|
||||
# (not SIGKILL 137), then the service is started again. Volumes are untouched.
|
||||
set -eu
|
||||
|
||||
cd "$(dirname "$0")/.."
|
||||
CID=$(docker compose ps -q api)
|
||||
[ -n "$CID" ] || { echo "FAIL: api is not running"; exit 1; }
|
||||
|
||||
START=$(date +%s)
|
||||
docker compose stop api >/dev/null 2>&1
|
||||
ELAPSED=$(( $(date +%s) - START ))
|
||||
EXIT_CODE=$(docker inspect -f '{{.State.ExitCode}}' "$CID")
|
||||
echo "api stopped in ${ELAPSED}s, exit code ${EXIT_CODE} (0 = graceful, 137 = SIGKILL)"
|
||||
docker compose logs --since 30s api 2>&1 | grep -q "Application shutdown complete" || { echo "FAIL: no graceful shutdown line"; exit 1; }
|
||||
[ "$ELAPSED" -le 10 ] && [ "$EXIT_CODE" -eq 0 ] || { docker compose up -d --wait api; echo "FAIL: not graceful"; exit 1; }
|
||||
|
||||
docker compose up -d --no-build --wait api >/dev/null 2>&1
|
||||
echo "PASS: graceful SIGTERM shutdown, api restarted"
|
||||
Executable
+47
@@ -0,0 +1,47 @@
|
||||
"""Shared helpers for student tests: public client + direct access to the compose stack."""
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT / "tests"))
|
||||
from client import Client, Failure, check # noqa: E402,F401
|
||||
|
||||
BASE_URL = os.environ.get("BASE_URL", "http://localhost:8080")
|
||||
SHORT_TTL_URL = os.environ.get("SHORT_TTL_URL", "http://localhost:8081")
|
||||
|
||||
PASSED = []
|
||||
|
||||
|
||||
def ok(label):
|
||||
PASSED.append(label)
|
||||
print("ok -", label)
|
||||
|
||||
|
||||
def compose(*args, check_rc=True):
|
||||
"""docker compose in the repository root; output is captured, never printed raw."""
|
||||
proc = subprocess.run(["docker", "compose", *args], cwd=ROOT, capture_output=True, text=True)
|
||||
if check_rc and proc.returncode != 0:
|
||||
raise Failure("docker compose %s failed: %s" % (args[0], proc.stderr.strip()[-300:]))
|
||||
return proc.stdout
|
||||
|
||||
|
||||
def psql(sql):
|
||||
"""Run SQL inside the postgres container (local socket, no password on the wire)."""
|
||||
return compose("exec", "-T", "postgres", "psql", "-U", "messenger", "-d", "messenger", "-tAX", "-c", sql).strip()
|
||||
|
||||
|
||||
def valkey(*args):
|
||||
"""valkey-cli with the password read inside the container, never on this host's argv."""
|
||||
cmd = 'valkey-cli --no-auth-warning -a "$(sed -n "s/^requirepass //p" /run/secrets/valkey_conf)" "$@"'
|
||||
return compose("exec", "-T", "valkey", "sh", "-c", cmd, "valkey-cli", *args).strip()
|
||||
|
||||
|
||||
def main_wrapper(main):
|
||||
try:
|
||||
main()
|
||||
except (AssertionError, Failure) as exc:
|
||||
print("FAIL:", exc)
|
||||
raise SystemExit(1)
|
||||
print("PASS: %d checks" % len(PASSED))
|
||||
Reference in New Issue
Block a user