ЛР 2 - БВТ2302 - Миронов - на 4 #65
@@ -0,0 +1,9 @@
|
||||
FROM python:3.12.11-slim-bookworm
|
||||
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1
|
||||
WORKDIR /app
|
||||
COPY requirements.txt .
|
||||
RUN pip install --no-cache-dir -r requirements.txt
|
||||
COPY src/ ./src/
|
||||
COPY migrations/ ./migrations/
|
||||
EXPOSE 8080
|
||||
CMD ["uvicorn", "src.main:app", "--host", "0.0.0.0", "--port", "8080", "--timeout-graceful-shutdown", "5"]
|
||||
@@ -0,0 +1,173 @@
|
||||
# Отчёт - лабораторная 2
|
||||
|
||||
- Автор, группа: Миронов Алексей, бвт2302
|
||||
- Целевая оценка: 4
|
||||
- Commit/PR: 4905353 lab2 condition inpostgres, sessions in redis / https://gitea.privcloud.tech/mtusi2026/mtusi2026lab2/pulls/65
|
||||
- Использованные источники, библиотеки и помощь ИИ: исходный учебный репозиторий; FastAPI, Pydantic, psycopg/psycopg-pool, PostgreSQL, Redis, Argon2; документация соответствующих библиотек; использовалась помощь ИИ
|
||||
|
||||
## Запуск из чистого clone
|
||||
|
||||
Требуются Docker Engine/Desktop с Docker Compose, Python 3.12+ и GNU Make. Команды ниже выполняются из корня репозитория.
|
||||
|
||||
1. Создать локальную конфигурацию и заменить демонстрационный пароль:
|
||||
```bash
|
||||
cp .env.example .env
|
||||
```
|
||||
Отредактировать `.env`, задав длинный случайный `POSTGRES_PASSWORD`. Не добавлять `.env` в Git.
|
||||
2. Выполнить офлайн-проверку структуры:
|
||||
```bash
|
||||
make check
|
||||
```
|
||||
3. Установить зависимости для валидации контракта и запустить её:
|
||||
```bash
|
||||
python -m pip install -r requirements-dev.txt
|
||||
make validate
|
||||
```
|
||||
4. Проверить конфигурацию Compose:
|
||||
```bash
|
||||
docker compose --env-file .env config --quiet
|
||||
```
|
||||
5. Собрать и запустить PostgreSQL, Redis и API:
|
||||
```bash
|
||||
docker compose --env-file .env up --build
|
||||
```
|
||||
Приложение применяет SQL-миграцию `migrations/001_init.sql` при инициализации. Дождаться готовности:
|
||||
```bash
|
||||
curl -i http://localhost:8080/health/ready
|
||||
```
|
||||
6. Запустить основной сценарий API:
|
||||
```bash
|
||||
make test
|
||||
```
|
||||
7. После подготовки исполняемого hook перезапуска проверить сохранность данных и сессии:
|
||||
```bash
|
||||
make test-persistence ACTION_SCRIPT=scripts/restart.sh
|
||||
```
|
||||
`scripts/restart.sh` перезапускает API, PostgreSQL и Redis, не удаляя именованные volumes. Запускать проверку только на локальном тестовом стенде.
|
||||
8. Дополнительные проверки уровня 4 и их ожидаемые результаты описаны в разделах «Доказательства критериев» и «Эксперименты и отказы».
|
||||
9. Безопасно остановить сервисы, сохранив данные:
|
||||
```bash
|
||||
docker compose --env-file .env down
|
||||
```
|
||||
Намеренное удаление учебных данных вместе с именованными volumes:
|
||||
```bash
|
||||
docker compose --env-file .env down -v
|
||||
```
|
||||
Последняя команда удаляет сохранённые данные PostgreSQL и Redis; выполнять её только при осознанном сбросе локального учебного стенда.
|
||||
|
||||
|
||||
|
||||
## Архитектура и решения
|
||||
|
||||
**Схема:** клиент / smoke- и нагрузочные тесты -> FastAPI -> PostgreSQL и Redis.
|
||||
|
||||
- **PostgreSQL - источник истины** для пользователей, чатов, участников чатов и сообщений. Данные хранятся в именованном volume `postgres_data`; схема создаётся SQL-миграцией `migrations/001_init.sql`.
|
||||
- **Redis** хранит серверные сессии. Ключ сессии строится как `session:<sha256(token)>`; значение содержит идентификатор пользователя и срок истечения. Исходный Bearer-токен не сохраняется. Redis настроен на AOF (`appendonly yes`, `appendfsync everysec`) и использует volume `redis_data`.
|
||||
- **Docker Compose** запускает PostgreSQL, Redis и API. API зависит от успешных healthcheck хранилищ. Наружу опубликован только API-порт `127.0.0.1:8080`; PostgreSQL и Redis не публикуют порты на хост.
|
||||
- **Пул PostgreSQL** ограничивается `DB_POOL_MIN_SIZE`, `DB_POOL_MAX_SIZE` и `DB_POOL_TIMEOUT_SECONDS`; значение по умолчанию для максимального размера - 10 соединений. В подключении заданы ограничения времени соединения/выполнения запросов и транзакций.
|
||||
- **Аутентификация:** пароль хранится в виде Argon2id-хеша. `Authorization: Basic` используется для входа на `POST /api/v1/auth/sessions`; после входа выдаётся случайный непрозрачный Bearer-токен. Защищённые запросы проверяют сессию в Redis. TTL настраивается через `SESSION_TTL_SECONDS`; logout удаляет запись сессии.
|
||||
- **Транзакции и целостность:** создание чата и добавление его участников выполняются в одной транзакции, чтобы ошибка не оставляла частично созданный чат. Пагинация использует стабильный составной порядок `(created_at, id)` и непрозрачный base64url cursor. API возвращает `X-Request-Id` и единый JSON-формат ошибок. Недоступность обязательного хранилища должна отражаться в readiness и ответах затронутых запросов.
|
||||
- **Конфигурация:** переменные `.env.example` соответствуют Compose-конфигурации: `APP_HOST`, `APP_PORT`, `SESSION_TTL_SECONDS`, параметры пула `DB_POOL_*`, `REDIS_URL` и параметры PostgreSQL `POSTGRES_DB`, `POSTGRES_USER`, `POSTGRES_PASSWORD`. Пароль должен быть заменён локально; секреты не включаются в отчёт или Git.
|
||||
- **Связь с lab1:** сохранены основные сущности и сценарии API - пользователи, чаты, участники, сообщения, пагинация, общий формат ошибок, `X-Request-Id` и health-endpoints. Ключевое изменение - перенос состояния из памяти процесса в PostgreSQL и введение серверных сессий вместо доверия к одному лишь `X-User-Id`.
|
||||
|
||||
## Доказательства критериев
|
||||
|
||||
Все команды выполнены на локальном стенде (Windows, Git Bash и WSL, Docker Desktop) на синтетических данных. Проверки `make check`/`make validate` и тесты, запускаемые через `python tests/...`, выполнялись из корня репозитория. Вывод сохранён в `evidence.txt`, план запроса - в `evidence/explain-level4.txt`, нагрузка - в `evidence/load-run.json`.
|
||||
|
||||
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
|
||||
| --- | --- | --- | --- |
|
||||
| Структура и синтаксис | `make check` | `PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.` Проверка офлайн, приложение ею не тестируется | evidence/evidence.txt |
|
||||
| OpenAPI и JSON Schema | `make validate` | `PASS: full OpenAPI 3.1 and JSON Schema validation; application NOT tested.` | evidence/evidence.txt |
|
||||
| Конфигурация Compose | `docker compose --env-file .env config --quiet` | Команда завершилась без вывода, сообщений об ошибках нет | evidence/evidence.txt |
|
||||
| Сборка и запуск | `docker compose --env-file .env up --build` | Образы `redis` и `postgres` загружены; образ `api` собран (12/12 шагов, 42.1 с, базовый образ `python:3.12.11-slim-bookworm`) | evidence/evidence.txt |
|
||||
| Живость и готовность | `curl -i http://localhost:8080/health/live`, `curl -i http://localhost:8080/health/ready` | Оба запроса: HTTP 200, тело `{"status":"ok"}`, заголовок `x-request-id` присутствует (16 Oct 2026, 16:32:06 и 16:33:35 GMT) | evidence/evidence.txt |
|
||||
| Состояние сервисов и порты | `docker compose --env-file .env ps` | `api`, `postgres` (`postgres:17.6-alpine`) и `redis` (`redis:7.4.2-alpine`) в статусе `healthy`. На хост опубликован только `127.0.0.1:8080->8080/tcp`; у PostgreSQL и Redis указаны лишь внутренние `5432/tcp` и `6379/tcp` | evidence/evidence.txt |
|
||||
| Публичный smoke | `make test` | `PASS: lab2 public API scenario; infrastructure and grading evidence remain separate.` | evidence/evidence.txt |
|
||||
| Персистентность при рестарте | `make test-persistence ACTION_SCRIPT=scripts/restart.sh` (hook выполняет `docker compose restart api postgres redis`) | `PASS: persistence preserved confirmed data and existing session.` Код возврата `0`. До теста все три сервиса `Up 22 minutes (healthy)`, после - `Up 13 seconds` (api) и `Up 14 seconds` (postgres, redis), все `healthy`; поле `CREATED` не изменилось (19-20 часов), то есть контейнеры перезапущены, а не пересозданы. Вывод hook тест скрывает намеренно, поэтому охват рестарта подтверждается этими `ps` | evidence/evidence.txt |
|
||||
| Redis AOF | `docker compose --env-file .env exec redis redis-cli CONFIG GET appendonly`, `... CONFIG GET appendfsync`, `... INFO persistence` | `appendonly` = `yes`, `appendfsync` = `everysec`. В `INFO persistence`: `aof_enabled:1`, `aof_last_write_status:ok`, `aof_last_bgrewrite_status:ok`, `aof_current_size:5969`, `aof_base_size:88`, `aof_rewrites:0`, `aof_rewrites_consecutive_failures:0`, `aof_delayed_fsync:0` | evidence/evidence.txt |
|
||||
| Истечение TTL сессии | `SESSION_TTL_SECONDS=3` в `.env`, `docker compose --env-file .env up -d --force-recreate api`, затем `python tests/session_ttl.py --base-url http://localhost:8080` | `PASS: expired Redis session is rejected with HTTP 401.` После теста TTL возвращён к 3600, API пересоздан тем же `up -d --force-recreate api` | evidence/evidence.txt |
|
||||
| Отказ Redis | `python tests/storage_failure.py --target redis --base-url http://localhost:8080` | `PASS: /health/ready returned HTTP 503 while redis was unavailable`; затем `PASS: readiness recovered to HTTP 200 after redis restart` | evidence/evidence.txt |
|
||||
| Отказ PostgreSQL | `python tests/storage_failure.py --target postgres --base-url http://localhost:8080` | `PASS: /health/ready returned HTTP 503 while postgres was unavailable`; затем `PASS: readiness recovered to HTTP 200 after postgres restart` | evidence/evidence.txt |
|
||||
| Конкурентная регистрация | `python tests/grade4_checks.py --base-url http://localhost:8080` | `PASS: concurrent same-username registration produced exactly one 201 and seven 409` (8 одновременных запросов с одним username) | evidence/evidence.txt, evidence/explain-level4.txt |
|
||||
| Откат транзакции | Та же команда `tests/grade4_checks.py` | `PASS: injected membership failure returned 503 and rolled back the chat row`. Ошибка внедряется временным триггером на `chat_members`; скрипт удаляет триггер и функцию в блоке `finally` | evidence/evidence.txt, evidence/explain-level4.txt |
|
||||
| План запроса для 10 000 сообщений | Та же команда `tests/grade4_checks.py` | `PASS: EXPLAIN (ANALYZE, BUFFERS) used messages_chat_created_id_idx; saved evidence\explain-level4.txt`. Подробный разбор плана ниже | evidence/evidence.txt, evidence/explain-level4.txt |
|
||||
| Нагрузочный прогон | `python tests/load.py --base-url http://localhost:8080 --duration 30 --concurrency 4 --output evidence/load-run.json` | 60 успешных запросов, 0 ошибок, 1.91 RPS, задержка успешных запросов p50 = 2079 мс, p95 = 2125 мс, p99 = 2140 мс. Один прогон; интерпретация ниже | evidence/evidence.txt, `evidence/load-run.json` |
|
||||
|
||||
**План запроса.** Набор данных: 10 000 синтетических сообщений в 100 чатах (по 100 сообщений в чате). Запрос: история одного чата в порядке `(created_at, id)`, `LIMIT 51`. Индекс `messages_chat_created_id_idx` построен по `messages(chat_id, created_at, id)`, то есть соответствует фильтру по чату и порядку выдачи. Фактический план:
|
||||
|
||||
- `Bitmap Index Scan` по `messages_chat_created_id_idx` с условием `chat_id = ...` вернул 100 строк (`shared hit=3`);
|
||||
- `Bitmap Heap Scan` прочитал 100 блоков, выбрав только строки этого чата;
|
||||
- `Sort` (`quicksort`, 38 kB) упорядочил эти 100 строк по `(created_at, id)`, затем `Limit` оставил 51;
|
||||
- суммарно `Buffers: shared hit=109`, `Planning Time: 0.974 ms`, `Execution Time: 0.696 ms`.
|
||||
|
||||
Из плана следует, что запрос не сканирует всю таблицу из 10 000 строк: индекс сужает выборку до сообщений одного чата. Ограничения вывода: планировщик выбрал bitmap-план с отдельной сортировкой 100 строк, а не `Index Scan`, отдающий порядок индекса; измерялась первая страница без keyset-условия `(created_at, id) > cursor`; чаты одинакового размера. Поведение на очень больших или неравномерных чатах и со следующими страницами этим прогоном не проверялось.
|
||||
|
||||
## Эксперименты и отказы
|
||||
|
||||
**Нагрузочный эксперимент.** Использован синтетический closed-loop сценарий `tests/load.py`: 4 параллельных исполнителя в течение 30 с отправляют `POST` сообщений в один чат от имени одного пользователя, без автоматических повторов. Фактическая длительность 31.406 с. Результат единственного прогона:
|
||||
|
||||
| Параметр | Значение |
|
||||
| --- | --- |
|
||||
| Успешных / неуспешных запросов | 60 / 0 (доля ошибок 0.0) |
|
||||
| Успешных запросов в секунду | 1.91 |
|
||||
| Задержка успешных запросов p50 / p95 / p99 | 2079 мс / 2125 мс / 2140 мс |
|
||||
|
||||
Задержка почти постоянна (2.08-2.14 с), поэтому при 4 исполнителях RPS примерно равен `4 / 2.08 ≈ 1.9`. Причина столь высокой задержки одного `POST` не исследовалась: не разделено время сервера, клиента и сетевого стека Docker Desktop на Windows, прогон не повторялся. Поэтому результат фиксируется как измерение одного локального профиля, а не как оценка пропускной способности системы.
|
||||
|
||||
OS: Майкрософт Windows 11 Pro 10.0.22631 (build 22631)
|
||||
CPU: Intel(R) Core(TM) i5-8350U CPU @ 1.70GHz, cores=4, threads=8
|
||||
RAM: 15.9 GB
|
||||
|
||||
Docker Compose version v2.32.4-desktop.1
|
||||
|
||||
docker version
|
||||
Client:
|
||||
Version: 27.5.1
|
||||
API version: 1.47
|
||||
Go version: go1.22.11
|
||||
Git commit: 9f9e405
|
||||
Built: Wed Jan 22 13:41:44 2025
|
||||
OS/Arch: windows/amd64
|
||||
Context: desktop-linux
|
||||
|
||||
Server: Docker Desktop 4.38.0 (181591)
|
||||
Engine:
|
||||
Version: 27.5.1
|
||||
API version: 1.47 (minimum version 1.24)
|
||||
Go version: go1.22.11
|
||||
Git commit: 4c9b3b0
|
||||
Built: Wed Jan 22 13:41:17 2025
|
||||
OS/Arch: linux/amd64
|
||||
Experimental: false
|
||||
containerd:
|
||||
Version: 1.7.25
|
||||
GitCommit: bcc810d6b9066471b0b6fa75f557a15a1cbf31bb
|
||||
runc:
|
||||
Version: 1.1.12
|
||||
GitCommit: v1.1.12-0-g51d5e946
|
||||
docker-init:
|
||||
Version: 0.19.0
|
||||
GitCommit: de40ad0
|
||||
|
||||
|
||||
**Контролируемые воздействия.** Время восстановления в тестах отказа не измеряется: `tests/storage_failure.py` ждёт HTTP 503 не дольше 10 с после остановки и восстановления HTTP 200 не дольше 45 с после запуска. Сообщения Docker `Stopped`/`Started` (0.3-0.4 с) относятся к самой команде, а не к готовности API.
|
||||
|
||||
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
|
||||
| --- | --- | --- | --- |
|
||||
| Истечение TTL (`SESSION_TTL_SECONDS=3`) | Сразу после входа токен работает, после TTL запрос с тем же токеном получает HTTP 401 | `tests/session_ttl.py`: PASS, просроченная сессия отклонена с HTTP 401 | Не относится к отказу сервиса; сессия истекает по TTL Redis без явного logout |
|
||||
| Остановка Redis | Readiness становится HTTP 503; после восстановления Redis возвращается HTTP 200 | `/health/ready` вернул 503 при остановленном Redis, после его запуска вернулся к 200 | Восстановление уложилось в таймаут теста 45 с, точное время не измерялось. Данные в этом тесте не создавались; сохранность сессий после рестарта проверена отдельным persistence-тестом |
|
||||
| Остановка PostgreSQL | Readiness становится HTTP 503; после восстановления БД возвращается HTTP 200 | `/health/ready` вернул 503 при остановленной БД, после её запуска вернулся к 200 | Восстановление уложилось в таймаут теста 45 с, точное время не измерялось. Данные в этом тесте не создавались; сохранность данных проверена persistence-тестом |
|
||||
| 8 параллельных регистраций одного username | Одна регистрация успешна (201), остальные получают конфликт (409) | Ровно один ответ 201 и семь 409 | Не относится к отказу сервиса; дубликаты не созданы, так как уникальность username обеспечивает БД |
|
||||
| Искусственная ошибка при добавлении участника в чат | Транзакция откатывается; частично созданный чат не остаётся | API вернул ограниченную ошибку 503, строка чата в БД отсутствует | Тестовые триггер и функция удаляются скриптом в `finally` |
|
||||
| Запрос истории на 10 000 сообщений | Получить и сохранить фактический `EXPLAIN (ANALYZE, BUFFERS)` | Использован индекс `messages_chat_created_id_idx`, время выполнения 0.696 мс; детали и ограничения - в разделе «Доказательства критериев» | Не является тестом восстановления; синтетические строки скрипт удаляет после проверки |
|
||||
| Перезапуск API, PostgreSQL и Redis через hook | Ранее созданные данные и действующая сессия остаются доступными, volumes сохраняются | `make test-persistence`: PASS, подтверждённые данные и существующая сессия сохранились; после перезапуска все сервисы `healthy` | Время восстановления отдельно не измерялось: через 13-14 с после перезапуска сервисы уже были `healthy`. Потери подтверждённых данных и действующей сессии не обнаружено |
|
||||
|
||||
|
||||
## Ограничения и следующие шаги
|
||||
|
||||
- **Нагрузочные измерения:** тест использует closed-loop модель с фиксированным числом параллельных исполнителей. Это не полноценный open-loop capacity-тест; результаты зависят от времени ответа сервера, клиентских накладных расходов и характеристик машины. Без повторов и фиксации окружения нельзя делать универсальные выводы по одному запуску.
|
||||
- **Персистентность Redis:** AOF с `appendfsync everysec` снижает риск потери данных, но при аварии хоста последние примерно секунду изменений теоретически можно потерять. AOF не заменяет резервное копирование и не обеспечивает высокую доступность.
|
||||
- **Отказы:** предусмотрены контролируемые остановки одного хранилища и последующее восстановление. Автоматический failover, кластеризация PostgreSQL/Redis и восстановление из резервной копии не реализованы в текущей версии.
|
||||
- **Масштабирование и безопасность:** rate limiting и горизонтальное масштабирование не реализованы. Порт API привязан к localhost в Compose; стенд предназначен для учебных синтетических данных.
|
||||
|
||||
|
||||
@@ -0,0 +1,61 @@
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:17.6-alpine
|
||||
environment:
|
||||
POSTGRES_DB: ${POSTGRES_DB:-messenger}
|
||||
POSTGRES_USER: ${POSTGRES_USER:-messenger}
|
||||
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?Set POSTGRES_PASSWORD in .env}
|
||||
volumes:
|
||||
- postgres_data:/var/lib/postgresql/data
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-messenger} -d ${POSTGRES_DB:-messenger}"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 20
|
||||
restart: unless-stopped
|
||||
redis:
|
||||
image: redis:7.4.2-alpine
|
||||
command: ["redis-server", "--appendonly", "yes", "--appendfsync", "everysec"]
|
||||
volumes:
|
||||
- redis_data:/data
|
||||
healthcheck:
|
||||
test: ["CMD", "redis-cli", "ping"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 20
|
||||
restart: unless-stopped
|
||||
api:
|
||||
build: .
|
||||
environment:
|
||||
APP_HOST: ${APP_HOST:-0.0.0.0}
|
||||
APP_PORT: ${APP_PORT:-8080}
|
||||
SESSION_TTL_SECONDS: ${SESSION_TTL_SECONDS:-3600}
|
||||
DB_POOL_MIN_SIZE: ${DB_POOL_MIN_SIZE:-1}
|
||||
DB_POOL_MAX_SIZE: ${DB_POOL_MAX_SIZE:-10}
|
||||
DB_POOL_TIMEOUT_SECONDS: ${DB_POOL_TIMEOUT_SECONDS:-5}
|
||||
# Use libpq variables rather than interpolating the password into a URL;
|
||||
# this supports long passwords containing URL-reserved characters.
|
||||
DATABASE_URL: ${DATABASE_URL:-}
|
||||
PGHOST: postgres
|
||||
PGPORT: "5432"
|
||||
PGDATABASE: ${POSTGRES_DB:-messenger}
|
||||
PGUSER: ${POSTGRES_USER:-messenger}
|
||||
PGPASSWORD: ${POSTGRES_PASSWORD:?Set POSTGRES_PASSWORD in .env}
|
||||
REDIS_URL: ${REDIS_URL:-redis://redis:6379/0}
|
||||
ports:
|
||||
- "127.0.0.1:${APP_PORT:-8080}:8080"
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
redis:
|
||||
condition: service_healthy
|
||||
healthcheck:
|
||||
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/health/ready', timeout=2)"]
|
||||
interval: 10s
|
||||
timeout: 3s
|
||||
retries: 5
|
||||
start_period: 10s
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
postgres_data:
|
||||
redis_data:
|
||||
|
After Width: | Height: | Size: 24 KiB |
|
After Width: | Height: | Size: 25 KiB |
|
After Width: | Height: | Size: 32 KiB |
|
After Width: | Height: | Size: 25 KiB |
|
After Width: | Height: | Size: 39 KiB |
|
After Width: | Height: | Size: 99 KiB |
|
After Width: | Height: | Size: 90 KiB |
|
After Width: | Height: | Size: 72 KiB |
|
After Width: | Height: | Size: 77 KiB |
|
After Width: | Height: | Size: 74 KiB |
|
After Width: | Height: | Size: 59 KiB |
|
After Width: | Height: | Size: 61 KiB |
|
After Width: | Height: | Size: 58 KiB |
|
After Width: | Height: | Size: 79 KiB |
@@ -0,0 +1,233 @@
|
||||
Базовая проверка структуры
|
||||
lab2/mtusi2026lab2-fork (solution/bvt2302-mironov)
|
||||
$ make check
|
||||
python scripts/check_contract.py
|
||||
PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.
|
||||
|
||||
lab2/mtusi2026lab2-fork (solution/bvt2302-mironov)
|
||||
$ make validate
|
||||
python scripts/validate_openapi.py
|
||||
PASS: full OpenAPI 3.1 and JSON Schema validation; application NOT tested.
|
||||
|
||||
Проверка конфигурации докер компоуз
|
||||
|
||||
lab2/mtusi2026lab2-fork (solution/bvt2302-mironov)
|
||||
$ docker compose --env-file .env config --quiet
|
||||
|
||||
User@DESKTOP-8Q26HPT MINGW64 ~/Desktop/MTUCI/4_course/highload_apps/lab2/mtusi2026lab2-fork (solution/bvt2302-mironov)
|
||||
|
||||
Запуск через докер компоуз наших сервисов с созданием образов через билд
|
||||
|
||||
User@DESKTOP-8Q26HPT MINGW64 ~/Desktop/MTUCI/4_course/highload_apps/lab2/mtusi2026lab2-fork (solution/bvt2302-mironov)
|
||||
$ docker compose --env-file .env up --build
|
||||
[+] Running 19/19
|
||||
✔ redis Pulled 19.1s
|
||||
✔ postgres Pulled 31.7s
|
||||
|
||||
[+] Building 42.1s (12/12) FINISHED docker:desktop-linux
|
||||
=> [api internal] load build definition from Dockerfile 0.1s
|
||||
=> => transferring dockerfile: 377B 0.0s
|
||||
=> [api internal] load metadata for docker.io/library/python:3.12.11-slim-bookworm 5.2s
|
||||
=> [api internal] load .dockerignore 0.0s
|
||||
=> => transferring context: 216B
|
||||
|
||||
|
||||
Проверка живости и готовности нашего сервиса:
|
||||
|
||||
mtusi2026lab2-fork (solution/bvt2302-mironov)
|
||||
$ curl -i http://localhost:8080/health/live
|
||||
HTTP/1.1 200 OK
|
||||
date: Fri, 16 Oct 2026 16:32:06 GMT
|
||||
server: uvicorn
|
||||
content-length: 15
|
||||
content-type: application/json
|
||||
x-request-id: 714ef0b982804520b901a1c998fdd3e9
|
||||
|
||||
{"status":"ok"}
|
||||
User@DESKTOP-8Q26HPT MINGW64 ~/Desktop/MTUCI/4_course/highload_apps/lab2/mtusi2026lab2-fork (solution/bvt2302-mironov)
|
||||
$ curl -i http://localhost:8080/health/ready
|
||||
HTTP/1.1 200 OK
|
||||
date: Fri, 16 Oct 2026 16:33:35 GMT
|
||||
server: uvicorn
|
||||
content-length: 15
|
||||
content-type: application/json
|
||||
x-request-id: 9e1c6a2baea14f51bb1d3689618f7aba
|
||||
|
||||
{"status":"ok"}
|
||||
|
||||
Прохождение публичных тестов:
|
||||
lab2/mtusi2026lab2-fork (solution/bvt2302-mironov)
|
||||
$ make test
|
||||
python tests/smoke.py --base-url "http://localhost:8080"
|
||||
PASS: lab2 public API scenario; infrastructure and grading evidence remain separate.
|
||||
|
||||
Прохождение тестов на персистентность системы с сохранением данных в постгрес и редис после рестарта
|
||||
(с демонстрацией запущенных докер-контейнеров до и после)
|
||||
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab2/mtusi2026lab2-fork$ docker compose --env-file .env ps
|
||||
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
|
||||
mtusi2026lab2-fork-api-1 mtusi2026lab2-fork-api "uvicorn src.main:ap…" api 19 hours ago Up 22 minutes (healthy) 127.0.0.1:8080->8080/tcp
|
||||
mtusi2026lab2-fork-postgres-1 postgres:17.6-alpine "docker-entrypoint.s…" postgres 20 hours ago Up 22 minutes (healthy) 5432/tcp
|
||||
mtusi2026lab2-fork-redis-1 redis:7.4.2-alpine "docker-entrypoint.s…" redis 20 hours ago Up 22 minutes (healthy) 6379/tcp
|
||||
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab2/mtusi2026lab2-fork$ make test-persistence ACTION_SCRIPT=scripts/restart.sh
|
||||
python3 tests/resilience.py --mode persistence --action-script "scripts/restart.sh" --base-url "http://localhost:8080"
|
||||
Executing explicit persistence hook; hook output suppressed to avoid leaking secrets.
|
||||
PASS: persistence preserved confirmed data and existing session. Hook scope needs operator evidence.
|
||||
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab2/mtusi2026lab2-fork$ echo $?
|
||||
0
|
||||
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab2/mtusi2026lab2-fork$ docker compose --env-file .env ps
|
||||
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
|
||||
mtusi2026lab2-fork-api-1 mtusi2026lab2-fork-api "uvicorn src.main:ap…" api 19 hours ago Up 13 seconds (healthy) 127.0.0.1:8080->8080/tcp
|
||||
mtusi2026lab2-fork-postgres-1 postgres:17.6-alpine "docker-entrypoint.s…" postgres 20 hours ago Up 14 seconds (healthy) 5432/tcp
|
||||
mtusi2026lab2-fork-redis-1 redis:7.4.2-alpine "docker-entrypoint.s…" redis 20 hours ago Up 14 seconds (healthy) 6379/tcp
|
||||
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab2/mtusi2026lab2-fork$
|
||||
|
||||
|
||||
Проверка электронного журналирования в redis AOF: корректной его работы и настройки
|
||||
|
||||
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab2/mtusi2026lab2-fork$ docker compose --env-file .env exec redis redis-cli CONFIG GET appendonly
|
||||
1) "appendonly"
|
||||
2) "yes"
|
||||
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab2/mtusi2026lab2-fork$ docker compose --env-file .env exec redis redis-cli INFO persistence
|
||||
# Persistence
|
||||
loading:0
|
||||
async_loading:0
|
||||
current_cow_peak:0
|
||||
current_cow_size:0
|
||||
current_cow_size_age:0
|
||||
current_fork_perc:0.00
|
||||
current_save_keys_processed:0
|
||||
current_save_keys_total:0
|
||||
rdb_changes_since_last_save:33
|
||||
rdb_bgsave_in_progress:0
|
||||
rdb_last_save_time:1792238782
|
||||
rdb_last_bgsave_status:ok
|
||||
rdb_last_bgsave_time_sec:-1
|
||||
rdb_current_bgsave_time_sec:-1
|
||||
rdb_saves:0
|
||||
rdb_last_cow_size:0
|
||||
rdb_last_load_keys_expired:0
|
||||
rdb_last_load_keys_loaded:0
|
||||
aof_enabled:1
|
||||
aof_rewrite_in_progress:0
|
||||
aof_rewrite_scheduled:0
|
||||
aof_last_rewrite_time_sec:-1
|
||||
aof_current_rewrite_time_sec:-1
|
||||
aof_last_bgrewrite_status:ok
|
||||
aof_rewrites:0
|
||||
aof_rewrites_consecutive_failures:0
|
||||
aof_last_write_status:ok
|
||||
aof_last_cow_size:0
|
||||
module_fork_in_progress:0
|
||||
module_fork_last_cow_size:0
|
||||
aof_current_size:5969
|
||||
aof_base_size:88
|
||||
aof_pending_rewrite:0
|
||||
aof_buffer_length:0
|
||||
aof_pending_bio_fsync:0
|
||||
aof_delayed_fsync:0
|
||||
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab2/mtusi2026lab2-fork$
|
||||
|
||||
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab2/mtusi2026lab2-fork$ docker compose --env-file .env exec redis redis-cli CONFIG GET appendfsync
|
||||
1) "appendfsync"
|
||||
2) "everysec"
|
||||
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab2/mtusi2026lab2-fork$
|
||||
|
||||
|
||||
Прохождение проверки корректного поведения при истечении ttl сессии в редис
|
||||
|
||||
Для начала изменим ttl на 3 в .env и пересоздадим контейнер апи
|
||||
User@DESKTOP-8Q26HPT MINGW64 ~/Desktop/MTUCI/4_course/highload_apps/lab2/mtusi2026lab2-fork (solution/bvt2302-mironov)
|
||||
$ docker compose --env-file .env up -d --force-recreate api
|
||||
[+] Running 3/3
|
||||
✔ Container mtusi2026lab2-fork-redis-1 He... 1.7s
|
||||
✔ Container mtusi2026lab2-fork-postgres-1 Healthy 1.7s
|
||||
✔ Container mtusi2026lab2-fork-api-1 Star... 2.0s
|
||||
|
||||
User@DESKTOP-8Q26HPT MINGW64 ~/Desktop/MTUCI/4_course/highload_apps/lab2/mtusi2026lab2-fork (solution/bvt2302-mironov)
|
||||
$ python tests/session_ttl.py --base-url http://localhost:8080
|
||||
PASS: expired Redis session is rejected with HTTP 401
|
||||
|
||||
User@DESKTOP-8Q26HPT MINGW64 ~/Desktop/MTUCI/4_course/highload_apps/lab2/mtusi2026lab2-fork (solution/bvt2302-mironov)
|
||||
$
|
||||
Затем после проверки изменим ttl обратно на 3600 (сек)
|
||||
User@DESKTOP-8Q26HPT MINGW64 ~/Desktop/MTUCI/4_course/highload_apps/lab2/mtusi2026lab2-fork (solution/bvt2302-mironov)
|
||||
$ docker compose --env-file .env up -d --force-recreate api
|
||||
[+] Running 3/3
|
||||
✔ Container mtusi2026lab2-fork-redis-1 He... 1.5s
|
||||
✔ Container mtusi2026lab2-fork-postgres-1 Healthy 1.5s
|
||||
✔ Container mtusi2026lab2-fork-api-1 Star... 1.9s
|
||||
|
||||
User@DESKTOP-8Q26HPT MINGW64 ~/Desktop/MTUCI/4_course/highload_apps/lab2/mtusi2026lab2-fork (solution/bvt2302-mironov)
|
||||
$
|
||||
|
||||
Проверка возвращения статуса 503 при проверки доступности, если редис не работает
|
||||
|
||||
User@DESKTOP-8Q26HPT MINGW64 ~/Desktop/MTUCI/4_course/highload_apps/lab2/mtusi2026lab2-fork (solution/bvt2302-mironov)
|
||||
$ python tests/storage_failure.py --target redis --base-url http://localhost:8080
|
||||
[+] Stopping 1/1
|
||||
✔ Container mtusi2026lab2-fork-redis-1 Stopp... 0.3s
|
||||
PASS: /health/ready returned HTTP 503 while redis was unavailable
|
||||
[+] Running 1/1
|
||||
✔ Container mtusi2026lab2-fork-redis-1 Start... 0.4s
|
||||
PASS: readiness recovered to HTTP 200 after redis restart
|
||||
|
||||
User@DESKTOP-8Q26HPT MINGW64 ~/Desktop/MTUCI/4_course/highload_apps/lab2/mtusi2026lab2-fork (solution/bvt2302-mironov)
|
||||
|
||||
Проверка возвращения статуса 503 при проверки доступности, если постгрес не работает
|
||||
|
||||
User@DESKTOP-8Q26HPT MINGW64 ~/Desktop/MTUCI/4_course/highload_apps/lab2/mtusi2026lab2-fork (solution/bvt2302-mironov)
|
||||
$ python tests/storage_failure.py --target postgres --base-url http://localhost:8080
|
||||
[+] Stopping 1/1
|
||||
✔ Container mtusi2026lab2-fork-postgres-1 Stopped 0.3s
|
||||
PASS: /health/ready returned HTTP 503 while postgres was unavailable
|
||||
[+] Running 1/1
|
||||
✔ Container mtusi2026lab2-fork-postgres-1 Started 0.3s
|
||||
PASS: readiness recovered to HTTP 200 after postgres restart
|
||||
|
||||
User@DESKTOP-8Q26HPT MINGW64 ~/Desktop/MTUCI/4_course/highload_apps/lab2/mtusi2026lab2-fork (solution/bvt2302-mironov)
|
||||
|
||||
Проверки для 4го уровня:
|
||||
User@DESKTOP-8Q26HPT MINGW64 ~/Desktop/MTUCI/4_course/highload_apps/lab2/mtusi2026lab2-fork (solution/bvt2302-mironov)
|
||||
$ python tests/grade4_checks.py --base-url http://localhost:8080
|
||||
PASS: concurrent same-username registration produced exactly one 201 and seven 409
|
||||
PASS: injected membership failure returned 503 and rolled back the chat row
|
||||
PASS: EXPLAIN (ANALYZE, BUFFERS) used messages_chat_created_id_idx; saved evidence\explain-level4.txt
|
||||
PASS: all automated level-4 checks in this script completed
|
||||
|
||||
User@DESKTOP-8Q26HPT MINGW64 ~/Desktop/MTUCI/4_course/highload_apps/lab2/mtusi2026lab2-fork (solution/bvt2302-mironov)
|
||||
|
||||
|
||||
Нагрузочное тестирование:
|
||||
|
||||
User@DESKTOP-8Q26HPT MINGW64 ~/Desktop/MTUCI/4_course/highload_apps/lab2/mtusi2026lab2-fork (solution/bvt2302-mironov)
|
||||
$ python tests/load.py --base-url http://localhost:8080 --duration 30 --concurrency 4 --output evidence/load-run.json
|
||||
{
|
||||
"lab": 2,
|
||||
"workload": "closed-loop POST messages; no automatic retries",
|
||||
"requested_duration_seconds": 30.0,
|
||||
"elapsed_seconds": 31.406,
|
||||
"concurrency": 4,
|
||||
"successful_requests": 60,
|
||||
"failed_requests": 0,
|
||||
"error_ratio": 0.0,
|
||||
"successful_rps": 1.91,
|
||||
"successful_latency_ms": {
|
||||
"p50": 2079.0,
|
||||
"p95": 2125.0,
|
||||
"p99": 2140.0
|
||||
},
|
||||
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
|
||||
}
|
||||
|
||||
User@DESKTOP-8Q26HPT MINGW64 ~/Desktop/MTUCI/4_course/highload_apps/lab2/mtusi2026lab2-fork (solution/bvt2302-mironov)
|
||||
|
||||
Завершение работы приложения и удаление контейнеров
|
||||
User@DESKTOP-8Q26HPT MINGW64 ~/Desktop/MTUCI/4_course/highload_apps/lab2/mtusi2026lab2-fork (solution/bvt2302-mironov)
|
||||
$ docker compose --env-file .env down
|
||||
[+] Running 4/4
|
||||
✔ Container mtusi2026lab2-fork-api-1 Remo... 3.0s
|
||||
✔ Container mtusi2026lab2-fork-redis-1 Re... 0.6s
|
||||
✔ Container mtusi2026lab2-fork-postgres-1 Removed 0.7s
|
||||
✔ Network mtusi2026lab2-fork_default Remo... 0.4s
|
||||
|
||||
User@DESKTOP-8Q26HPT MINGW64 ~/Desktop/MTUCI/4_course/highload_apps/lab2/mtusi2026lab2-fork (solution/bvt2302-mironov)
|
||||
@@ -0,0 +1,20 @@
|
||||
Dataset: at least 10,000 synthetic messages across 100 chats.
|
||||
Query: history ordered by (created_at, id), LIMIT 51.
|
||||
|
||||
Limit (cost=143.99..144.12 rows=51 width=79) (actual time=0.561..0.567 rows=51 loops=1)
|
||||
Buffers: shared hit=109
|
||||
-> Sort (cost=143.99..144.24 rows=100 width=79) (actual time=0.560..0.563 rows=51 loops=1)
|
||||
Sort Key: created_at, id
|
||||
Sort Method: quicksort Memory: 38kB
|
||||
Buffers: shared hit=109
|
||||
-> Bitmap Heap Scan on messages (cost=9.06..140.67 rows=100 width=79) (actual time=0.150..0.335 rows=100 loops=1)
|
||||
Recheck Cond: (chat_id = 'fb9f060e-91d3-4ee8-a3f3-adf6ffe42ba0'::uuid)
|
||||
Heap Blocks: exact=100
|
||||
Buffers: shared hit=103
|
||||
-> Bitmap Index Scan on messages_chat_created_id_idx (cost=0.00..9.04 rows=100 width=0) (actual time=0.128..0.129 rows=100 loops=1)
|
||||
Index Cond: (chat_id = 'fb9f060e-91d3-4ee8-a3f3-adf6ffe42ba0'::uuid)
|
||||
Buffers: shared hit=3
|
||||
Planning:
|
||||
Buffers: shared hit=107
|
||||
Planning Time: 0.974 ms
|
||||
Execution Time: 0.696 ms
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"lab": 2,
|
||||
"workload": "closed-loop POST messages; no automatic retries",
|
||||
"requested_duration_seconds": 30.0,
|
||||
"elapsed_seconds": 31.406,
|
||||
"concurrency": 4,
|
||||
"successful_requests": 60,
|
||||
"failed_requests": 0,
|
||||
"error_ratio": 0.0,
|
||||
"successful_rps": 1.91,
|
||||
"successful_latency_ms": {
|
||||
"p50": 2079.0,
|
||||
"p95": 2125.0,
|
||||
"p99": 2140.0
|
||||
},
|
||||
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
CREATE TABLE IF NOT EXISTS users (
|
||||
id UUID PRIMARY KEY,
|
||||
username VARCHAR(32) NOT NULL UNIQUE,
|
||||
display_name VARCHAR(100) NOT NULL,
|
||||
password_hash TEXT NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
CONSTRAINT users_username_format CHECK (username ~ '^[a-z0-9_]{3,32}$'),
|
||||
CONSTRAINT users_display_name_nonblank CHECK (length(btrim(display_name)) > 0)
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS chats (
|
||||
id UUID PRIMARY KEY,
|
||||
title VARCHAR(100) NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
CONSTRAINT chats_title_nonblank CHECK (length(btrim(title)) > 0)
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS chat_members (
|
||||
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
|
||||
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
position INTEGER NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (chat_id, user_id)
|
||||
);
|
||||
ALTER TABLE chat_members ADD COLUMN IF NOT EXISTS position INTEGER NOT NULL DEFAULT 0;
|
||||
CREATE INDEX IF NOT EXISTS chat_members_user_chat_idx ON chat_members(user_id, chat_id);
|
||||
CREATE TABLE IF NOT EXISTS messages (
|
||||
id UUID PRIMARY KEY,
|
||||
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
|
||||
sender_id UUID NOT NULL REFERENCES users(id),
|
||||
text VARCHAR(4000) NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
CONSTRAINT messages_text_nonblank CHECK (length(btrim(text)) > 0)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS messages_chat_created_id_idx ON messages(chat_id, created_at, id);
|
||||
@@ -0,0 +1,5 @@
|
||||
fastapi==0.116.1
|
||||
uvicorn[standard]==0.35.0
|
||||
psycopg[pool,binary]==3.2.9
|
||||
argon2-cffi==25.1.0
|
||||
redis==6.4.0
|
||||
@@ -0,0 +1,4 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
# Restart all stateful services and API while preserving PostgreSQL and Redis named volumes.
|
||||
docker compose --env-file .env restart api postgres redis
|
||||
@@ -0,0 +1,446 @@
|
||||
"""Lab 2 REST API: FastAPI + PostgreSQL persistence + Redis + opaque Bearer sessions."""
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import hashlib
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import secrets
|
||||
import uuid
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import psycopg
|
||||
from psycopg_pool import ConnectionPool, PoolTimeout
|
||||
import redis
|
||||
from redis.exceptions import RedisError
|
||||
from argon2 import PasswordHasher
|
||||
from argon2.exceptions import InvalidHashError, VerificationError, VerifyMismatchError
|
||||
from fastapi import FastAPI, Request
|
||||
from fastapi.exceptions import RequestValidationError
|
||||
from fastapi.responses import JSONResponse, Response
|
||||
from pydantic import BaseModel, ConfigDict, Field, ValidationError
|
||||
from starlette.exceptions import HTTPException as StarletteHTTPException
|
||||
|
||||
logging.basicConfig(level=os.getenv("LOG_LEVEL", "INFO"))
|
||||
log = logging.getLogger("messenger")
|
||||
DATABASE_URL = os.getenv("DATABASE_URL", "postgresql://messenger:messenger@localhost:5432/messenger")
|
||||
REDIS_URL = os.getenv("REDIS_URL", "redis://localhost:6379/0")
|
||||
SESSION_TTL_SECONDS = int(os.getenv("SESSION_TTL_SECONDS", "3600"))
|
||||
if SESSION_TTL_SECONDS < 1:
|
||||
raise RuntimeError("SESSION_TTL_SECONDS must be positive")
|
||||
# Redis uses its connection pool; session keys contain only a hash of the opaque token.
|
||||
session_store = redis.Redis.from_url(REDIS_URL, decode_responses=True, socket_connect_timeout=3, socket_timeout=3, health_check_interval=30)
|
||||
DB_POOL_MIN_SIZE = int(os.getenv("DB_POOL_MIN_SIZE", "1"))
|
||||
DB_POOL_MAX_SIZE = int(os.getenv("DB_POOL_MAX_SIZE", "10"))
|
||||
DB_POOL_TIMEOUT_SECONDS = float(os.getenv("DB_POOL_TIMEOUT_SECONDS", "5"))
|
||||
if DB_POOL_MIN_SIZE < 1 or DB_POOL_MAX_SIZE < DB_POOL_MIN_SIZE:
|
||||
raise RuntimeError("Invalid PostgreSQL pool size configuration")
|
||||
# Every pooled connection gets bounded connect/query/idle-transaction timeouts.
|
||||
db_pool = ConnectionPool(
|
||||
conninfo=DATABASE_URL,
|
||||
min_size=DB_POOL_MIN_SIZE,
|
||||
max_size=DB_POOL_MAX_SIZE,
|
||||
timeout=DB_POOL_TIMEOUT_SECONDS,
|
||||
kwargs={
|
||||
"connect_timeout": 5,
|
||||
"options": "-c statement_timeout=5000 -c idle_in_transaction_session_timeout=10000",
|
||||
},
|
||||
open=False,
|
||||
name="messenger-postgres",
|
||||
)
|
||||
USERNAME_RE = re.compile(r"^[a-z0-9_]{3,32}$")
|
||||
PASSWORDS = PasswordHasher()
|
||||
|
||||
class ApiError(Exception):
|
||||
def __init__(self, status: int, code: str, message: str, headers: dict[str, str] | None = None):
|
||||
self.status, self.code, self.message, self.headers = status, code, message, headers or {}
|
||||
|
||||
def fail(status: int, code: str, message: str, headers: dict[str, str] | None = None):
|
||||
raise ApiError(status, code, message, headers)
|
||||
|
||||
class StrictModel(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid", strict=True)
|
||||
|
||||
class RegisterBody(StrictModel):
|
||||
username: str
|
||||
display_name: str
|
||||
password: str
|
||||
|
||||
class ChatBody(StrictModel):
|
||||
title: str
|
||||
member_ids: list[str] = Field(min_length=1, max_length=100)
|
||||
|
||||
class MessageBody(StrictModel):
|
||||
text: str
|
||||
|
||||
app = FastAPI(docs_url=None, redoc_url=None, openapi_url=None)
|
||||
|
||||
@app.middleware("http")
|
||||
async def request_id_middleware(request: Request, call_next):
|
||||
request.state.request_id = uuid.uuid4().hex
|
||||
try:
|
||||
response = await call_next(request)
|
||||
except Exception:
|
||||
log.exception("Unhandled API exception request_id=%s", request.state.request_id)
|
||||
response = JSONResponse(status_code=500, content={"error": {"code": "internal_error", "message": "Internal server error", "request_id": request.state.request_id}})
|
||||
response.headers["X-Request-Id"] = request.state.request_id
|
||||
return response
|
||||
|
||||
def error_response(status: int, code: str, message: str, request_id: str, headers: dict[str, str] | None = None):
|
||||
return JSONResponse(status_code=status, content={"error": {"code": code, "message": message, "request_id": request_id}}, headers=headers or {})
|
||||
|
||||
@app.exception_handler(ApiError)
|
||||
async def api_error_handler(request: Request, exc: ApiError):
|
||||
return error_response(exc.status, exc.code, exc.message, request.state.request_id, exc.headers)
|
||||
|
||||
@app.exception_handler(PoolTimeout)
|
||||
async def database_pool_timeout_handler(request: Request, exc: PoolTimeout):
|
||||
log.warning("Database connection pool timed out request_id=%s", request.state.request_id)
|
||||
return error_response(503, "unavailable", "Required storage is unavailable", request.state.request_id)
|
||||
|
||||
@app.exception_handler(psycopg.Error)
|
||||
async def database_error_handler(request: Request, exc: psycopg.Error):
|
||||
log.warning("Database operation failed request_id=%s", request.state.request_id)
|
||||
return error_response(503, "unavailable", "Required storage is unavailable", request.state.request_id)
|
||||
|
||||
@app.exception_handler(RedisError)
|
||||
async def redis_error_handler(request: Request, exc: RedisError):
|
||||
log.warning("Session store operation failed request_id=%s", request.state.request_id)
|
||||
return error_response(503, "unavailable", "Required storage is unavailable", request.state.request_id)
|
||||
|
||||
@app.exception_handler(RequestValidationError)
|
||||
async def validation_error_handler(request: Request, exc: RequestValidationError):
|
||||
return error_response(400, "invalid_request", "Invalid request", request.state.request_id)
|
||||
|
||||
@app.exception_handler(StarletteHTTPException)
|
||||
async def http_error_handler(request: Request, exc: StarletteHTTPException):
|
||||
status = exc.status_code
|
||||
code = "not_found" if status == 404 else "invalid_request"
|
||||
message = "Object not found" if status == 404 else "Invalid request"
|
||||
headers = dict(exc.headers or {})
|
||||
return error_response(status, code, message, request.state.request_id, headers)
|
||||
|
||||
def db():
|
||||
"""Borrow a bounded pooled connection; context exit commits/rolls back and returns it."""
|
||||
return db_pool.connection(timeout=DB_POOL_TIMEOUT_SECONDS)
|
||||
|
||||
def initialize_database():
|
||||
migration = Path(__file__).resolve().parents[1] / "migrations" / "001_init.sql"
|
||||
sql = migration.read_text(encoding="utf-8")
|
||||
with db() as conn:
|
||||
with conn.cursor() as cur:
|
||||
for statement in sql.split(";"):
|
||||
if statement.strip():
|
||||
cur.execute(statement)
|
||||
|
||||
@app.on_event("startup")
|
||||
def startup():
|
||||
# Open the bounded pool before schema initialization and wait for both stores.
|
||||
import time
|
||||
db_pool.open(wait=False)
|
||||
last_error = None
|
||||
for _ in range(30):
|
||||
try:
|
||||
initialize_database()
|
||||
session_store.ping()
|
||||
log.info("Database schema and Redis session store are ready")
|
||||
return
|
||||
except Exception as exc:
|
||||
last_error = exc
|
||||
time.sleep(1)
|
||||
raise RuntimeError("PostgreSQL or Redis unavailable during startup") from last_error
|
||||
|
||||
@app.on_event("shutdown")
|
||||
def shutdown():
|
||||
session_store.close()
|
||||
db_pool.close()
|
||||
|
||||
def now_utc() -> datetime:
|
||||
return datetime.now(timezone.utc)
|
||||
|
||||
def iso(value: datetime) -> str:
|
||||
return value.astimezone(timezone.utc).isoformat(timespec="microseconds").replace("+00:00", "Z")
|
||||
|
||||
def user_dict(row: dict[str, Any]) -> dict[str, Any]:
|
||||
return {"id": str(row["id"]), "username": row["username"], "display_name": row["display_name"], "created_at": iso(row["created_at"])}
|
||||
|
||||
def chat_dict(row: dict[str, Any], member_ids: list[str]) -> dict[str, Any]:
|
||||
return {"id": str(row["id"]), "title": row["title"], "member_ids": member_ids, "created_at": iso(row["created_at"])}
|
||||
|
||||
def message_dict(row: dict[str, Any]) -> dict[str, Any]:
|
||||
return {"id": str(row["id"]), "chat_id": str(row["chat_id"]), "sender_id": str(row["sender_id"]), "text": row["text"], "created_at": iso(row["created_at"])}
|
||||
|
||||
def validate_unicode(value: str, label: str) -> str:
|
||||
"""Reject unpaired UTF-16 surrogates before sending text to PostgreSQL/KDF."""
|
||||
try:
|
||||
value.encode("utf-8")
|
||||
except UnicodeEncodeError:
|
||||
fail(400, "invalid_request", "Invalid Unicode in " + label)
|
||||
return value
|
||||
|
||||
|
||||
def parse_uuid(value: str, label: str = "UUID") -> uuid.UUID:
|
||||
try:
|
||||
# Keep the canonical UUID representation used by the Lab 1 API.
|
||||
if not isinstance(value, str) or len(value) != 36:
|
||||
raise ValueError("non-canonical UUID")
|
||||
parsed = uuid.UUID(value)
|
||||
if str(parsed) != value:
|
||||
raise ValueError("non-canonical UUID")
|
||||
return parsed
|
||||
except (ValueError, TypeError, AttributeError):
|
||||
fail(400, "invalid_request", "Invalid " + label)
|
||||
|
||||
def validate_body_type(request: Request):
|
||||
content_type = request.headers.get("content-type", "").split(";", 1)[0].strip().lower()
|
||||
if content_type != "application/json":
|
||||
fail(415, "unsupported_media_type", "Content-Type must be application/json")
|
||||
|
||||
async def parse_body(request: Request, model):
|
||||
validate_body_type(request)
|
||||
try:
|
||||
raw = await request.body()
|
||||
data = json.loads(raw.decode("utf-8"))
|
||||
return model.model_validate(data)
|
||||
except (UnicodeDecodeError, json.JSONDecodeError, ValidationError, TypeError):
|
||||
fail(400, "invalid_request", "Invalid JSON request body")
|
||||
|
||||
def challenge(scheme: str) -> dict[str, str]:
|
||||
if scheme == "Basic":
|
||||
return {"WWW-Authenticate": 'Basic realm="messenger", charset="UTF-8"'}
|
||||
return {"WWW-Authenticate": 'Bearer realm="messenger"'}
|
||||
|
||||
def current_user(request: Request) -> dict[str, Any]:
|
||||
auth = request.headers.get("authorization", "")
|
||||
if not auth.startswith("Bearer ") or not auth[7:].strip():
|
||||
fail(401, "unauthorized", "Valid Bearer session required", challenge("Bearer"))
|
||||
token_hash = hashlib.sha256(auth[7:].strip().encode("utf-8")).hexdigest()
|
||||
session_json = session_store.get("session:" + token_hash)
|
||||
if session_json is None:
|
||||
fail(401, "unauthorized", "Valid Bearer session required", challenge("Bearer"))
|
||||
try:
|
||||
session = json.loads(session_json)
|
||||
user_id = uuid.UUID(session["user_id"])
|
||||
expires_at = datetime.fromisoformat(session["expires_at"].replace("Z", "+00:00"))
|
||||
except (ValueError, KeyError, TypeError, json.JSONDecodeError):
|
||||
# Corrupt session data is invalidated instead of trusted.
|
||||
session_store.delete("session:" + token_hash)
|
||||
fail(401, "unauthorized", "Valid Bearer session required", challenge("Bearer"))
|
||||
if expires_at <= now_utc():
|
||||
session_store.delete("session:" + token_hash)
|
||||
fail(401, "unauthorized", "Valid Bearer session required", challenge("Bearer"))
|
||||
with db() as conn:
|
||||
row = conn.execute("SELECT id,username,display_name,created_at FROM users WHERE id=%s", (user_id,)).fetchone()
|
||||
if row is None:
|
||||
session_store.delete("session:" + token_hash)
|
||||
fail(401, "unauthorized", "Valid Bearer session required", challenge("Bearer"))
|
||||
return {"id": row[0], "username": row[1], "display_name": row[2], "created_at": row[3], "expires_at": expires_at, "user": {"id": str(row[0]), "username": row[1], "display_name": row[2], "created_at": iso(row[3])}, "token_hash": token_hash}
|
||||
|
||||
def ensure_member(actor_id: uuid.UUID, chat_id: uuid.UUID):
|
||||
with db() as conn:
|
||||
row = conn.execute("SELECT 1 FROM chats c JOIN chat_members m ON m.chat_id=c.id WHERE c.id=%s AND m.user_id=%s", (chat_id, actor_id)).fetchone()
|
||||
if row is None:
|
||||
fail(404, "not_found", "Object not found")
|
||||
|
||||
def parse_page_args(limit: int | None, cursor: str | None, resource: str):
|
||||
actual_limit = 50 if limit is None else limit
|
||||
if isinstance(actual_limit, bool) or actual_limit < 1 or actual_limit > 100:
|
||||
fail(400, "invalid_request", "limit must be an integer from 1 to 100")
|
||||
after = None
|
||||
if cursor is not None:
|
||||
if not cursor:
|
||||
fail(400, "invalid_request", "Invalid cursor")
|
||||
try:
|
||||
decoded = base64.urlsafe_b64decode(cursor + "=" * (-len(cursor) % 4)).decode("utf-8")
|
||||
obj = json.loads(decoded)
|
||||
if obj.get("resource") != resource:
|
||||
fail(400, "invalid_request", "Cursor belongs to another resource")
|
||||
after = (datetime.fromisoformat(obj["created_at"].replace("Z", "+00:00")), uuid.UUID(obj["id"]))
|
||||
except ApiError:
|
||||
raise
|
||||
except Exception:
|
||||
fail(400, "invalid_request", "Invalid cursor")
|
||||
return actual_limit, after
|
||||
|
||||
def make_cursor(resource: str, created_at: datetime, item_id: uuid.UUID) -> str:
|
||||
raw = json.dumps({"resource": resource, "created_at": iso(created_at), "id": str(item_id)}, separators=(",", ":")).encode()
|
||||
return base64.urlsafe_b64encode(raw).decode().rstrip("=")
|
||||
|
||||
def json_rows(cursor, query: str, params=()):
|
||||
with db() as conn:
|
||||
cur = conn.execute(query, params)
|
||||
names = [d.name for d in cur.description]
|
||||
return [dict(zip(names, row)) for row in cur.fetchall()]
|
||||
|
||||
@app.get("/health/live")
|
||||
def live():
|
||||
return {"status": "ok"}
|
||||
|
||||
@app.get("/health/ready")
|
||||
def ready():
|
||||
try:
|
||||
with db() as conn:
|
||||
conn.execute("SELECT 1")
|
||||
session_store.ping()
|
||||
return {"status": "ok"}
|
||||
except (psycopg.Error, PoolTimeout, RedisError):
|
||||
fail(503, "unavailable", "Required storage is unavailable")
|
||||
|
||||
@app.post("/api/v1/users", status_code=201)
|
||||
async def register(request: Request):
|
||||
body = await parse_body(request, RegisterBody)
|
||||
if not USERNAME_RE.fullmatch(body.username):
|
||||
fail(400, "invalid_request", "Invalid username")
|
||||
validate_unicode(body.display_name, "display_name")
|
||||
validate_unicode(body.password, "password")
|
||||
if not body.display_name.strip() or len(body.display_name) > 100:
|
||||
fail(400, "invalid_request", "Invalid display_name")
|
||||
if not 12 <= len(body.password) <= 128:
|
||||
fail(400, "invalid_request", "Password must contain 12 to 128 Unicode code points")
|
||||
user_id = uuid.uuid4()
|
||||
password_hash = PASSWORDS.hash(body.password)
|
||||
try:
|
||||
with db() as conn:
|
||||
row = conn.execute("INSERT INTO users(id,username,display_name,password_hash) VALUES (%s,%s,%s,%s) RETURNING id,username,display_name,created_at", (user_id, body.username, body.display_name, password_hash)).fetchone()
|
||||
except psycopg.errors.UniqueViolation:
|
||||
fail(409, "conflict", "Username already exists")
|
||||
return user_dict(dict(zip(("id", "username", "display_name", "created_at"), row)))
|
||||
|
||||
@app.get("/api/v1/users/me")
|
||||
def me(request: Request):
|
||||
return current_user(request)["user"]
|
||||
|
||||
@app.post("/api/v1/auth/sessions", status_code=201)
|
||||
async def create_session(request: Request):
|
||||
if await request.body():
|
||||
fail(400, "invalid_request", "Session creation does not accept a request body")
|
||||
auth = request.headers.get("authorization", "")
|
||||
if not auth[:6].lower() == "basic ":
|
||||
fail(401, "unauthorized", "Valid Basic credentials required", challenge("Basic"))
|
||||
try:
|
||||
decoded = base64.b64decode(auth[6:], validate=True).decode("utf-8")
|
||||
username, password = decoded.split(":", 1)
|
||||
except Exception:
|
||||
fail(401, "unauthorized", "Invalid credentials", challenge("Basic"))
|
||||
try:
|
||||
validate_unicode(password, "password")
|
||||
except ApiError:
|
||||
fail(401, "unauthorized", "Invalid credentials", challenge("Basic"))
|
||||
if not password or len(password) < 12 or len(password) > 128:
|
||||
fail(401, "unauthorized", "Invalid credentials", challenge("Basic"))
|
||||
with db() as conn:
|
||||
row = conn.execute("SELECT id,username,display_name,password_hash,created_at FROM users WHERE username=%s", (username,)).fetchone()
|
||||
if row is None:
|
||||
fail(401, "unauthorized", "Invalid credentials", challenge("Basic"))
|
||||
try:
|
||||
PASSWORDS.verify(row[3], password)
|
||||
except (VerifyMismatchError, VerificationError, InvalidHashError):
|
||||
fail(401, "unauthorized", "Invalid credentials", challenge("Basic"))
|
||||
token = secrets.token_urlsafe(32)
|
||||
token_hash = hashlib.sha256(token.encode()).hexdigest()
|
||||
expires_at = now_utc() + timedelta(seconds=SESSION_TTL_SECONDS)
|
||||
session_json = json.dumps({"user_id": str(row[0]), "expires_at": iso(expires_at)}, separators=(",", ":"))
|
||||
# Redis TTL enforces expiry; AOF + named volume make active sessions survive stack restarts.
|
||||
session_store.setex("session:" + token_hash, SESSION_TTL_SECONDS, session_json)
|
||||
user = user_dict({"id": row[0], "username": row[1], "display_name": row[2], "created_at": row[4]})
|
||||
return {"token": token, "token_type": "Bearer", "expires_at": iso(expires_at), "user": user}
|
||||
|
||||
@app.delete("/api/v1/auth/sessions/current", status_code=204)
|
||||
def revoke_session(request: Request):
|
||||
actor = current_user(request)
|
||||
session_store.delete("session:" + actor["token_hash"])
|
||||
return Response(status_code=204)
|
||||
|
||||
@app.post("/api/v1/chats", status_code=201)
|
||||
async def create_chat(request: Request):
|
||||
actor = current_user(request)
|
||||
body = await parse_body(request, ChatBody)
|
||||
validate_unicode(body.title, "title")
|
||||
if not body.title.strip() or len(body.title) > 100:
|
||||
fail(400, "invalid_request", "Invalid title")
|
||||
ids = [parse_uuid(value, "member UUID") for value in body.member_ids]
|
||||
if len(ids) != len(set(ids)) or actor["id"] not in ids:
|
||||
fail(400, "invalid_request", "Members must be unique and include the caller")
|
||||
chat_id = uuid.uuid4()
|
||||
try:
|
||||
with db() as conn:
|
||||
existing = conn.execute("SELECT id FROM users WHERE id = ANY(%s)", (ids,)).fetchall()
|
||||
if len(existing) != len(ids):
|
||||
fail(400, "invalid_request", "Unknown chat member")
|
||||
row = conn.execute("INSERT INTO chats(id,title) VALUES (%s,%s) RETURNING id,title,created_at", (chat_id, body.title)).fetchone()
|
||||
with conn.cursor() as cur:
|
||||
cur.executemany("INSERT INTO chat_members(chat_id,user_id,position) VALUES (%s,%s,%s)", [(chat_id, uid, position) for position, uid in enumerate(ids)])
|
||||
except psycopg.errors.ForeignKeyViolation:
|
||||
fail(400, "invalid_request", "Unknown chat member")
|
||||
return chat_dict({"id": row[0], "title": row[1], "created_at": row[2]}, [str(x) for x in ids])
|
||||
|
||||
@app.get("/api/v1/chats")
|
||||
def list_chats(request: Request, limit: int | None = None, cursor: str | None = None):
|
||||
actor = current_user(request)
|
||||
chat_list_resource = "chats:" + str(actor["id"])
|
||||
actual_limit, after = parse_page_args(limit, cursor, chat_list_resource)
|
||||
params: list[Any] = [actor["id"]]
|
||||
where = "m.user_id=%s"
|
||||
if after:
|
||||
where += " AND (c.created_at,c.id)>(%s,%s)"
|
||||
params.extend(after)
|
||||
params.append(actual_limit + 1)
|
||||
rows = json_rows(None, f"SELECT c.id,c.title,c.created_at FROM chats c JOIN chat_members m ON m.chat_id=c.id WHERE {where} ORDER BY c.created_at,c.id LIMIT %s", tuple(params))
|
||||
more = len(rows) > actual_limit
|
||||
rows = rows[:actual_limit]
|
||||
items = []
|
||||
for row in rows:
|
||||
members = json_rows(None, "SELECT user_id FROM chat_members WHERE chat_id=%s ORDER BY position,user_id", (row["id"],))
|
||||
items.append(chat_dict(row, [str(m["user_id"]) for m in members]))
|
||||
next_cursor = make_cursor(chat_list_resource, rows[-1]["created_at"], rows[-1]["id"]) if more and rows else None
|
||||
return {"items": items, "next_cursor": next_cursor}
|
||||
|
||||
@app.get("/api/v1/chats/{chat_id}")
|
||||
def get_chat(chat_id: str, request: Request):
|
||||
actor = current_user(request)
|
||||
parsed = parse_uuid(chat_id, "chat UUID")
|
||||
ensure_member(actor["id"], parsed)
|
||||
rows = json_rows(None, "SELECT id,title,created_at FROM chats WHERE id=%s", (parsed,))
|
||||
members = json_rows(None, "SELECT user_id FROM chat_members WHERE chat_id=%s ORDER BY position,user_id", (parsed,))
|
||||
return chat_dict(rows[0], [str(m["user_id"]) for m in members])
|
||||
|
||||
@app.post("/api/v1/chats/{chat_id}/messages", status_code=201)
|
||||
async def create_message(chat_id: str, request: Request):
|
||||
actor = current_user(request)
|
||||
parsed_chat = parse_uuid(chat_id, "chat UUID")
|
||||
ensure_member(actor["id"], parsed_chat)
|
||||
body = await parse_body(request, MessageBody)
|
||||
validate_unicode(body.text, "text")
|
||||
if not body.text.strip() or len(body.text) > 4000:
|
||||
fail(400, "invalid_request", "Message text must contain 1 to 4000 non-whitespace code points")
|
||||
message_id = uuid.uuid4()
|
||||
with db() as conn:
|
||||
row = conn.execute("INSERT INTO messages(id,chat_id,sender_id,text) VALUES (%s,%s,%s,%s) RETURNING id,chat_id,sender_id,text,created_at", (message_id, parsed_chat, actor["id"], body.text)).fetchone()
|
||||
return message_dict(dict(zip(("id", "chat_id", "sender_id", "text", "created_at"), row)))
|
||||
|
||||
@app.get("/api/v1/chats/{chat_id}/messages")
|
||||
def list_messages(chat_id: str, request: Request, limit: int | None = None, cursor: str | None = None):
|
||||
actor = current_user(request)
|
||||
parsed_chat = parse_uuid(chat_id, "chat UUID")
|
||||
ensure_member(actor["id"], parsed_chat)
|
||||
actual_limit, after = parse_page_args(limit, cursor, "messages:" + str(parsed_chat))
|
||||
params: list[Any] = [parsed_chat]
|
||||
where = "chat_id=%s"
|
||||
if after:
|
||||
where += " AND (created_at,id)>(%s,%s)"
|
||||
params.extend(after)
|
||||
params.append(actual_limit + 1)
|
||||
rows = json_rows(None, f"SELECT id,chat_id,sender_id,text,created_at FROM messages WHERE {where} ORDER BY created_at,id LIMIT %s", tuple(params))
|
||||
more = len(rows) > actual_limit
|
||||
rows = rows[:actual_limit]
|
||||
items = [message_dict(row) for row in rows]
|
||||
next_cursor = make_cursor("messages:" + str(parsed_chat), rows[-1]["created_at"], rows[-1]["id"]) if more and rows else None
|
||||
return {"items": items, "next_cursor": next_cursor}
|
||||
|
||||
if __name__ == "__main__":
|
||||
import uvicorn
|
||||
uvicorn.run("main:app", host=os.getenv("APP_HOST", "0.0.0.0"), port=int(os.getenv("APP_PORT", "8080")), timeout_graceful_shutdown=5)
|
||||
@@ -0,0 +1,190 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Level-4 checks: registration race, transactional rollback, and indexed 10k-row EXPLAIN.
|
||||
|
||||
Run from the repository root while the local Compose stack is healthy:
|
||||
python tests/grade4_checks.py --base-url http://localhost:8080
|
||||
The script uses `docker compose exec postgres psql` for controlled test data and
|
||||
removes its synthetic rows/triggers afterwards. It does not delete volumes.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import concurrent.futures
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
ENV_FILE = ROOT / ".env"
|
||||
EVIDENCE = ROOT / "evidence" / "explain-level4.txt"
|
||||
|
||||
|
||||
def read_env() -> dict[str, str]:
|
||||
result: dict[str, str] = {}
|
||||
if ENV_FILE.exists():
|
||||
for raw in ENV_FILE.read_text(encoding="utf-8").splitlines():
|
||||
line = raw.strip()
|
||||
if not line or line.startswith("#") or "=" not in line:
|
||||
continue
|
||||
key, value = line.split("=", 1)
|
||||
result[key.strip()] = value.strip().strip("\"'")
|
||||
return result
|
||||
|
||||
|
||||
ENV = read_env()
|
||||
|
||||
|
||||
def psql(sql: str) -> str:
|
||||
user = ENV.get("POSTGRES_USER", "messenger")
|
||||
database = ENV.get("POSTGRES_DB", "messenger")
|
||||
cmd = ["docker", "compose", "--env-file", str(ENV_FILE), "exec", "-T", "postgres",
|
||||
"psql", "-X", "-v", "ON_ERROR_STOP=1", "-A", "-t", "-U", user, "-d", database]
|
||||
completed = subprocess.run(cmd, input=sql, text=True, capture_output=True, cwd=ROOT)
|
||||
if completed.returncode:
|
||||
raise RuntimeError("psql failed: " + (completed.stderr.strip() or completed.stdout.strip()))
|
||||
return completed.stdout.strip()
|
||||
|
||||
|
||||
def api(method: str, base_url: str, path: str, body: dict | None = None,
|
||||
headers: dict[str, str] | None = None) -> tuple[int, dict | None]:
|
||||
request_headers = {"Accept": "application/json"}
|
||||
data = None
|
||||
if body is not None:
|
||||
data = json.dumps(body).encode("utf-8")
|
||||
request_headers["Content-Type"] = "application/json"
|
||||
request_headers.update(headers or {})
|
||||
req = urllib.request.Request(base_url.rstrip("/") + path, data=data,
|
||||
headers=request_headers, method=method)
|
||||
try:
|
||||
response = urllib.request.urlopen(req, timeout=15)
|
||||
except urllib.error.HTTPError as exc:
|
||||
response = exc
|
||||
with response:
|
||||
payload_bytes = response.read()
|
||||
try:
|
||||
payload = json.loads(payload_bytes) if payload_bytes else None
|
||||
except json.JSONDecodeError:
|
||||
payload = None
|
||||
return response.status, payload
|
||||
|
||||
|
||||
def register(base_url: str, username: str, display_name: str, password: str) -> tuple[int, dict | None]:
|
||||
return api("POST", base_url, "/api/v1/users", {
|
||||
"username": username, "display_name": display_name, "password": password
|
||||
})
|
||||
|
||||
|
||||
def login(base_url: str, username: str, password: str) -> str:
|
||||
import base64
|
||||
encoded = base64.b64encode(f"{username}:{password}".encode()).decode()
|
||||
status, payload = api("POST", base_url, "/api/v1/auth/sessions",
|
||||
headers={"Authorization": "Basic " + encoded})
|
||||
if status != 201 or not payload or not payload.get("token"):
|
||||
raise RuntimeError(f"Basic login failed: HTTP {status}")
|
||||
return payload["token"]
|
||||
|
||||
|
||||
def check_registration_race(base_url: str) -> None:
|
||||
username = "race_" + uuid.uuid4().hex[:12]
|
||||
password = "Grade4-Race-" + uuid.uuid4().hex[:16]
|
||||
def one(i: int) -> int:
|
||||
status, _ = register(base_url, username, f"Race probe {i}", password)
|
||||
return status
|
||||
with concurrent.futures.ThreadPoolExecutor(max_workers=8) as executor:
|
||||
statuses = list(executor.map(one, range(8)))
|
||||
if statuses.count(201) != 1 or statuses.count(409) != 7:
|
||||
raise RuntimeError(f"Registration race expected one 201 and seven 409, got {statuses}")
|
||||
print("PASS: concurrent same-username registration produced exactly one 201 and seven 409")
|
||||
|
||||
|
||||
def create_actor_and_chat(base_url: str) -> tuple[str, str, str]:
|
||||
username = "g4_" + uuid.uuid4().hex[:12]
|
||||
password = "Grade4-Probe-" + uuid.uuid4().hex[:16]
|
||||
status, user = register(base_url, username, "Level 4 test", password)
|
||||
if status != 201 or not user:
|
||||
raise RuntimeError(f"Probe user registration failed: HTTP {status}")
|
||||
token = login(base_url, username, password)
|
||||
title = "explain-probe-" + uuid.uuid4().hex[:10]
|
||||
status, chat = api("POST", base_url, "/api/v1/chats",
|
||||
{"title": title, "member_ids": [user["id"]]},
|
||||
{"Authorization": "Bearer " + token})
|
||||
if status != 201 or not chat:
|
||||
raise RuntimeError(f"Probe chat creation failed: HTTP {status}")
|
||||
return user["id"], chat["id"], token
|
||||
|
||||
|
||||
def check_rollback(base_url: str, user_id: str, token: str) -> None:
|
||||
psql("DROP TRIGGER IF EXISTS lab2_grade4_fail_membership_trigger ON chat_members;\n"
|
||||
"DROP FUNCTION IF EXISTS lab2_grade4_fail_membership();\n"
|
||||
"CREATE FUNCTION lab2_grade4_fail_membership() RETURNS trigger LANGUAGE plpgsql AS $$\n"
|
||||
"BEGIN IF EXISTS (SELECT 1 FROM chats WHERE id=NEW.chat_id AND title LIKE 'rollback-probe-%') THEN "
|
||||
"RAISE EXCEPTION 'intentional level-4 rollback probe'; END IF; RETURN NEW; END; $$;\n"
|
||||
"CREATE TRIGGER lab2_grade4_fail_membership_trigger BEFORE INSERT ON chat_members "
|
||||
"FOR EACH ROW EXECUTE FUNCTION lab2_grade4_fail_membership();\n")
|
||||
title = "rollback-probe-" + uuid.uuid4().hex[:10]
|
||||
try:
|
||||
status, _ = api("POST", base_url, "/api/v1/chats",
|
||||
{"title": title, "member_ids": [user_id]},
|
||||
{"Authorization": "Bearer " + token})
|
||||
if status != 503:
|
||||
raise RuntimeError(f"Rollback probe expected bounded HTTP 503, got {status}")
|
||||
count = psql("SELECT count(*) FROM chats WHERE title='" + title + "';")
|
||||
if count != "0":
|
||||
raise RuntimeError("Chat row remained after injected membership failure; transaction did not roll back")
|
||||
print("PASS: injected membership failure returned 503 and rolled back the chat row")
|
||||
finally:
|
||||
psql("DROP TRIGGER IF EXISTS lab2_grade4_fail_membership_trigger ON chat_members;\n"
|
||||
"DROP FUNCTION IF EXISTS lab2_grade4_fail_membership();\n")
|
||||
|
||||
|
||||
def check_explain(user_id: str, target_chat_id: str) -> None:
|
||||
# 99 other chats + the target chat, 100 rows per chat = 10,000 rows.
|
||||
psql("INSERT INTO chats(id,title) SELECT gen_random_uuid(), 'grade4-probe-chat-' || g "
|
||||
"FROM generate_series(1,99) AS g;\n")
|
||||
try:
|
||||
psql("WITH all_chats AS (SELECT '" + target_chat_id + "'::uuid AS id UNION ALL "
|
||||
"SELECT id FROM chats WHERE title LIKE 'grade4-probe-chat-%') "
|
||||
"INSERT INTO messages(id,chat_id,sender_id,text,created_at) "
|
||||
"SELECT gen_random_uuid(), c.id, '" + user_id + "'::uuid, "
|
||||
"'grade4-probe-message-' || g, now() + g * interval '1 microsecond' "
|
||||
"FROM all_chats c CROSS JOIN generate_series(1,100) AS g;\nANALYZE messages;\n")
|
||||
plan = psql("EXPLAIN (ANALYZE, BUFFERS) SELECT id,chat_id,sender_id,text,created_at "
|
||||
"FROM messages WHERE chat_id='" + target_chat_id + "'::uuid "
|
||||
"ORDER BY created_at,id LIMIT 51;\n")
|
||||
EVIDENCE.parent.mkdir(parents=True, exist_ok=True)
|
||||
EVIDENCE.write_text("Dataset: at least 10,000 synthetic messages across 100 chats.\n"
|
||||
"Query: history ordered by (created_at, id), LIMIT 51.\n\n" + plan + "\n",
|
||||
encoding="utf-8")
|
||||
if "messages_chat_created_id_idx" not in plan:
|
||||
raise RuntimeError("EXPLAIN did not use messages_chat_created_id_idx; inspect " + str(EVIDENCE))
|
||||
print(f"PASS: EXPLAIN (ANALYZE, BUFFERS) used messages_chat_created_id_idx; saved {EVIDENCE.relative_to(ROOT)}")
|
||||
finally:
|
||||
psql("DELETE FROM messages WHERE text LIKE 'grade4-probe-message-%';\n"
|
||||
"DELETE FROM chats WHERE title LIKE 'grade4-probe-chat-%';\n")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--base-url", default="http://localhost:8080")
|
||||
args = parser.parse_args()
|
||||
ready, _ = api("GET", args.base_url, "/health/ready")
|
||||
if ready != 200:
|
||||
raise RuntimeError(f"API must be ready before level-4 checks; got HTTP {ready}")
|
||||
check_registration_race(args.base_url)
|
||||
user_id, chat_id, token = create_actor_and_chat(args.base_url)
|
||||
check_rollback(args.base_url, user_id, token)
|
||||
check_explain(user_id, chat_id)
|
||||
print("PASS: all automated level-4 checks in this script completed")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
try:
|
||||
main()
|
||||
except (RuntimeError, OSError, subprocess.SubprocessError) as exc:
|
||||
raise SystemExit("FAIL: " + str(exc)) from None
|
||||
@@ -0,0 +1,34 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Verify that a Redis-backed Bearer session expires without logging out."""
|
||||
import time
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from client import Client, arguments, check, run, timestamp
|
||||
|
||||
|
||||
def main():
|
||||
args = arguments('Session TTL check; use SESSION_TTL_SECONDS=3 in .env for this test')
|
||||
c = Client(args.base_url, args.ca_file)
|
||||
actor = c.user('ttl')
|
||||
|
||||
# Confirm the session works before waiting for its server-declared expiry.
|
||||
c.call('GET', '/api/v1/users/me', 200, actor=actor, schema='User')
|
||||
# Client.user stores the registration payload, not the session response. Log in once
|
||||
# more to obtain the exact expires_at returned by the API and test that token.
|
||||
import base64
|
||||
credentials = actor['registration']
|
||||
basic = base64.b64encode((credentials['username'] + ':' + credentials['password']).encode()).decode()
|
||||
session = c.call('POST', '/api/v1/auth/sessions', 201,
|
||||
headers={'Authorization': 'Basic ' + basic}, schema='Session')['json']
|
||||
ttl_seconds = (timestamp(session['expires_at']) - datetime.now(timezone.utc)).total_seconds()
|
||||
check(0 < ttl_seconds <= 30,
|
||||
'Set SESSION_TTL_SECONDS=3 (or another value <=30) in .env and recreate the API container')
|
||||
token_actor = {'id': actor['id'], 'token': session['token']}
|
||||
c.call('GET', '/api/v1/users/me', 200, actor=token_actor, schema='User')
|
||||
time.sleep(max(0, ttl_seconds) + 1.0)
|
||||
c.call('GET', '/api/v1/users/me', 401, actor=token_actor)
|
||||
print('PASS: expired Redis session is rejected with HTTP 401')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
run(main)
|
||||
@@ -0,0 +1,69 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Controlled Redis outage check for the local Compose stack; restores Redis in finally."""
|
||||
import argparse
|
||||
import json
|
||||
import subprocess
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
|
||||
def request_status(url, timeout=15):
|
||||
try:
|
||||
with urllib.request.urlopen(url, timeout=timeout) as response:
|
||||
return response.status, response.read()
|
||||
except urllib.error.HTTPError as exc:
|
||||
return exc.code, exc.read()
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--base-url', default='http://localhost:8080')
|
||||
parser.add_argument('--timeout', type=int, default=45)
|
||||
#parser.add_argument('--outage-timeout', type=int, default=30)
|
||||
parser.add_argument('--compose-file', default='.env')
|
||||
parser.add_argument('--target', choices=('redis', 'postgres'), default='redis')
|
||||
args = parser.parse_args()
|
||||
compose = ['docker', 'compose', '--env-file', args.compose_file]
|
||||
stopped = False
|
||||
try:
|
||||
code, body = request_status(args.base_url.rstrip('/') + '/health/ready')
|
||||
if code != 200:
|
||||
raise RuntimeError('API must be ready before the test; got HTTP %s' % code)
|
||||
stopped = True
|
||||
subprocess.run(compose + ['stop', args.target], check=True)
|
||||
deadline = time.monotonic() + 30
|
||||
outage_status = None
|
||||
while time.monotonic() < deadline:
|
||||
try:
|
||||
outage_status, body = request_status(args.base_url.rstrip('/') + '/health/ready')
|
||||
if outage_status == 503:
|
||||
break
|
||||
except (TimeoutError, OSError):
|
||||
pass
|
||||
time.sleep(0.25)
|
||||
if outage_status != 503:
|
||||
raise RuntimeError('readiness did not return HTTP 503 while %s was stopped (last=%r)' % (args.target, outage_status))
|
||||
print('PASS: /health/ready returned HTTP 503 while %s was unavailable' % args.target)
|
||||
finally:
|
||||
if stopped:
|
||||
subprocess.run(compose + ['start', args.target], check=False)
|
||||
deadline = time.monotonic() + args.timeout
|
||||
recovered = False
|
||||
while time.monotonic() < deadline:
|
||||
try:
|
||||
code, body = request_status(args.base_url.rstrip('/') + '/health/ready')
|
||||
if code == 200:
|
||||
recovered = True
|
||||
break
|
||||
except (TimeoutError, OSError):
|
||||
pass
|
||||
time.sleep(0.5)
|
||||
if recovered:
|
||||
print('PASS: readiness recovered to HTTP 200 after %s restart' % args.target)
|
||||
else:
|
||||
raise RuntimeError('%s did not recover to ready state; inspect docker compose logs %s api' % (args.target, args.target))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||