ЛР2 — Дугина Александра, БПИ2301 — оценка 5 (трек 2) #131
@@ -30,3 +30,7 @@ coverage/
|
||||
.vscode/
|
||||
# Keep small, sanitized evidence summaries in evidence/ under version control.
|
||||
# Inspect git diff --cached: ignore rules are not a secret scanner.
|
||||
|
||||
/evidence/*.dump
|
||||
/evidence/*.backup
|
||||
/evidence/*.sql.gz
|
||||
|
||||
+26
@@ -0,0 +1,26 @@
|
||||
FROM python:3.11.16-slim-bookworm
|
||||
|
||||
ENV PYTHONDONTWRITEBYTECODE=1
|
||||
ENV PYTHONUNBUFFERED=1
|
||||
ENV APP_HOST=0.0.0.0
|
||||
ENV APP_PORT=8080
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY requirements.txt .
|
||||
|
||||
RUN python -m pip install --no-cache-dir -r requirements.txt
|
||||
|
||||
RUN useradd --uid 10001 --create-home --shell /usr/sbin/nologin appuser
|
||||
|
||||
COPY --chown=appuser:appuser src/ ./src/
|
||||
|
||||
USER appuser
|
||||
|
||||
EXPOSE 8080
|
||||
|
||||
STOPSIGNAL SIGTERM
|
||||
|
||||
HEALTHCHECK --interval=10s --timeout=3s --start-period=5s --retries=3 CMD python -c "import os,urllib.request; urllib.request.urlopen('http://127.0.0.1:' + os.getenv('APP_PORT', '8080') + '/health/ready', timeout=2).read()"
|
||||
|
||||
CMD ["python", "-m", "uvicorn", "src.main:app", "--host", "0.0.0.0", "--port", "8080", "--timeout-graceful-shutdown", "8"]
|
||||
@@ -1,37 +1,117 @@
|
||||
# Отчёт — лабораторная 2
|
||||
# Отчёт — лабораторная работа № 2
|
||||
|
||||
- Автор, группа: TODO
|
||||
- Целевая оценка; выбранный трек 5, если нужен: TODO
|
||||
- Commit/PR: TODO после публикации
|
||||
- Использованные источники, библиотеки и помощь ИИ: TODO
|
||||
**Автор:** Дугина Александра, БПИ2301
|
||||
|
||||
## Запуск из чистого clone
|
||||
**Дисциплина:** Высоконагруженные приложения
|
||||
|
||||
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
|
||||
**Целевая оценка:** 5; **трек:** 2 — планы запросов и восстановление данных
|
||||
|
||||
## Архитектура и решения
|
||||
## Запуск
|
||||
|
||||
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab1, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
|
||||
Требуются Docker Desktop / Docker Compose с поддержкой `up --wait` и Python 3.9+ для публичных проверок. Приложение использует Python 3.11.16, PostgreSQL 16.8 и Redis 7.4. API доступен на `127.0.0.1:8080`.
|
||||
|
||||
## Доказательства критериев
|
||||
Создать локально `secrets/postgres_password` и `secrets/database_url`. Первый файл содержит пароль пользователя `messenger`, второй — строку подключения `postgresql://messenger:<PASSWORD>@postgres:5432/messenger` с тем же паролем. Файлы исключены из Git и Docker build context. Не публиковать их содержимое.
|
||||
|
||||
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
|
||||
| --- | --- | --- | --- |
|
||||
| Публичный smoke | make test | TODO | TODO |
|
||||
| Остальные пункты уровня 3 | TODO | TODO | TODO |
|
||||
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
|
||||
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
|
||||
```sh
|
||||
docker compose up -d --build --wait
|
||||
python3 -m venv .venv
|
||||
.venv/bin/python -m pip install -r requirements-dev.txt
|
||||
make check
|
||||
make validate PYTHON=.venv/bin/python
|
||||
make test
|
||||
make test-persistence ACTION_SCRIPT=scripts/restart.sh
|
||||
```
|
||||
|
||||
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
|
||||
Миграция `db/migrations/001_initial.sql` автоматически применяется при инициализации пустого PostgreSQL volume. На существующем volume init-скрипты автоматически не повторяются; повторное применение текущей миграции вручную безопасно:
|
||||
|
||||
## Эксперименты и отказы
|
||||
```sh
|
||||
docker compose exec -T postgres psql -v ON_ERROR_STOP=1 -U messenger -d messenger < db/migrations/001_initial.sql
|
||||
```
|
||||
|
||||
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
|
||||
Остановка с сохранением данных: `docker compose stop`. Hook `scripts/restart.sh` перезапускает PostgreSQL, Redis и API, ждёт готовности сервисов и сохраняет именованные volumes.
|
||||
|
||||
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
|
||||
| --- | --- | --- | --- |
|
||||
| TODO | TODO | TODO | TODO |
|
||||
## Схема, транзакции и аутентификация
|
||||
|
||||
## Ограничения и следующие шаги
|
||||
PostgreSQL — источник истины. Таблицы: `users`, `chats`, `chat_members`, `messages`, `sessions`. UUID — первичные ключи; username и хеш токена уникальны; членство имеет составной PK `(chat_id, user_id)`. Внешние ключи исключают ссылки на отсутствующие сущности. Ограничения проверяют username и непустые поля. Индексы: история `(chat_id, created_at, id)`, членство по `user_id`, expiry сессий. Отдельный индекс `chat_id` исследуется в эксперименте; приложению достаточно составного индекса.
|
||||
|
||||
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
|
||||
Создание чата и всех участников выполняется в одной транзакции; ошибка вставки участника откатывает также чат. Отдельный тест принудительно вызывает ошибку после записи первого участника и проверяет отсутствие обеих записей. Регистрация опирается на UNIQUE username; гонка десяти запросов даёт один 201 и девять 409. Создание сообщения проверяет членство внутри INSERT. Успешный ответ формируется после выхода из контекста соединения и COMMIT.
|
||||
|
||||
SQL-запросы параметризованы; PostgreSQL pool: 1–10 соединений, ожидание 5 с, connect timeout 5 с, statement timeout 5 с. Потеря PostgreSQL обрабатывается HTTP 503, включая защищённые запросы и readiness. Эти ограничения не обеспечивают автоматический failover.
|
||||
|
||||
История использует keyset-пагинацию по `(created_at, id)`, запрашивает `limit + 1` строк. Для последующих страниц условие курсора включается отдельной формой SQL без nullable OR: это позволяет использовать range scan даже при generic prepared plan. Значения курсора остаются параметрами. В финальном тесте после 80 000 сообщений generic plan использует Index Cond по chat_id и курсору: 51 строка, 5 буферов, 0,054 мс. План сохранён в `evidence/pagination_plan.txt`.
|
||||
|
||||
Пароли хешируются Argon2id (time_cost=3, memory_cost=65536 KiB, parallelism=4). Basic применяется к созданию сессии; бизнес-запросы требуют Bearer. В PostgreSQL хранится SHA-256 случайного opaque токена, срок действия и признак отзыва. Неверный пароль, отозванный, просроченный, не-ASCII и синтаксически неверный токен дают 401. `X-User-Id` полномочий не предоставляет.
|
||||
|
||||
Redis кеширует профиль по хешу токена, TTL ограничен оставшимся временем сессии. Перед каждым обращением к кешу проверяются expiry и отзыв в PostgreSQL. Cache hit позволяет пропустить отдельное чтение профиля из `users`, но не проверку сессии. Logout сначала отзывает сессию в БД, затем удаляет кеш; недоступность Redis не отменяет отзыв. Профили через API не изменяются. После рестарта/потери кеша он заполняется повторно из PostgreSQL. Redis использует AOF и volume, однако сохранность сессий обеспечивается PostgreSQL. Проверка отказа подтверждает наличие ключа с TTL, чтение и запись через БД без Redis, logout и отклонение старого токена после возврата кеша.
|
||||
|
||||
## Проверки и соответствие критериям
|
||||
|
||||
Результаты финального аудита от 30.10.2026 сохранены в `evidence/audit_results.json`; там указаны команды, коды завершения и вывод. Публичные тесты и требования преподавателя не изменялись. Проверки добавляют синтетические записи, не удаляют существующие данные.
|
||||
|
||||
| Уровень | Требования и доказательства |
|
||||
| --- | --- |
|
||||
| 3 | PostgreSQL, PK/FK/UNIQUE, миграция с нуля и повторно; атомарное создание чата; KDF/Basic/Bearer/expiry/logout; smoke и persistence с прежней сессией; volumes и secrets исключены из Git |
|
||||
| 4 | Redis с TTL и fallback; ограниченный pool и таймауты; параметризованный SQL; гонка username и реальный rollback; EXPLAIN на ≥100 тыс. сообщений; автоматические проверки TTL, logout, неверных credentials, отказов Redis и БД |
|
||||
| 5, трек 2 | Неравномерные размеры чатов ≥100 тыс. сообщений; сравнение составного и одиночного индексов, измерение записи; backup и восстановление в новый volume, сверка количеств и выбранных сообщений; обсуждение ограничений и RPO/RTO |
|
||||
|
||||
Дополнительные проверки:
|
||||
|
||||
```sh
|
||||
python3 scripts/check_auth.py
|
||||
python3 scripts/check_advanced.py
|
||||
python3 scripts/check_messenger.py
|
||||
python3 scripts/check_redis_failure.py
|
||||
python3 scripts/check_db_failure.py
|
||||
python3 scripts/check_migrations.py
|
||||
docker compose exec -T api python < scripts/check_transactions.py
|
||||
# Отдельный API с TTL=2 с; основной API не перенастраивается:
|
||||
docker compose -f compose.yaml -f compose.ttl.yaml up -d --build --wait api-ttl
|
||||
python3 scripts/check_ttl.py
|
||||
docker compose -f compose.yaml -f compose.ttl.yaml stop api-ttl
|
||||
```
|
||||
|
||||
Сценарии отказов останавливают соответствующий сервис и восстанавливают его в `finally`. Запускать последовательно на локальном учебном стенде. `scripts/check_messenger.py` проверяет функциональность; сохранность после рестарта доказывает именно `tests/resilience.py`, а не название последнего шага messenger-теста.
|
||||
|
||||
## Нагрузка HTTP
|
||||
|
||||
Сохранённый результат `make load`: **10 942** успешных запросов, **0** ошибок, **364,483 RPS**, p50 **10,341 мс**, p95 **16,404 мс**, p99 **21,772 мс**. Источник: `evidence/load_results.txt`. Продолжительность 30 с (фактически 30,021), concurrency=4, closed-loop POST сообщений без автоматических повторов. Перцентили относятся только к успешным ответам.
|
||||
|
||||
Это ранее сохранённое измерение локального Docker Desktop стенда (MacBook Pro, Apple M1); после исправлений финального аудита нагрузка повторно не запускалась. Нет серии повторов, CPU/RAM и точного commit измерения. Closed-loop имеет coordinated omission и накладные расходы клиента, поэтому результат не является универсальным пределом производительности.
|
||||
|
||||
## Планы чтения и стоимость записи
|
||||
|
||||
Гипотеза: составной индекс `(chat_id, created_at, id)` позволяет получить упорядоченную ограниченную страницу без сортировки и чтения всей истории. Генератор `scripts/generate_100k.sql` добавляет 90 000 и 10 000 сообщений двум существующим чатам; требует минимум два чата и при повторном запуске добавляет ещё данные. Сохранённый `evidence/index_benchmark.txt` подтверждает размеры крупнейших чатов 90 005 и 10 005 сообщений, плюс другие чаты. Поэтому порог ≥100 000 подтверждён независимо от общего счётчика.
|
||||
|
||||
Методика: на локальной БД выполнить ANALYZE, сравнить планы первой страницы с составным индексом и без него при наличии индекса только по `chat_id`. `scripts/benchmark_indexes.sql` показывает, что добавление одиночного индекса при сохранении составного не меняет выбор оптимизатора; `scripts/compare_indexes.sql` временно убирает составной индекс внутри откатываемой транзакции. Эксперименты блокируют таблицу при DDL, поэтому выполнять на учебном стенде без параллельной нагрузки.
|
||||
|
||||
```sh
|
||||
docker compose exec -T postgres psql -v ON_ERROR_STOP=1 -U messenger -d messenger < scripts/benchmark_indexes.sql
|
||||
docker compose exec -T postgres psql -v ON_ERROR_STOP=1 -U messenger -d messenger < scripts/compare_indexes.sql
|
||||
docker compose exec -T postgres psql -v ON_ERROR_STOP=1 -U messenger -d messenger < scripts/benchmark_writes.sql
|
||||
docker compose exec -T postgres psql -v ON_ERROR_STOP=1 -U messenger -d messenger < scripts/explain_pagination.sql
|
||||
```
|
||||
|
||||
| Условие | План | Execution Time | Буферы |
|
||||
| --- | --- | ---: | ---: |
|
||||
| Составной индекс | Index Scan | 0,048 мс | 1 hit + 4 read |
|
||||
| Только одиночный вторичный индекс | Parallel Seq Scan + Sort | 13,525 мс | 182 hit + 1475 read |
|
||||
|
||||
Источник: `evidence/index_comparison.txt`. Выигрыш одного измерения около 282 раз; выбор seq scan при наличии одиночного индекса относится к данной селективности. `evidence/index_sizes.txt`: составной индекс 8552 kB, одиночный 912 kB, PK 4608 kB. Это размеры в момент прежнего измерения, они изменяются при новых записях.
|
||||
|
||||
Гипотеза записи: поддержка вторичных индексов увеличивает стоимость INSERT. Сохранённый `evidence/write_benchmark.txt`: 10 000 вставок без двух вторичных индексов **88,932 мс**, с ними **107,873 мс** (+21,3%). PK и проверки FK остаются в обеих конфигурациях; вставки выполняются в транзакциях с ROLLBACK. Для воспроизведения конфигурации с двумя вторичными индексами скрипт временно создаёт также `idx_messages_chat_id_only` внутри откатываемой транзакции. Результаты SQL INSERT нельзя напрямую сравнивать с HTTP RPS. Порядок прогонов и прогрев кеша не контролировались, статистической серии нет.
|
||||
|
||||
Повтор SQL-экспериментов при финальном аудите сохранён отдельно: `evidence/index_benchmark_audit.txt`, `evidence/index_comparison_audit.txt`, `evidence/write_benchmark_audit.txt`. Сравнение чтения: **0,048 мс** с составным индексом и **14,977 мс** без него; запись 10 000 сообщений: **111,483 мс** без вторичных индексов и **117,756 мс** с ними (+5,6%). Различие с прежним +21,3% показывает изменчивость единичных измерений; оба прогона сохранены, результаты не усредняются. DDL и тестовые вставки откатились.
|
||||
|
||||
## Backup → restore в отдельное хранилище
|
||||
|
||||
Повторяемая проверка: `python3 scripts/check_backup_restore.py`. Выполнять без параллельных писателей. Скрипт создаёт custom backup через `pg_dump`, запускает новый PostgreSQL 16.8 в контейнере с уникальным именем и отдельным новым volume, восстанавливает через `pg_restore --exit-on-error`. Контейнер не имеет сети (`--network none`) и опубликованных портов. После проверки он останавливается, volume и backup сохраняются; рабочая БД и старые volumes не перезаписываются. Backup содержит данные и хеши паролей, хранится только локально в игнорируемом `backups/` с правами 0600.
|
||||
|
||||
Результат аудита: **122 616 сообщений**, совпали количества во всех пяти таблицах и UUID/MD5 текста пяти выбранных сообщений; исходная сводка до/после dump также совпала. `pg_restore` занял **0,437 с**. Имена отдельного контейнера и volume, счётчики и обе выборки хешей сохранены в `evidence/backup_restore.json`. MD5 здесь используется для выборочной проверки текста, не для паролей или авторизации. Старые `original_samples.txt` / `restored_samples.txt` совпадают, но сами по себе не подтверждают прежнее время восстановления и общий счётчик; прежние неподтверждённые значения из отчёта исключены.
|
||||
|
||||
Измерено время команды восстановления, а не полный RTO: не включены запуск БД, ожидание готовности, переподключение API и сквозная проверка. RPO ручного snapshot определяется временем между backup и отказом; более поздние записи отсутствуют. Не реализованы регулярные backup, WAL/PITR, репликация и failover. Сверка количеств и пяти образцов не доказывает побайтовую идентичность всей БД.
|
||||
|
||||
## Ограничения и помощь ИИ
|
||||
|
||||
Доказательства соответствуют локальному стенду и перечисленным сценариям; smoke сам по себе не выставляет оценку. Полный RTO, длительная нагрузка и статистическая серия не измерены. Сессии проверяются в PostgreSQL на каждом запросе, поэтому Redis не снимает всю нагрузку аутентификации. Проверки сохраняют синтетические данные; число записей после новых запусков растёт.
|
||||
|
||||
Использованы README, контракт и публичные проверки преподавателя. ChatGPT помогал с кодом, SQL-экспериментами, аудитом и отчётом. Запуски и ограничения перечислены явно; сохранённые показатели нагрузки не выдаются за новый прогон.
|
||||
|
||||
@@ -0,0 +1,14 @@
|
||||
# Дополнительный API с коротким TTL; основное приложение и volumes не изменяются.
|
||||
services:
|
||||
api-ttl:
|
||||
build: .
|
||||
environment:
|
||||
DATABASE_URL_FILE: /run/secrets/database_url
|
||||
SESSION_TTL_SECONDS: 2
|
||||
secrets:
|
||||
- database_url
|
||||
ports:
|
||||
- "127.0.0.1:8081:8080"
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
@@ -0,0 +1,61 @@
|
||||
services:
|
||||
redis:
|
||||
image: redis:7.4-alpine
|
||||
command:
|
||||
- redis-server
|
||||
- --appendonly
|
||||
- "yes"
|
||||
volumes:
|
||||
- redis_data:/data
|
||||
healthcheck:
|
||||
test: ["CMD", "redis-cli", "ping"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 10
|
||||
restart: unless-stopped
|
||||
|
||||
postgres:
|
||||
image: postgres:16.8-bookworm
|
||||
environment:
|
||||
POSTGRES_DB: messenger
|
||||
POSTGRES_USER: messenger
|
||||
POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password
|
||||
secrets:
|
||||
- postgres_password
|
||||
volumes:
|
||||
- postgres_data:/var/lib/postgresql/data
|
||||
- ./db/migrations:/docker-entrypoint-initdb.d:ro
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U messenger -d messenger"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 10
|
||||
restart: unless-stopped
|
||||
|
||||
api:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
environment:
|
||||
DATABASE_URL_FILE: /run/secrets/database_url
|
||||
APP_HOST: 0.0.0.0
|
||||
APP_PORT: 8080
|
||||
SESSION_TTL_SECONDS: 3600
|
||||
secrets:
|
||||
- database_url
|
||||
ports:
|
||||
- "127.0.0.1:8080:8080"
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
restart: unless-stopped
|
||||
|
||||
volumes:
|
||||
redis_data:
|
||||
postgres_data:
|
||||
|
||||
secrets:
|
||||
postgres_password:
|
||||
file: ./secrets/postgres_password
|
||||
database_url:
|
||||
file: ./secrets/database_url
|
||||
@@ -0,0 +1,51 @@
|
||||
CREATE TABLE IF NOT EXISTS users (
|
||||
id UUID PRIMARY KEY,
|
||||
username VARCHAR(32) NOT NULL UNIQUE
|
||||
CHECK (username ~ '^[a-z0-9_]{3,32}$'),
|
||||
display_name VARCHAR(100) NOT NULL
|
||||
CHECK (length(btrim(display_name)) > 0),
|
||||
password_hash TEXT NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS chats (
|
||||
id UUID PRIMARY KEY,
|
||||
title VARCHAR(100) NOT NULL
|
||||
CHECK (length(btrim(title)) > 0),
|
||||
created_by UUID NOT NULL REFERENCES users(id),
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS chat_members (
|
||||
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
|
||||
user_id UUID NOT NULL REFERENCES users(id),
|
||||
joined_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
PRIMARY KEY (chat_id, user_id)
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS messages (
|
||||
id UUID PRIMARY KEY,
|
||||
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
|
||||
sender_id UUID NOT NULL REFERENCES users(id),
|
||||
text TEXT NOT NULL
|
||||
CHECK (length(text) <= 4000 AND length(btrim(text)) > 0),
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS sessions (
|
||||
id UUID PRIMARY KEY,
|
||||
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
token_hash TEXT NOT NULL UNIQUE,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
expires_at TIMESTAMPTZ NOT NULL,
|
||||
revoked_at TIMESTAMPTZ
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_messages_chat_created_id
|
||||
ON messages (chat_id, created_at, id);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_chat_members_user_id
|
||||
ON chat_members (user_id);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_sessions_expires_at
|
||||
ON sessions (expires_at);
|
||||
@@ -0,0 +1,130 @@
|
||||
{
|
||||
"date": "2026-10-30T10:56:54.044284+00:00",
|
||||
"checks": [
|
||||
{
|
||||
"command": "make check",
|
||||
"exit_code": 0,
|
||||
"stdout": "python3 scripts/check_contract.py\nPASS: offline skeleton structure, local refs and Python syntax; application NOT tested.\n",
|
||||
"stderr": ""
|
||||
},
|
||||
{
|
||||
"command": "make validate PYTHON=.venv/bin/python",
|
||||
"exit_code": 0,
|
||||
"stdout": ".venv/bin/python scripts/validate_openapi.py\nPASS: full OpenAPI 3.1 and JSON Schema validation; application NOT tested.\n",
|
||||
"stderr": ""
|
||||
},
|
||||
{
|
||||
"command": "make test",
|
||||
"exit_code": 0,
|
||||
"stdout": "python3 tests/smoke.py --base-url \"http://localhost:8080\" \nPASS: lab2 public API scenario; infrastructure and grading evidence remain separate.\n",
|
||||
"stderr": ""
|
||||
},
|
||||
{
|
||||
"command": "python3 scripts/check_auth.py",
|
||||
"exit_code": 0,
|
||||
"stdout": "1. Registration\nRegistration status: 201\nRegistration headers: {'Content-Length': '144', 'Connection': 'keep-alive', 'Content-Type': 'application/json', 'Date': 'Fri, 30 Oct 2026 10:56:55 GMT', 'Keep-Alive': 'timeout=60', 'Proxy-Connection': 'keep-alive', 'Server': 'uvicorn', 'X-Request-Id': 'e18774ae-f206-4216-8a6c-81648825f093'}\nRegistration body: {'id': 'abd4cc85-fe81-4dae-a33a-4b09fcc993a4', 'username': 'test_10537209d0', 'display_name': 'Test User', 'created_at': '2026-10-30T10:56:55.192161Z'}\nPASS: registration: 201 (expected 201)\nUser ID: abd4cc85-fe81-4dae-a33a-4b09fcc993a4\n\n2. Basic login\nPASS: login: 201 (expected 201)\n\n3. Access with Bearer token\nPASS: authorized request: 200 (expected 200)\n\n4. Access without token\nPASS: missing token: 401 (expected 401)\n\n5. X-User-Id cannot authorize\nPASS: X-User-Id rejected: 401 (expected 401)\n\n6. Logout\nPASS: logout: 204 (expected 204)\n\n7. Reuse revoked token\nPASS: revoked token rejected: 401 (expected 401)\n\nALL AUTH CHECKS PASSED\n",
|
||||
"stderr": ""
|
||||
},
|
||||
{
|
||||
"command": "python3 scripts/check_advanced.py",
|
||||
"exit_code": 0,
|
||||
"stdout": "1. Concurrent registration\nPASS: one registration, nine conflicts\n2. Incorrect password\nPASS: incorrect password rejected\n3. Invalid Bearer token\nPASS: invalid token rejected\nPASS: malformed and non-ASCII Bearer tokens rejected\n4. Unknown chat member\nPASS: unknown member rejected\n\nALL ADVANCED API CHECKS PASSED\n",
|
||||
"stderr": ""
|
||||
},
|
||||
{
|
||||
"command": "python3 scripts/check_messenger.py",
|
||||
"exit_code": 0,
|
||||
"stdout": "\n1. Register two users\nPASS: register alice_b35280a7: 201\nPASS: register bob_b35280a7: 201\n\n2. Login\nPASS: login alice_b35280a7: 201\nPASS: login bob_b35280a7: 201\n\n3. Create chat\nPASS: create chat: 201\nChat ID: 1b010c7a-727b-43cf-965c-4377c6c87f05\n\n4. Check chat visibility\nPASS: get chat: 200\nPASS: get chat: 200\n\n5. Send messages\nPASS: send message: 201\nPASS: send message: 201\nPASS: send message: 201\nPASS: send message: 201\nPASS: send message: 201\n\n6. Read first page\nPASS: first page: 200\n\n7. Read second page\nPASS: second page: 200\n\n8. Check chat list\nPASS: list chats: 200\n\n9. Check PostgreSQL persistence\nCreated chat: 1b010c7a-727b-43cf-965c-4377c6c87f05\nCreated messages: 5\n\nALL MESSENGER CHECKS PASSED\n",
|
||||
"stderr": ""
|
||||
},
|
||||
{
|
||||
"command": "python3 scripts/check_redis_failure.py",
|
||||
"exit_code": 0,
|
||||
"stdout": "PASS: Redis contains session profile with TTL\nPASS: PostgreSQL read/write and logout work without Redis\nPASS: stale cache cannot revive revoked session; Redis recovered\n",
|
||||
"stderr": " Container mtusi2026lab2-redis-1 Stopping\n Container mtusi2026lab2-redis-1 Stopped\ntime=\"2026-10-30T13:56:57+03:00\" level=warning msg=\"Found orphan containers ([mtusi2026lab2-api-ttl-1]) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.\"\n Container mtusi2026lab2-redis-1 Created\n Container mtusi2026lab2-redis-1 Starting\n Container mtusi2026lab2-redis-1 Started\n Container mtusi2026lab2-redis-1 Waiting\n Container mtusi2026lab2-redis-1 Healthy\n"
|
||||
},
|
||||
{
|
||||
"command": "python3 scripts/check_db_failure.py",
|
||||
"exit_code": 0,
|
||||
"stdout": "PASS: token works before database failure\nPASS: database failure does not grant access: 503\nALL DATABASE FAILURE CHECKS PASSED; original session recovered\n",
|
||||
"stderr": " Container mtusi2026lab2-postgres-1 Stopping\n Container mtusi2026lab2-postgres-1 Stopped\n Container mtusi2026lab2-postgres-1 Starting\n Container mtusi2026lab2-postgres-1 Started\n"
|
||||
},
|
||||
{
|
||||
"command": "python3 scripts/check_migrations.py",
|
||||
"exit_code": 0,
|
||||
"stdout": "PASS: migration creates five tables from zero and reapplies without errors; rolled back\n",
|
||||
"stderr": ""
|
||||
},
|
||||
{
|
||||
"command": "docker compose exec -T api python",
|
||||
"exit_code": 0,
|
||||
"stdout": "PASS: failure after first member insert rolls back chat and membership\n",
|
||||
"stderr": "",
|
||||
"stdin_file": "scripts/check_transactions.py"
|
||||
},
|
||||
{
|
||||
"command": "make test-persistence ACTION_SCRIPT=scripts/restart.sh",
|
||||
"exit_code": 0,
|
||||
"stdout": "python3 tests/resilience.py --mode persistence --action-script \"scripts/restart.sh\" --base-url \"http://localhost:8080\" \nExecuting explicit persistence hook; hook output suppressed to avoid leaking secrets.\nPASS: persistence preserved confirmed data and existing session. Hook scope needs operator evidence.\n",
|
||||
"stderr": ""
|
||||
},
|
||||
{
|
||||
"command": "docker compose -f compose.yaml -f compose.ttl.yaml up -d --build --wait api-ttl",
|
||||
"exit_code": 0,
|
||||
"stdout": "Compose now can delegate build to bake for better performances\nJust set COMPOSE_BAKE=true\n#0 building with \"desktop-linux\" instance using docker driver\n\n#1 [api-ttl internal] load build definition from Dockerfile\n#1 transferring dockerfile: 778B done\n#1 DONE 0.0s\n\n#2 [api-ttl internal] load metadata for docker.io/library/python:3.11.16-slim-bookworm\n#2 DONE 0.0s\n\n#3 [api-ttl internal] load .dockerignore\n#3 transferring context: 196B done\n#3 DONE 0.0s\n\n#4 [api-ttl internal] load build context\n#4 transferring context: 720B done\n#4 DONE 0.0s\n\n#5 [api-ttl 1/6] FROM docker.io/library/python:3.11.16-slim-bookworm@sha256:a36c24f9cbdf4fd0f52d67f0823eeac19c2028c637cecc392d97f980d4fec56b\n#5 resolve docker.io/library/python:3.11.16-slim-bookworm@sha256:a36c24f9cbdf4fd0f52d67f0823eeac19c2028c637cecc392d97f980d4fec56b\n#5 resolve docker.io/library/python:3.11.16-slim-bookworm@sha256:a36c24f9cbdf4fd0f52d67f0823eeac19c2028c637cecc392d97f980d4fec56b 0.7s done\n#5 DONE 0.7s\n\n#6 [api-ttl 3/6] COPY requirements.txt .\n#6 CACHED\n\n#7 [api-ttl 4/6] RUN python -m pip install --no-cache-dir -r requirements.txt\n#7 CACHED\n\n#8 [api-ttl 5/6] RUN useradd --uid 10001 --create-home --shell /usr/sbin/nologin appuser\n#8 CACHED\n\n#9 [api-ttl 2/6] WORKDIR /app\n#9 CACHED\n\n#10 [api-ttl 6/6] COPY --chown=appuser:appuser src/ ./src/\n#10 CACHED\n\n#11 [api-ttl] exporting to image\n#11 exporting layers done\n#11 exporting manifest sha256:a6a0f6f9fc66e2ad26dd5d141b27ab8e4d3a8ee26b43afa21aa58c6e5ef10fcd done\n#11 exporting config sha256:9232a3492de92095ab11f4877bc256cb3227c8ae3cc63430bd9f3664a4d357ce done\n#11 exporting attestation manifest sha256:4673ee5cb2ce9fedb997670011d04711c70452a44bc23d18c8313e17370062e2 done\n#11 exporting manifest list sha256:8f1054addd1e51ae837fa0a23e0e485839f97b15257f8aa53638fd848978c5c5 done\n#11 naming to docker.io/library/mtusi2026lab2-api-ttl:latest done\n#11 unpacking to docker.io/library/mtusi2026lab2-api-ttl:latest done\n#11 DONE 0.1s\n\n#12 [api-ttl] resolving provenance for metadata file\n#12 DONE 0.0s\n",
|
||||
"stderr": " api-ttl Built\n Container mtusi2026lab2-postgres-1 Running\n Container mtusi2026lab2-api-ttl-1 Recreate\n Container mtusi2026lab2-api-ttl-1 Recreated\n Container mtusi2026lab2-postgres-1 Waiting\n Container mtusi2026lab2-postgres-1 Healthy\n Container mtusi2026lab2-api-ttl-1 Starting\n Container mtusi2026lab2-api-ttl-1 Started\n Container mtusi2026lab2-postgres-1 Waiting\n Container mtusi2026lab2-api-ttl-1 Waiting\n Container mtusi2026lab2-postgres-1 Healthy\n Container mtusi2026lab2-api-ttl-1 Healthy\n"
|
||||
},
|
||||
{
|
||||
"command": "python3 scripts/check_ttl.py",
|
||||
"exit_code": 0,
|
||||
"stdout": "PASS: session expires after TTL\n",
|
||||
"stderr": ""
|
||||
},
|
||||
{
|
||||
"command": "docker compose -f compose.yaml -f compose.ttl.yaml stop api-ttl",
|
||||
"exit_code": 0,
|
||||
"stdout": "",
|
||||
"stderr": " Container mtusi2026lab2-api-ttl-1 Stopping\n Container mtusi2026lab2-api-ttl-1 Stopped\n"
|
||||
},
|
||||
{
|
||||
"command": "docker compose exec -T postgres psql -X -v ON_ERROR_STOP=1 -U messenger -d messenger",
|
||||
"exit_code": 0,
|
||||
"stdout": "BEGIN\nSET\nPREPARE\n QUERY PLAN \n-----------------------------------------------------------------------------------------------------------------------------------------------------\n Limit (cost=0.42..582.96 rows=584 width=78) (actual time=0.020..0.032 rows=51 loops=1)\n Buffers: shared hit=2 read=3\n -> Index Scan using idx_messages_chat_created_id on messages (cost=0.42..5820.85 rows=5835 width=78) (actual time=0.019..0.028 rows=51 loops=1)\n Index Cond: ((chat_id = $1) AND (ROW(created_at, id) > ROW($2, $3)))\n Buffers: shared hit=2 read=3\n Planning:\n Buffers: shared hit=6\n Planning Time: 0.128 ms\n Execution Time: 0.054 ms\n(9 rows)\n\nROLLBACK\n",
|
||||
"stderr": "",
|
||||
"stdin_file": "scripts/explain_pagination.sql"
|
||||
},
|
||||
{
|
||||
"command": "docker compose exec -T postgres psql -X -v ON_ERROR_STOP=1 -U messenger -d messenger",
|
||||
"exit_code": 0,
|
||||
"stdout": "BEGIN\nTiming is on.\nANALYZE\nTime: 117.334 ms\n chat_id | messages_count \n--------------------------------------+----------------\n f68eff95-e44f-4c79-997c-49eb725cfb4d | 90005\n b14d0a4f-3046-44b2-871e-4ebfb752c543 | 11613\n 41d9abbc-4d0b-44f7-a1a2-ea3de4faa0c3 | 10942\n 1b7b678e-d262-4d61-8167-7fc8b736f3ff | 10005\n 0d8c7368-f88a-4bdb-972e-e4d7e1cd437a | 5\n(5 rows)\n\nTime: 10.412 ms\nTime: 8.597 ms\nTEST A: existing composite index\n QUERY PLAN \n-------------------------------------------------------------------------------------------------------------------------------------------------------\n Limit (cost=0.42..6.74 rows=50 width=45) (actual time=0.025..0.036 rows=50 loops=1)\n Buffers: shared hit=3 read=2\n -> Index Scan using idx_messages_chat_created_id on messages (cost=0.42..11395.89 rows=90150 width=45) (actual time=0.024..0.033 rows=50 loops=1)\n Index Cond: (chat_id = 'f68eff95-e44f-4c79-997c-49eb725cfb4d'::uuid)\n Buffers: shared hit=3 read=2\n Planning:\n Buffers: shared hit=10\n Planning Time: 0.069 ms\n Execution Time: 0.044 ms\n(9 rows)\n\nTime: 0.360 ms\nTEST B: additional index\nCREATE INDEX\nTime: 0.285 ms\nANALYZE\nTime: 123.698 ms\n QUERY PLAN \n-------------------------------------------------------------------------------------------------------------------------------------------------------\n Limit (cost=0.42..6.75 rows=50 width=45) (actual time=0.007..0.014 rows=50 loops=1)\n Buffers: shared hit=5\n -> Index Scan using idx_messages_chat_created_id on messages (cost=0.42..11378.89 rows=89905 width=45) (actual time=0.006..0.011 rows=50 loops=1)\n Index Cond: (chat_id = 'f68eff95-e44f-4c79-997c-49eb725cfb4d'::uuid)\n Buffers: shared hit=5\n Planning:\n Buffers: shared hit=15\n Planning Time: 0.081 ms\n Execution Time: 0.023 ms\n(9 rows)\n\nTime: 0.279 ms\nIndex sizes\n indexrelname | size \n------------------------------+---------\n idx_messages_chat_created_id | 9328 kB\n idx_messages_chat_id_only | 1016 kB\n messages_pkey | 5392 kB\n(3 rows)\n\nTime: 1.366 ms\nROLLBACK\nTime: 0.119 ms\n",
|
||||
"stderr": "NOTICE: relation \"idx_messages_chat_id_only\" already exists, skipping\n",
|
||||
"stdin_file": "scripts/benchmark_indexes.sql"
|
||||
},
|
||||
{
|
||||
"command": "docker compose exec -T postgres psql -X -v ON_ERROR_STOP=1 -U messenger -d messenger",
|
||||
"exit_code": 0,
|
||||
"stdout": "Timing is on.\nTime: 8.859 ms\nCOMPOSITE INDEX\n QUERY PLAN \n-------------------------------------------------------------------------------------------------------------------------------------------------------\n Limit (cost=0.42..6.06 rows=50 width=46) (actual time=0.028..0.039 rows=50 loops=1)\n Buffers: shared hit=5\n -> Index Scan using idx_messages_chat_created_id on messages (cost=0.42..11162.90 rows=98962 width=46) (actual time=0.027..0.036 rows=50 loops=1)\n Index Cond: (chat_id = 'f68eff95-e44f-4c79-997c-49eb725cfb4d'::uuid)\n Buffers: shared hit=5\n Planning:\n Buffers: shared hit=14\n Planning Time: 0.097 ms\n Execution Time: 0.048 ms\n(9 rows)\n\nTime: 0.507 ms\nBEGIN\nTime: 0.083 ms\nCREATE INDEX\nTime: 0.205 ms\nDROP INDEX\nTime: 0.680 ms\nSINGLE COLUMN INDEX\n QUERY PLAN \n-----------------------------------------------------------------------------------------------------------------------------------------\n Limit (cost=5448.46..5454.21 rows=50 width=46) (actual time=13.704..14.946 rows=50 loops=1)\n Buffers: shared hit=114 read=1543\n -> Gather Merge (cost=5448.46..12142.95 rows=58213 width=46) (actual time=13.703..14.938 rows=50 loops=1)\n Workers Planned: 1\n Workers Launched: 1\n Buffers: shared hit=114 read=1543\n -> Sort (cost=4448.45..4593.98 rows=58213 width=46) (actual time=12.461..12.463 rows=50 loops=2)\n Sort Key: created_at, id\n Sort Method: top-N heapsort Memory: 28kB\n Buffers: shared hit=114 read=1543\n Worker 0: Sort Method: top-N heapsort Memory: 28kB\n -> Parallel Seq Scan on messages (cost=0.00..2514.65 rows=58213 width=46) (actual time=0.012..8.363 rows=45002 loops=2)\n Filter: (chat_id = 'f68eff95-e44f-4c79-997c-49eb725cfb4d'::uuid)\n Rows Removed by Filter: 16310\n Buffers: shared hit=70 read=1543\n Planning:\n Buffers: shared hit=4\n Planning Time: 0.038 ms\n Execution Time: 14.977 ms\n(19 rows)\n\nTime: 15.268 ms\nROLLBACK\nTime: 0.138 ms\nORIGINAL INDEX RESTORED\n",
|
||||
"stderr": "NOTICE: relation \"idx_messages_chat_id_only\" already exists, skipping\n",
|
||||
"stdin_file": "scripts/compare_indexes.sql"
|
||||
},
|
||||
{
|
||||
"command": "docker compose exec -T postgres psql -X -v ON_ERROR_STOP=1 -U messenger -d messenger",
|
||||
"exit_code": 0,
|
||||
"stdout": "Timing is on.\nBEGIN\nTime: 0.114 ms\nDROP INDEX\nTime: 0.510 ms\nDROP INDEX\nTime: 0.255 ms\nWRITE WITHOUT SECONDARY INDEXES\n QUERY PLAN \n----------------------------------------------------------------------------------------------------------------------------------------------\n Insert on messages (cost=0.00..275.41 rows=0 width=0) (actual time=38.420..38.421 rows=0 loops=1)\n Buffers: shared hit=39651 read=676 dirtied=815 written=143\n -> Subquery Scan on \"*SELECT*\" (cost=0.00..275.41 rows=10000 width=88) (actual time=0.387..11.577 rows=10000 loops=1)\n Buffers: shared hit=1\n -> Limit (cost=0.00..150.41 rows=10000 width=80) (actual time=0.386..10.826 rows=10000 loops=1)\n Buffers: shared hit=1\n -> Nested Loop (cost=0.00..42113.50 rows=2800000 width=80) (actual time=0.385..10.225 rows=10000 loops=1)\n Buffers: shared hit=1\n -> Function Scan on generate_series (cost=0.00..100.00 rows=10000 width=0) (actual time=0.358..0.388 rows=345 loops=1)\n -> Materialize (cost=0.00..14.20 rows=280 width=32) (actual time=0.000..0.001 rows=29 loops=345)\n Buffers: shared hit=1\n -> Seq Scan on chats c (cost=0.00..12.80 rows=280 width=32) (actual time=0.009..0.013 rows=29 loops=1)\n Buffers: shared hit=1\n Planning:\n Buffers: shared hit=65\n Planning Time: 0.148 ms\n Trigger for constraint messages_chat_id_fkey: time=26.385 calls=10000\n Trigger for constraint messages_sender_id_fkey: time=45.532 calls=10000\n Execution Time: 111.483 ms\n(19 rows)\n\nTime: 112.022 ms\nROLLBACK\nTime: 0.118 ms\nBEGIN\nTime: 0.125 ms\nCREATE INDEX\nTime: 0.326 ms\nWRITE WITH EXISTING INDEXES\n QUERY PLAN \n----------------------------------------------------------------------------------------------------------------------------------------------\n Insert on messages (cost=0.00..275.41 rows=0 width=0) (actual time=48.708..48.709 rows=0 loops=1)\n Buffers: shared hit=90737 read=23 dirtied=292 written=245\n -> Subquery Scan on \"*SELECT*\" (cost=0.00..275.41 rows=10000 width=88) (actual time=0.533..11.244 rows=10000 loops=1)\n Buffers: shared hit=1\n -> Limit (cost=0.00..150.41 rows=10000 width=80) (actual time=0.532..10.510 rows=10000 loops=1)\n Buffers: shared hit=1\n -> Nested Loop (cost=0.00..42113.50 rows=2800000 width=80) (actual time=0.531..9.910 rows=10000 loops=1)\n Buffers: shared hit=1\n -> Function Scan on generate_series (cost=0.00..100.00 rows=10000 width=0) (actual time=0.497..0.525 rows=345 loops=1)\n -> Materialize (cost=0.00..14.20 rows=280 width=32) (actual time=0.000..0.001 rows=29 loops=345)\n Buffers: shared hit=1\n -> Seq Scan on chats c (cost=0.00..12.80 rows=280 width=32) (actual time=0.008..0.011 rows=29 loops=1)\n Buffers: shared hit=1\n Planning Time: 0.055 ms\n Trigger for constraint messages_chat_id_fkey: time=26.086 calls=10000\n Trigger for constraint messages_sender_id_fkey: time=41.720 calls=10000\n Execution Time: 117.756 ms\n(17 rows)\n\nTime: 118.278 ms\nROLLBACK\nTime: 0.159 ms\n",
|
||||
"stderr": "NOTICE: relation \"idx_messages_chat_id_only\" already exists, skipping\n",
|
||||
"stdin_file": "scripts/benchmark_writes.sql"
|
||||
}
|
||||
],
|
||||
"restart_before": {
|
||||
"api": "2026-10-30T10:55:12.637263918Z",
|
||||
"postgres": "2026-10-30T10:57:10.385474751Z",
|
||||
"redis": "2026-10-30T10:56:57.171946925Z"
|
||||
},
|
||||
"restart_after": {
|
||||
"api": "2026-10-30T10:57:19.664978005Z",
|
||||
"postgres": "2026-10-30T10:57:13.61408121Z",
|
||||
"redis": "2026-10-30T10:57:13.675466502Z"
|
||||
},
|
||||
"all_three_services_restarted": true,
|
||||
"backup_restore_evidence": "evidence/backup_restore.json"
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
{
|
||||
"date_utc": "20261030T105422Z-43b52d",
|
||||
"container": "lab2-audit-restore-20261030t105422z-43b52d",
|
||||
"volume": "lab2-audit-restore-20261030t105422z-43b52d-data",
|
||||
"network": "none",
|
||||
"backup": "backups/20261030T105422Z-43b52d.dump",
|
||||
"pg_restore_seconds": 0.437,
|
||||
"source_counts_and_sample_md5": [
|
||||
"chat_members|50",
|
||||
"chats|25",
|
||||
"messages|122616",
|
||||
"sessions|84",
|
||||
"users|87",
|
||||
"00003673-6e8b-4225-a3a1-e15fd328e504|b1cb97941c6ae16d504db747a8e82214",
|
||||
"000074f3-3b8e-4165-94f3-1b5e99e16a2e|9f58a18ad0e8d3707ae9ab03cbef0866",
|
||||
"0000c11d-0b41-4b07-acbf-dae56b612638|6f9ecd02b934fd283b096ebd22f38442",
|
||||
"000297c9-7c10-42c5-a642-9dc384530091|dc792e0e8909c263f626e4205a5947dd",
|
||||
"0002df01-338a-4ab0-a7c3-6f895fad06d1|ce5e5167d711d1154349cf5f93def759"
|
||||
],
|
||||
"restored_counts_and_sample_md5": [
|
||||
"chat_members|50",
|
||||
"chats|25",
|
||||
"messages|122616",
|
||||
"sessions|84",
|
||||
"users|87",
|
||||
"00003673-6e8b-4225-a3a1-e15fd328e504|b1cb97941c6ae16d504db747a8e82214",
|
||||
"000074f3-3b8e-4165-94f3-1b5e99e16a2e|9f58a18ad0e8d3707ae9ab03cbef0866",
|
||||
"0000c11d-0b41-4b07-acbf-dae56b612638|6f9ecd02b934fd283b096ebd22f38442",
|
||||
"000297c9-7c10-42c5-a642-9dc384530091|dc792e0e8909c263f626e4205a5947dd",
|
||||
"0002df01-338a-4ab0-a7c3-6f895fad06d1|ce5e5167d711d1154349cf5f93def759"
|
||||
],
|
||||
"result": "PASS",
|
||||
"limitation": "counts for all five tables and five message id/text hashes; not full RTO or full DB checksum"
|
||||
}
|
||||
@@ -0,0 +1,57 @@
|
||||
Timing is on.
|
||||
ANALYZE
|
||||
Time: 119.924 ms
|
||||
chat_id | messages_count
|
||||
--------------------------------------+----------------
|
||||
f68eff95-e44f-4c79-997c-49eb725cfb4d | 90005
|
||||
b14d0a4f-3046-44b2-871e-4ebfb752c543 | 11613
|
||||
1b7b678e-d262-4d61-8167-7fc8b736f3ff | 10005
|
||||
baabd1bb-8b74-41dd-9bcf-d801d31c160c | 5
|
||||
aeed09d0-85e4-41c2-a77b-b2c2dd6a0de3 | 5
|
||||
(5 rows)
|
||||
|
||||
Time: 9.632 ms
|
||||
Time: 8.499 ms
|
||||
TEST A: existing composite index
|
||||
QUERY PLAN
|
||||
------------------------------------------------------------------------------------------------------------------------------------------------------
|
||||
Limit (cost=0.42..5.86 rows=50 width=46) (actual time=0.007..0.014 rows=50 loops=1)
|
||||
Buffers: shared hit=5
|
||||
-> Index Scan using idx_messages_chat_created_id on messages (cost=0.42..9781.28 rows=89792 width=46) (actual time=0.006..0.011 rows=50 loops=1)
|
||||
Index Cond: (chat_id = 'f68eff95-e44f-4c79-997c-49eb725cfb4d'::uuid)
|
||||
Buffers: shared hit=5
|
||||
Planning:
|
||||
Buffers: shared hit=9
|
||||
Planning Time: 0.042 ms
|
||||
Execution Time: 0.021 ms
|
||||
(9 rows)
|
||||
|
||||
Time: 0.314 ms
|
||||
TEST B: additional index
|
||||
CREATE INDEX
|
||||
Time: 40.624 ms
|
||||
ANALYZE
|
||||
Time: 116.772 ms
|
||||
QUERY PLAN
|
||||
------------------------------------------------------------------------------------------------------------------------------------------------------
|
||||
Limit (cost=0.42..5.84 rows=50 width=46) (actual time=0.018..0.026 rows=50 loops=1)
|
||||
Buffers: shared hit=5
|
||||
-> Index Scan using idx_messages_chat_created_id on messages (cost=0.42..9765.59 rows=90101 width=46) (actual time=0.017..0.023 rows=50 loops=1)
|
||||
Index Cond: (chat_id = 'f68eff95-e44f-4c79-997c-49eb725cfb4d'::uuid)
|
||||
Buffers: shared hit=5
|
||||
Planning:
|
||||
Buffers: shared hit=14 read=1
|
||||
Planning Time: 0.125 ms
|
||||
Execution Time: 0.043 ms
|
||||
(9 rows)
|
||||
|
||||
Time: 0.372 ms
|
||||
Index sizes
|
||||
indexrelname | size
|
||||
------------------------------+---------
|
||||
idx_messages_chat_created_id | 7424 kB
|
||||
idx_messages_chat_id_only | 784 kB
|
||||
messages_pkey | 4432 kB
|
||||
(3 rows)
|
||||
|
||||
Time: 1.309 ms
|
||||
@@ -0,0 +1,60 @@
|
||||
BEGIN
|
||||
Timing is on.
|
||||
ANALYZE
|
||||
Time: 117.334 ms
|
||||
chat_id | messages_count
|
||||
--------------------------------------+----------------
|
||||
f68eff95-e44f-4c79-997c-49eb725cfb4d | 90005
|
||||
b14d0a4f-3046-44b2-871e-4ebfb752c543 | 11613
|
||||
41d9abbc-4d0b-44f7-a1a2-ea3de4faa0c3 | 10942
|
||||
1b7b678e-d262-4d61-8167-7fc8b736f3ff | 10005
|
||||
0d8c7368-f88a-4bdb-972e-e4d7e1cd437a | 5
|
||||
(5 rows)
|
||||
|
||||
Time: 10.412 ms
|
||||
Time: 8.597 ms
|
||||
TEST A: existing composite index
|
||||
QUERY PLAN
|
||||
-------------------------------------------------------------------------------------------------------------------------------------------------------
|
||||
Limit (cost=0.42..6.74 rows=50 width=45) (actual time=0.025..0.036 rows=50 loops=1)
|
||||
Buffers: shared hit=3 read=2
|
||||
-> Index Scan using idx_messages_chat_created_id on messages (cost=0.42..11395.89 rows=90150 width=45) (actual time=0.024..0.033 rows=50 loops=1)
|
||||
Index Cond: (chat_id = 'f68eff95-e44f-4c79-997c-49eb725cfb4d'::uuid)
|
||||
Buffers: shared hit=3 read=2
|
||||
Planning:
|
||||
Buffers: shared hit=10
|
||||
Planning Time: 0.069 ms
|
||||
Execution Time: 0.044 ms
|
||||
(9 rows)
|
||||
|
||||
Time: 0.360 ms
|
||||
TEST B: additional index
|
||||
CREATE INDEX
|
||||
Time: 0.285 ms
|
||||
ANALYZE
|
||||
Time: 123.698 ms
|
||||
QUERY PLAN
|
||||
-------------------------------------------------------------------------------------------------------------------------------------------------------
|
||||
Limit (cost=0.42..6.75 rows=50 width=45) (actual time=0.007..0.014 rows=50 loops=1)
|
||||
Buffers: shared hit=5
|
||||
-> Index Scan using idx_messages_chat_created_id on messages (cost=0.42..11378.89 rows=89905 width=45) (actual time=0.006..0.011 rows=50 loops=1)
|
||||
Index Cond: (chat_id = 'f68eff95-e44f-4c79-997c-49eb725cfb4d'::uuid)
|
||||
Buffers: shared hit=5
|
||||
Planning:
|
||||
Buffers: shared hit=15
|
||||
Planning Time: 0.081 ms
|
||||
Execution Time: 0.023 ms
|
||||
(9 rows)
|
||||
|
||||
Time: 0.279 ms
|
||||
Index sizes
|
||||
indexrelname | size
|
||||
------------------------------+---------
|
||||
idx_messages_chat_created_id | 9328 kB
|
||||
idx_messages_chat_id_only | 1016 kB
|
||||
messages_pkey | 5392 kB
|
||||
(3 rows)
|
||||
|
||||
Time: 1.366 ms
|
||||
ROLLBACK
|
||||
Time: 0.119 ms
|
||||
@@ -0,0 +1,49 @@
|
||||
Timing is on.
|
||||
Time: 10.952 ms
|
||||
COMPOSITE INDEX
|
||||
QUERY PLAN
|
||||
-------------------------------------------------------------------------------------------------------------------------------------------------------
|
||||
Limit (cost=0.42..5.93 rows=50 width=46) (actual time=0.028..0.041 rows=50 loops=1)
|
||||
Buffers: shared hit=1 read=4
|
||||
-> Index Scan using idx_messages_chat_created_id on messages (cost=0.42..10839.86 rows=98331 width=46) (actual time=0.027..0.037 rows=50 loops=1)
|
||||
Index Cond: (chat_id = 'f68eff95-e44f-4c79-997c-49eb725cfb4d'::uuid)
|
||||
Buffers: shared hit=1 read=4
|
||||
Planning:
|
||||
Buffers: shared hit=14
|
||||
Planning Time: 0.060 ms
|
||||
Execution Time: 0.048 ms
|
||||
(9 rows)
|
||||
|
||||
Time: 0.417 ms
|
||||
BEGIN
|
||||
Time: 0.106 ms
|
||||
DROP INDEX
|
||||
Time: 0.657 ms
|
||||
SINGLE COLUMN INDEX
|
||||
QUERY PLAN
|
||||
-----------------------------------------------------------------------------------------------------------------------------------------
|
||||
Limit (cost=5430.38..5436.13 rows=50 width=46) (actual time=12.333..13.504 rows=50 loops=1)
|
||||
Buffers: shared hit=182 read=1475
|
||||
-> Gather Merge (cost=5430.38..12082.21 rows=57842 width=46) (actual time=12.331..13.500 rows=50 loops=1)
|
||||
Workers Planned: 1
|
||||
Workers Launched: 1
|
||||
Buffers: shared hit=182 read=1475
|
||||
-> Sort (cost=4430.37..4574.98 rows=57842 width=46) (actual time=11.217..11.219 rows=50 loops=2)
|
||||
Sort Key: created_at, id
|
||||
Sort Method: top-N heapsort Memory: 28kB
|
||||
Buffers: shared hit=182 read=1475
|
||||
Worker 0: Sort Method: top-N heapsort Memory: 28kB
|
||||
-> Parallel Seq Scan on messages (cost=0.00..2508.90 rows=57842 width=46) (actual time=0.009..7.022 rows=45002 loops=2)
|
||||
Filter: (chat_id = 'f68eff95-e44f-4c79-997c-49eb725cfb4d'::uuid)
|
||||
Rows Removed by Filter: 16293
|
||||
Buffers: shared hit=138 read=1475
|
||||
Planning:
|
||||
Buffers: shared hit=4
|
||||
Planning Time: 0.052 ms
|
||||
Execution Time: 13.525 ms
|
||||
(19 rows)
|
||||
|
||||
Time: 13.780 ms
|
||||
ROLLBACK
|
||||
Time: 0.088 ms
|
||||
ORIGINAL INDEX RESTORED
|
||||
@@ -0,0 +1,51 @@
|
||||
Timing is on.
|
||||
Time: 8.859 ms
|
||||
COMPOSITE INDEX
|
||||
QUERY PLAN
|
||||
-------------------------------------------------------------------------------------------------------------------------------------------------------
|
||||
Limit (cost=0.42..6.06 rows=50 width=46) (actual time=0.028..0.039 rows=50 loops=1)
|
||||
Buffers: shared hit=5
|
||||
-> Index Scan using idx_messages_chat_created_id on messages (cost=0.42..11162.90 rows=98962 width=46) (actual time=0.027..0.036 rows=50 loops=1)
|
||||
Index Cond: (chat_id = 'f68eff95-e44f-4c79-997c-49eb725cfb4d'::uuid)
|
||||
Buffers: shared hit=5
|
||||
Planning:
|
||||
Buffers: shared hit=14
|
||||
Planning Time: 0.097 ms
|
||||
Execution Time: 0.048 ms
|
||||
(9 rows)
|
||||
|
||||
Time: 0.507 ms
|
||||
BEGIN
|
||||
Time: 0.083 ms
|
||||
CREATE INDEX
|
||||
Time: 0.205 ms
|
||||
DROP INDEX
|
||||
Time: 0.680 ms
|
||||
SINGLE COLUMN INDEX
|
||||
QUERY PLAN
|
||||
-----------------------------------------------------------------------------------------------------------------------------------------
|
||||
Limit (cost=5448.46..5454.21 rows=50 width=46) (actual time=13.704..14.946 rows=50 loops=1)
|
||||
Buffers: shared hit=114 read=1543
|
||||
-> Gather Merge (cost=5448.46..12142.95 rows=58213 width=46) (actual time=13.703..14.938 rows=50 loops=1)
|
||||
Workers Planned: 1
|
||||
Workers Launched: 1
|
||||
Buffers: shared hit=114 read=1543
|
||||
-> Sort (cost=4448.45..4593.98 rows=58213 width=46) (actual time=12.461..12.463 rows=50 loops=2)
|
||||
Sort Key: created_at, id
|
||||
Sort Method: top-N heapsort Memory: 28kB
|
||||
Buffers: shared hit=114 read=1543
|
||||
Worker 0: Sort Method: top-N heapsort Memory: 28kB
|
||||
-> Parallel Seq Scan on messages (cost=0.00..2514.65 rows=58213 width=46) (actual time=0.012..8.363 rows=45002 loops=2)
|
||||
Filter: (chat_id = 'f68eff95-e44f-4c79-997c-49eb725cfb4d'::uuid)
|
||||
Rows Removed by Filter: 16310
|
||||
Buffers: shared hit=70 read=1543
|
||||
Planning:
|
||||
Buffers: shared hit=4
|
||||
Planning Time: 0.038 ms
|
||||
Execution Time: 14.977 ms
|
||||
(19 rows)
|
||||
|
||||
Time: 15.268 ms
|
||||
ROLLBACK
|
||||
Time: 0.138 ms
|
||||
ORIGINAL INDEX RESTORED
|
||||
@@ -0,0 +1,6 @@
|
||||
indexrelname | index_size | idx_scan
|
||||
------------------------------+------------+----------
|
||||
idx_messages_chat_created_id | 8552 kB | 26
|
||||
idx_messages_chat_id_only | 912 kB | 2
|
||||
messages_pkey | 4608 kB | 9
|
||||
(3 rows)
|
||||
@@ -0,0 +1,18 @@
|
||||
python3 tests/load.py --base-url "http://localhost:8080"
|
||||
{
|
||||
"lab": 2,
|
||||
"workload": "closed-loop POST messages; no automatic retries",
|
||||
"requested_duration_seconds": 30,
|
||||
"elapsed_seconds": 30.021,
|
||||
"concurrency": 4,
|
||||
"successful_requests": 10942,
|
||||
"failed_requests": 0,
|
||||
"error_ratio": 0.0,
|
||||
"successful_rps": 364.483,
|
||||
"successful_latency_ms": {
|
||||
"p50": 10.341,
|
||||
"p95": 16.404,
|
||||
"p99": 21.772
|
||||
},
|
||||
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
BEGIN
|
||||
SET
|
||||
PREPARE
|
||||
QUERY PLAN
|
||||
-----------------------------------------------------------------------------------------------------------------------------------------------------
|
||||
Limit (cost=0.42..582.96 rows=584 width=78) (actual time=0.020..0.032 rows=51 loops=1)
|
||||
Buffers: shared hit=2 read=3
|
||||
-> Index Scan using idx_messages_chat_created_id on messages (cost=0.42..5820.85 rows=5835 width=78) (actual time=0.019..0.028 rows=51 loops=1)
|
||||
Index Cond: ((chat_id = $1) AND (ROW(created_at, id) > ROW($2, $3)))
|
||||
Buffers: shared hit=2 read=3
|
||||
Planning:
|
||||
Buffers: shared hit=6
|
||||
Planning Time: 0.128 ms
|
||||
Execution Time: 0.054 ms
|
||||
(9 rows)
|
||||
|
||||
ROLLBACK
|
||||
@@ -0,0 +1,61 @@
|
||||
Timing is on.
|
||||
BEGIN
|
||||
Time: 0.100 ms
|
||||
DROP INDEX
|
||||
Time: 0.695 ms
|
||||
DROP INDEX
|
||||
Time: 0.189 ms
|
||||
WRITE WITHOUT SECONDARY INDEXES
|
||||
QUERY PLAN
|
||||
----------------------------------------------------------------------------------------------------------------------------------------------
|
||||
Insert on messages (cost=0.00..275.41 rows=0 width=0) (actual time=30.658..30.659 rows=0 loops=1)
|
||||
Buffers: shared hit=39804 read=579 dirtied=749 written=174
|
||||
-> Subquery Scan on "*SELECT*" (cost=0.00..275.41 rows=10000 width=88) (actual time=0.339..10.740 rows=10000 loops=1)
|
||||
Buffers: shared hit=1
|
||||
-> Limit (cost=0.00..150.41 rows=10000 width=80) (actual time=0.338..9.999 rows=10000 loops=1)
|
||||
Buffers: shared hit=1
|
||||
-> Nested Loop (cost=0.00..42113.50 rows=2800000 width=80) (actual time=0.337..9.452 rows=10000 loops=1)
|
||||
Buffers: shared hit=1
|
||||
-> Function Scan on generate_series (cost=0.00..100.00 rows=10000 width=0) (actual time=0.322..0.365 rows=667 loops=1)
|
||||
-> Materialize (cost=0.00..14.20 rows=280 width=32) (actual time=0.000..0.001 rows=15 loops=667)
|
||||
Buffers: shared hit=1
|
||||
-> Seq Scan on chats c (cost=0.00..12.80 rows=280 width=32) (actual time=0.004..0.006 rows=15 loops=1)
|
||||
Buffers: shared hit=1
|
||||
Planning:
|
||||
Buffers: shared hit=65
|
||||
Planning Time: 0.146 ms
|
||||
Trigger for constraint messages_chat_id_fkey: time=25.322 calls=10000
|
||||
Trigger for constraint messages_sender_id_fkey: time=32.144 calls=10000
|
||||
Execution Time: 88.932 ms
|
||||
(19 rows)
|
||||
|
||||
Time: 89.644 ms
|
||||
ROLLBACK
|
||||
Time: 0.131 ms
|
||||
BEGIN
|
||||
Time: 0.082 ms
|
||||
WRITE WITH EXISTING INDEXES
|
||||
QUERY PLAN
|
||||
----------------------------------------------------------------------------------------------------------------------------------------------
|
||||
Insert on messages (cost=0.00..275.41 rows=0 width=0) (actual time=50.655..50.656 rows=0 loops=1)
|
||||
Buffers: shared hit=90871 read=26 dirtied=348 written=320
|
||||
-> Subquery Scan on "*SELECT*" (cost=0.00..275.41 rows=10000 width=88) (actual time=0.446..11.088 rows=10000 loops=1)
|
||||
Buffers: shared hit=1
|
||||
-> Limit (cost=0.00..150.41 rows=10000 width=80) (actual time=0.445..10.291 rows=10000 loops=1)
|
||||
Buffers: shared hit=1
|
||||
-> Nested Loop (cost=0.00..42113.50 rows=2800000 width=80) (actual time=0.444..9.751 rows=10000 loops=1)
|
||||
Buffers: shared hit=1
|
||||
-> Function Scan on generate_series (cost=0.00..100.00 rows=10000 width=0) (actual time=0.410..0.455 rows=667 loops=1)
|
||||
-> Materialize (cost=0.00..14.20 rows=280 width=32) (actual time=0.000..0.001 rows=15 loops=667)
|
||||
Buffers: shared hit=1
|
||||
-> Seq Scan on chats c (cost=0.00..12.80 rows=280 width=32) (actual time=0.005..0.008 rows=15 loops=1)
|
||||
Buffers: shared hit=1
|
||||
Planning Time: 0.077 ms
|
||||
Trigger for constraint messages_chat_id_fkey: time=25.106 calls=10000
|
||||
Trigger for constraint messages_sender_id_fkey: time=31.329 calls=10000
|
||||
Execution Time: 107.873 ms
|
||||
(17 rows)
|
||||
|
||||
Time: 108.260 ms
|
||||
ROLLBACK
|
||||
Time: 0.109 ms
|
||||
@@ -0,0 +1,63 @@
|
||||
Timing is on.
|
||||
BEGIN
|
||||
Time: 0.114 ms
|
||||
DROP INDEX
|
||||
Time: 0.510 ms
|
||||
DROP INDEX
|
||||
Time: 0.255 ms
|
||||
WRITE WITHOUT SECONDARY INDEXES
|
||||
QUERY PLAN
|
||||
----------------------------------------------------------------------------------------------------------------------------------------------
|
||||
Insert on messages (cost=0.00..275.41 rows=0 width=0) (actual time=38.420..38.421 rows=0 loops=1)
|
||||
Buffers: shared hit=39651 read=676 dirtied=815 written=143
|
||||
-> Subquery Scan on "*SELECT*" (cost=0.00..275.41 rows=10000 width=88) (actual time=0.387..11.577 rows=10000 loops=1)
|
||||
Buffers: shared hit=1
|
||||
-> Limit (cost=0.00..150.41 rows=10000 width=80) (actual time=0.386..10.826 rows=10000 loops=1)
|
||||
Buffers: shared hit=1
|
||||
-> Nested Loop (cost=0.00..42113.50 rows=2800000 width=80) (actual time=0.385..10.225 rows=10000 loops=1)
|
||||
Buffers: shared hit=1
|
||||
-> Function Scan on generate_series (cost=0.00..100.00 rows=10000 width=0) (actual time=0.358..0.388 rows=345 loops=1)
|
||||
-> Materialize (cost=0.00..14.20 rows=280 width=32) (actual time=0.000..0.001 rows=29 loops=345)
|
||||
Buffers: shared hit=1
|
||||
-> Seq Scan on chats c (cost=0.00..12.80 rows=280 width=32) (actual time=0.009..0.013 rows=29 loops=1)
|
||||
Buffers: shared hit=1
|
||||
Planning:
|
||||
Buffers: shared hit=65
|
||||
Planning Time: 0.148 ms
|
||||
Trigger for constraint messages_chat_id_fkey: time=26.385 calls=10000
|
||||
Trigger for constraint messages_sender_id_fkey: time=45.532 calls=10000
|
||||
Execution Time: 111.483 ms
|
||||
(19 rows)
|
||||
|
||||
Time: 112.022 ms
|
||||
ROLLBACK
|
||||
Time: 0.118 ms
|
||||
BEGIN
|
||||
Time: 0.125 ms
|
||||
CREATE INDEX
|
||||
Time: 0.326 ms
|
||||
WRITE WITH EXISTING INDEXES
|
||||
QUERY PLAN
|
||||
----------------------------------------------------------------------------------------------------------------------------------------------
|
||||
Insert on messages (cost=0.00..275.41 rows=0 width=0) (actual time=48.708..48.709 rows=0 loops=1)
|
||||
Buffers: shared hit=90737 read=23 dirtied=292 written=245
|
||||
-> Subquery Scan on "*SELECT*" (cost=0.00..275.41 rows=10000 width=88) (actual time=0.533..11.244 rows=10000 loops=1)
|
||||
Buffers: shared hit=1
|
||||
-> Limit (cost=0.00..150.41 rows=10000 width=80) (actual time=0.532..10.510 rows=10000 loops=1)
|
||||
Buffers: shared hit=1
|
||||
-> Nested Loop (cost=0.00..42113.50 rows=2800000 width=80) (actual time=0.531..9.910 rows=10000 loops=1)
|
||||
Buffers: shared hit=1
|
||||
-> Function Scan on generate_series (cost=0.00..100.00 rows=10000 width=0) (actual time=0.497..0.525 rows=345 loops=1)
|
||||
-> Materialize (cost=0.00..14.20 rows=280 width=32) (actual time=0.000..0.001 rows=29 loops=345)
|
||||
Buffers: shared hit=1
|
||||
-> Seq Scan on chats c (cost=0.00..12.80 rows=280 width=32) (actual time=0.008..0.011 rows=29 loops=1)
|
||||
Buffers: shared hit=1
|
||||
Planning Time: 0.055 ms
|
||||
Trigger for constraint messages_chat_id_fkey: time=26.086 calls=10000
|
||||
Trigger for constraint messages_sender_id_fkey: time=41.720 calls=10000
|
||||
Execution Time: 117.756 ms
|
||||
(17 rows)
|
||||
|
||||
Time: 118.278 ms
|
||||
ROLLBACK
|
||||
Time: 0.159 ms
|
||||
@@ -0,0 +1,7 @@
|
||||
fastapi==0.115.12
|
||||
uvicorn==0.34.2
|
||||
pydantic==2.11.4
|
||||
psycopg[binary]==3.2.6
|
||||
psycopg-pool==3.2.6
|
||||
redis==5.2.1
|
||||
argon2-cffi==23.1.0
|
||||
@@ -0,0 +1,59 @@
|
||||
\set ON_ERROR_STOP on
|
||||
BEGIN;
|
||||
\timing on
|
||||
|
||||
ANALYZE messages;
|
||||
|
||||
SELECT
|
||||
chat_id,
|
||||
COUNT(*) AS messages_count
|
||||
FROM messages
|
||||
GROUP BY chat_id
|
||||
ORDER BY messages_count DESC
|
||||
LIMIT 5;
|
||||
|
||||
SELECT id AS benchmark_chat_id
|
||||
FROM chats
|
||||
WHERE id IN (
|
||||
SELECT chat_id
|
||||
FROM messages
|
||||
GROUP BY chat_id
|
||||
ORDER BY COUNT(*) DESC
|
||||
LIMIT 1
|
||||
)
|
||||
LIMIT 1
|
||||
\gset
|
||||
|
||||
\echo 'TEST A: existing composite index'
|
||||
|
||||
EXPLAIN (ANALYZE, BUFFERS)
|
||||
SELECT id, text, created_at
|
||||
FROM messages
|
||||
WHERE chat_id = :'benchmark_chat_id'
|
||||
ORDER BY created_at, id
|
||||
LIMIT 50;
|
||||
|
||||
\echo 'TEST B: additional index'
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_messages_chat_id_only
|
||||
ON messages (chat_id);
|
||||
|
||||
ANALYZE messages;
|
||||
|
||||
EXPLAIN (ANALYZE, BUFFERS)
|
||||
SELECT id, text, created_at
|
||||
FROM messages
|
||||
WHERE chat_id = :'benchmark_chat_id'
|
||||
ORDER BY created_at, id
|
||||
LIMIT 50;
|
||||
|
||||
\echo 'Index sizes'
|
||||
|
||||
SELECT
|
||||
indexrelname,
|
||||
pg_size_pretty(pg_relation_size(indexrelid)) AS size
|
||||
FROM pg_stat_user_indexes
|
||||
WHERE relname = 'messages'
|
||||
ORDER BY indexrelname;
|
||||
|
||||
ROLLBACK;
|
||||
@@ -0,0 +1,40 @@
|
||||
\set ON_ERROR_STOP on
|
||||
\timing on
|
||||
|
||||
BEGIN;
|
||||
|
||||
DROP INDEX IF EXISTS idx_messages_chat_created_id;
|
||||
DROP INDEX IF EXISTS idx_messages_chat_id_only;
|
||||
|
||||
\echo 'WRITE WITHOUT SECONDARY INDEXES'
|
||||
|
||||
EXPLAIN (ANALYZE, BUFFERS)
|
||||
INSERT INTO messages (id, chat_id, sender_id, text)
|
||||
SELECT
|
||||
gen_random_uuid(),
|
||||
c.id,
|
||||
c.created_by,
|
||||
'Write benchmark without index'
|
||||
FROM chats c
|
||||
CROSS JOIN generate_series(1, 10000)
|
||||
LIMIT 10000;
|
||||
|
||||
ROLLBACK;
|
||||
|
||||
BEGIN;
|
||||
CREATE INDEX IF NOT EXISTS idx_messages_chat_id_only ON messages (chat_id);
|
||||
|
||||
\echo 'WRITE WITH EXISTING INDEXES'
|
||||
|
||||
EXPLAIN (ANALYZE, BUFFERS)
|
||||
INSERT INTO messages (id, chat_id, sender_id, text)
|
||||
SELECT
|
||||
gen_random_uuid(),
|
||||
c.id,
|
||||
c.created_by,
|
||||
'Write benchmark with index'
|
||||
FROM chats c
|
||||
CROSS JOIN generate_series(1, 10000)
|
||||
LIMIT 10000;
|
||||
|
||||
ROLLBACK;
|
||||
@@ -0,0 +1,120 @@
|
||||
import base64
|
||||
import concurrent.futures
|
||||
import json
|
||||
import secrets
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
BASE = "http://" + "127.0.0.1:8080"
|
||||
PASSWORD = "StrongPassword123!"
|
||||
|
||||
|
||||
def call(method, path, data=None, authorization=None):
|
||||
headers = {}
|
||||
if data is not None:
|
||||
headers["Content-Type"] = "application/json"
|
||||
if authorization:
|
||||
headers["Authorization"] = authorization
|
||||
|
||||
req = urllib.request.Request(
|
||||
BASE + path,
|
||||
data=json.dumps(data).encode() if data is not None else None,
|
||||
headers=headers,
|
||||
method=method,
|
||||
)
|
||||
|
||||
try:
|
||||
response = urllib.request.urlopen(req, timeout=15)
|
||||
except urllib.error.HTTPError as exc:
|
||||
response = exc
|
||||
|
||||
with response:
|
||||
raw = response.read()
|
||||
return response.status, json.loads(raw) if raw else None
|
||||
|
||||
|
||||
def register(username):
|
||||
return call("POST", "/api/v1/users", {
|
||||
"username": username,
|
||||
"display_name": "Concurrent User",
|
||||
"password": PASSWORD,
|
||||
})[0]
|
||||
|
||||
|
||||
print("1. Concurrent registration")
|
||||
|
||||
username = "race_" + secrets.token_hex(5)
|
||||
|
||||
with concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor:
|
||||
statuses = list(executor.map(register, [username] * 10))
|
||||
|
||||
assert statuses.count(201) == 1, statuses
|
||||
assert statuses.count(409) == 9, statuses
|
||||
print("PASS: one registration, nine conflicts")
|
||||
|
||||
|
||||
print("2. Incorrect password")
|
||||
|
||||
credentials = base64.b64encode(
|
||||
(username + ":WrongPassword123!").encode()
|
||||
).decode()
|
||||
|
||||
status, _ = call(
|
||||
"POST",
|
||||
"/api/v1/auth/sessions",
|
||||
authorization="Basic " + credentials,
|
||||
)
|
||||
|
||||
assert status == 401, status
|
||||
print("PASS: incorrect password rejected")
|
||||
|
||||
|
||||
print("3. Invalid Bearer token")
|
||||
|
||||
status, _ = call(
|
||||
"GET",
|
||||
"/api/v1/users/me",
|
||||
authorization="Bearer invalid-token",
|
||||
)
|
||||
|
||||
assert status == 401, status
|
||||
print("PASS: invalid token rejected")
|
||||
for bad_token in ("é", "abc def", "", "x" * 513, "abc\tdef"):
|
||||
status, _ = call("GET", "/api/v1/users/me", authorization="Bearer " + bad_token)
|
||||
assert status == 401, (repr(bad_token), status)
|
||||
print("PASS: malformed and non-ASCII Bearer tokens rejected")
|
||||
|
||||
|
||||
print("4. Unknown chat member")
|
||||
|
||||
credentials = base64.b64encode(
|
||||
(username + ":" + PASSWORD).encode()
|
||||
).decode()
|
||||
|
||||
status, result = call(
|
||||
"POST",
|
||||
"/api/v1/auth/sessions",
|
||||
authorization="Basic " + credentials,
|
||||
)
|
||||
|
||||
assert status == 201, status
|
||||
token = result["token"]
|
||||
user_id = result["user"]["id"]
|
||||
|
||||
import uuid
|
||||
|
||||
status, _ = call(
|
||||
"POST",
|
||||
"/api/v1/chats",
|
||||
{
|
||||
"title": "Rollback test",
|
||||
"member_ids": [user_id, str(uuid.uuid4())],
|
||||
},
|
||||
authorization="Bearer " + token,
|
||||
)
|
||||
|
||||
assert status == 400, status
|
||||
print("PASS: unknown member rejected")
|
||||
|
||||
|
||||
print("\nALL ADVANCED API CHECKS PASSED")
|
||||
@@ -0,0 +1,149 @@
|
||||
import base64
|
||||
import json
|
||||
import secrets
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
BASE_URL = "http://" + "127.0.0.1:8080"
|
||||
|
||||
username = "test_" + secrets.token_hex(5)
|
||||
password = "StrongPassword123!"
|
||||
|
||||
def request(method, path, body=None, authorization=None):
|
||||
headers = {}
|
||||
|
||||
if body is not None:
|
||||
headers["Content-Type"] = "application/json"
|
||||
body = json.dumps(body).encode("utf-8")
|
||||
|
||||
if authorization:
|
||||
headers["Authorization"] = authorization
|
||||
|
||||
req = urllib.request.Request(
|
||||
BASE_URL + path,
|
||||
data=body,
|
||||
headers=headers,
|
||||
method=method,
|
||||
)
|
||||
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=10) as response:
|
||||
data = response.read()
|
||||
return response.status, dict(response.headers), (
|
||||
json.loads(data) if data else None
|
||||
)
|
||||
except urllib.error.HTTPError as error:
|
||||
data = error.read()
|
||||
return error.code, dict(error.headers), (
|
||||
json.loads(data) if data else None
|
||||
)
|
||||
|
||||
|
||||
def check(name, actual, expected):
|
||||
result = "PASS" if actual == expected else "FAIL"
|
||||
print(f"{result}: {name}: {actual} (expected {expected})")
|
||||
if actual != expected:
|
||||
raise SystemExit(1)
|
||||
|
||||
|
||||
print("1. Registration")
|
||||
|
||||
status, headers, user = request(
|
||||
"POST",
|
||||
"/api/v1/users",
|
||||
{
|
||||
"username": username,
|
||||
"display_name": "Test User",
|
||||
"password": password,
|
||||
},
|
||||
)
|
||||
print("Registration status:", status)
|
||||
print("Registration headers:", headers)
|
||||
print("Registration body:", user)
|
||||
check("registration", status, 201)
|
||||
|
||||
assert "password" not in user
|
||||
assert "password_hash" not in user
|
||||
assert "X-Request-Id" in headers
|
||||
|
||||
print("User ID:", user["id"])
|
||||
|
||||
|
||||
print("\n2. Basic login")
|
||||
|
||||
credentials = base64.b64encode(
|
||||
f"{username}:{password}".encode()
|
||||
).decode()
|
||||
|
||||
status, headers, session = request(
|
||||
"POST",
|
||||
"/api/v1/auth/sessions",
|
||||
authorization="Basic " + credentials,
|
||||
)
|
||||
check("login", status, 201)
|
||||
|
||||
token = session["token"]
|
||||
assert session["token_type"] == "Bearer"
|
||||
assert len(token) >= 40
|
||||
|
||||
|
||||
print("\n3. Access with Bearer token")
|
||||
|
||||
status, headers, me = request(
|
||||
"GET",
|
||||
"/api/v1/users/me",
|
||||
authorization="Bearer " + token,
|
||||
)
|
||||
check("authorized request", status, 200)
|
||||
assert me["id"] == user["id"]
|
||||
|
||||
|
||||
print("\n4. Access without token")
|
||||
|
||||
status, headers, body = request(
|
||||
"GET",
|
||||
"/api/v1/users/me",
|
||||
)
|
||||
check("missing token", status, 401)
|
||||
assert any(
|
||||
key.lower() == "www-authenticate"
|
||||
and value == 'Bearer realm="messenger"'
|
||||
for key, value in headers.items()
|
||||
), f"Неверный WWW-Authenticate: {headers}"
|
||||
|
||||
|
||||
print("\n5. X-User-Id cannot authorize")
|
||||
|
||||
req = urllib.request.Request(
|
||||
BASE_URL + "/api/v1/users/me",
|
||||
headers={"X-User-Id": user["id"]},
|
||||
)
|
||||
|
||||
try:
|
||||
urllib.request.urlopen(req, timeout=10)
|
||||
raise AssertionError("X-User-Id unexpectedly authorized")
|
||||
except urllib.error.HTTPError as error:
|
||||
check("X-User-Id rejected", error.code, 401)
|
||||
|
||||
|
||||
print("\n6. Logout")
|
||||
|
||||
status, headers, body = request(
|
||||
"DELETE",
|
||||
"/api/v1/auth/sessions/current",
|
||||
authorization="Bearer " + token,
|
||||
)
|
||||
check("logout", status, 204)
|
||||
|
||||
|
||||
print("\n7. Reuse revoked token")
|
||||
|
||||
status, headers, body = request(
|
||||
"GET",
|
||||
"/api/v1/users/me",
|
||||
authorization="Bearer " + token,
|
||||
)
|
||||
check("revoked token rejected", status, 401)
|
||||
|
||||
|
||||
print("\nALL AUTH CHECKS PASSED")
|
||||
@@ -0,0 +1,81 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Backup/restore в новый изолированный контейнер и новый volume.
|
||||
|
||||
Запускать без параллельных писателей. Backup остаётся в backups/ (игнорируется
|
||||
Git), контейнер после проверки останавливается, volume сохраняется.
|
||||
"""
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
import json
|
||||
import subprocess
|
||||
import time
|
||||
from uuid import uuid4
|
||||
|
||||
|
||||
def run(args, **kwargs):
|
||||
return subprocess.run(args, check=True, **kwargs)
|
||||
|
||||
|
||||
def main():
|
||||
suffix = datetime.now(timezone.utc).strftime('%Y%m%dT%H%M%SZ') + '-' + uuid4().hex[:6]
|
||||
container = 'lab2-audit-restore-' + suffix.lower()
|
||||
volume = container + '-data'
|
||||
source = ['docker', 'compose', 'exec', '-T', 'postgres']
|
||||
target = ['docker', 'exec', '-i', container]
|
||||
query = """SELECT 'users', count(*) FROM users
|
||||
UNION ALL SELECT 'chats', count(*) FROM chats
|
||||
UNION ALL SELECT 'chat_members', count(*) FROM chat_members
|
||||
UNION ALL SELECT 'messages', count(*) FROM messages
|
||||
UNION ALL SELECT 'sessions', count(*) FROM sessions ORDER BY 1;
|
||||
SELECT id, md5(text) FROM messages ORDER BY id LIMIT 5;"""
|
||||
|
||||
def summary(prefix):
|
||||
return run(prefix + ['psql', '-X', '-v', 'ON_ERROR_STOP=1', '-U', 'messenger',
|
||||
'-d', 'messenger', '-Atc', query],
|
||||
capture_output=True, text=True).stdout
|
||||
|
||||
before = summary(source)
|
||||
Path('backups').mkdir(exist_ok=True)
|
||||
backup = Path('backups') / (suffix + '.dump')
|
||||
with backup.open('xb') as out:
|
||||
backup.chmod(0o600)
|
||||
run(source + ['pg_dump', '-U', 'messenger', '-d', 'messenger', '-Fc'], stdout=out)
|
||||
run(['docker', 'run', '-d', '--name', container, '--network', 'none',
|
||||
'--mount', 'type=volume,src=' + volume + ',dst=/var/lib/postgresql/data',
|
||||
'-e', 'POSTGRES_HOST_AUTH_METHOD=trust', '-e', 'POSTGRES_USER=messenger',
|
||||
'-e', 'POSTGRES_DB=messenger', 'postgres:16.8-bookworm'], capture_output=True)
|
||||
try:
|
||||
for _ in range(60):
|
||||
ready = subprocess.run(target + ['pg_isready', '-U', 'messenger', '-d', 'messenger'],
|
||||
capture_output=True)
|
||||
if ready.returncode == 0:
|
||||
break
|
||||
time.sleep(1)
|
||||
else:
|
||||
raise AssertionError('restore PostgreSQL did not start')
|
||||
start = time.monotonic()
|
||||
with backup.open('rb') as src:
|
||||
run(target + ['pg_restore', '--exit-on-error', '-U', 'messenger',
|
||||
'-d', 'messenger'], stdin=src)
|
||||
elapsed = time.monotonic() - start
|
||||
restored = summary(target)
|
||||
after = summary(source)
|
||||
assert before == after, 'source changed during backup; retry without writers'
|
||||
assert before == restored, 'restored counts or samples differ'
|
||||
evidence = {
|
||||
'date_utc': suffix, 'container': container, 'volume': volume,
|
||||
'network': 'none', 'backup': str(backup),
|
||||
'pg_restore_seconds': round(elapsed, 3),
|
||||
'source_counts_and_sample_md5': before.splitlines(),
|
||||
'restored_counts_and_sample_md5': restored.splitlines(),
|
||||
'result': 'PASS',
|
||||
'limitation': 'counts for all five tables and five message id/text hashes; not full RTO or full DB checksum',
|
||||
}
|
||||
Path('evidence/backup_restore.json').write_text(json.dumps(evidence, indent=2) + '\n')
|
||||
print(json.dumps(evidence, indent=2))
|
||||
finally:
|
||||
run(['docker', 'stop', container], capture_output=True)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,99 @@
|
||||
import base64
|
||||
import json
|
||||
import secrets
|
||||
import subprocess
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
BASE = "http://" + "127.0.0.1:8080"
|
||||
username = "failure_" + secrets.token_hex(4)
|
||||
password = "StrongPassword123!"
|
||||
|
||||
|
||||
def request(method, path, body=None, authorization=None):
|
||||
headers = {}
|
||||
if body is not None:
|
||||
headers["Content-Type"] = "application/json"
|
||||
if authorization:
|
||||
headers["Authorization"] = authorization
|
||||
|
||||
req = urllib.request.Request(
|
||||
BASE + path,
|
||||
data=json.dumps(body).encode() if body is not None else None,
|
||||
headers=headers,
|
||||
method=method,
|
||||
)
|
||||
|
||||
try:
|
||||
response = urllib.request.urlopen(req, timeout=20)
|
||||
except urllib.error.HTTPError as exc:
|
||||
response = exc
|
||||
|
||||
with response:
|
||||
raw = response.read()
|
||||
return response.status, json.loads(raw) if raw else None
|
||||
|
||||
|
||||
status, _ = request("POST", "/api/v1/users", {
|
||||
"username": username,
|
||||
"display_name": "Failure Test",
|
||||
"password": password,
|
||||
})
|
||||
assert status == 201, status
|
||||
|
||||
basic = base64.b64encode(
|
||||
(username + ":" + password).encode()
|
||||
).decode()
|
||||
|
||||
status, result = request(
|
||||
"POST",
|
||||
"/api/v1/auth/sessions",
|
||||
authorization="Basic " + basic,
|
||||
)
|
||||
assert status == 201, status
|
||||
|
||||
bearer = "Bearer " + result["token"]
|
||||
|
||||
status, _ = request(
|
||||
"GET", "/api/v1/users/me", authorization=bearer
|
||||
)
|
||||
assert status == 200, status
|
||||
|
||||
print("PASS: token works before database failure")
|
||||
|
||||
try:
|
||||
subprocess.run(
|
||||
["docker", "compose", "stop", "postgres"],
|
||||
check=True,
|
||||
)
|
||||
|
||||
time.sleep(2)
|
||||
|
||||
status, _ = request(
|
||||
"GET", "/api/v1/users/me", authorization=bearer
|
||||
)
|
||||
|
||||
assert status == 503, status
|
||||
print("PASS: database failure does not grant access:", status)
|
||||
|
||||
status, _ = request("GET", "/health/ready")
|
||||
assert status == 503, status
|
||||
|
||||
finally:
|
||||
subprocess.run(
|
||||
["docker", "compose", "start", "postgres"],
|
||||
check=True,
|
||||
)
|
||||
|
||||
for _ in range(30):
|
||||
try:
|
||||
status, _ = request("GET", "/api/v1/users/me", authorization=bearer)
|
||||
if status == 200:
|
||||
break
|
||||
except (OSError, TimeoutError):
|
||||
pass
|
||||
time.sleep(1)
|
||||
else:
|
||||
raise AssertionError("PostgreSQL did not recover")
|
||||
print("ALL DATABASE FAILURE CHECKS PASSED; original session recovered")
|
||||
@@ -0,0 +1,179 @@
|
||||
import base64
|
||||
import json
|
||||
import secrets
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
BASE_URL = "http://" + "127.0.0.1:8080"
|
||||
|
||||
|
||||
def request(method, path, body=None, token=None, basic=None):
|
||||
headers = {}
|
||||
|
||||
if body is not None:
|
||||
headers["Content-Type"] = "application/json"
|
||||
|
||||
if token is not None:
|
||||
headers["Authorization"] = "Bearer " + token
|
||||
|
||||
if basic is not None:
|
||||
credentials = base64.b64encode(
|
||||
basic.encode("utf-8")
|
||||
).decode("ascii")
|
||||
headers["Authorization"] = "Basic " + credentials
|
||||
|
||||
req = urllib.request.Request(
|
||||
BASE_URL + path,
|
||||
data=json.dumps(body).encode("utf-8") if body is not None else None,
|
||||
headers=headers,
|
||||
method=method,
|
||||
)
|
||||
|
||||
try:
|
||||
response = urllib.request.urlopen(req, timeout=15)
|
||||
except urllib.error.HTTPError as error:
|
||||
response = error
|
||||
|
||||
with response:
|
||||
raw = response.read()
|
||||
data = json.loads(raw) if raw else None
|
||||
return response.status, data
|
||||
|
||||
|
||||
def check(label, actual, expected):
|
||||
if actual != expected:
|
||||
raise AssertionError(
|
||||
f"{label}: expected {expected}, got {actual}"
|
||||
)
|
||||
print(f"PASS: {label}: {actual}")
|
||||
|
||||
|
||||
password = "StrongPassword123!"
|
||||
suffix = secrets.token_hex(4)
|
||||
|
||||
print("\n1. Register two users")
|
||||
|
||||
users = []
|
||||
|
||||
for name in ["alice_" + suffix, "bob_" + suffix]:
|
||||
status, user = request(
|
||||
"POST",
|
||||
"/api/v1/users",
|
||||
{
|
||||
"username": name,
|
||||
"display_name": name,
|
||||
"password": password,
|
||||
},
|
||||
)
|
||||
check("register " + name, status, 201)
|
||||
users.append(user)
|
||||
|
||||
|
||||
print("\n2. Login")
|
||||
|
||||
tokens = []
|
||||
|
||||
for user in users:
|
||||
status, result = request(
|
||||
"POST",
|
||||
"/api/v1/auth/sessions",
|
||||
basic=user["username"] + ":" + password,
|
||||
)
|
||||
check("login " + user["username"], status, 201)
|
||||
tokens.append(result["token"])
|
||||
|
||||
|
||||
print("\n3. Create chat")
|
||||
|
||||
status, chat = request(
|
||||
"POST",
|
||||
"/api/v1/chats",
|
||||
{
|
||||
"title": "PostgreSQL test chat",
|
||||
"member_ids": [users[0]["id"], users[1]["id"]],
|
||||
},
|
||||
token=tokens[0],
|
||||
)
|
||||
|
||||
check("create chat", status, 201)
|
||||
chat_id = chat["id"]
|
||||
|
||||
print("Chat ID:", chat_id)
|
||||
|
||||
|
||||
print("\n4. Check chat visibility")
|
||||
|
||||
for token in tokens:
|
||||
status, result = request(
|
||||
"GET",
|
||||
"/api/v1/chats/" + chat_id,
|
||||
token=token,
|
||||
)
|
||||
check("get chat", status, 200)
|
||||
|
||||
|
||||
print("\n5. Send messages")
|
||||
|
||||
for index in range(5):
|
||||
status, message = request(
|
||||
"POST",
|
||||
"/api/v1/chats/" + chat_id + "/messages",
|
||||
{"text": f"Message number {index + 1}"},
|
||||
token=tokens[index % 2],
|
||||
)
|
||||
check("send message", status, 201)
|
||||
|
||||
|
||||
print("\n6. Read first page")
|
||||
|
||||
status, page1 = request(
|
||||
"GET",
|
||||
"/api/v1/chats/" + chat_id + "/messages?limit=2",
|
||||
token=tokens[0],
|
||||
)
|
||||
|
||||
check("first page", status, 200)
|
||||
|
||||
assert len(page1["items"]) == 2
|
||||
assert page1["next_cursor"] is not None
|
||||
|
||||
|
||||
print("\n7. Read second page")
|
||||
|
||||
cursor = page1["next_cursor"]
|
||||
|
||||
status, page2 = request(
|
||||
"GET",
|
||||
"/api/v1/chats/" + chat_id + "/messages?limit=2&cursor=" + cursor,
|
||||
token=tokens[0],
|
||||
)
|
||||
|
||||
check("second page", status, 200)
|
||||
|
||||
assert len(page2["items"]) == 2
|
||||
|
||||
first_ids = {item["id"] for item in page1["items"]}
|
||||
second_ids = {item["id"] for item in page2["items"]}
|
||||
|
||||
assert not first_ids.intersection(second_ids)
|
||||
|
||||
|
||||
print("\n8. Check chat list")
|
||||
|
||||
status, result = request(
|
||||
"GET",
|
||||
"/api/v1/chats",
|
||||
token=tokens[0],
|
||||
)
|
||||
|
||||
check("list chats", status, 200)
|
||||
|
||||
assert any(item["id"] == chat_id for item in result["items"])
|
||||
|
||||
|
||||
print("\n9. Check PostgreSQL persistence")
|
||||
|
||||
print("Created chat:", chat_id)
|
||||
print("Created messages: 5")
|
||||
|
||||
print("\nALL MESSENGER CHECKS PASSED")
|
||||
@@ -0,0 +1,16 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Миграция с нуля и повторно, в изолированной схеме с ROLLBACK."""
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
from uuid import uuid4
|
||||
|
||||
schema = 'audit_' + uuid4().hex
|
||||
migration = Path('db/migrations/001_initial.sql').read_text()
|
||||
sql = f'BEGIN; CREATE SCHEMA {schema}; SET LOCAL search_path TO {schema};\n'
|
||||
sql += migration + '\n' + migration
|
||||
sql += "\nSELECT count(*) = 5 AS five_tables FROM information_schema.tables WHERE table_schema = '" + schema + "';\nROLLBACK;\n"
|
||||
result = subprocess.run(['docker', 'compose', 'exec', '-T', 'postgres', 'psql',
|
||||
'-v', 'ON_ERROR_STOP=1', '-U', 'messenger', '-d', 'messenger', '-At'],
|
||||
input=sql, text=True, capture_output=True, check=True)
|
||||
assert 't' in result.stdout.splitlines(), result.stdout
|
||||
print('PASS: migration creates five tables from zero and reapplies without errors; rolled back')
|
||||
@@ -0,0 +1,47 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Проверяет заполнение кеша, fallback, запись и logout при отказе Redis."""
|
||||
import hashlib
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'tests'))
|
||||
from client import Client, check
|
||||
|
||||
|
||||
def compose(*args, **kwargs):
|
||||
return subprocess.run(['docker', 'compose', *args], check=True, **kwargs)
|
||||
|
||||
|
||||
def main():
|
||||
c = Client()
|
||||
alice, bob, _, chat = c.bootstrap()
|
||||
c.call('GET', '/api/v1/users/me', actor=alice, schema='User')
|
||||
key = 'session:' + hashlib.sha256(alice['token'].encode('ascii')).hexdigest()
|
||||
# Хеш ключа передаётся через stdin и не выводится.
|
||||
result = compose('exec', '-T', 'redis', 'redis-cli', '-x', 'EXISTS',
|
||||
input=key, text=True, capture_output=True)
|
||||
check(result.stdout.strip() == '1', 'session profile was not cached')
|
||||
result = compose('exec', '-T', 'redis', 'redis-cli', '-x', 'TTL',
|
||||
input=key, text=True, capture_output=True)
|
||||
check(0 < int(result.stdout) <= 3600, 'cache TTL missing')
|
||||
print('PASS: Redis contains session profile with TTL')
|
||||
try:
|
||||
compose('stop', 'redis')
|
||||
c.call('GET', '/api/v1/users/me', actor=alice, schema='User')
|
||||
message = c.message(alice, chat, 'Redis fallback check')['json']
|
||||
page = c.call('GET', '/api/v1/chats/' + chat['id'] + '/messages',
|
||||
actor=bob, schema='MessagePage')['json']
|
||||
check(any(m['id'] == message['id'] for m in page['items']), 'fallback lost message')
|
||||
c.call('DELETE', '/api/v1/auth/sessions/current', 204, actor=alice)
|
||||
c.call('GET', '/api/v1/users/me', 401, actor=alice)
|
||||
print('PASS: PostgreSQL read/write and logout work without Redis')
|
||||
finally:
|
||||
compose('up', '-d', '--wait', 'redis')
|
||||
c.call('GET', '/api/v1/users/me', 401, actor=alice)
|
||||
c.call('GET', '/api/v1/users/me', actor=bob, schema='User')
|
||||
print('PASS: stale cache cannot revive revoked session; Redis recovered')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,71 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Запуск: docker compose exec -T api python < scripts/check_transactions.py.
|
||||
|
||||
Принудительная ошибка после записи первого участника доказывает rollback
|
||||
всего чата. Созданный синтетический пользователь остаётся в базе.
|
||||
"""
|
||||
from contextlib import contextmanager
|
||||
from unittest.mock import patch
|
||||
from uuid import uuid4
|
||||
|
||||
from src import chat_repository
|
||||
from src.auth import hash_password
|
||||
from src.database import get_connection, open_database, close_database
|
||||
from src.user_repository import create_user
|
||||
|
||||
|
||||
open_database()
|
||||
try:
|
||||
user = create_user('rollback_' + uuid4().hex[:12], 'Rollback check',
|
||||
hash_password('SyntheticPassword123!'))
|
||||
title = 'rollback-' + uuid4().hex
|
||||
|
||||
class FailingCursor:
|
||||
def __init__(self, cursor):
|
||||
self.cursor = cursor
|
||||
|
||||
def execute(self, *args):
|
||||
return self.cursor.execute(*args)
|
||||
|
||||
def fetchall(self):
|
||||
return self.cursor.fetchall()
|
||||
|
||||
def fetchone(self):
|
||||
return self.cursor.fetchone()
|
||||
|
||||
def executemany(self, query, rows):
|
||||
self.cursor.execute(query, rows[0])
|
||||
raise RuntimeError('injected membership failure')
|
||||
|
||||
class Connection:
|
||||
def __init__(self, conn):
|
||||
self.conn = conn
|
||||
|
||||
def transaction(self):
|
||||
return self.conn.transaction()
|
||||
|
||||
@contextmanager
|
||||
def cursor(self):
|
||||
with self.conn.cursor() as cur:
|
||||
yield FailingCursor(cur)
|
||||
|
||||
@contextmanager
|
||||
def failing_connection():
|
||||
with get_connection() as conn:
|
||||
yield Connection(conn)
|
||||
|
||||
with patch.object(chat_repository, 'get_connection', failing_connection):
|
||||
try:
|
||||
chat_repository.create_chat(user['id'], title, [user['id']])
|
||||
except RuntimeError as exc:
|
||||
assert str(exc) == 'injected membership failure'
|
||||
else:
|
||||
raise AssertionError('injected error was not raised')
|
||||
with get_connection() as conn:
|
||||
assert conn.execute('SELECT count(*) AS n FROM chats WHERE title = %s',
|
||||
(title,)).fetchone()['n'] == 0
|
||||
assert conn.execute('SELECT count(*) AS n FROM chat_members WHERE user_id = %s',
|
||||
(user['id'],)).fetchone()['n'] == 0
|
||||
print('PASS: failure after first member insert rolls back chat and membership')
|
||||
finally:
|
||||
close_database()
|
||||
@@ -0,0 +1,69 @@
|
||||
import base64
|
||||
import json
|
||||
import secrets
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
BASE = "http://" + "127.0.0.1:8081"
|
||||
username = "ttl_" + secrets.token_hex(4)
|
||||
password = "StrongPassword123!"
|
||||
|
||||
|
||||
def call(method, path, data=None, authorization=None):
|
||||
headers = {}
|
||||
if data is not None:
|
||||
headers["Content-Type"] = "application/json"
|
||||
if authorization:
|
||||
headers["Authorization"] = authorization
|
||||
|
||||
req = urllib.request.Request(
|
||||
BASE + path,
|
||||
data=json.dumps(data).encode() if data is not None else None,
|
||||
headers=headers,
|
||||
method=method,
|
||||
)
|
||||
|
||||
try:
|
||||
response = urllib.request.urlopen(req, timeout=15)
|
||||
except urllib.error.HTTPError as exc:
|
||||
response = exc
|
||||
|
||||
with response:
|
||||
raw = response.read()
|
||||
return response.status, json.loads(raw) if raw else None
|
||||
|
||||
|
||||
status, _ = call("POST", "/api/v1/users", {
|
||||
"username": username,
|
||||
"display_name": "TTL Test",
|
||||
"password": password,
|
||||
})
|
||||
assert status == 201, status
|
||||
|
||||
basic = base64.b64encode(
|
||||
(username + ":" + password).encode()
|
||||
).decode()
|
||||
|
||||
status, result = call(
|
||||
"POST",
|
||||
"/api/v1/auth/sessions",
|
||||
authorization="Basic " + basic,
|
||||
)
|
||||
assert status == 201, status
|
||||
|
||||
bearer = "Bearer " + result["token"]
|
||||
|
||||
status, _ = call(
|
||||
"GET", "/api/v1/users/me", authorization=bearer
|
||||
)
|
||||
assert status == 200, status
|
||||
|
||||
time.sleep(3)
|
||||
|
||||
status, _ = call(
|
||||
"GET", "/api/v1/users/me", authorization=bearer
|
||||
)
|
||||
assert status == 401, status
|
||||
|
||||
print("PASS: session expires after TTL")
|
||||
@@ -0,0 +1,36 @@
|
||||
\set ON_ERROR_STOP on
|
||||
\timing on
|
||||
|
||||
SELECT chat_id
|
||||
FROM messages
|
||||
GROUP BY chat_id
|
||||
ORDER BY COUNT(*) DESC
|
||||
LIMIT 1
|
||||
\gset
|
||||
|
||||
\echo 'COMPOSITE INDEX'
|
||||
|
||||
EXPLAIN (ANALYZE, BUFFERS)
|
||||
SELECT id, text, created_at
|
||||
FROM messages
|
||||
WHERE chat_id = :'chat_id'
|
||||
ORDER BY created_at, id
|
||||
LIMIT 50;
|
||||
|
||||
BEGIN;
|
||||
CREATE INDEX IF NOT EXISTS idx_messages_chat_id_only ON messages (chat_id);
|
||||
|
||||
DROP INDEX IF EXISTS idx_messages_chat_created_id;
|
||||
|
||||
\echo 'SINGLE COLUMN INDEX'
|
||||
|
||||
EXPLAIN (ANALYZE, BUFFERS)
|
||||
SELECT id, text, created_at
|
||||
FROM messages
|
||||
WHERE chat_id = :'chat_id'
|
||||
ORDER BY created_at, id
|
||||
LIMIT 50;
|
||||
|
||||
ROLLBACK;
|
||||
|
||||
\echo 'ORIGINAL INDEX RESTORED'
|
||||
@@ -0,0 +1,15 @@
|
||||
\set ON_ERROR_STOP on
|
||||
SELECT chat_id FROM messages GROUP BY chat_id ORDER BY count(*) DESC LIMIT 1
|
||||
\gset
|
||||
SELECT created_at AS cursor_time, id AS cursor_id FROM messages
|
||||
WHERE chat_id = :'chat_id' ORDER BY created_at, id OFFSET 80000 LIMIT 1
|
||||
\gset
|
||||
BEGIN;
|
||||
SET LOCAL plan_cache_mode = force_generic_plan;
|
||||
PREPARE history(uuid, timestamptz, uuid, int) AS
|
||||
SELECT id, chat_id, sender_id, text, created_at FROM messages
|
||||
WHERE chat_id = $1 AND (created_at, id) > ($2, $3)
|
||||
ORDER BY created_at, id LIMIT $4;
|
||||
EXPLAIN (ANALYZE, BUFFERS)
|
||||
EXECUTE history(:'chat_id', :'cursor_time', :'cursor_id', 51);
|
||||
ROLLBACK;
|
||||
@@ -0,0 +1,42 @@
|
||||
\set ON_ERROR_STOP on
|
||||
BEGIN;
|
||||
DO $$
|
||||
BEGIN
|
||||
IF (SELECT count(*) FROM chats c WHERE EXISTS (
|
||||
SELECT 1 FROM chat_members cm
|
||||
WHERE cm.chat_id = c.id AND cm.user_id = c.created_by
|
||||
)) < 2 THEN
|
||||
RAISE EXCEPTION 'Need at least two chats with creator membership';
|
||||
END IF;
|
||||
END $$;
|
||||
|
||||
WITH selected AS (
|
||||
SELECT
|
||||
c.id AS chat_id,
|
||||
c.created_by AS sender_id,
|
||||
ROW_NUMBER() OVER (ORDER BY c.created_at, c.id) AS rn
|
||||
FROM chats c
|
||||
WHERE EXISTS (
|
||||
SELECT 1 FROM chat_members cm
|
||||
WHERE cm.chat_id = c.id
|
||||
AND cm.user_id = c.created_by
|
||||
)
|
||||
ORDER BY c.created_at, c.id
|
||||
LIMIT 2
|
||||
)
|
||||
INSERT INTO messages (id, chat_id, sender_id, text, created_at)
|
||||
SELECT
|
||||
gen_random_uuid(),
|
||||
s.chat_id,
|
||||
s.sender_id,
|
||||
'Benchmark message ' || g.n,
|
||||
NOW() - INTERVAL '1 day' + g.n * INTERVAL '1 millisecond'
|
||||
FROM selected s
|
||||
CROSS JOIN LATERAL generate_series(
|
||||
1,
|
||||
CASE WHEN s.rn = 1 THEN 90000 ELSE 10000 END
|
||||
) AS g(n);
|
||||
|
||||
COMMIT;
|
||||
|
||||
ANALYZE messages;
|
||||
Executable
+8
@@ -0,0 +1,8 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
cd "$(dirname "$0")/.."
|
||||
# Все хранилища сохраняют именованные volumes. API запускается после готовности БД.
|
||||
docker compose restart postgres redis
|
||||
docker compose up -d --wait postgres redis
|
||||
docker compose restart api
|
||||
docker compose up -d --wait api
|
||||
+91
@@ -0,0 +1,91 @@
|
||||
import base64
|
||||
import binascii
|
||||
import hashlib
|
||||
import os
|
||||
import re
|
||||
import secrets
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
from argon2 import PasswordHasher
|
||||
from argon2.exceptions import InvalidHashError, VerifyMismatchError
|
||||
|
||||
|
||||
password_hasher = PasswordHasher(
|
||||
time_cost=3,
|
||||
memory_cost=65536,
|
||||
parallelism=4,
|
||||
)
|
||||
|
||||
|
||||
def hash_password(password: str) -> str:
|
||||
return password_hasher.hash(password)
|
||||
|
||||
|
||||
def verify_password(password_hash: str, password: str) -> bool:
|
||||
try:
|
||||
return password_hasher.verify(password_hash, password)
|
||||
except (VerifyMismatchError, InvalidHashError):
|
||||
return False
|
||||
|
||||
|
||||
def parse_basic(authorization: str | None) -> tuple[str, str] | None:
|
||||
if not authorization:
|
||||
return None
|
||||
|
||||
scheme, separator, credentials = authorization.partition(" ")
|
||||
|
||||
if not separator or scheme.lower() != "basic":
|
||||
return None
|
||||
|
||||
try:
|
||||
raw = base64.b64decode(credentials, validate=True)
|
||||
decoded = raw.decode("utf-8")
|
||||
username, password = decoded.split(":", 1)
|
||||
except (ValueError, UnicodeError, binascii.Error):
|
||||
return None
|
||||
|
||||
if not re.fullmatch(r"[a-z0-9_]{3,32}", username):
|
||||
return None
|
||||
|
||||
if not 12 <= len(password) <= 128:
|
||||
return None
|
||||
|
||||
return username, password
|
||||
|
||||
|
||||
def parse_bearer(authorization: str | None) -> str | None:
|
||||
if not authorization:
|
||||
return None
|
||||
|
||||
scheme, separator, token = authorization.partition(" ")
|
||||
|
||||
if not separator or scheme.lower() != "bearer":
|
||||
return None
|
||||
|
||||
if not re.fullmatch(r"[A-Za-z0-9._~+/-]+={0,2}", token) or len(token) > 512:
|
||||
return None
|
||||
|
||||
return token
|
||||
|
||||
|
||||
def create_token() -> str:
|
||||
return secrets.token_urlsafe(32)
|
||||
|
||||
|
||||
def hash_token(token: str) -> str:
|
||||
return hashlib.sha256(token.encode("ascii")).hexdigest()
|
||||
|
||||
|
||||
def session_expiration() -> datetime:
|
||||
ttl = int(os.getenv("SESSION_TTL_SECONDS", "3600"))
|
||||
|
||||
if ttl < 1:
|
||||
raise ValueError("SESSION_TTL_SECONDS must be positive")
|
||||
|
||||
return datetime.now(timezone.utc) + timedelta(seconds=ttl)
|
||||
|
||||
|
||||
def format_datetime(value: datetime) -> str:
|
||||
return value.astimezone(timezone.utc).isoformat(
|
||||
timespec="microseconds"
|
||||
).replace("+00:00", "Z")
|
||||
@@ -0,0 +1,234 @@
|
||||
from uuid import UUID, uuid4
|
||||
|
||||
from .database import get_connection
|
||||
|
||||
|
||||
def serialize_chat(row):
|
||||
return {
|
||||
"id": str(row["id"]),
|
||||
"title": row["title"],
|
||||
"member_ids": [str(member_id) for member_id in row["member_ids"]],
|
||||
"created_at": row["created_at"].isoformat(
|
||||
timespec="microseconds"
|
||||
).replace("+00:00", "Z"),
|
||||
}
|
||||
|
||||
|
||||
def serialize_message(row):
|
||||
return {
|
||||
"id": str(row["id"]),
|
||||
"chat_id": str(row["chat_id"]),
|
||||
"sender_id": str(row["sender_id"]),
|
||||
"text": row["text"],
|
||||
"created_at": row["created_at"].isoformat(
|
||||
timespec="microseconds"
|
||||
).replace("+00:00", "Z"),
|
||||
}
|
||||
|
||||
|
||||
def create_chat(creator_id, title, member_ids):
|
||||
chat_id = uuid4()
|
||||
members = [UUID(member_id) for member_id in member_ids]
|
||||
|
||||
with get_connection() as conn:
|
||||
with conn.transaction():
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
SELECT id
|
||||
FROM users
|
||||
WHERE id = ANY(%s::uuid[])
|
||||
""",
|
||||
(members,),
|
||||
)
|
||||
existing = {row["id"] for row in cur.fetchall()}
|
||||
|
||||
if len(existing) != len(members):
|
||||
return None
|
||||
|
||||
cur.execute(
|
||||
"""
|
||||
INSERT INTO chats (id, title, created_by)
|
||||
VALUES (%s, %s, %s)
|
||||
RETURNING id, title, created_at
|
||||
""",
|
||||
(chat_id, title, UUID(creator_id)),
|
||||
)
|
||||
chat = cur.fetchone()
|
||||
|
||||
cur.executemany(
|
||||
"""
|
||||
INSERT INTO chat_members (chat_id, user_id)
|
||||
VALUES (%s, %s)
|
||||
""",
|
||||
[(chat_id, member_id) for member_id in members],
|
||||
)
|
||||
|
||||
return {
|
||||
"id": str(chat["id"]),
|
||||
"title": chat["title"],
|
||||
"member_ids": member_ids,
|
||||
"created_at": chat["created_at"].isoformat(
|
||||
timespec="microseconds"
|
||||
).replace("+00:00", "Z"),
|
||||
}
|
||||
|
||||
|
||||
def get_chat(chat_id, user_id):
|
||||
with get_connection() as conn:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
SELECT
|
||||
c.id,
|
||||
c.title,
|
||||
c.created_at,
|
||||
ARRAY(
|
||||
SELECT cm2.user_id
|
||||
FROM chat_members cm2
|
||||
WHERE cm2.chat_id = c.id
|
||||
ORDER BY cm2.user_id
|
||||
) AS member_ids
|
||||
FROM chats c
|
||||
JOIN chat_members cm
|
||||
ON cm.chat_id = c.id
|
||||
WHERE c.id = %s
|
||||
AND cm.user_id = %s
|
||||
""",
|
||||
(UUID(chat_id), UUID(user_id)),
|
||||
)
|
||||
row = cur.fetchone()
|
||||
return serialize_chat(row) if row else None
|
||||
|
||||
|
||||
def list_chats(user_id, limit, last_chat_id=None):
|
||||
with get_connection() as conn:
|
||||
with conn.cursor() as cur:
|
||||
last_key = None
|
||||
|
||||
if last_chat_id is not None:
|
||||
cur.execute(
|
||||
"""
|
||||
SELECT c.created_at, c.id
|
||||
FROM chats c
|
||||
JOIN chat_members cm
|
||||
ON cm.chat_id = c.id
|
||||
WHERE c.id = %s
|
||||
AND cm.user_id = %s
|
||||
""",
|
||||
(UUID(last_chat_id), UUID(user_id)),
|
||||
)
|
||||
last_key = cur.fetchone()
|
||||
|
||||
if last_key is None:
|
||||
return None
|
||||
|
||||
cur.execute(
|
||||
"""
|
||||
SELECT
|
||||
c.id,
|
||||
c.title,
|
||||
c.created_at,
|
||||
ARRAY(
|
||||
SELECT cm2.user_id
|
||||
FROM chat_members cm2
|
||||
WHERE cm2.chat_id = c.id
|
||||
ORDER BY cm2.user_id
|
||||
) AS member_ids
|
||||
FROM chats c
|
||||
JOIN chat_members cm
|
||||
ON cm.chat_id = c.id
|
||||
WHERE cm.user_id = %s
|
||||
AND (
|
||||
%s::timestamptz IS NULL
|
||||
OR (c.created_at, c.id) >
|
||||
(%s::timestamptz, %s::uuid)
|
||||
)
|
||||
ORDER BY c.created_at, c.id
|
||||
LIMIT %s
|
||||
""",
|
||||
(
|
||||
UUID(user_id),
|
||||
last_key["created_at"] if last_key else None,
|
||||
last_key["created_at"] if last_key else None,
|
||||
last_key["id"] if last_key else None,
|
||||
limit + 1,
|
||||
),
|
||||
)
|
||||
rows = cur.fetchall()
|
||||
|
||||
return [serialize_chat(row) for row in rows]
|
||||
|
||||
|
||||
def create_message(chat_id, sender_id, text):
|
||||
message_id = uuid4()
|
||||
|
||||
with get_connection() as conn:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
INSERT INTO messages (id, chat_id, sender_id, text)
|
||||
SELECT %s, %s, %s, %s
|
||||
WHERE EXISTS (
|
||||
SELECT 1
|
||||
FROM chat_members
|
||||
WHERE chat_id = %s
|
||||
AND user_id = %s
|
||||
)
|
||||
RETURNING id, chat_id, sender_id, text, created_at
|
||||
""",
|
||||
(
|
||||
message_id,
|
||||
UUID(chat_id),
|
||||
UUID(sender_id),
|
||||
text,
|
||||
UUID(chat_id),
|
||||
UUID(sender_id),
|
||||
),
|
||||
)
|
||||
row = cur.fetchone()
|
||||
return serialize_message(row) if row else None
|
||||
|
||||
|
||||
def list_messages(chat_id, limit, last_message_id=None):
|
||||
with get_connection() as conn:
|
||||
with conn.cursor() as cur:
|
||||
last_key = None
|
||||
|
||||
if last_message_id is not None:
|
||||
cur.execute(
|
||||
"""
|
||||
SELECT created_at, id
|
||||
FROM messages
|
||||
WHERE chat_id = %s
|
||||
AND id = %s
|
||||
""",
|
||||
(UUID(chat_id), UUID(last_message_id)),
|
||||
)
|
||||
last_key = cur.fetchone()
|
||||
|
||||
if last_key is None:
|
||||
return None
|
||||
|
||||
# Разные формы SQL сохраняют индексный range scan и для generic plan.
|
||||
# В SQL подставляется только константа; значения остаются параметрами.
|
||||
cursor_condition = ""
|
||||
params = [UUID(chat_id)]
|
||||
if last_key is not None:
|
||||
cursor_condition = "AND (created_at, id) > (%s, %s)"
|
||||
params.extend([last_key["created_at"], last_key["id"]])
|
||||
params.append(limit + 1)
|
||||
cur.execute(
|
||||
f"""
|
||||
SELECT id, chat_id, sender_id, text, created_at
|
||||
FROM messages
|
||||
WHERE chat_id = %s
|
||||
{cursor_condition}
|
||||
ORDER BY created_at, id
|
||||
LIMIT %s
|
||||
""",
|
||||
params,
|
||||
)
|
||||
rows = cur.fetchall()
|
||||
|
||||
return [serialize_message(row) for row in rows]
|
||||
@@ -0,0 +1,48 @@
|
||||
import os
|
||||
from contextlib import contextmanager
|
||||
from pathlib import Path
|
||||
|
||||
from psycopg.rows import dict_row
|
||||
from psycopg_pool import ConnectionPool
|
||||
|
||||
|
||||
def get_database_url() -> str:
|
||||
secret_file = os.getenv("DATABASE_URL_FILE")
|
||||
|
||||
if secret_file:
|
||||
return Path(secret_file).read_text().strip()
|
||||
|
||||
database_url = os.getenv("DATABASE_URL")
|
||||
|
||||
if database_url:
|
||||
return database_url
|
||||
|
||||
raise RuntimeError("Database connection is not configured")
|
||||
|
||||
|
||||
pool = ConnectionPool(
|
||||
conninfo=get_database_url(),
|
||||
min_size=1,
|
||||
max_size=10,
|
||||
timeout=5,
|
||||
open=False,
|
||||
kwargs={
|
||||
"row_factory": dict_row,
|
||||
"connect_timeout": 5,
|
||||
"options": "-c statement_timeout=5000",
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def open_database():
|
||||
pool.open(wait=True, timeout=15)
|
||||
|
||||
|
||||
def close_database():
|
||||
pool.close()
|
||||
|
||||
|
||||
@contextmanager
|
||||
def get_connection():
|
||||
with pool.connection() as connection:
|
||||
yield connection
|
||||
+478
@@ -0,0 +1,478 @@
|
||||
"""Мессенджер: PostgreSQL хранит данные и сессии, Redis кеширует профили."""
|
||||
|
||||
import base64
|
||||
import binascii
|
||||
import logging
|
||||
import re
|
||||
from uuid import UUID, uuid4
|
||||
|
||||
from fastapi import FastAPI, HTTPException, Request
|
||||
from fastapi.exceptions import RequestValidationError
|
||||
from fastapi.responses import JSONResponse
|
||||
from starlette.exceptions import HTTPException as StarletteHTTPException
|
||||
|
||||
|
||||
|
||||
from psycopg import OperationalError
|
||||
from psycopg_pool import PoolTimeout
|
||||
|
||||
from .auth import (
|
||||
create_token,
|
||||
format_datetime,
|
||||
hash_password,
|
||||
hash_token,
|
||||
parse_basic,
|
||||
parse_bearer,
|
||||
session_expiration,
|
||||
verify_password,
|
||||
)
|
||||
from .database import open_database, close_database, get_connection
|
||||
from .user_repository import (
|
||||
create_user as insert_user,
|
||||
create_session,
|
||||
find_session_user_cached,
|
||||
find_user_by_username,
|
||||
revoke_session,
|
||||
serialize_user,
|
||||
)
|
||||
|
||||
from .chat_repository import (
|
||||
create_chat as insert_chat,
|
||||
get_chat as find_chat,
|
||||
list_chats as find_chats,
|
||||
create_message as insert_message,
|
||||
list_messages as find_messages,
|
||||
)
|
||||
|
||||
app = FastAPI(title="MTUSI Messenger", version="1.2.0")
|
||||
|
||||
logger = logging.getLogger("uvicorn.error")
|
||||
|
||||
ERROR_CODES = {
|
||||
400: "invalid_request",
|
||||
401: "unauthorized",
|
||||
404: "not_found",
|
||||
409: "conflict",
|
||||
413: "payload_too_large",
|
||||
415: "unsupported_media_type",
|
||||
429: "rate_limited",
|
||||
500: "internal_error",
|
||||
503: "unavailable",
|
||||
}
|
||||
|
||||
|
||||
def error_response(request, status_code, code, message):
|
||||
return JSONResponse(
|
||||
status_code=status_code,
|
||||
content={
|
||||
"error": {
|
||||
"code": code,
|
||||
"message": message,
|
||||
"request_id": request.state.request_id,
|
||||
}
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
@app.middleware("http")
|
||||
async def request_id_middleware(request: Request, call_next):
|
||||
request.state.request_id = str(uuid4())
|
||||
try:
|
||||
response = await call_next(request)
|
||||
except Exception:
|
||||
# Логируем путь, метод и request_id, но НЕ тело сообщения.
|
||||
logger.error(
|
||||
"Unhandled error: %s %s request_id=%s",
|
||||
request.method,
|
||||
request.url.path,
|
||||
request.state.request_id,
|
||||
)
|
||||
response = error_response(
|
||||
request, 500, "internal_error", "Внутренняя ошибка сервера"
|
||||
)
|
||||
response.headers["X-Request-Id"] = request.state.request_id
|
||||
return response
|
||||
|
||||
|
||||
@app.exception_handler(StarletteHTTPException)
|
||||
async def http_error_handler(request: Request, exc: StarletteHTTPException):
|
||||
response = error_response(
|
||||
request,
|
||||
exc.status_code,
|
||||
ERROR_CODES.get(exc.status_code, "invalid_request"),
|
||||
str(exc.detail),
|
||||
)
|
||||
if exc.headers:
|
||||
response.headers.update(exc.headers)
|
||||
return response
|
||||
|
||||
|
||||
@app.exception_handler(OperationalError)
|
||||
@app.exception_handler(PoolTimeout)
|
||||
async def database_error_handler(request: Request, exc):
|
||||
return error_response(request, 503, "unavailable", "База данных временно недоступна")
|
||||
|
||||
|
||||
@app.exception_handler(RequestValidationError)
|
||||
async def validation_error_handler(request: Request, exc: RequestValidationError):
|
||||
return error_response(request, 400, "invalid_request", "Некорректные данные запроса")
|
||||
|
||||
|
||||
async def read_json(request: Request):
|
||||
"""415 — только для неправильного Content-Type; 400 — для неверного JSON."""
|
||||
content_type = (
|
||||
request.headers.get("content-type", "").split(";")[0].strip().lower()
|
||||
)
|
||||
if content_type != "application/json":
|
||||
raise HTTPException(415, "Неподдерживаемый Content-Type")
|
||||
|
||||
try:
|
||||
data = await request.json()
|
||||
except (ValueError, UnicodeError):
|
||||
raise HTTPException(400, "Неверный JSON") from None
|
||||
|
||||
# [], строка, число, true, false, null — это корректный JSON,
|
||||
# но НЕ объект, обязательный по контракту.
|
||||
if not isinstance(data, dict):
|
||||
raise HTTPException(400, "Ожидается JSON-объект")
|
||||
return data
|
||||
|
||||
|
||||
def get_current_user(request: Request):
|
||||
token = parse_bearer(request.headers.get("Authorization"))
|
||||
if token is None:
|
||||
raise HTTPException(
|
||||
401,
|
||||
"Требуется действующий Bearer-токен",
|
||||
headers={"WWW-Authenticate": 'Bearer realm="messenger"'},
|
||||
)
|
||||
|
||||
try:
|
||||
user = find_session_user_cached(hash_token(token))
|
||||
except (OperationalError, PoolTimeout):
|
||||
raise HTTPException(
|
||||
503,
|
||||
"База данных временно недоступна",
|
||||
) from None
|
||||
if user is None:
|
||||
raise HTTPException(
|
||||
401,
|
||||
"Недействительная или истёкшая сессия",
|
||||
headers={"WWW-Authenticate": 'Bearer realm="messenger"'},
|
||||
)
|
||||
|
||||
return user
|
||||
|
||||
|
||||
def get_limit(request: Request):
|
||||
"""Проверяем длину ДО int(): 5000 цифр должны дать 400, а не 500."""
|
||||
limit_raw = request.query_params.get("limit", "50")
|
||||
if not limit_raw.isascii() or not limit_raw.isdecimal():
|
||||
raise HTTPException(400, "Неверный limit")
|
||||
|
||||
significant = limit_raw.lstrip("0")
|
||||
if not significant or len(significant) > 3:
|
||||
raise HTTPException(400, "Неверный limit")
|
||||
|
||||
limit = int(significant) # Теперь строка содержит максимум 3 цифры.
|
||||
if not 1 <= limit <= 100:
|
||||
raise HTTPException(400, "limit должен быть от 1 до 100")
|
||||
return limit
|
||||
|
||||
|
||||
def encode_cursor(first_id, last_id):
|
||||
raw = f"{first_id}|{last_id}".encode("ascii")
|
||||
return base64.urlsafe_b64encode(raw).decode("ascii").rstrip("=")
|
||||
|
||||
|
||||
def decode_cursor(cursor):
|
||||
try:
|
||||
if not cursor or len(cursor) > 256:
|
||||
raise ValueError("empty or too long")
|
||||
padded = cursor + "=" * (-len(cursor) % 4)
|
||||
raw = base64.b64decode(
|
||||
padded, altchars=b"-_", validate=True
|
||||
).decode("ascii")
|
||||
first_id, last_id = raw.split("|")
|
||||
return str(UUID(first_id)), str(UUID(last_id))
|
||||
except (ValueError, UnicodeError, binascii.Error):
|
||||
raise HTTPException(400, "Неверный cursor") from None
|
||||
|
||||
|
||||
# ---- Health ----
|
||||
|
||||
@app.get("/health/live")
|
||||
def health_live():
|
||||
return {"status": "ok"}
|
||||
|
||||
|
||||
@app.get("/health/ready")
|
||||
def health_ready():
|
||||
try:
|
||||
with get_connection() as conn:
|
||||
conn.execute("SELECT 1")
|
||||
except (OperationalError, PoolTimeout):
|
||||
raise HTTPException(503, "База данных недоступна") from None
|
||||
return {"status": "ok"}
|
||||
|
||||
|
||||
# ---- Users ----
|
||||
|
||||
@app.post("/api/v1/users", status_code=201)
|
||||
async def create_user(request: Request):
|
||||
data = await read_json(request)
|
||||
|
||||
if set(data) != {"username", "display_name", "password"}:
|
||||
raise HTTPException(400, "Неверные поля пользователя")
|
||||
|
||||
username = data["username"]
|
||||
display_name = data["display_name"]
|
||||
password = data["password"]
|
||||
|
||||
if not isinstance(username, str) or not re.fullmatch(
|
||||
r"[a-z0-9_]{3,32}", username
|
||||
):
|
||||
raise HTTPException(400, "Недопустимый username")
|
||||
|
||||
if (
|
||||
not isinstance(display_name, str)
|
||||
or not 1 <= len(display_name) <= 100
|
||||
or not display_name.strip()
|
||||
):
|
||||
raise HTTPException(400, "Недопустимое имя пользователя")
|
||||
|
||||
if not isinstance(password, str) or not 12 <= len(password) <= 128:
|
||||
raise HTTPException(400, "Пароль должен содержать 12–128 символов")
|
||||
|
||||
user = insert_user(username, display_name, hash_password(password))
|
||||
if user is None:
|
||||
raise HTTPException(409, "Username уже занят")
|
||||
|
||||
return user
|
||||
|
||||
|
||||
@app.get("/api/v1/users/me")
|
||||
def get_me(request: Request):
|
||||
return get_current_user(request)
|
||||
|
||||
|
||||
@app.post("/api/v1/auth/sessions", status_code=201)
|
||||
async def login(request: Request):
|
||||
credentials = parse_basic(request.headers.get("Authorization"))
|
||||
|
||||
if credentials is None:
|
||||
raise HTTPException(
|
||||
401,
|
||||
"Неверные учётные данные",
|
||||
headers={
|
||||
"WWW-Authenticate":
|
||||
'Basic realm="messenger", charset="UTF-8"'
|
||||
},
|
||||
)
|
||||
|
||||
username, password = credentials
|
||||
user_row = find_user_by_username(username)
|
||||
|
||||
if user_row is None or not verify_password(
|
||||
user_row["password_hash"], password
|
||||
):
|
||||
raise HTTPException(
|
||||
401,
|
||||
"Неверные учётные данные",
|
||||
headers={
|
||||
"WWW-Authenticate":
|
||||
'Basic realm="messenger", charset="UTF-8"'
|
||||
},
|
||||
)
|
||||
|
||||
token = create_token()
|
||||
expires_at = session_expiration()
|
||||
|
||||
create_session(user_row["id"], hash_token(token), expires_at)
|
||||
|
||||
return {
|
||||
"token": token,
|
||||
"token_type": "Bearer",
|
||||
"expires_at": format_datetime(expires_at),
|
||||
"user": serialize_user(user_row),
|
||||
}
|
||||
|
||||
|
||||
@app.delete("/api/v1/auth/sessions/current", status_code=204)
|
||||
def logout(request: Request):
|
||||
get_current_user(request)
|
||||
token = parse_bearer(request.headers.get("Authorization"))
|
||||
|
||||
from .session_cache import invalidate_session
|
||||
|
||||
if not revoke_session(hash_token(token)):
|
||||
raise HTTPException(
|
||||
401,
|
||||
"Недействительная сессия",
|
||||
headers={"WWW-Authenticate": 'Bearer realm="messenger"'},
|
||||
)
|
||||
|
||||
invalidate_session(hash_token(token))
|
||||
|
||||
from fastapi import Response
|
||||
return Response(status_code=204)
|
||||
|
||||
|
||||
|
||||
# ---- Chats ----
|
||||
|
||||
|
||||
@app.post("/api/v1/chats", status_code=201)
|
||||
async def create_chat(request: Request):
|
||||
user = get_current_user(request)
|
||||
data = await read_json(request)
|
||||
|
||||
if set(data) != {"title", "member_ids"}:
|
||||
raise HTTPException(400, "Неверные поля чата")
|
||||
|
||||
title = data["title"]
|
||||
member_ids = data["member_ids"]
|
||||
|
||||
if not isinstance(title, str) or not 1 <= len(title) <= 100 or not title.strip():
|
||||
raise HTTPException(400, "Недопустимое название чата")
|
||||
|
||||
if not isinstance(member_ids, list) or not 1 <= len(member_ids) <= 100:
|
||||
raise HTTPException(400, "Неверный список участников")
|
||||
|
||||
normalized_ids = []
|
||||
|
||||
for member_id in member_ids:
|
||||
if not isinstance(member_id, str):
|
||||
raise HTTPException(400, "Неверный ID участника")
|
||||
|
||||
try:
|
||||
normalized_ids.append(str(UUID(member_id)))
|
||||
except ValueError:
|
||||
raise HTTPException(400, "Неверный UUID участника") from None
|
||||
|
||||
if len(normalized_ids) != len(set(normalized_ids)):
|
||||
raise HTTPException(400, "Участники повторяются")
|
||||
|
||||
if user["id"] not in normalized_ids:
|
||||
raise HTTPException(400, "Создатель должен быть участником")
|
||||
|
||||
chat = insert_chat(user["id"], title, normalized_ids)
|
||||
|
||||
if chat is None:
|
||||
raise HTTPException(400, "Неизвестный участник")
|
||||
|
||||
return chat
|
||||
|
||||
|
||||
@app.get("/api/v1/chats")
|
||||
def list_chats(request: Request):
|
||||
user = get_current_user(request)
|
||||
limit = get_limit(request)
|
||||
cursor = request.query_params.get("cursor")
|
||||
|
||||
last_chat_id = None
|
||||
|
||||
if cursor is not None:
|
||||
cursor_user_id, last_chat_id = decode_cursor(cursor)
|
||||
|
||||
if cursor_user_id != user["id"]:
|
||||
raise HTTPException(400, "Неверный cursor")
|
||||
|
||||
rows = find_chats(user["id"], limit, last_chat_id)
|
||||
|
||||
if rows is None:
|
||||
raise HTTPException(400, "Неверный cursor")
|
||||
|
||||
has_more = len(rows) > limit
|
||||
page = rows[:limit]
|
||||
|
||||
next_cursor = (
|
||||
encode_cursor(user["id"], page[-1]["id"])
|
||||
if has_more and page
|
||||
else None
|
||||
)
|
||||
|
||||
return {"items": page, "next_cursor": next_cursor}
|
||||
|
||||
|
||||
@app.get("/api/v1/chats/{chat_id}")
|
||||
def get_chat(chat_id: str, request: Request):
|
||||
user = get_current_user(request)
|
||||
|
||||
try:
|
||||
chat_id = str(UUID(chat_id))
|
||||
except ValueError:
|
||||
raise HTTPException(400, "Неверный UUID чата") from None
|
||||
|
||||
chat = find_chat(chat_id, user["id"])
|
||||
|
||||
if chat is None:
|
||||
raise HTTPException(404, "Чат не найден")
|
||||
|
||||
return chat
|
||||
|
||||
|
||||
# ---- Messages ----
|
||||
|
||||
|
||||
@app.post("/api/v1/chats/{chat_id}/messages", status_code=201)
|
||||
async def send_message(chat_id: str, request: Request):
|
||||
user = get_current_user(request)
|
||||
chat = get_chat(chat_id, request)
|
||||
data = await read_json(request)
|
||||
|
||||
if set(data) != {"text"}:
|
||||
raise HTTPException(400, "Неверные поля сообщения")
|
||||
|
||||
text = data["text"]
|
||||
|
||||
if not isinstance(text, str) or not 1 <= len(text) <= 4000 or not text.strip():
|
||||
raise HTTPException(400, "Недопустимый текст сообщения")
|
||||
|
||||
message = insert_message(chat["id"], user["id"], text)
|
||||
|
||||
if message is None:
|
||||
raise HTTPException(404, "Чат не найден")
|
||||
|
||||
return message
|
||||
|
||||
|
||||
@app.get("/api/v1/chats/{chat_id}/messages")
|
||||
def list_messages(chat_id: str, request: Request):
|
||||
chat = get_chat(chat_id, request)
|
||||
limit = get_limit(request)
|
||||
cursor = request.query_params.get("cursor")
|
||||
|
||||
last_message_id = None
|
||||
|
||||
if cursor is not None:
|
||||
cursor_chat_id, last_message_id = decode_cursor(cursor)
|
||||
|
||||
if cursor_chat_id != chat["id"]:
|
||||
raise HTTPException(400, "Неверный cursor")
|
||||
|
||||
rows = find_messages(chat["id"], limit, last_message_id)
|
||||
|
||||
if rows is None:
|
||||
raise HTTPException(400, "Неверный cursor")
|
||||
|
||||
has_more = len(rows) > limit
|
||||
page = rows[:limit]
|
||||
|
||||
next_cursor = (
|
||||
encode_cursor(chat["id"], page[-1]["id"])
|
||||
if has_more and page
|
||||
else None
|
||||
)
|
||||
|
||||
return {"items": page, "next_cursor": next_cursor}
|
||||
|
||||
|
||||
@app.on_event("startup")
|
||||
def start_database_pool():
|
||||
open_database()
|
||||
|
||||
|
||||
@app.on_event("shutdown")
|
||||
def stop_database_pool():
|
||||
close_database()
|
||||
@@ -0,0 +1,48 @@
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
import redis
|
||||
|
||||
|
||||
def get_cache_url():
|
||||
secret_file = os.getenv("CACHE_URL_FILE")
|
||||
if secret_file:
|
||||
return Path(secret_file).read_text().strip()
|
||||
return os.getenv("CACHE_URL", "redis://redis:6379/0")
|
||||
|
||||
|
||||
client = redis.Redis.from_url(
|
||||
get_cache_url(),
|
||||
socket_connect_timeout=1,
|
||||
socket_timeout=1,
|
||||
decode_responses=True,
|
||||
)
|
||||
|
||||
|
||||
def get_cached_user(token_hash):
|
||||
try:
|
||||
value = client.get("session:" + token_hash)
|
||||
return json.loads(value) if value else None
|
||||
except (redis.RedisError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def cache_user(token_hash, user, ttl):
|
||||
if ttl <= 0:
|
||||
return
|
||||
try:
|
||||
client.setex(
|
||||
"session:" + token_hash,
|
||||
ttl,
|
||||
json.dumps(user),
|
||||
)
|
||||
except redis.RedisError:
|
||||
pass
|
||||
|
||||
|
||||
def invalidate_session(token_hash):
|
||||
try:
|
||||
client.delete("session:" + token_hash)
|
||||
except redis.RedisError:
|
||||
pass
|
||||
@@ -0,0 +1,136 @@
|
||||
from uuid import uuid4
|
||||
|
||||
from psycopg.errors import UniqueViolation
|
||||
|
||||
from .database import get_connection
|
||||
|
||||
|
||||
def serialize_user(row):
|
||||
return {
|
||||
"id": str(row["id"]),
|
||||
"username": row["username"],
|
||||
"display_name": row["display_name"],
|
||||
"created_at": row["created_at"].isoformat(
|
||||
timespec="microseconds"
|
||||
).replace("+00:00", "Z"),
|
||||
}
|
||||
|
||||
|
||||
def create_user(username, display_name, password_hash):
|
||||
with get_connection() as conn:
|
||||
with conn.cursor() as cur:
|
||||
try:
|
||||
cur.execute(
|
||||
"""
|
||||
INSERT INTO users (
|
||||
id, username, display_name, password_hash
|
||||
)
|
||||
VALUES (%s, %s, %s, %s)
|
||||
RETURNING id, username, display_name, created_at
|
||||
""",
|
||||
(uuid4(), username, display_name, password_hash),
|
||||
)
|
||||
return serialize_user(cur.fetchone())
|
||||
except UniqueViolation:
|
||||
conn.rollback()
|
||||
return None
|
||||
|
||||
|
||||
def find_user_by_username(username):
|
||||
with get_connection() as conn:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
SELECT id, username, display_name,
|
||||
password_hash, created_at
|
||||
FROM users
|
||||
WHERE username = %s
|
||||
""",
|
||||
(username,),
|
||||
)
|
||||
return cur.fetchone()
|
||||
|
||||
|
||||
def create_session(user_id, token_hash, expires_at):
|
||||
with get_connection() as conn:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
INSERT INTO sessions (
|
||||
id, user_id, token_hash, expires_at
|
||||
)
|
||||
VALUES (%s, %s, %s, %s)
|
||||
""",
|
||||
(uuid4(), user_id, token_hash, expires_at),
|
||||
)
|
||||
|
||||
|
||||
def revoke_session(token_hash):
|
||||
with get_connection() as conn:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
UPDATE sessions
|
||||
SET revoked_at = NOW()
|
||||
WHERE token_hash = %s
|
||||
AND revoked_at IS NULL
|
||||
AND expires_at > NOW()
|
||||
RETURNING id
|
||||
""",
|
||||
(token_hash,),
|
||||
)
|
||||
return cur.fetchone() is not None
|
||||
|
||||
|
||||
def find_session_user_cached(token_hash):
|
||||
from datetime import datetime, timezone
|
||||
from .session_cache import cache_user, get_cached_user
|
||||
|
||||
with get_connection() as conn:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
SELECT
|
||||
s.user_id,
|
||||
s.expires_at
|
||||
FROM sessions s
|
||||
WHERE s.token_hash = %s
|
||||
AND s.revoked_at IS NULL
|
||||
AND s.expires_at > NOW()
|
||||
""",
|
||||
(token_hash,),
|
||||
)
|
||||
session = cur.fetchone()
|
||||
|
||||
if session is None:
|
||||
return None
|
||||
|
||||
cached = get_cached_user(token_hash)
|
||||
|
||||
if cached is not None and str(cached.get("id")) == str(session["user_id"]):
|
||||
return cached
|
||||
|
||||
with get_connection() as conn:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
SELECT id, username, display_name, created_at
|
||||
FROM users
|
||||
WHERE id = %s
|
||||
""",
|
||||
(session["user_id"],),
|
||||
)
|
||||
row = cur.fetchone()
|
||||
|
||||
if row is None:
|
||||
return None
|
||||
|
||||
user = serialize_user(row)
|
||||
ttl = int(
|
||||
(session["expires_at"] - datetime.now(timezone.utc)).total_seconds()
|
||||
)
|
||||
|
||||
if ttl > 0:
|
||||
cache_user(token_hash, user, ttl)
|
||||
|
||||
return user
|
||||
Reference in New Issue
Block a user