Author SHA1 Message Date
SAMURAI 6d1d7cbbc4 REPORT.md 2026-10-29 23:27:34 +03:00
SAMURAI 0406baab3f Lab2: PostgreSQL, Bearer sessions and Redis for grade 4 2026-10-29 23:22:15 +03:00
49 changed files with 3096 additions and 16 deletions
+12 -2
View File
@@ -5,7 +5,17 @@ APP_PORT=8080
APP_ENV=lab
SESSION_TTL_SECONDS=3600
# Choose credentials locally. File paths contain no secret values.
# scripts/bootstrap.sh writes secrets/ and .env; do not commit them.
DATABASE_URL_FILE=/run/secrets/database_url
# Optional on 3, required use on 4 in lab2; Redis OR Valkey:
SESSION_STORE=postgres
# Direct URLs override the files. The API container uses the files above.
# DATABASE_URL=postgresql+asyncpg://USER:PASSWORD@HOST:5432/messenger
# REDIS_URL=redis://HOST:6379/0
SESSION_STORE=redis
CACHE_URL_FILE=/run/secrets/cache_url
DB_POOL_SIZE=10
DB_MAX_OVERFLOW=10
DB_POOL_TIMEOUT_SECONDS=5
DB_CONNECT_TIMEOUT_SECONDS=2
DB_COMMAND_TIMEOUT_SECONDS=5
REDIS_SOCKET_TIMEOUT_SECONDS=1
REDIS_MAX_CONNECTIONS=50
+1
View File
@@ -0,0 +1 @@
*.sh text eol=lf
+28
View File
@@ -0,0 +1,28 @@
FROM python:3.13.16-slim-bookworm@sha256:a1165e272e578941b84abc79e4ab38a0305cd12803a5c4247979ac7655f4d641
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
PIP_DISABLE_PIP_VERSION_CHECK=1 \
PYTHONPATH=/app/src \
APP_HOST=0.0.0.0 \
APP_PORT=8080 \
APP_ENV=lab
WORKDIR /app
RUN useradd --create-home --uid 10001 --shell /usr/sbin/nologin appuser
COPY requirements.txt /app/requirements.txt
RUN pip install --no-cache-dir -r /app/requirements.txt
COPY src /app/src
COPY migrations /app/migrations
USER appuser
EXPOSE 8080
HEALTHCHECK --interval=5s --timeout=3s --start-period=40s --retries=12 \
CMD ["python", "-c", "import os,urllib.request; p=os.environ.get('APP_PORT','8080'); urllib.request.urlopen('http://127.0.0.1:%s/health/ready'%p, timeout=2).read()"]
CMD ["python", "-m", "app"]
+97 -14
View File
@@ -1,37 +1,120 @@
# Отчёт — лабораторная 2
- Автор, группа: TODO
- Целевая оценка; выбранный трек 5, если нужен: TODO
- Commit/PR: TODO после публикации
- Использованные источники, библиотеки и помощь ИИ: TODO
- Автор, группа: Щербацевич Владислав БПИ2302
- Целевая оценка: **4**. Трек 5 не выбран.
- Commit/PR: 0406baa
- Использованные источники, библиотеки и помощь ИИ: FastAPI, Uvicorn, Pydantic, psycopg3+pool, redis-py, argon2-cffi; контракт курса; помощь ИИ при переносе на PostgreSQL/сессии/Redis
## Запуск из чистого clone
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
Стенд: Windows 10, Docker Desktop 29.6.1, виртуальной машине Docker выделено 20 CPU и около 15.4 ГиБ RAM. Лимиты CPU/RAM на сервисы не ставились. Хостовый Python для проверок — 3.13.0. Образ API — `python:3.13.16-slim-bookworm` (digest зафиксирован в `Dockerfile`).
Образы хранилищ:
- PostgreSQL `16.10-bookworm`, digest `sha256:38471f330eb885e04de130b768d6db4e10469e2311879c7e5c699f6d2d8a1c74`
- Redis `7.4.6-alpine`, digest `sha256:3b73847e72874be07e6657b129a94761662b79bc0f679273757d4218573b2a98`
```sh
python -m venv .venv
.venv/bin/python -m pip install -r requirements.txt
sh scripts/bootstrap.sh
docker compose --env-file .env up -d --build
```
На Windows те же шаги: `powershell -File scripts/bootstrap.ps1`, затем `docker compose --env-file .env up -d --build`. Bootstrap создаёт неотслеживаемые `.env` и `secrets/`. В git их нет. Пароль учебный и только локальный; в отчёт и логи он не копируется.
Дождаться `GET /health/ready` → `200 {"status":"ok"}`. Затем:
```sh
python tests/smoke.py --base-url http://127.0.0.1:8080
python student-tests/run.py
python student-tests/level4.py
```
Сохранность:
```sh
chmod +x scripts/restart.sh
make test-persistence ACTION_SCRIPT=scripts/restart.sh
```
`scripts/restart.sh` делает `docker compose restart api postgres redis` и не вызывает `down -v`. На PowerShell процесс Windows не запускает `.sh` напрямую (WinError 193). Там тот же файл выполняется через Git Bash: `python student-tests/persistence.py`. На Linux достаточно `make test-persistence`.
Остановка без удаления данных: `docker compose --env-file .env stop`. Удаление учебных томов: `docker compose --env-file .env down -v`.
Хостовый порт PostgreSQL — **5433**. Внутри сети Compose база остаётся `postgres:5432`. На этой машине привязка `127.0.0.1:5432` запрещена (WinError 10013), Redis на `127.0.0.1:6379` и API на `127.0.0.1:8080` открываются. Порты не публикуются наружу интерфейса, только на loopback.
## Архитектура и решения
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab1, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
Клиент ходит в один процесс API. Пользователи, чаты, участники и сообщения лежат в PostgreSQL: это источник истины. Сессии лежат только в Redis. Кэша истории нет, поэтому после COMMIT следующее чтение видит ту же строку без инвалидации.
Из lab1 перенесены контрактные правила чата, keyset-пагинация и формат ошибок. Память процесса больше не хранит эти данные. Аутентификация заменена по переходу 1 → 2: регистрация принимает `password`, `POST /api/v1/auth/sessions` принимает только Basic, остальные защищённые методы — только Bearer. Заголовок `X-User-Id` не читается.
Схема — `migrations/001_initial.sql`. При старте API берёт `pg_advisory_xact_lock` и применяет файл, только если его имени ещё нет в `schema_migrations`. Повторный старт пишет в лог `migrations newly_applied=0` и не вставляет строки заново. После `scripts/restart.sh` в логе API было именно `newly_applied=0`, версия осталась `001_initial.sql`.
Транзакции:
- регистрация: Argon2 считается до захвата соединения; в транзакции одна вставка `users`; уникальность username держит ограничение `uq_users_username`;
- создание чата: в одной транзакции проверка участников, вставка `chats` и все строки `chat_members`;
- сообщение: в одной транзакции проверка членства и вставка `messages`.
Пароль — Argon2id, параметры `m=32768` (32 МиБ), `t=2`, `p=1`, длина хеша 32 байта. Это не SHA-256 и не усечение bcrypt. Для отсутствующего пользователя выполняется проверка фиктивного хеша, ответ тот же, что и при неверном пароле: `401` и `WWW-Authenticate: Basic`.
Сессия — 32 случайных байта (`token_urlsafe(32)`, 256 бит). В Redis ключ `sess:` + SHA-256 токена, значение — `user_id` и абсолютный `expires_at`. Команда `SET` с `PXAT` записывает абсолютное время в миллисекундах, поэтому повтор AOF после рестарта не сдвигает срок. Чтение делает `GET` и не продлевает TTL. Logout делает `GETDEL`. Пока Redis не отвечает, запрос с Bearer получает `503`, запрос без заголовка — `401`: анонимный клиент не становится доверенным.
Redis запущен с `--appendonly yes --appendfsync always` и томом `redis_data`. `CONFIG GET` показал `appendonly=yes`, `appendfsync=always`. Ответ `201` на вход возвращается после того, как Redis подтвердил запись.
Пул SQLAlchemy: `pool_size=10`, `max_overflow=10`, ожидание соединения 5 с, `pool_pre_ping=true`. asyncpg: таймаут подключения 2 с, `command_timeout` 5 с. На стороне PostgreSQL `statement_timeout` 5 с, `lock_timeout` 5 с, `idle_in_transaction_session_timeout` 10 с. Redis: таймаут сокета 1 с, не больше 50 соединений. Запросы параметризованы через SQLAlchemy; текст сообщения в SQL не склеивается.
Переменные из `.env.example` совпадают с `compose.yaml`: `SESSION_TTL_SECONDS` (в стенде 3600, для проверки истечения отдельно 2), `DATABASE_URL_FILE` и `CACHE_URL_FILE` внутри API, прямые `DATABASE_URL` и `REDIS_URL` для процессов на хосте. `SESSION_STORE=redis`.
`/health/live` смотрит только процесс. `/health/ready` требует и PostgreSQL, и Redis: без сессий и без источника истины обязательный путь не обслуживается.
## Доказательства критериев
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
| --- | --- | --- | --- |
| Публичный smoke | make test | TODO | TODO |
| Остальные пункты уровня 3 | TODO | TODO | TODO |
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
| Публичный smoke | `python tests/smoke.py --base-url http://127.0.0.1:8080` | `PASS: lab2 public API scenario` | вывод команды, повторён после рестарта |
| Данные и сессия переживают рестарт | `python student-tests/persistence.py` (`scripts/restart.sh` перезапускает api, postgres, redis, тома на месте) | тот же Bearer читает то же сообщение; лог API: `migrations newly_applied=0` | вывод команды |
| PostgreSQL, ключи, миграция с нуля | старт пустого тома, затем повторный старт | таблицы users/chats/chat_members/messages, PK/FK, `uq_users_username`; вторая миграция не применяется | `migrations/001_initial.sql`, лог `newly_applied=0` |
| Атомарное создание чата | неизвестный участник; отдельно обрыв после INSERT чата | число чатов не меняется, строки с заголовком проверки нет (`11` и `11`, `title_rows=0`) | `evidence/level4.json` |
| Basic только на создании сессии, Bearer на остальном, `X-User-Id` не даёт доступ | `student-tests/level4.py` | неверный и неизвестный пароль дают один и тот же текст `401`; Basic на `/users/me` — `401`; один `X-User-Id` — `401`; чужой `X-User-Id` вместе с Bearer не меняет пользователя | `evidence/level4.json` |
| Пароль — Argon2id, в том числе Unicode и двоеточие | пароль `Пароль:пример-12` | повторный Basic-вход успешен | вывод `PASS: unicode password with colon` |
| Срок сессии и logout | `DELETE /api/v1/auth/sessions/current`, затем отдельный процесс с `SESSION_TTL_SECONDS=2` | logout → `204`, повтор и чтение → `401`; после `expires_at` чтение → `401` | `evidence/level4.json` |
| Redis — хранилище сессий с устойчивым TTL | `PTTL` до и после `docker compose restart redis` | `3590337` мс → `3588378` мс, ключ жив, тот же токен принимается | `evidence/level4.json` |
| Гонка одинакового username | 16 параллельных `POST /users` | ровно один `201` и пятнадцать `409` | `evidence/level4.json` |
| Параметризованные запросы | текст `'; DROP TABLE messages; --` | таблица `messages` на месте, текст сохранён как данные | вывод level4 |
| Индекс истории и keyset | 10 000 сообщений, `EXPLAIN (ANALYZE, BUFFERS)` первой страницы и страницы после 5000-й строки | `Index Scan using ix_messages_history`, не `Seq Scan`; `shared hit=5` и `shared hit=4`; `actual rows=50`; 0.050 мс и 0.051 мс | `evidence/explain_messages.txt` |
| Отказ Redis и PostgreSQL | `docker compose stop redis`, затем postgres; потом `start` | live `200`; ready `503`; без заголовка `401`; с Bearer `503`; после возврата данные и сессия на месте, 10 000 строк целы | `evidence/level4.json` |
| Свои проверки lab1 на новом входе | `python student-tests/run.py` | пагинация, 100 сообщений от 10 клиентов, текст сообщения не попадает в лог | `evidence/concurrency.json` |
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
Проверка инфраструктурного hook: `docker compose ps` до и после `restart` показывает те же сервисы `api`, `postgres`, `redis`; тома `postgres_data` и `redis_data` не удалялись. Скрипт завершился с кодом 0.
## Эксперименты и отказы
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
Нагрузка уровня 5 не заявлялась. Ниже только обязательные отказы уровня 4. Прогрев — до `/health/ready`. Конкуренция гонки — 16 запросов регистрации. История для плана — 10 000 строк в одном чате. Повторы: smoke дважды (до и после рестарта), level4 и persistence по одному успешному прогону. RPS и перцентили нагрузки не измерялись: это не трек 5. CPU/RAM стенда — 20 CPU и 15.4 ГиБ у Docker Desktop, без лимитов сервисов.
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
| --- | --- | --- | --- |
| TODO | TODO | TODO | TODO |
| `restart` API, PostgreSQL и Redis | тот же токен и то же сообщение, TTL не сбрасывается в полный час | сообщение найдено тем же Bearer; `PTTL` уменьшился с 3590337 до 3588378 мс | готовность около 2 с после старта контейнеров; подтверждённые строки не пропали |
| `SESSION_TTL_SECONDS=2` | после `expires_at` сессия недействительна | `401` | новая сессия нужна сразу после срока; данные пользователя остаются |
| `stop redis` | live жив, ready нет, аноним не доверен, Bearer не обслуживается | 200 / 503 / 401 / 503 | после `start redis` тот же токен снова `200`; сессия не потеряна за счёт AOF |
| `stop postgres` | live жив, чтение истории невозможно, аноним не доверен | 200 / 503 / 401 / 503 | после `start postgres` история читается; 10 000 сообщений на месте |
## Ответы к вопросам защиты
COMMIT в PostgreSQL фиксирует транзакцию чата или сообщения. При `synchronous_commit=on` (значение по умолчанию у этого образа) сервер подтверждает COMMIT после записи WAL. Это не защита от потери всего диска хоста и не замена резервной копии. Том Docker сохраняет каталог данных между рестартом контейнера, но не между `down -v`.
Сессия после рестарта Redis читается из AOF. `appendfsync always` просит fsync на каждую запись. `PXAT` хранит абсолютный срок, поэтому рестарт его не продлевает. Если удалить том Redis или выключить AOF, сессии пропадут, а пользователи и сообщения в PostgreSQL останутся. Один только TTL без persistence ключ после рестарта не возвращает.
Аутентификация отвечает на вопрос «кто вызывает». Доступ к чату отвечает на вопрос «состоит ли этот пользователь в чате». Чужой чат для уже вошедшего пользователя — `404`, не `403`, чтобы не подтверждать существование. Неверный токен — `401` и возникает раньше проверки чата.
Хеш пароля должен быть медленным и со случайной солью: его задача — сделать перебор дорогим, если база утечёт. Контрольная сумма вроде SHA-256 быстро узнаёт, изменились ли байты. Для пароля она не подходит: одинаковые пароли дали бы одинаковый хеш, а проверка была бы слишком дешёвой.
## Ограничения и следующие шаги
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
Стенд выдерживает рестарт контейнеров на том же Docker-хосте и короткую остановку Redis или PostgreSQL. Он не выдерживает потерю самого хоста, порчу тома и одновременный отказ диска. Один процесс API — не горизонтальное масштабирование; это задача lab3.
Роль `messenger` в учебном образе PostgreSQL — суперпользователь. Отдельная роль только для DML здесь не выделялась. У Redis нет AUTH: порт открыт только на `127.0.0.1`, сеть Compose внутренняя. Argon2id запущен с учебными параметрами, не с промышленным профилем lab7. История не кэшируется, поэтому отдельной инвалидации кэша сообщений нет.
Не делался ни один трек оценки 5: ни измерение cache stampede, ни backup на 100 000 сообщений.
+109
View File
@@ -0,0 +1,109 @@
# Локальный учебный стенд лабораторной 2. Секреты создаёт scripts/bootstrap, не этот файл.
name: mtusi-lab2
services:
postgres:
# Хостовый порт 5433: на части машин Windows порт 5432 нельзя занять.
# Внутри сети стенда база по-прежнему доступна как postgres:5432.
image: postgres:16.10-bookworm@sha256:38471f330eb885e04de130b768d6db4e10469e2311879c7e5c699f6d2d8a1c74
environment:
POSTGRES_USER: messenger
POSTGRES_DB: messenger
POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password
secrets:
- postgres_password
volumes:
- postgres_data:/var/lib/postgresql/data
ports:
- "127.0.0.1:5433:5432"
networks:
- lab
healthcheck:
test: ["CMD-SHELL", "pg_isready -U messenger -d messenger"]
interval: 2s
timeout: 3s
retries: 30
start_period: 5s
restart: unless-stopped
redis:
image: redis:7.4.6-alpine@sha256:3b73847e72874be07e6657b129a94761662b79bc0f679273757d4218573b2a98
command:
- redis-server
- --appendonly
- yes
- --appendfsync
- always
- --dir
- /data
volumes:
- redis_data:/data
ports:
- "127.0.0.1:6379:6379"
networks:
- lab
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 2s
timeout: 3s
retries: 30
start_period: 3s
restart: unless-stopped
api:
build: .
environment:
APP_HOST: 0.0.0.0
APP_PORT: "8080"
APP_ENV: lab
SESSION_TTL_SECONDS: ${SESSION_TTL_SECONDS:-3600}
SESSION_STORE: redis
DATABASE_URL_FILE: /run/secrets/database_url
CACHE_URL_FILE: /run/secrets/cache_url
DB_POOL_SIZE: "10"
DB_MAX_OVERFLOW: "10"
DB_POOL_TIMEOUT_SECONDS: "5"
DB_CONNECT_TIMEOUT_SECONDS: "2"
DB_COMMAND_TIMEOUT_SECONDS: "5"
REDIS_SOCKET_TIMEOUT_SECONDS: "1"
REDIS_MAX_CONNECTIONS: "50"
secrets:
- database_url
- cache_url
ports:
- "127.0.0.1:8080:8080"
networks:
- lab
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
healthcheck:
test:
[
"CMD",
"python",
"-c",
"import os,urllib.request; p=os.environ.get('APP_PORT','8080'); urllib.request.urlopen('http://127.0.0.1:%s/health/ready'%p, timeout=2).read()",
]
interval: 5s
timeout: 3s
retries: 12
start_period: 40s
restart: unless-stopped
secrets:
postgres_password:
file: ./secrets/postgres_password
database_url:
file: ./secrets/database_url
cache_url:
file: ./secrets/cache_url
volumes:
postgres_data:
redis_data:
networks:
lab:
+11
View File
@@ -0,0 +1,11 @@
{
"date": "2026-10-25",
"command": "python student-tests/run.py",
"clients": 10,
"messages_per_client": 10,
"submitted": 100,
"unique_ids": 100,
"listed": 100,
"failed_requests": 0,
"pass": true
}
+23
View File
@@ -0,0 +1,23 @@
messages=10000
-- first page
Limit (cost=0.29..7.72 rows=50 width=32) (actual time=0.021..0.032 rows=50 loops=1)
Buffers: shared hit=5
-> Index Scan using ix_messages_history on messages (cost=0.29..1486.70 rows=10000 width=32) (actual time=0.021..0.029 rows=50 loops=1)
Index Cond: (chat_id = 'f3c38461-f7ec-49fd-a912-46e9cff8f2b4'::uuid)
Buffers: shared hit=5
Planning:
Buffers: shared hit=104
Planning Time: 0.210 ms
Execution Time: 0.050 ms
-- page after row 5000
Limit (cost=0.29..12.22 rows=50 width=32) (actual time=0.016..0.027 rows=50 loops=1)
Buffers: shared hit=4
-> Index Scan using ix_messages_history on messages (cost=0.29..1200.23 rows=5027 width=32) (actual time=0.015..0.023 rows=50 loops=1)
Index Cond: ((chat_id = 'f3c38461-f7ec-49fd-a912-46e9cff8f2b4'::uuid) AND (ROW(created_at, id) > ROW('2026-01-01 00:00:00.005001+00'::timestamp with time zone, 'd063df83-eabe-455a-b0dc-d8e1dfb1364d'::uuid)))
Buffers: shared hit=4
Planning:
Buffers: shared hit=105
Planning Time: 0.276 ms
Execution Time: 0.051 ms
+40
View File
@@ -0,0 +1,40 @@
{
"date": "2026-10-25T17:11:54Z",
"base_url": "http://127.0.0.1:8080",
"race": {
"created": 1,
"conflict": 15
},
"rollback": {
"chats_before": 11,
"chats_after": 11,
"title_rows": 0
},
"explain": {
"messages": 10000
},
"expiry_and_migrations": {
"users_before": 38,
"users_after": 39,
"migration": "001_initial.sql"
},
"redis_restart": {
"pttl_before_ms": 3590337,
"pttl_after_ms": 3588378
},
"dependency_loss": {
"redis_down": {
"live": 200,
"ready": 503,
"anonymous": 401,
"bearer": 503
},
"postgres_down": {
"live": 200,
"ready": 503,
"anonymous": 401,
"bearer": 503
}
},
"messages_after_postgres_restart": 10000
}
+52
View File
@@ -0,0 +1,52 @@
-- Пользователи, чаты, состав и сообщения. Пароли и тексты сообщений
-- не пишутся приложением в логи. password_hash — только Argon2id.
-- Сессии в этой схеме нет: их источник истины — Redis (см. compose.yaml).
CREATE TABLE users (
id UUID PRIMARY KEY,
username TEXT NOT NULL,
display_name TEXT NOT NULL,
password_hash TEXT NOT NULL,
created_at TIMESTAMPTZ NOT NULL,
CONSTRAINT uq_users_username UNIQUE (username),
CONSTRAINT ck_users_username_format CHECK (username ~ '^[a-z0-9_]{3,32}$'),
CONSTRAINT ck_users_display_name_len CHECK (
char_length(btrim(display_name)) >= 1 AND char_length(display_name) <= 100
),
CONSTRAINT ck_users_password_hash_len CHECK (char_length(password_hash) >= 20)
);
CREATE TABLE chats (
id UUID PRIMARY KEY,
title TEXT NOT NULL,
created_at TIMESTAMPTZ NOT NULL,
CONSTRAINT ck_chats_title CHECK (char_length(btrim(title)) >= 1 AND char_length(title) <= 100)
);
-- Состав чата пишется в той же транзакции, что и строка chats.
-- Откат до COMMIT не оставляет чат без участников.
CREATE TABLE chat_members (
chat_id UUID NOT NULL REFERENCES chats (id),
user_id UUID NOT NULL REFERENCES users (id),
ordinal SMALLINT NOT NULL,
PRIMARY KEY (chat_id, user_id),
CONSTRAINT ck_chat_members_ordinal CHECK (ordinal >= 0 AND ordinal < 100)
);
-- Список чатов пользователя: фильтр по user_id, затем порядок (created_at, id) чата.
CREATE INDEX ix_chat_members_user ON chat_members (user_id, chat_id);
CREATE TABLE messages (
id UUID PRIMARY KEY,
chat_id UUID NOT NULL REFERENCES chats (id),
sender_id UUID NOT NULL REFERENCES users (id),
body TEXT NOT NULL,
created_at TIMESTAMPTZ NOT NULL,
CONSTRAINT ck_messages_text CHECK (char_length(btrim(body)) >= 1 AND char_length(body) <= 4000)
);
-- История: WHERE chat_id = $1 AND (created_at, id) > ($2, $3)
-- ORDER BY created_at, id LIMIT $4.
-- Ведущий chat_id отсекает чужие чаты, пара (created_at, id) продолжает
-- keyset с cursor и не читает предыдущие строки этой истории.
CREATE INDEX ix_messages_history ON messages (chat_id, created_at, id);
+18
View File
@@ -0,0 +1,18 @@
annotated-doc==0.0.5
annotated-types==0.8.0
anyio==4.15.1
click==8.5.0
fastapi==0.143.0
h11==0.16.0
idna==3.20
opentelemetry-api==1.45.1
pydantic==2.14.0
pydantic-core==2.50.0
starlette==1.7.0
typing-inspection==0.4.4
typing-extensions==4.16.0
uvicorn==0.54.0
sqlalchemy==2.0.44
asyncpg==0.31.0
redis==6.4.0
argon2-cffi==25.1.0
+14
View File
@@ -0,0 +1,14 @@
# Создаёт локальные учебные секреты. Не печатает их и не кладёт в git.
$ErrorActionPreference = "Stop"
$root = Split-Path -Parent $PSScriptRoot
Set-Location $root
New-Item -ItemType Directory -Force -Path (Join-Path $root "secrets") | Out-Null
$pass = "lab-only-password"
$utf8 = New-Object System.Text.UTF8Encoding $false
[System.IO.File]::WriteAllText((Join-Path $root "secrets\postgres_password"), $pass, $utf8)
[System.IO.File]::WriteAllText((Join-Path $root "secrets\database_url"), "postgresql+asyncpg://messenger:${pass}@postgres:5432/messenger", $utf8)
[System.IO.File]::WriteAllText((Join-Path $root "secrets\cache_url"), "redis://redis:6379/0", $utf8)
[System.IO.File]::WriteAllText((Join-Path $root "secrets\host_database_url"), "postgresql+asyncpg://messenger:${pass}@127.0.0.1:5433/messenger", $utf8)
[System.IO.File]::WriteAllText((Join-Path $root "secrets\host_cache_url"), "redis://127.0.0.1:6379/0", $utf8)
[System.IO.File]::WriteAllText((Join-Path $root ".env"), "SESSION_TTL_SECONDS=3600`nAPP_ENV=lab`n", $utf8)
Write-Host "bootstrap: wrote .env and secrets/ for the local lab stand"
+13
View File
@@ -0,0 +1,13 @@
#!/bin/sh
# Создаёт локальные учебные секреты. Не печатает их и не кладёт в git.
set -eu
cd "$(dirname "$0")/.."
mkdir -p secrets
pass='lab-only-password'
printf '%s' "$pass" > secrets/postgres_password
printf '%s' "postgresql+asyncpg://messenger:${pass}@postgres:5432/messenger" > secrets/database_url
printf '%s' 'redis://redis:6379/0' > secrets/cache_url
printf '%s' "postgresql+asyncpg://messenger:${pass}@127.0.0.1:5433/messenger" > secrets/host_database_url
printf '%s' 'redis://127.0.0.1:6379/0' > secrets/host_cache_url
printf '%s\n' 'SESSION_TTL_SECONDS=3600' 'APP_ENV=lab' > .env
echo 'bootstrap: wrote .env and secrets/ for the local lab stand'
+9
View File
@@ -0,0 +1,9 @@
#!/bin/sh
# Перезапуск API и всех хранилищ лабораторной 2. Тома не удаляются.
set -eu
cd "$(dirname "$0")/.."
if [ -f .env ]; then
docker compose --env-file .env restart api postgres redis
else
docker compose restart api postgres redis
fi
View File
+38
View File
@@ -0,0 +1,38 @@
import logging
import sys
import uvicorn
from app.config import load_settings
def main() -> None:
settings = load_settings()
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s %(levelname)s %(name)s %(message)s",
stream=sys.stdout,
force=True,
)
logging.getLogger("sqlalchemy.engine").setLevel(logging.WARNING)
logging.getLogger("sqlalchemy.pool").setLevel(logging.WARNING)
logging.getLogger("asyncpg").setLevel(logging.WARNING)
logging.getLogger("app").info(
"starting host=%s port=%s env=%s",
settings.host,
settings.port,
settings.env,
)
uvicorn.run(
"app.main:app",
host=settings.host,
port=settings.port,
workers=1,
access_log=False,
timeout_graceful_shutdown=5,
log_config=None,
)
if __name__ == "__main__":
main()
+12
View File
@@ -0,0 +1,12 @@
from fastapi import APIRouter
from app.api.auth import router as auth_router
from app.api.chats import router as chats_router
from app.api.messages import router as messages_router
from app.api.users import router as users_router
api_router = APIRouter(prefix="/api/v1")
api_router.include_router(users_router)
api_router.include_router(auth_router)
api_router.include_router(chats_router)
api_router.include_router(messages_router)
+37
View File
@@ -0,0 +1,37 @@
from fastapi import APIRouter, Depends, Header
from starlette.responses import Response
from app.crud.users import authenticate
from app.deps import BASIC_CHALLENGE, BEARER_CHALLENGE, require_token, unauthorized
from app.schemas.session import SessionOut
from app.security import parse_basic, verify_password
from app.sessions import create_session, revoke_session
from app.timeutil import format_time
router = APIRouter()
@router.post("/auth/sessions", status_code=201, response_model=SessionOut)
async def create_session_route(authorization: str | None = Header(default=None)) -> dict:
parsed = parse_basic(authorization or "")
if parsed is None:
verify_password("invalid-basic-auth", None)
raise unauthorized(BASIC_CHALLENGE)
username, password = parsed
user = await authenticate(username, password)
if user is None:
raise unauthorized(BASIC_CHALLENGE)
token, expires_at = await create_session(user.id)
return {
"token": token,
"token_type": "Bearer",
"expires_at": format_time(expires_at),
"user": user.to_dict(),
}
@router.delete("/auth/sessions/current", status_code=204)
async def revoke_current_session(token: str = Depends(require_token)) -> Response:
if not await revoke_session(token):
raise unauthorized(BEARER_CHALLENGE)
return Response(status_code=204)
+31
View File
@@ -0,0 +1,31 @@
from uuid import UUID
from fastapi import APIRouter, Depends, Query
from app.crud.chats import chat_for_member, chats_for_user, create_chat as save_chat
from app.deps import get_current_user
from app.present import PublicUser
from app.schemas.chat import ChatOut, ChatPage, CreateChat
router = APIRouter()
@router.post("/chats", status_code=201, response_model=ChatOut)
async def create_chat(body: CreateChat, user: PublicUser = Depends(get_current_user)) -> dict:
chat = await save_chat(user.id, body.title, tuple(body.member_ids))
return chat.to_dict()
@router.get("/chats", response_model=ChatPage)
async def list_chats(
user: PublicUser = Depends(get_current_user),
limit: int = Query(default=50, ge=1, le=100),
cursor: str | None = Query(default=None, min_length=1),
) -> dict:
return await chats_for_user(user.id, limit, cursor)
@router.get("/chats/{chat_id}", response_model=ChatOut)
async def get_chat(chat_id: UUID, user: PublicUser = Depends(get_current_user)) -> dict:
chat = await chat_for_member(chat_id, user.id)
return chat.to_dict()
+39
View File
@@ -0,0 +1,39 @@
import logging
from fastapi import APIRouter, Request
from pydantic import BaseModel
from sqlalchemy import text
from app.db import is_dependency_error
from app.errors import AppError
router = APIRouter()
logger = logging.getLogger("app.health")
class HealthOut(BaseModel):
status: str
@router.get("/health/live", response_model=HealthOut)
async def live() -> dict:
return {"status": "ok"}
@router.get("/health/ready", response_model=HealthOut)
async def ready(request: Request) -> dict:
if not getattr(request.app.state, "accepting", False):
raise AppError(503, "unavailable", "Сервер не готов")
engine = getattr(request.app.state, "engine", None)
redis = getattr(request.app.state, "redis", None)
if engine is None or redis is None:
raise AppError(503, "unavailable", "Сервис временно недоступен")
try:
async with engine.connect() as conn:
await conn.execute(text("SELECT 1"))
await redis.ping()
except Exception as exc:
if not is_dependency_error(exc):
logger.warning("readiness check failed type=%s", type(exc).__name__)
raise AppError(503, "unavailable", "Сервис временно недоступен") from None
return {"status": "ok"}
+31
View File
@@ -0,0 +1,31 @@
from uuid import UUID
from fastapi import APIRouter, Depends, Query
from app.crud.messages import create_message as save_message
from app.crud.messages import messages_for_member
from app.deps import get_current_user
from app.present import PublicUser
from app.schemas.message import CreateMessage, MessageOut, MessagePage
router = APIRouter()
@router.post("/chats/{chat_id}/messages", status_code=201, response_model=MessageOut)
async def create_message(
chat_id: UUID,
body: CreateMessage,
user: PublicUser = Depends(get_current_user),
) -> dict:
message = await save_message(chat_id, user.id, body.text)
return message.to_dict()
@router.get("/chats/{chat_id}/messages", response_model=MessagePage)
async def list_messages(
chat_id: UUID,
user: PublicUser = Depends(get_current_user),
limit: int = Query(default=50, ge=1, le=100),
cursor: str | None = Query(default=None, min_length=1),
) -> dict:
return await messages_for_member(chat_id, user.id, limit, cursor)
+19
View File
@@ -0,0 +1,19 @@
from fastapi import APIRouter, Depends
from app.crud.users import create_user
from app.deps import get_current_user
from app.present import PublicUser
from app.schemas.user import CreateUser, UserOut
router = APIRouter()
@router.post("/users", status_code=201, response_model=UserOut)
async def register(body: CreateUser) -> dict:
user = await create_user(body.username, body.display_name, body.password)
return user.to_dict()
@router.get("/users/me", response_model=UserOut)
async def me(user: PublicUser = Depends(get_current_user)) -> dict:
return user.to_dict()
+97
View File
@@ -0,0 +1,97 @@
import os
from dataclasses import dataclass
from pathlib import Path
@dataclass(frozen=True)
class Settings:
host: str
port: int
env: str
session_ttl_seconds: int
database_url: str
cache_url: str
migrations_dir: Path
db_pool_size: int
db_max_overflow: int
db_pool_timeout_seconds: float
db_connect_timeout_seconds: float
db_command_timeout_seconds: float
redis_socket_timeout_seconds: float
redis_max_connections: int
def _int(name: str, default: int, low: int, high: int) -> int:
raw = os.environ.get(name, str(default)).strip()
try:
value = int(raw)
except ValueError as exc:
raise SystemExit(f"{name} must be an integer") from exc
if not low <= value <= high:
raise SystemExit(f"{name} is out of range")
return value
def _float(name: str, default: float, low: float, high: float) -> float:
raw = os.environ.get(name, str(default)).strip()
try:
value = float(raw)
except ValueError as exc:
raise SystemExit(f"{name} must be a number") from exc
if not low <= value <= high:
raise SystemExit(f"{name} is out of range")
return value
def _secret(direct_name: str, file_name: str) -> str:
direct = os.environ.get(direct_name, "").strip()
if direct:
return direct
path = os.environ.get(file_name, "").strip()
if not path:
raise SystemExit(f"set {direct_name} or {file_name}")
try:
value = Path(path).read_text(encoding="utf-8").strip()
except OSError as exc:
raise SystemExit(f"cannot read {file_name}") from exc
if not value:
raise SystemExit(f"{file_name} is empty")
return value
def _migrations_dir() -> Path:
configured = os.environ.get("MIGRATIONS_DIR", "").strip()
if configured:
return Path(configured)
return Path(__file__).resolve().parents[2] / "migrations"
def load_settings() -> Settings:
host = os.environ.get("APP_HOST", "0.0.0.0").strip() or "0.0.0.0"
env_name = os.environ.get("APP_ENV", "lab").strip() or "lab"
port = _int("APP_PORT", 8080, 1, 65535)
database_url = _secret("DATABASE_URL", "DATABASE_URL_FILE")
cache_url = _secret("REDIS_URL", "CACHE_URL_FILE")
if not database_url.startswith("postgresql+asyncpg://"):
raise SystemExit("DATABASE_URL must start with postgresql+asyncpg://")
if not cache_url.startswith(("redis://", "rediss://")):
raise SystemExit("REDIS_URL must start with redis:// or rediss://")
migrations = _migrations_dir()
if not migrations.is_dir():
raise SystemExit("migrations directory is missing")
return Settings(
host=host,
port=port,
env=env_name,
session_ttl_seconds=_int("SESSION_TTL_SECONDS", 3600, 1, 30 * 24 * 3600),
database_url=database_url,
cache_url=cache_url,
migrations_dir=migrations,
db_pool_size=_int("DB_POOL_SIZE", 10, 1, 50),
db_max_overflow=_int("DB_MAX_OVERFLOW", 10, 0, 50),
db_pool_timeout_seconds=_float("DB_POOL_TIMEOUT_SECONDS", 5, 0.1, 30),
db_connect_timeout_seconds=_float("DB_CONNECT_TIMEOUT_SECONDS", 2, 0.1, 30),
db_command_timeout_seconds=_float("DB_COMMAND_TIMEOUT_SECONDS", 5, 0.1, 60),
redis_socket_timeout_seconds=_float("REDIS_SOCKET_TIMEOUT_SECONDS", 1, 0.1, 30),
redis_max_connections=_int("REDIS_MAX_CONNECTIONS", 50, 1, 200),
)
+1
View File
@@ -0,0 +1 @@
"""Чтение и изменение PostgreSQL. Сессии живут в Redis, не в этих таблицах."""
+101
View File
@@ -0,0 +1,101 @@
from uuid import UUID, uuid4
from sqlalchemy import select, tuple_
from sqlalchemy.exc import IntegrityError
from app.db import map_integrity, session_scope
from app.errors import AppError
from app.models import Chat, ChatMember, User
from app.pagination import decode_position, slice_page
from app.present import PublicChat
from app.timeutil import utcnow
def _invalid_members() -> AppError:
return AppError(400, "invalid_request", "Некорректный список участников")
async def create_chat(
owner_id: UUID,
title: str,
member_ids: tuple[UUID, ...],
*,
fail_after_insert: bool = False,
) -> PublicChat:
if (
not member_ids
or len(member_ids) > 100
or len(set(member_ids)) != len(member_ids)
or owner_id not in member_ids
):
raise _invalid_members()
chat_id = uuid4()
created_at = utcnow()
async with session_scope() as session:
try:
async with session.begin():
found = set((await session.scalars(select(User.id).where(User.id.in_(member_ids)))).all())
if found != set(member_ids):
raise _invalid_members()
session.add(Chat(id=chat_id, title=title, created_at=created_at))
await session.flush()
# Точка для проверки отката: строка чата уже вставлена, COMMIT ещё нет.
if fail_after_insert:
raise RuntimeError("rollback-check")
for ordinal, member_id in enumerate(member_ids):
session.add(ChatMember(chat_id=chat_id, user_id=member_id, ordinal=ordinal))
await session.flush()
except IntegrityError as exc:
raise map_integrity(exc, "Конфликт записи") from None
return PublicChat(id=chat_id, title=title, member_ids=tuple(member_ids), created_at=created_at)
async def _member_ids(session, chat_id: UUID) -> tuple[UUID, ...]:
rows = await session.execute(
select(ChatMember.user_id).where(ChatMember.chat_id == chat_id).order_by(ChatMember.ordinal)
)
return tuple(row[0] for row in rows)
async def chat_for_member(chat_id: UUID, user_id: UUID) -> PublicChat:
async with session_scope() as session:
async with session.begin():
member = await session.scalar(
select(ChatMember.user_id).where(ChatMember.chat_id == chat_id, ChatMember.user_id == user_id)
)
chat = await session.get(Chat, chat_id)
if member is None or chat is None:
raise AppError(404, "not_found", "Объект не найден")
return PublicChat(
id=chat.id,
title=chat.title,
member_ids=await _member_ids(session, chat.id),
created_at=chat.created_at,
)
async def chats_for_user(user_id: UUID, limit: int, cursor: str | None) -> dict:
position = decode_position(cursor, "c", user_id) if cursor else None
async with session_scope() as session:
async with session.begin():
stmt = (
select(Chat)
.join(ChatMember, ChatMember.chat_id == Chat.id)
.where(ChatMember.user_id == user_id)
)
if position is not None:
created_at, cursor_id = position
stmt = stmt.where(tuple_(Chat.created_at, Chat.id) > tuple_(created_at, cursor_id))
stmt = stmt.order_by(Chat.created_at.asc(), Chat.id.asc()).limit(limit + 1)
chats = (await session.scalars(stmt)).unique().all()
public = []
for chat in chats:
public.append(
PublicChat(
id=chat.id,
title=chat.title,
member_ids=await _member_ids(session, chat.id),
created_at=chat.created_at,
)
)
return slice_page(public, limit, "c", user_id)
+72
View File
@@ -0,0 +1,72 @@
from uuid import UUID, uuid4
from sqlalchemy import select, tuple_
from sqlalchemy.exc import IntegrityError
from app.db import map_integrity, session_scope
from app.errors import AppError
from app.models import ChatMember, Message
from app.pagination import decode_position, slice_page
from app.present import PublicMessage
from app.timeutil import utcnow
async def _require_member(session, chat_id: UUID, user_id: UUID) -> None:
member = await session.scalar(
select(ChatMember.user_id).where(ChatMember.chat_id == chat_id, ChatMember.user_id == user_id)
)
if member is None:
raise AppError(404, "not_found", "Объект не найден")
async def create_message(chat_id: UUID, sender_id: UUID, text: str) -> PublicMessage:
message_id = uuid4()
created_at = utcnow()
async with session_scope() as session:
try:
async with session.begin():
# Проверка членства и вставка коммитятся вместе: чужому чату строка не видна.
await _require_member(session, chat_id, sender_id)
session.add(
Message(
id=message_id,
chat_id=chat_id,
sender_id=sender_id,
text=text,
created_at=created_at,
)
)
await session.flush()
except IntegrityError as exc:
raise map_integrity(exc, "Конфликт записи") from None
return PublicMessage(
id=message_id,
chat_id=chat_id,
sender_id=sender_id,
text=text,
created_at=created_at,
)
async def messages_for_member(chat_id: UUID, user_id: UUID, limit: int, cursor: str | None) -> dict:
async with session_scope() as session:
async with session.begin():
await _require_member(session, chat_id, user_id)
position = decode_position(cursor, "m", chat_id) if cursor else None
stmt = select(Message).where(Message.chat_id == chat_id)
if position is not None:
created_at, message_id = position
stmt = stmt.where(tuple_(Message.created_at, Message.id) > tuple_(created_at, message_id))
stmt = stmt.order_by(Message.created_at.asc(), Message.id.asc()).limit(limit + 1)
rows = (await session.scalars(stmt)).all()
public = [
PublicMessage(
id=row.id,
chat_id=row.chat_id,
sender_id=row.sender_id,
text=row.text,
created_at=row.created_at,
)
for row in rows
]
return slice_page(public, limit, "m", chat_id)
+78
View File
@@ -0,0 +1,78 @@
import re
from uuid import UUID, uuid4
from sqlalchemy import select
from sqlalchemy.exc import IntegrityError
from app.db import map_integrity, session_scope
from app.models import User
from app.present import PublicUser
from app.security import hash_password, verify_password
from app.timeutil import utcnow
_USERNAME = re.compile(r"^[a-z0-9_]{3,32}$")
async def get_user(user_id: UUID) -> PublicUser | None:
async with session_scope() as session:
async with session.begin():
row = (
await session.execute(
select(User.id, User.username, User.display_name, User.created_at).where(User.id == user_id)
)
).one_or_none()
if row is None:
return None
return PublicUser(id=row.id, username=row.username, display_name=row.display_name, created_at=row.created_at)
async def create_user(username: str, display_name: str, password: str) -> PublicUser:
# Argon2 считается до открытия транзакции, чтобы не держать соединение пула.
password_hash = hash_password(password)
user_id = uuid4()
created_at = utcnow()
async with session_scope() as session:
try:
async with session.begin():
session.add(
User(
id=user_id,
username=username,
display_name=display_name,
password_hash=password_hash,
created_at=created_at,
)
)
await session.flush()
except IntegrityError as exc:
raise map_integrity(exc, "Имя пользователя занято") from None
return PublicUser(id=user_id, username=username, display_name=display_name, created_at=created_at)
async def authenticate(username: str, password: str) -> PublicUser | None:
if _USERNAME.fullmatch(username) is None or not 12 <= len(password) <= 128:
verify_password("rejected-credential", None)
return None
async with session_scope() as session:
async with session.begin():
row = (
await session.execute(
select(User.id, User.username, User.display_name, User.created_at, User.password_hash).where(
User.username == username
)
)
).one_or_none()
if row is None:
stored = None
snapshot = None
else:
stored = row.password_hash
snapshot = PublicUser(
id=row.id,
username=row.username,
display_name=row.display_name,
created_at=row.created_at,
)
if not verify_password(password, stored):
return None
return snapshot
+79
View File
@@ -0,0 +1,79 @@
import logging
from contextlib import asynccontextmanager
from redis.exceptions import RedisError
from sqlalchemy.exc import DBAPIError, IntegrityError, InterfaceError, OperationalError
from sqlalchemy.exc import TimeoutError as SATimeout
from sqlalchemy.ext.asyncio import AsyncEngine, AsyncSession, async_sessionmaker, create_async_engine
from app.config import Settings
from app.errors import AppError
logger = logging.getLogger("app.db")
def build_engine(settings: Settings) -> AsyncEngine:
command_ms = str(int(settings.db_command_timeout_seconds * 1000))
lock_ms = str(int(settings.db_pool_timeout_seconds * 1000))
return create_async_engine(
settings.database_url,
pool_size=settings.db_pool_size,
max_overflow=settings.db_max_overflow,
pool_timeout=settings.db_pool_timeout_seconds,
pool_pre_ping=True,
pool_recycle=1800,
echo=False,
connect_args={
"timeout": settings.db_connect_timeout_seconds,
"command_timeout": settings.db_command_timeout_seconds,
"server_settings": {
"statement_timeout": command_ms,
"lock_timeout": lock_ms,
"idle_in_transaction_session_timeout": "10000",
},
},
)
def build_session_factory(engine: AsyncEngine) -> async_sessionmaker[AsyncSession]:
return async_sessionmaker(engine, expire_on_commit=False, autoflush=False)
def is_dependency_error(exc: BaseException) -> bool:
if isinstance(exc, (OperationalError, InterfaceError, SATimeout, RedisError, ConnectionError, TimeoutError)):
return True
return isinstance(exc, DBAPIError) and bool(getattr(exc, "connection_invalidated", False))
def map_integrity(exc: IntegrityError, duplicate_message: str) -> AppError:
orig = getattr(exc, "orig", None)
code = getattr(orig, "sqlstate", None) or getattr(orig, "pgcode", None)
if code == "23505":
return AppError(409, "conflict", duplicate_message)
if code in {"23503", "23514", "23502"}:
return AppError(400, "invalid_request", "Некорректный запрос")
return AppError(500, "internal_error", "Внутренняя ошибка")
@asynccontextmanager
async def session_scope():
from app.runtime import require_factory
factory = require_factory()
try:
session = factory()
except Exception as exc:
if is_dependency_error(exc):
logger.warning("database unavailable type=%s", type(exc).__name__)
raise AppError(503, "unavailable", "Сервис временно недоступен") from None
raise
try:
async with session:
yield session
except AppError:
raise
except Exception as exc:
if is_dependency_error(exc):
logger.warning("database unavailable type=%s", type(exc).__name__)
raise AppError(503, "unavailable", "Сервис временно недоступен") from None
raise
+50
View File
@@ -0,0 +1,50 @@
from fastapi import Depends, Header, Request
from app.crud.users import get_user
from app.errors import AppError
from app.present import PublicUser
from app.sessions import resolve_session
BEARER_CHALLENGE = 'Bearer realm="messenger"'
BASIC_CHALLENGE = 'Basic realm="messenger", charset="UTF-8"'
def unauthorized(challenge: str) -> AppError:
return AppError(
401,
"unauthorized",
"Требуется идентификация",
headers={"WWW-Authenticate": challenge},
)
async def get_current_user(
request: Request,
authorization: str | None = Header(default=None),
) -> PublicUser:
# Нет заголовка — клиент анонимный. Redis для этого не нужен и не делает его доверенным.
if authorization is None:
raise unauthorized(BEARER_CHALLENGE)
scheme, _, token = authorization.partition(" ")
token = token.strip()
if scheme.lower() != "bearer" or not token or len(token) > 512:
raise unauthorized(BEARER_CHALLENGE)
user_id = await resolve_session(token)
if user_id is None:
raise unauthorized(BEARER_CHALLENGE)
user = await get_user(user_id)
if user is None:
raise unauthorized(BEARER_CHALLENGE)
request.state.session_token = token
return user
def current_token(request: Request) -> str:
token = getattr(request.state, "session_token", "")
if not token:
raise unauthorized(BEARER_CHALLENGE)
return token
def require_token(request: Request, user: PublicUser = Depends(get_current_user)) -> str:
del user
return current_token(request)
+54
View File
@@ -0,0 +1,54 @@
import logging
from fastapi import Request
from fastapi.exceptions import RequestValidationError
from fastapi.responses import JSONResponse
from starlette.exceptions import HTTPException
class AppError(Exception):
def __init__(self, status: int, code: str, message: str, headers: dict | None = None) -> None:
super().__init__(message)
self.status = status
self.code = code
self.message = message
self.headers = headers
def request_id_of(request: Request) -> str:
return getattr(request.state, "request_id", "") or "missing"
def error_response(
status: int,
code: str,
message: str,
request_id: str,
headers: dict | None = None,
) -> JSONResponse:
return JSONResponse(
status_code=status,
content={"error": {"code": code, "message": message, "request_id": request_id}},
headers=headers,
)
async def handle_app_error(request: Request, exc: AppError) -> JSONResponse:
return error_response(exc.status, exc.code, exc.message, request_id_of(request), exc.headers)
async def handle_dependency(request: Request, exc: Exception) -> JSONResponse:
# Текст исключения может содержать SQL или параметры. В ответ и лог его не кладём.
logging.getLogger("app").warning("dependency unavailable type=%s", type(exc).__name__)
return error_response(503, "unavailable", "Сервис временно недоступен", request_id_of(request))
async def handle_validation_error(request: Request, _exc: RequestValidationError) -> JSONResponse:
return error_response(400, "invalid_request", "Некорректный запрос", request_id_of(request))
async def handle_http_error(request: Request, exc: HTTPException) -> JSONResponse:
# Тело не в UTF-8 FastAPI превращает в HTTPException 400 с полем detail.
if exc.status_code == 400:
return error_response(400, "invalid_request", "Некорректный запрос", request_id_of(request))
return JSONResponse(status_code=exc.status_code, content={"detail": exc.detail}, headers=exc.headers)
+93
View File
@@ -0,0 +1,93 @@
import asyncio
import logging
from contextlib import asynccontextmanager
from fastapi import FastAPI
from fastapi.exceptions import RequestValidationError
from redis.exceptions import RedisError
from sqlalchemy import text
from sqlalchemy.exc import InterfaceError, OperationalError, ProgrammingError
from sqlalchemy.exc import TimeoutError as SATimeout
from starlette.exceptions import HTTPException
from app.api import api_router
from app.api.health import router as health_router
from app.config import load_settings
from app.db import build_engine, build_session_factory, is_dependency_error
from app.errors import AppError, handle_app_error, handle_dependency, handle_http_error, handle_validation_error
from app.migrate import apply_migrations
from app.middleware import request_context
from app.runtime import configure
from app.security import init_security
from app.sessions import build_redis
logger = logging.getLogger("app")
async def open_stores(engine, redis, migrations_dir) -> None:
for attempt in range(40):
try:
async with engine.connect() as conn:
await conn.execute(text("SELECT 1"))
await redis.ping()
applied = await apply_migrations(engine, migrations_dir)
logger.info("migrations newly_applied=%s", applied)
return
except SystemExit:
raise
except ProgrammingError as exc:
sqlstate = getattr(getattr(exc, "orig", None), "sqlstate", "")
logger.error("migration rejected type=%s sqlstate=%s", type(exc).__name__, sqlstate)
raise SystemExit("migration failed") from None
except Exception as exc:
if not is_dependency_error(exc):
logger.error("startup failed type=%s", type(exc).__name__)
raise SystemExit("startup failed") from None
logger.warning("startup not ready type=%s attempt=%s", type(exc).__name__, attempt + 1)
if attempt == 39:
raise SystemExit("database or redis is not ready") from None
await asyncio.sleep(0.5)
@asynccontextmanager
async def lifespan(app: FastAPI):
settings = load_settings()
init_security()
engine = build_engine(settings)
redis = build_redis(settings)
app.state.engine = engine
app.state.redis = redis
app.state.settings = settings
app.state.accepting = False
try:
await open_stores(engine, redis, settings.migrations_dir)
configure(build_session_factory(engine), redis, settings)
app.state.accepting = True
logger.info(
"ready env=%s session_ttl_seconds=%s db_pool=%s",
settings.env,
settings.session_ttl_seconds,
settings.db_pool_size,
)
yield
finally:
app.state.accepting = False
logger.info("shutdown: new requests are no longer accepted")
await redis.aclose()
await engine.dispose()
def create_app() -> FastAPI:
app = FastAPI(title="MTUSI Messenger", version="1.2.0", lifespan=lifespan)
app.add_exception_handler(AppError, handle_app_error)
app.add_exception_handler(RequestValidationError, handle_validation_error)
app.add_exception_handler(HTTPException, handle_http_error)
for cls in (OperationalError, InterfaceError, SATimeout, RedisError):
app.add_exception_handler(cls, handle_dependency)
app.middleware("http")(request_context)
app.include_router(health_router)
app.include_router(api_router)
return app
app = create_app()
+44
View File
@@ -0,0 +1,44 @@
import logging
import time
from uuid import uuid4
from fastapi import Request
from app.db import is_dependency_error
from app.errors import error_response
logger = logging.getLogger("app.access")
async def request_context(request: Request, call_next):
request_id = uuid4().hex
request.state.request_id = request_id
started = time.perf_counter()
status = 500
try:
accepting = getattr(request.app.state, "accepting", True)
if not accepting and request.url.path != "/health/live":
response = error_response(503, "unavailable", "Сервер останавливается", request_id)
else:
try:
response = await call_next(request)
except Exception as exc:
if is_dependency_error(exc):
logger.warning("dependency unavailable type=%s", type(exc).__name__)
response = error_response(503, "unavailable", "Сервис временно недоступен", request_id)
else:
logger.error("internal error request_id=%s type=%s", request_id, type(exc).__name__)
response = error_response(500, "internal_error", "Внутренняя ошибка", request_id)
response.headers["X-Request-Id"] = request_id
status = response.status_code
return response
finally:
elapsed_ms = int((time.perf_counter() - started) * 1000)
logger.info(
"request method=%s path=%s status=%s request_id=%s duration_ms=%s",
request.method,
request.url.path,
status,
request_id,
elapsed_ms,
)
+60
View File
@@ -0,0 +1,60 @@
import logging
from pathlib import Path
from sqlalchemy import text
from sqlalchemy.ext.asyncio import AsyncEngine
logger = logging.getLogger("app.migrate")
# Один номер на все процессы API, чтобы две реплики не применяли DDL одновременно.
_ADVISORY_LOCK = 842021
def sql_statements(script: str) -> list[str]:
lines = []
for line in script.splitlines():
if line.strip().startswith("--"):
continue
lines.append(line)
statements = []
for part in "\n".join(lines).split(";"):
statement = part.strip()
if statement:
statements.append(statement)
return statements
async def apply_migrations(engine: AsyncEngine, directory: Path) -> int:
files = sorted(path for path in directory.glob("*.sql") if path.is_file())
if not files:
raise SystemExit("no SQL migrations found")
applied = 0
async with engine.begin() as conn:
await conn.execute(text("SELECT pg_advisory_xact_lock(:lock_id)"), {"lock_id": _ADVISORY_LOCK})
await conn.execute(
text(
"""
CREATE TABLE IF NOT EXISTS schema_migrations (
version TEXT PRIMARY KEY,
applied_at TIMESTAMPTZ NOT NULL DEFAULT now()
)
"""
)
)
for path in files:
version = path.name
exists = await conn.scalar(
text("SELECT version FROM schema_migrations WHERE version = :version"),
{"version": version},
)
if exists:
continue
for statement in sql_statements(path.read_text(encoding="utf-8")):
await conn.execute(text(statement))
await conn.execute(
text("INSERT INTO schema_migrations (version) VALUES (:version)"),
{"version": version},
)
applied += 1
logger.info("migration applied version=%s", version)
return applied
+89
View File
@@ -0,0 +1,89 @@
import uuid
from datetime import datetime
from sqlalchemy import (
CheckConstraint,
DateTime,
ForeignKey,
Index,
SmallInteger,
Text,
UniqueConstraint,
)
from sqlalchemy.dialects.postgresql import UUID
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column
class Base(DeclarativeBase):
pass
class User(Base):
__tablename__ = "users"
__table_args__ = (
UniqueConstraint("username", name="uq_users_username"),
CheckConstraint("username ~ '^[a-z0-9_]{3,32}$'", name="ck_users_username_format"),
CheckConstraint(
"char_length(btrim(display_name)) >= 1 AND char_length(display_name) <= 100",
name="ck_users_display_name_len",
),
CheckConstraint("char_length(password_hash) >= 20", name="ck_users_password_hash_len"),
)
id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True)
username: Mapped[str] = mapped_column(Text, nullable=False)
display_name: Mapped[str] = mapped_column(Text, nullable=False)
password_hash: Mapped[str] = mapped_column(Text, nullable=False)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
class Chat(Base):
__tablename__ = "chats"
__table_args__ = (
CheckConstraint(
"char_length(btrim(title)) >= 1 AND char_length(title) <= 100",
name="ck_chats_title",
),
)
id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True)
title: Mapped[str] = mapped_column(Text, nullable=False)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
class ChatMember(Base):
__tablename__ = "chat_members"
__table_args__ = (
CheckConstraint("ordinal >= 0 AND ordinal < 100", name="ck_chat_members_ordinal"),
Index("ix_chat_members_user", "user_id", "chat_id"),
)
chat_id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
ForeignKey("chats.id"),
primary_key=True,
)
user_id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
ForeignKey("users.id"),
primary_key=True,
)
ordinal: Mapped[int] = mapped_column(SmallInteger, nullable=False)
class Message(Base):
__tablename__ = "messages"
__table_args__ = (
CheckConstraint(
"char_length(btrim(body)) >= 1 AND char_length(body) <= 4000",
name="ck_messages_text",
),
# Keyset: WHERE chat_id = $1 AND (created_at, id) > ($2, $3) ORDER BY created_at, id LIMIT $4
Index("ix_messages_history", "chat_id", "created_at", "id"),
)
id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True)
chat_id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), ForeignKey("chats.id"), nullable=False)
sender_id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), ForeignKey("users.id"), nullable=False)
text: Mapped[str] = mapped_column("body", Text, nullable=False)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
+50
View File
@@ -0,0 +1,50 @@
import base64
import json
from datetime import datetime
from uuid import UUID
from app.errors import AppError
from app.timeutil import format_time, parse_time
def encode_cursor(kind: str, resource_id: UUID, created_at: datetime, entity_id: UUID) -> str:
payload = {
"i": str(entity_id),
"k": kind,
"r": str(resource_id),
"t": format_time(created_at),
}
raw = json.dumps(payload, separators=(",", ":"), sort_keys=True).encode("ascii")
return base64.urlsafe_b64encode(raw).decode("ascii").rstrip("=")
def decode_position(token: str, kind: str, resource_id: UUID) -> tuple[datetime, UUID]:
if len(token) > 1024:
raise AppError(400, "invalid_request", "Некорректный cursor")
padding = "=" * (-len(token) % 4)
try:
data = json.loads(base64.urlsafe_b64decode(token + padding))
if (
not isinstance(data, dict)
or data.get("k") != kind
or data.get("r") != str(resource_id)
or not isinstance(data.get("t"), str)
or not isinstance(data.get("i"), str)
):
raise ValueError("cursor")
created_at = parse_time(data["t"])
entity_id = UUID(data["i"])
except (KeyError, TypeError, ValueError, AttributeError, json.JSONDecodeError):
raise AppError(400, "invalid_request", "Некорректный cursor") from None
return created_at, entity_id
def slice_page(rows: list, limit: int, kind: str, resource_id: UUID) -> dict:
# SQL уже вернул не больше limit + 1 строк, отфильтрованных keyset-условием.
has_more = len(rows) > limit
page = rows[:limit]
next_cursor = None
if has_more and page:
last = page[-1]
next_cursor = encode_cursor(kind, resource_id, last.created_at, last.id)
return {"items": [row.to_dict() for row in page], "next_cursor": next_cursor}
+55
View File
@@ -0,0 +1,55 @@
from dataclasses import dataclass
from datetime import datetime
from uuid import UUID
from app.timeutil import format_time
@dataclass(frozen=True)
class PublicUser:
id: UUID
username: str
display_name: str
created_at: datetime
def to_dict(self) -> dict:
return {
"id": self.id,
"username": self.username,
"display_name": self.display_name,
"created_at": format_time(self.created_at),
}
@dataclass(frozen=True)
class PublicChat:
id: UUID
title: str
member_ids: tuple[UUID, ...]
created_at: datetime
def to_dict(self) -> dict:
return {
"id": self.id,
"title": self.title,
"member_ids": list(self.member_ids),
"created_at": format_time(self.created_at),
}
@dataclass(frozen=True)
class PublicMessage:
id: UUID
chat_id: UUID
sender_id: UUID
text: str
created_at: datetime
def to_dict(self) -> dict:
return {
"id": self.id,
"chat_id": self.chat_id,
"sender_id": self.sender_id,
"text": self.text,
"created_at": format_time(self.created_at),
}
+36
View File
@@ -0,0 +1,36 @@
from app.config import Settings
_factory = None
_redis = None
_settings: Settings | None = None
def configure(factory, redis, settings: Settings) -> None:
global _factory, _redis, _settings
_factory = factory
_redis = redis
_settings = settings
def require_factory():
if _factory is None:
from app.errors import AppError
raise AppError(503, "unavailable", "Сервис временно недоступен")
return _factory
def require_redis():
if _redis is None:
from app.errors import AppError
raise AppError(503, "unavailable", "Сервис временно недоступен")
return _redis
def require_settings() -> Settings:
if _settings is None:
from app.errors import AppError
raise AppError(503, "unavailable", "Сервис временно недоступен")
return _settings
View File
+35
View File
@@ -0,0 +1,35 @@
from uuid import UUID
from pydantic import BaseModel, ConfigDict, Field, field_validator
class CreateChat(BaseModel):
model_config = ConfigDict(extra="forbid")
title: str
member_ids: list[UUID] = Field(min_length=1, max_length=100)
@field_validator("title")
@classmethod
def check_title(cls, value: str) -> str:
if value.strip() == "" or len(value) > 100:
raise ValueError("title")
return value
@field_validator("member_ids")
@classmethod
def unique_members(cls, value: list[UUID]) -> list[UUID]:
if len(set(value)) != len(value):
raise ValueError("duplicate members")
return value
class ChatOut(BaseModel):
id: UUID
title: str
member_ids: list[UUID]
created_at: str
class ChatPage(BaseModel):
items: list[ChatOut]
next_cursor: str | None
+28
View File
@@ -0,0 +1,28 @@
from uuid import UUID
from pydantic import BaseModel, ConfigDict, field_validator
class CreateMessage(BaseModel):
model_config = ConfigDict(extra="forbid")
text: str
@field_validator("text")
@classmethod
def check_text(cls, value: str) -> str:
if value.strip() == "" or len(value) > 4000:
raise ValueError("text")
return value
class MessageOut(BaseModel):
id: UUID
chat_id: UUID
sender_id: UUID
text: str
created_at: str
class MessagePage(BaseModel):
items: list[MessageOut]
next_cursor: str | None
+10
View File
@@ -0,0 +1,10 @@
from pydantic import BaseModel
from app.schemas.user import UserOut
class SessionOut(BaseModel):
token: str
token_type: str
expires_at: str
user: UserOut
+41
View File
@@ -0,0 +1,41 @@
import re
from uuid import UUID
from pydantic import BaseModel, ConfigDict, field_validator
class CreateUser(BaseModel):
model_config = ConfigDict(extra="forbid")
username: str
display_name: str
password: str
@field_validator("username")
@classmethod
def check_username(cls, value: str) -> str:
if re.fullmatch(r"[a-z0-9_]{3,32}", value) is None:
raise ValueError("username")
return value
@field_validator("display_name")
@classmethod
def check_display_name(cls, value: str) -> str:
if value.strip() == "" or len(value) > 100:
raise ValueError("display_name")
return value
@field_validator("password")
@classmethod
def check_password(cls, value: str) -> str:
# Длина в code points. Пароль не обрезается и не нормализуется.
if not 12 <= len(value) <= 128:
raise ValueError("password")
return value
class UserOut(BaseModel):
id: UUID
username: str
display_name: str
created_at: str
+66
View File
@@ -0,0 +1,66 @@
import base64
import binascii
from argon2 import PasswordHasher
from argon2.exceptions import InvalidHashError, VerifyMismatchError
from argon2.low_level import Type
# Учебные параметры Argon2id: устойчивее SHA-256, но подъём на ноутбуке
# остаётся коротким. В lab7 параметры будут заданы курсом отдельно.
_TIME_COST = 2
_MEMORY_COST = 32768
_PARALLELISM = 1
_hasher: PasswordHasher | None = None
_dummy_hash: str | None = None
def get_hasher() -> PasswordHasher:
global _hasher
if _hasher is None:
_hasher = PasswordHasher(
time_cost=_TIME_COST,
memory_cost=_MEMORY_COST,
parallelism=_PARALLELISM,
hash_len=32,
salt_len=16,
type=Type.ID,
)
return _hasher
def init_security() -> None:
global _dummy_hash
if _dummy_hash is None:
_dummy_hash = get_hasher().hash("messenger-dummy-password-not-used-for-login")
def hash_password(password: str) -> str:
return get_hasher().hash(password)
def verify_password(password: str, password_hash: str | None) -> bool:
init_security()
try:
get_hasher().verify(password_hash or _dummy_hash, password)
matched = True
except (VerifyMismatchError, InvalidHashError):
matched = False
return password_hash is not None and matched
def parse_basic(header: str) -> tuple[str, str] | None:
if not header:
return None
scheme, _, rest = header.partition(" ")
if scheme.lower() != "basic" or not rest.strip():
return None
try:
decoded = base64.b64decode(rest.strip(), validate=True)
text = decoded.decode("utf-8")
except (ValueError, UnicodeError, binascii.Error):
return None
username, separator, password = text.partition(":")
if separator != ":" or len(username) > 256 or len(password) > 1024:
return None
return username, password
+90
View File
@@ -0,0 +1,90 @@
import hashlib
import json
import logging
import secrets
from datetime import timedelta
from uuid import UUID
from redis.asyncio import Redis
from redis.exceptions import RedisError
from app.config import Settings
from app.errors import AppError
from app.runtime import require_redis, require_settings
from app.timeutil import format_time, parse_time, utcnow
logger = logging.getLogger("app.sessions")
def build_redis(settings: Settings) -> Redis:
return Redis.from_url(
settings.cache_url,
decode_responses=True,
socket_timeout=settings.redis_socket_timeout_seconds,
socket_connect_timeout=settings.redis_socket_timeout_seconds,
health_check_interval=30,
max_connections=settings.redis_max_connections,
)
def session_key(token: str) -> str:
digest = hashlib.sha256(token.encode("utf-8")).hexdigest()
return "sess:" + digest
async def _call(awaitable):
try:
return await awaitable
except RedisError:
logger.warning("redis unavailable type=%s", "RedisError")
raise AppError(503, "unavailable", "Сервис временно недоступен") from None
def _read_payload(raw: str) -> tuple[UUID, object] | None:
try:
data = json.loads(raw)
if not isinstance(data, dict):
raise ValueError("payload")
return UUID(str(data["user_id"])), parse_time(data["expires_at"])
except (ValueError, KeyError, TypeError):
return None
async def create_session(user_id: UUID) -> tuple[str, object]:
settings = require_settings()
token = secrets.token_urlsafe(32)
expires_at = utcnow() + timedelta(seconds=settings.session_ttl_seconds)
payload = json.dumps(
{"user_id": str(user_id), "expires_at": format_time(expires_at)},
separators=(",", ":"),
)
# PXAT — абсолютное время. Повтор AOF после рестарта не продлевает TTL.
await _call(require_redis().set(session_key(token), payload, pxat=int(expires_at.timestamp() * 1000)))
return token, expires_at
async def resolve_session(token: str) -> UUID | None:
key = session_key(token)
raw = await _call(require_redis().get(key))
if raw is None:
return None
parsed = _read_payload(raw)
if parsed is None:
await _call(require_redis().delete(key))
return None
user_id, expires_at = parsed
if expires_at <= utcnow():
await _call(require_redis().delete(key))
return None
return user_id
async def revoke_session(token: str) -> bool:
raw = await _call(require_redis().getdel(session_key(token)))
if raw is None:
return False
parsed = _read_payload(raw)
if parsed is None:
return False
_, expires_at = parsed
return expires_at > utcnow()
+1
View File
@@ -0,0 +1 @@
"""Состояние лабораторной 2 хранится в PostgreSQL и Redis, не в памяти процесса."""
+5
View File
@@ -0,0 +1,5 @@
"""Память процесса больше не хранит пользователей, чаты и сообщения.
Источник истины для этих данных — PostgreSQL.
Источник истины для сессий — Redis с AOF и абсолютным PXAT.
"""
+20
View File
@@ -0,0 +1,20 @@
from datetime import datetime, timezone
def utcnow() -> datetime:
return datetime.now(timezone.utc)
def format_time(value: datetime) -> str:
if value.tzinfo is None:
value = value.replace(tzinfo=timezone.utc)
return value.astimezone(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S.%fZ")
def parse_time(value: str) -> datetime:
if not isinstance(value, str) or not value.endswith("Z"):
raise ValueError("time")
parsed = datetime.fromisoformat(value[:-1] + "+00:00")
if parsed.tzinfo is None:
raise ValueError("time")
return parsed.astimezone(timezone.utc)
+630
View File
@@ -0,0 +1,630 @@
#!/usr/bin/env python3
"""Проверки уровня 4: гонка, откат, EXPLAIN, TTL, logout и отказ Redis/PostgreSQL.
Стенд уже должен быть поднят: scripts/bootstrap и docker compose up.
Скрипт останавливает redis и postgres и снова их запускает.
Токены и пароли не печатаются и не пишутся в evidence.
"""
import asyncio
import base64
import json
import os
import socket
import subprocess
import sys
import threading
import time
import urllib.error
import urllib.request
import uuid
from datetime import datetime, timezone
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
EVIDENCE = ROOT / "evidence"
sys.path.insert(0, str(ROOT / "src"))
BASE = os.environ.get("BASE_URL", "http://127.0.0.1:8080").rstrip("/")
class Failure(AssertionError):
pass
def check(condition, message):
if not condition:
raise Failure(message)
def host_secret(name):
path = ROOT / "secrets" / name
if not path.is_file():
raise Failure("missing secrets/%s; run scripts/bootstrap first" % name)
value = path.read_text(encoding="utf-8").strip()
if not value:
raise Failure("empty secrets/%s" % name)
return value
def call(method, path, expected, body=None, token=None, authorization=None, user_id=None, timeout=15):
headers = {"Accept": "application/json"}
data = None
if body is not None:
data = json.dumps(body, ensure_ascii=False).encode("utf-8")
headers["Content-Type"] = "application/json"
if authorization:
headers["Authorization"] = authorization
elif token:
headers["Authorization"] = "Bearer " + token
if user_id:
headers["X-User-Id"] = user_id
request = urllib.request.Request(BASE + path, data=data, headers=headers, method=method)
try:
response = urllib.request.urlopen(request, timeout=timeout)
except urllib.error.HTTPError as error:
response = error
except (urllib.error.URLError, TimeoutError, OSError) as exc:
raise Failure(method + " " + path.split("?")[0] + " failed: " + type(exc).__name__) from None
with response:
status = response.status
payload = response.read()
header_id = response.headers.get("X-Request-Id")
challenge = response.headers.get("WWW-Authenticate", "")
allowed = [expected] if isinstance(expected, int) else list(expected)
check(status in allowed, "%s %s: expected %s, got %s" % (method, path.split("?")[0], allowed, status))
check(header_id and header_id.strip(), "missing X-Request-Id")
parsed = None
if payload:
try:
parsed = json.loads(payload)
except json.JSONDecodeError as exc:
raise Failure("response is not JSON") from exc
if status >= 400:
check(isinstance(parsed, dict) and set(parsed) == {"error"}, "error envelope")
error = parsed["error"]
check(set(error) == {"code", "message", "request_id"}, "error fields")
check(error["request_id"] == header_id, "request_id differs from header")
lowered = error["message"].lower()
for leaked in ("traceback", "select ", "password", "syntax", "asyncpg", "sqlalchemy"):
check(leaked not in lowered, "error message leaked internals")
if status == 204:
check(not payload, "204 must not contain a body")
return {"status": status, "json": parsed, "www": challenge}
def basic(username, password):
raw = base64.b64encode(("%s:%s" % (username, password)).encode("utf-8")).decode("ascii")
return "Basic " + raw
def register(label, password=None):
username = ("u_%s_%s" % (label, uuid.uuid4().hex[:10])).lower()
password = password or ("T3st-" + uuid.uuid4().hex)
created = call(
"POST",
"/api/v1/users",
201,
body={"username": username, "display_name": "Студент " + label, "password": password},
)["json"]
check("password" not in created, "password leaked")
session = call("POST", "/api/v1/auth/sessions", 201, authorization=basic(username, password))["json"]
check(session["token_type"] == "Bearer" and session["user"]["id"] == created["id"], "session shape")
check(len(session["token"]) >= 32, "token is too short")
return {"id": created["id"], "username": username, "password": password, "token": session["token"]}
def compose(*args, timeout=120):
proc = subprocess.run(
["docker", "compose", *args],
cwd=str(ROOT),
capture_output=True,
text=True,
timeout=timeout,
encoding="utf-8",
errors="replace",
)
if proc.returncode != 0:
raise Failure("docker compose %s failed with code %s" % (args[0], proc.returncode))
return proc.stdout
def psql(sql, timeout=60):
proc = subprocess.run(
[
"docker",
"compose",
"exec",
"-T",
"postgres",
"psql",
"-U",
"messenger",
"-d",
"messenger",
"-v",
"ON_ERROR_STOP=1",
"-At",
"-c",
sql,
],
cwd=str(ROOT),
capture_output=True,
text=True,
timeout=timeout,
encoding="utf-8",
errors="replace",
)
if proc.returncode != 0:
raise Failure("psql failed")
return proc.stdout.strip()
def redis_cli(*args):
proc = subprocess.run(
["docker", "compose", "exec", "-T", "redis", "redis-cli", *args],
cwd=str(ROOT),
capture_output=True,
text=True,
timeout=20,
encoding="utf-8",
errors="replace",
)
if proc.returncode != 0:
raise Failure("redis-cli failed")
return proc.stdout.strip()
def sql_quote(value):
text = str(value)
if "'" in text or ";" in text or "\n" in text:
raise Failure("refusing unsafe SQL literal")
return "'" + text + "'"
def free_port():
with socket.socket() as sock:
sock.bind(("127.0.0.1", 0))
return sock.getsockname()[1]
def wait_ready(timeout=90):
deadline = time.monotonic() + timeout
last = ""
while time.monotonic() < deadline:
try:
call("GET", "/health/ready", 200, timeout=5)
return
except Failure as exc:
last = str(exc)
time.sleep(0.5)
raise Failure("API did not become ready: " + last)
def test_credentials():
user = register("cred")
wrong = call(
"POST",
"/api/v1/auth/sessions",
401,
authorization=basic(user["username"], "wrong-password-value"),
)
missing = call(
"POST",
"/api/v1/auth/sessions",
401,
authorization=basic("no_such_user_zz", "wrong-password-value"),
)
check(wrong["json"]["error"]["code"] == "unauthorized", "wrong password code")
check(wrong["json"]["error"]["message"] == missing["json"]["error"]["message"], "credential errors differ")
check(wrong["www"].lower().startswith("basic "), "basic challenge")
bare = call("GET", "/api/v1/users/me", 401)
check(bare["www"].lower().startswith("bearer "), "bearer challenge")
only_header = call("GET", "/api/v1/users/me", 401, user_id=user["id"])
check(only_header["json"]["error"]["code"] == "unauthorized", "X-User-Id alone was accepted")
other = register("other")
me = call("GET", "/api/v1/users/me", 200, token=user["token"], user_id=other["id"])["json"]
check(me["id"] == user["id"], "X-User-Id overrode bearer")
basic_on_api = call("GET", "/api/v1/users/me", 401, authorization=basic(user["username"], user["password"]))
check(basic_on_api["www"].lower().startswith("bearer "), "basic accepted on business API")
print("PASS: credentials, same 401, X-User-Id ignored")
def test_unicode_password():
password = "Пароль:пример-12"
check(12 <= len(password) <= 128 and ":" in password, "fixture password")
user = register("uni", password=password)
again = call("POST", "/api/v1/auth/sessions", 201, authorization=basic(user["username"], password))["json"]
check(again["user"]["id"] == user["id"], "unicode password login failed")
print("PASS: unicode password with colon")
def test_logout():
user = register("bye")
call("DELETE", "/api/v1/auth/sessions/current", 204, token=user["token"])
rejected = call("GET", "/api/v1/users/me", 401, token=user["token"])
check(rejected["www"].lower().startswith("bearer "), "revoked token challenge")
again = call("DELETE", "/api/v1/auth/sessions/current", 401, token=user["token"])
check(again["json"]["error"]["code"] == "unauthorized", "second logout")
print("PASS: logout revokes the session")
def test_injection_and_rollback():
user = register("sql")
chat = call(
"POST",
"/api/v1/chats",
201,
token=user["token"],
body={"title": "SQL", "member_ids": [user["id"]]},
)["json"]
payload = "'; DROP TABLE messages; --"
sent = call(
"POST",
"/api/v1/chats/%s/messages" % chat["id"],
201,
token=user["token"],
body={"text": payload},
)["json"]
check(sent["text"] == payload, "text was rewritten")
check(psql("SELECT to_regclass('public.messages')") == "messages", "messages table disappeared")
listed = call("GET", "/api/v1/chats/%s/messages?limit=10" % chat["id"], 200, token=user["token"])["json"]
check(listed["items"][0]["text"] == payload, "stored text differs")
before = int(psql("SELECT count(*) FROM chats"))
title = "partial-" + uuid.uuid4().hex
rejected = call(
"POST",
"/api/v1/chats",
400,
token=user["token"],
body={"title": title, "member_ids": [user["id"], str(uuid.uuid4())]},
)
check(rejected["json"]["error"]["code"] == "invalid_request", "unknown member code")
after = int(psql("SELECT count(*) FROM chats"))
check(before == after, "unknown member committed a chat")
check(psql("SELECT count(*) FROM chats WHERE title = %s" % sql_quote(title)) == "0", "partial chat row exists")
print("PASS: parameterized text and unknown-member chat is not stored")
return user, chat
async def failpoint(owner_id, other_id, title):
os.environ["DATABASE_URL"] = host_secret("host_database_url")
os.environ["REDIS_URL"] = host_secret("host_cache_url")
from app.config import load_settings
from app.crud.chats import create_chat
from app.db import build_engine, build_session_factory
from app.runtime import configure
from app.sessions import build_redis
settings = load_settings()
engine = build_engine(settings)
redis = build_redis(settings)
configure(build_session_factory(engine), redis, settings)
try:
try:
await create_chat(
uuid.UUID(owner_id),
title,
(uuid.UUID(owner_id), uuid.UUID(other_id)),
fail_after_insert=True,
)
except RuntimeError as exc:
check(str(exc) == "rollback-check", "failpoint raised a different error")
else:
raise Failure("failpoint committed a chat")
finally:
await redis.aclose()
await engine.dispose()
def test_failpoint(owner_id, other_id):
title = "rollback-" + uuid.uuid4().hex
before = int(psql("SELECT count(*) FROM chats"))
try:
asyncio.run(failpoint(owner_id, other_id, title))
except Failure:
raise
except Exception as exc:
raise Failure("rollback check failed: " + type(exc).__name__) from None
after = int(psql("SELECT count(*) FROM chats"))
left = psql("SELECT count(*) FROM chats WHERE title = %s" % sql_quote(title))
check(before == after and left == "0", "chat insert survived the rollback")
print("PASS: composite chat insert rolls back before commit")
return {"chats_before": before, "chats_after": after, "title_rows": int(left)}
def test_race():
username = "race_" + uuid.uuid4().hex[:8]
password = "Race-password-" + uuid.uuid4().hex[:8]
body = {"username": username, "display_name": "Гонка", "password": password}
barrier = threading.Barrier(16)
statuses = []
lock = threading.Lock()
def worker():
try:
barrier.wait(20)
result = call("POST", "/api/v1/users", [201, 409], body=body, timeout=30)
with lock:
statuses.append(result["status"])
except Exception as exc:
with lock:
statuses.append(str(exc)[:120])
threads = [threading.Thread(target=worker) for _ in range(16)]
for thread in threads:
thread.start()
for thread in threads:
thread.join()
created = statuses.count(201)
conflict = statuses.count(409)
check(created == 1 and conflict == 15 and len(statuses) == 16, "race statuses %s" % statuses)
print("PASS: 16 parallel registrations, one 201 and fifteen 409")
return {"created": created, "conflict": conflict}
def assert_plan(name, plan):
check("ix_messages_history" in plan, name + " plan ignores ix_messages_history")
check("Seq Scan" not in plan, name + " plan seq-scans messages")
check("Index Scan" in plan or "Index Only Scan" in plan, name + " plan is not an index scan")
check("Buffers:" in plan, name + " plan has no buffer stats")
def test_explain(user):
chat = call(
"POST",
"/api/v1/chats",
201,
token=user["token"],
body={"title": "План", "member_ids": [user["id"]]},
)["json"]
chat_sql = sql_quote(chat["id"])
user_sql = sql_quote(user["id"])
psql(
"INSERT INTO messages (id, chat_id, sender_id, body, created_at) "
"SELECT gen_random_uuid(), %s, %s, 'row-' || g, "
"TIMESTAMPTZ '2026-01-01 00:00:00+00' + (g || ' microseconds')::interval "
"FROM generate_series(1, 10000) AS g" % (chat_sql, user_sql),
timeout=60,
)
count = int(psql("SELECT count(*) FROM messages WHERE chat_id = %s" % chat_sql))
check(count >= 10000, "expected at least 10000 messages, got %s" % count)
psql("ANALYZE messages")
midpoint = psql(
"SELECT to_char(created_at AT TIME ZONE 'UTC', 'YYYY-MM-DD HH24:MI:SS.US'), id::text "
"FROM messages WHERE chat_id = %s ORDER BY created_at, id OFFSET 5000 LIMIT 1" % chat_sql
)
created_at, message_id = midpoint.split("|")
late = psql(
"EXPLAIN (ANALYZE, BUFFERS) SELECT id, body FROM messages "
"WHERE chat_id = %s AND (created_at, id) > (%s::timestamptz, %s::uuid) "
"ORDER BY created_at, id LIMIT 50"
% (chat_sql, sql_quote(created_at + "+00"), sql_quote(message_id)),
timeout=60,
)
first = psql(
"EXPLAIN (ANALYZE, BUFFERS) SELECT id, body FROM messages "
"WHERE chat_id = %s ORDER BY created_at, id LIMIT 50" % chat_sql,
timeout=60,
)
EVIDENCE.mkdir(exist_ok=True)
(EVIDENCE / "explain_messages.txt").write_text(
"messages=%s\n\n-- first page\n%s\n\n-- page after row 5000\n%s\n" % (count, first, late),
encoding="utf-8",
)
assert_plan("first", first)
assert_plan("late", late)
print("PASS: EXPLAIN uses ix_messages_history on %s messages" % count)
return {"messages": count, "chat_id": chat["id"]}
def test_expiry_and_migrations():
from app.timeutil import parse_time
users_before = int(psql("SELECT count(*) FROM users"))
port = free_port()
env = os.environ.copy()
env["PYTHONPATH"] = str(ROOT / "src")
env["PYTHONUNBUFFERED"] = "1"
env["APP_HOST"] = "127.0.0.1"
env["APP_PORT"] = str(port)
env["APP_ENV"] = "lab"
env["DATABASE_URL"] = host_secret("host_database_url")
env["REDIS_URL"] = host_secret("host_cache_url")
env["SESSION_TTL_SECONDS"] = "2"
env.pop("DATABASE_URL_FILE", None)
env.pop("CACHE_URL_FILE", None)
proc = subprocess.Popen(
[sys.executable, "-m", "app"],
cwd=str(ROOT),
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True,
encoding="utf-8",
errors="replace",
)
logs = []
def read_logs():
if proc.stdout is None:
return
for line in proc.stdout:
if "://" not in line and "password" not in line.lower():
logs.append(line)
reader = threading.Thread(target=read_logs, daemon=True)
reader.start()
base = "http://127.0.0.1:%s" % port
try:
deadline = time.monotonic() + 40
ready = False
while time.monotonic() < deadline:
if proc.poll() is not None:
raise Failure("short-ttl server exited: " + "".join(logs[-8:]))
try:
urllib.request.urlopen(base + "/health/ready", timeout=2).read()
ready = True
break
except (urllib.error.URLError, TimeoutError, OSError):
time.sleep(0.2)
check(ready, "short-ttl server did not become ready: " + "".join(logs[-8:]))
username = "ttl_" + uuid.uuid4().hex[:8]
password = "Ttl-password-" + uuid.uuid4().hex[:8]
body = json.dumps(
{"username": username, "display_name": "TTL", "password": password},
ensure_ascii=False,
).encode("utf-8")
request = urllib.request.Request(
base + "/api/v1/users",
data=body,
headers={"Content-Type": "application/json", "Accept": "application/json"},
method="POST",
)
with urllib.request.urlopen(request, timeout=20) as response:
created = json.loads(response.read())
login = urllib.request.Request(
base + "/api/v1/auth/sessions",
data=b"",
headers={"Authorization": basic(username, password), "Accept": "application/json"},
method="POST",
)
with urllib.request.urlopen(login, timeout=20) as response:
session = json.loads(response.read())
expires = parse_time(session["expires_at"])
delta = (expires - datetime.now(timezone.utc)).total_seconds()
check(0 < delta <= 3, "SESSION_TTL_SECONDS=2 was not applied")
time.sleep(delta + 1.0)
probe = urllib.request.Request(
base + "/api/v1/users/me",
headers={"Authorization": "Bearer " + session["token"], "Accept": "application/json"},
)
try:
with urllib.request.urlopen(probe, timeout=10) as response:
status = response.status
except urllib.error.HTTPError as error:
status = error.status
check(status == 401, "expired session status %s" % status)
finally:
if proc.poll() is None:
proc.terminate()
try:
proc.wait(timeout=8)
except subprocess.TimeoutExpired:
proc.kill()
proc.wait(timeout=5)
reader.join(timeout=2)
users_after = int(psql("SELECT count(*) FROM users"))
versions = psql("SELECT count(*) FROM schema_migrations")
version_name = psql("SELECT version FROM schema_migrations")
check(users_after == users_before + 1, "migrations duplicated or dropped users")
check(versions == "1" and version_name == "001_initial.sql", "migration version changed")
print("PASS: short TTL expires and migrations stay idempotent")
return {"users_before": users_before, "users_after": users_after, "migration": version_name}
def test_redis_persistence(user):
from app.sessions import session_key
before = int(redis_cli("PTTL", session_key(user["token"])))
check(before > 300000, "session ttl is unexpectedly short")
compose("restart", "redis")
wait_ready()
after = int(redis_cli("PTTL", session_key(user["token"])))
check(after > 0, "session disappeared after redis restart")
check(after <= before + 1000, "redis restart extended the session ttl")
me = call("GET", "/api/v1/users/me", 200, token=user["token"])["json"]
check(me["id"] == user["id"], "token did not survive redis restart")
print("PASS: Redis AOF kept the session and did not refresh TTL")
return {"pttl_before_ms": before, "pttl_after_ms": after}
def test_dependency_loss(user, chat_id):
observed = {}
compose("stop", "redis")
try:
live = call("GET", "/health/live", 200)
ready = call("GET", "/health/ready", 503)
anonymous = call("GET", "/api/v1/users/me", 401)
trusted = call("GET", "/api/v1/users/me", 503, token=user["token"])
check(live["json"]["status"] == "ok", "live failed while redis is down")
check(ready["json"]["error"]["code"] == "unavailable", "ready code")
check(anonymous["json"]["error"]["code"] == "unauthorized", "anonymous became trusted")
check(trusted["json"]["error"]["code"] == "unavailable", "unverified token was accepted")
observed["redis_down"] = {
"live": live["status"],
"ready": ready["status"],
"anonymous": anonymous["status"],
"bearer": trusted["status"],
}
finally:
compose("start", "redis")
wait_ready()
me = call("GET", "/api/v1/users/me", 200, token=user["token"])["json"]
check(me["id"] == user["id"], "token lost after redis came back")
compose("stop", "postgres")
try:
live = call("GET", "/health/live", 200)
ready = call("GET", "/health/ready", 503, timeout=20)
anonymous = call("GET", "/api/v1/users/me", 401, timeout=20)
denied = call("GET", "/api/v1/chats/" + chat_id, 503, token=user["token"], timeout=20)
check(live["json"]["status"] == "ok", "live depends on postgres")
check(ready["json"]["error"]["code"] == "unavailable", "ready while postgres is down")
check(anonymous["json"]["error"]["code"] == "unauthorized", "anonymous trusted without postgres")
check(denied["json"]["error"]["code"] == "unavailable", "request served without postgres")
observed["postgres_down"] = {
"live": live["status"],
"ready": ready["status"],
"anonymous": anonymous["status"],
"bearer": denied["status"],
}
finally:
compose("start", "postgres")
wait_ready(timeout=90)
history = call("GET", "/api/v1/chats/%s/messages?limit=5" % chat_id, 200, token=user["token"])["json"]
check(history["items"], "messages lost after postgres restart")
print("PASS: redis and postgres outages return 503 and do not trust anonymous clients")
return observed
def main():
os.environ["DATABASE_URL"] = host_secret("host_database_url")
os.environ["REDIS_URL"] = host_secret("host_cache_url")
wait_ready(timeout=60)
test_credentials()
test_unicode_password()
test_logout()
user, chat = test_injection_and_rollback()
other = register("pair")
rollback = test_failpoint(user["id"], other["id"])
race = test_race()
explain = test_explain(user)
expiry = test_expiry_and_migrations()
pttl = test_redis_persistence(user)
faults = test_dependency_loss(user, chat["id"])
remaining = int(psql("SELECT count(*) FROM messages WHERE chat_id = %s" % sql_quote(explain["chat_id"])))
check(remaining == explain["messages"], "bulk history lost after postgres restart")
EVIDENCE.mkdir(exist_ok=True)
report = {
"date": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
"base_url": BASE,
"race": race,
"rollback": rollback,
"explain": {"messages": explain["messages"]},
"expiry_and_migrations": expiry,
"redis_restart": pttl,
"dependency_loss": faults,
"messages_after_postgres_restart": remaining,
}
(EVIDENCE / "level4.json").write_text(json.dumps(report, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
print("PASS: level 4 checks")
if __name__ == "__main__":
try:
main()
except Failure as exc:
print("FAIL:", exc)
sys.exit(1)
+69
View File
@@ -0,0 +1,69 @@
#!/usr/bin/env python3
"""Тот же сценарий, что tests/resilience.py, но hook запускается так, как умеет ОС.
Linux выполняет scripts/restart.sh напрямую.
Windows не умеет CreateProcess для shell-скрипта, поэтому здесь используется Git Bash.
Скрипт тот же: перезапуск api, postgres и redis без удаления томов.
"""
import os
import subprocess
import sys
import time
from pathlib import Path
from urllib.parse import quote
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "tests"))
from client import Client, Failure, check, run
def hook_command():
script = ROOT / "scripts" / "restart.sh"
if os.name != "nt":
return [str(script)]
bash = Path(os.environ.get("BASH", r"C:\Program Files\Git\bin\bash.exe"))
check(bash.is_file(), "Git Bash is required to launch scripts/restart.sh on Windows")
return [str(bash), "-c", "sh scripts/restart.sh"]
def history(client, actor, chat):
path = "/api/v1/chats/" + chat["id"] + "/messages?limit=100"
result = []
seen = set()
for _ in range(1000):
page = client.call("GET", path, actor=actor, schema="MessagePage")["json"]
result.extend(page["items"])
cursor = page["next_cursor"]
if cursor is None:
return result
check(cursor not in seen, "cursor cycle")
seen.add(cursor)
path = "/api/v1/chats/" + chat["id"] + "/messages?limit=100&cursor=" + quote(cursor, safe="")
raise Failure("history pagination exceeded safety bound")
def main():
client = Client("http://127.0.0.1:8080", timeout=5)
alice, bob, _eve, chat = client.bootstrap()
original = client.message(alice, chat, "Сохранить при отказе")["json"]
print("Executing scripts/restart.sh")
completed = subprocess.run(hook_command(), cwd=str(ROOT))
check(completed.returncode == 0, "restart hook failed")
deadline = time.monotonic() + 90
while True:
try:
client.call("GET", "/health/ready", schema="Health")
client.call("GET", "/api/v1/users/me", actor=alice, schema="User")
break
except Failure:
check(time.monotonic() < deadline, "stack/session did not recover within 90 seconds")
time.sleep(0.5)
messages = history(client, bob, chat)
check(any(item == original for item in messages), "confirmed message lost or altered")
print("PASS: persistence preserved confirmed data and existing session")
if __name__ == "__main__":
run(main)
+508
View File
@@ -0,0 +1,508 @@
#!/usr/bin/env python3
import base64
import json
import os
import socket
import subprocess
import sys
import threading
import time
import urllib.error
import urllib.request
import uuid
from pathlib import Path
from urllib.parse import quote
ROOT = Path(__file__).resolve().parents[1]
EVIDENCE = ROOT / "evidence"
SECRET = "SECRET_MESSAGE_DO_NOT_LOG_7f3a"
CLIENTS = 10
PER_CLIENT = 10
class Failure(AssertionError):
pass
def check(condition, message):
if not condition:
raise Failure(message)
def free_port():
with socket.socket() as sock:
sock.bind(("127.0.0.1", 0))
return sock.getsockname()[1]
class Server:
def __init__(self, port):
self.base = "http://127.0.0.1:%d" % port
env = os.environ.copy()
env["PYTHONPATH"] = str(ROOT / "src")
env["PYTHONUNBUFFERED"] = "1"
env["APP_HOST"] = "127.0.0.1"
env["APP_PORT"] = str(port)
env["APP_ENV"] = "lab"
env["DATABASE_URL"] = host_secret("host_database_url")
env["REDIS_URL"] = host_secret("host_cache_url")
env["SESSION_TTL_SECONDS"] = "3600"
env.pop("DATABASE_URL_FILE", None)
env.pop("CACHE_URL_FILE", None)
self.logs = []
self.proc = subprocess.Popen(
[sys.executable, "-m", "app"],
cwd=str(ROOT),
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True,
encoding="utf-8",
errors="replace",
)
self._reader = threading.Thread(target=self._read, daemon=True)
self._reader.start()
def _read(self):
assert self.proc.stdout is not None
for line in self.proc.stdout:
self.logs.append(line)
def wait_ready(self):
deadline = time.monotonic() + 40
last = ""
while time.monotonic() < deadline:
if self.proc.poll() is not None:
raise Failure("server exited early:\n" + "".join(self.logs[-40:]))
try:
call(self.base, "GET", "/health/ready", 200)
return
except Failure as exc:
last = str(exc)
except (urllib.error.URLError, TimeoutError, OSError) as exc:
last = type(exc).__name__
time.sleep(0.1)
raise Failure("server did not become ready: " + last + "\n" + "".join(self.logs[-40:]))
def stop(self):
if self.proc.poll() is None:
self.proc.terminate()
try:
self.proc.wait(timeout=8)
except subprocess.TimeoutExpired:
self.proc.kill()
self.proc.wait(timeout=5)
self._reader.join(timeout=2)
def output(self):
return "".join(self.logs)
def host_secret(name):
path = ROOT / "secrets" / name
if not path.is_file():
raise Failure("missing secrets/%s; run scripts/bootstrap.ps1 or scripts/bootstrap.sh" % name)
value = path.read_text(encoding="utf-8").strip()
if not value:
raise Failure("empty secrets/%s" % name)
return value
def call(
base,
method,
path,
expected,
body=None,
token=None,
user_id=None,
authorization=None,
raw=None,
content_type="application/json",
timeout=20,
):
headers = {"Accept": "application/json"}
data = None
if raw is not None:
data = raw
headers["Content-Type"] = content_type
elif body is not None:
data = json.dumps(body, ensure_ascii=False).encode("utf-8")
headers["Content-Type"] = content_type
if authorization:
headers["Authorization"] = authorization
elif token:
headers["Authorization"] = "Bearer " + token
if user_id:
headers["X-User-Id"] = user_id
request = urllib.request.Request(base + path, data=data, headers=headers, method=method)
try:
response = urllib.request.urlopen(request, timeout=timeout)
except urllib.error.HTTPError as error:
response = error
except (urllib.error.URLError, TimeoutError, OSError) as exc:
raise Failure(method + " " + path.split("?")[0] + " failed: " + type(exc).__name__) from None
with response:
status = response.status
payload = response.read()
header_id = response.headers.get("X-Request-Id")
content_type_value = response.headers.get_content_type()
allowed = [expected] if isinstance(expected, int) else list(expected)
check(status in allowed, "%s %s: expected %s, got %s" % (method, path.split("?")[0], allowed, status))
check(header_id and header_id.strip() and "," not in header_id, "missing or duplicated X-Request-Id")
parsed = None
if payload:
check(content_type_value == "application/json", "expected application/json")
try:
parsed = json.loads(payload)
except json.JSONDecodeError as exc:
raise Failure("response is not JSON") from exc
if status >= 400:
check(isinstance(parsed, dict) and set(parsed) == {"error"}, "error envelope")
error = parsed["error"]
check(set(error) == {"code", "message", "request_id"}, "error fields")
check(error["request_id"] == header_id, "request_id differs from header")
check(error["message"] and "Traceback" not in error["message"], "error message leaked internals")
return {"status": status, "json": parsed, "request_id": header_id}
def create_user(base, label):
username = ("u_%s_%s" % (label, uuid.uuid4().hex[:10])).lower()
password = "T3st-" + uuid.uuid4().hex
body = {"username": username, "display_name": "Студент " + label, "password": password}
created = call(base, "POST", "/api/v1/users", 201, body=body)["json"]
check(created["username"] == username, "username changed")
check("password" not in created, "password leaked in user response")
raw = base64.b64encode(("%s:%s" % (username, password)).encode("utf-8")).decode("ascii")
session = call(base, "POST", "/api/v1/auth/sessions", 201, authorization="Basic " + raw)["json"]
check(session["token_type"] == "Bearer", "token type")
check(session["user"]["id"] == created["id"], "session user")
check("password" not in session["user"], "password leaked in session")
created["token"] = session["token"]
return created
def replace_cursor(token, fields):
padding = "=" * (-len(token) % 4)
data = json.loads(base64.urlsafe_b64decode(token + padding))
data.update(fields)
raw = json.dumps(data, separators=(",", ":"), sort_keys=True).encode("ascii")
return base64.urlsafe_b64encode(raw).decode("ascii").rstrip("=")
def test_invalid_json(base):
broken = call(base, "POST", "/api/v1/users", 400, raw=b"{")
check(broken["json"]["error"]["code"] == "invalid_request", "malformed JSON code")
extra = call(
base,
"POST",
"/api/v1/users",
400,
body={"username": "abc_user", "display_name": "Имя", "extra": True},
)
check(extra["json"]["error"]["code"] == "invalid_request", "extra field code")
bad_utf = call(base, "POST", "/api/v1/users", 400, raw=b"\xff")
check(bad_utf["json"]["error"]["code"] == "invalid_request", "invalid utf-8 code")
alice = create_user(base, "utf")
chat = call(
base,
"POST",
"/api/v1/chats",
201,
token=alice["token"],
body={"title": "UTF", "member_ids": [alice["id"]]},
)["json"]
bad_message = call(
base,
"POST",
"/api/v1/chats/%s/messages" % chat["id"],
400,
token=alice["token"],
raw=b"\xff",
)
check(bad_message["json"]["error"]["code"] == "invalid_request", "invalid utf-8 message code")
print("PASS: invalid JSON")
def test_duplicate_username(base):
first = create_user(base, "dup")
again = call(
base,
"POST",
"/api/v1/users",
409,
body={
"username": first["username"],
"display_name": "Другой",
"password": "T3st-" + uuid.uuid4().hex,
},
)
check(again["json"]["error"]["code"] == "conflict", "duplicate username code")
me = call(base, "GET", "/api/v1/users/me", 200, token=first["token"])["json"]
check(me["id"] == first["id"], "first user disappeared after duplicate")
print("PASS: duplicate username")
def test_empty_and_foreign(base):
alice = create_user(base, "alice")
bob = create_user(base, "bob")
chat = call(
base,
"POST",
"/api/v1/chats",
201,
token=alice["token"],
body={"title": "Только Алиса", "member_ids": [alice["id"]]},
)["json"]
path = "/api/v1/chats/" + chat["id"]
for text in ("", " ", "\n\t"):
denied = call(base, "POST", path + "/messages", 400, token=alice["token"], body={"text": text})
check(denied["json"]["error"]["code"] == "invalid_request", "empty message code")
for method, target, payload in (
("GET", path, None),
("GET", path + "/messages", None),
("POST", path + "/messages", {"text": "чужое"}),
):
hidden = call(base, method, target, 404, token=bob["token"], body=payload)
check(hidden["json"]["error"]["code"] == "not_found", "foreign chat code")
check(chat["title"] not in hidden["json"]["error"]["message"], "foreign chat leaked title")
listed = call(base, "GET", "/api/v1/chats?limit=100", 200, token=bob["token"])["json"]
check(all(item["id"] != chat["id"] for item in listed["items"]), "foreign chat listed")
missing = call(base, "GET", "/api/v1/chats/" + str(uuid.uuid4()), 404, token=bob["token"])
check(missing["json"]["error"]["code"] == "not_found", "missing chat code")
bad_limit = call(base, "GET", path + "/messages?limit=0", 400, token=alice["token"])
check(bad_limit["json"]["error"]["code"] == "invalid_request", "limit=0 code")
print("PASS: empty message and foreign chat")
def test_pagination(base):
alice = create_user(base, "page")
other = create_user(base, "other")
chat = call(
base,
"POST",
"/api/v1/chats",
201,
token=alice["token"],
body={"title": "Страницы", "member_ids": [alice["id"]]},
)["json"]
other_chat = call(
base,
"POST",
"/api/v1/chats",
201,
token=other["token"],
body={"title": "Другой", "member_ids": [other["id"]]},
)["json"]
texts = ["p0", "Привет, мир 👋", "p2", "p3", "p4"]
created = []
for text in texts:
created.append(
call(
base,
"POST",
"/api/v1/chats/%s/messages" % chat["id"],
201,
token=alice["token"],
body={"text": text},
)["json"]
)
call(
base,
"POST",
"/api/v1/chats/%s/messages" % other_chat["id"],
201,
token=other["token"],
body={"text": "x"},
)
call(
base,
"POST",
"/api/v1/chats/%s/messages" % other_chat["id"],
201,
token=other["token"],
body={"text": "y"},
)
collected = []
cursor = None
pages = 0
while True:
target = "/api/v1/chats/%s/messages?limit=2" % chat["id"]
if cursor:
target += "&cursor=" + quote(cursor, safe="")
page = call(base, "GET", target, 200, token=alice["token"])["json"]
pages += 1
collected.extend(page["items"])
cursor = page["next_cursor"]
if cursor is None:
break
check(pages < 10, "pagination did not finish")
check(pages == 3, "expected 3 pages, got %d" % pages)
check(collected == created, "pagination lost, duplicated or reordered messages")
stolen = call(
base,
"GET",
"/api/v1/chats/%s/messages?limit=1" % other_chat["id"],
200,
token=other["token"],
)["json"]
check(stolen["next_cursor"], "foreign page should expose a cursor")
rejected = call(
base,
"GET",
"/api/v1/chats/%s/messages?limit=2&cursor=%s" % (chat["id"], quote(stolen["next_cursor"], safe="")),
400,
token=alice["token"],
)
check(rejected["json"]["error"]["code"] == "invalid_request", "foreign cursor code")
first_page = call(
base,
"GET",
"/api/v1/chats/%s/messages?limit=1" % chat["id"],
200,
token=alice["token"],
)["json"]
for fields in ({"t": None}, {"t": 123}, {"i": 123}):
broken = call(
base,
"GET",
"/api/v1/chats/%s/messages?limit=1&cursor=%s"
% (chat["id"], quote(replace_cursor(first_page["next_cursor"], fields), safe="")),
400,
token=alice["token"],
)
check(broken["json"]["error"]["code"] == "invalid_request", "tampered cursor code")
print("PASS: pagination")
def test_concurrency(base):
users = [create_user(base, "c%d" % index) for index in range(CLIENTS)]
chat = call(
base,
"POST",
"/api/v1/chats",
201,
token=users[0]["token"],
body={"title": "Гонка", "member_ids": [user["id"] for user in users]},
)["json"]
barrier = threading.Barrier(CLIENTS)
found = []
errors = []
lock = threading.Lock()
def worker(user):
try:
barrier.wait(10)
local = []
for index in range(PER_CLIENT):
message = call(
base,
"POST",
"/api/v1/chats/%s/messages" % chat["id"],
201,
token=user["token"],
body={"text": "m-%s-%d" % (user["id"][:8], index)},
)["json"]
local.append(message["id"])
with lock:
found.extend(local)
except Exception as exc:
with lock:
errors.append("%s: %s" % (type(exc).__name__, exc))
threads = [threading.Thread(target=worker, args=(user,)) for user in users]
for thread in threads:
thread.start()
for thread in threads:
thread.join()
check(not errors, "concurrent sends failed: " + "; ".join(errors[:5]))
check(len(found) == CLIENTS * PER_CLIENT, "submitted count")
check(len(set(found)) == CLIENTS * PER_CLIENT, "duplicate message ids")
listed = []
cursor = None
while True:
target = "/api/v1/chats/%s/messages?limit=40" % chat["id"]
if cursor:
target += "&cursor=" + quote(cursor, safe="")
page = call(base, "GET", target, 200, token=users[0]["token"])["json"]
listed.extend(item["id"] for item in page["items"])
cursor = page["next_cursor"]
if cursor is None:
break
check(len(listed) == 100 and len(set(listed)) == 100, "history does not contain 100 unique messages")
check(set(listed) == set(found), "history ids differ from accepted sends")
EVIDENCE.mkdir(exist_ok=True)
report = {
"date": "2026-10-25",
"command": "python student-tests/run.py",
"clients": CLIENTS,
"messages_per_client": PER_CLIENT,
"submitted": len(found),
"unique_ids": len(set(found)),
"listed": len(listed),
"failed_requests": len(errors),
"pass": True,
}
(EVIDENCE / "concurrency.json").write_text(
json.dumps(report, ensure_ascii=False, indent=2) + "\n",
encoding="utf-8",
)
print("PASS: concurrency 100 unique messages from 10 clients")
def test_logs(server):
alice = create_user(server.base, "logs")
chat = call(
server.base,
"POST",
"/api/v1/chats",
201,
token=alice["token"],
body={"title": "Логи", "member_ids": [alice["id"]]},
)["json"]
call(
server.base,
"POST",
"/api/v1/chats/%s/messages" % chat["id"],
201,
token=alice["token"],
body={"text": SECRET},
)
time.sleep(0.2)
output = server.output()
check("duration_ms=" in output and "method=POST" in output, "request log line missing")
check(SECRET not in output, "message text leaked into logs")
check(alice["token"] not in output, "session token leaked into logs")
check("lab-only-password" not in output and "postgresql+asyncpg://" not in output, "database url leaked into logs")
print("PASS: logs omit message text")
def main():
port = free_port()
server = Server(port)
try:
server.wait_ready()
call(server.base, "GET", "/health/live", 200)
call(server.base, "GET", "/api/v1/users/me", 401)
header_only = create_user(server.base, "header")
rejected = call(server.base, "GET", "/api/v1/users/me", 401, user_id=header_only["id"])
check(rejected["json"]["error"]["code"] == "unauthorized", "X-User-Id was accepted")
test_invalid_json(server.base)
test_duplicate_username(server.base)
test_empty_and_foreign(server.base)
test_pagination(server.base)
test_concurrency(server.base)
test_logs(server)
finally:
server.stop()
print("PASS: student tests")
if __name__ == "__main__":
try:
main()
except Failure as exc:
print("FAIL:", exc)
sys.exit(1)