forked from mtusi2026/mtusi2026lab2
Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
6d1d7cbbc4 | ||
|
|
0406baab3f |
+12
-2
@@ -5,7 +5,17 @@ APP_PORT=8080
|
|||||||
APP_ENV=lab
|
APP_ENV=lab
|
||||||
SESSION_TTL_SECONDS=3600
|
SESSION_TTL_SECONDS=3600
|
||||||
# Choose credentials locally. File paths contain no secret values.
|
# Choose credentials locally. File paths contain no secret values.
|
||||||
|
# scripts/bootstrap.sh writes secrets/ and .env; do not commit them.
|
||||||
DATABASE_URL_FILE=/run/secrets/database_url
|
DATABASE_URL_FILE=/run/secrets/database_url
|
||||||
# Optional on 3, required use on 4 in lab2; Redis OR Valkey:
|
# Direct URLs override the files. The API container uses the files above.
|
||||||
SESSION_STORE=postgres
|
# DATABASE_URL=postgresql+asyncpg://USER:PASSWORD@HOST:5432/messenger
|
||||||
|
# REDIS_URL=redis://HOST:6379/0
|
||||||
|
SESSION_STORE=redis
|
||||||
CACHE_URL_FILE=/run/secrets/cache_url
|
CACHE_URL_FILE=/run/secrets/cache_url
|
||||||
|
DB_POOL_SIZE=10
|
||||||
|
DB_MAX_OVERFLOW=10
|
||||||
|
DB_POOL_TIMEOUT_SECONDS=5
|
||||||
|
DB_CONNECT_TIMEOUT_SECONDS=2
|
||||||
|
DB_COMMAND_TIMEOUT_SECONDS=5
|
||||||
|
REDIS_SOCKET_TIMEOUT_SECONDS=1
|
||||||
|
REDIS_MAX_CONNECTIONS=50
|
||||||
|
|||||||
@@ -0,0 +1 @@
|
|||||||
|
*.sh text eol=lf
|
||||||
+28
@@ -0,0 +1,28 @@
|
|||||||
|
FROM python:3.13.16-slim-bookworm@sha256:a1165e272e578941b84abc79e4ab38a0305cd12803a5c4247979ac7655f4d641
|
||||||
|
|
||||||
|
ENV PYTHONDONTWRITEBYTECODE=1 \
|
||||||
|
PYTHONUNBUFFERED=1 \
|
||||||
|
PIP_DISABLE_PIP_VERSION_CHECK=1 \
|
||||||
|
PYTHONPATH=/app/src \
|
||||||
|
APP_HOST=0.0.0.0 \
|
||||||
|
APP_PORT=8080 \
|
||||||
|
APP_ENV=lab
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
RUN useradd --create-home --uid 10001 --shell /usr/sbin/nologin appuser
|
||||||
|
|
||||||
|
COPY requirements.txt /app/requirements.txt
|
||||||
|
RUN pip install --no-cache-dir -r /app/requirements.txt
|
||||||
|
|
||||||
|
COPY src /app/src
|
||||||
|
COPY migrations /app/migrations
|
||||||
|
|
||||||
|
USER appuser
|
||||||
|
|
||||||
|
EXPOSE 8080
|
||||||
|
|
||||||
|
HEALTHCHECK --interval=5s --timeout=3s --start-period=40s --retries=12 \
|
||||||
|
CMD ["python", "-c", "import os,urllib.request; p=os.environ.get('APP_PORT','8080'); urllib.request.urlopen('http://127.0.0.1:%s/health/ready'%p, timeout=2).read()"]
|
||||||
|
|
||||||
|
CMD ["python", "-m", "app"]
|
||||||
@@ -1,37 +1,120 @@
|
|||||||
# Отчёт — лабораторная 2
|
# Отчёт — лабораторная 2
|
||||||
|
|
||||||
- Автор, группа: TODO
|
- Автор, группа: Щербацевич Владислав БПИ2302
|
||||||
- Целевая оценка; выбранный трек 5, если нужен: TODO
|
- Целевая оценка: **4**. Трек 5 не выбран.
|
||||||
- Commit/PR: TODO после публикации
|
- Commit/PR: 0406baa
|
||||||
- Использованные источники, библиотеки и помощь ИИ: TODO
|
- Использованные источники, библиотеки и помощь ИИ: FastAPI, Uvicorn, Pydantic, psycopg3+pool, redis-py, argon2-cffi; контракт курса; помощь ИИ при переносе на PostgreSQL/сессии/Redis
|
||||||
|
|
||||||
## Запуск из чистого clone
|
## Запуск из чистого clone
|
||||||
|
|
||||||
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
|
Стенд: Windows 10, Docker Desktop 29.6.1, виртуальной машине Docker выделено 20 CPU и около 15.4 ГиБ RAM. Лимиты CPU/RAM на сервисы не ставились. Хостовый Python для проверок — 3.13.0. Образ API — `python:3.13.16-slim-bookworm` (digest зафиксирован в `Dockerfile`).
|
||||||
|
|
||||||
|
Образы хранилищ:
|
||||||
|
|
||||||
|
- PostgreSQL `16.10-bookworm`, digest `sha256:38471f330eb885e04de130b768d6db4e10469e2311879c7e5c699f6d2d8a1c74`
|
||||||
|
- Redis `7.4.6-alpine`, digest `sha256:3b73847e72874be07e6657b129a94761662b79bc0f679273757d4218573b2a98`
|
||||||
|
|
||||||
|
```sh
|
||||||
|
python -m venv .venv
|
||||||
|
.venv/bin/python -m pip install -r requirements.txt
|
||||||
|
sh scripts/bootstrap.sh
|
||||||
|
docker compose --env-file .env up -d --build
|
||||||
|
```
|
||||||
|
|
||||||
|
На Windows те же шаги: `powershell -File scripts/bootstrap.ps1`, затем `docker compose --env-file .env up -d --build`. Bootstrap создаёт неотслеживаемые `.env` и `secrets/`. В git их нет. Пароль учебный и только локальный; в отчёт и логи он не копируется.
|
||||||
|
|
||||||
|
Дождаться `GET /health/ready` → `200 {"status":"ok"}`. Затем:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
python tests/smoke.py --base-url http://127.0.0.1:8080
|
||||||
|
python student-tests/run.py
|
||||||
|
python student-tests/level4.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Сохранность:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
chmod +x scripts/restart.sh
|
||||||
|
make test-persistence ACTION_SCRIPT=scripts/restart.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
`scripts/restart.sh` делает `docker compose restart api postgres redis` и не вызывает `down -v`. На PowerShell процесс Windows не запускает `.sh` напрямую (WinError 193). Там тот же файл выполняется через Git Bash: `python student-tests/persistence.py`. На Linux достаточно `make test-persistence`.
|
||||||
|
|
||||||
|
Остановка без удаления данных: `docker compose --env-file .env stop`. Удаление учебных томов: `docker compose --env-file .env down -v`.
|
||||||
|
|
||||||
|
Хостовый порт PostgreSQL — **5433**. Внутри сети Compose база остаётся `postgres:5432`. На этой машине привязка `127.0.0.1:5432` запрещена (WinError 10013), Redis на `127.0.0.1:6379` и API на `127.0.0.1:8080` открываются. Порты не публикуются наружу интерфейса, только на loopback.
|
||||||
|
|
||||||
## Архитектура и решения
|
## Архитектура и решения
|
||||||
|
|
||||||
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab1, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
|
Клиент ходит в один процесс API. Пользователи, чаты, участники и сообщения лежат в PostgreSQL: это источник истины. Сессии лежат только в Redis. Кэша истории нет, поэтому после COMMIT следующее чтение видит ту же строку без инвалидации.
|
||||||
|
|
||||||
|
Из lab1 перенесены контрактные правила чата, keyset-пагинация и формат ошибок. Память процесса больше не хранит эти данные. Аутентификация заменена по переходу 1 → 2: регистрация принимает `password`, `POST /api/v1/auth/sessions` принимает только Basic, остальные защищённые методы — только Bearer. Заголовок `X-User-Id` не читается.
|
||||||
|
|
||||||
|
Схема — `migrations/001_initial.sql`. При старте API берёт `pg_advisory_xact_lock` и применяет файл, только если его имени ещё нет в `schema_migrations`. Повторный старт пишет в лог `migrations newly_applied=0` и не вставляет строки заново. После `scripts/restart.sh` в логе API было именно `newly_applied=0`, версия осталась `001_initial.sql`.
|
||||||
|
|
||||||
|
Транзакции:
|
||||||
|
|
||||||
|
- регистрация: Argon2 считается до захвата соединения; в транзакции одна вставка `users`; уникальность username держит ограничение `uq_users_username`;
|
||||||
|
- создание чата: в одной транзакции проверка участников, вставка `chats` и все строки `chat_members`;
|
||||||
|
- сообщение: в одной транзакции проверка членства и вставка `messages`.
|
||||||
|
|
||||||
|
Пароль — Argon2id, параметры `m=32768` (32 МиБ), `t=2`, `p=1`, длина хеша 32 байта. Это не SHA-256 и не усечение bcrypt. Для отсутствующего пользователя выполняется проверка фиктивного хеша, ответ тот же, что и при неверном пароле: `401` и `WWW-Authenticate: Basic`.
|
||||||
|
|
||||||
|
Сессия — 32 случайных байта (`token_urlsafe(32)`, 256 бит). В Redis ключ `sess:` + SHA-256 токена, значение — `user_id` и абсолютный `expires_at`. Команда `SET` с `PXAT` записывает абсолютное время в миллисекундах, поэтому повтор AOF после рестарта не сдвигает срок. Чтение делает `GET` и не продлевает TTL. Logout делает `GETDEL`. Пока Redis не отвечает, запрос с Bearer получает `503`, запрос без заголовка — `401`: анонимный клиент не становится доверенным.
|
||||||
|
|
||||||
|
Redis запущен с `--appendonly yes --appendfsync always` и томом `redis_data`. `CONFIG GET` показал `appendonly=yes`, `appendfsync=always`. Ответ `201` на вход возвращается после того, как Redis подтвердил запись.
|
||||||
|
|
||||||
|
Пул SQLAlchemy: `pool_size=10`, `max_overflow=10`, ожидание соединения 5 с, `pool_pre_ping=true`. asyncpg: таймаут подключения 2 с, `command_timeout` 5 с. На стороне PostgreSQL `statement_timeout` 5 с, `lock_timeout` 5 с, `idle_in_transaction_session_timeout` 10 с. Redis: таймаут сокета 1 с, не больше 50 соединений. Запросы параметризованы через SQLAlchemy; текст сообщения в SQL не склеивается.
|
||||||
|
|
||||||
|
Переменные из `.env.example` совпадают с `compose.yaml`: `SESSION_TTL_SECONDS` (в стенде 3600, для проверки истечения отдельно 2), `DATABASE_URL_FILE` и `CACHE_URL_FILE` внутри API, прямые `DATABASE_URL` и `REDIS_URL` для процессов на хосте. `SESSION_STORE=redis`.
|
||||||
|
|
||||||
|
`/health/live` смотрит только процесс. `/health/ready` требует и PostgreSQL, и Redis: без сессий и без источника истины обязательный путь не обслуживается.
|
||||||
|
|
||||||
## Доказательства критериев
|
## Доказательства критериев
|
||||||
|
|
||||||
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
|
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
|
||||||
| --- | --- | --- | --- |
|
| --- | --- | --- | --- |
|
||||||
| Публичный smoke | make test | TODO | TODO |
|
| Публичный smoke | `python tests/smoke.py --base-url http://127.0.0.1:8080` | `PASS: lab2 public API scenario` | вывод команды, повторён после рестарта |
|
||||||
| Остальные пункты уровня 3 | TODO | TODO | TODO |
|
| Данные и сессия переживают рестарт | `python student-tests/persistence.py` (`scripts/restart.sh` перезапускает api, postgres, redis, тома на месте) | тот же Bearer читает то же сообщение; лог API: `migrations newly_applied=0` | вывод команды |
|
||||||
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
|
| PostgreSQL, ключи, миграция с нуля | старт пустого тома, затем повторный старт | таблицы users/chats/chat_members/messages, PK/FK, `uq_users_username`; вторая миграция не применяется | `migrations/001_initial.sql`, лог `newly_applied=0` |
|
||||||
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
|
| Атомарное создание чата | неизвестный участник; отдельно обрыв после INSERT чата | число чатов не меняется, строки с заголовком проверки нет (`11` и `11`, `title_rows=0`) | `evidence/level4.json` |
|
||||||
|
| Basic только на создании сессии, Bearer на остальном, `X-User-Id` не даёт доступ | `student-tests/level4.py` | неверный и неизвестный пароль дают один и тот же текст `401`; Basic на `/users/me` — `401`; один `X-User-Id` — `401`; чужой `X-User-Id` вместе с Bearer не меняет пользователя | `evidence/level4.json` |
|
||||||
|
| Пароль — Argon2id, в том числе Unicode и двоеточие | пароль `Пароль:пример-12` | повторный Basic-вход успешен | вывод `PASS: unicode password with colon` |
|
||||||
|
| Срок сессии и logout | `DELETE /api/v1/auth/sessions/current`, затем отдельный процесс с `SESSION_TTL_SECONDS=2` | logout → `204`, повтор и чтение → `401`; после `expires_at` чтение → `401` | `evidence/level4.json` |
|
||||||
|
| Redis — хранилище сессий с устойчивым TTL | `PTTL` до и после `docker compose restart redis` | `3590337` мс → `3588378` мс, ключ жив, тот же токен принимается | `evidence/level4.json` |
|
||||||
|
| Гонка одинакового username | 16 параллельных `POST /users` | ровно один `201` и пятнадцать `409` | `evidence/level4.json` |
|
||||||
|
| Параметризованные запросы | текст `'; DROP TABLE messages; --` | таблица `messages` на месте, текст сохранён как данные | вывод level4 |
|
||||||
|
| Индекс истории и keyset | 10 000 сообщений, `EXPLAIN (ANALYZE, BUFFERS)` первой страницы и страницы после 5000-й строки | `Index Scan using ix_messages_history`, не `Seq Scan`; `shared hit=5` и `shared hit=4`; `actual rows=50`; 0.050 мс и 0.051 мс | `evidence/explain_messages.txt` |
|
||||||
|
| Отказ Redis и PostgreSQL | `docker compose stop redis`, затем postgres; потом `start` | live `200`; ready `503`; без заголовка `401`; с Bearer `503`; после возврата данные и сессия на месте, 10 000 строк целы | `evidence/level4.json` |
|
||||||
|
| Свои проверки lab1 на новом входе | `python student-tests/run.py` | пагинация, 100 сообщений от 10 клиентов, текст сообщения не попадает в лог | `evidence/concurrency.json` |
|
||||||
|
|
||||||
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
|
Проверка инфраструктурного hook: `docker compose ps` до и после `restart` показывает те же сервисы `api`, `postgres`, `redis`; тома `postgres_data` и `redis_data` не удалялись. Скрипт завершился с кодом 0.
|
||||||
|
|
||||||
## Эксперименты и отказы
|
## Эксперименты и отказы
|
||||||
|
|
||||||
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
|
Нагрузка уровня 5 не заявлялась. Ниже только обязательные отказы уровня 4. Прогрев — до `/health/ready`. Конкуренция гонки — 16 запросов регистрации. История для плана — 10 000 строк в одном чате. Повторы: smoke дважды (до и после рестарта), level4 и persistence по одному успешному прогону. RPS и перцентили нагрузки не измерялись: это не трек 5. CPU/RAM стенда — 20 CPU и 15.4 ГиБ у Docker Desktop, без лимитов сервисов.
|
||||||
|
|
||||||
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
|
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
|
||||||
| --- | --- | --- | --- |
|
| --- | --- | --- | --- |
|
||||||
| TODO | TODO | TODO | TODO |
|
| `restart` API, PostgreSQL и Redis | тот же токен и то же сообщение, TTL не сбрасывается в полный час | сообщение найдено тем же Bearer; `PTTL` уменьшился с 3590337 до 3588378 мс | готовность около 2 с после старта контейнеров; подтверждённые строки не пропали |
|
||||||
|
| `SESSION_TTL_SECONDS=2` | после `expires_at` сессия недействительна | `401` | новая сессия нужна сразу после срока; данные пользователя остаются |
|
||||||
|
| `stop redis` | live жив, ready нет, аноним не доверен, Bearer не обслуживается | 200 / 503 / 401 / 503 | после `start redis` тот же токен снова `200`; сессия не потеряна за счёт AOF |
|
||||||
|
| `stop postgres` | live жив, чтение истории невозможно, аноним не доверен | 200 / 503 / 401 / 503 | после `start postgres` история читается; 10 000 сообщений на месте |
|
||||||
|
|
||||||
|
## Ответы к вопросам защиты
|
||||||
|
|
||||||
|
COMMIT в PostgreSQL фиксирует транзакцию чата или сообщения. При `synchronous_commit=on` (значение по умолчанию у этого образа) сервер подтверждает COMMIT после записи WAL. Это не защита от потери всего диска хоста и не замена резервной копии. Том Docker сохраняет каталог данных между рестартом контейнера, но не между `down -v`.
|
||||||
|
|
||||||
|
Сессия после рестарта Redis читается из AOF. `appendfsync always` просит fsync на каждую запись. `PXAT` хранит абсолютный срок, поэтому рестарт его не продлевает. Если удалить том Redis или выключить AOF, сессии пропадут, а пользователи и сообщения в PostgreSQL останутся. Один только TTL без persistence ключ после рестарта не возвращает.
|
||||||
|
|
||||||
|
Аутентификация отвечает на вопрос «кто вызывает». Доступ к чату отвечает на вопрос «состоит ли этот пользователь в чате». Чужой чат для уже вошедшего пользователя — `404`, не `403`, чтобы не подтверждать существование. Неверный токен — `401` и возникает раньше проверки чата.
|
||||||
|
|
||||||
|
Хеш пароля должен быть медленным и со случайной солью: его задача — сделать перебор дорогим, если база утечёт. Контрольная сумма вроде SHA-256 быстро узнаёт, изменились ли байты. Для пароля она не подходит: одинаковые пароли дали бы одинаковый хеш, а проверка была бы слишком дешёвой.
|
||||||
|
|
||||||
## Ограничения и следующие шаги
|
## Ограничения и следующие шаги
|
||||||
|
|
||||||
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
|
Стенд выдерживает рестарт контейнеров на том же Docker-хосте и короткую остановку Redis или PostgreSQL. Он не выдерживает потерю самого хоста, порчу тома и одновременный отказ диска. Один процесс API — не горизонтальное масштабирование; это задача lab3.
|
||||||
|
|
||||||
|
Роль `messenger` в учебном образе PostgreSQL — суперпользователь. Отдельная роль только для DML здесь не выделялась. У Redis нет AUTH: порт открыт только на `127.0.0.1`, сеть Compose внутренняя. Argon2id запущен с учебными параметрами, не с промышленным профилем lab7. История не кэшируется, поэтому отдельной инвалидации кэша сообщений нет.
|
||||||
|
|
||||||
|
Не делался ни один трек оценки 5: ни измерение cache stampede, ни backup на 100 000 сообщений.
|
||||||
|
|||||||
+109
@@ -0,0 +1,109 @@
|
|||||||
|
# Локальный учебный стенд лабораторной 2. Секреты создаёт scripts/bootstrap, не этот файл.
|
||||||
|
name: mtusi-lab2
|
||||||
|
|
||||||
|
services:
|
||||||
|
postgres:
|
||||||
|
# Хостовый порт 5433: на части машин Windows порт 5432 нельзя занять.
|
||||||
|
# Внутри сети стенда база по-прежнему доступна как postgres:5432.
|
||||||
|
image: postgres:16.10-bookworm@sha256:38471f330eb885e04de130b768d6db4e10469e2311879c7e5c699f6d2d8a1c74
|
||||||
|
environment:
|
||||||
|
POSTGRES_USER: messenger
|
||||||
|
POSTGRES_DB: messenger
|
||||||
|
POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password
|
||||||
|
secrets:
|
||||||
|
- postgres_password
|
||||||
|
volumes:
|
||||||
|
- postgres_data:/var/lib/postgresql/data
|
||||||
|
ports:
|
||||||
|
- "127.0.0.1:5433:5432"
|
||||||
|
networks:
|
||||||
|
- lab
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "pg_isready -U messenger -d messenger"]
|
||||||
|
interval: 2s
|
||||||
|
timeout: 3s
|
||||||
|
retries: 30
|
||||||
|
start_period: 5s
|
||||||
|
restart: unless-stopped
|
||||||
|
|
||||||
|
redis:
|
||||||
|
image: redis:7.4.6-alpine@sha256:3b73847e72874be07e6657b129a94761662b79bc0f679273757d4218573b2a98
|
||||||
|
command:
|
||||||
|
- redis-server
|
||||||
|
- --appendonly
|
||||||
|
- yes
|
||||||
|
- --appendfsync
|
||||||
|
- always
|
||||||
|
- --dir
|
||||||
|
- /data
|
||||||
|
volumes:
|
||||||
|
- redis_data:/data
|
||||||
|
ports:
|
||||||
|
- "127.0.0.1:6379:6379"
|
||||||
|
networks:
|
||||||
|
- lab
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "redis-cli", "ping"]
|
||||||
|
interval: 2s
|
||||||
|
timeout: 3s
|
||||||
|
retries: 30
|
||||||
|
start_period: 3s
|
||||||
|
restart: unless-stopped
|
||||||
|
|
||||||
|
api:
|
||||||
|
build: .
|
||||||
|
environment:
|
||||||
|
APP_HOST: 0.0.0.0
|
||||||
|
APP_PORT: "8080"
|
||||||
|
APP_ENV: lab
|
||||||
|
SESSION_TTL_SECONDS: ${SESSION_TTL_SECONDS:-3600}
|
||||||
|
SESSION_STORE: redis
|
||||||
|
DATABASE_URL_FILE: /run/secrets/database_url
|
||||||
|
CACHE_URL_FILE: /run/secrets/cache_url
|
||||||
|
DB_POOL_SIZE: "10"
|
||||||
|
DB_MAX_OVERFLOW: "10"
|
||||||
|
DB_POOL_TIMEOUT_SECONDS: "5"
|
||||||
|
DB_CONNECT_TIMEOUT_SECONDS: "2"
|
||||||
|
DB_COMMAND_TIMEOUT_SECONDS: "5"
|
||||||
|
REDIS_SOCKET_TIMEOUT_SECONDS: "1"
|
||||||
|
REDIS_MAX_CONNECTIONS: "50"
|
||||||
|
secrets:
|
||||||
|
- database_url
|
||||||
|
- cache_url
|
||||||
|
ports:
|
||||||
|
- "127.0.0.1:8080:8080"
|
||||||
|
networks:
|
||||||
|
- lab
|
||||||
|
depends_on:
|
||||||
|
postgres:
|
||||||
|
condition: service_healthy
|
||||||
|
redis:
|
||||||
|
condition: service_healthy
|
||||||
|
healthcheck:
|
||||||
|
test:
|
||||||
|
[
|
||||||
|
"CMD",
|
||||||
|
"python",
|
||||||
|
"-c",
|
||||||
|
"import os,urllib.request; p=os.environ.get('APP_PORT','8080'); urllib.request.urlopen('http://127.0.0.1:%s/health/ready'%p, timeout=2).read()",
|
||||||
|
]
|
||||||
|
interval: 5s
|
||||||
|
timeout: 3s
|
||||||
|
retries: 12
|
||||||
|
start_period: 40s
|
||||||
|
restart: unless-stopped
|
||||||
|
|
||||||
|
secrets:
|
||||||
|
postgres_password:
|
||||||
|
file: ./secrets/postgres_password
|
||||||
|
database_url:
|
||||||
|
file: ./secrets/database_url
|
||||||
|
cache_url:
|
||||||
|
file: ./secrets/cache_url
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
postgres_data:
|
||||||
|
redis_data:
|
||||||
|
|
||||||
|
networks:
|
||||||
|
lab:
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
{
|
||||||
|
"date": "2026-10-25",
|
||||||
|
"command": "python student-tests/run.py",
|
||||||
|
"clients": 10,
|
||||||
|
"messages_per_client": 10,
|
||||||
|
"submitted": 100,
|
||||||
|
"unique_ids": 100,
|
||||||
|
"listed": 100,
|
||||||
|
"failed_requests": 0,
|
||||||
|
"pass": true
|
||||||
|
}
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
messages=10000
|
||||||
|
|
||||||
|
-- first page
|
||||||
|
Limit (cost=0.29..7.72 rows=50 width=32) (actual time=0.021..0.032 rows=50 loops=1)
|
||||||
|
Buffers: shared hit=5
|
||||||
|
-> Index Scan using ix_messages_history on messages (cost=0.29..1486.70 rows=10000 width=32) (actual time=0.021..0.029 rows=50 loops=1)
|
||||||
|
Index Cond: (chat_id = 'f3c38461-f7ec-49fd-a912-46e9cff8f2b4'::uuid)
|
||||||
|
Buffers: shared hit=5
|
||||||
|
Planning:
|
||||||
|
Buffers: shared hit=104
|
||||||
|
Planning Time: 0.210 ms
|
||||||
|
Execution Time: 0.050 ms
|
||||||
|
|
||||||
|
-- page after row 5000
|
||||||
|
Limit (cost=0.29..12.22 rows=50 width=32) (actual time=0.016..0.027 rows=50 loops=1)
|
||||||
|
Buffers: shared hit=4
|
||||||
|
-> Index Scan using ix_messages_history on messages (cost=0.29..1200.23 rows=5027 width=32) (actual time=0.015..0.023 rows=50 loops=1)
|
||||||
|
Index Cond: ((chat_id = 'f3c38461-f7ec-49fd-a912-46e9cff8f2b4'::uuid) AND (ROW(created_at, id) > ROW('2026-01-01 00:00:00.005001+00'::timestamp with time zone, 'd063df83-eabe-455a-b0dc-d8e1dfb1364d'::uuid)))
|
||||||
|
Buffers: shared hit=4
|
||||||
|
Planning:
|
||||||
|
Buffers: shared hit=105
|
||||||
|
Planning Time: 0.276 ms
|
||||||
|
Execution Time: 0.051 ms
|
||||||
@@ -0,0 +1,40 @@
|
|||||||
|
{
|
||||||
|
"date": "2026-10-25T17:11:54Z",
|
||||||
|
"base_url": "http://127.0.0.1:8080",
|
||||||
|
"race": {
|
||||||
|
"created": 1,
|
||||||
|
"conflict": 15
|
||||||
|
},
|
||||||
|
"rollback": {
|
||||||
|
"chats_before": 11,
|
||||||
|
"chats_after": 11,
|
||||||
|
"title_rows": 0
|
||||||
|
},
|
||||||
|
"explain": {
|
||||||
|
"messages": 10000
|
||||||
|
},
|
||||||
|
"expiry_and_migrations": {
|
||||||
|
"users_before": 38,
|
||||||
|
"users_after": 39,
|
||||||
|
"migration": "001_initial.sql"
|
||||||
|
},
|
||||||
|
"redis_restart": {
|
||||||
|
"pttl_before_ms": 3590337,
|
||||||
|
"pttl_after_ms": 3588378
|
||||||
|
},
|
||||||
|
"dependency_loss": {
|
||||||
|
"redis_down": {
|
||||||
|
"live": 200,
|
||||||
|
"ready": 503,
|
||||||
|
"anonymous": 401,
|
||||||
|
"bearer": 503
|
||||||
|
},
|
||||||
|
"postgres_down": {
|
||||||
|
"live": 200,
|
||||||
|
"ready": 503,
|
||||||
|
"anonymous": 401,
|
||||||
|
"bearer": 503
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"messages_after_postgres_restart": 10000
|
||||||
|
}
|
||||||
@@ -0,0 +1,52 @@
|
|||||||
|
-- Пользователи, чаты, состав и сообщения. Пароли и тексты сообщений
|
||||||
|
-- не пишутся приложением в логи. password_hash — только Argon2id.
|
||||||
|
-- Сессии в этой схеме нет: их источник истины — Redis (см. compose.yaml).
|
||||||
|
|
||||||
|
CREATE TABLE users (
|
||||||
|
id UUID PRIMARY KEY,
|
||||||
|
username TEXT NOT NULL,
|
||||||
|
display_name TEXT NOT NULL,
|
||||||
|
password_hash TEXT NOT NULL,
|
||||||
|
created_at TIMESTAMPTZ NOT NULL,
|
||||||
|
CONSTRAINT uq_users_username UNIQUE (username),
|
||||||
|
CONSTRAINT ck_users_username_format CHECK (username ~ '^[a-z0-9_]{3,32}$'),
|
||||||
|
CONSTRAINT ck_users_display_name_len CHECK (
|
||||||
|
char_length(btrim(display_name)) >= 1 AND char_length(display_name) <= 100
|
||||||
|
),
|
||||||
|
CONSTRAINT ck_users_password_hash_len CHECK (char_length(password_hash) >= 20)
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE TABLE chats (
|
||||||
|
id UUID PRIMARY KEY,
|
||||||
|
title TEXT NOT NULL,
|
||||||
|
created_at TIMESTAMPTZ NOT NULL,
|
||||||
|
CONSTRAINT ck_chats_title CHECK (char_length(btrim(title)) >= 1 AND char_length(title) <= 100)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Состав чата пишется в той же транзакции, что и строка chats.
|
||||||
|
-- Откат до COMMIT не оставляет чат без участников.
|
||||||
|
CREATE TABLE chat_members (
|
||||||
|
chat_id UUID NOT NULL REFERENCES chats (id),
|
||||||
|
user_id UUID NOT NULL REFERENCES users (id),
|
||||||
|
ordinal SMALLINT NOT NULL,
|
||||||
|
PRIMARY KEY (chat_id, user_id),
|
||||||
|
CONSTRAINT ck_chat_members_ordinal CHECK (ordinal >= 0 AND ordinal < 100)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Список чатов пользователя: фильтр по user_id, затем порядок (created_at, id) чата.
|
||||||
|
CREATE INDEX ix_chat_members_user ON chat_members (user_id, chat_id);
|
||||||
|
|
||||||
|
CREATE TABLE messages (
|
||||||
|
id UUID PRIMARY KEY,
|
||||||
|
chat_id UUID NOT NULL REFERENCES chats (id),
|
||||||
|
sender_id UUID NOT NULL REFERENCES users (id),
|
||||||
|
body TEXT NOT NULL,
|
||||||
|
created_at TIMESTAMPTZ NOT NULL,
|
||||||
|
CONSTRAINT ck_messages_text CHECK (char_length(btrim(body)) >= 1 AND char_length(body) <= 4000)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- История: WHERE chat_id = $1 AND (created_at, id) > ($2, $3)
|
||||||
|
-- ORDER BY created_at, id LIMIT $4.
|
||||||
|
-- Ведущий chat_id отсекает чужие чаты, пара (created_at, id) продолжает
|
||||||
|
-- keyset с cursor и не читает предыдущие строки этой истории.
|
||||||
|
CREATE INDEX ix_messages_history ON messages (chat_id, created_at, id);
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
annotated-doc==0.0.5
|
||||||
|
annotated-types==0.8.0
|
||||||
|
anyio==4.15.1
|
||||||
|
click==8.5.0
|
||||||
|
fastapi==0.143.0
|
||||||
|
h11==0.16.0
|
||||||
|
idna==3.20
|
||||||
|
opentelemetry-api==1.45.1
|
||||||
|
pydantic==2.14.0
|
||||||
|
pydantic-core==2.50.0
|
||||||
|
starlette==1.7.0
|
||||||
|
typing-inspection==0.4.4
|
||||||
|
typing-extensions==4.16.0
|
||||||
|
uvicorn==0.54.0
|
||||||
|
sqlalchemy==2.0.44
|
||||||
|
asyncpg==0.31.0
|
||||||
|
redis==6.4.0
|
||||||
|
argon2-cffi==25.1.0
|
||||||
@@ -0,0 +1,14 @@
|
|||||||
|
# Создаёт локальные учебные секреты. Не печатает их и не кладёт в git.
|
||||||
|
$ErrorActionPreference = "Stop"
|
||||||
|
$root = Split-Path -Parent $PSScriptRoot
|
||||||
|
Set-Location $root
|
||||||
|
New-Item -ItemType Directory -Force -Path (Join-Path $root "secrets") | Out-Null
|
||||||
|
$pass = "lab-only-password"
|
||||||
|
$utf8 = New-Object System.Text.UTF8Encoding $false
|
||||||
|
[System.IO.File]::WriteAllText((Join-Path $root "secrets\postgres_password"), $pass, $utf8)
|
||||||
|
[System.IO.File]::WriteAllText((Join-Path $root "secrets\database_url"), "postgresql+asyncpg://messenger:${pass}@postgres:5432/messenger", $utf8)
|
||||||
|
[System.IO.File]::WriteAllText((Join-Path $root "secrets\cache_url"), "redis://redis:6379/0", $utf8)
|
||||||
|
[System.IO.File]::WriteAllText((Join-Path $root "secrets\host_database_url"), "postgresql+asyncpg://messenger:${pass}@127.0.0.1:5433/messenger", $utf8)
|
||||||
|
[System.IO.File]::WriteAllText((Join-Path $root "secrets\host_cache_url"), "redis://127.0.0.1:6379/0", $utf8)
|
||||||
|
[System.IO.File]::WriteAllText((Join-Path $root ".env"), "SESSION_TTL_SECONDS=3600`nAPP_ENV=lab`n", $utf8)
|
||||||
|
Write-Host "bootstrap: wrote .env and secrets/ for the local lab stand"
|
||||||
Executable
+13
@@ -0,0 +1,13 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
# Создаёт локальные учебные секреты. Не печатает их и не кладёт в git.
|
||||||
|
set -eu
|
||||||
|
cd "$(dirname "$0")/.."
|
||||||
|
mkdir -p secrets
|
||||||
|
pass='lab-only-password'
|
||||||
|
printf '%s' "$pass" > secrets/postgres_password
|
||||||
|
printf '%s' "postgresql+asyncpg://messenger:${pass}@postgres:5432/messenger" > secrets/database_url
|
||||||
|
printf '%s' 'redis://redis:6379/0' > secrets/cache_url
|
||||||
|
printf '%s' "postgresql+asyncpg://messenger:${pass}@127.0.0.1:5433/messenger" > secrets/host_database_url
|
||||||
|
printf '%s' 'redis://127.0.0.1:6379/0' > secrets/host_cache_url
|
||||||
|
printf '%s\n' 'SESSION_TTL_SECONDS=3600' 'APP_ENV=lab' > .env
|
||||||
|
echo 'bootstrap: wrote .env and secrets/ for the local lab stand'
|
||||||
Executable
+9
@@ -0,0 +1,9 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
# Перезапуск API и всех хранилищ лабораторной 2. Тома не удаляются.
|
||||||
|
set -eu
|
||||||
|
cd "$(dirname "$0")/.."
|
||||||
|
if [ -f .env ]; then
|
||||||
|
docker compose --env-file .env restart api postgres redis
|
||||||
|
else
|
||||||
|
docker compose restart api postgres redis
|
||||||
|
fi
|
||||||
@@ -0,0 +1,38 @@
|
|||||||
|
import logging
|
||||||
|
import sys
|
||||||
|
|
||||||
|
import uvicorn
|
||||||
|
|
||||||
|
from app.config import load_settings
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
settings = load_settings()
|
||||||
|
logging.basicConfig(
|
||||||
|
level=logging.INFO,
|
||||||
|
format="%(asctime)s %(levelname)s %(name)s %(message)s",
|
||||||
|
stream=sys.stdout,
|
||||||
|
force=True,
|
||||||
|
)
|
||||||
|
logging.getLogger("sqlalchemy.engine").setLevel(logging.WARNING)
|
||||||
|
logging.getLogger("sqlalchemy.pool").setLevel(logging.WARNING)
|
||||||
|
logging.getLogger("asyncpg").setLevel(logging.WARNING)
|
||||||
|
logging.getLogger("app").info(
|
||||||
|
"starting host=%s port=%s env=%s",
|
||||||
|
settings.host,
|
||||||
|
settings.port,
|
||||||
|
settings.env,
|
||||||
|
)
|
||||||
|
uvicorn.run(
|
||||||
|
"app.main:app",
|
||||||
|
host=settings.host,
|
||||||
|
port=settings.port,
|
||||||
|
workers=1,
|
||||||
|
access_log=False,
|
||||||
|
timeout_graceful_shutdown=5,
|
||||||
|
log_config=None,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
from fastapi import APIRouter
|
||||||
|
|
||||||
|
from app.api.auth import router as auth_router
|
||||||
|
from app.api.chats import router as chats_router
|
||||||
|
from app.api.messages import router as messages_router
|
||||||
|
from app.api.users import router as users_router
|
||||||
|
|
||||||
|
api_router = APIRouter(prefix="/api/v1")
|
||||||
|
api_router.include_router(users_router)
|
||||||
|
api_router.include_router(auth_router)
|
||||||
|
api_router.include_router(chats_router)
|
||||||
|
api_router.include_router(messages_router)
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
from fastapi import APIRouter, Depends, Header
|
||||||
|
from starlette.responses import Response
|
||||||
|
|
||||||
|
from app.crud.users import authenticate
|
||||||
|
from app.deps import BASIC_CHALLENGE, BEARER_CHALLENGE, require_token, unauthorized
|
||||||
|
from app.schemas.session import SessionOut
|
||||||
|
from app.security import parse_basic, verify_password
|
||||||
|
from app.sessions import create_session, revoke_session
|
||||||
|
from app.timeutil import format_time
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/auth/sessions", status_code=201, response_model=SessionOut)
|
||||||
|
async def create_session_route(authorization: str | None = Header(default=None)) -> dict:
|
||||||
|
parsed = parse_basic(authorization or "")
|
||||||
|
if parsed is None:
|
||||||
|
verify_password("invalid-basic-auth", None)
|
||||||
|
raise unauthorized(BASIC_CHALLENGE)
|
||||||
|
username, password = parsed
|
||||||
|
user = await authenticate(username, password)
|
||||||
|
if user is None:
|
||||||
|
raise unauthorized(BASIC_CHALLENGE)
|
||||||
|
token, expires_at = await create_session(user.id)
|
||||||
|
return {
|
||||||
|
"token": token,
|
||||||
|
"token_type": "Bearer",
|
||||||
|
"expires_at": format_time(expires_at),
|
||||||
|
"user": user.to_dict(),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@router.delete("/auth/sessions/current", status_code=204)
|
||||||
|
async def revoke_current_session(token: str = Depends(require_token)) -> Response:
|
||||||
|
if not await revoke_session(token):
|
||||||
|
raise unauthorized(BEARER_CHALLENGE)
|
||||||
|
return Response(status_code=204)
|
||||||
@@ -0,0 +1,31 @@
|
|||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Depends, Query
|
||||||
|
|
||||||
|
from app.crud.chats import chat_for_member, chats_for_user, create_chat as save_chat
|
||||||
|
from app.deps import get_current_user
|
||||||
|
from app.present import PublicUser
|
||||||
|
from app.schemas.chat import ChatOut, ChatPage, CreateChat
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/chats", status_code=201, response_model=ChatOut)
|
||||||
|
async def create_chat(body: CreateChat, user: PublicUser = Depends(get_current_user)) -> dict:
|
||||||
|
chat = await save_chat(user.id, body.title, tuple(body.member_ids))
|
||||||
|
return chat.to_dict()
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/chats", response_model=ChatPage)
|
||||||
|
async def list_chats(
|
||||||
|
user: PublicUser = Depends(get_current_user),
|
||||||
|
limit: int = Query(default=50, ge=1, le=100),
|
||||||
|
cursor: str | None = Query(default=None, min_length=1),
|
||||||
|
) -> dict:
|
||||||
|
return await chats_for_user(user.id, limit, cursor)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/chats/{chat_id}", response_model=ChatOut)
|
||||||
|
async def get_chat(chat_id: UUID, user: PublicUser = Depends(get_current_user)) -> dict:
|
||||||
|
chat = await chat_for_member(chat_id, user.id)
|
||||||
|
return chat.to_dict()
|
||||||
@@ -0,0 +1,39 @@
|
|||||||
|
import logging
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Request
|
||||||
|
from pydantic import BaseModel
|
||||||
|
from sqlalchemy import text
|
||||||
|
|
||||||
|
from app.db import is_dependency_error
|
||||||
|
from app.errors import AppError
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
logger = logging.getLogger("app.health")
|
||||||
|
|
||||||
|
|
||||||
|
class HealthOut(BaseModel):
|
||||||
|
status: str
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/health/live", response_model=HealthOut)
|
||||||
|
async def live() -> dict:
|
||||||
|
return {"status": "ok"}
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/health/ready", response_model=HealthOut)
|
||||||
|
async def ready(request: Request) -> dict:
|
||||||
|
if not getattr(request.app.state, "accepting", False):
|
||||||
|
raise AppError(503, "unavailable", "Сервер не готов")
|
||||||
|
engine = getattr(request.app.state, "engine", None)
|
||||||
|
redis = getattr(request.app.state, "redis", None)
|
||||||
|
if engine is None or redis is None:
|
||||||
|
raise AppError(503, "unavailable", "Сервис временно недоступен")
|
||||||
|
try:
|
||||||
|
async with engine.connect() as conn:
|
||||||
|
await conn.execute(text("SELECT 1"))
|
||||||
|
await redis.ping()
|
||||||
|
except Exception as exc:
|
||||||
|
if not is_dependency_error(exc):
|
||||||
|
logger.warning("readiness check failed type=%s", type(exc).__name__)
|
||||||
|
raise AppError(503, "unavailable", "Сервис временно недоступен") from None
|
||||||
|
return {"status": "ok"}
|
||||||
@@ -0,0 +1,31 @@
|
|||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Depends, Query
|
||||||
|
|
||||||
|
from app.crud.messages import create_message as save_message
|
||||||
|
from app.crud.messages import messages_for_member
|
||||||
|
from app.deps import get_current_user
|
||||||
|
from app.present import PublicUser
|
||||||
|
from app.schemas.message import CreateMessage, MessageOut, MessagePage
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/chats/{chat_id}/messages", status_code=201, response_model=MessageOut)
|
||||||
|
async def create_message(
|
||||||
|
chat_id: UUID,
|
||||||
|
body: CreateMessage,
|
||||||
|
user: PublicUser = Depends(get_current_user),
|
||||||
|
) -> dict:
|
||||||
|
message = await save_message(chat_id, user.id, body.text)
|
||||||
|
return message.to_dict()
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/chats/{chat_id}/messages", response_model=MessagePage)
|
||||||
|
async def list_messages(
|
||||||
|
chat_id: UUID,
|
||||||
|
user: PublicUser = Depends(get_current_user),
|
||||||
|
limit: int = Query(default=50, ge=1, le=100),
|
||||||
|
cursor: str | None = Query(default=None, min_length=1),
|
||||||
|
) -> dict:
|
||||||
|
return await messages_for_member(chat_id, user.id, limit, cursor)
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
from fastapi import APIRouter, Depends
|
||||||
|
|
||||||
|
from app.crud.users import create_user
|
||||||
|
from app.deps import get_current_user
|
||||||
|
from app.present import PublicUser
|
||||||
|
from app.schemas.user import CreateUser, UserOut
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/users", status_code=201, response_model=UserOut)
|
||||||
|
async def register(body: CreateUser) -> dict:
|
||||||
|
user = await create_user(body.username, body.display_name, body.password)
|
||||||
|
return user.to_dict()
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/users/me", response_model=UserOut)
|
||||||
|
async def me(user: PublicUser = Depends(get_current_user)) -> dict:
|
||||||
|
return user.to_dict()
|
||||||
@@ -0,0 +1,97 @@
|
|||||||
|
import os
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class Settings:
|
||||||
|
host: str
|
||||||
|
port: int
|
||||||
|
env: str
|
||||||
|
session_ttl_seconds: int
|
||||||
|
database_url: str
|
||||||
|
cache_url: str
|
||||||
|
migrations_dir: Path
|
||||||
|
db_pool_size: int
|
||||||
|
db_max_overflow: int
|
||||||
|
db_pool_timeout_seconds: float
|
||||||
|
db_connect_timeout_seconds: float
|
||||||
|
db_command_timeout_seconds: float
|
||||||
|
redis_socket_timeout_seconds: float
|
||||||
|
redis_max_connections: int
|
||||||
|
|
||||||
|
|
||||||
|
def _int(name: str, default: int, low: int, high: int) -> int:
|
||||||
|
raw = os.environ.get(name, str(default)).strip()
|
||||||
|
try:
|
||||||
|
value = int(raw)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise SystemExit(f"{name} must be an integer") from exc
|
||||||
|
if not low <= value <= high:
|
||||||
|
raise SystemExit(f"{name} is out of range")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _float(name: str, default: float, low: float, high: float) -> float:
|
||||||
|
raw = os.environ.get(name, str(default)).strip()
|
||||||
|
try:
|
||||||
|
value = float(raw)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise SystemExit(f"{name} must be a number") from exc
|
||||||
|
if not low <= value <= high:
|
||||||
|
raise SystemExit(f"{name} is out of range")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _secret(direct_name: str, file_name: str) -> str:
|
||||||
|
direct = os.environ.get(direct_name, "").strip()
|
||||||
|
if direct:
|
||||||
|
return direct
|
||||||
|
path = os.environ.get(file_name, "").strip()
|
||||||
|
if not path:
|
||||||
|
raise SystemExit(f"set {direct_name} or {file_name}")
|
||||||
|
try:
|
||||||
|
value = Path(path).read_text(encoding="utf-8").strip()
|
||||||
|
except OSError as exc:
|
||||||
|
raise SystemExit(f"cannot read {file_name}") from exc
|
||||||
|
if not value:
|
||||||
|
raise SystemExit(f"{file_name} is empty")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _migrations_dir() -> Path:
|
||||||
|
configured = os.environ.get("MIGRATIONS_DIR", "").strip()
|
||||||
|
if configured:
|
||||||
|
return Path(configured)
|
||||||
|
return Path(__file__).resolve().parents[2] / "migrations"
|
||||||
|
|
||||||
|
|
||||||
|
def load_settings() -> Settings:
|
||||||
|
host = os.environ.get("APP_HOST", "0.0.0.0").strip() or "0.0.0.0"
|
||||||
|
env_name = os.environ.get("APP_ENV", "lab").strip() or "lab"
|
||||||
|
port = _int("APP_PORT", 8080, 1, 65535)
|
||||||
|
database_url = _secret("DATABASE_URL", "DATABASE_URL_FILE")
|
||||||
|
cache_url = _secret("REDIS_URL", "CACHE_URL_FILE")
|
||||||
|
if not database_url.startswith("postgresql+asyncpg://"):
|
||||||
|
raise SystemExit("DATABASE_URL must start with postgresql+asyncpg://")
|
||||||
|
if not cache_url.startswith(("redis://", "rediss://")):
|
||||||
|
raise SystemExit("REDIS_URL must start with redis:// or rediss://")
|
||||||
|
migrations = _migrations_dir()
|
||||||
|
if not migrations.is_dir():
|
||||||
|
raise SystemExit("migrations directory is missing")
|
||||||
|
return Settings(
|
||||||
|
host=host,
|
||||||
|
port=port,
|
||||||
|
env=env_name,
|
||||||
|
session_ttl_seconds=_int("SESSION_TTL_SECONDS", 3600, 1, 30 * 24 * 3600),
|
||||||
|
database_url=database_url,
|
||||||
|
cache_url=cache_url,
|
||||||
|
migrations_dir=migrations,
|
||||||
|
db_pool_size=_int("DB_POOL_SIZE", 10, 1, 50),
|
||||||
|
db_max_overflow=_int("DB_MAX_OVERFLOW", 10, 0, 50),
|
||||||
|
db_pool_timeout_seconds=_float("DB_POOL_TIMEOUT_SECONDS", 5, 0.1, 30),
|
||||||
|
db_connect_timeout_seconds=_float("DB_CONNECT_TIMEOUT_SECONDS", 2, 0.1, 30),
|
||||||
|
db_command_timeout_seconds=_float("DB_COMMAND_TIMEOUT_SECONDS", 5, 0.1, 60),
|
||||||
|
redis_socket_timeout_seconds=_float("REDIS_SOCKET_TIMEOUT_SECONDS", 1, 0.1, 30),
|
||||||
|
redis_max_connections=_int("REDIS_MAX_CONNECTIONS", 50, 1, 200),
|
||||||
|
)
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
"""Чтение и изменение PostgreSQL. Сессии живут в Redis, не в этих таблицах."""
|
||||||
@@ -0,0 +1,101 @@
|
|||||||
|
from uuid import UUID, uuid4
|
||||||
|
|
||||||
|
from sqlalchemy import select, tuple_
|
||||||
|
from sqlalchemy.exc import IntegrityError
|
||||||
|
|
||||||
|
from app.db import map_integrity, session_scope
|
||||||
|
from app.errors import AppError
|
||||||
|
from app.models import Chat, ChatMember, User
|
||||||
|
from app.pagination import decode_position, slice_page
|
||||||
|
from app.present import PublicChat
|
||||||
|
from app.timeutil import utcnow
|
||||||
|
|
||||||
|
|
||||||
|
def _invalid_members() -> AppError:
|
||||||
|
return AppError(400, "invalid_request", "Некорректный список участников")
|
||||||
|
|
||||||
|
|
||||||
|
async def create_chat(
|
||||||
|
owner_id: UUID,
|
||||||
|
title: str,
|
||||||
|
member_ids: tuple[UUID, ...],
|
||||||
|
*,
|
||||||
|
fail_after_insert: bool = False,
|
||||||
|
) -> PublicChat:
|
||||||
|
if (
|
||||||
|
not member_ids
|
||||||
|
or len(member_ids) > 100
|
||||||
|
or len(set(member_ids)) != len(member_ids)
|
||||||
|
or owner_id not in member_ids
|
||||||
|
):
|
||||||
|
raise _invalid_members()
|
||||||
|
chat_id = uuid4()
|
||||||
|
created_at = utcnow()
|
||||||
|
async with session_scope() as session:
|
||||||
|
try:
|
||||||
|
async with session.begin():
|
||||||
|
found = set((await session.scalars(select(User.id).where(User.id.in_(member_ids)))).all())
|
||||||
|
if found != set(member_ids):
|
||||||
|
raise _invalid_members()
|
||||||
|
session.add(Chat(id=chat_id, title=title, created_at=created_at))
|
||||||
|
await session.flush()
|
||||||
|
# Точка для проверки отката: строка чата уже вставлена, COMMIT ещё нет.
|
||||||
|
if fail_after_insert:
|
||||||
|
raise RuntimeError("rollback-check")
|
||||||
|
for ordinal, member_id in enumerate(member_ids):
|
||||||
|
session.add(ChatMember(chat_id=chat_id, user_id=member_id, ordinal=ordinal))
|
||||||
|
await session.flush()
|
||||||
|
except IntegrityError as exc:
|
||||||
|
raise map_integrity(exc, "Конфликт записи") from None
|
||||||
|
return PublicChat(id=chat_id, title=title, member_ids=tuple(member_ids), created_at=created_at)
|
||||||
|
|
||||||
|
|
||||||
|
async def _member_ids(session, chat_id: UUID) -> tuple[UUID, ...]:
|
||||||
|
rows = await session.execute(
|
||||||
|
select(ChatMember.user_id).where(ChatMember.chat_id == chat_id).order_by(ChatMember.ordinal)
|
||||||
|
)
|
||||||
|
return tuple(row[0] for row in rows)
|
||||||
|
|
||||||
|
|
||||||
|
async def chat_for_member(chat_id: UUID, user_id: UUID) -> PublicChat:
|
||||||
|
async with session_scope() as session:
|
||||||
|
async with session.begin():
|
||||||
|
member = await session.scalar(
|
||||||
|
select(ChatMember.user_id).where(ChatMember.chat_id == chat_id, ChatMember.user_id == user_id)
|
||||||
|
)
|
||||||
|
chat = await session.get(Chat, chat_id)
|
||||||
|
if member is None or chat is None:
|
||||||
|
raise AppError(404, "not_found", "Объект не найден")
|
||||||
|
return PublicChat(
|
||||||
|
id=chat.id,
|
||||||
|
title=chat.title,
|
||||||
|
member_ids=await _member_ids(session, chat.id),
|
||||||
|
created_at=chat.created_at,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def chats_for_user(user_id: UUID, limit: int, cursor: str | None) -> dict:
|
||||||
|
position = decode_position(cursor, "c", user_id) if cursor else None
|
||||||
|
async with session_scope() as session:
|
||||||
|
async with session.begin():
|
||||||
|
stmt = (
|
||||||
|
select(Chat)
|
||||||
|
.join(ChatMember, ChatMember.chat_id == Chat.id)
|
||||||
|
.where(ChatMember.user_id == user_id)
|
||||||
|
)
|
||||||
|
if position is not None:
|
||||||
|
created_at, cursor_id = position
|
||||||
|
stmt = stmt.where(tuple_(Chat.created_at, Chat.id) > tuple_(created_at, cursor_id))
|
||||||
|
stmt = stmt.order_by(Chat.created_at.asc(), Chat.id.asc()).limit(limit + 1)
|
||||||
|
chats = (await session.scalars(stmt)).unique().all()
|
||||||
|
public = []
|
||||||
|
for chat in chats:
|
||||||
|
public.append(
|
||||||
|
PublicChat(
|
||||||
|
id=chat.id,
|
||||||
|
title=chat.title,
|
||||||
|
member_ids=await _member_ids(session, chat.id),
|
||||||
|
created_at=chat.created_at,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return slice_page(public, limit, "c", user_id)
|
||||||
@@ -0,0 +1,72 @@
|
|||||||
|
from uuid import UUID, uuid4
|
||||||
|
|
||||||
|
from sqlalchemy import select, tuple_
|
||||||
|
from sqlalchemy.exc import IntegrityError
|
||||||
|
|
||||||
|
from app.db import map_integrity, session_scope
|
||||||
|
from app.errors import AppError
|
||||||
|
from app.models import ChatMember, Message
|
||||||
|
from app.pagination import decode_position, slice_page
|
||||||
|
from app.present import PublicMessage
|
||||||
|
from app.timeutil import utcnow
|
||||||
|
|
||||||
|
|
||||||
|
async def _require_member(session, chat_id: UUID, user_id: UUID) -> None:
|
||||||
|
member = await session.scalar(
|
||||||
|
select(ChatMember.user_id).where(ChatMember.chat_id == chat_id, ChatMember.user_id == user_id)
|
||||||
|
)
|
||||||
|
if member is None:
|
||||||
|
raise AppError(404, "not_found", "Объект не найден")
|
||||||
|
|
||||||
|
|
||||||
|
async def create_message(chat_id: UUID, sender_id: UUID, text: str) -> PublicMessage:
|
||||||
|
message_id = uuid4()
|
||||||
|
created_at = utcnow()
|
||||||
|
async with session_scope() as session:
|
||||||
|
try:
|
||||||
|
async with session.begin():
|
||||||
|
# Проверка членства и вставка коммитятся вместе: чужому чату строка не видна.
|
||||||
|
await _require_member(session, chat_id, sender_id)
|
||||||
|
session.add(
|
||||||
|
Message(
|
||||||
|
id=message_id,
|
||||||
|
chat_id=chat_id,
|
||||||
|
sender_id=sender_id,
|
||||||
|
text=text,
|
||||||
|
created_at=created_at,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
await session.flush()
|
||||||
|
except IntegrityError as exc:
|
||||||
|
raise map_integrity(exc, "Конфликт записи") from None
|
||||||
|
return PublicMessage(
|
||||||
|
id=message_id,
|
||||||
|
chat_id=chat_id,
|
||||||
|
sender_id=sender_id,
|
||||||
|
text=text,
|
||||||
|
created_at=created_at,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def messages_for_member(chat_id: UUID, user_id: UUID, limit: int, cursor: str | None) -> dict:
|
||||||
|
async with session_scope() as session:
|
||||||
|
async with session.begin():
|
||||||
|
await _require_member(session, chat_id, user_id)
|
||||||
|
position = decode_position(cursor, "m", chat_id) if cursor else None
|
||||||
|
stmt = select(Message).where(Message.chat_id == chat_id)
|
||||||
|
if position is not None:
|
||||||
|
created_at, message_id = position
|
||||||
|
stmt = stmt.where(tuple_(Message.created_at, Message.id) > tuple_(created_at, message_id))
|
||||||
|
stmt = stmt.order_by(Message.created_at.asc(), Message.id.asc()).limit(limit + 1)
|
||||||
|
rows = (await session.scalars(stmt)).all()
|
||||||
|
public = [
|
||||||
|
PublicMessage(
|
||||||
|
id=row.id,
|
||||||
|
chat_id=row.chat_id,
|
||||||
|
sender_id=row.sender_id,
|
||||||
|
text=row.text,
|
||||||
|
created_at=row.created_at,
|
||||||
|
)
|
||||||
|
for row in rows
|
||||||
|
]
|
||||||
|
return slice_page(public, limit, "m", chat_id)
|
||||||
@@ -0,0 +1,78 @@
|
|||||||
|
import re
|
||||||
|
from uuid import UUID, uuid4
|
||||||
|
|
||||||
|
from sqlalchemy import select
|
||||||
|
from sqlalchemy.exc import IntegrityError
|
||||||
|
|
||||||
|
from app.db import map_integrity, session_scope
|
||||||
|
from app.models import User
|
||||||
|
from app.present import PublicUser
|
||||||
|
from app.security import hash_password, verify_password
|
||||||
|
from app.timeutil import utcnow
|
||||||
|
|
||||||
|
_USERNAME = re.compile(r"^[a-z0-9_]{3,32}$")
|
||||||
|
|
||||||
|
|
||||||
|
async def get_user(user_id: UUID) -> PublicUser | None:
|
||||||
|
async with session_scope() as session:
|
||||||
|
async with session.begin():
|
||||||
|
row = (
|
||||||
|
await session.execute(
|
||||||
|
select(User.id, User.username, User.display_name, User.created_at).where(User.id == user_id)
|
||||||
|
)
|
||||||
|
).one_or_none()
|
||||||
|
if row is None:
|
||||||
|
return None
|
||||||
|
return PublicUser(id=row.id, username=row.username, display_name=row.display_name, created_at=row.created_at)
|
||||||
|
|
||||||
|
|
||||||
|
async def create_user(username: str, display_name: str, password: str) -> PublicUser:
|
||||||
|
# Argon2 считается до открытия транзакции, чтобы не держать соединение пула.
|
||||||
|
password_hash = hash_password(password)
|
||||||
|
user_id = uuid4()
|
||||||
|
created_at = utcnow()
|
||||||
|
async with session_scope() as session:
|
||||||
|
try:
|
||||||
|
async with session.begin():
|
||||||
|
session.add(
|
||||||
|
User(
|
||||||
|
id=user_id,
|
||||||
|
username=username,
|
||||||
|
display_name=display_name,
|
||||||
|
password_hash=password_hash,
|
||||||
|
created_at=created_at,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
await session.flush()
|
||||||
|
except IntegrityError as exc:
|
||||||
|
raise map_integrity(exc, "Имя пользователя занято") from None
|
||||||
|
return PublicUser(id=user_id, username=username, display_name=display_name, created_at=created_at)
|
||||||
|
|
||||||
|
|
||||||
|
async def authenticate(username: str, password: str) -> PublicUser | None:
|
||||||
|
if _USERNAME.fullmatch(username) is None or not 12 <= len(password) <= 128:
|
||||||
|
verify_password("rejected-credential", None)
|
||||||
|
return None
|
||||||
|
async with session_scope() as session:
|
||||||
|
async with session.begin():
|
||||||
|
row = (
|
||||||
|
await session.execute(
|
||||||
|
select(User.id, User.username, User.display_name, User.created_at, User.password_hash).where(
|
||||||
|
User.username == username
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).one_or_none()
|
||||||
|
if row is None:
|
||||||
|
stored = None
|
||||||
|
snapshot = None
|
||||||
|
else:
|
||||||
|
stored = row.password_hash
|
||||||
|
snapshot = PublicUser(
|
||||||
|
id=row.id,
|
||||||
|
username=row.username,
|
||||||
|
display_name=row.display_name,
|
||||||
|
created_at=row.created_at,
|
||||||
|
)
|
||||||
|
if not verify_password(password, stored):
|
||||||
|
return None
|
||||||
|
return snapshot
|
||||||
@@ -0,0 +1,79 @@
|
|||||||
|
import logging
|
||||||
|
from contextlib import asynccontextmanager
|
||||||
|
|
||||||
|
from redis.exceptions import RedisError
|
||||||
|
from sqlalchemy.exc import DBAPIError, IntegrityError, InterfaceError, OperationalError
|
||||||
|
from sqlalchemy.exc import TimeoutError as SATimeout
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncEngine, AsyncSession, async_sessionmaker, create_async_engine
|
||||||
|
|
||||||
|
from app.config import Settings
|
||||||
|
from app.errors import AppError
|
||||||
|
|
||||||
|
logger = logging.getLogger("app.db")
|
||||||
|
|
||||||
|
|
||||||
|
def build_engine(settings: Settings) -> AsyncEngine:
|
||||||
|
command_ms = str(int(settings.db_command_timeout_seconds * 1000))
|
||||||
|
lock_ms = str(int(settings.db_pool_timeout_seconds * 1000))
|
||||||
|
return create_async_engine(
|
||||||
|
settings.database_url,
|
||||||
|
pool_size=settings.db_pool_size,
|
||||||
|
max_overflow=settings.db_max_overflow,
|
||||||
|
pool_timeout=settings.db_pool_timeout_seconds,
|
||||||
|
pool_pre_ping=True,
|
||||||
|
pool_recycle=1800,
|
||||||
|
echo=False,
|
||||||
|
connect_args={
|
||||||
|
"timeout": settings.db_connect_timeout_seconds,
|
||||||
|
"command_timeout": settings.db_command_timeout_seconds,
|
||||||
|
"server_settings": {
|
||||||
|
"statement_timeout": command_ms,
|
||||||
|
"lock_timeout": lock_ms,
|
||||||
|
"idle_in_transaction_session_timeout": "10000",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def build_session_factory(engine: AsyncEngine) -> async_sessionmaker[AsyncSession]:
|
||||||
|
return async_sessionmaker(engine, expire_on_commit=False, autoflush=False)
|
||||||
|
|
||||||
|
|
||||||
|
def is_dependency_error(exc: BaseException) -> bool:
|
||||||
|
if isinstance(exc, (OperationalError, InterfaceError, SATimeout, RedisError, ConnectionError, TimeoutError)):
|
||||||
|
return True
|
||||||
|
return isinstance(exc, DBAPIError) and bool(getattr(exc, "connection_invalidated", False))
|
||||||
|
|
||||||
|
|
||||||
|
def map_integrity(exc: IntegrityError, duplicate_message: str) -> AppError:
|
||||||
|
orig = getattr(exc, "orig", None)
|
||||||
|
code = getattr(orig, "sqlstate", None) or getattr(orig, "pgcode", None)
|
||||||
|
if code == "23505":
|
||||||
|
return AppError(409, "conflict", duplicate_message)
|
||||||
|
if code in {"23503", "23514", "23502"}:
|
||||||
|
return AppError(400, "invalid_request", "Некорректный запрос")
|
||||||
|
return AppError(500, "internal_error", "Внутренняя ошибка")
|
||||||
|
|
||||||
|
|
||||||
|
@asynccontextmanager
|
||||||
|
async def session_scope():
|
||||||
|
from app.runtime import require_factory
|
||||||
|
|
||||||
|
factory = require_factory()
|
||||||
|
try:
|
||||||
|
session = factory()
|
||||||
|
except Exception as exc:
|
||||||
|
if is_dependency_error(exc):
|
||||||
|
logger.warning("database unavailable type=%s", type(exc).__name__)
|
||||||
|
raise AppError(503, "unavailable", "Сервис временно недоступен") from None
|
||||||
|
raise
|
||||||
|
try:
|
||||||
|
async with session:
|
||||||
|
yield session
|
||||||
|
except AppError:
|
||||||
|
raise
|
||||||
|
except Exception as exc:
|
||||||
|
if is_dependency_error(exc):
|
||||||
|
logger.warning("database unavailable type=%s", type(exc).__name__)
|
||||||
|
raise AppError(503, "unavailable", "Сервис временно недоступен") from None
|
||||||
|
raise
|
||||||
@@ -0,0 +1,50 @@
|
|||||||
|
from fastapi import Depends, Header, Request
|
||||||
|
from app.crud.users import get_user
|
||||||
|
from app.errors import AppError
|
||||||
|
from app.present import PublicUser
|
||||||
|
from app.sessions import resolve_session
|
||||||
|
|
||||||
|
BEARER_CHALLENGE = 'Bearer realm="messenger"'
|
||||||
|
BASIC_CHALLENGE = 'Basic realm="messenger", charset="UTF-8"'
|
||||||
|
|
||||||
|
|
||||||
|
def unauthorized(challenge: str) -> AppError:
|
||||||
|
return AppError(
|
||||||
|
401,
|
||||||
|
"unauthorized",
|
||||||
|
"Требуется идентификация",
|
||||||
|
headers={"WWW-Authenticate": challenge},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def get_current_user(
|
||||||
|
request: Request,
|
||||||
|
authorization: str | None = Header(default=None),
|
||||||
|
) -> PublicUser:
|
||||||
|
# Нет заголовка — клиент анонимный. Redis для этого не нужен и не делает его доверенным.
|
||||||
|
if authorization is None:
|
||||||
|
raise unauthorized(BEARER_CHALLENGE)
|
||||||
|
scheme, _, token = authorization.partition(" ")
|
||||||
|
token = token.strip()
|
||||||
|
if scheme.lower() != "bearer" or not token or len(token) > 512:
|
||||||
|
raise unauthorized(BEARER_CHALLENGE)
|
||||||
|
user_id = await resolve_session(token)
|
||||||
|
if user_id is None:
|
||||||
|
raise unauthorized(BEARER_CHALLENGE)
|
||||||
|
user = await get_user(user_id)
|
||||||
|
if user is None:
|
||||||
|
raise unauthorized(BEARER_CHALLENGE)
|
||||||
|
request.state.session_token = token
|
||||||
|
return user
|
||||||
|
|
||||||
|
|
||||||
|
def current_token(request: Request) -> str:
|
||||||
|
token = getattr(request.state, "session_token", "")
|
||||||
|
if not token:
|
||||||
|
raise unauthorized(BEARER_CHALLENGE)
|
||||||
|
return token
|
||||||
|
|
||||||
|
|
||||||
|
def require_token(request: Request, user: PublicUser = Depends(get_current_user)) -> str:
|
||||||
|
del user
|
||||||
|
return current_token(request)
|
||||||
@@ -0,0 +1,54 @@
|
|||||||
|
import logging
|
||||||
|
|
||||||
|
from fastapi import Request
|
||||||
|
from fastapi.exceptions import RequestValidationError
|
||||||
|
from fastapi.responses import JSONResponse
|
||||||
|
from starlette.exceptions import HTTPException
|
||||||
|
|
||||||
|
|
||||||
|
class AppError(Exception):
|
||||||
|
def __init__(self, status: int, code: str, message: str, headers: dict | None = None) -> None:
|
||||||
|
super().__init__(message)
|
||||||
|
self.status = status
|
||||||
|
self.code = code
|
||||||
|
self.message = message
|
||||||
|
self.headers = headers
|
||||||
|
|
||||||
|
|
||||||
|
def request_id_of(request: Request) -> str:
|
||||||
|
return getattr(request.state, "request_id", "") or "missing"
|
||||||
|
|
||||||
|
|
||||||
|
def error_response(
|
||||||
|
status: int,
|
||||||
|
code: str,
|
||||||
|
message: str,
|
||||||
|
request_id: str,
|
||||||
|
headers: dict | None = None,
|
||||||
|
) -> JSONResponse:
|
||||||
|
return JSONResponse(
|
||||||
|
status_code=status,
|
||||||
|
content={"error": {"code": code, "message": message, "request_id": request_id}},
|
||||||
|
headers=headers,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def handle_app_error(request: Request, exc: AppError) -> JSONResponse:
|
||||||
|
return error_response(exc.status, exc.code, exc.message, request_id_of(request), exc.headers)
|
||||||
|
|
||||||
|
|
||||||
|
async def handle_dependency(request: Request, exc: Exception) -> JSONResponse:
|
||||||
|
# Текст исключения может содержать SQL или параметры. В ответ и лог его не кладём.
|
||||||
|
logging.getLogger("app").warning("dependency unavailable type=%s", type(exc).__name__)
|
||||||
|
return error_response(503, "unavailable", "Сервис временно недоступен", request_id_of(request))
|
||||||
|
|
||||||
|
|
||||||
|
async def handle_validation_error(request: Request, _exc: RequestValidationError) -> JSONResponse:
|
||||||
|
return error_response(400, "invalid_request", "Некорректный запрос", request_id_of(request))
|
||||||
|
|
||||||
|
|
||||||
|
async def handle_http_error(request: Request, exc: HTTPException) -> JSONResponse:
|
||||||
|
# Тело не в UTF-8 FastAPI превращает в HTTPException 400 с полем detail.
|
||||||
|
if exc.status_code == 400:
|
||||||
|
return error_response(400, "invalid_request", "Некорректный запрос", request_id_of(request))
|
||||||
|
return JSONResponse(status_code=exc.status_code, content={"detail": exc.detail}, headers=exc.headers)
|
||||||
@@ -0,0 +1,93 @@
|
|||||||
|
import asyncio
|
||||||
|
import logging
|
||||||
|
from contextlib import asynccontextmanager
|
||||||
|
|
||||||
|
from fastapi import FastAPI
|
||||||
|
from fastapi.exceptions import RequestValidationError
|
||||||
|
from redis.exceptions import RedisError
|
||||||
|
from sqlalchemy import text
|
||||||
|
from sqlalchemy.exc import InterfaceError, OperationalError, ProgrammingError
|
||||||
|
from sqlalchemy.exc import TimeoutError as SATimeout
|
||||||
|
from starlette.exceptions import HTTPException
|
||||||
|
|
||||||
|
from app.api import api_router
|
||||||
|
from app.api.health import router as health_router
|
||||||
|
from app.config import load_settings
|
||||||
|
from app.db import build_engine, build_session_factory, is_dependency_error
|
||||||
|
from app.errors import AppError, handle_app_error, handle_dependency, handle_http_error, handle_validation_error
|
||||||
|
from app.migrate import apply_migrations
|
||||||
|
from app.middleware import request_context
|
||||||
|
from app.runtime import configure
|
||||||
|
from app.security import init_security
|
||||||
|
from app.sessions import build_redis
|
||||||
|
|
||||||
|
logger = logging.getLogger("app")
|
||||||
|
|
||||||
|
|
||||||
|
async def open_stores(engine, redis, migrations_dir) -> None:
|
||||||
|
for attempt in range(40):
|
||||||
|
try:
|
||||||
|
async with engine.connect() as conn:
|
||||||
|
await conn.execute(text("SELECT 1"))
|
||||||
|
await redis.ping()
|
||||||
|
applied = await apply_migrations(engine, migrations_dir)
|
||||||
|
logger.info("migrations newly_applied=%s", applied)
|
||||||
|
return
|
||||||
|
except SystemExit:
|
||||||
|
raise
|
||||||
|
except ProgrammingError as exc:
|
||||||
|
sqlstate = getattr(getattr(exc, "orig", None), "sqlstate", "")
|
||||||
|
logger.error("migration rejected type=%s sqlstate=%s", type(exc).__name__, sqlstate)
|
||||||
|
raise SystemExit("migration failed") from None
|
||||||
|
except Exception as exc:
|
||||||
|
if not is_dependency_error(exc):
|
||||||
|
logger.error("startup failed type=%s", type(exc).__name__)
|
||||||
|
raise SystemExit("startup failed") from None
|
||||||
|
logger.warning("startup not ready type=%s attempt=%s", type(exc).__name__, attempt + 1)
|
||||||
|
if attempt == 39:
|
||||||
|
raise SystemExit("database or redis is not ready") from None
|
||||||
|
await asyncio.sleep(0.5)
|
||||||
|
|
||||||
|
|
||||||
|
@asynccontextmanager
|
||||||
|
async def lifespan(app: FastAPI):
|
||||||
|
settings = load_settings()
|
||||||
|
init_security()
|
||||||
|
engine = build_engine(settings)
|
||||||
|
redis = build_redis(settings)
|
||||||
|
app.state.engine = engine
|
||||||
|
app.state.redis = redis
|
||||||
|
app.state.settings = settings
|
||||||
|
app.state.accepting = False
|
||||||
|
try:
|
||||||
|
await open_stores(engine, redis, settings.migrations_dir)
|
||||||
|
configure(build_session_factory(engine), redis, settings)
|
||||||
|
app.state.accepting = True
|
||||||
|
logger.info(
|
||||||
|
"ready env=%s session_ttl_seconds=%s db_pool=%s",
|
||||||
|
settings.env,
|
||||||
|
settings.session_ttl_seconds,
|
||||||
|
settings.db_pool_size,
|
||||||
|
)
|
||||||
|
yield
|
||||||
|
finally:
|
||||||
|
app.state.accepting = False
|
||||||
|
logger.info("shutdown: new requests are no longer accepted")
|
||||||
|
await redis.aclose()
|
||||||
|
await engine.dispose()
|
||||||
|
|
||||||
|
|
||||||
|
def create_app() -> FastAPI:
|
||||||
|
app = FastAPI(title="MTUSI Messenger", version="1.2.0", lifespan=lifespan)
|
||||||
|
app.add_exception_handler(AppError, handle_app_error)
|
||||||
|
app.add_exception_handler(RequestValidationError, handle_validation_error)
|
||||||
|
app.add_exception_handler(HTTPException, handle_http_error)
|
||||||
|
for cls in (OperationalError, InterfaceError, SATimeout, RedisError):
|
||||||
|
app.add_exception_handler(cls, handle_dependency)
|
||||||
|
app.middleware("http")(request_context)
|
||||||
|
app.include_router(health_router)
|
||||||
|
app.include_router(api_router)
|
||||||
|
return app
|
||||||
|
|
||||||
|
|
||||||
|
app = create_app()
|
||||||
@@ -0,0 +1,44 @@
|
|||||||
|
import logging
|
||||||
|
import time
|
||||||
|
from uuid import uuid4
|
||||||
|
|
||||||
|
from fastapi import Request
|
||||||
|
|
||||||
|
from app.db import is_dependency_error
|
||||||
|
from app.errors import error_response
|
||||||
|
|
||||||
|
logger = logging.getLogger("app.access")
|
||||||
|
|
||||||
|
|
||||||
|
async def request_context(request: Request, call_next):
|
||||||
|
request_id = uuid4().hex
|
||||||
|
request.state.request_id = request_id
|
||||||
|
started = time.perf_counter()
|
||||||
|
status = 500
|
||||||
|
try:
|
||||||
|
accepting = getattr(request.app.state, "accepting", True)
|
||||||
|
if not accepting and request.url.path != "/health/live":
|
||||||
|
response = error_response(503, "unavailable", "Сервер останавливается", request_id)
|
||||||
|
else:
|
||||||
|
try:
|
||||||
|
response = await call_next(request)
|
||||||
|
except Exception as exc:
|
||||||
|
if is_dependency_error(exc):
|
||||||
|
logger.warning("dependency unavailable type=%s", type(exc).__name__)
|
||||||
|
response = error_response(503, "unavailable", "Сервис временно недоступен", request_id)
|
||||||
|
else:
|
||||||
|
logger.error("internal error request_id=%s type=%s", request_id, type(exc).__name__)
|
||||||
|
response = error_response(500, "internal_error", "Внутренняя ошибка", request_id)
|
||||||
|
response.headers["X-Request-Id"] = request_id
|
||||||
|
status = response.status_code
|
||||||
|
return response
|
||||||
|
finally:
|
||||||
|
elapsed_ms = int((time.perf_counter() - started) * 1000)
|
||||||
|
logger.info(
|
||||||
|
"request method=%s path=%s status=%s request_id=%s duration_ms=%s",
|
||||||
|
request.method,
|
||||||
|
request.url.path,
|
||||||
|
status,
|
||||||
|
request_id,
|
||||||
|
elapsed_ms,
|
||||||
|
)
|
||||||
@@ -0,0 +1,60 @@
|
|||||||
|
import logging
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from sqlalchemy import text
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncEngine
|
||||||
|
|
||||||
|
logger = logging.getLogger("app.migrate")
|
||||||
|
|
||||||
|
# Один номер на все процессы API, чтобы две реплики не применяли DDL одновременно.
|
||||||
|
_ADVISORY_LOCK = 842021
|
||||||
|
|
||||||
|
|
||||||
|
def sql_statements(script: str) -> list[str]:
|
||||||
|
lines = []
|
||||||
|
for line in script.splitlines():
|
||||||
|
if line.strip().startswith("--"):
|
||||||
|
continue
|
||||||
|
lines.append(line)
|
||||||
|
statements = []
|
||||||
|
for part in "\n".join(lines).split(";"):
|
||||||
|
statement = part.strip()
|
||||||
|
if statement:
|
||||||
|
statements.append(statement)
|
||||||
|
return statements
|
||||||
|
|
||||||
|
|
||||||
|
async def apply_migrations(engine: AsyncEngine, directory: Path) -> int:
|
||||||
|
files = sorted(path for path in directory.glob("*.sql") if path.is_file())
|
||||||
|
if not files:
|
||||||
|
raise SystemExit("no SQL migrations found")
|
||||||
|
applied = 0
|
||||||
|
async with engine.begin() as conn:
|
||||||
|
await conn.execute(text("SELECT pg_advisory_xact_lock(:lock_id)"), {"lock_id": _ADVISORY_LOCK})
|
||||||
|
await conn.execute(
|
||||||
|
text(
|
||||||
|
"""
|
||||||
|
CREATE TABLE IF NOT EXISTS schema_migrations (
|
||||||
|
version TEXT PRIMARY KEY,
|
||||||
|
applied_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||||
|
)
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
)
|
||||||
|
for path in files:
|
||||||
|
version = path.name
|
||||||
|
exists = await conn.scalar(
|
||||||
|
text("SELECT version FROM schema_migrations WHERE version = :version"),
|
||||||
|
{"version": version},
|
||||||
|
)
|
||||||
|
if exists:
|
||||||
|
continue
|
||||||
|
for statement in sql_statements(path.read_text(encoding="utf-8")):
|
||||||
|
await conn.execute(text(statement))
|
||||||
|
await conn.execute(
|
||||||
|
text("INSERT INTO schema_migrations (version) VALUES (:version)"),
|
||||||
|
{"version": version},
|
||||||
|
)
|
||||||
|
applied += 1
|
||||||
|
logger.info("migration applied version=%s", version)
|
||||||
|
return applied
|
||||||
@@ -0,0 +1,89 @@
|
|||||||
|
import uuid
|
||||||
|
from datetime import datetime
|
||||||
|
|
||||||
|
from sqlalchemy import (
|
||||||
|
CheckConstraint,
|
||||||
|
DateTime,
|
||||||
|
ForeignKey,
|
||||||
|
Index,
|
||||||
|
SmallInteger,
|
||||||
|
Text,
|
||||||
|
UniqueConstraint,
|
||||||
|
)
|
||||||
|
from sqlalchemy.dialects.postgresql import UUID
|
||||||
|
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column
|
||||||
|
|
||||||
|
|
||||||
|
class Base(DeclarativeBase):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class User(Base):
|
||||||
|
__tablename__ = "users"
|
||||||
|
__table_args__ = (
|
||||||
|
UniqueConstraint("username", name="uq_users_username"),
|
||||||
|
CheckConstraint("username ~ '^[a-z0-9_]{3,32}$'", name="ck_users_username_format"),
|
||||||
|
CheckConstraint(
|
||||||
|
"char_length(btrim(display_name)) >= 1 AND char_length(display_name) <= 100",
|
||||||
|
name="ck_users_display_name_len",
|
||||||
|
),
|
||||||
|
CheckConstraint("char_length(password_hash) >= 20", name="ck_users_password_hash_len"),
|
||||||
|
)
|
||||||
|
|
||||||
|
id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True)
|
||||||
|
username: Mapped[str] = mapped_column(Text, nullable=False)
|
||||||
|
display_name: Mapped[str] = mapped_column(Text, nullable=False)
|
||||||
|
password_hash: Mapped[str] = mapped_column(Text, nullable=False)
|
||||||
|
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||||
|
|
||||||
|
|
||||||
|
class Chat(Base):
|
||||||
|
__tablename__ = "chats"
|
||||||
|
__table_args__ = (
|
||||||
|
CheckConstraint(
|
||||||
|
"char_length(btrim(title)) >= 1 AND char_length(title) <= 100",
|
||||||
|
name="ck_chats_title",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True)
|
||||||
|
title: Mapped[str] = mapped_column(Text, nullable=False)
|
||||||
|
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||||
|
|
||||||
|
|
||||||
|
class ChatMember(Base):
|
||||||
|
__tablename__ = "chat_members"
|
||||||
|
__table_args__ = (
|
||||||
|
CheckConstraint("ordinal >= 0 AND ordinal < 100", name="ck_chat_members_ordinal"),
|
||||||
|
Index("ix_chat_members_user", "user_id", "chat_id"),
|
||||||
|
)
|
||||||
|
|
||||||
|
chat_id: Mapped[uuid.UUID] = mapped_column(
|
||||||
|
UUID(as_uuid=True),
|
||||||
|
ForeignKey("chats.id"),
|
||||||
|
primary_key=True,
|
||||||
|
)
|
||||||
|
user_id: Mapped[uuid.UUID] = mapped_column(
|
||||||
|
UUID(as_uuid=True),
|
||||||
|
ForeignKey("users.id"),
|
||||||
|
primary_key=True,
|
||||||
|
)
|
||||||
|
ordinal: Mapped[int] = mapped_column(SmallInteger, nullable=False)
|
||||||
|
|
||||||
|
|
||||||
|
class Message(Base):
|
||||||
|
__tablename__ = "messages"
|
||||||
|
__table_args__ = (
|
||||||
|
CheckConstraint(
|
||||||
|
"char_length(btrim(body)) >= 1 AND char_length(body) <= 4000",
|
||||||
|
name="ck_messages_text",
|
||||||
|
),
|
||||||
|
# Keyset: WHERE chat_id = $1 AND (created_at, id) > ($2, $3) ORDER BY created_at, id LIMIT $4
|
||||||
|
Index("ix_messages_history", "chat_id", "created_at", "id"),
|
||||||
|
)
|
||||||
|
|
||||||
|
id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True)
|
||||||
|
chat_id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), ForeignKey("chats.id"), nullable=False)
|
||||||
|
sender_id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), ForeignKey("users.id"), nullable=False)
|
||||||
|
text: Mapped[str] = mapped_column("body", Text, nullable=False)
|
||||||
|
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||||
@@ -0,0 +1,50 @@
|
|||||||
|
import base64
|
||||||
|
import json
|
||||||
|
from datetime import datetime
|
||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
from app.errors import AppError
|
||||||
|
from app.timeutil import format_time, parse_time
|
||||||
|
|
||||||
|
|
||||||
|
def encode_cursor(kind: str, resource_id: UUID, created_at: datetime, entity_id: UUID) -> str:
|
||||||
|
payload = {
|
||||||
|
"i": str(entity_id),
|
||||||
|
"k": kind,
|
||||||
|
"r": str(resource_id),
|
||||||
|
"t": format_time(created_at),
|
||||||
|
}
|
||||||
|
raw = json.dumps(payload, separators=(",", ":"), sort_keys=True).encode("ascii")
|
||||||
|
return base64.urlsafe_b64encode(raw).decode("ascii").rstrip("=")
|
||||||
|
|
||||||
|
|
||||||
|
def decode_position(token: str, kind: str, resource_id: UUID) -> tuple[datetime, UUID]:
|
||||||
|
if len(token) > 1024:
|
||||||
|
raise AppError(400, "invalid_request", "Некорректный cursor")
|
||||||
|
padding = "=" * (-len(token) % 4)
|
||||||
|
try:
|
||||||
|
data = json.loads(base64.urlsafe_b64decode(token + padding))
|
||||||
|
if (
|
||||||
|
not isinstance(data, dict)
|
||||||
|
or data.get("k") != kind
|
||||||
|
or data.get("r") != str(resource_id)
|
||||||
|
or not isinstance(data.get("t"), str)
|
||||||
|
or not isinstance(data.get("i"), str)
|
||||||
|
):
|
||||||
|
raise ValueError("cursor")
|
||||||
|
created_at = parse_time(data["t"])
|
||||||
|
entity_id = UUID(data["i"])
|
||||||
|
except (KeyError, TypeError, ValueError, AttributeError, json.JSONDecodeError):
|
||||||
|
raise AppError(400, "invalid_request", "Некорректный cursor") from None
|
||||||
|
return created_at, entity_id
|
||||||
|
|
||||||
|
|
||||||
|
def slice_page(rows: list, limit: int, kind: str, resource_id: UUID) -> dict:
|
||||||
|
# SQL уже вернул не больше limit + 1 строк, отфильтрованных keyset-условием.
|
||||||
|
has_more = len(rows) > limit
|
||||||
|
page = rows[:limit]
|
||||||
|
next_cursor = None
|
||||||
|
if has_more and page:
|
||||||
|
last = page[-1]
|
||||||
|
next_cursor = encode_cursor(kind, resource_id, last.created_at, last.id)
|
||||||
|
return {"items": [row.to_dict() for row in page], "next_cursor": next_cursor}
|
||||||
@@ -0,0 +1,55 @@
|
|||||||
|
from dataclasses import dataclass
|
||||||
|
from datetime import datetime
|
||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
from app.timeutil import format_time
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class PublicUser:
|
||||||
|
id: UUID
|
||||||
|
username: str
|
||||||
|
display_name: str
|
||||||
|
created_at: datetime
|
||||||
|
|
||||||
|
def to_dict(self) -> dict:
|
||||||
|
return {
|
||||||
|
"id": self.id,
|
||||||
|
"username": self.username,
|
||||||
|
"display_name": self.display_name,
|
||||||
|
"created_at": format_time(self.created_at),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class PublicChat:
|
||||||
|
id: UUID
|
||||||
|
title: str
|
||||||
|
member_ids: tuple[UUID, ...]
|
||||||
|
created_at: datetime
|
||||||
|
|
||||||
|
def to_dict(self) -> dict:
|
||||||
|
return {
|
||||||
|
"id": self.id,
|
||||||
|
"title": self.title,
|
||||||
|
"member_ids": list(self.member_ids),
|
||||||
|
"created_at": format_time(self.created_at),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class PublicMessage:
|
||||||
|
id: UUID
|
||||||
|
chat_id: UUID
|
||||||
|
sender_id: UUID
|
||||||
|
text: str
|
||||||
|
created_at: datetime
|
||||||
|
|
||||||
|
def to_dict(self) -> dict:
|
||||||
|
return {
|
||||||
|
"id": self.id,
|
||||||
|
"chat_id": self.chat_id,
|
||||||
|
"sender_id": self.sender_id,
|
||||||
|
"text": self.text,
|
||||||
|
"created_at": format_time(self.created_at),
|
||||||
|
}
|
||||||
@@ -0,0 +1,36 @@
|
|||||||
|
from app.config import Settings
|
||||||
|
|
||||||
|
_factory = None
|
||||||
|
_redis = None
|
||||||
|
_settings: Settings | None = None
|
||||||
|
|
||||||
|
|
||||||
|
def configure(factory, redis, settings: Settings) -> None:
|
||||||
|
global _factory, _redis, _settings
|
||||||
|
_factory = factory
|
||||||
|
_redis = redis
|
||||||
|
_settings = settings
|
||||||
|
|
||||||
|
|
||||||
|
def require_factory():
|
||||||
|
if _factory is None:
|
||||||
|
from app.errors import AppError
|
||||||
|
|
||||||
|
raise AppError(503, "unavailable", "Сервис временно недоступен")
|
||||||
|
return _factory
|
||||||
|
|
||||||
|
|
||||||
|
def require_redis():
|
||||||
|
if _redis is None:
|
||||||
|
from app.errors import AppError
|
||||||
|
|
||||||
|
raise AppError(503, "unavailable", "Сервис временно недоступен")
|
||||||
|
return _redis
|
||||||
|
|
||||||
|
|
||||||
|
def require_settings() -> Settings:
|
||||||
|
if _settings is None:
|
||||||
|
from app.errors import AppError
|
||||||
|
|
||||||
|
raise AppError(503, "unavailable", "Сервис временно недоступен")
|
||||||
|
return _settings
|
||||||
@@ -0,0 +1,35 @@
|
|||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
from pydantic import BaseModel, ConfigDict, Field, field_validator
|
||||||
|
|
||||||
|
|
||||||
|
class CreateChat(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
title: str
|
||||||
|
member_ids: list[UUID] = Field(min_length=1, max_length=100)
|
||||||
|
|
||||||
|
@field_validator("title")
|
||||||
|
@classmethod
|
||||||
|
def check_title(cls, value: str) -> str:
|
||||||
|
if value.strip() == "" or len(value) > 100:
|
||||||
|
raise ValueError("title")
|
||||||
|
return value
|
||||||
|
|
||||||
|
@field_validator("member_ids")
|
||||||
|
@classmethod
|
||||||
|
def unique_members(cls, value: list[UUID]) -> list[UUID]:
|
||||||
|
if len(set(value)) != len(value):
|
||||||
|
raise ValueError("duplicate members")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
class ChatOut(BaseModel):
|
||||||
|
id: UUID
|
||||||
|
title: str
|
||||||
|
member_ids: list[UUID]
|
||||||
|
created_at: str
|
||||||
|
|
||||||
|
|
||||||
|
class ChatPage(BaseModel):
|
||||||
|
items: list[ChatOut]
|
||||||
|
next_cursor: str | None
|
||||||
@@ -0,0 +1,28 @@
|
|||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
from pydantic import BaseModel, ConfigDict, field_validator
|
||||||
|
|
||||||
|
|
||||||
|
class CreateMessage(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
text: str
|
||||||
|
|
||||||
|
@field_validator("text")
|
||||||
|
@classmethod
|
||||||
|
def check_text(cls, value: str) -> str:
|
||||||
|
if value.strip() == "" or len(value) > 4000:
|
||||||
|
raise ValueError("text")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
class MessageOut(BaseModel):
|
||||||
|
id: UUID
|
||||||
|
chat_id: UUID
|
||||||
|
sender_id: UUID
|
||||||
|
text: str
|
||||||
|
created_at: str
|
||||||
|
|
||||||
|
|
||||||
|
class MessagePage(BaseModel):
|
||||||
|
items: list[MessageOut]
|
||||||
|
next_cursor: str | None
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
from pydantic import BaseModel
|
||||||
|
|
||||||
|
from app.schemas.user import UserOut
|
||||||
|
|
||||||
|
|
||||||
|
class SessionOut(BaseModel):
|
||||||
|
token: str
|
||||||
|
token_type: str
|
||||||
|
expires_at: str
|
||||||
|
user: UserOut
|
||||||
@@ -0,0 +1,41 @@
|
|||||||
|
import re
|
||||||
|
|
||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
from pydantic import BaseModel, ConfigDict, field_validator
|
||||||
|
|
||||||
|
|
||||||
|
class CreateUser(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
username: str
|
||||||
|
display_name: str
|
||||||
|
password: str
|
||||||
|
|
||||||
|
@field_validator("username")
|
||||||
|
@classmethod
|
||||||
|
def check_username(cls, value: str) -> str:
|
||||||
|
if re.fullmatch(r"[a-z0-9_]{3,32}", value) is None:
|
||||||
|
raise ValueError("username")
|
||||||
|
return value
|
||||||
|
|
||||||
|
@field_validator("display_name")
|
||||||
|
@classmethod
|
||||||
|
def check_display_name(cls, value: str) -> str:
|
||||||
|
if value.strip() == "" or len(value) > 100:
|
||||||
|
raise ValueError("display_name")
|
||||||
|
return value
|
||||||
|
|
||||||
|
@field_validator("password")
|
||||||
|
@classmethod
|
||||||
|
def check_password(cls, value: str) -> str:
|
||||||
|
# Длина в code points. Пароль не обрезается и не нормализуется.
|
||||||
|
if not 12 <= len(value) <= 128:
|
||||||
|
raise ValueError("password")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
class UserOut(BaseModel):
|
||||||
|
id: UUID
|
||||||
|
username: str
|
||||||
|
display_name: str
|
||||||
|
created_at: str
|
||||||
@@ -0,0 +1,66 @@
|
|||||||
|
import base64
|
||||||
|
import binascii
|
||||||
|
|
||||||
|
from argon2 import PasswordHasher
|
||||||
|
from argon2.exceptions import InvalidHashError, VerifyMismatchError
|
||||||
|
from argon2.low_level import Type
|
||||||
|
|
||||||
|
# Учебные параметры Argon2id: устойчивее SHA-256, но подъём на ноутбуке
|
||||||
|
# остаётся коротким. В lab7 параметры будут заданы курсом отдельно.
|
||||||
|
_TIME_COST = 2
|
||||||
|
_MEMORY_COST = 32768
|
||||||
|
_PARALLELISM = 1
|
||||||
|
|
||||||
|
_hasher: PasswordHasher | None = None
|
||||||
|
_dummy_hash: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
def get_hasher() -> PasswordHasher:
|
||||||
|
global _hasher
|
||||||
|
if _hasher is None:
|
||||||
|
_hasher = PasswordHasher(
|
||||||
|
time_cost=_TIME_COST,
|
||||||
|
memory_cost=_MEMORY_COST,
|
||||||
|
parallelism=_PARALLELISM,
|
||||||
|
hash_len=32,
|
||||||
|
salt_len=16,
|
||||||
|
type=Type.ID,
|
||||||
|
)
|
||||||
|
return _hasher
|
||||||
|
|
||||||
|
|
||||||
|
def init_security() -> None:
|
||||||
|
global _dummy_hash
|
||||||
|
if _dummy_hash is None:
|
||||||
|
_dummy_hash = get_hasher().hash("messenger-dummy-password-not-used-for-login")
|
||||||
|
|
||||||
|
|
||||||
|
def hash_password(password: str) -> str:
|
||||||
|
return get_hasher().hash(password)
|
||||||
|
|
||||||
|
|
||||||
|
def verify_password(password: str, password_hash: str | None) -> bool:
|
||||||
|
init_security()
|
||||||
|
try:
|
||||||
|
get_hasher().verify(password_hash or _dummy_hash, password)
|
||||||
|
matched = True
|
||||||
|
except (VerifyMismatchError, InvalidHashError):
|
||||||
|
matched = False
|
||||||
|
return password_hash is not None and matched
|
||||||
|
|
||||||
|
|
||||||
|
def parse_basic(header: str) -> tuple[str, str] | None:
|
||||||
|
if not header:
|
||||||
|
return None
|
||||||
|
scheme, _, rest = header.partition(" ")
|
||||||
|
if scheme.lower() != "basic" or not rest.strip():
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
decoded = base64.b64decode(rest.strip(), validate=True)
|
||||||
|
text = decoded.decode("utf-8")
|
||||||
|
except (ValueError, UnicodeError, binascii.Error):
|
||||||
|
return None
|
||||||
|
username, separator, password = text.partition(":")
|
||||||
|
if separator != ":" or len(username) > 256 or len(password) > 1024:
|
||||||
|
return None
|
||||||
|
return username, password
|
||||||
@@ -0,0 +1,90 @@
|
|||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import secrets
|
||||||
|
from datetime import timedelta
|
||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
from redis.asyncio import Redis
|
||||||
|
from redis.exceptions import RedisError
|
||||||
|
|
||||||
|
from app.config import Settings
|
||||||
|
from app.errors import AppError
|
||||||
|
from app.runtime import require_redis, require_settings
|
||||||
|
from app.timeutil import format_time, parse_time, utcnow
|
||||||
|
|
||||||
|
logger = logging.getLogger("app.sessions")
|
||||||
|
|
||||||
|
|
||||||
|
def build_redis(settings: Settings) -> Redis:
|
||||||
|
return Redis.from_url(
|
||||||
|
settings.cache_url,
|
||||||
|
decode_responses=True,
|
||||||
|
socket_timeout=settings.redis_socket_timeout_seconds,
|
||||||
|
socket_connect_timeout=settings.redis_socket_timeout_seconds,
|
||||||
|
health_check_interval=30,
|
||||||
|
max_connections=settings.redis_max_connections,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def session_key(token: str) -> str:
|
||||||
|
digest = hashlib.sha256(token.encode("utf-8")).hexdigest()
|
||||||
|
return "sess:" + digest
|
||||||
|
|
||||||
|
|
||||||
|
async def _call(awaitable):
|
||||||
|
try:
|
||||||
|
return await awaitable
|
||||||
|
except RedisError:
|
||||||
|
logger.warning("redis unavailable type=%s", "RedisError")
|
||||||
|
raise AppError(503, "unavailable", "Сервис временно недоступен") from None
|
||||||
|
|
||||||
|
|
||||||
|
def _read_payload(raw: str) -> tuple[UUID, object] | None:
|
||||||
|
try:
|
||||||
|
data = json.loads(raw)
|
||||||
|
if not isinstance(data, dict):
|
||||||
|
raise ValueError("payload")
|
||||||
|
return UUID(str(data["user_id"])), parse_time(data["expires_at"])
|
||||||
|
except (ValueError, KeyError, TypeError):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
async def create_session(user_id: UUID) -> tuple[str, object]:
|
||||||
|
settings = require_settings()
|
||||||
|
token = secrets.token_urlsafe(32)
|
||||||
|
expires_at = utcnow() + timedelta(seconds=settings.session_ttl_seconds)
|
||||||
|
payload = json.dumps(
|
||||||
|
{"user_id": str(user_id), "expires_at": format_time(expires_at)},
|
||||||
|
separators=(",", ":"),
|
||||||
|
)
|
||||||
|
# PXAT — абсолютное время. Повтор AOF после рестарта не продлевает TTL.
|
||||||
|
await _call(require_redis().set(session_key(token), payload, pxat=int(expires_at.timestamp() * 1000)))
|
||||||
|
return token, expires_at
|
||||||
|
|
||||||
|
|
||||||
|
async def resolve_session(token: str) -> UUID | None:
|
||||||
|
key = session_key(token)
|
||||||
|
raw = await _call(require_redis().get(key))
|
||||||
|
if raw is None:
|
||||||
|
return None
|
||||||
|
parsed = _read_payload(raw)
|
||||||
|
if parsed is None:
|
||||||
|
await _call(require_redis().delete(key))
|
||||||
|
return None
|
||||||
|
user_id, expires_at = parsed
|
||||||
|
if expires_at <= utcnow():
|
||||||
|
await _call(require_redis().delete(key))
|
||||||
|
return None
|
||||||
|
return user_id
|
||||||
|
|
||||||
|
|
||||||
|
async def revoke_session(token: str) -> bool:
|
||||||
|
raw = await _call(require_redis().getdel(session_key(token)))
|
||||||
|
if raw is None:
|
||||||
|
return False
|
||||||
|
parsed = _read_payload(raw)
|
||||||
|
if parsed is None:
|
||||||
|
return False
|
||||||
|
_, expires_at = parsed
|
||||||
|
return expires_at > utcnow()
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
"""Состояние лабораторной 2 хранится в PostgreSQL и Redis, не в памяти процесса."""
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
"""Память процесса больше не хранит пользователей, чаты и сообщения.
|
||||||
|
|
||||||
|
Источник истины для этих данных — PostgreSQL.
|
||||||
|
Источник истины для сессий — Redis с AOF и абсолютным PXAT.
|
||||||
|
"""
|
||||||
@@ -0,0 +1,20 @@
|
|||||||
|
from datetime import datetime, timezone
|
||||||
|
|
||||||
|
|
||||||
|
def utcnow() -> datetime:
|
||||||
|
return datetime.now(timezone.utc)
|
||||||
|
|
||||||
|
|
||||||
|
def format_time(value: datetime) -> str:
|
||||||
|
if value.tzinfo is None:
|
||||||
|
value = value.replace(tzinfo=timezone.utc)
|
||||||
|
return value.astimezone(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S.%fZ")
|
||||||
|
|
||||||
|
|
||||||
|
def parse_time(value: str) -> datetime:
|
||||||
|
if not isinstance(value, str) or not value.endswith("Z"):
|
||||||
|
raise ValueError("time")
|
||||||
|
parsed = datetime.fromisoformat(value[:-1] + "+00:00")
|
||||||
|
if parsed.tzinfo is None:
|
||||||
|
raise ValueError("time")
|
||||||
|
return parsed.astimezone(timezone.utc)
|
||||||
@@ -0,0 +1,630 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Проверки уровня 4: гонка, откат, EXPLAIN, TTL, logout и отказ Redis/PostgreSQL.
|
||||||
|
|
||||||
|
Стенд уже должен быть поднят: scripts/bootstrap и docker compose up.
|
||||||
|
Скрипт останавливает redis и postgres и снова их запускает.
|
||||||
|
Токены и пароли не печатаются и не пишутся в evidence.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import base64
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import socket
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
import urllib.error
|
||||||
|
import urllib.request
|
||||||
|
import uuid
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
EVIDENCE = ROOT / "evidence"
|
||||||
|
sys.path.insert(0, str(ROOT / "src"))
|
||||||
|
|
||||||
|
BASE = os.environ.get("BASE_URL", "http://127.0.0.1:8080").rstrip("/")
|
||||||
|
|
||||||
|
|
||||||
|
class Failure(AssertionError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def check(condition, message):
|
||||||
|
if not condition:
|
||||||
|
raise Failure(message)
|
||||||
|
|
||||||
|
|
||||||
|
def host_secret(name):
|
||||||
|
path = ROOT / "secrets" / name
|
||||||
|
if not path.is_file():
|
||||||
|
raise Failure("missing secrets/%s; run scripts/bootstrap first" % name)
|
||||||
|
value = path.read_text(encoding="utf-8").strip()
|
||||||
|
if not value:
|
||||||
|
raise Failure("empty secrets/%s" % name)
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def call(method, path, expected, body=None, token=None, authorization=None, user_id=None, timeout=15):
|
||||||
|
headers = {"Accept": "application/json"}
|
||||||
|
data = None
|
||||||
|
if body is not None:
|
||||||
|
data = json.dumps(body, ensure_ascii=False).encode("utf-8")
|
||||||
|
headers["Content-Type"] = "application/json"
|
||||||
|
if authorization:
|
||||||
|
headers["Authorization"] = authorization
|
||||||
|
elif token:
|
||||||
|
headers["Authorization"] = "Bearer " + token
|
||||||
|
if user_id:
|
||||||
|
headers["X-User-Id"] = user_id
|
||||||
|
request = urllib.request.Request(BASE + path, data=data, headers=headers, method=method)
|
||||||
|
try:
|
||||||
|
response = urllib.request.urlopen(request, timeout=timeout)
|
||||||
|
except urllib.error.HTTPError as error:
|
||||||
|
response = error
|
||||||
|
except (urllib.error.URLError, TimeoutError, OSError) as exc:
|
||||||
|
raise Failure(method + " " + path.split("?")[0] + " failed: " + type(exc).__name__) from None
|
||||||
|
with response:
|
||||||
|
status = response.status
|
||||||
|
payload = response.read()
|
||||||
|
header_id = response.headers.get("X-Request-Id")
|
||||||
|
challenge = response.headers.get("WWW-Authenticate", "")
|
||||||
|
allowed = [expected] if isinstance(expected, int) else list(expected)
|
||||||
|
check(status in allowed, "%s %s: expected %s, got %s" % (method, path.split("?")[0], allowed, status))
|
||||||
|
check(header_id and header_id.strip(), "missing X-Request-Id")
|
||||||
|
parsed = None
|
||||||
|
if payload:
|
||||||
|
try:
|
||||||
|
parsed = json.loads(payload)
|
||||||
|
except json.JSONDecodeError as exc:
|
||||||
|
raise Failure("response is not JSON") from exc
|
||||||
|
if status >= 400:
|
||||||
|
check(isinstance(parsed, dict) and set(parsed) == {"error"}, "error envelope")
|
||||||
|
error = parsed["error"]
|
||||||
|
check(set(error) == {"code", "message", "request_id"}, "error fields")
|
||||||
|
check(error["request_id"] == header_id, "request_id differs from header")
|
||||||
|
lowered = error["message"].lower()
|
||||||
|
for leaked in ("traceback", "select ", "password", "syntax", "asyncpg", "sqlalchemy"):
|
||||||
|
check(leaked not in lowered, "error message leaked internals")
|
||||||
|
if status == 204:
|
||||||
|
check(not payload, "204 must not contain a body")
|
||||||
|
return {"status": status, "json": parsed, "www": challenge}
|
||||||
|
|
||||||
|
|
||||||
|
def basic(username, password):
|
||||||
|
raw = base64.b64encode(("%s:%s" % (username, password)).encode("utf-8")).decode("ascii")
|
||||||
|
return "Basic " + raw
|
||||||
|
|
||||||
|
|
||||||
|
def register(label, password=None):
|
||||||
|
username = ("u_%s_%s" % (label, uuid.uuid4().hex[:10])).lower()
|
||||||
|
password = password or ("T3st-" + uuid.uuid4().hex)
|
||||||
|
created = call(
|
||||||
|
"POST",
|
||||||
|
"/api/v1/users",
|
||||||
|
201,
|
||||||
|
body={"username": username, "display_name": "Студент " + label, "password": password},
|
||||||
|
)["json"]
|
||||||
|
check("password" not in created, "password leaked")
|
||||||
|
session = call("POST", "/api/v1/auth/sessions", 201, authorization=basic(username, password))["json"]
|
||||||
|
check(session["token_type"] == "Bearer" and session["user"]["id"] == created["id"], "session shape")
|
||||||
|
check(len(session["token"]) >= 32, "token is too short")
|
||||||
|
return {"id": created["id"], "username": username, "password": password, "token": session["token"]}
|
||||||
|
|
||||||
|
|
||||||
|
def compose(*args, timeout=120):
|
||||||
|
proc = subprocess.run(
|
||||||
|
["docker", "compose", *args],
|
||||||
|
cwd=str(ROOT),
|
||||||
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
timeout=timeout,
|
||||||
|
encoding="utf-8",
|
||||||
|
errors="replace",
|
||||||
|
)
|
||||||
|
if proc.returncode != 0:
|
||||||
|
raise Failure("docker compose %s failed with code %s" % (args[0], proc.returncode))
|
||||||
|
return proc.stdout
|
||||||
|
|
||||||
|
|
||||||
|
def psql(sql, timeout=60):
|
||||||
|
proc = subprocess.run(
|
||||||
|
[
|
||||||
|
"docker",
|
||||||
|
"compose",
|
||||||
|
"exec",
|
||||||
|
"-T",
|
||||||
|
"postgres",
|
||||||
|
"psql",
|
||||||
|
"-U",
|
||||||
|
"messenger",
|
||||||
|
"-d",
|
||||||
|
"messenger",
|
||||||
|
"-v",
|
||||||
|
"ON_ERROR_STOP=1",
|
||||||
|
"-At",
|
||||||
|
"-c",
|
||||||
|
sql,
|
||||||
|
],
|
||||||
|
cwd=str(ROOT),
|
||||||
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
timeout=timeout,
|
||||||
|
encoding="utf-8",
|
||||||
|
errors="replace",
|
||||||
|
)
|
||||||
|
if proc.returncode != 0:
|
||||||
|
raise Failure("psql failed")
|
||||||
|
return proc.stdout.strip()
|
||||||
|
|
||||||
|
|
||||||
|
def redis_cli(*args):
|
||||||
|
proc = subprocess.run(
|
||||||
|
["docker", "compose", "exec", "-T", "redis", "redis-cli", *args],
|
||||||
|
cwd=str(ROOT),
|
||||||
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
timeout=20,
|
||||||
|
encoding="utf-8",
|
||||||
|
errors="replace",
|
||||||
|
)
|
||||||
|
if proc.returncode != 0:
|
||||||
|
raise Failure("redis-cli failed")
|
||||||
|
return proc.stdout.strip()
|
||||||
|
|
||||||
|
|
||||||
|
def sql_quote(value):
|
||||||
|
text = str(value)
|
||||||
|
if "'" in text or ";" in text or "\n" in text:
|
||||||
|
raise Failure("refusing unsafe SQL literal")
|
||||||
|
return "'" + text + "'"
|
||||||
|
|
||||||
|
|
||||||
|
def free_port():
|
||||||
|
with socket.socket() as sock:
|
||||||
|
sock.bind(("127.0.0.1", 0))
|
||||||
|
return sock.getsockname()[1]
|
||||||
|
|
||||||
|
|
||||||
|
def wait_ready(timeout=90):
|
||||||
|
deadline = time.monotonic() + timeout
|
||||||
|
last = ""
|
||||||
|
while time.monotonic() < deadline:
|
||||||
|
try:
|
||||||
|
call("GET", "/health/ready", 200, timeout=5)
|
||||||
|
return
|
||||||
|
except Failure as exc:
|
||||||
|
last = str(exc)
|
||||||
|
time.sleep(0.5)
|
||||||
|
raise Failure("API did not become ready: " + last)
|
||||||
|
|
||||||
|
|
||||||
|
def test_credentials():
|
||||||
|
user = register("cred")
|
||||||
|
wrong = call(
|
||||||
|
"POST",
|
||||||
|
"/api/v1/auth/sessions",
|
||||||
|
401,
|
||||||
|
authorization=basic(user["username"], "wrong-password-value"),
|
||||||
|
)
|
||||||
|
missing = call(
|
||||||
|
"POST",
|
||||||
|
"/api/v1/auth/sessions",
|
||||||
|
401,
|
||||||
|
authorization=basic("no_such_user_zz", "wrong-password-value"),
|
||||||
|
)
|
||||||
|
check(wrong["json"]["error"]["code"] == "unauthorized", "wrong password code")
|
||||||
|
check(wrong["json"]["error"]["message"] == missing["json"]["error"]["message"], "credential errors differ")
|
||||||
|
check(wrong["www"].lower().startswith("basic "), "basic challenge")
|
||||||
|
bare = call("GET", "/api/v1/users/me", 401)
|
||||||
|
check(bare["www"].lower().startswith("bearer "), "bearer challenge")
|
||||||
|
only_header = call("GET", "/api/v1/users/me", 401, user_id=user["id"])
|
||||||
|
check(only_header["json"]["error"]["code"] == "unauthorized", "X-User-Id alone was accepted")
|
||||||
|
other = register("other")
|
||||||
|
me = call("GET", "/api/v1/users/me", 200, token=user["token"], user_id=other["id"])["json"]
|
||||||
|
check(me["id"] == user["id"], "X-User-Id overrode bearer")
|
||||||
|
basic_on_api = call("GET", "/api/v1/users/me", 401, authorization=basic(user["username"], user["password"]))
|
||||||
|
check(basic_on_api["www"].lower().startswith("bearer "), "basic accepted on business API")
|
||||||
|
print("PASS: credentials, same 401, X-User-Id ignored")
|
||||||
|
|
||||||
|
|
||||||
|
def test_unicode_password():
|
||||||
|
password = "Пароль:пример-12"
|
||||||
|
check(12 <= len(password) <= 128 and ":" in password, "fixture password")
|
||||||
|
user = register("uni", password=password)
|
||||||
|
again = call("POST", "/api/v1/auth/sessions", 201, authorization=basic(user["username"], password))["json"]
|
||||||
|
check(again["user"]["id"] == user["id"], "unicode password login failed")
|
||||||
|
print("PASS: unicode password with colon")
|
||||||
|
|
||||||
|
|
||||||
|
def test_logout():
|
||||||
|
user = register("bye")
|
||||||
|
call("DELETE", "/api/v1/auth/sessions/current", 204, token=user["token"])
|
||||||
|
rejected = call("GET", "/api/v1/users/me", 401, token=user["token"])
|
||||||
|
check(rejected["www"].lower().startswith("bearer "), "revoked token challenge")
|
||||||
|
again = call("DELETE", "/api/v1/auth/sessions/current", 401, token=user["token"])
|
||||||
|
check(again["json"]["error"]["code"] == "unauthorized", "second logout")
|
||||||
|
print("PASS: logout revokes the session")
|
||||||
|
|
||||||
|
|
||||||
|
def test_injection_and_rollback():
|
||||||
|
user = register("sql")
|
||||||
|
chat = call(
|
||||||
|
"POST",
|
||||||
|
"/api/v1/chats",
|
||||||
|
201,
|
||||||
|
token=user["token"],
|
||||||
|
body={"title": "SQL", "member_ids": [user["id"]]},
|
||||||
|
)["json"]
|
||||||
|
payload = "'; DROP TABLE messages; --"
|
||||||
|
sent = call(
|
||||||
|
"POST",
|
||||||
|
"/api/v1/chats/%s/messages" % chat["id"],
|
||||||
|
201,
|
||||||
|
token=user["token"],
|
||||||
|
body={"text": payload},
|
||||||
|
)["json"]
|
||||||
|
check(sent["text"] == payload, "text was rewritten")
|
||||||
|
check(psql("SELECT to_regclass('public.messages')") == "messages", "messages table disappeared")
|
||||||
|
listed = call("GET", "/api/v1/chats/%s/messages?limit=10" % chat["id"], 200, token=user["token"])["json"]
|
||||||
|
check(listed["items"][0]["text"] == payload, "stored text differs")
|
||||||
|
|
||||||
|
before = int(psql("SELECT count(*) FROM chats"))
|
||||||
|
title = "partial-" + uuid.uuid4().hex
|
||||||
|
rejected = call(
|
||||||
|
"POST",
|
||||||
|
"/api/v1/chats",
|
||||||
|
400,
|
||||||
|
token=user["token"],
|
||||||
|
body={"title": title, "member_ids": [user["id"], str(uuid.uuid4())]},
|
||||||
|
)
|
||||||
|
check(rejected["json"]["error"]["code"] == "invalid_request", "unknown member code")
|
||||||
|
after = int(psql("SELECT count(*) FROM chats"))
|
||||||
|
check(before == after, "unknown member committed a chat")
|
||||||
|
check(psql("SELECT count(*) FROM chats WHERE title = %s" % sql_quote(title)) == "0", "partial chat row exists")
|
||||||
|
print("PASS: parameterized text and unknown-member chat is not stored")
|
||||||
|
return user, chat
|
||||||
|
|
||||||
|
|
||||||
|
async def failpoint(owner_id, other_id, title):
|
||||||
|
os.environ["DATABASE_URL"] = host_secret("host_database_url")
|
||||||
|
os.environ["REDIS_URL"] = host_secret("host_cache_url")
|
||||||
|
from app.config import load_settings
|
||||||
|
from app.crud.chats import create_chat
|
||||||
|
from app.db import build_engine, build_session_factory
|
||||||
|
from app.runtime import configure
|
||||||
|
from app.sessions import build_redis
|
||||||
|
|
||||||
|
settings = load_settings()
|
||||||
|
engine = build_engine(settings)
|
||||||
|
redis = build_redis(settings)
|
||||||
|
configure(build_session_factory(engine), redis, settings)
|
||||||
|
try:
|
||||||
|
try:
|
||||||
|
await create_chat(
|
||||||
|
uuid.UUID(owner_id),
|
||||||
|
title,
|
||||||
|
(uuid.UUID(owner_id), uuid.UUID(other_id)),
|
||||||
|
fail_after_insert=True,
|
||||||
|
)
|
||||||
|
except RuntimeError as exc:
|
||||||
|
check(str(exc) == "rollback-check", "failpoint raised a different error")
|
||||||
|
else:
|
||||||
|
raise Failure("failpoint committed a chat")
|
||||||
|
finally:
|
||||||
|
await redis.aclose()
|
||||||
|
await engine.dispose()
|
||||||
|
|
||||||
|
|
||||||
|
def test_failpoint(owner_id, other_id):
|
||||||
|
title = "rollback-" + uuid.uuid4().hex
|
||||||
|
before = int(psql("SELECT count(*) FROM chats"))
|
||||||
|
try:
|
||||||
|
asyncio.run(failpoint(owner_id, other_id, title))
|
||||||
|
except Failure:
|
||||||
|
raise
|
||||||
|
except Exception as exc:
|
||||||
|
raise Failure("rollback check failed: " + type(exc).__name__) from None
|
||||||
|
after = int(psql("SELECT count(*) FROM chats"))
|
||||||
|
left = psql("SELECT count(*) FROM chats WHERE title = %s" % sql_quote(title))
|
||||||
|
check(before == after and left == "0", "chat insert survived the rollback")
|
||||||
|
print("PASS: composite chat insert rolls back before commit")
|
||||||
|
return {"chats_before": before, "chats_after": after, "title_rows": int(left)}
|
||||||
|
|
||||||
|
|
||||||
|
def test_race():
|
||||||
|
username = "race_" + uuid.uuid4().hex[:8]
|
||||||
|
password = "Race-password-" + uuid.uuid4().hex[:8]
|
||||||
|
body = {"username": username, "display_name": "Гонка", "password": password}
|
||||||
|
barrier = threading.Barrier(16)
|
||||||
|
statuses = []
|
||||||
|
lock = threading.Lock()
|
||||||
|
|
||||||
|
def worker():
|
||||||
|
try:
|
||||||
|
barrier.wait(20)
|
||||||
|
result = call("POST", "/api/v1/users", [201, 409], body=body, timeout=30)
|
||||||
|
with lock:
|
||||||
|
statuses.append(result["status"])
|
||||||
|
except Exception as exc:
|
||||||
|
with lock:
|
||||||
|
statuses.append(str(exc)[:120])
|
||||||
|
|
||||||
|
threads = [threading.Thread(target=worker) for _ in range(16)]
|
||||||
|
for thread in threads:
|
||||||
|
thread.start()
|
||||||
|
for thread in threads:
|
||||||
|
thread.join()
|
||||||
|
created = statuses.count(201)
|
||||||
|
conflict = statuses.count(409)
|
||||||
|
check(created == 1 and conflict == 15 and len(statuses) == 16, "race statuses %s" % statuses)
|
||||||
|
print("PASS: 16 parallel registrations, one 201 and fifteen 409")
|
||||||
|
return {"created": created, "conflict": conflict}
|
||||||
|
|
||||||
|
|
||||||
|
def assert_plan(name, plan):
|
||||||
|
check("ix_messages_history" in plan, name + " plan ignores ix_messages_history")
|
||||||
|
check("Seq Scan" not in plan, name + " plan seq-scans messages")
|
||||||
|
check("Index Scan" in plan or "Index Only Scan" in plan, name + " plan is not an index scan")
|
||||||
|
check("Buffers:" in plan, name + " plan has no buffer stats")
|
||||||
|
|
||||||
|
|
||||||
|
def test_explain(user):
|
||||||
|
chat = call(
|
||||||
|
"POST",
|
||||||
|
"/api/v1/chats",
|
||||||
|
201,
|
||||||
|
token=user["token"],
|
||||||
|
body={"title": "План", "member_ids": [user["id"]]},
|
||||||
|
)["json"]
|
||||||
|
chat_sql = sql_quote(chat["id"])
|
||||||
|
user_sql = sql_quote(user["id"])
|
||||||
|
psql(
|
||||||
|
"INSERT INTO messages (id, chat_id, sender_id, body, created_at) "
|
||||||
|
"SELECT gen_random_uuid(), %s, %s, 'row-' || g, "
|
||||||
|
"TIMESTAMPTZ '2026-01-01 00:00:00+00' + (g || ' microseconds')::interval "
|
||||||
|
"FROM generate_series(1, 10000) AS g" % (chat_sql, user_sql),
|
||||||
|
timeout=60,
|
||||||
|
)
|
||||||
|
count = int(psql("SELECT count(*) FROM messages WHERE chat_id = %s" % chat_sql))
|
||||||
|
check(count >= 10000, "expected at least 10000 messages, got %s" % count)
|
||||||
|
psql("ANALYZE messages")
|
||||||
|
midpoint = psql(
|
||||||
|
"SELECT to_char(created_at AT TIME ZONE 'UTC', 'YYYY-MM-DD HH24:MI:SS.US'), id::text "
|
||||||
|
"FROM messages WHERE chat_id = %s ORDER BY created_at, id OFFSET 5000 LIMIT 1" % chat_sql
|
||||||
|
)
|
||||||
|
created_at, message_id = midpoint.split("|")
|
||||||
|
late = psql(
|
||||||
|
"EXPLAIN (ANALYZE, BUFFERS) SELECT id, body FROM messages "
|
||||||
|
"WHERE chat_id = %s AND (created_at, id) > (%s::timestamptz, %s::uuid) "
|
||||||
|
"ORDER BY created_at, id LIMIT 50"
|
||||||
|
% (chat_sql, sql_quote(created_at + "+00"), sql_quote(message_id)),
|
||||||
|
timeout=60,
|
||||||
|
)
|
||||||
|
first = psql(
|
||||||
|
"EXPLAIN (ANALYZE, BUFFERS) SELECT id, body FROM messages "
|
||||||
|
"WHERE chat_id = %s ORDER BY created_at, id LIMIT 50" % chat_sql,
|
||||||
|
timeout=60,
|
||||||
|
)
|
||||||
|
EVIDENCE.mkdir(exist_ok=True)
|
||||||
|
(EVIDENCE / "explain_messages.txt").write_text(
|
||||||
|
"messages=%s\n\n-- first page\n%s\n\n-- page after row 5000\n%s\n" % (count, first, late),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
assert_plan("first", first)
|
||||||
|
assert_plan("late", late)
|
||||||
|
print("PASS: EXPLAIN uses ix_messages_history on %s messages" % count)
|
||||||
|
return {"messages": count, "chat_id": chat["id"]}
|
||||||
|
|
||||||
|
|
||||||
|
def test_expiry_and_migrations():
|
||||||
|
from app.timeutil import parse_time
|
||||||
|
|
||||||
|
users_before = int(psql("SELECT count(*) FROM users"))
|
||||||
|
port = free_port()
|
||||||
|
env = os.environ.copy()
|
||||||
|
env["PYTHONPATH"] = str(ROOT / "src")
|
||||||
|
env["PYTHONUNBUFFERED"] = "1"
|
||||||
|
env["APP_HOST"] = "127.0.0.1"
|
||||||
|
env["APP_PORT"] = str(port)
|
||||||
|
env["APP_ENV"] = "lab"
|
||||||
|
env["DATABASE_URL"] = host_secret("host_database_url")
|
||||||
|
env["REDIS_URL"] = host_secret("host_cache_url")
|
||||||
|
env["SESSION_TTL_SECONDS"] = "2"
|
||||||
|
env.pop("DATABASE_URL_FILE", None)
|
||||||
|
env.pop("CACHE_URL_FILE", None)
|
||||||
|
proc = subprocess.Popen(
|
||||||
|
[sys.executable, "-m", "app"],
|
||||||
|
cwd=str(ROOT),
|
||||||
|
env=env,
|
||||||
|
stdout=subprocess.PIPE,
|
||||||
|
stderr=subprocess.STDOUT,
|
||||||
|
text=True,
|
||||||
|
encoding="utf-8",
|
||||||
|
errors="replace",
|
||||||
|
)
|
||||||
|
logs = []
|
||||||
|
|
||||||
|
def read_logs():
|
||||||
|
if proc.stdout is None:
|
||||||
|
return
|
||||||
|
for line in proc.stdout:
|
||||||
|
if "://" not in line and "password" not in line.lower():
|
||||||
|
logs.append(line)
|
||||||
|
|
||||||
|
reader = threading.Thread(target=read_logs, daemon=True)
|
||||||
|
reader.start()
|
||||||
|
base = "http://127.0.0.1:%s" % port
|
||||||
|
try:
|
||||||
|
deadline = time.monotonic() + 40
|
||||||
|
ready = False
|
||||||
|
while time.monotonic() < deadline:
|
||||||
|
if proc.poll() is not None:
|
||||||
|
raise Failure("short-ttl server exited: " + "".join(logs[-8:]))
|
||||||
|
try:
|
||||||
|
urllib.request.urlopen(base + "/health/ready", timeout=2).read()
|
||||||
|
ready = True
|
||||||
|
break
|
||||||
|
except (urllib.error.URLError, TimeoutError, OSError):
|
||||||
|
time.sleep(0.2)
|
||||||
|
check(ready, "short-ttl server did not become ready: " + "".join(logs[-8:]))
|
||||||
|
username = "ttl_" + uuid.uuid4().hex[:8]
|
||||||
|
password = "Ttl-password-" + uuid.uuid4().hex[:8]
|
||||||
|
body = json.dumps(
|
||||||
|
{"username": username, "display_name": "TTL", "password": password},
|
||||||
|
ensure_ascii=False,
|
||||||
|
).encode("utf-8")
|
||||||
|
request = urllib.request.Request(
|
||||||
|
base + "/api/v1/users",
|
||||||
|
data=body,
|
||||||
|
headers={"Content-Type": "application/json", "Accept": "application/json"},
|
||||||
|
method="POST",
|
||||||
|
)
|
||||||
|
with urllib.request.urlopen(request, timeout=20) as response:
|
||||||
|
created = json.loads(response.read())
|
||||||
|
login = urllib.request.Request(
|
||||||
|
base + "/api/v1/auth/sessions",
|
||||||
|
data=b"",
|
||||||
|
headers={"Authorization": basic(username, password), "Accept": "application/json"},
|
||||||
|
method="POST",
|
||||||
|
)
|
||||||
|
with urllib.request.urlopen(login, timeout=20) as response:
|
||||||
|
session = json.loads(response.read())
|
||||||
|
expires = parse_time(session["expires_at"])
|
||||||
|
delta = (expires - datetime.now(timezone.utc)).total_seconds()
|
||||||
|
check(0 < delta <= 3, "SESSION_TTL_SECONDS=2 was not applied")
|
||||||
|
time.sleep(delta + 1.0)
|
||||||
|
probe = urllib.request.Request(
|
||||||
|
base + "/api/v1/users/me",
|
||||||
|
headers={"Authorization": "Bearer " + session["token"], "Accept": "application/json"},
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(probe, timeout=10) as response:
|
||||||
|
status = response.status
|
||||||
|
except urllib.error.HTTPError as error:
|
||||||
|
status = error.status
|
||||||
|
check(status == 401, "expired session status %s" % status)
|
||||||
|
finally:
|
||||||
|
if proc.poll() is None:
|
||||||
|
proc.terminate()
|
||||||
|
try:
|
||||||
|
proc.wait(timeout=8)
|
||||||
|
except subprocess.TimeoutExpired:
|
||||||
|
proc.kill()
|
||||||
|
proc.wait(timeout=5)
|
||||||
|
reader.join(timeout=2)
|
||||||
|
users_after = int(psql("SELECT count(*) FROM users"))
|
||||||
|
versions = psql("SELECT count(*) FROM schema_migrations")
|
||||||
|
version_name = psql("SELECT version FROM schema_migrations")
|
||||||
|
check(users_after == users_before + 1, "migrations duplicated or dropped users")
|
||||||
|
check(versions == "1" and version_name == "001_initial.sql", "migration version changed")
|
||||||
|
print("PASS: short TTL expires and migrations stay idempotent")
|
||||||
|
return {"users_before": users_before, "users_after": users_after, "migration": version_name}
|
||||||
|
|
||||||
|
|
||||||
|
def test_redis_persistence(user):
|
||||||
|
from app.sessions import session_key
|
||||||
|
|
||||||
|
before = int(redis_cli("PTTL", session_key(user["token"])))
|
||||||
|
check(before > 300000, "session ttl is unexpectedly short")
|
||||||
|
compose("restart", "redis")
|
||||||
|
wait_ready()
|
||||||
|
after = int(redis_cli("PTTL", session_key(user["token"])))
|
||||||
|
check(after > 0, "session disappeared after redis restart")
|
||||||
|
check(after <= before + 1000, "redis restart extended the session ttl")
|
||||||
|
me = call("GET", "/api/v1/users/me", 200, token=user["token"])["json"]
|
||||||
|
check(me["id"] == user["id"], "token did not survive redis restart")
|
||||||
|
print("PASS: Redis AOF kept the session and did not refresh TTL")
|
||||||
|
return {"pttl_before_ms": before, "pttl_after_ms": after}
|
||||||
|
|
||||||
|
|
||||||
|
def test_dependency_loss(user, chat_id):
|
||||||
|
observed = {}
|
||||||
|
compose("stop", "redis")
|
||||||
|
try:
|
||||||
|
live = call("GET", "/health/live", 200)
|
||||||
|
ready = call("GET", "/health/ready", 503)
|
||||||
|
anonymous = call("GET", "/api/v1/users/me", 401)
|
||||||
|
trusted = call("GET", "/api/v1/users/me", 503, token=user["token"])
|
||||||
|
check(live["json"]["status"] == "ok", "live failed while redis is down")
|
||||||
|
check(ready["json"]["error"]["code"] == "unavailable", "ready code")
|
||||||
|
check(anonymous["json"]["error"]["code"] == "unauthorized", "anonymous became trusted")
|
||||||
|
check(trusted["json"]["error"]["code"] == "unavailable", "unverified token was accepted")
|
||||||
|
observed["redis_down"] = {
|
||||||
|
"live": live["status"],
|
||||||
|
"ready": ready["status"],
|
||||||
|
"anonymous": anonymous["status"],
|
||||||
|
"bearer": trusted["status"],
|
||||||
|
}
|
||||||
|
finally:
|
||||||
|
compose("start", "redis")
|
||||||
|
wait_ready()
|
||||||
|
me = call("GET", "/api/v1/users/me", 200, token=user["token"])["json"]
|
||||||
|
check(me["id"] == user["id"], "token lost after redis came back")
|
||||||
|
|
||||||
|
compose("stop", "postgres")
|
||||||
|
try:
|
||||||
|
live = call("GET", "/health/live", 200)
|
||||||
|
ready = call("GET", "/health/ready", 503, timeout=20)
|
||||||
|
anonymous = call("GET", "/api/v1/users/me", 401, timeout=20)
|
||||||
|
denied = call("GET", "/api/v1/chats/" + chat_id, 503, token=user["token"], timeout=20)
|
||||||
|
check(live["json"]["status"] == "ok", "live depends on postgres")
|
||||||
|
check(ready["json"]["error"]["code"] == "unavailable", "ready while postgres is down")
|
||||||
|
check(anonymous["json"]["error"]["code"] == "unauthorized", "anonymous trusted without postgres")
|
||||||
|
check(denied["json"]["error"]["code"] == "unavailable", "request served without postgres")
|
||||||
|
observed["postgres_down"] = {
|
||||||
|
"live": live["status"],
|
||||||
|
"ready": ready["status"],
|
||||||
|
"anonymous": anonymous["status"],
|
||||||
|
"bearer": denied["status"],
|
||||||
|
}
|
||||||
|
finally:
|
||||||
|
compose("start", "postgres")
|
||||||
|
wait_ready(timeout=90)
|
||||||
|
history = call("GET", "/api/v1/chats/%s/messages?limit=5" % chat_id, 200, token=user["token"])["json"]
|
||||||
|
check(history["items"], "messages lost after postgres restart")
|
||||||
|
print("PASS: redis and postgres outages return 503 and do not trust anonymous clients")
|
||||||
|
return observed
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
os.environ["DATABASE_URL"] = host_secret("host_database_url")
|
||||||
|
os.environ["REDIS_URL"] = host_secret("host_cache_url")
|
||||||
|
wait_ready(timeout=60)
|
||||||
|
test_credentials()
|
||||||
|
test_unicode_password()
|
||||||
|
test_logout()
|
||||||
|
user, chat = test_injection_and_rollback()
|
||||||
|
other = register("pair")
|
||||||
|
rollback = test_failpoint(user["id"], other["id"])
|
||||||
|
race = test_race()
|
||||||
|
explain = test_explain(user)
|
||||||
|
expiry = test_expiry_and_migrations()
|
||||||
|
pttl = test_redis_persistence(user)
|
||||||
|
faults = test_dependency_loss(user, chat["id"])
|
||||||
|
remaining = int(psql("SELECT count(*) FROM messages WHERE chat_id = %s" % sql_quote(explain["chat_id"])))
|
||||||
|
check(remaining == explain["messages"], "bulk history lost after postgres restart")
|
||||||
|
EVIDENCE.mkdir(exist_ok=True)
|
||||||
|
report = {
|
||||||
|
"date": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
|
||||||
|
"base_url": BASE,
|
||||||
|
"race": race,
|
||||||
|
"rollback": rollback,
|
||||||
|
"explain": {"messages": explain["messages"]},
|
||||||
|
"expiry_and_migrations": expiry,
|
||||||
|
"redis_restart": pttl,
|
||||||
|
"dependency_loss": faults,
|
||||||
|
"messages_after_postgres_restart": remaining,
|
||||||
|
}
|
||||||
|
(EVIDENCE / "level4.json").write_text(json.dumps(report, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||||
|
print("PASS: level 4 checks")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
try:
|
||||||
|
main()
|
||||||
|
except Failure as exc:
|
||||||
|
print("FAIL:", exc)
|
||||||
|
sys.exit(1)
|
||||||
@@ -0,0 +1,69 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Тот же сценарий, что tests/resilience.py, но hook запускается так, как умеет ОС.
|
||||||
|
|
||||||
|
Linux выполняет scripts/restart.sh напрямую.
|
||||||
|
Windows не умеет CreateProcess для shell-скрипта, поэтому здесь используется Git Bash.
|
||||||
|
Скрипт тот же: перезапуск api, postgres и redis без удаления томов.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import os
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
from urllib.parse import quote
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
sys.path.insert(0, str(ROOT / "tests"))
|
||||||
|
|
||||||
|
from client import Client, Failure, check, run
|
||||||
|
|
||||||
|
|
||||||
|
def hook_command():
|
||||||
|
script = ROOT / "scripts" / "restart.sh"
|
||||||
|
if os.name != "nt":
|
||||||
|
return [str(script)]
|
||||||
|
bash = Path(os.environ.get("BASH", r"C:\Program Files\Git\bin\bash.exe"))
|
||||||
|
check(bash.is_file(), "Git Bash is required to launch scripts/restart.sh on Windows")
|
||||||
|
return [str(bash), "-c", "sh scripts/restart.sh"]
|
||||||
|
|
||||||
|
|
||||||
|
def history(client, actor, chat):
|
||||||
|
path = "/api/v1/chats/" + chat["id"] + "/messages?limit=100"
|
||||||
|
result = []
|
||||||
|
seen = set()
|
||||||
|
for _ in range(1000):
|
||||||
|
page = client.call("GET", path, actor=actor, schema="MessagePage")["json"]
|
||||||
|
result.extend(page["items"])
|
||||||
|
cursor = page["next_cursor"]
|
||||||
|
if cursor is None:
|
||||||
|
return result
|
||||||
|
check(cursor not in seen, "cursor cycle")
|
||||||
|
seen.add(cursor)
|
||||||
|
path = "/api/v1/chats/" + chat["id"] + "/messages?limit=100&cursor=" + quote(cursor, safe="")
|
||||||
|
raise Failure("history pagination exceeded safety bound")
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
client = Client("http://127.0.0.1:8080", timeout=5)
|
||||||
|
alice, bob, _eve, chat = client.bootstrap()
|
||||||
|
original = client.message(alice, chat, "Сохранить при отказе")["json"]
|
||||||
|
print("Executing scripts/restart.sh")
|
||||||
|
completed = subprocess.run(hook_command(), cwd=str(ROOT))
|
||||||
|
check(completed.returncode == 0, "restart hook failed")
|
||||||
|
deadline = time.monotonic() + 90
|
||||||
|
while True:
|
||||||
|
try:
|
||||||
|
client.call("GET", "/health/ready", schema="Health")
|
||||||
|
client.call("GET", "/api/v1/users/me", actor=alice, schema="User")
|
||||||
|
break
|
||||||
|
except Failure:
|
||||||
|
check(time.monotonic() < deadline, "stack/session did not recover within 90 seconds")
|
||||||
|
time.sleep(0.5)
|
||||||
|
messages = history(client, bob, chat)
|
||||||
|
check(any(item == original for item in messages), "confirmed message lost or altered")
|
||||||
|
print("PASS: persistence preserved confirmed data and existing session")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
run(main)
|
||||||
@@ -0,0 +1,508 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
|
||||||
|
import base64
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import socket
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
import urllib.error
|
||||||
|
import urllib.request
|
||||||
|
import uuid
|
||||||
|
from pathlib import Path
|
||||||
|
from urllib.parse import quote
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
EVIDENCE = ROOT / "evidence"
|
||||||
|
SECRET = "SECRET_MESSAGE_DO_NOT_LOG_7f3a"
|
||||||
|
CLIENTS = 10
|
||||||
|
PER_CLIENT = 10
|
||||||
|
|
||||||
|
|
||||||
|
class Failure(AssertionError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def check(condition, message):
|
||||||
|
if not condition:
|
||||||
|
raise Failure(message)
|
||||||
|
|
||||||
|
|
||||||
|
def free_port():
|
||||||
|
with socket.socket() as sock:
|
||||||
|
sock.bind(("127.0.0.1", 0))
|
||||||
|
return sock.getsockname()[1]
|
||||||
|
|
||||||
|
|
||||||
|
class Server:
|
||||||
|
def __init__(self, port):
|
||||||
|
self.base = "http://127.0.0.1:%d" % port
|
||||||
|
env = os.environ.copy()
|
||||||
|
env["PYTHONPATH"] = str(ROOT / "src")
|
||||||
|
env["PYTHONUNBUFFERED"] = "1"
|
||||||
|
env["APP_HOST"] = "127.0.0.1"
|
||||||
|
env["APP_PORT"] = str(port)
|
||||||
|
env["APP_ENV"] = "lab"
|
||||||
|
env["DATABASE_URL"] = host_secret("host_database_url")
|
||||||
|
env["REDIS_URL"] = host_secret("host_cache_url")
|
||||||
|
env["SESSION_TTL_SECONDS"] = "3600"
|
||||||
|
env.pop("DATABASE_URL_FILE", None)
|
||||||
|
env.pop("CACHE_URL_FILE", None)
|
||||||
|
self.logs = []
|
||||||
|
self.proc = subprocess.Popen(
|
||||||
|
[sys.executable, "-m", "app"],
|
||||||
|
cwd=str(ROOT),
|
||||||
|
env=env,
|
||||||
|
stdout=subprocess.PIPE,
|
||||||
|
stderr=subprocess.STDOUT,
|
||||||
|
text=True,
|
||||||
|
encoding="utf-8",
|
||||||
|
errors="replace",
|
||||||
|
)
|
||||||
|
self._reader = threading.Thread(target=self._read, daemon=True)
|
||||||
|
self._reader.start()
|
||||||
|
|
||||||
|
def _read(self):
|
||||||
|
assert self.proc.stdout is not None
|
||||||
|
for line in self.proc.stdout:
|
||||||
|
self.logs.append(line)
|
||||||
|
|
||||||
|
def wait_ready(self):
|
||||||
|
deadline = time.monotonic() + 40
|
||||||
|
last = ""
|
||||||
|
while time.monotonic() < deadline:
|
||||||
|
if self.proc.poll() is not None:
|
||||||
|
raise Failure("server exited early:\n" + "".join(self.logs[-40:]))
|
||||||
|
try:
|
||||||
|
call(self.base, "GET", "/health/ready", 200)
|
||||||
|
return
|
||||||
|
except Failure as exc:
|
||||||
|
last = str(exc)
|
||||||
|
except (urllib.error.URLError, TimeoutError, OSError) as exc:
|
||||||
|
last = type(exc).__name__
|
||||||
|
time.sleep(0.1)
|
||||||
|
raise Failure("server did not become ready: " + last + "\n" + "".join(self.logs[-40:]))
|
||||||
|
|
||||||
|
def stop(self):
|
||||||
|
if self.proc.poll() is None:
|
||||||
|
self.proc.terminate()
|
||||||
|
try:
|
||||||
|
self.proc.wait(timeout=8)
|
||||||
|
except subprocess.TimeoutExpired:
|
||||||
|
self.proc.kill()
|
||||||
|
self.proc.wait(timeout=5)
|
||||||
|
self._reader.join(timeout=2)
|
||||||
|
|
||||||
|
def output(self):
|
||||||
|
return "".join(self.logs)
|
||||||
|
|
||||||
|
|
||||||
|
def host_secret(name):
|
||||||
|
path = ROOT / "secrets" / name
|
||||||
|
if not path.is_file():
|
||||||
|
raise Failure("missing secrets/%s; run scripts/bootstrap.ps1 or scripts/bootstrap.sh" % name)
|
||||||
|
value = path.read_text(encoding="utf-8").strip()
|
||||||
|
if not value:
|
||||||
|
raise Failure("empty secrets/%s" % name)
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def call(
|
||||||
|
base,
|
||||||
|
method,
|
||||||
|
path,
|
||||||
|
expected,
|
||||||
|
body=None,
|
||||||
|
token=None,
|
||||||
|
user_id=None,
|
||||||
|
authorization=None,
|
||||||
|
raw=None,
|
||||||
|
content_type="application/json",
|
||||||
|
timeout=20,
|
||||||
|
):
|
||||||
|
headers = {"Accept": "application/json"}
|
||||||
|
data = None
|
||||||
|
if raw is not None:
|
||||||
|
data = raw
|
||||||
|
headers["Content-Type"] = content_type
|
||||||
|
elif body is not None:
|
||||||
|
data = json.dumps(body, ensure_ascii=False).encode("utf-8")
|
||||||
|
headers["Content-Type"] = content_type
|
||||||
|
if authorization:
|
||||||
|
headers["Authorization"] = authorization
|
||||||
|
elif token:
|
||||||
|
headers["Authorization"] = "Bearer " + token
|
||||||
|
if user_id:
|
||||||
|
headers["X-User-Id"] = user_id
|
||||||
|
request = urllib.request.Request(base + path, data=data, headers=headers, method=method)
|
||||||
|
try:
|
||||||
|
response = urllib.request.urlopen(request, timeout=timeout)
|
||||||
|
except urllib.error.HTTPError as error:
|
||||||
|
response = error
|
||||||
|
except (urllib.error.URLError, TimeoutError, OSError) as exc:
|
||||||
|
raise Failure(method + " " + path.split("?")[0] + " failed: " + type(exc).__name__) from None
|
||||||
|
with response:
|
||||||
|
status = response.status
|
||||||
|
payload = response.read()
|
||||||
|
header_id = response.headers.get("X-Request-Id")
|
||||||
|
content_type_value = response.headers.get_content_type()
|
||||||
|
allowed = [expected] if isinstance(expected, int) else list(expected)
|
||||||
|
check(status in allowed, "%s %s: expected %s, got %s" % (method, path.split("?")[0], allowed, status))
|
||||||
|
check(header_id and header_id.strip() and "," not in header_id, "missing or duplicated X-Request-Id")
|
||||||
|
parsed = None
|
||||||
|
if payload:
|
||||||
|
check(content_type_value == "application/json", "expected application/json")
|
||||||
|
try:
|
||||||
|
parsed = json.loads(payload)
|
||||||
|
except json.JSONDecodeError as exc:
|
||||||
|
raise Failure("response is not JSON") from exc
|
||||||
|
if status >= 400:
|
||||||
|
check(isinstance(parsed, dict) and set(parsed) == {"error"}, "error envelope")
|
||||||
|
error = parsed["error"]
|
||||||
|
check(set(error) == {"code", "message", "request_id"}, "error fields")
|
||||||
|
check(error["request_id"] == header_id, "request_id differs from header")
|
||||||
|
check(error["message"] and "Traceback" not in error["message"], "error message leaked internals")
|
||||||
|
return {"status": status, "json": parsed, "request_id": header_id}
|
||||||
|
|
||||||
|
|
||||||
|
def create_user(base, label):
|
||||||
|
username = ("u_%s_%s" % (label, uuid.uuid4().hex[:10])).lower()
|
||||||
|
password = "T3st-" + uuid.uuid4().hex
|
||||||
|
body = {"username": username, "display_name": "Студент " + label, "password": password}
|
||||||
|
created = call(base, "POST", "/api/v1/users", 201, body=body)["json"]
|
||||||
|
check(created["username"] == username, "username changed")
|
||||||
|
check("password" not in created, "password leaked in user response")
|
||||||
|
raw = base64.b64encode(("%s:%s" % (username, password)).encode("utf-8")).decode("ascii")
|
||||||
|
session = call(base, "POST", "/api/v1/auth/sessions", 201, authorization="Basic " + raw)["json"]
|
||||||
|
check(session["token_type"] == "Bearer", "token type")
|
||||||
|
check(session["user"]["id"] == created["id"], "session user")
|
||||||
|
check("password" not in session["user"], "password leaked in session")
|
||||||
|
created["token"] = session["token"]
|
||||||
|
return created
|
||||||
|
|
||||||
|
|
||||||
|
def replace_cursor(token, fields):
|
||||||
|
padding = "=" * (-len(token) % 4)
|
||||||
|
data = json.loads(base64.urlsafe_b64decode(token + padding))
|
||||||
|
data.update(fields)
|
||||||
|
raw = json.dumps(data, separators=(",", ":"), sort_keys=True).encode("ascii")
|
||||||
|
return base64.urlsafe_b64encode(raw).decode("ascii").rstrip("=")
|
||||||
|
|
||||||
|
|
||||||
|
def test_invalid_json(base):
|
||||||
|
broken = call(base, "POST", "/api/v1/users", 400, raw=b"{")
|
||||||
|
check(broken["json"]["error"]["code"] == "invalid_request", "malformed JSON code")
|
||||||
|
extra = call(
|
||||||
|
base,
|
||||||
|
"POST",
|
||||||
|
"/api/v1/users",
|
||||||
|
400,
|
||||||
|
body={"username": "abc_user", "display_name": "Имя", "extra": True},
|
||||||
|
)
|
||||||
|
check(extra["json"]["error"]["code"] == "invalid_request", "extra field code")
|
||||||
|
bad_utf = call(base, "POST", "/api/v1/users", 400, raw=b"\xff")
|
||||||
|
check(bad_utf["json"]["error"]["code"] == "invalid_request", "invalid utf-8 code")
|
||||||
|
alice = create_user(base, "utf")
|
||||||
|
chat = call(
|
||||||
|
base,
|
||||||
|
"POST",
|
||||||
|
"/api/v1/chats",
|
||||||
|
201,
|
||||||
|
token=alice["token"],
|
||||||
|
body={"title": "UTF", "member_ids": [alice["id"]]},
|
||||||
|
)["json"]
|
||||||
|
bad_message = call(
|
||||||
|
base,
|
||||||
|
"POST",
|
||||||
|
"/api/v1/chats/%s/messages" % chat["id"],
|
||||||
|
400,
|
||||||
|
token=alice["token"],
|
||||||
|
raw=b"\xff",
|
||||||
|
)
|
||||||
|
check(bad_message["json"]["error"]["code"] == "invalid_request", "invalid utf-8 message code")
|
||||||
|
print("PASS: invalid JSON")
|
||||||
|
|
||||||
|
|
||||||
|
def test_duplicate_username(base):
|
||||||
|
first = create_user(base, "dup")
|
||||||
|
again = call(
|
||||||
|
base,
|
||||||
|
"POST",
|
||||||
|
"/api/v1/users",
|
||||||
|
409,
|
||||||
|
body={
|
||||||
|
"username": first["username"],
|
||||||
|
"display_name": "Другой",
|
||||||
|
"password": "T3st-" + uuid.uuid4().hex,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
check(again["json"]["error"]["code"] == "conflict", "duplicate username code")
|
||||||
|
me = call(base, "GET", "/api/v1/users/me", 200, token=first["token"])["json"]
|
||||||
|
check(me["id"] == first["id"], "first user disappeared after duplicate")
|
||||||
|
print("PASS: duplicate username")
|
||||||
|
|
||||||
|
|
||||||
|
def test_empty_and_foreign(base):
|
||||||
|
alice = create_user(base, "alice")
|
||||||
|
bob = create_user(base, "bob")
|
||||||
|
chat = call(
|
||||||
|
base,
|
||||||
|
"POST",
|
||||||
|
"/api/v1/chats",
|
||||||
|
201,
|
||||||
|
token=alice["token"],
|
||||||
|
body={"title": "Только Алиса", "member_ids": [alice["id"]]},
|
||||||
|
)["json"]
|
||||||
|
path = "/api/v1/chats/" + chat["id"]
|
||||||
|
for text in ("", " ", "\n\t"):
|
||||||
|
denied = call(base, "POST", path + "/messages", 400, token=alice["token"], body={"text": text})
|
||||||
|
check(denied["json"]["error"]["code"] == "invalid_request", "empty message code")
|
||||||
|
for method, target, payload in (
|
||||||
|
("GET", path, None),
|
||||||
|
("GET", path + "/messages", None),
|
||||||
|
("POST", path + "/messages", {"text": "чужое"}),
|
||||||
|
):
|
||||||
|
hidden = call(base, method, target, 404, token=bob["token"], body=payload)
|
||||||
|
check(hidden["json"]["error"]["code"] == "not_found", "foreign chat code")
|
||||||
|
check(chat["title"] not in hidden["json"]["error"]["message"], "foreign chat leaked title")
|
||||||
|
listed = call(base, "GET", "/api/v1/chats?limit=100", 200, token=bob["token"])["json"]
|
||||||
|
check(all(item["id"] != chat["id"] for item in listed["items"]), "foreign chat listed")
|
||||||
|
missing = call(base, "GET", "/api/v1/chats/" + str(uuid.uuid4()), 404, token=bob["token"])
|
||||||
|
check(missing["json"]["error"]["code"] == "not_found", "missing chat code")
|
||||||
|
bad_limit = call(base, "GET", path + "/messages?limit=0", 400, token=alice["token"])
|
||||||
|
check(bad_limit["json"]["error"]["code"] == "invalid_request", "limit=0 code")
|
||||||
|
print("PASS: empty message and foreign chat")
|
||||||
|
|
||||||
|
|
||||||
|
def test_pagination(base):
|
||||||
|
alice = create_user(base, "page")
|
||||||
|
other = create_user(base, "other")
|
||||||
|
chat = call(
|
||||||
|
base,
|
||||||
|
"POST",
|
||||||
|
"/api/v1/chats",
|
||||||
|
201,
|
||||||
|
token=alice["token"],
|
||||||
|
body={"title": "Страницы", "member_ids": [alice["id"]]},
|
||||||
|
)["json"]
|
||||||
|
other_chat = call(
|
||||||
|
base,
|
||||||
|
"POST",
|
||||||
|
"/api/v1/chats",
|
||||||
|
201,
|
||||||
|
token=other["token"],
|
||||||
|
body={"title": "Другой", "member_ids": [other["id"]]},
|
||||||
|
)["json"]
|
||||||
|
texts = ["p0", "Привет, мир 👋", "p2", "p3", "p4"]
|
||||||
|
created = []
|
||||||
|
for text in texts:
|
||||||
|
created.append(
|
||||||
|
call(
|
||||||
|
base,
|
||||||
|
"POST",
|
||||||
|
"/api/v1/chats/%s/messages" % chat["id"],
|
||||||
|
201,
|
||||||
|
token=alice["token"],
|
||||||
|
body={"text": text},
|
||||||
|
)["json"]
|
||||||
|
)
|
||||||
|
call(
|
||||||
|
base,
|
||||||
|
"POST",
|
||||||
|
"/api/v1/chats/%s/messages" % other_chat["id"],
|
||||||
|
201,
|
||||||
|
token=other["token"],
|
||||||
|
body={"text": "x"},
|
||||||
|
)
|
||||||
|
call(
|
||||||
|
base,
|
||||||
|
"POST",
|
||||||
|
"/api/v1/chats/%s/messages" % other_chat["id"],
|
||||||
|
201,
|
||||||
|
token=other["token"],
|
||||||
|
body={"text": "y"},
|
||||||
|
)
|
||||||
|
collected = []
|
||||||
|
cursor = None
|
||||||
|
pages = 0
|
||||||
|
while True:
|
||||||
|
target = "/api/v1/chats/%s/messages?limit=2" % chat["id"]
|
||||||
|
if cursor:
|
||||||
|
target += "&cursor=" + quote(cursor, safe="")
|
||||||
|
page = call(base, "GET", target, 200, token=alice["token"])["json"]
|
||||||
|
pages += 1
|
||||||
|
collected.extend(page["items"])
|
||||||
|
cursor = page["next_cursor"]
|
||||||
|
if cursor is None:
|
||||||
|
break
|
||||||
|
check(pages < 10, "pagination did not finish")
|
||||||
|
check(pages == 3, "expected 3 pages, got %d" % pages)
|
||||||
|
check(collected == created, "pagination lost, duplicated or reordered messages")
|
||||||
|
stolen = call(
|
||||||
|
base,
|
||||||
|
"GET",
|
||||||
|
"/api/v1/chats/%s/messages?limit=1" % other_chat["id"],
|
||||||
|
200,
|
||||||
|
token=other["token"],
|
||||||
|
)["json"]
|
||||||
|
check(stolen["next_cursor"], "foreign page should expose a cursor")
|
||||||
|
rejected = call(
|
||||||
|
base,
|
||||||
|
"GET",
|
||||||
|
"/api/v1/chats/%s/messages?limit=2&cursor=%s" % (chat["id"], quote(stolen["next_cursor"], safe="")),
|
||||||
|
400,
|
||||||
|
token=alice["token"],
|
||||||
|
)
|
||||||
|
check(rejected["json"]["error"]["code"] == "invalid_request", "foreign cursor code")
|
||||||
|
first_page = call(
|
||||||
|
base,
|
||||||
|
"GET",
|
||||||
|
"/api/v1/chats/%s/messages?limit=1" % chat["id"],
|
||||||
|
200,
|
||||||
|
token=alice["token"],
|
||||||
|
)["json"]
|
||||||
|
for fields in ({"t": None}, {"t": 123}, {"i": 123}):
|
||||||
|
broken = call(
|
||||||
|
base,
|
||||||
|
"GET",
|
||||||
|
"/api/v1/chats/%s/messages?limit=1&cursor=%s"
|
||||||
|
% (chat["id"], quote(replace_cursor(first_page["next_cursor"], fields), safe="")),
|
||||||
|
400,
|
||||||
|
token=alice["token"],
|
||||||
|
)
|
||||||
|
check(broken["json"]["error"]["code"] == "invalid_request", "tampered cursor code")
|
||||||
|
print("PASS: pagination")
|
||||||
|
|
||||||
|
|
||||||
|
def test_concurrency(base):
|
||||||
|
users = [create_user(base, "c%d" % index) for index in range(CLIENTS)]
|
||||||
|
chat = call(
|
||||||
|
base,
|
||||||
|
"POST",
|
||||||
|
"/api/v1/chats",
|
||||||
|
201,
|
||||||
|
token=users[0]["token"],
|
||||||
|
body={"title": "Гонка", "member_ids": [user["id"] for user in users]},
|
||||||
|
)["json"]
|
||||||
|
barrier = threading.Barrier(CLIENTS)
|
||||||
|
found = []
|
||||||
|
errors = []
|
||||||
|
lock = threading.Lock()
|
||||||
|
|
||||||
|
def worker(user):
|
||||||
|
try:
|
||||||
|
barrier.wait(10)
|
||||||
|
local = []
|
||||||
|
for index in range(PER_CLIENT):
|
||||||
|
message = call(
|
||||||
|
base,
|
||||||
|
"POST",
|
||||||
|
"/api/v1/chats/%s/messages" % chat["id"],
|
||||||
|
201,
|
||||||
|
token=user["token"],
|
||||||
|
body={"text": "m-%s-%d" % (user["id"][:8], index)},
|
||||||
|
)["json"]
|
||||||
|
local.append(message["id"])
|
||||||
|
with lock:
|
||||||
|
found.extend(local)
|
||||||
|
except Exception as exc:
|
||||||
|
with lock:
|
||||||
|
errors.append("%s: %s" % (type(exc).__name__, exc))
|
||||||
|
|
||||||
|
threads = [threading.Thread(target=worker, args=(user,)) for user in users]
|
||||||
|
for thread in threads:
|
||||||
|
thread.start()
|
||||||
|
for thread in threads:
|
||||||
|
thread.join()
|
||||||
|
check(not errors, "concurrent sends failed: " + "; ".join(errors[:5]))
|
||||||
|
check(len(found) == CLIENTS * PER_CLIENT, "submitted count")
|
||||||
|
check(len(set(found)) == CLIENTS * PER_CLIENT, "duplicate message ids")
|
||||||
|
|
||||||
|
listed = []
|
||||||
|
cursor = None
|
||||||
|
while True:
|
||||||
|
target = "/api/v1/chats/%s/messages?limit=40" % chat["id"]
|
||||||
|
if cursor:
|
||||||
|
target += "&cursor=" + quote(cursor, safe="")
|
||||||
|
page = call(base, "GET", target, 200, token=users[0]["token"])["json"]
|
||||||
|
listed.extend(item["id"] for item in page["items"])
|
||||||
|
cursor = page["next_cursor"]
|
||||||
|
if cursor is None:
|
||||||
|
break
|
||||||
|
check(len(listed) == 100 and len(set(listed)) == 100, "history does not contain 100 unique messages")
|
||||||
|
check(set(listed) == set(found), "history ids differ from accepted sends")
|
||||||
|
EVIDENCE.mkdir(exist_ok=True)
|
||||||
|
report = {
|
||||||
|
"date": "2026-10-25",
|
||||||
|
"command": "python student-tests/run.py",
|
||||||
|
"clients": CLIENTS,
|
||||||
|
"messages_per_client": PER_CLIENT,
|
||||||
|
"submitted": len(found),
|
||||||
|
"unique_ids": len(set(found)),
|
||||||
|
"listed": len(listed),
|
||||||
|
"failed_requests": len(errors),
|
||||||
|
"pass": True,
|
||||||
|
}
|
||||||
|
(EVIDENCE / "concurrency.json").write_text(
|
||||||
|
json.dumps(report, ensure_ascii=False, indent=2) + "\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
print("PASS: concurrency 100 unique messages from 10 clients")
|
||||||
|
|
||||||
|
|
||||||
|
def test_logs(server):
|
||||||
|
alice = create_user(server.base, "logs")
|
||||||
|
chat = call(
|
||||||
|
server.base,
|
||||||
|
"POST",
|
||||||
|
"/api/v1/chats",
|
||||||
|
201,
|
||||||
|
token=alice["token"],
|
||||||
|
body={"title": "Логи", "member_ids": [alice["id"]]},
|
||||||
|
)["json"]
|
||||||
|
call(
|
||||||
|
server.base,
|
||||||
|
"POST",
|
||||||
|
"/api/v1/chats/%s/messages" % chat["id"],
|
||||||
|
201,
|
||||||
|
token=alice["token"],
|
||||||
|
body={"text": SECRET},
|
||||||
|
)
|
||||||
|
time.sleep(0.2)
|
||||||
|
output = server.output()
|
||||||
|
check("duration_ms=" in output and "method=POST" in output, "request log line missing")
|
||||||
|
check(SECRET not in output, "message text leaked into logs")
|
||||||
|
check(alice["token"] not in output, "session token leaked into logs")
|
||||||
|
check("lab-only-password" not in output and "postgresql+asyncpg://" not in output, "database url leaked into logs")
|
||||||
|
print("PASS: logs omit message text")
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
port = free_port()
|
||||||
|
server = Server(port)
|
||||||
|
try:
|
||||||
|
server.wait_ready()
|
||||||
|
call(server.base, "GET", "/health/live", 200)
|
||||||
|
call(server.base, "GET", "/api/v1/users/me", 401)
|
||||||
|
header_only = create_user(server.base, "header")
|
||||||
|
rejected = call(server.base, "GET", "/api/v1/users/me", 401, user_id=header_only["id"])
|
||||||
|
check(rejected["json"]["error"]["code"] == "unauthorized", "X-User-Id was accepted")
|
||||||
|
test_invalid_json(server.base)
|
||||||
|
test_duplicate_username(server.base)
|
||||||
|
test_empty_and_foreign(server.base)
|
||||||
|
test_pagination(server.base)
|
||||||
|
test_concurrency(server.base)
|
||||||
|
test_logs(server)
|
||||||
|
finally:
|
||||||
|
server.stop()
|
||||||
|
print("PASS: student tests")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
try:
|
||||||
|
main()
|
||||||
|
except Failure as exc:
|
||||||
|
print("FAIL:", exc)
|
||||||
|
sys.exit(1)
|
||||||
Reference in New Issue
Block a user