ЛР 3 — БВТ2301 — Гамаюнов — на 3 #169

Closed
gamayunov2004 wants to merge 2 commits from gamayunov2004/mtusi2026lab3:solution/bvt2301-gamayunov into main
31 changed files with 1491 additions and 21 deletions
+2
View File
@@ -0,0 +1,2 @@
scripts/restart.sh text eol=lf
scripts/stop-one.sh text eol=lf
+27
View File
@@ -0,0 +1,27 @@
name: Student application
on: [push, pull_request]
permissions:
contents: read
jobs:
application:
runs-on: ubuntu-24.04
timeout-minutes: 15
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- name: Generate local synthetic configuration
shell: pwsh
run: ./scripts/init-env.ps1
- name: Build and start
run: |
docker compose config --quiet
docker compose up -d --build
- name: Verify both replicas and run the public smoke test
run: |
python3 student-tests/grade3.py ready
make test PYTHON=python3
- name: Stop the CI stack
if: always()
run: docker compose down
+12
View File
@@ -0,0 +1,12 @@
FROM python:3.12.10-slim-bookworm
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1 APP_HOST=0.0.0.0 APP_PORT=8080
WORKDIR /app
COPY requirements.txt requirements.lock ./
RUN pip install --no-cache-dir -r requirements.lock && useradd --create-home --uid 10001 appuser
COPY --chown=appuser:appuser src ./src
COPY --chown=appuser:appuser migrations ./migrations
USER appuser
EXPOSE 8080
STOPSIGNAL SIGTERM
HEALTHCHECK --interval=3s --timeout=3s --start-period=10s --retries=5 CMD python -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/health/ready',timeout=2).read()" || exit 1
CMD ["sh", "-c", "exec uvicorn src.main:app --host \"$APP_HOST\" --port \"$APP_PORT\" --no-access-log --timeout-graceful-shutdown 3"]
+60 -21
View File
@@ -1,37 +1,76 @@
# Отчёт — лабораторная 3
- Автор, группа: TODO
- Целевая оценка; выбранный трек 5, если нужен: TODO
- Commit/PR: TODO после публикации
- Использованные источники, библиотеки и помощь ИИ: TODO
- Автор — Гамаюнов Артём, БВТ2301.
- Целевая оценка — 3.
- Исходная ветка — `solution/bvt2301-gamayunov`, основа ЛР3 `0ab3a34`.
- Публикация — PR №169, коммит реализации `be96c39`.
- Источники — контракт ЛР3, собственная ЛР2 `207e380`, документация PostgreSQL, Docker Compose и HAProxy. [Контракт ЛР3](contracts/README.md), [HAProxy 3.0](https://docs.haproxy.org/3.0/configuration.html)
- Codex участвовал в переносе, реализации, запуске тестов, сборе доказательств и подготовке архива.
## Запуск из чистого clone
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
Откройте PowerShell в корне репозитория. Запустите Docker Desktop в режиме Linux containers с WSL2. Проверенный адрес — `http://127.0.0.1:8080`, имя проекта — `mtusi-lab3-grade3`. При занятом порте задайте `$env:LAB3_PORT = '18083'` перед запуском и используйте этот порт в запросах с Windows. Внутренний адрес тестов задаётся Compose как `http://lb:8080`.
## Архитектура и решения
```powershell
powershell.exe -NoProfile -ExecutionPolicy Bypass -File scripts/init-env.ps1
docker compose config --quiet
docker compose up -d --build
docker compose build tools
docker compose run --rm -T tools python student-tests/grade3.py ready
docker compose run --rm -T tools make check validate test
docker compose run --rm -T tools python student-tests/grade3.py core
docker compose run --rm -T tools python student-tests/grade3.py persistence
docker compose run --rm -T tools python student-tests/grade3.py failover
docker compose ps -a
```
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab2, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
При ненулевом коде завершения разберите вывод соответствующей команды. Для `core` и сценариев сохранности используйте учебную конфигурацию с TTL сессии 3600 секунд. Проверки создают синтетические данные. `persistence` вызывает преподавательский `make test-persistence ACTION_SCRIPT=scripts/restart.sh`. `failover` дважды вызывает `make test-failover ACTION_SCRIPT=scripts/stop-one.sh`, проверяет состояние контейнеров и восстанавливает остановленную реплику.
`init-env.ps1` создаёт локальный `.env` из шаблона и генерирует случайный пароль PostgreSQL. Сервисам доступны нужные файлы Compose secrets. Существующий локальный пароль сохраняется при повторной инициализации. `tools` — отдельный профиль для локальных проверок с доступом к Docker socket. Этот доступ даёт права управления Docker Engine. API, HAProxy и PostgreSQL сокет не монтируют. Для проверок на Windows достаточно Docker Desktop и PowerShell. Linux-образ `tools` предоставляет Python, make, POSIX shell и Docker CLI.
Для остановки с сохранением учебной БД выполните `docker compose down`. Справочная команда намеренного удаления учебных данных — `docker compose down --volumes`. Она удаляет volume проекта и уничтожает хранящиеся в нём данные.
## Архитектура и перенос
```text
Windows client / test runner -> HAProxy (:8080) -> api1, api2 -> PostgreSQL
migrate -> schema
```
- Из ЛР2 перенесены модели, HTTP-обработчики, Argon2id, логика сессий, пагинация, слой PostgreSQL, исходная миграция `001_initial.sql`, Dockerfile и lock-файлы. Эффективным источником профиля пользователя, истории и сессий служит PostgreSQL. Valkey и исследования ЛР2 для этого стенда не требуются.
- Сервис `migrate` стартует после готовности PostgreSQL и выполняет версионированные SQL-миграции. API запускаются после кода 0 у `migrate`. Advisory lock защищает от одновременных миграционных запусков, а `schema_migrations` — от повторного применения версии. В процессах API SQL-миграций нет.
- HAProxy использует round robin и обновляет адреса через встроенный DNS Docker. Проверка `/health/ready` идёт примерно раз в секунду, при переходе — раз в 500 мс. Два неуспеха исключают backend, один успех возвращает. Readiness проверяет запрос к PostgreSQL, liveness проверяет процесс. Повтор HAProxy ограничен ошибкой установления backend-соединения. Uvicorn получает SIGTERM и даёт активным задачам до 3 секунд на завершение.
- Сессии хранятся по SHA-256 непрозрачного случайного токена с энтропией 256 бит. Пароли защищены Argon2id. Общая БД проверяет срок и факт отзыва при каждом защищённом запросе. Basic используется только для создания сессии.
- Ключ POST сообщения имеет область `(user_id, method, chat_id, key)`. UUID чата приводится к канонической форме. Перед резервированием ключа проверяются Bearer и право записи. PostgreSQL сериализует одинаковый ключ уникальным индексом. Победитель создаёт запись ключа с отложенным внешним ключом и вставляет сообщение в одной транзакции. При конфликте активного ключа новый SQL-запрос читает уже зафиксированное сообщение. В ЛР3 валидное тело содержит только `text`, поэтому сравнивается декодированная строка, независимо от пробелов и экранирования в JSON. Другой текст даёт 409, тот же — исходные 201 и Message. Дополнительные поля запрещены схемой ЛР3.
- После 24 часов ключ можно занять заново. Ошибка до commit откатывает запись ключа вместе с сообщением. Когда клиент не увидел подтверждение, он повторяет запрос с исходным ключом и телом. Формируемые API ответы, включая health и ошибки, получают `X-Instance-Id` и `X-Request-Id` в middleware.
- `.env.example` сохранён. `SESSION_TTL_SECONDS` и `SESSION_STORE` используются Compose, значения `APP_HOST`, `APP_PORT` и `DATABASE_URL_FILE` заданы внутри контейнеров. Две реплики и TTL ключа 86400 с закреплены топологией и SQL. `APP_ENV` и `CACHE_URL_FILE` — подсказки шаблона. Порт хоста задаётся через `LAB3_PORT`.
## Доказательства критериев
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
| --- | --- | --- | --- |
| Публичный smoke | make test | TODO | TODO |
| Остальные пункты уровня 3 | TODO | TODO | TODO |
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
| Критерий | Команда или сценарий | Результат |
| --- | --- | --- |
| A. Стек | `docker compose config --quiet`, сборка, запуск, `docker compose ps -a` | Две API-реплики и PostgreSQL готовы, отдельная миграция завершилась с 0. Только HAProxy публикует порт. [Окружение](evidence/grade3/environment.txt) |
| B. Балансировка | `grade3.py core` | Один Bearer принят на `api1` и `api2`. Smoke проверил служебные заголовки. [Core](evidence/grade3/core.txt) |
| C. Идемпотентность | `grade3.py core` | Одновременные повторы на обеих репликах вернули 201 и один Message. Разные тела дали 201/409. Проверены область ключа, expiry, нечитанный ответ и откат при 503. [Core](evidence/grade3/core.txt) |
| D. Failover | `grade3.py failover` | Поочерёдно остановлены `api1` и `api2`, пережившая отказ реплика обслужила чтение, запись и повтор без дубля. [Failover](evidence/grade3/failover.txt) |
| E. Проверки | `make check validate test`, обе make-цели с hooks | Все команды завершились с кодом 0. В Linux-образе оба hook имеют режим 0755. [Публичные проверки](evidence/grade3/checks.txt), [persistence](evidence/grade3/persistence.txt), [failover](evidence/grade3/failover.txt) |
| F. Минимум ЛР2 | Smoke, `grade3.py core` и `grade3.py persistence` | Подтверждены регистрация, Basic/Bearer, Argon2id, TTL и отзыв сессии, доступ, пагинация и сохранность данных со старым токеном после рестарта API и PostgreSQL. [Core](evidence/grade3/core.txt), [persistence](evidence/grade3/persistence.txt) |
## Эксперименты и отказы
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
Проверки выполнены на Windows с Docker Desktop 4.99.0, Engine 29.9.3 и Compose 5.7.0. Приложение собрано на Python 3.12.10, PostgreSQL 16.8, HAProxy 3.0.11. Образы и Python-зависимости закреплены. Версии инструментов и ресурсы Docker VM приведены в [выводе окружения](evidence/grade3/environment.txt). Публичный тест пишет через LB около 15 секунд на каждую остановку. Отсчёт записи ведётся от запуска hook, наблюдение чтения — от строки публичного теста непосредственно перед запуском hook. ID и время запуска процессов фиксируются через Docker inspect; история проверяется поверх HTTP.
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
| --- | --- | --- | --- |
| TODO | TODO | TODO | TODO |
| Остановлена | Чтение, с | Первая запись после hook, с | Успешные записи / ошибки |
| --- | ---: | ---: | ---: |
| api1 | 0,892 | 1,893 | 118 / 0 |
| api2 | 1,002 | 1,889 | 116 / 0 |
## Ограничения и следующие шаги
Обе реплики вернулись в балансировку с прежней сессией. На время остановки пережившая отказ реплика повторила уже зафиксированное сообщение без дубля. Перезапуск PostgreSQL и обеих API подтверждён сменой StartedAt у тех же контейнеров. У сессии сохранилось время истечения.
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
В опубликованном коммите файлы `scripts/restart.sh` и `scripts/stop-one.sh` имеют режим Git `100755`. ZIP хранит режим 0755, а Linux-образ устанавливает его при сборке.
## Ограничения
Граница отказоустойчивости — одна API-реплика. PostgreSQL, HAProxy и Docker host остаются единичными компонентами; перезапуск PostgreSQL кратко прерывает доступность. История использует keyset по `(created_at, id)`. При вставках между страницами сообщение с ключом сортировки до уже выданного cursor может стать видимым только при новом обходе. Истёкшие записи ключей переиспользуются при новом запросе с тем же ключом; фоновой сборки этих строк нет. Стенд работает по HTTP на loopback с синтетическими данными.
Приложен отдельный smoke workflow для CI. Удалённый запуск workflow не выполнялся. Доказательства относятся к локальному Docker Desktop.
+79
View File
@@ -0,0 +1,79 @@
name: mtusi-lab3-grade3
x-runtime: &runtime
image: mtusi-lab3-api:grade3
pull_policy: never
environment: &api-env
APP_HOST: 0.0.0.0
APP_PORT: 8080
SESSION_TTL_SECONDS: ${SESSION_TTL_SECONDS:-3600}
SESSION_STORE: ${SESSION_STORE:-postgres}
DATABASE_URL_FILE: /run/secrets/database_url
secrets: [database_url]
x-api: &api
<<: *runtime
depends_on:
migrate: {condition: service_completed_successfully}
postgres: {condition: service_healthy}
restart: unless-stopped
stop_grace_period: 6s
services:
postgres:
image: postgres:16.8-bookworm
environment:
POSTGRES_USER: messenger
POSTGRES_DB: messenger
POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password
secrets: [postgres_password]
volumes: [postgres_data:/var/lib/postgresql/data]
healthcheck:
test: [CMD-SHELL, "pg_isready -h 127.0.0.1 -U messenger -d messenger"]
interval: 2s
timeout: 2s
retries: 20
restart: unless-stopped
migrate:
<<: *runtime
build: .
command: ["python", "-m", "src.db"]
healthcheck: {disable: true}
depends_on:
postgres: {condition: service_healthy}
api1:
<<: *api
environment:
<<: *api-env
INSTANCE_ID: api1
api2:
<<: *api
environment:
<<: *api-env
INSTANCE_ID: api2
lb:
image: haproxy:3.0.11-alpine
ports: ["127.0.0.1:${LAB3_PORT:-8080}:8080"]
volumes:
- ./infra/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro
restart: unless-stopped
tools:
profiles: [test]
image: mtusi-lab3-tools:grade3
build: {context: ., dockerfile: infra/Dockerfile.tools}
volumes:
- /var/run/docker.sock:/var/run/docker.sock
environment:
BASE_URL: http://lb:8080
LAB3_PROJECT: ${COMPOSE_PROJECT_NAME}
secrets:
postgres_password: {file: ./secrets/postgres_password}
database_url: {file: ./secrets/database_url}
volumes:
postgres_data:
+10
View File
@@ -0,0 +1,10 @@
PS> docker compose run --rm -T tools make check validate test
Container mtusi-lab3-grade3-tools-run-8d9dad6c38c7 Creating
Container mtusi-lab3-grade3-tools-run-8d9dad6c38c7 Created
python3 scripts/check_contract.py
PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.
python3 scripts/validate_openapi.py
PASS: full OpenAPI 3.1 and JSON Schema validation; application NOT tested.
python3 tests/smoke.py --base-url "http://lb:8080"
PASS: lab3 public API scenario; infrastructure and grading evidence remain separate.
exit=0
+11
View File
@@ -0,0 +1,11 @@
PS> docker compose run --rm -T tools python student-tests/grade3.py core
Container mtusi-lab3-grade3-tools-run-309ba1f80a2b Creating
Container mtusi-lab3-grade3-tools-run-309ba1f80a2b Created
PASS: one Bearer accepted by api1 and api2
{"same_key_race": [{"instance": "api2", "status": 201, "message_id": "0f879955-e2e3-41da-a213-66a6105d1048"}, {"instance": "api1", "status": 201, "message_id": "0f879955-e2e3-41da-a213-66a6105d1048"}], "history_count": 1}
{"different_body_race": {"api1": 201, "api2": 409}}
PASS: independent keys/users/chats, invalid key/body, authorization and revocation on replay
PASS: Argon2id, session TTL and expiry, 24h key TTL and concurrent key reuse after expiry
PASS: retry of committed POST with unread response returned ae7b9594-56af-4e3b-a31e-3f400c28c36c
PASS: message insert lock timeout produced 503 and rolled back reservation, same key then committed once
exit=0
+64
View File
@@ -0,0 +1,64 @@
Windows build 10.0.19045.0
PS> docker version
Client:
Version: 29.9.3
API version: 1.56
Go version: go1.26.10
Git commit: 5457199
Built: Tue Nov 3 21:46:10 2026
OS/Arch: windows/amd64
Context: desktop-linux
Server: Docker Desktop 4.99.0 (245822)
Engine:
Version: 29.9.3
API version: 1.56 (minimum version 1.40)
Go version: go1.26.10
Git commit: 70efc0e
Built: Tue Nov 3 21:41:42 2026
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: v2.4.2
GitCommit: b7a25261e029d68978bbf290df0a62fc819b3c51
runc:
Version: 1.5.2
GitCommit: v1.5.2-0-g29dd3dc2
docker-init:
Version: 0.19.0
GitCommit: de40ad0
PS> docker compose version
Docker Compose version v5.7.0
PS> docker info (Docker Desktop Linux VM)
linux x86_64 CPUs=4 MemoryBytes=8242380800
PS> docker compose --profile test config --quiet
PS> docker compose ps -a
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
mtusi-lab3-grade3-api1-1 mtusi-lab3-api:grade3 "sh -c 'exec uvicorn…" api1 7 minutes ago Up About a minute (healthy) 8080/tcp
mtusi-lab3-grade3-api2-1 mtusi-lab3-api:grade3 "sh -c 'exec uvicorn…" api2 7 minutes ago Up 51 seconds (healthy) 8080/tcp
mtusi-lab3-grade3-lb-1 haproxy:3.0.11-alpine "docker-entrypoint.s…" lb 9 minutes ago Up 9 minutes 127.0.0.1:8080->8080/tcp
mtusi-lab3-grade3-migrate-1 mtusi-lab3-api:grade3 "python -m src.db" migrate 7 minutes ago Exited (0) 3 minutes ago
mtusi-lab3-grade3-postgres-1 postgres:16.8-bookworm "docker-entrypoint.s…" postgres 9 minutes ago Up About a minute (healthy) 5432/tcp
PS> docker compose logs --no-color migrate
migrate-1 | PASS: versioned migrations applied (already applied versions skipped)
migrate-1 | PASS: versioned migrations applied (already applied versions skipped)
migrate-1 | PASS: versioned migrations applied (already applied versions skipped)
migrate-1 | PASS: versioned migrations applied (already applied versions skipped)
migrate-1 | PASS: versioned migrations applied (already applied versions skipped)
PS> Applied migration versions
001_initial.sql
002_message_idempotency.sql
PS> API runtime
Python 3.12.10
PS> HAProxy configuration check
PS> Tool versions and both executable hooks
Container mtusi-lab3-grade3-tools-run-e0c2ceb3000d Creating
Container mtusi-lab3-grade3-tools-run-e0c2ceb3000d Created
Python 3.12.15
GNU Make 4.4.1
Docker Compose version v2.33.0
755
755
PS> Windows loopback readiness
PS> Windows loopback public smoke
PASS: lab3 public API scenario; infrastructure and grading evidence remain separate.
+22
View File
@@ -0,0 +1,22 @@
PS> docker compose run --rm -T tools python student-tests/grade3.py failover
Container mtusi-lab3-grade3-tools-run-e2ebc343dc86 Creating
Container mtusi-lab3-grade3-tools-run-e2ebc343dc86 Created
python3 tests/resilience.py --mode failover --action-script "scripts/stop-one.sh" --base-url "http://lb:8080"
Executing explicit failover hook; hook output suppressed to avoid leaking secrets.
{"observed_instances_before": 2, "successful_writes": 118, "failed_attempts": 0, "first_recovery_seconds": 1.893}
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
PASS: api2 read confirmed history after api1 stopped in 0.892s
{"target": "api1", "before": {"api1": "9b898d6720dca8eb1cfb84fde08090f64b808a495578c0185c77bd63892fdf1f running 2026-11-20T11:44:11.521174934Z", "api2": "ab7c05600399d2c7dc9c5beb0dc661eb222032d32a9758f9cee945b4ae196a02 running 2026-11-20T11:44:11.248047203Z"}, "after": {"api1": "9b898d6720dca8eb1cfb84fde08090f64b808a495578c0185c77bd63892fdf1f exited 2026-11-20T11:44:11.521174934Z", "api2": "ab7c05600399d2c7dc9c5beb0dc661eb222032d32a9758f9cee945b4ae196a02 running 2026-11-20T11:44:11.248047203Z"}}
PASS: api2 replayed e580ba9a-ae15-4763-89a5-4985eaec6d0c while api1 was stopped, history count 1
9b898d6720dca8eb1cfb84fde08090f64b808a495578c0185c77bd63892fdf1f
PASS: api1 rejoined the LB with the same Bearer; migrator was not restarted
python3 tests/resilience.py --mode failover --action-script "scripts/stop-one.sh" --base-url "http://lb:8080"
Executing explicit failover hook; hook output suppressed to avoid leaking secrets.
{"observed_instances_before": 2, "successful_writes": 116, "failed_attempts": 0, "first_recovery_seconds": 1.889}
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
PASS: api1 read confirmed history after api2 stopped in 1.002s
{"target": "api2", "before": {"api1": "9b898d6720dca8eb1cfb84fde08090f64b808a495578c0185c77bd63892fdf1f running 2026-11-20T11:44:35.925512672Z", "api2": "ab7c05600399d2c7dc9c5beb0dc661eb222032d32a9758f9cee945b4ae196a02 running 2026-11-20T11:44:11.248047203Z"}, "after": {"api1": "9b898d6720dca8eb1cfb84fde08090f64b808a495578c0185c77bd63892fdf1f running 2026-11-20T11:44:35.925512672Z", "api2": "ab7c05600399d2c7dc9c5beb0dc661eb222032d32a9758f9cee945b4ae196a02 exited 2026-11-20T11:44:11.248047203Z"}}
PASS: api1 replayed 8ee32cdc-1ad9-46f6-8f0c-fa5c3854353c while api2 was stopped, history count 1
ab7c05600399d2c7dc9c5beb0dc661eb222032d32a9758f9cee945b4ae196a02
PASS: api2 rejoined the LB with the same Bearer; migrator was not restarted
exit=0
+72
View File
@@ -0,0 +1,72 @@
Baseline lab3: 0ab3a34, solution/bvt2301-gamayunov, initially clean.
Read-only source lab2: 207e380, solution/bvt2301-gamayunov, initially and finally clean.
M = modified; A = added to working tree. Index remained empty.
M REPORT.md
A .gitattributes
A .github/workflows/application.yml
A Dockerfile
A PR_GRADE3_DRAFT.md
A compose.yaml
A evidence/grade3/checks.txt
A evidence/grade3/core.txt
A evidence/grade3/environment.txt
A evidence/grade3/failover.txt
A evidence/grade3/files.txt
A evidence/grade3/persistence.txt
A infra/Dockerfile.tools
A infra/haproxy.cfg
A infra/requirements-tools.lock
A migrations/001_initial.sql
A migrations/002_message_idempotency.sql
A requirements.lock
A requirements.txt
A scripts/init-env.ps1
A scripts/restart.sh
A scripts/stop-one.sh
A src/__init__.py
A src/config.py
A src/db.py
A src/errors.py
A src/main.py
A src/models.py
A src/pagination.py
A src/security.py
A src/store.py
A student-tests/grade3.py
Future staging allowlist (on publication):
.gitattributes
.github/workflows/application.yml
Dockerfile
REPORT.md
compose.yaml
evidence/grade3/checks.txt
evidence/grade3/core.txt
evidence/grade3/environment.txt
evidence/grade3/failover.txt
evidence/grade3/files.txt
evidence/grade3/persistence.txt
infra/Dockerfile.tools
infra/haproxy.cfg
infra/requirements-tools.lock
migrations/001_initial.sql
migrations/002_message_idempotency.sql
requirements.lock
requirements.txt
scripts/init-env.ps1
scripts/restart.sh
scripts/stop-one.sh
src/__init__.py
src/config.py
src/db.py
src/errors.py
src/main.py
src/models.py
src/pagination.py
src/security.py
src/store.py
student-tests/grade3.py
The two action hooks must receive file mode 100755 during future staging.
PR_GRADE3_DRAFT.md is local review material included in the ZIP.
+11
View File
@@ -0,0 +1,11 @@
PS> docker compose run --rm -T tools python student-tests/grade3.py persistence
Container mtusi-lab3-grade3-tools-run-6baa53019397 Creating
Container mtusi-lab3-grade3-tools-run-6baa53019397 Created
python3 tests/resilience.py --mode persistence --action-script "scripts/restart.sh" --base-url "http://lb:8080"
Executing explicit persistence hook; hook output suppressed to avoid leaking secrets.
PASS: persistence preserved confirmed data and existing session. Hook scope needs operator evidence.
postgres before 5d254f0d75a9a9f9467e73ff3618215cf6535830043b347fb402ca84b8a63094 running 2026-11-20T11:39:51.532308399Z after 5d254f0d75a9a9f9467e73ff3618215cf6535830043b347fb402ca84b8a63094 running 2026-11-20T11:44:07.540264913Z
api1 before 9b898d6720dca8eb1cfb84fde08090f64b808a495578c0185c77bd63892fdf1f running 2026-11-20T11:41:43.244757785Z after 9b898d6720dca8eb1cfb84fde08090f64b808a495578c0185c77bd63892fdf1f running 2026-11-20T11:44:11.521174934Z
api2 before ab7c05600399d2c7dc9c5beb0dc661eb222032d32a9758f9cee945b4ae196a02 running 2026-11-20T11:42:05.596639197Z after ab7c05600399d2c7dc9c5beb0dc661eb222032d32a9758f9cee945b4ae196a02 running 2026-11-20T11:44:11.248047203Z
PASS: old Bearer on both replicas after PostgreSQL restart, unchanged expiry, replay 443b9676-28b5-43f3-a2f7-2df69ab3cf76, history count 1
exit=0
+14
View File
@@ -0,0 +1,14 @@
FROM docker:27.5.1-cli
RUN apk add --no-cache python3 py3-pip make && python3 -m venv /opt/venv
ENV PATH=/opt/venv/bin:$PATH PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1
WORKDIR /workspace
COPY infra/requirements-tools.lock /tmp/requirements-tools.lock
RUN pip install --no-cache-dir -r /tmp/requirements-tools.lock
COPY Makefile lab.json compose.yaml README.md COURSE.md CONTRIBUTING.md REPORT.md ./
COPY contracts/ ./contracts/
COPY tests/ ./tests/
COPY scripts/check_contract.py scripts/validate_openapi.py ./scripts/
COPY --chmod=0755 scripts/restart.sh scripts/stop-one.sh ./scripts/
COPY student-tests/ ./student-tests/
ENTRYPOINT []
CMD ["make", "check"]
+29
View File
@@ -0,0 +1,29 @@
defaults
mode http
timeout http-request 5s
timeout connect 500ms
timeout client 15s
timeout server 10s
timeout check 1s
retries 1
retry-on conn-failure
option redispatch
resolvers docker
nameserver embedded 127.0.0.11:53
timeout resolve 500ms
timeout retry 500ms
hold valid 1s
hold nx 1s
frontend public_http
bind :8080
default_backend messenger
backend messenger
balance roundrobin
option httpchk GET /health/ready
http-check expect status 200
default-server check inter 1s fastinter 500ms downinter 1s fall 2 rise 1 resolvers docker init-addr none resolve-prefer ipv4
server api1 api1:8080
server api2 api2:8080
+19
View File
@@ -0,0 +1,19 @@
attrs==26.1.0
certifi==2026.10.23
charset-normalizer==3.5.2
idna==3.20
jsonschema==4.26.0
jsonschema-path==0.3.4
jsonschema-specifications==2025.9.1
lazy-object-proxy==1.13.0
openapi-schema-validator==0.6.3
openapi-spec-validator==0.7.2
pathable==0.4.4
PyYAML==6.0.3
referencing==0.36.2
requests==2.34.2
rfc3339-validator==0.1.4
rpds-py==2026.9.1
six==1.17.0
typing_extensions==4.16.0
urllib3==2.8.0
+36
View File
@@ -0,0 +1,36 @@
CREATE TABLE users (
id uuid PRIMARY KEY,
username text NOT NULL UNIQUE CHECK (username ~ '^[a-z0-9_]{3,32}$'),
display_name text NOT NULL CHECK (char_length(display_name) BETWEEN 1 AND 100 AND btrim(display_name) <> ''),
password_hash text NOT NULL,
created_at timestamptz NOT NULL DEFAULT clock_timestamp()
);
CREATE TABLE chats (
id uuid PRIMARY KEY,
title text NOT NULL CHECK (char_length(title) BETWEEN 1 AND 100 AND btrim(title) <> ''),
created_at timestamptz NOT NULL DEFAULT clock_timestamp()
);
CREATE TABLE chat_members (
chat_id uuid NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
user_id uuid NOT NULL REFERENCES users(id),
PRIMARY KEY (chat_id, user_id)
);
CREATE INDEX chat_members_user ON chat_members(user_id, chat_id);
CREATE INDEX chats_order ON chats(created_at, id);
CREATE TABLE messages (
id uuid PRIMARY KEY,
chat_id uuid NOT NULL REFERENCES chats(id),
sender_id uuid NOT NULL,
text text NOT NULL CHECK (char_length(text) BETWEEN 1 AND 4000 AND btrim(text) <> ''),
created_at timestamptz NOT NULL DEFAULT clock_timestamp(),
FOREIGN KEY (chat_id, sender_id) REFERENCES chat_members(chat_id, user_id)
);
CREATE INDEX messages_history ON messages(chat_id, created_at, id);
CREATE TABLE sessions (
token_hash bytea PRIMARY KEY CHECK (octet_length(token_hash) = 32),
user_id uuid NOT NULL REFERENCES users(id),
expires_at timestamptz NOT NULL,
created_at timestamptz NOT NULL DEFAULT clock_timestamp(),
CHECK (expires_at > created_at)
);
CREATE INDEX sessions_expiry ON sessions(expires_at);
+9
View File
@@ -0,0 +1,9 @@
CREATE TABLE message_idempotency (
user_id uuid NOT NULL REFERENCES users(id),
method text NOT NULL,
chat_id uuid NOT NULL REFERENCES chats(id),
key text NOT NULL CHECK (key COLLATE "C" ~ '^[A-Za-z0-9._:-]{1,128}$'),
message_id uuid NOT NULL REFERENCES messages(id) DEFERRABLE INITIALLY DEFERRED,
expires_at timestamptz NOT NULL,
PRIMARY KEY (user_id, method, chat_id, key)
);
+19
View File
@@ -0,0 +1,19 @@
annotated-types==0.8.0
anyio==4.15.1
argon2-cffi==23.1.0
argon2-cffi-bindings==26.1.0
cffi==2.1.1
click==8.5.0
fastapi==0.115.12
h11==0.16.0
idna==3.20
psycopg==3.2.6
psycopg-binary==3.2.6
psycopg-pool==3.2.6
pycparser==3.12
pydantic==2.11.3
pydantic_core==2.33.1
starlette==0.46.2
typing-inspection==0.4.4
typing_extensions==4.16.0
uvicorn==0.34.2
+7
View File
@@ -0,0 +1,7 @@
fastapi==0.115.12
starlette==0.46.2
pydantic==2.11.3
uvicorn==0.34.2
psycopg[binary]==3.2.6
psycopg-pool==3.2.6
argon2-cffi==23.1.0
+27
View File
@@ -0,0 +1,27 @@
$ErrorActionPreference = 'Stop'
$root = Split-Path -Parent $PSScriptRoot
$utf8 = [Text.UTF8Encoding]::new($false)
$secretDir = Join-Path $root 'secrets'
[void][IO.Directory]::CreateDirectory($secretDir)
$passwordFile = Join-Path $secretDir 'postgres_password'
if (-not (Test-Path -LiteralPath $passwordFile)) {
$bytes = New-Object byte[] 32
$rng = [Security.Cryptography.RandomNumberGenerator]::Create()
try { $rng.GetBytes($bytes) } finally { $rng.Dispose() }
$password = [BitConverter]::ToString($bytes).Replace('-', '').ToLowerInvariant()
[IO.File]::WriteAllText($passwordFile, $password, $utf8)
}
$password = [IO.File]::ReadAllText($passwordFile).Trim()
if ($password -cnotmatch '^[a-f0-9]{64}$') {
throw 'Expected the locally generated PostgreSQL password in secrets/postgres_password.'
}
[IO.File]::WriteAllText(
(Join-Path $secretDir 'database_url'),
"postgresql://messenger:${password}@postgres:5432/messenger",
$utf8
)
$envFile = Join-Path $root '.env'
if (-not (Test-Path -LiteralPath $envFile)) {
Copy-Item -LiteralPath (Join-Path $root '.env.example') -Destination $envFile
}
Write-Output 'Local configuration ready.'
+8
View File
@@ -0,0 +1,8 @@
#!/bin/sh
set -eu
cd "$(dirname "$0")/.."
if [ -n "${LAB3_PROJECT:-}" ]; then
set -- --project-name "$LAB3_PROJECT"
fi
docker compose "$@" restart --no-deps --timeout 4 postgres
docker compose "$@" restart --no-deps --timeout 4 api1 api2
+11
View File
@@ -0,0 +1,11 @@
#!/bin/sh
set -eu
case "${TARGET_INSTANCE_ID:-}" in
api1|api2) ;;
*) echo 'TARGET_INSTANCE_ID must be api1 or api2' >&2; exit 2 ;;
esac
cd "$(dirname "$0")/.."
if [ -n "${LAB3_PROJECT:-}" ]; then
set -- --project-name "$LAB3_PROJECT"
fi
docker compose "$@" stop --timeout 4 "$TARGET_INSTANCE_ID"
+1
View File
@@ -0,0 +1 @@
+16
View File
@@ -0,0 +1,16 @@
import os
from pathlib import Path
def secret(name):
path = os.environ.get(name + "_FILE")
if path:
return Path(path).read_text().strip()
raise RuntimeError(f"{name}_FILE is required")
SESSION_TTL = int(os.getenv("SESSION_TTL_SECONDS", "3600"))
if not 1 <= SESSION_TTL <= 604800:
raise RuntimeError("SESSION_TTL_SECONDS must be 1..604800")
if os.getenv("SESSION_STORE", "postgres") != "postgres":
raise RuntimeError("This implementation uses PostgreSQL sessions")
+40
View File
@@ -0,0 +1,40 @@
from contextlib import contextmanager
from pathlib import Path
import psycopg
from psycopg.rows import dict_row
from psycopg_pool import ConnectionPool
from src.config import secret
pool = ConnectionPool(
conninfo=secret("DATABASE_URL"), min_size=1, max_size=8,
timeout=2, max_waiting=32, reconnect_timeout=5, open=False,
kwargs={"connect_timeout": 2, "row_factory": dict_row,
"options": "-c statement_timeout=2000 -c lock_timeout=1500 -c idle_in_transaction_session_timeout=5000 -c timezone=UTC"},
)
def migrate():
# Serialize migration jobs if two operators start the stack concurrently.
with psycopg.connect(secret("DATABASE_URL"), connect_timeout=3) as conn:
conn.execute("SET LOCAL lock_timeout = '10s'")
conn.execute("SET LOCAL statement_timeout = '15s'")
conn.execute("SELECT pg_advisory_xact_lock(202602)")
conn.execute("CREATE TABLE IF NOT EXISTS schema_migrations (version text PRIMARY KEY, applied_at timestamptz NOT NULL DEFAULT now())")
for path in sorted((Path(__file__).resolve().parents[1] / "migrations").glob("*.sql")):
if not conn.execute("SELECT 1 FROM schema_migrations WHERE version=%s", (path.name,)).fetchone():
conn.execute(path.read_text(encoding="utf-8"))
conn.execute("INSERT INTO schema_migrations(version) VALUES (%s)", (path.name,))
@contextmanager
def connection():
with pool.connection() as conn:
# psycopg context commits on success, rolls back on EVERY exception.
yield conn
if __name__ == "__main__":
migrate()
print("PASS: versioned migrations applied (already applied versions skipped)")
+34
View File
@@ -0,0 +1,34 @@
from fastapi import Request
from fastapi.responses import JSONResponse
class ApiError(Exception):
def __init__(
self,
status_code: int,
code: str,
message: str,
):
self.status_code = status_code
self.code = code
self.message = message
def error_response(
request: Request,
status_code: int,
code: str,
message: str,
) -> JSONResponse:
request_id = request.state.request_id
return JSONResponse(
status_code=status_code,
content={
"error": {
"code": code,
"message": message,
"request_id": request_id,
}
},
)
+170
View File
@@ -0,0 +1,170 @@
import hashlib
import logging
import os
import re
import secrets
from contextlib import asynccontextmanager
from uuid import UUID, uuid4
import anyio
import psycopg
from fastapi import Depends, FastAPI, Query, Request, Response
from fastapi.exceptions import RequestValidationError
from psycopg_pool import PoolTimeout, TooManyRequests
from starlette.exceptions import HTTPException
from src import store
from src.config import SESSION_TTL
from src.db import connection, pool
from src.errors import ApiError, error_response
from src.models import Chat, ChatPage, CreateChat, CreateMessage, CreateUser, Message, MessagePage, User
from src.security import basic_credentials, hasher, token_hash, unauthorized, verify
INSTANCE_ID = os.getenv("INSTANCE_ID") or uuid4().hex
IDEMPOTENCY_KEY = re.compile(r"[A-Za-z0-9._:-]{1,128}")
@asynccontextmanager
async def lifespan(app):
anyio.to_thread.current_default_thread_limiter().total_tokens = 16
pool.open()
try:
pool.wait(timeout=10)
yield
finally:
pool.close()
app = FastAPI(title="MTUSI Messenger", version="1.3.0", lifespan=lifespan,
docs_url=None, redoc_url=None, openapi_url=None)
def failure(request, status, code, message):
response = error_response(request, status, code, message)
if status == 401:
basic = request.url.path == "/api/v1/auth/sessions"
response.headers['WWW-Authenticate'] = ('Basic realm="messenger", charset="UTF-8"' if basic else 'Bearer realm="messenger"')
return response
@app.middleware("http")
async def request_context(request, call_next):
request.state.request_id = str(uuid4())
try:
if request.method == "POST" and request.url.path == "/api/v1/auth/sessions" and await request.body():
raise ApiError(400, "invalid_request", "Вход не принимает тело запроса")
response = await call_next(request)
except ApiError as exc:
response = failure(request, exc.status_code, exc.code, exc.message)
except (psycopg.Error, PoolTimeout, TooManyRequests):
response = failure(request, 503, "unavailable", "Хранилище временно недоступно")
except Exception as exc:
logging.error("Unhandled request error: %s", type(exc).__name__)
response = failure(request, 500, "internal_error", "Внутренняя ошибка сервера")
response.headers['X-Request-Id'] = request.state.request_id
response.headers['X-Instance-Id'] = INSTANCE_ID
return response
@app.exception_handler(ApiError)
async def api_error(request, exc):
return failure(request, exc.status_code, exc.code, exc.message)
@app.exception_handler(RequestValidationError)
async def validation_error(request, exc):
return failure(request, 400, "invalid_request", "Некорректный запрос")
@app.exception_handler(HTTPException)
async def http_error(request, exc):
return failure(request, exc.status_code, "not_found" if exc.status_code == 404 else "invalid_request", "Недопустимый HTTP-запрос")
def require_user(request: Request):
digest = token_hash(request.headers.get('Authorization'))
with connection() as conn:
user = conn.execute(
"""SELECT u.id,u.username,u.display_name,u.created_at
FROM sessions s JOIN users u ON u.id=s.user_id
WHERE s.token_hash=%s AND s.expires_at > clock_timestamp()""",
(digest,),
).fetchone()
if user is None:
unauthorized()
request.state.token_hash = digest
return User(**store.public(user))
@app.get('/health/live')
def live():
return {"status": "ok"}
@app.get('/health/ready')
def ready():
with connection() as conn:
conn.execute("SELECT 1").fetchone()
return {"status": "ok"}
@app.post('/api/v1/users', status_code=201, response_model=User)
def register(data: CreateUser):
return store.create_user(data, hasher.hash(data.password))
@app.post('/api/v1/auth/sessions', status_code=201)
def login(request: Request):
username, password = basic_credentials(request.headers.get('Authorization'))
with connection() as conn:
row = conn.execute("SELECT id,username,display_name,created_at,password_hash FROM users WHERE username=%s", (username,)).fetchone()
valid = verify(row['password_hash'] if row else None, password)
if not row or not valid:
unauthorized()
token = secrets.token_urlsafe(32)
digest = hashlib.sha256(token.encode('ascii')).digest()
with connection() as conn:
result = conn.execute("INSERT INTO sessions(token_hash,user_id,expires_at) VALUES (%s,%s,clock_timestamp() + %s * interval '1 second') RETURNING expires_at", (digest, row['id'], SESSION_TTL)).fetchone()
del row['password_hash']
return {"token": token, "token_type": "Bearer", "expires_at": store.public(result['expires_at']), "user": store.public(row)}
@app.delete('/api/v1/auth/sessions/current', status_code=204)
def logout(request: Request, user: User = Depends(require_user)):
with connection() as conn:
conn.execute("DELETE FROM sessions WHERE token_hash=%s", (request.state.token_hash,))
return Response(status_code=204)
@app.get('/api/v1/users/me', response_model=User)
def me(user: User = Depends(require_user)):
return user
@app.post('/api/v1/chats', status_code=201, response_model=Chat)
def create_chat(data: CreateChat, user: User = Depends(require_user)):
return store.create_chat(data, user.id)
@app.get('/api/v1/chats', response_model=ChatPage)
def chats(user: User = Depends(require_user), limit: int = Query(50, ge=1, le=100), cursor: str | None = Query(None, min_length=1, max_length=2048)):
return store.list_chats(user.id, limit, cursor)
@app.get('/api/v1/chats/{chat_id}', response_model=Chat)
def chat(chat_id: UUID, user: User = Depends(require_user)):
return store.get_chat(str(chat_id), user.id)
@app.post('/api/v1/chats/{chat_id}/messages', status_code=201, response_model=Message)
def send(request: Request, chat_id: UUID, data: CreateMessage, user: User = Depends(require_user)):
keys = request.headers.getlist("Idempotency-Key")
if len(keys) != 1 or not IDEMPOTENCY_KEY.fullmatch(keys[0]):
raise ApiError(400, "invalid_request", "Требуется корректный Idempotency-Key")
return store.create_message(str(chat_id), user.id, request.method, keys[0], data.text)
@app.get('/api/v1/chats/{chat_id}/messages', response_model=MessagePage)
def history(chat_id: UUID, user: User = Depends(require_user), limit: int = Query(50, ge=1, le=100), cursor: str | None = Query(None, min_length=1, max_length=2048)):
return store.list_messages(str(chat_id), user.id, limit, cursor)
+140
View File
@@ -0,0 +1,140 @@
from pydantic import BaseModel, ConfigDict, Field, field_validator
from uuid import UUID
class CreateUser(BaseModel):
model_config = ConfigDict(
extra="forbid",
strict=True,
)
username: str = Field(
min_length=3,
max_length=32,
pattern=r"^[a-z0-9_]{3,32}$",
)
display_name: str = Field(
min_length=1,
max_length=100,
)
@field_validator("display_name")
@classmethod
def display_name_must_not_be_blank(cls, value: str) -> str:
if not value.strip() or "\x00" in value:
raise ValueError("display_name must not be blank")
return value
password: str = Field(min_length=12, max_length=128)
@field_validator("password")
@classmethod
def password_must_not_be_blank(cls, value: str) -> str:
if not value.strip():
raise ValueError("password must not be blank")
return value
class User(BaseModel):
model_config = ConfigDict(extra="forbid")
id: str
username: str
display_name: str
created_at: str
class CreateChat(BaseModel):
model_config = ConfigDict(
extra="forbid",
strict=True,
)
title: str = Field(
min_length=1,
max_length=100,
)
member_ids: list[str] = Field(
min_length=1,
max_length=100,
)
@field_validator("title")
@classmethod
def title_must_not_be_blank(cls, value: str) -> str:
if not value.strip() or "\x00" in value:
raise ValueError("title must not be blank")
return value
@field_validator("member_ids")
@classmethod
def validate_member_ids(cls, value: list[str]) -> list[str]:
normalized = []
for member_id in value:
try:
normalized.append(str(UUID(member_id)))
except (ValueError, TypeError):
raise ValueError("member_ids must contain valid UUIDs") from None
if len(normalized) != len(set(normalized)):
raise ValueError("member_ids must be unique")
return normalized
class Chat(BaseModel):
model_config = ConfigDict(extra="forbid")
id: str
title: str
member_ids: list[str]
created_at: str
class ChatPage(BaseModel):
model_config = ConfigDict(extra="forbid")
items: list[Chat]
next_cursor: str | None
class CreateMessage(BaseModel):
model_config = ConfigDict(
extra="forbid",
strict=True,
)
text: str = Field(
min_length=1,
max_length=4000,
)
@field_validator("text")
@classmethod
def text_must_not_be_blank(cls, value: str) -> str:
if not value.strip() or "\x00" in value:
raise ValueError("text must not be blank")
return value
class Message(BaseModel):
model_config = ConfigDict(extra="forbid")
id: str
chat_id: str
sender_id: str
text: str
created_at: str
class MessagePage(BaseModel):
model_config = ConfigDict(extra="forbid")
items: list[Message]
next_cursor: str | None
+74
View File
@@ -0,0 +1,74 @@
import base64
import json
import re
from datetime import datetime
from uuid import UUID
from src.errors import ApiError
def encode_cursor(
resource: str,
owner_id: str,
created_at: str,
object_id: str,
) -> str:
payload = {
"resource": resource,
"owner_id": owner_id,
"created_at": created_at,
"id": object_id,
}
raw = json.dumps(
payload,
separators=(",", ":"),
).encode("utf-8")
return base64.urlsafe_b64encode(raw).decode("ascii").rstrip("=")
def decode_cursor(
cursor: str,
expected_resource: str,
expected_owner_id: str,
) -> tuple[str, str]:
try:
if not re.fullmatch(r"[A-Za-z0-9_-]+", cursor):
raise ValueError
padding = "=" * (-len(cursor) % 4)
raw = base64.b64decode(
(cursor + padding).encode("ascii"), altchars=b"-_", validate=True
)
payload = json.loads(raw.decode("utf-8"))
if payload["resource"] != expected_resource:
raise ValueError
if payload["owner_id"] != expected_owner_id:
raise ValueError
created_at = payload["created_at"]
object_id = payload["id"]
# The cursor must carry the same UTC timestamp format produced
# by the server. Invalid timestamps must return HTTP 400 before a SQL cast.
if not isinstance(created_at, str) or not re.fullmatch(
r"\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d+)?Z",
created_at,
):
raise ValueError
datetime.fromisoformat(created_at.replace("Z", "+00:00"))
object_id = str(UUID(object_id))
return created_at, object_id
except Exception:
raise ApiError(
status_code=400,
code="invalid_request",
message="Некорректный cursor",
)
+47
View File
@@ -0,0 +1,47 @@
import base64
import hashlib
import re
import secrets
from argon2 import PasswordHasher
from argon2.exceptions import VerificationError, InvalidHashError
from src.errors import ApiError
# Explicit bounded Argon2id parameters: 19 MiB, 2 passes, one lane.
hasher = PasswordHasher(time_cost=2, memory_cost=19456, parallelism=1)
DUMMY_HASH = hasher.hash(secrets.token_urlsafe(32))
def unauthorized():
raise ApiError(401, "unauthorized", "Требуется действительная сессия")
def basic_credentials(header):
try:
scheme, value = header.split(" ", 1)
if scheme.lower() != "basic":
raise ValueError
username, password = base64.b64decode(value, validate=True).decode("utf-8").split(":", 1)
if not re.fullmatch(r"[a-z0-9_]{3,32}", username) or not 12 <= len(password) <= 128:
raise ValueError
return username, password
except (ValueError, UnicodeError, AttributeError):
unauthorized()
def verify(encoded, password):
try:
return hasher.verify(encoded or DUMMY_HASH, password)
except (VerificationError, InvalidHashError):
return False
def token_hash(header):
try:
scheme, token = header.split(" ", 1)
if scheme.lower() != "bearer" or not re.fullmatch(r"[A-Za-z0-9_-]{43}", token):
raise ValueError
return hashlib.sha256(token.encode("ascii")).digest()
except (ValueError, AttributeError):
unauthorized()
+144
View File
@@ -0,0 +1,144 @@
from datetime import datetime
from uuid import UUID, uuid4
from psycopg.errors import ForeignKeyViolation, UniqueViolation
from src.db import connection
from src.errors import ApiError
from src.pagination import decode_cursor, encode_cursor
def public(value):
if isinstance(value, dict):
return {key: public(item) for key, item in value.items()}
if isinstance(value, list):
return [public(item) for item in value]
if isinstance(value, UUID):
return str(value)
if isinstance(value, datetime):
return value.isoformat().replace("+00:00", "Z")
return value
def create_user(data, password_hash):
try:
with connection() as conn:
row = conn.execute(
"INSERT INTO users(id,username,display_name,password_hash) VALUES (%s,%s,%s,%s) RETURNING id,username,display_name,created_at",
(uuid4(), data.username, data.display_name, password_hash),
).fetchone()
return public(row)
except UniqueViolation:
raise ApiError(409, "conflict", "Username уже занят") from None
def create_chat(data, caller):
if caller not in data.member_ids:
raise ApiError(400, "invalid_request", "Создатель должен входить в member_ids")
try:
with connection() as conn:
chat = conn.execute("INSERT INTO chats(id,title) VALUES (%s,%s) RETURNING id,title,created_at", (uuid4(), data.title)).fetchone()
# FK errors here MUST roll back the already inserted parent chat.
with conn.cursor() as cursor:
cursor.executemany("INSERT INTO chat_members(chat_id,user_id) VALUES (%s,%s)", [(chat['id'], member) for member in data.member_ids])
chat['member_ids'] = data.member_ids
return public(chat)
except ForeignKeyViolation:
raise ApiError(400, "invalid_request", "Указан неизвестный участник") from None
CHAT_SELECT = """SELECT c.id,c.title,c.created_at,
ARRAY(SELECT user_id FROM chat_members WHERE chat_id=c.id ORDER BY user_id) AS member_ids
FROM chats c WHERE EXISTS (SELECT 1 FROM chat_members m WHERE m.chat_id=c.id AND m.user_id=%s)"""
def get_chat(chat_id, user_id):
with connection() as conn:
row = conn.execute(CHAT_SELECT + " AND c.id=%s", (user_id, chat_id)).fetchone()
if not row:
raise ApiError(404, "not_found", "Объект не найден")
return public(row)
def page(rows, limit, resource, owner):
items = public(rows[:limit])
cursor = None
if len(rows) > limit:
last = items[-1]
cursor = encode_cursor(resource, owner, last['created_at'], last['id'])
return {"items": items, "next_cursor": cursor}
def list_chats(user_id, limit, cursor):
params = [user_id]
sql = CHAT_SELECT
if cursor is not None:
timestamp, object_id = decode_cursor(cursor, "chats", user_id)
sql += " AND (c.created_at,c.id) > (%s::timestamptz,%s::uuid)"
params += [timestamp, object_id]
sql += " ORDER BY c.created_at,c.id LIMIT %s"
params.append(limit + 1)
with connection() as conn:
rows = conn.execute(sql, params).fetchall()
return page(rows, limit, "chats", user_id)
def create_message(chat_id, user_id, method, key, text):
scope = (user_id, method, chat_id, key)
message_id = uuid4()
with connection() as conn:
if not conn.execute(
"SELECT 1 FROM chat_members WHERE chat_id=%s AND user_id=%s FOR KEY SHARE",
(chat_id, user_id),
).fetchone():
raise ApiError(404, "not_found", "Объект не найден")
# A concurrent ON CONFLICT waits for the first transaction. The deferred
# FK allows reserving the proposed message ID before inserting the row.
claimed = conn.execute(
"""INSERT INTO message_idempotency (user_id,method,chat_id,key,message_id,expires_at)
VALUES (%s,%s,%s,%s,%s,clock_timestamp() + interval '24 hours')
ON CONFLICT (user_id,method,chat_id,key) DO UPDATE
SET message_id=EXCLUDED.message_id, expires_at=clock_timestamp() + interval '24 hours'
WHERE message_idempotency.expires_at <= clock_timestamp()
RETURNING message_id""",
(*scope, message_id),
).fetchone()
if claimed:
row = conn.execute(
"""INSERT INTO messages (id,chat_id,sender_id,text)
VALUES (%s,%s,%s,%s)
RETURNING id,chat_id,sender_id,text,created_at""",
(message_id, chat_id, user_id, text),
).fetchone()
else:
# READ COMMITTED gives this statement a new snapshot. The UPSERT
# still holds the key row locked, including when its WHERE was false.
row = conn.execute(
"""SELECT m.id,m.chat_id,m.sender_id,m.text,m.created_at
FROM message_idempotency i JOIN messages m ON m.id=i.message_id
WHERE (i.user_id,i.method,i.chat_id,i.key)=(%s,%s,%s,%s)""",
scope,
).fetchone()
# Lab 3 has one semantic input field, the validated text string.
if row["text"] != text:
raise ApiError(409, "conflict", "Ключ уже использован с другим сообщением")
return public(row)
def list_messages(chat_id, user_id, limit, cursor):
with connection() as conn:
# Access precedes cursor decoding, including malformed foreign cursors.
if not conn.execute("SELECT 1 FROM chat_members WHERE chat_id=%s AND user_id=%s", (chat_id, user_id)).fetchone():
raise ApiError(404, "not_found", "Объект не найден")
params = [chat_id]
sql = "SELECT id,chat_id,sender_id,text,created_at FROM messages WHERE chat_id=%s"
if cursor is not None:
timestamp, object_id = decode_cursor(cursor, "messages", chat_id)
sql += " AND (created_at,id) > (%s::timestamptz,%s::uuid)"
params += [timestamp, object_id]
sql += " ORDER BY created_at,id LIMIT %s"
params.append(limit + 1)
rows = conn.execute(sql, params).fetchall()
return page(rows, limit, "messages", chat_id)
+276
View File
@@ -0,0 +1,276 @@
"""Additional grade-3 checks. Run only on the isolated synthetic Compose stack."""
from concurrent.futures import ThreadPoolExecutor
from http.client import HTTPConnection
import json
import os
from pathlib import Path
import subprocess
import sys
from threading import Barrier
import time
from urllib.parse import urlsplit
from uuid import uuid4
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tests"))
from client import Client, Failure, check, run
from resilience import history
BASE = os.environ.get("BASE_URL", "http://localhost:8080")
REPLICAS = {"api1", "api2"}
COMPOSE = ["docker", "compose", "-p", os.environ.get("LAB3_PROJECT", "mtusi-lab3-grade3")]
PSQL = [*COMPOSE, "exec", "-T", "postgres", "psql", "-XAtq", "-v", "ON_ERROR_STOP=1",
"-U", "messenger", "-d", "messenger"]
def instance(response):
return response["headers"]["X-Instance-Id"]
def wait_for_both(client, actor=None):
observed = set()
path, schema = ("/api/v1/users/me", "User") if actor else ("/health/ready", "Health")
deadline = time.monotonic() + 40
while time.monotonic() < deadline:
try:
response = client.call("GET", path, actor=actor, schema=schema)
if actor:
check(response["json"] == actor["user"], "same Bearer changed identity")
observed.add(instance(response))
if observed == REPLICAS:
return
except Failure:
pass
time.sleep(0.2)
raise Failure("both real replicas did not respond through the LB")
def pair(left, right):
barrier = Barrier(2)
def send(operation):
barrier.wait(timeout=5)
return operation()
with ThreadPoolExecutor(max_workers=2) as executor:
futures = [executor.submit(send, operation) for operation in (left, right)]
results = [future.result() for future in futures]
check({instance(result) for result in results} == REPLICAS, "parallel requests did not hit two replicas")
return results
def database(sql):
return subprocess.check_output(PSQL, input=sql + "\n", text=True, timeout=12).strip()
def core(client):
wait_for_both(client)
alice, bob, eve, chat = client.bootstrap()
wait_for_both(client, alice)
print("PASS: one Bearer accepted by api1 and api2")
path = "/api/v1/chats/" + chat["id"] + "/messages"
key = "grade3.same." + uuid4().hex
headers = {"Idempotency-Key": key, "Content-Type": "application/json"}
def post(raw, expected=201, actor=alice, request_headers=headers):
return Client(BASE, timeout=5).call(
"POST", path, expected, actor=actor, raw=raw,
headers=request_headers, schema="Message",
)
first, second = pair(lambda: post(b'{"text":"parallel same"}'),
lambda: post(b'{ "text" : "parallel \\u0073ame" }'))
original = first["json"]
check(original == second["json"] and history(client, bob, chat) == [original],
"same-key race did not produce one identical Message")
print(json.dumps({"same_key_race": [
{"instance": instance(r), "status": r["status"], "message_id": r["json"]["id"]}
for r in (first, second)], "history_count": 1}))
rival_key = "grade3.conflict." + uuid4().hex
rival_headers = dict(headers, **{"Idempotency-Key": rival_key})
left, right = pair(lambda: post(b'{"text":"left"}', [201, 409], request_headers=rival_headers),
lambda: post(b'{"text":"right"}', [201, 409], request_headers=rival_headers))
check(sorted([left["status"], right["status"]]) == [201, 409], "different body race must conflict")
check(len(history(client, bob, chat)) == 2, "different body race added a duplicate")
print(json.dumps({"different_body_race": {instance(r): r["status"] for r in (left, right)}}))
distinct = pair(lambda: client.message(alice, chat, "same text, independent keys"),
lambda: client.message(alice, chat, "same text, independent keys"))
check(len({r["json"]["id"] for r in distinct}) == 2, "different keys were deduplicated")
other_chat = client.call("POST", "/api/v1/chats", 201, actor=alice,
body={"title": "Scope", "member_ids": [alice["id"]]}, schema="Chat")["json"]
check(client.message(bob, chat, original["text"], key=key)["json"]["id"] != original["id"],
"users must have independent key spaces")
check(client.message(alice, other_chat, original["text"], key=key)["json"]["id"] != original["id"],
"chats must have independent key spaces")
for invalid_key in (None, "", "a" * 129, "with space", "comma,key", "é"):
bad = {"Content-Type": "application/json"}
if invalid_key is not None:
bad["Idempotency-Key"] = invalid_key
post(b'{"text":"parallel same"}', 400, request_headers=bad)
for bad_body in (b"{", b"{}", b'{"text":4}', b'{"text":" "}', b'{"text":"x","sender_id":"x"}'):
post(bad_body, 400)
post(b'{"text":"parallel same"}', 404, actor=eve)
post(b'{"text":"parallel same"}', 401, actor=None)
client.call("DELETE", "/api/v1/auth/sessions/current", 204, actor=alice)
post(b'{"text":"parallel same"}', 401)
print("PASS: independent keys/users/chats, invalid key/body, authorization and revocation on replay")
check(database(f"SELECT password_hash LIKE '$argon2id$%' FROM users WHERE id='{bob['id']}';") == "t",
"password is not stored using Argon2id")
check(database(f"""SELECT extract(epoch FROM expires_at - created_at) BETWEEN 3590 AND 3600
FROM sessions WHERE user_id='{bob['id']}';""") == "t", "unexpected default session TTL")
database(f"UPDATE sessions SET created_at=clock_timestamp()-interval '1 hour', expires_at=clock_timestamp()-interval '1 second' WHERE user_id='{eve['id']}';")
client.call("GET", "/api/v1/users/me", 401, actor=eve)
check(database(f"""SELECT extract(epoch FROM i.expires_at - m.created_at) BETWEEN 86390 AND 86400
FROM message_idempotency i JOIN messages m ON m.id=i.message_id
WHERE m.id='{original['id']}';""") == "t", "unexpected idempotency TTL")
previous = next(m for m in history(client, bob, chat) if m["sender_id"] == bob["id"])
database(f"""UPDATE message_idempotency SET expires_at=clock_timestamp()-interval '1 second'
WHERE message_id='{previous['id']}';""")
after = pair(lambda: client.message(bob, chat, "after expiry", key=key),
lambda: client.message(bob, chat, "after expiry", key=key))
check(after[0]["json"] == after[1]["json"] and after[0]["json"]["id"] != previous["id"],
"expired key was not reclaimed exactly once")
check(sum(m["text"] == "after expiry" for m in history(client, bob, chat)) == 1, "expired-key race duplicated")
print("PASS: Argon2id, session TTL and expiry, 24h key TTL and concurrent key reuse after expiry")
unknown_key = "uncertain." + uuid4().hex
url = urlsplit(BASE)
connection = HTTPConnection(url.hostname, url.port, timeout=5)
connection.request("POST", path, body=b'{"text":"unread response"}', headers={
"Authorization": "Bearer " + bob["token"], "Idempotency-Key": unknown_key,
"Content-Type": "application/json",
})
try:
# Observe the committed write through another connection without reading its response.
deadline = time.monotonic() + 5
while True:
observed = [m for m in history(client, bob, chat) if m["text"] == "unread response"]
if observed:
break
check(time.monotonic() < deadline, "unacknowledged write did not reach history")
time.sleep(0.05)
finally:
connection.close()
replay = client.message(bob, chat, "unread response", key=unknown_key)["json"]
check(observed == [replay] and history(client, bob, chat).count(replay) == 1, "unconfirmed POST duplicated")
print(f"PASS: retry of committed POST with unread response returned {replay['id']}")
blocked_key = "transient." + uuid4().hex
with subprocess.Popen(PSQL, stdin=subprocess.PIPE, stdout=subprocess.PIPE, text=True) as lock:
lock.stdin.write("BEGIN; LOCK TABLE messages IN ACCESS EXCLUSIVE MODE; SELECT 'locked';\n")
lock.stdin.flush()
try:
check(lock.stdout.readline().strip() == "locked", "could not acquire test lock")
post(b'{"text":"retry after 503"}', 503, actor=bob,
request_headers=dict(headers, **{"Idempotency-Key": blocked_key}))
finally:
lock.stdin.write("ROLLBACK;\n")
lock.stdin.close()
lock.wait(timeout=10)
check(database(f"SELECT count(*) FROM message_idempotency WHERE key='{blocked_key}';") == "0",
"lock timeout left a committed key reservation")
recovered = client.message(bob, chat, "retry after 503", key=blocked_key)["json"]
check(history(client, bob, chat).count(recovered) == 1, "503 poisoned key or duplicated history")
print("PASS: message insert lock timeout produced 503 and rolled back reservation, same key then committed once")
def container_state(service):
container = subprocess.check_output([*COMPOSE, "ps", "-a", "-q", service], text=True, timeout=10).strip()
template = "{{.Id}} {{.State.Status}} {{.State.StartedAt}}"
return subprocess.check_output(["docker", "inspect", "--format", template, container],
text=True, timeout=10).strip()
def persistence(client):
wait_for_both(client)
alice, bob, _, chat = client.bootstrap()
key = uuid4().hex
original = client.message(alice, chat, "replay across restart", key=key)["json"]
expiry_query = f"SELECT expires_at FROM sessions WHERE user_id='{alice['id']}';"
expiry = database(expiry_query)
check(bool(expiry), "expected a stored session before restart")
services = ("postgres", "api1", "api2")
before = {service: container_state(service) for service in services}
subprocess.run(["make", "test-persistence", "ACTION_SCRIPT=scripts/restart.sh"], check=True, timeout=120)
for service in services:
after = container_state(service)
check(before[service].split()[0] == after.split()[0] and before[service] != after,
"hook did not restart existing " + service)
print(f"{service} before {before[service]} after {after}")
wait_for_both(client, alice)
check(client.message(alice, chat, original["text"], key=key)["json"] == original
and history(client, bob, chat) == [original], "idempotency did not survive restart")
check(database(expiry_query) == expiry, "restart changed the session expiry")
print(f"PASS: old Bearer on both replicas after PostgreSQL restart, unchanged expiry, replay {original['id']}, history count 1")
def run_public_failover(client, actor, chat, original, target, survivor):
command = ["make", "test-failover", "ACTION_SCRIPT=scripts/stop-one.sh"]
recovery = None
with subprocess.Popen(command, stdout=subprocess.PIPE, text=True) as process:
for line in process.stdout:
print(line, end="")
if not line.startswith("Executing explicit failover hook"):
continue
started = time.monotonic()
while time.monotonic() - started <= 10:
if container_state(target).split()[1] == "exited":
try:
path = "/api/v1/chats/" + chat["id"] + "/messages"
response = client.call("GET", path, actor=actor, schema="MessagePage")
if instance(response) == survivor and original in response["json"]["items"]:
recovery = time.monotonic() - started
break
except Failure:
pass
time.sleep(0.1)
check(process.wait(timeout=5) == 0, "public failover scenario failed")
check(recovery is not None and recovery <= 10, "history did not recover within 10 seconds")
print(f"PASS: {survivor} read confirmed history after {target} stopped in {recovery:.3f}s")
def failover(client):
wait_for_both(client)
alice, bob, _, chat = client.bootstrap()
migration = container_state("migrate")
for target in sorted(REPLICAS):
survivor = (REPLICAS - {target}).pop()
key = uuid4().hex
original = client.message(alice, chat, "keep " + target, key=key)["json"]
before = {service: container_state(service) for service in sorted(REPLICAS)}
check(all(state.split()[1] == "running" for state in before.values()), "expected two running containers")
# Public setup sends ten requests through round robin before choosing its target
# from request number ten. Prime the last response to select that replica.
for _ in range(4):
if instance(client.call("GET", "/api/v1/users/me", actor=alice, schema="User")) == target:
break
else:
raise Failure("could not prime failover target")
try:
run_public_failover(client, bob, chat, original, target, survivor)
after = {service: container_state(service) for service in sorted(REPLICAS)}
print(json.dumps({"target": target, "before": before, "after": after}))
check(after[target].split()[1] == "exited" and after[survivor] == before[survivor],
"public hook did not stop exactly the selected container")
replay = client.message(alice, chat, original["text"], key=key)
check(instance(replay) == survivor and replay["json"] == original
and history(client, bob, chat).count(original) == 1, "surviving replica replay failed")
print(f"PASS: {survivor} replayed {original['id']} while {target} was stopped, history count 1")
finally:
subprocess.run(["docker", "start", before[target].split()[0]], check=True, timeout=25)
wait_for_both(client, alice)
check(container_state("migrate") == migration, "replica recovery restarted the migrator")
print(f"PASS: {target} rejoined the LB with the same Bearer; migrator was not restarted")
def main():
scenario = {"ready": wait_for_both, "core": core, "persistence": persistence, "failover": failover}
check(len(sys.argv) == 2 and sys.argv[1] in scenario, "choose ready, core, persistence or failover")
scenario[sys.argv[1]](Client(BASE, timeout=5))
if __name__ == "__main__":
run(main)