ЛР 5 — БВТ2301 — Гамаюнов — на 3 #107

Closed
gamayunov2004 wants to merge 1 commits from gamayunov2004/mtusi2026lab5:solution/bvt2301-gamayunov into main
50 changed files with 3388 additions and 39 deletions
+2
View File
@@ -18,3 +18,5 @@ __pycache__
node_modules
evidence
.DS_Store
/PR_GRADE3_DRAFT.md
/dist
+3 -22
View File
@@ -1,25 +1,6 @@
# Copy to .env for your LOCAL synthetic test environment; never commit .env.
# These are logical interface suggestions; map actual framework names in REPORT.md.
APP_HOST=0.0.0.0
APP_PORT=8080
APP_ENV=lab
# Run scripts/init-env.ps1 from PowerShell before starting the stack.
COMPOSE_PROJECT_NAME=mtusi-lab5-grade3
LAB5_PORT=8085
SESSION_TTL_SECONDS=3600
# Choose credentials locally. File paths contain no secret values.
DATABASE_URL_FILE=/run/secrets/database_url
# Optional on 3, required use on 4 in lab2; Redis OR Valkey:
SESSION_STORE=postgres
CACHE_URL_FILE=/run/secrets/cache_url
API_REPLICAS=2
IDEMPOTENCY_TTL_SECONDS=86400
S3_ENDPOINT=http://s3:9000
S3_REGION=us-east-1
S3_BUCKET=messenger-attachments
S3_ACCESS_KEY_ID_FILE=/run/secrets/s3_access_key_id
S3_SECRET_ACCESS_KEY_FILE=/run/secrets/s3_secret_access_key
MAX_UPLOAD_BYTES=10485760
MAX_IMAGE_PIXELS=40000000
BROKER_URL_FILE=/run/secrets/broker_url
WORKER_CONCURRENCY=2
JOB_MAX_ATTEMPTS=5
JOB_TIMEOUT_SECONDS=60
STAGING_RETENTION_SECONDS=86400
+5
View File
@@ -0,0 +1,5 @@
*.sh text eol=lf
*.py text eol=lf
*.yaml text eol=lf
*.sql text eol=lf
*.ps1 text eol=lf
+2
View File
@@ -30,3 +30,5 @@ coverage/
.vscode/
# Keep small, sanitized evidence summaries in evidence/ under version control.
# Inspect git diff --cached: ignore rules are not a secret scanner.
/PR_GRADE3_DRAFT.md
+12
View File
@@ -0,0 +1,12 @@
FROM python:3.12.10-slim-bookworm
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1 APP_HOST=0.0.0.0 APP_PORT=8080
WORKDIR /app
COPY requirements.txt requirements.lock ./
RUN pip install --no-cache-dir -r requirements.lock && useradd --create-home --uid 10001 appuser
COPY --chown=appuser:appuser src ./src
COPY --chown=appuser:appuser migrations ./migrations
USER appuser
EXPOSE 8080
STOPSIGNAL SIGTERM
HEALTHCHECK --interval=3s --timeout=3s --start-period=10s --retries=5 CMD python -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/health/ready',timeout=2).read()" || exit 1
CMD ["sh", "-c", "exec uvicorn src.main:app --host \"$APP_HOST\" --port \"$APP_PORT\" --no-access-log --timeout-graceful-shutdown 3"]
+78 -17
View File
@@ -1,37 +1,98 @@
# Отчёт — лабораторная 5
- Автор, группа: TODO
- Целевая оценка; выбранный трек 5, если нужен: TODO
- Commit/PR: TODO после публикации
- Использованные источники, библиотеки и помощь ИИ: TODO
- Автор и группа — Гамаюнов Артём, БВТ2301.
- Целевая оценка — 3.
- Ветка — `solution/bvt2301-gamayunov`. Результат подготовлен локально для ревью.
- Основа — собственная реализация ЛР4 и преподавательские материалы ЛР5. Codex помог перенести код, реализовать обработку вложений, запустить проверки Docker и оформить отчёт. Нормативные источники — [контракт API](contracts/README.md), [состояния](contracts/attachment-states.md), [формат задания](contracts/jobs.schema.json).
## Запуск из чистого clone
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
Нужны Windows PowerShell и работающий Docker Desktop в режиме Linux containers с Docker Compose. Сборка при пустом кэше скачивает образы и зависимости. Приложение использует Python 3.12.10, FastAPI 0.115.12, psycopg 3.2.6, Pillow 12.2.0, boto3 1.40.45, Pika 1.3.2 и argon2-cffi 23.1.0. Образы PostgreSQL 16.8-bookworm, RabbitMQ 4.1.4-alpine, HAProxy 3.0.11-alpine и RustFS 1.0.0-alpha.85 закреплены в Compose, RustFS дополнительно задан digest. В tools зафиксирован фактически полученный Python 3.12.15. Tools собран на `docker:27.5.1-cli`; зависимости Python закреплены в lock-файлах.
Команды выполняются из корня ЛР5 в PowerShell.
```powershell
powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\scripts\init-env.ps1
docker compose --env-file .env config --quiet
docker compose --profile test build
docker compose --env-file .env up -d --wait --wait-timeout 150
curl.exe -fsS -i http://127.0.0.1:8085/health/ready
docker compose ps -a
docker compose run --rm --no-deps tools make check validate test
```
Bootstrap создаёт локальный `.env` по шаблону и генерирует синтетические секреты в `secrets/`. Повторный запуск использует прежние файлы и проверяет согласованность URL с паролями. Compose подключает их как файлы. В `.env` можно изменить `LAB5_PORT` при конфликте порта. `COMPOSE_PROJECT_NAME` отличает стенд и именованные тома, `SESSION_TTL_SECONDS` задаёт срок новых сессий, `S3_BUCKET` задаёт имя приватного бакета, `JOB_MAX_ATTEMPTS` задаёт бюджет новых заданий (1–20, по умолчанию 5). Лимиты изображения фиксированы контрактом.
`migrate` применяет версионные SQL-миграции 001–004, `bootstrap` создаёт или проверяет бакет и снимает bucket policy. Оба сервиса завершаются с кодом 0. PostgreSQL проверяется через `pg_isready`, RustFS через `/health`, RabbitMQ через TCP 5672. API проверяет PostgreSQL и приватный бакет через `/health/ready`. HAProxy проверяет обе реплики, собственная проверка HAProxy идёт через тот же маршрут readiness. Подключение worker к очереди подтверждается строкой `consumer ready prefetch=1` и результатами обработки.
Tools использует `BASE_URL=http://lb:8080` внутри Compose. Проверки перезапуска выполняются последовательно на отдельном учебном стенде. Tools имеет локальный доступ к Docker socket для действительных stop/restart. Используйте его в доверенном окружении. Hook-скрипты фильтруют контейнеры по меткам проекта ЛР5 и сервиса. На Windows `COPY --chmod=0755` задаёт права hook-скриптов внутри tools.
```powershell
docker compose run --rm --no-deps tools make test-persistence ACTION_SCRIPT=scripts/restart.sh
docker compose run --rm --no-deps tools make test-failover ACTION_SCRIPT=scripts/stop-one.sh
docker compose ps -a api1 api2
docker compose start api1 api2
docker compose run --rm --no-deps tools python student-tests/grade3.py
```
Публичный failover выбирает реплику по `X-Instance-Id` исходного POST. Для проверки обеих реплик повторите его после восстановления первой и сопоставьте остановленное имя с `docker compose ps -a`. Собственный сценарий явно проверяет обе стороны.
Остановка с сохранением контейнеров и томов — `docker compose stop`. Повторный запуск — `docker compose up -d --wait --wait-timeout 150`. Команда намеренного удаления учебных томов этого Compose-проекта — `docker compose down --volumes`; она приведена для осознанного сброса стенда и в проверках не использовалась.
## Архитектура и решения
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab4, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
| Компонент | Роль и данные |
| --- | --- |
| `lb` и `api1`, `api2` | HAProxy публикует единственный хостовый порт на `127.0.0.1`. Две FastAPI-реплики работают с общими хранилищами. |
| `postgres` | Пользователи, Argon2id-хеши, хеши Bearer-токенов, чаты, сообщения, идемпотентность, метаданные вложений и попыток. Именованный том `postgres_data`. |
| `s3` | Приватный RustFS, один бакет. `staging/` хранит принятый оригинал, `originals/` хранит опубликованный оригинал, `variants/` хранит JPEG. Именованный том `s3_data`. |
| `broker` | Durable classic queue `attachment.process.v1`, persistent delivery mode 2. Именованный том `broker_data` и постоянное имя узла. |
| `worker` | Отдельный процесс, один consumer с prefetch=1. Управляет состояниями и записывает объекты и их метаданные. |
| `migrate`, `bootstrap`, `tools` | Стартовые миграции, настройка S3 и тестовые команды соответственно. |
Из ЛР4 перенесены FastAPI, SQL-схема сообщений и вложений, Bearer-сессии, Argon2id, ACL, keyset pagination, балансировка и приватный RustFS. Отправка сообщения сохраняет 201 и атомарный результат по `Idempotency-Key`. Миграция сохраняет ключи, статус и ссылки существующих строк ЛР4. Для таких записей отсутствующая миниатюра даёт 404. Обновление заполненной базы ЛР4 отдельно не проверялось. Все прогоны использовали тома ЛР5.
До 202 API принимает полный HTTP body, проверяет ACL, filename, MIME, декодируемый PNG/JPEG, 10 MiB и 40 млн пикселей. Затем API записывает полный оригинал в приватный staging, одним SQL COMMIT сохраняет `attachments` и `attachment_jobs` и получает publisher confirm для `mandatory` persistent-публикации в durable-очередь. Ответ содержит снимок queued. При неподтверждённой публикации клиент получает 503 с безопасным JSON. Broker и worker находятся вне маршрута текстовых сообщений.
Carrier содержит `schema_version`, `type`, `job_id`, `attachment_id`, `pipeline_version`, `attempt` и `created_at`. Идентичность работы сверяется с PostgreSQL. Worker под session advisory lock фиксирует `processing` и текущую попытку, получает staging по доверенному ключу из БД и проверяет длину и SHA-256. Миниатюра — JPEG quality 85, EXIF orientation, белый фон для прозрачности, LANCZOS, вписывание в 256×256 с округлением вниз до 1 px и без увеличения. Оригинал записывается побайтно. Затем одной транзакцией фиксируются вариант и `ready`. Ключ эффекта — `(attachment_id, pipeline_version, variant)`, одинаковый в SQL и префиксе объектов. В `ready` дубль находит завершённый вариант и подтверждается без записи.
Publisher confirm подтверждает сохранение сообщения брокером. Manual consumer ack отправляется после устойчивого результата или после durable COMMIT состояния queued и confirm следующей попытки. При временной ошибке задержка до следующей попытки составляет 3 секунды. Бюджет хранится в БД отдельно для каждого задания. Если worker погиб во время processing, unacked-доставка возвращается в очередь, блокировка освобождается вместе с SQL-сессией, предыдущая попытка засчитывается. При сбое после записи retry и до подтверждения новой публикации старая доставка восстанавливает текущую попытку. При повторе после записи объектов успешная работа заново использует те же ключи, при повторе после COMMIT worker подтверждает готовый результат. При исчерпании попыток API возвращает `failed` с `retry_exhausted`, при несовпадении staging с сохранённой контрольной суммой — `invalid_image`. Подробности исключений скрыты.
В метаданных доступны все состояния. Для queued/processing можно отправлять сообщения с вложением, оба варианта content возвращают 409. В failed создание такой связи возвращает 409. Получение метаданных и content требует Bearer и участия в чате, клиентские ответы исключают внутренние ключи объектов.
## Доказательства критериев
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
| Критерий README | Команда/сценарий | Наблюдаемый результат | Доказательство |
| --- | --- | --- | --- |
| Публичный smoke | make test | TODO | TODO |
| Остальные пункты уровня 3 | TODO | TODO | TODO |
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
| Сборка, готовность, миграции и публичный smoke | `docker compose --profile test build`, `up --wait`, `make check validate test` в tools | PASS. Startup и HTTP readiness 200. Миграции 001–004, bootstrap exit 0, smoke дождался ready и прочитал оригинал и JPEG. | [startup](evidence/grade3/startup.txt), [checks](evidence/grade3/checks.txt) |
| Минимум ЛР1–ЛР4 | Публичные smoke, persistence, failover и собственные проверки | PASS. Argon2id, Bearer без повторного входа, ACL, keyset, приватный RustFS. 8 конкурентных POST с одним ключом вернули тот же ID через две реплики. Текст и вложения пережили перезапуски. | [checks](evidence/grade3/checks.txt), [persistence](evidence/grade3/persistence.txt), [student](evidence/grade3/student.txt) |
| Устойчивый приём при остановке worker | `grade3.py`, остановка worker и два upload | PASS. Два 202 queued, исходный staging закрыт для unsigned GET, queued content 409, обычное сообщение и сообщение с вложением 201. | [student](evidence/grade3/student.txt) |
| Broker, worker и состояния | `grade3.py`, перезапуск api1/api2 и broker с тем же томом, запуск worker | PASS. Два durable задания с delivery mode 2 и carrier по схеме остались в очереди. Видимы queued и processing, content 409 до атомарной фиксации ready. Оба ID стали ready без нового upload. | [student](evidence/grade3/student.txt), [worker](evidence/grade3/worker.txt) |
| Оригинал и JPEG 256×256 bounding box | Публичный smoke и `grade3.py` | PASS. PNG 320×200 дал JPEG 256×160. EXIF-поворот дал 81×256, прозрачный 37×19 остался 37×19 с белым фоном. Сверены байты, Content-Type, SHA-256 и размеры. | [checks](evidence/grade3/checks.txt), [student](evidence/grade3/student.txt) |
| Повторная доставка и ограниченный retry | `grade3.py`, два дубликата, stop/start S3 и повреждение staging | PASS. Два дубля подтверждены, вариант один в SQL и S3. После возврата S3 тот же ID обработан. Повреждённый staging дал `invalid_image`, недоступный staging после 5 попыток — `retry_exhausted`. | [student](evidence/grade3/student.txt), [worker](evidence/grade3/worker.txt) |
| Обработка отказов и проверка доступа | `grade3.py`, повреждённые/слишком большие изображения, чужой чат, отсутствующий Bearer и остановленный broker | PASS. 400/401/404/409/413/415 по сценариям, неизвестный ready-вариант 404, непринятая загрузка при недоступном AMQP 503, текстовое сообщение 201. | [student](evidence/grade3/student.txt) |
| Failover обеих реплик | Два публичных `make test-failover` и собственные загрузки с остановленной второй репликой | PASS. Hook по очереди оставил api2 и api1 остановленными. Прежний Bearer, новая загрузка и чтение нового JPEG работают через каждую выжившую реплику. | [failover](evidence/grade3/failover.txt), [student](evidence/grade3/student.txt) |
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
Сборку, bootstrap и публичные check/validate/smoke повторили с чистой копией исходников и новыми томами отдельного проекта на порту 18085. Готовность через HAProxy вернула HTTP 200. После прогона контейнеры этой копии сняты, тома сохранены. [Журнал воспроизведения](evidence/grade3/clean-reproduction.txt)
## Эксперименты и отказы
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
Все прогоны используют одиночный Docker Desktop в Linux-режиме, 4 vCPU и 8 242 380 800 байт доступной памяти. Тесты функциональные, без искусственного профиля нагрузки. В клиенте одновременно выполнялись 8 одинаковых POST с одним ключом идемпотентности. Остальные прогоны — короткие последовательные сценарии на синтетическом PNG 320×200, JPEG 319×101 с EXIF orientation 6 и RGBA PNG 37×19. Синтетический файл размером 10 MiB + 1 байт и компактный PNG с 40 005 000 пикселей проверяют разные ограничения. Время в текстовых журналах — время хоста и UTC из `docker inspect`. Время восстановления public failover считается от старта hook до первого успешного POST после его завершения. В собственном тесте время ожидания ready начинается непосредственно перед опросом HTTP, поэтому оно не включает предшествующие перезапуски.
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
| --- | --- | --- | --- |
| TODO | TODO | TODO | TODO |
| Воздействие | Ожидание | Наблюдение и время |
| --- | --- | --- |
| Перезапуск PostgreSQL, RustFS, RabbitMQ, двух API, HAProxy и worker | Прежний токен и подтверждённые данные доступны | Публичный persistence PASS. `StartedAt` изменился у всех семи контейнеров, имена трёх томов совпали. Финальный прогон занял около 40 с. |
| Остановка api2 и api1 по очереди | Сохраняются подтверждения, сессия и запись в оставшуюся реплику | Публичный failover PASS. Первый POST после hook через 2,089 с при остановке api2 и через 1,863 с при остановке api1. За 15 с подтверждены 98 и 116 записей соответственно, неуспешных попыток 0. |
| Перезапуск API и RabbitMQ при двух queued jobs | Те же объекты должны стать ready при запуске worker | После рестарта API readiness вернулся за 3,197 с от начала опроса. Через 16,607 с опроса broker снова healthy, очередь сохранила два сообщения и тот же том. После снятия контрольной блокировки SQL ready наблюдался через 0,181 и 0,155 с от начала опроса. |
| Остановка RustFS | Запись попытки остаётся, после возврата S3 тот же ID доходит до ready | После первой ошибки SQL показал `queued\|2\|storage_unavailable\|5`. После восстановления S3 readiness и ready исходного ID вернулись. |
| Отсутствующий и повреждённый staging | Предсказуемые безопасные ошибки | При отсутствии объекта `failed` зафиксирован на попытке 5, при повреждении — на попытке 1. Content и новое связывание с сообщением возвращали 409. |
Первый Compose-запуск выявил медленную проверку здоровья RabbitMQ. После замены probe на проверку AMQP-порта startup прошёл. Публичный smoke, запущенный до готовности, и первая попытка failover сразу после рестарта брокера получили 503. Ожидание readiness при запуске и ожидание broker health в restart hook устранили эти ошибки в повторных прогонах.
## Ограничения и следующие шаги
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
- Стенд однохостовый. HAProxy, PostgreSQL, RabbitMQ и RustFS имеют по одному экземпляру. Проверялись перезапуски с теми же томами и отказ одной API-реплики. Предполагается исправность диска и файловой системы Docker.
- До HTTP 202 есть окно между staging и SQL COMMIT, а также между COMMIT и publisher confirm. Если публикация не произошла, клиент получает ошибку либо разрыв соединения. Если confirm или HTTP-ответ потерялся после приёма брокером, результат для клиента двусмыслен. Повторный upload создаёт отдельное вложение. Автоматическое закрытие разрыва исходной публикации требует outbox или равносильной схемы.
- Успешный worker пробует очистить staging после COMMIT `ready`. Для failed и непривязанных ключей предусмотрена ручная ревизия через 24 часа. Проверку проводят в окне обслуживания с остановленным приёмом upload и worker: сверяют кандидатов в S3 с БД и удаляют только staging у завершённых записей и объекты старше 24 часов без SQL-ссылки. Staging queued/processing сохраняют до разрешения конкретного задания. Доступ к таблицам и бакету для такого обслуживания нужен только оператору стенда.
- Временная недоступность PostgreSQL и брокера приводит к повторному подключению consumer. При зависшем живом процессе автоматического lease/тайм-аута задачи нет. Остановка worker освобождает AMQP-доставку и SQL-блокировку. S3 участвует в readiness API, поэтому полная недоступность S3 может временно вывести обе реплики из маршрутизации HAProxy.
- Преподавательские файлы и публичные тесты сохранены. Трек уровня 4 или 5 не заявлен.
+156
View File
@@ -0,0 +1,156 @@
name: mtusi-lab5-grade3
x-s3-env: &s3-env
S3_ENDPOINT: http://s3:9000
S3_REGION: us-east-1
S3_BUCKET: ${S3_BUCKET:-messenger-attachments}
S3_ACCESS_KEY_ID_FILE: /run/secrets/s3_access_key_id
S3_SECRET_ACCESS_KEY_FILE: /run/secrets/s3_secret_access_key
x-app: &app
image: mtusi-lab5-api:grade3
pull_policy: never
x-api: &api
<<: *app
environment: &api-env
<<: *s3-env
BROKER_URL_FILE: /run/secrets/broker_url
JOB_MAX_ATTEMPTS: ${JOB_MAX_ATTEMPTS:-5}
APP_HOST: 0.0.0.0
APP_PORT: 8080
SESSION_TTL_SECONDS: ${SESSION_TTL_SECONDS:-3600}
DATABASE_URL_FILE: /run/secrets/database_url
secrets: [database_url, s3_access_key_id, s3_secret_access_key, broker_url]
depends_on:
migrate: {condition: service_completed_successfully}
bootstrap: {condition: service_completed_successfully}
restart: unless-stopped
stop_grace_period: 6s
services:
postgres:
image: postgres:16.8-bookworm
environment:
POSTGRES_USER: messenger
POSTGRES_DB: messenger
POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password
secrets: [postgres_password]
volumes: [postgres_data:/var/lib/postgresql/data]
healthcheck:
test: [CMD-SHELL, "pg_isready -h 127.0.0.1 -U messenger -d messenger"]
interval: 2s
timeout: 2s
retries: 20
restart: unless-stopped
s3:
image: rustfs/rustfs:1.0.0-alpha.85@sha256:23ff2af1ec2d64e0fb9608a1c39b5eb96fcff3a8efecf332986c948b0e3e4f8a
entrypoint: [sh, -ec, "unset RUSTFS_ACCESS_KEY RUSTFS_SECRET_KEY; exec /entrypoint.sh rustfs"]
environment:
RUSTFS_VOLUMES: /data
RUSTFS_ADDRESS: ":9000"
RUSTFS_CONSOLE_ENABLE: "false"
RUSTFS_ACCESS_KEY_FILE: /run/secrets/s3_access_key_id
RUSTFS_SECRET_KEY_FILE: /run/secrets/s3_secret_access_key
secrets: [s3_access_key_id, s3_secret_access_key]
volumes: [s3_data:/data]
healthcheck:
test: [CMD, curl, -fsS, --max-time, "2", "http://127.0.0.1:9000/health"]
interval: 2s
timeout: 3s
retries: 20
restart: unless-stopped
migrate:
<<: *app
build: .
command: [python, -m, src.db]
environment:
DATABASE_URL_FILE: /run/secrets/database_url
secrets: [database_url]
healthcheck: {disable: true}
depends_on:
postgres: {condition: service_healthy}
bootstrap:
<<: *app
command: [python, -m, src.s3]
environment: *s3-env
secrets: [s3_access_key_id, s3_secret_access_key]
healthcheck: {disable: true}
depends_on:
s3: {condition: service_healthy}
broker:
image: rabbitmq:4.1.4-alpine
hostname: broker
entrypoint: [sh, -ec, 'export RABBITMQ_DEFAULT_PASS="$$(cat /run/secrets/broker_password)"; exec docker-entrypoint.sh rabbitmq-server']
environment:
RABBITMQ_DEFAULT_USER: messenger
secrets: [broker_password]
volumes: [broker_data:/var/lib/rabbitmq]
healthcheck:
test: [CMD, nc, -z, -w, "2", "127.0.0.1", "5672"]
interval: 3s
timeout: 3s
start_period: 60s
retries: 15
restart: unless-stopped
worker:
<<: *api
command: [python, -m, src.worker]
healthcheck: {disable: true}
depends_on:
migrate: {condition: service_completed_successfully}
bootstrap: {condition: service_completed_successfully}
broker: {condition: service_healthy}
api1:
<<: *api
environment:
<<: *api-env
INSTANCE_ID: api1
api2:
<<: *api
environment:
<<: *api-env
INSTANCE_ID: api2
lb:
image: haproxy:3.0.11-alpine
ports: ["127.0.0.1:${LAB5_PORT:-8085}:8080"]
volumes:
- ./infra/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro
healthcheck:
test: [CMD, wget, -q, -O, /dev/null, "http://127.0.0.1:8080/health/ready"]
interval: 3s
timeout: 3s
retries: 30
restart: unless-stopped
tools:
profiles: [test]
image: mtusi-lab5-tools:grade3
build: {context: ., dockerfile: infra/Dockerfile.tools}
volumes:
- /var/run/docker.sock:/var/run/docker.sock
environment:
BASE_URL: http://lb:8080
LAB5_PROJECT: ${COMPOSE_PROJECT_NAME:-mtusi-lab5-grade3}
S3_BUCKET: ${S3_BUCKET:-messenger-attachments}
secrets:
postgres_password: {file: ./secrets/postgres_password}
database_url: {file: ./secrets/database_url}
s3_access_key_id: {file: ./secrets/s3_access_key_id}
s3_secret_access_key: {file: ./secrets/s3_secret_access_key}
broker_password: {file: ./secrets/broker_password}
broker_url: {file: ./secrets/broker_url}
volumes:
postgres_data:
s3_data:
broker_data:
+20
View File
@@ -0,0 +1,20 @@
# Доказательства ЛР5, оценка 3
Прогоны выполнены в Windows PowerShell и Docker Desktop на отдельном проекте `mtusi-lab5-grade3`. Все запросы используют синтетические данные и локальный стенд. Сценарии повторяются по REPORT.md и student-tests/README.md.
| Журнал | Содержание |
| --- | --- |
| `environment.txt` | Версии Docker и ресурсы VM, состояние старых стендов и их томов перед запуском. |
| `startup-initial.txt` | Первый запуск с истёкшей проверкой готовности RabbitMQ. |
| `build-and-early-smoke.txt` | Успешная сборка и ранний smoke, начатый до готовности стенда. |
| `startup.txt` | Успешный запуск после изменения probe, health, миграции и bootstrap. |
| `checks.txt` | Повторный запуск с ожиданием health, публичные check/validate/smoke и Python runtime. |
| `persistence.txt` | Публичная проверка persistence и сравнение StartedAt и томов. |
| `failover.txt` | Публичные прогоны failover и `docker inspect` остановленных реплик. |
| `student.txt` | Свои проверки доставки, worker, S3, изображений, ACL, попыток и обеих реплик. |
| `persistence-initial.txt` и `failover-early.txt` | Первый persistence сохранил данные, но сразу после него ранний failover получил 503 до готовности AMQP. |
| `worker.txt` | Журнал worker с отметками UTC, набор опубликованных портов, состояние соседних стендов и томов после проверок. |
| `clean-reproduction.txt` | Повтор сборки, установки и smoke из чистой копии исходников на новых томах. |
| `git.txt` | Проверка рабочего дерева и список файлов перед упаковкой. |
Probe и ранний smoke оставлены как история диагностики. Результаты берутся из завершённых прогонов и наблюдений Docker, приведённых в отчёте.
+154
View File
@@ -0,0 +1,154 @@
2026-11-20T18:56:10.0437266+03:00
PS> docker compose --progress plain --profile test build
Image mtusi-lab5-api:grade3 Building
Image mtusi-lab5-tools:grade3 Building
#1 [internal] load local bake definitions
#1 reading from stdin 948B 0.3s done
#1 DONE 0.3s
#2 [migrate internal] load build definition from Dockerfile
#2 transferring dockerfile:
#2 transferring dockerfile: 773B 0.1s done
#2 DONE 0.6s
#3 [tools internal] load build definition from Dockerfile.tools
#3 transferring dockerfile: 720B 0.1s done
#3 DONE 0.6s
#4 [tools internal] load metadata for docker.io/library/docker:27.5.1-cli
#4 ...
#5 [migrate internal] load metadata for docker.io/library/python:3.12.10-slim-bookworm
#5 DONE 0.5s
#6 [migrate internal] load .dockerignore
#6 transferring context: 216B 0.1s done
#6 ...
#4 [tools internal] load metadata for docker.io/library/docker:27.5.1-cli
#4 DONE 0.7s
#6 [tools internal] load .dockerignore
#6 transferring context: 216B 0.1s done
#6 DONE 0.3s
#7 [tools internal] load build context
#7 ...
#8 [migrate internal] load build context
#8 transferring context: 52.75kB 0.2s done
#8 DONE 0.2s
#9 [tools 1/11] FROM docker.io/library/docker:27.5.1-cli@sha256:851f91d241214e7c6db86513b270d58776379aacc5eb9c4a87e5b47115e3065c
#9 resolve docker.io/library/docker:27.5.1-cli@sha256:851f91d241214e7c6db86513b270d58776379aacc5eb9c4a87e5b47115e3065c 0.3s done
#9 DONE 0.3s
#7 [tools internal] load build context
#7 transferring context: 364.55kB 0.3s done
#7 DONE 0.3s
#10 [migrate 1/6] FROM docker.io/library/python:3.12.10-slim-bookworm@sha256:fd95fa221297a88e1cf49c55ec1828edd7c5a428187e67b5d1805692d11588db
#10 resolve docker.io/library/python:3.12.10-slim-bookworm@sha256:fd95fa221297a88e1cf49c55ec1828edd7c5a428187e67b5d1805692d11588db 0.3s done
#10 DONE 0.4s
#11 [migrate 2/6] WORKDIR /app
#11 CACHED
#12 [migrate 3/6] COPY requirements.txt requirements.lock ./
#12 CACHED
#13 [migrate 4/6] RUN pip install --no-cache-dir -r requirements.lock && useradd --create-home --uid 10001 appuser
#13 CACHED
#14 [migrate 5/6] COPY --chown=appuser:appuser src ./src
#14 CACHED
#15 [migrate 6/6] COPY --chown=appuser:appuser migrations ./migrations
#15 CACHED
#16 [migrate] exporting to image
#16 exporting layers
#16 ...
#17 [tools 9/11] COPY scripts/check_contract.py scripts/validate_openapi.py ./scripts/
#17 CACHED
#18 [tools 2/11] RUN apk add --no-cache python3 py3-pip make && python3 -m venv /opt/venv
#18 CACHED
#19 [tools 4/11] COPY infra/requirements-tools.lock /tmp/requirements-tools.lock
#19 CACHED
#20 [tools 6/11] COPY Makefile lab.json README.md COURSE.md CONTRIBUTING.md REPORT.md ./
#20 CACHED
#21 [tools 8/11] COPY tests/ ./tests/
#21 CACHED
#22 [tools 5/11] RUN pip install --no-cache-dir -r /tmp/requirements-tools.lock
#22 CACHED
#23 [tools 3/11] WORKDIR /workspace
#23 CACHED
#24 [tools 7/11] COPY contracts/ ./contracts/
#24 CACHED
#25 [tools 10/11] COPY --chmod=0755 scripts/restart.sh scripts/stop-one.sh scripts/lab-containers.sh ./scripts/
#25 CACHED
#16 [migrate] exporting to image
#16 exporting layers 0.1s done
#16 exporting manifest sha256:20ede3477485db65afa1ae6381b03dfe77a92092f77ed7c29ebedd9ca0c8661b 0.0s done
#16 exporting config sha256:223781fc15f3341c55f85a5d978d21b20b08752a0fb6b3068bce324d146e5662 0.0s done
#16 exporting attestation manifest sha256:ccc967481226eb5a7f6cc6cf8a357b0ff67a47f57ba02202f3e983cfa70bdbda
#16 ...
#26 [tools 11/11] COPY student-tests/ ./student-tests/
#26 DONE 0.5s
#27 [tools] exporting to image
#27 ...
#16 [migrate] exporting to image
#16 exporting attestation manifest sha256:ccc967481226eb5a7f6cc6cf8a357b0ff67a47f57ba02202f3e983cfa70bdbda 0.3s done
#16 exporting manifest list sha256:0273efc48eb65d316dbe13b15bf7226c403f454f9b5ed03b85fec2cca64e692c 0.2s done
#16 naming to docker.io/library/mtusi-lab5-api:grade3 0.0s done
#16 unpacking to docker.io/library/mtusi-lab5-api:grade3 0.0s done
#16 DONE 1.4s
#27 [tools] exporting to image
#27 exporting layers 1.6s done
#27 exporting manifest sha256:81174ce6991023487617687a7bff7e325042712860cfaca14b972050a0b8d8d9
#27 exporting manifest sha256:81174ce6991023487617687a7bff7e325042712860cfaca14b972050a0b8d8d9 0.3s done
#27 exporting config sha256:e1a03b497e2afdb6c62eeffccca951a5748407b7fb0539e668249062025b00de
#27 exporting config sha256:e1a03b497e2afdb6c62eeffccca951a5748407b7fb0539e668249062025b00de 0.2s done
#27 exporting attestation manifest sha256:c0e4feb89d4eebb61e20cd5663435019057c6c38769dd1aa5242c78c5f689258
#27 exporting attestation manifest sha256:c0e4feb89d4eebb61e20cd5663435019057c6c38769dd1aa5242c78c5f689258 0.2s done
#27 exporting manifest list sha256:121557e657f6fa1a020e4bdf7d8033846dbe25d60fc51adf0fe38dc70c37c611
#27 exporting manifest list sha256:121557e657f6fa1a020e4bdf7d8033846dbe25d60fc51adf0fe38dc70c37c611 0.1s done
#27 naming to docker.io/library/mtusi-lab5-tools:grade3 0.0s done
#27 unpacking to docker.io/library/mtusi-lab5-tools:grade3
#27 ...
#28 [migrate] resolving provenance for metadata file
#28 DONE 0.1s
#27 [tools] exporting to image
#27 unpacking to docker.io/library/mtusi-lab5-tools:grade3 0.2s done
#27 DONE 3.3s
#29 [tools] resolving provenance for metadata file
#29 DONE 0.2s
Image mtusi-lab5-tools:grade3 Built
Image mtusi-lab5-api:grade3 Built
PS> docker compose run --rm --no-deps tools make check validate test
Container mtusi-lab5-grade3-tools-run-bcd94ea191bb Creating
Container mtusi-lab5-grade3-tools-run-bcd94ea191bb Created
python3 scripts/check_contract.py
PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.
python3 scripts/validate_openapi.py
PASS: full OpenAPI 3.1 and JSON Schema validation; application NOT tested.
python3 tests/smoke.py --base-url "http://lb:8080"
FAIL: GET /health/live: expected [200], received 503
make: *** [Makefile:19: test] Error 1
+57
View File
@@ -0,0 +1,57 @@
2026-11-20T18:57:06.5773372+03:00
PS> docker compose --env-file .env up -d --wait --wait-timeout 150
Container mtusi-lab5-grade3-broker-1 Running
Container mtusi-lab5-grade3-postgres-1 Running
Container mtusi-lab5-grade3-lb-1 Running
Container mtusi-lab5-grade3-s3-1 Running
Container mtusi-lab5-grade3-worker-1 Running
Container mtusi-lab5-grade3-api1-1 Running
Container mtusi-lab5-grade3-api2-1 Running
Container mtusi-lab5-grade3-broker-1 Waiting
Container mtusi-lab5-grade3-postgres-1 Waiting
Container mtusi-lab5-grade3-s3-1 Waiting
Container mtusi-lab5-grade3-postgres-1 Healthy
Container mtusi-lab5-grade3-migrate-1 Starting
Container mtusi-lab5-grade3-broker-1 Healthy
Container mtusi-lab5-grade3-s3-1 Healthy
Container mtusi-lab5-grade3-bootstrap-1 Starting
Container mtusi-lab5-grade3-migrate-1 Started
Container mtusi-lab5-grade3-migrate-1 Waiting
Container mtusi-lab5-grade3-bootstrap-1 Started
Container mtusi-lab5-grade3-bootstrap-1 Waiting
Container mtusi-lab5-grade3-migrate-1 Exited
Container mtusi-lab5-grade3-bootstrap-1 Exited
Container mtusi-lab5-grade3-lb-1 Waiting
Container mtusi-lab5-grade3-migrate-1 Waiting
Container mtusi-lab5-grade3-s3-1 Waiting
Container mtusi-lab5-grade3-worker-1 Waiting
Container mtusi-lab5-grade3-api1-1 Waiting
Container mtusi-lab5-grade3-api2-1 Waiting
Container mtusi-lab5-grade3-postgres-1 Waiting
Container mtusi-lab5-grade3-bootstrap-1 Waiting
Container mtusi-lab5-grade3-broker-1 Waiting
Container mtusi-lab5-grade3-bootstrap-1 Exited
Container mtusi-lab5-grade3-migrate-1 Exited
Container mtusi-lab5-grade3-s3-1 Healthy
Container mtusi-lab5-grade3-api1-1 Healthy
Container mtusi-lab5-grade3-postgres-1 Healthy
Container mtusi-lab5-grade3-worker-1 Healthy
Container mtusi-lab5-grade3-lb-1 Healthy
Container mtusi-lab5-grade3-api2-1 Healthy
Container mtusi-lab5-grade3-broker-1 Healthy
PS> docker compose run --rm --no-deps tools make check validate test
Container mtusi-lab5-grade3-tools-run-61012d51d077 Creating
Container mtusi-lab5-grade3-tools-run-61012d51d077 Created
python3 scripts/check_contract.py
PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.
python3 scripts/validate_openapi.py
PASS: full OpenAPI 3.1 and JSON Schema validation; application NOT tested.
python3 tests/smoke.py --base-url "http://lb:8080"
PASS: lab5 public API scenario; infrastructure and grading evidence remain separate.
PS> docker compose run --rm --no-deps tools python --version
Container mtusi-lab5-grade3-tools-run-998d936cea6b Creating
Container mtusi-lab5-grade3-tools-run-998d936cea6b Created
Python 3.12.15
PS> docker compose exec -T api1 python --version
Python 3.12.10
2026-11-20T18:57:31.6435644+03:00
+287
View File
@@ -0,0 +1,287 @@
2026-11-20T19:06:04.7856039+03:00
Fresh source snapshot at C:\dev\mtusi\lab5\dist\review-clean-20261120 with empty Docker project mtusi-lab5-grade3-review-20261120; port 18085
PS> powershell.exe -NoProfile -ExecutionPolicy Bypass -File scripts/init-env.ps1
Lab 5 local configuration ready.
PS> docker compose --env-file .env config --quiet
PS> docker compose --profile test build
Image mtusi-lab5-api:grade3 Building
Image mtusi-lab5-tools:grade3 Building
#1 [internal] load local bake definitions
#1 reading from stdin 1.10kB 0.2s done
#1 DONE 0.2s
#2 [tools internal] load build definition from Dockerfile.tools
#2 transferring dockerfile:
#2 transferring dockerfile: 720B 0.0s done
#2 DONE 1.0s
#3 [tools internal] load metadata for docker.io/library/docker:27.5.1-cli
#3 ...
#4 [migrate internal] load build definition from Dockerfile
#4 transferring dockerfile: 773B 0.0s done
#4 DONE 1.0s
#5 [migrate internal] load metadata for docker.io/library/python:3.12.10-slim-bookworm
#5 DONE 0.2s
#3 [tools internal] load metadata for docker.io/library/docker:27.5.1-cli
#3 DONE 0.2s
#6 [migrate internal] load .dockerignore
#6 transferring context:
#6 transferring context: 222B 0.1s done
#6 DONE 0.3s
#7 [migrate internal] load build context
#7 ...
#8 [tools 1/11] FROM docker.io/library/docker:27.5.1-cli@sha256:851f91d241214e7c6db86513b270d58776379aacc5eb9c4a87e5b47115e3065c
#8 resolve docker.io/library/docker:27.5.1-cli@sha256:851f91d241214e7c6db86513b270d58776379aacc5eb9c4a87e5b47115e3065c 1.0s done
#8 DONE 1.0s
#7 [migrate internal] load build context
#7 ...
#9 [migrate 1/6] FROM docker.io/library/python:3.12.10-slim-bookworm@sha256:fd95fa221297a88e1cf49c55ec1828edd7c5a428187e67b5d1805692d11588db
#9 resolve docker.io/library/python:3.12.10-slim-bookworm@sha256:fd95fa221297a88e1cf49c55ec1828edd7c5a428187e67b5d1805692d11588db 1.2s done
#9 DONE 1.3s
#7 [migrate internal] load build context
#7 ...
#10 [tools internal] load build context
#10 transferring context: 385.23kB 1.0s done
#10 DONE 1.6s
#7 [migrate internal] load build context
#7 transferring context: 52.75kB 0.7s done
#7 DONE 1.6s
#11 [migrate 2/6] WORKDIR /app
#11 CACHED
#12 [migrate 3/6] COPY requirements.txt requirements.lock ./
#12 CACHED
#13 [migrate 5/6] COPY --chown=appuser:appuser src ./src
#13 CACHED
#14 [migrate 4/6] RUN pip install --no-cache-dir -r requirements.lock && useradd --create-home --uid 10001 appuser
#14 CACHED
#15 [migrate 6/6] COPY --chown=appuser:appuser migrations ./migrations
#15 CACHED
#16 [tools 4/11] COPY infra/requirements-tools.lock /tmp/requirements-tools.lock
#16 CACHED
#17 [tools 3/11] WORKDIR /workspace
#17 CACHED
#18 [tools 2/11] RUN apk add --no-cache python3 py3-pip make && python3 -m venv /opt/venv
#18 CACHED
#19 [tools 5/11] RUN pip install --no-cache-dir -r /tmp/requirements-tools.lock
#19 CACHED
#20 [tools 6/11] COPY Makefile lab.json README.md COURSE.md CONTRIBUTING.md REPORT.md ./
#20 DONE 1.5s
#21 [migrate] exporting to image
#21 exporting layers 0.0s done
#21 exporting manifest sha256:ddaa9a114523c63e5fa5f6a0175d1ce8117d9b77fce7e6fd3edbc4021e858a93 0.4s done
#21 exporting config sha256:4b0a0ad6357825596db6169c6969e73c72a955de3ba39b64d72e20ff7dd079c4 0.3s done
#21 exporting attestation manifest sha256:b08e28f94f92205af6c4ac2a88bfb64e1b04e43a3410a7cdc09e864dd4fca688
#21 exporting attestation manifest sha256:b08e28f94f92205af6c4ac2a88bfb64e1b04e43a3410a7cdc09e864dd4fca688 0.4s done
#21 exporting manifest list sha256:e7a1fba73ef6ec49666cadea6e978c240ee21ac8e3900f50e60df58e52b15c08
#21 exporting manifest list sha256:e7a1fba73ef6ec49666cadea6e978c240ee21ac8e3900f50e60df58e52b15c08 0.2s done
#21 naming to docker.io/library/mtusi-lab5-api:grade3
#21 naming to docker.io/library/mtusi-lab5-api:grade3 0.0s done
#21 unpacking to docker.io/library/mtusi-lab5-api:grade3 0.1s done
#21 ...
#22 [tools 7/11] COPY contracts/ ./contracts/
#22 DONE 0.6s
#21 [migrate] exporting to image
#21 DONE 2.2s
#23 [tools 8/11] COPY tests/ ./tests/
#23 DONE 1.7s
#24 [tools 9/11] COPY scripts/check_contract.py scripts/validate_openapi.py ./scripts/
#24 DONE 0.4s
#25 [migrate] resolving provenance for metadata file
#25 DONE 0.1s
#26 [tools 10/11] COPY --chmod=0755 scripts/restart.sh scripts/stop-one.sh scripts/lab-containers.sh ./scripts/
#26 DONE 0.2s
#27 [tools 11/11] COPY student-tests/ ./student-tests/
#27 DONE 0.3s
#28 [tools] exporting to image
#28 exporting layers 2.2s done
#28 exporting manifest sha256:9b69ca3ad724e9dc16e6823ef9a8521da23df23baa58b1e5b113ac1bcf6aa647 0.1s done
#28 exporting config sha256:2d901ea44174375fb550a94afae49ac24f9ac6ac13f99c17e15a60aab0c9d87e
#28 exporting config sha256:2d901ea44174375fb550a94afae49ac24f9ac6ac13f99c17e15a60aab0c9d87e 0.3s done
#28 exporting attestation manifest sha256:9589c586d191d066d5902dfc0010fa0bbf8e369b4c1a31bddd951badeeec8334
#28 exporting attestation manifest sha256:9589c586d191d066d5902dfc0010fa0bbf8e369b4c1a31bddd951badeeec8334 0.3s done
#28 exporting manifest list sha256:a180c7ff7857e9f71f68235ad884ffcb3c3342d7b8ffdeb86722241515370f1e
#28 exporting manifest list sha256:a180c7ff7857e9f71f68235ad884ffcb3c3342d7b8ffdeb86722241515370f1e 0.3s done
#28 naming to docker.io/library/mtusi-lab5-tools:grade3
#28 naming to docker.io/library/mtusi-lab5-tools:grade3 0.1s done
#28 unpacking to docker.io/library/mtusi-lab5-tools:grade3
#28 unpacking to docker.io/library/mtusi-lab5-tools:grade3 1.8s done
#28 DONE 6.2s
#29 [tools] resolving provenance for metadata file
#29 DONE 0.1s
Image mtusi-lab5-tools:grade3 Built
Image mtusi-lab5-api:grade3 Built
PS> docker compose --env-file .env up -d --wait --wait-timeout 150
Network mtusi-lab5-grade3-review-20261120_default Creating
Network mtusi-lab5-grade3-review-20261120_default Creating
Volume mtusi-lab5-grade3-review-20261120_s3_data Creating
Volume mtusi-lab5-grade3-review-20261120_s3_data Creating
Volume mtusi-lab5-grade3-review-20261120_broker_data Creating
Volume mtusi-lab5-grade3-review-20261120_broker_data Creating
Volume mtusi-lab5-grade3-review-20261120_postgres_data Creating
Volume mtusi-lab5-grade3-review-20261120_postgres_data Creating
Volume mtusi-lab5-grade3-review-20261120_postgres_data Created
Volume mtusi-lab5-grade3-review-20261120_postgres_data Created
Volume mtusi-lab5-grade3-review-20261120_s3_data Created
Volume mtusi-lab5-grade3-review-20261120_s3_data Created
Volume mtusi-lab5-grade3-review-20261120_broker_data Created
Volume mtusi-lab5-grade3-review-20261120_broker_data Created
Network mtusi-lab5-grade3-review-20261120_default Created
Network mtusi-lab5-grade3-review-20261120_default Created
Container mtusi-lab5-grade3-review-20261120-broker-1 Creating
Container mtusi-lab5-grade3-review-20261120-lb-1 Creating
Container mtusi-lab5-grade3-review-20261120-s3-1 Creating
Container mtusi-lab5-grade3-review-20261120-postgres-1 Creating
Container mtusi-lab5-grade3-review-20261120-s3-1 Created
Container mtusi-lab5-grade3-review-20261120-bootstrap-1 Creating
Container mtusi-lab5-grade3-review-20261120-broker-1 Created
Container mtusi-lab5-grade3-review-20261120-lb-1 Created
Container mtusi-lab5-grade3-review-20261120-postgres-1 Created
Container mtusi-lab5-grade3-review-20261120-migrate-1 Creating
Container mtusi-lab5-grade3-review-20261120-bootstrap-1 Created
Container mtusi-lab5-grade3-review-20261120-migrate-1 Created
Container mtusi-lab5-grade3-review-20261120-worker-1 Creating
Container mtusi-lab5-grade3-review-20261120-api2-1 Creating
Container mtusi-lab5-grade3-review-20261120-api1-1 Creating
Container mtusi-lab5-grade3-review-20261120-api2-1 Created
Container mtusi-lab5-grade3-review-20261120-api1-1 Created
Container mtusi-lab5-grade3-review-20261120-worker-1 Created
Container mtusi-lab5-grade3-review-20261120-broker-1 Starting
Container mtusi-lab5-grade3-review-20261120-lb-1 Starting
Container mtusi-lab5-grade3-review-20261120-postgres-1 Starting
Container mtusi-lab5-grade3-review-20261120-s3-1 Starting
Container mtusi-lab5-grade3-review-20261120-broker-1 Started
Container mtusi-lab5-grade3-review-20261120-broker-1 Waiting
Container mtusi-lab5-grade3-review-20261120-lb-1 Started
Container mtusi-lab5-grade3-review-20261120-postgres-1 Started
Container mtusi-lab5-grade3-review-20261120-postgres-1 Waiting
Container mtusi-lab5-grade3-review-20261120-s3-1 Started
Container mtusi-lab5-grade3-review-20261120-s3-1 Waiting
Container mtusi-lab5-grade3-review-20261120-s3-1 Healthy
Container mtusi-lab5-grade3-review-20261120-bootstrap-1 Starting
Container mtusi-lab5-grade3-review-20261120-bootstrap-1 Started
Container mtusi-lab5-grade3-review-20261120-bootstrap-1 Waiting
Container mtusi-lab5-grade3-review-20261120-postgres-1 Healthy
Container mtusi-lab5-grade3-review-20261120-migrate-1 Starting
Container mtusi-lab5-grade3-review-20261120-bootstrap-1 Exited
Container mtusi-lab5-grade3-review-20261120-migrate-1 Started
Container mtusi-lab5-grade3-review-20261120-migrate-1 Waiting
Container mtusi-lab5-grade3-review-20261120-migrate-1 Exited
Container mtusi-lab5-grade3-review-20261120-api2-1 Starting
Container mtusi-lab5-grade3-review-20261120-api1-1 Starting
Container mtusi-lab5-grade3-review-20261120-api2-1 Started
Container mtusi-lab5-grade3-review-20261120-api1-1 Started
Container mtusi-lab5-grade3-review-20261120-broker-1 Healthy
Container mtusi-lab5-grade3-review-20261120-worker-1 Starting
Container mtusi-lab5-grade3-review-20261120-worker-1 Started
Container mtusi-lab5-grade3-review-20261120-api1-1 Waiting
Container mtusi-lab5-grade3-review-20261120-api2-1 Waiting
Container mtusi-lab5-grade3-review-20261120-broker-1 Waiting
Container mtusi-lab5-grade3-review-20261120-bootstrap-1 Waiting
Container mtusi-lab5-grade3-review-20261120-lb-1 Waiting
Container mtusi-lab5-grade3-review-20261120-postgres-1 Waiting
Container mtusi-lab5-grade3-review-20261120-worker-1 Waiting
Container mtusi-lab5-grade3-review-20261120-migrate-1 Waiting
Container mtusi-lab5-grade3-review-20261120-s3-1 Waiting
Container mtusi-lab5-grade3-review-20261120-bootstrap-1 Exited
Container mtusi-lab5-grade3-review-20261120-migrate-1 Exited
Container mtusi-lab5-grade3-review-20261120-broker-1 Healthy
Container mtusi-lab5-grade3-review-20261120-s3-1 Healthy
Container mtusi-lab5-grade3-review-20261120-lb-1 Healthy
Container mtusi-lab5-grade3-review-20261120-api2-1 Healthy
Container mtusi-lab5-grade3-review-20261120-api1-1 Healthy
Container mtusi-lab5-grade3-review-20261120-worker-1 Healthy
Container mtusi-lab5-grade3-review-20261120-postgres-1 Healthy
PS> curl.exe -fsS -i http://127.0.0.1:18085/health/ready
HTTP/1.1 200 OK
date: Fri, 20 Nov 2026 16:07:23 GMT
server: uvicorn
content-length: 15
content-type: application/json
x-request-id: 88b585b1-5abf-4fcd-8eff-a073206f4844
x-instance-id: api2
{"status":"ok"}
PS> docker compose run --rm --no-deps tools make check validate test
Container mtusi-lab5-grade3-review-20261120-tools-run-302ea695e183 Creating
Container mtusi-lab5-grade3-review-20261120-tools-run-302ea695e183 Created
python3 scripts/check_contract.py
PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.
python3 scripts/validate_openapi.py
PASS: full OpenAPI 3.1 and JSON Schema validation; application NOT tested.
python3 tests/smoke.py --base-url "http://lb:8080"
PASS: lab5 public API scenario; infrastructure and grading evidence remain separate.
PS> docker compose down
Container mtusi-lab5-grade3-review-20261120-api2-1 Stopping
Container mtusi-lab5-grade3-review-20261120-lb-1 Stopping
Container mtusi-lab5-grade3-review-20261120-worker-1 Stopping
Container mtusi-lab5-grade3-review-20261120-api1-1 Stopping
Container mtusi-lab5-grade3-review-20261120-lb-1 Stopped
Container mtusi-lab5-grade3-review-20261120-lb-1 Removing
Container mtusi-lab5-grade3-review-20261120-lb-1 Removed
Container mtusi-lab5-grade3-review-20261120-api2-1 Stopped
Container mtusi-lab5-grade3-review-20261120-api2-1 Removing
Container mtusi-lab5-grade3-review-20261120-api2-1 Removed
Container mtusi-lab5-grade3-review-20261120-api1-1 Stopped
Container mtusi-lab5-grade3-review-20261120-api1-1 Removing
Container mtusi-lab5-grade3-review-20261120-api1-1 Removed
Container mtusi-lab5-grade3-review-20261120-worker-1 Stopped
Container mtusi-lab5-grade3-review-20261120-worker-1 Removing
Container mtusi-lab5-grade3-review-20261120-worker-1 Removed
Container mtusi-lab5-grade3-review-20261120-bootstrap-1 Stopping
Container mtusi-lab5-grade3-review-20261120-broker-1 Stopping
Container mtusi-lab5-grade3-review-20261120-migrate-1 Stopping
Container mtusi-lab5-grade3-review-20261120-migrate-1 Stopped
Container mtusi-lab5-grade3-review-20261120-migrate-1 Removing
Container mtusi-lab5-grade3-review-20261120-bootstrap-1 Stopped
Container mtusi-lab5-grade3-review-20261120-bootstrap-1 Removing
Container mtusi-lab5-grade3-review-20261120-bootstrap-1 Removed
Container mtusi-lab5-grade3-review-20261120-s3-1 Stopping
Container mtusi-lab5-grade3-review-20261120-migrate-1 Removed
Container mtusi-lab5-grade3-review-20261120-postgres-1 Stopping
Container mtusi-lab5-grade3-review-20261120-postgres-1 Stopped
Container mtusi-lab5-grade3-review-20261120-postgres-1 Removing
Container mtusi-lab5-grade3-review-20261120-postgres-1 Removed
Container mtusi-lab5-grade3-review-20261120-broker-1 Stopped
Container mtusi-lab5-grade3-review-20261120-broker-1 Removing
Container mtusi-lab5-grade3-review-20261120-s3-1 Stopped
Container mtusi-lab5-grade3-review-20261120-s3-1 Removing
Container mtusi-lab5-grade3-review-20261120-broker-1 Removed
Container mtusi-lab5-grade3-review-20261120-s3-1 Removed
Network mtusi-lab5-grade3-review-20261120_default Removing
Network mtusi-lab5-grade3-review-20261120_default Removed
PS> docker volume ls -q --filter <review project label>
mtusi-lab5-grade3-review-20261120_broker_data
mtusi-lab5-grade3-review-20261120_postgres_data
mtusi-lab5-grade3-review-20261120_s3_data
2026-11-20T19:07:45.9725744+03:00
+21
View File
@@ -0,0 +1,21 @@
2026-11-20T18:52:37.4175271+03:00
PS> docker version --format ...
Client=29.9.3 Server=29.9.3 OS=linux
PS> docker compose version
Docker Compose version v5.7.0
PS> docker info --format ...
NCPU=4 MemTotal=8242380800
PS> docker ps --format ... (existing labs)
mtusi-lab3-grade3-api2-1 8080/tcp
mtusi-lab3-grade3-api1-1 8080/tcp
mtusi-lab3-grade3-postgres-1 5432/tcp
mtusi-lab3-grade3-lb-1 127.0.0.1:8080->8080/tcp
mtusi-lab4-grade3-api2-1 8080/tcp
mtusi-lab4-grade3-api1-1 8080/tcp
mtusi-lab4-grade3-s3-1 9000-9001/tcp
mtusi-lab4-grade3-postgres-1 5432/tcp
mtusi-lab4-grade3-lb-1 127.0.0.1:8084->8080/tcp
PS> docker volume ls --format ... (existing labs)
mtusi-lab3-grade3_postgres_data
mtusi-lab4-grade3_postgres_data
mtusi-lab4-grade3_s3_data
+11
View File
@@ -0,0 +1,11 @@
2026-11-20T18:58:26.7192993+03:00
Run 1
PS> docker compose run --rm --no-deps tools make test-failover ACTION_SCRIPT=scripts/stop-one.sh
Container mtusi-lab5-grade3-tools-run-c6ea2334065d Creating
Container mtusi-lab5-grade3-tools-run-c6ea2334065d Created
python3 tests/resilience.py --mode failover --action-script "scripts/stop-one.sh" --base-url "http://lb:8080"
FAIL: POST /api/v1/chats/8a88176c-35ac-4877-b966-d2766e4b6c73/attachments: expected [202], received 503
make: *** [Makefile:30: test-failover] Error 1
/mtusi-lab5-grade3-api1-1 id=4996fc725084706ef645e554bdc3fda0c5ff6798b55b357ac8be3af364cc3042 state=running finished=2026-11-20T15:58:16.447625327Z
/mtusi-lab5-grade3-api2-1 id=413b207f215f54f30477b2e672940dc30867f4b7bc4d383af4948e7ae7c39c0f state=running finished=2026-11-20T15:58:18.595409421Z
+56
View File
@@ -0,0 +1,56 @@
2026-11-20T19:01:35.9035577+03:00
Run 1
PS> docker compose run --rm --no-deps tools make test-failover ACTION_SCRIPT=scripts/stop-one.sh
Container mtusi-lab5-grade3-tools-run-3420ada0e362 Creating
Container mtusi-lab5-grade3-tools-run-3420ada0e362 Created
python3 tests/resilience.py --mode failover --action-script "scripts/stop-one.sh" --base-url "http://lb:8080"
Executing explicit failover hook; hook output suppressed to avoid leaking secrets.
{"observed_instances_before": 2, "successful_writes": 98, "failed_attempts": 0, "first_recovery_seconds": 2.089}
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
/mtusi-lab5-grade3-api1-1 id=4996fc725084706ef645e554bdc3fda0c5ff6798b55b357ac8be3af364cc3042 state=running finished=2026-11-20T16:01:17.87099914Z
/mtusi-lab5-grade3-api2-1 id=413b207f215f54f30477b2e672940dc30867f4b7bc4d383af4948e7ae7c39c0f state=exited finished=2026-11-20T16:01:44.419893971Z
PS> docker compose start api2
Container mtusi-lab5-grade3-postgres-1 Waiting
Container mtusi-lab5-grade3-s3-1 Waiting
Container mtusi-lab5-grade3-s3-1 Healthy
Container mtusi-lab5-grade3-bootstrap-1 Starting
Container mtusi-lab5-grade3-postgres-1 Healthy
Container mtusi-lab5-grade3-migrate-1 Starting
Container mtusi-lab5-grade3-bootstrap-1 Started
Container mtusi-lab5-grade3-bootstrap-1 Waiting
Container mtusi-lab5-grade3-migrate-1 Started
Container mtusi-lab5-grade3-migrate-1 Waiting
Container mtusi-lab5-grade3-migrate-1 Exited
Container mtusi-lab5-grade3-bootstrap-1 Exited
Container mtusi-lab5-grade3-api2-1 Starting
Container mtusi-lab5-grade3-api2-1 Started
Restored api2 health=healthy
2026-11-20T19:02:15.1173667+03:00
Run 2
PS> docker compose run --rm --no-deps tools make test-failover ACTION_SCRIPT=scripts/stop-one.sh
Container mtusi-lab5-grade3-tools-run-75e4f022f605 Creating
Container mtusi-lab5-grade3-tools-run-75e4f022f605 Created
python3 tests/resilience.py --mode failover --action-script "scripts/stop-one.sh" --base-url "http://lb:8080"
Executing explicit failover hook; hook output suppressed to avoid leaking secrets.
{"observed_instances_before": 2, "successful_writes": 116, "failed_attempts": 0, "first_recovery_seconds": 1.863}
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
/mtusi-lab5-grade3-api1-1 id=4996fc725084706ef645e554bdc3fda0c5ff6798b55b357ac8be3af364cc3042 state=exited finished=2026-11-20T16:02:19.742212537Z
/mtusi-lab5-grade3-api2-1 id=413b207f215f54f30477b2e672940dc30867f4b7bc4d383af4948e7ae7c39c0f state=running finished=2026-11-20T16:01:44.419893971Z
PS> docker compose start api1
Container mtusi-lab5-grade3-s3-1 Waiting
Container mtusi-lab5-grade3-postgres-1 Waiting
Container mtusi-lab5-grade3-s3-1 Healthy
Container mtusi-lab5-grade3-bootstrap-1 Starting
Container mtusi-lab5-grade3-postgres-1 Healthy
Container mtusi-lab5-grade3-migrate-1 Starting
Container mtusi-lab5-grade3-bootstrap-1 Started
Container mtusi-lab5-grade3-bootstrap-1 Waiting
Container mtusi-lab5-grade3-migrate-1 Started
Container mtusi-lab5-grade3-migrate-1 Waiting
Container mtusi-lab5-grade3-migrate-1 Exited
Container mtusi-lab5-grade3-bootstrap-1 Exited
Container mtusi-lab5-grade3-api1-1 Starting
Container mtusi-lab5-grade3-api1-1 Started
Restored api1 health=healthy
Covered stopped replicas: api1, api2
2026-11-20T19:02:47.0267819+03:00
+93
View File
@@ -0,0 +1,93 @@
2026-11-20T19:08:30.3345331+03:00
PS> git status -sb
## solution/bvt2301-gamayunov
M .dockerignore
M .env.example
M .gitignore
M REPORT.md
?? .gitattributes
?? Dockerfile
?? compose.yaml
?? evidence/grade3/
?? infra/Dockerfile.tools
?? infra/haproxy.cfg
?? infra/requirements-tools.lock
?? migrations/
?? requirements.lock
?? requirements.txt
?? scripts/init-env.ps1
?? scripts/lab-containers.sh
?? scripts/restart.sh
?? scripts/stop-one.sh
?? src/
?? student-tests/
PS> git diff --check
warning: in the working copy of '.dockerignore', LF will be replaced by CRLF the next time Git touches it
warning: in the working copy of '.env.example', LF will be replaced by CRLF the next time Git touches it
warning: in the working copy of '.gitignore', LF will be replaced by CRLF the next time Git touches it
warning: in the working copy of 'REPORT.md', LF will be replaced by CRLF the next time Git touches it
exit=0
PS> git diff --cached --name-only
Git index is empty.
PS> git diff --exit-code HEAD -- (teacher files and published checks)
exit=0
PS> git status --short --untracked-files=all
Modified and new files eligible for staging:
M .dockerignore
M .env.example
M .gitignore
M REPORT.md
?? .gitattributes
?? Dockerfile
?? compose.yaml
?? evidence/grade3/README.md
?? evidence/grade3/build-and-early-smoke.txt
?? evidence/grade3/checks.txt
?? evidence/grade3/clean-reproduction.txt
?? evidence/grade3/environment.txt
?? evidence/grade3/failover-early.txt
?? evidence/grade3/failover.txt
?? evidence/grade3/git.txt
?? evidence/grade3/persistence-initial.txt
?? evidence/grade3/persistence.txt
?? evidence/grade3/startup-initial.txt
?? evidence/grade3/startup.txt
?? evidence/grade3/student.txt
?? evidence/grade3/worker.txt
?? infra/Dockerfile.tools
?? infra/haproxy.cfg
?? infra/requirements-tools.lock
?? migrations/001_initial.sql
?? migrations/002_message_idempotency.sql
?? migrations/003_attachments.sql
?? migrations/004_attachment_jobs.sql
?? requirements.lock
?? requirements.txt
?? scripts/init-env.ps1
?? scripts/lab-containers.sh
?? scripts/restart.sh
?? scripts/stop-one.sh
?? src/__init__.py
?? src/attachments.py
?? src/config.py
?? src/db.py
?? src/errors.py
?? src/jobs.py
?? src/main.py
?? src/models.py
?? src/pagination.py
?? src/s3.py
?? src/security.py
?? src/store.py
?? src/thumbnail.py
?? src/worker.py
?? student-tests/README.md
?? student-tests/grade3.py
PS> git check-ignore .env secrets/ PR_GRADE3_DRAFT.md dist/lab5-grade3-review.zip
.env
secrets/
PR_GRADE3_DRAFT.md
dist/lab5-grade3-review.zip
PS> curl.exe -fsS http://127.0.0.1:8085/health/ready
{"status":"ok"}
2026-11-20T19:08:31.0561230+03:00
+24
View File
@@ -0,0 +1,24 @@
2026-11-20T18:57:58.5752878+03:00
PS> docker inspect ... (before persistence)
/mtusi-lab5-grade3-api1-1 started=2026-11-20T15:56:33.725735351Z state=running
/mtusi-lab5-grade3-api2-1 started=2026-11-20T15:56:33.218626831Z state=running
/mtusi-lab5-grade3-broker-1 started=2026-11-20T15:56:07.794042633Z state=running volume=mtusi-lab5-grade3_broker_data
/mtusi-lab5-grade3-lb-1 started=2026-11-20T15:56:11.955927736Z state=running
/mtusi-lab5-grade3-postgres-1 started=2026-11-20T15:53:27.139250265Z state=running volume=mtusi-lab5-grade3_postgres_data
/mtusi-lab5-grade3-s3-1 started=2026-11-20T15:53:24.604658124Z state=running volume=mtusi-lab5-grade3_s3_data
/mtusi-lab5-grade3-worker-1 started=2026-11-20T15:56:50.702171974Z state=running
PS> docker compose run --rm --no-deps tools make test-persistence ACTION_SCRIPT=scripts/restart.sh
Container mtusi-lab5-grade3-tools-run-9cd7ccf616b6 Creating
Container mtusi-lab5-grade3-tools-run-9cd7ccf616b6 Created
python3 tests/resilience.py --mode persistence --action-script "scripts/restart.sh" --base-url "http://lb:8080"
Executing explicit persistence hook; hook output suppressed to avoid leaking secrets.
PASS: persistence preserved confirmed data and existing session. Hook scope needs operator evidence.
PS> docker inspect ... (after persistence)
/mtusi-lab5-grade3-api1-1 started=2026-11-20T15:58:16.841393519Z state=running
/mtusi-lab5-grade3-api2-1 started=2026-11-20T15:58:18.98534691Z state=running
/mtusi-lab5-grade3-broker-1 started=2026-11-20T15:58:14.442587438Z state=running volume=mtusi-lab5-grade3_broker_data
/mtusi-lab5-grade3-lb-1 started=2026-11-20T15:58:20.689150902Z state=running
/mtusi-lab5-grade3-postgres-1 started=2026-11-20T15:58:10.112911153Z state=running volume=mtusi-lab5-grade3_postgres_data
/mtusi-lab5-grade3-s3-1 started=2026-11-20T15:58:12.509192703Z state=running volume=mtusi-lab5-grade3_s3_data
/mtusi-lab5-grade3-worker-1 started=2026-11-20T15:58:21.622295267Z state=running
2026-11-20T18:58:26.6539850+03:00
+24
View File
@@ -0,0 +1,24 @@
2026-11-20T19:00:56.3535184+03:00
PS> docker inspect ... (before persistence)
/mtusi-lab5-grade3-api1-1 id=4996fc725084706ef645e554bdc3fda0c5ff6798b55b357ac8be3af364cc3042 started=2026-11-20T15:58:16.841393519Z state=running
/mtusi-lab5-grade3-api2-1 id=413b207f215f54f30477b2e672940dc30867f4b7bc4d383af4948e7ae7c39c0f started=2026-11-20T15:58:18.98534691Z state=running
/mtusi-lab5-grade3-broker-1 id=ab4a43edf14a8270449e7343d760613a2b7d9aaf1fe11e1745126c5527f390e6 started=2026-11-20T15:58:14.442587438Z state=running volume=mtusi-lab5-grade3_broker_data
/mtusi-lab5-grade3-lb-1 id=6ebe3cf6c8d5d9200655f5509ab7b01da84755eb1591cc85b4b643fe580532a5 started=2026-11-20T15:58:20.689150902Z state=running
/mtusi-lab5-grade3-postgres-1 id=3651a5577643e7b359f03250b6e6949ee5a8a06fcfb77f0c2c8dfcf0eb588e32 started=2026-11-20T15:58:10.112911153Z state=running volume=mtusi-lab5-grade3_postgres_data
/mtusi-lab5-grade3-s3-1 id=39235ab07c3f2c642b5c0cecd9a3a026ab0a7ccf5a426e98379302fe8d0fc6cf started=2026-11-20T15:58:12.509192703Z state=running volume=mtusi-lab5-grade3_s3_data
/mtusi-lab5-grade3-worker-1 id=c3377a8231eeb4150c513577714cb1d25efe9b454594d666bab846946bf7b505 started=2026-11-20T15:58:21.622295267Z state=running
PS> docker compose run --rm --no-deps tools make test-persistence ACTION_SCRIPT=scripts/restart.sh
Container mtusi-lab5-grade3-tools-run-015fbb76f95e Creating
Container mtusi-lab5-grade3-tools-run-015fbb76f95e Created
python3 tests/resilience.py --mode persistence --action-script "scripts/restart.sh" --base-url "http://lb:8080"
Executing explicit persistence hook; hook output suppressed to avoid leaking secrets.
PASS: persistence preserved confirmed data and existing session. Hook scope needs operator evidence.
PS> docker inspect ... (after persistence)
/mtusi-lab5-grade3-api1-1 id=4996fc725084706ef645e554bdc3fda0c5ff6798b55b357ac8be3af364cc3042 started=2026-11-20T16:01:18.081246421Z state=running
/mtusi-lab5-grade3-api2-1 id=413b207f215f54f30477b2e672940dc30867f4b7bc4d383af4948e7ae7c39c0f started=2026-11-20T16:01:19.314559585Z state=running
/mtusi-lab5-grade3-broker-1 id=ab4a43edf14a8270449e7343d760613a2b7d9aaf1fe11e1745126c5527f390e6 started=2026-11-20T16:01:16.448104113Z state=running volume=mtusi-lab5-grade3_broker_data
/mtusi-lab5-grade3-lb-1 id=6ebe3cf6c8d5d9200655f5509ab7b01da84755eb1591cc85b4b643fe580532a5 started=2026-11-20T16:01:22.844314329Z state=running
/mtusi-lab5-grade3-postgres-1 id=3651a5577643e7b359f03250b6e6949ee5a8a06fcfb77f0c2c8dfcf0eb588e32 started=2026-11-20T16:01:11.993585569Z state=running volume=mtusi-lab5-grade3_postgres_data
/mtusi-lab5-grade3-s3-1 id=39235ab07c3f2c642b5c0cecd9a3a026ab0a7ccf5a426e98379302fe8d0fc6cf started=2026-11-20T16:01:14.052049209Z state=running volume=mtusi-lab5-grade3_s3_data
/mtusi-lab5-grade3-worker-1 id=c3377a8231eeb4150c513577714cb1d25efe9b454594d666bab846946bf7b505 started=2026-11-20T16:01:33.558869996Z state=running
2026-11-20T19:01:35.8683544+03:00
+63
View File
@@ -0,0 +1,63 @@
2026-11-20T18:52:48.8918342+03:00
PS> docker compose --env-file .env config --quiet
PS> docker compose --env-file .env up -d --no-build --wait --wait-timeout 90
Network mtusi-lab5-grade3_default Creating
Volume mtusi-lab5-grade3_broker_data Creating
Volume mtusi-lab5-grade3_s3_data Creating
Volume mtusi-lab5-grade3_postgres_data Creating
Volume mtusi-lab5-grade3_postgres_data Creating
Network mtusi-lab5-grade3_default Creating
Volume mtusi-lab5-grade3_s3_data Creating
Volume mtusi-lab5-grade3_broker_data Creating
Volume mtusi-lab5-grade3_s3_data Created
Volume mtusi-lab5-grade3_s3_data Created
Volume mtusi-lab5-grade3_broker_data Created
Volume mtusi-lab5-grade3_broker_data Created
Volume mtusi-lab5-grade3_postgres_data Created
Volume mtusi-lab5-grade3_postgres_data Created
Network mtusi-lab5-grade3_default Created
Network mtusi-lab5-grade3_default Created
Container mtusi-lab5-grade3-lb-1 Creating
Container mtusi-lab5-grade3-s3-1 Creating
Container mtusi-lab5-grade3-broker-1 Creating
Container mtusi-lab5-grade3-postgres-1 Creating
Container mtusi-lab5-grade3-lb-1 Created
Container mtusi-lab5-grade3-s3-1 Created
Container mtusi-lab5-grade3-bootstrap-1 Creating
Container mtusi-lab5-grade3-broker-1 Created
Container mtusi-lab5-grade3-postgres-1 Created
Container mtusi-lab5-grade3-migrate-1 Creating
Container mtusi-lab5-grade3-migrate-1 Created
Container mtusi-lab5-grade3-bootstrap-1 Created
Container mtusi-lab5-grade3-api1-1 Creating
Container mtusi-lab5-grade3-api2-1 Creating
Container mtusi-lab5-grade3-worker-1 Creating
Container mtusi-lab5-grade3-api1-1 Created
Container mtusi-lab5-grade3-api2-1 Created
Container mtusi-lab5-grade3-worker-1 Created
Container mtusi-lab5-grade3-broker-1 Starting
Container mtusi-lab5-grade3-lb-1 Starting
Container mtusi-lab5-grade3-postgres-1 Starting
Container mtusi-lab5-grade3-s3-1 Starting
Container mtusi-lab5-grade3-broker-1 Started
Container mtusi-lab5-grade3-broker-1 Waiting
Container mtusi-lab5-grade3-s3-1 Started
Container mtusi-lab5-grade3-s3-1 Waiting
Container mtusi-lab5-grade3-postgres-1 Started
Container mtusi-lab5-grade3-postgres-1 Waiting
Container mtusi-lab5-grade3-s3-1 Healthy
Container mtusi-lab5-grade3-bootstrap-1 Starting
Container mtusi-lab5-grade3-lb-1 Started
Container mtusi-lab5-grade3-bootstrap-1 Started
Container mtusi-lab5-grade3-bootstrap-1 Waiting
Container mtusi-lab5-grade3-bootstrap-1 Exited
Container mtusi-lab5-grade3-postgres-1 Healthy
Container mtusi-lab5-grade3-migrate-1 Starting
Container mtusi-lab5-grade3-migrate-1 Started
Container mtusi-lab5-grade3-migrate-1 Waiting
Container mtusi-lab5-grade3-broker-1 Error dependency broker failed to start
Container mtusi-lab5-grade3-api2-1 not started: condition "service_completed_successfully" of service "migrate" not reached: interrupted by the unmet condition "service_healthy" of service "broker"
Container mtusi-lab5-grade3-api1-1 not started: condition "service_completed_successfully" of service "migrate" not reached: interrupted by the unmet condition "service_healthy" of service "broker"
Container mtusi-lab5-grade3-worker-1 not started: condition "service_healthy" of service "broker" not met
Container mtusi-lab5-grade3-migrate-1 Skipped: condition "service_completed_successfully" of service "migrate" not reached: interrupted by the unmet condition "service_healthy" of service "broker"
dependency failed to start: container mtusi-lab5-grade3-broker-1 is unhealthy
+83
View File
@@ -0,0 +1,83 @@
2026-11-20T18:55:52.3556172+03:00
PS> docker compose --env-file .env config --quiet
PS> docker compose --env-file .env up -d --no-build --wait --wait-timeout 150
Container mtusi-lab5-grade3-s3-1 Running
Container mtusi-lab5-grade3-postgres-1 Running
Container mtusi-lab5-grade3-broker-1 Recreate
Container mtusi-lab5-grade3-lb-1 Recreate
Container mtusi-lab5-grade3-lb-1 Recreated
Container mtusi-lab5-grade3-broker-1 Recreated
Container mtusi-lab5-grade3-broker-1 Starting
Container mtusi-lab5-grade3-lb-1 Starting
Container mtusi-lab5-grade3-postgres-1 Waiting
Container mtusi-lab5-grade3-s3-1 Waiting
Container mtusi-lab5-grade3-s3-1 Healthy
Container mtusi-lab5-grade3-bootstrap-1 Starting
Container mtusi-lab5-grade3-postgres-1 Healthy
Container mtusi-lab5-grade3-migrate-1 Starting
Container mtusi-lab5-grade3-broker-1 Started
Container mtusi-lab5-grade3-broker-1 Waiting
Container mtusi-lab5-grade3-lb-1 Started
Container mtusi-lab5-grade3-bootstrap-1 Started
Container mtusi-lab5-grade3-bootstrap-1 Waiting
Container mtusi-lab5-grade3-migrate-1 Started
Container mtusi-lab5-grade3-migrate-1 Waiting
Container mtusi-lab5-grade3-migrate-1 Exited
Container mtusi-lab5-grade3-bootstrap-1 Exited
Container mtusi-lab5-grade3-api2-1 Starting
Container mtusi-lab5-grade3-api1-1 Starting
Container mtusi-lab5-grade3-api2-1 Started
Container mtusi-lab5-grade3-api1-1 Started
Container mtusi-lab5-grade3-broker-1 Healthy
Container mtusi-lab5-grade3-worker-1 Starting
Container mtusi-lab5-grade3-worker-1 Started
Container mtusi-lab5-grade3-migrate-1 Waiting
Container mtusi-lab5-grade3-broker-1 Waiting
Container mtusi-lab5-grade3-s3-1 Waiting
Container mtusi-lab5-grade3-postgres-1 Waiting
Container mtusi-lab5-grade3-lb-1 Waiting
Container mtusi-lab5-grade3-bootstrap-1 Waiting
Container mtusi-lab5-grade3-worker-1 Waiting
Container mtusi-lab5-grade3-api2-1 Waiting
Container mtusi-lab5-grade3-api1-1 Waiting
Container mtusi-lab5-grade3-migrate-1 Exited
Container mtusi-lab5-grade3-bootstrap-1 Exited
Container mtusi-lab5-grade3-lb-1 Healthy
Container mtusi-lab5-grade3-worker-1 Healthy
Container mtusi-lab5-grade3-postgres-1 Healthy
Container mtusi-lab5-grade3-s3-1 Healthy
Container mtusi-lab5-grade3-api1-1 Healthy
Container mtusi-lab5-grade3-api2-1 Healthy
Container mtusi-lab5-grade3-broker-1 Healthy
PS> docker compose ps -a
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
mtusi-lab5-grade3-api1-1 sha256:b62e53a94c49d16d79bf0ed20a0cf3f6962eadc669898f6a1756dcd6abe3b928 "sh -c 'exec uvicorn…" api1 3 minutes ago Up 20 seconds (healthy) 8080/tcp
mtusi-lab5-grade3-api2-1 sha256:b62e53a94c49d16d79bf0ed20a0cf3f6962eadc669898f6a1756dcd6abe3b928 "sh -c 'exec uvicorn…" api2 3 minutes ago Up 21 seconds (healthy) 8080/tcp
mtusi-lab5-grade3-bootstrap-1 sha256:b62e53a94c49d16d79bf0ed20a0cf3f6962eadc669898f6a1756dcd6abe3b928 "python -m src.s3" bootstrap 3 minutes ago Exited (0) 21 seconds ago
mtusi-lab5-grade3-broker-1 rabbitmq:4.1.4-alpine "sh -ec 'export RABB…" broker 56 seconds ago Up 46 seconds (healthy) 4369/tcp, 5671-5672/tcp, 15691-15692/tcp, 25672/tcp
mtusi-lab5-grade3-lb-1 haproxy:3.0.11-alpine "docker-entrypoint.s…" lb 57 seconds ago Up 42 seconds (healthy) 127.0.0.1:8085->8080/tcp
mtusi-lab5-grade3-migrate-1 sha256:b62e53a94c49d16d79bf0ed20a0cf3f6962eadc669898f6a1756dcd6abe3b928 "python -m src.db" migrate 3 minutes ago Exited (0) 23 seconds ago
mtusi-lab5-grade3-postgres-1 postgres:16.8-bookworm "docker-entrypoint.s…" postgres 3 minutes ago Up 3 minutes (healthy) 5432/tcp
mtusi-lab5-grade3-s3-1 rustfs/rustfs:1.0.0-alpha.85@sha256:23ff2af1ec2d64e0fb9608a1c39b5eb96fcff3a8efecf332986c948b0e3e4f8a "sh -ec 'unset RUSTF…" s3 4 minutes ago Up 3 minutes (healthy) 9000-9001/tcp
mtusi-lab5-grade3-worker-1 sha256:b62e53a94c49d16d79bf0ed20a0cf3f6962eadc669898f6a1756dcd6abe3b928 "python -m src.worker" worker 3 minutes ago Up 3 seconds 8080/tcp
PS> docker compose logs --no-color migrate bootstrap worker
migrate-1 | PASS: versioned migrations applied (already applied versions skipped)
migrate-1 | PASS: versioned migrations applied (already applied versions skipped)
bootstrap-1 | PASS: private attachment bucket bootstrapped
bootstrap-1 | PASS: private attachment bucket bootstrapped
PS> curl.exe -fsS -i http://127.0.0.1:8085/health/ready
HTTP/1.1 200 OK
date: Fri, 20 Nov 2026 15:56:58 GMT
server: uvicorn
content-length: 15
content-type: application/json
x-request-id: b6376ffe-3ac6-42b5-85d6-71b9b5d22622
x-instance-id: api1
{"status":"ok"}
PS> docker compose exec -T postgres psql -XAt -U messenger -d messenger -c "SELECT version FROM schema_migrations ORDER BY version"
001_initial.sql
002_message_idempotency.sql
003_attachments.sql
004_attachment_jobs.sql
2026-11-20T18:57:00.8458798+03:00
+83
View File
@@ -0,0 +1,83 @@
2026-11-20T19:02:47.0621819+03:00
PS> docker compose run --rm --no-deps tools python student-tests/grade3.py
Container mtusi-lab5-grade3-tools-run-2cd33497c352 Creating
Container mtusi-lab5-grade3-tools-run-2cd33497c352 Created
LB ready after 0.037s
PASS Argon2id verifier confirmed without reading hash
docker stop worker containers=c3377a8231ee
RabbitMQ persistent carrier schema valid job=77c585c8-1f45-4d50-a5d8-45a1f8c013f6 attempt=1 fields=attachment_id,attempt,created_at,job_id,pipeline_version,schema_version,type
private RustFS object unsigned GET HTTP 403
PASS stopped worker: uploads=202 ids=55fab679-9499-4063-8c01-1592f5e3a2ac,a13af220-d5a2-443e-9857-9924cc59a812; queued content=409; linked and ordinary POST=201; concurrent same-key response identical via api1/api2
docker restart api1 api2 containers=4996fc725084,413b207f215f
LB ready after 3.197s
api1 started before=2026-11-20T16:02:40.858740406Z after=2026-11-20T16:03:01.947238391Z
api2 started before=2026-11-20T16:02:08.921034559Z after=2026-11-20T16:03:02.835520972Z
broker queue=attachment.process.v1 durable=true ready=2 unacked=0
docker restart broker containers=ab4a43edf14a
broker healthy after 16.607s
broker started before=2026-11-20T16:01:16.448104113Z after=2026-11-20T16:03:11.348310151Z volume=mtusi-lab5-grade3_broker_data
broker queue=attachment.process.v1 durable=true ready=2 unacked=0
PASS both APIs + RabbitMQ restarted with queued work and unchanged RabbitMQ volume
docker start worker containers=c3377a8231ee
HTTP processing state after 2.974s
PASS processing metadata visible, content=409, linking=201 before final SQL commit
ready id=55fab679-9499-4063-8c01-1592f5e3a2ac wait=0.181s original_sha256=99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75 jpeg=256x160 thumb_sha256=b8e59b0cad429b1c49c27c4a1c79e52bfc23e2fab7c7238d18d5addd4df4b546 read=api2/api1
ready id=a13af220-d5a2-443e-9857-9924cc59a812 wait=0.155s original_sha256=99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75 jpeg=256x160 thumb_sha256=b8e59b0cad429b1c49c27c4a1c79e52bfc23e2fab7c7238d18d5addd4df4b546 read=api1/api2
worker acknowledged two ready duplicates after 0.053s
PASS duplicate delivery: ready snapshot unchanged, one SQL row and one S3 variant
private RustFS object unsigned GET HTTP 403
docker start worker containers=c3377a8231ee
negative upload wrong-mime HTTP 415
negative upload corrupt HTTP 415
negative upload oversize HTTP 413
negative upload private HTTP 404
negative upload anonymous HTTP 401
negative upload invalid%2Fname.png HTTP 400
negative upload 40,005,000 decoded pixels HTTP 413
negative truncated JPEG HTTP 415
ready id=2c61975d-3c8a-4f84-9c16-1afa6a74c28f wait=0.381s original_sha256=91dad4332c3984d2f019280575a82fac7a0d7f859cf57e860814a6f786622a71 jpeg=81x256 thumb_sha256=edd37f18bfa4ad50788f2ae8c8bf97473d433e326531ca92e942c357af90770f read=api1/api2
ready id=70ef8801-e722-48b6-ac56-2fb26e1dd176 wait=0.371s original_sha256=b15af0b56245cde806601ecff6e8f7fa1df12e7b6cd55530d356fc01a08c0aaf jpeg=37x19 thumb_sha256=8db1b1571a2b9333f7fe23f2a825770cdf9a18e6166a3fa08937fd786d6c50d8 read=api2/api1
PASS true JPEG, EXIF orientation, floor rounding, no upscale, alpha pixel=(255, 255, 255)
docker stop worker containers=c3377a8231ee
docker stop s3 containers=39235ab07c3f
docker start worker containers=c3377a8231ee
persistent retry after real S3 stop after 7.544s
storage outage SQL status|attempt|reason|max=queued|2|storage_unavailable|5
docker start s3 worker containers=39235ab07c3f,c3377a8231ee
LB ready after 1.644s
ready id=4094085a-efc6-4f68-9af0-a355e714aee9 wait=0.775s original_sha256=99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75 jpeg=256x160 thumb_sha256=b8e59b0cad429b1c49c27c4a1c79e52bfc23e2fab7c7238d18d5addd4df4b546 read=api2/api1
PASS temporary S3 outage recovered the same attachment
docker stop worker containers=c3377a8231ee
docker start worker containers=c3377a8231ee
failed 3d0244f4-47e0-44d4-a4ad-7acd064642fa after 3.665s
PASS safe failure code=invalid_image; status|attempt|reason|max=failed|1|invalid_image|5; content/link HTTP 409
failed fbb4c4c1-a9c0-4e4f-9936-79800f5915f7 after 11.594s
PASS safe failure code=retry_exhausted; status|attempt|reason|max=failed|5|storage_unavailable|5; content/link HTTP 409
docker stop broker containers=ab4a43edf14a
PASS unavailable broker upload=503 (no false 202), ordinary message=201
docker start broker containers=ab4a43edf14a
broker healthy after failure case after 15.616s
docker stop api1 containers=4996fc725084
api2 handles existing Bearer with api1 stopped after 0.011s
ready id=716795d6-9d70-41a4-beec-78438fbfc02f wait=1.065s original_sha256=99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75 jpeg=256x160 thumb_sha256=b8e59b0cad429b1c49c27c4a1c79e52bfc23e2fab7c7238d18d5addd4df4b546 read=api2/api2
docker start api1 containers=4996fc725084
api1 container healthy after 5.905s
docker stop api2 containers=413b207f215f
api1 handles existing Bearer with api2 stopped after 0.015s
ready id=e2cb5a62-673b-4602-8d67-4e4cd2623f62 wait=1.583s original_sha256=99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75 jpeg=256x160 thumb_sha256=b8e59b0cad429b1c49c27c4a1c79e52bfc23e2fab7c7238d18d5addd4df4b546 read=api1/api1
docker start api2 containers=413b207f215f
api2 container healthy after 5.826s
PASS authorized new uploads and thumbnail reads with either replica stopped
broker queue=attachment.process.v1 durable=true ready=0 unacked=0
PASS Lab 5 grade 3 Docker scenarios
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
mtusi-lab5-grade3-api1-1 sha256:b62e53a94c49d16d79bf0ed20a0cf3f6962eadc669898f6a1756dcd6abe3b928 "sh -c 'exec uvicorn…" api1 11 minutes ago Up 18 seconds (healthy) 8080/tcp
mtusi-lab5-grade3-api2-1 sha256:b62e53a94c49d16d79bf0ed20a0cf3f6962eadc669898f6a1756dcd6abe3b928 "sh -c 'exec uvicorn…" api2 11 minutes ago Up 9 seconds (healthy) 8080/tcp
mtusi-lab5-grade3-bootstrap-1 sha256:b62e53a94c49d16d79bf0ed20a0cf3f6962eadc669898f6a1756dcd6abe3b928 "python -m src.s3" bootstrap 12 minutes ago Exited (0) 2 minutes ago
mtusi-lab5-grade3-broker-1 rabbitmq:4.1.4-alpine "sh -ec 'export RABB…" broker 9 minutes ago Up 37 seconds (healthy) 4369/tcp, 5671-5672/tcp, 15691-15692/tcp, 25672/tcp
mtusi-lab5-grade3-lb-1 haproxy:3.0.11-alpine "docker-entrypoint.s…" lb 9 minutes ago Up 3 minutes (healthy) 127.0.0.1:8085->8080/tcp
mtusi-lab5-grade3-migrate-1 sha256:b62e53a94c49d16d79bf0ed20a0cf3f6962eadc669898f6a1756dcd6abe3b928 "python -m src.db" migrate 12 minutes ago Exited (0) 2 minutes ago
mtusi-lab5-grade3-postgres-1 postgres:16.8-bookworm "docker-entrypoint.s…" postgres 12 minutes ago Up 3 minutes (healthy) 5432/tcp
mtusi-lab5-grade3-s3-1 rustfs/rustfs:1.0.0-alpha.85@sha256:23ff2af1ec2d64e0fb9608a1c39b5eb96fcff3a8efecf332986c948b0e3e4f8a "sh -ec 'unset RUSTF…" s3 12 minutes ago Up About a minute (healthy) 9000-9001/tcp
mtusi-lab5-grade3-worker-1 sha256:b62e53a94c49d16d79bf0ed20a0cf3f6962eadc669898f6a1756dcd6abe3b928 "python -m src.worker" worker 11 minutes ago Up 59 seconds 8080/tcp
2026-11-20T19:05:08.1218907+03:00
+83
View File
@@ -0,0 +1,83 @@
2026-11-20T19:06:23.2953704+03:00
PS> docker compose logs --no-color --timestamps worker
worker-1 | 2026-11-20T15:57:05.826717851Z INFO consumer ready prefetch=1
worker-1 | 2026-11-20T15:57:27.273018642Z INFO job=219e13a4-6d58-4d01-a06f-2de157567b83 attempt=1 status=processing
worker-1 | 2026-11-20T15:57:27.535019947Z INFO job=219e13a4-6d58-4d01-a06f-2de157567b83 attempt=1 outcome=ready reason=None
worker-1 | 2026-11-20T15:58:05.375025003Z INFO job=3fce5374-3d31-413f-887d-f52b0a007418 attempt=1 status=processing
worker-1 | 2026-11-20T15:58:05.416747428Z INFO job=3fce5374-3d31-413f-887d-f52b0a007418 attempt=1 outcome=ready reason=None
worker-1 | 2026-11-20T15:58:35.523919049Z WARNING consumer reconnecting after AMQPConnectionError
worker-1 | 2026-11-20T15:58:37.723014978Z WARNING consumer reconnecting after AMQPConnectionError
worker-1 | 2026-11-20T15:58:39.808151823Z WARNING consumer reconnecting after AMQPConnectionError
worker-1 | 2026-11-20T15:58:41.807173804Z WARNING consumer reconnecting after AMQPConnectionError
worker-1 | 2026-11-20T15:58:43.910406970Z INFO consumer ready prefetch=1
worker-1 | 2026-11-20T16:01:06.313598472Z INFO job=3ca0f2ea-dee9-4325-80e2-c190591e4da7 attempt=1 status=processing
worker-1 | 2026-11-20T16:01:06.433167528Z INFO job=3ca0f2ea-dee9-4325-80e2-c190591e4da7 attempt=1 outcome=ready reason=None
worker-1 | 2026-11-20T16:01:42.244711391Z INFO consumer ready prefetch=1
worker-1 | 2026-11-20T16:01:42.435406895Z INFO job=482fdde8-2214-4927-80b1-2f2b45c50e09 attempt=1 status=processing
worker-1 | 2026-11-20T16:01:42.707011142Z INFO job=482fdde8-2214-4927-80b1-2f2b45c50e09 attempt=1 outcome=ready reason=None
worker-1 | 2026-11-20T16:02:18.581394520Z INFO job=b02e2857-fe40-4444-82b6-b84b9038fc45 attempt=1 status=processing
worker-1 | 2026-11-20T16:02:18.641345046Z INFO job=b02e2857-fe40-4444-82b6-b84b9038fc45 attempt=1 outcome=ready reason=None
worker-1 | 2026-11-20T16:03:36.096107554Z INFO consumer ready prefetch=1
worker-1 | 2026-11-20T16:03:36.149576180Z INFO job=77c585c8-1f45-4d50-a5d8-45a1f8c013f6 attempt=1 status=processing
worker-1 | 2026-11-20T16:03:36.328370067Z INFO job=77c585c8-1f45-4d50-a5d8-45a1f8c013f6 attempt=1 outcome=ready reason=None
worker-1 | 2026-11-20T16:03:36.397185201Z INFO job=c5cca485-a6af-4ba2-bd94-5f8c15ed35b8 attempt=1 status=processing
worker-1 | 2026-11-20T16:03:36.528083165Z INFO job=c5cca485-a6af-4ba2-bd94-5f8c15ed35b8 attempt=1 outcome=ready reason=None
worker-1 | 2026-11-20T16:03:38.223656394Z INFO job=77c585c8-1f45-4d50-a5d8-45a1f8c013f6 attempt=1 outcome=duplicate_ready reason=None
worker-1 | 2026-11-20T16:03:38.263636714Z INFO job=77c585c8-1f45-4d50-a5d8-45a1f8c013f6 attempt=1 outcome=duplicate_ready reason=None
worker-1 | 2026-11-20T16:03:45.285276333Z INFO job=5a8e2010-5337-49d5-a3cb-b9f70c1e61be attempt=1 status=processing
worker-1 | 2026-11-20T16:03:45.387728788Z INFO job=5a8e2010-5337-49d5-a3cb-b9f70c1e61be attempt=1 outcome=ready reason=None
worker-1 | 2026-11-20T16:03:45.744061783Z INFO job=3a717b4e-ef2b-4470-8614-c707ba5d00a5 attempt=1 status=processing
worker-1 | 2026-11-20T16:03:45.775216500Z INFO job=3a717b4e-ef2b-4470-8614-c707ba5d00a5 attempt=1 outcome=ready reason=None
worker-1 | 2026-11-20T16:03:54.837844874Z INFO consumer ready prefetch=1
worker-1 | 2026-11-20T16:03:54.877224890Z INFO job=7c5f651d-75e9-4d4a-89b6-ef655865503a attempt=1 status=processing
worker-1 | 2026-11-20T16:03:58.213689574Z INFO job=7c5f651d-75e9-4d4a-89b6-ef655865503a attempt=1 outcome=retry reason=storage_unavailable
worker-1 | 2026-11-20T16:04:01.222619395Z INFO job=7c5f651d-75e9-4d4a-89b6-ef655865503a attempt=2 status=processing
worker-1 | 2026-11-20T16:04:01.369642070Z INFO job=7c5f651d-75e9-4d4a-89b6-ef655865503a attempt=2 outcome=ready reason=None
worker-1 | 2026-11-20T16:04:12.157881938Z INFO consumer ready prefetch=1
worker-1 | 2026-11-20T16:04:12.184229951Z INFO job=58861b50-21c9-4caa-97cf-45f987c02c61 attempt=1 status=processing
worker-1 | 2026-11-20T16:04:12.211661364Z INFO job=58861b50-21c9-4caa-97cf-45f987c02c61 attempt=1 outcome=retry reason=storage_unavailable
worker-1 | 2026-11-20T16:04:12.245057180Z INFO job=da8a7190-82a3-4e15-a1df-c0dc60549c06 attempt=1 status=processing
worker-1 | 2026-11-20T16:04:12.258017987Z INFO job=da8a7190-82a3-4e15-a1df-c0dc60549c06 attempt=1 outcome=failed reason=invalid_image
worker-1 | 2026-11-20T16:04:15.213649323Z INFO job=58861b50-21c9-4caa-97cf-45f987c02c61 attempt=2 status=processing
worker-1 | 2026-11-20T16:04:15.245260338Z INFO job=58861b50-21c9-4caa-97cf-45f987c02c61 attempt=2 outcome=retry reason=storage_unavailable
worker-1 | 2026-11-20T16:04:18.236171493Z INFO job=58861b50-21c9-4caa-97cf-45f987c02c61 attempt=3 status=processing
worker-1 | 2026-11-20T16:04:18.249030599Z INFO job=58861b50-21c9-4caa-97cf-45f987c02c61 attempt=3 outcome=retry reason=storage_unavailable
worker-1 | 2026-11-20T16:04:21.250297456Z INFO job=58861b50-21c9-4caa-97cf-45f987c02c61 attempt=4 status=processing
worker-1 | 2026-11-20T16:04:21.263213863Z INFO job=58861b50-21c9-4caa-97cf-45f987c02c61 attempt=4 outcome=retry reason=storage_unavailable
worker-1 | 2026-11-20T16:04:24.271258705Z INFO job=58861b50-21c9-4caa-97cf-45f987c02c61 attempt=5 status=processing
worker-1 | 2026-11-20T16:04:24.277817408Z INFO job=58861b50-21c9-4caa-97cf-45f987c02c61 attempt=5 outcome=failed reason=retry_exhausted
worker-1 | 2026-11-20T16:04:24.814054065Z WARNING consumer reconnecting after ConnectionClosedByBroker
worker-1 | 2026-11-20T16:04:30.262356818Z WARNING consumer reconnecting after gaierror
worker-1 | 2026-11-20T16:04:32.278870347Z WARNING consumer reconnecting after AMQPConnectionError
worker-1 | 2026-11-20T16:04:34.284613670Z WARNING consumer reconnecting after AMQPConnectionError
worker-1 | 2026-11-20T16:04:36.296397096Z WARNING consumer reconnecting after AMQPConnectionError
worker-1 | 2026-11-20T16:04:38.305148312Z WARNING consumer reconnecting after AMQPConnectionError
worker-1 | 2026-11-20T16:04:40.317572976Z WARNING consumer reconnecting after AMQPConnectionError
worker-1 | 2026-11-20T16:04:42.323362436Z WARNING consumer reconnecting after AMQPConnectionError
worker-1 | 2026-11-20T16:04:44.380425325Z INFO consumer ready prefetch=1
worker-1 | 2026-11-20T16:04:48.206171250Z INFO job=137dba80-e862-4a04-842e-94f0f85f984d attempt=1 status=processing
worker-1 | 2026-11-20T16:04:48.282533195Z INFO job=137dba80-e862-4a04-842e-94f0f85f984d attempt=1 outcome=ready reason=None
worker-1 | 2026-11-20T16:04:57.020529822Z INFO job=e1b4a0e4-9ee2-4dca-a907-a7ee4c52e059 attempt=1 status=processing
worker-1 | 2026-11-20T16:04:57.068149044Z INFO job=e1b4a0e4-9ee2-4dca-a907-a7ee4c52e059 attempt=1 outcome=ready reason=None
PS> docker inspect --format ... (published ports)
/mtusi-lab5-grade3-api1-1 published={}
/mtusi-lab5-grade3-api2-1 published={}
/mtusi-lab5-grade3-broker-1 published={}
/mtusi-lab5-grade3-lb-1 published={"8080/tcp":[{"HostIp":"127.0.0.1","HostPort":"8085"}]}
/mtusi-lab5-grade3-postgres-1 published={}
/mtusi-lab5-grade3-s3-1 published={}
/mtusi-lab5-grade3-worker-1 published={}
PS> docker ps ... (existing lab3/lab4)
mtusi-lab3-grade3-api2-1 8080/tcp
mtusi-lab3-grade3-api1-1 8080/tcp
mtusi-lab3-grade3-postgres-1 5432/tcp
mtusi-lab3-grade3-lb-1 127.0.0.1:8080->8080/tcp
mtusi-lab4-grade3-api2-1 8080/tcp
mtusi-lab4-grade3-api1-1 8080/tcp
mtusi-lab4-grade3-s3-1 9000-9001/tcp
mtusi-lab4-grade3-postgres-1 5432/tcp
mtusi-lab4-grade3-lb-1 127.0.0.1:8084->8080/tcp
PS> docker volume ls ... (existing lab3/lab4)
mtusi-lab3-grade3_postgres_data
mtusi-lab4-grade3_postgres_data
mtusi-lab4-grade3_s3_data
+14
View File
@@ -0,0 +1,14 @@
FROM docker:27.5.1-cli
RUN apk add --no-cache python3 py3-pip make && python3 -m venv /opt/venv
ENV PATH=/opt/venv/bin:$PATH PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1
WORKDIR /workspace
COPY infra/requirements-tools.lock /tmp/requirements-tools.lock
RUN pip install --no-cache-dir -r /tmp/requirements-tools.lock
COPY Makefile lab.json README.md COURSE.md CONTRIBUTING.md REPORT.md ./
COPY contracts/ ./contracts/
COPY tests/ ./tests/
COPY scripts/check_contract.py scripts/validate_openapi.py ./scripts/
COPY --chmod=0755 scripts/restart.sh scripts/stop-one.sh scripts/lab-containers.sh ./scripts/
COPY student-tests/ ./student-tests/
ENTRYPOINT []
CMD ["make", "check"]
+29
View File
@@ -0,0 +1,29 @@
defaults
mode http
timeout http-request 5s
timeout connect 500ms
timeout client 15s
timeout server 10s
timeout check 1s
retries 1
retry-on conn-failure
option redispatch
resolvers docker
nameserver embedded 127.0.0.11:53
timeout resolve 500ms
timeout retry 500ms
hold valid 1s
hold nx 1s
frontend public_http
bind :8080
default_backend messenger
backend messenger
balance roundrobin
option httpchk GET /health/ready
http-check expect status 200
default-server check inter 1s fastinter 500ms downinter 1s fall 2 rise 1 resolvers docker init-addr none resolve-prefer ipv4
server api1 api1:8080
server api2 api2:8080
+20
View File
@@ -0,0 +1,20 @@
attrs==26.1.0
certifi==2026.10.23
charset-normalizer==3.5.2
idna==3.20
jsonschema==4.26.0
jsonschema-path==0.3.4
jsonschema-specifications==2025.9.1
lazy-object-proxy==1.13.0
openapi-schema-validator==0.6.3
openapi-spec-validator==0.7.2
pathable==0.4.4
PyYAML==6.0.3
referencing==0.36.2
requests==2.34.2
rfc3339-validator==0.1.4
rpds-py==2026.9.1
six==1.17.0
typing_extensions==4.16.0
urllib3==2.8.0
Pillow==12.2.0
+36
View File
@@ -0,0 +1,36 @@
CREATE TABLE users (
id uuid PRIMARY KEY,
username text NOT NULL UNIQUE CHECK (username ~ '^[a-z0-9_]{3,32}$'),
display_name text NOT NULL CHECK (char_length(display_name) BETWEEN 1 AND 100 AND btrim(display_name) <> ''),
password_hash text NOT NULL,
created_at timestamptz NOT NULL DEFAULT clock_timestamp()
);
CREATE TABLE chats (
id uuid PRIMARY KEY,
title text NOT NULL CHECK (char_length(title) BETWEEN 1 AND 100 AND btrim(title) <> ''),
created_at timestamptz NOT NULL DEFAULT clock_timestamp()
);
CREATE TABLE chat_members (
chat_id uuid NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
user_id uuid NOT NULL REFERENCES users(id),
PRIMARY KEY (chat_id, user_id)
);
CREATE INDEX chat_members_user ON chat_members(user_id, chat_id);
CREATE INDEX chats_order ON chats(created_at, id);
CREATE TABLE messages (
id uuid PRIMARY KEY,
chat_id uuid NOT NULL REFERENCES chats(id),
sender_id uuid NOT NULL,
text text NOT NULL CHECK (char_length(text) BETWEEN 1 AND 4000 AND btrim(text) <> ''),
created_at timestamptz NOT NULL DEFAULT clock_timestamp(),
FOREIGN KEY (chat_id, sender_id) REFERENCES chat_members(chat_id, user_id)
);
CREATE INDEX messages_history ON messages(chat_id, created_at, id);
CREATE TABLE sessions (
token_hash bytea PRIMARY KEY CHECK (octet_length(token_hash) = 32),
user_id uuid NOT NULL REFERENCES users(id),
expires_at timestamptz NOT NULL,
created_at timestamptz NOT NULL DEFAULT clock_timestamp(),
CHECK (expires_at > created_at)
);
CREATE INDEX sessions_expiry ON sessions(expires_at);
+9
View File
@@ -0,0 +1,9 @@
CREATE TABLE message_idempotency (
user_id uuid NOT NULL REFERENCES users(id),
method text NOT NULL,
chat_id uuid NOT NULL REFERENCES chats(id),
key text NOT NULL CHECK (key COLLATE "C" ~ '^[A-Za-z0-9._:-]{1,128}$'),
message_id uuid NOT NULL REFERENCES messages(id) DEFERRABLE INITIALLY DEFERRED,
expires_at timestamptz NOT NULL,
PRIMARY KEY (user_id, method, chat_id, key)
);
+35
View File
@@ -0,0 +1,35 @@
-- Preserve existing message rows and stored idempotency keys.
ALTER TABLE messages DROP CONSTRAINT messages_text_check;
ALTER TABLE messages ADD CHECK (char_length(text) <= 4000);
ALTER TABLE messages ADD UNIQUE (chat_id, id);
CREATE TABLE attachments (
id uuid PRIMARY KEY,
chat_id uuid NOT NULL REFERENCES chats(id),
owner_id uuid NOT NULL,
filename text NOT NULL CHECK (char_length(filename) BETWEEN 1 AND 255
AND strpos(filename, '/') = 0 AND strpos(filename, chr(92)) = 0
AND filename !~ '[[:cntrl:]]'),
content_type text NOT NULL CHECK (content_type IN ('image/png', 'image/jpeg')),
size_bytes integer NOT NULL CHECK (size_bytes BETWEEN 1 AND 10485760),
sha256 text NOT NULL CHECK (sha256 COLLATE "C" ~ '^[a-f0-9]{64}$'),
object_key text NOT NULL UNIQUE,
status text NOT NULL DEFAULT 'ready' CHECK (status IN ('ready', 'failed')),
error_code text,
created_at timestamptz NOT NULL DEFAULT transaction_timestamp(),
updated_at timestamptz NOT NULL DEFAULT transaction_timestamp(),
CHECK ((status = 'ready') = (error_code IS NULL)),
FOREIGN KEY (chat_id, owner_id) REFERENCES chat_members(chat_id, user_id),
UNIQUE (chat_id, id)
);
CREATE TABLE message_attachments (
chat_id uuid NOT NULL,
message_id uuid NOT NULL,
attachment_id uuid NOT NULL,
position integer NOT NULL CHECK (position BETWEEN 1 AND 10),
PRIMARY KEY (message_id, position),
UNIQUE (message_id, attachment_id),
FOREIGN KEY (chat_id, message_id) REFERENCES messages(chat_id, id) ON DELETE CASCADE,
FOREIGN KEY (chat_id, attachment_id) REFERENCES attachments(chat_id, id)
);
+32
View File
@@ -0,0 +1,32 @@
-- Existing Lab 4 rows keep their original keys, metadata and message links.
ALTER TABLE attachments DROP CONSTRAINT attachments_status_check;
ALTER TABLE attachments DROP CONSTRAINT attachments_check;
ALTER TABLE attachments ALTER COLUMN status SET DEFAULT 'queued';
ALTER TABLE attachments ADD CHECK (status IN ('queued', 'processing', 'ready', 'failed'));
ALTER TABLE attachments ADD CHECK ((status = 'failed') = (error_code IS NOT NULL));
CREATE TABLE attachment_jobs (
job_id uuid PRIMARY KEY,
attachment_id uuid NOT NULL UNIQUE REFERENCES attachments(id),
pipeline_version integer NOT NULL CHECK (pipeline_version = 1),
staging_key text NOT NULL UNIQUE,
created_at timestamptz NOT NULL,
attempt integer NOT NULL DEFAULT 1,
max_attempts integer NOT NULL CHECK (max_attempts BETWEEN 1 AND 20),
retry_at timestamptz NOT NULL DEFAULT now(),
last_error text,
CHECK (attempt BETWEEN 1 AND max_attempts)
);
CREATE TABLE attachment_variants (
attachment_id uuid NOT NULL REFERENCES attachments(id),
pipeline_version integer NOT NULL CHECK (pipeline_version = 1),
name text NOT NULL CHECK (name = 'thumbnail'),
content_type text NOT NULL CHECK (content_type = 'image/jpeg'),
size_bytes integer NOT NULL CHECK (size_bytes > 0),
sha256 text NOT NULL CHECK (sha256 COLLATE "C" ~ '^[a-f0-9]{64}$'),
width integer NOT NULL CHECK (width BETWEEN 1 AND 256),
height integer NOT NULL CHECK (height BETWEEN 1 AND 256),
object_key text NOT NULL UNIQUE,
PRIMARY KEY (attachment_id, pipeline_version, name)
);
+28
View File
@@ -0,0 +1,28 @@
annotated-types==0.8.0
anyio==4.15.1
argon2-cffi==23.1.0
argon2-cffi-bindings==26.1.0
cffi==2.1.1
click==8.5.0
fastapi==0.115.12
h11==0.16.0
idna==3.20
psycopg==3.2.6
psycopg-binary==3.2.6
psycopg-pool==3.2.6
pycparser==3.12
pydantic==2.11.3
pydantic_core==2.33.1
starlette==0.46.2
typing-inspection==0.4.4
typing_extensions==4.16.0
uvicorn==0.34.2
boto3==1.40.45
botocore==1.40.45
jmespath==1.0.1
Pillow==12.2.0
python-dateutil==2.9.0.post0
s3transfer==0.14.0
six==1.17.0
urllib3==2.6.3
pika==1.3.2
+10
View File
@@ -0,0 +1,10 @@
fastapi==0.115.12
starlette==0.46.2
pydantic==2.11.3
uvicorn==0.34.2
psycopg[binary]==3.2.6
psycopg-pool==3.2.6
argon2-cffi==23.1.0
boto3==1.40.45
Pillow==12.2.0
pika==1.3.2
+63
View File
@@ -0,0 +1,63 @@
$ErrorActionPreference = 'Stop'
$root = Split-Path -Parent $PSScriptRoot
$utf8 = [Text.UTF8Encoding]::new($false)
$secretDir = Join-Path $root 'secrets'
[void][IO.Directory]::CreateDirectory($secretDir)
# Compose bind-mounts these files via Docker Desktop. Keep the host ACL private.
$sid = [Security.Principal.WindowsIdentity]::GetCurrent().User
if (Test-Path -LiteralPath $secretDir -PathType Container) {
$security = [Security.AccessControl.DirectorySecurity]::new()
$security.SetOwner($sid)
$security.SetAccessRuleProtection($true, $false)
$inheritance = [Security.AccessControl.InheritanceFlags]'ContainerInherit, ObjectInherit'
foreach ($principal in @($sid, [Security.Principal.SecurityIdentifier]::new('S-1-5-18'))) {
$rule = [Security.AccessControl.FileSystemAccessRule]::new(
$principal, 'FullControl', $inheritance, 'None', 'Allow'
)
$security.AddAccessRule($rule)
}
Set-Acl -LiteralPath $secretDir -AclObject $security
}
function New-HexSecret([string]$name, [int]$length) {
$file = Join-Path $secretDir $name
if (-not (Test-Path -LiteralPath $file)) {
$bytes = New-Object byte[] $length
$rng = [Security.Cryptography.RandomNumberGenerator]::Create()
try { $rng.GetBytes($bytes) } finally { $rng.Dispose() }
$value = [BitConverter]::ToString($bytes).Replace('-', '').ToLowerInvariant()
[IO.File]::WriteAllText($file, $value, $utf8)
}
}
New-HexSecret 'broker_password' 32
$brokerPassword = [IO.File]::ReadAllText((Join-Path $secretDir 'broker_password')).Trim()
if ($brokerPassword -cnotmatch '^[a-f0-9]{64}$') { throw 'Invalid local RabbitMQ secret format.' }
$brokerUrl = Join-Path $secretDir 'broker_url'
$brokerExpected = "amqp://messenger:$brokerPassword@broker:5672/%2F"
if (-not (Test-Path -LiteralPath $brokerUrl)) {
[IO.File]::WriteAllText($brokerUrl, $brokerExpected, $utf8)
}
if ([IO.File]::ReadAllText($brokerUrl).Trim() -cne $brokerExpected) {
throw 'Existing broker_url and broker_password disagree.'
}
New-HexSecret 'postgres_password' 32
New-HexSecret 's3_access_key_id' 10
New-HexSecret 's3_secret_access_key' 24
$password = [IO.File]::ReadAllText((Join-Path $secretDir 'postgres_password')).Trim()
if ($password -cnotmatch '^[a-f0-9]{64}$') { throw 'Invalid local PostgreSQL secret format.' }
$url = Join-Path $secretDir 'database_url'
$expected = "postgresql://messenger:${password}@postgres:5432/messenger"
if (-not (Test-Path -LiteralPath $url)) {
[IO.File]::WriteAllText($url, $expected, $utf8)
}
if ([IO.File]::ReadAllText($url).Trim() -cne $expected) {
throw 'Existing database_url and postgres_password disagree.'
}
$envFile = Join-Path $root '.env'
if (-not (Test-Path -LiteralPath $envFile)) {
Copy-Item -LiteralPath (Join-Path $root '.env.example') -Destination $envFile
}
Write-Output 'Lab 5 local configuration ready.'
+18
View File
@@ -0,0 +1,18 @@
#!/bin/sh
# Docker socket actions are scoped to the explicitly named course stack.
case "${LAB5_PROJECT:-}" in
mtusi-lab5-grade3|mtusi-lab5-grade3-*) ;;
*) echo 'LAB5_PROJECT must identify the Lab 5 test stack' >&2; exit 2 ;;
esac
container() {
ids=$(docker ps -aq --filter "label=com.docker.compose.project=$LAB5_PROJECT" \
--filter "label=com.docker.compose.service=$1" \
--filter 'label=com.docker.compose.oneoff=False')
set -- $ids
if [ "$#" -ne 1 ]; then
echo 'Expected one Lab 5 service container' >&2
return 2
fi
printf '%s\n' "$1"
}
+19
View File
@@ -0,0 +1,19 @@
#!/bin/sh
set -eu
. "$(dirname "$0")/lab-containers.sh"
broker=$(container broker)
docker stop --time 3 "$(container worker)"
docker restart --time 4 "$(container postgres)" "$(container s3)" "$broker"
docker restart --time 4 "$(container api1)" "$(container api2)" "$(container lb)"
# API readiness covers PostgreSQL and S3. Let AMQP recover before returning.
remaining=55
while [ "$(docker inspect --format '{{.State.Health.Status}}' "$broker")" != healthy ]; do
remaining=$((remaining - 1))
if [ "$remaining" -le 0 ]; then
echo 'RabbitMQ did not recover after restart' >&2
exit 1
fi
sleep 1
done
docker start "$(container worker)"
+9
View File
@@ -0,0 +1,9 @@
#!/bin/sh
set -eu
case "${TARGET_INSTANCE_ID:-}" in
api1|api2) ;;
*) echo 'TARGET_INSTANCE_ID must be api1 or api2' >&2; exit 2 ;;
esac
. "$(dirname "$0")/lab-containers.sh"
id=$(container "$TARGET_INSTANCE_ID")
docker stop --time 4 "$id"
+1
View File
@@ -0,0 +1 @@
+165
View File
@@ -0,0 +1,165 @@
import hashlib
import re
import unicodedata
from io import BytesIO
from uuid import uuid4
import pika
from PIL import Image
from src import jobs, s3
from src.db import connection
from src.errors import ApiError
from src.store import public
MAX_UPLOAD_BYTES = 10 * 1024 * 1024
MAX_IMAGE_PIXELS = 40_000_000
Image.MAX_IMAGE_PIXELS = MAX_IMAGE_PIXELS
FORMATS = {"image/png": "PNG", "image/jpeg": "JPEG"}
METADATA = """id, chat_id, owner_id, filename, content_type, size_bytes, sha256,
status, error_code, created_at, updated_at"""
def oversized():
return ApiError(413, "payload_too_large", "Превышен допустимый размер изображения")
def invalid_image():
return ApiError(415, "unsupported_media_type", "Ожидается корректный PNG или JPEG")
def validate_filename(filename):
if any(char in "/\\" or unicodedata.category(char) == "Cc" for char in filename):
raise ApiError(400, "invalid_request", "Недопустимое имя файла")
async def read_upload(request):
lengths = request.headers.getlist("content-length")
if lengths:
if len(lengths) != 1 or not re.fullmatch(r"[0-9]+", lengths[0]):
raise ApiError(400, "invalid_request", "Некорректный Content-Length")
length = lengths[0].lstrip("0") or "0"
if len(length) > len(str(MAX_UPLOAD_BYTES)) or int(length) > MAX_UPLOAD_BYTES:
raise oversized()
body = BytesIO()
async for chunk in request.stream():
if len(chunk) > MAX_UPLOAD_BYTES - body.tell():
raise oversized()
body.write(chunk)
body.seek(0)
return body
def validate_image(body, content_type):
try:
if content_type == "image/png":
# Pillow's verify stops at the IEND chunk before reading its CRC.
with body.getbuffer() as data:
if data[-12:] != b"\x00\x00\x00\x00IEND\xae\x42\x60\x82":
raise invalid_image()
with Image.open(body, formats=(FORMATS[content_type],)) as image:
if image.width * image.height > MAX_IMAGE_PIXELS:
raise oversized()
image.verify()
with Image.open(body, formats=(FORMATS[content_type],)) as image:
# Check each frame of a PNG while bounding the total decode work.
for frame in range(getattr(image, "n_frames", 1)):
if (frame + 1) * image.width * image.height > MAX_IMAGE_PIXELS:
raise oversized()
image.seek(frame)
image.load()
except Image.DecompressionBombError:
raise oversized() from None
except (OSError, ValueError, SyntaxError, IndexError, EOFError):
raise invalid_image() from None
finally:
body.seek(0)
def upload(chat_id, owner_id, filename, content_type, body):
validate_image(body, content_type)
with body.getbuffer() as data:
digest = hashlib.sha256(data).hexdigest()
size = len(data)
attachment_id = uuid4()
job_id = uuid4()
staging_key = f"staging/{attachment_id}"
object_key = f"originals/{attachment_id}"
s3.put(staging_key, body, size, content_type)
# Both rows commit before publish, so a fast consumer can always find them.
with connection() as conn:
row = conn.execute(
f"""INSERT INTO attachments
(id, chat_id, owner_id, filename, content_type, size_bytes, sha256, object_key)
VALUES (%s,%s,%s,%s,%s,%s,%s,%s) RETURNING {METADATA}""",
(attachment_id, chat_id, owner_id, filename, content_type, size, digest, object_key),
).fetchone()
conn.execute(
"""INSERT INTO attachment_jobs
(job_id,attachment_id,pipeline_version,staging_key,created_at,max_attempts)
VALUES (%s,%s,%s,%s,%s,%s)""",
(job_id, attachment_id, jobs.PIPELINE_VERSION, staging_key, row["created_at"],
jobs.MAX_ATTEMPTS),
)
try:
with jobs.broker() as channel:
jobs.publish(channel, jobs.carrier(job_id, attachment_id, row["created_at"]))
except (pika.exceptions.AMQPError, OSError):
# The outcome can be ambiguous. Keep staging and metadata for a consumer
# that already has the message. Never return 202 without a confirm.
raise ApiError(503, "unavailable", "Очередь вложений временно недоступна") from None
return public({**row, "variants": []})
VARIANT_METADATA = "name, content_type, size_bytes, sha256, width, height"
def get(attachment_id, user_id):
with connection() as conn:
row = visible(conn, attachment_id, user_id)
variants = []
if row["status"] == "ready":
variants = conn.execute(
f"""SELECT {VARIANT_METADATA} FROM attachment_variants
WHERE attachment_id=%s AND pipeline_version=%s ORDER BY name""",
(attachment_id, jobs.PIPELINE_VERSION),
).fetchall()
del row["object_key"]
return public({**row, "variants": variants})
def visible(conn, attachment_id, user_id):
row = conn.execute(
f"""SELECT {METADATA}, object_key FROM attachments a
WHERE id=%s AND EXISTS (
SELECT 1 FROM chat_members m WHERE m.chat_id=a.chat_id AND m.user_id=%s
)""",
(attachment_id, user_id),
).fetchone()
if row is None:
raise ApiError(404, "not_found", "Объект не найден")
return row
def content(attachment_id, user_id, variant):
with connection() as conn:
row = visible(conn, attachment_id, user_id)
if row["status"] != "ready":
raise ApiError(409, "conflict", "Вложение недоступно")
if variant != "original":
row = conn.execute(
"""SELECT object_key, content_type, size_bytes, sha256 FROM attachment_variants
WHERE (attachment_id,pipeline_version,name)=(%s,%s,%s)""",
(attachment_id, jobs.PIPELINE_VERSION, variant),
).fetchone()
if row is None:
raise ApiError(404, "not_found", "Объект не найден")
data = s3.download(row["object_key"], row["size_bytes"])
if len(data) != row["size_bytes"] or hashlib.sha256(data).hexdigest() != row["sha256"]:
raise s3.unavailable()
return data, row["content_type"]
+16
View File
@@ -0,0 +1,16 @@
import os
from pathlib import Path
def secret(name):
path = os.environ.get(name + "_FILE")
if path:
return Path(path).read_text().strip()
raise RuntimeError(f"{name}_FILE is required")
SESSION_TTL = int(os.getenv("SESSION_TTL_SECONDS", "3600"))
if not 1 <= SESSION_TTL <= 604800:
raise RuntimeError("SESSION_TTL_SECONDS must be 1..604800")
if os.getenv("SESSION_STORE", "postgres") != "postgres":
raise RuntimeError("This implementation uses PostgreSQL sessions")
+40
View File
@@ -0,0 +1,40 @@
from contextlib import contextmanager
from pathlib import Path
import psycopg
from psycopg.rows import dict_row
from psycopg_pool import ConnectionPool
from src.config import secret
pool = ConnectionPool(
conninfo=secret("DATABASE_URL"), min_size=1, max_size=8,
timeout=2, max_waiting=32, reconnect_timeout=5, open=False,
kwargs={"connect_timeout": 2, "row_factory": dict_row,
"options": "-c statement_timeout=2000 -c lock_timeout=1500 -c idle_in_transaction_session_timeout=5000 -c timezone=UTC"},
)
def migrate():
# Serialize migration jobs if two operators start the stack concurrently.
with psycopg.connect(secret("DATABASE_URL"), connect_timeout=3) as conn:
conn.execute("SET LOCAL lock_timeout = '10s'")
conn.execute("SET LOCAL statement_timeout = '15s'")
conn.execute("SELECT pg_advisory_xact_lock(202602)")
conn.execute("CREATE TABLE IF NOT EXISTS schema_migrations (version text PRIMARY KEY, applied_at timestamptz NOT NULL DEFAULT now())")
for path in sorted((Path(__file__).resolve().parents[1] / "migrations").glob("*.sql")):
if not conn.execute("SELECT 1 FROM schema_migrations WHERE version=%s", (path.name,)).fetchone():
conn.execute(path.read_text(encoding="utf-8"))
conn.execute("INSERT INTO schema_migrations(version) VALUES (%s)", (path.name,))
@contextmanager
def connection():
with pool.connection() as conn:
# psycopg context commits on success, rolls back on EVERY exception.
yield conn
if __name__ == "__main__":
migrate()
print("PASS: versioned migrations applied (already applied versions skipped)")
+34
View File
@@ -0,0 +1,34 @@
from fastapi import Request
from fastapi.responses import JSONResponse
class ApiError(Exception):
def __init__(
self,
status_code: int,
code: str,
message: str,
):
self.status_code = status_code
self.code = code
self.message = message
def error_response(
request: Request,
status_code: int,
code: str,
message: str,
) -> JSONResponse:
request_id = request.state.request_id
return JSONResponse(
status_code=status_code,
content={
"error": {
"code": code,
"message": message,
"request_id": request_id,
}
},
)
+81
View File
@@ -0,0 +1,81 @@
"""AMQP carrier and durable publishing, shared by API and worker."""
import json
import logging
import os
from contextlib import contextmanager
from datetime import datetime
from typing import Literal
from uuid import UUID
import pika
from pydantic import AwareDatetime, BaseModel, ConfigDict, Field
from src.config import secret
QUEUE = "attachment.process.v1"
PIPELINE_VERSION = 1
MAX_ATTEMPTS = int(os.getenv("JOB_MAX_ATTEMPTS", "5"))
if not 1 <= MAX_ATTEMPTS <= 20:
raise RuntimeError("JOB_MAX_ATTEMPTS must be 1..20")
# Report dependency failures at the call site without logging connection details.
logging.getLogger("pika").setLevel(logging.CRITICAL)
class Job(BaseModel):
model_config = ConfigDict(extra="forbid", strict=True)
schema_version: Literal[1]
type: Literal["attachment.process.v1"]
job_id: UUID
attachment_id: UUID
pipeline_version: int = Field(ge=1)
attempt: int = Field(ge=1)
created_at: AwareDatetime
traceparent: str = Field(default="", pattern=r"^00-[0-9a-f]{32}-[0-9a-f]{16}-[0-9a-f]{2}$")
tracestate: str = Field(default="", max_length=512)
def carrier(job_id: UUID, attachment_id: UUID, created_at: datetime, attempt: int = 1):
return Job(
schema_version=1, type=QUEUE, job_id=job_id, attachment_id=attachment_id,
pipeline_version=PIPELINE_VERSION, attempt=attempt, created_at=created_at,
)
def parse(body):
# Validate JSON primitive types before Pydantic's UUID/time parsing.
values = json.loads(body)
if not isinstance(values, dict) or any(
type(values.get(k)) is not int for k in ("schema_version", "pipeline_version", "attempt")
) or any(type(values.get(k)) is not str for k in ("job_id", "attachment_id", "created_at")):
raise ValueError("Bad job carrier")
return Job.model_validate_json(body)
@contextmanager
def broker():
params = pika.URLParameters(secret("BROKER_URL"))
params.socket_timeout = 2
params.stack_timeout = 4
params.blocked_connection_timeout = 4
params.heartbeat = 30
conn = pika.BlockingConnection(params)
try:
channel = conn.channel()
channel.queue_declare(queue=QUEUE, durable=True, arguments={"x-queue-type": "classic"})
channel.confirm_delivery()
yield channel
finally:
if conn.is_open:
conn.close()
def publish(channel, job):
channel.basic_publish(
exchange="", routing_key=QUEUE, body=job.model_dump_json(exclude_unset=True).encode(),
properties=pika.BasicProperties(delivery_mode=2, content_type="application/json",
message_id=str(job.job_id)),
mandatory=True,
)
+211
View File
@@ -0,0 +1,211 @@
import hashlib
import logging
import os
import re
import secrets
from contextlib import asynccontextmanager
from uuid import UUID, uuid4
import anyio
import psycopg
from fastapi import Depends, FastAPI, Query, Request, Response
from starlette.requests import ClientDisconnect
from fastapi.exceptions import RequestValidationError
from psycopg_pool import PoolTimeout, TooManyRequests
from starlette.exceptions import HTTPException
from src import attachments, s3, store
from src.config import SESSION_TTL
from src.db import connection, pool
from src.errors import ApiError, error_response
from src.models import Attachment, Chat, ChatPage, CreateChat, CreateMessage, CreateUser, Message, MessagePage, User
from src.security import basic_credentials, hasher, token_hash, unauthorized, verify
INSTANCE_ID = os.getenv("INSTANCE_ID") or uuid4().hex
IDEMPOTENCY_KEY = re.compile(r"[A-Za-z0-9._:-]{1,128}")
@asynccontextmanager
async def lifespan(app):
anyio.to_thread.current_default_thread_limiter().total_tokens = 16
app.state.upload_slots = anyio.Semaphore(2)
pool.open()
try:
pool.wait(timeout=10)
yield
finally:
pool.close()
app = FastAPI(title="MTUSI Messenger", version="1.5.0", lifespan=lifespan,
docs_url=None, redoc_url=None, openapi_url=None)
def failure(request, status, code, message):
response = error_response(request, status, code, message)
if status == 401:
basic = request.url.path == "/api/v1/auth/sessions"
response.headers['WWW-Authenticate'] = ('Basic realm="messenger", charset="UTF-8"' if basic else 'Bearer realm="messenger"')
return response
@app.middleware("http")
async def request_context(request, call_next):
request.state.request_id = str(uuid4())
try:
if request.method == "POST" and request.url.path == "/api/v1/auth/sessions" and await request.body():
raise ApiError(400, "invalid_request", "Вход не принимает тело запроса")
response = await call_next(request)
except ApiError as exc:
response = failure(request, exc.status_code, exc.code, exc.message)
except (psycopg.Error, PoolTimeout, TooManyRequests):
response = failure(request, 503, "unavailable", "Хранилище временно недоступно")
except Exception as exc:
logging.error("Unhandled request error: %s", type(exc).__name__)
response = failure(request, 500, "internal_error", "Внутренняя ошибка сервера")
response.headers['X-Request-Id'] = request.state.request_id
response.headers['X-Instance-Id'] = INSTANCE_ID
return response
@app.exception_handler(ApiError)
async def api_error(request, exc):
return failure(request, exc.status_code, exc.code, exc.message)
@app.exception_handler(RequestValidationError)
async def validation_error(request, exc):
return failure(request, 400, "invalid_request", "Некорректный запрос")
@app.exception_handler(HTTPException)
async def http_error(request, exc):
return failure(request, exc.status_code, "not_found" if exc.status_code == 404 else "invalid_request", "Недопустимый HTTP-запрос")
def require_user(request: Request):
digest = token_hash(request.headers.get('Authorization'))
with connection() as conn:
user = conn.execute(
"""SELECT u.id,u.username,u.display_name,u.created_at
FROM sessions s JOIN users u ON u.id=s.user_id
WHERE s.token_hash=%s AND s.expires_at > clock_timestamp()""",
(digest,),
).fetchone()
if user is None:
unauthorized()
request.state.token_hash = digest
return User(**store.public(user))
@app.get('/health/live')
def live():
return {"status": "ok"}
@app.get('/health/ready')
def ready():
with connection() as conn:
conn.execute("SELECT 1").fetchone()
s3.ready()
return {"status": "ok"}
@app.post('/api/v1/users', status_code=201, response_model=User)
def register(data: CreateUser):
return store.create_user(data, hasher.hash(data.password))
@app.post('/api/v1/auth/sessions', status_code=201)
def login(request: Request):
username, password = basic_credentials(request.headers.get('Authorization'))
with connection() as conn:
row = conn.execute("SELECT id,username,display_name,created_at,password_hash FROM users WHERE username=%s", (username,)).fetchone()
valid = verify(row['password_hash'] if row else None, password)
if not row or not valid:
unauthorized()
token = secrets.token_urlsafe(32)
digest = hashlib.sha256(token.encode('ascii')).digest()
with connection() as conn:
result = conn.execute("INSERT INTO sessions(token_hash,user_id,expires_at) VALUES (%s,%s,clock_timestamp() + %s * interval '1 second') RETURNING expires_at", (digest, row['id'], SESSION_TTL)).fetchone()
del row['password_hash']
return {"token": token, "token_type": "Bearer", "expires_at": store.public(result['expires_at']), "user": store.public(row)}
@app.delete('/api/v1/auth/sessions/current', status_code=204)
def logout(request: Request, user: User = Depends(require_user)):
with connection() as conn:
conn.execute("DELETE FROM sessions WHERE token_hash=%s", (request.state.token_hash,))
return Response(status_code=204)
@app.get('/api/v1/users/me', response_model=User)
def me(user: User = Depends(require_user)):
return user
@app.post('/api/v1/chats', status_code=201, response_model=Chat)
def create_chat(data: CreateChat, user: User = Depends(require_user)):
return store.create_chat(data, user.id)
@app.get('/api/v1/chats', response_model=ChatPage)
def chats(user: User = Depends(require_user), limit: int = Query(50, ge=1, le=100), cursor: str | None = Query(None, min_length=1, max_length=2048)):
return store.list_chats(user.id, limit, cursor)
@app.get('/api/v1/chats/{chat_id}', response_model=Chat)
def chat(chat_id: UUID, user: User = Depends(require_user)):
return store.get_chat(str(chat_id), user.id)
@app.post('/api/v1/chats/{chat_id}/messages', status_code=201, response_model=Message)
def send(request: Request, chat_id: UUID, data: CreateMessage, user: User = Depends(require_user)):
keys = request.headers.getlist("Idempotency-Key")
if len(keys) != 1 or not IDEMPOTENCY_KEY.fullmatch(keys[0]):
raise ApiError(400, "invalid_request", "Требуется корректный Idempotency-Key")
return store.create_message(
str(chat_id), user.id, request.method, keys[0], data.text, data.attachment_ids,
)
@app.get('/api/v1/chats/{chat_id}/messages', response_model=MessagePage)
def history(chat_id: UUID, user: User = Depends(require_user), limit: int = Query(50, ge=1, le=100), cursor: str | None = Query(None, min_length=1, max_length=2048)):
return store.list_messages(str(chat_id), user.id, limit, cursor)
@app.post('/api/v1/chats/{chat_id}/attachments', status_code=202, response_model=Attachment)
async def upload_attachment(
request: Request,
chat_id: UUID,
filename: str = Query(..., min_length=1, max_length=255),
user: User = Depends(require_user),
):
await anyio.to_thread.run_sync(store.get_chat, str(chat_id), user.id)
attachments.validate_filename(filename)
types = request.headers.getlist("content-type")
if len(types) != 1 or types[0] not in attachments.FORMATS:
raise attachments.invalid_image()
async with request.app.state.upload_slots:
try:
body = await attachments.read_upload(request)
except ClientDisconnect:
raise ApiError(400, "invalid_request", "Загрузка прервана") from None
with body:
return await anyio.to_thread.run_sync(
attachments.upload, chat_id, user.id, filename, types[0], body,
)
@app.get('/api/v1/attachments/{attachment_id}', response_model=Attachment)
def attachment_metadata(attachment_id: UUID, user: User = Depends(require_user)):
return attachments.get(attachment_id, user.id)
@app.get('/api/v1/attachments/{attachment_id}/content')
def attachment_content(
attachment_id: UUID, user: User = Depends(require_user), variant: str = Query("original"),
):
data, content_type = attachments.content(attachment_id, user.id, variant)
return Response(data, media_type=content_type,
headers={"Cache-Control": "private, no-store"})
+171
View File
@@ -0,0 +1,171 @@
from typing import Literal
from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
from uuid import UUID
class CreateUser(BaseModel):
model_config = ConfigDict(
extra="forbid",
strict=True,
)
username: str = Field(
min_length=3,
max_length=32,
pattern=r"^[a-z0-9_]{3,32}$",
)
display_name: str = Field(
min_length=1,
max_length=100,
)
@field_validator("display_name")
@classmethod
def display_name_must_not_be_blank(cls, value: str) -> str:
if not value.strip() or "\x00" in value:
raise ValueError("display_name must not be blank")
return value
password: str = Field(min_length=12, max_length=128)
@field_validator("password")
@classmethod
def password_must_not_be_blank(cls, value: str) -> str:
if not value.strip():
raise ValueError("password must not be blank")
return value
class User(BaseModel):
model_config = ConfigDict(extra="forbid")
id: str
username: str
display_name: str
created_at: str
class CreateChat(BaseModel):
model_config = ConfigDict(
extra="forbid",
strict=True,
)
title: str = Field(
min_length=1,
max_length=100,
)
member_ids: list[str] = Field(
min_length=1,
max_length=100,
)
@field_validator("title")
@classmethod
def title_must_not_be_blank(cls, value: str) -> str:
if not value.strip() or "\x00" in value:
raise ValueError("title must not be blank")
return value
@field_validator("member_ids")
@classmethod
def validate_member_ids(cls, value: list[str]) -> list[str]:
normalized = []
for member_id in value:
try:
normalized.append(str(UUID(member_id)))
except (ValueError, TypeError):
raise ValueError("member_ids must contain valid UUIDs") from None
if len(normalized) != len(set(normalized)):
raise ValueError("member_ids must be unique")
return normalized
class Chat(BaseModel):
model_config = ConfigDict(extra="forbid")
id: str
title: str
member_ids: list[str]
created_at: str
class ChatPage(BaseModel):
model_config = ConfigDict(extra="forbid")
items: list[Chat]
next_cursor: str | None
class CreateMessage(BaseModel):
model_config = ConfigDict(
extra="forbid",
strict=True,
)
text: str = Field(max_length=4000)
attachment_ids: list[str] = Field(default_factory=list, max_length=10)
@field_validator("attachment_ids")
@classmethod
def validate_attachment_ids(cls, value: list[str]) -> list[str]:
normalized = [str(UUID(attachment_id)) for attachment_id in value]
if len(normalized) != len(set(normalized)):
raise ValueError("attachment_ids must be unique")
return normalized
@model_validator(mode="after")
def validate_content(self):
if "\x00" in self.text or (not self.text.strip() and not self.attachment_ids):
raise ValueError("Message requires text or attachments")
return self
class Message(BaseModel):
model_config = ConfigDict(extra="forbid")
id: str
chat_id: str
sender_id: str
text: str
created_at: str
attachment_ids: list[str]
class MessagePage(BaseModel):
model_config = ConfigDict(extra="forbid")
items: list[Message]
next_cursor: str | None
class AttachmentVariant(BaseModel):
name: Literal["thumbnail"]
content_type: Literal["image/jpeg"]
size_bytes: int
sha256: str
width: int
height: int
class Attachment(BaseModel):
model_config = ConfigDict(extra="forbid")
id: str
chat_id: str
owner_id: str
filename: str
content_type: Literal["image/png", "image/jpeg"]
size_bytes: int
sha256: str
status: Literal["queued", "processing", "ready", "failed"]
variants: list[AttachmentVariant]
error_code: str | None
created_at: str
updated_at: str
+74
View File
@@ -0,0 +1,74 @@
import base64
import json
import re
from datetime import datetime
from uuid import UUID
from src.errors import ApiError
def encode_cursor(
resource: str,
owner_id: str,
created_at: str,
object_id: str,
) -> str:
payload = {
"resource": resource,
"owner_id": owner_id,
"created_at": created_at,
"id": object_id,
}
raw = json.dumps(
payload,
separators=(",", ":"),
).encode("utf-8")
return base64.urlsafe_b64encode(raw).decode("ascii").rstrip("=")
def decode_cursor(
cursor: str,
expected_resource: str,
expected_owner_id: str,
) -> tuple[str, str]:
try:
if not re.fullmatch(r"[A-Za-z0-9_-]+", cursor):
raise ValueError
padding = "=" * (-len(cursor) % 4)
raw = base64.b64decode(
(cursor + padding).encode("ascii"), altchars=b"-_", validate=True
)
payload = json.loads(raw.decode("utf-8"))
if payload["resource"] != expected_resource:
raise ValueError
if payload["owner_id"] != expected_owner_id:
raise ValueError
created_at = payload["created_at"]
object_id = payload["id"]
# The cursor must carry the same UTC timestamp format produced
# by the server. Invalid timestamps must return HTTP 400 before a SQL cast.
if not isinstance(created_at, str) or not re.fullmatch(
r"\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d+)?Z",
created_at,
):
raise ValueError
datetime.fromisoformat(created_at.replace("Z", "+00:00"))
object_id = str(UUID(object_id))
return created_at, object_id
except Exception:
raise ApiError(
status_code=400,
code="invalid_request",
message="Некорректный cursor",
)
+90
View File
@@ -0,0 +1,90 @@
"""Synchronous access to the single private object bucket."""
import os
from contextlib import closing
import boto3
from botocore.config import Config
from botocore.exceptions import BotoCoreError, ClientError
from src.config import secret
from src.errors import ApiError
BUCKET = os.environ["S3_BUCKET"]
client = boto3.client(
"s3",
endpoint_url=os.environ["S3_ENDPOINT"],
region_name=os.environ["S3_REGION"],
aws_access_key_id=secret("S3_ACCESS_KEY_ID"),
aws_secret_access_key=secret("S3_SECRET_ACCESS_KEY"),
config=Config(
signature_version="s3v4",
s3={"addressing_style": "path"},
connect_timeout=0.5,
read_timeout=2,
retries={"total_max_attempts": 1},
request_checksum_calculation="when_required",
response_checksum_validation="when_required",
proxies={},
),
)
def unavailable():
return ApiError(503, "unavailable", "Хранилище вложений временно недоступно")
def ready():
try:
client.head_bucket(Bucket=BUCKET)
except (ClientError, BotoCoreError):
raise unavailable() from None
def put(key, content, size, content_type):
try:
client.put_object(
Bucket=BUCKET, Key=key, Body=content, ContentLength=size,
ContentType=content_type,
)
except (ClientError, BotoCoreError):
raise unavailable() from None
def download(key, size):
try:
result = client.get_object(Bucket=BUCKET, Key=key)
with closing(result["Body"]) as body:
if result["ContentLength"] != size:
raise unavailable()
return body.read(size)
except (ClientError, BotoCoreError):
raise unavailable() from None
def delete(key):
try:
client.delete_object(Bucket=BUCKET, Key=key)
except (ClientError, BotoCoreError):
raise unavailable() from None
def bootstrap():
try:
client.head_bucket(Bucket=BUCKET)
except ClientError as exc:
if exc.response["ResponseMetadata"]["HTTPStatusCode"] != 404:
raise
client.create_bucket(Bucket=BUCKET)
# New buckets are private by default. Remove any old public bucket policy.
client.delete_bucket_policy(Bucket=BUCKET)
client.head_bucket(Bucket=BUCKET)
if __name__ == "__main__":
try:
bootstrap()
except (ClientError, BotoCoreError):
raise SystemExit("S3 bootstrap failed") from None
print("PASS: private attachment bucket bootstrapped")
+47
View File
@@ -0,0 +1,47 @@
import base64
import hashlib
import re
import secrets
from argon2 import PasswordHasher
from argon2.exceptions import VerificationError, InvalidHashError
from src.errors import ApiError
# Explicit bounded Argon2id parameters: 19 MiB, 2 passes, one lane.
hasher = PasswordHasher(time_cost=2, memory_cost=19456, parallelism=1)
DUMMY_HASH = hasher.hash(secrets.token_urlsafe(32))
def unauthorized():
raise ApiError(401, "unauthorized", "Требуется действительная сессия")
def basic_credentials(header):
try:
scheme, value = header.split(" ", 1)
if scheme.lower() != "basic":
raise ValueError
username, password = base64.b64decode(value, validate=True).decode("utf-8").split(":", 1)
if not re.fullmatch(r"[a-z0-9_]{3,32}", username) or not 12 <= len(password) <= 128:
raise ValueError
return username, password
except (ValueError, UnicodeError, AttributeError):
unauthorized()
def verify(encoded, password):
try:
return hasher.verify(encoded or DUMMY_HASH, password)
except (VerificationError, InvalidHashError):
return False
def token_hash(header):
try:
scheme, token = header.split(" ", 1)
if scheme.lower() != "bearer" or not re.fullmatch(r"[A-Za-z0-9_-]{43}", token):
raise ValueError
return hashlib.sha256(token.encode("ascii")).digest()
except (ValueError, AttributeError):
unauthorized()
+174
View File
@@ -0,0 +1,174 @@
from datetime import datetime
from uuid import UUID, uuid4
from psycopg.errors import ForeignKeyViolation, UniqueViolation
from src.db import connection
from src.errors import ApiError
from src.pagination import decode_cursor, encode_cursor
def public(value):
if isinstance(value, dict):
return {key: public(item) for key, item in value.items()}
if isinstance(value, list):
return [public(item) for item in value]
if isinstance(value, UUID):
return str(value)
if isinstance(value, datetime):
return value.isoformat().replace("+00:00", "Z")
return value
def create_user(data, password_hash):
try:
with connection() as conn:
row = conn.execute(
"INSERT INTO users(id,username,display_name,password_hash) VALUES (%s,%s,%s,%s) RETURNING id,username,display_name,created_at",
(uuid4(), data.username, data.display_name, password_hash),
).fetchone()
return public(row)
except UniqueViolation:
raise ApiError(409, "conflict", "Username уже занят") from None
def create_chat(data, caller):
if caller not in data.member_ids:
raise ApiError(400, "invalid_request", "Создатель должен входить в member_ids")
try:
with connection() as conn:
chat = conn.execute("INSERT INTO chats(id,title) VALUES (%s,%s) RETURNING id,title,created_at", (uuid4(), data.title)).fetchone()
# FK errors here MUST roll back the already inserted parent chat.
with conn.cursor() as cursor:
cursor.executemany("INSERT INTO chat_members(chat_id,user_id) VALUES (%s,%s)", [(chat['id'], member) for member in data.member_ids])
chat['member_ids'] = data.member_ids
return public(chat)
except ForeignKeyViolation:
raise ApiError(400, "invalid_request", "Указан неизвестный участник") from None
CHAT_SELECT = """SELECT c.id,c.title,c.created_at,
ARRAY(SELECT user_id FROM chat_members WHERE chat_id=c.id ORDER BY user_id) AS member_ids
FROM chats c WHERE EXISTS (SELECT 1 FROM chat_members m WHERE m.chat_id=c.id AND m.user_id=%s)"""
def get_chat(chat_id, user_id):
with connection() as conn:
row = conn.execute(CHAT_SELECT + " AND c.id=%s", (user_id, chat_id)).fetchone()
if not row:
raise ApiError(404, "not_found", "Объект не найден")
return public(row)
def page(rows, limit, resource, owner):
items = public(rows[:limit])
cursor = None
if len(rows) > limit:
last = items[-1]
cursor = encode_cursor(resource, owner, last['created_at'], last['id'])
return {"items": items, "next_cursor": cursor}
def list_chats(user_id, limit, cursor):
params = [user_id]
sql = CHAT_SELECT
if cursor is not None:
timestamp, object_id = decode_cursor(cursor, "chats", user_id)
sql += " AND (c.created_at,c.id) > (%s::timestamptz,%s::uuid)"
params += [timestamp, object_id]
sql += " ORDER BY c.created_at,c.id LIMIT %s"
params.append(limit + 1)
with connection() as conn:
rows = conn.execute(sql, params).fetchall()
return page(rows, limit, "chats", user_id)
MESSAGE_SELECT = """SELECT m.id,m.chat_id,m.sender_id,m.text,m.created_at,
ARRAY(SELECT ma.attachment_id FROM message_attachments ma
WHERE ma.message_id=m.id ORDER BY ma.position) AS attachment_ids
FROM messages m"""
def check_attachments(conn, chat_id, attachment_ids):
if not attachment_ids:
return
rows = conn.execute(
"""SELECT id,status FROM attachments
WHERE chat_id=%s AND id=ANY(%s::uuid[]) ORDER BY id FOR SHARE""",
(chat_id, attachment_ids),
).fetchall()
if len(rows) != len(attachment_ids):
raise ApiError(404, "not_found", "Объект не найден")
if any(row["status"] == "failed" for row in rows):
raise ApiError(409, "conflict", "Вложение недоступно")
def create_message(chat_id, user_id, method, key, text, attachment_ids):
scope = (user_id, method, chat_id, key)
message_id = uuid4()
with connection() as conn:
if not conn.execute(
"SELECT 1 FROM chat_members WHERE chat_id=%s AND user_id=%s FOR KEY SHARE",
(chat_id, user_id),
).fetchone():
raise ApiError(404, "not_found", "Объект не найден")
# A concurrent ON CONFLICT waits for the first transaction. The deferred
# FK allows reserving the proposed message ID before inserting the row.
claimed = conn.execute(
"""INSERT INTO message_idempotency (user_id,method,chat_id,key,message_id,expires_at)
VALUES (%s,%s,%s,%s,%s,clock_timestamp() + interval '24 hours')
ON CONFLICT (user_id,method,chat_id,key) DO UPDATE
SET message_id=EXCLUDED.message_id, expires_at=clock_timestamp() + interval '24 hours'
WHERE message_idempotency.expires_at <= clock_timestamp()
RETURNING message_id""",
(*scope, message_id),
).fetchone()
if claimed:
check_attachments(conn, chat_id, attachment_ids)
row = conn.execute(
"""INSERT INTO messages (id,chat_id,sender_id,text)
VALUES (%s,%s,%s,%s)
RETURNING id,chat_id,sender_id,text,created_at""",
(message_id, chat_id, user_id, text),
).fetchone()
if attachment_ids:
with conn.cursor() as cursor:
cursor.executemany(
"""INSERT INTO message_attachments
(chat_id,message_id,attachment_id,position) VALUES (%s,%s,%s,%s)""",
[
(chat_id, message_id, attachment_id, position)
for position, attachment_id in enumerate(attachment_ids, 1)
],
)
row["attachment_ids"] = attachment_ids
else:
# READ COMMITTED gives this statement a new snapshot. The UPSERT
# still holds the key row locked, including when its WHERE was false.
row = conn.execute(
MESSAGE_SELECT + """ JOIN message_idempotency i ON i.message_id=m.id
WHERE (i.user_id,i.method,i.chat_id,i.key)=(%s,%s,%s,%s)""",
scope,
).fetchone()
if row["text"] != text or [str(value) for value in row["attachment_ids"]] != attachment_ids:
raise ApiError(409, "conflict", "Ключ уже использован с другим сообщением")
return public(row)
def list_messages(chat_id, user_id, limit, cursor):
with connection() as conn:
# Access precedes cursor decoding, including malformed foreign cursors.
if not conn.execute("SELECT 1 FROM chat_members WHERE chat_id=%s AND user_id=%s", (chat_id, user_id)).fetchone():
raise ApiError(404, "not_found", "Объект не найден")
params = [chat_id]
sql = MESSAGE_SELECT + " WHERE m.chat_id=%s"
if cursor is not None:
timestamp, object_id = decode_cursor(cursor, "messages", chat_id)
sql += " AND (created_at,id) > (%s::timestamptz,%s::uuid)"
params += [timestamp, object_id]
sql += " ORDER BY created_at,id LIMIT %s"
params.append(limit + 1)
rows = conn.execute(sql, params).fetchall()
return page(rows, limit, "messages", chat_id)
+25
View File
@@ -0,0 +1,25 @@
"""Create the single promised grade 3 variant."""
from io import BytesIO
from PIL import Image, ImageOps
def thumbnail(data):
with Image.open(BytesIO(data), formats=("PNG", "JPEG")) as image:
oriented = ImageOps.exif_transpose(image)
width, height = oriented.size
divisor = max(width, height, 256)
size = (max(1, width * 256 // divisor), max(1, height * 256 // divisor))
if "A" in oriented.getbands() or "transparency" in oriented.info:
rgba = oriented.convert("RGBA")
white = Image.new("RGB", rgba.size, "white")
white.paste(rgba, mask=rgba.getchannel("A"))
rgb = white
else:
rgb = oriented.convert("RGB")
if rgb.size != size:
rgb = rgb.resize(size, Image.Resampling.LANCZOS)
result = BytesIO()
rgb.save(result, format="JPEG", quality=85)
return result.getvalue(), size
+164
View File
@@ -0,0 +1,164 @@
"""One AMQP consumer with durable attempt accounting and idempotent effects."""
import hashlib
import logging
import time
from io import BytesIO
import pika
import psycopg
from psycopg.rows import dict_row
from src import jobs, s3
from src.config import secret
from src.errors import ApiError
from src.thumbnail import thumbnail
LOG = logging.getLogger(__name__)
RETRY_SECONDS = 3
def select_job(conn, job):
return conn.execute(
"""SELECT j.*, a.status, a.object_key, a.content_type, a.size_bytes, a.sha256,
EXISTS (SELECT 1 FROM attachment_variants v
WHERE (v.attachment_id, v.pipeline_version, v.name) =
(j.attachment_id, j.pipeline_version, 'thumbnail')) AS has_thumbnail
FROM attachment_jobs j JOIN attachments a ON a.id=j.attachment_id
WHERE (j.job_id,j.attachment_id,j.pipeline_version,j.created_at)=(%s,%s,%s,%s)""",
(job.job_id, job.attachment_id, job.pipeline_version, job.created_at),
).fetchone()
def state(conn, row, status, error=None):
conn.execute(
"""UPDATE attachments SET status=%s,error_code=%s,updated_at=clock_timestamp()
WHERE id=%s""",
(status, error, row["attachment_id"]),
)
def retry_or_fail(conn, row, reason, permanent=False):
attempt = row["attempt"]
conn.execute("UPDATE attachment_jobs SET last_error=%s WHERE job_id=%s", (reason, row["job_id"]))
if permanent or attempt == row["max_attempts"]:
code = reason if permanent else "retry_exhausted"
state(conn, row, "failed", code)
return ("failed", code)
conn.execute(
"""UPDATE attachment_jobs SET attempt=attempt+1,
retry_at=clock_timestamp()+(%s * interval '1 second') WHERE job_id=%s""",
(RETRY_SECONDS, row["job_id"]),
)
state(conn, row, "queued")
return ("retry", reason)
def objects(row):
original = s3.download(row["staging_key"], row["size_bytes"])
if len(original) != row["size_bytes"] or hashlib.sha256(original).hexdigest() != row["sha256"]:
raise ValueError("Staging checksum mismatch")
data, (width, height) = thumbnail(original)
key = f"variants/{row['attachment_id']}/v{row['pipeline_version']}/thumbnail.jpg"
s3.put(row["object_key"], BytesIO(original), len(original), row["content_type"])
s3.put(key, BytesIO(data), len(data), "image/jpeg")
return key, len(data), hashlib.sha256(data).hexdigest(), width, height
def process(conn, row):
try:
variant = objects(row)
except ApiError:
with conn.transaction():
return retry_or_fail(conn, row, "storage_unavailable")
except (ValueError, OSError, SyntaxError, IndexError):
with conn.transaction():
return retry_or_fail(conn, row, "invalid_image", permanent=True)
with conn.transaction():
conn.execute(
"""INSERT INTO attachment_variants
(attachment_id,pipeline_version,name,content_type,object_key,size_bytes,sha256,width,height)
VALUES (%s,%s,'thumbnail','image/jpeg',%s,%s,%s,%s,%s)""",
(row["attachment_id"], row["pipeline_version"], *variant),
)
state(conn, row, "ready")
try:
s3.delete(row["staging_key"])
except ApiError:
LOG.warning("staging retained job=%s", row["job_id"])
return ("ready", None)
def deliver(channel, method, _properties, body):
try:
job = jobs.parse(body)
except ValueError:
LOG.warning("rejected invalid carrier")
channel.basic_reject(method.delivery_tag, requeue=False)
return
with psycopg.connect(secret("DATABASE_URL"), row_factory=dict_row, connect_timeout=3,
options="-c statement_timeout=5000 -c lock_timeout=3000",
autocommit=True) as conn:
# The session lock spans state commits and S3 writes. Closing the database
# session releases it, including on worker crash. Prefetch is one.
conn.execute("SELECT pg_advisory_lock(hashtextextended(%s, 0))", (str(job.job_id),))
row = select_job(conn, job)
if row is None or job.attempt > row["attempt"]:
LOG.warning("rejected unregistered carrier job=%s", job.job_id)
channel.basic_reject(method.delivery_tag, requeue=False)
return
status = row["status"]
if status == "ready":
if not row["has_thumbnail"]:
raise RuntimeError("Missing committed thumbnail")
outcome, reason = "duplicate_ready", None
elif status == "failed":
outcome, reason = "duplicate_failed", None
elif status == "processing":
# A prior consumer committed its claim but lost its session before
# finalizing. Its attempt counts against the same persistent budget.
with conn.transaction():
outcome, reason = retry_or_fail(conn, row, "interrupted")
elif job.attempt < row["attempt"]:
# Covers interruption after committing retry but before publisher confirm.
outcome, reason = "republish_retry", None
else:
delay = conn.execute(
"SELECT GREATEST(0,EXTRACT(EPOCH FROM (%s-clock_timestamp()))) AS delay",
(row["retry_at"],),
).fetchone()["delay"]
channel.connection.sleep(float(delay))
with conn.transaction():
state(conn, row, "processing")
LOG.info("job=%s attempt=%s status=processing", job.job_id, job.attempt)
outcome, reason = process(conn, row)
if outcome in {"retry", "republish_retry"}:
attempt = row["attempt"] + (outcome == "retry")
jobs.publish(channel, jobs.carrier(job.job_id, job.attachment_id, job.created_at, attempt))
channel.basic_ack(method.delivery_tag)
LOG.info("job=%s attempt=%s outcome=%s reason=%s", job.job_id, job.attempt, outcome, reason)
def main():
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
while True:
try:
with jobs.broker() as channel:
channel.basic_qos(prefetch_count=1)
channel.basic_consume(queue=jobs.QUEUE, on_message_callback=deliver, auto_ack=False)
LOG.info("consumer ready prefetch=1")
channel.start_consuming()
except (pika.exceptions.AMQPError, psycopg.Error, OSError) as exc:
LOG.warning("consumer reconnecting after %s", type(exc).__name__)
time.sleep(2)
if __name__ == "__main__":
main()
+12
View File
@@ -0,0 +1,12 @@
# Проверки ЛР5
В Windows PowerShell поднимите стенд по REPORT.md, затем выполните:
```powershell
docker compose --profile test build tools
docker compose run --rm --no-deps tools python student-tests/grade3.py
```
Тест использует Docker socket из профиля tools и действует только на сервисы локального Compose-проекта `mtusi-lab5-grade3` или его именованного варианта с суффиксом. Он останавливает и запускает worker, API, S3 и RabbitMQ, временно блокирует запись одного типа данных в PostgreSQL для наблюдения processing и изменяет staging двух собственных вложений для инъекции ошибок. Не запускайте его на стенде с чужой нагрузкой или одновременно с публичными сценариями отказов.
Сохраняются тестовые пользователи, чаты, файлы и журнал попыток. Токены используются в памяти. Сценарий сам восстанавливает затронутые сервисы при проверяемых сбоях. После прерывания процесса состояние стенда можно привести к норме командой `docker compose up -d --wait --wait-timeout 150`.
+444
View File
@@ -0,0 +1,444 @@
"""Reproduce Lab 5 acceptance scenarios using only local Compose services."""
import hashlib
import json
import os
from concurrent.futures import ThreadPoolExecutor
from contextlib import contextmanager
from io import BytesIO
from pathlib import Path
import re
import subprocess
import sys
import time
from uuid import UUID, uuid4
from jsonschema import Draft202012Validator, FormatChecker
from PIL import Image
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "tests"))
from client import Client, Failure, check, run
PROJECT = os.getenv("LAB5_PROJECT", "")
SERVICES = {"api1", "api2", "broker", "worker", "postgres", "s3", "lb"}
JOB_SCHEMA = json.loads((ROOT / "contracts/jobs.schema.json").read_text(encoding="utf-8"))
SAMPLE = (ROOT / "tests/fixtures/sample.png").read_bytes()
def docker(*args):
completed = subprocess.run(["docker", *args], capture_output=True, text=True, timeout=45)
check(completed.returncode == 0, f"docker {args[0]} returned {completed.returncode}")
return (completed.stdout + (completed.stderr if args[0] == "logs" else "")).strip()
def container(service):
check(service in SERVICES, "unexpected service")
ids = docker("ps", "-aq", "--filter", f"label=com.docker.compose.project={PROJECT}",
"--filter", f"label=com.docker.compose.service={service}",
"--filter", "label=com.docker.compose.oneoff=False").splitlines()
check(len(ids) == 1, f"expected one container for {service}")
return ids[0]
def action(verb, *services):
ids = [container(service) for service in services]
print(f"docker {verb} {' '.join(services)} containers={','.join(ids)}")
extra = ("--time", "2") if verb in {"stop", "restart"} else ()
docker(verb, *extra, *ids)
def inspect(service, fmt):
return docker("inspect", "--format", fmt, container(service))
def inside(service, *args):
return docker("exec", container(service), *args)
def psql_args():
return ("psql", "-XAt", "-v", "ON_ERROR_STOP=1", "-U", "messenger", "-d", "messenger")
def sql(statement):
return inside("postgres", *psql_args(), "-c", statement)
@contextmanager
def block_variant_insert():
# Compatible with metadata SELECT, blocks the worker's final INSERT only.
args = ["docker", "exec", "-i", container("postgres"), *psql_args()]
proc = subprocess.Popen(args, stdin=subprocess.PIPE, stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL, text=True)
try:
proc.stdin.write("BEGIN; LOCK attachment_variants IN SHARE MODE; SELECT 'locked';\n")
proc.stdin.flush()
for _ in range(3):
if proc.stdout.readline().strip() == "locked":
break
else:
raise Failure("could not lock variants")
yield
finally:
try:
proc.communicate("COMMIT;\n", timeout=5)
except subprocess.TimeoutExpired:
proc.kill()
proc.wait(timeout=5)
raise Failure("could not release variants lock") from None
check(proc.returncode == 0, "variant lock session failed")
def eventually(description, fn, seconds=45, interval=0.2):
start = time.monotonic()
last = ""
while time.monotonic() - start < seconds:
try:
value = fn()
if value:
print(f"{description} after {time.monotonic()-start:.3f}s")
return value
except Failure as exc:
last = str(exc)
time.sleep(interval)
raise Failure(f"timeout waiting for {description}: {last}")
def ready(client):
eventually("LB ready", lambda: client.call("GET", "/health/ready", schema="Health"))
def metadata(client, actor, attachment):
path = f"/api/v1/attachments/{attachment['id']}"
return client.call("GET", path, actor=actor, schema="Attachment")["json"]
def expect_unavailable(client, actor, attachment):
path = f"/api/v1/attachments/{attachment['id']}/content"
for variant in ("original", "thumbnail"):
client.call("GET", f"{path}?variant={variant}", 409, actor=actor)
def upload(client, actor, chat, data=SAMPLE, mime="image/png", expected=202, filename="fixture"):
response = client.call("POST", f"/api/v1/chats/{chat['id']}/attachments?filename={filename}",
expected, actor=actor, raw=data, headers={"Content-Type": mime},
schema="Attachment" if expected == 202 else None)
if expected == 202:
row = response["json"]
check(row["status"] == "queued" and row["variants"] == [] and row["error_code"] is None,
"expected queued snapshot")
check((row["size_bytes"], row["sha256"], row["content_type"]) ==
(len(data), hashlib.sha256(data).hexdigest(), mime), "staging metadata mismatch")
return response
def assert_ready(client, actor, attachment, data, dimensions, mime="image/png"):
start = time.monotonic()
row = client.wait_ready(actor, attachment)
check(row["error_code"] is None and len(row["variants"]) == 1, "invalid ready result")
check((row["content_type"], row["size_bytes"], row["sha256"]) ==
(mime, len(data), hashlib.sha256(data).hexdigest()), "original metadata changed")
content_path = f"/api/v1/attachments/{attachment['id']}/content"
original = client.call("GET", content_path, actor=actor)
check(original["bytes"] == data and original["headers"].get_content_type() == mime,
"original content changed")
thumb = client.call("GET", content_path + "?variant=thumbnail", actor=actor)
check(thumb["headers"].get_content_type() == "image/jpeg", "thumbnail HTTP MIME")
variant = row["variants"][0]
check(variant["name"] == "thumbnail" and variant["content_type"] == "image/jpeg"
and (variant["width"], variant["height"]) == dimensions, "thumbnail metadata")
check((variant["sha256"], variant["size_bytes"]) ==
(hashlib.sha256(thumb["bytes"]).hexdigest(), len(thumb["bytes"])), "thumbnail checksum")
with Image.open(BytesIO(thumb["bytes"])) as im:
im.load()
check(im.format == "JPEG" and im.size == dimensions and not im.getexif(),
"invalid thumbnail format, dimensions or EXIF")
pixel = im.getpixel((0, 0))
print(f"ready id={row['id']} wait={time.monotonic()-start:.3f}s original_sha256={row['sha256']} "
f"jpeg={dimensions[0]}x{dimensions[1]} thumb_sha256={variant['sha256']} "
f"read={original['headers']['X-Instance-Id']}/{thumb['headers']['X-Instance-Id']}")
return row, pixel
def job_state(attachment):
aid = UUID(attachment["id"])
return sql(f"""SELECT a.status,j.attempt,COALESCE(j.last_error,''),j.max_attempts
FROM attachments a JOIN attachment_jobs j ON a.id=j.attachment_id
WHERE a.id='{aid}'""")
def private(service, key):
status = inside(service, "python", "-c", """
import sys
from urllib.error import HTTPError
from urllib.request import urlopen
from src import s3
try:
urlopen(f"http://s3:9000/{s3.BUCKET}/{sys.argv[1]}", timeout=3).close()
except HTTPError as error:
print(error.code)
else:
print(200)
""", key)
check(status == "403", "unsigned S3 request was not rejected")
print("private RustFS object unsigned GET HTTP 403")
def queued_carrier():
# Observe the real persistent AMQP message and put it back without ack.
raw = inside("api1", "python", "-c", """
import json
from src import jobs
with jobs.broker() as channel:
method, props, body = channel.basic_get(jobs.QUEUE, auto_ack=False)
if method:
channel.basic_nack(method.delivery_tag, requeue=True)
print(json.dumps({"delivery_mode":props.delivery_mode, "job":json.loads(body)}))
""")
check(bool(raw), "no queued job in RabbitMQ")
observed = json.loads(raw)
check(observed["delivery_mode"] == 2, "AMQP message is not persistent")
carrier = observed["job"]
Draft202012Validator(JOB_SCHEMA, format_checker=FormatChecker()).validate(carrier)
print(f"RabbitMQ persistent carrier schema valid job={carrier['job_id']} "
f"attempt={carrier['attempt']} fields={','.join(sorted(carrier))}")
return carrier
def queue_stats():
text = inside("broker", "rabbitmqctl", "-q", "list_queues",
"name", "durable", "messages_ready", "messages_unacknowledged")
for line in text.splitlines():
fields = line.split()
if fields and fields[0] == "attachment.process.v1":
check(fields[1] == "true", "queue durability is off")
print(f"broker queue={fields[0]} durable={fields[1]} ready={fields[2]} unacked={fields[3]}")
return tuple(map(int, fields[2:]))
raise Failure("RabbitMQ queue absent")
def duplicate(client, actor, attachment, carrier, ready_snapshot):
inside("api1", "python", "-c", """
import sys
from src import jobs
job = jobs.parse(sys.argv[1])
with jobs.broker() as channel:
jobs.publish(channel, job)
jobs.publish(channel, job)
""", json.dumps(carrier))
worker = container("worker")
job_id = carrier["job_id"]
eventually("worker acknowledged two ready duplicates", lambda:
docker("logs", worker).count(f"job={job_id} attempt=1 outcome=duplicate_ready") >= 2)
check(metadata(client, actor, attachment) == ready_snapshot, "duplicate changed ready metadata")
aid = UUID(attachment["id"])
check(sql(f"SELECT count(*) FROM attachment_variants WHERE attachment_id='{aid}'") == "1",
"duplicate created another variant row")
count = inside("api1", "python", "-c", """
import sys
from src import s3
result = s3.client.list_objects_v2(Bucket=s3.BUCKET, Prefix=f"variants/{sys.argv[1]}/")
print(result["KeyCount"])
""", str(aid))
check(count == "1", "duplicate created another variant object")
print("PASS duplicate delivery: ready snapshot unchanged, one SQL row and one S3 variant")
def restart_scenarios(c, alice, bob, eve, chat):
action("stop", "worker")
try:
a = upload(c, alice, chat)["json"]
carrier = queued_carrier()
check(carrier["attachment_id"] == a["id"] and carrier["attempt"] == 1,
"RabbitMQ job identity differs from upload")
b = upload(c, bob, chat)["json"]
check(a["id"] != b["id"], "distinct uploads reused ID")
key = uuid4().hex
with ThreadPoolExecutor(max_workers=8) as pool:
calls = list(pool.map(lambda _: c.message(bob, chat, "", key, [a["id"]]), range(8)))
check(len({x["json"]["id"] for x in calls}) == 1, "idempotency race produced duplicates")
observed = {x["headers"]["X-Instance-Id"] for x in calls}
check(observed == {"api1", "api2"}, "idempotency requests did not use both replicas")
check(calls[0]["json"]["attachment_ids"] == [a["id"]], "queued attachment link lost")
c.message(alice, chat, "ordinary text with worker stopped")
check(metadata(c, bob, a) == a, "queued metadata changed with worker stopped")
expect_unavailable(c, bob, a)
private("api1", f"staging/{a['id']}")
print(f"PASS stopped worker: uploads=202 ids={a['id']},{b['id']}; queued content=409; "
"linked and ordinary POST=201; concurrent same-key response identical via api1/api2")
starts = {s: inspect(s, "{{.State.StartedAt}}") for s in ("api1", "api2")}
action("restart", "api1", "api2")
ready(c)
for service, old in starts.items():
new = inspect(service, "{{.State.StartedAt}}")
check(new != old, f"{service} did not actually restart")
print(f"{service} started before={old} after={new}")
check(metadata(c, bob, a) == a and job_state(a).startswith("queued|1|"),
"accepted job lost after API restart")
volume_format = "{{range .Mounts}}{{if eq .Destination \"/var/lib/rabbitmq\"}}{{.Name}}{{end}}{{end}}"
volume = inspect("broker", volume_format)
check(volume == f"{PROJECT}_broker_data", "unexpected RabbitMQ volume")
check(queue_stats() == (2, 0), "expected two pending jobs before broker restart")
broker_start = inspect("broker", "{{.State.StartedAt}}")
action("restart", "broker")
eventually("broker healthy", lambda: inspect("broker", "{{.State.Health.Status}}") == "healthy")
broker_after = inspect("broker", "{{.State.StartedAt}}")
check(broker_start != broker_after and inspect("broker", volume_format) == volume,
"broker restart or volume identity mismatch")
print(f"broker started before={broker_start} after={broker_after} volume={volume}")
check(queue_stats() == (2, 0), "broker restart lost accepted jobs")
check(metadata(c, bob, b) == b, "second job metadata lost")
print("PASS both APIs + RabbitMQ restarted with queued work and unchanged RabbitMQ volume")
path = f"/api/v1/attachments/{a['id']}"
c.call("GET", path, 401)
c.call("GET", path + "/content?variant=thumbnail", 404, actor=eve)
with block_variant_insert():
action("start", "worker")
processing = eventually(
"HTTP processing state",
lambda: next((row for row in (a, b) if metadata(c, bob, row)["status"] == "processing"), None),
6, interval=0.05,
)
expect_unavailable(c, bob, processing)
c.message(bob, chat, "linked during processing", attachments=[processing["id"]])
print("PASS processing metadata visible, content=409, linking=201 before final SQL commit")
a_ready, _ = assert_ready(c, bob, a, SAMPLE, (256, 160))
assert_ready(c, alice, b, SAMPLE, (256, 160))
duplicate(c, bob, a, carrier, a_ready)
private("api1", f"originals/{a['id']}")
c.call("GET", path + "/content?variant=unknown", 404, actor=bob)
finally:
action("start", "worker")
def image_bytes(image, fmt, **kwargs):
data = BytesIO()
image.save(data, fmt, **kwargs)
return data.getvalue()
def input_and_image_cases(c, alice, eve, chat):
for data, mime, status, actor, filename in (
(SAMPLE, "image/jpeg", 415, alice, "wrong-mime"),
(SAMPLE[:-8], "image/png", 415, alice, "corrupt"),
(b"x" * (10 * 1024 * 1024 + 1), "image/png", 413, alice, "oversize"),
(SAMPLE, "image/png", 404, eve, "private"),
(SAMPLE, "image/png", 401, None, "anonymous"),
(SAMPLE, "image/png", 400, alice, "invalid%2Fname.png"),
):
upload(c, actor, chat, data, mime, status, filename)
print(f"negative upload {filename} HTTP {status}")
huge = image_bytes(Image.new("1", (8001, 5000)), "PNG")
upload(c, alice, chat, huge, expected=413)
print("negative upload 40,005,000 decoded pixels HTTP 413")
exif = Image.Exif()
exif[274] = 6
original = image_bytes(Image.new("RGB", (319, 101), "navy"), "JPEG", exif=exif)
broken = original[:100]
upload(c, alice, chat, broken, mime="image/jpeg", expected=415)
print("negative truncated JPEG HTTP 415")
oriented = upload(c, alice, chat, original, "image/jpeg")["json"]
assert_ready(c, alice, oriented, original, (81, 256), "image/jpeg")
original = image_bytes(Image.new("RGBA", (37, 19), (255, 0, 0, 0)), "PNG")
alpha = upload(c, alice, chat, original)["json"]
_, pixel = assert_ready(c, alice, alpha, original, (37, 19))
check(min(pixel) >= 250, "transparent pixel was not composited on white")
print(f"PASS true JPEG, EXIF orientation, floor rounding, no upscale, alpha pixel={pixel}")
def failure_cases(c, alice, bob, chat):
action("stop", "worker")
try:
a = upload(c, alice, chat)["json"]
action("stop", "s3")
action("start", "worker")
state = eventually("persistent retry after real S3 stop", lambda:
(s := job_state(a)) and "storage_unavailable" in s and s)
print(f"storage outage SQL status|attempt|reason|max={state}")
finally:
action("start", "s3", "worker")
ready(c)
assert_ready(c, bob, a, SAMPLE, (256, 160))
check(int(job_state(a).split("|")[1]) >= 2, "no durable retries recorded")
print("PASS temporary S3 outage recovered the same attachment")
action("stop", "worker")
try:
missing = upload(c, alice, chat)["json"]
corrupt = upload(c, alice, chat)["json"]
inside("api1", "python", "-c", """
from io import BytesIO
import sys
from src import s3
s3.delete(f"staging/{sys.argv[1]}")
damaged = BytesIO(b"X" * int(sys.argv[3]))
s3.put(f"staging/{sys.argv[2]}", damaged, damaged.getbuffer().nbytes, "image/png")
""", missing["id"], corrupt["id"], str(corrupt["size_bytes"]))
finally:
action("start", "worker")
for row, error in ((corrupt, "invalid_image"), (missing, "retry_exhausted")):
failed = eventually(f"failed {row['id']}", lambda:
(meta := metadata(c, bob, row))["status"] == "failed" and meta)
check(failed["error_code"] == error and failed["variants"] == [], "unsafe failed result")
state = job_state(row)
status, current, _, maximum = state.split("|")
check(status == "failed" and int(current) == (1 if error == "invalid_image" else int(maximum)),
"unexpected number of attempts")
expect_unavailable(c, bob, row)
c.call("POST", f"/api/v1/chats/{chat['id']}/messages", 409, actor=bob,
body={"text": "", "attachment_ids": [row["id"]]},
headers={"Idempotency-Key": uuid4().hex})
print(f"PASS safe failure code={error}; status|attempt|reason|max={state}; content/link HTTP 409")
action("stop", "broker")
try:
response = upload(c, alice, chat, expected=503)
check(response["json"]["error"]["code"] == "unavailable", "unexpected queue-down response")
c.message(alice, chat, "normal API works when broker is stopped")
print("PASS unavailable broker upload=503 (no false 202), ordinary message=201")
finally:
action("start", "broker")
eventually("broker healthy after failure case", lambda:
inspect("broker", "{{.State.Health.Status}}") == "healthy")
def replica_cases(c, alice, bob, chat):
for stopped, survivor in (("api1", "api2"), ("api2", "api1")):
action("stop", stopped)
try:
eventually(f"{survivor} handles existing Bearer with {stopped} stopped", lambda:
c.call("GET", "/api/v1/users/me", actor=bob, schema="User")["headers"]
["X-Instance-Id"] == survivor)
response = upload(c, alice, chat)
check(response["headers"]["X-Instance-Id"] == survivor, "wrong upload replica")
assert_ready(c, bob, response["json"], SAMPLE, (256, 160))
finally:
action("start", stopped)
eventually(f"{stopped} container healthy", lambda:
inspect(stopped, "{{.State.Health.Status}}") == "healthy")
print("PASS authorized new uploads and thumbnail reads with either replica stopped")
def main():
check(re.fullmatch(r"mtusi-lab5-grade3(?:-[a-z0-9-]+)?", PROJECT), "unsafe project name")
c = Client("http://lb:8080")
ready(c)
alice, bob, eve, chat = c.bootstrap()
alice_id = UUID(alice["id"])
check(sql(f"SELECT password_hash LIKE '$argon2id$%' FROM users WHERE id='{alice_id}'") == "t",
"stored verifier is not Argon2id")
print("PASS Argon2id verifier confirmed without reading hash")
restart_scenarios(c, alice, bob, eve, chat)
input_and_image_cases(c, alice, eve, chat)
failure_cases(c, alice, bob, chat)
replica_cases(c, alice, bob, chat)
check(queue_stats() == (0, 0), "queue did not drain")
print("PASS Lab 5 grade 3 Docker scenarios")
if __name__ == "__main__":
run(main)