ЛР 4 - БВТ2302 - Миронов - на 4 #48

Closed
ALMiron wants to merge 2 commits from ALMiron/mtusi2026lab4-fork:solution/bvt2302-mironov into main
27 changed files with 1945 additions and 0 deletions
+13
View File
@@ -0,0 +1,13 @@
FROM python:3.12.11-slim-bookworm
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1 PIP_DISABLE_PIP_VERSION_CHECK=1
WORKDIR /app
RUN addgroup --system app && adduser --system --ingroup app app
COPY requirements.txt ./requirements.txt
RUN python -m pip install --no-cache-dir -r requirements.txt
COPY src ./src
COPY migrations ./migrations
COPY scripts ./scripts
USER app
EXPOSE 8080
STOPSIGNAL SIGTERM
CMD ["uvicorn", "src.main:app", "--host", "0.0.0.0", "--port", "8080", "--timeout-graceful-shutdown", "10", "--proxy-headers"]
+17
View File
@@ -28,3 +28,20 @@ test-persistence:
test-failover:
@test -n "$(ACTION_SCRIPT)" || (echo "Set ACTION_SCRIPT to your implemented executable stop-one hook"; exit 2)
$(PYTHON) tests/resilience.py --mode failover --action-script "$(ACTION_SCRIPT)" --base-url "$(BASE_URL)" $(TLS_ARGS)
# ---- student additions (lab4); public targets above are unchanged ----
.PHONY: test-lab3 test-level4 check-s3 sweep-dry memory rolling-restart
test-lab3:
$(PYTHON) student-tests/lab3_idempotency_checks.py --base-url "$(BASE_URL)"
test-level4:
$(PYTHON) student-tests/level4_checks.py --base-url "$(BASE_URL)"
$(PYTHON) -m unittest discover -s student-tests -p "test_*.py"
check-s3:
sh scripts/check_s3_exposure.sh
docker compose run --rm --no-deps --entrypoint python api scripts/check_s3_private.py
sweep-dry:
docker compose run --rm --no-deps s3_sweeper python scripts/s3_sweep.py --grace-seconds 3600
memory:
$(PYTHON) student-tests/memory_experiment.py --base-url "$(BASE_URL)" --concurrency 10
rolling-restart:
sh scripts/rolling-restart.sh
+139
View File
@@ -0,0 +1,139 @@
# Отчёт - лабораторная 4
- Автор, группа: Алексей Миронов, бвт2302
- Целевая оценка: **4**; трек 5 не заявлен
- Commit/PR: 6c0a3d8, https://gitea.privcloud.tech/mtusi2026/mtusi2026lab4/pulls/48
- Использованные источники, библиотеки и помощь ИИ: скелет lab4 (README, COURSE, contracts, tests); собственная реализация lab3 (FastAPI, PostgreSQL, Redis, HAProxy, идемпотентность); FastAPI, Pydantic, psycopg 3, boto3, Pillow, MinIO + mc, HAProxy, Docker Compose; помощь ИИ
## Запуск из чистого clone
Нужны Docker Engine + Compose v2, GNU Make, Python 3.9+ (для тестов).
```bash
cp .env.example .env
make check
make validate
docker compose --env-file .env config --quiet
docker compose --env-file .env up -d --build --scale api=2
curl -i http://localhost:8080/health/ready # дождаться 200
make test # публичный smoke lab4
make test-lab3 # перенесённые проверки lab3 (идемпотентность, 2 реплики) - опционально
make test-level4 # негативные тесты вложений + unit-тест компенсации
make check-s3 # private bucket вне API, права app-пользователя, консоль закрыта
python3 student-tests/orphan_sweep_check.py # сверка/очистка: grace period, dry-run, идемпотентность
(Скачивает настоящее вложение, сверяет байты и одновременно создаёт сироту (orphan) в Minio и проверяет, что в grace период она жива, dry-run её показывает, apply удаляет, а нужное вложение цело (сторож (sweeper) уничтожает только сирот, у которых закончился период помилования) и повтор ничего не делает (идемпотентность))
make memory # >=10 параллельных загрузок, пик RAM реплик
make test-persistence ACTION_SCRIPT=scripts/restart.sh (пишет данные, перезапускает постгрес, редис, minio, api и проверяет, что данные, токен и вложения остались)
make test-failover ACTION_SCRIPT=scripts/stop-one.sh (загружает вложение, останавливает одну реплику api и проверяет, что скачивание работает через другую) # затем: docker compose up -d --scale api=2 api; повторить
```
Объяснение: какие рукотворные мейки что запускают
make test-lab3:
$(PYTHON) student-tests/lab3_idempotency_checks.py --base-url "$(BASE_URL)"
make test-level4:
$(PYTHON) student-tests/level4_checks.py --base-url "$(BASE_URL)"
(негативные сценарии вложений. Файл больше 10 мб (413), пиксельная бомба, ровно 40 Мпикс принимается, битые Png/jpg (415), подмена MIME, плохие filename, чужое и неизвестное вложение, вложение из другого чата, порядок и идемпотентность с attachment_ids, 10 параллельных загрузок, в конце проверяется, что ни один ответ не был 5xx)
$(PYTHON) -m unittest discover -s student-tests -p "test_compensation.py"
(Если запись БД падает после S3 PUT, удаляется ровно тот объект, что записали. Битый файл до S3 не доходит. Сбой PUT тоже компенсируется и даёт 503)
make check-s3:
sh scripts/check_s3_exposure.sh
(порты S3 не опубликованы, с хоста :9000 и :9001 недоступны)
docker compose run --rm --no-deps --entrypoint python api scripts/check_s3_private.py
(из контейнера, мимо API. Анонимные GET/, HEAD/список/PUT дают 403. У ключа приложения нет прав на список бакетов, создание или удаление bucket, смену политики и запись вне attachments/. Порты консоли закрыты. )
make memory:
$(PYTHON) student-tests/memory_experiment.py --base-url "$(BASE_URL)" --concurrency 10
(10 одновременных загрузок, читает пик RAM каждой реплики. Есть 1 профиль: 9.6 мб несжимаемых байт. Доказывает, что даже 10 одновременных загрузок не забивают память (благодаря стримингу/чанкингу))
Остановка с сохранением данных: `docker compose --env-file .env down`. Намеренное удаление учебных данных (БД, Redis, S3): `docker compose --env-file .env down -v`. Идемпотентность bootstrap: `docker compose run --rm s3_init` можно запускать повторно (exit 0).
## Архитектура и решения
Клиент -> HAProxy (единственный опубликованный порт, 127.0.0.1:8080) -> API×2 (FastAPI) -> PostgreSQL (метаданные) + Redis (сессии, AOF) + S3 MinIO-совместимый сервер Silo (форк MinIO) (байты, приватный bucket). Сети: `frontend` (только HAProxy) и `backend` (`internal: true`, без маршрута наружу) - S3, PostgreSQL, Redis и API недоступны с хоста.
| Роль | Сервис | Версия | Health | Volume | Порты наружу |
| --- | --- | --- | --- | --- | --- |
| LB | load_balancer | haproxy:3.2.5-alpine | `haproxy -c` | - | 127.0.0.1:8080 |
| API | api (×2) | python:3.12.11-slim-bookworm | /health/ready | - (без локального состояния) | нет |
| БД | postgres | postgres:17.6-alpine | pg_isready | postgres_data | нет |
| Сессии | redis | redis:7.4.2-alpine | redis-cli ping | redis_data | нет |
| S3 | s3 | pgsty/silo:RELEASE.2026-10-15T00-00-00Z | mc ready local | s3_data | нет; консоль выключена (`MINIO_BROWSER=off`) |
| Bootstrap S3 | s3_init (one-shot) | pgsty/silo:RELEASE.2026-10-15T00-00-00Z | - | - | нет |
| Миграции | migrate (one-shot) | образ API | - | - | нет |
| Сверка | s3_sweeper | образ API | - | - | нет |
**Перенесено из lab3:** src/main.py (пользователи, чаты, сессии, пагинация, идемпотентность), миграции 001–002, migrate.py, HAProxy-конфиг, restart/stop-one/rolling-restart, Dockerfile, проверка grade4 (теперь `student-tests/lab3_idempotency_checks.py`).
**Новое в lab4:** миграция 003 (`attachments`, `message_attachments`, снят CHECK непустого текста - текст может быть пустым при вложении); эндпоинты POST/GET вложений; потоковая приёмка с лимитами; boto3-клиент с таймаутами; readiness проверяет S3; `s3_init`, `s3_sweeper`; изолированная сеть.
**Границы транзакций.** Порядок: тело -> валидация -> S3 PUT -> INSERT метаданных/COMMIT -> 201. 201 возвращается только после обоих. Обрыв клиента до PUT ничего не оставляет в S3. Сбой между PUT и COMMIT: (1) компенсация - немедленный DELETE ключа; (2) если и она/процесс упали - `s3_sweeper` удаляет объект, которому нет строки в БД, но только старше `ORPHAN_GRACE_SECONDS` (3600 с ≫ секунд между PUT и COMMIT), с повторной проверкой БД перед удалением. Обратный разрыв (строка без объекта) этим порядком не создаётся; при обнаружении download пишет INTEGRITY в лог и отвечает 500 (полный двусторонний reconciler - трек 5, не заявлен).
**Readiness:** критичны PostgreSQL, Redis и S3 (HeadObject, таймаут 1 с): загрузка/скачивание входят в обязательный путь. Недоступность S3 -> /health/ready 503 -> HAProxy исключает реплики; запросы, дошедшие до API, получают JSON `503 unavailable`.
**Решения по контракту:** Content-Type должен совпадать с содержимым (PNG под видом JPEG и наоборот - 415); filename `.` и `..` отклоняются; несуществующий/чужой/из другого чата attachment_id - 404; повтор с тем же ключом сравнивается по каноническому телу (`attachment_ids` входят в хеш только если непусты - записи lab3 остаются валидны).
## Доказательства критериев
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
| --- | --- | --- | --- |
| Публичный smoke | `make test` | `PASS: lab4 public API scenario` | evidence/evidence.txt |
| Уровень 3: S3 в стенде, байты не в БД/локально; стенд поднят, `/health/ready` отвечает | `docker compose --env-file .env up -d --build --scale api=2`; `curl -i /health/ready`; `make check-s3` | Стенд собран и запущен; `HTTP/1.1 200 OK`, `{"status":"ok"}`, `x-instance-id: 464805954a42`; `make check-s3` - 11 PASS | evidence/evidence.txt |
| 201 после сохранения; побайтный download; чужой - 404 | `make test`; `make test-level4` | PASS smoke; в `test-level4` `statuses seen: [201, 400, 404, 413, 415]`, 5xx нет | evidence/evidence.txt |
| PNG/JPEG ≤10 MiB, ≤40 Мпикс, MIME по содержимому, filename | `make test-level4` | PASS; `statuses seen: [201, 400, 404, 413, 415]`, ни одного 5xx | evidence/evidence.txt |
| Upload -> стоп реплики -> download через другую | `make test-failover ACTION_SCRIPT=scripts/stop-one.sh` | `PASS: failover preserved confirmed data and existing session`; `{"observed_instances_before": 2, "successful_writes": 108, "failed_attempts": 1, "first_recovery_seconds": 2.727}` | evidence/evidence.txt |
| Restart стенда сохраняет данные и сессию | `make test-persistence ACTION_SCRIPT=scripts/restart.sh` | `PASS: persistence preserved confirmed data and existing session` | evidence/evidence.txt |
| Ур.4: тесты ошибок, нет 500 | `make test-level4` (включает `test_compensation.py`) | `PASS: level-4 attachment checks (no 5xx for user errors)`; в unit - 415 (битый файл), 503 (сбой БД после PUT), 503 (`type=BotoCoreError`), `Ran 3 tests … OK` | evidence/evidence.txt |
| Ур.4: компенсация/сверка с grace | `test_compensation.py`; `orphan_sweep_check.py` | unit 3/3 OK; orphan_sweep - grace/dry-run/apply/referenced/idempotent | evidence/evidence.txt |
| Ур.4: ≥10 загрузок, пик RAM | `make memory` | 10×10 050 123 Б; VmRSS 91->126 MiB и 94->116 MiB; `naive_estimate_MiB: 192`; wall 2.39 с; latency 1.05–1.87 с | evidence/evidence.txt |
| Ур.4: идемпотентный bootstrap, отдельные права, консоль закрыта, private bucket вне API | `make check-s3` (после `s3_init` из `up`) | 11 PASS: anonymous GET/HEAD/LIST/PUT -> 403; app creds - ListBuckets/CreateBucket/DeleteBucket/PutBucketPolicy/PutObject outside attachments denied; console 9001/9090 closed; `s3 publishers: []` | evidence/evidence.txt |
## Эксперименты и отказы
OS: Майкрософт Windows 11 Pro 10.0.22631 (build 22631)
CPU: Intel(R) Core(TM) i5-8350U CPU @ 1.70GHz, cores=4, threads=8
RAM: 15.9 GB
Docker Compose version v2.32.4-desktop.1
docker version
Client:
Version: 27.5.1
API version: 1.47
Go version: go1.22.11
Git commit: 9f9e405
Built: Wed Jan 22 13:41:44 2025
OS/Arch: windows/amd64
Context: desktop-linux
Server: Docker Desktop 4.38.0 (181591)
Engine:
Version: 27.5.1
API version: 1.47 (minimum version 1.24)
Go version: go1.22.11
Git commit: 4c9b3b0
Built: Wed Jan 22 13:41:17 2025
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.7.25
GitCommit: bcc810d6b9066471b0b6fa75f557a15a1cbf31bb
runc:
Version: 1.1.12
GitCommit: v1.1.12-0-g51d5e946
docker-init:
Version: 0.19.0
GitCommit: de40ad0
| Воздействие | Что ожидали | Что наблюдали | Время восстановления / потеря данных |
| --- | --- | --- | --- |
| Остановка одной реплики API | download через другую; подтверждённые данные целы | `make test-failover` PASS: `observed_instances_before: 2`, `successful_writes: 108`, `failed_attempts: 1`, `first_recovery_seconds: 2.727` | восстановление 2.727 с; 1 неудачный запрос - попал на остановленную реплику, ожидаемо; данные и сессия целы |
| Restart postgres/redis/s3/api | данные и токен сохранены | `make test-persistence` PASS: `persistence preserved confirmed data and existing session` | 0 потерь |
| PUT без COMMIT (сироты) | удаляется только после grace; referenced не трогается | `orphan_sweep_check.py` PASS: свежий orphan переживает grace 3600; dry-run печатает `WOULD-DELETE`; `--apply` удаляет; referenced attachment байт-в-байт цел; повтор - `orphans=0` (no-op) | 0 потерь |
| 10 параллельных загрузок 10 MiB | пик RAM не растёт линейно с размером файла | `make memory`: `payload_bytes: 10050123`, `concurrency: 10`; VmRSS 91->126 MiB (реплика 1), 94->116 MiB (реплика 2); `naive_estimate_MiB: 192`; `wall_seconds: 2.39`; `latency_s: 1.05–1.87` | отказов нет; запас до `mem_limit` (~512 MiB) сохраняется |
| Остановка S3 | upload/download -> 503 JSON, `/health/ready` -> 503, после старта - снова 201 | в `evidence/evidence.txt` не прогонялся (сценарий описан в «Ограничения и следующие шаги»); проверено только, что S3 недоступен снаружи (`s3 publishers: []`) | - |
## Ограничения и следующие шаги
Не реализованы: двусторонний reconciler и presigned upload (треки 5). Вложения, не прикреплённые к сообщению, не удаляются (учтённые, но «висящие»). S3-сервер - Silo (pgsty/silo, форк MinIO), закреплён на теге RELEASE.2026-10-15T00-00-00Z; для учебной изолированной сети допустимо, для production нужен актуальный патч-релиз. Один Docker-host - не отказоустойчивость; PostgreSQL, Redis, S3 и HAProxy - единичные. Временный файл буферизации (диск контейнера, анонимный, удаляется сразу) - не хранилище.
+192
View File
@@ -0,0 +1,192 @@
name: mtusi-lab4
services:
postgres:
image: postgres:17.6-alpine
environment:
POSTGRES_DB: ${POSTGRES_DB:-messenger}
POSTGRES_USER: ${POSTGRES_USER:-messenger}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?Set POSTGRES_PASSWORD in .env}
volumes:
- postgres_data:/var/lib/postgresql/data
networks: [backend]
healthcheck:
test: ["CMD-SHELL", "pg_isready -U $${POSTGRES_USER} -d $${POSTGRES_DB}"]
interval: 5s
timeout: 3s
retries: 20
cpus: 1.0
mem_limit: 768m
restart: unless-stopped
redis:
image: redis:7.4.2-alpine
command: ["redis-server", "--appendonly", "yes", "--appendfsync", "everysec"]
volumes:
- redis_data:/data
networks: [backend]
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 5s
timeout: 3s
retries: 20
cpus: 0.25
mem_limit: 256m
restart: unless-stopped
s3:
image: pgsty/silo:RELEASE.2026-10-15T00-00-00Z
command: ["server", "/data", "--address", ":9000"]
environment:
MINIO_ROOT_USER: ${MINIO_ROOT_USER:?Set MINIO_ROOT_USER in .env}
MINIO_ROOT_PASSWORD: ${MINIO_ROOT_PASSWORD:?Set MINIO_ROOT_PASSWORD in .env}
MINIO_BROWSER: "off" # admin web console disabled; no ports are published at all
volumes:
- s3_data:/data
networks: [backend]
healthcheck:
test: ["CMD", "mc", "ready", "local"]
interval: 5s
timeout: 5s
retries: 20
cpus: 1.0
mem_limit: 512m
restart: unless-stopped
s3_init:
image: pgsty/silo:RELEASE.2026-10-15T00-00-00Z
entrypoint: ["/bin/sh", "/infra/s3-init.sh"]
environment:
S3_ENDPOINT: http://s3:9000
S3_BUCKET: ${S3_BUCKET:-messenger-attachments}
MINIO_ROOT_USER: ${MINIO_ROOT_USER:?Set MINIO_ROOT_USER in .env}
MINIO_ROOT_PASSWORD: ${MINIO_ROOT_PASSWORD:?Set MINIO_ROOT_PASSWORD in .env}
S3_ACCESS_KEY_ID: ${S3_ACCESS_KEY_ID:?Set S3_ACCESS_KEY_ID in .env}
S3_SECRET_ACCESS_KEY: ${S3_SECRET_ACCESS_KEY:?Set S3_SECRET_ACCESS_KEY in .env}
SWEEPER_S3_ACCESS_KEY_ID: ${SWEEPER_S3_ACCESS_KEY_ID:?Set SWEEPER_S3_ACCESS_KEY_ID in .env}
SWEEPER_S3_SECRET_ACCESS_KEY: ${SWEEPER_S3_SECRET_ACCESS_KEY:?Set SWEEPER_S3_SECRET_ACCESS_KEY in .env}
volumes:
- ./infra/s3-init.sh:/infra/s3-init.sh:ro
networks: [backend]
depends_on:
s3:
condition: service_started
restart: "no"
migrate:
build: .
image: mtusi-lab4-api:1.0.0
command: ["python", "scripts/migrate.py"]
environment:
PGHOST: postgres
PGPORT: "5432"
PGDATABASE: ${POSTGRES_DB:-messenger}
PGUSER: ${POSTGRES_USER:-messenger}
PGPASSWORD: ${POSTGRES_PASSWORD:?Set POSTGRES_PASSWORD in .env}
networks: [backend]
depends_on:
postgres:
condition: service_healthy
restart: "no"
cpus: 0.5
mem_limit: 384m
api:
build: .
image: mtusi-lab4-api:1.0.0
environment:
APP_HOST: 0.0.0.0
APP_PORT: "8080"
SESSION_TTL_SECONDS: ${SESSION_TTL_SECONDS:-3600}
DB_POOL_MIN_SIZE: ${DB_POOL_MIN_SIZE:-1}
DB_POOL_MAX_SIZE: ${DB_POOL_MAX_SIZE:-10}
DB_POOL_TIMEOUT_SECONDS: ${DB_POOL_TIMEOUT_SECONDS:-5}
PGHOST: postgres
PGPORT: "5432"
PGDATABASE: ${POSTGRES_DB:-messenger}
PGUSER: ${POSTGRES_USER:-messenger}
PGPASSWORD: ${POSTGRES_PASSWORD:?Set POSTGRES_PASSWORD in .env}
REDIS_URL: redis://redis:6379/0
S3_ENDPOINT: http://s3:9000
S3_REGION: ${S3_REGION:-us-east-1}
S3_BUCKET: ${S3_BUCKET:-messenger-attachments}
S3_ACCESS_KEY_ID: ${S3_ACCESS_KEY_ID:?Set S3_ACCESS_KEY_ID in .env}
S3_SECRET_ACCESS_KEY: ${S3_SECRET_ACCESS_KEY:?Set S3_SECRET_ACCESS_KEY in .env}
MAX_UPLOAD_BYTES: ${MAX_UPLOAD_BYTES:-10485760}
MAX_IMAGE_PIXELS: ${MAX_IMAGE_PIXELS:-40000000}
IMAGE_DECODE_CONCURRENCY: ${IMAGE_DECODE_CONCURRENCY:-1}
expose: ["8080"]
networks: [backend]
depends_on:
migrate:
condition: service_completed_successfully
redis:
condition: service_healthy
s3_init:
condition: service_completed_successfully
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/health/ready', timeout=2)"]
interval: 3s
timeout: 3s
retries: 3
start_period: 5s
stop_grace_period: 15s
cpus: 1.0
mem_limit: 512m
restart: unless-stopped
s3_sweeper:
build: .
image: mtusi-lab4-api:1.0.0
command: ["python", "scripts/s3_sweep.py", "--loop", "--apply"]
environment:
PGHOST: postgres
PGPORT: "5432"
PGDATABASE: ${POSTGRES_DB:-messenger}
PGUSER: ${POSTGRES_USER:-messenger}
PGPASSWORD: ${POSTGRES_PASSWORD:?Set POSTGRES_PASSWORD in .env}
S3_ENDPOINT: http://s3:9000
S3_REGION: ${S3_REGION:-us-east-1}
S3_BUCKET: ${S3_BUCKET:-messenger-attachments}
S3_ACCESS_KEY_ID: ${SWEEPER_S3_ACCESS_KEY_ID:?Set SWEEPER_S3_ACCESS_KEY_ID in .env}
S3_SECRET_ACCESS_KEY: ${SWEEPER_S3_SECRET_ACCESS_KEY:?Set SWEEPER_S3_SECRET_ACCESS_KEY in .env}
SWEEP_GRACE_SECONDS: ${SWEEP_GRACE_SECONDS:-3600}
SWEEP_INTERVAL_SECONDS: ${SWEEP_INTERVAL_SECONDS:-600}
networks: [backend]
depends_on:
migrate:
condition: service_completed_successfully
s3_init:
condition: service_completed_successfully
cpus: 0.25
mem_limit: 128m
restart: unless-stopped
load_balancer:
image: haproxy:3.2.5-alpine
ports:
- "127.0.0.1:${APP_PORT:-8080}:8080"
volumes:
- ./infra/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro
networks: [frontend, backend]
depends_on:
api:
condition: service_healthy
healthcheck:
test: ["CMD", "haproxy", "-c", "-f", "/usr/local/etc/haproxy/haproxy.cfg"]
interval: 10s
timeout: 3s
retries: 3
cpus: 0.5
mem_limit: 128m
restart: unless-stopped
networks:
frontend:
driver: bridge
backend:
driver: bridge
internal: true # no route to/from the host or internet: S3, PostgreSQL, Redis and API are reachable only inside
volumes:
postgres_data:
redis_data:
s3_data:
+153
View File
@@ -0,0 +1,153 @@
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab4/mtusi2026lab4-fork$
make check
python3 scripts/check_contract.py
PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab4/mtusi2026lab4-fork$
docker compose --env-file .env config --quiet
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab4/mtusi2026lab4-fork$
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab4/mtusi2026lab4-fork$
make validate
python3 scripts/validate_openapi.py
PASS: full OpenAPI 3.1 and JSON Schema validation; application NOT tested.
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab4/mtusi2026lab4-fork$
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab4/mtusi2026lab4-fork$
docker compose --env-file .env up --build --scale api=2
[+] Running 13/13
✔ s3_init Pulled 29.1s
✔ s3 Pulled 29.1s
[+] Building 54.6s (29/34) docker:default
=> [migrate internal] load build definition from Dockerfile 0.2s
=> => transferring dockerfile: 567B 0.1s
=> [api internal] load metadata for docker.io/library/python:3.12.11-slim-bookworm 4.7s
=> [migrate internal] load .dockerignore 0.1s
=> => transferring context: 216B
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab4/mtusi2026lab4-fork$ curl -i http://localhost:8080/health/ready
HTTP/1.1 200 OK
date: Sat, 31 Oct 2026 17:06:30 GMT
server: uvicorn
content-length: 15
content-type: application/json
x-request-id: 241b13832e55470c83632d148975e85a
x-instance-id: 464805954a42
{"status":"ok"}thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab4/mtusi2026lab4-fork$
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab4/mtusi2026lab4-fork$ make test
python3 tests/smoke.py --base-url "http://localhost:8080"
PASS: lab4 public API scenario; infrastructure and grading evidence remain separate.
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab4/mtusi2026lab4-fork$
(проверки 3ей лабы)
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab4/mtusi2026lab4-fork$ make test-lab3
python3 student-tests/lab3_idempotency_checks.py --base-url "http://localhost:8080"
{"distinct_instances": ["464805954a42", "e269ce4a6969"], "parallel_requests": 12, "distinct_message_ids": 1, "changed_body_status": 409, "missing_key_status": 400, "invalid_key_status": 400}
PASS: concurrent cross-replica idempotency, key validation, and instance visibility
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab4/mtusi2026lab4-fork$
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab4/mtusi2026lab4-fork$ make test-level4
python3 student-tests/level4_checks.py --base-url "http://localhost:8080"
statuses seen: [201, 400, 404, 413, 415]
PASS: level-4 attachment checks (no 5xx for user errors)
python3 -m unittest discover -s student-tests -p "test_*.py"
INFO:httpx:HTTP Request: POST http://testserver/api/v1/chats/68da9b78-1806-4f38-bffa-f73b9499a1ef/attachments?filename=a.png "HTTP/1.1 415 Unsupported Media Type"
.WARNING:messenger:Database operation failed request_id=7dce87b3b62643cc890fe3e40a8c15dd
INFO:httpx:HTTP Request: POST http://testserver/api/v1/chats/9c8e1f04-5772-4854-8297-08482a1bef15/attachments?filename=a.png "HTTP/1.1 503 Service Unavailable"
.WARNING:messenger:Object storage failure request_id=eb68ec5f6e774fab8463cada45862daa type=BotoCoreError
INFO:httpx:HTTP Request: POST http://testserver/api/v1/chats/97cd6e8a-3e68-4c94-add5-5e0aac4e0e4f/attachments?filename=a.png "HTTP/1.1 503 Service Unavailable"
.
----------------------------------------------------------------------
Ran 3 tests in 0.071s
OK
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab4/mtusi2026lab4-fork$
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab4/mtusi2026lab4-fork$ make check-s3
sh scripts/check_s3_exposure.sh
s3 publishers: []
PASS: S3 and console unreachable from host; run 'make check-s3' for in-network checks
docker compose run --rm --no-deps --entrypoint python api scripts/check_s3_private.py
PASS anonymous GET object -> 403
PASS anonymous HEAD object -> 403
PASS anonymous list bucket -> 403
PASS anonymous PUT -> 403
PASS app creds: ListBuckets denied
PASS app creds: CreateBucket denied
PASS app creds: DeleteBucket denied
PASS app creds: PutBucketPolicy denied
PASS app creds: PutObject outside attachments/ denied
PASS admin console port 9001 closed
PASS admin console port 9090 closed
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab4/mtusi2026lab4-fork$
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab4/mtusi2026lab4-fork$ python3 student-tests/orphan_sweep_check.py
PASS: grace period, dry-run, orphan removal, referenced object preserved, idempotent re-run
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab4/mtusi2026lab4-fork$
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab4/mtusi2026lab4-fork$ make memory
python3 student-tests/memory_experiment.py --base-url "http://localhost:8080" --concurrency 10
{
"profile": "big-bytes",
"concurrency": 10,
"payload_bytes": 10050123,
"wall_seconds": 2.39,
"latency_s": {
"min": 1.05,
"max": 1.87
},
"before": {
"c6a05147ec62": {
"VmRSS_MiB": 91,
"VmHWM_MiB": 91
},
"f76cf025a087": {
"VmRSS_MiB": 94,
"VmHWM_MiB": 94
}
},
"after_peak": {
"c6a05147ec62": {
"VmRSS_MiB": 126,
"VmHWM_MiB": 126
},
"f76cf025a087": {
"VmRSS_MiB": 116,
"VmHWM_MiB": 116
}
},
"naive_estimate_MiB": 192
}
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab4/mtusi2026lab4-fork$
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab4/mtusi2026lab4-fork$ make test-persistence ACTION_SCRIPT=scripts/restart.sh
python3 tests/resilience.py --mode persistence --action-script "scripts/restart.sh" --base-url "http://localhost:8080"
Executing explicit persistence hook; hook output suppressed to avoid leaking secrets.
PASS: persistence preserved confirmed data and existing session. Hook scope needs operator evidence.
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab4/mtusi2026lab4-fork$ make test-failover ACTION_SCRIPT=scripts/stop-one.sh
python3 tests/resilience.py --mode failover --action-script "scripts/stop-one.sh" --base-url "http://localhost:8080"
Executing explicit failover hook; hook output suppressed to avoid leaking secrets.
{"observed_instances_before": 2, "successful_writes": 108, "failed_attempts": 1, "first_recovery_seconds": 2.727}
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
(восстановление реплики после стоп ван сш)
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab4/mtusi2026lab4-fork$ docker compose up -d --scale api=2 api
[+] Running 7/7
✔ Container mtusi-lab4-s3-1 Running 0.0s
✔ Container mtusi-lab4-postgres-1 Healthy 0.6s
✔ Container mtusi-lab4-migrate-1 Exited 3.5s
✔ Container mtusi-lab4-redis-1 Healthy 1.6s
✔ Container mtusi-lab4-s3_init-1 Exited 2.5s
✔ Container mtusi-lab4-api-1 Running 0.0s
✔ Container mtusi-lab4-api-2 Started 3.7s
thereal_alyoname@DESKTOP-8Q26HPT:/mnt/c/Users/User/Desktop/MTUCI/4_course/highload_apps/lab4/mtusi2026lab4-fork$
+32
View File
@@ -0,0 +1,32 @@
global
log stdout format raw local0
maxconn 1024
defaults
log global
mode http
option httplog
option http-keep-alive
timeout connect 2s
timeout client 30s
timeout server 30s
timeout check 2s
resolvers docker
nameserver docker-dns 127.0.0.11:53
resolve_retries 3
timeout resolve 1s
timeout retry 1s
hold valid 2s
hold obsolete 2s
frontend public_http
bind :8080
default_backend api_replicas
backend api_replicas
balance roundrobin
option httpchk GET /health/ready
http-check expect status 200
default-server inter 1s fall 2 rise 1
server-template api 10 api:8080 check resolvers docker init-addr libc,none
+56
View File
@@ -0,0 +1,56 @@
#!/bin/sh
# Idempotent bootstrap of the private bucket, the restricted application user
# and the separate sweeper user (safe to re-run).
set -eu
: "${S3_ENDPOINT:?}" "${S3_BUCKET:?}" "${MINIO_ROOT_USER:?}" "${MINIO_ROOT_PASSWORD:?}"
: "${S3_ACCESS_KEY_ID:?}" "${S3_SECRET_ACCESS_KEY:?}"
: "${SWEEPER_S3_ACCESS_KEY_ID:?}" "${SWEEPER_S3_SECRET_ACCESS_KEY:?}"
[ "$S3_ACCESS_KEY_ID" != "$MINIO_ROOT_USER" ] || { echo "app key must differ from root user" >&2; exit 1; }
[ "$SWEEPER_S3_ACCESS_KEY_ID" != "$MINIO_ROOT_USER" ] || { echo "sweeper key must differ from root user" >&2; exit 1; }
[ "$S3_ACCESS_KEY_ID" != "$SWEEPER_S3_ACCESS_KEY_ID" ] || { echo "app and sweeper keys must differ" >&2; exit 1; }
i=0
until mc alias set local "$S3_ENDPOINT" "$MINIO_ROOT_USER" "$MINIO_ROOT_PASSWORD" >/dev/null 2>&1; do
i=$((i+1)); [ "$i" -lt 60 ] || { echo "S3 not reachable" >&2; exit 1; }; sleep 1
done
mc mb --ignore-existing "local/$S3_BUCKET"
mc anonymous set none "local/$S3_BUCKET"
printf ok | mc pipe "local/$S3_BUCKET/attachments/.ready"
# --- app policy: exact keys only, NO ListBucket ---
cat > /tmp/policy-app.json <<POLICY
{"Version":"2012-10-17","Statement":[
{"Effect":"Allow","Action":["s3:PutObject","s3:GetObject","s3:DeleteObject"],"Resource":["arn:aws:s3:::$S3_BUCKET/attachments/*"]}]}
POLICY
# --- sweeper policy: ListBucket for enumeration + Get/Delete, NO PutObject ---
cat > /tmp/policy-sweeper.json <<POLICY
{"Version":"2012-10-17","Statement":[
{"Effect":"Allow","Action":["s3:ListBucket","s3:GetBucketLocation"],"Resource":["arn:aws:s3:::$S3_BUCKET"]},
{"Effect":"Allow","Action":["s3:GetObject","s3:DeleteObject"],"Resource":["arn:aws:s3:::$S3_BUCKET/attachments/*"]}]}
POLICY
# MinIO's `policy create` does not overwrite an existing policy, so remove first.
mc admin policy remove local messenger-app --force >/dev/null 2>&1 || true
mc admin policy remove local messenger-sweeper --force >/dev/null 2>&1 || true
mc admin policy create local messenger-app /tmp/policy-app.json
mc admin policy create local messenger-sweeper /tmp/policy-sweeper.json
# `user add` is idempotent (re-running updates the secret).
mc admin user add local "$S3_ACCESS_KEY_ID" "$S3_SECRET_ACCESS_KEY"
mc admin user add local "$SWEEPER_S3_ACCESS_KEY_ID" "$SWEEPER_S3_SECRET_ACCESS_KEY"
out=$(mc admin policy attach local messenger-app --user "$S3_ACCESS_KEY_ID" 2>&1) || {
echo "$out" | grep -qi "already" || { echo "$out" >&2; exit 1; }
}
out=$(mc admin policy attach local messenger-sweeper --user "$SWEEPER_S3_ACCESS_KEY_ID" 2>&1) || {
echo "$out" | grep -qi "already" || { echo "$out" >&2; exit 1; }
}
echo "PASS: bucket $S3_BUCKET private"
echo " app=$S3_ACCESS_KEY_ID (Put/Get/Delete on attachments/*, no ListBucket)"
echo " sweeper=$SWEEPER_S3_ACCESS_KEY_ID (ListBucket + Get/Delete, no PutObject)"
+32
View File
@@ -0,0 +1,32 @@
CREATE TABLE IF NOT EXISTS users (
id UUID PRIMARY KEY,
username VARCHAR(32) NOT NULL UNIQUE,
display_name VARCHAR(100) NOT NULL,
password_hash TEXT NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
CONSTRAINT users_username_format CHECK (username ~ '^[a-z0-9_]{3,32}$'),
CONSTRAINT users_display_name_nonblank CHECK (length(btrim(display_name)) > 0)
);
CREATE TABLE IF NOT EXISTS chats (
id UUID PRIMARY KEY,
title VARCHAR(100) NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
CONSTRAINT chats_title_nonblank CHECK (length(btrim(title)) > 0)
);
CREATE TABLE IF NOT EXISTS chat_members (
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
position INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (chat_id, user_id)
);
ALTER TABLE chat_members ADD COLUMN IF NOT EXISTS position INTEGER NOT NULL DEFAULT 0;
CREATE INDEX IF NOT EXISTS chat_members_user_chat_idx ON chat_members(user_id, chat_id);
CREATE TABLE IF NOT EXISTS messages (
id UUID PRIMARY KEY,
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
sender_id UUID NOT NULL REFERENCES users(id),
text VARCHAR(4000) NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
CONSTRAINT messages_text_nonblank CHECK (length(btrim(text)) > 0)
);
CREATE INDEX IF NOT EXISTS messages_chat_created_id_idx ON messages(chat_id, created_at, id);
+11
View File
@@ -0,0 +1,11 @@
CREATE TABLE IF NOT EXISTS idempotency_records (
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
idempotency_key VARCHAR(128) NOT NULL,
request_hash CHAR(64) NOT NULL,
response_body JSONB NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
PRIMARY KEY (user_id, chat_id, idempotency_key),
CONSTRAINT idempotency_key_format CHECK (idempotency_key ~ '^[A-Za-z0-9._:-]{1,128}$')
);
CREATE INDEX IF NOT EXISTS idempotency_records_created_at_idx ON idempotency_records(created_at);
+24
View File
@@ -0,0 +1,24 @@
CREATE TABLE IF NOT EXISTS attachments (
id UUID PRIMARY KEY,
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
owner_id UUID NOT NULL REFERENCES users(id),
filename VARCHAR(255) NOT NULL,
content_type TEXT NOT NULL,
size_bytes BIGINT NOT NULL,
sha256 CHAR(64) NOT NULL,
s3_key TEXT NOT NULL UNIQUE,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
CONSTRAINT attachments_filename_nonblank CHECK (length(btrim(filename)) > 0),
CONSTRAINT attachments_content_type CHECK (content_type IN ('image/png', 'image/jpeg')),
CONSTRAINT attachments_size CHECK (size_bytes BETWEEN 1 AND 10485760),
CONSTRAINT attachments_sha256 CHECK (sha256 ~ '^[a-f0-9]{64}$')
);
CREATE INDEX IF NOT EXISTS attachments_chat_idx ON attachments(chat_id);
CREATE TABLE IF NOT EXISTS message_attachments (
message_id UUID NOT NULL REFERENCES messages(id) ON DELETE CASCADE,
attachment_id UUID NOT NULL REFERENCES attachments(id),
position INTEGER NOT NULL,
PRIMARY KEY (message_id, attachment_id)
);
CREATE INDEX IF NOT EXISTS message_attachments_attachment_idx ON message_attachments(attachment_id);
ALTER TABLE messages DROP CONSTRAINT IF EXISTS messages_text_nonblank;
+7
View File
@@ -0,0 +1,7 @@
fastapi==0.116.1
uvicorn[standard]==0.35.0
psycopg[pool,binary]==3.2.9
argon2-cffi==25.1.0
redis==6.4.0
boto3==1.40.20
Pillow==11.3.0
+10
View File
@@ -0,0 +1,10 @@
#!/usr/bin/env bash
# Host-side: S3 (9000/9001) must not be published anywhere; the network must be internal.
set -eu
pub=$(docker compose ps s3 --format '{{.Publishers}}')
echo "s3 publishers: ${pub:-none}"
case "$pub" in ""|"[]") ;; *) echo "FAIL: s3 publishes ports" >&2; exit 1;; esac
for port in 9000 9001; do
if (exec 3<>/dev/tcp/127.0.0.1/$port) 2>/dev/null; then echo "FAIL: host can reach :$port" >&2; exit 1; fi
done
echo "PASS: S3 and console unreachable from host; run 'make check-s3' for in-network checks"
+36
View File
@@ -0,0 +1,36 @@
#!/usr/bin/env python3
"""Runs INSIDE the backend network but WITHOUT the API: anonymous access is denied, app credentials are least-privilege."""
import os, secrets, sys, urllib.request, urllib.error, socket
import boto3
from botocore.config import Config
from botocore.exceptions import ClientError
EP, B = os.environ["S3_ENDPOINT"], os.environ["S3_BUCKET"]
s3 = boto3.client("s3", endpoint_url=EP, region_name=os.getenv("S3_REGION", "us-east-1"), aws_access_key_id=os.environ["S3_ACCESS_KEY_ID"],
aws_secret_access_key=os.environ["S3_SECRET_ACCESS_KEY"], config=Config(signature_version="s3v4", s3={"addressing_style": "path"}, retries={"max_attempts": 1}))
bad = []
def expect(name, ok): print(("PASS " if ok else "FAIL ") + name); (ok or bad.append(name))
def anon(method, path, data=None):
try: return urllib.request.urlopen(urllib.request.Request(EP + path, method=method, data=data), timeout=5).status
except urllib.error.HTTPError as e: return e.code
def denied(fn):
try: fn(); return False
except ClientError as e: return e.response["Error"]["Code"] in ("AccessDenied", "403")
key = "attachments/probe-" + secrets.token_hex(8)
s3.put_object(Bucket=B, Key=key, Body=b"probe")
try:
expect("anonymous GET object -> 403", anon("GET", f"/{B}/{key}") == 403)
expect("anonymous HEAD object -> 403", anon("HEAD", f"/{B}/{key}") == 403)
expect("anonymous list bucket -> 403", anon("GET", f"/{B}/") == 403)
expect("anonymous PUT -> 403", anon("PUT", f"/{B}/attachments/anon-{secrets.token_hex(4)}", b"x") == 403)
finally:
s3.delete_object(Bucket=B, Key=key)
expect("app creds: ListBuckets denied", denied(lambda: s3.list_buckets()))
expect("app creds: CreateBucket denied", denied(lambda: s3.create_bucket(Bucket="x-" + secrets.token_hex(4))))
expect("app creds: DeleteBucket denied", denied(lambda: s3.delete_bucket(Bucket=B)))
expect("app creds: PutBucketPolicy denied", denied(lambda: s3.put_bucket_policy(Bucket=B, Policy="{}")))
expect("app creds: PutObject outside attachments/ denied", denied(lambda: s3.put_object(Bucket=B, Key="other/x", Body=b"x")))
host = EP.split("//")[1].split(":")[0]
for port in (9001, 9090):
with socket.socket() as sk:
sk.settimeout(2); expect(f"admin console port {port} closed", sk.connect_ex((host, port)) != 0)
sys.exit(1 if bad else 0)
+37
View File
@@ -0,0 +1,37 @@
#!/usr/bin/env python3
"""Apply SQL migrations once, before any API replica is started."""
from pathlib import Path
import os
import time
import psycopg
ROOT = Path(__file__).resolve().parents[1]
CONNINFO = os.getenv("DATABASE_URL", "")
def main():
last_error = None
for _ in range(30):
try:
with psycopg.connect(CONNINFO, connect_timeout=3) as conn:
conn.execute("CREATE TABLE IF NOT EXISTS schema_migrations (version TEXT PRIMARY KEY, applied_at TIMESTAMPTZ NOT NULL DEFAULT now())")
conn.commit()
for path in sorted((ROOT / "migrations").glob("*.sql")):
version = path.name
if conn.execute("SELECT 1 FROM schema_migrations WHERE version=%s", (version,)).fetchone():
continue
sql = path.read_text(encoding="utf-8")
with conn.transaction():
for statement in sql.split(";"):
if statement.strip():
conn.execute(statement)
conn.execute("INSERT INTO schema_migrations(version) VALUES (%s)", (version,))
print(f"APPLIED {version}", flush=True)
print("PASS: migrations complete", flush=True)
return
except Exception as exc:
last_error = exc
time.sleep(1)
raise SystemExit(f"Migration failed after retries: {type(last_error).__name__}")
if __name__ == "__main__":
main()
+5
View File
@@ -0,0 +1,5 @@
#!/bin/sh
set -eu
# Restart ALL state stores (PostgreSQL, Redis, S3), then every API replica; named volumes are preserved.
docker compose restart postgres redis s3
docker compose restart api
+24
View File
@@ -0,0 +1,24 @@
#!/bin/sh
set -eu
# Sequential replacement-equivalent exercise for Compose (which has no native rolling-update primitive).
# Run a load generator in another terminal while this script executes.
replicas="$(docker compose ps -q --all api)"
[ -n "$replicas" ] || { echo "No API containers found; start the stack first" >&2; exit 1; }
for id in $replicas; do
# Remove one replica; the LB excludes it after health-check failure.
docker stop --time 10 "$id" >/dev/null || true
docker rm -f "$id" >/dev/null
docker compose up -d --no-deps --scale api=2 api
# Wait for at least two healthy API containers before moving to the next one.
tries=0
while [ "$tries" -lt 30 ]; do
healthy=0
for candidate in $(docker compose ps -q api); do
status=$(docker inspect --format '{{.State.Health.Status}}' "$candidate" 2>/dev/null || echo starting)
[ "$status" = healthy ] && healthy=$((healthy+1))
done
[ "$healthy" -ge 2 ] && break
tries=$((tries+1)); sleep 1
done
[ "$healthy" -ge 2 ] || { echo "Fewer than two healthy replicas after replacement" >&2; exit 1; }
done
+10
View File
@@ -0,0 +1,10 @@
#!/usr/bin/env python3
"""TEST-ONLY helper: puts an orphan object (as if the API died between S3 PUT and COMMIT)."""
import os, secrets, boto3
from botocore.config import Config
s3 = boto3.client("s3", endpoint_url=os.environ["S3_ENDPOINT"], region_name=os.getenv("S3_REGION", "us-east-1"),
aws_access_key_id=os.environ["S3_ACCESS_KEY_ID"], aws_secret_access_key=os.environ["S3_SECRET_ACCESS_KEY"],
config=Config(signature_version="s3v4", s3={"addressing_style": "path"}))
key = "attachments/" + secrets.token_hex(16)
s3.put_object(Bucket=os.environ["S3_BUCKET"], Key=key, Body=b"orphan-test", ContentType="image/png")
print(key)
+57
View File
@@ -0,0 +1,57 @@
#!/usr/bin/env python3
"""Orphan sweeper: deletes S3 objects under attachments/ that have no row in `attachments`.
Safety: only objects older than --grace-seconds are candidates (default 3600, far above the few seconds between
PUT and COMMIT), and the DB is re-checked immediately before each delete. Default mode is --dry-run.
"""
import argparse, os, sys, time
from datetime import datetime, timezone
import boto3, psycopg
from botocore.config import Config
def run_once(s3, conn, bucket, grace, apply):
now = datetime.now(timezone.utc)
seen = old = orphans = deleted = 0
for page in s3.get_paginator("list_objects_v2").paginate(Bucket=bucket, Prefix="attachments/"):
for obj in page.get("Contents", []):
seen += 1
if obj["Key"] == "attachments/.ready":
continue
if (now - obj["LastModified"]).total_seconds() < grace:
continue # fresh: may still be between PUT and COMMIT
old += 1
if conn.execute("SELECT 1 FROM attachments WHERE s3_key=%s", (obj["Key"],)).fetchone():
continue
orphans += 1
print(("DELETE " if apply else "WOULD-DELETE ") + obj["Key"], flush=True)
if apply:
# re-check right before delete (closes the race with a concurrent COMMIT)
if not conn.execute("SELECT 1 FROM attachments WHERE s3_key=%s", (obj["Key"],)).fetchone():
s3.delete_object(Bucket=bucket, Key=obj["Key"]); deleted += 1
print(f"sweep: seen={seen} older_than_grace={old} orphans={orphans} deleted={deleted} apply={apply}", flush=True)
return orphans, deleted
def main():
p = argparse.ArgumentParser()
p.add_argument("--apply", action="store_true", help="really delete (default: dry-run)")
p.add_argument("--grace-seconds", type=int, default=int(os.getenv("SWEEP_GRACE_SECONDS", "3600")))
p.add_argument("--loop", action="store_true"); p.add_argument("--interval", type=int, default=int(os.getenv("SWEEP_INTERVAL_SECONDS", "600")))
a = p.parse_args()
if a.grace_seconds < 0: sys.exit("grace must be >= 0")
s3 = boto3.client("s3", endpoint_url=os.environ["S3_ENDPOINT"], region_name=os.getenv("S3_REGION", "us-east-1"),
aws_access_key_id=os.environ["S3_ACCESS_KEY_ID"], aws_secret_access_key=os.environ["S3_SECRET_ACCESS_KEY"],
config=Config(signature_version="s3v4", s3={"addressing_style": "path"}, connect_timeout=3, read_timeout=15, retries={"max_attempts": 2}))
bucket = os.environ["S3_BUCKET"]
while True:
try:
with psycopg.connect(connect_timeout=5, autocommit=True) as conn:
run_once(s3, conn, bucket, a.grace_seconds, a.apply)
except Exception as exc: # keep looping through outages
print("sweep failed:", type(exc).__name__, flush=True)
if not a.loop: raise
if not a.loop: return
time.sleep(a.interval)
if __name__ == "__main__":
main()
+17
View File
@@ -0,0 +1,17 @@
#!/bin/sh
set -eu
: "${TARGET_INSTANCE_ID:?TARGET_INSTANCE_ID must be supplied by tests/resilience.py}"
found=""
for id in $(docker compose ps -q api); do
host=$(docker inspect --format '{{.Config.Hostname}}' "$id")
if [ "$host" = "$TARGET_INSTANCE_ID" ]; then
found="$id"
break
fi
done
if [ -z "$found" ]; then
echo "No running API container matches TARGET_INSTANCE_ID" >&2
exit 1
fi
# Stop exactly one API replica and intentionally leave it stopped for the failover check.
docker stop --time 10 "$found" >/dev/null
+1
View File
@@ -0,0 +1 @@
"""Lab 3 FastAPI application package."""
+739
View File
@@ -0,0 +1,739 @@
"""Lab 4 REST API: FastAPI + PostgreSQL + Redis sessions + idempotency + S3 image attachments."""
from __future__ import annotations
import asyncio
import anyio
import base64
import hashlib
import json
import logging
import os
import re
import secrets
import socket
import tempfile
import unicodedata
import uuid
from datetime import datetime, timedelta, timezone
from pathlib import Path
from typing import Any
from urllib.parse import quote
import psycopg
from psycopg_pool import ConnectionPool, PoolTimeout
import anyio
import boto3
import redis
from botocore.config import Config as BotoConfig
from botocore.exceptions import BotoCoreError, ClientError
from PIL import Image
from redis.exceptions import RedisError
from argon2 import PasswordHasher
from argon2.exceptions import InvalidHashError, VerificationError, VerifyMismatchError
from fastapi import FastAPI, Request
from fastapi.exceptions import RequestValidationError
from fastapi.responses import JSONResponse, Response, StreamingResponse
from starlette.concurrency import iterate_in_threadpool, run_in_threadpool
from starlette.requests import ClientDisconnect
from pydantic import BaseModel, ConfigDict, Field, ValidationError
from starlette.exceptions import HTTPException as StarletteHTTPException
logging.basicConfig(level=os.getenv("LOG_LEVEL", "INFO"))
log = logging.getLogger("messenger")
DATABASE_URL = os.getenv("DATABASE_URL", "")
INSTANCE_ID = os.getenv("INSTANCE_ID") or socket.gethostname()
IDEMPOTENCY_KEY_RE = re.compile(r"^[A-Za-z0-9._:-]{1,128}$")
REDIS_URL = os.getenv("REDIS_URL", "redis://localhost:6379/0")
SESSION_TTL_SECONDS = int(os.getenv("SESSION_TTL_SECONDS", "3600"))
if SESSION_TTL_SECONDS < 1:
raise RuntimeError("SESSION_TTL_SECONDS must be positive")
# Redis uses its connection pool; session keys contain only a hash of the opaque token.
session_store = redis.Redis.from_url(REDIS_URL, decode_responses=True, socket_connect_timeout=3, socket_timeout=3, health_check_interval=30)
DB_POOL_MIN_SIZE = int(os.getenv("DB_POOL_MIN_SIZE", "1"))
DB_POOL_MAX_SIZE = int(os.getenv("DB_POOL_MAX_SIZE", "10"))
DB_POOL_TIMEOUT_SECONDS = float(os.getenv("DB_POOL_TIMEOUT_SECONDS", "5"))
if DB_POOL_MIN_SIZE < 1 or DB_POOL_MAX_SIZE < DB_POOL_MIN_SIZE:
raise RuntimeError("Invalid PostgreSQL pool size configuration")
# Every pooled connection gets bounded connect/query/idle-transaction timeouts.
db_pool = ConnectionPool(
conninfo=DATABASE_URL,
min_size=DB_POOL_MIN_SIZE,
max_size=DB_POOL_MAX_SIZE,
timeout=DB_POOL_TIMEOUT_SECONDS,
kwargs={
"connect_timeout": 5,
"options": "-c statement_timeout=5000 -c idle_in_transaction_session_timeout=10000",
},
open=False,
name="messenger-postgres",
)
# ---- Lab 4: S3 / attachment limits (names follow .env.example of the lab) ----
S3_ENDPOINT = os.getenv("S3_ENDPOINT", "http://s3:9000")
S3_REGION = os.getenv("S3_REGION", "us-east-1")
S3_BUCKET = os.getenv("S3_BUCKET", "messenger-attachments")
S3_ACCESS_KEY_ID = os.getenv("S3_ACCESS_KEY_ID", "")
S3_SECRET_ACCESS_KEY = os.getenv("S3_SECRET_ACCESS_KEY", "")
S3_KEY_PREFIX = "attachments/"
MAX_UPLOAD_BYTES = int(os.getenv("MAX_UPLOAD_BYTES", "10485760"))
MAX_IMAGE_PIXELS = int(os.getenv("MAX_IMAGE_PIXELS", "40000000"))
UPLOAD_IDLE_TIMEOUT_SECONDS = float(os.getenv("UPLOAD_IDLE_TIMEOUT_SECONDS", "10"))
UPLOAD_TOTAL_TIMEOUT_SECONDS = float(os.getenv("UPLOAD_TOTAL_TIMEOUT_SECONDS", "60"))
IMAGE_DECODE_CONCURRENCY = int(os.getenv("IMAGE_DECODE_CONCURRENCY", "1"))
SPOOL_MEMORY_BYTES = 1024 * 1024 # larger bodies roll over to an anonymous temp file, never to RAM
CHUNK_BYTES = 64 * 1024
# Pillow's own bomb guard is a second line of defence behind the explicit check below.
Image.MAX_IMAGE_PIXELS = MAX_IMAGE_PIXELS
# Credentials are the restricted application user created by s3_init, never the MinIO root user.
_s3_common = dict(endpoint_url=S3_ENDPOINT, region_name=S3_REGION, aws_access_key_id=S3_ACCESS_KEY_ID, aws_secret_access_key=S3_SECRET_ACCESS_KEY)
s3 = boto3.client("s3", **_s3_common, config=BotoConfig(
signature_version="s3v4", s3={"addressing_style": "path"}, connect_timeout=2, read_timeout=20,
retries={"max_attempts": 2, "mode": "standard"}, max_pool_connections=32,
request_checksum_calculation="when_required", response_checksum_validation="when_required"))
# Readiness probe client: one attempt and tight timeouts so HAProxy's 2 s check never hangs on it.
s3_probe = boto3.client("s3", **_s3_common, config=BotoConfig(
signature_version="s3v4", s3={"addressing_style": "path"}, connect_timeout=1, read_timeout=1,
retries={"max_attempts": 1}, request_checksum_calculation="when_required", response_checksum_validation="when_required"))
decode_limiter = anyio.CapacityLimiter(IMAGE_DECODE_CONCURRENCY)
IMAGE_TYPES = {"PNG": "image/png", "JPEG": "image/jpeg"}
USERNAME_RE = re.compile(r"^[a-z0-9_]{3,32}$")
PASSWORDS = PasswordHasher()
class ApiError(Exception):
def __init__(self, status: int, code: str, message: str, headers: dict[str, str] | None = None):
self.status, self.code, self.message, self.headers = status, code, message, headers or {}
def fail(status: int, code: str, message: str, headers: dict[str, str] | None = None):
raise ApiError(status, code, message, headers)
class StrictModel(BaseModel):
model_config = ConfigDict(extra="forbid", strict=True)
class RegisterBody(StrictModel):
username: str
display_name: str
password: str
class ChatBody(StrictModel):
title: str
member_ids: list[str] = Field(min_length=1, max_length=100)
class MessageBody(StrictModel):
text: str
attachment_ids: list[str] = Field(default_factory=list, max_length=10)
app = FastAPI(docs_url=None, redoc_url=None, openapi_url=None)
@app.middleware("http")
async def request_id_middleware(request: Request, call_next):
request.state.request_id = uuid.uuid4().hex
try:
response = await call_next(request)
except Exception:
log.exception("Unhandled API exception request_id=%s", request.state.request_id)
response = JSONResponse(status_code=500, content={"error": {"code": "internal_error", "message": "Internal server error", "request_id": request.state.request_id}})
response.headers["X-Request-Id"] = request.state.request_id
response.headers["X-Instance-Id"] = INSTANCE_ID
return response
def error_response(status: int, code: str, message: str, request_id: str, headers: dict[str, str] | None = None):
return JSONResponse(status_code=status, content={"error": {"code": code, "message": message, "request_id": request_id}}, headers=headers or {})
@app.exception_handler(ApiError)
async def api_error_handler(request: Request, exc: ApiError):
return error_response(exc.status, exc.code, exc.message, request.state.request_id, exc.headers)
@app.exception_handler(PoolTimeout)
async def database_pool_timeout_handler(request: Request, exc: PoolTimeout):
log.warning("Database connection pool timed out request_id=%s", request.state.request_id)
return error_response(503, "unavailable", "Required storage is unavailable", request.state.request_id)
@app.exception_handler(psycopg.Error)
async def database_error_handler(request: Request, exc: psycopg.Error):
log.warning("Database operation failed request_id=%s", request.state.request_id)
return error_response(503, "unavailable", "Required storage is unavailable", request.state.request_id)
@app.exception_handler(RedisError)
async def redis_error_handler(request: Request, exc: RedisError):
log.warning("Session store operation failed request_id=%s", request.state.request_id)
return error_response(503, "unavailable", "Required storage is unavailable", request.state.request_id)
@app.exception_handler(BotoCoreError)
async def s3_core_error_handler(request: Request, exc: BotoCoreError):
log.warning("Object storage failure request_id=%s type=%s", request.state.request_id, type(exc).__name__)
return error_response(503, "unavailable", "Required storage is unavailable", request.state.request_id)
@app.exception_handler(ClientError)
async def s3_client_error_handler(request: Request, exc: ClientError):
log.warning("Object storage error request_id=%s code=%s", request.state.request_id, exc.response.get("Error", {}).get("Code"))
return error_response(503, "unavailable", "Required storage is unavailable", request.state.request_id)
@app.exception_handler(RequestValidationError)
async def validation_error_handler(request: Request, exc: RequestValidationError):
return error_response(400, "invalid_request", "Invalid request", request.state.request_id)
@app.exception_handler(StarletteHTTPException)
async def http_error_handler(request: Request, exc: StarletteHTTPException):
status = exc.status_code
code = "not_found" if status == 404 else "invalid_request"
message = "Object not found" if status == 404 else "Invalid request"
headers = dict(exc.headers or {})
return error_response(status, code, message, request.state.request_id, headers)
def db():
"""Borrow a bounded pooled connection; context exit commits/rolls back and returns it."""
return db_pool.connection(timeout=DB_POOL_TIMEOUT_SECONDS)
@app.on_event("startup")
def startup():
# Migrations run once in the dedicated `migrate` service before API replicas.
import time
db_pool.open(wait=False)
last_error = None
for _ in range(30):
try:
with db() as conn:
conn.execute("SELECT 1")
conn.execute("SELECT 1 FROM idempotency_records LIMIT 0")
session_store.ping()
s3.head_object(Bucket=S3_BUCKET, Key=S3_KEY_PREFIX + ".ready")
log.info("PostgreSQL, Redis and S3 bucket are ready; instance=%s", INSTANCE_ID)
return
except Exception as exc:
last_error = exc
time.sleep(1)
raise RuntimeError("PostgreSQL schema, Redis or S3 bucket unavailable during startup") from last_error
@app.on_event("shutdown")
def shutdown():
session_store.close()
db_pool.close()
def now_utc() -> datetime:
return datetime.now(timezone.utc)
def iso(value: datetime) -> str:
return value.astimezone(timezone.utc).isoformat(timespec="microseconds").replace("+00:00", "Z")
def user_dict(row: dict[str, Any]) -> dict[str, Any]:
return {"id": str(row["id"]), "username": row["username"], "display_name": row["display_name"], "created_at": iso(row["created_at"])}
def chat_dict(row: dict[str, Any], member_ids: list[str]) -> dict[str, Any]:
return {"id": str(row["id"]), "title": row["title"], "member_ids": member_ids, "created_at": iso(row["created_at"])}
def message_dict(row: dict[str, Any], attachment_ids: list[str] | None = None) -> dict[str, Any]:
return {"id": str(row["id"]), "chat_id": str(row["chat_id"]), "sender_id": str(row["sender_id"]), "text": row["text"], "created_at": iso(row["created_at"]), "attachment_ids": attachment_ids or []}
def validate_unicode(value: str, label: str) -> str:
"""Reject unpaired UTF-16 surrogates before sending text to PostgreSQL/KDF."""
try:
value.encode("utf-8")
except UnicodeEncodeError:
fail(400, "invalid_request", "Invalid Unicode in " + label)
return value
def parse_uuid(value: str, label: str = "UUID") -> uuid.UUID:
try:
# Keep the canonical UUID representation used by the Lab 1 API.
if not isinstance(value, str) or len(value) != 36:
raise ValueError("non-canonical UUID")
parsed = uuid.UUID(value)
if str(parsed) != value:
raise ValueError("non-canonical UUID")
return parsed
except (ValueError, TypeError, AttributeError):
fail(400, "invalid_request", "Invalid " + label)
def validate_body_type(request: Request):
content_type = request.headers.get("content-type", "").split(";", 1)[0].strip().lower()
if content_type != "application/json":
fail(415, "unsupported_media_type", "Content-Type must be application/json")
async def parse_body(request: Request, model):
validate_body_type(request)
try:
raw = await request.body()
data = json.loads(raw.decode("utf-8"))
return model.model_validate(data)
except (UnicodeDecodeError, json.JSONDecodeError, ValidationError, TypeError):
fail(400, "invalid_request", "Invalid JSON request body")
def challenge(scheme: str) -> dict[str, str]:
if scheme == "Basic":
return {"WWW-Authenticate": 'Basic realm="messenger", charset="UTF-8"'}
return {"WWW-Authenticate": 'Bearer realm="messenger"'}
def current_user(request: Request) -> dict[str, Any]:
auth = request.headers.get("authorization", "")
if not auth.startswith("Bearer ") or not auth[7:].strip():
fail(401, "unauthorized", "Valid Bearer session required", challenge("Bearer"))
token_hash = hashlib.sha256(auth[7:].strip().encode("utf-8")).hexdigest()
session_json = session_store.get("session:" + token_hash)
if session_json is None:
fail(401, "unauthorized", "Valid Bearer session required", challenge("Bearer"))
try:
session = json.loads(session_json)
user_id = uuid.UUID(session["user_id"])
expires_at = datetime.fromisoformat(session["expires_at"].replace("Z", "+00:00"))
except (ValueError, KeyError, TypeError, json.JSONDecodeError):
# Corrupt session data is invalidated instead of trusted.
session_store.delete("session:" + token_hash)
fail(401, "unauthorized", "Valid Bearer session required", challenge("Bearer"))
if expires_at <= now_utc():
session_store.delete("session:" + token_hash)
fail(401, "unauthorized", "Valid Bearer session required", challenge("Bearer"))
with db() as conn:
row = conn.execute("SELECT id,username,display_name,created_at FROM users WHERE id=%s", (user_id,)).fetchone()
if row is None:
session_store.delete("session:" + token_hash)
fail(401, "unauthorized", "Valid Bearer session required", challenge("Bearer"))
return {"id": row[0], "username": row[1], "display_name": row[2], "created_at": row[3], "expires_at": expires_at, "user": {"id": str(row[0]), "username": row[1], "display_name": row[2], "created_at": iso(row[3])}, "token_hash": token_hash}
def ensure_member(actor_id: uuid.UUID, chat_id: uuid.UUID):
with db() as conn:
row = conn.execute("SELECT 1 FROM chats c JOIN chat_members m ON m.chat_id=c.id WHERE c.id=%s AND m.user_id=%s", (chat_id, actor_id)).fetchone()
if row is None:
fail(404, "not_found", "Object not found")
def parse_page_args(limit: int | None, cursor: str | None, resource: str):
actual_limit = 50 if limit is None else limit
if isinstance(actual_limit, bool) or actual_limit < 1 or actual_limit > 100:
fail(400, "invalid_request", "limit must be an integer from 1 to 100")
after = None
if cursor is not None:
if not cursor:
fail(400, "invalid_request", "Invalid cursor")
try:
decoded = base64.urlsafe_b64decode(cursor + "=" * (-len(cursor) % 4)).decode("utf-8")
obj = json.loads(decoded)
if obj.get("resource") != resource:
fail(400, "invalid_request", "Cursor belongs to another resource")
after = (datetime.fromisoformat(obj["created_at"].replace("Z", "+00:00")), uuid.UUID(obj["id"]))
except ApiError:
raise
except Exception:
fail(400, "invalid_request", "Invalid cursor")
return actual_limit, after
def make_cursor(resource: str, created_at: datetime, item_id: uuid.UUID) -> str:
raw = json.dumps({"resource": resource, "created_at": iso(created_at), "id": str(item_id)}, separators=(",", ":")).encode()
return base64.urlsafe_b64encode(raw).decode().rstrip("=")
def json_rows(cursor, query: str, params=()):
with db() as conn:
cur = conn.execute(query, params)
names = [d.name for d in cur.description]
return [dict(zip(names, row)) for row in cur.fetchall()]
@app.get("/health/live")
def live():
return {"status": "ok"}
@app.get("/health/ready")
def ready():
try:
with db() as conn:
conn.execute("SELECT 1")
session_store.ping()
s3_probe.head_object(Bucket=S3_BUCKET, Key=S3_KEY_PREFIX + ".ready") # S3 is critical: uploads/downloads are part of the mandatory path
return {"status": "ok"}
except (psycopg.Error, PoolTimeout, RedisError, BotoCoreError, ClientError):
fail(503, "unavailable", "Required storage is unavailable")
@app.post("/api/v1/users", status_code=201)
async def register(request: Request):
body = await parse_body(request, RegisterBody)
if not USERNAME_RE.fullmatch(body.username):
fail(400, "invalid_request", "Invalid username")
validate_unicode(body.display_name, "display_name")
validate_unicode(body.password, "password")
if not body.display_name.strip() or len(body.display_name) > 100:
fail(400, "invalid_request", "Invalid display_name")
if not 12 <= len(body.password) <= 128:
fail(400, "invalid_request", "Password must contain 12 to 128 Unicode code points")
user_id = uuid.uuid4()
password_hash = PASSWORDS.hash(body.password)
try:
with db() as conn:
row = conn.execute("INSERT INTO users(id,username,display_name,password_hash) VALUES (%s,%s,%s,%s) RETURNING id,username,display_name,created_at", (user_id, body.username, body.display_name, password_hash)).fetchone()
except psycopg.errors.UniqueViolation:
fail(409, "conflict", "Username already exists")
return user_dict(dict(zip(("id", "username", "display_name", "created_at"), row)))
@app.get("/api/v1/users/me")
def me(request: Request):
return current_user(request)["user"]
@app.post("/api/v1/auth/sessions", status_code=201)
async def create_session(request: Request):
if await request.body():
fail(400, "invalid_request", "Session creation does not accept a request body")
auth = request.headers.get("authorization", "")
if not auth[:6].lower() == "basic ":
fail(401, "unauthorized", "Valid Basic credentials required", challenge("Basic"))
try:
decoded = base64.b64decode(auth[6:], validate=True).decode("utf-8")
username, password = decoded.split(":", 1)
except Exception:
fail(401, "unauthorized", "Invalid credentials", challenge("Basic"))
try:
validate_unicode(password, "password")
except ApiError:
fail(401, "unauthorized", "Invalid credentials", challenge("Basic"))
if not password or len(password) < 12 or len(password) > 128:
fail(401, "unauthorized", "Invalid credentials", challenge("Basic"))
with db() as conn:
row = conn.execute("SELECT id,username,display_name,password_hash,created_at FROM users WHERE username=%s", (username,)).fetchone()
if row is None:
fail(401, "unauthorized", "Invalid credentials", challenge("Basic"))
try:
PASSWORDS.verify(row[3], password)
except (VerifyMismatchError, VerificationError, InvalidHashError):
fail(401, "unauthorized", "Invalid credentials", challenge("Basic"))
token = secrets.token_urlsafe(32)
token_hash = hashlib.sha256(token.encode()).hexdigest()
expires_at = now_utc() + timedelta(seconds=SESSION_TTL_SECONDS)
session_json = json.dumps({"user_id": str(row[0]), "expires_at": iso(expires_at)}, separators=(",", ":"))
# Redis TTL enforces expiry; AOF + named volume make active sessions survive stack restarts.
session_store.setex("session:" + token_hash, SESSION_TTL_SECONDS, session_json)
user = user_dict({"id": row[0], "username": row[1], "display_name": row[2], "created_at": row[4]})
return {"token": token, "token_type": "Bearer", "expires_at": iso(expires_at), "user": user}
@app.delete("/api/v1/auth/sessions/current", status_code=204)
def revoke_session(request: Request):
actor = current_user(request)
session_store.delete("session:" + actor["token_hash"])
return Response(status_code=204)
@app.post("/api/v1/chats", status_code=201)
async def create_chat(request: Request):
actor = current_user(request)
body = await parse_body(request, ChatBody)
validate_unicode(body.title, "title")
if not body.title.strip() or len(body.title) > 100:
fail(400, "invalid_request", "Invalid title")
ids = [parse_uuid(value, "member UUID") for value in body.member_ids]
if len(ids) != len(set(ids)) or actor["id"] not in ids:
fail(400, "invalid_request", "Members must be unique and include the caller")
chat_id = uuid.uuid4()
try:
with db() as conn:
existing = conn.execute("SELECT id FROM users WHERE id = ANY(%s)", (ids,)).fetchall()
if len(existing) != len(ids):
fail(400, "invalid_request", "Unknown chat member")
row = conn.execute("INSERT INTO chats(id,title) VALUES (%s,%s) RETURNING id,title,created_at", (chat_id, body.title)).fetchone()
# psycopg 3: executemany exists on cursors, not on Connection (fix of a latent lab3 bug)
with conn.cursor() as cur:
cur.executemany("INSERT INTO chat_members(chat_id,user_id,position) VALUES (%s,%s,%s)", [(chat_id, uid, position) for position, uid in enumerate(ids)])
except psycopg.errors.ForeignKeyViolation:
fail(400, "invalid_request", "Unknown chat member")
return chat_dict({"id": row[0], "title": row[1], "created_at": row[2]}, [str(x) for x in ids])
@app.get("/api/v1/chats")
def list_chats(request: Request, limit: int | None = None, cursor: str | None = None):
actor = current_user(request)
chat_list_resource = "chats:" + str(actor["id"])
actual_limit, after = parse_page_args(limit, cursor, chat_list_resource)
params: list[Any] = [actor["id"]]
where = "m.user_id=%s"
if after:
where += " AND (c.created_at,c.id)>(%s,%s)"
params.extend(after)
params.append(actual_limit + 1)
rows = json_rows(None, f"SELECT c.id,c.title,c.created_at FROM chats c JOIN chat_members m ON m.chat_id=c.id WHERE {where} ORDER BY c.created_at,c.id LIMIT %s", tuple(params))
more = len(rows) > actual_limit
rows = rows[:actual_limit]
items = []
for row in rows:
members = json_rows(None, "SELECT user_id FROM chat_members WHERE chat_id=%s ORDER BY position,user_id", (row["id"],))
items.append(chat_dict(row, [str(m["user_id"]) for m in members]))
next_cursor = make_cursor(chat_list_resource, rows[-1]["created_at"], rows[-1]["id"]) if more and rows else None
return {"items": items, "next_cursor": next_cursor}
@app.get("/api/v1/chats/{chat_id}")
def get_chat(chat_id: str, request: Request):
actor = current_user(request)
parsed = parse_uuid(chat_id, "chat UUID")
ensure_member(actor["id"], parsed)
rows = json_rows(None, "SELECT id,title,created_at FROM chats WHERE id=%s", (parsed,))
members = json_rows(None, "SELECT user_id FROM chat_members WHERE chat_id=%s ORDER BY position,user_id", (parsed,))
return chat_dict(rows[0], [str(m["user_id"]) for m in members])
@app.post("/api/v1/chats/{chat_id}/messages", status_code=201)
async def create_message(chat_id: str, request: Request):
actor = current_user(request)
parsed_chat = parse_uuid(chat_id, "chat UUID")
ensure_member(actor["id"], parsed_chat)
body = await parse_body(request, MessageBody)
validate_unicode(body.text, "text")
attachment_ids = [parse_uuid(value, "attachment UUID") for value in body.attachment_ids]
if len(attachment_ids) != len(set(attachment_ids)):
fail(400, "invalid_request", "attachment_ids must be unique")
if len(body.text) > 4000 or (not body.text.strip() and not attachment_ids):
fail(400, "invalid_request", "Message needs 1 to 4000 non-whitespace code points of text or at least one attachment")
key = request.headers.get("Idempotency-Key", "")
if not IDEMPOTENCY_KEY_RE.fullmatch(key):
fail(400, "invalid_request", "Idempotency-Key must contain 1 to 128 ASCII letters, digits, '.', '_', ':', or '-'")
# Hash semantic JSON rather than raw bytes, so insignificant JSON formatting does not matter.
# attachment_ids is part of the hash only when non-empty: absent == [] and lab3 records stay valid.
canonical = {"text": body.text}
if attachment_ids:
canonical["attachment_ids"] = [str(x) for x in attachment_ids]
canonical_body = json.dumps(canonical, ensure_ascii=False, sort_keys=True, separators=(",", ":"))
request_hash = hashlib.sha256(canonical_body.encode("utf-8")).hexdigest()
scope = f"{actor['id']}|POST|/api/v1/chats/{parsed_chat}/messages|{key}"
# The advisory transaction lock serializes the same key across every API replica.
# Message and idempotency result are committed atomically in the same PostgreSQL transaction.
with db() as conn:
conn.execute("SELECT pg_advisory_xact_lock(hashtextextended(%s, 0))", (scope,))
existing = conn.execute(
"SELECT request_hash, response_body FROM idempotency_records "
"WHERE user_id=%s AND chat_id=%s AND idempotency_key=%s AND created_at >= now() - interval '24 hours'",
(actor["id"], parsed_chat, key),
).fetchone()
if existing is not None:
if existing[0] != request_hash:
fail(409, "conflict", "Idempotency-Key was already used with a different request body")
replay = dict(existing[1])
replay.setdefault("attachment_ids", [])
return JSONResponse(status_code=201, content=replay)
# Keys are guaranteed for 24 hours; expired records may be replaced by a new attempt.
conn.execute(
"DELETE FROM idempotency_records WHERE user_id=%s AND chat_id=%s AND idempotency_key=%s AND created_at < now() - interval '24 hours'",
(actor["id"], parsed_chat, key),
)
if attachment_ids:
# Foreign, unknown and other-chat attachments are indistinguishable: all -> 404.
found = conn.execute("SELECT count(*) FROM attachments WHERE id = ANY(%s) AND chat_id=%s", (attachment_ids, parsed_chat)).fetchone()[0]
if found != len(attachment_ids):
fail(404, "not_found", "Object not found")
row = conn.execute(
"INSERT INTO messages(id,chat_id,sender_id,text) VALUES (%s,%s,%s,%s) "
"RETURNING id,chat_id,sender_id,text,created_at",
(uuid.uuid4(), parsed_chat, actor["id"], body.text),
).fetchone()
result = message_dict(dict(zip(("id", "chat_id", "sender_id", "text", "created_at"), row)), [str(x) for x in attachment_ids])
if attachment_ids:
with conn.cursor() as cur:
cur.executemany("INSERT INTO message_attachments(message_id,attachment_id,position) VALUES (%s,%s,%s)", [(row[0], aid, i) for i, aid in enumerate(attachment_ids)])
conn.execute(
"INSERT INTO idempotency_records(user_id,chat_id,idempotency_key,request_hash,response_body) "
"VALUES (%s,%s,%s,%s,%s::jsonb)",
(actor["id"], parsed_chat, key, request_hash, json.dumps(result, ensure_ascii=False)),
)
return result
@app.get("/api/v1/chats/{chat_id}/messages")
def list_messages(chat_id: str, request: Request, limit: int | None = None, cursor: str | None = None):
actor = current_user(request)
parsed_chat = parse_uuid(chat_id, "chat UUID")
ensure_member(actor["id"], parsed_chat)
actual_limit, after = parse_page_args(limit, cursor, "messages:" + str(parsed_chat))
params: list[Any] = [parsed_chat]
where = "chat_id=%s"
if after:
where += " AND (created_at,id)>(%s,%s)"
params.extend(after)
params.append(actual_limit + 1)
rows = json_rows(None, f"SELECT id,chat_id,sender_id,text,created_at FROM messages WHERE {where} ORDER BY created_at,id LIMIT %s", tuple(params))
more = len(rows) > actual_limit
rows = rows[:actual_limit]
links: dict[Any, list[str]] = {}
if rows:
link_rows = json_rows(None, "SELECT message_id, attachment_id FROM message_attachments WHERE message_id = ANY(%s) ORDER BY message_id, position", ([r["id"] for r in rows],))
for link in link_rows:
links.setdefault(link["message_id"], []).append(str(link["attachment_id"]))
items = [message_dict(row, links.get(row["id"])) for row in rows]
next_cursor = make_cursor("messages:" + str(parsed_chat), rows[-1]["created_at"], rows[-1]["id"]) if more and rows else None
return {"items": items, "next_cursor": next_cursor}
# ======================= Lab 4: attachments =======================
def attachment_dict(row: dict[str, Any]) -> dict[str, Any]:
created = iso(row["created_at"])
return {"id": str(row["id"]), "chat_id": str(row["chat_id"]), "owner_id": str(row["owner_id"]), "filename": row["filename"],
"content_type": row["content_type"], "size_bytes": row["size_bytes"], "sha256": row["sha256"], "status": "ready",
"variants": [], "error_code": None, "created_at": created, "updated_at": created}
def validate_filename(values: list[str]) -> str:
"""filename is display-only: it never reaches a path or an S3 key, but dangerous values are still rejected."""
if len(values) != 1:
fail(400, "invalid_request", "Exactly one filename query parameter is required")
name = values[0]
validate_unicode(name, "filename")
if not 1 <= len(name) <= 255 or not name.strip() or name in (".", ".."):
fail(400, "invalid_request", "Invalid filename")
if "/" in name or "\\" in name or any(unicodedata.category(ch) == "Cc" for ch in name):
fail(400, "invalid_request", "Invalid filename")
return name
def sniff_image_type(head: bytes) -> str | None:
if head.startswith(b"\x89PNG\r\n\x1a\n"):
return "image/png"
if head.startswith(b"\xff\xd8\xff"):
return "image/jpeg"
return None
async def read_body_to_spool(request: Request, declared: str):
"""Stream the request body into a bounded spool while hashing; abort as soon as a limit is broken.
RAM per upload stays ~1 MiB: bodies above SPOOL_MEMORY_BYTES roll over to an anonymous (already unlinked) temp file.
"""
spool = tempfile.SpooledTemporaryFile(max_size=SPOOL_MEMORY_BYTES)
digest, total, head = hashlib.sha256(), 0, b""
try:
iterator = request.stream().__aiter__()
with anyio.fail_after(UPLOAD_TOTAL_TIMEOUT_SECONDS):
while True:
try:
chunk = await asyncio.wait_for(iterator.__anext__(), UPLOAD_IDLE_TIMEOUT_SECONDS)
except StopAsyncIteration:
break
if not chunk:
continue
total += len(chunk)
if total > MAX_UPLOAD_BYTES:
fail(413, "payload_too_large", "Attachment exceeds the size limit")
if len(head) < 8:
head += chunk[: 8 - len(head)]
if len(head) >= 8 and sniff_image_type(head) != declared:
fail(415, "unsupported_media_type", "Content does not match the declared image type")
digest.update(chunk)
spool.write(chunk)
except ApiError:
spool.close()
raise
except (TimeoutError, asyncio.TimeoutError):
spool.close()
fail(400, "invalid_request", "Upload timed out")
except BaseException: # ClientDisconnect, cancellation: nothing was written to S3 yet
spool.close()
raise
if total == 0 or sniff_image_type(head) != declared:
spool.close()
fail(415, "unsupported_media_type", "Content is not a supported image")
return spool, total, digest.hexdigest()
def decode_image_checked(spool, declared: str) -> None:
"""Runs in a worker thread: header -> pixel limit -> full decode. Raises ApiError for every user mistake."""
try:
spool.seek(0)
with Image.open(spool) as img:
if IMAGE_TYPES.get(img.format or "") != declared:
fail(415, "unsupported_media_type", "Content does not match the declared image type")
width, height = img.size
if width < 1 or height < 1:
fail(415, "unsupported_media_type", "Corrupt image")
if width * height > MAX_IMAGE_PIXELS: # decided from the header, before any pixel buffer exists
fail(413, "payload_too_large", "Image has too many pixels")
img.load() # full decode detects truncated/corrupt data
except ApiError:
raise
except Image.DecompressionBombError:
fail(413, "payload_too_large", "Image has too many pixels")
except MemoryError:
fail(503, "unavailable", "Temporarily unable to process the image")
except Exception: # Pillow raises OSError/SyntaxError/ValueError/struct.error/... for corrupt input
fail(415, "unsupported_media_type", "Corrupt or unsupported image")
finally:
spool.seek(0)
def s3_put(key: str, spool, size: int, content_type: str) -> None:
spool.seek(0)
s3.put_object(Bucket=S3_BUCKET, Key=key, Body=spool, ContentLength=size, ContentType=content_type)
def s3_delete_quietly(key: str) -> None:
try:
s3.delete_object(Bucket=S3_BUCKET, Key=key)
except Exception:
# The periodic sweeper (scripts/s3_sweep.py) removes it after the grace period.
log.warning("Compensation delete failed; orphan left for sweeper key=%s", key)
async def compensate(key: str) -> None:
with anyio.CancelScope(shield=True):
await run_in_threadpool(s3_delete_quietly, key)
@app.post("/api/v1/chats/{chat_id}/attachments", status_code=201)
async def upload_attachment(chat_id: str, request: Request):
actor = current_user(request)
parsed_chat = parse_uuid(chat_id, "chat UUID")
ensure_member(actor["id"], parsed_chat)
filename = validate_filename(request.query_params.getlist("filename"))
declared = request.headers.get("content-type", "").split(";", 1)[0].strip().lower()
if declared not in ("image/png", "image/jpeg"):
fail(415, "unsupported_media_type", "Content-Type must be image/png or image/jpeg")
length = request.headers.get("content-length")
if length is not None:
if not length.isascii() or not length.isdigit():
fail(400, "invalid_request", "Invalid Content-Length")
if int(length) > MAX_UPLOAD_BYTES: # reject before reading a single body byte
fail(413, "payload_too_large", "Attachment exceeds the size limit")
spool, size, sha256 = await read_body_to_spool(request, declared)
key = S3_KEY_PREFIX + secrets.token_hex(16) # server-chosen random key, unrelated to filename or attachment id
try:
await anyio.to_thread.run_sync(decode_image_checked, spool, declared, limiter=decode_limiter)
await run_in_threadpool(s3_put, key, spool, size, declared)
except ApiError: # user error found by validation, before any S3 write
spool.close()
raise
except BaseException:
spool.close()
await compensate(key) # a PUT whose reply was lost may still have stored the object
raise
spool.close()
attachment_id = uuid.uuid4()
try:
with db() as conn:
row = conn.execute(
"INSERT INTO attachments(id,chat_id,owner_id,filename,content_type,size_bytes,sha256,s3_key) VALUES (%s,%s,%s,%s,%s,%s,%s,%s) "
"RETURNING id,chat_id,owner_id,filename,content_type,size_bytes,sha256,created_at",
(attachment_id, parsed_chat, actor["id"], filename, declared, size, sha256, key)).fetchone()
except BaseException:
await compensate(key) # S3 PUT succeeded but the metadata COMMIT did not: remove the unaccounted object
raise
return attachment_dict(dict(zip(("id", "chat_id", "owner_id", "filename", "content_type", "size_bytes", "sha256", "created_at"), row)))
def load_attachment(attachment_id: str, request: Request) -> dict[str, Any]:
actor = current_user(request)
parsed = parse_uuid(attachment_id, "attachment UUID")
rows = json_rows(None, "SELECT a.id,a.chat_id,a.owner_id,a.filename,a.content_type,a.size_bytes,a.sha256,a.s3_key,a.created_at FROM attachments a "
"JOIN chat_members m ON m.chat_id=a.chat_id WHERE a.id=%s AND m.user_id=%s", (parsed, actor["id"]))
if not rows:
fail(404, "not_found", "Object not found")
return rows[0]
@app.get("/api/v1/attachments/{attachment_id}")
def get_attachment(attachment_id: str, request: Request):
return attachment_dict(load_attachment(attachment_id, request))
@app.get("/api/v1/attachments/{attachment_id}/content")
def download_attachment(attachment_id: str, request: Request, variant: str = "original"):
row = load_attachment(attachment_id, request)
if variant != "original":
fail(400, "invalid_request", "Only the original variant exists in lab4")
try:
obj = s3.get_object(Bucket=S3_BUCKET, Key=row["s3_key"])
except ClientError as exc:
if exc.response.get("Error", {}).get("Code") in ("NoSuchKey", "404"):
log.error("INTEGRITY: metadata without object attachment=%s", row["id"])
fail(500, "internal_error", "Internal server error")
raise
body = obj["Body"]
async def stream():
try:
async for chunk in iterate_in_threadpool(body.iter_chunks(CHUNK_BYTES)):
yield chunk
finally:
body.close()
headers = {"Content-Length": str(row["size_bytes"]), "X-Content-Type-Options": "nosniff", "Cache-Control": "private, no-store",
"Content-Disposition": "inline; filename*=UTF-8''" + quote(row["filename"], safe="")}
return StreamingResponse(stream(), media_type=row["content_type"], headers=headers)
if __name__ == "__main__":
import uvicorn
uvicorn.run("main:app", host=os.getenv("APP_HOST", "0.0.0.0"), port=int(os.getenv("APP_PORT", "8080")), timeout_graceful_shutdown=5)
+19
View File
@@ -0,0 +1,19 @@
"""Synthetic image builders (stdlib only). No large fixtures are stored in Git."""
import base64, os, struct, zlib
JPEG_SMALL = base64.b64decode("/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAYEBQYFBAYGBQYHBwYIChAKCgkJChQODwwQFxQYGBcUFhYaHSUfGhsjHBYWICwgIyYnKSopGR8tMC0oMCUoKSj/2wBDAQcHBwoIChMKChMoGhYaKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCj/wAARCAAgADADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwDxnTpHgRuQjSbiwtyoTgYAH1PoeNx6ZONrSw5jiaJmMo+ZtwXYNpP93sCC3Q4OOck7q1m+CRA4WSIhQ24KSA3QcEHO1u3f252YCp8oBEm2ZYFAQQoxhl3Hk4DEDJxj8+Wvl1/sn2uO93XnbS/Vrbz3vp1+69ptqbo4mjR8kFiig7cn16YPYjtgc4xXU6UzW0ku0rIWLIGdG+XG4tuJP3M57/8A18bT4Xi8mU5+Zvkj+YKHP+6TnsRzxjgda2dKdQiSxwtthAEi4IUq27oRngZHbOPxz4eKy3mT00Pk681Rkm9Jd3vb/hm338k9DX0eVHxHKpDhzgSk5LZBIyAVHBK8jpweuB01vui2wSAqq5L7QMuMffBA+6Ap75G7HGKxrSUtCkN5CXkR1dYIVDHGOBg9F6cckZIwOK2bWBImjnmWSF8FAFYNuOCCeo75PXnOexrwcTlib1R8xi60oxtF32v00638/Vq60tqfOWjvPEiywzJOZF2ggqdrg7TkjBwc8cHBI9TnobMAs1ybbYFKklgVbgbdpYkD0PXnpg55w9PeLz3V5FCJKWVs5AcEDO3r0PY8kck5zXR6TE/2qAG3kMasjs0hzgkdiCR0I68cfl+94rLLXdj9GxFXmSgndX9bPv18/L7tNnT5VadkFxKUAKFEADLkfMcZyDw3GPl4Arbs0ihDt9lCFSjNHJzxyGI7cnH8zwax9KFwbeQQXE0nzEKNpY44PTOF6n5hxkdRgVvBBcRSJhmTfuZtxbYNuASeeNuDkZ3evyivCxGV62t/X4Hy2Nrtw5+urV/nt8W3y0N20jS6sTIINszsxXYAvmgnJxnJPHsD+tbemxS2MqybAsG7IyQAOpHOOD8zc8HAIwOKzdPtpI4vMYPIxYLHOF2OMcFWJx9OTzz7Ct+xZbaeLehec9WWMLyflGPUkMxwD2+pr5/EZdukro+dxr1VSfuyVtdP6be9lpazuf/Z")
def _chunk(kind, data):
return struct.pack(">I", len(data)) + kind + data + struct.pack(">I", zlib.crc32(kind + data) & 0xFFFFFFFF)
def png(width, height, color_type=0, bit_depth=8, noise=False, level=6):
"""Valid PNG. color_type 0=gray, 2=RGB. noise=True -> incompressible rows (big files)."""
channels = {0: 1, 2: 3}[color_type]
row_bytes = (width * channels * bit_depth + 7) // 8
comp = zlib.compressobj(level)
parts = []
zero = b"\x00" * (row_bytes + 1)
for _ in range(height):
parts.append(comp.compress(b"\x00" + os.urandom(row_bytes) if noise else zero))
parts.append(comp.flush())
ihdr = struct.pack(">IIBBBBB", width, height, bit_depth, color_type, 0, 0, 0)
return b"\x89PNG\r\n\x1a\n" + _chunk(b"IHDR", ihdr) + _chunk(b"IDAT", b"".join(parts)) + _chunk(b"IEND", b"")
+42
View File
@@ -0,0 +1,42 @@
#!/usr/bin/env python3
"""Focused Lab 3 level-4 checks; run only against the local synthetic Compose stack."""
from concurrent.futures import ThreadPoolExecutor
import json, time, uuid
import sys, pathlib
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1] / "tests"))
from client import Client, Failure, arguments, check, run
def main():
args = arguments("Concurrent idempotency and replica diversity checks")
c = Client(args.base_url, args.ca_file, timeout=5)
alice, bob, eve, chat = c.bootstrap()
instances = set()
for _ in range(40):
instances.add(c.call('GET', '/api/v1/users/me', actor=alice, schema='User')['headers'].get('X-Instance-Id',''))
instances.discard('')
check(len(instances) >= 2, 'load balancer did not expose at least two distinct API instances')
key = 'parallel-' + uuid.uuid4().hex
body = {'text':'Одна запись при конкурентном повторе'}
def send(_):
return c.call('POST', '/api/v1/chats/'+chat['id']+'/messages', expected=201, actor=alice, body=body,
headers={'Idempotency-Key':key}, schema='Message')
with ThreadPoolExecutor(max_workers=12) as pool:
responses = list(pool.map(send, range(12)))
ids = {r['json']['id'] for r in responses}
check(len(ids) == 1, 'concurrent identical keys produced different message IDs')
check(all(r['status'] == 201 for r in responses), 'identical retry must return HTTP 201')
changed = c.call('POST', '/api/v1/chats/'+chat['id']+'/messages', expected=409, actor=alice,
body={'text':'Другой текст'}, headers={'Idempotency-Key':key})
missing = c.call('POST', '/api/v1/chats/'+chat['id']+'/messages', expected=400, actor=alice,
body={'text':'Без ключа'}, headers={})
invalid = c.call('POST', '/api/v1/chats/'+chat['id']+'/messages', expected=400, actor=alice,
body={'text':'Неверный ключ'}, headers={'Idempotency-Key':'bad key'})
print(json.dumps({'distinct_instances':sorted(instances), 'parallel_requests':len(responses),
'distinct_message_ids':len(ids), 'changed_body_status':changed['status'],
'missing_key_status':missing['status'], 'invalid_key_status':invalid['status']}, ensure_ascii=False))
print('PASS: concurrent cross-replica idempotency, key validation, and instance visibility')
if __name__ == '__main__':
run(main)
+102
View File
@@ -0,0 +1,102 @@
#!/usr/bin/env python3
"""Level-4 negative/safety suite for attachments. Run only against the synthetic local stack."""
import hashlib, sys, uuid, urllib.error, urllib.request
from concurrent.futures import ThreadPoolExecutor
from pathlib import Path
from urllib.parse import quote
ROOT = Path(__file__).resolve().parents[1]
sys.path[:0] = [str(ROOT / "tests"), str(Path(__file__).parent)]
from client import Client, arguments, check, run
import imagegen
MAX = 10 * 1024 * 1024
statuses = []
def up(c, actor, chat, data, ctype="image/png", name="a.png", expected=201, raw_name=None):
q = raw_name if raw_name is not None else quote(name)
r = c.call("POST", f"/api/v1/chats/{chat['id']}/attachments?filename={q}", expected, actor=actor, raw=data, headers={"Content-Type": ctype},
schema="Attachment" if expected == 201 else None)
statuses.append(r["status"])
return r
def main():
args = arguments("Level 4 attachment checks")
c = Client(args.base_url, args.ca_file, timeout=30)
alice, bob, eve, chat = c.bootstrap()
sample = (ROOT / "tests/fixtures/sample.png").read_bytes()
jpg = imagegen.JPEG_SMALL
# --- happy paths: png, jpeg, unicode filename, sha256 + headers, no redirect
a1 = up(c, alice, chat, sample)["json"]
a2 = up(c, alice, chat, jpg, "image/jpeg", "фото №1.jpg")["json"]
check(a2["filename"] == "фото №1.jpg" and a2["content_type"] == "image/jpeg", "jpeg/unicode filename")
d = c.call("GET", f"/api/v1/attachments/{a1['id']}/content", actor=bob)
check(hashlib.sha256(d["bytes"]).hexdigest() == a1["sha256"] and "Location" not in d["headers"], "download checksum / redirect")
big_ok = imagegen.png(1500, 1500, color_type=2, noise=True, level=0) # ~6.4 MiB: spool rolls over to temp file
check(len(big_ok) < MAX, "fixture size")
b = up(c, alice, chat, big_ok)["json"]
check(c.call("GET", f"/api/v1/attachments/{b['id']}/content", actor=bob)["bytes"] == big_ok, "big image roundtrip")
edge = up(c, alice, chat, imagegen.png(8000, 5000, bit_depth=1))["json"] # exactly 40 000 000 px: accepted
# --- size / pixels
up(c, alice, chat, b"\x89PNG\r\n\x1a\n" + b"0" * MAX, expected=413) # MAX+8 bytes
up(c, alice, chat, imagegen.png(8000, 5001, bit_depth=1), expected=413) # 40 008 000 px bomb, ~1 KB
up(c, alice, chat, imagegen.png(30000, 30000, bit_depth=1), expected=413) # far beyond Pillow's own limit
# --- corrupted
up(c, alice, chat, sample[: len(sample) // 2], expected=415) # truncated PNG
flipped = bytearray(sample); flipped[5000:5016] = b"\xde\xad\xbe\xef" * 4
up(c, alice, chat, bytes(flipped), expected=415) # CRC/zlib damage
up(c, alice, chat, b"\x89PNG\r\n\x1a\n" + bytes(range(256)) * 8, expected=415) # signature + garbage
up(c, alice, chat, jpg[: len(jpg) // 2], "image/jpeg", expected=415) # truncated JPEG
up(c, alice, chat, b"", expected=415)
# --- MIME spoofing
up(c, alice, chat, b"<html><script>alert(1)</script></html>", expected=415)
up(c, alice, chat, b"<svg xmlns='http://www.w3.org/2000/svg'/>", "image/png", expected=415)
up(c, alice, chat, b"GIF89a" + b"\x00" * 64, "image/png", expected=415)
up(c, alice, chat, jpg, "image/png", expected=415) # real JPEG declared PNG
up(c, alice, chat, sample, "image/jpeg", expected=415) # real PNG declared JPEG
up(c, alice, chat, sample, "text/plain", expected=415)
up(c, alice, chat, sample, "application/octet-stream", expected=415)
# --- filename
for bad in ("a%2Fb.png", "a%5Cb.png", "a%00b.png", "a%0Ab.png", "..", "%20%20", "x" * 256, ""):
up(c, alice, chat, sample, raw_name=bad, expected=400)
c.call("POST", f"/api/v1/chats/{chat['id']}/attachments", 400, actor=alice, raw=sample, headers={"Content-Type": "image/png"})
up(c, alice, chat, sample, raw_name="x" * 255)
# --- access control on upload / metadata / bytes
up(c, eve, chat, sample, expected=404)
for suffix in ("", "/content"):
c.call("GET", f"/api/v1/attachments/{a1['id']}{suffix}", 404, actor=eve)
c.call("GET", f"/api/v1/attachments/{uuid.uuid4()}", 404, actor=alice)
c.call("GET", f"/api/v1/attachments/{a1['id']}", 401)
c.call("GET", f"/api/v1/attachments/{a1['id']}/content?variant=thumbnail", 400, actor=alice)
# --- attachment binding rules
chat_b = c.call("POST", "/api/v1/chats", 201, actor=bob, body={"title": "B", "member_ids": [bob["id"], eve["id"]]}, schema="Chat")["json"]
chat_c = c.call("POST", "/api/v1/chats", 201, actor=alice, body={"title": "C", "member_ids": [alice["id"], bob["id"]]}, schema="Chat")["json"]
def msg(actor, ch, body, key=None, expected=201):
return c.call("POST", f"/api/v1/chats/{ch['id']}/messages", expected, actor=actor, body=body, headers={"Idempotency-Key": key or uuid.uuid4().hex},
schema="Message" if expected == 201 else None)
msg(bob, chat_b, {"text": "x", "attachment_ids": [a1["id"]]}, expected=404) # foreign chat's attachment (Bob not a member of chat)
msg(alice, chat_c, {"text": "x", "attachment_ids": [a1["id"]]}, expected=404) # same user, other chat
msg(alice, chat, {"text": "x", "attachment_ids": [str(uuid.uuid4())]}, expected=404)
msg(alice, chat, {"text": "x", "attachment_ids": [a1["id"], str(uuid.uuid4())]}, expected=404)
msg(alice, chat, {"text": "x", "attachment_ids": [a1["id"], a1["id"]]}, expected=400)
msg(alice, chat, {"text": "x", "attachment_ids": ["not-a-uuid"]}, expected=400)
msg(alice, chat, {"text": "x", "attachment_ids": [str(uuid.uuid4()) for _ in range(11)]}, expected=400)
msg(alice, chat, {"text": " ", "attachment_ids": []}, expected=400)
msg(alice, chat, {"text": "x", "attachment_ids": None}, expected=400)
# --- success + idempotency + order + history
key = uuid.uuid4().hex
m1 = msg(bob, chat, {"text": "", "attachment_ids": [a2["id"], a1["id"]]}, key)["json"]
check(m1["attachment_ids"] == [a2["id"], a1["id"]], "attachment order lost")
check(msg(bob, chat, {"attachment_ids": [a2["id"], a1["id"]], "text": ""}, key)["json"] == m1, "idempotent replay differs")
msg(bob, chat, {"text": "", "attachment_ids": [a1["id"]]}, key, expected=409)
hist = c.call("GET", f"/api/v1/chats/{chat['id']}/messages?limit=100", actor=alice, schema="MessagePage")["json"]["items"]
check(any(m == m1 for m in hist), "history lost attachment_ids")
# --- 10 parallel uploads stay correct
def one(_): return up(c, alice, chat, sample)["json"]["sha256"]
with ThreadPoolExecutor(10) as pool:
check(set(pool.map(one, range(10))) == {hashlib.sha256(sample).hexdigest()}, "parallel uploads")
check(max(statuses) < 500, "a user mistake produced HTTP 5xx: " + str(sorted(set(statuses))))
print("statuses seen:", sorted(set(statuses)))
print("PASS: level-4 attachment checks (no 5xx for user errors)")
if __name__ == "__main__":
run(main)
+69
View File
@@ -0,0 +1,69 @@
#!/usr/bin/env python3
"""Peak-RAM experiment: N concurrent uploads, reads VmHWM (peak RSS) of every API replica process.
Docker mode (default): `docker compose exec -T api` is not enough to pick each replica, so we iterate
`docker compose ps -q api` and `docker exec <id> cat /proc/1/status`. Local mode: --pids 123,456.
Run on a fresh stack (restart api first) so VmHWM starts from the idle baseline.
"""
import argparse, json, re, subprocess, sys, time
from concurrent.futures import ThreadPoolExecutor
from pathlib import Path
sys.path[:0] = [str(Path(__file__).resolve().parents[1] / "tests"), str(Path(__file__).parent)]
from client import Client, check, run
import imagegen
def reset_peak(args):
"""Writing 5 to /proc/PID/clear_refs resets VmHWM to the current RSS (own process, no privileges)."""
if args.pids:
for pid in args.pids.split(","):
Path(f"/proc/{pid}/clear_refs").write_text("5")
else:
for cid in subprocess.check_output(["docker", "compose", "ps", "-q", "api"], text=True).split():
subprocess.check_call(["docker", "exec", cid, "sh", "-c", "echo 5 > /proc/1/clear_refs"])
def read_status(args):
out = {}
if args.pids:
for pid in args.pids.split(","):
out[pid] = Path(f"/proc/{pid}/status").read_text()
else:
for cid in subprocess.check_output(["docker", "compose", "ps", "-q", "api"], text=True).split():
out[cid[:12]] = subprocess.check_output(["docker", "exec", cid, "cat", "/proc/1/status"], text=True)
res = {}
for k, t in out.items():
g = lambda n: int(re.search(n + r":\s+(\d+) kB", t).group(1)) // 1024
res[k] = {"VmRSS_MiB": g("VmRSS"), "VmHWM_MiB": g("VmHWM")}
return res
def main():
p = argparse.ArgumentParser()
p.add_argument("--base-url", default=None); p.add_argument("--ca-file", default=None)
p.add_argument("--concurrency", type=int, default=10); p.add_argument("--pids", default="")
p.add_argument("--profile", choices=["big-bytes", "max-pixels"], default="big-bytes")
p.add_argument("--output", default=None)
args = p.parse_args()
check(args.concurrency >= 10, "the lab requires >= 10 concurrent uploads")
c = Client(args.base_url, args.ca_file, timeout=120)
alice, bob, eve, chat = c.bootstrap()
if args.profile == "big-bytes": # ~9.6 MiB incompressible RGB PNG: stresses body streaming
data = imagegen.png(1830, 1830, color_type=2, noise=True, level=0)
else: # exactly 40 MP, tiny file: stresses the decoder
data = imagegen.png(8000, 5000, bit_depth=8, color_type=0)
check(len(data) <= 10 * 1024 * 1024, "payload above the limit")
reset_peak(args)
before = read_status(args)
def one(_):
t = time.monotonic(); r = c.call("POST", f"/api/v1/chats/{chat['id']}/attachments?filename=m.png", 201, actor=alice, raw=data, headers={"Content-Type": "image/png"}, schema="Attachment")
return time.monotonic() - t
t0 = time.monotonic()
with ThreadPoolExecutor(args.concurrency) as pool:
lat = sorted(pool.map(one, range(args.concurrency)))
after = read_status(args)
rep = {"profile": args.profile, "concurrency": args.concurrency, "payload_bytes": len(data), "wall_seconds": round(time.monotonic() - t0, 2),
"latency_s": {"min": round(lat[0], 2), "max": round(lat[-1], 2)}, "before": before, "after_peak": after,
"naive_estimate_MiB": round(args.concurrency * len(data) / 2**20 * 2)}
print(json.dumps(rep, indent=2))
if args.output: Path(args.output).write_text(json.dumps(rep, indent=2) + "\n")
if __name__ == "__main__":
run(main)
+41
View File
@@ -0,0 +1,41 @@
#!/usr/bin/env python3
"""Docker-stack check of the orphan sweeper: fresh survives, old orphan removed, referenced object never removed.
Run on the local synthetic compose stack: python3 student-tests/orphan_sweep_check.py"""
import subprocess, sys, time
from pathlib import Path
sys.path[:0] = [str(Path(__file__).resolve().parents[1] / "tests")]
from client import Client, check, run
def compose(container: str, *cmd: str) -> str:
return subprocess.check_output(
["docker", "compose", "run", "--rm", "--no-deps", "-T", container, *cmd],
text=True,
)
def main():
c = Client(timeout=30)
alice, bob, eve, chat = c.bootstrap()
att, content = c.attachment(alice, chat)
# injector → api (app-креды, есть PutObject)
key = compose("api", "python", "scripts/s3_fault_inject.py").strip().splitlines()[-1]
# sweeper → s3_sweeper (sweeper-креды, есть ListBucket)
out = compose("s3_sweeper", "python", "scripts/s3_sweep.py", "--apply", "--grace-seconds", "3600")
check("deleted=0" in out, "fresh orphan was deleted inside the grace period: " + out)
time.sleep(6)
out = compose("s3_sweeper", "python", "scripts/s3_sweep.py", "--grace-seconds", "5")
check("WOULD-DELETE " + key in out and "deleted=0" in out, "dry-run must report the orphan and delete nothing: " + out)
out = compose("s3_sweeper", "python", "scripts/s3_sweep.py", "--apply", "--grace-seconds", "5")
check("DELETE " + key in out, "orphan not deleted: " + out)
check(c.call("GET", f"/api/v1/attachments/{att['id']}/content", actor=bob)["bytes"] == content, "referenced attachment damaged by sweeper")
out = compose("s3_sweeper", "python", "scripts/s3_sweep.py", "--apply", "--grace-seconds", "5")
check("orphans=0" in out, "second run must be a no-op: " + out)
print("PASS: grace period, dry-run, orphan removal, referenced object preserved, idempotent re-run")
if __name__ == "__main__":
run(main)
+60
View File
@@ -0,0 +1,60 @@
"""Deterministic proof of compensation: S3 PUT succeeded but the metadata INSERT/COMMIT failed.
Needs: pip install -r requirements.txt httpx (no running services: S3, PostgreSQL are replaced by fakes)."""
import sys, unittest, uuid
from contextlib import contextmanager
from pathlib import Path
from unittest import mock
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT))
sys.path.insert(0, str(Path(__file__).parent))
import imagegen
from fastapi.testclient import TestClient
from src import main
import psycopg
class Compensation(unittest.TestCase):
def setUp(self):
self.puts, self.deletes = [], []
self.client = TestClient(main.app, raise_server_exceptions=False)
self.patches = [
mock.patch.object(main, "current_user", lambda r: {"id": uuid.uuid4()}),
mock.patch.object(main, "ensure_member", lambda a, c: None),
mock.patch.object(main, "s3_put", lambda key, spool, size, ct: self.puts.append(key)),
mock.patch.object(main, "s3_delete_quietly", lambda key: self.deletes.append(key)),
]
for p in self.patches: p.start()
self.url = f"/api/v1/chats/{uuid.uuid4()}/attachments?filename=a.png"
self.png = imagegen.png(16, 16)
def tearDown(self):
for p in self.patches: p.stop()
def post(self, data=None, ctype="image/png"):
return self.client.post(self.url, content=data or self.png, headers={"Content-Type": ctype})
def test_db_failure_after_put_deletes_object(self):
@contextmanager
def broken_db():
raise psycopg.OperationalError("simulated COMMIT failure")
yield
with mock.patch.object(main, "db", broken_db):
r = self.post()
self.assertEqual(r.status_code, 503)
self.assertEqual(len(self.puts), 1)
self.assertEqual(self.deletes, self.puts) # exactly the object that was written
def test_corrupt_image_never_reaches_s3(self):
r = self.post(self.png[:40])
self.assertEqual(r.status_code, 415)
self.assertEqual((self.puts, self.deletes), ([], []))
def test_put_failure_is_compensated_and_503(self):
def failing_put(key, spool, size, ct):
self.puts.append(key); raise main.BotoCoreError()
with mock.patch.object(main, "s3_put", failing_put):
r = self.post()
self.assertEqual(r.status_code, 503)
self.assertEqual(self.deletes, self.puts) # a PUT with a lost reply may have stored the object
if __name__ == "__main__":
unittest.main()