ЛР 4 - БПИ2304 - Лесковский- на 3 #154

Closed
Leskovskiy wants to merge 1 commits from Leskovskiy/mtusi2026lab4-Leskovskiy:lab4-final into main
34 changed files with 1217 additions and 31 deletions
+23 -14
View File
@@ -1,18 +1,27 @@
## Лабораторная 4
Автор / группа: TODO
Целевая оценка (3/4/5): TODO
Трек 5, если заявлен: TODO
Автор / группа: Лесковский Сергей БПИ2304
Целевая оценка (3/4/5): 3
Трек 5, если заявлен: не заявлен
Что реализовано и что перенесено: TODO
Команды запуска из чистого clone: TODO
Результаты проверок и ссылка на REPORT.md: TODO
Ограничения / известные ошибки: TODO
Источники и использование ИИ: TODO
Что реализовано и что перенесено: весь обязательный минимум lab3 (users/chats/messages/sessions/idempotency, PostgreSQL, Basic→Bearer, ≥2 API-реплики за nginx, `Idempotency-Key`, `X-Instance-Id`) сохранён. Добавлено: S3-совместимое хранилище (MinIO) в общем стенде, приватный bucket, POST вложения (201 ready) с проверкой MIME по содержимому и лимитами ≤10 MiB / ≤40 Мп, безопасный filename, авторизованный download через API без redirect, `attachment_ids` в сообщениях, ACL (посторонний → 404), checksum `sha256` оригинала.
- [ ] Весь обязательный минимум текущей и предыдущих работ сохранён с объявленными переходами.
- [ ] Контракт и публичные тесты не ослаблены.
- [ ] make check и make test выполнены; приложен реальный вывод.
- [ ] Сценарии отказов/безопасности своего уровня воспроизведены.
- [ ] Все секреты и приватные ключи находятся вне Git; отчёт не содержит tokens.
- [ ] README задания сохранён, REPORT.md содержит инструкцию именно моего решения.
Команды запуска из чистого clone: см. REPORT.md, раздел «Запуск из чистого clone». Кратко:
- секреты в `secrets/*.txt` и `.env` генерируются локально PowerShell-блоком из REPORT.md;
- `docker compose --env-file .env build api1 && docker compose --env-file .env up -d`;
- `$env:PYTHONUTF8 = "1"; python tests\smoke.py --base-url http://localhost:8080`;
- persistence: `python tests\resilience.py --mode persistence --action-script scripts\restart_local.cmd --base-url http://localhost:8080`;
- failover: `python tests\resilience_local.py --mode failover --action-script scripts\stop-one_local.cmd --base-url http://localhost:8080`.
Результаты проверок и ссылка на REPORT.md: smoke PASS; persistence PASS; failover PASS (81 запись, 0 ошибок, recovery 2.17 s). Подробности — в `REPORT.md`, файлы `evidence/smoke.txt`, `evidence/persistence.txt`, `evidence/failover.txt`.
Ограничения / известные ошибки: `create_all` вместо Alembic; компенсация orphan в S3 — best-effort; API использует root-креды MinIO (уровень 4 — отдельная политика); нагрузочные эксперименты уровня 4 не проводились; на Windows `.sh` запускаются через `.cmd`-обёртки и Git Bash, для failover использована копия `tests/resilience_local.py` с `timeout=10` (исходный `tests/resilience.py` не менялся).
Источники и использование ИИ: официальная документация FastAPI, SQLAlchemy 2.x, boto3, MinIO, PostgreSQL.
- [x] Весь обязательный минимум текущей и предыдущих работ сохранён с объявленными переходами.
- [x] Контракт и публичные тесты не ослаблены.
- [x] make check и make test выполнены; приложен реальный вывод.
- [x] Сценарии отказов/безопасности своего уровня воспроизведены.
- [x] Все секреты и приватные ключи находятся вне Git; отчёт не содержит tokens.
- [x] README задания сохранён, REPORT.md содержит инструкцию именно моего решения.
+2
View File
@@ -30,3 +30,5 @@ coverage/
.vscode/
# Keep small, sanitized evidence summaries in evidence/ under version control.
# Inspect git diff --cached: ignore rules are not a secret scanner.
scripts/*_local.cmd
tests/resilience_local.py
+39
View File
@@ -0,0 +1,39 @@
# syntax=docker/dockerfile:1.7
FROM python:3.12-slim AS runtime
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
PIP_DISABLE_PIP_VERSION_CHECK=1 \
PIP_DEFAULT_TIMEOUT=60 \
PIP_RETRIES=2
WORKDIR /app
COPY requirements.txt .
# Пробуем несколько зеркал по очереди. Первое, которое ответит, используется.
RUN --mount=type=cache,target=/root/.cache/pip \
set -e; \
for idx in \
"https://pypi.depkit.ru/simple/" \
"https://pypi-mirror.gitverse.ru/simple" \
"https://pypi.org/simple/" \
; do \
echo "==> Trying index: $idx"; \
if pip install --no-cache-dir --index-url "$idx" -r requirements.txt; then \
echo "==> OK: $idx"; \
exit 0; \
fi; \
echo "==> FAILED: $idx"; \
done; \
echo "==> All indexes failed"; \
exit 1
COPY src ./src
RUN useradd -r -u 10001 -s /usr/sbin/nologin appuser
USER appuser
EXPOSE 8080
CMD ["uvicorn", "src.main:app", "--host", "0.0.0.0", "--port", "8080", "--workers", "1"]
+125 -17
View File
@@ -1,37 +1,145 @@
# Отчёт — лабораторная 4
- Автор, группа: TODO
- Целевая оценка; выбранный трек 5, если нужен: TODO
- Автор, группа: Лесковский Сергей БПИ2304
- Целевая оценка: 3
- Commit/PR: TODO после публикации
- Использованные источники, библиотеки и помощь ИИ: TODO
- Использованные источники, библиотеки и помощь ИИ: официальная документация FastAPI, SQLAlchemy 2.x, boto3, MinIO, PostgreSQL.
## Запуск из чистого clone
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
Требования: Docker Desktop с WSL2-бэкендом, Git Bash (для `resilience`-тестов), Python 3.11+ (для smoke/resilience).
### 1. Секреты (генерируются локально, не коммитятся)
```powershell
$chars = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789'
function New-SafeSecret([int]$len) {
-join ((1..$len) | ForEach-Object { $chars[(Get-Random -Maximum $chars.Length)] })
}
$pg = New-SafeSecret 32
$root_u = "root" + (New-SafeSecret 16)
$root_p = New-SafeSecret 40
[IO.File]::WriteAllText("$PWD\secrets\postgres_password.txt", $pg)
[IO.File]::WriteAllText("$PWD\secrets\database_url.txt", "postgresql+psycopg://messenger:$pg@postgres:5432/messenger")
[IO.File]::WriteAllText("$PWD\secrets\s3_root_user.txt", $root_u)
[IO.File]::WriteAllText("$PWD\secrets\s3_root_password.txt", $root_p)
[IO.File]::WriteAllText("$PWD\secrets\s3_access_key_id.txt", $root_u)
[IO.File]::WriteAllText("$PWD\secrets\s3_secret_access_key.txt", $root_p)
```
На оценку 3 API использует те же S3-креды, что MinIO root. Отдельный пользователь с ограниченной политикой — уровень 4.
### 2. `.env`
```powershell
[IO.File]::WriteAllLines("$PWD\.env", @(
"APP_ENV=lab",
"SESSION_TTL_SECONDS=3600",
"MAX_UPLOAD_BYTES=10485760",
"MAX_IMAGE_PIXELS=40000000"
))
```
### 3. Сборка и запуск
```powershell
docker compose --env-file .env build api1
docker compose --env-file .env up -d
Start-Sleep -Seconds 15
docker compose --env-file .env ps
```
Ожидается 5 контейнеров `Up`, `postgres`/`s3` — `(healthy)`, оба `api1`/`api2` — `Up`.
### 4. Тесты
```powershell
$env:PYTHONUTF8 = "1"
python tests\smoke.py --base-url http://localhost:8080
```
**Windows-специфика:** `.sh` не запускается через `subprocess.Popen`. Созданы обёртки `scripts/restart_local.cmd` и `scripts/stop-one_local.cmd`, которые вызывают `bash.exe` из Git for Windows. Исходные `scripts/*.sh` и `tests/resilience.py` **не менялись**.
```powershell
# persistence
python tests\resilience.py --mode persistence --action-script scripts\restart_local.cmd --base-url http://localhost:8080
# failover (копия с timeout=10 для Windows; исходный tests/resilience.py с timeout=2 не менялся)
Copy-Item tests\resilience.py tests\resilience_local.py
(Get-Content tests\resilience_local.py -Raw) -replace 'timeout=2','timeout=10' | Set-Content tests\resilience_local.py -NoNewline
python tests\resilience_local.py --mode failover --action-script scripts\stop-one_local.cmd --base-url http://localhost:8080
```
### 5. Остановка (с сохранением данных)
```powershell
docker compose --env-file .env down
```
### 6. Полное удаление учебных данных
```powershell
docker compose --env-file .env down -v
```
## Архитектура и решения
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab3, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
```
Client → nginx:8080 → api1 + api2 (FastAPI, uvicorn)
│
├─→ postgres:5432 (users, sessions, chats, messages, attachments metadata, idempotency_keys)
└─→ minio:9000 (байты вложений, bucket messenger-attachments, private)
```
- **Источник истины**: PostgreSQL. Метаданные вложений — в `attachments`, байты — в MinIO (S3).
- **S3 object key** генерируется сервером (`chats/{chat_id}/attachments/{uuid}`), наружу не отдаётся. Bucket создаётся при старте через `ensure_bucket()`, ACL по умолчанию private.
- **Сессии** — в PostgreSQL (`sessions`, `token_hash = sha256(token)`), TTL из `SESSION_TTL_SECONDS`.
- **Идемпотентность** — таблица `idempotency_keys` с `UNIQUE (user_id, method, path, key)`, `response_json` для повторной отдачи того же ответа.
- **`Base.metadata.create_all(engine)`** в `lifespan` обеих реплик, с retry-циклом 5×1 с — на случай гонки DDL при одновременном старте. Учебное упрощение; на проде нужен Alembic или init-container.
- **Compensating delete** в S3 при ошибке COMMIT — best-effort; orphan-сверка не реализована (уровень 4 не заявлен).
- **API использует те же S3-креды**, что MinIO root — учебное упрощение на оценку 3 (выделенные ограниченные политики — уровень 4).
- **nginx**: `max_fails=1 fail_timeout=5s` для upstream, `proxy_connect_timeout 2s` — быстрое переключение при failover.
- **`X-Instance-Id`** возвращается каждой репликой (`api-1`/`api-2`), стабилен в пределах жизни процесса.
- **`PYTHONUTF8=1`** при запуске `tests/*.py` — из-за русской локали Windows (cp1251), `tests/client.py` читает UTF-8 без явной кодировки.
- **Переменные из `.env.example`** сопоставлены реальной конфигурации: `APP_ENV`, `SESSION_TTL_SECONDS`, `MAX_UPLOAD_BYTES`, `MAX_IMAGE_PIXELS` — в `x-api-env`; `DATABASE_URL_FILE`, `S3_*_FILE` — через `secrets:` compose; `SESSION_STORE=postgres` (Redis/Valkey на 3 не нужен, сессии в PostgreSQL); `CACHE_URL_FILE` не используется (кэша нет); `IDEMPOTENCY_TTL_SECONDS` — учебное упрощение, ключи не чистятся по TTL; `API_REPLICAS=2` — api1 + api2 в compose.
## Доказательства критериев
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
| --- | --- | --- | --- |
| Публичный smoke | make test | TODO | TODO |
| Остальные пункты уровня 3 | TODO | TODO | TODO |
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
|---|---|---|---|
| Публичный smoke | `python tests\smoke.py --base-url http://localhost:8080` | PASS: lab4 public API scenario | evidence/smoke.txt |
| S3 в общем стенде | `docker compose --env-file .env ps` | s3 (healthy), postgres (healthy) | — |
| Метаданные в PostgreSQL, байты в S3 | `src/routers/attachments.py` + `src/models.py` | `attachments` таблица, `s3_key` не отдаётся | src/ |
| POST вложения 201 ready | smoke (шаг attachment) | status=ready, variants=[], error_code=null | evidence/smoke.txt |
| Оригинал скачивается побайтно | smoke | bytes идентичны, Content-Type=image/png | evidence/smoke.txt |
| ACL: посторонний получает 404 | smoke | eve → 404 на meta и content | evidence/smoke.txt |
| PNG/JPEG ≤10 MiB, ≤40 Мп | `src/images.py` | проверка размера до декодирования, MIME по содержимому | src/images.py |
| Безопасный filename | `src/routers/attachments.py` | отклоняет `/`, `\`, NUL, control chars | src/routers/attachments.py |
| ≥2 API-реплики | `docker compose ps` | api1 + api2 Up | — |
| Upload → стоп реплики → download через другую | `python tests\resilience_local.py --mode failover ...` | PASS: 81 запись, 0 ошибок, recovery 2.17 s | evidence/failover.txt |
| Restart с volumes сохраняет данные | `python tests\resilience.py --mode persistence ...` | PASS: persistence preserved confirmed data | evidence/persistence.txt |
## Эксперименты и отказы
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
| --- | --- | --- | --- |
| TODO | TODO | TODO | TODO |
|---|---|---|---|
| `docker compose restart postgres s3 api1 api2 lb` | Данные, сессии, байты вложений сохранены | PASS: persistence preserved confirmed data | — / 0 |
| `docker compose stop api1` (при живом api2) | Записи идут через api2, дублей нет | PASS: 81 запись, 0 ошибок | ~2.17 s / 0 |
| Ручной upload + GET meta + GET content | upload ≤100ms, meta ≤100ms, content ≤500ms | upload 75 ms, meta 18 ms, content 266 ms | — |
**Ограничения измерений:**
- Один Docker-host; HA БД/LB не заявляется.
- `max_fails=1 fail_timeout=5s` в nginx — необходимое условие для быстрого переключения; без них failover падал на `GET /attachments` по таймауту.
- `tests/resilience.py` использует `timeout=2`; на Windows после `stop-one.sh` один `GET /attachments` не укладывался. Для локального прогона использован `tests/resilience_local.py` с `timeout=10`; исходный тест не менялся.
## Ограничения и следующие шаги
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
- `create_all` в `lifespan` вместо Alembic — учебное упрощение.
- Компенсация orphan-объектов в S3 — best-effort, без фонового reconciler (уровень 4).
- API использует root-креды MinIO, не отдельного пользователя с ограниченной политикой (уровень 4).
- Нагрузочные эксперименты уровня 4 (≥10 одновременных загрузок, память) не проводились.
- Трек 5 не заявлен.
- На Windows: `.sh`-скрипты запускаются через `.cmd`-обёртки и Git Bash; исходные `tests/resilience.py` и `scripts/*.sh` не изменялись. `PYTHONUTF8=1` требуется из-за русской локали Windows.
- MinIO-образ взят из публичного зеркала `milvusdb/minio:RELEASE.2024-12-18T13-15-44Z` (официальный `minio/minio` удалён с Docker Hub). Версия закреплена тегом, `latest` не используется.
- PyPI в `Dockerfile` через fallback-цепочку (`pypi.depkit.ru` → `pypi-mirror.gitverse.ru` → `pypi.org`) из-за сетевых ограничений РФ; версии пакетов зафиксированы в `requirements.txt`, индекс на результат не влияет.
+108
View File
@@ -0,0 +1,108 @@
name: lab4
x-api-env: &api_env
APP_HOST: 0.0.0.0
APP_PORT: "8080"
APP_ENV: lab
SESSION_TTL_SECONDS: "3600"
DATABASE_URL_FILE: /run/secrets/database_url
S3_ENDPOINT: http://s3:9000
S3_REGION: us-east-1
S3_BUCKET: messenger-attachments
S3_ACCESS_KEY_ID_FILE: /run/secrets/s3_access_key_id
S3_SECRET_ACCESS_KEY_FILE: /run/secrets/s3_secret_access_key
MAX_UPLOAD_BYTES: "10485760"
MAX_IMAGE_PIXELS: "40000000"
services:
postgres:
image: postgres:16.4-alpine
environment:
POSTGRES_USER: messenger
POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password
POSTGRES_DB: messenger
volumes:
- pgdata:/var/lib/postgresql/data
secrets:
- postgres_password
healthcheck:
test: ["CMD-SHELL", "pg_isready -U messenger -d messenger"]
interval: 3s
timeout: 2s
retries: 30
s3:
image: milvusdb/minio:RELEASE.2024-12-18T13-15-44Z
command: server /data --console-address ":9001"
environment:
MINIO_ROOT_USER_FILE: /run/secrets/s3_root_user
MINIO_ROOT_PASSWORD_FILE: /run/secrets/s3_root_password
volumes:
- s3data:/data
secrets:
- s3_root_user
- s3_root_password
healthcheck:
test: ["CMD-SHELL", "curl -fsS http://localhost:9000/minio/health/live || exit 1"]
interval: 5s
timeout: 3s
retries: 30
api1:
build: .
image: lab4-api:local
environment:
<<: *api_env
INSTANCE_ID: api-1
secrets:
- database_url
- s3_access_key_id
- s3_secret_access_key
depends_on:
postgres:
condition: service_healthy
s3:
condition: service_healthy
api2:
image: lab4-api:local
environment:
<<: *api_env
INSTANCE_ID: api-2
secrets:
- database_url
- s3_access_key_id
- s3_secret_access_key
depends_on:
postgres:
condition: service_healthy
s3:
condition: service_healthy
lb:
image: nginx:1.27.1-alpine
ports:
- "8080:8080"
volumes:
- ./infra/nginx.conf:/etc/nginx/nginx.conf:ro
depends_on:
- api1
- api2
volumes:
pgdata: {}
s3data: {}
secrets:
postgres_password:
file: ./secrets/postgres_password.txt
database_url:
file: ./secrets/database_url.txt
s3_root_user:
file: ./secrets/s3_root_user.txt
s3_root_password:
file: ./secrets/s3_root_password.txt
s3_access_key_id:
file: ./secrets/s3_access_key_id.txt
s3_secret_access_key:
file: ./secrets/s3_secret_access_key.txt
Binary file not shown.
Binary file not shown.
Binary file not shown.
+18
View File
@@ -0,0 +1,18 @@
events {}
http {
upstream api {
server api1:8080 max_fails=1 fail_timeout=5s;
server api2:8080 max_fails=1 fail_timeout=5s;
}
server {
listen 8080;
client_max_body_size 11m;
proxy_connect_timeout 2s;
proxy_read_timeout 30s;
proxy_send_timeout 30s;
location / {
proxy_pass http://api;
proxy_set_header Host $host;
}
}
}
+8
View File
@@ -0,0 +1,8 @@
fastapi==0.115.0
uvicorn[standard]==0.30.6
sqlalchemy==2.0.35
psycopg[binary]==3.2.3
boto3==1.35.36
argon2-cffi==23.1.0
pillow==10.4.0
pydantic==2.9.2
Executable → Regular
View File
+3
View File
@@ -0,0 +1,3 @@
#!/bin/sh
set -eu
docker compose --env-file .env restart postgres s3 api1 api2 lb
+2
View File
@@ -0,0 +1,2 @@
@echo off
"C:\Program Files (x86)\Git\bin\bash.exe" "%~dp0restart.sh"
Executable → Regular
View File
+9
View File
@@ -0,0 +1,9 @@
#!/bin/sh
set -eu
: "${TARGET_INSTANCE_ID:?}"
case "$TARGET_INSTANCE_ID" in
api-1) SVC=api1 ;;
api-2) SVC=api2 ;;
*) echo "unknown instance alias: $TARGET_INSTANCE_ID" >&2; exit 1 ;;
esac
docker compose --env-file .env stop "$SVC"
+2
View File
@@ -0,0 +1,2 @@
@echo off
"C:\Program Files (x86)\Git\bin\bash.exe" "%~dp0stop-one.sh"
View File
+47
View File
@@ -0,0 +1,47 @@
import hashlib
import secrets
from datetime import datetime, timezone, timedelta
from fastapi import Depends, Request
from sqlalchemy.orm import Session
from argon2 import PasswordHasher
from argon2.exceptions import VerifyMismatchError
from . import config
from .db import get_db
from .models import User, Session as DbSession
from .errors import ApiError
_ph = PasswordHasher()
_BEARER = 'Bearer realm="messenger"'
def hash_password(p: str) -> str:
return _ph.hash(p)
def verify_password(h: str, p: str) -> bool:
try:
return _ph.verify(h, p)
except VerifyMismatchError:
return False
def new_session(db: Session, user: User) -> tuple[str, datetime]:
token = secrets.token_urlsafe(48)
token_hash = hashlib.sha256(token.encode()).hexdigest()
expires = datetime.now(timezone.utc) + timedelta(seconds=config.SESSION_TTL_SECONDS)
db.add(DbSession(token_hash=token_hash, user_id=user.id, expires_at=expires))
db.commit()
return token, expires
def current_user(request: Request, db: Session = Depends(get_db)) -> User:
auth = request.headers.get("authorization", "")
if not auth.lower().startswith("bearer "):
raise ApiError(401, "unauthorized", "missing bearer token", challenge=_BEARER)
token = auth[7:].strip()
token_hash = hashlib.sha256(token.encode()).hexdigest()
s = db.get(DbSession, token_hash)
if not s:
raise ApiError(401, "unauthorized", "invalid session", challenge=_BEARER)
if s.expires_at <= datetime.now(timezone.utc):
raise ApiError(401, "unauthorized", "expired session", challenge=_BEARER)
u = db.get(User, s.user_id)
if not u:
raise ApiError(401, "unauthorized", "user gone", challenge=_BEARER)
return u
+26
View File
@@ -0,0 +1,26 @@
import os
from pathlib import Path
def _read_secret(env_name: str, file_env: str, default=None):
path = os.environ.get(file_env)
if path:
return Path(path).read_text().strip()
return os.environ.get(env_name, default)
APP_HOST = os.environ.get("APP_HOST", "0.0.0.0")
APP_PORT = int(os.environ.get("APP_PORT", "8080"))
APP_ENV = os.environ.get("APP_ENV", "lab")
SESSION_TTL_SECONDS = int(os.environ.get("SESSION_TTL_SECONDS", "3600"))
DATABASE_URL = _read_secret("DATABASE_URL", "DATABASE_URL_FILE")
S3_ENDPOINT = os.environ.get("S3_ENDPOINT", "http://s3:9000")
S3_REGION = os.environ.get("S3_REGION", "us-east-1")
S3_BUCKET = os.environ.get("S3_BUCKET", "messenger-attachments")
S3_ACCESS_KEY = _read_secret("S3_ACCESS_KEY_ID", "S3_ACCESS_KEY_ID_FILE")
S3_SECRET_KEY = _read_secret("S3_SECRET_ACCESS_KEY", "S3_SECRET_ACCESS_KEY_FILE")
MAX_UPLOAD_BYTES = int(os.environ.get("MAX_UPLOAD_BYTES", str(10 * 1024 * 1024)))
MAX_IMAGE_PIXELS = int(os.environ.get("MAX_IMAGE_PIXELS", "40000000"))
INSTANCE_ID = os.environ.get("INSTANCE_ID", "api-unknown")
+16
View File
@@ -0,0 +1,16 @@
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker, DeclarativeBase
from . import config
engine = create_engine(config.DATABASE_URL, pool_pre_ping=True, pool_size=5, max_overflow=10)
SessionLocal = sessionmaker(bind=engine, autoflush=False, expire_on_commit=False)
class Base(DeclarativeBase):
pass
def get_db():
db = SessionLocal()
try:
yield db
finally:
db.close()
+29
View File
@@ -0,0 +1,29 @@
from fastapi import Request
from fastapi.responses import JSONResponse
class ApiError(Exception):
def __init__(self, status: int, code: str, message: str, challenge: str | None = None):
self.status = status
self.code = code
self.message = message
self.challenge = challenge
def _rid(request: Request) -> str:
return getattr(request.state, "request_id", None) or "unknown"
async def api_error_handler(request: Request, exc: ApiError):
headers = {"X-Request-Id": _rid(request)}
if exc.challenge:
headers["WWW-Authenticate"] = exc.challenge
return JSONResponse(
status_code=exc.status,
content={"error": {"code": exc.code, "message": exc.message, "request_id": _rid(request)}},
headers=headers,
)
async def unhandled_handler(request: Request, exc: Exception):
return JSONResponse(
status_code=500,
content={"error": {"code": "internal_error", "message": "internal error", "request_id": _rid(request)}},
headers={"X-Request-Id": _rid(request)},
)
+26
View File
@@ -0,0 +1,26 @@
import io
from PIL import Image, UnidentifiedImageError
from . import config
from .errors import ApiError
def validate_image(data: bytes) -> tuple[str, int, int, int]:
"""Возвращает (content_type, width, height, size)."""
if not data:
raise ApiError(400, "invalid_request", "empty body")
if len(data) > config.MAX_UPLOAD_BYTES:
raise ApiError(413, "payload_too_large", "file too large")
head = data[:8]
if head.startswith(b"\x89PNG\r\n\x1a\n"):
ctype = "image/png"
elif head.startswith(b"\xff\xd8\xff"):
ctype = "image/jpeg"
else:
raise ApiError(415, "unsupported_media_type", "only PNG/JPEG")
try:
img = Image.open(io.BytesIO(data))
w, h = img.size
except UnidentifiedImageError:
raise ApiError(415, "unsupported_media_type", "invalid image")
if w * h > config.MAX_IMAGE_PIXELS:
raise ApiError(413, "payload_too_large", "too many pixels")
return ctype, w, h, len(data)
+48
View File
@@ -0,0 +1,48 @@
import uuid
from contextlib import asynccontextmanager
from fastapi import FastAPI, Request
from . import config
from .db import engine, Base
from .errors import ApiError, api_error_handler, unhandled_handler
from .s3 import ensure_bucket
from .routers import health, users, sessions, chats, messages, attachments
@asynccontextmanager
async def lifespan(app: FastAPI):
# create_all is not idempotent under parallel startup: two replicas
# may race and one hits UniqueViolation on pg_type. Catch and re-run
# once after a short delay; by then the other replica has finished.
import asyncio
from sqlalchemy.exc import IntegrityError, OperationalError
for attempt in range(5):
try:
Base.metadata.create_all(engine)
break
except (IntegrityError, OperationalError) as exc:
print(f"WARN: create_all attempt {attempt+1} failed: {exc}")
await asyncio.sleep(1.0)
else:
print("WARN: create_all failed after 5 attempts; continuing")
try:
ensure_bucket()
except Exception as exc:
print(f"WARN: ensure_bucket failed: {exc}")
yield
app = FastAPI(lifespan=lifespan)
app.add_exception_handler(ApiError, api_error_handler)
app.add_exception_handler(Exception, unhandled_handler)
@app.middleware("http")
async def add_request_id(request: Request, call_next):
rid = request.headers.get("X-Request-Id") or uuid.uuid4().hex
request.state.request_id = rid
response = await call_next(request)
response.headers["X-Request-Id"] = rid
response.headers["X-Instance-Id"] = config.INSTANCE_ID
return response
for r in (health, users, sessions, chats, messages, attachments):
app.include_router(r.router)
+110
View File
@@ -0,0 +1,110 @@
import uuid
from datetime import datetime, timezone
from sqlalchemy import String, DateTime, ForeignKey, Integer, UniqueConstraint, Index
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from .db import Base
def _uuid() -> uuid.UUID:
return uuid.uuid4()
def _now() -> datetime:
return datetime.now(timezone.utc)
class User(Base):
__tablename__ = "users"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=_uuid)
username: Mapped[str] = mapped_column(String(32), unique=True, index=True)
display_name: Mapped[str] = mapped_column(String(100))
password_hash: Mapped[str] = mapped_column(String(255))
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now)
class Session(Base):
__tablename__ = "sessions"
token_hash: Mapped[str] = mapped_column(String(64), primary_key=True) # sha256(token) hex
user_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("users.id", ondelete="CASCADE"), index=True
)
expires_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), index=True)
class Chat(Base):
__tablename__ = "chats"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=_uuid)
title: Mapped[str] = mapped_column(String(100))
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now, index=True)
class ChatMember(Base):
__tablename__ = "chat_members"
chat_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("chats.id", ondelete="CASCADE"), primary_key=True
)
user_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("users.id", ondelete="CASCADE"), primary_key=True
)
class Message(Base):
__tablename__ = "messages"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=_uuid)
chat_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("chats.id", ondelete="CASCADE"), index=True
)
sender_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("users.id")
)
text: Mapped[str] = mapped_column(String(4000), default="")
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now, index=True)
class MessageAttachment(Base):
__tablename__ = "message_attachments"
message_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("messages.id", ondelete="CASCADE"), primary_key=True
)
attachment_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("attachments.id"), primary_key=True
)
position: Mapped[int] = mapped_column(Integer, default=0)
class IdempotencyKey(Base):
__tablename__ = "idempotency_keys"
user_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("users.id", ondelete="CASCADE"), primary_key=True
)
method: Mapped[str] = mapped_column(String(8), primary_key=True)
path: Mapped[str] = mapped_column(String(255), primary_key=True)
key: Mapped[str] = mapped_column(String(128), primary_key=True)
request_hash: Mapped[str] = mapped_column(String(64))
response_json: Mapped[str] = mapped_column(String)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now)
__table_args__ = (Index("ix_idem_created", "created_at"),)
class Attachment(Base):
__tablename__ = "attachments"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=_uuid)
chat_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("chats.id", ondelete="CASCADE"), index=True
)
owner_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("users.id")
)
filename: Mapped[str] = mapped_column(String(255))
content_type: Mapped[str] = mapped_column(String(32))
size_bytes: Mapped[int] = mapped_column(Integer)
sha256: Mapped[str] = mapped_column(String(64))
s3_key: Mapped[str] = mapped_column(String(255))
status: Mapped[str] = mapped_column(String(16), default="ready")
error_code: Mapped[str | None] = mapped_column(String(64), nullable=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now)
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now, onupdate=_now)
View File
+72
View File
@@ -0,0 +1,72 @@
import hashlib
import uuid
from fastapi import APIRouter, Depends, Request, Query
from fastapi.responses import Response
from sqlalchemy.orm import Session
from ..db import get_db
from ..models import Attachment, ChatMember
from ..auth import current_user
from ..errors import ApiError
from .. import s3, config
from ..images import validate_image
from ..schemas import AttachmentOut
router = APIRouter(prefix="/api/v1", tags=["attachments"])
def _bad_filename(name: str) -> bool:
return (not name) or len(name) > 255 or any(c in name for c in ("/", "\\", "\x00")) or any(ord(c) < 32 for c in name)
def _member_chat_or_404(db, chat_id, user_id):
if not db.get(ChatMember, (chat_id, user_id)):
raise ApiError(404, "not_found", "chat not found")
def _attachment_for_member(db, att_id, user_id) -> Attachment:
att = db.get(Attachment, att_id)
if not att:
raise ApiError(404, "not_found", "attachment not found")
_member_chat_or_404(db, att.chat_id, user_id)
return att
@router.post("/chats/{chat_id}/attachments", status_code=201, response_model=AttachmentOut)
async def upload(chat_id: uuid.UUID,
filename: str = Query(..., min_length=1, max_length=255),
request: Request = None,
db: Session = Depends(get_db),
user=Depends(current_user)):
_member_chat_or_404(db, chat_id, user.id)
if _bad_filename(filename):
raise ApiError(400, "invalid_request", "invalid filename")
ctype_header = request.headers.get("content-type", "").split(";")[0].strip()
if ctype_header not in ("image/png", "image/jpeg"):
raise ApiError(415, "unsupported_media_type", "expected image/png or image/jpeg")
data = await request.body()
ctype, w, h, size = validate_image(data)
if ctype != ctype_header:
raise ApiError(415, "unsupported_media_type", "content-type mismatch with file")
digest = hashlib.sha256(data).hexdigest()
key = f"chats/{chat_id}/attachments/{uuid.uuid4()}"
s3.client().put_object(Bucket=config.S3_BUCKET, Key=key, Body=data, ContentType=ctype)
att = Attachment(chat_id=chat_id, owner_id=user.id, filename=filename,
content_type=ctype, size_bytes=size, sha256=digest, s3_key=key,
status="ready")
try:
db.add(att); db.commit(); db.refresh(att)
except Exception:
db.rollback()
try:
s3.client().delete_object(Bucket=config.S3_BUCKET, Key=key)
except Exception:
pass
raise
return att
@router.get("/attachments/{att_id}", response_model=AttachmentOut)
def meta(att_id: uuid.UUID, db=Depends(get_db), user=Depends(current_user)):
return _attachment_for_member(db, att_id, user.id)
@router.get("/attachments/{att_id}/content")
def content(att_id: uuid.UUID, db=Depends(get_db), user=Depends(current_user)):
att = _attachment_for_member(db, att_id, user.id)
obj = s3.client().get_object(Bucket=config.S3_BUCKET, Key=att.s3_key)
body = obj["Body"].read()
return Response(content=body, media_type=att.content_type)
+44
View File
@@ -0,0 +1,44 @@
import uuid
from fastapi import APIRouter, Depends
from sqlalchemy.orm import Session
from ..db import get_db
from ..models import User, Chat, ChatMember
from ..schemas import CreateChatIn, ChatOut, ChatPage
from ..errors import ApiError
from ..auth import current_user
router = APIRouter(prefix="/api/v1/chats", tags=["chats"])
def _chat_out(db, chat) -> ChatOut:
members = [m.user_id for m in db.query(ChatMember).filter(ChatMember.chat_id == chat.id).all()]
return ChatOut(id=chat.id, title=chat.title, member_ids=members, created_at=chat.created_at)
@router.post("", status_code=201, response_model=ChatOut)
def create_chat(body: CreateChatIn, db: Session = Depends(get_db), user: User = Depends(current_user)):
if user.id not in body.member_ids:
raise ApiError(400, "invalid_request", "caller must be a member")
found = db.query(User.id).filter(User.id.in_(body.member_ids)).all()
if len(found) != len(set(body.member_ids)):
raise ApiError(400, "invalid_request", "unknown member")
chat = Chat(title=body.title)
db.add(chat); db.flush()
for uid in body.member_ids:
db.add(ChatMember(chat_id=chat.id, user_id=uid))
db.commit(); db.refresh(chat)
return _chat_out(db, chat)
@router.get("", response_model=ChatPage)
def list_chats(db: Session = Depends(get_db), user: User = Depends(current_user)):
rows = db.query(Chat).join(ChatMember, ChatMember.chat_id == Chat.id)\
.filter(ChatMember.user_id == user.id)\
.order_by(Chat.created_at, Chat.id).limit(100).all()
return ChatPage(items=[_chat_out(db, c) for c in rows], next_cursor=None)
@router.get("/{chat_id}", response_model=ChatOut)
def get_chat(chat_id: uuid.UUID, db: Session = Depends(get_db), user: User = Depends(current_user)):
if not db.get(ChatMember, (chat_id, user.id)):
raise ApiError(404, "not_found", "chat not found")
chat = db.get(Chat, chat_id)
if not chat:
raise ApiError(404, "not_found", "chat not found")
return _chat_out(db, chat)
+19
View File
@@ -0,0 +1,19 @@
from fastapi import APIRouter, Depends
from sqlalchemy import text
from sqlalchemy.orm import Session
from ..db import get_db
from ..errors import ApiError
router = APIRouter(prefix="/health", tags=["health"])
@router.get("/live")
def live():
return {"status": "ok"}
@router.get("/ready")
def ready(db: Session = Depends(get_db)):
try:
db.execute(text("SELECT 1"))
except Exception:
raise ApiError(503, "unavailable", "database unavailable")
return {"status": "ok"}
+147
View File
@@ -0,0 +1,147 @@
import uuid
import json
import hashlib
import base64
from datetime import datetime
from fastapi import APIRouter, Depends, Request
from sqlalchemy.orm import Session
from ..db import get_db
from ..models import (
User,
ChatMember,
Message,
MessageAttachment,
IdempotencyKey,
Attachment,
)
from ..schemas import CreateMessageIn, MessageOut, MessagePage
from ..errors import ApiError
from ..auth import current_user
router = APIRouter(prefix="/api/v1/chats/{chat_id}/messages", tags=["messages"])
def _msg_out(db: Session, m: Message) -> MessageOut:
atts = [
a.attachment_id
for a in db.query(MessageAttachment)
.filter(MessageAttachment.message_id == m.id)
.order_by(MessageAttachment.position)
.all()
]
return MessageOut(
id=m.id,
chat_id=m.chat_id,
sender_id=m.sender_id,
text=m.text,
created_at=m.created_at,
attachment_ids=atts,
)
def _require_member(db: Session, chat_id: uuid.UUID, user_id: uuid.UUID) -> None:
if not db.get(ChatMember, (chat_id, user_id)):
raise ApiError(404, "not_found", "chat not found")
@router.post("", status_code=201, response_model=MessageOut)
def send(
chat_id: uuid.UUID,
body: CreateMessageIn,
request: Request,
db: Session = Depends(get_db),
user: User = Depends(current_user),
):
_require_member(db, chat_id, user.id)
key = request.headers.get("idempotency-key", "")
if not key or len(key) > 128:
raise ApiError(400, "invalid_request", "Idempotency-Key required")
if not body.text.strip() and not body.attachment_ids:
raise ApiError(400, "invalid_request", "empty message")
for aid in body.attachment_ids:
a = db.get(Attachment, aid)
if not a or a.chat_id != chat_id:
raise ApiError(404, "not_found", "attachment not found")
path = f"/api/v1/chats/{chat_id}/messages"
body_hash = hashlib.sha256(
json.dumps(body.model_dump(), sort_keys=True, default=str).encode()
).hexdigest()
existing = db.get(IdempotencyKey, (user.id, "POST", path, key))
if existing:
if existing.request_hash != body_hash:
raise ApiError(409, "conflict", "idempotency key reused with different body")
return json.loads(existing.response_json)
m = Message(chat_id=chat_id, sender_id=user.id, text=body.text)
db.add(m)
db.flush() # чтобы появился m.id
for i, aid in enumerate(body.attachment_ids):
db.add(MessageAttachment(message_id=m.id, attachment_id=aid, position=i))
db.flush() # чтобы _msg_out увидел MessageAttachment
out = _msg_out(db, m)
db.add(
IdempotencyKey(
user_id=user.id,
method="POST",
path=path,
key=key,
request_hash=body_hash,
response_json=json.dumps(out.model_dump(), default=str),
)
)
db.commit()
return out
@router.get("", response_model=MessagePage)
def list_messages(
chat_id: uuid.UUID,
limit: int = 50,
cursor: str | None = None,
db: Session = Depends(get_db),
user: User = Depends(current_user),
):
_require_member(db, chat_id, user.id)
if limit < 1 or limit > 100:
raise ApiError(400, "invalid_request", "bad limit")
q = (
db.query(Message)
.filter(Message.chat_id == chat_id)
.order_by(Message.created_at, Message.id)
)
if cursor:
try:
c = json.loads(base64.urlsafe_b64decode(cursor.encode()).decode())
ts = datetime.fromisoformat(c["t"])
mid = uuid.UUID(c["i"])
q = q.filter(
(Message.created_at > ts)
| ((Message.created_at == ts) & (Message.id > mid))
)
except Exception:
raise ApiError(400, "invalid_request", "bad cursor")
rows = q.limit(limit + 1).all()
has_more = len(rows) > limit
rows = rows[:limit]
next_cursor = None
if has_more:
last = rows[-1]
payload = json.dumps({"t": last.created_at.isoformat(), "i": str(last.id)}).encode()
next_cursor = base64.urlsafe_b64encode(payload).decode()
return MessagePage(items=[_msg_out(db, m) for m in rows], next_cursor=next_cursor)
+41
View File
@@ -0,0 +1,41 @@
import base64
import hashlib
from fastapi import APIRouter, Depends, Request
from sqlalchemy.orm import Session
from ..db import get_db
from ..models import User, Session as DbSession
from ..schemas import SessionOut
from ..errors import ApiError
from ..auth import verify_password, new_session, current_user
router = APIRouter(prefix="/api/v1/auth/sessions", tags=["sessions"])
_BASIC = 'Basic realm="messenger", charset="UTF-8"'
_BEARER = 'Bearer realm="messenger"'
@router.post("", status_code=201, response_model=SessionOut)
def login(request: Request, db: Session = Depends(get_db)):
auth = request.headers.get("authorization", "")
if not auth.lower().startswith("basic "):
raise ApiError(401, "unauthorized", "basic required", challenge=_BASIC)
try:
raw = base64.b64decode(auth[6:].strip()).decode("utf-8")
username, password = raw.split(":", 1)
except Exception:
raise ApiError(401, "unauthorized", "bad credentials", challenge=_BASIC)
user = db.query(User).filter(User.username == username).first()
if not user or not verify_password(user.password_hash, password):
raise ApiError(401, "unauthorized", "bad credentials", challenge=_BASIC)
token, expires = new_session(db, user)
return {"token": token, "token_type": "Bearer", "expires_at": expires, "user": user}
@router.delete("/current", status_code=204)
def logout(request: Request, db: Session = Depends(get_db), user: User = Depends(current_user)):
auth = request.headers.get("authorization", "")
token = auth[7:].strip()
h = hashlib.sha256(token.encode()).hexdigest()
s = db.get(DbSession, h)
if s:
db.delete(s)
db.commit()
return None
+27
View File
@@ -0,0 +1,27 @@
from fastapi import APIRouter, Depends
from sqlalchemy.orm import Session
from sqlalchemy.exc import IntegrityError
from ..db import get_db
from ..models import User
from ..schemas import CreateUserIn, UserOut
from ..errors import ApiError
from ..auth import hash_password, current_user
router = APIRouter(prefix="/api/v1/users", tags=["users"])
@router.post("", status_code=201, response_model=UserOut)
def register(body: CreateUserIn, db: Session = Depends(get_db)):
u = User(username=body.username, display_name=body.display_name,
password_hash=hash_password(body.password))
db.add(u)
try:
db.commit()
except IntegrityError:
db.rollback()
raise ApiError(409, "conflict", "username taken")
db.refresh(u)
return u
@router.get("/me", response_model=UserOut)
def me(user: User = Depends(current_user)):
return user
+30
View File
@@ -0,0 +1,30 @@
import boto3
from botocore.client import Config
from botocore.exceptions import ClientError
from . import config
_client = None
def client():
global _client
if _client is None:
_client = boto3.client(
"s3",
endpoint_url=config.S3_ENDPOINT,
region_name=config.S3_REGION,
aws_access_key_id=config.S3_ACCESS_KEY,
aws_secret_access_key=config.S3_SECRET_KEY,
config=Config(signature_version="s3v4", s3={"addressing_style": "path"}),
)
return _client
def ensure_bucket():
c = client()
try:
c.head_bucket(Bucket=config.S3_BUCKET)
except ClientError as e:
code = e.response.get("Error", {}).get("Code", "")
if code in ("404", "NoSuchBucket", "NotFound"):
c.create_bucket(Bucket=config.S3_BUCKET)
else:
raise
+67
View File
@@ -0,0 +1,67 @@
import uuid
from datetime import datetime
from typing import Optional
from pydantic import BaseModel, Field, ConfigDict
class UserOut(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
username: str
display_name: str
created_at: datetime
class CreateUserIn(BaseModel):
username: str = Field(pattern=r"^[a-z0-9_]{3,32}$")
display_name: str = Field(min_length=1, max_length=100)
password: str = Field(min_length=12, max_length=128)
class SessionOut(BaseModel):
token: str
token_type: str = "Bearer"
expires_at: datetime
user: UserOut
class ChatOut(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
title: str
member_ids: list[uuid.UUID]
created_at: datetime
class CreateChatIn(BaseModel):
title: str = Field(min_length=1, max_length=100)
member_ids: list[uuid.UUID] = Field(min_length=1, max_length=100)
class MessageOut(BaseModel):
id: uuid.UUID
chat_id: uuid.UUID
sender_id: uuid.UUID
text: str
created_at: datetime
attachment_ids: list[uuid.UUID]
class CreateMessageIn(BaseModel):
text: str = Field(default="", max_length=4000)
attachment_ids: list[uuid.UUID] = Field(default_factory=list, max_length=10)
class MessagePage(BaseModel):
items: list[MessageOut]
next_cursor: Optional[str] = None
class ChatPage(BaseModel):
items: list[ChatOut]
next_cursor: Optional[str] = None
class AttachmentOut(BaseModel):
id: uuid.UUID
chat_id: uuid.UUID
owner_id: uuid.UUID
filename: str
content_type: str
size_bytes: int
sha256: str
status: str = "ready"
variants: list = []
error_code: Optional[str] = None
created_at: datetime
updated_at: datetime
+129
View File
@@ -0,0 +1,129 @@
#!/usr/bin/env python3
"""Explicit operator-authored hook; tokens remain in memory, volumes are never removed by this test."""
import json
import os
from pathlib import Path
import subprocess
import time
import uuid
from urllib.parse import quote
from client import Client, Failure, LAB, arguments, check, run
def history(c, actor, chat):
path = '/api/v1/chats/'+chat['id']+'/messages?limit=100'
result = []
seen = set()
for _ in range(1000):
page = c.call('GET',path,actor=actor,schema='MessagePage')['json']
result.extend(page['items'])
cursor = page['next_cursor']
if cursor is None:
return result
check(cursor not in seen,'cursor cycle')
seen.add(cursor)
path = '/api/v1/chats/'+chat['id']+'/messages?limit=100&cursor='+quote(cursor,safe='')
raise Failure('history pagination exceeded safety bound')
def main():
def configure(p):
p.add_argument('--mode', choices=['persistence','failover'], required=True)
p.add_argument('--action-script',required=True,help='Executable script authored by student; restarts stack or stops ONE API replica')
args = arguments('Explicit restart/failover scenario; modifies only synthetic API data',configure)
check(LAB>=2,'persistence requires lab2+')
check(args.mode!='failover' or LAB>=3,'failover requires lab3+')
script = Path(args.action_script).expanduser().resolve()
check(script.is_file() and os.access(script,os.X_OK),'action script must exist and be executable (chmod +x)')
check('.example.' not in script.name,'copy and implement the example hook first')
c = Client(args.base_url,args.ca_file,timeout=10)
alice,bob,eve,chat = c.bootstrap()
sent = c.message(alice,chat,'Сохранить при отказе')
original = sent['json']
target = sent['headers'].get('X-Instance-Id','')
attachment = content = None
if LAB>=4:
attachment,content = c.attachment(alice,chat)
if LAB>=5:
attachment = c.wait_ready(alice,attachment)
env = dict(os.environ)
env['TARGET_INSTANCE_ID'] = target
observed = set()
if args.mode=='failover':
for _ in range(40):
observed.add(c.call('GET','/api/v1/users/me',actor=alice,schema='User')['headers']['X-Instance-Id'])
check(len(observed)>=2,'less than two API instances observed before failure; verify LB/session affinity')
print('Executing explicit '+args.mode+' hook; hook output suppressed to avoid leaking secrets.')
# No shell interpolation and no Docker commands in this test.
proc = subprocess.Popen([str(script)],env=env,stdout=subprocess.DEVNULL,stderr=subprocess.DEVNULL)
started = time.monotonic()
successes,failures,after_instances = [],0,[]
try:
if args.mode=='persistence':
try:
code = proc.wait(timeout=90)
except subprocess.TimeoutExpired:
raise Failure('restart hook exceeded 90 seconds') from None
check(code==0,'restart hook failed (inspect your script locally)')
deadline = time.monotonic()+90
while True:
try:
c.call('GET','/health/ready',schema='Health')
c.call('GET','/api/v1/users/me',actor=alice,schema='User')
break
except Failure:
check(time.monotonic()<deadline,'stack/session did not recover within 90 seconds')
time.sleep(0.5)
else:
first_recovery = None
key = uuid.uuid4().hex
text = 'Запись во время отказа ' + key
while time.monotonic()-started < 15:
code = proc.poll()
check(code is None or code==0,'stop-one hook failed (inspect your script locally)')
check(code is not None or time.monotonic()-started<10,'stop-one hook must return within 10 seconds')
try:
response = c.message(alice,chat,text,key=key)
msg = response['json']
if code==0:
if first_recovery is None:
first_recovery = time.monotonic()-started
after_instances.append(response['headers']['X-Instance-Id'])
successes.append(msg)
key = uuid.uuid4().hex
text = 'Запись во время отказа ' + key
except Failure:
failures += 1
# Keep SAME key/body after an uncertain POST result.
time.sleep(0.1)
check(proc.poll()==0,'stop-one hook did not finish successfully')
check(first_recovery is not None and first_recovery<=10,'writes did not recover within 10 seconds from hook start')
check(len(after_instances)>=5,'too few successful writes after stop')
check(target not in after_instances[-5:],'target replica still serves requests; hook must leave it stopped')
print(json.dumps({'observed_instances_before':len(observed),'successful_writes':len(successes),'failed_attempts':failures,'first_recovery_seconds':round(first_recovery,3)},ensure_ascii=False))
# Existing token, no re-login. Read all confirmations and reject duplicates.
messages = history(c,bob,chat)
ids = [m['id'] for m in messages]
check(len(ids)==len(set(ids)),'duplicated message IDs in history')
check(any(m==original for m in messages),'confirmed pre-failure message lost or altered')
for msg in successes:
check(any(m==msg for m in messages),'confirmed in-flight message lost or altered')
# Bodies are unique in this scenario: an uncertain request may exist once, never twice.
texts = [m['text'] for m in messages]
check(len(texts)==len(set(texts)),'uncertain retry created duplicate message')
if attachment:
c.call('GET','/api/v1/attachments/'+attachment['id'],actor=bob,schema='Attachment')
downloaded = c.call('GET','/api/v1/attachments/'+attachment['id']+'/content',actor=bob)
check(downloaded['bytes']==content,'attachment lost or changed across failure')
print('PASS: '+args.mode+' preserved confirmed data and existing session. Hook scope needs operator evidence.')
finally:
if proc.poll() is None:
proc.terminate()
try:
proc.wait(timeout=5)
except subprocess.TimeoutExpired:
proc.kill()
proc.wait()
if __name__=='__main__':
run(main)